This post is inspired by an issue I encountered while trying to connect two docker containers located in two different docker-compose files. It briefly describes how this situation can happen and how to handle it.

The post assumes that the reader is familiar with Docker and Docker compose.

Context

I was given a project and tasked to make the local dev setup work.
The project architecture was consisting of three components; two Java applications and a MongoDB database. There were two different GIT repositories, one for each of the Java applications.
Both Java applications were dockerized and a docker-compose file was also present in each of the repositories to help spin up the containers. One of the two docker-compose also had a MongoDB container defined.

The issue

There were a couple of minor configuration issues, but after resolving those all three containers were running. I was able to successfully connect to each of them, and the Java application that was using MongoDB as the database was also successfully connected. The problem was the communication between the two Java apps.

After some debugging, I eventually figured out that the problem was that the java containers didn’t see each other.

That is because both of the docker-compose files were configured without network(s), which meant that each docker-compose created its own default network and connected the containers defined in that docker-compose file to that default network.
Effectively this means that there were two separate networks, one had just one Java app container connected and the other had the other Java app and the MongoDB containers connected.
And naturally, the container from one network couldn’t communicate to a container in the other network.

The solution

To solve the issue we have to put the docker containers that should talk to each other on the same network.
In my case, that meant that the two Java containers had to be part of the same network. I achieved that by defining a named network in one of the docker-compose files and defining an external network in the other. With this configuration, one docker-compose created a network and the other was looking for that existing network and connecting the containers to that instead of creating a network.

This means that in every docker-compose we need to define the networks and then add every service to whatever networks it should join.
The final docker-compose looks something like:

version: '3.7'

services:
    service-1:
        .. other parts of the service ...
        # The next part joins the container to the listed networks
        networks:
            - internal_network_name
    service-2:
        .. other parts of the service ...
        # The next part joins the container to the listed networks
        networks:
            - internal_network_name
            - external_network_name

networks:        
    internal_network_name:
        # This signals docker-compose to create a new network
        driver: bridge            
    external_network_name:
        # This signals docker-compose to search for an existing network
        external: true

An important thing is that we define a name for the network we want others to join because, without an explicit name, docker-compose creates a network with the folder name as a prefix. For the previous docker-compose file to work, we need to define the network like so in another docker-compose:

networks:
    external_network_name:
        # Without this part, the network name would be {folderName}_external_network_name
        name: external_network_name
        driver: bridge

Example project

For the example, I am using a real-world analogy of the NBA.
In the NBA there are 30 teams. Every team has departments, some departments work only inside the team, while some departments need to work with other teams or the league itself.

For this example, we will use two teams (Celtics and Lakers) with two departments (HR and Trade). Let’s imagine the HR department only works inside the team, while the Trade department needs to talk to other teams to agree on player trades. So we will want to protect the HR department from unwarranted messages from the other teams’ departments, and on the other side allow the Trade department to be contacted for inquiries.

We will represent each department with a Java Spring Boot application and an extra one for the NBA. In the end, we will have 5 applications:

• Celtics HR
• Celtics Trade
• Lakers HR
• Lakers Trade
• NBA

We can imagine each team (and the NBA) would have its own GIT repository, but for simplicity, I put everything in the same GIT repository under different folders. Each of the team folders contains two dockerized Java Spring boot applications, one for the HR department and one for the Trade department. The NBA folder contains a single dockerized Java Spring Boot application.

The Spring Boot applications are very simple, they only have a single controller and endpoint that says hello from the different departments:

@RestController()
public class HumanResourcesController {

    @GetMapping("/hr")
    @ResponseStatus(HttpStatus.OK)
    @CrossOrigin(origins = "*")
    public String health() {
        return "Hello from the Celtics HR Department!";
    }

}

The Dockerfile for all five applications is the same, quite a standard java application Dockerfile

FROM maven:3.6.0-jdk-11-slim AS build
COPY src /app/src
COPY pom.xml /app
RUN mvn -f /app/pom.xml clean package

FROM openjdk:11-jre-slim
COPY --from=build /app/target/hr-0.0.1-SNAPSHOT.jar /usr/local/lib/celtics-hr.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/usr/local/lib/celtics-hr.jar"]

Both team folders also contain docker-compose files that define these two applications.

version: '3.7'

services:
    celtics-hr:
        container_name: celtics-hr
        build:
            context: ./hr/
            dockerfile: Dockerfile
        volumes:
            - '.:/app'
        ports:
            - 8081:8080
        networks:
            - celtics_internal
    celtics-trade:
        container_name: celtics-trade
        build:
            context: ./trade/
            dockerfile: Dockerfile
        volumes:
            - '.:/app'
        ports:
            - 8082:8080
        networks:
            - celtics_internal
            - nba

networks:
    celtics_internal:
        driver: bridge    
    nba:
        external: true

The NBA folder also has a docker-compose file.

version: '3.7'

services:
    nba:
        container_name: nba
        build:
            context: .
            dockerfile: Dockerfile
        volumes:
            - '.:/app'
        ports:
            - 9999:8080
        networks:
            - nba

networks:
    nba:
        # We need this to keep the network name 'nba',
        # if not present the network gets prefixed '{prefix}_nba'
        # and the network wouldn't be found in the other docker-compose
        name: nba
        driver: bridge

If you were following along, you will recognize that there are three networks defined:

• nba (created in the NBA docker-compose)
• celtics_internal (created in the Celtics docker-compose)
• lakers_internal (created in the Lakers docker-compose)

On top of that, the Celtics and Lakers docker-compose files define the external nba network.

The result is that we have the following networks and containers connected to them:

• celtics_internal
  • Celtics Trade
  • Celtics HR
• lakers_internal
  • Lakers Trade
  • Lakers HR
• nba
  • NBA
  • Celtics Trade
  • Lakers Trade

To test this, we can connect to the individual docker containers and try to make CURL calls to the other container endpoints.

First, we spin up everything:

cd nba
docker-compose up -d
cd ..
cd celtics
docker-compose up -d
cd ..
cd lakers
docker-compose up -d
cd ..

After all the containers are running we can connect to the container with

docker exec -it <container_name> /bin/bash

We want to execute GET CURL calls, so we need to install CURL on the container with

apt-get update; apt-get install curl

After that is done, we can try to contact other containers with:

curl -X GET "<container_name>:8080/{path}"

For instance, if we connect to the lakers-hr container and execute a call to the lakers-trade container we get a successful response:

docker exec -it lakers-hr /bin/bash

root@3cfb0c31b323:/# curl -X GET "lakers-trade:8080/trade"
Hello from the Lakers Trade Department!    
root@3cfb0c31b323:/#

Here is the full connectivity matrix:

You can browse the full example repository on GitHub.

Conclusion

We can see that with the network section of the docker-compose file we can control which containers can communicate with each other. If there are no networks defined in the docker-compose, all the services join a default network specific to that docker-compose file.
This is useful if our docker-compose file contains all the services, but if any external services should be visible to us, we need to make sure that the containers are on the same network. And for that, we need to define an external network and connect our container to it.

This is it for a quick overview of how to connect two docker containers located in different docker-compose files.
Thank you for reading through, I hope you found this article useful.