Orchestrate containers using Docker compose

This tutorial is a step-by-step guide to teach you how to use the Docker compose tool to manage multiple contains from a single YAML configuration file.

Managing multiple containers individually can soon become troublesome if you have lots of containers. For a simpler Docker administration you can use a more advanced tool like docker-compose which lets you manage multiple containers from a single configuration file.

Behind the scenes, the Docker compose tool leverages the Docker engine for pulling images, building the images, starting the containers in a correct order, and making the right links among the containers and services.

Install Docker Compose

You can download the latest release of Docker Compose at: https://github.com/docker/compose/releases
You can find more details on the installation process on the Docker Compose site at: https://docs.docker.com/compose/install/

After the installation, make sure that the docker-compose has the right permissions and then start it:

$ chmod +x /usr/local/bin/docker-compose

The docker-compose tool orchestrates containers using the docker-compose.yml configuration file. This file defines the services to be executed, the relationships between them and their runtime properties. You need to use the YAML syntax in this file, which specifies for each service a list of keys:values; hence it is an human readable file.

A sample Docker Compose file

As an example, we will show how to start a Container Image of MySQL. Firstly, create a file docker-compose.yml file with the following content:

version: '3'
services:
    mysql:
        container_name: mysql
        image: mysql
        extra_hosts: [ 'host.docker.internal:host-gateway' ]
        command: --default-authentication-plugin=mysql_native_password
        restart: always
        environment:
            MYSQL_ROOT_PASSWORD: password
        volumes:
            - ./config/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
            - ./local-data:/var/lib/mysql
        ports:
            - 3306:3306
    adminer:
        container_name: adminer
        image: adminer
        extra_hosts: [ 'host.docker.internal:host-gateway' ]
        restart: always
        environment:
            ADMINER_DEFAULT_SERVER: mysql
        depends_on:
            - mysql
        ports:
            - 8888:8080

# credentials: root:password
# http://localhost:8888/?server=mysql&username=root

The above file defines two services:

A mysql service which is bound to the mysql image. This file configures a volume for the storage of MySQL Tables and a config directory where we will upload the init.sql script to add some sample data
A adminer Service which is an HTTP interface to access MySQL Database. You can access it at localhost:8888 using the credentials “root/password”

Finally, create a file init.sql in the config folder with the following content:

CREATE DATABASE testdb;
USE testdb;

CREATE TABLE users (
    id INT NOT NULL AUTO_INCREMENT,
    username VARCHAR(255) NOT NULL,
    email VARCHAR(255) NOT NULL,
    PRIMARY KEY (id)
);

INSERT INTO users (username, email)
VALUES
    ('jane.doe', '[email protected]'),
    ('john.doe', '[email protected]');

To run the services you can simply execute:

docker-compose up

Here is the expected outcome:

Finally, it is essential to execute docker-compose down in order to stop containers, removes containers, networks, volumes, and images created by up. The simple Control+C is definitely not recommended!

docker-compose down

Executing Commands in Containers

You can use the docker-compose exec command to execute a command within a running Docker container that you manage through Docker Compose. It’s particularly useful when you want to run a specific command in a container without starting a new instance of that container.

For example, take the following snippet of docker-compose-postgres.yaml file:

version: '2'
services:
  zookeeper:
    image: quay.io/debezium/zookeeper:${DEBEZIUM_VERSION}
    ports:
     - 2181:2181
     - 2888:2888
     - 3888:3888
  kafka:
    image: quay.io/debezium/kafka:${DEBEZIUM_VERSION}
    ports:
     - 9092:9092
    links:
     - zookeeper
    environment:
     - ZOOKEEPER_CONNECT=zookeeper:2181

The above definition sets up a Kafka Cluster. To execute the kafka-topics.sh command on the “kafka” service, you can use the following command:

docker-compose -f docker-compose-postgres.yaml exec kafka /kafka/bin/kafka-topics.sh \
    --bootstrap-server kafka:9092 \
    --list

How to build a Dockerfile with Docker Compose

It us worth mentioning that you can include in your service definition the build option. This allows to build a Container from a Dockerfile. For example, if you have a Dockerfile in the debezium-jdbc-es folder, you can add the following build section in your docker-compose.yaml file:

  connect:
    image: debezium/connect-jdbc-es:${DEBEZIUM_VERSION}
    build:
      context: debezium-jdbc-es
      args:
        DEBEZIUM_VERSION: ${DEBEZIUM_VERSION}
    ports:
     - 8083:8083
     - 5005:5005

Then, use the –build option when launching the Docker Compose:

docker compose -up --build

Including other YAML files

Since version of 2.20 of Docker-compose it it possible to include compose sub-projects via the include attribute. This is comparable to language dependency management. For example:

include:
   - ../commons/compose.yaml
   - ../another-project/compose.yaml

services:
   foo:
      depends_on:
         - imported-service

Benefits of using Docker Compose

In conclusion, let’s discuss which are the main benefits of using Docker Compose with containerized applications:

Simplified Application Deployment: Docker Compose allows you to define and manage multi-container applications using a single configuration file. It simplifies the deployment process by automating the creation and configuration of multiple containers, including their networks and volumes.
Easy Service Orchestration: Docker Compose provides a straightforward way to define and manage the relationships and dependencies between services.

Reproducible Development Environments: Docker Compose allows developers to define their development environments as code. With a Compose file, you can specify the necessary services, their versions, dependencies, and configurations.
Scalability and Load Balancing: Docker Compose simplifies scaling your services horizontally. By specifying the desired number of service replicas in the Compose file, you can easily scale your application to handle increased traffic or workload.
Integration Testing and Continuous Integration: Docker Compose is an excellent tool for integration testing and continuous integration workflows. By defining the required services and their configurations in the Compose file, you can easily spin up the necessary environment for running tests and verifying the interactions between different components of your application.