docker-compose.yml: A Closer Look
Anyone working with Docker should bookmark the official reference document for
docker compose: https://docs.docker.com/compose/compose-file/Open up the
docker-compose.yml file in the root of the repository, or view it online:A YML (or properly, 'YAML' for Yet Another Markup Language) file is just a text file with strict formatting rules that govern how the file is divided into sections and how one line relates to the next. YAML files use two spaces (not tabs) for each level of indentation, and Docker is very particular about YAML syntax. Throughout this guide, we'll refer to "levels" of indentation, e.g. "2nd level" would mean an indentation of four spaces.
Your IDE should have a plugin to facilitate YAML syntax, like Visual Studio's YAML by Adam Voss or Sublime Text's Pretty YAML.
docker-compose up looks for a docker-compose.yml file in the current directory and translates the instructions in that file to a series of docker run CLI commands (behind-the-scenes; you won't see this part). The stdout output of each container will go to your console window unless you add the option-d for --detach(ed); that output will still be available to you in Docker's logs but you'll get your console back right away.Docker won't read anything after
# in a compose file, so we can comment on the same line as our directives.Let's go through each line of
docker-compose.yml:version: '3.6' # if no version is specificed then v1 is assumed. Recommend v2 minimum
The version directive indirectly specifies the version of Docker required to execute the file. It's indirect because the version number doesn't refer to a Docker version, but the reference edition of
docker-compose.yml that tracks which directives are included and deprecated over time. Which versions of docker-compose.yml map to which versions of the Docker engine are available in the official document reference, but unless you know you have to support an older version of Docker, use a version corresponding to a recent stable Docker CE release.volumes:
sql-data:
Persistent storage is one of the more difficult concepts in Docker -- especially in a Swarm. We'll spend more time on this subject later, but for a simple development stack, it's a straightforward topic.
Since containers are stateless, anything in their filesystem will be destroyed when the container is destroyed. Stopping and restarting a container doesn't destroy it (though in older versions of Docker, it used to); a simple restart will preserve its filesystem, but we need to be able to destroy and re-create our MySQL container without losing our database. The solution is a
docker volume, a virtual, persistent storage device managed by the Docker daemon. We can remove our container entirely and it will persist unless we explicitly delete the volume using docker volume delete.Some of the terminology around Docker volumes can be confusing. "Volumes" can refer to one of two things:
- A docker volume, which is a persistent data storage device managed entirely by the Docker daemon, or
- The
volumesdirective indocker composeanddocker stack, which governs both docker volumes and bind mounts. Both involve persistent storage, but a bind mount takes a directory that already exists on the host and mounts it into the container. This means that the host (rather than Docker) is managing the filesystem. Bind mounts are slower than docker volumes -- especially on Windows and MacOS. The benefit of bind mounts is that they allow for easy read and write access between the host and the container, which is what we want for a development environment; on a regular docker volume, you can only copy files between the host and the container through thedocker copycommand, unless your container supports some other form of file sharing (e.g. NFS).
Every service that makes use of a volume will have a service-level
volumes directive that specifies the source and target mount point for each volume it requires.networks:
cfswarm-simple:
Docker's default behavior without any
networks directive is to create a "project" network that applies to any service created in the same directory. This would work just fine, but it's best to name your networks so that you can assign each container to one or more of them explicitly. Docker networking is fairly complicated under the hood, but for our purposes, we just need to know which containers need to be able to access which other containers (and so should share a network). Our simple solution here places all of our containers on one network so that each container can access any other container by its name; we'll take advantage of this when configuring our nginx proxy and our CF datasources.The top-level
networks directive just names the network we'll be referring to later on.secrets:
cfconfig:
file: ./config/cfml/cfconfig/cfconfig.json
Like the
networks directive, the top-level secrets directive defines a "Docker Secret" that we'll refer to in each of our services. We'll cover this in more detail below; for now, it's sufficient to know that we want to define individual pieces of data or files that we will mount into one or more containers, and that they're called secrets because this is a mechanism that is used to great effect for storing credentials and any other sensitive information we don't want accessible to anything outside our container.services:
The services directive indicates that the next (first) level indentation will be the names of the services we're asking Docker to create. We can refer to these services through the Docker CLI via docker service commands.
For our purposes, the difference between a container and a service is simple: a container is a single "instance" of a service, but a service may be replicated more than once -- that is, it may have more than one container of the same type.
You can have a container without a service, but every service needs a container.
Since our sample app needs a database, let's define that service first.
cfswarm-mysql: # a friendly name. this is also DNS name inside network
The first level of indentation is reserved for service name definitions. Each service must have a unique name, and everything underneath it at the second or higher indentation level describes some aspect of the service. It's easy to break a docker compose file (or a docker stack file, as you'll see later) into digestible sections once you know how the file is divided.
In this case, we're specifying a service for our MySQL container. Note that the service name is also the the DNS name: when our CF containers need to connect to MySQL, the hostname they'll use is cfswarm-mysql.
image: mysql:5.7
Every container is created from an image, and this line specifies the image for our MySQL service:
[registry address]/<image name>:[tag]If no registry address is specified, Docker will first check to see if the image exists locally, and if it doesn't, pull the image from Docker Hub. Docker Hub allows unlimited storage of public images and is typically where you'll pull from unless and until you make your own images -- even then, those images usually extend public images from Docker Hub.
If no tag is specified, Docker will default to pulling the
:latest tag. This can have unintended consequences -- you might upgrade your database or your web server without realizing that you've done so. :latest for MySQL is currently version 8, but we'll get MySQL 5.7 because we've specified that tag. The README for a Docker image will usually document available tags (like the official Docker Hub page for MySQL does/), and a full list is often available under the tags link on that page.Only the name of the image is required. For ubiquitous software like mysql or nginx, it may just be the name of the software, but it's often a path to an image (e.g. ortussolutions/commandbox)
container_name: cfswarm-mysql
Without this directive, Docker will assign a randomly generated name to the container for our MySQL service. While it's easy to see the names of running containers with
docker container ls, we want to be able to refer to our containers quickly and easily, whether we're outputting logs (docker logs cfswarm-mysql) or logging in to the container (docker exec -it cfswarm-mysql bash).Note that you can only use the
container_name directive when you have a single container in your service. If we were replicating our service -- running multiple, identical containers in parallel -- container_name would be forbidden. environment:
MYSQL_ROOT_PASSWORD: 'myAwesomePassword'
MYSQL_DATABASE: 'cfswarm-simple-dev'
MYSQL_ROOT_HOST: '%'
MYSQL_LOG_CONSOLE: 'true'
You can specify environment variables one-at-a-time in
docker-compose.yml. These particular variables set the root password, create a database called cfswarm-simple-dev (if it doesn't already exist), routes the log to the console, and allows connections from any host. volumes:
- type: volume
source: sql-data
target: /var/lib/mysql
The above example is a "long form" definition of a single volume; the compose reference explains the difference between the long and shorthand expression for the volume directive.
- type specifies a volume rather than a bind mount, which will mount a directory on the host into the container. Since volumes perform much better than bind mounts, they are the best choice for a database container.
- source refers to the name of a volume defined in the top-level volumes directive. Multiple containers can share the same volume, so we're referring back to our top-level definition that would (but in this case, does not) contain any volume options like driver specifications.
- target specifies the mount point in the container for the volume.
Whenever you see a
volume directive under a service, you'll know to look for a top-level volume configuration reference. Because volumes can be shared between services, only the service-specific information goes in the service-level definition. ports:
- 3306:3306
The
ports directive maps one or more ports on the service container to the specified port on the host. In this example, we're mapping MySQL's port 3306 to the same port on our host. Without this directive, our other containers could access MySQL since they all share a network, but we wouldn't be able to connect to MySQL using any tools on our host unless we ran those tools in Docker as well.Like
volumes and some other docker compose directives, ports has both a short form (shown above) and a long form that can be easier to read. Here is the same directive in long form: ports:
- target: 3306
published: 3306
protocol: tcp
mode: host
networks:
- cfswarm-simple
The service-level
networks directive refers back to the top-level network name we defined earlier. Every container that needs to access that network should have a service-level directive referring to it.Our CFML service definition uses some of the same directives as the
mysql service, but with a few alternatives and additions. cfswarm-cfml:
image: ortussolutions/commandbox:alpine
container_name: cfswarm-cfml
volumes:
- type: bind
source: ./app-one
target: /app
env_file:
- ./config/cfml/simple-cfml.env
secrets:
- source: cfconfig # this isn't really a secret but non-stack deploys don't support configs so let's make it one
target: cfconfig.json
networks:
- cfswarm-simple
depends_on:
- cfswarm-mysql
- cfswarm-nginx
volumes:
- type: bind
source: ./app-one
target: /app
This directive is the same as our MySQL
volume, but instead of letting Docker create and manage the volume, we're "binding" a directory on the host into the container. This is generally not suitable for production, but if we're actively developing an app then Docker isn't much use to us if we can't read and write to our app folder while it's running. env_file:
- ./config/cfml/simple-cfml.env
In our MySQL service, we specified each environment variable on its own line. Since environment variables are likely to change more often than our service definitions, it's cleaner to store the environment variables in a separate file and just refer to that file in your
docker compose.yml. The effect is the same -- the container created for your service doesn't care whether you put your environment variables in your compose file or somewhere else, but the best practice is to keep them separate. secrets:
- source: cfconfig # this isn't really a secret but non-stack deploys don't support configs so let's make it one
target: cfconfig.json
The service-level
secrets directive refers back to the secret we defined at the top level. But what is a secret, and why are we using it?docker configsare not (yet) supported bydocker compose-- they're meant fordocker stackdeployments, which we'll use in production.docker secretsare for files containing sensitive information: credentials, encryption keys, anything that you don't want to put somewhere easily accessible (such as an environment variable or a layer of your Docker image). These are meant to be supplied to Docker once and then referenced in deployments, rather than read from a file.
Our CFConfig JSON file doesn't have anything sensitive in it -- it defines a datasource but relies on environment variables for the credentials. We could have built our own Docker image and copied it directly into the image, but it's conceivable that different services might have different CFConfig options; we also want to be able to update CFConfig without rebuilding an entire image. A
config is really what we need, but since it's not supported in docker compose, we'll use a secret instead. The syntax is nearly identical; much like volumes and networks, secrets can be shared between services, so this service-level definition only supplies the source (the name of the secret in the top-level secrets config) and the target (name of the file to mount into the /run/secrets folder in our container). depends_on:
- cfswarm-mysql
- cfswarm-nginx
depends_on tells
docker compose that this service requires other services to be running before it can start. Since our CF engine isn't much use to us without NGINX and MySQL, docker compose will start these services prior to starting cfswarm-cfml. Note that Docker doesn't check to see if the services are actually ready -- health checks are more involved and beyond the scope of our development stack. Even so, we want Docker to bring up MySQL and NGINX before starting the CF container.The NGINX container will proxy HTTP and HTTPS requests made to the host on ports 80 and 443 to one or more CF containers. There's only one new directive in this service:
cfswarm-nginx:
image: nginx
command: [nginx-debug, '-g', 'daemon off;']
container_name: cfswarm-nginx
ports:
- 80:80
- 443:443
volumes:
- type: bind
source: ./app-one
target: /var/www/app-one
- type: bind
source: ./app-two
target: /var/www/app-two
- type: bind
source: ./nginx/
target: /etc/nginx
networks:
- cfswarm-simple
Every Docker image executes a command when the container is started. This command is specified in the
Dockerfile that governs how the image is built. Our CF containers are running a script that triggers Commandbox's server start. NGINX's Dockerfile starts the container with the nginx command, but for development we'll run NGINX in DEBUG mode.We can override the
CMD directive from the Dockerfile with the command directive in docker-compose.yml:command: [nginx-debug, '-g', 'daemon off;']
This replaces the default startup command with
nginx-debug -g daemon off.