Dockerize a Rails App with MySQL and Sidekiq
Dockerising an application is creating an isolated environment with containers for different moving parts in an application which makes the development and deployment easier and you don’t have to repeat yourself everytime when you need to deploy it locally for development or in production. It’s one command away and it’s that’s easy when you dockerize it. Dockerising is fun and tricky in the beginning when you define volumes for containers and working out an architecture that’ll suit your application. Okay, let’s just stop the talking and dive right into it. I’ll first start with the introduction to Docker and what you’ll need to dockerize an application. I take up a Rails App for this blog post but you can dockerize literally anything.
The prerequisites are to install Docker and make sure it’s running. Install Ruby & Rails. Don’t worry we’re not gonna deal with ruby.
“Hey, Docker. Who are you?”
I’m not gonna describe it to you. I’ll ask Docker to do that part. It’ll be so good if he does that by himself. So here he is.
“Hey! I’m a tool designed to make it easier to
create, deploy, and run applications by using containers. A container is a part of me. A container is a standard unit of software that
packages up the code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A container image is a
lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings. By doing so, thanks to my container, you can rest assured that the application will run on any other Linux machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code. The important thing to note is that I’m Open source and you can contribute to making me better. See how awesome I’m! :D”
So, how was his introduction? Okay, Let’s move to the other aspects in detail. Let’s talk about Docker images and Containers.
Volumes, Docker Images & Containers
Volumes are the preferred mechanism for
persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure of the host machine, volumes are completely managed by Docker.
containers are based on Docker images. A Docker image is a binary that includes all of the requirements for running a single Docker container, as well as metadata describing its needs and capabilities. You can think of it as a packaging technology. Docker containers only have access to resources defined in the image, unless you give the container additional access when creating it.
Dockerfile and docker-compose.yml
A Dockerfile is used to build an image. A Compose file is used to deploy a container from an image.
If you’re using a public image, such as Nginx or MySQL, then there’s no need for a Dockerfile since you don’t need to build it yourself - it’s already built and accessible via Docker Hub, so your Compose file can just pull it from there.
You usually don’t need a Dockerfile unless you’re creating a custom container image from scratch, or customizing a public image for some reason.
So, we’ll need Dockerfile to build our custom images for the web-app, sidekiq for our ruby application. Since we have pre-built images for MySQL, Ruby, and Redis, we’ll just pull the image from Docker Hub. We’ll talk in detail about everything.
We’re half-way through and let’s get started in dockerizing a rails application.
rails new app_name
As we discussed earlier, we’ll work on a rails application. Let’s create a new application with the command
rails new rails-mysql-docker where
rails-mysql-docker is the name of the project. This will create a scaffold and once it’s complete, open the folder in a code editor. Now, let’s get the basics done. We’ll install the MySQL gem as we’ll be using MySQL as the database and sidekiq for background processing. At the later point of the blog, we’ll talk about sidekiq. Now, add the required gems to the Gemfile and run
bundle install in the command line.
Now, I assume that you’ve installed Docker on your computer and made sure that it’s up and running. We now created a rails app and installed mysql gem in it.
“It’s Docker Time!!”
We’ll now be writing the
Dockerfile and the
docker-compose.yml file. As I said previously, A Dockerfile is used to build an image. A Compose file is used to deploy a container from an image. We’ll first write the Dockerfile. The Dockerfile contains commands that need to be executed while building the image. That may include system libraries and other stuff. Create a file named
Dockerfile at the root of the project and add the following to it.
FROM ruby:2.5-alpine RUN apk update && apk upgrade && apk add ruby ruby-json ruby-io-console ruby-bundler ruby-irb ruby-bigdecimal tzdata postgresql-dev && apk add nodejs && apk add curl-dev ruby-dev build-base libffi-dev && apk add build-base libxslt-dev libxml2-dev ruby-rdoc mysql-dev sqlite-dev RUN mkdir /app WORKDIR /app COPY Gemfile Gemfile.lock ./ RUN gem install ovirt-engine-sdk -v '4.3.0' --source 'https://rubygems.org/' RUN bundle install --binstubs COPY . . EXPOSE 3000 ENTRYPOINT ["sh", "./config/docker/startup.sh"]
The above Dockerfile contains some commands that are run to build a Docker image. We’ll now break out the above docker file.
We’re asking it to use the base image
ruby for the
RUN apk update && apk upgrade && \ apk add ruby ruby-json ruby-io-console ruby-bundler ruby-irb ruby-bigdecimal tzdata && \ apk add nodejs && \ apk add curl-dev ruby-dev build-base libffi-dev && \ apk add build-base libxslt-dev libxml2-dev ruby-rdoc mysql-dev sqlite-dev
This will install the required system libraries. The above has some extra libraries which are not quite needed for a small rails application.
RUN mkdir /app WORKDIR /app
We now run a mkdir command which is used to create a new directory. The
WORKDIR line specifies a new default directory within the image’s file system which is the app directory.
COPY Gemfile Gemfile.lock ./ RUN bundle install --binstubs COPY . .
Now, copy the Gemfile, lock file and the current directory and run bundle install. When you use COPY it will copy the files from the local source, in this case
. meaning the files in the current directory, to the location defined by
WORKDIR. In the above example, the second. refers to the current directory in the working directory within the image.
EXPOSE 3000 ENTRYPOINT ["sh", "./config/docker/startup.sh"]
This will export the
port 3000 and run a startup.sh shell script. We use some shell scripts to build the application. We’ll talk about it more in some time.
ENTRYPOINT allows you to configure a container that will run as an executable. In our case, it will run
startup.sh that will build our app.
We also don’t want unwanted files inside a container. Take git tracking for example. We don’t need it to run our app. Thus, it is not needed. So, we create a
.dockerignore file that contains the following,
.dockerignore .git logs/ tmp/
The Compose file is a YAML file which defines services, networks, and volumes. It usually helps us defining the containers and, volumes with ENV variables. We’ll break out every container.
version: "3.7" services: db: image: "mysql:5.7" environment: MYSQL_ROOT_PASSWORD: root MYSQL_USERNAME: root MYSQL_PASSWORD: root ports: - "3307:3306" redis: image: "redis:4.0-alpine" command: redis-server volumes: - "redis:/data" website: depends_on: - "db" - "redis" build: . ports: - "3000:3000" environment: DB_USERNAME: root DB_PASSWORD: root DB_DATABASE: sample DB_PORT: 3306 DB_HOST: db RAILS_ENV: production RAILS_MAX_THREADS: 5 volumes: - ".:/app" - "./config/docker/database.yml:/app/config/database.yml" sidekiq: depends_on: - "db" - "redis" build: . command: sidekiq -C config/sidekiq.yml volumes: - ".:/app" environment: REDIS_URL: redis://redis:6379/0 volumes: redis: db:
“Breaking it up is better than breakup”. Ok, I get it. It’s so dumb. Now we’ll break out our docker-compose.yml.
In the docker-compose.yml file, the
version depends on the docker release. There is a table in the docker documentation that maps different version with their respective docker releases. The
services are what that makes up your application. It consists of all the services that will be built as different containers that act as the organs of your application. We’ll have about 4 services namely -
db, redis, website, sidekiq.
db: image: "mysql:5.7" environment: MYSQL_ROOT_PASSWORD: root MYSQL_USERNAME: root MYSQL_PASSWORD: root ports: - "3307:3306"
This container is for installing our MySQL and running it as a container. We pull an already built image
mysql:5.7 from docker hub. We also pass the required environment variables.
Ports mentioned in docker-compose.yml will be shared among different services started by the docker-compose. Ports will be exposed to the host machine to a random port or a given port.
redis: image: "redis:4.0-alpine" command: redis-server volumes: - "redis:/data"
Same as MySql, we pull a redis image from docker hub. Compose includes the ability to attach
volumes to any service that has persistent storage requirements. We’ll have volumes to contain our persistent data like the above. The
command will run after the container is built. Docker will create the volume for you in the
/var/lib/docker/volumes folder. This volume persists as long as you are not typing
docker-compose down -v.
website: depends_on: - "db" - "redis" build: . ports: - "3000:3000" environment: DB_USERNAME: root DB_PASSWORD: root DB_DATABASE: sample DB_PORT: 3306 DB_HOST: db RAILS_ENV: production RAILS_MAX_THREADS: 5 volumes: - ".:/app" - "./config/docker/database.yml:/app/config/database.yml"
The website service is where our rails application resides.
docker-compose up will start services in dependency order as defined with
depends_on. In the following example, db and redis will be started before the website. The build key tells to build with the docker-compose.yml present in the current directory.
sidekiq: depends_on: - "db" - "redis" build: . command: sidekiq -C config/sidekiq.yml volumes: - ".:/app" environment: REDIS_URL: redis://redis:6379/0
Sidekiq is a simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs in the same process simultaneously. It does not require Rails but will integrate tightly with Rails to make background processing dead simple. A rails app will definitely need redis and sidekiq for background processing.
Docker Utility Folder
We’ll have a docker utility folder inside config that’ll contain a
database.yml which will be later mounted into the config folder that’ll be used by the application. We’ll have four shell scripts.
If you remember the
Dockerfile (If not, you can scroll back though :p), we would’ve referred startup.sh in our ENTRYPOINT. Yes, you guessed it right. We’re nearly at the end of the tutorial.
#! /bin/sh # Wait for DB services sh ./config/docker/wait-for-services.sh # Prepare DB (Migrate - If not? Create db & Migrate) sh ./config/docker/prepare-db.sh # Pre-comple app assets sh ./config/docker/asset-pre-compile.sh # Start Application bundle exec puma -C config/puma.rb
This script will run three more scripts and starts our application. We’ll discuss that below.
This script polls and waits for the MySQL to be up and running with the help of the host and the port it is running.
#! /bin/sh # Wait for MySQL until nc -z -v -w30 $DB_HOST $DB_PORT; do echo 'Waiting for MySQL...' sleep 1 done echo "MySQL is up and running!"
This script handles the database migrations for the application. It’ll create the database and migrate if the migrations failed in the first try.
#! /bin/sh # If the database exists, migrate. Otherwise setup (create and migrate) bundle exec rake db:migrate 2>/dev/null || bundle exec rake db:create db:migrate echo "Done!"
We use rake assets:precompile to precompile our assets before pushing code to production. This command precompiles assets and places them under the public/assets directory in our Rails application.
#! /bin/sh # Precompile assets for production bundle exec rake assets:precompile echo "Assets Pre-compiled!"
As said before we’ll have the database configurations in our docker folder which will then be mounted into the config folder that’ll be then used by the rails application.
default: &default adapter: mysql2 encoding: utf8mb4 collation: utf8mb4_bin reconnect: false pool: 50 username: <%= ENV['DB_USERNAME'] %> password: <%= ENV['DB_PASSWORD'] %> port: <%= ENV['DB_PORT'] %> host: <%= ENV['DB_HOST'] %> socket: /var/run/mysqld/mysqlx.sock development: <<: *default database: <%= ENV['DB_DATABASE'] %>_development production: <<: *default database: <%= ENV['DB_DATABASE'] %>
Now, all we need to do is run
docker-compose up -d. The -d is Detached mode which will run the containers in the background. The app will be built with all the containers and you can view the containers with
docker ps -a. You can also find the GitHub Repo or follow me on Twitter for some retweet spam. In my next article, we’ll talk about CI/CD with Jenkins. Until next time, have this Gif for free.