Using Docker Containers for C++ Builds

Why Docker?

1. Allows you to use your host machine to build applications for different OSes, libraries, and complete OS setup based ecosystems after just running a script
2. Setup for a new machine is easy
3. Easy to extend yours and other’s work
4. Easy to document tools needed and dependencies
5. Easy to standardise on a Docker image across a team/company.
   This allows everyone to build on the same system without question or doubt
6. Easy to share files with the host system thanks to volumes
   This helps you gain access to the source code and specify where the generated output will be stored
7. Easy to share a Docker images (thanks to Docker Hub and Custom Docker registries some of which can be self hosted even)
8. Cost savings as no longer do you need a new machine for build and testing

With Docker, you get the full advantages of having created a new VM in its entirety (not factually true of course), while with mounting having an easy ability to share files across systems, giving you the best of both worlds while allowing you to code on a modern OS, while seamlessly allowing you to build, test and release on an older OS without requiring the setup of a new machine.

Docker Images are the VM/Machine

1. Gross oversimplification
2. Unlike a VM, building a Docker image is in my experience, more convenient.
3. Once built, you can’t change a Docker Image but you can build a new one.
4. Docker Images contain all your dependencies
5. They can be thought of as a setup for a machine, an ISO for your exe to use to run everything
6. They can be shared across the organisation
7. Can be used to standardise the setup/machine that would be used for building across the organisation
   Except everyone can run it on their own actual machine at the same time with no sharing of resources if you choose it to be.

Docker Containers are the EXE

1. Gross Oversimplification
2. Build upon the Image by actually running it
3. Can access your machine (as much as you choose to share) and are easily modified
4. Can be used as the standard system to build our OS

Docker always runs as root

1. Root is synonymous with security concerns
2. Loss of convenience
   Files made by root cannot be easily accessed or modified by non-root users due to Linux privileges

However, Docker can be configured to be run as the user of your choice. By default, it is true, that Docker prefers root though.

Why One Image Can’t Rule them all?

It would be awesome if instead of this boring article which is part philosophy/system design, we could save your time and give you a single magical command which does everything and solves all your problems.

Sadly, C++ does not work that way given its heritage and we certainly couldn’t pull off with ease with C++ which we could with Rust or Node.JS.

1. C++ ecosystem is just a bit complicated
2. No Single Build System (need to support all the build systems known in the world)
3. No Single way to manage dependencies
4. No single way to store codebases

As such, we would have to support every build system known on the planet. Not just that, we need to write our code in a way such that all of them work without issues, questions or problems.
The code could be built on a modern machine while expectation would be that support needs to exist on a legacy machine. And it all needs to work.

This requires a certain focus on styling very absent with most other tools in question of course.

What does a simple Docker Build System look like?

Let’s assume we are using Ubuntu.
For the projects in question, we have had a discussion and found out we need Python, Boost and the compilers.

So let’s see what a Docker file looks like

With these simple steps, we would be able to have a fully custom Dockerfile which supports Python3, Boost and C++ build tools.

Yes that’s it. But it’s not quiet ready for what we need yet.
Heck, all it has are the dependencies but it can’t build something or even obtain our code yet.

We already have an OS specific setup script

Let’s say you have an OS specific setup script ready already by the name of deps-install.sh which has been explicitly designed to support your app, AguriApp.exe

This script is responsible for installing and setting up all dependencies required to build AguriApp.exe

Thanks to Docker Images mapping almost very well theoretically to machines, there is a 100% chance as long as you remember to target the right OS, your scripts would work.

Not only do you have a simple way to add your dependency scripts but with Multi-Stage builds, thinks have already become so much more clearer.

Designing enterprise style

Okay, there isn’t anything special about enterprise style but what I want right now is to propose a design to you with a focus on:-

1. Ease of use
2. Easy to read
3. Easy to scale
4. Easy to modify
5. Easy to handover

So the focus here is going to split everything into separate sections.

When it comes to images, as you can see above, Multi-Stage builds are going to provide substantial assistance to you when it comes to dividing the code in sections.

But the split needs to be done in a manner that it is most configurable because, as a C++ supportive design, we need to support everything under the sun. Or at least our design does.
The end result can be quite specific to the build system you use.

Build Scripts. Why not?

Wait, before you get angry that we are going to be using Bash or Python scripts, wait for a second and think about it.

A simple Bash script can awesomely gel across builds, run unit tests (if you have any :-P), change directories and move results from Point A to Point B with more convenience than any other solution while perfectly integrating with your existing approach.

If you have experienced the horrors of what some call Makefiles, the following scripts are probably going to be simple for you to understand, implement and something that you already use at work.

The code it is pretty basic isn’t it?

1. Change to source directory
2. Perform clean and build
3. Move output to /Output directory

This script could just as easily be 100 lines of calling your custom Python builders, or your own builders or tools interfacing with different coding styles already designed by you.

The other advantages of build scripts are:-

1. Assuming that your code has been currently configured to assume a specifc path which is different in the Docker Container
2. You could run a sed command to perform the fix
3. You can use it to execute everything cleanly and efficiently.

What’s missing?

Now if you were running this within the Docker Container, you notice we need to: -

1. Have a ready image (although I guess we built one)
2. Have a way to share /source and /Output directories and mention where they point to on our file system.
3. Have a way to specify to our script where to start executing.
   Do note that in the image above, we never mention a starting point.

Using Docker Compose

Docker Compose helps us call our scripts in a manner more cleaner and easier to understand than a CLI.

Run it with

USER_ID=$(id -u) GROUP_ID=$(id -g) docker-compose up build

Imp Points

1. EntryPoint specifies what the EXE should execute when the app starts.
   This is where you want your build script.
2. local:container:type is the way volumes are configured
   Local is the path on your local machine. Hence ./code because that is where my code is
   container is the path on the machine, hence /code
   Your build script should try to find the code at /code not ./code.
   Type is more interesting.
   While there are a lot of types of access specifiers, I would recommend using either of the 2
   ro or z. With
   ro when you want Read Only access and
   z when you want to modify
3. Need to specify the 2 Environment variables
4. Using build and specifying context we can generate the image with the App.
   To do otherwise, utilise the image parameter within a Dockerfile

Managing Temporary Files

1. It is important to sometimes manage temporary files which will no doubt be generated as part of our build system
2. But would be of no use if they were to be made available on our host system
3. CMakeCache for example would be one thing that belongs to this directory
4. Temporarily generated files which will be reused across builds will be another
5. For this, we would use another Docker feature by the name of volumes.
6. The CMake example makes that a lot more clearer.

But here is a sample code

Handling ownership issues

Named volumes are by default, created as root. However, our builds have been designed primarily to be run as the same user as on the host machine.
So we need to find a way to fix this.

With this simple hack, everything starts working.

Fix thanks to Michael McQuade

Read an article by me on the same

CMake and Make

1. Let us view a sample Docker Compose which helps us add support for CMake and Make.
2. Here we are going to write a simple CMakeLists.txt based code which on configure is going to generate Makefiles for us
3. Our compose Makefile builds will depend on using these generated Makefiles.
4. We will have our own CMake Build based step which could generalise
5. The CMake Cache will be stored in Named Volumes. Because we don’t actually need to access it on our host machine.
6. The Named Volumes permission issues would have to be fixed. (Which is easy thanks to Michael’s approach)

Docker Image

We use ARG to make it easier for you to specify the version of CMake you wish to use.
Everything as is tradition with our codebases, remains pretty simple and generic.

CMakeLists.txt

Again everything is super simple.

Code

Nothing could be simpler than Hello Docker

The Compose

Okay this is a bit complicated

1. Create a named volume cmake-cache
2. Have a container to fix the permission issues for cmake-cache on non-root machines
3. Share the source code with the container
4. Have entrypoint based commands which need to be run
   The CMake Configure, Build and Makefile builder
5. Have an image (which we built using the Dockerfile above)
6. You yourself would need to write code for extracting data due to how weird CMake builds can be
   I would recommend modifying your CMakeLists.txt for the same.

CMake Configure

USER_ID=$(id -u) GROUP_ID=$(id -g) docker-compose up configure

CMake Build

USER_ID=$(id -u) GROUP_ID=$(id -g) docker-compose up build

Make Build

USER_ID=$(id -u) GROUP_ID=$(id -g) docker-compose up make-build

Repo with the CMake code

You can refer to it here.

Tips

Keep a focus on clean and easy to maintain code.

This would make it easier to understand

Never run as root

There almost is never a point to build as root.
Try to avoid doing the same.

Same Source Mount Point on Host can mount to different locations in the container

1. ./code maps to both /buildtools and /code
2. Use this.
3. This is the most important Docker feature for you as far as writing clean code is concerned.
4. If your codebase is in a monorepo fashion with dependencies, codebase, utilities etc. all as one, you could access the same directory under different mounts and use them that way
5. This would clarify exactly what the point of each and every call was
   This would help make your scripts cleaner and easier to maintain.
6. If your build script is present within ./code as mine was, this allows you to use /buildtools/build.sh which makes it abundantly clear what the task at hand is

Use ro and z access control correctly

If there is no need for your script to edit that specific mounting, disable it.
You can never be too sure.

List of dependencies

Get a list of all your organisation’s dependencies before you attempt this journey in the wasteland

Multi-Stage Build improves readability

1. Use this.
2. Improved readability makes it easier for you to work, modify and support multiple Operating Systems

Setup a Container Registry

You can either use Public paid ones or something self hosted but have a professional and easy to way to distribute Docker images within your organisation.
Otherwise the Project Unified Build, is dead before it begins.
Also if you actually call it that :-P.

Use Named Volumes for Temporary Files not required outside

This would help reduce the load on your own machine and let the EXE handle everything of value.

Perform the Named Volumes Permission Fixer

Because we run everything as non-root

Repo with a sample tutorial

You can refer here.
The README contains another attempt at a tutorial I had made.

Contact me

You can contact me via LinkedIn