CentOS Atomic Host Customization Using cloud-init
Published on 1 Jun 2017 · Filed in Tutorial · 688 words (estimated 4 minutes to read)Back in early March of this year, I wrote a post on customizing the Docker Engine on CentOS Atomic Host. In that post, I showed how you could use systemd constructs like drop-in units to customize the behavior of the Docker Engine when running on CentOS Atomic Host. In this post, I’m going to build on that information to show how this can be done using cloud-init
on a public cloud provider (AWS, in this case).
Although I haven’t really blogged about it, I’d already taken the information in that first post and written some Ansible playbooks to do the same thing (see here for more information). Thus, one could use Ansible to do this when running CentOS Atomic Host on a public cloud provider. However, much like the original post, I wanted to find a very “cloud-native” way of doing this, and cloud-init
seemed like a pretty good candidate.
All in all, it was pretty straightforward—with one significant exception. As I was testing this, I ran into an issue where the Docker daemon wouldn’t start after cloud-init
had finished. Convinced I’d done something wrong, I kept going over the files, testing and re-testing (I’ve been working on this, off and on, since early March). Finally, I turned to the #atomic
IRC channel on Freenode, and after some help debugging the scenario we discovered that there was an unexpected interaction between how I’d configured Docker and an Atomic Host-specific script named docker-storage-setup
. As it turns out, docker-storage-setup
calls docker
in order to gather some version information. Well, docker
waits on docker-storage-setup
to finish…and thus you can see the problem. With the help of the folks in the #atomic
IRC channel, I submitted this bug to track the problem.
Fortunately, there’s a workaround (credit to Dusty Mabe for the workaround). The final version of the cloud-init
configuration looks like this (I’ll explain the workaround after this code):
#cloud-config
groups:
- docker: [centos,root]
write_files:
- content: |
[Unit]
Description=UNIX Socket for the Docker API
[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
Service=docker.service
[Install]
WantedBy=sockets.target
path: /etc/systemd/system/docker.socket
owner: root:root
permissions: '0644'
- content: |
[Unit]
Description=TCP Socket for the Docker API
[Socket]
ListenStream=2375
BindIPv6Only=both
Service=docker.service
[Install]
WantedBy=sockets.target
path: /etc/systemd/system/docker-tcp.socket
owner: root:root
permissions: '0644'
- content: |
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd-current -H fd:// \
--add-runtime docker-runc=/usr/libexec/docker/docker-runc-current \
--default-runtime=docker-runc \
--exec-opt native.cgroupdriver=systemd \
--userland-proxy-path=/usr/libexec/docker/docker-proxy-current \
$OPTIONS \
$DOCKER_STORAGE_OPTIONS \
$DOCKER_NETWORK_OPTIONS \
$ADD_REGISTRY \
$BLOCK_REGISTRY \
$INSECURE_REGISTRY
path: /etc/systemd/system/docker.service.d/docker-socket.conf
owner: root:root
permissions: '0644'
runcmd:
- [ systemctl, start, docker-storage-setup ]
- [ systemctl, mask, docker-storage-setup ]
- [ systemctl, daemon-reload ]
- [ systemctl, enable, docker.service ]
- [ systemctl, enable, docker.socket ]
- [ systemctl, enable, docker-tcp.socket ]
- [ systemctl, stop, docker.service ]
- [ systemctl, start, docker.socket ]
- [ systemctl, start, docker-tcp.socket ]
- [ systemctl, start, docker.service ]
For the most part, this is all pretty straightforward stuff—create a group, add some users to it (although it won’t add the “centos” user for some reason; still working on a resolution for that), create some systemd units, etc. The workaround is at the end:
-
First, we’ll explicitly call the
docker-storage-setup
unit, so that it goes ahead and runs. Now, because the system hasn’t yet reloaded the systemd units, the old Docker configuration is still in place, so we don’t run into the loop condition described earlier. -
Then we mask the
docker-storage-setup
unit so that it can’t be inadvertently called/launched elsewhere. -
Finally, we reload the systemd units, stop the Docker service, and then enable and restart the Docker socket units and Docker service.
There was some discussion as to whether the use of --no-block
on the final systemctl
commands was necessary (see Dusty’s post here), but additional testing showed that in this case it was not needed.
So there you have it: a way to use cloud-init
to configure Docker Engine on CentOS Atomic Host running a public cloud provider.
Additional Resources
If you’d like to play around with this yourself, check out the centos-atomic/docker-cloudinit
directory in my GitHub “learning-tools” repository. The cloud-init
configuration is there, as is a simple Bash shell script to launch an instance on AWS to try this out.
Enjoy!