I have found the root cause of a strange problem which I have encountered intermittently over the past couple years. I run my small website, and some demo websites off a single cloud instance of CoreOS. I have ran into an issue every so often where the website seems to go down for no reason. I had figured out that the website goes down when my host reboots the VM the cloud instance runs on. I had recognized that upon logging into my server, the website becomes functional again. Initially, I figured, maybe there was some sort of "lazy-loading" feature the host had, where if a cloud instance wasn't being used actively, it wasn't actually allowed to run fully. After thinking about how this would be implemented, I realized that it was unlikely, so I investigated further. What I realized, was that my Docker containers would remain inactive until a docker command was run. It did not matter what command. So for instance, if my server restarted, my website would be inaccessible, until I logged in and ran
docker ps , but after that single command, the containers would start up and I could access them.
After a bit of searching, what I figured out, is that CoreOS, despite being an OS built purely for running Docker containers, does not enable the Docker systemd service. Instead, it enables the docker socket, which runs the service on-demand any time a program (such as the docker executable) tries to access it. Apparently this is because, at one point in time, systems with very many docker images took a very long time to boot when the Docker service was enabled. The development team for CoreOS decided to disable the service, and keep the socket enabled. This doesn't cause an issue for many adminstrators, because they manage their cloud instances remotely, and upon attempting to connect to the instance, the docker service runs, essentially keeping it at maximal uptime. However, in the case of users like myself, that run a rather simple setup with just a single cloud instance that is directly managed from the server console, I had this issue where occasionally my host would reboot the machine for maintenance, and I had to start the Docker daemon myself after each reboot.
There is no obvious notice that this is the preconfigured behavior. I had to figure it out by getting errors trying to connect to my own website. The solution is very simple though. If you run CoreOS and do not control your Docker daemon remotely, this is an ideal bit of configuration:
systemctl enable docker.service
This can also be put into a cloud-config
#cloud-config coreos: units: - name: "docker.service" command: "start" enable: true
which will configure your CoreOS instance to automatically have systemd start docker every boot.