If you work on a Mac (or Windows) and use Docker then you probably have heard of boot2docker. If you haven’t heard of it before, boot2docker is basically a super lightweight Linux VM that is designed to run Docker containers. Unfortunately there is no support (yet) in Mac OS X or Windows kernels for running Docker natively so this lightweight VM must be used as an intermediary layer that allows the host Operating Systems to communicate with the Docker daemon running inside the VM. This solution really is not that limiting once you get introduced to and become comfortable with boot2docker and how to work around some of the current limitations.
Because Docker itself is such a new piece of software, the ecosystem and surrounding environment is still expanding and growing rapidly. As such, the tooling has not had a great deal of time to mature. So with pretty much anything that’s new, especially in the software and Open Source world, there are definitely some nuances and some things to be aware of when working with boot2docker.
That being said, the boot2docker project bridges a gap and does a great job of bringing Docker to an otherwise incompatible platform as well as making it easy to use across platforms, which especially useful for furthering the adoption of Docker among Mac and Windows users.
When getting started with boot2docker, it is important to note that there are a few different things going on under the hood.
Components
The first component is VirtualBox. If you are familiar with virtual machines, there’s pretty much nothing new here. It is the underpinning of running the VM and is a common tool for creating and managing VM’s. One important note here about VBox. This is currently the key to make volume sharing work with boot2docker to allow a user to pass local directories and files in to containers using its shared folder implementation. Unfortunately it has been pretty well documented that vboxsf (shared folders) have not great performance when compared to other solutions. This is something that the boot2docker team is aware of and working on for future reference. I have a workaround that I will outline below if anyone happens to hit some of these performance issues.
The next component is the VM. This is a super light weight image based on Tiny Core Linux and the 3.16.4 Linux kernel with AUFS to leverage Docker. Other than that there is pretty much nothing else to it. The TCL image is about 27MB and boots up in about 5 seconds, making it very fast, which is nice to get going with quickly. There are instructions on the boot2docker site for creating custom .iso’s if you are interested as well if you are’t interested in building your own customized TCL.
The final component is called boot2docker-cli, which is normally referred to as the management tool. This tools does a lot of the magic to get your host talking to the VM with minimal interaction. It is basically the glue or the duct tape that allows users to pass commands from a local shell in to the container and get Docker to do stuff.
Installation
It is pretty dead simple to get boot2docker set up and configured. You can download everything in one shot from the links on their site http://boot2docker.io or you can install manually on OSX with brew and a few other tools. I highly recommend the packaged installer, it is very straight forward and easy to follow and there is a good video depiction of the process on the boot2docker site.
If you choose to install everything with brew you can use the following commands as a reference. Obviously it will be assumed that brew is already installed and set up on your OSX system. The first step is to install boot2docker.
brew install boot2docker
You might need to install Virtualbox separately using this method, depending on whether or not you already have a good version of Virtualbox to use with boot2docker.
The following commands will assume you are starting from scratch and do not have VBox installed.
brew update brew tap phinze/homebrew-cask brew install brew-cask brew cask install virtualbox
That is pretty much it for installation.
Usage
The boot2docker CLI is pretty straight forward to use. There are a bunch of commands to help users interface with the boot2docker VM from the command line. The most basic and simple usage to initialize and create a vanilla boot2docker VM can be done with the following command.
boot2docker init
This will pull down the correct image and get the environment set up. Once the VM has been created (see the tricks sections for a bit of customization) you are ready to bring up the VM.
boot2docker start
This command will simply start up the boot2docker VM and run some behind the scenes tasks to help make using the VM seamless. Sometimes you will be asked to set ENV variables here, just go ahead and follow the instructions to add them.
There are a few other nice commands that help you interact with the boot2docker VM. For example if you are having trouble communicating with the VM you can run the ip command to gather network information.
boot2docker ip
If the VM somehow gets shut off or you cannot access it you can check its status.
boot2docker status
Finally there is a nice help command that serves as a good guide for interacting with the VM in various ways.
boot2docker help
The commands listed in this section will for the most part cover 90% of interaction and usage of the boot2docker VM. There is a little bit of advanced usage with the cli covered below in the tricks section.
Tricks
You can actually modify some of the default the behavior of your boot2docker VM by altering some of the underlying boot2docker configurations. For example, boot2docker will look in $HOME/.boot2docker/profile for any custom settings you may have. If you want to change any network settings, adjust memory or cpu or a number of settings, you would simply change the profile to reflect the updated changes.
You can also override the defaults when you create your boot2docker VM by passing some arguments in. If you want to change the memory or disk size by default, you would run something like
boot2docker init --memory=4096 --disksize=60000
Notice the –disksize=60000. Docker likes to take up a lot of disk space for some of its operations, so if you can afford to, I would very highly recommending that you adjust the default disk size for the VM to avoid any strange running out of disk issues. Most Macbooks or Windows machines have plenty of extra resources and big disks so usually there isn’t a good reason to not leverage the extra horsepower for your VM.
Troubleshooting
One very useful command for gathering information about your boot2docker environment is the boot2docker config command. This command will give you all the basic information about the running config. This can be very valuable when you are trying to troubleshoot different types of errors.
If you are familiar with boot2docker already you might have noticed that it isn’t a perfect solution and there are some weird quirks and nuances. For example, if you put your host machine to sleep while the boot2docker VM is still running and then attempt to run things in Docker you may get some quirky results or things just won’t work. This is due to the time skew that occurs when you put the machine to sleep and wake it up again, you can check the github issue for details. You can quickly check if the boot2docker VM is out of sync with this command.
date -u; boot2docker ssh date -u
If you notice that the times don’t match up then you know to update your time settings. The best fix for now that I have found for now is to basically reset the time settings by wrapping the following commands in to a script.
#!/bin/sh # Fix NTP/Time boot2docker ssh -- sudo killall -9 ntpd boot2docker ssh -- sudo ntpclient -s -h pool.ntp.org boot2docker ssh -- sudo ntpd -p pool.ntp.org
For about 95% of the time skew issues you can simply run sudo ntpclient -s -h pool.ntp.org to take care of the issue.
Another interesting boot2docker oddity is that sometimes you will not be able to connect to the Docker daemon or will sometimes receive other strange errors. Usually this indicates that the environment variables that get set by boot2docker have disappeared, if you close your terminal window or something similar for example. Both of the following errors indicate the issue.
dial unix /var/run/docker.sock: no such file or directory
or
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
The solution is to either add the ENV variables back in to the terminal session by hand or just as easily modify your bashrc config file to read the values in when the terminal loads up. Here are the variables that need to be reset, or appended to your bashrc.
export DOCKER_CERT_PATH=/Users/jmreicha/.boot2docker/certs/boot2docker-vm export DOCKER_TLS_VERIFY=1 export DOCKER_HOST=tcp://192.168.59.103:2376
Assuming your boot2docker VM has an address of 192.168.59.103 and a port of 2376 for communication.
Shared folders
This has been my biggest gripe so far with boot2docker as I’m sure it has been for others as well. Mostly I am upset that vboxsf are horrible and in all fairness the boot2docker people have been awesome getting this far to get things working with vboxsf as of release 1.3. Another caveat to note if you aren’t aware is that currently, if you pass volumes to docker with “-v”, the directory you share must be located within the “/Users” directory on OSX. Obviously not a huge issue but something to be aware if you happen to have problems with volume sharing.
The main issue with vboxsf is that it does not do any sort of caching sort of caching so when you are attempting to share a large amount of small files (big git repo’s) or anything that is filesystem read heavy (grunt) performance becomes a factor. I have been exploring different workarounds because of this limitation but have not found very many that I could convince our developers to use. Others have had luck by creating a container that runs SMB or have been able to share a host directory in to the boot2docker vm with sshfs but I have not had great success with these options. If anybody has these working please let me know I’d love to see how to get them working.
The best solution I have come up with so far is using vagrant with a customized version of boot2docker with NFS support enabled, which has very little “hacking” to get working which is nice. And a good enough selling point for me is the speed increase by using NFS instead of vboxsf, it’s pretty staggering actually.
This is the project that I have been using https://vagrantcloud.com/yungsang/boxes/boot2docker. Thanks to @yungsang for putting this project together. Basically it uses a custom vagrant-box based off of the boot2docker iso to accomplish its folder sharing with the awesome customization that Vagrant provides.
To get this workaround to work, grab the vagrantfile from the link provided above and put that in to the location you would like to run Vagrant from. The magic sauce in the volume sharing is in this line.
onfig.vm.synced_folder ".", "/vagrant", type: "nfs"
Which tells Vagrant to share your current directory in to the boot2docker VM in the /vagrant directory, using NFS. I would suggest modifying the default CPU and memory as well if your machine is beefy enough.
v.cpus = 4 v.memory = 4096
After you make your adjustments, you just need to spin up the yungsang version of boot2docker and jump in to the VM.
vagrant up vagrant ssh
From within the VM you can run your docker commands just like you normally would. Ports get forwarded through to your local machine like magic and everybody is happy.
