Lint your Dockerfiles with Hadolint

April 13, 2016 Josh Reichardt 1 Comment

If you haven’t already gotten in to the habit of linting your Dockerfiles you should. Code linting is a common practice in software development which helps find, identify and eliminate issues and bugs before they are ever able to become a problem. One of the main benefits of linting your code is that it helps identify and eliminate nasty little bugs before they ever have a chance to become a problem.

Taking the concept of linting and appyling it to Dockerfiles gives you a quick and easy way to identify errors quickly before you ever build any Docker images. By running your Dockerfile through a linter first, you can ensure that there aren’t any structural problems with the logic and instructions specified in your Dockerfiles. Linting your files is fast and easy (as I will demonstrate) and getting in the habit of adding a linting step to your development workflow is often very useful because not only will the linter help identify hidden issues which you might not otherwise catch right away but it can potentially save hours of troubleshoot later on, so there is some pretty good effort to benefit ratio there.

There are serveral other Docker linting tools around:

But in my experience, these tools have either been overly complicated, don’t detect/catch as many errors and in general just don’t seem to work as well or have as much polish as Hadolint. I may just have a skewed perspective of these tools but this was my experience when I tried them, so take my evaluation with a grain of salt. Definitely let me know if you have experience with any of these tools, I may just need to revisit them to get a better perspective.

With that being said, Hadolint offers everything I need and is very easy to use and get started with and makes linting your Dockerfiles is trivially easy, which counts for the most points in my experience. Another bonus of Hadolint is that the project is fairly active and the author is friendly, so if there are things you’d like to see get added, it shouldn’t be too hard to get some movement on. You can check out the project on Github for details about how to install and run Hadolint.

Below, I will go over how to get setup and started as well as some basic usage.

Install

If you use Mac OS X there is a brew formula for installing Hadolint.

brew update
brew install hadolint

If you are a Windows user, for now you will have to run Hadolint from within a Docker conainer.

docker run --rm -i lukasmartinelli/hadolint < Dockerfile

If you feel comfortable with the source code you can try building the code locally. I haven’t attempted that method, so I don’t have instructions here for how to do it. Go check out the project if you are interested.

Usage

Hadolint helps you find syntax errors and other mistakes that you may not notice in your Dockerfiles otherwise. It’s easy to get started. To run Hadolint run the following.

hadolint Dockerfile

If there are any issues, Hadolint will print out the rule number as well as a blurb describing what could potentially be wrong.

DL4000 Specify a maintainer of the Dockerfile
L1 DL3007 Using latest is prone to errors if the image will ever update. Pin the version explicitly to a release tag.
L3 DL3013 Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>`

As with any linting tool, you will definitely get some false positives at some point so just be aware of items that can potentially be ignored. There is an issue open on Github right now to allow Hadolint to ignore certain rules, which will help eliminate some of the false positives. For example, in the above snippet, we don’t necessarily care about the maintainer missing so it will be nice to be able to ignore that line.

Here is a complete reference for the all of the linting rules. The wiki gives examples of what is wrong and how to fix things, which is very helpful. Additionally, the author is welcoming ideas for additional things to check, so if you have a good idea for a linting rule open up an issue.

That’s it for now. Good luck and happy Docker linting!

Useful Vim Plugins

March 1, 2016March 1, 2016 Josh Reichardt Leave a comment

This post is mostly a reference for folks that are interested in adding a little bit of extra polish and functionality to the stock version of Vim. The plugin system in Vim is a little bit confusing at first but is really powerful once you get past the initial learning curve. I know this topic has been covered a million times but having a centralized reference for how to set up each plugin is a little bit harder to find.

Below I have highlighted a sample list of my favorite Vim plugins. I suggest that you go try as many plugins that you can to figure out what suits your needs and workflow best. The following plugins are the most useful to me, but certainly I don’t think will be the best for everybody so use this post as a reference to getting started with plugins and try some out to decide which ones are the best for your own environment.

Vundle

This is a package manager of sorts for Vim plugins. Vundle allows you to download, install, search and otherwise manage plugins for Vim in an easy and straight forward way.

To get started with Vundle, put the following configuration at THE VERY TOP of your vimrc.

set nocompatible              " be iMproved, required
filetype off                  " required
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#rc()
"" let Vundle manage Vundle
Bundle 'gmarik/Vundle.vim'
...

Then you need to clone the Vundle project in to the path specified in the vimrc from above.

git clone https://github.com/gmarik/Vundle.vim.git ~/.vim/bundle/Vundle.vim

Now you can install any other defined plugins from within Vim by running :BundleInstall. This should trigger Vundle to start downloading/updating its list of plugins based on your vimrc.

To install additional plugins, update your vimrc with the plugins you want to install, similar to how Vundle installs itself as shown below.

"" Example plugin
Bundle 'flazz/vim-colorschemes'

Color Schemes

Customizing the look and feel of Vim is a very personal experience. Luckily there are a lot of options to choose from.

The vim-colorschemes plugin allows you to pick from a huge list of custom color schemes that users have put together and published. As illustrated above you can simply add the repo to your vimrc to gain access to a large number of color options. Then to pick one just add the following to your vimrc (after the Bundle command).

colorscheme xoria256

Next time you open up Vim you should see color output for the scheme you like.

Syntastic

Syntastic is a fantastic syntax highlighter and linting tool and is easily the best syntax checker I have found for Vim. Syntastic offers support for tons of different languages and styles and even offers support for third party syntax checking plugins.

Here is how to install and configure Syntastic using Vundle. The first step is to ddd Syntastic to your vimrc,

" Syntax highlighting
 Bundle 'scrooloose/syntastic'

There are a few basic settings that also need to get added to your vimrc to get Syntastic to work well.

" Syntastic statusline
 set statusline+=%#warningmsg#
 set statusline+=%{SyntasticStatuslineFlag()}
 set statusline+=%*
 " Sytnastic settings
 let g:syntastic_always_populate_loc_list = 1
 let g:syntastic_auto_loc_list = 1
 let g:syntastic_check_on_open = 1
 let g:syntastic_loc_list_height=5
 let g:syntastic_check_on_wq = 0
 " Better symbols
 let g:syntastic_error_symbol = 'XX'
 let g:syntastic_warning_symbol = '!!'

That’s pretty much it. Having a syntax highlighter and automatic code linter has been a wonderful boon for productivity. I have saved myself so much time chasing down syntax errors and other bad code. I definitely recommend this tool.

YouCompleteMe

This plugin is an autocompletion tool that adds tab completion to Vim, giving it a really nice IDE feel. I’ve only tested YCM out for a few weeks now but have to say it doesn’t seem to slow anything down very much at all, which is nice. An added bonus to using YCM with Syntastic is that they work together so if there are problems with the functions entered by YCM, Syntastic will pick them up.

Here are the installation instructions for Vundle. The first thing you will need to do is add a Vundle reference to your vimrc.

"" Autocomplete
Bundle 'Valloric/YouCompleteMe'

Then, in Vim, run :BundleInstall – this will download the git repo for YouCompleteMe. Once the repo is downloaded you will need a few other tools installed to get things working correctly. Check the official documentation for installation instruction for your system. On OS X you will need to have Python, cmake, MacVim and clang support.

xcode-select --install
brew install cmake

Then, to install YouCompleteMe.

cd ~/.vim/bundle/YouCompleteMe
git submodule update --init --recursive (not needed if you use Vundle)
./install.py --clang-completer

vim-better-whitespace

Highlights pesky whitespace automatically. This one is really useful to just have on in the background to help you catch whitespace mistakes. I know I make a lot of mistakes with regards to missing whitespace so having this is just really nice.

To install it.

"" Whitespace highlighting
Bundle 'ntpeters/vim-better-whitespace'

That’s it. Vundle should handle the rest.

ctrlp / nerdtree

These tools are useful for file management and traversal. These plugins become more powersful when you work with a lot of files and move around different directories a lot. There is some debate about whether or not to use nerdtree in favor of the built in netrw. Nonetheless, it is still worth checking out different file browsers and see how they work.

Check out Vim Unite for a sort of hybrid file manager for fuzzy finding like ctrlp with additional functionality, like the ability to grep files from within Vim using a mapped key.

Bonus – Shellcheck

This is a shell and bash linting tool that integrates with vim and is great. Bash is notoriously difficult to read and debug and the shellcheck tools helps out with that a lot.

Install shellcheck on your system and syntastic will automatically pick up the installation and automatically do its linting whenever you save a file. I have been writing a lot of bash lately and the shellcheck tool has been a godsend for catching mistakes, and especially useful in Vim since it runs all the time.

By combining the powers of a good syntax highlighter and a good solid understanding of Bash you should be able to be that much more productive once you get used to having a build in to syntax and style checker for your scripts.

Thoughts on Working Remotely

September 24, 2015 Josh Reichardt 5 Comments

I’d like to share a few nuggets that I have learned so far in my experience working as a remote employee. I have been working from home for around a year and a half and have learned some lessons in my experience thus far. While I absolutely recommend trying the remote option if possible, there are a few things that are important to know.

Working remotely is definitely not for everybody. In order to be an effective remote employee you have to have a certain amount of discipline, internal drive and self motivation. Additionally, you need to be a good communicator (covered below). If you have trouble staying on task or finding things to do at work or even have issues working by yourself in an isolated environment, you will quickly discover that working remotely may be more stressful than working in an office where you get the daily interactions and guidance from others.

Benefits

That being said, I feel that in most cases, the positives outweigh the negatives. Below are a few of the biggest benefits that I have discovered.

There is little to no commute. Long commutes, especially in big cities create a certain amount of stress and strain that you simply don’t have to deal with when working from home. As a bonus you save some cash on gas and miles of wear and tear on your vehicle.
It is easier to avoid distractions. This of course depends on how you handle your work but if you are disciplined it becomes much easier to get work done with less distractions. At home, if you manage to separate home from work (more on that topic below) then you don’t need to worry about random people stopping over to your desk to shoot the shit or bother you. By avoiding simple distractions you can become much more productive in shorter periods of time.
No dress code. This is a surprisingly simple but powerful bonus to working from home. Having a criteria for dress code was actually stressful for me in previous jobs. I always disagreed with having a dress code and didn’t understand why I couldn’t wear a t-shirt and jeans to work. Now that I can wear whatever I want I feel more comfortable and more relaxed which leads to better productivity.
Schedule can be more flexible. I can pick my own hours for the most part. Obviously it is best to get in to a routine of working the same hours each day but if something comes up I can step out for a few hours and just make the hours up in the evening most in most cases and it won’t be a big deal to coworkers. This flexibility is a great perk to working remotely and it allows you much more time to yourself when needed because you aren’t restricted to a set schedule.

Achieving a Work/life balance

Maintaining a balance between life at home and life at work can get very blurry when working from home as a telecommuter. I would argue that finding a balance between personal life and work is the number one most important thing to work towards when making the transition from an on site employment because it directly leads to your happiness (or sorrow), which in turn influences all other aspects of your life, including activities and relationships outside of work.

It is super easy to get in to the habit of “always being around” and working extra and often time crazy hours when you are at home. One thing that has helped in my own experience to improve the work/life balance and alleviate this always working thing is by creating routines.

I try to start work and end work at the same time of the day each day during the week. Likewise, I make a point to take breaks throughout the day to break up the time. A few things I like to do are take a 30ish minute walk around the same time every day and I also have a coffee ritual in the morning that always precedes work time. These daily cues help me get in to the flow of the day and to get my day started the same way every day.

Another mechanism I have discovered to help cope with the work hours is to leave work at work. Find a way to create clear distinctions between home and work, either by creating an office at home where work stays or consider finding a coffee shop or co-working space. As a side note, I have found 2-3 days working at a coffee shop/co-working space to be the best middle ground for me, but everybody is different so if you are new to remote work you will need to experiment. That way you can have a place that represents what a workplace should be, and you when you leave that place, the work stays there. It is very important to separate home from work if you don’t have a clear distinction between the two.

Some folks mention that it can get lonely. I definitely agree with this sentiment. On the up side, working in this type of environment can sort of force you to find ways to interact with people. It can feel uncomfortable at first, but finding social activities will help alleviate the loneliness. Coffee shops and co-working spaces are a great place to start. I find that working in an environment with others helps mix things up and having the extra interaction really helps feeling like you are a part of a community. These environments are a great solution if you are introverted and have a hard time getting out and meeting people.

Regardless of what exactly you do, it is absolutely critical to get out of your house. This should be a no brainer but I can’t stress the importance enough. Even if you’re just taking walks or going to the store, you need to make sure that you find things to do to get out of the house. I have found some things that work but it is something again that you will need to experiment with.

If you are ambitious then I suggest getting involved in some other communities outside of work. Meeting new people (outside of a work environment) is a very powerful tool in managing your work/life balance. Obviously this advice works as well in more scenario’s than working remotely but I think it becomes much more important. If you want some ideas for ways to get out or communities to join, feel free to email or comment and I can let you know what has worked for me.

Communicate

Another important piece of the social aspect that I have discovered is that it is VERY important to have many open communication channels with coworkers. Google Hangouts, Slack, Screenhero, WebEx, Skype, email, IRC and any other collaboration tools you can find are super important for communicating with coworkers and for building relationships and culture in distributed work environments. In my experience, if you are working as part of a team and aren’t a great communicator, relationships with coworkers can quickly become strained.

Also, having regular meetings with key members of your team is important. A nice once a week check in with any managers is a good starting point. It helps you keep track of what you’re doing and it helps others on your team understand the type of work you’re doing so you’re not as isolated. Gaining the trust of your coworkers is always very important.

Conclusion

The most difficult balance to achieve when transitioning to a work from home opportunity for me, was maintaining a good work/life balance. You are 100% responsible for how you choose to spend your time so it becomes important to make the right decisions when it comes to how to prioritize.

For example, one thing I have struggled with is how to work the right amount of time. There was a stretch where I was working 12-14 days just because I kept finding more and more things to do. While that is good for your employer, it is not good for you or anyone around you. The work will always be there, so you have to find strategies to help you step away from work when you have put in enough hours for the day.

Everybody is different so if you are new to telecommuting/working remotely I encourage you to experiment with different techniques for managing your work/life balance. While I feel that working remotely is for the most part a bonus, it still has its own set of issues so please be careful and don’t work too much, and especially don’t expend extra energy or get too stressed out about things you can’t control.

CLI hotkey and navigation guide

November 24, 2014August 31, 2015 Josh Reichardt 1 Comment

I have been meaning to write this post for quite a while now but have always managed to forget. I have been piecing together useful terminal shortcuts, commands and productivity tools since I started using Linux back in the day. If you spend any amount of time in the terminal you should hopefully know about some of these tricks already but more importantly, if you’re like me, are always looking for ways to improve the efficiency of your bash workflow and making your life easier.

There are a few things that I would quickly like to note. If you use tmux as your CLI session manager you may not be able to use some of the mentioned hotkeys to get around by default if you don’t have some settings turned on in your configuration file.

You can take a look at my custom .tmux.conf file if you’re interested in screen style bindings plus configuration for hotkeys. If you simply want to add the option that turns on the correct hotkey bindings for your terminal, add this line to your ~/.tmux.conf file

set-window-option -g xterm-keys on

Also, if you are a Mac user, and don’t already know about it, I highly recommend checking out iTerm2. Coming primarily from a Linux background the hotkey bindings in Mac OS X are a little bit different than what I am used to and were initially a challenge for me to get accustomed to. The transition for me took a little bit but iTerm has definitely helped me out immensely, as well as a few other ticks learned along the way. I really haven’t dug through all the options in iTerm but there are a huge number of options and customizations that can made.

The only thing I have been interested in so far is the navigation which I will highlight below.

Adjust iTerm keybindings – As I mentioned, I am used to using Linux keybinding so a natural fit for my purposes is the option key. The first step is to disable the custom binding in the iTerm preferences. To do this, click iTerm -> Preferences -> Profiles -> Keys and find the binding for option left arrow and option right arrow and remove them from the default profile.

Next, add the following to your global key bindings, iTerm -> Preferences -> Keys.

Move left one word

Keyboard shortcut: ??
Action: Send Escape Sequence
Escape: b

Move right one word

Keyboard shortcut: ??
Action: Send Escape Sequence
Escape: f

Finally, it is also worth pointing out that I use zsh for my default shell. There are some really nice additions that zsh offers over vanilla bash. I recently ran across this blog post which has some awesome tips. I have also written about switching to zsh here. Anyway, here is the lis. It will grow as I find more tips.

Basic navigation:

Ctrl-left/right arrow – Jump between words quickly.
Opt-left/right arrow – Custom iTerm binding for jumping between words quickly.
Alt-left/right arrow – Linux only. Jump between words quickly.
Esc-b/f – Mac OS. Similar to arrow keys, move between words quickly.
Alt-b – Linux only. Jump back one word. Handy with other hotkeys overridden.
Ctrl-a – Jump to the beginning of a line (doesn’t work with tmux mappings).
Ctrl-e – Jump to the end of a line.
End – SImilar to ctrl-e this will send your cursor to the end of the line.
Home – Similar to End, except jumps to the beginning of the line.

Intermediate navigation:

Ctrl-u – Copy entire command to clipboard.
Ctrl-y – Paste previously copied ctrl-u command in to the terminal.
Ctrl-w – Cut a word to the left of the cursor.
Alt-d – Cut after word after the cursor position

Advanced use:

Ctrl-x Ctrl-e – Zsh command. Edit the current command in your $EDITOR, which for me is vim
Ctrl-r – Everybody hopefully should know this one. It is basically recursive search history
Ctrl-k – Erase everything after the current cursor position. Handy for long commands
!<command>:p – Print the last command
cd … – Zsh command. This can be easily aliased but will jump up two directories
!$ – Quickly access the last argument of the last command.

Zsh tab completion

Tab completion with Zsh is awesome, it’s like bash completion on steroids. I will attempt to highlight some of my favorite tab completion tricks with Zsh.

Directory shorthand – Say you need to get to a directory that is nested deeply. You can use the first few characters that will uniquely match to that directory to navigate instead of typing out the whole command. So for example, cd /u/lo/b will expand out to /usr/local/bin.

command specific history – This one comes in handy all the time. If you need to grab a command that you don’t use very often you can user Ctrl-r to match the first part of the command and from there you can up arrow to locate the command you typed.

Spelling and case correction – Bash by default can get annoying if you have a long command typed out but somehow managed to typo part of the command. In zsh this is (sometimes) corrected for you automatically when you <tab> to complete the command. For example if you are changing dirs in to the ‘Documents’ directory you can type ‘cd ~/doc/’ and the correct location will be expanded for you.

This list will continue to grow as I find more handy shortcuts, hotkeys or generally other useful tips and tricks that I find in my day to day command line work. I really want to build a similar list for things in Vim but my Vim skills are unfortunately lacking plus there is already some really nice documentation and guidance out there already. If you are interested in writing up a Vim productivity post I would love to post it. Likewise, if you have any other nice shortcuts or tips you think are worth mentioning, post them in the comments and I will try to get them added to the list.

DevOpsDays Chicago

October 12, 2014August 31, 2015 Josh Reichardt 16 Comments

As a first timer to this event, and first timer to any devopsdays event I’d like to write up a quick summary of the event and write about a few of the key takeaways that I got from the event. For anybody that isn’t familiar, the devopsdays events are basically 2 day events spread out through the year at different locations around the world. You can find more information on their site here:

http://devopsdays.org/

The nice thing about the devopsdays events is that they are small enough that they are very affordable ($100) if you register early. One unique thing about the events is their format, which I really liked. Basically the first half of the day is split up in to talks given by various leaders in the industry followed by “Ignite” talks which are very brief but informative talks on a certain subject, followed by “open spaces”, which are pretty much open group discussions about topics suggested by event participants. I spoke to a number of individuals that enjoyed the open spaces, even though they didn’t like all of the subjects covered in the talks. So I thought there was a very nice balance to the format of the conference and how everything was laid out.

I noticed that a number of the talks focused on broad cultural topics as well as a few technical subjects as well. Even if you don’t like the format or topics you will more than likely at least learn a few things from speakers or participants that will help you moving forward in your career. Obviously you will get more out of a conference if you are more involved so go out of your way to introduce yourself and try to talk to as many people as you can. The hallway tracks are really good to introduce yourself and to meet people.

So much to learn

One thing that really stood out to me was how different the composition and background of attendees really was. I met people from gigantic organizations all the way down to small startup companies and basically everything in between. The balance and mixture of attendees was really cool to see and it was great to get some different perspective on different topics.

Another fact that really stood out to me was that the topics covered were really well balanced, although some may disagree. I thought heading in to the conference that most of the talks were going to be super technical in nature but it turned out a lot these talks revolved more around the concepts and ideas that sort of drive the DevOps movement rather than just tools that are associated with DevOps.

One massive takeaway that I got from the conference was that DevOps is really just a buzzword. The definition that I Have of DevOps at my organization may be totally different than somebody else’s definition at a different company. What is important is that even though there will be differences in implementation at different scales, a lot of the underlying concepts and ideas will be similar and can be used to drive change and improve processes as well as efficiency.

DevOps is really not just about a specific tool or set of tools you may use to get something accomplished, it is more complicated than that. DevOps is about solving a problem or set of problems first and foremost, the tooling to do these tasks is secondary. Before this event I had these two distinguishing traits of DevOps backwards, I thought I could drive change with tools, but now I understand that it is much more important to drive the change of culture first and then to retool you environment once you have the buy in to do so.

Talk to people

One of the more underrated aspects of this conference (and any conference for that matter) is the amount of knowledge you can pick up from the hallway track. The hallway track is basically just a way to talk to people that you may or may not have met yet who are doing interesting work or have solved problems that you are trying to solve. I ran in to a few people who were working on some interesting challenges *cough* docker *cough* in the hallway track and I really got a chance for the first time to see what others are doing with Docker which I thought was really cool.

Open spaces were another nice way to get people to intermingle. The open spaces allowed people working on similar issues to put heads together to discuss specific topics that attendees either found interesting or were actively working on. A lot of good discussion occurred in these open spaces and a good amount of knowledge was spread around.

Conclusion

DevOps is not one thing. It is not a set of tools but rather a shift in thinking and therefore involves various cultural aspects, which can get very complicated. I think in the years to come, as DevOps evolves, a lot more of these aspects will become much more clear and will hopefully make it easier people to get involved in embracing the changes that come along with the DevOps mentality.

Current thought leaders in the DevOps space (many of them in attendance at devopsdays Chichago) are doing a great job of moving the discussion forward and there are some awesome discussions at the devopsdays events. Podcasts like Arrested DevOps, The Ship Show and DevOps Cafe are definitely creating a lot of good discussion around the subject as well. Judging from my observations from the event, it seems there is still a lot of work to be done before DevOps becomes more common and mainstream.