Category Archives: Productivity

CLI hotkey and navigation guide

I have been meaning to write this post for quite a while now but have always managed to forget.  I have been piecing together useful terminal shortcuts, commands and productivity tools since I started using Linux back in the day.  If you spend any amount of time in the terminal you should hopefully know about some of these tricks already but more importantly, if you’re like me, are always looking for ways to improve the efficiency of your bash workflow and making your life easier.

There are a few things that I would quickly like to note.  If you use tmux as your CLI session manager you may not be able to use some of the mentioned hotkeys to get around by default if you don’t have some settings turned on in your configuration file.

You can take a look at my custom .tmux.conf file if you’re interested in screen style bindings plus configuration for hotkeys.  If you simply want to add the option that turns on the correct hotkey bindings for your terminal, add this line to your ~/.tmux.conf file

set-window-option -g xterm-keys on

Also, if you are a Mac user, and don’t already know about it, I highly recommend checking out iTerm2.  Coming primarily from a Linux background the hotkey bindings in Mac OS X are a little bit different than what I am used to and were initially a challenge for me to get accustomed to.  The transition for me took a little bit but iTerm has definitely helped me out immensely, as well as a few other ticks learned along the way.  I really haven’t dug through all the options in iTerm but there are a huge number of options and customizations that can made.

The only thing I have been interested in so far is the navigation which I will highlight below.

Adjust iTerm keybindings – As I mentioned, I am used to using Linux keybinding so a natural fit for my purposes is the option key.  The first step is to disable the custom binding in the iTerm preferences.  To do this, click iTerm -> Preferences -> Profiles -> Keys and find the binding for option left arrow and option right arrow and remove them from the default profile.

Next, add the following to your global key bindings, iTerm -> Preferences -> Keys.

iterm2

 

 

 

Move left one word

  • Keyboard shortcut: ??
  • Action: Send Escape Sequence
  • Escape: b

Move right one word

  • Keyboard shortcut: ??
  • Action: Send Escape Sequence
  • Escape: f

Finally, it is also worth pointing out that I use zsh for my default shell.  There are some really nice additions that zsh offers over vanilla bash.  I recently ran across this blog post which has some awesome tips.  I have also written about switching to zsh here.  Anyway, here is the lis.  It will grow as I find more tips.

Basic navigation:

  • Ctrl-left/right arrow – Jump between words quickly.
  • Opt-left/right arrow – Custom iTerm binding for jumping between words quickly.
  • Alt-left/right arrow – Linux only.  Jump between words quickly.
  • Esc-b/f – Mac OS.  Similar to arrow keys, move between words quickly.
  • Alt-b – Linux only.  Jump back one word.  Handy with other hotkeys overridden.
  • Ctrl-a – Jump to the beginning of a line (doesn’t work with tmux mappings).
  • Ctrl-e – Jump to the end of a line.
  • End – SImilar to ctrl-e this will send your cursor to the end of the line.
  • Home – Similar to End, except jumps to the beginning of the line.

Intermediate navigation:

  • Ctrl-u – Copy entire command to clipboard.
  • Ctrl-y – Paste previously copied ctrl-u command in to the terminal.
  • Ctrl-w – Cut a word to the left of the cursor.
  • Alt-d – Cut after word after the cursor position

Advanced use:

  • Ctrl-x Ctrl-e – Zsh command.  Edit the current command in your $EDITOR, which for me is vim
  • Ctrl-r – Everybody hopefully should know this one.  It is basically recursive search history
  • Ctrl-k – Erase everything after the current cursor position.  Handy for long commands
  • !<command>:p – Print the last command
  • cd … – Zsh command.  This can be easily aliased but will jump up two directories
  • !$ – Quickly access the last argument of the last command.

Zsh tab completion

Tab completion with Zsh is awesome, it’s like bash completion on steroids.  I will attempt to highlight some of my favorite tab completion tricks with Zsh.

Directory shorthand – Say you need to get to a directory that is nested deeply.  You can use the first few characters that will uniquely match to that directory to navigate instead of typing out the whole command.  So for example, cd /u/lo/b will expand out to /usr/local/bin.

command specific history – This one comes in handy all the time.  If you need to grab a command that you don’t use very often you can user Ctrl-r to match the first part of the command and from there you can up arrow to locate the command you typed.

Spelling and case correction – Bash by default can get annoying if you have a long command typed out but somehow managed to typo part of the command.  In zsh this is (sometimes) corrected for you automatically when you <tab> to complete the command.  For example if you are changing dirs in to the ‘Documents’ directory you can type ‘cd ~/doc/’ and the correct location will be expanded for you.

This list will continue to grow as I find more handy shortcuts, hotkeys or generally other useful tips and tricks that I find in my day to day command line work.  I really want to build a similar list for things in Vim but my Vim skills are unfortunately lacking plus there is already some really nice documentation and guidance out there already.  If you are interested in writing up a Vim productivity post I would love to post it.  Likewise, if you have any other nice shortcuts or tips you think are worth mentioning, post them in the comments and I will try to get them added to the list.

DevOpsDays Chicago

devopsdays chicago

As a first timer to this event, and first timer to any devopsdays event I’d like to write up a quick summary of the event and write about a few of the key takeaways that I got from the event.  For anybody that isn’t familiar, the devopsdays events are basically 2 day events spread out through the year at different locations around the world.  You can find more information on their site here:

http://devopsdays.org/

The nice thing about the devopsdays events is that they are small enough that they are very affordable ($100) if you register early.  One unique thing about the events is their format, which I really liked.  Basically the first half of the day is split up in to talks given by various leaders in the industry followed by “Ignite” talks which are very brief but informative talks on a certain subject, followed by “open spaces”, which are pretty much open group discussions about topics suggested by event participants.  I spoke to a number of individuals that enjoyed the open spaces, even though they didn’t like all of the subjects covered in the talks.  So I thought there was a very nice balance to the format of the conference and how everything was laid out.

I noticed that a number of the talks focused on broad cultural topics as well as a few technical subjects as well.  Even if you don’t like the format or topics you will more than likely at least learn a few things from speakers or participants that will help you moving forward in your career.  Obviously you will get more out of a conference if you are more involved so go out of your way to introduce yourself and try to talk to as many people as you can.  The hallway tracks are really good to introduce yourself and to meet people.

So much to learn

One thing that really stood out to me was how different the composition and background of attendees really was.  I met people from gigantic organizations all the way down to small startup companies and basically everything in between.  The balance and mixture of attendees was really cool to see and it was great to get some different perspective on different topics.

Another fact that really stood out to me was that the topics covered were really well balanced, although some may disagree.  I thought heading in to the conference that most of the talks were going to be super technical in nature but it turned out a lot these talks revolved more around the concepts and ideas that sort of drive the DevOps movement rather than just tools that are associated with DevOps.

One massive takeaway that I got from the conference was that DevOps is really just a buzzword.  The definition that I Have of DevOps at my organization may be totally different than somebody else’s definition at a different company.  What is important is that even though there will be differences in implementation at different scales, a lot of the underlying concepts and ideas will be similar and can be used to drive change and improve processes as well as efficiency.

DevOps is really not just about a specific tool or set of tools you may use to get something accomplished, it is more complicated than that.  DevOps is about solving a problem or set of problems first and foremost, the tooling to do these tasks is secondary.  Before this event I had these two distinguishing traits of DevOps backwards, I thought I could drive change with tools, but now I understand that it is much more important to drive the change of culture first and then to retool you environment once you have the buy in to do so.

Talk to people

One of the more underrated aspects of this conference (and any conference for that matter) is the amount of knowledge you can pick up from the hallway track.  The hallway track is basically just a way to talk to people that you may or may not have met yet who are doing interesting work or have solved problems that you are trying to solve.  I ran in to a few people who were working on some interesting challenges *cough* docker *cough* in the hallway track and I really got a chance for the first time to see what others are doing with Docker which I thought was really cool.

Open spaces were another nice way to get people to intermingle.  The open spaces allowed people working on similar issues to put heads together to discuss specific topics that attendees either found interesting or were actively working on.  A lot of good discussion occurred in these open spaces and a good amount of knowledge was spread around.

Conclusion

DevOps is not one thing.  It is not a set of tools but rather a shift in thinking and therefore involves various cultural aspects, which can get very complicated.  I think in the years to come, as DevOps evolves, a lot more of these aspects will become much more clear and will hopefully make it easier people to get involved in embracing the changes that come along with the DevOps mentality.

Current thought leaders in the DevOps space (many of them in attendance at devopsdays Chichago) are doing a great job of moving the discussion forward and there are some awesome discussions at the devopsdays events.  Podcasts like Arrested DevOps, The Ship Show and DevOps Cafe are definitely creating a lot of good discussion around the subject as well.  Judging from my observations from the event, it seems there is still a lot of work to be done before DevOps becomes more common and mainstream.

Transitioning from bash to zsh

oh-my-zsh

I have know about zsh for a long time now but have never really had a compelling reason to switch my default shell from bash until just recently, I have been hearing more and more people talking about how powerful and awesome zsh is.  So I thought I might as well take the dive and get started since that’s what all the cool kids seem to be doing these days.  At first I thought that changing my shell was going to be a PITA with all the customizations and idiosyncrasies that I have grown accustomed to using bash but I didn’t find that to be the case at all when switching to zsh.

First and foremost, I used a tool called oh-my-zsh to help with the transition.  If you haven’t heard about it yet, oh-my-zsh aims to be a sort of framework for zsh.  This project is a nice clean way to get started with zsh because it give you a nice set of defaults out of the the box without having to do much configuration or tweaking and I found that many of my little tricks and shortcuts were already baked in to to oh-my-zsh, along with a ton of other settings and customizations that I did not have using bash.

From their github page:

oh-my-zsh is an open source, community-driven framework for managing your ZSH configuration. It comes bundled with a ton of helpful functions, helpers, plugins, themes, and few things that make you shout…

Here are just a few of the improvement that zsh/oh-my-zsh offer:

  • Improved tab completion
  • persistent history across all shells
  • Easy to use plugin system
  • Easy to use theme system
  • Autocorrect

The most obvious difference that I have noticed is the improved, out of the box tab completion, which I think should be enough on its own to convince you!  On top of that, most of my tricks and customizations were already turned on with oh-my-zsh.  Another nice touch is that themes and plugins come along as part of the package, which is really nice for easing the transition.

So after spending an afternoon with zsh I am convinced that it is the way to go (at least for my own workfolw).  Of course there are always caveats and hiccups along the way as I’ve learned there are with pretty much everything.

Tuning up tmux

Out of the box, my tmux config uses the default shell, which happens to be bash.  So I needed to modify my ~/.tmux.conf to reflect the switch over the zsh.  It is a pretty straight forward change but is something that you will need to make note of kif you use tmux and are transitioning in to using zsh.

set-option -g default-shell /usr/bin/zsh

I am using Ubuntu 14.04, so my zsh is installed to /usr/bin/zsh.  The other thing that you will need to do is make sure you kill any stale tmux processes after updating to zsh.  I found one running in the background that was blocking me from using the new coonfig.

Goodies

There is a nice command cheat sheet for zsh.  Take some time to learn these shortcuts and aliases, they are great time savers and are very usefull.

oh-my-zsh comes bundled up with a large number of goodies.  At the time of this writing there were 135 plugins as well as a variety of themes.  You can check the plugins wiki page for descriptions for the various plugins.  To turn on a specific plugin you will need to add it to your ~/.zshrc config file.  Find the following line in your config.

plugins=(git)

and add plugins separated by spaces

plugins (git vagrant chef)

You will need to reload the config for the changes to be picked up.

source ~/.zshrc

Most themes are hosted on the wiki, but there is also a web site dedicated to displaying the various themes, which is really cool.  It does a much better job of showing differences between various themes.  You can check it out here.  Themes function in a similar way to plugins.  If you want to change your theme, edit your ~/.zshrc file and select the desired them.

ZSH_THEME="clean"

You will need to reload your config for this option as well.

source ~/.zshrc

Conclusion

If you haven’t already made the switch to zsh I recommend that you at least experiment and play around with it before you make any final decisions.  You may be set in your ways and happy with bash or any other shell that you are used to but for me, all the awesomeness changed my opinion and decide to reevaluate my biases.  If you’re worried about making the switchin, using oh-my-zsh makes the transition so painless there is practically no reason not to try it out.

This post is really just the tip of the iceberg for the capabilities of this shell, I just wanted to expose readers to all of its glory.  Zsh offers so much more power and customization than I have covered in this post and is an amazing productivity tool with little learning overhead.

Let me know if you have any awesome zsh tips or tweaks that folks should know about.

7 useful but hard to remember Linux commands

I have found myself using these commands over and over so I decided I’d take the time to go ahead and document them for future me as well as readers because I find these commands pretty useful.  I just always manage to forget them, hence the title of the post.  The smart thing to do would be to create aliases for these commands but I have just been too lazy and some of them are run across different servers so it isn’t always a convenient option.

Anyway, let’s go ahead and run through the commands before I forget…

1) du -ah / | sort -n -r | head -n 50

This one is really handy for debugging space issues.  It will list the top 50 files according to file size, with the largest at the top of the list.Notice the “/” will specify the location to search so you can easily modify this one to search different locations, like “/var/log” for example if you are having trouble with growing log files.

1.5) du -sh /*

This will quickly give you an idea of how how your disk space has been allocated.  Definitely handy when you are troubleshooting.

2) git checkout — .

I don’t use this one very often, which is probably why I manage to forget it so easily.  But I really like it.  Sometimes I will be working on a git repo across different machines at the same time and will run in to conflicts committing to the repo or more likely I committed changes on one machine and just need to pull down the newest changes but can’t since I have made modifications.  For those scenarios you can run the above command quickly reset your git changes quickly and easily.

3) tmux kill-window -t 3

i use tmux for my terminal and window manager on all my workstations and love it.  If you haven’t heard of it, take a look here.  Sometimes the sessions can get stuck so it becomes necessary to close the window without destroying the tmux session.  Again, this doesn’t happen very often so it is sometimes hard for me to remember the exact syntax but this one is a handy little trick for managing tmux windows and sessions.

4) grep -r “text”

I know, I should really have this one memorized by now.  I am trying to remember but I don’t find myself using this one all that often even though it is really powerful and useful.  This will essentially search through every file recursively and spit out the text pattern that you feed to it.

5) kill $(pgrep process)

This one is handy when there are a large number of stuck processes and you need to blow them all out with one command.  For example if the chrome browser ever gets stuck with a million tabs open, there are likely a large number of processes all with the same – or similar names.  If you pass all or part of the process name in to this command pgrep will find them and kill will destroy them

6) docker rm $(docker ps -a -q)

I have been using Docker more and more recently and every once in awhile I find myself with a large number of dead Docker processes that need to be cleaned up.  This command will blow out all of these stale processes at once.  This is nice because Docker processes take up a large amount of disk space and often times can fill up your drives without you being aware.  I have been able to reclaim large amounts of disk space with this command.

7) watch -n 10 df -ah

This is another good one for checking disk space issues.  It will update you every ten seconds with the disk utilization of the system.  Pretty straight forward but a great tool to help troubleshooting space issues.

That’s all I have for now, there are lots more but these are the most useful ones that I find myself forgetting the most often, hopefully this post will serve as a nice reminder.  If you have any cool or useful commands that you would like to share feel free to comment and I will update the post to include them.

What is DevOps?

Since landing myself in a new and unexplored terrain as a freshly minted DevOps admin, I have been thinking a lot about what exactly DevOps is and how I will translate my skills moving into the position.  I am very excited to have the opportunity to work in such a new and powerful area of IT (and at such a sweet company!) but really think I need to lay out some of the groundwork behind what DevOps is, to help strengthen my own understanding and hopefully to help others grasp some of the concepts and ideas behind it.

I have been hearing more and more about DevOps philosophy and its growing influence and adoption in the world of IT, especially in fast paced, cloud and start up companies.  From what I have seen so far, I think I people really need to start looking at the impact that DevOps is making in the realm of system administration and how to set themselves up to succeed in this profession moving forward.

Here is the official DevOps description on Wikipedia:

DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals.  DevOps is a response to the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services.

While this is a solid description, there still seems to be a large amount of confusion about what exactly DevOps is so I’d like to address some of the key ideas and views that go along with its mentality and application to system administration.  To me, DevOps can be thought of as a combination of the best practices that a career in operations has to offer with many of the concepts and ideas that are used in the world of development.  Especially those derived from Agile and Scrum.

The great thing about DevOps is that since it is so new, there is really no universally accepted definition of what it is limited to.  This means that those who are currently involved in the DevOps development and adoption are essentially creating a new discipline, adding to it as they go.  A current DevOps admin can be described in simple terms as a systems admin that works closely with developers to decrease the gap between operations and development.  But that is not the main strength that DevOps offers and really just hits the tip of the ice burg for what DevOps actually is and means.

For one, DevOps offers a sort of cultural shift in the IT environment.  Traditionally in IT landscapes, there has been somewhat of a divide between operations and development.  You can think of this divide as a wall built between the dev and the ops teams either due to siloing of job skills and responsibilities or how the organization at broader perspective operates.  Because of this dissection of duties, there is typically little to no overlap between the tool sets or thought process between the dev or ops teams, which can cause serious headaches trying to get products out the door.

So how do you fix this?

In practical application, the principals of DevOps can put into practice using things like Continuous Integration tools , configuration management, logging and monitoring, creating a standardized test, dev and QA environments, etc.  The DevOps mindset and culture has many of its roots in environments of rapid growth and change.  An example of this philosophy put in to practice is at start up companies that rely on getting their product to market as quickly as smoothly as possible.  The good news is that larger enterprise IT environments are beginning to look at some of the benefits of this approach and starting to tear down the walls of the silos.

Some of the benefits of DevOps include:

  • Increased stability in your environment (embracing config management and version control)
  • Faster resolution of problems (decrease MTT)
  • Continuous software delivery (increasing release frequency brings ideas to market faster)
  • Much faster software development life cycles
  • Quicker interaction and feedback loops for key business stakeholders
  • Automate otherwise cumbersome and tedious tasks to free up time for devs and ops teams

These are some powerful concepts.  And the benefits here cannot be underestimated because at the end of the day the company you work for is in the business of making money.  And the faster they can make changes to become more marketable and competitive in the market the better.

One final topic I’d like to cover is programming.  If you are even remotely interested in DevOps you should learn to program, if you don’t know already.  This is the general direction of the discipline and if you don’t have a solid foundation to work from you will not be putting yourself into the best position to progress your career.  This doesn’t mean you have to be a developer, but IMO you have have to at least know and understand what the developers are talking about.  It is also very useful to know programming for all of the various scripting and automation tasks that are involved in DevOps.  Not only will you be able to debug issues with other software, scripts and programs but you will be a much more valuable asset to your team if you can be trusted to get things done and help get product shipped out the door.