Category Archives: Productivity

Transitioining from bash to zsh

oh-my-zsh

I have know about zsh for a long time now but have never really had a compelling reason to switch my default shell from bash until just recently, I have been hearing more and more people talking about how powerful and awesome zsh is.  So I thought I might as well take the dive and get started since that’s what all the cool kids seem to be doing these days.  At first I thought that changing my shell was going to be a PITA with all the customizations and idiosyncrasies that I have grown accustomed to using bash but I didn’t find that to be the case at all when switching to zsh.

First and foremost, I used a tool called oh-my-zsh to help with the transition.  If you haven’t heard about it yet, oh-my-zsh aims to be a sort of framework for zsh.  This project is a nice clean way to get started with zsh because it give you a nice set of defaults out of the the box without having to do much configuration or tweaking and I found that many of my little tricks and shortcuts were already baked in to to oh-my-zsh, along with a ton of other settings and customizations that I did not have using bash.

From their github page:

oh-my-zsh is an open source, community-driven framework for managing your ZSH configuration. It comes bundled with a ton of helpful functions, helpers, plugins, themes, and few things that make you shout…

Here are just a few of the improvement that zsh/oh-my-zsh offer:

  • Improved tab completion
  • persistent history across all shells
  • Easy to use plugin system
  • Easy to use theme system
  • Autocorrect

The most obvious difference that I have noticed is the improved, out of the box tab completion, which I think should be enough on its own to convince you!  On top of that, most of my tricks and customizations were already turned on with oh-my-zsh.  Another nice touch is that themes and plugins come along as part of the package, which is really nice for easing the transition.

So after spending an afternoon with zsh I am convinced that it is the way to go (at least for my own workfolw).  Of course there are always caveats and hiccups along the way as I’ve learned there are with pretty much everything.

Tuning up tmux

Out of the box, my tmux config uses the default shell, which happens to be bash.  So I needed to modify my ~/.tmux.conf to reflect the switch over the zsh.  It is a pretty straight forward change but is something that you will need to make note of kif you use tmux and are transitioning in to using zsh.

set-option -g default-shell /usr/bin/zsh

I am using Ubuntu 14.04, so my zsh is installed to /usr/bin/zsh.  The other thing that you will need to do is make sure you kill any stale tmux processes after updating to zsh.  I found one running in the background that was blocking me from using the new coonfig.

Goodies

There is a nice command cheat sheet for zsh.  Take some time to learn these shortcuts and aliases, they are great time savers and are very usefull.

oh-my-zsh comes bundled up with a large number of goodies.  At the time of this writing there were 135 plugins as well as a variety of themes.  You can check the plugins wiki page for descriptions for the various plugins.  To turn on a specific plugin you will need to add it to your ~/.zshrc config file.  Find the following line in your config.

plugins=(git)

and add plugins separated by spaces

plugins (git vagrant chef)

You will need to reload the config for the changes to be picked up.

source ~/.zshrc

Most themes are hosted on the wiki, but there is also a web site dedicated to displaying the various themes, which is really cool.  It does a much better job of showing differences between various themes.  You can check it out here.  Themes function in a similar way to plugins.  If you want to change your theme, edit your ~/.zshrc file and select the desired them.

ZSH_THEME="clean"

You will need to reload your config for this option as well.

source ~/.zshrc

Conclusion

If you haven’t already made the switch to zsh I recommend that you at least experiment and play around with it before you make any final decisions.  You may be set in your ways and happy with bash or any other shell that you are used to but for me, all the awesomeness changed my opinion and decide to reevaluate my biases.  If you’re worried about making the switchin, using oh-my-zsh makes the transition so painless there is practically no reason not to try it out.

This post is really just the tip of the iceberg for the capabilities of this shell, I just wanted to expose readers to all of its glory.  Zsh offers so much more power and customization than I have covered in this post and is an amazing productivity tool with little learning overhead.

Let me know if you have any awesome zsh tips or tweaks that folks should know about.

7 useful but hard to remember Linux commands

I have found myself using these commands over and over so I decided I’d take the time to go ahead and document them for future me as well as readers because I find these commands pretty useful.  I just always manage to forget them, hence the title of the post.  The smart thing to do would be to create aliases for these commands but I have just been too lazy and some of them are run across different servers so it isn’t always a convenient option.

Anyway, let’s go ahead and run through the commands before I forget…

1) du -ah / | sort -n -r | head -n 50

This one is really handy for debugging space issues.  It will list the top 50 files according to file size, with the largest at the top of the list.Notice the “/” will specify the location to search so you can easily modify this one to search different locations, like “/var/log” for example if you are having trouble with growing log files.

2) git checkout — .

I don’t use this one very often, which is probably why I manage to forget it so easily.  But I really like it.  Sometimes I will be working on a git repo across different machines at the same time and will run in to conflicts committing to the repo or more likely I committed changes on one machine and just need to pull down the newest changes but can’t since I have made modifications.  For those scenarios you can run the above command quickly reset your git changes quickly and easily.

3) tmux kill-window -t 3

i use tmux for my terminal and window manager on all my workstations and love it.  If you haven’t heard of it, take a look here.  Sometimes the sessions can get stuck so it becomes necessary to close the window without destroying the tmux session.  Again, this doesn’t happen very often so it is sometimes hard for me to remember the exact syntax but this one is a handy little trick for managing tmux windows and sessions.

4) grep -r “text”

I know, I should really have this one memorized by now.  I am trying to remember but I don’t find myself using this one all that often even though it is really powerful and useful.  This will essentially search through every file recursively and spit out the text pattern that you feed to it.

5) kill $(pgrep process)

This one is handy when there are a large number of stuck processes and you need to blow them all out with one command.  For example if the chrome browser ever gets stuck with a million tabs open, there are likely a large number of processes all with the same – or similar names.  If you pass all or part of the process name in to this command pgrep will find them and kill will destroy them

6) docker rm $(docker ps -a -q)

I have been using Docker more and more recently and every once in awhile I find myself with a large number of dead Docker processes that need to be cleaned up.  This command will blow out all of these stale processes at once.  This is nice because Docker processes take up a large amount of disk space and often times can fill up your drives without you being aware.  I have been able to reclaim large amounts of disk space with this command.

7) watch -n 10 df -ah

This is another good one for checking disk space issues.  It will update you every ten seconds with the disk utilization of the system.  Pretty straight forward but a great tool to help troubleshooting space issues.

That’s all I have for now, there are lots more but these are the most useful ones that I find myself forgetting the most often, hopefully this post will serve as a nice reminder.  If you have any cool or useful commands that you would like to share feel free to comment and I will update the post to include them.

What is DevOps?

Since landing myself in a new and unexplored terrain as a freshly minted DevOps admin, I have been thinking a lot about what exactly DevOps is and how I will translate my skills moving into the position.  I am very excited to have the opportunity to work in such a new and powerful area of IT (and at such a sweet company!) but really think I need to lay out some of the groundwork behind what DevOps is, to help strengthen my own understanding and hopefully to help others grasp some of the concepts and ideas behind it.

I have been hearing more and more about DevOps philosophy and its growing influence and adoption in the world of IT, especially in fast paced, cloud and start up companies.  From what I have seen so far, I think I people really need to start looking at the impact that DevOps is making in the realm of system administration and how to set themselves up to succeed in this profession moving forward.

Here is the official DevOps description on Wikipedia:

DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals.  DevOps is a response to the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services.

While this is a solid description, there still seems to be a large amount of confusion about what exactly DevOps is so I’d like to address some of the key ideas and views that go along with its mentality and application to system administration.  To me, DevOps can be thought of as a combination of the best practices that a career in operations has to offer with many of the concepts and ideas that are used in the world of development.  Especially those derived from Agile and Scrum.

The great thing about DevOps is that since it is so new, there is really no universally accepted definition of what it is limited to.  This means that those who are currently involved in the DevOps development and adoption are essentially creating a new discipline, adding to it as they go.  A current DevOps admin can be described in simple terms as a systems admin that works closely with developers to decrease the gap between operations and development.  But that is not the main strength that DevOps offers and really just hits the tip of the ice burg for what DevOps actually is and means.

For one, DevOps offers a sort of cultural shift in the IT environment.  Traditionally in IT landscapes, there has been somewhat of a divide between operations and development.  You can think of this divide as a wall built between the dev and the ops teams either due to siloing of job skills and responsibilities or how the organization at broader perspective operates.  Because of this dissection of duties, there is typically little to no overlap between the tool sets or thought process between the dev or ops teams, which can cause serious headaches trying to get products out the door.

So how do you fix this?

In practical application, the principals of DevOps can put into practice using things like Continuous Integration tools , configuration management, logging and monitoring, creating a standardized test, dev and QA environments, etc.  The DevOps mindset and culture has many of its roots in environments of rapid growth and change.  An example of this philosophy put in to practice is at start up companies that rely on getting their product to market as quickly as smoothly as possible.  The good news is that larger enterprise IT environments are beginning to look at some of the benefits of this approach and starting to tear down the walls of the silos.

Some of the benefits of DevOps include:

  • Increased stability in your environment (embracing config management and version control)
  • Faster resolution of problems (decrease MTT)
  • Continuous software delivery (increasing release frequency brings ideas to market faster)
  • Much faster software development life cycles
  • Quicker interaction and feedback loops for key business stakeholders
  • Automate otherwise cumbersome and tedious tasks to free up time for devs and ops teams

These are some powerful concepts.  And the benefits here cannot be underestimated because at the end of the day the company you work for is in the business of making money.  And the faster they can make changes to become more marketable and competitive in the market the better.

One final topic I’d like to cover is programming.  If you are even remotely interested in DevOps you should learn to program, if you don’t know already.  This is the general direction of the discipline and if you don’t have a solid foundation to work from you will not be putting yourself into the best position to progress your career.  This doesn’t mean you have to be a developer, but IMO you have have to at least know and understand what the developers are talking about.  It is also very useful to know programming for all of the various scripting and automation tasks that are involved in DevOps.  Not only will you be able to debug issues with other software, scripts and programs but you will be a much more valuable asset to your team if you can be trusted to get things done and help get product shipped out the door.

How I do internal documentation

Intro

Whether you are new to system administration or have spent some time in the role, getting a grip on documentation can be tricky and often times challenging.  I can speak from experience because when I started my career I had no idea how to do this stuff, but slowly over time I have been able to develop some useful techniques.  It also helps that I am obsessed with documentation.  Therefore the techniques I use are always changing and I am always looking for ways to improve my documentation skills.  I think the most important take away from what I have learned, is that there is no end all be all way to do documentation and everybody will probably find slightly different ways.  My method may or may not work for you but hopefully you can  look at it and use some of the underlying concepts to help with your own style.  I have been able to use my own documentation procedures over the past 3 years to really improve my job performance as well as other areas in my job so hopefully there is something in here you can apply to your own situation.

Once you get past the initial stage of how uncomfortable it is to document everything by your mindset to how useful it is to having a living reference for everything on the network, then documentation really becomes much more manageable.  The added bonus of solid documentation is that once you have a decent repository built up you don’t have to worry about constantly adding new items to your documentation.  I am at a point now that I primarily use my documentation as a reference and don’t find myself having to create or update things nearly as often as I used to.  The other main benefit of having solid documentation is to help others on your team, especially junior members so they can go out and learn about certain intricacies or specific details of systems in your environment.  It’s also good for avoiding situations like the hit by a bus scenarios.  The general idea is that by having a great piece of documentation, somebody can pick up where you left of with minimal downtime and impact.  Hence, if you were to get hit by a bus and/or become incapacitated then somebody else could get up to speed with the way things work (what, when, why, how) using your documentation as a reference without having to spend hours and hours of time to figure out what exact information that they needed to know.

Documentation Overview

There are really two main types of internal documentation, internal and external.  Public facing (external) documentation is an entirely separate topic and I will not be discussing it here.  Network and technical documentation (internal) are the two big ones that I use to reference typically.  Network documentation is essentially working with Visio to produce anything and everything related to the network.  If you want to be a good system/network administrator a great way to help is to get good at working with Visio.  The other type of documentation I like to refer to as “technical documentation” is typically anything and everything that I have found to be helpful in solving a problem.  As I mentioned earlier, the longer your work on your environment and building your documentation the less you will have to produce new documentation. This information comes from the documentation section in Limoncelli’s book Time Management for System Administrators which is a fantastic book for sysadmins if you haven’t already taken a look.

My own technical documentation has grown organically in the 3 years, from just a few pieces of information to essentially a living library of all of the different systems I and others on my team work with on a regular basis.  Everything I do is in there; from technical procedures, links to external blogs and websites, PDF’s, to pretty much anything else you can imagine that would be useful.  One of the hardest things about working on documentation in a collaborative environment is getting people to buy in to the system.  The best way I have found to handle this is to take the initiative, start the process and then let everyone know about it what tools you are using and how awesome this documentation is and hope that it sticks.  I am not a fan of forcing people to do things but by showing them an amazingly simple system that is also incredibly useful the goal is to get them excited and motivated to use your documentation system.  Sometimes it works, sometimes not, but I have found it to be the best approach when attempting to change people’s habits and get them used to working with something new.

In the remainder of this post I will go into some of the details of how I organize my technical documentation and use it to help quickly identify and fix problems quickly.  I don’t plan on covering network documentation in this post as it is a slightly different beast.

The Tool

I have used pretty much every documentation system you can think of, from SharePoint, EverNote, Dropbox, various Wiki’s, Office documents, etc. and through my time and experience with all of these various tools I have found one to be the best for syadmin documentation.  I really believe that many of these tools serve a certain purpose or function but in my time as a sysadmin I have found OneNote to be the hands down best tools for system administration documentation.  As a side note, Wiki’s are a great alternative if you don’t have access to OneNote but my preference lies with OneNote.

OneNote is flexible, you can set up a location on a file share inside your company and allow access by certain AD users or groups to the share to control who can see and write to and collaborate on your documentation system.  It is very handy to be able to work on some notes while in a meeting on my laptop then move back to my desktop and have these notes current and in sync between my machines.  It is also possible for multiple people to be in OneNote working live on different notes, which is also nice when you are working in a busy team environment.  I know there are many other products that allow this so even if you don’t choose OneNote I would highly recommend a tool that offers these features.

Organization

I have OneNote grouped in a specific way to help increase my productivity.  I have different notebooks for various items.  I have a notebook for my technical documentation, a task log, one for projects and meetings that eventually get merged into my tech docs if they grow and the project/meeting ever materializes into anything more than just a few notes and I also have a notebook for personal work items.  The personal notebook contains a few different notes for personal accomplishments, personal agenda and ideas for projects and improvement.

Personal Notebook

Within the tech docs notebook I have notes for all of the major technologies that I or members of my team work with on a regular basis.  Basically, anything that we deem noteworthy gets thrown into one of the categories somewhere inside the tech doc.  The structure of this document grows organically for me with a few minor pain points here and there along the way.  For example, when a new solution to a problem is found and there isn’t a specific category for the fix, sometimes it is necessary to either create a new category or just use your judgement when determining the best place to store the fix.  Luckily the search feature in OneNote works well so if you can’t remember where you put something then you can just search through everything to find it.  Coincidentally, the larger your docs grow to be the better the search function works.

Each category has a basic general structure guideline that I have found to improve the workflow and allows things to work more smoothly.  The structure applies to every page I can use it on and is a template that includes the  most common and hard to find information that I use as a template.  The items in each category each get their own note and are as follows:

  • Network Information – IP’s, DNS names, server and network roles, anything that plany a part in the network goes here.
  • Support/Contact – Direct software and product support numbers, vendor contacts, important email addresses, account ID’s and service agreements, licenses, contract numbers, etc.
  • Resources – Any links to relevant and important technical docs, deployment guides, implementation guide or admin docs that are either internally or online.
  • Commands – I work in a Windows environment so often times PowerShell is the go to resource.  I keep a table of all the commands that I deem to be useful here for that particular category.
  • Useful tools – This one is optional but for some categories is a nice little reference.  I use this section to help identify anything and everything that I can use as a tool to help with a particular category.

This information is the skeleton that I begin to model all of the other documentation around.  From there, the rest is easy and should essentially fall into place.  Most of the time for me, these other items are one off fixes for problems that I have found or have solved.  Sometimes a quick link to the blog post that helped will be a good enough reference and other times a painfully detailed set of steps is necessary for documenting the procedure for the fix.

Documentation template

Style

The actual documentation is more of an art form than anything else that gets better with practice.  Sometimes technical notes require an obscene amount of detail because of how complex they are and because the procedure  only needs to take place once or twice in a year.  Other notes just need a rough explanation and therefore don’t need any detail at all, it all depends on the situation.  As has been pointed out in the comments section, one good approach to documentation sometimes is to assume the reader has no prior knowledge about the topic and therefore painful explanation and detail are necessary to convey your materials.  The point is, there is no cookie cutter way to do all of your technical documentation and various tactics and techniques need to be developed for different types of issues, that is why documentation becomes an art form in my view.

Going back to my own technique, one of the most helpful tricks I have found is to create the commands section, which I throw all of the most useful and common commands into for the particular category I am referencing.  The more work you do from the command line, the more useful this note will be to you.  For example, being in a Windows environment, I work with Exchange pretty much daily and having a repository of all the useful commands that I need in one place is one of the best ways to save time rather then going through Google looking for the specific information I am looking for every time.  Here is a glimpse of what I am talking about.

PowerShell commands

Another thing that will help your documentation immensely is consistency.  I am talking broadly here but when you create a living document you will want to have some sort of consistent style across your documentation.  Items that come to mind here are things like having a consistent template as mentioned earlier, consistent naming conventions, fonts, naming conventions, established criteria for creating topics and notes, capitalization, fonts and organization, etc.  Having this general style guide established early on will help to make finding and reading information in your documents much easier and will consequently help to save time when you are looking for specific information.

Closing

That’s all I’ve got for now.  I just wanted to point out a few things and get people pointed in the right direction that are looking for ideas of how to get going on technical documentation.  As I mentioned, there’s more than one way to skin a cat – meaning there are a large number of ways to do documentation and there isn’t necessarily one best way to do things.  I have shown you my preferred way, but it may not be the best way for you.

Documentation is as much of a learning process as anything else and sometimes you just need to experiment with things and just spend some time wading around to get the best results.  One great way to check if your documentation is working or not is to get somebody from your team to attempt to use your documentation to fix something you have instructions for.  If you just want to get some practice writing clear and concise instructions, try this method on yourself.  Write out the doc and procedure and come back to the instructions a week or month later and see how hard it is to figure out how to fix the problem.  Chances are, if things are unclear or your teammate is unable to fix the issue then it is probably a good idea to take a look at reevaluating how you’re doing your documentation.

If you have any thoughts, suggestions, ideas or anything else just let me know.  Like I said earlier, I am obsessed with this kind of stuff and I am always looking for ways to improve my own processes and procedures.  I plan on coming back to this post and update it in the future if any of my documentation practices change but for the time being, I hope you find this information useful and applicable to your own documentation processes.

Some quick tips for getting ahead

Do you want to get ahead in this field?  In my experience in the industry, there are a lot of great ways to improve yourself and to put yourself in position to get ahead.  One of the best ways is to show that you are interested in your craft.  There are numerous ways to demonstrate this desire, and dedication; a simple yet great way to do so is by attending industry related conferences and networking events.  Pay for these events as well as certification courses out of pocket if your employer is unwilling to do so.  Paying for your own career betterment out of your own pocket shows your level of commitment to those around you that you may meet at these events as well as those that you work with.  I guarantee you people will notice this.  I would also like to mention that if you are interested in training, certification, or anything the will help you do your job better and employers aren’t willing to help pay for any of these opportunities then often times it is a sign that you have outgrown your current position or will be in the not so distant future.

Another great area to focus on is to never stop learning.  This applies not only to IT but to everything that you do in your life.  Taking the time to learn something new is a great way to help boost your career.  Interested in business?  Take a business class or at least start researching different aspects of business and find what interests you.  Then leverage that newly found knowledge in every day aspects of your life.  I am a firm believer in education overlapping and translating skills from one area to contribute to others.  If you learn a seemingly unrelated skill you never know when and where it will be applicable and it will all contribute to furthering your education and improving your knowledge.  It can be devotion to skills like Documentation, attention to detail, writing, programming, whatever it may be will help to improve and strengthen your IT skills, I promise.  Many of these skills are things that I work on improving outside of my life in IT but certainly they contribute to the success I have experienced in my career.

The final tip I’d like to mention is doing things that others either aren’t willing to do or are not aware of doing.  This is a more subtle point but I believe it is something that will separate people that want to be good at something from those that are content with where they are at in life.  For example, many individuals in IT don’t like to leave their comfort zone for many things.  Learn how to talk to people and network with others in the industry who have already discovered how to get ahead.  They will give you so many great pieces of information and can potentially help you get into new positions as well.  We all know that social skills are not a strong suit for many that are in the IT profession so learning how to talk to people is incredibly valuable.  Yes it will feel awkward and unnatural at first but the more you do it and the more you practice it (just like anything else) the better you will get at it.  And since it is so uncomfortable for many individuals they simply won’t attempt it because it is out of their comfort zone.  There are many other examples of simple ways to get ahead but just by knowing how to leverage and utilize things like networking and being social will help your career more than you might realize.

I am taking a different approach from the usual subject matter, I have just found it difficult recently to find anything technically interesting enough to write about.  I would love to cover more of these areas if there is any interest in the future, I think its an oft missed subject in the IT profession and I happen to like writing about it so hopefully I can get some positive feedback.  As always, if you have anything that you find interesting or think would be a positive contribution let me know, I’m always looking for guest posters and guest content and I welcome the fresh perspective and ideas.