Category Archives: Programming

What is DevOps?

Since landing myself in a new and unexplored terrain as a freshly minted DevOps admin, I have been thinking a lot about what exactly DevOps is and how I will translate my skills moving into the position.  I am very excited to have the opportunity to work in such a new and powerful area of IT (and at such a sweet company!) but really think I need to lay out some of the groundwork behind what DevOps is, to help strengthen my own understanding and hopefully to help others grasp some of the concepts and ideas behind it.

I have been hearing more and more about DevOps philosophy and its growing influence and adoption in the world of IT, especially in fast paced, cloud and start up companies.  From what I have seen so far, I think I people really need to start looking at the impact that DevOps is making in the realm of system administration and how to set themselves up to succeed in this profession moving forward.

Here is the official DevOps description on Wikipedia:

DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals.  DevOps is a response to the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services.

While this is a solid description, there still seems to be a large amount of confusion about what exactly DevOps is so I’d like to address some of the key ideas and views that go along with its mentality and application to system administration.  To me, DevOps can be thought of as a combination of the best practices that a career in operations has to offer with many of the concepts and ideas that are used in the world of development.  Especially those derived from Agile and Scrum.

The great thing about DevOps is that since it is so new, there is really no universally accepted definition of what it is limited to.  This means that those who are currently involved in the DevOps development and adoption are essentially creating a new discipline, adding to it as they go.  A current DevOps admin can be described in simple terms as a systems admin that works closely with developers to decrease the gap between operations and development.  But that is not the main strength that DevOps offers and really just hits the tip of the ice burg for what DevOps actually is and means.

For one, DevOps offers a sort of cultural shift in the IT environment.  Traditionally in IT landscapes, there has been somewhat of a divide between operations and development.  You can think of this divide as a wall built between the dev and the ops teams either due to siloing of job skills and responsibilities or how the organization at broader perspective operates.  Because of this dissection of duties, there is typically little to no overlap between the tool sets or thought process between the dev or ops teams, which can cause serious headaches trying to get products out the door.

So how do you fix this?

In practical application, the principals of DevOps can put into practice using things like Continuous Integration tools , configuration management, logging and monitoring, creating a standardized test, dev and QA environments, etc.  The DevOps mindset and culture has many of its roots in environments of rapid growth and change.  An example of this philosophy put in to practice is at start up companies that rely on getting their product to market as quickly as smoothly as possible.  The good news is that larger enterprise IT environments are beginning to look at some of the benefits of this approach and starting to tear down the walls of the silos.

Some of the benefits of DevOps include:

  • Increased stability in your environment (embracing config management and version control)
  • Faster resolution of problems (decrease MTT)
  • Continuous software delivery (increasing release frequency brings ideas to market faster)
  • Much faster software development life cycles
  • Quicker interaction and feedback loops for key business stakeholders
  • Automate otherwise cumbersome and tedious tasks to free up time for devs and ops teams

These are some powerful concepts.  And the benefits here cannot be underestimated because at the end of the day the company you work for is in the business of making money.  And the faster they can make changes to become more marketable and competitive in the market the better.

One final topic I’d like to cover is programming.  If you are even remotely interested in DevOps you should learn to program, if you don’t know already.  This is the general direction of the discipline and if you don’t have a solid foundation to work from you will not be putting yourself into the best position to progress your career.  This doesn’t mean you have to be a developer, but IMO you have have to at least know and understand what the developers are talking about.  It is also very useful to know programming for all of the various scripting and automation tasks that are involved in DevOps.  Not only will you be able to debug issues with other software, scripts and programs but you will be a much more valuable asset to your team if you can be trusted to get things done and help get product shipped out the door.

Why Computer Science degrees translate to System Administration

I run across a lot of articles and posts that talk about how a degree in Computer Science is usually irrelevant to system administration and that you are just as well off with another degree or no degree at all. I think that line of logic is very short sighted and today I am going (or at least attempt) to explain why. By no means am I criticizing these approaches, in fact I believe in the logic that there is more than one way to skin a cat, and I have found many other highly successful admins that have reached their positions by these alternate means. I just want to quickly clarify that I am not advising readers that taking the CS route to becoming a system admin is the only, right way to go, I am simply relating my own experiences in system administration to my background in CS and making a case of why pursuing a degree in Computer Science, or any other degree in engineering for that matter isn’t going to hurt your chances of becoming a sysadmin.

When you think of Computer Science you think of programming or maybe math, at least I do. Most CS programs these days have a heavy orientation towards programming and the scientific and mathematic applications of programming as it applies to the world around us. As an aside, I am beginning to see many more programs that are tailored to specific disciplines inside the realm of IT which looks promising. This is a great hybrid approach in my opinion because it gives students a chance to look at a few alternate options. Coding isn’t my passion so having an option to become a system administrator without the amount of intense coding from a CS program looks like an attractive approach.

It is true that many of the mundane daily tasks related to system administration don’t involve 8 hours a day of reading and writing code. Because of this I think it is important to characterize and distinguish a sysadmin as somebody who relies on software tools and programming to solve problems and technical challenges but doesn’t necessarily devote all of their time and energy to living in and interacting with code. The relationship of the sysadmin to programming is more of an indirect one, though still very important.

The farther along I wander on in my journey as a sysadmin the more I realize how the CS background is helping me.  I have a solid foundation in many of the core concepts that were taught through the CS program, which in turn  have indirectly influenced my abilities as a system administrator for the better. The first and most valuable asset my CS background has given me is the ability to write and understand code.  This is extremely useful in my daily slew of activities.  It allows me to approach problems with a programmatic methodology, it allows me to automate redundant and repeatable tasks with scripts, it gives me intuition into why databases or programs are slow, it allows me to debug issues systematically, and on and on.  Obviously these skills can be learned elsewhere but having them rolled up into your education when you learn about Computer Science as part of the package deal is very convenient.  I would much rather have this set of skills and have the ability to look at things from a different perspective than have to learn each of these techniques separately.  There is no way that somebody coming from a business or other similar background will know about silly things like big O notation or how different algorithms work at a fundamental level, it just isn’t part of their background so they don’t spend time thinking about these things.

This really parlays into other areas well and you are setting yourself up for a diversified and broad horizon for future employment prospects. For example, take a pure sysadmin that knows no programming or CS; at their core they know system administration. But what if they either get burnt out (which is common in this profession) or they don’t keep up the skills to match their position? There is nowhere in the industry for these individuals to turn, unless they want to go into management. That is why I believe individuals that choose not to further their careers are essentially crippling themselves and their future prospects by not knowing how or learning to program, or to at least understand how system administration and programming can relate to each other. With a diverse background the CS sysadmin could potentially move into a Devops role, a pure programming and development role or a management role. With the diverse IT ecosystem, programming and development skills are very much saught after and so the demand is high for these other types of positions and sets of skills.

Another well known fact in the IT industry, which I don’t necessarily agree with but nonetheless exists, is the fact that just having a CS degree will open doors that may not otherwise be open without a degree. I personally believe that a degree shouldn’t dictate your position but by having a degree you set yourself up for some unique opportunities and certainly are not hurting yourself. For example, all other things being equal, somebody scanning through resumes has to select an individual applicant that either has a degree in Computer Science or a degree in Philosophy. Which do you think will be picked? Like I said, I don’t think the hiring process is fair or even has anything to do with skill but can be used as a way to get ahead of the competition in the hiring process and can therefore a degree be valuable by itself as well as viewed as a strategic component in the hiring process if nothing else.

Here’s what I am saying. You don’t have to have a degree in Computer Science to be a great System Administrator. But the CS background definitely equips you with the tools to both understand some of the more abstract technical concepts and ideas and give you a robust framework working through and solving these difficult and complex problems. Ultimately the most important factors in being a good sysadmin (let alone anything else) is a combination of many different things, including a willingness to learn and the amount of experience an individual possesses. There is no cookie cutter way to build the perfect sysadmin and you will invariably find a very diverse group of people in this profession, but a head start with a CS degree is certainly one path that won’t hurt you and is a good attribute of many good sysadmins.

Using Find-String to grep in Powershell

For the longest time I have not been a fan of embracing the shell in the Windows world, but more and more I find that Powershell is able to do the things that I need.  I suppose my seething hatred was in part due to my negative bias towards Windows and the lack of useful tools from the command line in Windows.  Increasingly lately, I have been changing the way I think about and utilize the command line in Windows with Powershell.  And to be honest, I’m really beginning to fall in love with Powershell the more I get work with it and the more I get to see how to apply it in Windows based environments.  The good news is that Microsoft has put a lot of effort into this and are adding improvements and features all the time.  It still has a long way to go, but I can already see this as an alternative to GUI based administration in the Windows world and finally begin to feel like I can see the promise land on the horizon.  Using Powershell has been somewhat of a paradigm shift (in my opinion anyways) to how to do Windows administration recently and I feel like it will only get stronger and more common in the years to come.  So in this post I will try to show you some of the flexibility as well as some of the power that Powershell has to offer with some great tools from the community.

To highlight what I am talking about, let’s talk about grep, a well known and loved tool in the *nix universe.  I love grep.  For the longest time, I hadn’t known of a way to grep in the Windows world until just recently with a wonderful third party Powershell module called Find-String.  There are a few commands that you need to get this working.  To start, we need to get a module installed that is basically used for package management.  This tool is called PsGet. Installation is super simple, just run the following from a Powershell prompt, and ensure that your execution policy is set at least at remotesigned (Set-ExecutionPolicy RemoteSigned if you don’t have this turned on).

(new-object Net.WebClient).DownloadString(“http://psget.net/GetPsGet.ps1“) | iex

This should go out, download and install PsGet for you.  Once all that has completed you should be able to install Powershell modules that are contained in the PsGet repository.  With this installed we can just reach out to the PsGet repo and install our module.

Install-Module Find-String

Easy, simple, and clean.  That is the best part, there isn’t a ton of leg work to get this stuff working correctly and is why I’m enjoying Powershell so much these days.  Now we have a very functional grep clone!  Usage is quite a bit similar to grep, fore help you can do a Get-Help Find-String or look at the link I posted earlier to the author’s github page.  If you want to look for a word or substring in a file just use something like this:

Find-String word example.txt

This will output all occurrences of the word you are looking for in the file example.txt.  Here is a screen shot to show you.

Find-String in action

What other cool Powershell stuff are you doing?  I would love to hear about other cool uses that can be beneficial in every day use.

Make MacVim’s mvim script use tabs and play nice with the command line

tl;dr: Replace the mvim script with this modified version: https://gist.github.com/3780676

MacVim comes with a really sweet script called mvim, which lets you launch MacVim and edit files from the command line. Unfortunately, this script is a little weak in a few ways:

  • It doesn’t let you edit multiple files.
  • It doesn’t let you pass in command line options.
  • It doesn’t let you use new tabs for opening new files into an existing window.
  • It doesn’t let you pipe stdin into vim for viewing (great with diffs).

All those things are awesome, so let’s make the mvim script better! How do we do that?

Well, first we add some extra command line options parsing to detect if we’re in diff mode, if we’re using stdin, and to preserve options for passing back into MacVim later. At Line 60 we add the following:

# Add new flags for different modes
stdin=false
diffmode=false
# Preserve command line options lazily
while [ -n "$1" ]; do
  case $1 in 
  -d) diffmode=true; shift;;
  -?*) opts="$opts $1"; shift;;
  -) stdin=true; break;;
  *) echo "*"; break;; 
  esac
done

This is a pretty normal bash argument getting loop. We look for -d (diff mode) and – (stdin) separate from other arguments. We also need to modify the command that starts MacVim to handle our different modes, etc. So we replace that command (originally on line 69):

# Last step:  fire up vim.
# The program should fork by default when started in GUI mode, but it does
# not; we work around this when this script is invoked as "gvim" or "rgview"
# etc., but not when it is invoked as "vim -g".
if [ "$gui" ]; then
	# Note: this isn't perfect, because any error output goes to the
	# terminal instead of the console log.
	# But if you use open instead, you will need to fully qualify the
	# path names for any filenames you specify, which is hard.
	exec "$binary" -g $opts ${1:+"$@"}
else
	exec "$binary" $opts ${1:+"$@"}
fi

With this better command:

# Last step:  fire up vim.
# The program should fork by default when started in GUI mode, but it does
# not; we work around this when this script is invoked as "gvim" or "rgview"
# etc., but not when it is invoked as "vim -g".
if [ "$gui" ]; then
  # Note: this isn't perfect, because any error output goes to the
  # terminal instead of the console log.
  # But if you use open instead, you will need to fully qualify the
  # path names for any filenames you specify, which is hard.

  # Handle stdin
  if $stdin; then
    exec "$binary" -g $opts -
  elif $diffmode; then
    exec "$binary" -f -g -d $opts $*
  elif $tabs && [[ `$binary --serverlist` = "VIM" ]]; then
    #make macvim open stuff in the same window instead of new ones
    exec "$binary" -g $opts --remote-tab-silent ${1:+"$@"} & 
    wait
  else
    exec "$binary" -g $opts ${1:+"$@"}
  fi
else
  exec "$binary" $opts ${1:+"$@"}
fi

The first two branches are pretty clear – they just invoke the MacVim binary in the correct way, for our different modes. The third one uses the very awesome –remote-tab-silent option, which gives us the ability to reuse the same window with new tabs when we edit multiple files. Neato!

Finally, if you don’t want to do the modifications yourself, it’s available as a gist, so you can download it and use it as a drop-in replacement for the vanilla mvim script: https://gist.github.com/3780676