Resetting SCCM Agent if patch installation fails

From time to time, a ticket will be created in regards to System Patches failing in an SCCM environment. To fix this, there are really only two major steps:

  1. Rename the C:\Windows\SoftwareDistribution folder to SoftwareDistribution.old (stop Windows Update service before renaming, then restart the service).
  2. Rename C:\Windows\System32\catroot2 to catroot2.old (stop the Cryptography service before renaming, then restart the service).

After this is done, run these actions from the configuration manager:

  1. Discovery Data Collection Cycle
  2. Software Updates Deployment Evaluation Cycle
  3. Software Updates Scan Cycle

The procedure above has taken care of the issue pretty reliably. If the updates still don’t install properly, you may have to download the specific updates and install them manually.

About the Author: Mike Erps

Michael Erps currently provides IT support and consulting to gov’t contractors in D.C. He is also interested in Internet Marketing and helping others build the online presence they are looking for. You can find him on about.me

OpenFlow and the Future of Networking

OpenFlow is all the rage right now and since I just got done doing a product overview of it and its relation to the HP product line we just recently purchased, I thought I would get in a quick post about all of it while the topics and ideas are still fresh in my mind.  So this post will be less of a technical post than usual and more of a detour about my thoughts on networking and the effect OpenFlow is having on it.

I am still trying to wrap my head around some of the key concepts and applications that OpenFlow has to offer but I think I am beginning to understand the core concepts behind it, and honestly I don’t understand all the OpenFlow hate and SDN bashing from other network professionals.

Even thought OpenFlow is a fresh concept for me I can already see potential benefits and possible use cases and I think that there is some great potential with SDN in general.  There must be some interesting value here, otherwise there wouldn’t be so much interest by all of the heavy hitting networking industry leaders like IBM, Cisco, HP, Google, etc. collaborating and working on projects like OpenDaylight and Floodlight. Since the concepts and ideas behind OpenFlow are so new and are largely unexplored there is a very mysterious and exciting quality behind the technology and because of this I believe that creativity can help drive its development and adoption.  The other nice part about OpenFlow is that it is an open standard so it can be developed and extended by whomever feels like participating or contributing (Cisco and its OnePK API and other vendor specific API’s are a different story) to the project and the code base.  I am a huge proponent of Open Source and I feel like having an open standard creates better code and more opportunities for everybody involved, it doesn’t benefit one but rather the collective.

I also want to touch briefly on the technical side of OpenFlow for all the IT pros.  Technology evolves and changes all the time, we’ve seen it time and again in our industry.  If you are stubborn to the point that you won’t dedicate the time to learn something new just because its not what you are familiar with then you probably won’t have much of a future in IT and ops or at least a future going forward in the networking world.  Sure you’ve built a career on your niche ability and skill set to solve complex and challenging networking problems, but that is not a unique quality.  All IT professionals build their careers on their ability to do this (at least the good ones I’ve seen so far), and every other area of IT is subject to these same types of issues that new technology brings.  In my opinion the haters just need to grow up and accept the fact that they will need to remodel their skills from time to time.  It’s not that big of a deal.  And besides, OpenFlow actually looks promising and looks like it will be a great tool for IT pros to utilize to solve interesting problems.

Rather than complain and find fault, embrace OpenFlow, because whether you like it or not, it will have its place in the networking world moving forward.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

Wireshark Reference Guide

Based on a strange network problem recently I decided to put together some quick notes and a few tips on ways to improve your Wireshark experience based on my own experience with it.  There are many, many more features that Wireshark has to offer, these just happen to be the most apparent ones I have found so far.  Wireshark is extremely powerful and therefore extremely useful if used properly.  At first it takes a while to get used to everything Wireshark has to offer but once you start to get the hang of how things work then it can be a great network troubleshooting tool.  Basic knowledge of networking concepts should be assumed as well as familiarity of Wireshark for those who attempt to debug network problems using this tool.

Here is a list of some of the most common and handy features that you can utilize in Wireshark.  I am not going to dive into great detail with most of  these items because I honestly don’t have a ton of experience with all of them, I basically just wanted to point out the highlights.

  • Filtering in Wireshark is very handy.
  • Create custom profiles for different use cases (quickly select from bottom right hand corner).
  • Color filters are useful!  (Right click a field in the packet trace and selelct colorized rule)  The bottom left bar will tell you what variable you are looking at to make things easier when customizing.
  • Use Regex in wireshark using the “matches” clause to turn on regex patterns.
  • You can extract specific information from trace files on the command line using tshark.
  • Right click a packet and select “follow TCP/UDP stream” to debug a single network conversation.
  • Low delta times are good.  If you see high deltas you should probably investigate things.

Here are some more concrete examples and a few basics of how to put these tips into practice.  In Wireshark you can use either English or code like operators when filtering to help narrow down traffic and interesting networking patterns and issues.  So for example, “==” and “eq” will behave in the same manner when applying filters.  Other operators include <,>, !=, <=, >=, etc.  Just like you would see in a typical programming language.

Use custom configuration profiles.  If you look at packet traces often this will save you a tremendous amount of time if you are looking at specific types of traffic or are only interested in certain traffic patterns.  For example, you spend a lot of time looking at many traces that fit the same type of criteria; by using custom profiles you can quickly adjust and modify the view in Wireshark to help quickly identify patterns and potentially issues by cycling through different, specific views.  To begin creating custom profiles go to Edit -> Configuration Profiles and then either select the custom profile or create a new one to begin changing.

One handy trick is to disable TCP offload checks.  If your packet captures are getting clogged up with a bunch of red and black with offload errors, this is place you should go to look first.  There are a few places where this option can either be enabled or disabled.  The easiest way to check these options is under Edit -> Preferences and then under the Protocols tree for UDP, TCP and IPv4 protocols.  The example below shows what the options should look like for the TCP protocol.  The TCP and UDP offload checks are disabled by default but the IPv4 needs to be manually unchecked.  The specific option under the IPv4 protocol is labeled “Validate the IPv4 checksum if possible”, simply uncheck this and the red and black errors should disappear.

There is a capture option that allows you to resolve IP addresses to hostname, which I find can be very useful.  To enable this option open up the Show capture options screen, there should be an option in there under name resolution called “Resolve network-layer names”.  Simply check that box and you should have name resolution.

As mentioned in the bullets above, the “follow UDP/TCP stream” option can be extremely useful and is a very quick way to glean information.  It is so useful because it is so easy to use.  Simply find a traffic conversation you would like to debug and right click the packet number in the top Wireshark pane and choose the follow UDP/TCP stream option and you can get an idea of everything that happened during a particular conversation.  For example, using this technique you can follow FTP transactions.

Viewing a breakdown of the packet flow and traffic patterns can be a useful tool as well when diagnosing various network issues.  There is an option in Wireshark that shows in good detail the breakdown of various packets and protocols that can be used to troubelshoot the network.  This option is called Protocol Hierarchy Statistics and can be found un te Statistics -> Protocol Hierarchy Statistics page.

Only look at traffic for one IP address:

ip.addr==192.168.103.104

Likewise, filter out all traffic from an IP address:

!(ip.addr == x.x.x.x)
!(ip.addr == 1.2.3.4)

filter out all traffic for a specific port

!(tcp.port eq 2222)

Resources:

http://ask.wireshark.org/questions/
http://ask.wireshark.org/questions/
http://www.wireshark.org/docs/

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

A Git Primer for Sysadmins

I am writing this post with the intention of giving readers (and other sysadmins) that are unfamiliar with Git a brief introduction to get up and running as quickly as possible.  There are numerous guides out there but none I have found so far with the spin of getting things working quickly for system administrators.  I recently went through the process of submitting some work of mine to an Open Source project and went through all of the work myself and thought I would write up a little guide to getting started, just in case anyone is interested.  Because of this experience, I thought the process could be streamlined if I presented a simple set of steps and examples for other sysadmins so that they can get started and get comfortable using Git as quickly as possible.  By no means am I advocating that I am a Git expert, I simply know enough to “do stuff” so take this guide for what it is, a simple introduction.  Hopefully it is enough to get the ball rolling and get you hooked on revision control!

Step 1:  If you haven’t already, head over to github and set yourself up with a free account.  The steps are trivial, just like setting up any other account.  You will need to install Git on you machine locally so that you can do everything, sudo apt-get install git in Ubuntu, or, with a little more legwork if you are using Windows.

Step 2:  Once you are all set up and running you can either create a new repository or fork off of somebody else’s repo, which will essentially make a copy of their work to your user account in github.  Since I was contributing to a public project I forked their repo.  This is typically (for beginners at least) done through the website by browsing to the project you are interested in and then clicking the fork button at the top right.

Fork a repo

Once you have copied or forked the project you want, you can begin working on making your own changes and updates to the project.  This is where much of the initial confusion came in for me.  I should make a note here; there are a few things that must be done through the website and others that should be done with the git tool you downloaded and installed earlier.

Step 3:  Clone the repo down to your machine locally to begin making changes.  This is done via command line with the following:

git clone https://github.com/$username/repository.git
git clone https://github.com/jmreicha/curriculum.git

Once you have the repo cloned down to your machine, let’s create a branch which we will use to work on the specific piece of the project we are working on.  There are a few more common commands that are used for working with repos.  In some scenarios the public project may change or get updated while you are working on it so you might want to pull in the most recent changes to your own fork.  The following commands are used to work with public repos and forks.

Fetch new changes from original repo:

git fetch upstream

Merge the fetched changes into your working files:

git merge upstream/master

Pull newest updates and changes from a for or repo:

git pull

Okay, let’s assume that our repository is all up to date and we are ready to start working on the new changes that we want to make.  First, we need to create a local branch to make the changes on.  The command for creating a branch is:

git checkout -b $topic
git checkout -b my_topic

Where $topic is the specific change you will be working on.  Then go ahead and make your changes and/or updates with your favorite text editor.  If you have multiple changes to make it may be a good idea to create a few different branches using the command above.  To switch between branches use the following:

git checkout $branchname
git checkout my_topic

If you are working on a branch and either don’t want to commit the changes or the branch becomes obsolete, you can delete the branch with either of the following commands:

git reset --hard
git branch -d $branchname
git branch -d my_topic

Step 4:  Assuming you are all done with your branches and are finished making your changes the next step is to commit the changes.  This will basically record a snapshot of (the state) the files.  Typically for a commit you will want to do one thing at a time to keep track of things more easily.  So you will make one change and then commit and then repeat if there are numerous changes that need to be made.  The command is:

git commit -am "update goes in here"

Now that we have created a branch, made our changes, written our changes so that git knows about them and can keep track of them, we are ready to go ahead and merge the code that we have changed locally back up to github.  Start by pushing the branch up to the github repo.

git push $remote $branchname
git push origin my_topic

Next, merge your changes.  This is done by changing to the “master” branch and then issuing a merge command on the branch that was created to make changes into.  So here is what this process would look like:

git checkout master
git merge $branchname
git merge my_topic

alternatively if you just want to quickly push out a change to the master branch you can issue a git commit locally and then a git push to update the master branch without merging the items first.

git commit -am "updated message"
git push

I like to use this for local repositories where I just need to get things up quickly.

Step 5:  At this point everything should be accurate on your own personal github page.  You can double check your commit history or look at the last update time to make sure your changes worked their way into your public fork or repo.  If you are contributing to a public project the final step is to issue a pull request on the github site.  Once this is done, wait to hear back from a moderator of the project if your changes need to be fixed before they accept them or wait for a response saying that your changes were good and get merged to the project.

Git, to me, was a little bit confusing at first.  Like I said earlier, I still only have a generic and basic understanding but I think it is important for system administrators to know how to commit code to projects and how to keep track of their own code and projects as well.  Even if the code is only small scripting projects it is still beneficial to publish since github is a public site.  Potential colleagues and employers can browse through your work, which increases your visibility and can even lead to job offers.

If you’re interested in learning more, I found this site to be a great resource.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

Enable telnet with PowerShell

Every once and awhile you will probably encounter a situation where you need to enable and then use telnet in a security focused environment.  In certain situations telnet can be a great tool to test the functionality of firewall rule. Iif you aren’t certain whether or not a rule is working telnet can be a great way to help debug.  The problem in Server 2008 and above is that telnet isn’t enabled by default.  Luckily with PowerShell it is easy to enable the telnet functionality.

The following set of commands is a quick depiction of how you can enable telnet from a PowerShell prompt to ensure the ability of testing certain ports.  Try it out.

Import-Module servermanager
Add-WindowsFeature telnet-client

Bam!  As always, it is always easier to stay in command prompt and this is a great way to test port connectivity.  I can understand why telnet is disabled by default on fresh server builds but sometimes it can become useful to have telnet as a tool to test connectivity.  If you would like to debate the merits of disabling/enabling telnet on a server just drop me a line, I obviously will not be focusing on this aspect here.  Anyway, just as easily as it is to enable telnet through PowerShell it can be disabled with the following command.  If you already have the server manager module imported, skip to the second command.

Import-Module servermanager
Remove-WindowsFeature telnet-client

That’s all it takes.  Very simple and very straightforward.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.