Design Group Policy for easy troubleshooting

I tend to see a lot of one off fixes for setting up and fixing group policies that either don’t exist or are intended for policies that are broken the majority of the time when I am looking up GP answers on teh google’s.  I recently watched a great video over at the channel9 website by Daren Mar-Elia of GPOguy fame about using best practices and design principles for managing your Group Policy environment.  Here is the link to that video.

That video really got me thinking about the topic of how I could improve my GP management skills in my day to day environment.  So I decided that I would take as many offerings from his talk and elsewhere in my searches across the interwebz to help come up with some of my own best practices and guidelines for managing Group Policy.

The following is an overview of the ideas and techniques that I came up with and what has worked well in my experience with regards to managing Group Policy.

Group Policy organizational best practices:

  • Use either a “U” “S” or “C” to denote whether Group policy is User, Server or Computer
  • Tack on a version at the end of the specific Group Policy.  Brand new Group Policies begin at v1.0
  • Every time a policy changes increment the version number.  It makes things easier to troubleshoot when using gpresult with this method
  • Each GPO has one specific use case.  DO NOT LUMP MULTIPLE FUNCTIONS INTO ONE POLICY
  • Use very detailed and descriptive names to denote what a GPO is and does

Here are some example policies that I have been working on in a test environment.  I think it captures many of these above best practices quite nicely.  Please feel free to adapt this technique to suit your own specific needs, this is only a template and I’d like to see how it can be improved.

Group Policy best practices

As you can see, using this format it is easy to tell whether or not this is a computer policy, what specifically the policy is doing and which version of the policy we’re at currently.

The most crucial part of using this system is to get other Group Policy admins to buy in to this technique.  If you don’t clearly lay out your expectations then keeping policies up to date and organized could potentially become a pain point looking on down the road.  The other caveat is to get the other GP admins in the habit of creating policies that address only one specific task, that are broken into either user or computer policies and have descriptive names.  If the environment utilizes multi-purpose policies that contain both user and computer specific settings then this may be a new concept for many of the admins but the extra effort in setting this type of environment up will be totally worth the extra overhead initially.

I definitely think that this technique can be improved and I am always tinkering with it to see how I can get it to work better but for now it is at a good point.  If you make the transition to organizing and improving your management of Group Policy or just have some solid best practices of your own already let me know, I would love to hear about what you are doing and how to incorporate more techniques into my own management style.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

Gathering Exchange 2010 mail flow statistics

There are times when it can be useful and beneficial to have a good grasp on the details of what kind of mail traffic is running through your Exchange environment.  Recently I have been tasked with coming up with some environmental statistics for our Exchange 2010 servers to help size a new project we are starting soon.  There are a few different tools to help gather this information that I’d like to briefly go over today.  Before I start I’d like to point out that most of this stuff I am borrowing from others, however I think it is valuable to know how to do this type of thing.  With that said, I’m definitely not trying to take credit for any of these techniques, just trying to show the benefits.

There are a few different tools that will help to get a handle on your Exchange environment.  The first and quickest way to peer into your Exchange environment for some quick high level overview statistics is to use PowerShell.

The following command can be used to grab some basics stastics such as the total mailbox size, average maiblbox size, the max and the minimum sizes in your environment.

Get-Mailbox -Database MBDB1 | Get-MailboxStatistics | %{$_.TotalItemSize.Value.ToMB()} | Measure-Object -sum -average -max -min

It is important to note however that this command can take some time to complete and can be an intensive process because there are so many calculations going on, just be careful that you don’t crash anything.  This command may not be viable if the environment is enormous but if that is the case you probably don’t need to use any of these techniques anyway.

The next useful tool to gather up mail flow information uses the Microsoft Log Parser tool, which can be downloaded here.  The log parser basically allows us to query the Exchange message transport logs to pull out interesting information.  I found a great blog post that describes the process of using the log parser tool to query the message tracking logs to help determine daily send and receive traffic in your Exchange environment.  You can find the blog post here and I have it reference at the end of this article as well.

There are a few tricks however that I would like to mention because a few things in the blog post aren’t exactly obvious.  After downloading and installing the Log Parser you must run the command he has listed on his site using CMD, otherwise you will have to modify his commands to use PowerShell.

For this command to work correctly you must also navigate to the correct location where the transport logs are being stored.  In the default install of Exchange they are stored in:

C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\MessageTracking

So after you navigate to the correct location you run the command:

"C:\Program Files (x86)\Log Parser 2.2\logparser.exe" "SELECT TO_LOCALTIME(TO_TIMESTAMP(EXTRACT_PREFIX(TO_STRING([#Fields: date-time]),0,'T'), 'yyyy-MM-dd')) AS Date, COUNT(*) AS Hits from *.log where (event-id='RECEIVE') GROUP BY Date ORDER BY Date ASC" -i:CSV -nSkipLines:4 -rtp:-1

This will output the total number of send/receive messages for each date for the last 30 days on that particular server.  Another important thing to keep in mind is that you need to run this command on each server that has either the Hub Transport or Edge Transport role installed because each server houses a unique set of log files.

The last technique I’d like to go over for gathering interesting Exchange mail flow information is a script I found online, which can found here.  This is a very robust script that gathers a lot of specific information for a particular set of logs files.  Essentially this script functions similarly to the above Log Parser, except it grabs a lot more detail for a particular date.

This is easy to get working, just copy the script from the link into a .ps1 file and save it to a server that has the Exchange Management Shell installed on it.  If the EMS is not installed then this script will not function correctly.  The script will output some interesting details for each individual user including things like:

  • Username
  • Messages sent/received
  • Total MB sent/received
  • Internal sent/received stats
  • Unique messages sent

And output this information into a CSV file so it easy to manipulate the data at that point.  This kind of stuff is very useful in helping to determine things like average sent and received message size for example, I have not been able to provide that information to management easily until I found this script.

There are more techniques out there I’m sure, maybe even software that helps gather these sorts of stastics and information but for a quick and dirty way to grab some high level statistics you can’t really beat these techniques.  These methods are quick and will get you the information you need, which more often than not seems to be at least as detailed as the people requesting this information are looking for which is a win-win for everybody.  If you have any other input or questions about mail flow statistics feel free to let me know.

Resources:

http://exchangeserverpro.com/daily-email-traffic-message-tracking-log-parser/
http://exchangeserverpro.com/exchange-2010-message-tracking/
http://gallery.technet.microsoft.com/scriptcenter/bb94b422-eb9e-4c53-a454-f7da6ddfb5d6

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

.NET framework 3.5 source files could not be found

If you are having trouble installing previous versions of the .NET framework on Windows Server 2012 and above, or are receiving the following error listed below, use this set of instructions.

Installation failing

Find and mount the original Windows Server 2012 ISO.  An important thing to make note of is the drive letter that the image gets mounted as.

Mount the ISO

Once the ISO is mounted and we have correctly identified the drive path we need to enable the .NET Framework.  This can be done through command line using the dism.exe command or alternatively throught the Windows GUI.

First we need to enable the .NET features on the server.  Notice in the command I am using the F: drive.

dism.exe /online /enable-feature /featurename:NetFX3ServerFeatures /Source:F:\sources\sxs

Next, to enable the specific .NET framework we want just use the following command:

dism.exe /online /enable-feature /featurename:NetFX3 /Source:F:\sources\sxs /LimitAccess

Successful completion of this command should look similar to the output below.

enable .net

Using the GUI the instructions are similar and pretty straightforward.

  1. Insert Windows Server 2012 DVD or ISO.
  2. Open Add Roles and Features Wizard.
  3. Select .NET Framework 3.5 Features and then click next.
  4. Select “Specify an alternate source path link in the Confirm installation selections screen”.
  5. Path: F:\Sources\SxS and then click OK.
  6. Click the Install button.

That should be it.  Definitely an easy fix but not very obvious.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

How I do internal documentation

Intro

Whether you are new to system administration or have spent some time in the role, getting a grip on documentation can be tricky and often times challenging.  I can speak from experience because when I started my career I had no idea how to do this stuff, but slowly over time I have been able to develop some useful techniques.  It also helps that I am obsessed with documentation.  Therefore the techniques I use are always changing and I am always looking for ways to improve my documentation skills.  I think the most important take away from what I have learned, is that there is no end all be all way to do documentation and everybody will probably find slightly different ways.  My method may or may not work for you but hopefully you can  look at it and use some of the underlying concepts to help with your own style.  I have been able to use my own documentation procedures over the past 3 years to really improve my job performance as well as other areas in my job so hopefully there is something in here you can apply to your own situation.

Once you get past the initial stage of how uncomfortable it is to document everything by your mindset to how useful it is to having a living reference for everything on the network, then documentation really becomes much more manageable.  The added bonus of solid documentation is that once you have a decent repository built up you don’t have to worry about constantly adding new items to your documentation.  I am at a point now that I primarily use my documentation as a reference and don’t find myself having to create or update things nearly as often as I used to.  The other main benefit of having solid documentation is to help others on your team, especially junior members so they can go out and learn about certain intricacies or specific details of systems in your environment.  It’s also good for avoiding situations like the hit by a bus scenarios.  The general idea is that by having a great piece of documentation, somebody can pick up where you left of with minimal downtime and impact.  Hence, if you were to get hit by a bus and/or become incapacitated then somebody else could get up to speed with the way things work (what, when, why, how) using your documentation as a reference without having to spend hours and hours of time to figure out what exact information that they needed to know.

Documentation Overview

There are really two main types of internal documentation, internal and external.  Public facing (external) documentation is an entirely separate topic and I will not be discussing it here.  Network and technical documentation (internal) are the two big ones that I use to reference typically.  Network documentation is essentially working with Visio to produce anything and everything related to the network.  If you want to be a good system/network administrator a great way to help is to get good at working with Visio.  The other type of documentation I like to refer to as “technical documentation” is typically anything and everything that I have found to be helpful in solving a problem.  As I mentioned earlier, the longer your work on your environment and building your documentation the less you will have to produce new documentation. This information comes from the documentation section in Limoncelli’s book Time Management for System Administrators which is a fantastic book for sysadmins if you haven’t already taken a look.

My own technical documentation has grown organically in the 3 years, from just a few pieces of information to essentially a living library of all of the different systems I and others on my team work with on a regular basis.  Everything I do is in there; from technical procedures, links to external blogs and websites, PDF’s, to pretty much anything else you can imagine that would be useful.  One of the hardest things about working on documentation in a collaborative environment is getting people to buy in to the system.  The best way I have found to handle this is to take the initiative, start the process and then let everyone know about it what tools you are using and how awesome this documentation is and hope that it sticks.  I am not a fan of forcing people to do things but by showing them an amazingly simple system that is also incredibly useful the goal is to get them excited and motivated to use your documentation system.  Sometimes it works, sometimes not, but I have found it to be the best approach when attempting to change people’s habits and get them used to working with something new.

In the remainder of this post I will go into some of the details of how I organize my technical documentation and use it to help quickly identify and fix problems quickly.  I don’t plan on covering network documentation in this post as it is a slightly different beast.

The Tool

I have used pretty much every documentation system you can think of, from SharePoint, EverNote, Dropbox, various Wiki’s, Office documents, etc. and through my time and experience with all of these various tools I have found one to be the best for syadmin documentation.  I really believe that many of these tools serve a certain purpose or function but in my time as a sysadmin I have found OneNote to be the hands down best tools for system administration documentation.  As a side note, Wiki’s are a great alternative if you don’t have access to OneNote but my preference lies with OneNote.

OneNote is flexible, you can set up a location on a file share inside your company and allow access by certain AD users or groups to the share to control who can see and write to and collaborate on your documentation system.  It is very handy to be able to work on some notes while in a meeting on my laptop then move back to my desktop and have these notes current and in sync between my machines.  It is also possible for multiple people to be in OneNote working live on different notes, which is also nice when you are working in a busy team environment.  I know there are many other products that allow this so even if you don’t choose OneNote I would highly recommend a tool that offers these features.

Organization

I have OneNote grouped in a specific way to help increase my productivity.  I have different notebooks for various items.  I have a notebook for my technical documentation, a task log, one for projects and meetings that eventually get merged into my tech docs if they grow and the project/meeting ever materializes into anything more than just a few notes and I also have a notebook for personal work items.  The personal notebook contains a few different notes for personal accomplishments, personal agenda and ideas for projects and improvement.

Personal Notebook

Within the tech docs notebook I have notes for all of the major technologies that I or members of my team work with on a regular basis.  Basically, anything that we deem noteworthy gets thrown into one of the categories somewhere inside the tech doc.  The structure of this document grows organically for me with a few minor pain points here and there along the way.  For example, when a new solution to a problem is found and there isn’t a specific category for the fix, sometimes it is necessary to either create a new category or just use your judgement when determining the best place to store the fix.  Luckily the search feature in OneNote works well so if you can’t remember where you put something then you can just search through everything to find it.  Coincidentally, the larger your docs grow to be the better the search function works.

Each category has a basic general structure guideline that I have found to improve the workflow and allows things to work more smoothly.  The structure applies to every page I can use it on and is a template that includes the  most common and hard to find information that I use as a template.  The items in each category each get their own note and are as follows:

  • Network Information – IP’s, DNS names, server and network roles, anything that plany a part in the network goes here.
  • Support/Contact – Direct software and product support numbers, vendor contacts, important email addresses, account ID’s and service agreements, licenses, contract numbers, etc.
  • Resources – Any links to relevant and important technical docs, deployment guides, implementation guide or admin docs that are either internally or online.
  • Commands – I work in a Windows environment so often times PowerShell is the go to resource.  I keep a table of all the commands that I deem to be useful here for that particular category.
  • Useful tools – This one is optional but for some categories is a nice little reference.  I use this section to help identify anything and everything that I can use as a tool to help with a particular category.

This information is the skeleton that I begin to model all of the other documentation around.  From there, the rest is easy and should essentially fall into place.  Most of the time for me, these other items are one off fixes for problems that I have found or have solved.  Sometimes a quick link to the blog post that helped will be a good enough reference and other times a painfully detailed set of steps is necessary for documenting the procedure for the fix.

Documentation template

Style

The actual documentation is more of an art form than anything else that gets better with practice.  Sometimes technical notes require an obscene amount of detail because of how complex they are and because the procedure  only needs to take place once or twice in a year.  Other notes just need a rough explanation and therefore don’t need any detail at all, it all depends on the situation.  As has been pointed out in the comments section, one good approach to documentation sometimes is to assume the reader has no prior knowledge about the topic and therefore painful explanation and detail are necessary to convey your materials.  The point is, there is no cookie cutter way to do all of your technical documentation and various tactics and techniques need to be developed for different types of issues, that is why documentation becomes an art form in my view.

Going back to my own technique, one of the most helpful tricks I have found is to create the commands section, which I throw all of the most useful and common commands into for the particular category I am referencing.  The more work you do from the command line, the more useful this note will be to you.  For example, being in a Windows environment, I work with Exchange pretty much daily and having a repository of all the useful commands that I need in one place is one of the best ways to save time rather then going through Google looking for the specific information I am looking for every time.  Here is a glimpse of what I am talking about.

PowerShell commands

Another thing that will help your documentation immensely is consistency.  I am talking broadly here but when you create a living document you will want to have some sort of consistent style across your documentation.  Items that come to mind here are things like having a consistent template as mentioned earlier, consistent naming conventions, fonts, naming conventions, established criteria for creating topics and notes, capitalization, fonts and organization, etc.  Having this general style guide established early on will help to make finding and reading information in your documents much easier and will consequently help to save time when you are looking for specific information.

Closing

That’s all I’ve got for now.  I just wanted to point out a few things and get people pointed in the right direction that are looking for ideas of how to get going on technical documentation.  As I mentioned, there’s more than one way to skin a cat – meaning there are a large number of ways to do documentation and there isn’t necessarily one best way to do things.  I have shown you my preferred way, but it may not be the best way for you.

Documentation is as much of a learning process as anything else and sometimes you just need to experiment with things and just spend some time wading around to get the best results.  One great way to check if your documentation is working or not is to get somebody from your team to attempt to use your documentation to fix something you have instructions for.  If you just want to get some practice writing clear and concise instructions, try this method on yourself.  Write out the doc and procedure and come back to the instructions a week or month later and see how hard it is to figure out how to fix the problem.  Chances are, if things are unclear or your teammate is unable to fix the issue then it is probably a good idea to take a look at reevaluating how you’re doing your documentation.

If you have any thoughts, suggestions, ideas or anything else just let me know.  Like I said earlier, I am obsessed with this kind of stuff and I am always looking for ways to improve my own processes and procedures.  I plan on coming back to this post and update it in the future if any of my documentation practices change but for the time being, I hope you find this information useful and applicable to your own documentation processes.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.

Some quick tips for getting ahead

Do you want to get ahead in this field?  In my experience in the industry, there are a lot of great ways to improve yourself and to put yourself in position to get ahead.  One of the best ways is to show that you are interested in your craft.  There are numerous ways to demonstrate this desire, and dedication; a simple yet great way to do so is by attending industry related conferences and networking events.  Pay for these events as well as certification courses out of pocket if your employer is unwilling to do so.  Paying for your own career betterment out of your own pocket shows your level of commitment to those around you that you may meet at these events as well as those that you work with.  I guarantee you people will notice this.  I would also like to mention that if you are interested in training, certification, or anything the will help you do your job better and employers aren’t willing to help pay for any of these opportunities then often times it is a sign that you have outgrown your current position or will be in the not so distant future.

Another great area to focus on is to never stop learning.  This applies not only to IT but to everything that you do in your life.  Taking the time to learn something new is a great way to help boost your career.  Interested in business?  Take a business class or at least start researching different aspects of business and find what interests you.  Then leverage that newly found knowledge in every day aspects of your life.  I am a firm believer in education overlapping and translating skills from one area to contribute to others.  If you learn a seemingly unrelated skill you never know when and where it will be applicable and it will all contribute to furthering your education and improving your knowledge.  It can be devotion to skills like Documentation, attention to detail, writing, programming, whatever it may be will help to improve and strengthen your IT skills, I promise.  Many of these skills are things that I work on improving outside of my life in IT but certainly they contribute to the success I have experienced in my career.

The final tip I’d like to mention is doing things that others either aren’t willing to do or are not aware of doing.  This is a more subtle point but I believe it is something that will separate people that want to be good at something from those that are content with where they are at in life.  For example, many individuals in IT don’t like to leave their comfort zone for many things.  Learn how to talk to people and network with others in the industry who have already discovered how to get ahead.  They will give you so many great pieces of information and can potentially help you get into new positions as well.  We all know that social skills are not a strong suit for many that are in the IT profession so learning how to talk to people is incredibly valuable.  Yes it will feel awkward and unnatural at first but the more you do it and the more you practice it (just like anything else) the better you will get at it.  And since it is so uncomfortable for many individuals they simply won’t attempt it because it is out of their comfort zone.  There are many other examples of simple ways to get ahead but just by knowing how to leverage and utilize things like networking and being social will help your career more than you might realize.

I am taking a different approach from the usual subject matter, I have just found it difficult recently to find anything technically interesting enough to write about.  I would love to cover more of these areas if there is any interest in the future, I think its an oft missed subject in the IT profession and I happen to like writing about it so hopefully I can get some positive feedback.  As always, if you have anything that you find interesting or think would be a positive contribution let me know, I’m always looking for guest posters and guest content and I welcome the fresh perspective and ideas.

About the Author: Josh Reichardt

Josh is the creator of this blog, a system administrator and a contributor to other technology communities such as /r/sysadmin and Ops School. You can also find him on Twitter and Facebook.