Security

Patching CVE-6271 (shellshock) with Chef

Josh Reichardt 2 Comments

If you haven’t heard the news yet, a recently disclosed vulnerability has been released that exploits environmental variables in bash.  This has some far reaching implications because bash is so widespread and runs on many different types of devices, for example network gear, routers, switches, firewalls, etc.  If that doesn’t scare you then you probably don’t need to finish reading this article.  For more information you can check out this article that helped to break the story.

I have been seeing a lot “OMG the world is on fire, patch patch patch!” posts and sentiment surrounding this recently disclosed vulnerability, but basically have not seen anybody taking the time to explain how to patch and fix this issue.  It is not a difficult fix but it might not be obvious to the more casual user or those who do not have a sysadmin or security background.

Debian/Ubuntu:

Use the following commands to search through your installed packages for the correct package release.  You can check the Ubuntu USN for versions.

dpkg -l | grep '^ii' or
dpkg-query --show bash

If you are on Ubuntu 12.04 you will need update to the following version:

bash    4.2-2ubuntu2.3

If you are on Ubuntu 13.10, and have this package (or below), you are vulnerable.  Update to 14.04!

 bash 4.2-5ubuntu3

If you are on Ubuntu 14.04, be sure to update to the most recently patched patch.

bash 4.3-7ubuntu1.3

Luckily, the update process is pretty straight forward.

apt-get update
apt-get --only-upgrade install bash

If you have the luxury of managing your environment with some sort of automation or configuration management tool (get this in place if you don’t have it already!) then this process can be managed quite efficiently.

It is easy to check if a server that is being managed by Chef has the vulnerability by using knife search:

knife search node 'bash_shellshock_vulnerable:true'

From here you could createa a recipe to patch the servers or fix each one by hand.  Another cool trick is that you can blast out the update to Debian based servers with the following command:

knife ssh 'platform_family:debian' 'sudo apt-get update; sudo apt-get install -y bash'; knife ssh 'platform_family:redhat' 'sudo yum -y install bash'

This will iterate over every server in your Chef server environment that is in the Debain family (including Ubuntu) or RHEL family (including CentOS) and update the server packages so that the latest patched bash version gets pulled down and then gets updated to the latest version.

You may need to tweak the syntax a little, -x to override the ssh user and -i to feed an identity file.  This is so much faster than manually installing the update on all your servers or even fiddling around with a tool like Fabric, which is still better than nothing.

One caveat to note:  If you are not on an LTS version of Ubuntu, you will need to upgrade your server(s) first to an LTS, either 12.04 or 14.04 to qualify for this patch.  Ubuntu 13.10 went out of support in August which was about a month ago as per the time of this writing so you will want to get your OS up date.

One more thing:  The early patches to address this vulnerability did not entirely fix the issue, so make sure that you have the correct patch installed.  If you patched right away there is a good chance you may still be vulnerable, so simply rerun your knife ssh command to reapply the newest patch, now that the dust is beginning to settle.

Outside of this vulnerability, it is a good idea to get your OS on an Ubuntu LTS version anyway to continue receiving critical updates for software as well as security patches for a longer duration than the normal, 6 month release cycle of the server distribution.

Read More

Set up PEM key authentication

Josh Reichardt 1 Comment

Many times it is useful to keys to authenticate to your servers.  This can dramatically improve security and is a great way to manage servers in bulk as well.  You just need to keep track of your keys rather than having to remember a large number of passwords.  The steps to get PEM key authentication are fairly straight forward but it never hurts to walk through the process of getting them set up correctly.

Side Note: I’d like to also mention briefly, that I have these steps set up to work with Chef, so every server that gets deployed using Chef will use PEM keys out of the box, which works out very nicely.  If you’re interested I can expound on that topic a little more, just let me know.

The first step in the process is to generate some keys using openssl.  If you don’t have openssl go download and install it.  If you do have openssl but haven’t updated in ahwile, please update to avoid the heartbleed vulnerability that was recently exploited (nearly all distributors have released the patched version at this point so it should be trivial).

We want to generate our key and create a PEM file out of it.  Here are the steps:

cd ~/.ssh
ssh-keygen -t dsa -b 1024
openssl dsa -in id_dsa -outform pem > test.pem
cat ida_dsa >> authorized_keys

You can leave the values blank (default) in the ssh-keygen.

Now you should have similar listings in your ~/.ssh directory:

ssh keys

  • authorized_keys – This is the public key that the pem file gets authenticated against
  • id_dsa – This is the private portion of the key that we generated in the steps above
  • id_dsa.pub – This is the public key section that is used when authenticating
  • test.pem – this is the file that will be used to authenticate.  Essentially the private key minus the pass phrase

Now you just need to copy the test.pem file that was just generated to a different host in order to log in with your PEM key using scp or rsync.  Once that is done, the command to connect to the remote host using  your key should look similar to the following:

ssh -i /path/to/pem user@server-name

Next steps.  At this point you should have a working pem authentication on your server.  It is probably a good idea at this point to start looking at hardening the security as well as the SSH configuration on the host.  Small things can go a long way.  For example disabling root login, disabling password authentication, etc. will stop a very large amount of attacks from hitting your server now that you are authenticating with pem keys.

Read More

Using a self signed cert with Nginx

Josh Reichardt 7 Comments

After the recent heart bleed incident (which I’m sure many of you well remember) I had to reassign some certificates. It turns out that this was a great opportunity to create a blog post.  Since I do not create and assign certs very frequently it is a good opportunity to take some notes and hopefully ease the process for others.  After patching the vulnerable version of Openssl, there are really only a few steps needed to accomplish this.  Assuming you already have nginx installed, which is trivial to do on Ubuntu, the first step is to create the necessary crt and key files.

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/cert.key -out /etc/nginx/cert.crt

Next you will need to tell nginx to load up you new certs in its config.  Here is an example of what the server block in you /etc/nginx/site-available config might look like.  Notice the ssl_certificate and ssl_certificate_key files correspond to the cert files we created above, which we stuck in the /etc/nginx directory.  If you decide to place these certs in a different location you will need to modify your config file to reflect the location.

server {

listen *:443; 
ssl on; 
ssl_certificate cert.crt; 
ssl_certificate_key cert.key; 
ssl_session_timeout 5m; 
ssl_protocols SSLv3 TLSv1; 
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP; 
ssl_prefer_server_ciphers on;

}

Just to cover all our bases here we will also redirect any requests that come in to port 80 (default web) back to 443 for ssl.  The is a simple addition and will add an additional layer of security.

server { 
listen 80; 
return 301 https://$host$request_uri; 
}

The final step is to reload your configuration and test to make sure everything works.

sudo service nginx reload

If your nginx fails to reload, more than likely there is some sort of configuration or syntax error in your config file.  Comb through it for any potential errors or mistakes.  Once your config is loaded properly you can check your handy work by attempting to hit your site using http://.  If your config is working properly it should automatically redirect you to https://.

That’s all it takes.  I think it might be a good exercise to try something like this with Chef but for now this process works okay by hand.  Let me know what you think or if this can be improved.

Read More

Wireshark Reference Guide

Josh Reichardt 10 Comments

Based on a strange network problem recently I decided to put together some quick notes and a few tips on ways to improve your Wireshark experience based on my own experience with it.  There are many, many more features that Wireshark has to offer, these just happen to be the most apparent ones I have found so far.  Wireshark is extremely powerful and therefore extremely useful if used properly.  At first it takes a while to get used to everything Wireshark has to offer but once you start to get the hang of how things work then it can be a great network troubleshooting tool.  Basic knowledge of networking concepts should be assumed as well as familiarity of Wireshark for those who attempt to debug network problems using this tool.

Here is a list of some of the most common and handy features that you can utilize in Wireshark.  I am not going to dive into great detail with most of  these items because I honestly don’t have a ton of experience with all of them, I basically just wanted to point out the highlights.

  • Filtering in Wireshark is very handy.
  • Create custom profiles for different use cases (quickly select from bottom right hand corner).
  • Color filters are useful!  (Right click a field in the packet trace and selelct colorized rule)  The bottom left bar will tell you what variable you are looking at to make things easier when customizing.
  • Use Regex in wireshark using the “matches” clause to turn on regex patterns.
  • You can extract specific information from trace files on the command line using tshark.
  • Right click a packet and select “follow TCP/UDP stream” to debug a single network conversation.
  • Low delta times are good.  If you see high deltas you should probably investigate things.

Here are some more concrete examples and a few basics of how to put these tips into practice.  In Wireshark you can use either English or code like operators when filtering to help narrow down traffic and interesting networking patterns and issues.  So for example, “==” and “eq” will behave in the same manner when applying filters.  Other operators include <,>, !=, <=, >=, etc.  Just like you would see in a typical programming language.

Use custom configuration profiles.  If you look at packet traces often this will save you a tremendous amount of time if you are looking at specific types of traffic or are only interested in certain traffic patterns.  For example, you spend a lot of time looking at many traces that fit the same type of criteria; by using custom profiles you can quickly adjust and modify the view in Wireshark to help quickly identify patterns and potentially issues by cycling through different, specific views.  To begin creating custom profiles go to Edit -> Configuration Profiles and then either select the custom profile or create a new one to begin changing.

One handy trick is to disable TCP offload checks.  If your packet captures are getting clogged up with a bunch of red and black with offload errors, this is place you should go to look first.  There are a few places where this option can either be enabled or disabled.  The easiest way to check these options is under Edit -> Preferences and then under the Protocols tree for UDP, TCP and IPv4 protocols.  The example below shows what the options should look like for the TCP protocol.  The TCP and UDP offload checks are disabled by default but the IPv4 needs to be manually unchecked.  The specific option under the IPv4 protocol is labeled “Validate the IPv4 checksum if possible”, simply uncheck this and the red and black errors should disappear.

There is a capture option that allows you to resolve IP addresses to hostname, which I find can be very useful.  To enable this option open up the Show capture options screen, there should be an option in there under name resolution called “Resolve network-layer names”.  Simply check that box and you should have name resolution.

As mentioned in the bullets above, the “follow UDP/TCP stream” option can be extremely useful and is a very quick way to glean information.  It is so useful because it is so easy to use.  Simply find a traffic conversation you would like to debug and right click the packet number in the top Wireshark pane and choose the follow UDP/TCP stream option and you can get an idea of everything that happened during a particular conversation.  For example, using this technique you can follow FTP transactions.

Viewing a breakdown of the packet flow and traffic patterns can be a useful tool as well when diagnosing various network issues.  There is an option in Wireshark that shows in good detail the breakdown of various packets and protocols that can be used to troubelshoot the network.  This option is called Protocol Hierarchy Statistics and can be found un te Statistics -> Protocol Hierarchy Statistics page.

Only look at traffic for one IP address:

ip.addr==192.168.103.104

Likewise, filter out all traffic from an IP address:

!(ip.addr == x.x.x.x)
!(ip.addr == 1.2.3.4)

filter out all traffic for a specific port

!(tcp.port eq 2222)

Resources:

http://ask.wireshark.org/questions/
http://ask.wireshark.org/questions/
http://www.wireshark.org/docs/

Read More

Locking down your WordPress site

Josh Reichardt 8 Comments

Since I don’t really want to get in trouble for this, I need to put in a disclaimer.  Some of these tools can be invasive and if you are running them against somebody then I take no responsibility for their actions against you.  I am testing these tools against my own site so the consequences are minimal.  Just be aware that there can be serious consequences for using these tools on sites and companies against their will.  I don’t want anybody going to jail.

The tools

Let’s take a poke around with WP Scan.  This tool is a WordPress vulnerability scanner, often packaged together with Backtrack or the newer Kali Linux pentesting distro.   WPScan helps find and eliminate security weaknesses in your WordPress site.  More information about this tool can be found here.

There are many other tools out there but for basic WordPress scanning this tool should suffice, because it offers a number of things that are of interest in a nice single tidy and clean interface.  Other tools that may be of interest include tools like Burp Suite, SQLmap, username enumeration through Metasploit and other reconnaissance tools.

The process

Most real world attacks will reach for the low hanging fruit when it comes to exploiting WordPress sites, typically gaining access to a site through password exploitation.  With so many WordPress sites going up it becomes easy to move from site to site trying different password brute forcing attacks, so that’s where you will see a large number of attacks.  There are others as well, such as vulnerability attacks, SQL injection attacks, XSS, etc.

To begin the process let’s start gathering some information about the WordPress site that will be the focus of this attack, my blog.  Here, I am running WPScan through Kali Linux, so the syntax may change depending on how you decide to use this tool.  Let’s see what basic information we can get about my blog.  This site scan will attempt to gather the basics of the site it is scanning.  For help just type ‘wpscan –help’.

wpscan --url http://thepracticalsysadmin.com

Let’s see how far we can get with the password brute forcing method.  To enumerate a list of user account names use the following,

wpscan --url http://thepracticalsysadmin.com --enumerate u

If you get any interesting results from this scan, for example the result returns the username admin, go ahead and see if you can brute force the account.

wpscan --url http://thepracticalsysadmin.com --wordlist /pentest/passwords/wordlists/darkc0de.lst --username admin

There are more features packed in this tool so take some time to explore what all it can do (preferably on a test box).  Odds are that on a site that hasn’t been properly locked down you can probably get in, one way or another.  I wouldn’t recommend running wpscan against this site though because I have already beefed up the security and temporarily block access if users run malicious scans against the site.

Locking it down

There are a number of techniques to help reduce the attack surface for your WordPress site as well as methods to increase the difficulty of breaching your site.  The first and foremost is the use of strong passwords.  That should be a given and I won’t get into the details here of how important strong passwords are.  Another (hopefully) obvious technique is to keep up to date with your patches.  Whether it be on the Operating System or your WordPress site/plugins you should try to be proactive about patching your systems.  The third and final obvious solution I will mention are getting good backups.  If your site does get compromised then it is incredibly helpful to have a point in time to go back to rather than starting over from square one.  There are plugins designed to help with this process and even doing it by hand isn’t that difficult.  You can get back on your feet even if you only have a database dump from your site at some point in the past.

I’d like to specifically mention some good tools to use if you have publicly facing SSH; one of which is fail2ban.  This tool can be used as a layer of defense to slow attackers down by detecting malicious activity and banning IP addresses.  Another great tool, a handy plugin for WordPress  sites is called Better WP Security.  This is an easy to use site hardening tool that can fill up weaknesses and security holes quickly for somebody that doesn’t necessarily have security in the foreground of their minds.

By utilizing  these basic techniques you will infinitely increase your WordPress site’s security and make it much more difficult to attack and exploit.  There are of course other techniques to improve security but at a certain point it can become a balancing act.  *Most* site admins aren’t overly conscious about security and so do not spend a lot of time on their security efforts, they are more concerned about the content and getting things up.  Likewise, some are probably more prone to lock things down more than they perhaps need to.  It is important to maximize your effort, and to cover the most important security aspects by implementing the basics.

Read More