Document Storage: Part 3

November 29, 2012August 31, 2015 James Cort 15 Comments

Document Storage Project

This is Part 3: Configuring Apache.

We’re only looking for a fairly simple interface to browse through documents. Apache already gives us that – you just need to enable a feature called “Indexes”. But the default indexing is pretty ugly; it’d be nice to make it look a little prettier and maybe add scope for expanding on functionality.

Initially I was going to design my own style, but it turns out someone’s already done that and he’s done a better job than I could ever hope to. So I took the style setup from Recursive Design and tweaked it slightly to fit in with what we’re doing here.

  cd /home/http/assets
  svn co http://recursive-design.com/svn/misc/apache/index-style
  mv index-style/* . 
  rmdir index-style

The stylesheet really ought to be referenced with a specific path so it can always be found. Edit /home/http/assets/header.html and change the stylesheet reference thus:

Next up, we need to configure Apache. Ignore the instructions on the Recursive Design blog; things are slightly different here. Edit /etc/apache2/sites-available/default as follows:


        ServerAdmin (YOUR EMAIL ADDRESS HERE)

        DocumentRoot /home/http/documents
        
                Options FollowSymLinks
                AllowOverride None
        
        
                AllowOverride None
                Options Indexes
                DirectoryIndex index.html index.php
                IndexOptions FancyIndexing
                IndexOptions VersionSort
                IndexOptions HTMLTable
                IndexOptions FoldersFirst
                IndexOptions IconsAreLinks
                IndexOptions IgnoreCase
                IndexOptions SuppressDescription
                IndexOptions SuppressHTMLPreamble
                IndexOptions XHTML
                IndexOptions IconWidth=16
                IndexOptions IconHeight=16
                IndexOptions NameWidth=*
                IndexOrderDefault Descending Name
                HeaderName /assets/header.html
                ReadmeName /assets/footer.html
                Order allow,deny
                Allow from all
        

        Alias /assets /home/http/assets
        Alias /search /home/http/search

        
                AllowOverride None
                Order allow,deny
                Allow from all
        

        
                AllowOverride None
                Order allow,deny
                Allow from all
        

        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/

 ..... LEAVE THE REST OF THE FILE ALONE ......

Restart Apache.

Put something – ideally a PDF that was NOT generated from a scan but instead contains searchable text – into /home/http/documents and browse to http://(hostname).local from a separate PC on the network. If all goes according to plan, you should see something a bit like this:

There’s a lot more to do: we still need something that can index this little lot (so we can just punch in search terms) and we need some easy way to get documents onto the server. But they’re a topic for a future post…

Protip December: Customize your Powershell profile

November 27, 2012August 31, 2015 Josh Reichardt 12 Comments

Powershell is great but is a little boring if anything, out of the box, with its drab white foreground and all. It isn’t exactly informative either, so I wanted to show everybody a quick trick to customize this look and feel to make things look a little bit cleaner. Hopefully this introduction will demonstrate one of the many features that makes Powershell a great tool for Windows admins, which is its flexibility.

This customization file, called profile.ps1 can be located in one of two places.

The first location is the global location and would be useful when you want all users to have a customized Powershell profile. This profile should be placed in C:\WINDOWS\system32\WindowsPowerShell\v1.0\Profile.ps1.
The second location is for the local profile and would be specific to each user account. This file overrides the global configuration file and should be placed in C:\Username\My Documents\WindowsPowerShell\Profile.ps1.

By default these files don’t exist so you will have to navigate to the respective directory and create the initial, empty profile.ps1 file.

Once you have the file created just pop this chunk of code into your profile.ps1 file.

function prompt {
	$path = ""
	$pathbits = ([string]$pwd).split("\", [System.StringSplitOptions]::RemoveEmptyEntries)
	if($pathbits.length -eq 1) {
		$path = $pathbits[0] + "\"
	} else {
		$path = $pathbits[$pathbits.length - 1]
	}
	$userLocation = $env:username + '@' + [System.Environment]::MachineName + ' ' + $path
	$host.UI.RawUi.WindowTitle = $userLocation
    Write-Host($userLocation) -nonewline -foregroundcolor Green 

	Write-Host('>') -nonewline -foregroundcolor Green    
	return " "
}

Once you have this bit added you will need to reload Powershell and voila,

Customized powershell look — Here’s our customized Powershell look.

Much better. As you can see we now have some additional, out of the box information including:

Our current username – jreichardt
Our computer name – JOSH-TEST
Our current directory – separated by the | (pipe) symbol
As well as a nice green foreground font to help with the readability.

Pretty simple to get set up but definitely adds a lot to the default look and feel. Let me know if you have any other cool Powershell customization’s or tricks that you think are worth sharing.

Configure SNMP in Debian

November 25, 2012August 31, 2015 Josh Reichardt 11 Comments

This post is pretty straight forward but I want to mention there is a trick you have to use in Debian to get everything working correctly after you have all your SNMP packages installed. I didn’t realize this when I was setting this up the other day and it tripped me up for awhile.

So to start things off, we need SNMP and SNMPD on our systems.

sudo aptitude install snmp snmpd

We also need to update our SNMP settings to reflect the read only SNMP community string that we want to use. The default is public but it has been criticized for being susceptible to security breaches so you should probably keep that in mind when setting up SNMP in your environment.

At the very minimum your snmpd.conf file should look something like the following:

rocommunity mysnmpstring

Once you have updated this you need to unbind your localhost so that it can be read by others on the network. This is what tripped me up initially on my Debian box, I do not believe it is an issue in Ubuntu but if it is then you should be able to use these instructions as well. To fix this problem you need to edit the /etc/default/snmpd file and chop off the 127.0.0.1 from the SNMPDOPTS section. When it is fixed it should look like this:

SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -g snmp -I -smux -p /var/run/snmpd.pid'

Now you just need to restart the SNMP service:

service snmpd restart

You can check your handy work when you are done to make sure everything is working correctly by using this command from either the local host or another machine with SNMP installed on it.

snmpwalk -v1 -cpublic HOSTNAME/IP

Hopefully this will save time for somebody in the future, it certainly tricked me.

Document Storage: Part 2

November 23, 2012August 31, 2015 James Cort 12 Comments

Document Storage Project

Part 1: What we’re doing

This is Part 2: Setting up our base system.

I’m assuming that readers are already reasonably familiar with Linux and can generally find their way around OK. If I didn’t assume that, this set of instructions would probably wind up becoming a book!

I’m keeping it simple here by installing this on a spare PC I have hanging around. Things would be a little more complicated if this was on a shared host or a virtual server in a datacentre, but that’s beyond the scope of this project.

Install a base Debian Wheezy system. At the time of writing this is the “Testing” branch, which I wouldn’t ever deploy to a client. But this project is for me personally so I’m rather less bothered. You don’t need any extra software, so untick as much as you can.

Give the bulk of the disk space over to /home; keep 15-20 GB left over for /var.

In /home, create the following directories:

http
http/assets
http/search
http/documents
scripts
incoming

Run the following command to install the software we’ll need:

apt-get install tesseract-ocr bzip2 make ocaml gawk apache2 unzip php5 zip php5-gd mysql-server php5-mysql subversion inotify-tools imagemagick ghostscript exactimage openssh-server avahi-daemon

We now have:

A Linux box running Apache – and we shouldn’t even need DNS if we’re on the same subnet. Check it works by typing http://(hostname).local into your web browser.
Directories for our scripts, our static HTML, the document repository, incoming files for OCR’ing and scripts to carry out the OCR work.
Most of the software we’re going to need. There’s one or two things missing, but they’re so trivial that it’s hardly worth losing any sleep over them.

We still need:

To configure Apache to act as our file browser.
To integrate search functionality.
To sort out the scripts that are going to OCR incoming files.

Introduction to Irssi

November 21, 2012August 31, 2015 Josh Reichardt 10 Comments

If you have been a follower this blog, I wrote a post awhile back that described my preferred settings in tmux and just recently wrote a post about getting set up with bitlbee. Today we will be adding on another piece to what I will be calling my ultimate command line theme by introducing another useful command line tool for communications called Irssi. Here are the back posts to these if you missed them.

My Favorite Browser is Windows (guide to tmux)
Introduction to bitlbee

Now that we have that out of the way let’s talk about Irssi. Irssi is a console based IRC client that has been around for quite a while now. There is somewhat of a ~~debate~~ holy war as to which console based IRC client is the best. There are a number of hardcore Irssi users around that tout it is as the best, with the likewise being true for Weechat fans. Before going any further I will say that there is definitely a certain amount of leg work to get Irssi up and running with the full set of customizations and features. That said, I believe the extra work is worth every minute of time and effort if you are looking for a fully featured, rich IRC experience.

I want to present both of these clients (Irssi and Weechat) to readers and let each person decide for themselves which is the best, because saying one is better than the other wouldn’t be a fair comparison, and is really like comparing apples and orages. With that said, in a future post I will be going over the basics of using Weechat, the other touted console based IRC client.

Bitlbee in Irssi

Add an alias for Bitlbee.

/network add bitlbee
/server add -auto -network bitlbee localhost
/connect bitlbee

register
/oper
<desired password>

Automatically join and identify when Irssi starts.

/channel add -auto -botcmd '/say identify\; /oper' &bitlbee bitlbee

Add in your Gtalk account.

account add jabber [email protected]
/oper
<gmail password>

Set up correct port and ssl stuff for gtalk.

account jabber server talk.google.com:5223:ssl

Getting used to Irssi

Here we will assume that you have created and set up a user with irc.freenode.net. Once that step has been completed you should be able to follow these instructions without any issues.

/SERVER ADD -auto -network freenode irc.freenode.net

You may have to shutdown and restart Irssi at this point for it to recognize the network name “freenode” in the next step.

/CHANNEL ADD -auto ##/r/sysadmin freenode

Adding advanced_windowlist.pl

First we need to download the script and put it into the appropriate place. If you haven’t created your Irssi scripts directory and your autorun directory go ahead and make them quickly.

mkdir ~/.irssi/scripts
mkdir ~/.irssi/scripts/autorun

Change directories to your scripts directory and download the script.

cd ~/.irssi/scripts
wget http://anti.teamidiot.de/static/nei/*/Code/Irssi/adv_windowlist.pl

Let’s quickly set it to be executable.

sudo chmod +x adv_windowlist.pl

Now we need to symlink this script and then run it in Irssi. To symlink it run the following,

cd ~/.irssi/scripts/autorun
ln -s ../adv_windowlist.pl

And finally to load it into Irrsi.

/run adv_windowlist.pl

That should be it. This can come in handy when you have any more than a handful of windows and can’t keep your conversations straight. If we take a look at our Irssi session we can see that there is a name associated with each window number now.

As you can see there is now a name associated with each of the windows that we have open. This looks pretty good but there are some cool features in this script that we are going to leverage to make it look even better. In your Irssi session run the following commands to customize your display even further,

/statusbar window remove act
/set awl_display_key $Q%K|$N%n $H$C$S
/set awl_display_key_active $Q%K|$N%n $H%U$C%n$S
/set awl_display_nokey [$N]$H$C$S
/set awl_block 1

OK, this is looking better. We now have our current conversation underlined, our windows named and numbered with decent formatting and have set windows with activity to update and change colors. There are more options if you look at the script itself but this is a pretty good start.

Setting up hilight.pl

This script will add in the ability to check messages that contain your nick. This is a good way to easily check messages while you were away or didn’t get a chance to respond to. First we need to make sure that the new window will split correctly.

SET autostick_split_windows ON
/hilight <nickname>

Now we add in and configure our new notification window.

/window new split
/window name hilight
/window size 4

Set up nm.pl

The description from its creator is “right aligned nicks depending on longest nick”. This script will help with the readability and organization of your different chats. I’m not sure if it requires nickcolor.pl but I have it in my scripts folder and symlinked to my autorun folder just in case. Just load nm.pl in like you do for all other scripts and it will start doing its things.

/run nm.pl

Setting up themes

Definitely not a necessity but can help to make things cleaner and easier to read. So far I have played around with the xchat and fear2 themes but will come back and update this post if I happen to find a better theme. The good thing is that thems are really easy to set up and use. So to load a specific theme just copy it into your /.irssi directory and turn it on in Irssi.

wget http://irssi.org/themefiles/xchat.theme
/set theme xchat

That’s all I have on Irssi for now. If there is one complaint that I have about Irssi it is that the nicklist.pl script doesn’t play nicely in tmux (however it should be fine in screen). It is a manual process and is a pain in the ass to get set up so I have chosen not to cover it in this post. It is possible I know, but for me, it just wasn’t worth the trouble. If you know of an easy way to get this working inside of tmux let me know.

Resources:
http://irssi.org/beginner/
http://quadpoint.org/articles/irssi/#channel_statusbar_using_advanced_windowlist
http://www.antonfagerberg.com/archive/my-perfect-irssi-setup
http://quadpoint.org/articles/irssisplit/