Centralising logs for fun and profit

It’s one of those things that usually gets pushed to the back burner because it seems like too much work for too little gain: setting up a central syslog server which all your other systems can report back to.

This is a shame, because there’s lots of benefits to having such a server:

  • You can analyse what’s going on in your network from a single, central location – saving you from having to log into a variety of devices for troubleshooting.
  • Improved security – if you have a security breach, the offender has to break into the logging server as well if they’re to cover their tracks properly. (I wouldn’t recommend re-purposing an existing server for precisely this reason – you want your syslog server to be as secure as possible, which means it needs to be running as few services as possible).
  • You only need to remember one set of tools to manage logs from a range of devices. Most routers will happily send logs back to a remote syslog server; there are also third-party products you can install on Windows.

It’s trivially easy to set this up in any reasonably modern Linux distribution. Once again, I’m going to use Debian for this example.

Out of the box, Debian uses rsyslog and stores the configuration file in /etc/rsyslog.conf. Fortunately, the default configuration only needs minor changes to two lines as shown in this excerpt:

# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

Uncomment the lines beginning $ModLoad and $UDPServerRun by removing the # symbols:

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

Restart rsyslogd (service rsyslog restart) and…. that’s it. Done.

Well, that’s not quite it. A remote syslog server isn’t much good unless you have equipment sending logs to it.  On any other Debian servers you may have, this is just a matter of adding a line to /etc/rsyslog.conf:

*.* @192.168.42.39:514

(substitute your own logging server’s IP address or hostname for 192.168.42.39).

Restart rsyslog on the server that will be sending logs to your remote syslog server. Now when you check your remote syslog server, you should find logs appearing from both itself and anything else that’s configured to send logs to it.

Advanced Tweaks

Once you’ve got this done, there’s all sorts of things you can add. You can separate logfiles according to the host that generated them, you can have new logfiles created every day with an appropriate filename… or you can just stick with the basic configuration which will put everything in the same set of log files and just use grep to separate the interesting information.

Whatever you do, keep a sharp eye on disk space on your logfile server. Logs can grow very large very quickly, and a syslog server with a full disk won’t log anything at all.

James Cort

James Cort is Managing Director of Bediwin Information Services, providing IT management and integration services in the South West of England.