Log management in an IT infrastructure is always going to be a challenge…

I’ve attempted to work with several “Enterprise” tools including GFI’s Languard, LogLogic, EventTracker, and a few more that aren’t even worth mentioning.

No matter the tool, the story is always the same: It’s a pain to work with logs, and a “pretty” interface and some neat looking graphs don’t offer that much benefit. Adding onto this general problem is the time and effort it takes to manage and maintain a log management solution. Some may even make the argument that it could be faster to manually review logs in some cases. Making a long story short – Log management sucks.

Which brings me to a solution that completely negates the first part of this post. It makes log management quick, easy, and even a little bit fun. Better yet, it’s completely free!

If you want to have a solid, scalable, and reliable log management solution in LITERALLY 10 minutes, then follow these steps.

1: Download Splunk.
Splunk is an awesome product. Plain and simple. It will index log data from almost anything you can throw at it. It can take syslogs, application logs, Windows event logs, and even straight up files/directories. Whether you have a Windows server, a Unix/Linux server, or even a Windows XP workstation laying around (with decent specs) then you can install Splunk in just a couple minutes. It’s painfully simple to install and get set up with an initial and default configuration. The only thing to configure before moving on to step 2 below is to make sure you have port 514 defined as a datasource on both TCP and UDP. This will allow your Splunk server to index log data we’re going to throw at it. Note that Splunk does have both a free version and a enterprise version which costs money. The free version is adequate for most small/medium deployments as it can index up to 500 megs daily. The enterprise version is definitely not a waste of money though if you’re looking at a large deployment.

2: Download Lasso Server.
LogLogic has released a very powerful tool for log collection. This tool is designed to run on a Windows Server or XP/Vista workstation, and it makes it VERY simple to gather log data from multiple Windows Server systems. The install takes only seconds: just add your log destination (read: Splunk server you just set up), and then add your Windows server hosts that you want to collect logs from. The Lasso service takes care of the rest. It will poll all your Windows servers, grab the event logs from them via WMI calls, convert those logs into syslog format, and then send them off to your Splunk server on TCP port 514. Splunk takes care of the rest by indexing all that log data and making it usable, searchable, and actionable. Note that If you’re running mostly Unix/Linux servers, then you don’t need a tool like Lasso to collect your logs. Just set them up to forward their own syslog data to your Splunk server directly.

3: Tuning.
By default, Splunk provides an easy to use search interface for you to easily find and locate log data. Depending on your needs, you may want to spend some time tuning Splunk to do scheduled searches, send emails if it finds certain results, outline custom search queries for specific messages, etc. Splunk is a very flexible tool that can be scaled to meet the needs of even large enterprise organizations. It can do all kinds of cool things if you want, but even the simple fact of having a log repository that you can search if/when you need to is priceless.

This combination of tools gives system administrators a quick and simple way to tackle the task of log management. And really, there’s no downside. It’s free (assuming you have access to some decent hardware to run it on), it’s easy, but most of all it’s very powerful. My thanks to Splunk and LogLogic for providing these tools to the community. It’s good to know that there’s companies out there making solid products and standing behind them.