I'm playing around with splunk today. Installed the "universal forwarder" on my server, which was easy... And then wasted an hour trying to figure out where the data was. Apparently I just needed to search for "index=main". Now that I can see the data though, it looks really cool. I set up event log forwarding and I can really easily search for things like login failures.

The main reason for all this is to understand other ways people do debugging and diagnostics.