With the rise of SIEM and IT operation data mining use-cases many organizations are investing or deciding on how to invest in centralized log management. I'm sure people wonder is there interesting open source alternatives or shall I even bother?
Well it depends, I think there are some great new options to test or keep an eye on, some fantastic mature commercial options and something in-between (Splunk).
So here are the latest a greatest open source options to commercial products such as:
Splunk, Qradar Log Manager , ArcSight Logger, Logrythem, etc.
enterprise-log-search-archive (ELSA)
Perl, MySQL and SOLR based solution which is said to be faster than Splunk at large data sets and seems to have a large following. There is currently no commercial backing or support and it looks that the web interface is highly usable but not rich with visualization options (a trait you will find common in the open source offerings).
http://code.google.com/p/enterprise-log-search-and-archive/
http://vimeo.com/39722091
Sentry
Python and Django based solution by the guys from DISQUS. Seems to be an interesting HTTP based approach and looks high performance. Lots of documentation available, great option for the Python hackers (like me).
http://sentry.readthedocs.org/en/latest/
https://www.wunki.org/posts/2012-01-19-centralized-logging-with-sentry.html
Graylog2
Looks to have the best interface of the group, this is a ROR and ruby based project by the smart folks at XING. Looks to be well maintained and feature rich. Would love to know how it scales.
http://graylog2.org/
LogStash
Java + ROR based streaming log aggregation. Looks very cool, it was created by an ex-google engineer now working at Loggly. Very actively maintained project with lots of documentation and some cool features. This one is definitely worth checking out.
http://logstash.net/
http://www.oscon.com/oscon2012/public/schedule/detail/26347
Logsandra
Another Python based project using Cassandra (NOSQL) backend database. Cool looking project but early stage and does not seem to have a mature UI.
https://github.com/thobbs/logsandra
http://my.safaribooksonline.com/book/databases/9781849515122/libraries-and-applications/ch10lvl1sec12
Scribe
Python based log aggregation tool used by Facebook. Does not seem to be actively maintained or updated in several years :(
https://github.com/facebook/scribe
Unfortunately I cannot speak to the performance, feature or overall quality these solutions. Perhaps when I have more time I can try them out and create some reviews. Enjoy!
