Top free and open source log management software

As mentioned in the previous post, in my quest to find an alternative to Kiwi Syslog, I looked at a few Software as a Service (SaaS) offerings first, and then started exploring open source log managment projects. I compiled the list below of all useful open source log management software I have found:

• Graylog2 — free open source self-hosted log management and exception tracking. Graylog2 enables you to unleash the power that lays inside your logs. Use it to run analytics, alerting, monitoring and powerful searches over your whole log base. It is licensed under the GNU General Public License v3 (GPLv3) and all source code can be browsed on GitHub. The web interface is using Ruby On Rails, the server is written in Java.
• log2timeline — a framework for automatic creation of a super timeline. The main purpose is to provide a single tool to parse various log files and artifacts found on suspect systems (and supporting systems, such as network equipment) and produce a timeline that can be analysed by forensic investigators/analysts.
• LogHound — is a tool that was designed for finding frequent patterns from event log data sets with the help of a breadth-first frequent itemset mining algorithm. LogHound can be employed for mining frequent line patterns from raw event logs, but also for mining frequent event type patterns from preprocessed event logs.
• LogReport — the LogReport project serves a dual purpose: developing and maintaining Lire, our Open Source reporting and analysis software, and serving as a nexus of documentation, ideas, and thought on the topic of log files and their potential applications.
• Logstash — is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs. It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
• Logsurfer — is a program for monitoring system logs in real-time, and reporting on the occurrence of events. It is similar to the well-known swatch program on which it is based, but offers a number of advanced features which swatch does not support. Logsurfer is capable of grouping related log entries together – for instance, when a system boots it usually creates a high number of log messages. In this case, logsurfer can be setup to group boot-time messages together and forward them in a single Email message to the system administrator under the subject line “Host xxx has just booted”. Swatch just couldn’t do this properly.
• Logwatch — is a customizable log analysis system. Logwatch parses through your system’s logs and creates a report analyzing areas that you specify. Logwatch is easy to use and will work right out of the package on most systems.
• OSSEC — is a full platform to monitor and control your systems. It mixes together all the aspects of HIDS (host-based intrusion detection), log monitoring and SIM/SIEM together in a simple, powerful and open source solution. OSSEC is an Open Source Host-based Intrusion Detection System that performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response. It runs on most operating systems, including Linux, MacOS, Solaris, HP-UX, AIX and Windows.
• OSSIM — provides all of the features that a security professional needs from a SIEM offering – event collection, normalization, and correlation. Established and launched by security engineers out of necessity, OSSIM was created with an understanding of the reality many security professionals face: a SIEM is useless without the basic security controls necessary for security visibility. OSSIM addresses this reality by providing the essential security capabilities built into a unified platform. Standing on the shoulders of the many proven open source security controls built into the platform, OSSIM continues to be the fastest way to make the first steps towards unified security visibility.
• Php-Syslog-ng — is a frontend for viewing syslog-ng messages logged to MySQL in realtime. It features customized searches based on device, priority, date, time, and message.
• RSyslog — is an enhanced syslogd supporting, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part. It is quite compatible to stock sysklogd and can be used as a drop-in replacement. Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user. The project was initiated in 2003 and seriouosly begun in 2004 by Rainer Gerhards and is currently being maintained by him.
• Sawmill — is a Open Source Unix Syslog log analyzer (it also supports the 956 other log formats listed to the left). It can process log files in Open Source Unix Syslog format, and generate dynamic statistics from them, analyzing and reporting events. Sawmill can parse Open Source Unix Syslog logs, import them into a MySQL, Microsoft SQL Server, or Oracle database (or its own built-in database), aggregate them, and generate dynamically filtered reports, all through a web interface. Sawmill can perform Open Source Unix Syslog log analysis on any platform, including Window, Linux, FreeBSD, OpenBSD, Mac OS, Solaris, other UNIX, and others.
• SEC — simple event correlator is a tool for accomplishing event correlation tasks in the domains of log analysis, system monitoring, network and security management, etc. SEC reads lines from files, named pipes, or standard input, matches the lines with patterns (like regular expressions or Perl subroutines) for recognizing input events, and correlates events according to the rules in its configuration file(s). SEC can produce output by executing external programs (e.g., snmptrap or mail), by writing to files, by calling precompiled Perl subroutines, etc.
• SLCT – simple logfile clustering tool is a tool that was designed to find clusters in logfile(s), so that each cluster corresponds to a certain line pattern that occurs frequently enough. With the help of SLCT, one can quickly build a model of logfile(s), and also identify rare lines that do not fit the model (and are possibly anomalous).

 

• Snare BackLog — is a program that provides a central collection facility for a variety of log sources, including Snare Agents for Windows, Solaris, AIX, Irix, ISA Server, IIS Server, Lotus Notes (and others), plus any device capable of sending data to a syslog server. The SNARE BackLog is free software (freeware), released under the terms of the GNU Public Licence (GPL).
• syslog-ng Open Source Edition — The syslog-ng application is a high-performance syslog server with advanced log processing services and direct database access. The syslog-ng project is a continuous community effort to create the best log management tool. The project is an advocate and early adopter of open standards, including the syslog RFCs developed by the IETF or the Common Event Expression (CEE) message-description standard of the MITRE Corporation

From： http://baudlabs.com/top-free-and-open-source-log-management-software/