Sometimes it is desirable to report on the fact that several events happend at a similar time, possibly in a particular order. As of version 2.6.1, samhain supports this in the following way:

16.1.2. Correlating the marked events

To correlate events labelled label_one, label_two, etc., just build a regular expression that matches the labels, in the temporal order you want to check for. E.g. if the temporal order is irrelevant, you may want to match (label_one.*label_two)|(label_two.*label_one). Use this expression in a rule maked as CORRELATE( description), like this:

LogmonRule=CORRELATE( description): queue_label: (perl)regex

	Old records in existing logfiles
	Because the 'keep' timeout is relative to the current time, correlation of old entries in logfiles (i.e. when, at startup, an existing logfile with old entries is scanned) will only work if you specify 'keep' timeouts that are long enough to cover the whole timespan from the first logfile record until now.

To check whether a given event occurs at least once within some given interval, the rule for matching an event can be modified like this:

LogmonRule=MARK( seconds,description): queue_label: (perl)regex matches a logfile entry against the provided regular expression, AND checks whether is occurs at least once within the specified interval (seconds).

Processing of this rule will be no different than other rules otherwise, so if you e.g. only want a report for this event if it is missing, just assign it to the trash queue. However, in the latter case the severity for reporting the messages must be set separately with the LogmonMarkSeverity directive, because the 'trash' queue has no severity assigned:

LogmonMarkSeverity= severity — Severity for reports on missing heartbeat messages if the messages themselves are assigned to the 'trash' queue (default: crit).

Samhain can automatically detect and report bursts of similar, repeated events in the monitored logfiles. Here similar, repeated events refers to events that differ (only) in details that can be expected to differ for events of the same kind: IP adresses, FQDNs, email adresses, and numbers. The event history goes back 12 minutes, and thus a report is triggered if the number of similar events within the last 12 minutes exceeds a given threshold (default: 24).

This feature is off by default. In order to switch it on, you need to set a reporting queue:

LogmonBurstQueue= queue — Set the reporting queue for reporting bursts of similar log messages (default: don't report).

In addition, there are two more configurable parameters, one to set the triggering threshold (i.e. the number of messages within 12 minutes that need to be exceeded to raise an alert), and another one to indicate whether messages from the cron daemon should be considered as well (default: no):

LogmonBurstThreshold= number — The number of repeated messages within 12 minutes that must be exceeded to report a burst of repeated messages (default: 24).

LogmonBurstCron= boolean — Whether to report also on bursts of repeated cron messages (default: false).

LogmonActive= boolean switches this module on or off (default: off).

LogmonSaveDir= /absolute/path sets the directory where checkpoint data for logfiles is stored (default: same as for database file).

LogmonClean= boolean delete old checkpoint data unmodified for 30 days or more (default: off).

LogmonInterval= seconds sets the interval for logfile checking (default: 10 seconds).

LogmonMarkSeverity= severity — Severity for reports on missing heartbeat messages if the messages themselves are assigned to the 'trash' queue (default: crit).

LogmonBurstThreshold= number — The number of repeated messages within 12 minutes that must be exceeded to report a burst of repeated messages (default: 24).

LogmonBurstQueue= queue — Set the reporting queue for reporting bursts of similar log messages (default: don't report).

LogmonBurstCron= boolean — Whether to report also on bursts of repeated cron messages (defaul: false).

LogmonDeadtime= seconds — Do not report a correlated event again within the given time (default: 60 seconds).

LogmonWatch= TYPE:path[:format] advises the module to monitor the logfile with the specified path, which is of type 'TYPE' (logfile types are uppercase). Some logfile types (e.g. Apache access logs) can be customized, and hence some format information must be provided.

	Do not quote the format
	Please note that it's neither required nor supported to add quotes around the format string. Likewise, quotes within the format should not be escaped. Wrong: LogmonWatch= APACHE:/var/log/apache/access.log:"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"" Correct: LogmonWatch= APACHE:/var/log/apache/access.log:%h %l %u %t "%r" %>s %b "%{Referer}i"

Currently ( samhain 2.6.4) the following logfile types are supported

LogmonHidePID= boolean is an option that only affects logfiles of type SYSLOG. It causes the PID to be stripped from the log line (before matching against the RE).

LogmonQueue= label:[interval]:(sum|report):severity[:alias] defines an output queue. Here, label is an arbitrary name which is used to assign rules to this queue; interval is the timespan over which messages are summarized if the queue is of type 'sum'; sum(summarize over some interval) or report(report each event separately and immediately) are the two queue type supported, and severity is the severity assigned to an event. Furthermore, optionally it is possible to specify an alias(must be defined in the email configuration) to direct email for this rule to a specific list of recipients.

	Email
	If you spefify a list alias, email will still go to all defined email recipients unless filtered, e.g. with SetMailFilterNot = \[Logfile\] I.e. you may want to define recipients, filter them as above, and then define list aliases to be used in an event queue. See Section 4 for more information.

LogmonHost= (perl)regex causes the following rules to be applied only to entries for this host(s). It is ended implicitely by another LogmonHost directive, or explicitely by a LogmonEndHost directive.

LogmonEndHost explicitely ends a preceding LogmonHost directive.

LogmonGroup= group_label:(perl)regex causes the following rules to be applied only if the group regex matches (i.e. rules within the group are skipped if the group regex doesn't match. This can be used to improve speed/efficiency of matching, i.e. you can group regexes by a common prefix. A group is ended implicitely by another LogmonGroup directive, or explicitely by a LogmonEndGroup directive.

LogmonEndGroup explicitely ends a preceding LogmonGroup directive.

LogmonRule= queue_label:(perl)regex matches a logfile entry against the provided regular expression. If the expression matches, then captured subexpressions are replaced by '___', and the logfile entry is reported as specified for the queue referenced by queue_label. Non-captured subexpressions (i.e. subexpressions where the opening bracket is followed by '?:') are not replaced by '___', but reported literally.

LogmonRule=KEEP( seconds,label): queue_label:(perl)regex as above, but additionally keep the event label for seconds to perform event correlation.

LogmonRule=CORRELATE( description): queue_label: (perl)regex perform event correlation by matching the labels(as specified in KEEP rules) of a sequence of events against the given regular expression.

LogmonRule=MARK( seconds,description): queue_label: (perl)regex matches a logfile entry against the provided regular expression, AND checks whether is occurs at least once within the specified interval (seconds).

	  [Logmon]
	  
	  #
	  # Switch on the module
	  #
	  LogmonActive = yes
	  
	  # Check every second
	  #
	  LogmonInterval = 1
	  
	  # Strip PIDs from syslog messages
	  #
	  Logmonhidepid = true
	  
	  # Define a queue with severity 'crit'.
	  # This is a 'report' queue, hence 'interval' (10)
	  # will be ignored.
	  #
	  LogmonQueue = q1:10:report:crit
	  
	  # Define a second queue with severity 'alert'
	  # 
	  LogmonQueue = q2:10:report:alert
	  
	  # Monitor /var/log/messages, which is a syslog file
	  #
	  LogmonWatch = SYSLOG:/var/log/messages
	  
	  # Monitor /var/log/samba/log.nmbd, which is a samba
	  # logfile
	  #
	  LogmonWatch = SAMBA:/var/log/samba/log.nmbd
	  
	  # Monitor /var/log/apache2/access.log, which is
	  # an Apache logfile in 'combined' format
	  #
	  LogmonWatch = APACHE:/var/log/apache2/access.log:combined
	  
	  # Monitor disks to check for full /dev/sda1
	  #
	  LogmonWatch = SHELL:df -h
	  
	  # Syslog messages for the pppd deamon
	  #
	  LogmonGroup = g1:pppd.*
	    #
	    # Rules in this group
	    #
	    LogmonRule     = q1:pppd:\s+primary.*
	    LogmonRule     = q1:pppd:\s+secondary.*
	    #
	  LogmonEndGroup

	  # Warn about disk /dev/sda1 nearly full (80% or more. Use a 
	  # non-capturing subexpression [the (?:8|9)] for the percentage full.
	  #
	  LogmonRule = q1:/dev/sda1\s+[0-9GM.]+\s+[0-9GM.]+\s+[0-9GM.]+\s+(?:8|9).%.*

	  # Messages starting with WARNING (some samba stuff)
	  #
	  LogmonGroup = g2:WARNING.*
	  LogmonRule     = q2:.*interfaces.*
	  LogmonEndGroup

	  # Report on these events if happening within 120 seconds.
	  # Set LogmonDeadtime to 120 seconds to avoid multiple reports.
	  # Use the 'trash' queue for the keep rules to avoid reports on
	  #   the individual events.
	  #
	  LogmonRule = KEEP(120,event1):trash:sshd: Accepted publickey for root.*
	  LogmonRule = KEEP(120,event2):trash:sshd: pam_unix\(sshd:session\).*
	  LogmonRule = CORRELATE(root_login):q1:(event1.*event2)|(event2.*event1)

	  LogmonDeadtime = 120

	  # Throw away all non-matching entries. This amounts
	  # to a blacklist policy (only report known bad).
	  #
	  # Usually considered bad practice!!! Use whitelisting!
	  #
	  # 'trash' is a built in queue, no definition needed.
	  #
	  LogmonRule = trash:.*