Filters
Incoming data streams can be filtered before they are written to the final store in the S3 / S3 compatible bucket. Filter expressions allow fine grained matching on key attributes from the ingested data.
Filters are defined using the "filters" keyword in the LOGIQ configuration file. A filter definition consists of a name and a condition. E.g.
1
filters:
2
-
3
name: f_debug
4
condition: severity == 'debug'
5
-
6
name: f_macbook_ubuntu_vm
7
condition: app_name == 'macbook_ubuntu_vm'
Copied!

name [required]

This name key is used to specify a unique name for the filter
1
filters:
2
-
3
name: f_ubuntu_vm
4
condition: app_name == 'ubuntu_vm'
Copied!
Filter for a rule is specified by including the filter name in the rule definition.
1
rules:
2
-
3
source: s_ubuntuvm
4
filter: f_ubuntu_vm
5
destination: t_default_destination
Copied!
Once an incoming stream matches the source definition in the rule, the filter, if one is specified, is applied to determine a rule match.

condition [required]

The condition key is used to specify a boolean expression that is used for computing a rule match. The boolean condition is specified on key fields that are extracted from the ingested data.
1
filters:
2
-
3
name: f_ubuntu_vm
4
condition: app_name == 'ubuntu_vm'
Copied!

Syslog data attributes for condition expressions

The attribute values are of type string unless otherwise specified
  • severity, Severity
  • facility, Facility
  • priority, Priority
  • app_name, appname, Appname, AppName
  • timestamp, Timestamp
  • message, Message
  • host_name, hostname, Hostname, HostName
  • boolean: tag.Exists, Tag.Exists
  • string: tag.Value, Tag.Value (RFC3164 only)
  • boolean: protocol.RFC5424, Protocol.RFC5424, Protocol.RFC3164, protocol.RFC3164
  • MsgId, Msgid, msg_id (RFC5424 only )

Condition expression syntax

  • Modifiers: + - / * & | ^ ** % >> <<
  • Comparators: > >= < <= == != =~ !~
  • Logical ops: || &&
  • Numeric constants, as 64-bit floating point (12345.678)
  • String constants (single quotes: 'foobar')
  • Date constants (single quotes, using any permutation of RFC3339, ISO8601, ruby date, or unix date; date parsing is automatically tried with any string constant)
  • Boolean constants: true false
  • Parenthesis to control order of evaluation ( )
  • Arrays (anything separated by , within parenthesis: (1, 2, 'foo'))
  • Prefixes: ! - ~
  • Regex comparators =~ !~
Last modified 2yr ago