Use fluentd to parse Juniper Firewall log

First of all I’ll show a sample syslog message from Juniper SSG-550:

SSG550: NetScreen device_id=JN1115722ADB [Root]system-notification-00257(traffic): start_time="2015-05-30 18:39:44"
duration=0 policy_id=320000 service=udp/port:26 proto=17 src_zone=Null dst_zone=Null action=Deny sent=0 rcvd=1500
src=x.y.c.a dst=x.t.q.z src_port=0 dst_port=26 session_id=0 reason=Traffic Denied

The source part of fluent.conf:

<source>
type syslog
port 7000
tag juniper
format none
</source>

format none disables the parsing by in_syslog plugin.

Then we need to replace src zone with src_zone, so that later the field parser use src_zone as field name rather than just zone, this is done with the rewrite plugin (fluent-plugin-rewrite):

<match juniper.**>
type rewrite
add_prefix key_replace_space

<rule>
key message
pattern "(src|dst) zone"
replace \1_zone
</rule>
<rule>
key message
pattern "(src|dst)-xlated ip"
replace \1-xlated_ip
</rule>
</match>

Then the field parser (fluent-plugin-fields-parser):

<match key_replace_space.juniper.**>
type fields_parser
remove_tag_prefix key_replace_space
add_tag_prefix field_parsed
</match>

At last, output to Elasticsearch:

<match field_parsed.juniper.**>
type elasticsearch
logstash_format true
logstash_prefix juniper
time_key parsed_time
</match>