Simple config testing

Create a simple configuration in a tmp directory. Only include the necessary bits.

input {

    file {
        path => "/home/isdadmin/logstash/junos.log"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        tags => "parse"
    }

}


filter {

    if [tags] == "parse" {
    grok {
        match => {
            "message" => "%{RT_FLOW2}"
        }
        remove_tag => "parse"

    }
    }

    if [tags] == "parse" {
    grok {
        match => {
            "message" => "%{RT_FLOW1}"
        }
        remove_tag => "parse"
    }
    }

    if [tags] == "parse" {
        grok {
            match => {
                "message" => "%{RT_FLOW3}"
            }
            remove_tag => "parse"
        }
    }

}


output {
    stdout { codec => rubydebug }
}

Next create a file (junos.log) with a small selection of log samples.

Sep 26 06:49:39 192.168.1.29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.168.1.4/123->10.1.1.1/123 junos-ntp 10.2.2.2/26300->10.1.1.1/123 nsw-src-interface None 17 Hyper_V_dns ISD_Srv Internet 432293 N/A(N/A) vlan.2 UNKNOWN UNKNOWN UNKNOWN
Sep 26 06:49:40 192.168.1.29 RT_FLOW: RT_FLOW_SESSION_CLOSE: session closed TCP RST: 192.168.1.42/61816->192.168.1.20/88 None 192.168.1.42/61816->192.168.1.20/88 None None 6 All_ISD_Wks_ISD_Srv ISD_Wks ISD_Srv 499914 6(1659) 6(1653) 2 UNKNOWN UNKNOWN N/A(N/A) vlan.1 UNKNOWN

Run logstash.

Warning

Not sure this works exactly. It worked at one point, but seemed to cause issues with the latest logstash?

/opt/logstash/bin/logstash -f ./logstash.conf --verbose

This is working with 6.5

bin/logstash --path.config ./config/conf.d --path.settings ./config -t

Observe output.

sincedb path

Set the sincedb_path to /dev/null to keep it from recording where it is in the file. If this isn’t set to something like /dev/null, delete the data dir after running to reset the state.

plugin statistics

Look at the [Metrics Logstash] Logstash Single Pipeline View dashboard. Not sure what provides this, maybe metricbeat or agent?