IIS Log Monitoring Update - Logstash 5.4

Would appear that since my first post, some additional GROK parameters are needed to properly index the IIS fields, well, probably any field parsed through GROK.

I recently setup a new instance of ELK, on the 5.4 stack. I setup all my filters as I normally had, in fact, I had SCP'd the files over to my workstation and simply copied them back down.

I noticed, when I went to graph status codes and bytes in Kibana, the fields were not available for aggregation. Checking my index in Kibana, sure enough, all the fields were being indexed as strings. After some back and forth with the support guys, it was discovered that additional information is needed to properly index numerical values.

Here is my original post for full, IIS logging:

filter {
  if [type] == "iis" {
if [message] =~ "^#" {
  drop {}
  }
grok {
  match => { "message" => "%{DATESTAMP:Event_Time} %{WORD:site_name} %{HOSTNAME:host_name} %{IP:host_ip} %{URIPROTO:method} %{URIPATH:uri_target} (?:%{NOTSPACE:uri_query}|-) %{NUMBER:port} (?:%{WORD:username}|-) %{IP:client_ip} %{NOTSPACE:http_version} %{NOTSPACE:user_agent} (?:%{NOTSPACE:cookie}|-) (?:%{NOTSPACE:referer}|-) (?:%{HOSTNAME:host}|-) %{NUMBER:status} %{NUMBER:substatus} %{NUMBER:win32_status} %{NUMBER:bytes_received} %{NUMBER:bytes_sent} %{NUMBER:time_taken}"}
}
}
}

The new format looks something like this:

filter {
  if [type] == "iis" {
if [message] =~ "^#" {
  drop {}
  }
grok {
  match => { "message" => "%{DATESTAMP:Event_Time} %{WORD:site_name} %{HOSTNAME:host_name} %{IP:host_ip} %{URIPROTO:method} %{URIPATH:uri_target} (?:%{NOTSPACE:uri_query}|-) %{NUMBER:port:int} (?:%{WORD:username}|-) %{IP:client_ip} %{NOTSPACE:http_version} %{NOTSPACE:user_agent} (?:%{NOTSPACE:cookie}|-) (?:%{NOTSPACE:referer}|-) (?:%{HOSTNAME:host}|-) %{NUMBER:status} %{NUMBER:substatus:float} %{NUMBER:win32_status:float} %{NUMBER:bytes_received:float} %{NUMBER:bytes_sent:float} %{NUMBER:time_taken:float}"}
}
}
}

Error: Which: no javac in (/sbin:/bin:/usr/sbin:/usr/bin) when installing a Plugin for Logstash

This took me down a rabbit hole, but ended up being an easy solution.

The solution was, in addition to the JRE environment, I had to install the JDK as well.

If you do both and still have the issue, ensure you have the path variable in your profile.

Mine is as such:

JAVA_HOME=/usr/java/jre1.8.0_131
PATH=$JAVA_HOME/bin:$PATH
export PATH JAVA_HOME

Check your profile. I did so by sudo nano /etc/profile

added those lines just about the "pathmunge ()" line

When I only had the JRE installed, if I ran "which javac" I'd receive a path error. After installing the JDK, that command worked successfully and my Logstash Plugins installed without error as well.