区配错误原因
匹配规则:
%{IPORHOST:client_ip} (%{WORD:ident}|-) (%{WORD:auth}|-) \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response_status} (?:%{NUMBER:response_bytes}|-) (?:"(?:%{URI:referrer}|-)"|%{QS:referrer}) %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time} %{NUMBER:upstream_response_time}
当日志中 upstream_response_time 或 request_time 字段出现 – 字符,而不是数字时,以上匹配规则会出现匹配错误, 改进如下:
%{IPORHOST:client_ip} (%{WORD:ident}|-) (%{WORD:auth}|-) \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response_status} (?:%{NUMBER:response_bytes}|-) (?:"(?:%{URI:referrer}|-)"|%{QS:referrer}) %{QS:agent} %{QS:xforwardedfor} (%{NUMBER:request_time}|-) (%{NUMBER:upstream_response_time}|-)
以上两条规则不同之处:
%{NUMBER:request_time} %{NUMBER:upstream_response_time}
更换为:
(%{NUMBER:request_time}|-) (%{NUMBER:upstream_response_time}|-)
但是,还是这样匹配到的字段会有 – 字符,而es中使用的字段类型是数字类型,所以需要以下解决办法:
解决方法
filter {
if [upstream_response_time] == "-" {
mutate {
replace => { "upstream_response_time" => 0 }
}
}
if [request_time] == "-" {
mutate {
replace => { "request_time" => 0 }
}
}
}