lostash收集tomcat access使用Grok过滤自定义格式日志

作者: root007 分类: EFK,中间件 发布时间: 2019-03-18 11:34

修改tomcataccess日志格式

[15/Mar/2019:23:57:32 +0800] 192.168.200.78 10.10.43.12 6A0ABEC0C2BE46507278C0B65D3FC992 HTTP/1.1 POST /peopleCredit/zxNotify 200 7 264 http-bio-8080-exec-2

sed -i ‘s/^.pattern=.%b\”/pattern=\”%t %h %A %S %H %m %U %s %B %D %I %q\”/g’ /opt/tomcat7/conf/server.xml

pattern参数

%a - 远程IP地址
%A - 本地IP地址
%b - 发送的字节数(Bytes sent), 不包括HTTP headers的字节,如果为0则展示'-'
%B - 发送的字节数(Bytes sent), 不包括HTTP headers的字节
%h - 远程主机名称(如果resolveHosts为false则展示IP)
%H - 请求协议
%l - 远程用户名,始终为'-'(Remote logical username from identd)
%m - 请求的方法(GET, POST等)
%p - 接受请求的本地端口
%q - 查询字符串,如果存在,有一个前置的'?'
%r - 请求的第一行(包括请求方法和请求的URI)
%s - response的HTTP状态码(200,404等)
%S - 用户的session ID
%t - 日期和时间,Common Log Format格式
%u - 被认证的远程用户, 不存在则展示'-'
%U - 请求URL路径
%v - 本地服务名
%D - 处理请求的时间,单位为毫秒
%T - 处理请求的时间,单位为秒
%I - 当前请求的线程名(can compare later with stacktraces)

提供Grok过滤日志字段https://grokdebug.herokuapp.com/

\[%{HTTPDATE:timestamp}\] %{IPORHOST:clientip} %{IPV4:localip} %{NOTSPACE:session} HTTP/%{NUMBER:httpversion} %{WORD:verb} %{NOTSPACE:request} %{NUMBER:response} %{NUMBER:bytes} %{NUMBER:processingtime} %{JAVAFILE:thread}

修改logstash提供grok插件过滤日志

filter {
    if [fields][appname] == "tomcataccess"{
    grok {
        match => {
            "message" => ["message","\[%{HTTPDATE:timestamp}\] %{IPORHOST:clientip} %{IPV4:localip} %{NOTSPACE:session} HTTP/%{NUMBER:httpversion} %{WORD:verb} %{NOTSPACE:request} %{NUMBER:response} %{NUMBER:bytes} %{NUMBER:processingtime} %{JAVAFILE:thread}"]
                 }
         }
    date {
       match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
       }
    mutate{
       convert => {"response" => "integer"}
       convert => {"bytes" => "integer"}
       convert => {"processingtime" => "integer"}
         }
    }

发表评论

邮箱地址不会被公开。 必填项已用*标注


  • http://ediconcept.odns.fr/maillotdefootpascher.html
  • http://ediconcept.odns.fr/maillotsfootpascher.html
  • http://ediconcept.odns.fr/maillotsfootpascher2022.html
  • https://sergiofajardo.co/maillotdefootpascher.html
  • https://sergiofajardo.co/maillotsfootballpascher.html
  • https://sergiofajardo.co/maillotsfootpascherfrance.html
  • https://allphonetoys.com/maillotdefootpascher.html
  • https://allphonetoys.com/nouveaumaillotdefootpascher.html
  • https://allphonetoys.com/nouveaumaillotsdefootball.html
  • http://samosystem.nichost.ru/maillotdefootpascher.html
  • http://samosystem.nichost.ru/nouveaumodèlemaillotdefoot.html
  • http://samosystem.nichost.ru/pascherfemmesfootballjersey.html
  • http://w96397on.beget.tech/maillotdefootpascher.html
  • http://w96397on.beget.tech/pascherfootballt-shirts.html
  • http://w96397on.beget.tech/paschermaillotsdefootpromo.html
  • http://wesleywang.acsite.org/maillotdefootpascher.html
  • https://wesleywang.acsite.org/repliquemaillotsdefootballlivraisongratuite.html
  • https://wesleywang.acsite.org/retrovintagesoccerjerseys.html
  • https://www.rnkhabri.com/maillotdefootpascher.html
  • https://www.rnkhabri.com/personnaliséenfantssoccerjersey.html
  • https://www.rnkhabri.com/personnaliséfemmessoccervêtements.html
  • https://www.oujdacity.net/maillotdefootpascher.html
  • https://www.oujdacity.net/sitedemaillotdefootpascher.html
  • https://www.oujdacity.net/survetementdefoot.html
  • https://infendo.com/maillotdefootpascher.html
  • https://infendo.com/survetementdefootpascher.html
  • https://infendo.com/uniformedefootballdesjeunes.html
  • https://planitagency.com/maillotdefootpascher.html
  • https://planitagency.com/ventemaillotdefootpascherthailande2022.html
  • https://planitagency.com/vesteequipedefoot.html
  • http://immergut.com.br
  • https://immobiliarelerosemarine.com
  • https://immobilienfotografie24.de
  • http://immobilier-lemaroc.com
  • https://imobilideranca.com.br
  • https://impactdigital.space
  • https://imperibiza.com
  • https://impexworldltd.com
  • https://implant.gen.tr
  • https://impread.com
  • http://improveyourdance.com
  • http://improvisamos.com
  • http://impulsarh.com.ar
  • http://inamakethat.com
  • http://inanna.be
  • https://incomementorbox.com
  • https://incomerepairpro.com
  • https://indiatvnewss.com
  • https://indoranews.com
  • https://infendo.com
  • https://infinitebankingagenttraining.com
  • https://infinnium.com
  • http://influenceyourway.com
  • http://infocarbrokers.net
  • http://infohealth-usa.com
  • https://informatic-crew.fr
  • https://informatiotech.com
  • https://informeblumenau.com
  • https://infos-grossesse.fr
  • https://ingatlanmagus.com