filebeat+redis+ELK收集Springboot的Logback日志

一、背景

  • 當(dāng)項(xiàng)目中用到集群環(huán)境時(shí)，一個(gè)springboot的應(yīng)用，會(huì)發(fā)布到多個(gè)tomcat中，在排除故障的時(shí)候，必須要每個(gè)tomcat都登錄上去查看，非常麻煩。所以就有了用filebeat+redis+ELK將分布式日志集中起來(lái)，分析的想法。

二、環(huán)境介紹

  1. 系統(tǒng)： Centos7
  2. 框架：springboot+logback
  3. 在springboot項(xiàng)目中做到了分不同環(huán)境執(zhí)行不同log配置。dev開(kāi)發(fā)環(huán)境只是簡(jiǎn)單的輸出日志在控制臺(tái)，test以及production環(huán)境輸出到文件中，每個(gè)項(xiàng)目的日志都放在自己相應(yīng)的文件夾下，并且做到按天分隔日志，當(dāng)前正在寫日志的log文件命令為info.log，使用filebeat去抓取此文件日志。

  4. 結(jié)構(gòu)


    image.png
  5. log-test.xml配置
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <property resource="application.properties" />

    <!-- 項(xiàng)目名， 在application.properties中定義的 -->
    <contextName>${projectName}</contextName>
    <!-- 日志級(jí)別 -->
    <property name="logLevel" value="INFO"></property>
    <!-- 最大保存時(shí)間 -->
    <property name="maxHistory" value="180"/>
    <!-- 異步緩沖隊(duì)列的深度,該值會(huì)影響性能.默認(rèn)值為256 -->
    <property name="queueSize" value="512"></property>

    <!-- 日志路徑，${catalina.home}是該項(xiàng)目當(dāng)前tomcat的路徑。 -->
    <property name="logPath" value="${catalina.home}/logs/${projectName}"/>

    <property name = "CONSOLE_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %-40.40logger{35} - %msg%n"></property>

    <!-- 控制臺(tái)輸出 -->
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <!--格式化輸出：%d表示日期，%thread表示線程名，%-5level：級(jí)別從左顯示5個(gè)字符寬度%msg：日志消息，%n是換行符-->
            <pattern>${CONSOLE_PATTERN}</pattern>
        </encoder>
    </appender>


    <!-- 日志記錄器，日期滾動(dòng)記錄 -->
    <appender name="FILE_INFO" class="ch.qos.logback.core.rolling.RollingFileAppender">

        <!-- 正在記錄的日志文件的路徑及文件名 -->
        <file>${logPath}/info.log</file>

        <!-- 日志記錄器的滾動(dòng)策略，按日期記錄 -->
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- 歸檔的日志文件的路徑-->
            <fileNamePattern>${logPath}/info.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>${maxHistory}</maxHistory>
        </rollingPolicy>

        <!-- 追加方式記錄日志 -->
        <append>true</append>

        <!-- 日志文件的格式 -->
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>${CONSOLE_PATTERN}</pattern>
        </encoder>

        <!-- 此日志文件只記錄info、warn、error 級(jí)別的 -->
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>info</level>
        </filter>

    </appender>

    <appender name="ASYNC_LOG_INFO" class="ch.qos.logback.classic.AsyncAppender">
        <!-- 不丟失日志.默認(rèn)的,如果隊(duì)列的80%已滿,則會(huì)丟棄TRACT、DEBUG、INFO級(jí)別的日志 -->
        <discardingThreshold>0</discardingThreshold>
        <!-- 更改默認(rèn)的隊(duì)列的深度,該值會(huì)影響性能.默認(rèn)值為256 -->
        <queueSize>${queueSize}</queueSize>
        <appender-ref ref="FILE_INFO"/>
    </appender>

    <root level="${logLevel}">
        <!-- appender referenced after it is defined -->
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="ASYNC_LOG_INFO"/>
    </root>

</configuration>

  • 貼一下application.properties中配置的projectName


    image.png

  • 最終的日志文件路徑如下


    image.png
  • 日志內(nèi)容如下
2019-04-12 15:20:31.828  [o-9099-exec-702]  INFO o.s.web.servlet.DispatcherServlet        - FrameworkServlet 'dispatcherServlet': initialization started
2019-04-12 15:20:31.853  [o-9099-exec-702]  INFO o.s.web.servlet.DispatcherServlet        - FrameworkServlet 'dispatcherServlet': initialization completed in 24 ms
2019-04-12 15:20:31.902  [o-9099-exec-702] ERROR o.s.b.w.servlet.support.ErrorPageFilter  - Forwarding to error page from request [/user/test1] due to exception [null]

java.lang.NullPointerException: null
    at com.my.test.controller.UserController.test(UserController.java:61)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:130)
    at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:66)
    at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:105)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:123)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:341)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1441)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)

三、配置filebeat收集info文件日志

  1. 編輯filebeat.yml
vi /etc/filebeat/filebeat.yml
  1. 輸入以下內(nèi)容
###################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common
# options. The filebeat.reference.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/filebeat/index.html

# For more available modules and options, please see the filebeat.reference.yml sample
# configuration file.

#=========================== Filebeat inputs =============================

filebeat.inputs:


#=========== nginx access-json日志 ==============

- type: log

  enabled: true

  paths:
    - /var/log/nginx/access-json.log   #指明讀取文件的位置
  tags: ["nginx-access"]       #用于logstash過(guò)濾


#=========== nginx error日志 ==============

- type: log

  enabled: true

  paths:
    - /var/log/nginx/error.log   #指明讀取文件的位置
  tags: ["nginx-error"]      #用于logstash過(guò)濾


#=========== tomcat access日志 ==============

- type: log

  enabled: true

  paths:
    - /root/server/apache-tomcat-aic/logs/localhost_access_log.Y-M-D.txt   #指明tomcat-access日志文件的位置
  tags: ["tomcat-access"]      #用于logstash過(guò)濾


#=========== tomcat job項(xiàng)目日志 ==============
- type: log

  enabled: true

  paths:
    - /root/server/apache-tomcat-aic/logs/aic-job/info.log   #指明Springboot項(xiàng)目日志文件的位置
  multiline:
    pattern: '^\s*(\d{4}|\d{2})\-(\d{2}|[a-zA-Z]{3})\-(\d{2}|\d{4})'   # 指定匹配的表達(dá)式（匹配以 2017-11-15 08:04:23:889 時(shí)間格式開(kāi)頭的字符串）
    negate: true                                # 是否匹配到
    match: after                                # 合并到上一行的末尾, 為了error日志
    max_lines: 1000                             # 最大的行數(shù)
    timeout: 30s                                # 如果在規(guī)定的時(shí)候沒(méi)有新的日志事件就不等待后面的日志

  tags: ["aic-job"]      #用于logstash過(guò)濾

#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Redis output ------------------------------
output.redis:
   hosts: ["192.168.1.110:6379"]   #輸出到redis的機(jī)器
   password: "123456"
   key: "filebeat:test16"   #redis中日志數(shù)據(jù)的key值?
   db: 0
   timeout: 5


#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

四、配置logstash過(guò)濾項(xiàng)目info文件日志

  1. 新建一個(gè)logstash的配置文件
vi /etc/logstash/conf.d/myJob.conf
  1. 輸入以下內(nèi)容
input {
    redis {
        data_type =>"list"
        key =>"filebeat:test16"
        host =>"192.168.1.110"
        port => 6379
        password => "123456"
        threads => "8"
        db => 0
        #codec => json
        }
}

filter {
    
    if "aic-job" in [tags]{
        grok {
            match => ["message", "%{TIMESTAMP_ISO8601:time}\s* \s*%{NOTSPACE:thread-id}\s* \s*%{LOGLEVEL:level}\s* \s*%{JAVACLASS:class}\s* \- \s*%{JAVALOGMESSAGE:logmessage}\s*"]
        }
        
    }
    mutate {
        remove_field => "log"
        remove_field => "beat"
        remove_field => "meta"
        remove_field => "prospector"
        remove_field => "[host][os]"
    }
}


output {
    if "aic-job" in [tags]{
        elasticsearch {
            hosts => ["192.168.1.110:9200"]      
            index => "logstash-test16-tomcat-aic-job-%{+yyyy.MM.dd}"      
        }
    }
    
}
  1. 重啟logstash，查看啟動(dòng)日志是否報(bào)錯(cuò)。
systemctl restart logstash   #重啟

tail -f /var/log/logstash/logstash-plain.log      #查看運(yùn)行日志

五、配置Kibana展示應(yīng)用日志

image.png

image.png
最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容