前言

logstash是一個(gè)開(kāi)源的日志統(tǒng)一處理數(shù)據(jù)收集器,屬于ELK中的L,在日志收集領(lǐng)域應(yīng)用廣泛.

docker默認(rèn)的日志驅(qū)動(dòng)是json-file,每一個(gè)容器都會(huì)在本地生成一個(gè)/var/lib/docker/containers/containerID/containerID-json.log,而日志驅(qū)動(dòng)是支持?jǐn)U展的,本章主要講解的是使用logstash收集docker日志.

docker是沒(méi)有l(wèi)ogstash這個(gè)驅(qū)動(dòng)的,但是可以通過(guò)logstash的gelf input插件收集gelf驅(qū)動(dòng)的日志.

前提

1. docker
2. 了解logstash配置
3. docker-compose

準(zhǔn)備配置文件

docker-compose.yml

version: '3.7'

x-logging:
  &default-logging
  driver: gelf
  options:
    gelf-address: "udp://localhost:12201"
    mode: non-blocking
    max-buffer-size: 4m
    tag: "kafeidou.{{.Name}}"  #配置容器的tag,以kafeidou.為前綴,容器名稱為后綴,docker-compose會(huì)給容器添加副本后綴,>如 logstash_1

services:

  logstash:
    ports:
      - 12201:12201/udp
    image: docker.elastic.co/logstash/logstash:7.5.1
    volumes:
      - ./logstash.yml:/usr/share/logstash/config/logstash.yml
      - /var/log/logstash:/var/log/logstash
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf

  logstash-worker:
    image: docker.elastic.co/logstash/logstash:7.5.1
    depends_on:
      - logstash
    logging:
      driver: "gelf"
      options:
        gelf-address: "udp://localhost:12201"

logstash.yml

http.host: "0.0.0.0"

logstash.conf

input {
 gelf{
  use_udp => true
  port_tcp => 12202
 }
}

 output {
   file {
        path => "/var/log/logstash/%{+yyyy-MM-dd-HH}/%{container_name}.log"
   }
 }

由于logstash需要在配置的目錄中有寫入的權(quán)限,所以需要先準(zhǔn)備好存放log的目錄以及給予權(quán)限.
創(chuàng)建目錄

mkdir /var/log/logstash

給予權(quán)限,這里用于實(shí)驗(yàn)演示,直接授權(quán)777

chmod -R 777 /var/log/logstash

在docker-compose.yml,logstash.conf和logstash.yml文件的目錄中執(zhí)行命令:
docker-compose up -d

[root@master logstash]# docker-compose up -d
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Starting logstash_logstash_1 ... done
Starting logstash_logstash-worker_1 ... done

logstash啟動(dòng)較慢,我實(shí)驗(yàn)的效果是90秒左右,所以更推薦使用fluentd收集日志

查看一下日志目錄下,應(yīng)該就有對(duì)應(yīng)的容器日志文件了:

[root@master logstash]# ls /var/log/logstash/
2020-02-16
[root@master logstash]# ls /var/log/logstash/2020-02-16/
logstash_logstash-worker_1.log

也可以直接下載我的文件:

1. docker-compose.yml
2. logstash.conf
3. logstash.yml

總結(jié)

技術(shù)選型更推薦fluentd，為什么?

fluentd更加輕量級(jí)并且更靈活,并且目前屬于CNCF,活躍度和可靠性都更上一層樓.

為什么還要介紹logstash收集docker日志?

在一個(gè)公司或者業(yè)務(wù)中如果已經(jīng)在使用ELK技術(shù)棧,也許就沒(méi)有必要再引入一個(gè)fluentd,而是繼續(xù)用logstash打通docker日志這塊.這里主要做一個(gè)分享,讓遇到這種情況的同學(xué)能夠有多一個(gè)選擇.

推薦閱讀:

使用fluentd作為docker日志驅(qū)動(dòng)收集日志

始發(fā)于 四顆咖啡豆 ,轉(zhuǎn)載請(qǐng)聲明出處. 歡迎交流