架構(gòu)
switch --> rsyslog --> filebeat --> logstash --> elasticsearch --> kibana
image.png
關(guān)閉selinux和防火墻
setenforce 0 # 臨時(shí)關(guān)閉
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config # 永久關(guān)閉
systemctl stop firewalld.service
systemctl disable firewalld.service
檢查是否已安裝rsyslog（CeontOS 7一般默認(rèn)安裝了此軟件）
rpm -qa |grep rsyslog
修改rsyslog.conf配置文件，如下
[root@testhost mnt]# grep -v "^#\|^$" /etc/rsyslog.conf
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
$IMJournalStateFile imjournal.state
*.info;mail.none;authpriv.none;cron.none;local6.none /var/log/messages
$template h3c,"/mnt/h3c/%FROMHOST-IP%.log"
local6.* ?h3c
authpriv.* /var/log/secure
mail.* -/var/log/maillog
cron.* /var/log/cron
*.emerg :omusrmsg:*
uucp,news.crit /var/log/spooler
local7.* /var/log/boot.log
重啟rsyslog服務(wù)
systemctl restart rsyslog.service
檢查服務(wù)端口
[root@testhost mnt]# netstat -antupl |grep syslog
tcp 0 0 0.0.0.0:514 0.0.0.0:* LISTEN 4772/rsyslogd
tcp6 0 0 :::514 :::* LISTEN 4772/rsyslogd
udp 0 0 0.0.0.0:514 0.0.0.0:* 4772/rsyslogd
udp6 0 0 :::514 :::* 4772/rsyslogd
創(chuàng)建日志存放目錄
[root@testhost mnt]# ll
總用量 0
drwxrwxrwx. 2 root root 29 10月 28 20:50 h3c
[root@testhost mnt]# pwd
/mnt
網(wǎng)絡(luò)設(shè)備（H3C）交換機(jī)配置
<H3C>dis curr | inc info-center
undo info-center logfile enable
info-center loghost source Vlan-interface3
info-center loghost 192.168.10.100 facility local6
在交換機(jī)端輸入命令出發(fā)產(chǎn)生日志后即可在/mnt/h3c/目錄下看到對(duì)應(yīng)的日志文件和交換機(jī)日志記錄
[root@testhost h3c]# ll
總用量 4
-rw-------. 1 root root 925 10月 28 21:02 192.168.10.111.log
[root@testhost h3c]# pwd
/mnt/h3c
下載并安裝filebeat（elk之前已安裝）
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.8.1-x86_64.rpm
rpm -ivh filebeat-7.8.1-x86_64.rpm
編輯filebeat配置文件，收集rsyslog的日志文件到logstash
[root@testhost ~]# grep -v "^#\|^$\|^ #" /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /mnt/h3c/*
tags: ["h3c"]
include_lines: ['LOGIN','Failed','failed','error','ERROR','\bDOWN\b','\bdown\b','\bUP\b','\bup\b']
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3
output.logstash:
hosts: ["localhost:5044"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
添加logstash配置文件networklog.conf（沒(méi)有此文件需要自行創(chuàng)建）
[root@testhost ~]# grep -v "^#\|^$" /etc/logstash/conf.d/networklog.conf
input {
beats {
port => 5044
}
}
filter {
if "huawei" in [tags] {
grok{
match => {"message" => "%{SYSLOGTIMESTAMP:time} %{DATA:hostname} %{GREEDYDATA:info}"}
}
}
else if "h3c" in [tags] {
grok{
match => {"message" => "%{SYSLOGTIMESTAMP:time} %{YEAR:year} %{DATA:hostname} %{GREEDYDATA:info}"}
}
}
mutate {
remove_field => ["message","time","year","offset","tags","path","host","@version","[log]","[prospector]","[beat]","[input][type]","[source]"]
}
}
output{
stdout {codec => rubydebug}
elasticsearch {
index => "networklogs-%{+YYYY.MM.dd}"
hosts => ["127.0.0.1:9200"]
sniffing => false
}
}
調(diào)整logstash管道配置文件
vim /etc/logstash/pipelines.yml
- pipeline.id: main
path.config: "/etc/logstash/conf.d/*.conf" # 加載networklog.conf配置
- pipeline.id: elastiflow
path.config: "/etc/logstash/elastiflow/conf.d/*.conf" # 加載elastiflow配置（sflow使用）
重啟logstash，systemctl restart filebeat.service，檢查networklog.conf中定義的5044端口是否正常監(jiān)聽(tīng)
[root@testhost ~]# netstat -antupl |grep 5044
tcp6 0 0 :::5044 :::* LISTEN 5957/java
kibana配置
瀏覽器打開(kāi)http://127.0.0.1:5601
打開(kāi)管理頁(yè)面：Home --> Management --> Stack Management
image.png
image.png
image.png
image.png
image.png
image.png
image.png
參考鏈接：
https://cloud.tencent.com/developer/article/1539522
https://elasticsearch.cn/question/8247
https://blog.csdn.net/tladagio/article/details/120436242
