越來越多的企業(yè)已經(jīng)采用ELK解決方案來對其公司產(chǎn)生的日志進(jìn)行分析，筆者最近著手在生產(chǎn)環(huán)境部署自己的ELK stack，本文介紹ELK中elasticsearch5.2集群的實(shí)現(xiàn)。
一、環(huán)境準(zhǔn)備
1、系統(tǒng)：CentOS 6.8
ip及角色：192.168.1.121（master node） 192.168.122（data node） 192.168.123(client node)
2、JDK
# 筆者使用的jdk版本jdk-8u121-linux-x64.rpm，下面給出可以直接下載的JDK版本
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u77-b02/jdk-8u77-linux-x64.rpm"
3、elasticsearch安裝
# 分別在三臺服務(wù)器上安裝elastic，以yum安裝為例
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch# 添加elasticsearch源echo '[elasticsearch-5.x]name=Elasticsearch repository for 5.x packagesbaseurl=https://artifacts.elastic.co/packages/5.x/yumgpgcheck=1gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearchenabled=1autorefresh=1type=rpm-md' | sudo tee /etc/yum.repos.d/elasticsearch.repo# yum makecache && yum install elasticsearch -y
二、elastic配置詳解
1、Elasticsearch cluster 三種角色
master node：master節(jié)點(diǎn)主要用于元數(shù)據(jù)（metadata）處理，如、索引的新增、刪除、分片
data node: data節(jié)點(diǎn)上保存了數(shù)據(jù)片
client node: client節(jié)點(diǎn)起到路由請求的作用，可看做負(fù)載均衡器
2、節(jié)點(diǎn)選擇
# 配置文件中給出了三種配置高性能集群拓?fù)浣Y(jié)構(gòu)的模式,如下：
- 如果你想讓節(jié)點(diǎn)從不選舉為主節(jié)點(diǎn),只用來存儲數(shù)據(jù),可作為負(fù)載器
node.master: false
node.data: true
- 如果想讓節(jié)點(diǎn)成為主節(jié)點(diǎn),且不存儲任何數(shù)據(jù),并保有空閑資源,可作為協(xié)調(diào)器
node.master: true
node.data: false
- 如果想讓節(jié)點(diǎn)既不稱為主節(jié)點(diǎn),又不成為數(shù)據(jù)節(jié)點(diǎn),那么可將他作為搜索器,從節(jié)點(diǎn)中獲取數(shù)據(jù),生成搜索結(jié)果等
node.master: false
node.data: false
3、elasticsearch.yaml配置詳解
# elastic-a1（192.168.1.121），master節(jié)點(diǎn)
[root@elastic-a1 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-node-a1
node.master: true
node.data: true
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: false
network.host: 192.168.1.121
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"]
discovery.zen.minimum_master_nodes: 1
gateway.recover_after_nodes: 2
gateway.recover_after_time: 5m
gateway.expected_nodes: 1
bootstrap.system_call_filter: false
script.engine.groovy.inline.search: on
script.engine.groovy.inline.aggs: on
indices.recovery.max_bytes_per_sec: 20mb
http.cors.enabled: true
http.cors.allow-origin: "*"
# elastic-a2 ，data節(jié)點(diǎn)（192.168.1.122）
[root@elastic-a2 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-node-a2
node.master: false
node.data: true
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: false
network.host: 192.168.1.122
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"]
discovery.zen.minimum_master_nodes: 1
gateway.recover_after_nodes: 2
gateway.recover_after_time: 5m
gateway.expected_nodes: 1
bootstrap.system_call_filter: false
script.engine.groovy.inline.search: on
script.engine.groovy.inline.aggs: on
indices.recovery.max_bytes_per_sec: 20mb
# elastic-a3，client節(jié)點(diǎn)（192.168.1.123）
[root@elastic-a3 ~]# egrep -v "^#|^$" /etc/elasticsearch/elasticsearch.yml
cluster.name: es-cluster
node.name: es-node-a3
node.master: false
node.data: false
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: false
network.host: 192.168.1.123
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"]
discovery.zen.minimum_master_nodes: 1
gateway.recover_after_nodes: 2
gateway.recover_after_time: 5m
gateway.expected_nodes: 1
bootstrap.system_call_filter: false
script.engine.groovy.inline.search: on
script.engine.groovy.inline.aggs: on
indices.recovery.max_bytes_per_sec: 20mb
# 注，建議配置參考本文配置，集體到?jīng)]想?yún)?shù)的含義，這里不具體給你，自行g(shù)oogle，baidu,如果讀者做實(shí)驗(yàn)時沒有足夠多的主機(jī)來實(shí)現(xiàn)es-cluster，可以在同一主機(jī)上進(jìn)行配置，這里只需要修改下面一行：
discovery.zen.ping.unicast.hosts: ["192.168.1.121", "192.168.1.122", "192.168.1.123"] 改成 discovery.zen.ping.unicast.hosts: ["0.0.0.0:9300", "0.0.0.0:9301", "0.0.0.0:9302"]
三、elastic集群啟動
1、分別在三個主機(jī)上啟動elasticsearch服務(wù)，并查看啟動日志是否報(bào)錯
/etc/init.d/elasticsearch start tailf
/var/log/elasticsearch/es-cluster.log
2、啟動成功，瀏覽器訪問 http://192.168.1.121:9200/_cluster/health?pretty=true
# 到這里elasticsearch集群已經(jīng)部署完成，具體使用詳情請自行移步到elsatic官網(wǎng)，查看官方文檔
四、head插件安裝
#（安裝在elastic-a1節(jié)點(diǎn)）
1、參考：https://github.com/mobz/elasticsearch-head
yum install npm git -ycd /usr/share/elasticsearch/gitclonegit://github.com/mobz/elasticsearch-head.gitcdelasticsearch-head && npm installnpm install -g grunt
2、插件配置
cd /usr/share/elasticsearch/elasticsearch-head/ # 移動到插件項(xiàng)目目錄下
vim +4329 _site/app.js #修改app.js中l(wèi)ocalhost為節(jié)點(diǎn)IP，便于連接es-cluster
3、后臺運(yùn)行插件
cd /usr/share/elasticsearch/elasticsearch-head/
nohup grunt server &
4、通過插件es-cluster狀態(tài) # http://192.168.1.121:9100/ （）
五、總結(jié)
1、?unable to install syscall filter
echo "bootstrap.system_call_filter: false" >> /etc/elasticsearch/elasticsearch.yml # 解決報(bào)錯 ！
使用本地 IP（127.0.0.1）時，Elasticsearch 進(jìn)入 dev mode，只能從本機(jī)訪問，只顯示警告。
使用局域網(wǎng)IP后，可以從其他機(jī)器訪問，但啟動時進(jìn)入 production mode，并進(jìn)行 bootstrap check，有可能對不合適的系統(tǒng)參數(shù)報(bào)錯。
2、安裝注意事項(xiàng)主要是elasticsearch配置文件，確保配置文件正確，然后再去啟動elastic節(jié)點(diǎn) 參考鏈接： http://blog.csdn.net/gamer_gyt/article/details/59077189#reply
https://www.gitbook.com/book/chenryn/elk-stack-guide-cn/details
https://www.elastic.co/guide/index.html
