KAFKA

先安裝完zookeeper，小規(guī)模用可以用新版kafka自帶的zookeeper。

下載

cd /usr/local

wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.7.0/kafka_2.12-2.7.0.tgz

#解壓
tar -xzvf kafka_2.12-2.7.0.tgz

#重命名
mv kafka_2.12-2.7 kafka

修改配置

cd kafka/config

vim server.properties

advertised.listeners=PLAINTEXT://外網(wǎng)IP:9092

log.dirs=/usr/local/kafka/logs

啟動(dòng)（注意要先啟動(dòng)zk）

../bin/kafka-server-start.sh config/server.properties

常用命令：

#在kafka安裝根目錄
#啟動(dòng)
bin/kafka-server-start.sh -daemon config/server.properties
ps:-daemon是為了讓在后臺(tái)運(yùn)行
 
#創(chuàng)建tocic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
 
#查詢(xún)topic列表
bin/kafka-topics.sh --list --zookeeper localhost:2181
 
#啟動(dòng) Producer
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
 
#啟動(dòng) Consumer
bin/kafka-console-consumer.sh --bootstrap-server local:9092 --topic test --from-beginning
 
#topic的信息
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test

KAFKA-EAGLE可視化界面

下載&解壓

cd /usr/local

wget https://github.com/smartloli/kafka-eagle-bin/archive/v2.0.4.tar.gz

tar -xvf kafka-eagle-bin-1.2.4.tar.gz

mv kafka-eagle-bin-1.2.4 kafka-eagle

配置環(huán)境變量(需要配置java_home，yum安裝看
https://blog.csdn.net/a704397849/article/details/103809206)

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64

export KE_HOME=/usr/local/kafka-eagle-bin-1.2.4/kafka-eagle-web-1.2.4

export PATH=$PATH:$KE_HOME/bin:$JAVA_HOME/bin

改好后更新配置

source /etc/profile

修改Kafka-Eagle配置文件

cd /usr/local/kafka-eagle/kafka-eagle-web-1.2.4

vi ./conf/system-config.properties

修改節(jié)點(diǎn)如下：

kafka.eagle.url=jdbc:sqlite:/usr/local/kafka-eagle/kafka-eagle-web-1.2.4/bin/db/ke.db

######################################
# multi zookeeper&kafka cluster list
######################################
#kafka.eagle.zk.cluster.alias=cluster1,cluster2
#cluster1.zk.list=tdn1:2181,tdn2:2181,tdn3:2181
#cluster2.zk.list=xdn10:2181,xdn11:2181,xdn12:2181

#單節(jié)點(diǎn)
kafka.eagle.zk.cluster.alias=cluster1
cluster1.zk.list=192.168.1.104:2181

創(chuàng)建數(shù)據(jù)庫(kù)目錄以及啟動(dòng)

cd /usr/local/kafka-eagle/kafka-eagle-web-1.2.4/bin
mkdir db

chmod +x ke.sh

./ke.sh start

啟動(dòng)成功打印日志如下;

*******************************************************************
* Kafka Eagle system monitor port successful...
*******************************************************************
[2020-02-07 17:05:57] INFO: Status Code[0]
[2020-02-07 17:05:57] INFO: [Job done!]
Welcome to
__ __ ___ ____ __ __ ___ ______ ___ ______ __ ______
/ //_/ / | / __/ / //_/ / | / ____/ / | / ____/ / / / ____/
/ ,< / /| | / /_ / ,< / /| | / __/ / /| | / / __ / / / __/
/ /| | / ___ | / __/ / /| | / ___ | / /___ / ___ |/ /_/ / / /___ / /___
/_/ |_| /_/ |_|/_/ /_/ |_| /_/ |_| /_____/ /_/ |_|\____/ /_____//_____/
Version 1.3.8
*******************************************************************
* Kafka Eagle Service has started success.
* Welcome, Now you can visit 'http://10.0.0.200:8048/ke'
* Account:admin ,Password:123456
*******************************************************************
* ke.sh [start|status|stop|restart|stats]
* https://www.kafka-eagle.org/
*******************************************************************

如果出現(xiàn)錯(cuò)誤，請(qǐng)查看日志日志是否出問(wèn)題

如果沒(méi)問(wèn)題，則直接登錄

• http://host:8048/ke
• 默認(rèn)用戶(hù)名:admin

• 默認(rèn)密碼:123456

  
  

監(jiān)控趨勢(shì)圖

Kafka系統(tǒng)默認(rèn)是沒(méi)有開(kāi)啟JMX端口的，所以Kafka Eagle的監(jiān)控趨勢(shì)圖默認(rèn)采用不啟用的方式，即kafka.eagle.metrics.charts=false。如果需要查看監(jiān)控趨勢(shì)圖，需要開(kāi)啟Kafka系統(tǒng)的JMX端口，設(shè)置該端口在$KAFKA_HOME/bin/kafka-server-start.sh腳本中，設(shè)置內(nèi)容如下：

vim  /opt/software/kafka-eagle/conf/system-config.properties

######################################
# enable kafka metrics
######################################
kafka.eagle.metrics.charts=true

開(kāi)啟JMX監(jiān)控

eagle通過(guò)jmx收集一些指標(biāo)，在使用jmx之前需要確保kafka開(kāi)啟了jmx監(jiān)控

開(kāi)啟方法1：kafka啟動(dòng)時(shí)要添加JMX_PORT=9999

開(kāi)啟方法2：vim bin/kafka-server-start.sh

...
vi /opt/software/kafka/bin/kafka-server-start.sh

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"
    # 這里的端口不一定非要設(shè)置成9999，端口只要可用，均可。
    export JMX_PORT="9999" 
    #export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

Kafka Eagle系統(tǒng)會(huì)自動(dòng)獲取這個(gè)JMX端口，采集結(jié)果如下：

配置文件參考鏈接：
https://www.cnblogs.com/weixiuli/p/6413109.html

集群部署

3臺(tái)機(jī)器，分別是192.168.1.104，192.168.1.105，192.168.1.107
kafka配置文件加上

#三臺(tái)機(jī)器必須不同
broker-id = 1
zookeeper.connect=192.168.88.21:2181,192.168.88.22:2181,192.168.88.23:2181

之后先啟動(dòng)zk集群，再啟動(dòng)kafka集群，啟動(dòng)完成后，登錄eagle查看即可。