kakfa exporter v1.0.0于今日正式發(fā)布，Prometheus官方推薦，新增多項特性，歡迎大家使用，多提寶貴意見。

項目地址

Github: https://github.com/danielqsj/kafka_exporter
Docker Hub: https://hub.docker.com/r/danielqsj/kafka-exporter

項目狀態(tài)

自v0.2.0版本被Prometheus項目官方推薦以來，鏡像累計下載量已超過6.1k，已經(jīng)穩(wěn)定運行在多個平臺。

特點

kafka exporter 通過 Kafka Protocol Specification 收集 Brokers, Topics 以及 Consumer Groups的相關指標，使用簡單，運行高效，相比于以往通過kafka內置的腳本進行收集，由于沒有了JVM的運行開銷，指標收集時間從分鐘級別降到秒級別，便于大規(guī)模集群的監(jiān)控。

使用方法

詳見項目說明文檔。

指標

Brokers

Metrics details

Metrics output example

# HELP kafka_brokers Number of Brokers in the Kafka Cluster.
# TYPE kafka_brokers gauge
kafka_brokers 3

Topics

Metrics details

Metrics output example

# HELP kafka_topic_partitions Number of partitions for this Topic
# TYPE kafka_topic_partitions gauge
kafka_topic_partitions{topic="__consumer_offsets"} 50

# HELP kafka_topic_partition_current_offset Current Offset of a Broker at Topic/Partition
# TYPE kafka_topic_partition_current_offset gauge
kafka_topic_partition_current_offset{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_oldest_offset Oldest Offset of a Broker at Topic/Partition
# TYPE kafka_topic_partition_oldest_offset gauge
kafka_topic_partition_oldest_offset{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_in_sync_replica Number of In-Sync Replicas for this Topic/Partition
# TYPE kafka_topic_partition_in_sync_replica gauge
kafka_topic_partition_in_sync_replica{partition="0",topic="__consumer_offsets"} 3

# HELP kafka_topic_partition_leader Leader Broker ID of this Topic/Partition
# TYPE kafka_topic_partition_leader gauge
kafka_topic_partition_leader{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_leader_is_preferred 1 if Topic/Partition is using the Preferred Broker
# TYPE kafka_topic_partition_leader_is_preferred gauge
kafka_topic_partition_leader_is_preferred{partition="0",topic="__consumer_offsets"} 1

# HELP kafka_topic_partition_replicas Number of Replicas for this Topic/Partition
# TYPE kafka_topic_partition_replicas gauge
kafka_topic_partition_replicas{partition="0",topic="__consumer_offsets"} 3

# HELP kafka_topic_partition_under_replicated_partition 1 if Topic/Partition is under Replicated
# TYPE kafka_topic_partition_under_replicated_partition gauge
kafka_topic_partition_under_replicated_partition{partition="0",topic="__consumer_offsets"} 0

Consumer Groups

Metrics details

Metrics output example

# HELP kafka_consumergroup_current_offset Current Offset of a ConsumerGroup at Topic/Partition
# TYPE kafka_consumergroup_current_offset gauge
kafka_consumergroup_current_offset{consumergroup="KMOffsetCache-kafka-manager-3806276532-ml44w",partition="0",topic="__consumer_offsets"} -1

# HELP kafka_consumergroup_lag Current Approximate Lag of a ConsumerGroup at Topic/Partition
# TYPE kafka_consumergroup_lag gauge
kafka_consumergroup_lag{consumergroup="KMOffsetCache-kafka-manager-3806276532-ml44w",partition="0",topic="__consumer_offsets"} 1

Release Notes

v1.0.0

• [FEATURE] Support TLS
• [FEATURE] Support disable SASL handshake
• [FEATURE] Support enable Sarama logging for detailed connection management events
• [ENHANCEMENT] Provide specific string for Kafka logging, debugging, and auditing purposes
• [BUGFIX] Fix multiple prometheus servers scraping competition issue
• [BUGFIX] Fix overwrite configuration issue

v0.3.0

• [FEATURE] Support kafka SASL/PLAIN authentication
• [FEATURE] Support topic filter
• [BUGFIX] Fix topics not sync when modifying topic list

v0.2.0

• [CHANGE] Change default port to 9308
• [ENHANCEMENT] Support multiple addresses for kafka servers

v0.1.0

• Initial release