danielqsj/kafka-exporter

kafka_exporter

https://app.travis-ci.com/danielqsj/kafka_exporter.svg?branch=master](https://app.travis-ci.com/danielqsj/kafka_exporter)https://img.shields.io/docker/pulls/danielqsj/kafka-exporter.svg](https://hub.docker.com/r/danielqsj/kafka-exporter)https://goreportcard.com/badge/github.com/danielqsj/kafka_exporter](https://goreportcard.com/report/github.com/danielqsj/kafka_exporter)https://img.shields.io/badge/language-Go-red.svg](https://github.com/danielqsj/kafka-exporter)https://img.shields.io/badge/release-1.4.1-green.svg](https://github.com/alibaba/derrick/releases)https://img.shields.io/badge/license-Apache%202-4EB1BA.svg](https://www.apache.org/licenses/LICENSE-2.0.html)

Kafka exporter for Prometheus. For other metrics from Kafka, have a look at the https://github.com/prometheus/jmx_exporter.

Table of Contents

• Compatibility
• Dependency
• Download
• Compile
  • Build Binary
  • Build Docker Image
• Run
  • Run Binary
  • Run Docker Image
• Flags
  • Notes
• Metrics
  • Brokers
  • Topics
  • Consumer Groups
• Grafana Dashboard
• Contribute

• ---

• License

Compatibility

Support https://kafka.apache.org version 0.10.1.0 (and later).

Dependency

• https://prometheus.io
• https://shopify.github.io/sarama
• https://golang.org
• https://github.com/golang/dep

Download

Binary can be downloaded from https://github.com/danielqsj/kafka_exporter/releases page.

Compile

Build Binary

shell
make

Build Docker Image

shell
make docker

Docker Hub Image

shell
docker pull danielqsj/kafka-exporter:latest

It can be used directly instead of having to build the image yourself. (https://hub.docker.com/r/danielqsj/kafka-exporter%5C)

Run

Run Binary

shell
kafka_exporter --kafka.server=kafka:9092 [--kafka.server=another-server ...]

Run Docker Image

docker run -ti --rm -p 9308:9308 danielqsj/kafka-exporter --kafka.server=kafka:9092 [--kafka.server=another-server ...]

Flags

This image is configurable using different flags

Notes

Boolean values are uniquely managed by https://github.com/alecthomas/kingpin/blob/master/README.md#boolean-values. Each boolean flag will have a negative complement: --<name> and --no-<name>.

For example:

If you need to disable sasl.handshake, you could add flag --no-sasl.handshake

Metrics

Documents about exposed Prometheus metrics.

For details on the underlying metrics please see https://kafka.apache.org/documentation.

Brokers

Metrics details

Metrics output example

txt
# HELP kafka_brokers Number of Brokers in the Kafka Cluster.
# TYPE kafka_brokers gauge
kafka_brokers 3

Topics

Metrics details

Metrics output example

txt
# HELP kafka_topic_partitions Number of partitions for this Topic
# TYPE kafka_topic_partitions gauge
kafka_topic_partitions{topic="__consumer_offsets"} 50

# HELP kafka_topic_partition_current_offset Current Offset of a Broker at Topic/Partition
# TYPE kafka_topic_partition_current_offset gauge
kafka_topic_partition_current_offset{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_oldest_offset Oldest Offset of a Broker at Topic/Partition
# TYPE kafka_topic_partition_oldest_offset gauge
kafka_topic_partition_oldest_offset{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_in_sync_replica Number of In-Sync Replicas for this Topic/Partition
# TYPE kafka_topic_partition_in_sync_replica gauge
kafka_topic_partition_in_sync_replica{partition="0",topic="__consumer_offsets"} 3

# HELP kafka_topic_partition_leader Leader Broker ID of this Topic/Partition
# TYPE kafka_topic_partition_leader gauge
kafka_topic_partition_leader{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_leader_is_preferred 1 if Topic/Partition is using the Preferred Broker
# TYPE kafka_topic_partition_leader_is_preferred gauge
kafka_topic_partition_leader_is_preferred{partition="0",topic="__consumer_offsets"} 1

# HELP kafka_topic_partition_replicas Number of Replicas for this Topic/Partition
# TYPE kafka_topic_partition_replicas gauge
kafka_topic_partition_replicas{partition="0",topic="__consumer_offsets"} 3

# HELP kafka_topic_partition_under_replicated_partition 1 if Topic/Partition is under Replicated
# TYPE kafka_topic_partition_under_replicated_partition gauge
kafka_topic_partition_under_replicated_partition{partition="0",topic="__consumer_offsets"} 0

Consumer Groups

Metrics details

Metrics output example

txt
# HELP kafka_consumergroup_current_offset Current Offset of a ConsumerGroup at Topic/Partition
# TYPE kafka_consumergroup_current_offset gauge
kafka_consumergroup_current_offset{consumergroup="KMOffsetCache-kafka-manager-3806276532-ml44w",partition="0",topic="__consumer_offsets"} -1

# HELP kafka_consumergroup_lag Current Approximate Lag of a ConsumerGroup at Topic/Partition
# TYPE kafka_consumergroup_lag gauge
kafka_consumergroup_lag{consumergroup="KMOffsetCache-kafka-manager-3806276532-ml44w",partition="0",topic="__consumer_offsets"} 1

Grafana Dashboard

Grafana Dashboard ID: 7589, name: Kafka Exporter Overview.

For details of the dashboard please see https://grafana.com/dashboards/7589.

Contribute

If you like Kafka Exporter, please give me a star. This will help more people know Kafka Exporter.

Please feel free to send me https://github.com/danielqsj/kafka_exporter/pulls.

Contributors ✨

Thanks goes to these wonderful people:

https://github.com/danielqsj/kafka_exporter/graphs/contributors

---

---

Your *** will encourage me to continue to improve Kafka Exporter. Support Alipay ***.

License

Code is licensed under the https://github.com/danielqsj/kafka_exporter/blob/master/LICENSE.