nodefluent/kafka-rest

Node.js Kafka REST API 镜像

镜像概述

本镜像基于Node.js实现Kafka REST API，目标是模拟Confluent Kafka REST Proxy，为kafka-topics-ui等应用提供更高效的Kafka主题数据访问接口，解决Confluent代理在处理大消息 payload 或大量主题消息时的性能问题。

核心功能

• 提供REST接口访问Kafka主题数据
• 支持通过环境变量自定义Kafka REST参数配置
• 兼容kafka-topics-ui等依赖Kafka REST接口的应用

使用场景

适用于需要快速访问Kafka主题数据的场景，例如：

• kafka-topics-ui展示Kafka主题内容时
• 应用程序通过REST接口读取或写入Kafka主题数据时

配置说明

可通过环境变量自定义Kafka REST参数，示例如下：

• 设置HTTP端口：KAFKA_REST_HTTP_PORT: 8082
• 开启消费者和生产者调试模式：KAFKA_REST_DEBUG: "all"
• 设置消费者组ID：KAFKA_REST_CONSUMER_GROUP_ID: "kafka-rest-consumer-group"

完整配置参数列表及说明可参考https://github.com/edenhill/librdkafka/blob/0.9.5.x/CONFIGURATION.md%E3%80%82

部署示例（Docker Compose）

yaml
---
version: "2"
services:
  zookeeper:
    image: wurstmeister/zookeeper:latest
    ports:
      - 2181:2181

  kafka:
    image: wurstmeister/kafka:0.10.2.1
    ports:
      - "9092:9092"
    links:
      - zookeeper
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_PORT: 9092
      KAFKA_ADVERTISED_HOST_NAME: "kafka"
      KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka:9092"
      KAFKA_LISTENERS: "PLAINTEXT://:9092"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "test:1:1"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"

  kafka-rest:
    image: nodefluent/kafka-rest
    ports:
      - 8082:8082
    links:
      - kafka
      - zookeeper
    depends_on:
      - kafka
      - zookeeper
    environment:
      # DEBUG: "*"
      KAFKA_REST_DEBUG: "all"
      KAFKA_REST_HTTP_PORT: 8082
      KAFKA_REST_CONSUMER_METADATA_BROKER_LIST: "kafka:9092"
      KAFKA_REST_PRODUCER_METADATA_BROKER_LIST: "kafka:9092"

  kafka-rest-ui:
    image: nodefluent/kafka-rest-ui
    ports:
      - 8000:8000
    links:
      - kafka-rest
    depends_on:
      - kafka-rest
    environment:
      DEBUG: "*"
      REACT_APP_KAFKA_REST_URL: "[***]"
      REACT_APP_TIMEOUT: "3000"
      PROXY: "yes"