Kafka Source

Kafka Source

Intro: To add data from a Kafka source, you configure a new pool source within Mezmo. This allows ingestion of data from Kafka topics without affecting your production environment.

Steps:

1. Navigate to the Pipelines page, select your pipeline, and click Edit Pipeline.
2. Click Add Source, scroll down to the Kafka Source, and click it.
3. Optionally add a title and description.
4. Add brokers as an array of host:port pairs (e.g., ["broker1:9092", "broker2:9092"]).
5. Add Kafka topics to subscribe to (supporting multiple topics).
6. Define a consumer group for the Kafka source.
7. Select an authentication method (plain, SCRAM-SHA-256, or SCRAM-SHA-512) and provide a username/password.
8. Click Save, then click Deploy Pipeline to activate the Kafka source.