Create a Kafka Data Source

[amit_kothari]

Introduction

This article will review how to create a Kafka Data Source.

Let's Go

In Incorta Analytics, go to  "Data Sources & Data Files"

In the "Add New Data Source" window:

1. Select "Kafka" from the Data Source drop-down list.
2. Provide the data source name in the "Data Source Name" field.
3. Provide the topic in the "Topic" field.
4. Type the brokers list in the "Brokers List" field.
5. In the "Message Type" field, it is optional to provide the name which is 
   the root table name, and would also be used as the schema name.
6. Switch the "Trim messageType after dash" feature on/off, by switching 
   the radio button to blue/grey, respectively.
7. Select a Kafka version from the "Select Kafka Version" drop-down list.
8. Click Choose File to select the JSON file containing the Avro schema
   generated in the "Create an Avro schema using the Avro Extractor
   tool" section.
9. Click Add Data Source.

Now that you created the Kafka data source, you are ready to navigate to the "Schemas" page and create a schema that consumes data from the Kafka data source.  Because the datasource knows about the tables and joins via the avro file, the "Schema Wizard" will help you create the necessary tables for this connection.

Related Material

• Kafka Overview 
• JSON - Avro Extractor Tool