Build Kafka Client Applications on Confluent Cloud¶
Developers can build a Producer or Consumer Client application for Confluent Cloud using a preferred programming language and a Apache Kafka® client library.
TLS SNI extension requirement¶
The client libraries you use to connect to Confluent Cloud must include a Server Name Indication (SNI) extension in the Transport Layer Security (TLS) handshake.
Note that the client libraries that Confluent provides do include the SNI extension in the TLS handshake.
For Kafka protocol connections (which use port 9092), the SNI extension must be set to the DNS hostname of the bootstrap endpoint or one of the Kafka brokers.
- For initial connections to the cluster, they include the configured bootstrap endpoint as the SNI extension.
- For the subsequent connections to individual brokers which are made after bootstrapping, they include that specific broker’s endpoint as the SNI extension.
These broker endpoints are automatically discovered as part of client bootstrapping and do not need to be configured.
For REST connections (which use port 443), the SNI extension must be set to the DNS hostname of the REST endpoint.
Any forward proxies deployed in your environment that are in the path of the network traffic to Confluent Cloud must be configured to forward the SNI extension unmodified.
For testing conformance with TLS SNI requirements, see TLS connectivity testing.
See Transport Layer Security (TLS) Extensions: Server Name Indication and The Transport Layer Security (TLS) Protocol Version 1.3: Server Name Indication.
Clients overview¶
You can use Apache Kafka® clients to write distributed applications and microservices that read, write, and process streams of events in parallel, at scale, and in a fault-tolerant manner, even in the case of network problems or machine failures.
The Kafka client library provides functions, classes, and utilities that you can use to create Kafka producer clients and consumer clients using various programming languages. The primary way to build production-ready producers and consumers is by using a programming language and a Kafka client library.
The official Confluent supported clients are:
- Java: The official Java client library supports the producer, consumer, Streams, and Connect APIs.
- librdkafka and derived
clients:
- C/C++: A C/C++ client library supporting the Producer and Consumer APIs.
- Python: A Python client library supporting the Producer and Consumer APIs.
- Go: A Go client library supporting the Producer and Consumer APIs.
- .NET: A .NET client library supporting the Producer and Consumer APIs.
When you use the official Confluent-supported clients, you get the same enterprise-level support that you get with the rest of Confluent Platform:
- The release cycle for Confluent-provided clients follow the Confluent release cycle, as opposed to the Apache Kafka release cycle.
- Confluent Platform maintenance fixes are provided for the 2-3 years (2 years with the Standard Support and 3 years with the Platinum Support) after the initial release of a minor version.
Additional open-source and community-developed Kafka client libraries are available for other programming languages. Some of these include Scala, Node.js, Ruby, Rust, JMS, PHP, and Elixir.
The core APIs in the Kafka client library are:
- Producer API: This API provides classes and methods for creating and sending messages to Kafka topics. It allows developers to specify message payloads, keys, and metadata and to control message delivery and acknowledgment.
- Consumer API: This API provides classes and methods for consuming messages from Kafka topics. It allows developers to subscribe to one or more topics, receive messages in batches or individually, and process messages using custom logic.
- Streams API: This API provides a high-level abstraction for building real-time data processing applications that consume, transform, and produce data streams from Kafka topics.
- Connector API: This API provides a framework for building connectors that can transfer data between Kafka topics and external data systems, such as databases, message queues, and cloud storage services.
- Admin API: This API provides functions for managing Kafka topics, partitions, and configurations. It allows developers to create, delete, and update topics and retrieve metadata about Kafka clusters and brokers.
In addition to these core APIs, the Kafka client library includes various tools and utilities for configuring and monitoring Kafka clients and clusters, handling errors and exceptions and optimizing client performance and scalability.
Client guides¶
Kafka Java Client
Docs | Tutorial | API | GitHub
Java producer and consumer shipped with Apache Kafka.
Client versions and support¶
Java client support¶
The matrix shown below follows the same support timeline as mentioned in the Confluent Platform and Apache Kafka Compatibility matrix. The Kafka clients are available for download on the org.apache.kafka/kafka-clients site.
The Kafka client version matches and maps to the version of Kafka that supports it. To learn more, see the Apache Kafka Clients Maven Repository.
Confluent supports Kafka clients included with new releases of Kafka in the interval before a corresponding Confluent Platform release, and when connecting to Confluent Cloud.
For example, on May 1, 2024, Confluent would support Java clients 3.7.x even though the corresponding Confluent Platform release, 7.7.x would not be released on May 1, 2024.
| Java clients version | Release Date | Standard End of Support | Platinum End of Support |
|---|---|---|---|
| 3.8.x | July 26, 2024 | July 26, 2026 | July 26, 2027 |
| 3.7.x | February 27, 2024 | February 27, 2026 | February 27, 2027 |
| 3.6.x | October 11, 2023 | October 11, 2025 | October 11, 2026 |
| 3.5.x | August 25, 2023 | August 25, 2025 | August 25, 2026 |
| 3.4.x | May 4, 2023 | May 4, 2025 | May 4, 2026 |
| 3.3.x | November 4, 2022 | November 4, 2024 | November 4, 2025 |
| 3.2.x | July 6, 2022 | July 6, 2024 | July 6, 2025 |
| 3.1.x | April 5, 2022 | April 5, 2024 | April 5, 2025 |
| 3.0.x | October 27, 2021 | October 27, 2023 | October 27, 2024 |
| 2.8.x | June 8, 2021 | June 8, 2023 | June 8, 2024 |
| 2.7.x | February 9, 2021 | February 9, 2023 | February 9, 2024 |
| 2.6.x | September 24, 2020 | September 24, 2022 | September 24, 2023 |
| 2.5.x | April 24, 2020 | April 24, 2022 | April 24, 2023 |
| 2.4.x | January 10, 2020 | January 10, 2022 | January 10, 2023 |
| 2.3.x | July 19, 2019 | July 19, 2021 | July 19, 2022 |
| 2.2.x | March 28, 2019 | March 28, 2021 | March 28, 2022 |
| 2.1.x | December 14, 2018 | December 14, 2020 | December 14, 2021 |
| 2.0.x | July 31, 2018 | July 31, 2020 | July 31, 2021 |
| 1.1.x | April 16, 2018 | April 16, 2020 | April 16, 2021 |
- The end of support date applies to the stated minor version, and all maintenance versions associated with that minor version. For example, support for Kafka 2.6.4 ended on September 24, 2022.
- The two year support policy is standard. End of support is extended one year for Platinum support agreements.
librdkafka and derived client support¶
librdkafka is an open source community project available at: https://github.com/confluentinc/librdkafka.
Confluent maintains its own official, signed set of binaries for librdkafka, available to you here: https://packages.confluent.io/clients/.
| Programming Language | Library Name | Documentation |
|---|---|---|
| C/C++ Client | librdkafka | Kafka C/C++ Client |
| Go Client | confluent-kafka-go | Kafka Go Client |
| .NET Client | confluent-kafka-dotnet | Kafka .NET Client |
| Python Client | confluent-kafka-python | Kafka Python Client |
| librdkafka Client Version | Release Date | Standard End of Support | Platinum End of Support |
|---|---|---|---|
| 2.6.x | October 12, 2024 | October 12, 2026 | October 12, 2027 |
| 2.5.x | July 10, 2024 | July 10, 2026 | July 10, 2027 |
| 2.4.x | May 7, 2024 | May 7, 2026 | May 7, 2027 |
| 2.3.x | October 26, 2023 | October 26, 2025 | October 26, 2026 |
| 2.2.x | July 13, 2023 | July 13, 2025 | July 13, 2026 |
| 2.1.x | April 6, 2023 | April 6, 2025 | April 6, 2026 |
| 2.0.x | January 24, 2023 | January 24, 2025 | January 24, 2026 |
| 1.9.x | June 16, 2022 | June 16, 2024 | June 16, 2025 |
| 1.8.x | September 22, 2021 | September 22, 2023 | September 22, 2024 |
| 1.7.x | August 8, 2021 | August 8, 2023 | August 8, 2024 |
| 1.6.x | February 21, 2021 | February 21, 2023 | February 21, 2024 |
- An end-of-support date is based on the first release of the specified minor version. The support date windows apply to that minor version and any maintenance/patch versions that come under the minor version. For example, 1.9.0 and 1.9.1 would have the same support windows as specified for 1.9.x.
- The two-year support policy is standard. End of support is extended one year for Platinum support agreements.
Client examples¶
To learn how to build Kafka clients apps using your favorite language, including Java, .NET, Python, Node.js, C/C++, and Go, see Kafka Client Examples for Confluent Cloud.
Note
Confluent recommends upgrading to the latest client since current fixes are generally not found in older clients.