License for Self-Managed Connectors

With a Developer license, you can use Confluent Platform commercial connectors on an unlimited basis, in Connect clusters that use a single-broker Apache Kafka® cluster. For Connect clusters that use a multi-broker Kafka cluster, you can use commercial connectors for a 30-day trial without an enterprise license key. After 30 days, Confluent Platform software stops working, as described in commercial components post-expiry. You must purchase an enterprise license to continue using commercial connectors in Connect clusters that use a multi-broker Kafka cluster.

Refer to the individual supported connector documentation for connector-specific information about license requirements.

Centralized License in the Kafka Connect Worker

License-related properties should be placed in the Connect worker configuration. Note that if you use a Developer license, a Trial (Evaluation) License, or a Control Center License , you do not need to add a license key to the Connect worker or connector configuration.

The Centralized License is a proprietary feature and is not available in Community-licensed distributions. This feature doesn’t apply to open-source Apache Kafka® or Confluent Platform using only Confluent community components–that is, CCS or cp-kafka-connect–which require licensing configurations at the connector level. The Centralized License feature doesn’t work with the cp-kafka-connect-base Docker image. Ensure you use the cp-server-connect-base Docker image to leverage the centralized license feature as this image contains Confluent’s proprietary code.

When the Connect worker configuration includes the license-related properties, the worker injects the license-related properties into each connector configuration that requires a license.

Important

The Kafka Connect worker only injects license-related properties into connector configurations that require these properties and when adding commercial connectors specifically. It doesn’t inject license-related properties if the connector configuration already includes license-related properties, or if the connector doesn’t require a license.

The license-related properties are injected when the connector is started. License properties are never persisted outside of the worker configuration. To change any of the license-related properties passed to the connectors, simply change the license-related properties in the worker configuration and restart the worker. The worker will restart any assigned connectors after injecting the worker’s latest license-related properties.

Note that it is important to set the license-related properties in each Connect worker configuration. You can change the license-related properties and use a rolling restart to have those properties take effect across the cluster.

Connect Worker License Properties

Both Connect standalone and distributed configurations support the same license-related properties that are used in commercial connector configurations. These license-related properties include:

  • confluent.license: The Confluent enterprise license key string for your organization. You don’t need to add this if you are using a Developer license, a Trial (Evaluation) License, or a Control Center License.

  • confluent.topic: The name of the Kafka topic where the license is stored. This defaults to _confluent-command.

  • confluent.topic.replication.factor: This is used only if the license topic specified in confluent.topic does not exist when the license checks are performed. This specifies the replication factor that is used if the license check creates the topic. This property defaults to 3.

  • confluent.topic.bootstrap.servers: The bootstrap addresses for the Kafka cluster where the license topic is stored. By default, the worker uses the value of bootstrap.servers, which is the Kafka cluster that Connect uses. Note that this entry is only needed if the license topic is stored on a different Kafka cluster than the Kafka cluster that Connect uses.

  • confluent.topic.*: Other producer and consumer properties that are required to read and write to the topic specified in confluent.topic at the Kafka cluster specified in confluent.topic.bootstrap.servers. By default, all of the producer and consumer properties are configured without prefixes in the Connect worker configuration. For example, if the broker uses TLS for connections and the worker configuration includes properties such as ssl.keystore.location and ssl.keystore.password, then these will be inferred as confluent.topic.ssl.keystore.location and confluent.topic.ssl.keystore.password, respectively.

    By default the Connect worker uses the same settings and the same Kafka principal for reading and writing to the _confluent-command license topic that it uses to read and write Connect’s internal topics. If the topic does not exist when it’s needed, the topic is created with a replication factor of 3 (a sensible default for production).

    When using Connect in development with a Kafka cluster that has one broker, include confluent.topic.replication.factor=1 so that if a license topic is needed it is created with a single replica.

You can always set any of the confluent.topic.* properties in the worker configuration, even if the value is the same as what would be inferred by the worker from its other properties.

Add or Change Connect Worker License Properties

The default license-properties inferred by the Connect worker may not be sufficient. You may need to set specific license-related properties in all your worker configurations. To do this on a running cluster, change the worker configuration for one worker, restart that worker, and verify all currently-defined connectors and tasks are still healthy. Repeat this on the other workers in the same Connect cluster.

Note that it is important to set the license-related properties in each worker configuration in the Connect cluster. If you don’t, it’s possible that a connector will run successfully when assigned to one worker, but then fails when assigned to a different worker.

You can then start a new licensed connector without defining the license-related properties in the individual connector configurations.

Also note that you can remove the license-related properties from any of your existing licensed connector configurations. Do this one connector at a time. Make sure that after you remove the license-related properties from a connector, that you verify the connector starts with no license-related properties specified in the connector configuration.

If a connector fails to start because of a license-related problem, check the worker logs for the connector configuration. The logs will include the license-related properties injected by the worker. Determine which license-related properties need to be changed and then apply the changes to those worker configs as described above.

Security configuration properties

confluent.topic.ssl.truststore.location

The location of the trust store file.

  • Type: string
  • Default: null
  • Importance: high
confluent.topic.ssl.truststore.password

The password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.

  • Type: password
  • Default: null
  • Importance: high
confluent.topic.ssl.keystore.location

The location of the key store file. This is optional for client and can be used for two-way authentication for client.

  • Type: string
  • Default: null
  • Importance: high
confluent.topic.ssl.keystore.password

The store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured.

  • Type: password
  • Default: null
  • Importance: high
confluent.topic.ssl.key.password

The password of the private key in the key store file. This is optional for client.

  • Type: password
  • Default: null
  • Importance: high
confluent.topic.security.protocol

Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.

  • Type: string
  • Default: “PLAINTEXT”
  • Importance: medium

License topic configuration

A Confluent enterprise license is stored in the _confluent-command topic. This topic is created by default and contains the license that corresponds to the license key supplied through the confluent.license property. No public keys are stored in Kafka topics.

The following describes how the default _confluent-command topic is generated under different scenarios:

  • A 30-day trial license is automatically generated for the _confluent command topic if you do not add the confluent.license property or leave this property empty (for example, confluent.license=).
  • Adding a valid license key (for example, confluent.license=<valid-license-key>) adds a valid license in the _confluent-command topic.

Here is an example of the minimal properties for development and testing.

You can change the name of the _confluent-command topic using the confluent.topic property (for instance, if your environment has strict naming conventions). The example below shows this change and the configured Kafka bootstrap server.

confluent.topic=foo_confluent-command
confluent.topic.bootstrap.servers=localhost:9092

The example above shows the minimally required bootstrap server property that you can use for development and testing. For a production environment, you add the normal producer, consumer, and topic configuration properties to the connector properties, prefixed with confluent.topic..

License topic ACLs

The _confluent-command topic contains the license that corresponds to the license key supplied through the confluent.license property. It is created by default. Connectors that access this topic require the following ACLs configured:

  • CREATE and DESCRIBE on the resource cluster, if the connector needs to create the topic.

  • DESCRIBE, READ, and WRITE on the _confluent-command topic.

    Important

    You can also use DESCRIBE and READ without WRITE to restrict access to read-only for license topic ACLs. If a topic exists, the LicenseManager will not try to create the topic.

You can provide access either individually for each principal that will use the license or use a wildcard entry to allow all clients. The following examples show commands that you can use to configure ACLs for the resource cluster and _confluent-command topic.

  1. Set a CREATE and DESCRIBE ACL on the resource cluster:

    kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf \
--add --allow-principal User:<principal> \
--operation CREATE --operation DESCRIBE --cluster

  2. Set a DESCRIBE, READ, and WRITE ACL on the _confluent-command topic:

    kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf \
--add --allow-principal User:<principal> \
--operation DESCRIBE --operation READ --operation WRITE --topic _confluent-command