Kafka Connect TopicRegexRouter SMT for Confluent Cloud

The following provides usage information for the Confluent SMT io.confluent.connect.cloud.transforms.TopicRegexRouter.

Important

TopicRegexRouter is only available for managed Source connectors. For the equivalent SMT for self-managed connectors, see RegexRouter.

Description

Update the record’s topic using the configured Re2j regular expression and replacement string.

Under the hood, the regex is compiled to a com.google.re2j.Pattern. If the pattern matches the input topic, com.google.re2j.Matcher#replaceFirst() is used with the replacement string to obtain the new topic.

Installation

This transformation is developed by Confluent and does not ship by default with Apache Kafka® or Confluent Cloud. You can install this transformation using the confluent connect plugin install command:

confluent connect plugin install confluentinc/connect-transforms:latest

Examples

The following examples show how to configure and use TopicRegexRouter.

Tip

You enclose part of the regular expression (regex) with parentheses to capture this part of the regex in the replacement string. In the replacement string, the enclosed part is represented by a variable $n. The $n variable used for each captured part is $1 for the first part, $2 for the second part, and so on. The variable $0 is used to capture the entire matched string.

Remove a topic prefix

This configuration snippet shows how to remove the prefix soe- from the beginning of a topic.

"transforms": "dropPrefix",
"transforms.dropPrefix.type": "io.confluent.connect.cloud.transforms.TopicRegexRouter",
"transforms.dropPrefix.regex": "soe-(.*)",
"transforms.dropPrefix.replacement": "$1"

Before: soe-Order

After: Order

Add a topic prefix

This configuration snippet shows how to add the prefix acme_ to the beginning of a topic.

"transforms": "AddPrefix",
"transforms.AddPrefix.type": "io.confluent.connect.cloud.transforms.TopicRegexRouter",
"transforms.AddPrefix.regex": ".*",
"transforms.AddPrefix.replacement": "acme_$0"

Before: Order

After: acme_Order

Remove part of a topic name

This configuration snippet shows how to remove a string in a topic name.

"transforms": "RemoveString",
"transforms.RemoveString.type": "io.confluent.connect.cloud.transforms.TopicRegexRouter",
"transforms.RemoveString.regex": "(.)Stream_(.)",
"transforms.RemoveString.replacement": "$1$2"

Before: Order_Stream_Data

After: Order_Data

Properties

Name	Description	Type	Default	Valid Values	Importance
regex	Regular expression to use for matching.	string		valid Re2j regex	high
replacement	Replacement string.	string			high

Predicates

Transformations can be configured with predicates so that the transformation is applied only to records which satisfy a condition. You can use predicates in a transformation chain and, when combined with the Kafka Connect Filter (Kafka) SMT for Confluent Cloud, predicates can conditionally filter out specific records. For details and examples, see Predicates.