Factor House release v93.2 brings a new minor version to our suite of products for Apache Kafka and Apache Flink.
This is the 117th release of Factor House products, and marks more than 2.5M downloads of Kpow and Flex from Dockerhub in the past five years.
The major new feature in v93.2 is the connector auto-restart which provides a hassle-free way for organizations to make sure that their connectors are always up and running.
This release also includes:
- Improve ksqlDB UI: import sql files, better error handling and timing metrics
- Allow RBAC wildcards at both start and end of resource, such as *foo*
- Introduce download capabilities in krepl
- Preserve the last visited page when navigating between resources
- Add keybindings table to data inspect + ksqldb forms
- Fix bug starting Kpow with PERSISTENCE_MODE=audit
- Improve mutation notification UX
- Fix bug with tenancy resource handling for fine-grained Schema resources
Connector Auto-restart
Kpow has supported the ability to restart connectors including bulk-restart since its inception, but we now offers the ability to automatically restart specified connectors and failing tasks, providing enhanced reliability and uptime for your data integration workflows.
To enable this feature, simply set the CONNECT_AUTO_RESTART environment variable in your configuration.
When activated, Kpow will monitor for failed connectors at one-minute intervals. After attempting to restart a connector, Kpow will wait for the user-configured amount of time before trying again. By default, this interval is set to 10 minutes, but you can adjust it using the CONNECT_AUTO_RESTART_WINDOW_MS environment variable.
Additionally, Kpow allows you to specify a limit on the number of connectors that can be restarted automatically. The default limit is 50 connectors, but you can modify this setting using the CONNECT_AUTO_RESTART_LIMIT environment variable. This feature helps prevent excessive restart requests being sent to the Kafka Connect cluster.
All restart attempts will be logged in the audit log as actions performed by the kpow_system user.
If you have configured Kpow's Slack integration, all restart attempts will also be sent to a Slack webhook. This integration enhances real-time monitoring and alerting capabilities, ensuring that your team is promptly notified of any restart actions taken by Kpow.
Note: If the connector is failing more frequently than CONNECT_AUTO_RESTART_WINDOW_MS limit then it may require manual intervention.
Example configuration
CONNECT_AUTO_RESTART="*" # restart all failed connectors CONNECT_AUTO_RESTART="dbz-connector-1" # restart **only** the connector dbz-connector-1 when it enters a failed state CONNECT_AUTO_RESTART="mysql-prod-us*" # restart any connector that matches the wildcard (mysql-prod-us-east1, mysql-prod-us-west2, etc) CONNECT_AUTO_RESTART="payments-*,*-stage" # you can specify many filters by providing a comma-separated list
ksqlDB UI improvements
ksqlDB UI has been streamlined. Users can now import a file into the query editor by simply clicking at Load SQL from file. Error messages has been improved. Also, you can now see metrics about timing.
