Beyond the CLI: Operating Kafka at Enterprise Scale

As Kafka usage grows within an organisation, the tooling teams rely on day-to-day often doesn't scale with it. CLI-based workflows that work at small scale tend to become slower and less reliable as the number of clusters, teams, and topics increases. Visibility into consumer behaviour requires running commands that capture a single point in time. Governance depends on manual processes. Access control becomes harder to audit and delegate consistently.

This workshop is designed for platform and engineering teams operating Kafka in production environments. It covers five practical exercises, each focused on a specific operational challenge:

• Tracing and resolving a poison pill message blocking a partition, using staged mutations to skip the bad offset without redeploying
• Configuring webhook integration to capture state-changing administrative actions and route them to Slack in real time
• Enforcing role-based access control and tenant isolation across different user personas, from read-only inspection through to staged operations requiring approval
• Deploying and managing Kafka Connect pipelines via both the Kpow UI and API, and verifying data flow end-to-end
• Exploring Kpow's Prometheus integration and how it surfaces business-level metrics for topics, consumer groups, and Connect clusters in Grafana

The session is suited to platform engineers, data engineers, and DevOps practitioners who manage Kafka infrastructure for other teams and are looking for more consistent, auditable ways to handle day-to-day operations.

This is a hands-on workshop, so please bring your laptop. Morning tea will be provided.