How NORD/LB transformed Kafka operations and cut debugging time in half

When Growth Outpaces Your Tools

For Norddeutsche Landesbank Girozentrale (NORD/LB), a leading German regional bank, Apache Kafka had become essential infrastructure. But as Erik Schumann, a member of the bank's central Kafka team, discovered, having the right streaming platform wasn't enough.

The bank runs their Kafka infrastructure on Confluent's Kubernetes operator, a relationship that continues today. Initially, they used Confluent Control Center for observability and management. But as their data architectures grew more complex and their compliance requirements tightened, they realized they needed purpose-built companion tooling. "In general, you definitely need to have a user interface for Kafka just for the developers, testers, and so on," Erik explains.

The Gap Between General Tools and Financial Services Requirements

As a financial institution with complex data structures and strict regulatory requirements, NORD/LB needed specialized tooling built for their use case. Three critical challenges were creating operational drag and compliance risk.

Search capabilities couldn't handle financial-scale complexity. With business partner IDs spanning 10 digits and financial statement schemas exceeding 10,000 lines of JSON, standard exact-match-only search meant developers needed complete identifiers to find anything. The impact on operations was significant. "We had some incidents in production where I had spent a lot of time figuring out what was wrong in the first place," Erik remembers. Teams resorted to manual workarounds: "We started copying the payload into Notepad and then we were trying to manually search, but it was just very messy."

Test data quality posed another critical challenge. The team discovered their tooling wasn't properly validating against schemas during test data production, allowing malformed data to enter their topics. "If you would produce a Kafka message, they weren't grabbing the schema, and then you get bad data into your Kafka topic," Erik explains. Without reliable test data, quality suffered. "When you don't have the options to properly test your data stream, you're not doing it, and by that you're delivering bad software."

Most critically, authorization controls needed to meet enterprise-grade requirements. As NORD/LB's Kafka adoption grew, they needed more granular separation between user types and finer control over production environment access. Erik explains the challenge: "We really had no separation between natural and technical users," which made it harder to enforce the principle of least privilege across their growing Kafka ecosystem.

For a bank navigating strict regulatory oversight, these weren't just operational inefficiencies. They were architectural gaps that needed addressing as they scaled.

Finding the Right Companion Product

In 2021, NORD/LB became one of Kpow's earliest adopters. After working with Confluent Control Center since their Kafka journey began, the team decided to find a specialized companion tool built for financial services observability and governance while keeping Confluent as their streaming platform.

"We do not want to rely on many different tools, and Factor House was just a good fit," Erik explains. Kpow's value proposition aligned perfectly with their gaps: advanced search for complex schemas, schema-aware data production, and enterprise authorization controls.

Kpow's search capabilities were built for the complexity NORD/LB faced. Teams could search by partial keys and filter through massive JSON structures. "It was pretty dope that we were just able to filter the exact point of it out," Erik says of working with those complex financial statement schemas.

The platform ensures schema validation during data production. "With Kpow, you aren't able to produce bad data because Kpow always fetches the schema and serializes it with the actual schema, and with that you don't get any issues," Erik explains.

Kpow's two-step authorization architecture solved their compliance problem. Users first authenticate into Kpow, then the system maps their permissions to technical users with appropriate access levels. This separation finally gave NORD/LB the control they needed: "In production, we really want to narrow what people can see, and with Kpow's two-step authorization, we could finally enforce that."

The platform also removed technical limitations. NORD/LB works with KSQL DB for stream processing but faced a 3,000-line query limit in their existing interface. "Kpow doesn’t have this limitation, so we just switched the complete development of KSQL DB queries to Kpow."

Halving Debugging Time While Solving Compliance Gaps

Today, Kpow serves three distinct user groups totaling more than 20 people: a five-person operations team handling monitoring and incidents, a 16-person central Kafka team building producers and managing the platform, and decentralized development teams across the bank.

Debugging time was cut in half. "We halved the time needed for debugging," Erik states simply. When bad data appears, teams can now identify root causes quickly and effectively. "So when it's bad data, we're very effective at figuring out why it's bad."

Software quality improved significantly. "The overall quality of software development improved significantly, and with that, the amount of incidents associated with bad Kafka schemas and so on, decreased," Erik reports.

The authorization problems that had posed serious risk were resolved. The bank now has proper separation between users, appropriate access controls for production environments, and audit trails that meet financial services requirements.

Beyond solving their initial challenges, teams discovered additional value. "We do like the visualization of the consumer lag," Erik notes. The platform's ACL management proved valuable for troubleshooting: "If you want to check if your SSL certificate is authorized to access a topic and so on, it's really nice to just have it displayed in Kpow."

NORD/LB currently has 25-30 business applications connected to Kafka, but they're planning significant expansion. Over the next few years, the bank expects to more than triple their Kafka-connected applications, bringing most of their critical systems onto the platform. That kind of scale-up demands confidence in your observability and governance tooling. With Kpow providing the foundation for debugging, testing, and compliance across their growing ecosystem, they're ready.

As the bank prepares to significantly scale their Kafka deployment, they're expanding with Factor House's platform evolution, planning to use upcoming data catalog features for compliance tags and interface documentation. For financial institutions facing similar challenges, NORD/LB's experience demonstrates how the right observability companion makes enterprise-scale growth possible.