Release 94.5: New Factor House docs, enhanced data inspection and URP & KRaft improvements

Introducing our new docs site

All Factor House product documentation has been migrated to a new, unified site. This new hub, Factor House Docs, provides a single, streamlined resource for all users.

Key improvements you'll find on the new site include:

• Unified product content: All documentation is now in one place with a simplified structure, consolidating what was previously separate community and enterprise docs.
• Clear feature availability: COMMUNITY, TEAM, and ENTERPRISE badges have been added to clearly indicate which features are available in each edition.
• Improved organization: Content is now grouped into more relevant sections, making it easier to find the information you need.
• Powerful search, instant answers: Instantly find any configuration, example, or guide with our new Algolia-powered, site-wide search.
• Hands-on playground: A new section featuring interactive labs and projects to help you explore product capabilities.
• Ready for the future: The documentation for the new Factor Platform will be added and expanded upon release, ensuring this hub remains the most up-to-date resource for all product information.

The documentation has been updated and is live at https://docs.factorhouse.io.

Data inspect enhancements

kJQ Projection expressions & search

• Comma-Separated kJQ Projection expressions: We've added support for comma-separated projection expressions (e.g., .value.base, .value.rates). This allows you to extract multiple fields from Kafka records in a single query, providing targeted data views without cluttering your output. This works for both key and value sub-paths.

• In-Browser Search (Ctrl + F): You can now use in-browser search (Ctrl + F) with kJQ filters to quickly find records by JSON path or value without re-running queries. The results component is now fully keyboard-friendly and follows the Listbox pattern, making it easier for everyone to navigate. Screen reader users can understand the list structure, and keyboard users can move through and select items smoothly and predictably.

Schema & deserialization insights

Data Inspect now provides detailed schema metadata for each message, including schema IDs and deserializer types. It also identifies misaligned schemas and poison messages, offering the following deserialization options:

• Drop record (default): Ignores erroneous records and displays only well-formatted records.
• Retain record: Includes both well-formatted and erroneous records. Instead of displaying the raw, poisonous value for problematic records, the system flags them with the message 'Deserialization exception'.
• Poison only: Displays only erroneous records, where the value is recorded as 'Deserialization exception'.

Sorting by Attribute

Selecting the 'Pretty printed (sorted)' display option sorts the attributes of the key or value alphabetically by name, improving readability and consistency during inspection.

High-performance streaming

Data Inspect can stream over 500,000 records smoothly without UI lag, enabling efficient analysis of large datasets.

kJQ improvements

Expanded kJQ capabilities with new transforms including parse-json, floor, ceil, upper-case, lower-case, trim, ltrim, rtrim, reverse, sort, unique, first, last, keys, values, is-empty, and flatten.

Also added new functions: within, split, and join, enabling richer data manipulation directly within kJQ queries.

For more details on these new features, please refer to the updated kJQ manual. Also, be sure to visit the new interactive examples page on our new Factor House docs site—it's a great way to quickly verify your kJQ queries.

Consumer group management

Empty group member assignments

Previously, EMPTY consumer groups showed no offset information in the reset offset UI, preventing customers from resetting their offsets. This was a critical issue when a poison message caused an entire consumer group to go offline. The fix now fetches offsets directly from the AdminClient instead of relying on the snapshot, ensuring offsets can be reset in these scenarios.

Cluster & platform enhancements

Improved Under-Replicated Partition (URP) Detection

We've enhanced our calculation for under-replicated partitions to provide more accurate health monitoring for your Kafka clusters. The system now correctly detects partitions with fewer in-sync replicas than the configured replication factor, even when brokers are offline and not reported by the AdminClient.

You can find URP details on the Brokers and Topics pages. The summary statistics will display the total number of under-replicated partitions. If this count is greater than zero, a new table will appear with details on all applicable topics.

Brokers

Topics

To further strengthen monitoring and alerting, new Prometheus metrics have been introduced to track under-replicated partitions. These metrics integrate seamlessly with your existing observability stack and provide more granular insights:

• broker_urp: The number of under-replicated topic partitions on this broker.
• topic_urp: The number of under-replicated partitions for this specific topic.
• topic_urp_total: The total number of under-replicated partitions across all topics in the Kafka cluster.

KRaft improvements and fixes

The following improvements and bug fixes have been made to KRaft:

• Polished KRaft tables with improved sorting and corrected display of the End Offset (now showing timestamps instead of offsets).
• Fixed an issue where the controller broker was incorrectly displayed in broker details.
• Added new KRaft-related metrics to Prometheus for enhanced observability.
• Introduced new KRaft-specific Prometheus metrics to strengthen observability:
  • kraft_high_watermark: The high watermark of the metadata log in the KRaft cluster, indicating the highest committed offset.
  • kraft_leader_epoch: The current leader epoch in the KRaft cluster, incremented each time a new leader is elected.
  • kraft_leader_id: The broker ID of the current leader in the KRaft cluster responsible for handling metadata operations.
  • kraft_observer_count: The number of observer replicas in the KRaft cluster. Observers replicate the metadata log but do not participate in leader election.
  • kraft_replicas_count: The total number of replicas (voters + observers) in the KRaft cluster responsible for maintaining the metadata log.
  • kraft_voter_count: The number of voting replicas in the KRaft cluster. Voters participate in leader election and maintain the metadata log.

Custom SerDes per cluster

Kpow supports to configure custom SerDes on a per-cluster basis, providing the flexibility to handle different data formats and compatibility requirements across your Kafka environments. This approach ensures smoother integration with diverse data pipelines, reduces serialization errors, and improves overall system reliability.

Here is an example of a per-cluster custom SerDes configuration:

serdes:
  - name: "PROTO 1"
    format: "json"
    isKey: true
    config:
      bootstrap: "some-value"
      limit: 22
      display: another-value
      abc: $SOME_ENV
  - name: "PROTO 2"
    cluster: "Trade Book (Staging)" # THIS IS A NEW KEY
    format: "json"
    isKey: false
    config:
      bootstrap: "some-value"
      limit: "100"
      display: another-value
      abc: $ANOTHER_ENV

Bug fixes

This release addresses several key issues improving UI stability, integrations, and navigation consistency across the platform.

• Resolved an edge case in the temporary policy UI display
• Fixed Microsoft Teams webhook integration
• Fixed a buggy textarea inside the Data Masking Playground.
• Fixed regression affecting ordering of ksqlDB, Schema Registry, and Connect resources in navigation dropdown
• Fixed ordering of Kafka resources. Examples:
  • CONNECT_RESOURCE_IDS=QA1,DEV1,DEV2 → showing in order: QA1, DEV1, DEV2
  • SCHEMA_REGISTRY_RESOURCE_IDS=QA1,DEV1,DEV2 → showing in order: QA1, DEV1, DEV2
  • KSQLDB_RESOURCE_IDS=QA1,DEV1,DEV2 → showing in order: QA1, DEV1, DEV2

Updated Help menu

Introduced a redesigned Help menu featuring an improved What's New section for quick access to the latest product updates. The menu also now includes direct links to join our Slack community, making it easier to connect with other users, share feedback, and get support right from within the product.