How to diagnose and fix a failed Kafka Connect connector
KafkaA failed Kafka Connect connector usually announces itself one of two ways: an alert on failed tasks, or a downstream team asking why their table stopped updating while the Connect UI still says RUNNING. Both have the same starting point. A connector has failed when the connector instance or one or more of its tasks has stopped processing records and entered the FAILED state, and Connect will not bring it back on its own.
The distinction that matters most in the first five minutes is between the connector and its tasks. The connector instance only coordinates, and the tasks move the data. A connector can report RUNNING with every one of its tasks FAILED, so a status check that reads only the top-level state tells you nothing. This page walks the sequence: read the status, read the trace, classify the failure, and only then decide whether a restart can help. It sits under the Kafka Connect topic page, which covers what a Connect deployment is made of.
Check connector and task status through the REST API
Every Connect worker exposes a REST API, and the status endpoint answers the first question on its own: the connector’s state, which worker runs it, and the state of every task, with the error attached to anything that failed. The call is:
curl -s http://connect-worker:8083/connectors/orders-sink/status | jq{
"name": "orders-sink",
"connector": { "state": "RUNNING", "worker_id": "10.0.1.12:8083" },
"tasks": [
{ "id": 0, "state": "RUNNING", "worker_id": "10.0.1.12:8083" },
{ "id": 1, "state": "FAILED", "worker_id": "10.0.1.14:8083",
"trace": "org.apache.kafka.connect.errors.ConnectException: ..." }
],
"type": "sink"
}The endpoint is documented in the Connect REST API reference as returning whether the connector “is running, failed, paused, etc., which worker it is assigned to, error information if it has failed, and the state of all its tasks”. A connector can be RUNNING, PAUSED, STOPPED, FAILED, RESTARTING or UNASSIGNED, and a task can be in any of those except STOPPED (Connect monitoring reference). UNASSIGNED is worth knowing: UNASSIGNED means no worker has picked the task up yet, which points at the Connect cluster (a worker that left, a rebalance in progress) rather than at the connector.
With more than a handful of connectors, list them all with their status in one call and filter for anything that is not healthy:
curl -s "http://connect-worker:8083/connectors?expand=status" \
| jq -r '.[].status
| select(.connector.state != "RUNNING" or any(.tasks[]; .state != "RUNNING"))
| [.name, .connector.state, ([.tasks[].state] | join(","))] | @tsv'Two things come out of this step: which tasks failed, and which worker each one was running on. You need the second one for the logs.
Find and read the stack trace
The trace field on a failed task is the full Java stack trace as a single string. Read it from the bottom up. The top line is usually a generic wrapper such as ConnectException or “Tolerance exceeded in error handler”, and the useful sentence is the last Caused by: in the chain. That line names the real failure: a refused connection, an authentication error, a schema or converter error, a constraint violation in the target database, or an exception from the target system’s own API.
When the trace is truncated or does not explain itself, go to the logs of the worker named in worker_id. Connect prefixes each log line with the connector and task it came from, a context added by KIP-449 and included in the default Connect log pattern, so a grep for the connector name pulls out one task’s history, including the retries and warnings in the minutes before it failed:
grep 'orders-sink|task-1' /var/log/kafka/connect.log | tail -200The minutes before the failure often matter more than the final exception. A task that logged ten timeouts and then died has a different problem from one that died on its first record.
Decide what kind of failure it is
Failed connectors fall into five groups, and the group decides the fix. The Caused by: line usually settles it.
Configuration and credentials. Wrong hostnames, expired passwords, missing permissions on the target table or bucket. The trace names the authentication or authorization error. These fail on start, fail again on every restart, and are fixed in the connector config.
Conversion and transformation. The converter could not read a record (the same failures as a Kafka deserialization error, seen from inside Connect) or a single message transform threw. This is the only group the connector’s own error-handling settings can absorb, which the next section covers.
Connector or plugin faults. A bug in the connector code, a plugin version that does not match the worker, a class conflict on the plugin path. The trace points inside the connector’s own packages. Restarts sometimes clear it for a while.
The Connect cluster. Tasks in UNASSIGNED, tasks that move between workers, workers that fail their health checks. The problem is the worker group, and restarting a connector moves it without fixing it.
Limits in the external system. The database is down or refusing connections, the target API does not support an operation the connector needs, a rate limit or quota is exhausted. The trace names the external system’s error. No action inside Kafka Connect fixes these, and a restart fails in exactly the same way.
The last group is the one that burns the most time, because a restart looks like progress. The next sections come back to it with a real example.
errors.tolerance and the dead letter queue
Connect has built-in error handling, but its scope is narrower than most people assume. The Apache documentation states the default plainly: “By default, any error encountered during conversion or within transformations will cause the connector to fail.” The error-handling properties change what happens to errors in the converter and in single message transforms. Beyond that, a sink connector can report a record that failed on write to the same dead letter topic, but only if the connector’s own code calls the errant record reporter added in KIP-610. Source connectors have no equivalent, and a failure outside a single record, such as the source refusing a connection or the target being down, still fails the task.
The settings that matter are:
errors.tolerance:none(the default) fails the task on the first bad record, andallskips problem records.errors.deadletterqueue.topic.name: for sink connectors, the topic that skipped records are written to.errors.deadletterqueue.context.headers.enable: adds headers describing the failure (the exception, and the original topic, partition and offset) to each dead-lettered record.errors.log.enableanderrors.log.include.messages: log each tolerated error, optionally with the record content.errors.retry.timeout: how long Connect retries a failed operation before giving up.
The properties and their defaults are in the Kafka Connect configuration reference. Two warnings go with these settings. First, errors.tolerance=all without a dead letter topic and without logging silently drops records, which turns a visible failure into missing data. Second, a DLQ that nobody reads is the same thing with extra steps. Whatever consumes the dead letter topic also needs a stop condition, because a job that writes its own failures back to the topic it reads from loops forever. Our guide to dead letter queues in Kafka covers the Connect DLQ alongside the consumer-side options, and Kafka DLQ tools compares the tools for inspecting and replaying a dead letter topic.
Restart semantics: what a restart does, and what it cannot do
Restarting is the right move for a transient fault, and the REST API is more precise about it than many tools expose. The older form of the call restarts only the connector instance, which does not restart failed tasks. Since KIP-745, which shipped in Kafka 3.0, one call can restart the connector and just its failed tasks:
curl -s -X POST \
"http://connect-worker:8083/connectors/orders-sink/restart?includeTasks=true&onlyFailed=true"Both parameters default to false, which keeps the old behaviour of restarting the connector instance alone. A single task restarts with:
curl -s -X POST http://connect-worker:8083/connectors/orders-sink/tasks/1/restartApache Kafka Connect does not restart failed tasks automatically. A task that fails stays FAILED until something or someone restarts it, which is why most teams end up with a script, an operator or a tool that does it for them.
A restart can only fix a failure whose cause went away. If the cause is still there, the task fails again, usually within seconds, and an automated restart loop turns one failure into a steady stream of them. So before restarting, answer one question from the trace: has the thing that failed changed since it failed? If the answer is no, a restart only confirms the failure.
When restarting will not help: three connectors down
I ran into a clean example of this while recording a demo on 11 September 2026. Three connectors in the demo environment were not moving data. The diagnosis, made from the task traces, split them into two external-system problems and no connector problems at all.
Two of the three were Iceberg sink connectors I had just deployed against a Databricks Unity Catalog. Both tasks were exiting on an unrecoverable exception. The trace showed the connector calling a catalog endpoint to create the destination Iceberg table, and the catalog, at the time of the demo, rejecting that call because Unity Catalog did not support that endpoint. The connector was doing what it was configured to do. Restarting it would have failed identically, because nothing about the catalog had changed. The fix was on the catalog side of the boundary: create the tables by hand in Unity Catalog and turn off table auto-creation in the connector. In the Apache Iceberg sink connector, for example, that setting is iceberg.tables.auto-create-enabled, which defaults to false (Iceberg Kafka Connect documentation). With the tables in place, the connectors start.
The third was a Debezium Postgres source connector that could not obtain a connection. The trace said connection refused, which meant the Postgres instance was down and refusing TCP connections. Again a restart could not help, and the fix belonged to whoever owned the database.
The lesson I took from it: the useful output of a Connect diagnosis is a sentence of the form “this is a catalog issue, not a connector issue”, because it tells you who fixes it and stops anyone restarting the connector in a loop in the meantime. Three connectors in the same failed state had two different owners, neither of them the Connect team.
The diagnosis in that demo was produced by an AI assistant working through a Kpow command-line interface and agent skills that I was previewing. That CLI, terminal UI and the agent skills are a preview and are not released at the time of writing, and the reasoning above does not depend on them: it is the same read of the task traces you can do with the REST calls earlier on this page. The recording is on the Kpow CLI, terminal UI and agentic skills demo page.
Once you are past the incident and choosing what should watch your connectors in future, the comparison of Kafka Connect monitoring tools scores the options on exactly these failure modes: task state, restarts and multi-cluster visibility. If the failing connectors are MirrorMaker 2 running on Connect, the MirrorMaker 2 on Connect migration guide covers the Connect-specific catches for that workload.
How Factor House approaches it
Kpow puts the status endpoint, the trace and the restart call behind one screen, across every Connect cluster it is configured for. From the Connect view you can see each connector’s state next to each task’s state, open the stack trace of any task in an error state, restart an individual task, and pause, restart, stop or delete a connector (Kpow docs, Kafka Connect management). Sensitive config values stay redacted when you view or edit a connector’s config.
For the transient faults where a restart is the right answer, Kpow can do the restarting. Auto-restart is enabled per connector name or wildcard pattern with CONNECT_AUTO_RESTART. Kpow checks for failed connectors at one-minute intervals, waits 10 minutes between attempts on the same connector by default, and caps restarts at 50 connectors per interval, so a fleet-wide outage does not become a restart storm against the Connect cluster (Kpow docs, Kafka Connect configuration). Every automatic restart is recorded in the audit log as the kpow_system user and can be posted to Slack. The same docs note that a connector failing more often than the restart window “may require manual intervention”, which is the external-system case from the example above.
Access is controlled per action with RBAC: CONNECT_ALTER_STATE covers pause, stop, resume and restart, separately from CONNECT_EDIT_CONFIG and CONNECT_DELETE, so an application team can restart its own connectors without being able to change or delete anyone else’s. Multiple Connect clusters per Kafka cluster are an Enterprise feature, and Kpow also works with Amazon MSK Connect and Confluent Cloud managed connectors.
The test worth running in a Kpow demo is the sequence on this page: find a connector whose tasks are not all RUNNING, open the failed task’s stack trace, and restart only that task, then compare how long that took with the curl-and-grep version above.
Product demo · 2 min
Apache Kafka Connect monitoring & task management: Kpow demo
Chad Harris walks through monitoring and managing Kafka Connect in Kpow: connector and task state at a glance, historical health charts, deploying new connector instances from the UI, and filtering to bulk-restart a subset of tasks.
Kpow live demo
See connector and task state live
Open the Kpow demo to check connector and task status side by side, the first split this page asks you to make.
Built for platform and data engineers running Kafka in production.
Try the Kpow demoFAQ
Why does my Kafka connector show RUNNING when no data is flowing?
The connector state only describes the connector instance, which coordinates. The tasks move the data, and they can all be FAILED while the connector is RUNNING. Check tasks[].state in the status response, not only connector.state.
Does Kafka Connect restart failed tasks automatically?
No. A failed task stays FAILED until it is restarted through the REST API, by an operator such as Strimzi, or by a management tool. Restart with POST /connectors/<name>/restart?includeTasks=true&onlyFailed=true to restart only what failed.
Where do I find the error for a failed connector?
In the trace field of the failed task in GET /connectors/<name>/status, and in the log of the worker named in that task’s worker_id. Read the last Caused by: line of the trace first.
Will errors.tolerance=all stop my connector failing?
Only for errors in conversion and transformations, plus write failures in sink connectors that implement the KIP-610 errant record reporter. Connection failures and outages in the external system still fail the task. Pair errors.tolerance=all with a dead letter topic and error logging, or failed records disappear without a trace.
More on how Connect fits into a streaming platform is in the complete Kafka guide.