33,431
questions
0
votes
0
answers
9
views
DefaultAfterRollbackProcessor without BackOff for batch listener
I'm trying to configure the error handling for a batch listener, such that if a single record fails to be processed, the entire batch is sent to the dead letter topic (with some additional logging). I'...
- 13
1
vote
0
answers
15
views
Kafka Debezium Oracle Source Connector Intermittent Lag & Data Delay
I am working with Kafka, and we are using Debezium to capture CDC (Change Data Capture) data from an Oracle source database. Below are my Debezium configuration parameters:
{
"name": &...
- 125
0
votes
1
answer
12
views
Kafka KRaft Mode - Access Denied Exception When Deleting Snapshot Files
I'm facing an issue with Kafka running in KRaft mode (without Zookeeper) on a Windows environment, and I keep getting an AccessDeniedException in the logs when Kafka attempts to delete specific ...
0
votes
0
answers
11
views
Kafka message compression not working (broker level)
I added compression for log messages on broker level compression.type=zstd on all three brokers with no other changes on the broker level or producer level.
When I tried to read messages from my ...
- 555
0
votes
1
answer
13
views
Count mismatch in AKHQ UI 0.24.0
I am producing 10 records to kafka topic and able to consume all 10 but in AKHQ UI I am seeing count as 12 . If I count the values manually its 10 only (same as what's produced).
PFA
Tried changing ...
- 13
0
votes
0
answers
11
views
Kafka consumer seek() losses some data in consmer.pool() with single consumer assign to multiple partitions
I am running one consumer and assigned 4 partitions(0,1,2,3) to it.
consumer.assign(0,1,2,3)
I have send A1,B2,C3,D4 data on partition 0,1,2,3 on the topic consumer is running.
Now, i pool the data ...
0
votes
0
answers
16
views
Kafka Streams: Persisting RocksDB state on shared disk across multiple instances
I have a Kafka Streams application running as a StatefulSet in a Kubernetes cluster. Each instance uses 10 threads. Due to the volatile nature of the data load, I use a scaling mechanism to ...
- 1,661
0
votes
1
answer
21
views
Debezium KafkaConnect, kafka_connect_run.sh No such file or directory
I am getting the error when attempting to setup Debezium in kubernetes (minikube for now) using Strimzi. What is wrong with my manifest? I may be confusing how to use the debezium connect image. I am ...
- 113
0
votes
0
answers
11
views
Kafka Streams - State Store changelog topic deleteion
I have some corrupted keys in a state store. I dont want to reset the whole application. Can I simply delete and recreate the changelong topic in order to purge the in memory state store. Is this safe?...
0
votes
0
answers
13
views
How to list most used topics
I want to list the most used topics by input/output of messages. How can I do this? I'm trying with sum delta but it isn't what I want
sum(delta(kafka_consumergroup_current_offset{})) by (topic, ...
1
vote
0
answers
16
views
ksqldb explode multiple list elements
I have data like this:
One Kafka Message:
[{"source": "858256_6052+571", "numericValue": null, "created": 1725969039288, "textValue": "mytestData&...
- 63
-1
votes
1
answer
21
views
Unable to sent records to Db from kafka topic using Microsoft sql server sink connector
Summary: I need to send records to kafka topic and using microsoft sql server sink connector have to fetch the record from topic and send to database.
**
Approaches i tried for sending records to ...
0
votes
0
answers
15
views
Nifi send metadata to Kafka topic
The task is to send some metadata to kafka topic in Nifi-pipeline.
For example, Java ProducerRecord api allow to send custom headers, which may be used as metadata container.
But in Nifi ...
- 1,210
1
vote
1
answer
18
views
Kraft inter-broker communication over SSL
I'm wondering if I'm missing something about configuring SSL for Kraft.
We currently have a Zookeeper cluster where all inter-broker & client communication uses SSL. We manage our own CA and ...
- 19
0
votes
0
answers
24
views
Kafka Consumer fails to rejoin its group in a single Kafka broker
I've been using Apache Kafka 3.3.2 for a while now as a single broker and not a cluster (I know, this started as a testing stuff) and now I bumped into some consumer error which I couldn't solve by ...
- 355