Data Stream = Kafka Topic + Data Governance |
Admin Client and Data Governance
Problems related to Schemas, Lineage,… cannot be solved by the Admin Client itself.
Problems related to Schemas, Lineage,… cannot be solved by the Admin Client itself.
How to set up Admin Client
Define Properties : bootstrap.servers=broker-1:8684
Create Admin Client : adminClient = new AdminClient<T>(properties)
Perform Action such as create topic = adminClient.createTopic(topic)
Deleting Records by Streaming Application
This action is more complex because specific records in topic have been incorrectly produced. Kafka does not store data like a database. So, we can’t delete individual records. We need to implement a filter pattern. |
Deleting Records by Offset
We can remove all records before a certain offset. With a single partition, it’s pretty simple, but with several partitions, it could become more complex.
If several partitions, we need to specify the partition as well.
If several partitions, we need to specify the partition as well.
Deleting Records by Key
