How significantly to decrease the throughput for Kafka Producer

Safety vs Throughput : Setting MaxInFlight to 1 can significantly decrease the throughput. So, we can guarantee that Kafka will preserve message order on the event that some messages will require multiple retries before they are successfully acknowledged.

var producerConfig = new ProducerConfig
{
      Acks = Acks.All,
      BootstrapServers = "localhost:9092",
      BatchSize = 32768,
      LingerMs = 20,
      CompressionType = CompressionType.Snappy,
      MaxInFlight = 1
};

Maximum number of in-flight requests per broker connection. This is a generic property applied to all broker communication, however it is primarily relevant to produce requests.

Handling Streaming Data with Messaging Systems ( Kafka & Azure Event Hub)

How significantly to decrease the throughput for Kafka Producer

Analytics

Related content