rust-rdkafka v0.29.0 Release Notes

Release Date: 2022-10-29 // 3 months ago
    • πŸ’₯ Breaking change. Pass through errors from librdkafka in BaseProducer::flush, StreamProducer::flush, and FutureProducer::flush.

    Thanks, @cjubb39.

    • πŸ’₯ Breaking change. Only provide NaiveRuntime if the naive-runtime feature is enabled. This elides a dependency on futures-executor when the naive-runtime feature is disabled.

    • πŸ’₯ Breaking change. Remove the deprecated StreamConsumer::start method. Use the more clearly named StreamConsumer::stream method instead.

    • πŸ’₯ Breaking change. Rework the Headers trait to distinguish between headers with null values and headers with empty values. The Headers::get and Headers::get_as methods now return a Header struct with the following definition:

      pub struct Header<'a, V> {
          pub key: &'a str,
          pub value: Option<V>,

    Previously, these methods operated in terms of key–value pair (&str, &[u8]).

    These methods now panic if presented with an out-of-bounds index. This simplifies their usage in the common case where the index is known to be valid. Use the new Headers::try_get and Headers::try_get_as methods if you need the old behavior of returning None if the index is invalid.

    • πŸ“‡ Rename the OwnedHeader::add method to OwnedHeader::insert, for parity with the equivalent method for the map types in std::collection and to avoid confusion with the add method of the std::ops::Add trait. The method now takes the Header type mentioned above as an argument, rather than the key and value as separate arguments.

    • βž• Add the Headers::iter method to iterate over all message headers in order.

    • Add the PartitionQueue::set_nonempty_callback method to register a callback for a specific partition queue that will run when that queue becomes nonempty. This is a more flexible replacement for the ConsumerContext::message_queue_nonempty_callback method that was removed in the last release.

    • In BaseConsumer::rebalance_protocol and StreamConsumer::rebalance_protocol, handle null return values from the underlying librdkakfa API (#417). This avoids an occasional segfault in the rebalance callback.

    Thanks, @bruceg.

    • βž• Add a tracing feature which, when enabled, emits log messages using the tracing crate rather than the log crate.

    • βž• Add support for the OAUTHBEARER SASL authentication mechanism via the new ClientContext::ENABLE_REFRESH_OAUTH_TOKEN constant and the new ClientContext::generate_oauth_token method.

    Thanks, @jsurany-bloomberg.

Previous changes from v0.28.0

    • Add the StreamConsumer::split_partition_queue method to mirror BaseConsumer::split_partition_queue (#411).

    Thanks to @davidblewett, @duarten, and @nemosupremo for contributing to the implementation.

    • πŸ’₯ Breaking change. Remove the StreamConsumerContext type and the ConsumerContext::message_queue_nonempty_callback method. These were essentially implementation details of StreamConsumer that had leaked into the public API. The vast majority of users should be unaffected.

    • πŸ’₯ Breaking change. Remove the type parameters from the MessageStream type.

    • πŸ’₯ Breaking change. Add the received TopicPartitionList to the Rebalance::Revoke variant, which is useful when using incremental cooperative rebalancing (#398).

    • Avoid crashing if librdkafka invokes the commit callback with a null topic partition list (#406).

    Thanks, @thijsc.

    • βž• Add the new statistics fields in librdkafka v1.7.0 to the various statistics types. The new fields are:

      • Partition::consumer_lag_stored
      • Broker::txidle
      • Broker::rxidle
      • Statistics::age
    • πŸ’₯ Breaking change. Change the type of the following statistics fields from i64 to u64 to reflect the signedness of the upstream types:

      • Statistics::msg_cnt
      • Statistics::msg_size
      • Statistics::msg_max
      • Statistics::msg_size_max
      • Broker::tx
      • Broker::txbytes
      • Broker::txretries
      • Broker::req_timeouts
      • Broker::rx
      • Broker::rxbytes
      • Broker::rxerrs
      • Broker::rxcorriderrs
      • Broker::rxpartial
      • Broker::zbuf_grow
      • Broker::buf_grow
      • Broker::wakeups
      • Broker::msgq_bytes
      • Broker::xmit_msgq_bytes
      • Broker::fetchq_size
      • Partition::txmsgs
      • Partition::txbytes
      • Partition::rxmsgs
      • Partition::rxbytes
      • Partition::msgs
      • Partition::rx_ver_drops
      • Partition::acked_msgid
    • βž• Add the ClientContext::stats_raw method to consume the JSON-encoded statistics from librdkafka. The default implementation calls ClientContext::stats with the decoded statistics.

    • βž• Add the Default trait to the statistics types: Statistics, Broker, Window, TopicPartition, Topic, Partition, ConsumerGroup, and ExactlyOnceSemantics (#410).

    Thanks, @scanterog.

    • βž• Add the Debug trait to DefaultClientContext and DefaultConsumerContext (#401).

    Thanks, @DXist.