rust-rdkafka v0.28.0 Release Notes

Release Date: 2021-11-27 // 6 months ago
    • Add the StreamConsumer::split_partition_queue method to mirror BaseConsumer::split_partition_queue (#411).

    Thanks to @davidblewett, @duarten, and @nemosupremo for contributing to the implementation.

    • ๐Ÿ’ฅ Breaking change. Remove the StreamConsumerContext type and the ConsumerContext::message_queue_nonempty_callback method. These were essentially implementation details of StreamConsumer that had leaked into the public API. The vast majority of users should be unaffected.

    • ๐Ÿ’ฅ Breaking change. Remove the type parameters from the MessageStream type.

    • ๐Ÿ’ฅ Breaking change. Add the received TopicPartitionList to the Rebalance::Revoke variant, which is useful when using incremental cooperative rebalancing (#398).

    • Avoid crashing if librdkafka invokes the commit callback with a null topic partition list (#406).

    Thanks, @thijsc.

    • โž• Add the new statistics fields in librdkafka v1.7.0 to the various statistics types. The new fields are:

      • Partition::consumer_lag_stored
      • Broker::txidle
      • Broker::rxidle
      • Statistics::age
    • ๐Ÿ’ฅ Breaking change. Change the type of the following statistics fields from i64 to u64 to reflect the signedness of the upstream types:

      • Statistics::msg_cnt
      • Statistics::msg_size
      • Statistics::msg_max
      • Statistics::msg_size_max
      • Broker::tx
      • Broker::txbytes
      • Broker::txretries
      • Broker::req_timeouts
      • Broker::rx
      • Broker::rxbytes
      • Broker::rxerrs
      • Broker::rxcorriderrs
      • Broker::rxpartial
      • Broker::zbuf_grow
      • Broker::buf_grow
      • Broker::wakeups
      • Broker::msgq_bytes
      • Broker::xmit_msgq_bytes
      • Broker::fetchq_size
      • Partition::txmsgs
      • Partition::txbytes
      • Partition::rxmsgs
      • Partition::rxbytes
      • Partition::msgs
      • Partition::rx_ver_drops
      • Partition::acked_msgid
    • โž• Add the ClientContext::stats_raw method to consume the JSON-encoded statistics from librdkafka. The default implementation calls ClientContext::stats with the decoded statistics.

    • โž• Add the Default trait to the statistics types: Statistics, Broker, Window, TopicPartition, Topic, Partition, ConsumerGroup, and ExactlyOnceSemantics (#410).

    Thanks, @scanterog.

    • โž• Add the Debug trait to DefaultClientContext and DefaultConsumerContext (#401).

    Thanks, @DXist.

Previous changes from v0.27.0

    • ๐Ÿ‘ Allow offset 0 in Offset::to_raw.

    Thanks, [@roignpar].

    • ๐Ÿ›  Fix a segfault when calling Consumer::position on a consumer that was improperly configured (#360).

    • ๐Ÿ›ฐ Provide a mutable accessor (Message::payload_mut) for a message's payload (#95).

    • Implement std::iter::Extend<(String, String)> and std::iter::FromIterator<(String, String) for ClientConfig (#367).

    Thanks, @djKooks.

    • ๐Ÿ’ฅ Breaking change. Change Consumer::store_offset to accept the topic, partition, and offset directly (#89, #368). The old API, which took a BorrowedMessage, is still accessible as Consumer::store_offset_from_message.

    • ๐Ÿ‘Œ Support incremental cooperative rebalancing (#364). There are two changes of note:

      • The addition of Consumer::rebalance_protocol to determine the rebalance protocol in use.
      • The modification of the default rebalance callback (ConsumerContext::rebalance) to perform incremental assignments and unassignments when the rebalance protocol in use is [RebalanceProtocol::Cooperative].

    Thanks, @SreeniIO.

    • ๐Ÿ‘Œ Support reading and writing commit metadata via TopicPartitionListElem::metadata and TopicPartitionListElem::set_metadata, respectively (#391).

    Thanks, @phaazon.