Skip to content

Conversation

@zheguang
Copy link
Contributor

@zheguang zheguang commented Oct 23, 2025

This patch clarifies SinkTask.precommit, which may see revoked partitions during rebalancing and stopping.

As pointed out in KAFKA-19685, there is a difference in whether SinkTask.precommit sees revoked partitions during iteration or stopping.

This difference appears to be intentional for correctness w.r.t. rebalance

  • During iteration, partition ownership may change due to rebalanced, so revoked partitions should be cleaned up at each poll, to avoid committing offsets for partitions no longer owned
  • During stopping, all current offsets are committed regardless of partition ownership to record the final state. The task is ending its lifecycle and no new records will be processed.

@github-actions github-actions bot added triage PRs from the community connect small Small PRs labels Oct 23, 2025
@zheguang zheguang changed the title Add possibility of revoked partitions in task precommit Add possibility of passing in revoked partitions in task precommit Oct 23, 2025
@zheguang zheguang changed the title Add possibility of passing in revoked partitions in task precommit KAFKA-19685: Add possibility of passing in revoked partitions in task precommit Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

connect small Small PRs triage PRs from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant