Skip to content

fix(subscription): avoid per-provider poll backoff - #18448

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-multi-provider-poll
Open

fix(subscription): avoid per-provider poll backoff#18448
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/subscription-multi-provider-poll

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Problem

The subscription consumer round-robins all available DataNode providers. After any provider returned an empty poll response, it slept for 50-150 ms before trying the next provider. In a multi-DataNode deployment where only some providers own active subscription queues, these per-provider sleeps made consumption time grow with the provider count even when another provider already had data ready.

Fix

Track the number of currently available providers in each serial poll round. An empty response now immediately advances to the next provider, and the existing random backoff runs only after a complete round returns no user message. Provider RPCs remain serial, and single-provider behavior is unchanged.

Tests

Added unit coverage for:

  • finding data on the next provider without backing off;
  • backing off exactly once after all three providers return empty, then continuing the next round.

Verified with:

  • mvn spotless:apply -pl iotdb-client/subscription
  • mvn -Ddevelocity.off=true test -pl iotdb-client/subscription (50 tests)
  • mvn -Ddevelocity.off=true -DskipTests test-compile -pl iotdb-client/subscription -P with-zh-locale

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added comments explaining the why and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.

Key changed/added classes (or packages if there are too many classes) in this PR
  • AbstractSubscriptionConsumer
  • AbstractSubscriptionProviders
  • SubscriptionConsumerMultiProviderPollTest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants