Fix GH-11591: AppendIterator with generators that yield nothing - #23270
Open
matthiasgoergens wants to merge 8 commits into
Open
Fix GH-11591: AppendIterator with generators that yield nothing#23270matthiasgoergens wants to merge 8 commits into
matthiasgoergens wants to merge 8 commits into
Conversation
Reinserting the same generator at the same outer key after an unset inherited a stale empty marker keyed by (object, key), silently skipping the entry where reinsertion at any other key threw. A generator can only enter the iterator through append()'s own probe, so marking the object itself is equivalent and cannot go stale.
Moving a probed-empty generator to another key previously threw, because the marker followed the entry key rather than the generator object.
An empty generator appended after a valid entry was never probed by append(), so its first exhaustion during traversal left it unmarked and a second traversal still threw. Mark generators that finish while initialising (they never yielded) at every exhaustion point, keyed by object so reinsertion cannot inherit a stale marker.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Appending a generator probes it by rewinding, which runs an empty generator to
completion; traversing it later then throws "Cannot traverse an already closed
generator". Mark generators exhausted by this probe and skip them during
traversal, including after the inner array is modified. A deliberate behaviour
change: appending the same probed-empty generator twice now adds two empty
entries instead of throwing on the second append; consumed (once-yielded)
generators still throw on re-traversal.