Repository navigation
Conversation
Query.stream() and AggregationQuery.stream() read the default retry policy from the raw gRPC stub, which has no _retry attribute. Any retryable error partway through a stream on the default retry therefore raised AttributeError instead of resuming or raising the original error. retry=None failed the same way. Read the policy from the transport's wrapped method instead, and fix the resume path that this enables: back off and give up at the retry deadline instead of looping, keep the caller's cursor when nothing was yielded yet, resume with the remaining limit and no offset, and do not re-run an aggregation whose result was already yielded. Fixes googleapis#19129
There was a problem hiding this comment.
Code Review
This pull request introduces robust stream resumption and exponential backoff mechanisms for Firestore queries and aggregation queries when encountering retriable errors. Specifically, it adds helper functions and classes (resolve_stream_retry and StreamResumeBackoff) to resolve retry policies from wrapped GAPIC methods and manage backoff timing and deadlines. It integrates these helpers into the stream generators for both queries and aggregation queries, ensuring that queries resume from the last yielded snapshot with adjusted limits and offsets, and that backoff resets upon making progress. Comprehensive unit tests have also been added to verify these behaviors. I have no additional feedback to provide as there are no review comments.
Fixes #19129
When a
Query.stream()orAggregationQuery.stream()call on the default retry hits a retryable gRPC error partway through,_retry_query_after_exceptionreadstransport.run_query._retry. That is the raw gRPC stub, which has no_retry, so the caller gets anAttributeErrorinstead of a resume or the originalGoogleAPICallError.retry=Nonefails the same way.The existing unit tests didn't catch this because they gave the stub a fake
_retryattribute.Changes
transport._wrapped_methods[...].retry=Nonenow raises the original error.start_after(None)dropped the caller's cursor or raisedValueError.limitand set nooffset. If the limit was already reached, the stream ends.Behaviour change
With the default retry, a stream that fails partway with
ServiceUnavailable,InternalServerErrororDeadlineExceedednow resumes after the last document instead of crashing.Testing
lint,mypy, and coverage of the changed code are clean.Follow-ups (not in this PR)
read_time.explain_optionsis not passed to a resumed query.