Repository navigation
fix(firestore): store non-native app: and user: state as JSON-safe values - #8232
Open
AliMuhammadAslam wants to merge 1 commit into
Open
AliMuhammadAslam wants to merge 1 commit into
AliMuhammadAslam wants to merge 1 commit into
Conversation
…lues FirestoreSessionService writes app: and user: state to Firestore natively, and passed values the Firestore client cannot encode, such as a date, UUID, Decimal, plain Enum or Pydantic model, through unchanged. The client raised TypeError inside the write transaction, so append_event lost the event and create_session failed, although the same values are stored under session-level keys and by the other session services. _encode_firestore_value now keeps the values Firestore stores natively and coerces any other value with _session_util.make_json_safe_state, the same form the session bucket and the other backends store. Fixes google#8231
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
VertexAiSessionService)Problem:
FirestoreSessionServicewritesapp:anduser:state to Firestore natively. Values the Firestore client can't encode reached it unchanged:datetime.date,uuid.UUID,decimal.Decimal, a plainEnum, a Pydantic model. The client raisedTypeError: Cannot convert to a Firestore Valueinside the write transaction, soappend_eventfailed and lost the event, andcreate_sessionfailed. The same values work under session-level keys and in the other session services, which store them in JSON-safe form.Solution:
_encode_firestore_valueis the single function that bothcreate_sessionandappend_eventuse for app and user state. It now leaves values the client stores natively unchanged:None,bool,int,float,str,bytes,datetime,GeoPointand document references. Any other value is converted with_session_util.make_json_safe_state, the same function used for the session bucket and the other backends. The result is then encoded again, so nested arrays inside a converted value, such as a list of lists in a Pydantic model, are still wrapped.Native types keep the current behaviour, including the existing tests that check a
datetimestays adatetime. Values that used to abort the transaction are now stored, and read back, as they are in the session bucket and in the database and SQLite backends:"2024-06-15", the UUID string,"1.50".Testing Plan
Unit Tests:
New tests in
tests/unittests/integrations/firestore/test_firestore_session_service.py:test_append_event_stores_non_native_app_and_user_stateandtest_create_session_stores_non_native_app_and_user_state, each parametrized overdate,UUID,Decimal,Enumand a Pydantic model holding a date and a list of lists. Each test passes what the service writes for app and user state through the Firestore client's ownencode_dict/decode_dict, so a value the client can't encode raises as it would in the real transaction. It then reads the result back throughget_user_state, and asserts that app, user and session state all hold the same JSON-safe value.test_create_session_keeps_firestore_native_app_and_user_statechecks thatGeoPoint,bytesand a list ofdatetimeare written unchanged.Before the fix, the 10 parametrized cases fail with
TypeError: ('Cannot convert to a Firestore Value', ...). After it:isort,pyinkandmypypass on the changed files.Manual End-to-End (E2E) Tests:
I don't have a live Firestore database. Instead I ran the reproduction script from the issue, which mocks the client but encodes each
transaction.set()with the client's realpbs_for_set_no_merge:I also checked that
SqliteSessionServiceandDatabaseSessionServicestore the sameapp:anduser:values and read them back as'2024-06-15','00000000-0000-0000-0000-000000000001'and'1.50'. Those are the values the new tests expect from Firestore.Checklist
Additional context
A value that was stored natively before this change, such as a
datetime,bytesor a nested list, is written exactly as it was. Nothing already in Firestore needs migrating.Vectorisn't treated as native because its module isn't in every supportedgoogle-cloud-firestorerelease (>=2.11). Firestore's vector fields are for vector search, not for session state.