You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Renaming a struct that holds identifier fields fails with "Cannot find identifier field" #4373
Renaming a struct column that contains an identifier field fails at commit:
schema=Schema(
NestedField(1, "user", StructType(NestedField(2, "id", LongType(), required=True)), required=True),
identifier_field_ids=[2],
)
tbl=catalog.create_table("db.t", schema=schema)
withtbl.update_schema() asupdate:
update.rename_column("user", "account")
# ValueError: Cannot find identifier field user.id. In case of deletion, update the identifier fields first.
The rename keeps every field ID, and the spec stores identifier fields by ID, so the identifier should follow the rename. UpdateSchema keeps the identifier fields as dotted names while the update is staged, and rename_column only updates a name that equals the renamed column itself, so a renamed parent struct leaves user.id behind. Java's SchemaUpdate.renameColumn has the same exact-name check.
I tried a few fixes and would like a direction before opening a PR, because the names-while-staging design makes each one trade something:
Rewrite the names nested under the renamed column in rename_column. Small, but staged names can collide, for example when two structs swap names, or when a case-insensitive update meets columns that differ only in case. Some of these updates work today.
Keep identifier fields by field ID while staging. Renames then just work, but set_identifier_fields needs a rule for which schema its names refer to (the starting schema like the other UpdateSchema methods, or the final one like Java), and either rule changes the result of some updates that work today.
Keep today's name lookup, and fall back to field IDs recorded at rename time only when a name no longer resolves. Updates that work today give the same result, but the fallback needs care so that it never picks a different field, which takes a few dozen lines.
Activity