Repository navigation
feat(nodes): propagate conditioning prompt and text encoder metadata (Stage 2) - #10172
skunkworxdark wants to merge 4 commits into
Conversation
Initial Check-in & Status Update (
|
eabd7ab to
578968a
Compare
Rebase & Sync Update (
|
578968a to
05cf76f
Compare
Rebase & Sync Update (
|
05cf76f to
c5cefae
Compare
Rebase & Sync Update (
|
c5cefae to
088f416
Compare
Rebase & Upstream Sync UpdateBranch Audit & Integration Summary:
Test Gates:
|
088f416 to
33e2891
Compare
3e7f33f to
b8fb814
Compare
Review Parity & Defensive Audit Fixes UpdateBranch Audit Fixes Summary:
Test Gates:
|
Stage 2 Update: Centralized Metadata Label Registry (
|
81df465 to
085295b
Compare
Rebase & Sync Update (
|
Registry Completeness & Type-Safety Update (
|
|
Rebased Stage 2 ( Rebase Summary
Local Verification (
|
5ee9a62 to
b8b6d1c
Compare
|
Rebased Stage 2 ( Rebase Summary
Local Verification (
|
b8b6d1c to
1ddb7c8
Compare
|
Rebased Stage 2 ( Rebase Summary
Local Verification (
|
f322e7b to
b54e65e
Compare
|
Synced
|
…data upstream Add optional metadata fields to conditioning classes and populate them across 18 text encoder invocations. Preserve provenance in conditioning modifiers using model_copy and extract in denoise invocations. Support multi-prompt collection extraction and document modifier provenance guidelines.
…gistry Register Stage 2-exclusive component_source key (QwenImageConditioningField.component_source) in CORE_METADATA_REGISTRY, regenerate API contracts, and add Stage 2 coverage in test_core_metadata.py.
…age 1 Add Stage 1 Wan LoRA target and low_noise_weight fields to canonical LoRAMetadataField in fields.py, remove duplicate definition in metadata.py while preserving re-export, regenerate API contracts, and restore extract_conditioning_metadata() documentation in media-metadata.mdx.
b54e65e to
9d5592c
Compare
Summary
In Stage 1 (#10083), native metadata building and emission was unified across all 19 image and video denoise invocations for generation-level parameters (dimensions, steps, CFG/guidance, scheduler, seeds, main model, VAE, transformer LoRAs, and ControlNets/adapters). However, prompts, standalone text encoders, encoder-specific LoRAs, and conditioning modifiers still required manual metadata wiring or wrapper nodes because conditioning edges carried only raw tensor names.
Stage 2 extends metadata propagation upstream along the conditioning edge (
*ConditioningField), enabling text encoders, modifier nodes, and denoisers to propagate provenance automatically without extra graph wiring or breaking existing workflows:prompt,style_prompt,loras, standalone text encoders, and modifier flags) withdefault=Noneacross all 16*ConditioningFieldclasses ininvokeai/app/invocations/fields.py.*ConditioningFieldwithprompt, standalone text encoder models (t5_encoder,clip_embed_model,clip_l_model,clip_g_model,mistral_encoder,qwen3_encoder,qwen3_vl_encoder, etc.), and text-encoder LoRAs.Krea2ConditioningRebalanceInvocation,Krea2SeedVarianceInvocation,ZImageSeedVarianceEnhancerInvocation, andConditioningInvocation(masking) now use non-mutatingself.conditioning.model_copy(update={...}), ensuring that upstream prompt, model, and LoRA provenance flows unbroken through arbitrarily chained modifiers.extract_conditioning_metadata): Implemented ininvokeai/app/invocations/metadata.pyand integrated into_build_metadata()across all 19 base denoise invocations:positive_conditioningsetspositive_prompt;negative_conditioningsetsnegative_prompt.loras.rebalance_*(Krea-2),seed_variance_*(Krea-2 and Z-Image), and layout/caption metadata (flux3_caption,ideogram4_caption,moss_image_layout_prompt).positive_style_promptandnegative_style_prompt.Collectnode into denoise), the primary unmasked prompt is recorded aspositive_prompt(ensuring existing UI gallery recall and single-string consumers function seamlessly).positive_prompts: list[str](andnegative_prompts: list[str]), avoiding any delimiter joining (such as" | ") that could corrupt user prompts containing delimiter characters.positive_promptsandnegative_promptstoCORE_LABELSandCORE_LABELS_STRINGinmetadata_linked.py, enabling workflows to extract and reuse the exact collection usingMetadataToStringCollectionInvocation.metadata_constants.py): Consolidated 9 duplicate hand-maintained label sets and literals into a single, type-safeCORE_METADATA_REGISTRYusingMetadataType(str, Enum). Eliminates label drift across modules, guarantees* CUSTOM LABEL *as index 0 for UI dropdown defaults, and automatically derives all UI dropdown literals (CORE_LABELS*) and runtime model unwrapping sets (CORE_MODEL_LABELS).docs/src/content/docs/development/Guides/creating-nodes.mdxillustrating themodel_copy(update={...})pattern for third-party custom node developers.docs/src/content/docs/development/Architecture/media-metadata.mdxto reflect automatic conditioning metadata extraction and the newpositive_prompts/negative_promptsand style prompt schema fields.docs/src/content/docs/Users Guide/08.Workflows/editor-reference.mdxexplaining Denoise-to-Decode metadata wiring, multi-prompt collections, and workflow metadata recovery viaMetadata to StringandMetadata to String Collection.Architectural Roadmap: 3-Stage Metadata Unification
This PR is Stage 2 of the three-stage initiative:
Related Issues / Discussions
QA Instructions
Automated Verification
uv run --no-sync pytest tests/app/invocations/test_conditioning_metadata.py tests/app/invocations/test_denoise_metadata.py tests/app/invocations/test_core_metadata.py(78 passed in 0.43s)uv run --no-sync python .scratch/prs/pr-10172-conditioning-metadata/audit_workflows.py(8/8 workflows PASS with 0 issues)uv run --no-sync ruff check invokeai tests docs(passed with 0 errors)uv run --no-sync ruff format --check invokeai tests docs(passed, all clean)pnpm lintininvokeai/frontend/api(passed, OpenAPI schema and TS types in sync)Test Workflows Validated (8/8 PASS)
Audited and verified in
.scratch/prs/pr-10172-conditioning-metadata/test_workflows/:01_flux_multi_prompt_collect_metadata.json: FLUX multi-prompt conditioning viaCollectnode intoFLUX Denoise.02_krea2_rebalance_and_seed_variance_metadata.json: Chained Krea-2 modifiers (rebalance->seed_variance).03_z_image_seed_variance_metadata.json: Z-Image seed variance modifier.04_sdxl_base_and_refiner_multi_pass_metadata.json: SDXL base and refiner multi-pass with style prompts.05_metadata_string_collection_extraction.json: Extractingpositive_promptscollection usingMetadataToStringCollectionInvocation.06_sd15_txt2img_prompt_metadata.json: Classic SD1.5 text encoder prompt propagation.07_wan_video_conditioning_metadata.json: Wan video text conditioning propagation.08_flux3_layout_prompt_metadata.json: FLUX.3 layout prompt and conditioning.Review
Noneif optional metadata is omitted or mocked.Compatibility / Rollout
*ConditioningFielddefault toNone. Existing saved workflows, custom nodes, and cached graph executions run without modification or errors.ConditioningField(conditioning_name=...)) continue to instantiate cleanly. If a custom node does not usemodel_copy, downstream denoise nodes gracefully skip prompt metadata without failing.positive_promptandnegative_promptidentically to existing behavior; multi-prompt workflows exposepositive_promptsandnegative_promptsas collections.Checklist
What's Newcopy (if doing a release after this PR)