You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Trial Windows build output reuse for cross-platform CI - #12410
Linux/macOS CI currently rebuilds MSTest, MTP, and extensions even though Windows produces the packages that ship. Add a disabled-by-default UseWindowsBuildOutputs trial to compare running the same Windows-built outputs across operating systems without changing test selection.
Windows exports configuration-matched binaries, dependencies, test data, packages, and coverage settings in a commit-bound, SHA-256-verified archive. Public Linux/macOS jobs and the official all-products Linux job download that run's exact producer attempt, restore their own SDK/runtime/workload and NuGet dependencies, rebase coverage paths and source-root mapping sidecars, and use DLL-based no-build execution. Generated acceptance applications still build on the destination OS. Existing native-build paths, SkipTests, and affected-test collection/fallback remain available.
The focused Linux coverage issue is now fixed: stale Windows source-root mappings caused the misleading no_symbols skip. The same Windows-built test selection now collects 18.9% line coverage on Linux, matching the Windows runner summary. The trial stays off pending full acceptance, macOS, and hosted transport validation; no compute saving has been measured.
Keep the existing project-based runner and reporting configuration rather than introducing a second module-selection runner. Consumers wait for the existing Windows job, so less aggregate computation does not necessarily mean shorter elapsed CI time. Missing or invalid artifacts fail closed; no silent rebuild fallback is introduced. Symbol extraction and symbol-format experiments that did not fix coverage were removed from the delivered change.
The importer also rebases the physical project/source/package paths in .msCoverageExtensionSourceRootsMapping_*, preserving virtual map keys and encoding. Hidden sidecars are included on Unix. DLL/PDB bytes and all four existing SymbolsRestrict* defaults remain unchanged.
Demonstration
Infrastructure-only change; screenshots are not applicable. The initial composed experiment, before the source-root repair, used SDK 12.0.100-alpha.1.26556.115, Release/net11.0, and a Windows-produced Microsoft.Testing.Platform.3.0.0-ci.nupkg payload:
Producer / execution
DeadlineHelperTests
Coverage
Windows / Windows
35 passed
19.0% line coverage
Linux / Linux, clean native checkout
35 passed
19.0% line coverage
Windows / Linux, old importer
35 passed
No coverable data; skipReason: no_symbols
Both data rows of TryGetDotnetPathByArchitecture_LinuxConfiguredElfMuxer_IsRejectedWithoutFallback also passed on Linux from the Windows-built assembly; they were present and skipped on Windows as expected. All 2,722 transported DLLs remained byte-identical. Restore and no-build test binlog replay found zero Csc/Vbc/Fsc tasks. TRX, CTRF, and JUnit reports were produced; coverage-file existence was not accepted as proof of nonempty coverage.
Source-root repair, validated 2026-11-12: a fresh Windows producer from f50f73e66dd136c88c213795ffcf207ad8bf3321, with CodeCoverage 18.13.0-preview.26560.3, was compared using the same archive with old and repaired importers. All 35 selected deadline cases passed on Windows and both Linux variants, with exact TRX names reconciled. The old importer still produced empty coverage; the repaired importer produced an 18.9% runner line-coverage summary, matching the current Windows baseline. All 2,836 DLL/PDB files stayed byte-identical, and four restore/test binlog replays found zero compiler tasks. The repaired Cobertura contains 5,223 covered line entries out of 25,000, including correctly relocated source paths. This is local composed evidence, not hosted CI or a full-matrix result.
Validation and regression proof
Rechecked before committing the original trial as 4d62268d8d5ce932013b07e745846211cbe324ed:
Command
Result
.\eng\test-windows-build-outputs.ps1
Exit 0; originally 12 transport cases passed. The same cases also passed on Linux during implementation.
.\eng\validate-affected-tests.ps1 -SelfTest
Exit 0; originally 71 affected-test/collection/rollback wiring cases passed.
python -c "import yaml; from pathlib import Path; paths = [Path('azure-pipelines.yml'), Path('azure-pipelines-official.yml'), *Path('eng').joinpath('pipelines','steps').glob('*.yml')]; [yaml.safe_load(p.read_text(encoding='utf-8-sig')) for p in paths]; print(f'Parsed {len(paths)} pipeline YAML files')"
Earlier real product builds/pack commands and the detailed Windows/Linux comparisons are recorded in the decision record. The original source baseline was 32a6ed6e1511ae9b61c9ce307e22ed5594b239cc; the original implementation patch has SHA-256 FD084840F3EAB0B94FD597F068DBB54DD30F8303C31C6F8209D8E6B25B6C266B.
The source-root repair is committed as d73ad4e8c3cb942a00ef547f9849dbb2f081dc2d. Its exact tested patch on f50f73e66dd136c88c213795ffcf207ad8bf3321 has SHA-256 74158C913CE0ABF21C83BA10137DDF6DA4C6783BF47A20393F378987F243EDCF. .\eng\test-windows-build-outputs.ps1 now passes 13 cases on Windows and Linux; .\eng\validate-affected-tests.ps1 -SelfTest passes the updated 80 cases. The new mapping regression fails against the old importer at the intended stale-source-path assertion. Windows build/pack and both real Linux no-build test commands exited zero; exact commands and limitations are in the maintained decision record.
Remaining gaps: the focused Linux source-resolution defect is repaired, but full artifact-based acceptance execution, macOS execution, and live Azure/1ES artifact transport have not been verified. No CI performance improvement is claimed. Keep the trial disabled until these checks succeed; do not suppress coverage or accept empty results to enable it.
Test-value evidence
Contract / test
Without the behavior
Correct implementation
Reject stale producer commits / RejectsStaleSourceBeforeExtraction
Executed: removing only the commit comparison in an isolated helper copy made the unchanged test fail its rejection assertion, exit 1.
Executed: passed as part of the transport suite, exit 0.
Rebase nested Windows coverage paths / WindowsCoveragePathsUseConsumerDirectorySeparators
Executed: the initial prefix-only implementation failed the exact consumer source-root regex assertion on Linux.
Executed: passed on Windows and Linux after normalizing the rebased path separators.
Rebase coverage source maps / CoverageSourceMappingsUseConsumerPaths
Executed: the unchanged importer failed the expected consumer repository/source mapping assertion.
Executed: passed on Windows and Linux, preserving virtual keys and UTF-16 encoding, including nested package caches.
Executed: exact marker assertions passed on Windows and Linux.
Preserve composed execution and coverage
Executed: the old importer ran all 35 selected cases but produced no coverable data.
Executed: the repaired importer ran the same 35 cases and collected real source coverage with an 18.9% runner summary; all 2,836 DLL/PDB files remained unchanged. Broader rollout is not yet validated.
Detailed reports/binlogs remain local session evidence, not publicly hosted artifacts or a CI verification claim.
Add an opt-in same-run Windows Release payload for public Linux/macOS and official Linux test jobs. Verify payload provenance, preserve target-OS restore and test selection, and document the unresolved coverage parity gap with the trial disabled by default.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve the pipeline conflicts by keeping the Windows-output reuse trial alongside main's affected-test provider comment and report-comparison validation. Preserve the auto-merged report and affected-test pipeline updates.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
Note
This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.
Relocate coverage source-map sidecars as well as coverage.config, preserving virtual path keys and encoding. Include hidden output metadata on Unix and cover repository, nested package-cache, and cross-OS mappings. Verify the same Windows-built tests collect coverage on Linux without changing DLLs, PDBs, or symbol-access defaults.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Compared merge base 50b513022be7c356cb342866a3b755a4b18bf97d with head d73ad4e8c3cb942a00ef547f9849dbb2f081dc2d. Evidence applies to this head, not every intermediate commit.
Test-value evidence
This PR changes only pipeline YAML, PowerShell scripts under eng/ and a decision record. It has no changed C# or MSTest test methods.
Static: the cases should fail with a prefix-only or unrebased importer.
Unverified. The PR description reports red and green runs, but no accessible records were checked here.
Preserve binary, package and test-data bytes
RoundTripPreservesBinariesPackagesAndTestData
Expected to pass with the correct implementation.
The PR description says no counterfactual was run.
Pipeline wiring (UseWindowsBuildOutputs, which defaults to off)
None found by static inspection
Not assessed.
Hosted Azure/1ES transport and macOS execution are unvalidated, as the PR states.
This workflow ran no code. Green CI or an author claim is not paired execution proof.
🧵 Parallel safety
NOT_APPLICABLE. No changed tests or parallelization configuration were in scope.
Per-test assessment
No changed C# test methods were found, so nothing is graded (Not applicable). The PowerShell harness was not graded against the C# rubric.
I have no inline findings.
This advisory review is heuristic and non-blocking. Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.
Scope official coverage validation to the restored non-Windows graph, transport staged acceptance packages, and normalize reporter test line endings across execution hosts.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
eng/test-windows-build-outputs.ps1 is wired into the pipeline as a validation step
Not assessed: requires an AzDO run
Unverified (non-executable here). Existing validation script not inspected in depth.
🧵 Parallel safety
STATUS: PARTIAL. The specialist subtask was not run in this session, so coverage is incomplete and this is not an all-clear. The only changed tests are pure string normalizations of local variables. No shared state, resource, or parallelization configuration changed (config_count=0).
Per-test assessment
Pseudo-mutation evidence below consists of static predictions, not executed mutation results.
Result
Quality
Test
Notes
How to improve
Pass
A (90–100)
mod TerminalTestReporterTests.NonAnsiTerminal_OutputFormattingIsCorrect
Exact multiline equality is kept, with line endings normalized.
None
Pass
A (90–100)
mod TerminalTestReporterTests.InProcessRetry_AnnotatesAttemptsAndCountsTestOnce
Exact multiline equality is kept, with line endings normalized.
None
Pass
A (90–100)
mod TerminalTestReporterTests.SimpleAnsiTerminal_OutputFormattingIsCorrect
Exact multiline equality is kept, with line endings normalized.
None
Pass
A (90–100)
mod TerminalTestReporterTests.AnsiTerminal_OutputFormattingIsCorrect
Exact multiline equality is kept, with line endings normalized.
None
Pass
A (90–100)
mod TerminalTestReporterTests.AnsiTerminal_OutputProgressFrameIsCorrect
Exact multiline equality is kept, with line endings normalized.
None
No inline findings. The normalization matches the repository guidance for platform-native newlines.
<sub>This advisory review is heuristic and non-blocking.
Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Reusing the Windows-built test assembly on Linux/macOS leaves three newline-sensitive assertions comparing CRLF expectations with LF output, causing test failures until both operands are normalized.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Normalize line endings in reused helper test assertions
With a CRLF Windows checkout, the reused Microsoft.Testing.Platform.MSBuild.UnitTests assembly still has three newline-sensitive assertions in FailedTestHelperTests.cs:95–145. On Linux/macOS, the compiled raw-string expectations retain CRLF while the generated or normalized messages use LF, so these selected tests fail despite correct output. Normalize both operands in the comparisons at lines 95–103, 117–124, and 138–145, as this PR already does for the terminal-reporter tests.
Final test-triage resolution — Azure Pipelines build 1677661
Triggering check: microsoft.testfx (Build Windows Release). Mode: full. Evidence is incomplete (1 artifact download failure, history download budget exhausted), so this is an inconclusive resolution, not a clean bill of health.
Failure signatures (Linux Release / "Test" task, exit code 2; the Windows "Test affected changes" task reports the same tests)
OpenTelemetry acceptance tests (OpenTelemetryTests, OpenTelemetryActivityTopologyTests, 17 results) fail in ClassInitialize. The dotnet build of the generated ApplicationOwnedOpenTelemetry asset fails with NU1015: The following PackageReference item(s) do not have a version specified: Microsoft.Extensions.Hosting. This is deterministic and a test-asset/package-version issue, not a flake. Confidence: high on the immediate cause. The link to this PR's changes (build output reuse, central package versions) is likely but unverified.
PlaywrightSdkTests.EnablePlaywrightProperty_* (4 results: Runner and VSTest) fail with Unexpected exit code / missing expected output. The root cause is not visible in the compact evidence, so confidence is low.
Windows App Model diagnostics probes (RetryFailsFirstAttempt, RetryTestMethod_PassesOnSecondApp, NativeTests.Outcome) fail with deliberate probe messages such as "This probe deliberately fails the first retry attempt". These look like intentional diagnostic probes, not product regressions, and are not treated as actionable here.
No retry/flaky recovery, crash/hang, or duration-regression evidence was found. No issue is created: this is a PR-specific failure and no recurrence is proven (history was incomplete).
Next step: check whether the PR changes how Microsoft.Extensions.Hosting gets its version in generated test assets (for example, Directory.Packages / central package management or reused build output). Then inspect the Playwright test logs from the Test task and rerun the Linux leg.
🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · copilot · auto · ~38 AIC · ⌖ ~1 AIC · ⊞ 10.1K
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request
Summary
Linux/macOS CI currently rebuilds MSTest, MTP, and extensions even though Windows produces the packages that ship. Add a disabled-by-default
UseWindowsBuildOutputstrial to compare running the same Windows-built outputs across operating systems without changing test selection.Windows exports configuration-matched binaries, dependencies, test data, packages, and coverage settings in a commit-bound, SHA-256-verified archive. Public Linux/macOS jobs and the official all-products Linux job download that run's exact producer attempt, restore their own SDK/runtime/workload and NuGet dependencies, rebase coverage paths and source-root mapping sidecars, and use DLL-based no-build execution. Generated acceptance applications still build on the destination OS. Existing native-build paths,
SkipTests, and affected-test collection/fallback remain available.The focused Linux coverage issue is now fixed: stale Windows source-root mappings caused the misleading
no_symbolsskip. The same Windows-built test selection now collects 18.9% line coverage on Linux, matching the Windows runner summary. The trial stays off pending full acceptance, macOS, and hosted transport validation; no compute saving has been measured.Design and decision record
Windows build output reuse: research and decisions records the artifact/lifecycle contract, alternatives, successful and unsuccessful experiments, and rollout prerequisites.
Keep the existing project-based runner and reporting configuration rather than introducing a second module-selection runner. Consumers wait for the existing Windows job, so less aggregate computation does not necessarily mean shorter elapsed CI time. Missing or invalid artifacts fail closed; no silent rebuild fallback is introduced. Symbol extraction and symbol-format experiments that did not fix coverage were removed from the delivered change.
The importer also rebases the physical project/source/package paths in
.msCoverageExtensionSourceRootsMapping_*, preserving virtual map keys and encoding. Hidden sidecars are included on Unix. DLL/PDB bytes and all four existingSymbolsRestrict*defaults remain unchanged.Demonstration
Infrastructure-only change; screenshots are not applicable. The initial composed experiment, before the source-root repair, used SDK
12.0.100-alpha.1.26556.115, Release/net11.0, and a Windows-producedMicrosoft.Testing.Platform.3.0.0-ci.nupkgpayload:skipReason: no_symbolsBoth data rows of
TryGetDotnetPathByArchitecture_LinuxConfiguredElfMuxer_IsRejectedWithoutFallbackalso passed on Linux from the Windows-built assembly; they were present and skipped on Windows as expected. All 2,722 transported DLLs remained byte-identical. Restore and no-build test binlog replay found zero Csc/Vbc/Fsc tasks. TRX, CTRF, and JUnit reports were produced; coverage-file existence was not accepted as proof of nonempty coverage.Source-root repair, validated 2026-11-12: a fresh Windows producer from
f50f73e66dd136c88c213795ffcf207ad8bf3321, with CodeCoverage18.13.0-preview.26560.3, was compared using the same archive with old and repaired importers. All 35 selected deadline cases passed on Windows and both Linux variants, with exact TRX names reconciled. The old importer still produced empty coverage; the repaired importer produced an 18.9% runner line-coverage summary, matching the current Windows baseline. All 2,836 DLL/PDB files stayed byte-identical, and four restore/test binlog replays found zero compiler tasks. The repaired Cobertura contains 5,223 covered line entries out of 25,000, including correctly relocated source paths. This is local composed evidence, not hosted CI or a full-matrix result.Validation and regression proof
Rechecked before committing the original trial as
4d62268d8d5ce932013b07e745846211cbe324ed:.\eng\test-windows-build-outputs.ps1.\eng\validate-affected-tests.ps1 -SelfTestgit diff --checkYAML syntax check, exit 0, parsed 13 pipeline/template files:
Earlier real product builds/pack commands and the detailed Windows/Linux comparisons are recorded in the decision record. The original source baseline was
32a6ed6e1511ae9b61c9ce307e22ed5594b239cc; the original implementation patch has SHA-256FD084840F3EAB0B94FD597F068DBB54DD30F8303C31C6F8209D8E6B25B6C266B.The source-root repair is committed as
d73ad4e8c3cb942a00ef547f9849dbb2f081dc2d. Its exact tested patch onf50f73e66dd136c88c213795ffcf207ad8bf3321has SHA-25674158C913CE0ABF21C83BA10137DDF6DA4C6783BF47A20393F378987F243EDCF..\eng\test-windows-build-outputs.ps1now passes 13 cases on Windows and Linux;.\eng\validate-affected-tests.ps1 -SelfTestpasses the updated 80 cases. The new mapping regression fails against the old importer at the intended stale-source-path assertion. Windows build/pack and both real Linux no-build test commands exited zero; exact commands and limitations are in the maintained decision record.Remaining gaps: the focused Linux source-resolution defect is repaired, but full artifact-based acceptance execution, macOS execution, and live Azure/1ES artifact transport have not been verified. No CI performance improvement is claimed. Keep the trial disabled until these checks succeed; do not suppress coverage or accept empty results to enable it.
Test-value evidence
RejectsStaleSourceBeforeExtractionWindowsCoveragePathsUseConsumerDirectorySeparatorsCoverageSourceMappingsUseConsumerPathsRoundTripPreservesBinariesPackagesAndTestDataDetailed reports/binlogs remain local session evidence, not publicly hosted artifacts or a CI verification claim.