Skip to content

[tool] Set up the script/ci_info package and add initial find_builds tool - #13580

Merged
auto-submit[bot] merged 2 commits into
flutter:mainfrom
elliette:ci-info-find-builds
Nov 18, 2026
Merged

auto-submit[bot] merged 2 commits into
flutter:mainfrom
elliette:ci-info-find-builds

Conversation

@elliette

@elliette elliette commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Sets up a new ci_info package in /script that can be used to analyze runs on CI.

Adds the first script in the package, find_builds which can be used to find all the LUCI builds for a given PR or commit hash.

Example:

Command: script/ci_info/run.sh find_builds --pr 13505

Screenshot 2026-11-06 at 11 17 20 AM

Command: script/ci_info/run.sh find_builds --pr 13505 --failed-only

Screenshot 2026-11-06 at 11 27 55 AM

Work towards flutter/flutter#196131

Pre-Review Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

Footnotes

  1. Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. ↩ ↩2

Adds the script/ci_info tooling package, CI checks, and the find_builds CLI script to query LUCI Buildbucket for PR or commit builds.
@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Nov 6, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Dart package, ci_info, under script/ci_info to analyze LUCI/Buildbucket CI runs for the repository, along with corresponding CI workflow updates to run its tests, formatting, and analysis. A critical syntax error was identified in script/ci_info/lib/ci_lib.dart at line 154, where the invalid 'pageToken': ?pageToken syntax is used, which will cause a compilation failure and should be replaced with a collection-if statement.

Comment thread script/ci_info/lib/ci_lib.dart Resolved

@tarrinneal tarrinneal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few possible issues, I'm open to push back on some if there is a decent reason

Comment thread script/ci_info/run.sh Outdated
Comment on lines +39 to +41
if [[ ! -f "$dir/.dart_tool/package_config.json" ]]; then
"$dart" pub get -C "$dir" >&2
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since run.sh is designed so agents can invoke multiple scripts concurrently in a single turn (L11-12), two parallel invocations on a fresh checkout will both see ! -f "$dir/.dart_tool/package_config.json" and run dart pub get -C "$dir" simultaneously—allowing one pub get to truncate/rewrite package_config.json while the other is starting dart run.

Could we guard the initial pub get bootstrap with a portable lock directory (e.g. mkdir "$dir/.dart_tool/pub_get.lock" spin-wait + trap cleanup, re-checking package_config.json inside) so parallel first-use calls wait cleanly without locking subsequent runs?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Instead of adding a lock, I decided instead to simply exit and tell the user to run pub get if .dart_tool/package_config.json is missing or older than pubspec.yaml. FWIW I couldn't reproduce the race, but this is simpler than locking anyway.

Comment on lines +124 to +135
// pRPC prefixes JSON responses with the XSSI guard `)]}'`.
final String raw = response.body;
final int brace = raw.indexOf('{');
if (brace == -1) {
throw Exception('Unexpected Buildbucket response: ${truncate(raw, 200)}');
}
final String decoded = raw.substring(brace);
if (cacheAge != null) {
_writeCache('prpc', cacheKey, decoded);
}
return jsonDecode(decoded) as Map<String, dynamic>;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two resilience issues in prpc:

  1. _writeCache (L132) writes decoded to disk before jsonDecode(decoded) as Map<String, dynamic> (L134) validates it—if a captive portal or truncated response returns a 200 body containing {, the invalid payload is cached and breaks subsequent runs for 5 minutes with FormatException. Decoding before _writeCache (and treating FormatException / FileSystemException on cache read/write as a cache miss) avoids poisoning the cache.
  2. http.post (L100) has no default timeout, so a stalled connection hangs indefinitely instead of entering the retry loop (TimeoutException implements Exception, so .timeout(...) would flow straight into on Exception on L108).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed! The response is now decoded and checked before it's cached, and each request has a 30s timeout. I also added tests for the caching behavior, though not the timeout (to test the timeout, we would need to make it configurable so that we aren't waiting 30s in the test). Let me know what you think.

Comment on lines +133 to +136
final String summary = ci_lib
.stringField(build, 'summaryMarkdown', defaultValue: '')
.replaceAll('\n', ' ')
.trim();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

summaryMarkdown from Buildbucket often contains \r\n line endings, so .replaceAll('\n', ' ') leaves \r intact and causes print('$indent-> ...') below to carriage-return over the line prefix. Consider .replaceAll(RegExp(r'[\r\n]+'), ' ') (and ci_lib.stringField(build, 'id') on L132 for consistency with the other fields).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to .replaceAll(RegExp(r'[\r\n]+'), ' ')

I didn't call stringField on the id because its the one field that identifies the build. If it's ever missing, I'd rather fail immediately than print ? (which is what stringField falls back to) and a broken link. The other fields are mostly descriptive, so a '?' placeholder is fine there.

Comment thread script/ci_info/bin/find_builds.dart Outdated
Comment on lines +24 to +25
const Set<String> _terminalBad = <String>{'FAILURE', 'INFRA_FAILURE', 'CANCELED'};
const Set<String> _terminalFailed = <String>{'FAILURE', 'INFRA_FAILURE'};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small CLI questions around --failed-only:

  1. The PR description's second example uses --failures-only, whereas the parser defines --failed-only (worth updating the PR description, or adding an alias if both were intended).
  2. --failed-only filters by _terminalBad (including 'CANCELED'), whereas the summary counts 'CANCELED' separately from failed (_terminalFailed). On a multi-push PR where earlier pushes were canceled and the latest push passed, --failed-only will still list all superseded CANCELED builds under 0 failed | N canceled. Is including CANCELED in --failed-only intentional?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Updated the PR description, thanks for catching!
  2. That was not intentional :) Fixed

@tarrinneal tarrinneal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg

@elliette elliette added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 18, 2026
@auto-submit
auto-submit Bot merged commit d57744b into flutter:main Nov 18, 2026
14 checks passed
BeatusMomentum pushed a commit to BeatusMomentum/flutter that referenced this pull request Nov 19, 2026
…r#196992)

flutter/packages@dca2e58...81b7c44

2026-11-19 36861262+QuncCccccc@users.noreply.github.com [material_ui]
Generalize ButtonTemplateM3E to read the tokens of each button variant
(flutter/packages#13684)
2026-11-19 engine-flutter-autoroll@skia.org Roll Flutter from
06f32f8 to e8cc6f3 (30 revisions) (flutter/packages#13719)
2026-11-18 katelovett@google.com [ci] Fix workflow permissions
(flutter/packages#13713)
2026-11-18 engine-flutter-autoroll@skia.org Roll Flutter from
2f09f96 to 06f32f8 (18 revisions) (flutter/packages#13712)
2026-11-18 53801679+ChandraChaan@users.noreply.github.com [material_ui]
Include layoutBehavior in ButtonThemeData equality
(flutter/packages#13517)
2026-11-18 21270878+elliette@users.noreply.github.com [tool] Set up the
`script/ci_info` package and add initial `find_builds` tool
(flutter/packages#13580)

If this roll has caused a breakage, revert this CL and set the roller
to dry run mode using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC flutter-ecosystem@google.com on the revert to ensure that a
human
is aware of the problem.

To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autosubmit Merge PR when tree becomes green via auto submit App CICD Run CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants