Repository navigation
ci: harden auto-fix path validation, prompt isolation, and PR token scope - #1088
Conversation
create_pr.py now only writes .kt files under the library and demo source directories, derives the branch name from the issue number and records the files it changed. The workflow checks out without persisted credentials, commits only those files and authenticates just for the push.
…cope - Restrict EDITABLE_SOURCE_DIRS to src/main/ (excluding test source sets and generated baseline profiles) and enforce a strict character allowlist regex on relative paths so embedded newlines cannot split mapfile in auto-fix.yml. - Scope list_project_files() to editable .kt files, cap modified/created files per run (MAX_FILES_PER_RUN = 10), add HTTP request timeouts, and abort if no files were changed. - Separate system_instruction from <issue_content>-delimited user input across all Gemini API calls and sanitize PR metadata (single-line commit/title, approved link domains, code-wrapped @mentions, and same-issue closing keywords). - Use GITHUB_TOKEN instead of SYNCED_GITHUB_TOKEN_REPO and create auto-fix PRs as drafts so downstream pull_request workflows do not execute unreviewed AI-generated code with repository secrets. - Remove committed create_pr.cpython-315.pyc bytecode file and block __pycache__/ and *.py[cod] in .gitignore and check_agent_leftovers.py.
Coverage (unit tests)Compared against
Line and branch coverage from unit test reports. History is recorded on the |
Coverage (instrumentation tests)
|
…llowups-merge # Conflicts: # .github/scripts/create_pr.py # .github/workflows/auto-fix.yml
| return ( | ||
| text.replace("</issue_content>", "</issue_content>") | ||
| .replace("<issue_content>", "<issue_content>") | ||
| ) |
There was a problem hiding this comment.
This only catches the exact lowercase tags, so </Issue_Content> or </issue_content > would still close the block from the model's point of view. Could we neutralise them with a case-insensitive regex?
| return ( | |
| text.replace("</issue_content>", "</issue_content>") | |
| .replace("<issue_content>", "<issue_content>") | |
| ) | |
| return re.sub( | |
| r"<\s*/?\s*issue_content\b[^>]*>", | |
| lambda m: m.group(0).replace("<", "<").replace(">", ">"), | |
| text, | |
| flags=re.IGNORECASE, | |
| ) |
There was a problem hiding this comment.
Good catch! Updated sanitize_content() in 21b891a to use a case-insensitive regex (r"<\s*/?\s*issue_content\b[^>]*>?") so mixed-case, spaced, attributed, or unclosed delimiter tags are all neutralized.
| --base main | ||
|
|
||
| git fetch origin "$BRANCH_NAME" 2>/dev/null || true | ||
| git push --force-with-lease origin "$BRANCH_NAME" |
There was a problem hiding this comment.
Since the fetch above updates origin/$BRANCH_NAME, the lease will always match and this behaves like a plain --force. If a maintainer pushed commits on top of a previous auto-fix PR, re-labelling the issue would wipe them out. Should we bail out when the remote branch already exists?
There was a problem hiding this comment.
Agreed — bailing out is much safer so we never wipe out commits a maintainer pushed onto an existing auto-fix branch. Added an early Check for Existing Auto-Fix Branch guard right after checkout in 21b891a (so we also avoid burning Gemini/Gradle quota on re-labels) plus a git ls-remote check right before a plain git push origin "$BRANCH_NAME".
| # Add a comment to the issue | ||
| gh issue comment "$ISSUE_NUMBER" --body "🚀 The **Auto-Fix Agent** has successfully generated a patch and created a Pull Request to address this issue! Please review the proposed changes." | ||
| gh issue comment "$ISSUE_NUMBER" --body "🚀 The **Auto-Fix Agent** has generated a patch and opened a draft Pull Request to address this issue. A maintainer will review the proposed changes before running full CI." |
There was a problem hiding this comment.
Out of curiosity, how do we run the CI afterwards? Since the PR is opened with GITHUB_TOKEN, nothing triggers on opened, and marking it as ready for review only starts instrumentation-test.yml (test.yml and lint-report.yml don't listen to ready_for_review). Would it make sense to mention in the comment that a maintainer needs to push or close and reopen it?
There was a problem hiding this comment.
Great point! Captured PR_URL from gh pr create in 21b891a and updated the issue comment to link directly to the draft PR and explain that a maintainer should close and reopen the PR (or push a commit to $BRANCH_NAME) and mark it ready for review to trigger the full CI suite.
- Use a case-insensitive regex in sanitize_content() to neutralize mixed-case, spaced, attributed, or unclosed <issue_content> delimiter tags. - Bail out if auto-fix/issue-<N> already exists on origin (both right after checkout and before git push) instead of force-pushing over potential maintainer commits. - Include the draft PR URL and instructions to close/reopen (or push a commit) and mark ready for review to trigger the full CI suite.
Follow-up security hardening on top of #1087 (
ci/harden-auto-fix) to close remaining CI execution, path-injection, and prompt-injection vectors:Why these additional guardrails are needed
Downstream
on: pull_requestCI execution with repository secrets (auto-fix.yml&create_pr.py)auto-fix.ymlruns./gradlew compileDebugKotlinwithpersist-credentials: false, pushing the branchauto-fix/issue-<N>directly tooriginusingsecrets.SYNCED_GITHUB_TOKEN_REPOand opening a non-draft PR immediately triggers same-repoon: pull_requestworkflows (test.yml,instrumentation-test.yml,demo-smoke-test.yml) wherehead.repo.full_name == github.repository.src/test/) and instrumentation tests (src/androidTest/)—both of which were previously reachable under*/src/—with access tosecrets.ACTIONS_API_KEY,secrets.SYNCED_GITHUB_TOKEN_REPO, andsecrets.GITHUB_TOKEN.auto-fix.ymlfromSYNCED_GITHUB_TOKEN_REPOtosecrets.GITHUB_TOKENand open the PR with--draftso downstreampull_requestworkflows do not automatically execute unreviewed AI-generated code until a maintainer reviews the diff.EDITABLE_SOURCE_DIRSincreate_pr.pyfrom*/src/to*/src/main/(and excludesrc/main/generated/), keepingsrc/test/,src/androidTest/, andsrc/screenshotTest/off-limits.Filename newline / control-character injection (
create_pr.py&auto-fix.yml)validate_source_pathcheckedrelative.startswith(EDITABLE_SOURCE_DIRS)andrelative.endswith(".kt"), which still allowed literal newlines (\n) inside path segments (valid on Linuxext4). Whenauto-fix.ymlrunsmapfile -t CHANGED_FILES < <(jq -r '.files[]' pr_metadata.json), an embedded newline splits one.files[]entry into multiple arguments togit add.SAFE_PATH_PATTERN(^[A-Za-z0-9_][A-Za-z0-9_.-]*(/[A-Za-z0-9_][A-Za-z0-9_.-]*)*\.kt$) and reject symlinks viaos.path.islink.Prompt isolation & PR metadata sanitization (
create_pr.py)system_instructionfrom<issue_content>-sanitized user messages across all 4 Gemini API calls (matchingtriage_issue.pyandanswer_issue.py).commit_messageandpr_titleas single-line strings without leading-flags.pr_bodyagainstALLOWED_DOMAINS, wrap@mentionsin code spans sogooglemaps-botcannot be used to mass-ping users/teams, and rewrite cross-issue closing keywords (Fixes #999->Fixes issue 999when999 != issue_number).Operational & hygiene fixes
list_project_files()to editable.ktfiles underEDITABLE_SOURCE_DIRS, cap modified/created files per run (MAX_FILES_PER_RUN = 10), addtimeout=60tourllib.request.urlopen, and abort ifchanged_filesis empty.concurrency:group toauto-fix.ymland support idempotent re-runs viagit push --force-with-leaseafter verifying[[ "$BRANCH_NAME" =~ ^auto-fix/issue-[0-9]+$ ]]..github/scripts/__pycache__/create_pr.cpython-315.pycbinary and add__pycache__/and*.py[cod]to.gitignoreandcheck_agent_leftovers.py.