Repository navigation
Conversation
|
@gocamille is on triage rotation this week and is the reviewer for this one. Labels come from the PR labeler. Please review the change. Automated triage. Labels are a first pass and the assigned maintainer has the final say. |
dcb9597 to
7e73113
Compare
A 401 or 403 may carry several `WWW-Authenticate` challenges, in one field or across repeated fields (RFC 9110 §11.6.1). The reqwest and Unix socket Streamable HTTP clients read only the first field, so a `Basic` challenge from a proxy hid the `Bearer` challenge that carries `resource_metadata` and `scope`, and the challenge parsers scanned the raw string for `scope=` and friends, matching text inside quoted values and parameters of other schemes. Combine every `WWW-Authenticate` field in order with `", "`, which RFC 9110 §5.3 defines as equivalent to the separate fields, so the existing string API carries all challenges. Parse that string with one RFC 9110 §11 challenge parser shared by the transport's `scope` extraction and the auth manager's `WWW-Authenticate` parameter extraction: challenge boundaries, `auth-param` vs `token68`, quoted-string unescaping, and case-insensitive scheme and parameter names. Parameters are read from `Bearer` challenges only and never borrowed across challenges. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
7e73113 to
338f908
Compare
There was a problem hiding this comment.
Community review. gocamille has the final call. I've been working on SEP-1932 (DPoP) in ext-auth, conformance and the TS/Python SDKs, so I'll answer the open question about non-Bearer schemes from that angle.
The parser is a clear improvement. A single RFC 9110 challenge parser replaces two substring scanners, the tests cover the cases that broke before (quoted text, foreign schemes, token68 padding), and keeping the ; and missing-comma leniencies on purpose is the right call.
On reading resource_metadata only from Bearer challenges (I'd treat this as blocking, but it's your and the maintainers' call):
The old substring scanner found resource_metadata in any scheme by accident. That included a DPoP challenge, so DPoP algs="ES256", resource_metadata="https://…/prm" worked on main. With this PR it doesn't. That matters because the SEP-1932 server rules in modelcontextprotocol/ext-auth#38 have a server that requires DPoP-bound tokens answer 401 with a Bearer or DPoP challenge. RFC 9728 §5.1 doesn't limit resource_metadata to Bearer. A DPoP-only server would then send only a DPoP challenge, and an rmcp client would stop finding its metadata even though rmcp doesn't speak DPoP yet. The same goes for a DPoP-only 403 insufficient_scope: on main it yields scope and error, so scope upgrade starts; with this PR it yields neither. (DPoP support for rmcp is being proposed separately.)
Suggestion: read parameters from the challenges of schemes the client can act on, not from Bearer alone. Today that's Bearer first, then DPoP. Basic and the other proxy schemes from the motivating case stay excluded. A bearer_challenges that takes a list of schemes (or a challenges_for(&[..])) would do it, along with one test case: DPoP resource_metadata="/prm" → found. I'm happy to send that as a small follow-up instead if you'd rather keep this PR Bearer-only.
Non-blocking:
- The branch conflicts with
maininunix_socket.rs, imports only:3e0156c(bounded response bodies) addedTryStreamExt, and this PR dropsheader::WWW_AUTHENTICATE. Keeping both resolves it; with that resolution the repeated-challenge tests (68) pass on top of currentmain. test_streamable_http_repeated_auth_challengesdrives the reqwest client only. The Unix-socket backend gets the same change without an end-to-end test.- Looking ahead:
Challengeandparse_challengesarepub(crate). A DPoP client will need the same parser forDPoPchallenges (algs,error="use_dpop_nonce"), and a server needs to build challenges. Nothing to change now. I'm only noting it so the type stays easy to make public later.
Evidence. The same probe test, run through the public WWWAuthenticateParams::parse on main (53e64a9) and on this branch (338f908), base https://mcp.example.com/mcp:
| Challenge | main |
this PR |
|---|---|---|
Bearer resource_metadata=…, scope=… |
metadata + scope | metadata + scope |
DPoP algs="ES256", resource_metadata=…, scope=… |
metadata + scope | nothing |
DPoP algs="ES256", error="insufficient_scope", scope="mcp:write" |
scope + error | nothing |
DPoP algs="ES256", resource_metadata=…, Bearer scope=… |
metadata + scope | scope only |
Basic realm="proxy", scope="basic", Bearer resource_metadata=…, scope="mcp:read" |
scope basic (wrong) |
scope mcp:read (fixed) |
Bearer error_description="scope=wrong", …, scope="right" |
scope wrong" (wrong) |
scope right (fixed) |
AI disclosure: this review was drafted with Claude Code, which also ran the probe above on both branches and checked the conflict resolution against the PR's tests.
Generated by Claude Code
Problem
The reqwest and Unix socket Streamable HTTP clients read
WWW-AuthenticatewithHeaderMap::get, which returns only the first field. A 401 or 403 whose first field is another scheme's challenge (for example aBasicchallenge added by a proxy) loses theBearerchallenge that carriesresource_metadataandscope, so the reactive OAuth flow cannot start from the real connection's response.The challenge parsers compounded this:
extract_scope_from_headerandAuthorizationManager::extract_www_authenticate_paramssearched the raw string forscope=,error=,resource_metadata=, so they matched text inside quoted values (error_description="scope=…") and parameters of other schemes (Basic scope="…").Changes
WWW-Authenticatefield is kept, in order, joined with", "into the existingStringAPI (AuthRequiredError::www_authenticate_header,InsufficientScopeError::www_authenticate_header). No public type changes. The same shared helper now serves both the reqwest and the Unix socket backend.parse_challenges/bearer_challengesintransport/common/http_header.rs) replaces the two substring scanners: challenge boundaries,auth-paramvstoken68, quoted-string unescaping, BWS around=, case-insensitive scheme and parameter names.Bearerchallenges only, never from another scheme's challenge or from text inside a quoted value. Each parameter takes its first occurrence across theBearerchallenges in field order, which is the rule the TypeScript SDK (extractFieldFromWwwAuth, first regex match) and the Python SDK (extract_field_from_www_auth, first regex match) apply, minus their false positives on quoted text and foreign schemes.resource_metadatatakes the first URL the existing metadata host policy accepts.;is accepted as a parameter separator (realm="x"; scope="y", see fix(oauth): three oauth discovery and registration issues #489), and parameters separated by whitespace alone (error="insufficient_scope" scope="x") are still read.AuthorizationManager::parse_next_header_valueis removed; its escape handling lives in the shared parser and its tests moved there.Before / after on the wire
Reproduction: a minimal MCP server (rmcp's own Streamable HTTP server behind a token gate) and a client probe that prints what the SDK hands back, committed on this fork at
tadasant/rust-sdk@5b3f78a(README). It is not a workspace member and depends on../crates/rmcpby path, so the same binaries run against any checkout. Captured output: before, upstreammainc164688 · after, this PR 338f908. Ports and the worker type path are elided below.Scenario
basic-then-bearer. A proxy addsWWW-Authenticate: Basic realm="proxy"ahead of the MCP server's own field: on 401Bearer resource_metadata="…/.well-known/oauth-protected-resource", scope="mcp:read", on 403Bearer error="insufficient_scope", scope="mcp:write".Before (upstream
main):sequenceDiagram participant C as rmcp client participant S as MCP server behind proxy C->>S: POST /mcp initialize (no token) S-->>C: 401<br/>WWW-Authenticate: Basic realm="proxy"<br/>WWW-Authenticate: Bearer resource_metadata="…", scope="mcp:read" Note over C: HeaderMap::get reads the first field only C->>C: AuthRequired { www_authenticate_header: "Basic realm=\"proxy\"" } C->>C: WWWAuthenticateParams::parse → resource_metadata_url=None, scope=None Note over C: nothing to follow, discovery can only guess well-known URIs<br/>scope hint lost C->>S: POST /mcp (under-scoped token) S-->>C: 403<br/>WWW-Authenticate: Basic realm="proxy"<br/>WWW-Authenticate: Bearer error="insufficient_scope", scope="mcp:write" C->>C: InsufficientScope { required_scope: None } Note over C: step-up cannot compute the scope unionAfter (this PR):
sequenceDiagram participant C as rmcp client participant S as MCP server behind proxy C->>S: POST /mcp initialize (no token) S-->>C: 401<br/>WWW-Authenticate: Basic realm="proxy"<br/>WWW-Authenticate: Bearer resource_metadata="…", scope="mcp:read" Note over C: get_all, joined with ", " (RFC 9110 §5.3) C->>C: AuthRequired { "Basic realm=\"proxy\", Bearer resource_metadata=\"…\", scope=\"mcp:read\"" } C->>C: WWWAuthenticateParams::parse → resource_metadata_url=…/.well-known/oauth-protected-resource, scope="mcp:read" C->>S: GET /.well-known/oauth-protected-resource (RFC 9728 §5.1) S-->>C: protected resource metadata C->>S: POST /mcp (under-scoped token) S-->>C: 403 (same two fields, Bearer error="insufficient_scope", scope="mcp:write") C->>C: InsufficientScope { required_scope: Some("mcp:write") }Scenario
quoted-text. OneBearerfield whoseerror_descriptionquotes parameter-like text:Bearer error_description="use scope=mcp:read or resource_metadata=/elsewhere", resource_metadata="…/.well-known/oauth-protected-resource", scope="mcp:read"on 401, andBearer error="insufficient_scope", error_description="scope=mcp:read is not enough", scope="mcp:write"on 403.main(c164688)basic-then-bearer,initializewith no token →auth_challenge()Basic realm="proxy"Basic realm="proxy", Bearer resource_metadata="…/.well-known/oauth-protected-resource", scope="mcp:read"WWWAuthenticateParams::parseresource_metadata_url=None scope=Noneresource_metadata_url=Some(…/.well-known/oauth-protected-resource) scope=Some("mcp:read")basic-then-bearer, POST and GET with under-scoped tokenInsufficientScope { required_scope: None, www_authenticate_header: "Basic realm=\"proxy\"" }InsufficientScope { required_scope: Some("mcp:write"), www_authenticate_header: "Basic realm=\"proxy\", Bearer error=\"insufficient_scope\", scope=\"mcp:write\"" }quoted-text,initializewith no token →WWWAuthenticateParams::parseresource_metadata_url=Some("http://…/elsewhere%22")(text insideerror_description, with the closing quote swallowed)scope=Some("mcp:read")resource_metadata_url=Some(…/.well-known/oauth-protected-resource) scope=Some("mcp:read")quoted-text, POST and GET with under-scoped tokenInsufficientScope { required_scope: Some("mcp:read") }(fromerror_description)InsufficientScope { required_scope: Some("mcp:write") }Unchanged in both revisions:
is_authorization_required()istrueon the 401, and theno-challenge*scenarios (the subject of #1419) behave the same with and without this PR.How the fix maps to the specs
www_authenticate_headerreadsget_all, notget; the parser splits challenges and their parameter lists.tests/test_streamable_http_repeated_auth_challenges.rs::retains_every_auth_challenge(GET and POST, 401 and 403, 13 header layouts)", "in received order. This is why the existingStringfield can carry all challenges without an API change.http_header::tests::www_authenticate_header_joins_repeated_fields_in_order; the integration test asserts the joined string equals the fields joined with", ".Challenge::is_bearerandChallenge::paramcompare witheq_ignore_ascii_case;paramreturns the first occurrence.parse_challenges_follows_the_grammar::case_insensitive_lookup,extract_scope_keeps_challenges_apart::case_insensitive,auth::tests::extract_www_authenticate_params_reads_bearer_challenges::case_insensitive_namesauth-param = token BWS "=" BWS ( token / quoted-string ),token68 = 1*( ALPHA / DIGIT / "-" / "." / "_" / "~" / "+" / "/" ) *"="; "Authentication scheme definitions need to accept both notations" (link)split_auth_paramaccepts BWS around=and both value forms;is_token68keeps a base64 blob's=padding from being read as a parameter.parse_challenges_follows_the_grammar::{bad_whitespace_around_equals, token68_is_not_a_parameter, token68_padding_then_params, unquoted_url_and_semicolon_separator}unquoteresolves\xtoxand ends the value at the first unescaped DQUOTE;split_listdoes not split on commas inside a quoted-string.parse_challenges_rfc9110_example(the RFC's owntitle="Login to \"apps\""example),parse_challenges_follows_the_grammar::quoted_comma_and_scheme_names,auth::tests::extract_www_authenticate_params_unescapes_quoted_values, integration caseescaped_quotes_and_case_insensitive_namesparse_challenges_follows_the_grammar::empty_membersBearerchallenges are consulted forscope,error,error_descriptionandresource_metadata; aBasic scope="…"is ignored. Scope values are passed through untouched.extract_scope_keeps_challenges_apart::{other_scheme_first, only_other_scheme},auth::tests::extract_www_authenticate_params_ignores_other_schemes, integration casesbasic_field_before_bearer_field,basic_only,no_bearer_scopeinsufficient_scope: "The resource server SHOULD respond with the HTTP 403 (Forbidden) status code and MAY include the "scope" attribute with the scope necessary to access the protected resource." and §3: the server "SHOULD include the "error" attribute to provide the client with the reason why the access request was declined" (§3.1)required_scopeis the firstBearerscope, andWWWAuthenticateParams::is_insufficient_scope()sees anerror="insufficient_scope"in anyBearerchallenge, not only in the first field.extract_scope_keeps_challenges_apart::{first_bearer_scope_wins, later_bearer_challenge},auth::tests::extract_www_authenticate_params_finds_error_in_any_bearer_challenge, integration casesfirst_bearer_scope_wins,bearer_scope_in_a_later_fieldresource_metadatais read fromBearerchallenges wherever they sit in the field list; the first URL that passes the SDK's metadata host policy is used, as before.auth::tests::extract_www_authenticate_params_reads_bearer_challenges,…_skips_rejected_metadata_url_and_tries_the_nextWWW-Authenticateheaders when present" and "MCP clients MUST be able to parseWWW-Authenticateheaders and respond appropriately toHTTP 401 Unauthorizedresponses from the MCP server." (2025-11-25, unchanged in 2026-07-28)Bearerchallenge occupies, so the header-based mechanism works behind proxies that add their own challenge. Note the spec's plural "headers".basic_field_before_bearer_field(401 branch asserts the full joined header reachesAuthRequiredError),auth::tests::…::other_scheme_first,…::repeated_fields_joinedscopeis read from oneBearerchallenge and never assembled from text inside quoted values or from other schemes.extract_scope_keeps_challenges_apart::ignores_quoted_text,auth::tests::…::quoted_text_is_not_a_parameter, integration casesquoted_commas,quoted_text_is_not_a_parameterWWW-Authenticateheader" (2025-11-25, 2026-07-28)WWWAuthenticateParams::is_insufficient_scope()is true when anyBearerchallenge in the response sayserror="insufficient_scope", not only when the first field does, and the 403required_scopefollows the same first-occurrence rule, so the two extractors agree on one header.auth::tests::extract_www_authenticate_params_finds_error_in_any_bearer_challenge,auth::tests::…::semicolon_separator_and_missing_comma, integration casesemicolon_and_missing_commasWhere the specs are silent
Bearerchallenges in one response. Neither RFC 6750 nor the MCP spec says which one a client should act on; both describe a singleBearerchallenge carrying all parameters. Taking each parameter's first occurrence acrossBearerchallenges is a judgment call, chosen because it is what the previous code, the TypeScript SDK and the Python SDK effectively do, so no deployment that worked before loses a parameter. It can pair ascopefrom oneBearerchallenge with anerrorfrom another; a per-challenge selection rule would avoid that but needs an arbitrary ranking and givesrequired_scopeandWWWAuthenticateParams::parsedifferent answers on the same header.resource_metadataon non-Bearerschemes. RFC 9728 §5.1 says the parameter "MAY also be used in WWW-Authenticate responses using authorization schemes other than "Bearer", such as the DPoP scheme". The SDK only sendsBearertokens and the MCP spec only describesBearerchallenges, so other schemes are not consulted. This is an observable change toWWWAuthenticateParams::parsefor aDPoP resource_metadata="…"challenge or a scheme-lessresource_metadata="…"value, both of which the old scanner accepted.obs-textinside a quoted-string, but theStringchallenge API cannot carry such bytes. Behaviour is unchanged: any such field still yieldsUnexpectedServerResponse("invalid www-authenticate header value"), now whichever field it occurs in (rejects_invalid_auth_header_values).Relationship to #1419
#1419 (challenge-less 401/403 as
HttpStatusError) touches the same 401/403 region oftransport/common/reqwest/streamable_http_client.rs. The two are independent: this PR replaces the challenge-present branches with the sharedauth_challenge_errorhelper, #1419 changeshttp_status_errorand the POST body path below them. Whichever lands second needs a small rebase. Together they give: challenge present →AuthRequired/InsufficientScopecarrying every field; no challenge →HttpStatus.Validation
Run locally on Rust 1.96 (the pinned toolchain) with the commands CI uses:
cargo clippy --all-targets --all-features -- -D warningscargo clippy -p rmcp --lib --no-default-features --features <f> -- -D warningsforauth,client-side-sse,transport-streamable-http-client-reqwest,transport-streamable-http-client-unix-socket,client, and no featuresparse_challenges,extract_scope_from_headerandWWWAuthenticateParams::parsewith 300k random strings over",\,,,;,=, tabs and multi-byte characters: no panics. It also caught the;and missing-comma regressions, which is why those leniencies are now explicit and tested.cargo test -p rmcp --features client,auth,transport-streamable-http-client-reqwest,transport-streamable-http-client-unix-socket --lib --test test_streamable_http_repeated_auth_challenges --test test_streamable_http_get_stream_auth_challenge --test test_streamable_http_status_metadata --test test_streamable_http_4xx_error_body --test test_auth_error_classification --test test_unix_socket_transportcargo +nightly fmt --all -- --check,typos,git diff --checkOn unmodified
main, every multi-field case ofretains_every_auth_challengefails and the single-field cases pass, which confirms the defect is the first-field read.Scope / review notes
AuthorizationManager::handle_responseandextract_www_authenticate_challengealready iteratedget_all; they now parse each field with the shared parser.Noticed, not filed
classify_reconnect_errortreats a challenge-less 401/403 on the GET stream as retryable; repeating the GET with the same credentials cannot help. Left alone here, same behaviour as before.🤖 Generated with Claude Code