SMART Backend Services: System-to-System FHIR Authorization in .NET 10

A nightly reporting job, a laboratory feed, a demographics synchronisation: much of the traffic into a FHIR server comes from software with no person at the keyboard. There is nobody to sign in and nobody to consent. In practice these services are often given a shared API key or a client secret that never expires, with access to everything, because that is the quickest thing to set up.

SMART Backend Services is the standard alternative. The service proves its identity by signing a short-lived statement with a private key that never leaves it. In exchange it receives an access token that lasts five minutes and names exactly which resource types it may read or write. Nothing that crosses the network is worth stealing for longer than those five minutes.

The earlier articles in this series, starting with FHIR API Security Part 1, covered the threat landscape and sender-constrained tokens. This one implements the Backend Services profile end to end in .NET 10: the client, a local token endpoint, and a gateway that enforces scopes in front of a HAPI FHIR server. Most of the tests are about what has to be refused, because that is where implementations of this profile go wrong.

What the profile requires

The specification is short, and almost every sentence in it is a test case. These are the rules the sample implements, in the words of the two relevant pages of the guide.

Architecture: decide twice, and keep the FHIR server out of reach

HAPI FHIR, like most FHIR servers, will answer any request that reaches it. It has no built-in notion of a SMART scope. The design therefore has three parts and one network rule: a client library, a token endpoint, a gateway that every FHIR request passes through, and a FHIR server that accepts connections from the gateway and from nothing else.

Authorisation is decided in two places on purpose. The token endpoint decides what a client may ever ask for, from its registration. The gateway decides whether one request fits inside the token it carries. A token is a claim about a client; only the gateway knows what a particular request will make the FHIR server do.

The sample has no package references. Signatures use the RSA and ECDSA classes in .NET, limited to the two algorithms the profile names. That keeps every check visible in a few hundred lines, which suits an article. It is not advice to write your own token endpoint: use your identity platform and its JWT library, and use the test list here as an acceptance test for them.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','signalColor':'#0073BB','signalTextColor':'#0F172A','actorBkg':'#FFF3E0','actorBorder':'#FF9900','actorTextColor':'#232F3E','noteBkgColor':'#FFF8E7','noteBorderColor':'#FF9900','noteTextColor':'#232F3E'}}}%%
sequenceDiagram
    participant C as Backend service
    participant T as Token endpoint
    participant G as FHIR gateway
    C->>G: GET .well-known/smart-configuration
    G-->>C: Token endpoint, algorithms, capabilities
    C->>C: Sign a one-time assertion
    C->>T: POST client_credentials with the assertion
    T->>T: Verify key, signature, audience, expiry, jti, scope
    T-->>C: Access token, 300 seconds, granted scope
    C->>G: GET Patient with Bearer token
    G->>G: Verify token, work out what the request needs
    G-->>C: 200 with data, or 403 naming the missing scope

The client: an assertion, a cached token, one host

The assertion is a JWT with five claims, valid here for two minutes and used once.

    public async Task<string> BuildAssertionAsync(Uri tokenEndpoint, CancellationToken cancel = default)
    {
        DateTimeOffset now = clock.GetUtcNow();
        JsonObject header = new() { ["alg"] = signer.Algorithm, ["kid"] = signer.KeyId, ["typ"] = "JWT" };
        JsonObject claims = new()
        {
            ["iss"] = options.ClientId,
            ["sub"] = options.ClientId,
            ["aud"] = tokenEndpoint.ToString(),
            ["exp"] = now.Add(options.AssertionLifetime).ToUnixTimeSeconds(),
            ["jti"] = Guid.NewGuid().ToString("N"),
        };
        string input = Jose.SigningInput(header, claims);
        return Jose.Compact(input, await signer.SignAsync(Encoding.ASCII.GetBytes(input), cancel));
    }

Signing goes through an interface with one method. That is the most important line of the client’s design: the private key does not have to be in the process. A key vault, a cloud key management service or a hardware module can hold a key that cannot be exported, sign the bytes it is given and return the signature. The sample’s signer holds the key in memory, which is right for a test and wrong for production.

public interface IAssertionSigner
{
    string Algorithm { get; }
    string KeyId { get; }
    ValueTask<byte[]> SignAsync(ReadOnlyMemory<byte> data, CancellationToken cancel);
}

A token is fetched once and shared. Without the gate, twenty requests starting at the same moment on a cold service would each build an assertion and each call the token endpoint.

    public async Task<AccessToken> GetTokenAsync(CancellationToken cancel = default)
    {
        if (Usable(_token)) { return _token!; }
        // One caller fetches; the rest wait and then find the token in the cache.
        await _gate.WaitAsync(cancel);
        try
        {
            if (Usable(_token)) { return _token!; }
            _tokenEndpoint ??= await DiscoverAsync(cancel);
            _token = await RequestTokenAsync(_tokenEndpoint, await BuildAssertionAsync(_tokenEndpoint, cancel), cancel);
            return _token;
        }
        finally { _gate.Release(); }
    }

The handler that attaches the token has two rules. It sends the token to the FHIR base URL and to no other host, because a FHIR resource can contain a reference to any address and following one must not hand over a credential. And when the server answers 401, it discards the token, gets one new token and tries once more.

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancel)
    {
        Uri fhir = client.Options.FhirBase;
        // "Ours" is the FHIR base and everything under it. A server writes its paging links as [base]?_getpages=..., without the slash.
        bool ours = request.RequestUri is Uri uri && uri.Scheme == fhir.Scheme && uri.Authority == fhir.Authority && $"{uri.AbsolutePath}/".StartsWith(fhir.AbsolutePath, StringComparison.Ordinal);
        if (!ours) { return await base.SendAsync(request, cancel); }

        if (request.Content is not null) { await request.Content.LoadIntoBufferAsync(cancel); }      // so the request can be sent twice
        AccessToken token = await client.GetTokenAsync(cancel);
        request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token.Value);
        HttpResponseMessage response = await base.SendAsync(request, cancel);
        if (response.StatusCode != System.Net.HttpStatusCode.Unauthorized) { return response; }

        // The token was refused: expired early, revoked, or the server restarted with new keys. Get one new token and try once.
        response.Dispose();
        client.Invalidate(token);
        token = await client.GetTokenAsync(cancel);
        request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token.Value);
        return await base.SendAsync(request, cancel);
    }

The client also checks what it was granted. The server is allowed to grant less than was asked for, and a service that discovers this at its first 403 in the middle of a batch is harder to operate than one that fails at start-up with the two scope lists side by side.

The token endpoint: every check, in order

This is the whole of client authentication. The order matters as much as the checks.

    private async Task<(ClientRegistration? Client, string Claimed, string? Failure)> AuthenticateAsync(string assertion, CancellationToken cancel)
    {
        if (Jose.Parse(assertion) is not ParsedJws jws) { return (null, "?", "the assertion is not a compact JWS"); }
        string issuer = jws.ClaimString("iss") ?? "?";

        // The algorithm comes from a fixed list, never from trust in the header.
        string? algorithm = jws.HeaderString("alg");
        if (algorithm is not (Jose.RS384 or Jose.ES384)) { return (null, issuer, $"alg '{algorithm}' is not RS384 or ES384"); }
        if (jws.HeaderString("typ") != "JWT") { return (null, issuer, "typ is not JWT"); }
        if (jws.HeaderString("kid") is not string keyId) { return (null, issuer, "no kid in the header"); }

        if (issuer != jws.ClaimString("sub")) { return (null, issuer, "iss and sub differ"); }
        if (!clients.TryGetValue(issuer, out ClientRegistration? client)) { return (null, issuer, "unknown client"); }
        if (jws.HeaderString("jku") is string jku && jku != client.JwksUri?.ToString()) { return (null, issuer, "jku is not the registered JWK Set URL"); }

        // Exactly one key with this ID and of the type the algorithm needs. The key set is the registered one: the token cannot bring its own.
        AsymmetricAlgorithm? key = await FindKeyAsync(client, keyId, algorithm, refresh: false, cancel) ?? await FindKeyAsync(client, keyId, algorithm, refresh: true, cancel);
        if (key is null) { return (null, issuer, $"no single {algorithm} key with kid '{keyId}' in the registered key set"); }
        using (key)
        {
            if (!Jose.Verify(key, algorithm, jws.SigningInput, jws.Signature)) { return (null, issuer, "the signature does not verify"); }
        }

        DateTimeOffset now = clock.GetUtcNow();
        if (!jws.Audiences.Contains(tokenEndpoint.ToString())) { return (null, issuer, "aud is not this token endpoint"); }
        if (jws.ClaimNumber("exp") is not long seconds) { return (null, issuer, "no exp"); }
        DateTimeOffset expires = DateTimeOffset.FromUnixTimeSeconds(seconds);
        if (expires <= now - Skew) { return (null, issuer, "the assertion has expired"); }
        if (expires > now + MaximumAssertionLifetime + Skew) { return (null, issuer, "exp is more than five minutes away"); }
        if (jws.ClaimNumber("nbf") is long notBefore && DateTimeOffset.FromUnixTimeSeconds(notBefore) > now + Skew) { return (null, issuer, "the assertion is not valid yet"); }

        // One use only. The entry is kept until the assertion could no longer be accepted anyway.
        if (jws.ClaimString("jti") is not { Length: > 0 and <= 128 } id) { return (null, issuer, "no usable jti"); }
        foreach ((var entry, DateTimeOffset until) in _seen) { if (until < now) { _seen.TryRemove(entry, out _); } }
        if (!_seen.TryAdd((issuer, id), expires + Skew)) { return (null, issuer, "this assertion was used before"); }

        return (client, issuer, null);
    }

Three decisions in that method are the ones most often got wrong.

• The algorithm comes from a fixed list. A verifier that does whatever the alg header says can be handed a token marked none, or one marked as HMAC and “signed” with the client’s public key, which is public. Both are in the test list below.
• The key comes from the registration. The assertion may carry a jku header naming a key set URL. If the server fetched whatever URL the token named, anyone could sign as any client. It must match the registered URL or the request is refused, without a fetch.
• The signature is verified before the replay cache is touched. If jti values were recorded first, an unauthenticated caller could fill the cache, or burn a real client’s identifiers before it used them.

Every failure returns the same thing to the caller: HTTP 401 and invalid_client, with no description. The reason goes to the audit log. These are the twenty-two requests the test sends, with the response and the audit line for each.

  refused 401 invalid_client         an assertion signed with a key that is not the registered one
             audit: invalid_client: the signature does not verify
  refused 401 invalid_client         alg none with an empty signature
             audit: invalid_client: alg 'none' is not RS384 or ES384
  refused 401 invalid_client         alg HS384, signed with the public key as if it were a shared secret
             audit: invalid_client: alg 'HS384' is not RS384 or ES384
  refused 401 invalid_client         alg RS384 in the header of an assertion signed with an EC key
             audit: invalid_client: no single RS384 key with kid 'bbe9pQ3R4gLmOE8D8AqHhySUDDTPi15yci-fuvH9k8s' in the registered key set
  refused 401 invalid_client         a key ID that is not in the registered key set
             audit: invalid_client: no single ES384 key with kid 'some-other-key' in the registered key set
  refused 401 invalid_client         no typ in the header
             audit: invalid_client: typ is not JWT
  refused 401 invalid_client         a jku header pointing at a key set the caller controls
             audit: invalid_client: jku is not the registered JWK Set URL
  refused 401 invalid_client         a client that was never registered
             audit: invalid_client: unknown client
  refused 401 invalid_client         iss and sub that differ
             audit: invalid_client: iss and sub differ
  refused 401 invalid_client         an audience that is the FHIR base, not the token endpoint
             audit: invalid_client: aud is not this token endpoint
  refused 401 invalid_client         an assertion that expired a minute ago
             audit: invalid_client: the assertion has expired
  refused 401 invalid_client         an assertion valid for ten minutes
             audit: invalid_client: exp is more than five minutes away
  refused 401 invalid_client         an assertion with no exp
             audit: invalid_client: no exp
  refused 401 invalid_client         an assertion with no jti
             audit: invalid_client: no usable jti
  refused 401 invalid_client         the same assertion presented a second time
             audit: invalid_client: this assertion was used before
  refused 401 invalid_client         a payload changed after signing
             audit: invalid_client: the signature does not verify
  refused 401 invalid_client         text that is not a JWT
             audit: invalid_client: the assertion is not a compact JWS
  refused 400 unsupported_grant_type grant_type authorization_code
             audit: unsupported_grant_type: grant_type is not client_credentials
  refused 400 invalid_request        no client assertion
             audit: invalid_request: client assertion missing or of the wrong type
  refused 400 invalid_scope          a v1 scope, system/*.read
             audit: invalid_scope: 'system/*.read' is not a list of SMART v2 system scopes
  refused 400 invalid_scope          a patient-level scope
             audit: invalid_scope: 'patient/Patient.rs' is not a list of SMART v2 system scopes
  refused 400 invalid_scope          a scope outside the registration
             audit: invalid_scope: nothing in 'system/Observation.rs' is within the registered 'system/Patient.rs'

The last three are scope errors, and they do carry a description, because by then the client has proved who it is and needs to know what to fix.

Scopes: what may be asked for

A v2 system scope is a resource type, or *, and a subset of the letters cruds in that order: create, read, update, delete, search. Two properties surprise people who come from the older read and write scopes. Writing does not imply reading, so a loader with Observation.cu cannot read back what it wrote. And reading does not imply searching, so a service with Patient.r can fetch a patient it knows the id of and cannot go looking for others.

A request is granted the part of it that lies within the registration. A request with nothing inside the registration is an error.

    public ScopeSet GrantFrom(ScopeSet registered)
    {
        ScopeSet granted = new();
        foreach ((string type, HashSet<char> letters) in _byType)
        {
            // A request for every resource type is only met by a registration for every resource type.
            string allowed = string.Concat(letters.Where(letter => type == "*" ? registered._byType.TryGetValue("*", out HashSet<char>? any) && any.Contains(letter) : registered.Allows(type, letter)));
            if (allowed.Length > 0) { granted.Add(type, allowed); }
        }
        return granted;
    }

Against a client registered for system/Patient.rs system/Observation.rs:

  asked system/Patient.rs                            -> system/Patient.rs
  asked system/Patient.cruds                         -> system/Patient.rs
  asked system/Patient.r system/Patient.s            -> system/Patient.rs
  asked system/Encounter.rs                          -> (nothing)
  asked system/*.rs                                  -> (nothing)
  asked system/Observation.s system/Encounter.r      -> system/Observation.s
  asked system/*.read                                -> not a v2 system scope
  asked patient/Patient.rs                           -> not a v2 system scope
  asked system/Patient.sr                            -> not a v2 system scope
  asked system/Patient.                              -> not a v2 system scope
  asked system/Observation.rs?category=laboratory    -> not a v2 system scope
  asked openid                                       -> not a v2 system scope

The fifth row deserves a second look. A request for system/*.rs from a client registered for two resource types is granted nothing. It would be easy to grant “the two types you do have”, and it would be wrong: a wildcard is a request for every type that exists now or is added later, and a token carrying it would be honoured for all of them.

Key rotation, and how long a removed key lives

With a key set URL the client rotates keys without telling anyone: it publishes the new key beside the old one, starts signing with the new one, and later removes the old one. The server’s side of that is a cache with two rules.

    public async Task<JsonObject?> GetAsync(Uri uri, bool unknownKeyId, CancellationToken cancel)
    {
        DateTimeOffset now = clock.GetUtcNow();
        if (_cache.TryGetValue(uri, out var cached))
        {
            // An unknown key ID earns a refetch, but not one per request: that would let anyone make this server hammer the client.
            bool refresh = unknownKeyId ? now - cached.FetchedAt >= MinimumBetweenRefreshes : now >= cached.FreshUntil;
            if (!refresh) { return cached.Jwks; }
        }
        try
        {
            Fetches++;
            using HttpResponseMessage response = await http.GetAsync(uri, cancel);
            if (!response.IsSuccessStatusCode || JsonNode.Parse(await response.Content.ReadAsStringAsync(cancel)) is not JsonObject jwks) { return null; }
            CacheControlHeaderValue? control = response.Headers.CacheControl;
            TimeSpan lifetime = control is { NoStore: true } or { NoCache: true } ? TimeSpan.Zero : control?.MaxAge ?? DefaultLifetime;
            _cache[uri] = (jwks, now, now + (lifetime < Ceiling ? lifetime : Ceiling));
            return jwks;
        }
        catch (Exception error) when (error is HttpRequestException or TaskCanceledException or System.Text.Json.JsonException) { return null; }
    }

An unknown key ID is a reason to fetch the key set again, since the client may have rotated. It is also something any caller can produce at will, so a refetch is allowed at most once in ten seconds per client. From the run:

== key rotation
ok    a second token for the same client does not fetch its key set again
ok    an unknown key ID within ten seconds of the last fetch is refused without another fetch, however often it is tried
ok    after that, an unknown key ID triggers one fetch and the new key is accepted
ok    a key removed from the published set is still accepted while the cached set is fresh
ok    once the cache lifetime the client asked for has passed, the removed key is refused

The fourth check is the operational fact to take away. Removing a key from the published set does not revoke it. The server goes on accepting it until its cached copy expires, which is as long as the client’s own Cache-Control header said. In the test that is 300 seconds. If a private key is believed stolen, removing it from the key set is the slow path; the fast path is disabling the client’s registration at the token endpoint, and that needs to be a documented procedure with a named owner.

The gateway: what a request needs

A valid token with system/Patient.rs arrives with GET Patient?_revinclude=Observation:subject. The path says Patient. The token says Patient. A gateway that compares the two forwards the request, and the FHIR server returns the patients together with every Observation that refers to them.

That is the central problem of scope enforcement on FHIR. The resource type in the path is the type the search starts from. Search parameters can pull in other types, or answer questions about them, and each of those needs its own permission.

    private static string? SearchNeeds(IReadOnlyList<KeyValuePair<string, string>> query, List<Need> needs)
    {
        foreach ((string key, string value) in query)
        {
            string name = key.Split(':')[0];
            switch (name)
            {
                case "_include":
                    // Patient?_include=Patient:general-practitioner:Practitioner returns Practitioner resources.
                    if (value == "*") { needs.Add(new Need("*", 'r')); break; }
                    string[] include = value.Split(':');
                    if (include.Length < 3 || !ResourceType().IsMatch(include[2])) { return $"_include must name its target type, as Source:parameter:Type ('{value}')"; }
                    needs.Add(new Need(include[2], 'r'));
                    break;
                case "_revinclude":
                    // Patient?_revinclude=Observation:subject returns every Observation that points at the patients found.
                    if (value == "*") { needs.Add(new Need("*", 'r')); break; }
                    string source = value.Split(':')[0];
                    if (!ResourceType().IsMatch(source)) { return $"_revinclude '{value}' does not name a resource type"; }
                    needs.Add(new Need(source, 'r'));
                    break;
                case "_has":
                    // Patient?_has:Observation:subject:code=X answers a question about Observations.
                    string[] has = key.Split(':');
                    if (has.Length != 4 || !ResourceType().IsMatch(has[1])) { return $"'{key}' is not a single-level _has"; }
                    needs.Add(new Need(has[1], 's'));
                    break;
                case "_type":
                    foreach (string type in value.Split(',')) { if (!ResourceType().IsMatch(type)) { return "_type lists something that is not a resource type"; } needs.Add(new Need(type, 's')); }
                    break;
                case "_filter" or "_query":
                    return $"{name} is not exposed through this gateway";
                default:
                    if (!key.Contains('.')) { break; }
                    // Observation?subject:Patient.name=Smith answers a question about Patients.
                    if (TypedChain().Match(key) is not { Success: true } chain) { return $"a chained parameter must name its type and go one level deep, as subject:Patient.name ('{key}')"; }
                    needs.Add(new Need(chain.Groups[1].Value, 's'));
                    break;
            }
        }
        return null;
    }

The classifier turns a request into the list of permissions it needs, or refuses it. These are the twenty-six shapes in the test, with what each needs.

  GET    Patient/123                                                    system/Patient.r
  GET    Patient/123/_history/2                                         system/Patient.r
  GET    Patient?family=Byrne                                           system/Patient.s
  GET    Patient/_history                                               system/Patient.s
  GET    Patient?_revinclude=Observation:subject                        system/Patient.s system/Observation.r
  GET    Patient?_include=Patient:general-practitioner:Practitioner     system/Patient.s system/Practitioner.r
  GET    Patient?_include=Patient:general-practitioner                  refused: _include must name its target type, as Source:parameter:Type ('Patient:general-practitioner')
  GET    Patient?_include:iterate=*                                     system/Patient.s system/*.r
  GET    Observation?subject:Patient.family=Byrne                       system/Observation.s system/Patient.s
  GET    Observation?subject.family=Byrne                               refused: a chained parameter must name its type and go one level deep, as subject:Patient.name ('subject.family')
  GET    Patient?_has:Observation:subject:code=1234-5                   system/Patient.s system/Observation.s
  GET    Patient/123/Observation?code=1234-5                            system/Patient.r system/Observation.s
  GET    Patient/123/$everything                                        refused: operations are not exposed through this gateway
  GET    ?_type=Patient,Observation                                     system/*.s system/Patient.s system/Observation.s
  GET    Patient?_filter=name eq x                                      refused: _filter is not exposed through this gateway
  POST   Patient/_search                                                refused: search by POST is not exposed through this gateway; use GET
  POST   Observation                                                    system/Observation.c
  PUT    Patient/123                                                    system/Patient.u
  PUT    Patient?identifier=x|1                                         system/Patient.u system/Patient.s
  PATCH  Patient/123                                                    system/Patient.u
  DELETE Patient/123                                                    system/Patient.d
  DELETE Patient?identifier=x|1                                         system/Patient.d system/Patient.s
  POST   (base) Bundle of POST Observation, PUT Patient/123, GET Encounter?subject=Patient/123 system/Observation.c system/Patient.u system/Encounter.s
  POST   (base) Bundle of POST Patient                                  system/Patient.c system/Patient.s
  POST   (base) Bundle of POST Patient/$match                           refused: Bundle entry POST Patient/$match: operations are not exposed through this gateway
  POST   (base) Bundle of a Patient                                     refused: a POST to the base must be a batch or transaction Bundle

Some of the refusals are design decisions, and each has a reason.

• An _include must name its target type. Patient:general-practitioner can return a Practitioner, an Organization or a PractitionerRole. The gateway cannot grant permission for a type it cannot name, so the client has to say which one it wants.
• A chained search must name its type and go one level deep. Observation?subject:Patient.family=Byrne is a search of patients by name, answered through observations, and needs Patient.s.
• Operations are not exposed. $everything returns a patient’s whole record across every resource type. An operation should be allowed one at a time, with its own rule, not by default.
• A transaction is judged by its entries. The guide says batches and transactions should be validated on the requests inside them. The gateway requires every entry’s permission and refuses the whole Bundle if one is missing.
• A conditional update or delete needs search as well. It finds its target by a query before it writes.

Then the same clients against HAPI FHIR, through the gateway:

== the gateway, in front of a FHIR server
ok    a client with system/*.cruds creates a patient and three observations through the gateway
ok    the Location header names the public address, not the server behind the gateway
ok    no token: 401 with a Bearer challenge
ok    a token whose scope was edited after signing: 401 invalid_token
ok    a client assertion presented as an access token: 401
ok    Patient.rs: read and search both succeed
ok    Patient.rs and Observation.rs: a search that pulls in the patient's observations returns all three
ok    Patient.rs only: reading a patient succeeds; searching observations is 403 and names the missing scope
  asked of the FHIR server directly, the same search returns 3 Observation resources
ok    Patient.rs only: the same patient search with _revinclude=Observation:subject is 403, though the FHIR server alone would answer it
ok    Patient.rs only: a chained search and a _has search that ask about observations are 403
ok    Patient.rs only: create, update and delete are 403
ok    Patient.r only: read succeeds and search is 403
ok    an operation and a search by POST are refused whatever the scope
ok    Observation.cu: create succeeds, and reading back what was just written is 403
ok    Observation.cu: a transaction of two observations succeeds; one that also updates a patient is 403 as a whole
ok    nothing from the refused transaction was written: six observations exist
ok    a paged search returns a next link at the public address, with a page token
ok    the client that ran the search can follow it
ok    another client, with every scope, cannot follow that link, and nobody can follow it without the page token
ok    a request to another host through the same HttpClient carries no token

The line without an “ok” in the middle is the control experiment. The test sends the _revinclude search straight to HAPI, bypassing the gateway, and HAPI returns the three observations. The server is behaving correctly; it was asked a valid question. Deciding who may ask it is not its job in this architecture, which is why the network rule matters as much as the code: if the FHIR server can be reached directly, none of the above applies.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','clusterBkg':'#F8FAFC','clusterBorder':'#CBD5E1','edgeLabelBackground':'#FFFFFF','nodeTextColor':'#232F3E'}}}%%
flowchart TD
    R["FHIR request"] --> T{"Token valid, for this
server, not expired?"}
    T -- "no" --> U["401 with a Bearer challenge"]
    T -- "yes" --> C["Work out every permission
the request needs"]
    C --> X{"Is the request shape
one the gateway exposes?"}
    X -- "no" --> F1["403 with the reason"]
    X -- "yes" --> S{"Does the token grant
all of them?"}
    S -- "no" --> F2["403 naming the
missing scope"]
    S -- "yes" --> W["Forward without the token,
rewrite addresses in the reply"]
    style U fill:#FEF2F2,stroke:#D13212,stroke-width:2px,color:#991B1B
    style F1 fill:#FEF2F2,stroke:#D13212,stroke-width:2px,color:#991B1B
    style F2 fill:#FEF2F2,stroke:#D13212,stroke-width:2px,color:#991B1B
    style W fill:#F0FFF4,stroke:#1D8102,stroke-width:2px,color:#145202
    style C fill:#FFF3E0,stroke:#FF9900,stroke-width:2px,color:#232F3E

Paging links are capabilities

A paged search returns a link to the next page. On HAPI FHIR that link is the base URL with a _getpages parameter holding a search identifier. It names no resource type, so there is nothing in it to check a scope against, and whoever holds the link can fetch the page. A client with a valid token for something else entirely could replay a link it was never given.

The gateway closes that in two steps. When it returns a search result it rewrites the server’s internal address to the public one and adds a page token: a keyed hash of the client ID, the token’s scope and the search identifier. When a paging request comes back, the page token must match the caller.

        // A next-page link is only good for the client, and the scopes, that ran the search it belongs to.
        if (method == "GET" && path.Length == 0 && query.FirstOrDefault(p => p.Key == "_getpages").Value is { Length: > 0 } pageId)
        {
            byte[] presented = Encoding.ASCII.GetBytes(query.FirstOrDefault(p => p.Key == "_pt").Value ?? ""), expected = Encoding.ASCII.GetBytes(PageToken(clientId, rawScope, pageId));
            if (!CryptographicOperations.FixedTimeEquals(presented, expected))
            {
                await DenyAsync(http, clientId, $"{method} ?_getpages", "this page link was not issued to this client with these scopes", null);
                return;
            }
            string paging = string.Join('&', query.Where(p => p.Key != "_pt").Select(p => $"{Uri.EscapeDataString(p.Key)}={Uri.EscapeDataString(p.Value)}"));
            audit(new AuditEvent(clock.GetUtcNow(), "fhir", clientId, "allowed", "GET ?_getpages"));
            await ForwardAsync(http, method, $"?{paging}", null, (clientId, rawScope));
            return;
        }

In the run, the client that ran the search follows its next link. A different client with system/*.cruds, the widest scope there is, is refused the same link, and so is anyone who strips the page token off.

The client over time

== the client over time
ok    twenty concurrent requests on a cold client result in one token request
ok    thirty seconds before expiry the client replaces the token without being asked
ok    the old token, presented after its five minutes, is 401
ok    after the server restarts with a new signing key, the cached token gets 401 and the handler recovers with one new token

The token endpoint in the sample creates its signing key when it starts and keeps it in memory. A restart therefore invalidates every token in circulation. With five-minute tokens and a client that retries once on 401, the cost of that is one extra token request per client, as the last line shows. That trade is available only because the tokens are short-lived.

A second client, in Python

Agreement between a client and a server written by the same person proves little, since both can share a misunderstanding. The build therefore runs a second client written in Python with PyJWT. It signs with RS384 where the .NET tests mostly use ES384, and it verifies the server’s access token with a different library.

def assertion(token_endpoint: str, lifetime: int = 120) -> str:
    claims = {"iss": client_id, "sub": client_id, "aud": token_endpoint, "exp": int(time.time()) + lifetime, "jti": uuid.uuid4().hex}
    return jwt.encode(claims, private_key, algorithm="RS384", headers={"kid": key_id, "typ": "JWT"})

  ok    discovery offers private_key_jwt with RS384
  ok    an RS384 assertion built with PyJWT is accepted by the .NET token endpoint
  ok    the ES384 access token issued by .NET verifies with PyJWT against the server's published key
  ok    the token reads patients through the gateway
  ok    the same token is refused for observations
  ok    the same assertion a second time is refused as invalid_client, with no description
  ok    an assertion valid for ten minutes is refused
  ok    a request for every resource type is refused for a client registered for patients only
  8 of 8 checks passed

Running it

• Transport. The sample runs on loopback over plain HTTP. The profile requires TLS 1.2 or later for every exchange, and the client in the sample refuses any address that is neither https nor loopback.
• Private keys. One key pair per client per environment, generated where it will be used, held in a key vault or key management service as non-exportable, and reached through the signing interface. A private key in a configuration file or a container image is a client secret with extra steps.
• The replay cache. In the sample it is a dictionary in one process. With two instances of a token endpoint behind a load balancer, an assertion refused by one would be accepted by the other. It has to be a shared store with entries that expire.
• Clocks. Every time check assumes the client and the server agree on the time to within the allowed skew, ten seconds here. Unsynchronised clocks show up as intermittent invalid_client on a client that worked yesterday.
• Network. The FHIR server accepts connections from the gateway only. Test that from outside, as part of the deployment.
• Audit. One record per decision: time, client, what was asked, what was decided and why. The gateway logs parameter names and not their values, since a search value can identify a patient, and no token or assertion is ever logged. The test checks both.
• Monitoring. Watch invalid_client by client (a rise means an expired key, a clock problem or someone guessing), 403 by missing scope (a client doing more than it was registered for, or a registration that is too narrow), and failed key set fetches.
• Rate limits. The token endpoint and the gateway both need them. The sample has only the one that protects the client’s key set URL from refetch storms.

Cost

The protocol itself adds almost nothing at run time: one token request per client every five minutes, and one signature verification per FHIR request, done locally. The costs are in ownership.

Decision framework

Key takeaways

• Authenticate services with a signed assertion and a private key that stays in a vault. The token that crosses the network lasts five minutes.
• At the token endpoint, take the algorithm from a fixed list and the key from the registration, verify the signature before recording the jti, and tell a failed caller nothing but invalid_client.
• Enforce scopes on what a request does, not on its path. _include, _revinclude, _has, chained parameters, transactions and paging links each reach beyond the resource type in the URL.
• Removing a key from a key set is not revocation. Plan for the cache lifetime, and keep a faster way to disable a client.
• The FHIR server must be unreachable except through the gateway, and that is a deployment test, not an assumption.

If you run Backend Services against a commercial EHR or a managed FHIR service, I would like to know how it treats _revinclude with a narrow scope. Leave a comment.

Glossary

• SMART Backend Services— The SMART App Launch profile for a system that calls a FHIR server with no user present.
• Client assertion— A short-lived JWT, signed by the client, that proves its identity to the token endpoint. Also called private_key_jwt.
• JWK Set— A JSON document listing public keys. A client publishes one so the server can verify its signatures.
• kid— The key ID in a JWT header, which says which key in the set signed it.
• jti— A unique identifier in a JWT. The server remembers it so the same assertion cannot be used twice.
• RS384, ES384— RSA and elliptic-curve signature algorithms with SHA-384, the two the profile names.
• System scope— A permission for a client acting as itself, such as system/Patient.rs, as opposed to one tied to a user or a patient.
• cruds— Create, read, update, delete, search: the letters of a SMART v2 scope, each granted separately.
• _revinclude— A search parameter that adds resources which refer to the ones found, for example every Observation about a patient.
• Chained search— A search on one resource type by a property of another it refers to.
• Algorithm confusion— An attack in which a verifier is tricked into checking a signature with a different algorithm from the one the key was issued for.

References & Further Reading

• → SMART App Launch: Backend Services— The profile: token request, scopes, token response
• → SMART App Launch: client authentication with asymmetric keys— Assertion format, key registration, server validation
• → SMART App Launch: scopes— The v2 scope syntax and what each letter covers
• → SMART App Launch: conformance— The discovery document and capability names
• → RFC 7523— JWT profile for OAuth 2.0 client authentication
• → RFC 7638— JWK thumbprint, used here as the key ID
• → RFC 8725— JSON Web Token best current practices, including algorithm verification
• → RFC 9068— JWT profile for OAuth 2.0 access tokens
• → FHIR R4: search— _include, _revinclude, _has and chained parameters
• → HAPI FHIR documentation— The server behind the gateway in the test run
• → Sample code for this article— Client, token endpoint, gateway, tests and the Python client