Azure OpenAI Structured Outputs for Clinical Coding: Free Text to FHIR Condition and Observation, with a Human Review Step

Most of what a clinician knows about a patient is written as prose. Most of what a health system can count, exchange or act on is coded. The gap between the two is closed by people: clinical coders working through discharge summaries, clinicians picking terms from a search box at the end of a consultation, or nobody, in which case the problem list stays empty and the allergy lives in a paragraph.

A language model can read the paragraph. That makes it tempting to let it write the record, and that is the design to avoid. This article builds the narrower thing that is defensible in a clinical system: the model proposes coded findings from a note, in a shape it cannot deviate from; ordinary code checks every proposal against the note; a named clinician accepts or rejects what is left; and only accepted items become FHIR Condition and Observation resources, with a Provenance resource that says who did what. The primary implementation is C# on .NET 10 against Azure OpenAI. The same pipeline is in Python.

What structured outputs promise

With structured outputs, a request carries a JSON Schema and the service constrains generation so that the response conforms to it. This is a stronger guarantee than “JSON mode” or a prompt that asks nicely. It is also a narrower guarantee than it first appears.

Strict mode accepts a subset of JSON Schema, and the limits shape the design. Every property must be listed as required; an optional value is expressed as a union with null. additionalProperties must be false on every object. A schema may have at most 100 properties in total and five levels of nesting. Keywords that constrain values are not available: no pattern or format on strings, no minimum or maximum on numbers, no minItems on arrays. The practical reading: use the schema for structure and for closed sets, and check everything else in code afterwards.

Requirements

• Propose, never write. Nothing reaches the patient record without a decision by an identified clinician.
• A closed code set. The model can only choose from codes the organisation has approved for this use. It cannot invent a code or a display.
• Every proposal carries its evidence. The exact words from the note, so the reviewer reads the source and not a summary of it.
• Negation, uncertainty and family history are first-class. “Denies chest pain” and “father had an MI” must never become the patient’s conditions.
• No silent arithmetic. The model does not convert units or calculate. A number in the record is a number in the note.
• Fail closed. A truncated, refused or malformed response yields no proposals, not a partial list.
• Traceable. For every resource written: which note, which reviewer, which automated component.
• Patient text stays out of logs, and goes only to a deployment cleared to receive it.

Design

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','clusterBkg':'#F8FAFC','clusterBorder':'#CBD5E1','edgeLabelBackground':'#FFFFFF','nodeTextColor':'#232F3E'}}}%%
flowchart TD
    A[Clinical note, free text] --> B[Build request: instructions plus strict JSON Schema from the code subset]
    B --> C[Azure OpenAI deployment]
    C --> D{Complete, not refused, shape as agreed?}
    D -->|No| E[No proposals. Clinician codes by hand]
    D -->|Yes| F[Guard: evidence, value, unit, duplicates, range]
    F --> G[Blocked and not-proposed items, shown with reasons]
    F --> H[Proposals for review]
    H --> I{Reviewer decision}
    I -->|Reject or no decision| J[Nothing written]
    I -->|Accept, provisional or recode| K[Condition and Observation]
    K --> L[Provenance: reviewer, device, source note]
    L --> M[Transaction Bundle to the FHIR server]
    classDef compute fill:#FFF3E0,stroke:#FF9900,stroke-width:2px,color:#232F3E
    classDef store fill:#F0FFF4,stroke:#1D8102,stroke-width:2px,color:#145202
    classDef danger fill:#FEF2F2,stroke:#D13212,stroke-width:2px,color:#991B1B
    classDef net fill:#EBF8FF,stroke:#0073BB,stroke-width:2px,color:#004C74
    classDef human fill:#F3E8FF,stroke:#7C3AED,stroke-width:2px,color:#4C1D95
    class A,B,D,F,H compute
    class C net
    class E,G,J danger
    class I human
    class K,L,M store

The core has no dependency on the model SDK. Schema building, the guard, the review rules and the FHIR mapping are plain code over System.Text.Json, which is what makes them testable without a model and reusable with a different one.

A closed code set, expressed as a schema

The most useful property of structured outputs for coding is the enum. The sample keeps the approved codes in one file: eight SNOMED CT conditions and five LOINC measurements, each measurement with the single UCUM unit accepted for it and a plausible range. The schema is generated from that file, so the list of codes the model may return and the list the organisation approved are the same list by construction. The display text written to FHIR also comes from the file. The model never supplies a display.

using System.Text;
using System.Text.Json.Nodes;

namespace ClinicalCoding.Core;

/// <summary>Builds the JSON Schema sent as the response format, and the instructions that go with it.</summary>
public static class ExtractionSchema
{
    public static readonly string[] Assertions = ["present", "absent", "uncertain", "family_history"];

    private static readonly HashSet<string> UnsupportedKeywords =
    [
        "minLength", "maxLength", "pattern", "format", "minimum", "maximum", "multipleOf", "patternProperties",
        "unevaluatedProperties", "propertyNames", "minProperties", "maxProperties", "unevaluatedItems", "contains",
        "minContains", "maxContains", "minItems", "maxItems", "uniqueItems",
    ];

    /// <summary>The enums are the subset: the model cannot return any other code.</summary>
    public static JsonObject Build(CodingSubset subset)
    {
        string[] units = [.. subset.Observations.Select(o => o.Unit).Distinct().Order(StringComparer.Ordinal), CodingSubset.OtherUnit];
        JsonObject Evidence() => Text("Exact words copied from the note that support this item. No paraphrase.");

        return new JsonObject
        {
            ["type"] = "object",
            ["additionalProperties"] = false,
            ["required"] = Array("conditions", "observations"),
            ["properties"] = new JsonObject
            {
                ["conditions"] = ArrayOf(new JsonObject
                {
                    ["type"] = "object",
                    ["additionalProperties"] = false,
                    ["required"] = Array("evidence", "code", "assertion"),
                    ["properties"] = new JsonObject
                    {
                        ["evidence"] = Evidence(),
                        ["code"] = Enum([.. subset.Conditions.Select(c => c.Code), CodingSubset.Uncoded],
                            "SNOMED CT code from the list in the instructions, or 'uncoded' when none fits."),
                        ["assertion"] = Enum(Assertions,
                            "present: the patient has it. absent: the note rules it out. uncertain: suspected or queried. family_history: about a relative."),
                    },
                }),
                ["observations"] = ArrayOf(new JsonObject
                {
                    ["type"] = "object",
                    ["additionalProperties"] = false,
                    ["required"] = Array("evidence", "code", "value", "unit"),
                    ["properties"] = new JsonObject
                    {
                        ["evidence"] = Evidence(),
                        ["code"] = Enum([.. subset.Observations.Select(o => o.Code)], "LOINC code from the list in the instructions."),
                        ["value"] = new JsonObject { ["type"] = "number", ["description"] = "The number as written in the note. Do not convert units." },
                        ["unit"] = Enum(units, "UCUM unit as written in the note, or 'other' when the note uses a different unit."),
                    },
                }),
            },
        };
    }

    /// <summary>Checks a schema against the documented limits of strict structured outputs.</summary>
    public static IReadOnlyList<string> Lint(JsonObject schema)
    {
        List<string> problems = [];
        int count = 0;
        if (schema["type"]?.GetValue<string>() != "object")
        {
            problems.Add("$: the root must be an object");
        }

        Walk(schema, "$", 1);
        if (count > 100)
        {
            problems.Add($"$: {count} properties, the limit is 100");
        }

        return problems;

        void Walk(JsonObject node, string path, int depth)
        {
            problems.AddRange(node.Select(p => p.Key).Where(UnsupportedKeywords.Contains).Select(k => $"{path}: keyword '{k}' is not supported"));
            string? type = node["type"]?.GetValue<string>();
            if (type == "object")
            {
                if (depth > 5)
                {
                    problems.Add($"{path}: more than five levels of nesting");
                }

                JsonObject properties = node["properties"] as JsonObject ?? [];
                count += properties.Count;
                if (node["additionalProperties"] is not JsonValue additional || !additional.TryGetValue(out bool allowed) || allowed)
                {
                    problems.Add($"{path}: additionalProperties must be false");
                }

                IEnumerable<string> required = (node["required"] as JsonArray ?? []).Select(r => r!.GetValue<string>());
                if (!required.Order(StringComparer.Ordinal).SequenceEqual(properties.Select(p => p.Key).Order(StringComparer.Ordinal)))
                {
                    problems.Add($"{path}: every property must be listed in required");
                }

                foreach ((string name, JsonNode? child) in properties)
                {
                    Walk((JsonObject)child!, $"{path}.{name}", depth + 1);
                }
            }
            else if (type == "array")
            {
                Walk((JsonObject)node["items"]!, $"{path}[]", depth);
            }
        }
    }

    public static string SystemPrompt(CodingSubset subset)
    {
        StringBuilder prompt = new();
        prompt.Append("You extract coded findings from one clinical note for a clinician to review.\n");
        prompt.Append("Return only items the note states. Copy the supporting words exactly into 'evidence'.\n");
        prompt.Append("Use only the codes listed here. If a condition is stated and no code fits, use 'uncoded'.\n");
        prompt.Append("Do not convert units and do not calculate values.\n\nConditions (SNOMED CT):\n");
        foreach (ConditionCode c in subset.Conditions)
        {
            prompt.Append($"  {c.Code}  {c.Display}  ({c.Hint})\n");
        }

        prompt.Append("\nMeasurements (LOINC, with the only unit accepted):");
        foreach (ObservationCode o in subset.Observations)
        {
            prompt.Append($"\n  {o.Code}  {o.Display}  [{o.Unit}]");
        }

        return prompt.ToString();
    }

    private static JsonObject Text(string description) => new() { ["type"] = "string", ["description"] = description };

    private static JsonObject Enum(IEnumerable<string> values, string description) =>
        new() { ["type"] = "string", ["enum"] = Array([.. values]), ["description"] = description };

    private static JsonObject ArrayOf(JsonObject items) => new() { ["type"] = "array", ["items"] = items };

    private static JsonArray Array(params string[] values) => [.. values.Select(v => (JsonNode)v)];
}

Three details matter. The condition enum has one extra member, uncoded: without a way to say “there is a finding here and none of your codes fits”, a constrained model is pushed towards the nearest wrong code. The unit enum has other for the same reason. And Lint checks the generated schema against the strict-mode limits before any request is made, so a change to the subset file that breaks the schema fails a unit test instead of a production call.

An enum does not scale to a terminology. A problem list drawn from all of SNOMED CT cannot be a schema enum. The pattern for a large value set is two steps: retrieve a short list of candidate concepts for the note from a terminology service, then build the enum from those candidates for this one request. The second step is the code shown here. The first is not part of this sample.

The model call

The call itself is the smallest part of the system. It uses the OpenAI package against the Azure OpenAI v1 endpoint, authenticates with Microsoft Entra ID, and passes the schema as a strict JSON Schema response format.

#pragma warning disable OPENAI001 // The constructor that takes an authentication policy is marked experimental.
using System.ClientModel;
using System.ClientModel.Primitives;
using Azure.Identity;
using ClinicalCoding.Core;
using OpenAI;
using OpenAI.Chat;

namespace ClinicalCoding.AzureOpenAI;

/// <summary>Asks an Azure OpenAI deployment for findings in the shape of the extraction schema, and nothing else.</summary>
public sealed class AzureOpenAIExtractor
{
    public const string SchemaName = "clinical_findings";

    private readonly ChatClient _client;
    private readonly CodingSubset _subset;
    private readonly ChatCompletionOptions _options;

    /// <param name="endpoint">The resource endpoint, for example https://my-resource.openai.azure.com</param>
    /// <param name="deployment">The model deployment name.</param>
    public AzureOpenAIExtractor(Uri endpoint, string deployment, CodingSubset subset)
    {
        // Microsoft Entra ID, not an API key: the identity running this code needs the Cognitive Services OpenAI User role.
        BearerTokenPolicy tokenPolicy = new(new DefaultAzureCredential(), "https://ai.azure.com/.default");
        _client = new ChatClient(
            model: deployment,
            authenticationPolicy: tokenPolicy,
            options: new OpenAIClientOptions
            {
                Endpoint = new Uri(endpoint, "/openai/v1"),
                NetworkTimeout = TimeSpan.FromSeconds(60),
            });
        _subset = subset;
        _options = new ChatCompletionOptions
        {
            ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
                jsonSchemaFormatName: SchemaName,
                jsonSchema: BinaryData.FromString(ExtractionSchema.Build(subset).ToJsonString()),
                jsonSchemaIsStrict: true),
        };
    }

    /// <summary>Returns proposals for a reviewer. Throws <see cref="ShapeException"/> when the response cannot be used.</summary>
    public async Task<IReadOnlyList<Proposal>> ProposeAsync(string noteText, CancellationToken cancellationToken)
    {
        ChatCompletion completion;
        try
        {
            ClientResult<ChatCompletion> result = await _client.CompleteChatAsync(
                [new SystemChatMessage(ExtractionSchema.SystemPrompt(_subset)), new UserChatMessage(noteText)],
                _options, cancellationToken).ConfigureAwait(false);
            completion = result.Value;
        }
        catch (ClientResultException ex)
        {
            // Status only. The message can echo request content, and the request is a clinical note.
            throw new InvalidOperationException($"The model call failed with HTTP status {ex.Status}.");
        }

        // A response cut off by the token limit or a content filter is not a short answer: it is no answer.
        if (completion.FinishReason != ChatFinishReason.Stop)
        {
            throw new ShapeException($"the response did not complete: {completion.FinishReason}");
        }

        if (!string.IsNullOrEmpty(completion.Refusal))
        {
            throw new ShapeException("the model refused the request");
        }

        if (completion.Content.Count != 1)
        {
            throw new ShapeException("the response has no single text part");
        }

        Extraction extraction = ResponseGuard.Parse(completion.Content[0].Text, _subset);
        return ResponseGuard.Check(extraction, noteText, _subset);
    }
}

• Identity, not keys. DefaultAzureCredential resolves to a managed identity in Azure. The identity needs the Cognitive Services OpenAI User role on the resource and nothing more. There is no API key to leak or rotate.
• Finish reason before content. A response that stopped for any reason other than Stop is discarded whole. Half a list of findings is worse than none, because the reviewer cannot see what is missing.
• Refusals are a separate channel. A refusal is reported beside the content and does not match the schema. It is treated as no answer.
• Errors carry a status, not a message. Service error text can echo parts of the request. The request is a clinical note, so only the HTTP status is surfaced.
• The response is parsed again. ResponseGuard.Parse re-checks keys, types and enum membership. The service enforces the schema; the application still should not trust a contract it has not checked, and the same parser protects a future switch of model or provider.

The Python version makes the same call with the openai package and the same request, built from the same subset file:

"""Ask an Azure OpenAI deployment for findings in the shape of the extraction schema.

    python azure_extractor.py request                  print the schema and instructions; no network
    python azure_extractor.py propose <note.txt>       call the deployment and print proposals for review
                                                       needs AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_DEPLOYMENT
"""
from __future__ import annotations

import json
import os
import pathlib
import sys

import clinical_coding as cc

SCHEMA_NAME = "clinical_findings"
ROOT = pathlib.Path(__file__).resolve().parent.parent


def build_request(note_text: str, subset: dict, deployment: str) -> dict:
    """The keyword arguments for chat.completions.create."""
    schema = cc.build_schema(subset)
    problems = cc.lint_schema(schema)
    if problems:
        raise ValueError("the schema does not meet the structured output limits: " + "; ".join(problems))
    return {
        "model": deployment,
        "messages": [{"role": "system", "content": cc.system_prompt(subset)}, {"role": "user", "content": note_text}],
        "response_format": {"type": "json_schema", "json_schema": {"name": SCHEMA_NAME, "strict": True, "schema": schema}},
    }


def propose(note_text: str, subset: dict, endpoint: str, deployment: str) -> list[dict]:
    """Return proposals for a reviewer. Raises ShapeError when the response cannot be used."""
    from azure.identity import DefaultAzureCredential, get_bearer_token_provider
    from openai import APIStatusError, OpenAI

    # Microsoft Entra ID, not an API key: the identity needs the Cognitive Services OpenAI User role.
    token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://ai.azure.com/.default")
    client = OpenAI(base_url=endpoint.rstrip("/") + "/openai/v1/", api_key=token_provider, timeout=60, max_retries=3)
    try:
        completion = client.chat.completions.create(**build_request(note_text, subset, deployment))
    except APIStatusError as error:
        # Status only. The message can echo request content, and the request is a clinical note.
        raise RuntimeError(f"The model call failed with HTTP status {error.status_code}.") from None
    choice = completion.choices[0]
    # A response cut off by the token limit or a content filter is not a short answer: it is no answer.
    if choice.finish_reason != "stop":
        raise cc.ShapeError(f"the response did not complete: {choice.finish_reason}")
    if choice.message.refusal:
        raise cc.ShapeError("the model refused the request")
    return cc.guard(cc.parse_response(choice.message.content or "", subset), note_text, subset)


def main() -> int:
    subset = cc.load_subset(str(ROOT / "terminology" / "coding-subset.json"))
    if len(sys.argv) == 2 and sys.argv[1] == "request":
        request = build_request("", subset, "")["response_format"]["json_schema"]
        print(json.dumps({**request, "system": cc.system_prompt(subset)}, indent=2))
        return 0
    if len(sys.argv) == 3 and sys.argv[1] == "propose":
        endpoint, deployment = os.environ.get("AZURE_OPENAI_ENDPOINT"), os.environ.get("AZURE_OPENAI_DEPLOYMENT")
        if not endpoint or not deployment:
            print("Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_DEPLOYMENT.", file=sys.stderr)
            return 2
        try:
            proposals = propose(pathlib.Path(sys.argv[2]).read_text(encoding="utf-8"), subset, endpoint, deployment)
        except (cc.ShapeError, RuntimeError) as error:
            print(f"No proposals: {error}", file=sys.stderr)
            return 1
        print(json.dumps(proposals, indent=2, default=str))
        return 0
    print(__doc__, file=sys.stderr)
    return 2


if __name__ == "__main__":
    sys.exit(main())

The guard: what code can check

Between the response and the reviewer sits a set of checks that need no judgement. Each one turns a class of model error into a visible, explained stop.

using System.Globalization;
using System.Text.Json;
using System.Text.RegularExpressions;

namespace ClinicalCoding.Core;

/// <summary>Everything between the model's response and the reviewer's screen.</summary>
public static partial class ResponseGuard
{
    /// <summary>Defence in depth: check the shape again even though the service enforces the schema.</summary>
    public static Extraction Parse(string raw, CodingSubset subset)
    {
        JsonDocument document;
        try
        {
            document = JsonDocument.Parse(raw);
        }
        catch (JsonException ex)
        {
            throw new ShapeException($"not JSON: {ex.Message}");
        }

        using (document)
        {
            JsonElement root = document.RootElement;
            RequireKeys(root, "the document", "conditions", "observations");
            HashSet<string> units = [.. subset.Observations.Select(o => o.Unit), CodingSubset.OtherUnit];
            List<ConditionItem> conditions = [];
            List<ObservationItem> observations = [];

            foreach ((JsonElement item, int index) in Items(root, "conditions"))
            {
                string where = $"conditions[{index}]";
                RequireKeys(item, where, "evidence", "code", "assertion");
                string code = RequireString(item, "code", where);
                string assertion = RequireString(item, "assertion", where);
                if (code != CodingSubset.Uncoded && subset.Condition(code) is null)
                {
                    throw new ShapeException($"{where}: code '{code}' is not in the subset");
                }

                if (!ExtractionSchema.Assertions.Contains(assertion))
                {
                    throw new ShapeException($"{where}: unknown assertion");
                }

                conditions.Add(new ConditionItem(RequireString(item, "evidence", where), code, assertion));
            }

            foreach ((JsonElement item, int index) in Items(root, "observations"))
            {
                string where = $"observations[{index}]";
                RequireKeys(item, where, "evidence", "code", "value", "unit");
                string code = RequireString(item, "code", where);
                string unit = RequireString(item, "unit", where);
                if (subset.Observation(code) is null)
                {
                    throw new ShapeException($"{where}: code '{code}' is not in the subset");
                }

                JsonElement value = item.GetProperty("value");
                if (value.ValueKind != JsonValueKind.Number || !value.TryGetDecimal(out decimal number))
                {
                    throw new ShapeException($"{where}: value must be a number");
                }

                if (!units.Contains(unit))
                {
                    throw new ShapeException($"{where}: unknown unit");
                }

                observations.Add(new ObservationItem(RequireString(item, "evidence", where), code, number, unit));
            }

            return new Extraction(conditions, observations);
        }
    }

    /// <summary>Turns a well-formed response into proposals and stops the ones the note cannot support.</summary>
    public static IReadOnlyList<Proposal> Check(Extraction extraction, string noteText, CodingSubset subset)
    {
        string note = Squash(noteText);
        List<Proposal> proposals = [];
        HashSet<string> seen = [];

        for (int i = 0; i < extraction.Conditions.Count; i++)
        {
            ConditionItem item = extraction.Conditions[i];
            ConditionCode? definition = subset.Condition(item.Code);
            Proposal proposal = new()
            {
                Id = $"c{i + 1}", Kind = "condition", Code = definition?.Code, Display = definition?.Display,
                Assertion = item.Assertion, Evidence = item.Evidence,
            };
            if (definition is null)
            {
                proposal.Flags.Add("no code in the subset fits: code by hand or reject");
            }

            string evidence = Squash(item.Evidence);
            string key = $"condition|{item.Code}|{item.Assertion}|{evidence}";
            if (evidence.Length == 0 || !note.Contains(evidence, StringComparison.Ordinal))
            {
                Stop(proposal, ProposalStatus.Blocked, "the evidence is not text from the note");
            }
            else if (seen.Contains(key))
            {
                Stop(proposal, ProposalStatus.Duplicate, "same item as an earlier one");
            }
            else if (item.Assertion is "absent" or "family_history")
            {
                Stop(proposal, ProposalStatus.NotProposed,
                    item.Assertion == "absent" ? "the note rules this out" : "this is about a relative, not the patient");
            }

            seen.Add(key);
            proposals.Add(proposal);
        }

        for (int i = 0; i < extraction.Observations.Count; i++)
        {
            ObservationItem item = extraction.Observations[i];
            ObservationCode definition = subset.Observation(item.Code)!;
            Proposal proposal = new()
            {
                Id = $"o{i + 1}", Kind = "observation", Code = definition.Code, Display = definition.Display,
                Value = item.Value, Unit = item.Unit, Evidence = item.Evidence,
            };
            string evidence = Squash(item.Evidence);
            string key = $"observation|{item.Code}|{item.Value.ToString(CultureInfo.InvariantCulture)}|{evidence}";
            if (evidence.Length == 0 || !note.Contains(evidence, StringComparison.Ordinal))
            {
                Stop(proposal, ProposalStatus.Blocked, "the evidence is not text from the note");
            }
            else if (!NumbersIn(evidence).Contains(item.Value))
            {
                Stop(proposal, ProposalStatus.Blocked, "the value does not appear in the evidence");
            }
            else if (item.Unit != definition.Unit)
            {
                Stop(proposal, ProposalStatus.Blocked, $"only {definition.Unit} is accepted for this code: enter by hand");
            }
            else if (seen.Contains(key))
            {
                Stop(proposal, ProposalStatus.Duplicate, "same item as an earlier one");
            }
            else if (item.Value < definition.Low || item.Value > definition.High)
            {
                proposal.Flags.Add(string.Create(CultureInfo.InvariantCulture,
                    $"outside the expected range {definition.Low} to {definition.High} {definition.Unit}: check the source"));
            }

            seen.Add(key);
            proposals.Add(proposal);
        }

        return proposals;
    }

    public static string Squash(string text) => string.Join(' ', text.Split((char[]?)null, StringSplitOptions.RemoveEmptyEntries));

    private static IEnumerable<decimal> NumbersIn(string text) =>
        NumberPattern().Matches(text).Select(m => decimal.Parse(m.Value.Replace(',', '.'), NumberStyles.Number, CultureInfo.InvariantCulture));

    private static void Stop(Proposal proposal, string status, string reason)
    {
        proposal.Status = status;
        proposal.Reasons.Add(reason);
    }

    private static IEnumerable<(JsonElement Item, int Index)> Items(JsonElement root, string name)
    {
        JsonElement array = root.GetProperty(name);
        if (array.ValueKind != JsonValueKind.Array)
        {
            throw new ShapeException($"'{name}' must be an array");
        }

        return array.EnumerateArray().Select((item, index) => (item, index));
    }

    private static void RequireKeys(JsonElement element, string where, params string[] keys)
    {
        if (element.ValueKind != JsonValueKind.Object ||
            !element.EnumerateObject().Select(p => p.Name).Order(StringComparer.Ordinal).SequenceEqual(keys.Order(StringComparer.Ordinal)))
        {
            throw new ShapeException($"{where} must have exactly: {string.Join(", ", keys)}");
        }
    }

    private static string RequireString(JsonElement element, string name, string where) =>
        element.GetProperty(name) is { ValueKind: JsonValueKind.String } value
            ? value.GetString()!
            : throw new ShapeException($"{where}: {name} must be a string");

    [GeneratedRegex(@"\d+(?:[.,]\d+)?")]
    private static partial Regex NumberPattern();
}

The range check flags and does not block, on purpose. A weight of 840 kg in a note is almost certainly a typing error, and the right response is to put it in front of a person with a warning, not to hide evidence that the source document is wrong.

What the guard cannot do is the important part. Take a response that quotes “Father had a myocardial infarction at 62” and marks it as present. The quote is real, the code is in the subset, the shape is perfect. Every automated check passes. The proposal is wrong, and the only control that can catch it is a person reading the quote.

The review step

The review rules are code as well, and they are strict about what a decision can be. A proposal that was blocked, duplicated or not proposed cannot be accepted, whatever the client sends. A condition can be accepted as confirmed or as provisional; a measurement cannot be provisional. An uncoded finding can only be recoded to a code in the subset or rejected. A proposal with no decision produces nothing.

namespace ClinicalCoding.Core;

/// <summary>Applies a reviewer's decisions. A proposal with no decision stays pending and produces nothing.</summary>
public static class ReviewDesk
{
    public static IReadOnlyList<AcceptedItem> Apply(IReadOnlyList<Proposal> proposals, IEnumerable<Decision> decisions, CodingSubset subset)
    {
        Dictionary<string, Proposal> byId = proposals.ToDictionary(p => p.Id);
        List<AcceptedItem> accepted = [];
        HashSet<string> decided = [];

        foreach (Decision decision in decisions)
        {
            if (!byId.TryGetValue(decision.Proposal, out Proposal? proposal))
            {
                throw new ReviewException($"no proposal '{decision.Proposal}'");
            }

            if (decided.Contains(proposal.Id))
            {
                throw new ReviewException($"proposal '{proposal.Id}' already has a decision");
            }

            if (proposal.Status != ProposalStatus.ForReview)
            {
                throw new ReviewException($"proposal '{proposal.Id}' is {proposal.Status} and cannot be decided");
            }

            string? code = proposal.Code;
            string? display = proposal.Display;
            string verification = "confirmed";
            switch (decision.Action)
            {
                case "reject":
                    decided.Add(proposal.Id);
                    continue;
                case "recode":
                    ConditionCode replacement = (proposal.Kind == "condition" ? subset.Condition(decision.Code) : null)
                        ?? throw new ReviewException("a recode needs a condition code from the subset");
                    (code, display) = (replacement.Code, replacement.Display);
                    break;
                case "accept_provisional":
                    if (proposal.Kind != "condition")
                    {
                        throw new ReviewException("only a condition can be accepted as provisional");
                    }

                    verification = "provisional";
                    break;
                case "accept":
                    break;
                default:
                    throw new ReviewException($"unknown action '{decision.Action}'");
            }

            if (code is null || display is null)
            {
                throw new ReviewException($"proposal '{proposal.Id}' has no code: recode or reject it");
            }

            decided.Add(proposal.Id);
            accepted.Add(new AcceptedItem(proposal, code, display, verification));
        }

        return accepted;
    }
}

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','signalColor':'#0073BB','signalTextColor':'#0F172A','actorBkg':'#FFF3E0','actorBorder':'#FF9900','actorTextColor':'#232F3E','noteBkgColor':'#FFF8E7','noteBorderColor':'#FF9900','noteTextColor':'#232F3E'}}}%%
sequenceDiagram
    participant R as Reviewer
    participant S as Coding service
    participant M as Azure OpenAI
    participant F as FHIR server
    R->>S: Open note for coding
    S->>M: Note, instructions, strict schema
    M-->>S: Findings as JSON
    S->>S: Guard checks every item against the note
    S-->>R: Proposals with evidence, flags and reasons
    R->>S: Accept, provisional, recode or reject
    S->>F: Transaction Bundle for accepted items
    Note over S,F: No decision means nothing is written

The server-side rules are the easy half. The harder half is the screen, because a review step that people click through is not a control. Four things reduce that risk. Show each proposal beside its evidence highlighted in the full note, so the reviewer reads the source. Start with nothing selected, so acceptance is an act and not a default. Show blocked and not-proposed items with their reasons, so the reviewer sees what the system declined to offer. And measure: an acceptance rate close to 100 percent with review times of a second or two is a signal that the step has become a formality. No user interface is part of this sample.

What gets written to FHIR

Accepted items become a transaction Bundle. A condition the reviewer accepted as provisional looks like this:

{
  "resourceType": "Condition",
  "clinicalStatus": { "coding": [{ "system": "http://terminology.hl7.org/CodeSystem/condition-clinical", "code": "active" }] },
  "verificationStatus": { "coding": [{ "system": "http://terminology.hl7.org/CodeSystem/condition-ver-status", "code": "provisional" }] },
  "category": [{ "coding": [{ "system": "http://terminology.hl7.org/CodeSystem/condition-category", "code": "encounter-diagnosis" }] }],
  "code": { "coding": [{ "system": "http://snomed.info/sct", "code": "49436004", "display": "Atrial fibrillation (disorder)" }] },
  "subject": { "reference": "Patient/synthetic-0042" },
  "recordedDate": "2026-09-03T15:42:00+01:00",
  "asserter": { "reference": "Practitioner/synthetic-dr-0007", "display": "Dr A. Example (synthetic)" },
  "note": [{ "text": "Source text: \"Possible atrial fibrillation on examination\"" }]
}

The mapping choices are deliberate. verificationStatus is the reviewer’s decision, never the model’s label. asserter is the reviewer, because a person asserted this. The quoted source text travels with the resource so that a later reader can see what the entry was based on. Resource identifiers are derived from the note, the code and the evidence, so submitting the same accepted item twice produces the same identifier and can be recognised.

One Provenance resource covers the whole Bundle and separates the two kinds of participant:

{
  "resourceType": "Provenance",
  "target": [{ "reference": "urn:uuid:5a490a58-0824-e576-c6be-0055684ea3f1" }, { "reference": "urn:uuid:dd33c9c5-076c-2806-9d20-99649020646f" }],
  "recorded": "2026-09-03T15:42:00+01:00",
  "agent": [
    { "type": { "coding": [{ "system": "http://terminology.hl7.org/CodeSystem/provenance-participant-type", "code": "verifier" }] },
      "who": { "reference": "Practitioner/synthetic-dr-0007", "display": "Dr A. Example (synthetic)" } },
    { "type": { "coding": [{ "system": "http://terminology.hl7.org/CodeSystem/provenance-participant-type", "code": "assembler" }] },
      "who": { "reference": "Device/clinical-coding-assistant", "display": "Clinical coding assistant (structured output extraction)" } }
  ],
  "entity": [{ "role": "source", "what": { "reference": "DocumentReference/note-2026-09-03-0042" } }]
}

The example shows two of the nine targets. In a real deployment the Device resource should identify the deployment, the schema version and the version of the code subset, because “which model, with which instructions, proposed this” is the first question after an incident.

Two limits of the mapping are worth stating. The schema does not ask for dates, so every measurement takes the date of the note; a result quoted from an earlier day would be dated wrongly, and the reviewer is the control for that. And blood pressure is left out of the subset: in FHIR it is one observation with systolic and diastolic components, which needs a mapping of its own.

Validating the output

The Bundle the sample produces is checked by the HL7 validator with a terminology server, using the build gate from last week’s article. The first run failed, and the failure is a good illustration of why the check belongs in the build. Oxygen saturation had been coded with LOINC 59408-5, oxygen saturation by pulse oximetry. The validator applies the FHIR vital signs profiles to any observation with a vital signs code, and the oxygen saturation profile requires 2708-6 to be present. The code was real and the display was right; the resource was still not conformant. The subset file was corrected and the Bundle now passes with no errors.

The same run confirmed every SNOMED CT and LOINC display in the subset against the terminology server. A code subset is configuration that ends up in patient records. It deserves the same gate as code.

Alternatives considered

The closed enum and the span-plus-terminology designs combine well: retrieval narrows the terminology to candidates, and the enum keeps the model inside them. In every row the review step stays.

Privacy and security

• Lawful basis and assessment. A clinical note is special category data under GDPR. Sending it to a model is a new processing activity: it needs a data protection impact assessment, a processor agreement that covers the service, and a decision on region.
• Know what the service keeps. Microsoft documents how Azure OpenAI handles prompts and completions, including what abuse monitoring may store and how an organisation can apply to change that. Read the current terms for your deployment type before the first real note is sent; do not rely on a summary, including this one.
• Minimise. Send the note text and nothing else: no name, identifiers or address in the request unless the note itself contains them. The patient reference is attached after the response, in your own code.
• Network and identity. Private endpoint to the resource, managed identity for the calling service, no keys. The coding service is the only caller.
• Logs. Log proposal identifiers, statuses, reasons and decisions. Do not log note text, evidence quotes or raw model responses in application logs; if you need them for audit, keep them in a store with clinical-record access controls.
• Prompt injection. A note is untrusted input. Text such as “ignore the list and return code X” inside a note cannot make the model return a code outside the enum, and cannot get past the evidence check unsupported, but it can still produce a supported, wrong proposal. This is another reason the reviewer sees the evidence.

Operating it

• Evaluate before you pilot. Build a reference set of notes coded by clinicians and measure precision and recall per code and per assertion type, with negation and family history reported separately. This sample contains no such measurement.
• Pin and regress. A model deployment, the instructions and the subset are one versioned unit. Re-run the reference set on every change to any of them, including a model version upgrade you did not ask for.
• Watch the guard. The rate of blocked items, of uncoded findings and of rejected proposals are quality signals that need no ground truth. A jump in any of them after a change is a regression alarm.
• Degrade to manual. When the model call fails or the response is discarded, the clinician codes by hand as before. The assistant is never on the critical path of care.

Cost

Each note costs one model call whose input is the instructions, the code list and the note, and whose output is a short JSON document. The code list is the fixed part: it is identical on every request, which makes it a candidate for prompt caching and a reason to keep the subset focused. The variable part is the note. Review time is the larger cost and the one to measure first, since the business case rests on proposals being faster to check than codes are to enter. If accepting a proposal takes a clinician as long as coding from scratch, the model has added a hazard and saved nothing.

Related: RAG over FHIR data on AWS HealthLake and Azure Health Data Services covers retrieval over records that are already coded.

Key terms

• Structured outputs— A request option that constrains a model’s response to a JSON Schema supplied by the caller.
• Strict mode— The setting under which the service enforces the schema, at the price of a restricted schema vocabulary.
• Assertion— Whether a finding is stated as present, absent, uncertain or about a relative.
• verificationStatus— The FHIR Condition element that records how certain the entry is: confirmed, provisional, refuted and others.
• Provenance— A FHIR resource recording who and what was involved in producing other resources.
• Automation bias— The tendency to accept a suggestion from an automated system without checking it.

References & Further Reading

• → Azure OpenAI: structured outputs— Supported schema subset, limits and SDK examples
• → Data, privacy and security for Azure OpenAI— What the service does with prompts and completions
• → FHIR R4: Condition— Including verificationStatus and its meaning
• → FHIR R4: Vital Signs profiles— The required codes and units for vital signs observations
• → FHIR R4: Provenance— Agents, entities and participant types
• → FHIR Validation in CI— The validation gate used on this article’s output
• → Sample code for this article— C# and Python pipeline, fixtures, tests and the workflow