Gateway identity (last_call)

A refusal already tells you which gateway said no. donkey.last_call is the success-path counterpart: after every governed call it records which gateway served the request, what it actually routed to, and what the call cost — without parsing headers or running a span backend. That matters because a silent model substitution is otherwise invisible to your cost model and your evals, and reading only total_tokens misses cached and reasoning tokens. If your assumptions are pinned to one model, on_model_substitution="raise" turns a swap into a hard ModelSubstituted error.

Example	Shows	Needs
Narrative demo 10	A cold `UNOBSERVED` record, one call populating identity, routing and usage, the substitution flag, and `on_model_substitution="raise"`	Nothing (simulator)
OpenAI script 08	The full `last_call` record on a live call, then `on_model_substitution="raise"`	Proxy credentials

Run it


make demo N=10

Expected output


════════════════════════════════════════════════════════════════════════════════════════
Demo 10 — last_call, routing, and per-call usage
The success-path counterpart to a typed refusal: who served this, what they served, and
what it cost.
════════════════════════════════════════════════════════════════════════════════════════

Run context
───────────
  target                 mock
  proxy base_url         http://127.0.0.1:8080/
  output masking         on
  credentials            fake — the simulator enforces no auth

[1] A new process has not observed a call yet — and it says so
  status                 unobserved
  observed               False
  available              True
  request_id             None

  PASS  UNOBSERVED — not None, not 0, not 'unknown'. A cold read is a named state.
  A bare None would be a lie of omission: you could not tell 'the gateway sent no id'
  from 'we never saw a response'. Budget uses the same honesty rule for an unobserved
  window (demo 03). UNAVAILABLE is the third state, for adapters that never route
  through our transport — LiteLLM-backed ADK and CrewAI, or default_headers-only
  LlamaIndex. Those surfaces report UNAVAILABLE by name rather than looking like a cold
  read.

[2] One governed call, and the record is the success-path counterpart

    await client.responses.create(model=..., input=...)
    donkey.last_call.served_model
    donkey.last_call.total_tokens
    donkey.last_call.substituted

  status                 observed
    observed             True
    request_id           req_f85003861d5348c9a1d152c276082b07
    requested_model      gpt-4o
    served_model         gpt-5.1
    served_provider      openai
    routing_type         ModelBased
    fallback             False
    substituted          True
    input_tokens         17
    output_tokens        51
    total_tokens         68
    cached_tokens        0
    cache_write_tokens   0
    reasoning_tokens     0

  PASS  OBSERVED — the SDK saw the response, even if some fields stayed None
  request_id is the gateway's own id (x-request-id) — quote it in a ticket. It is the
  same field classify() puts on a DonkeyError after a refusal, now present on the 200 as
  well. api_instance_id and environment_id are parsed from x-envoy-decorator-operation;
  they are masked in this output.

[3] Routing, fallback, and the cost-relevant token counts


What the gateway did with the request
─────────────────────────────────────
  requested              gpt-4o
  served                 openai/gpt-5.1
  routing_type           ModelBased
  fallback               False
  substituted            True

  PASS  substituted — asked for gpt-4o, gateway served gpt-5.1
  Against the simulator this is the captured happy-path fixture talking: it was recorded
  against gpt-5.1, and we asked for a different id. That is not a live failover — and it
  is exactly the mismatch last_call is for. A silent substitution is otherwise invisible
  to your cost model, your eval, and your latency dashboard.


What this call cost
───────────────────
  input / output / total 17 / 51 / 68
  cached_tokens          0
  cache_write_tokens     0
  reasoning_tokens       0

  cached_tokens are billed at the cached rate; reasoning_tokens are output the developer
  never sees. Reading only total_tokens draws the wrong conclusion about both cost and
  latency. An absent count is None, never 0 — 0 here means the gateway reported zero,
  which is a different statement. These are per-call; donkey.budget is the shared window
  (demo 03).
  The SDK never double-retries a fallback. It retries 502/503/504 with backoff, but a
  503 the gateway already marked as a failover is left alone — a second recovery layer
  stacked on a working first one just multiplies latency against an outage the gateway
  already handled.

[4] Opt in, and a substitution is a hard error instead of a flag

    donkey = Donkey.from_env(on_model_substitution="raise")
    # raises ModelSubstituted when served_model != requested_model

  Off by default: the call succeeds and last_call.substituted is True. Raise is for
  callers whose eval, cost model and token assumptions are pinned to one model.
  ModelSubstituted is deliberately not a PolicyViolation — the request was neither
  refused nor failed, it succeeded against a model you did not choose. Same shape as
  BudgetReserveReached: a client-side signal you opted into.

  PASS  ModelSubstituted — the 200 never reached the caller
  requested_model        gpt-4o
  served_model           gpt-5.1
  served_provider        openai
  request_id             req_f85003861d5348c9a1d152c276082b07
  last_call.substituted  True
  The record still populated — observe happens before the raise — so a handler that
  decides to accept the served completion can read last_call the same way. The exception
  also carries the response.


The point
─────────
  The refusal path already told you which gateway said no. The success path now tells
  you which gateway said yes, what it actually served, and what that call cost — without
  a span backend, without parsing headers, and without a second accessor for routing or
  usage.

────────────────────────────────────────────────────────────────────────────────────────


python "demos/human-made/openai/08 - last-call.py"   # needs proxy credentials

Key code

Reading the record after a call (OpenAI script 08):


donkey = Donkey.from_env()
client = donkey.openai(sync=True)
 
print("before any call ", donkey.last_call.status.value)
 
reply = client.responses.create(model="gpt-4o", input="Say hello in exactly three words.")
 
last = donkey.last_call
print("status          ", last.status.value)
print("request_id      ", last.request_id)
print("requested_model ", last.requested_model)
print("served_model    ", last.served_model)
print("served_provider ", last.served_provider)
print("routing_type    ", last.routing_type)
print("fallback        ", last.fallback)
print("substituted     ", last.substituted)
print("total_tokens    ", last.total_tokens)
print("cached_tokens   ", last.cached_tokens)
print("reasoning_tokens", last.reasoning_tokens)

Opting in to a hard error. The OpenAI client wraps the transport error, so the typed ModelSubstituted is on __cause__:


strict = Donkey.from_env(on_model_substitution="raise")
strict_client = strict.openai(sync=True)
try:
    strict_client.responses.create(model="gpt-4o", input="Say hello in exactly three words.")
    print("NO RAISE  substituted", strict.last_call.substituted)
except Exception as err:
    hit = err if isinstance(err, ModelSubstituted) else err.__cause__
    if not isinstance(hit, ModelSubstituted):
        raise
    print("requested_model   ", hit.requested_model)
    print("served_model      ", hit.served_model)

Three named states, never a bare None. UNOBSERVED is a cold read; OBSERVED means the SDK saw a response (fields may still be None if the gateway said nothing); UNAVAILABLE marks adapter surfaces that never route through the SDK’s transport.
request_id is the upstream provider’s id, passed through by the gateway — x-request-id for OpenAI, x-amzn-requestid for Amazon Bedrock, apim-request-id for Azure OpenAI. Quote it to the provider’s support team. It is the same field classify() puts on a refusal, now present on a 200 too, and None on a route whose provider forwards no id.
Absent counts are None, never 0. cached_tokens are billed at the cached rate; reasoning_tokens are output you never see.
ModelSubstituted is not a PolicyViolation. The request succeeded, against a model you did not choose. last_call is still populated before the raise.
A provider/ prefix is not a substitution. On a model-based routing proxy the gateway reports the served model without the provider/ prefix; substituted ignores a prefix that names the served provider (see Telemetry & cost).

The simulator’s captured success response was recorded against gpt-5.1, so asking for any other model id shows up as a substitution in narrative demo 10. That is the fixture, not a live failover — and it is exactly the mismatch last_call exists to surface.

Semantic routing: the matched topic and score

The call above went through a model-based proxy, where routing_type reads ModelBased. A semantic-routing proxy instead classifies each prompt by meaning and routes it to the matched topic’s provider and model — there routing_type reads Semantic, and last_call carries two more fields the model-based path leaves None:

matched_topic — which topic the prompt matched (e.g. Finance).
routing_score — how close that match was, a bare 0.xx similarity score.

Both come from the live-verified, semantic-only x-llm-proxy-semantic-routing-success response header. The four routing fields you already read (routing_type, fallback, served_provider, served_model) are emitted identically to the model-based case, so the rest of the record reads the same way — only these two are added.

You can exercise this branch offline: point the simulator at the captured Semantic response by requesting the donkey-sim/success-semantic model id (the same sentinel mechanism the simulator uses to force a refusal shape, here forcing a happy-path variant).


client = donkey.openai(sync=True)
client.responses.create(
    model="donkey-sim/success-semantic",
    input="How does compound interest work?",
)
 
last = donkey.last_call
print("routing_type ", last.routing_type)   # Semantic
print("matched_topic", last.matched_topic)  # Finance
print("routing_score", last.routing_score)  # 0.62

matched_topic and routing_score are None on a model-based proxy — the x-llm-proxy-semantic-routing-success header is semantic-only. An unparseable message leaves each field None rather than guessing a value.

Learn more: Feature overview · Telemetry & cost

Source: narrative demo 10 · script 08