Stub the TypeSafe API in your tests without fixtures or a network.

Add plug to your test dependencies ({:plug, "~> 1.16", only: :test}), then build a test client and describe the answers you want by question id:

# test/test_helper.exs
ExUnit.start()

# in a test
setup :typesafe_stubs

test "routes billing tickets" do
  client = TypeSafeAPI.Test.client()

  TypeSafeAPI.Test.stub(client,
    dept: {:choice, :billing, 0.9},
    urgent: {:noul, 0.3},
    anger: {:score, 1, 0.8}
  )

  assert {:ok, result} = MyApp.Triage.run(client, "Where is my refund?")
  assert result.answers.dept.choice == :billing
end

The stub reads the questions in each request and builds a response the way the API would, so the decoded structs are identical to those from a real call: probabilities are spread over exactly the options or levels you sent, they sum to one, the winning option or level is the argmax, Score answers carry level, label and a legend echoing your own labels, and keys come back as atoms or strings exactly as you sent them.

Answer specs

• {:noul, probability} - probability is in 0.0..1.0
• {:choice, option, confidence} - option is one of the question's keys; the remaining probability is spread evenly across the other options
• {:score, level, confidence} - level is a 0-based index; the remaining probability is spread evenly across the other levels

confidence has to beat the uniform baseline, which is 1 / n for a question with n options or levels. Below it the option you named would not be the argmax and the answer would contradict itself, so the stub rejects it with a message naming the minimum for that question. A shape that is wrong whatever the question (a confidence above 1.0, a negative level) is rejected by stub/2 itself, before any request is made.

A request that asks a question you did not stub is answered with HTTP 422 and a message naming the question, so TypeSafeAPI.evaluate/4 returns {:error, %TypeSafeAPI.Error{type: :validation}} and a test cannot silently pass on a default answer. It is an error response rather than a raise so that a missing stub under TypeSafeAPI.evaluate_many/4 surfaces as one error per state instead of an exception inside a task that takes the test process with it.

Composing stubs

Every helper adds to the same stub, so a client can serve several endpoints at once and a later call refines an earlier one:

client =
  TypeSafeAPI.Test.client()
  |> TypeSafeAPI.Test.stub_models([%{name: "jev-1.13.0"}])
  |> TypeSafeAPI.Test.stub(urgent: {:noul, 0.3})
  |> TypeSafeAPI.Test.stub(dept: {:choice, :billing, 0.9})

Requests are dispatched on method and path: GET /v1/models is served by stub_models/2 and POST /v1/systemone by stub/2. Anything else gets a 422 naming the route.

Errors

stub_error/4 queues an error response. By default it answers every request from then on; times: limits it to the next call or calls, which is how you test a retry or a recovery path:

client
|> TypeSafeAPI.Test.stub(urgent: {:noul, 0.3})
|> TypeSafeAPI.Test.stub_error(429, %{"error" => "slow down"},
  headers: [{"retry-after", "1"}],
  times: 1
)

The first call gets the 429; the retry gets the stubbed answers.

Recording fixtures

When the answers you want are the ones the real model gives, record them once and replay them forever after. record/2 wraps a client so every response is appended to a fixture file; replay/2 turns that file back into a client:

# once, against the real API
TypeSafeAPI.new()
|> TypeSafeAPI.Test.record("test/fixtures/triage.jsonl")
|> MyApp.Triage.run("Where is my refund?")

# in every test run afterwards, with no network
test "routes billing tickets" do
  client = TypeSafeAPI.Test.replay("test/fixtures/triage.jsonl")

  assert {:ok, result} = MyApp.Triage.run(client, "Where is my refund?")
  assert result.answers.dept.choice == :billing
end

The replayed TypeSafeAPI.Result is the one the API sent, decoded by the same code, so it is identical to the recorded run. Requests are matched by method, path and body, and the same request twice gets the two recordings in order; pass match: [:method, :path] to ignore the body. A request that matches nothing raises with the request, the closest unserved recording and the first field they differ on.

Fixture format

A fixture is JSON Lines: one JSON object per line, in the order the responses arrived, so appending one entry is one write and a recorded diff reads one response per line. Each line is

{"request":{"method":...,"path":...,"body":...},
 "response":{"status":...,"headers":...,"body":...}}

Only the request method, path and JSON body are written, never request headers, so an API key cannot reach a fixture that you commit. Of the response headers only content-type and x-typesafe-request-id are kept. The format is what record/2 writes and replay/2 reads; treat a fixture as generated output and re-record it rather than editing it by hand. A fixture written as a whole-file JSON array by an earlier version still loads.

Every response with an HTTP status is recorded, including 4xx and 5xx, so a test can replay the error path too. A transport failure never reached the server and is not recorded. A run that was retried records each attempt, so replaying it reproduces that same sequence: give the replay client a retry policy (replay(path, retry: [max_retries: 2])) when you want the retry to be replayed rather than surfaced.

Ordering

Entries are appended as responses arrive. Under TypeSafeAPI.evaluate_many/4 that order is the order the concurrent calls finished in, not the order of the input states, and it differs from run to run. Body matching (the default) is unaffected, because each request finds its own recording wherever it sits. match: [:method, :path] pairs requests to recordings by position instead, so use it only for fixtures recorded by sequential calls.

Unlike a missing stub, a request that matches no recording raises rather than answering 422, because the message is the whole value of a fixture mismatch. Under TypeSafeAPI.evaluate_many/4 that raise happens inside a task, so replay a fan-out only from a fixture that holds a recording for every state.

Concurrency

Stubs use Req.Test, which follows the ownership model of Mox: call Req.Test.set_req_test_from_context/1 in setup (or the typesafe_stubs/1 helper here) and stubs are private to each async test. The stubs a client accumulates are tracked per process, so build them from the test process; the tasks TypeSafeAPI.evaluate_many/4 starts reach them through $callers the same way Req.Test ownership does.