Design notes

Why this library is shaped the way it is, and the decisions behind the parts that are not obvious from the code alone.

Architecture

The library is two layers. TypeSafeAPI.HTTP is the thin layer: it knows about Req, retries, and the TypeSafe AI System One HTTP API, and returns raw-ish results. TypeSafeAPI (the facade) and TypeSafeAPI.SystemOne are the typed layer: they validate input, encode it in the shape the API expects, call the HTTP layer, and decode the response into typed structs (TypeSafeAPI.Result, TypeSafeAPI.Answer.*, TypeSafeAPI.Error). Callers who want raw maps can use TypeSafeAPI.HTTP directly; everyone else uses TypeSafeAPI.

TypeSafeAPI.Client is a plain struct, not a supervised process. There is no GenServer, no application-managed state, and no client registry: a client is just configuration (base URL, API key, Req options, retry policy) threaded explicitly through calls. This keeps the library usable from anywhere — LiveView, Task, Oban job, IEx — without asking the caller to start anything under a supervision tree first.

TypeSafeAPI.SystemOne holds the evaluate/4 pipeline (validate, encode, post, decode) alongside prepare/1 and evaluate_prepared/4, which let evaluate_many/4 validate and encode a question set once instead of once per state. The prepared set is a TypeSafeAPI.SystemOne.Prepared struct, so the functions that take one match on it instead of trusting the shape of a bare map.

Why Req

The library is built on Req, and that is not a free choice: Req brings Finch, Mint, NimblePool, Jason, and MIME along with it. Counting only runtime dependencies (excluding credo, dialyxir, and ex_doc, which are dev/test tooling, and excluding plug, which this library also declares but only as optional: true, needed solely for TypeSafeAPI.Test), mix deps.tree --format plain shows this client pulling in 9 packages:

For comparison, a bare Mint-based client would pull in 2 packages (mint and hpax), and a codegen-based rival built against the OpenAPI spec pulls in roughly 23, once its generated request/response validation, its own HTTP client choice, and their transitive dependencies are counted. Req sits in between: heavier than hand-rolled Mint, much lighter than a full codegen stack.

That tradeoff is worth it for three reasons:

• Req is the Elixir ecosystem's default. Most Elixir services that talk HTTP already depend on it, directly or transitively, so for many users this library adds no new dependency tree at all — and for everyone else, Req is a dependency they are likely to meet again.
• The offline test stubs are built on it. TypeSafeAPI.Test stubs requests with Req.Test, which is what makes TypeSafeAPI.evaluate/4 testable against canned answers with no fixtures, no HTTP mocking library, and no real network access. That convenience comes from Req specifically, not from HTTP clients in general.
• It keeps this library's own transport code small. Connection pooling, HTTP/2, timeouts, and TLS are Finch and Mint's problem, not this library's. The alternative — hand-rolled connection ownership on top of bare Mint — would trade five dependencies for a few hundred lines of code this library would then have to maintain and get right.

A few things worth knowing if you're weighing this yourself:

• Req is pre-1.0, but stable in practice: its public API has been steady across the 0.x line this library has tracked, and the underlying Finch/Mint stack is used in production well beyond Req itself (Phoenix's own test client, for one).
• Jason ships transitively but is unused. This library encodes and decodes JSON with Elixir's built-in JSON module (see "Differences from the official SDKs" below); Req 0.7 still declares Jason as a hard dependency, so it appears in mix.lock regardless.
• You can share an existing Finch pool instead of letting this library start its own, with the finch: client option. That's the escape hatch if your application already runs Finch and you'd rather not spin up a second pool, and the only way to control pool size. It is a first-class option rather than something passed through req_options because Req refuses :finch and :connect_options together: the client has to know, so it can stop sending the connect timeout it otherwise sets on every request.
• The library starts one process. TypeSafeAPI.Application supervises a single Task.Supervisor for TypeSafeAPI.FanOut, so batch tasks are unlinked from the caller. An async_stream over linked tasks propagates any exit other than a kill-on-timeout, so one raising task would take down the calling process and every sibling request with it — the opposite of what on_error: :collect promises.

Atom-key safety and the Keys registry

Decoded API responses are handed to callers as structs with atom keys, which means turning JSON object keys into atoms somewhere. Elixir atoms are not garbage collected, so converting arbitrary server-controlled strings to atoms is a memory-exhaustion risk. TypeSafeAPI.Keys is a small fixed registry of the atoms this library will ever need (:model, :input_tokens, criteria keys, and so on); decoding only ever looks up already-known atoms, never calls String.to_atom/1 on response data. A response field this library does not recognize is dropped rather than atomized.

Ordered JSON encoding

Choice option order and question order matter to the model: the API reads them as presented, not as an unordered set. Elixir's Map does not preserve insertion order, so anything that needs to survive to the wire in caller order needs its own carrier. TypeSafeAPI.JSON.OrderedObject is a tiny struct with a JSON.Encoder implementation for exactly this; Question.encode_all/1 also uses it for the top-level questions object so question order on the wire matches caller order. Because the questions object is a list of pairs rather than a map, two ids that collide on the wire (:billing and "billing", or the same id twice) would emit a duplicate JSON key instead of overwriting one another, so Question.normalize/1 rejects them with a :validation error, the way Choice.validate/1 already rejected duplicate option keys.

Question.encode_all/1 returns a TypeSafeAPI.JSON.Encoded, which serializes the ordered object once at construction and encodes as those cached bytes wherever it appears. That is what makes a prepared question set cheap across states: evaluate_many/4 splices the same iodata into every request body instead of walking and serializing the question set once per state. The bytes and the term are captured together and nothing mutates the struct afterwards, so they cannot drift.

Choice.new/2 rejects maps outright. It once accepted one up to 32 entries, on the belief that small maps preserve insertion order; they do not — a small atom-keyed map iterates by atom index, so the order was arbitrary at any size. Option order is what the model sees, so criteria are keyword lists or [{key, description}] lists only (with no limit of their own, aside from the API's 255-option cap), and a map raises ArgumentError at construction.

Score level shapes

A Score level may be a plain string or a {label, description} pair. The API itself has no label concept — a level is just a description string — so a {label, description} level is sent to the wire as a structured description ({"label": ..., "description": ...}), which the API accepts as an opaque description value, and the label is what TypeSafeAPI.Answer.Score.label and levels report back. Plain string levels report the description itself as the label. label is always a string: a structured level without a label falls back to a truncated inspect/1 of itself, and the answer's description field always carries the winning level exactly as written, independent of what the label computation does with it.

TypeSafeAPI.Answer.Score.normalized/1 (score divided by the top level index) exists because the composite-scoring pattern needs it and every caller would otherwise write it by hand.

Answer.gate/2 on a Noul answer uses max(noul, 1 - noul), since Noul answers carry no separate confidence field — a confident "no" gates the same as a confident "yes".

Errors and retries

503 maps to :overloaded, alongside 529 (the header TypeSafe AI's own docs call out for overload). Other 5xx map to :server_error. An HTTP 408 maps to :timeout: the server is reporting the same thing a transport timeout does, and making callers match two types for "nothing came back in time" buys nothing. That leaves :unexpected for its real meaning — a response this library could not interpret — rather than as a bucket holding both a broken API and a broken parse. TypeSafeAPI.Error.retryable?/1 tells the transient types apart without a caller re-deriving the retry policy from status.

A connection-pool checkout timeout deserves its own note. Finch does not return an error for it; Finch's HTTP/1 pool reraises a bare RuntimeError, which Req never normalizes and which therefore escapes the call as an exception — breaking the "every failure is an {:error, %Error{}}" promise exactly when the pool is saturated. This library wraps the Req adapter to turn it back into %Req.HTTPError{reason: :pool_timeout}, which maps to :timeout and is always retried regardless of method, since the request was never sent.

Two response shapes carry error detail: 422 responses have detail as a list, 400 responses have it as a map (error_type, message). TypeSafeAPI.Error reads both. Every 400 seen in practice was a caller mistake (unknown model, too many Choice options), so 400 maps to :validation.

Per-call options are validated with NimbleOptions.validate!/2 and raise on misuse, while questions and state return {:error, %TypeSafeAPI.Error{type: :validation}} instead of raising. The distinction is deliberate: options are code written by the developer calling the library, while questions and state are data that may ultimately come from end users and should fail as a value, not a crash.

Constructors never validate. noul/2, choice/2, and score/2 build structs and nothing more; evaluate/4 validates the whole question set once and returns an error value. An earlier idiom pass briefly made noul/2 raise on a misspelled criteria key so a typo could not be dropped silently; that was reverted in favor of the validator reporting the bad key, plus TypeSafeAPI.Question.validate!/1 for callers who build questions at compile time and want the raise at that line.

instructions is optional on every question type, matching the API: only type (plus criteria, for a Choice or a Score) is required. Rather than adding a separate option, the existing first positional argument simply accepts nil — and noul/0,1 default it — so every existing call site keeps working. A nil is dropped from the encoded wire map rather than sent as null, since an absent key and a null one are not the same request.

Timeout options for batch callers

evaluate_many/4 has two separate timeout knobs. :timeout is the single HTTP attempt timeout, exactly as in evaluate/4; :task_timeout caps one state's whole task. They default so that a task is never killed while a legitimate retry is still in flight: :task_timeout defaults to the retry budget plus one attempt timeout (40 s with defaults), or 60 s when the budget is disabled. This split exists because a caller who wants to shorten a single HTTP attempt (say, to fail fast on one slow question) should not have to build a second client with a different Req timeout just to do it.

The names matter more than the split. An earlier version had :timeout mean the task and :attempt_timeout mean the attempt, so evaluate_many(client, states, qs, timeout: 5_000) written by analogy with evaluate/4 silently meant something else — same name, same unit, different thing, no error. The batch function now keeps :timeout meaning what it means everywhere else, and the new concept got the new name.

The retry budget is a gate on starting a retry, not a deadline on an attempt: worst-case wall time is budget plus one attempt timeout, and TypeSafeAPI.FanOut derives its default task timeout from exactly that sum, so the two defaults have to move together.

Neither knob touches the connect timeout. Req picks a Finch pool by connect_options, so feeding a per-call timeout into it starts a pool per distinct value and throws connection reuse away. The connect timeout is therefore a client setting, connect_timeout (default 5 000 ms), and the per-call :timeout sets receive_timeout only: calls that differ in timeout still share one pool.

Differences from the official SDKs

• Milliseconds everywhere. The official Python SDK's retry policy uses float seconds (backoff_initial: 0.5, timeout: 30.0). Elixir convention is integer milliseconds (Process.sleep/1, Req timeouts, Task.async_stream), so TypeSafeAPI.Retry uses backoff_initial: 500, backoff_max: 5_000, budget: 30_000. The semantics are identical.
• Jason is present transitively, but unused. The library uses Elixir's built-in JSON module for all encoding and decoding, but Req 0.7 declares Jason as a hard dependency, so it appears in mix.lock. Nothing here calls it. Req's own body decoding is disabled (decode_body: false) so responses go through JSON.decode/1.
• Connection-error retries are off for POST /v1/systemone. The official SDKs retry them unconditionally, but a connection that is reset or times out may still have reached the server, the endpoint is not idempotent, and there is no idempotency key, so a retry can double-bill an evaluation. retry_connection_errors defaults to :auto, which TypeSafeAPI.Retry.for_method/2 resolves to true for GET /v1/models and false for POST when the policy is attached. retry_timeout_errors defaults to :auto for the same reason and a stronger one: a timed-out attempt is the failure most likely to have reached the server. An explicit boolean, on the client or per call, always wins, and status-driven retries (429, 503, 529, Retry-After) are untouched.
• A server-requested delay is clamped, not obeyed. retry-after: 0, an empty header and an HTTP date a skewed clock reads as past all parse to zero, and honoring that verbatim turns the policy into a tight loop against a server that is already struggling while the wall-clock budget never advances. The honored delay is floored at retry_after_min (100 ms) and capped at retry_after_max (60 s), and otherwise obeyed: a server asking for less than the backoff schedule gets it, since it knows its own load.
• plug is an optional dependency. TypeSafeAPI.Test builds on Req.Test, which needs Plug to construct responses. Users who want the stubs add {:plug, "~> 1.16", only: :test}; everyone else does not pay for it.

Verified against the live API (2026-09-17)

A live run with a real key settled several things the docs left open:

• jev-latest and jev-preview both resolve to a versioned id such as jev-1.13.0, which is what result.model reports. The bare id jev, shown in the Python usage docs, is rejected with a 400 "Unknown model".
• Limits are enforced server-side: 256 Choice options and 11 Score levels are rejected, so Choice.validate/1 enforces 255 and Score keeps its 2 to 10 bound. A one-level Score is accepted by the API but still rejected locally, since it cannot produce a meaningful score.
• 400 responses carry detail as a map (error_type, message), unlike 422s whose detail is a list. TypeSafeAPI.Error reads both, and 400 maps to :validation since every 400 seen was a caller mistake (unknown model, too many options).
• x-typesafe-request-id is on every response, success included, so TypeSafeAPI.Result carries request_id and TypeSafeAPI.HTTP.request/5 exposes the full response for raw callers.
• release_date on the models endpoint is a full ISO 8601 datetime, while the schema describes it as "formatted as YYYY-MM-DD". TypeSafeAPI.Model normalizes it to the Date the schema promises, reading the date part as written rather than shifting it into UTC, and keeps the original string on release_date_raw. An alias with an unparseable date leaves release_date nil.
• The API knows a bounding_box question type that is not in the public docs. This library does not model it; a request with an unknown type is rejected locally.

Verified against the live docs (2026-09-16)

• Models endpoint is GET /v1/models, returning {"models": [{name, description, release_date}]}. The public API reference does not document it; the path and shape come from the official Python SDK's constants and generated schema.
• retry-after-ms beats Retry-After when both are present. The Python SDK checks retry-after-ms first. Retry-After may be seconds (fractional allowed) or an HTTP date; negative or unparseable values are ignored and backoff applies.
• Usage tokens are required, and this library still tolerates their absence. The OpenAPI schema requires usage on every response, with input_tokens and output_tokens both required non-nullable integers. TypeSafeAPI.Usage nonetheless reports nil for a missing or malformed count, and TypeSafeAPI.Result decodes a missing usage to an empty one: a correct set of answers is not worth failing over a token count that drifted. That is a defensive choice, not something the spec permits, and test/typesafe_api/result_test.exs pins the spec so the drift stays visible.
• Score takes 2 to 10 levels, not just "at least 2". Local validation enforces both bounds.
• Descriptions may be structured. Instructions, Choice option descriptions, Score level descriptions, and Noul criteria all accept a string, map, or list. Validation allows all three.
• The API accepts no per-request timeout or extra behavioural headers. The official SDKs send X-TypeSafe-SDK, X-TypeSafe-Runtime, and X-TypeSafe-Retry-Count for diagnostics. This library sends User-Agent and X-TypeSafe-Retry-Count on retries.

Documentation conventions

Every public options keyword list is a NimbleOptions schema exposed through an options_schema/0, and the facade's docs interpolate NimbleOptions.docs/1 at the call site rather than linking away, so TypeSafeAPI.evaluate/4 reads the same as the module that implements it. Every module belongs to a groups_for_modules group in mix.exs; a module with no group is a docs bug.

An adversarial docs review (three personas: first-time integrator, on-call engineer, SDK expert) drove the README's onboarding section, the error and telemetry field prose, the end-to-end timeout timeline in TypeSafeAPI.Retry, and the "Writing good questions" section.

Toolchain

CI runs OTP 27 and 28. Nothing in the library is OTP-version specific.

Things worth revisiting

• Rate limits are unpublished. evaluate_many/4 defaults to max_concurrency: 8 as a guess. When TypeSafe publishes limits, raise the default and document it.
• Retry counting on Req.Test transport errors. The custom retry step re-runs the full Req pipeline, including request steps, on each retry. That is what Req's own step does; it is worth keeping in mind if anyone adds a request step with side effects.
• HTTP-date Retry-After parsing is a small RFC 7231 fixdate parser in TypeSafeAPI.Retry, so the library does not depend on inets.

Refreshing the OpenAPI snapshot

priv/openapi.json is a vendored, pretty-printed copy of TypeSafe's OpenAPI document (currently info.version 0.2.0). It ships in the Hex package (see files in mix.exs) and is the source of truth for test/typesafe_api/openapi_spec_test.exs, which asserts the request/response shapes this client relies on: the two paths and their bearer security, which question/answer fields are actually required versus optional, the minItems/minProperties bounds, and the 422 HTTPValidationError shape. If TypeSafe changes the API in a way that would break this client, that test is meant to fail first.

To refresh it: obtain the current openapi.json from TypeSafe AI (their docs site or OpenAPI endpoint), replace priv/openapi.json with it, then run

mix test test/typesafe_api/openapi_spec_test.exs

and fix any failures — either the client (if the API genuinely changed underneath it) or the test (if a local policy, like the Score 2-to-10 level bound, is intentionally stricter than the spec and needs a fresh comment explaining why).