TypeSafeAPI.Telemetry (TypeSafe AI v0.1.0-alpha.3)

Telemetry events emitted by this library, plus a small logger handler.

Every HTTP call is wrapped in a :telemetry.span/3, which yields:

[:typesafe_api, :request, :start] with measurements %{system_time: integer}
[:typesafe_api, :request, :stop] with measurements %{duration: integer}
[:typesafe_api, :request, :exception] with %{duration: integer} and the usual kind, reason, stacktrace metadata

Durations are in native time units; convert with System.convert_time_unit/3.

Metadata

The start event carries method, path, model, and question_count. The stop event adds:

status - HTTP status, or nil when no response arrived
retry_count - retries performed by TypeSafeAPI.Retry
input_tokens / output_tokens - from the response usage, or nil
error - a TypeSafeAPI.Error when the call failed, otherwise nil

question_count and model describe the request as sent, so a raw TypeSafeAPI.HTTP.post/4 call still reports them when the body has them.

Failures are stop events, not exception events

Every failure this library knows how to name is a normal outcome of the span, so it arrives as a :stop event with metadata.error set to a %TypeSafeAPI.Error{}. That covers :auth, :rate_limited, :overloaded, :timeout, :connection, :unexpected and :validation alike: an expired key, a 429 that outlived its retries, a socket that never opened, and a body the library could not decode all look the same to a handler, and all of them come with a duration. The :exception event fires only when code raises, which in practice means a bug in a handler, in a Req step, or in this library. A handler that watches only :exception will see none of the failures that matter operationally.

Attaching a handler

:telemetry.attach("typesafe-watch", [:typesafe_api, :request, :stop], fn _event, _m, meta, tokens ->
  if meta.error, do: Logger.warning("typesafe #{meta.error.type}: #{Exception.message(meta.error)}")
  :counters.add(tokens, 1, meta[:input_tokens] || 0)
  :counters.add(tokens, 2, meta[:output_tokens] || 0)
end, :counters.new(2, [:write_concurrency]))

Read the running totals back with :counters.get(tokens, 1) and :counters.get(tokens, 2). The duration in the measurements map is in native time units; turn it into milliseconds with System.convert_time_unit(duration, :native, :millisecond).

Metrics

metrics/0 returns Telemetry.Metrics definitions for these events, ready to drop into a Phoenix LiveDashboard or a Telemetry.Metrics.ConsoleReporter. It needs {:telemetry_metrics, "~> 1.0"}, which this library lists as an optional dependency. See the LiveDashboard guide.

Logging

attach_logger/1 attaches a handler that logs one line per request: at :level when the call succeeded, and at :error_level (:warning by default) when it failed. Failures arrive as :stop events, so logging them at the success level is how they disappear from a production :warning logger; the two levels keep the happy path quiet without hiding the failures.

The default success level comes from config :typesafe_api, log_level: first, then TYPESAFE_LOG_LEVEL, then :info. warn is accepted as a spelling of :warning; anything else raises rather than silently logging every request at :info.

Summary

Functions

attach_logger(opts \\ [])

Attaches a Logger handler for request stop and exception events.

detach_logger()

Detaches the handler attached by attach_logger/1.

metrics()

Telemetry.Metrics definitions for every [:typesafe_api, :request] event.

prefix()

The telemetry event prefix, [:typesafe_api, :request].

span(metadata, fun)

Runs fun inside a [:typesafe_api, :request] span.

Functions

attach_logger(opts \\ [])

@spec attach_logger(keyword()) :: :ok | {:error, :already_exists}

Attaches a Logger handler for request stop and exception events.

Calling it again replaces the handler rather than returning {:error, :already_exists}, so the levels can be changed at runtime.

Options

:level - level for calls that succeeded. Defaults to config :typesafe_api, log_level:, then TYPESAFE_LOG_LEVEL, then :info.
:error_level - level for calls that failed, i.e. :stop events carrying a %TypeSafeAPI.Error{}. Defaults to :warning.

Both accept any Logger level, plus warn as a spelling of :warning. An unrecognised level raises ArgumentError.

detach_logger()

@spec detach_logger() :: :ok | {:error, :not_found}

Detaches the handler attached by attach_logger/1.

metrics()

@spec metrics() :: [struct()]

Telemetry.Metrics definitions for every [:typesafe_api, :request] event.

Pass them to a LiveDashboard metrics: list, or to any reporter:

Telemetry.Metrics.ConsoleReporter.start_link(metrics: TypeSafeAPI.Telemetry.metrics())

The list covers request counts, durations, retries, errors by type and token usage; see the LiveDashboard guide for the names and for where to put them. Every definition reads the :stop or :exception event this library already emits, so nothing else has to be instrumented.

Requires {:telemetry_metrics, "~> 1.0"}; the function raises without it.

prefix()

@spec prefix() :: [atom()]

The telemetry event prefix, [:typesafe_api, :request].

span(metadata, fun)

@spec span(map(), (-> {result, map()})) :: result when result: term()

Runs fun inside a [:typesafe_api, :request] span.

fun must return {result, extra_metadata}; the extra metadata is merged into the stop event.