Error handling and the response envelope

Two audiences. Part 1 is how errors are produced inside the API. Part 2 is the contract a client can rely on.

The rules distilled from this document live in AGENTS.md; this is the reasoning behind them.

---

Part 1 — Contributors

One rule underpins everything: throw typed errors, never build an error response by hand. Handlers, services, and normalizers throw; a single onError hook (app.onError(handleGlobalError) in app.ts) catches everything, normalizes it, logs it, and shapes the response. Don't try/catch in a controller to format a response — let it bubble.

The flow of one request

1. Validation — zod validates params. On failure the hook in helpers/zodError.ts throws BadRequestError with the flattened issues in details (public — safe to show, it's just which field was wrong). You don't call it; registering the schema on the route is enough.
2. Controller — reads ctx.req.valid("query" | "json"), calls the service, returns ctx.json(ok(ctx, data), 200). No error handling of its own.
3. Service → UpstreamHandler.call() — any upstream failure (transport, non-2xx, or a provider error hidden in a 200 body) is translated into ServiceUnavailableError (503, retryable) or SourceError (500, terminal) for you. The upstream status/body never reach the client — see UpstreamHandler below.
4. Normalizer — throws ContractError (via contractFail(route, reason) in utils/normalizer.utils.ts) on a shape mismatch, never echoes the raw upstream body.
5. handleGlobalError (helpers/globalErrorHandler.ts) — flattens any thrown value into { code, status, message, retryable, details } (defaults for a non-AppError throw: INTERNAL_ERROR / 500 / not retryable / details: null), logs at error (status ≥ 500) or warn (4xx) with error.context attached for diagnostics, then calls fail() to build the body.

The public/private split

Every AppError carries two optional bags:

• context — private: diagnostics, upstream status codes, provider names. Logged only, never sent to the client.
• details — public: safe, client-facing extras like which field failed validation. Echoed verbatim in the response.

There is no trusted/expose flag — the split is just "which field did you put it in." Never put a raw third-party body, upstream status, or provider name in details.

Error classes

All from @thenetaji/packages, defined in packages/errors/. Every constructor takes (message, { cause?, context?, details?, retryable? }).

The one deliberate deviation

apify.controller.ts breaks "let it bubble" on purpose: it is fire-and-forget telemetry, so it swallows ClickHouse errors, drops invalid events, and still returns 202. Don't copy that pattern for data routes.

UpstreamHandler

Every third-party route call goes through an UpstreamHandler subclass (apps/api/src/helpers/upstream/, base class in upstream-handler.ts). call(url, headers?) has exactly one job: on success return the raw upstream body; on any failure, throw the right typed error. It never returns an error to the caller — it only throws or returns data.

Three failure paths, all in upstream-handler.ts:

1. Transport failure (timeout/connection error after the HTTP client's own retries) → ServiceUnavailableError (503, retryable). The upstream status is never involved — there wasn't one.
2. Provider non-2xx → 429 or 5xx become ServiceUnavailableError (503, retryable); everything else becomes SourceError (500, terminal). The real upstream status is logged in context (stage: "provider_http"), never echoed to the client.
3. A 200 OK whose body encodes a provider failure (e.g. TikHub's envelope code field) → the subclass's detectProviderError(body) hook returns a ProviderFailure { providerCode, retryable } or null. retryable decides 503 vs 500, same as above.

Concrete clients, all re-exported from index.ts:

• worker (worker.ts) — our own scraper backend. No retries configured (retryStatuses: []), propagates request/trace context to it.
• tikhub (tikhub.ts) — implements detectProviderError: a non-200 code in the JSON envelope is a provider failure (429/5xx retryable, everything else — including 400 — terminal). Injects Authorization: Bearer from TIK_API. Retries 429/500/502/503/504 at the transport layer. (There is no separate tikhubLinkedin handler — linkedin-pro uses the same tikhub instance.)
• rapidapi (rapidapi.ts) — injects x-rapidapi-host/x-rapidapi-key. No detectProviderError override (RapidAPI failures are HTTP-status-coded, handled by the base class).

Adding a new site: subclass UpstreamHandler, set provider, override detectProviderError only if the provider hides failures inside a 200 body.

Normalizers — null vs throw

Shared helpers live in apps/api/src/utils/normalizer.utils.ts: requireObject(data, route, reason), validateAndReturn(data, schema, route), contractFail(route, reason) (throws ContractError with { route, reason } in context), and removeTikHubInternals(data) (strips TikHub envelope metadata — request_id, cache_*, time*, docs, support, router, params, message*, code — before validating/returning).

The razor: null is only for a documented empty signal, decided by the site's own contract, not a generic "couldn't parse" fallback.

An upstream that answered is a 200. When a provider or the worker replies that the post, profile or video does not exist, the call succeeded and we paid for it. Return 200 with data: null and details: { reason: "record_not_found", message } (ok(ctx, null, details), as in #316 and instagram-v2 media info). Do not turn that answer into a NotFoundError (404) or a SourceError (500): callers, Apify actors and alerts read both as failures.

• Worker-backed sites (instagram, meta): raw === null → return null (the worker's own not-found signal — see Worker contract).
• tikhub sites: an envelope data: null is the documented not-found (see linkedin-pro.normalizer.ts).
• tiktok-shop product detail: a non-zero envelope-level error_code with no payload → null (see isProductNotFound in tiktok-shop.normalizer.ts). This is the only not-found signal for that endpoint — any other missing field is a contract break.
• Lists always return an empty array/collection on zero results — never null.
• Anything else missing or shape-invalid — a required field absent, or validateAndReturn's zod schema failing — throws ContractError via contractFail. Services do not try/catch-swallow this into null; a broken contract surfaces as a 500.

Worker contract

worker is our own scraper backend, and it is the one upstream that is not masked. The masking rule exists so a resold vendor's error text cannot leak and tell a caller who we buy from; there is no vendor here, so its message, reason and retry_after_ms are forwarded.

It signals not-found as HTTP 200 with a JSON body of null; normalizers/services recognize raw === null and return null, which the API wraps as { "data": null }.

The worker sends { type, reason, message, retry_after_ms, context }. type selects a row in WORKER_ERRORS (helpers/upstream/worker.ts); context is diagnostics and stays in logs. Every type in apps/worker/src/api/errors.py should have a row, which is what gives it the right status and retryability. An unmapped type no longer falls through to a masked 500: it publishes the worker's own message under a retryable 503, because a type we have not mapped is still a type we named.

PROXY_EXHAUSTED carries two reasons. proxy_account_unpaid is the proxy answering CONNECT with 402 — our account is out of credit, and no exit identity, retry or rotation can change that. proxy_exhausted is every other way the tier can refuse to give us a URL. They share a type and a code because clients act on both the same way; they differ in reason because on 2026-08-19 an unpaid balance was published to callers as "the source challenged this request" and cost hours of anti-bot diagnosis. Neither message is forwarded, and neither publishes details.worker: an unpaid invoice is our infrastructure. This is the publicAs rule below, not a per-row opt-out.

UPSTREAM_BLOCKED narrows the same way, and its narrower words were in production long before this table described them. The row's own blocked_by_source is what an unclassified refusal publishes and stays valid; the two below say which refusal, because they are fixed by different things:

replay_blocked is the one to recognise, because it is published by nine site route modules (clutch, indeed, yelp, wellfound, trustpilot, tripadvisor, google_maps, glassdoor and more) and it means the mint was not the problem. Reading it as a mint failure sends the next reader to the browser and the proxy tier when the credential in hand was good.

soft_block_shell is the worst failure shape available and the reason it has its own word: every check short of reading the content calls a shell a success. See is_shell in apps/worker/src/sites/yelp/parsers.py.

source_unreachable is the opposite direction and it matters just as much: Clutch publishes it when a request exhausts its exit-rotation budget without the site ever answering. That is our tunnel, not their anti-bot, and it is kept distinct for the same reason proxy_account_unpaid is — a dead proxy reported as a block sends a reader to the wrong system.

mint_failed narrows further on every route that mints in a browser. This is not a Google Maps-only vocabulary — the shared credential pool (BrowserCredentialPool._mint, src/core/browser_credential.py) publishes it for every site built on that pool (glassdoor, booking, trustpilot, tripadvisor, yelp and more), and BrowserSessionPool._mint (src/core/browser_fetcher.py, e.g. Indeed) publishes the reasons that apply to a browser it keeps in the request path. The row's own word is still what an unclassified failure publishes; the narrower ones say which half of the mint failed, because the halves point at different suspects and are not distinguishable from outside the container otherwise:

Google Maps reviews has its own _mint and publishes mint_capture_miss, mint_incomplete, mint_unvalidated and mint_transport — it has no mint_challenge_unsolved case, because it never separates "no proof cookie" from "missing header" the way the pool does. Every pool-backed site is the opposite: it has no mint_incomplete case, because a captured request with a missing proof cookie there IS the challenge-unsolved case. A route that has not been wired to read reason off its mint exception keeps publishing mint_failed, which stays valid.

SITE_UNDER_MAINTENANCE is the one row that is nobody's failure. The site is parked on purpose — SITE_MAINTENANCE names it, the API refuses before it calls the worker, and the worker refuses again for direct callers. It is deliberately NOT enabled: false in sites.yaml, which means there is no working implementation at all: that unmounts the route, so a caller gets a 404 saying "this never existed", with no retry hint and nothing to bring a well-behaved actor back. Retryable, unlike PROXY_EXHAUSTED, because a park ends without the caller doing anything; the hour-long Retry-After is what stops that becoming a hot loop. Its message IS forwarded — under_maintenance is not an internal reason, and naming the parked surface is more useful to a caller than the house sentence.

Two rows are deliberately non-retryable despite a 5xx. CONTRACT_ERROR is a shape change — rotating identity cannot fix it, and a retryable classification would have callers retrying a broken parser forever. PROXY_EXHAUSTED clears only when a human tops the account up, so inviting backoff sends clients at a wall.

Message forwarding is the default. It used to be opt-in for three rows, and the other fifteen published the registry's sentence — so "tripadvisor: detail refused after 3 exits" reached a caller as "The data source could not complete the request." and an operator had to read a log to learn anything. Every string in apps/worker/src/api/errors.py is one we wrote, so it is published.

Three things still hold it back, and only three:

• A source path or a vendor name. A message matching either falls back to the row's sentence. This is the whole remaining secret: which proxy network, solver or reseller sits behind a route.
• An internal reason — one the registry marks publicAs because the true slug discloses infrastructure (proxy_exhausted) or how we detect a block (replay_blocked, soft_block_shell). Those four publish the PUBLIC sibling's sentence and no details.worker, because publishing "our proxy account is out of credit" beside reason: "operator_action_required" would undo the swap in the same response.
• /fetch, which never forwards at all; echoing its SSRF validator would confirm what a prober reached.

The safe half of the worker's context is published as details.worker: site, route, operation, status, attempt, retried. A whitelist, not a denylist — context is where the worker is told to put everything private. error_type is deliberately excluded even though the logs carry it: its values are the ones that name infrastructure (proxy_out_of_credit is a real one).

The worker stamps site and route onto every AppError centrally, from the attribution ContextVar, so a raise site does not have to remember to.

Parsing ownership differs by site: LinkedIn's worker responses are already parsed by the worker itself, so linkedin.service.ts calls worker.call(...) directly and there is no linkedin.normalizer.ts — the API is a thin proxy. Instagram and Meta responses are parsed and validated in the API's own normalizers (instagram.normalizer.ts, meta.normalizer.ts).

Route-level error docs

Every route documents 400, 429, 500, and 503 via defaultError (utils/openapi.utils.ts), so /openapi.json and /llm.txt — and therefore docs.toolzerhub.com, which is built from the spec — already carry this contract. Don't hand-write per-route error docs.

---

Part 2 — API consumers

Defined in types/response.schema.ts, built by response.utils.ts. This is ToolzerHub's own stable error envelope (RFC-9457-inspired, not a strict implementation).

There is no success boolean — the HTTP status line is the source of truth.

• Success: { "data": ... } — data can legitimately be null (a documented not-found for a single-item lookup). Don't treat null data as an error.
• Error: { "error": { "code", "status", "message", "retryable", "details" } }
  • code — stable category slug (catalog below); the thing to switch on in client code.
  • status — mirrors the HTTP status.
  • message — human, developer-facing detail. Not for end-user display, not guaranteed stable.
  • retryable — true means retry with backoff is sensible (SOURCE_UNAVAILABLE, RATE_LIMITED, internal NETWORK_ERROR); false means don't retry as-is.
  • details — structured extras or null (e.g. field-level validation issues on a 400). Third-party/upstream internals are never here.

A retryable error that knows how long to wait also sends a Retry-After header (whole seconds, never 0), carrying the same interval as details.retryAfter. Honour it rather than the body if you already have retry machinery that reads the header.

Every response carries an X-Toolzer-Request-Id header (never in the body) — quote it in support requests.

Code catalog

Client rules

• Auth: every /v1 route requires the x-api-key header (apiKeyAuth middleware, middlewares/apiKeyAuth.middleware.ts). A missing/wrong key returns 401 UNAUTHORIZED.
• Handle errors by error.code, not by status alone.
• Retry only when error.retryable === true (i.e. SOURCE_UNAVAILABLE or RATE_LIMITED) — with backoff. Everything else (INVALID_REQUEST, UNAUTHORIZED, FORBIDDEN, NOT_FOUND, SOURCE_ERROR, INTERNAL_ERROR) will not succeed on retry without changing the request.
• { "data": null } on a lookup endpoint is a successful not-found, not an error — don't treat it as a failure or retry it.
• Query params use human-friendly enums, not raw upstream codes — e.g. LinkedIn date_posted: "past_week", job_types: "full_time,contract", workplace: "remote"; TikTok Shop filter_value: "5_star", sort_field: "price". Pass the enum string; the service maps it to the upstream's integer/code internally.
• Discover the contract from generated docs, don't hand-maintain a client spec: /openapi.json (full OpenAPI doc), /llm.txt (plain-text summary meant for LLM consumption), or docs.toolzerhub.com (the rendered reference, built from the spec).
• Quote X-Toolzer-Request-Id (a response header, never in the body) when reporting an issue.