Runtime API

POST/v1/evaluate/output

Evaluate an LLM response before returning it to the user

Evaluate output text from LLM before returning to user.

Bearer token or X-API-Keyscope: org memberSubject to per-plan eval quotaoperation_id: runtime.evaluateOutput

Authentication

Send either Authorization: Bearer <token> or X-API-Key: <token>. Runtime token — create via POST /v1/orgs/{org_id}/tokens/runtime. Scoped to one project and environment. Requests without a valid credential are rejected with 401.

Where this runs

This endpoint is served by the ZNYX runtime you host, so the base URL is your own runtime host, not api.znyx.ai. Prompts, responses, retrieved context, and tool payloads are evaluated inside your boundary, and there is no supported production endpoint on our side that evaluates your application’s traffic. Text you paste into the console’s playground is the one exception, and it is not persisted.

SDK install

pip install znyx-sdknpm install @znyx/sdk

Code samples

Request

curl -X POST "$ZNYX_RUNTIME_URL/v1/evaluate/output" \
  -H "Authorization: Bearer $ZNYX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "request_id": "string",
  "tenant_id": "string",
  "app_id": "string",
  "agent_id": "default",
  "env": "prod",
  "text": "string",
  "metadata": null,
  "trace_id": null,
  "session_id": null,
  "span_id": null
}'

Response

application/json

Successful Response

{
  "request_id": "string",
  "decision": "ALLOW",
  "risk_score": 0,
  "policy_version": "string",
  "rule_hits": [
    {
      "rule_id": "string",
      "severity": "low",
      "message": "string"
    }
  ],
  "sanitized_text": null,
  "sanitized_tool_args": null,
  "user_message": null,
  "developer_message": null,
  "latency_ms": null,
  "trace_id": null,
  "session_id": null,
  "span_id": null,
  "detector_results": [
    {
      "detector_name": "string",
      "decision": null,
      "risk_score": 0,
      "latency_ms": 0,
      "rule_hits": [
        {
          "rule_id": "string",
          "severity": "low",
          "message": "string"
        }
      ],
      "transformed": false,
      "confidence": null,
      "calibrated_score": null,
      "label_scores": null,
      "model_version": null,
      "execution_mode": null,
      "fallback_path": null,
      "external_egress": false,
      "threshold": null,
      "layer_results": [
        {
          "execution_mode": "string",
          "decision": null,
          "native_score": null,
          "normalized_score": null,
          "confidence": null,
          "calibrated_score": null,
          "label_scores": null,
          "category": null,
          "threshold": null,
          "model_id": null,
          "model_version": null,
          "rubric_version": null,
          "latency_ms": null,
          "token_count": null,
          "cost_usd": null,
          "rationale": null,
          "evidence_spans": null,
          "external_egress": false,
          "fallback_reason": null,
          "selected": false
        }
      ]
    }
  ],
  "quality": null,
  "field_errors": [
    {
      "path": "string",
      "message": "string",
      "expected": null,
      "actual": null
    }
  ],
  "remediation": null,
  "pending_review_id": null
}

Schema: object

Header parameters

Name	Type	Required	Description
X-API-Key#header	string \| null	optional	-
authorization#header	string \| null	optional	-

Request bodyrequired

Field	Type	Required	Description
request_id	string	required	Unique identifier for this request
tenant_id	string	required	Tenant identifier
app_id	string	required	Application identifier
agent_id	string	optional	Agent identifier
env	string	optional	Environment (prod, staging, dev)
text	string	required	Text to evaluate
metadata	object \| null	optional	Optional metadata
trace_id	string \| null	optional	Distributed trace ID for correlation
session_id	string \| null	optional	Session/conversation ID for grouping
span_id	string \| null	optional	Span ID within a trace

Responses

Status	Description
200	Successful Response
422	Validation Error

Response schema

request_idrequiredstring

decisionrequiredDecision

risk_scorerequiredinteger

Risk score from 0-100

policy_versionrequiredstring

rule_hits

sanitized_textstring | null

Sanitized text if REDACT/TRANSFORM

sanitized_tool_argsobject | null

Sanitized tool args (for tool evaluation)

user_messagestring | null

Safe message to show end-user when blocked

developer_messagestring | null

Developer-facing explanation

latency_msinteger | null

Total evaluation latency in milliseconds

trace_idstring | null

Trace ID for distributed tracing correlation

session_idstring | null

Session/conversation ID echoed from request

span_idstring | null

Span ID within a trace echoed from request

detector_results

Per-detector timing breakdown

qualityQualityReport | null

Response quality scores (output context only)

field_errors

Field-level errors from structured output validation

remediationRemediationResult | null

Remediation action applied after detector decision

pending_review_idstring | null

Human review queue ID if ask_human remediation was triggered

Errors & what triggers them

Code	Trigger	Fix
401	Missing or invalid X-API-Key / Authorization header.	Check the token is still active - rotated tokens return 401 after the grace period ends.
403	Token does not have the `evaluate` scope.	Use a runtime token (POST /v1/orgs/{org_id}/tokens/runtime).
422	Request body failed Pydantic validation (missing tenant_id, bad context, etc.).	-
429	Monthly evaluation quota hit for your plan.	Upgrade via POST /v1/billing/checkout, or wait for the next monthly reset.
500	Detector crashed or resolver timed out. Typically transient.	Retry with backoff. If it persists, check Traces for the request_id.

Notes & examples

When to use this

Call /v1/evaluate/output after the LLM responds but before you send the response to the user. Output-context detectors check things input-context detectors cannot:

Hallucination — does the response cite sources that don't exist?
Exfiltration — is the model leaking parts of the system prompt?
Output PII — did the model regurgitate training-data PII?
Quality scoring — 7-dimension scoring (helpfulness, groundedness, etc.) for Growth+ plans.

Minimum viable pipeline

user_input → evaluate/input → (if ALLOW) → LLM → evaluate/output → (if ALLOW) → return to user

Add tool-call guardrails by inserting evaluate/tool between the LLM and the tool dispatcher.

Common pitfalls

Output detectors cost more than input detectors — hallucination and quality scoring both invoke a judge model. If p99 latency matters, disable quality scoring on the hot path and run it async via traces.
If you're using TRANSFORM on output (e.g. PII redaction on a customer-support bot), return the transformed text to the user, not the original.

POST /v1/evaluate/input
POST /v1/evaluate/stream — for streaming LLMs (evaluate tokens as they arrive).