Runtime API

POST/v1/evaluate/tool

Evaluate a tool / function invocation

Evaluate tool invocation against governance policies.

Bearer token or X-API-Keyscope: org memberSubject to per-plan eval quotaoperation_id: runtime.evaluateTool

Authentication

Send either Authorization: Bearer <token> or X-API-Key: <token>. Runtime token — create via POST /v1/orgs/{org_id}/tokens/runtime. Scoped to one project and environment. Requests without a valid credential are rejected with 401.

Where this runs

This endpoint is served by the ZNYX runtime you host, so the base URL is your own runtime host, not api.znyx.ai. Prompts, responses, retrieved context, and tool payloads are evaluated inside your boundary, and there is no supported production endpoint on our side that evaluates your application’s traffic. Text you paste into the console’s playground is the one exception, and it is not persisted.

SDK install

pip install znyx-sdknpm install @znyx/sdk

Code samples

Request

curl -X POST "$ZNYX_RUNTIME_URL/v1/evaluate/tool" \
  -H "Authorization: Bearer $ZNYX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "request_id": "string",
  "tenant_id": "string",
  "app_id": "string",
  "agent_id": "default",
  "env": "prod",
  "tool_name": "string",
  "tool_args": {},
  "tool_result": null,
  "metadata": null,
  "trace_id": null,
  "session_id": null,
  "span_id": null
}'

Response

application/json

Successful Response

{
  "request_id": "string",
  "decision": "ALLOW",
  "risk_score": 0,
  "policy_version": "string",
  "rule_hits": [
    {
      "rule_id": "string",
      "severity": "low",
      "message": "string"
    }
  ],
  "sanitized_text": null,
  "sanitized_tool_args": null,
  "user_message": null,
  "developer_message": null,
  "latency_ms": null,
  "trace_id": null,
  "session_id": null,
  "span_id": null,
  "detector_results": [
    {
      "detector_name": "string",
      "decision": null,
      "risk_score": 0,
      "latency_ms": 0,
      "rule_hits": [
        {
          "rule_id": "string",
          "severity": "low",
          "message": "string"
        }
      ],
      "transformed": false,
      "confidence": null,
      "calibrated_score": null,
      "label_scores": null,
      "model_version": null,
      "execution_mode": null,
      "fallback_path": null,
      "external_egress": false,
      "threshold": null,
      "layer_results": [
        {
          "execution_mode": "string",
          "decision": null,
          "native_score": null,
          "normalized_score": null,
          "confidence": null,
          "calibrated_score": null,
          "label_scores": null,
          "category": null,
          "threshold": null,
          "model_id": null,
          "model_version": null,
          "rubric_version": null,
          "latency_ms": null,
          "token_count": null,
          "cost_usd": null,
          "rationale": null,
          "evidence_spans": null,
          "external_egress": false,
          "fallback_reason": null,
          "selected": false
        }
      ]
    }
  ],
  "quality": null,
  "field_errors": [
    {
      "path": "string",
      "message": "string",
      "expected": null,
      "actual": null
    }
  ],
  "remediation": null,
  "pending_review_id": null
}

Schema: object

Header parameters

Name	Type	Required	Description
X-API-Key#header	string \| null	optional	-
authorization#header	string \| null	optional	-

Request bodyrequired

Field	Type	Required	Description
request_id	string	required	Unique identifier for this request
tenant_id	string	required	Tenant identifier
app_id	string	required	Application identifier
agent_id	string	optional	Agent identifier
env	string	optional	Environment (prod, staging, dev)
tool_name	string	required	Name of the tool being invoked
tool_args	object	required	Tool arguments (arbitrary JSON)
tool_result	string \| null	optional	Optional tool-result text re-entering context. When present, it is scanned for prompt injection (tool_output_injection, LLM01).
metadata	object \| null	optional	Optional metadata
trace_id	string \| null	optional	Distributed trace ID for correlation
session_id	string \| null	optional	Session/conversation ID for grouping
span_id	string \| null	optional	Span ID within a trace

Responses

Status	Description
200	Successful Response
422	Validation Error

Response schema

request_idrequiredstring

decisionrequiredDecision

risk_scorerequiredinteger

Risk score from 0-100

policy_versionrequiredstring

rule_hits

sanitized_textstring | null

Sanitized text if REDACT/TRANSFORM

sanitized_tool_argsobject | null

Sanitized tool args (for tool evaluation)

user_messagestring | null

Safe message to show end-user when blocked

developer_messagestring | null

Developer-facing explanation

latency_msinteger | null

Total evaluation latency in milliseconds

trace_idstring | null

Trace ID for distributed tracing correlation

session_idstring | null

Session/conversation ID echoed from request

span_idstring | null

Span ID within a trace echoed from request

detector_results

Per-detector timing breakdown

qualityQualityReport | null

Response quality scores (output context only)

field_errors

Field-level errors from structured output validation

remediationRemediationResult | null

Remediation action applied after detector decision

pending_review_idstring | null

Human review queue ID if ask_human remediation was triggered

Errors & what triggers them

Code	Trigger	Fix
401	Missing or invalid X-API-Key / Authorization header.	Check the token is still active - rotated tokens return 401 after the grace period ends.
403	Token does not have the `evaluate` scope.	Use a runtime token (POST /v1/orgs/{org_id}/tokens/runtime).
422	Request body failed Pydantic validation (missing tenant_id, bad context, etc.).	-
429	Monthly evaluation quota hit for your plan.	Upgrade via POST /v1/billing/checkout, or wait for the next monthly reset.
500	Detector crashed or resolver timed out. Typically transient.	Retry with backoff. If it persists, check Traces for the request_id.

Notes & examples

When to use this

Call this immediately before dispatching a tool call the LLM proposed. Input:

tool_name — e.g. search_customers, send_email, run_sql.
tool_arguments — the JSON arguments the LLM emitted.
tenant_id / app_id / agent_id / env — policy scope.

The endpoint returns the standard evaluation response. If it returns BLOCK, do not dispatch the tool — return the user_message to the user (or surface developer_message to the developer).

Policy knobs

In your policy under tools.*, declare which tools are allowed for each agent:

tools:
  allowed:
    - search_customers
    - summarize_document
  denied:
    - run_sql
    - exec_shell
  schemas:
    search_customers:
      required: [query]
      max_length: 500

Common pitfalls

Argument validation runs against the schemas block, not your tool's OpenAPI spec. Keep both in sync or you'll drift.
Unknown tools default to BLOCK under policy.tools.mode = "deny-unknown". This is the safer default for production.