Build on a governed AI gateway

Introduction#

Sophy is an OpenAI-compatible gateway and operator console. Applications send input in familiar wire formats; Sophy authenticates the key, resolves its centrally managed configuration, calls the bound model through that key’s project-owned Vercel AI Gateway credential, and records project-attributed usage.

Compatibility is intentionally scoped. Chat Completions and Responses cover text, streaming, multimodal input, and function-tool loops. Dedicated routes cover audio transcription, embeddings, image generation, short-lived file uploads, and the primary model bound to a key. Evaluation models are reached through a native Sophy route rather than an OpenAI client method. The sections below call out differences from the full OpenAI platform.

Supported API surfaces#

Authentication#

Send the Sophy key as a Bearer token. Keys look like mw_live_… and are issued inside a project in the Sophy console. The console and CLI use email codes for operator sign-in. A verified email can create a renameable My Project or accept an Admin invitation.

Authorization: Bearer mw_live_…

Missing Bearer authentication returns 401 missing_api_key. A malformed, unknown, revoked, expired, or inactive-project key returns 401 invalid_api_key. Keys are checked against the database on every request, so edits and revocation apply immediately.

Quickstart#

Change the base URL and API key, then call one of the supported methods. The placeholder model is required by some OpenAI SDK methods but Sophy uses the model bound to the key.

from openai import OpenAI

client = OpenAI(
    base_url="https://sophy.in/v1",
    api_key="mw_live_…",
)

resp = client.chat.completions.create(
    model="sophy",  # ignored — the key owns the model
    messages=[{"role": "user", "content": "Hello, Sophy!"}],
)
print(resp.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://sophy.in/v1",
  apiKey: "mw_live_…",
});

const resp = await client.chat.completions.create({
  model: "sophy",  // ignored — the key owns the model
  messages: [{ role: "user", content: "Hello, Sophy!" }],
});
console.log(resp.choices[0].message.content);

curl https://sophy.in/v1/chat/completions \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sophy",
    "messages": [{"role": "user", "content": "Hello, Sophy!"}]
  }'

Chat Completions#

Key-owned or ignored: model, temperature, top_p, max_tokens, max_completion_tokens, response_format, n, and user. Sophy applies the key’s configured model and generation parameters instead.

Buffered response

{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "created": 1735689600,
  "model": "anthropic/claude-sonnet-4.6",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "Hello! How can I help?" },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 7,
    "total_tokens": 16,
    "prompt_tokens_details": { "cached_tokens": 0 }
  }
}

Streaming

A stream begins with an assistant-role chunk, emits text and/or tool-call deltas, finishes with a reason chunk, optionally emits usage, and ends with data: [DONE].

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1735689600,"model":"anthropic/claude-sonnet-4.6","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1735689600,"model":"anthropic/claude-sonnet-4.6","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1735689600,"model":"anthropic/claude-sonnet-4.6","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Responses API#

Use client.responses.create(...) for a Responses-compatible text, multimodal, streaming, or function-tool workflow. Sophy is stateless: send the full conversation and any prior function calls/results in input on every request.

Key-owned or ignored: model, temperature, top_p, max_output_tokens, and text.format.

from openai import OpenAI

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

resp = client.responses.create(
    model="sophy",
    input="Write a haiku about gateways.",
)
print(resp.output_text)

curl https://sophy.in/v1/responses \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{"model":"sophy","input":"Write a haiku about gateways."}'

{
  "id": "resp_…",
  "object": "response",
  "created_at": 1735689600,
  "status": "completed",
  "model": "anthropic/claude-sonnet-4.6",
  "store": false,
  "output": [{
    "id": "msg_…",
    "type": "message",
    "status": "completed",
    "role": "assistant",
    "content": [{ "type": "output_text", "text": "Silent routes converge…", "annotations": [] }]
  }],
  "usage": {
    "input_tokens": 11,
    "input_tokens_details": { "cached_tokens": 0 },
    "output_tokens": 9,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 20
  }
}

Streaming uses events such as response.created, response.output_text.delta, response.function_call_arguments.delta, and response.completed, each with an increasing sequence_number. There is no [DONE] sentinel.

Multimodal input#

Language keys can send public image/file URLs or URLs returned by POST /v1/files, provided the bound model supports the content type. Sophy ownership-checks its own uploads; public external URLs pass through to the model provider.

uploaded_url = "https://…/uploads/<project>/<key>/invoice-….pdf"
uploaded_filename = "invoice.pdf"

resp = client.chat.completions.create(
    model="sophy",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Summarize this document."},
            {
                "type": "file",
                "file": {
                    "file_url": uploaded_url,
                    "filename": uploaded_filename,
                },
            },
        ],
    }],
)

uploaded_url = "https://…/uploads/<project>/<key>/photo-….png"

resp = client.responses.create(
    model="sophy",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "What is shown here?"},
            {"type": "input_image", "image_url": uploaded_url},
        ],
    }],
)

Tool / function calling#

Sophy passes function definitions to the bound language model and maps its requested calls back to the selected OpenAI wire format. Sophy never executes a tool. Your application runs it, then sends the result and full conversation state on the next call.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

first = client.chat.completions.create(
    model="sophy",
    tools=tools,
    messages=[{"role": "user", "content": "Weather in Paris?"}],
)
call = first.choices[0].message.tool_calls[0]

second = client.chat.completions.create(model="sophy", tools=tools, messages=[
    {"role": "user", "content": "Weather in Paris?"},
    first.choices[0].message,
    {"role": "tool", "tool_call_id": call.id, "content": "18°C, sunny"},
])

tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get weather for a city",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

first = client.responses.create(
    model="sophy",
    tools=tools,
    input="Weather in Paris?",
)
call = next(item for item in first.output if item.type == "function_call")

second = client.responses.create(model="sophy", tools=tools, input=[
    {"role": "user", "content": "Weather in Paris?"},
    {
        "type": "function_call",
        "call_id": call.call_id,
        "name": call.name,
        "arguments": call.arguments,
    },
    {
        "type": "function_call_output",
        "call_id": call.call_id,
        "output": "18°C, sunny",
    },
])

Only function tools are supported; other Responses tool types are ignored. Chat uses the nested { type: "function", function: { ... } } shape; Responses uses the flat { type: "function", name, parameters } shape. Legacy Chat functions returns 400 functions_unsupported.

Structured output#

An operator can attach a JSON Schema to a language key. Sophy normalizes the schema across supported model providers. Buffered calls use constrained generation, tolerant JSON extraction and retry, then validate the completed object; streaming uses schema-constrained generation with post-hoc validation.

The schema lives on the key. Client-sent Chat response_format and Responses text.format do not replace it. Successful JSON is returned as the assistant text, so parse choices[0].message.content or response.output_text.

Audio transcription#

Bind the key’s primary model to a transcription model and send audio as multipart/form-data. Sophy returns raw text when no processing policy is set, or applies centrally managed instructions in a second language-model step and returns only that processed text.

Accepted audio formats are MP3/MPEG/MPGA, MP4/M4A, WAV, WebM, FLAC, and OGG. Sophy identifies the format from the file bytes instead of trusting its name or client-declared MIME type. Files outside that allowlist return 400 unsupported_audio_format; files over the limit return 413 file_too_large.

from openai import OpenAI

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

with open("meeting.m4a", "rb") as audio:
    resp = client.audio.transcriptions.create(
        model="sophy",
        file=audio,
        language="en",
        response_format="json",
    )

print(resp.text)

curl https://sophy.in/v1/audio/transcriptions \
  -H "Authorization: Bearer mw_live_…" \
  -F "file=@meeting.m4a" \
  -F "model=sophy" \
  -F "language=en" \
  -F "response_format=json"

{
  "text": "Decisions: launch on Friday and send the checklist today.",
  "processed": true,
  "language": "en",
  "duration": 8.4
}

One uploaded clip consumes one RPM slot and counts as one client request. Sophy attributes transcription and processing to their actual models, while the processor component does not add a second client request. If content logging is enabled on the key, Sophy stores the filename, byte count, language hint, raw transcript, and final text—but never the audio bytes. Turn content logging off when unprocessed text must not be retained.

Embeddings#

Bind the key to an embedding model and use the standard OpenAI embeddings method. The key-owned model is returned in the response, and usage records input tokens and estimated cost.

Pre-tokenized integer arrays are not portable across a gateway-bound model and return 400 token_input_unsupported.

from openai import OpenAI

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

resp = client.embeddings.create(
    model="sophy",
    input=["First document", "Second document"],
    encoding_format="float",
)
print(resp.data[0].embedding)

curl https://sophy.in/v1/embeddings \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sophy",
    "input": ["First document", "Second document"],
    "encoding_format": "float"
  }'

{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.012, -0.044, 0.008] },
    { "object": "embedding", "index": 1, "embedding": [-0.031, 0.017, 0.052] }
  ],
  "model": "openai/text-embedding-3-small",
  "usage": { "prompt_tokens": 5, "total_tokens": 5 }
}

Evaluation models#

Evaluation models are not exposed through the OpenAI-compatible client methods, so call this native Sophy route directly with an HTTP client. Bind the key to an evaluation model, send the state to judge together with a named set of typed questions, and Sophy returns one answer per question plus token usage.

Questions are judged against the same shared state, so keep each one self-contained. The call is a single buffered round trip; there is no streaming.

Question types and answers

Requests are validated before any paid call. Missing or oversized state returns 400 missing_state or 400 state_too_large; a bad question set returns 400 missing_questions, 400 invalid_questions, 400 questions_empty, 400 too_many_questions, 400 invalid_question, 400 missing_instructions, 400 unsupported_question_type, or 400 invalid_criteria. Out-of-namespace knobs return 400 unsupported_provider_options, and a key whose model is known to be something other than an evaluation model returns 400 model_not_evaluation.

import httpx

resp = httpx.post(
    "https://sophy.in/v1/evaluate",
    headers={"Authorization": "Bearer mw_live_…"},
    json={
        "state": {
            "question": "How do I reset my password?",
            "answer": "Open Settings and use the reset link we email you.",
        },
        "questions": {
            "grounded": {
                "type": "boolean",
                "instructions": "Is the answer supported by the question?",
            },
            "tone": {
                "type": "choice",
                "instructions": "Classify the tone of the answer.",
                "criteria": {
                    "formal": "Professional and impersonal.",
                    "casual": "Friendly and conversational.",
                },
            },
            "helpfulness": {
                "type": "score",
                "instructions": "Rate how helpful the answer is.",
                "criteria": ["useless", "partial", "complete"],
            },
        },
    },
    timeout=120,
)
print(resp.json()["answers"])

curl https://sophy.in/v1/evaluate \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Q: How do I reset my password? A: Open Settings and use the reset link.",
    "questions": {
      "grounded": {
        "type": "boolean",
        "instructions": "Is the answer supported by the question?"
      },
      "tone": {
        "type": "choice",
        "instructions": "Classify the tone of the answer.",
        "criteria": {
          "formal": "Professional and impersonal.",
          "casual": "Friendly and conversational."
        }
      },
      "helpfulness": {
        "type": "score",
        "instructions": "Rate how helpful the answer is.",
        "criteria": ["useless", "partial", "complete"]
      }
    }
  }'

{
  "model": "typesafe-ai/jev",
  "answers": {
    "grounded": { "type": "boolean", "probability": 0.93 },
    "tone": {
      "type": "choice",
      "choice": "casual",
      "probabilities": { "formal": 0.18, "casual": 0.82 }
    },
    "helpfulness": {
      "type": "score",
      "score": 2,
      "probabilities": { "useless": 0.04, "partial": 0.21, "complete": 0.75 }
    }
  },
  "usage": { "inputTokens": 412, "outputTokens": 36, "totalTokens": 448 }
}

Images (TypeSafe Jev)

Keys bound to TypeSafe Jev 1.13.0 or later can send still images in assets. Give each image a key, then use that key as a value in state and refer to it by its state name in your questions. Images must be PNG or JPEG, base64-encoded with no data: prefix, 280–1,568 pixels on each side, and already rotated upright (EXIF orientation is not applied). The whole request must stay under 4.5 MB. TypeSafe still labels image input experimental, so validate results before you depend on them.

curl https://sophy.in/v1/evaluate \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": { "selfie": "selfie-1" },
    "assets": {
      "selfie-1": { "type": "image", "format": "jpg", "data_base64": "/9j/4AAQ…" }
    },
    "questions": {
      "groomed": {
        "type": "score",
        "instructions": "Rate the grooming of the person in `selfie`.",
        "criteria": ["poor", "acceptable", "good"]
      }
    }
  }'

Bad image input is rejected before any paid call with 400 assets_not_supported, 400 invalid_assets, 400 too_many_assets, 400 invalid_asset_key, 400 invalid_asset, 400 unsupported_asset_format, 400 invalid_asset_data, 400 asset_format_mismatch, or 400 unreferenced_asset. An image the model rejects, for example one outside the size range, returns 400 upstream_invalid_request with TypeSafe’s message.

One evaluate call consumes one RPM slot and is checked against the key’s monthly cost cap before the paid model call. When content logging is enabled on the key, Sophy stores the evaluated state, the questions, and the returned answers. For images it logs only each one’s format and size, never the image itself.

Image generation#

Bind the key to a supported image model. Generated images are returned inline as base64 and are not stored by Sophy.

from openai import OpenAI
import base64

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

resp = client.images.generate(
    model="sophy",
    prompt="A red panda coding at a desk, watercolor",
    n=1,
    size="1024x1024",
)
with open("out.png", "wb") as f:
    f.write(base64.b64decode(resp.data[0].b64_json))

curl https://sophy.in/v1/images/generations \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{"prompt":"A red panda coding at a desk, watercolor","size":"1024x1024"}'

{
  "created": 1735689600,
  "data": [{ "b64_json": "iVBORw0KGgoAAAANSUhEUgAA…" }]
}

Files#

Upload a file as multipart/form-data using the field name file. Success returns 201. Use the returned URL in a Chat file/image part or a Responses input_file/input_image part.

Accepted content types are PDF, PNG, JPEG, WebP, GIF, plain text, CSV, and JSON. The maximum size is 4 MiB (4,194,304 bytes). Uploads become eligible for batched cleanup after 24 hours, so they are request staging—not durable file storage or a precise expiry service.

curl https://sophy.in/v1/files \
  -H "Authorization: Bearer mw_live_…" \
  -F "file=@./invoice.pdf"

{
  "id": "https://…/uploads/<project>/<key>/invoice-….pdf",
  "url": "https://…/uploads/<project>/<key>/invoice-….pdf",
  "pathname": "uploads/<project>/<key>/invoice-….pdf",
  "filename": "invoice.pdf",
  "bytes": 48213,
  "contentType": "application/pdf"
}

Upload URLs are public but unguessable so model providers can fetch them; treat each URL as a bearer URL, not a secret store. When a Sophy URL is submitted back through the API, Sophy checks its issuing key, and cross-key reuse returns 403 file_access_denied.

Models#

Returns exactly the primary model currently bound to the authenticated key in OpenAI list shape. This route does not return the full console catalog or an optional transcript processor, and it does not select a model for a later request.

{
  "object": "list",
  "data": [
    {
      "id": "anthropic/claude-sonnet-4.6",
      "object": "model",
      "created": 0,
      "owned_by": "sophy"
    }
  ]
}

Key-owned policy#

The key is Sophy’s unit of configuration. Operators can change its policy without changing or redeploying the client. The next request reads the latest values.

Client values for model and generation parameters are silently ignored. Chat and Responses prompt behavior is controlled separately by Agent mode. Audio transcription rejects a client prompt; only the key-owned system prompt can request post-processing.

When a language or transcript-processing call has an effective system prompt, Sophy prepends a fixed platform security preamble before the key and any applicable Agent-mode client instructions. On Anthropic models, Chat and Responses requests mark that system prompt (with any tools) as an ephemeral prompt-cache breakpoint, so requests on the same key within the cache window re-read it instead of paying full input price; knowledgebase context is sent after the breakpoint so it doesn’t break the shared prefix. Multi-turn requests on keys without a knowledgebase also mark the latest message, caching the conversation so far. Prompts shorter than the model’s cacheable minimum are not cached. Cache reads appear in usage as prompt_tokens_details.cached_tokens (Chat) and input_tokens_details.cached_tokens (Responses).

Agent mode#

Standard keys are governed mode: Sophy drops client system and developer messages plus Responses instructions, then uses the key-owned prompt.

For a trusted server-side agent whose instructions must vary per request, an operator can enable Agent mode on the key. Sophy keeps the key prompt ahead of client instructions (after Sophy’s fixed security preamble), then appends:

• Chat: consecutive leading system/developer messages, up to the first non-system turn.
• Responses: instructions, followed by leading system/developer input items.

Later system-role items in the conversation are still dropped. Model, generation parameters, schema, knowledge, budgets, and rate limits remain key-owned.

Knowledgebases#

Project Admins can manage every project knowledgebase. Editors can create and manage knowledgebases they own. Both roles can build them from PDF, DOCX, Markdown, plain text, CSV, and JSON documents up to 4 MiB each. Sophy extracts, chunks, and embeds sources in the background.

Knowledgebases are shareable inside their project. An Admin can attach one to any project key, while an Editor can attach one only to a key they own. The same collection can ground multiple permitted keys.

On Chat or Responses requests with usable user text, Sophy embeds the latest user text (capped at its first 8,000 characters), retrieves the six closest chunks, and adds them to the effective prompt. Image- or file-only turns have no retrieval query. A transient retrieval timeout can continue without grounding. Missing, invalid, or billing-blocked project Gateway credentials fail closed before the model call; they never fall back to Sophy’s deployment credentials or another project.

Knowledgebase ingestion and query-embedding spend is tracked separately from client proxy traffic in the console.

Operator console#

One passwordless identity can belong to multiple projects with an independent Admin or Editor role in each. The project in the URL is the tenant boundary, and every read, mutation, key, knowledgebase, log, evaluation, and audit event stays inside it.

Operator sign-in with email codes#

The console and CLI use the same email identity. Sophy sends a six-digit one-time password (OTP) to your inbox. Each code expires after 10 minutes and permits at most five attempts. A successful attempt consumes the code.

Sign-in links no longer authenticate an account. Invitation links identify the invitation, but acceptance requires sign-in with the matching email account.

{ "email": "operator@example.com" }

{ "ok": true, "challengeId": "<challenge-id>" }

This response does not reveal whether an account exists. Email delivery and verification are subject to rate limits. When Sophy issues a new code, it invalidates the previous code for that email. The optional next field preserves a local invitation path through sign-in.

Code requests require 60 seconds between deliveries. Each email permits five codes per 15 minutes and 10 per hour. Each IP permits 10 requests per 15 minutes for each sign-in endpoint. Sign-in JSON bodies permit at most 4 KiB.

{ "challengeId": "<challenge-id>", "code": "012345", "client": "cli" }

For web, verification sets the browser session cookie and returns the next local path. For cli, it returns token, expiresAt, and user. A CLI session expires after five days. Each request checks the current account, project membership, and project status.

Send the CLI token in the Authorization: Bearer header to revoke that session. The CLI removes its local session after a successful logout.

Sign-in endpoints return { "error": "code" } on errors. Malformed verification requests return 400 invalid_request. Invalid emails return 400 invalid_email. Invalid, used, or expired codes return 401 invalid_or_expired_code. Request limits return 429 too_many_attempts. An invalid CLI logout token returns 401 unauthorized.

Sophy CLI#

Manage Sophy from a terminal or a coding agent that can run terminal commands. The CLI uses the same project roles and resource ownership as the console. Admins manage project resources. Editors manage the keys and knowledgebases they own.

Install the public @sophyai/sophy-cli npm package. A source checkout is not required.

Use Node.js 22 or newer. Install the CLI from npm:

npm install -g @sophyai/sophy-cli
sophy --help

To run a command without a global installation, use npx:

npx --package @sophyai/sophy-cli sophy --help
npx --package @sophyai/sophy-cli sophy login --email operator@example.com

The following examples use the installed sophy command. With npx, prefix each command with npx --package @sophyai/sophy-cli.

Sign in with your email. Then choose a project from your project list:

sophy login --email operator@example.com
sophy whoami
sophy projects list
sophy projects use <project-id>
sophy keys list

Enter the six-digit email code at the hidden login prompt. The default server is https://sophy.in. The installed command and npx use the same local session. The --project flag selects a project for one command. Run sophy logout to revoke the session.

For development from a source checkout, run npm install -g ./cli from the repository root.

Update key instructions#

The systemPrompt field stores the instructions for a key. An update applies to subsequent requests with that key. Read the current settings before you change them:

sophy keys get <key-id> --project <project-id> --json

For a short instruction, use the --system-prompt flag:

sophy keys update <key-id> --project <project-id> \
  --system-prompt "Answer support questions using the attached knowledgebase."

For longer instructions, save a JSON file named key-instructions.json. Use \n for line breaks inside the JSON string:

{
  "systemPrompt": "You are the support assistant.\nUse the attached knowledgebase.\nIf the answer is unavailable, say so."
}

Apply the file. Then read the key again to confirm the saved instructions:

sophy keys update <key-id> --project <project-id> \
  --data @key-instructions.json --json
sophy keys get <key-id> --project <project-id> --json

Use Sophy with coding agents#

A coding agent can use the CLI through its terminal. Commands run with the account signed in to that environment. A remote agent needs its own login in its environment. Your browser session or another computer's CLI session does not sign in that agent.

Use --json for compact JSON output. Use --data @file.json for nested input or long prompts. Project roles and resource ownership still apply to every command. Admin-only changes remain restricted to project admins.

Copy these instructions into a task for your coding agent:

Use the Sophy CLI for this task.

1. Run sophy --help, then sophy keys update --help to read the supported fields.
2. Run sophy whoami --json. If sign-in is required, ask me to complete
   sophy login in this environment. Do not copy a session from another machine.
3. Run sophy projects list --json. Use the project ID and key ID I identify.
   Include --project <project-id> on every project command.
4. Before editing, run:
   sophy keys get <key-id> --project <project-id> --json
5. Put only the requested changes in changes.json. Use systemPrompt for instructions.
   Preserve omitted settings. If changing params, include all parameters to retain.
6. Apply the changes:
   sophy keys update <key-id> --project <project-id> --data @changes.json --json
7. Read the key again with keys get. Confirm the requested changes and preserved settings.
   Report the saved result. Treat any nonzero exit code as incomplete work.
8. Use --yes only for an action I explicitly authorized. Do not include secrets in reports.

The CLI returns exit code 1 for errors. Exit code 2 means a bulk action skipped items or a key update requires evaluation-stop confirmation. The JSON result identifies the incomplete work.

CLI commands and management API#

Discover commands, command groups, and input fields through the built-in help:

sophy --help
sophy keys --help
sophy keys update --help

Key creation and rotation show the new secret once. Sophy cannot return an existing secret. Rotation and revocation require confirmation.

To read one key's recent traffic, supply its ID. The command returns the key's 100 most recent logs from every log source, newest first. Add --source or a smaller --limit to narrow the list:

sophy logs list --key-id <key-id> --project <project-id> --json
sophy logs get <log-id> --project <project-id> --json

A log list contains metadata. logs get returns one log with any retained content. Admins can read logs for any key in the project, and editors for the keys they own. Any other key returns 404 not_found. The management API takes the key as keyId in the logs.list input.

The management API requires a CLI session token in the Authorization: Bearer header. Browser cookies and application API keys cannot authenticate this route. CLI tokens cannot authenticate the AI routes under /v1.

{ "operation": "keys.list", "projectId": "<project-id>", "input": {} }

{ "ok": true, "data": [] }

{ "error": { "code": "forbidden", "message": "Your current project role or resource ownership does not allow this action." } }

Management responses use Cache-Control: no-store. Access decisions happen on the server. A local project choice or a stale CLI session never grants additional project rights.

JSON requests permit at most 1 MiB. Knowledgebase uploads use multipart fields operation=knowledgebases.upload, projectId, kbId, and file. Files permit at most 4 MiB, with 64 KiB for the multipart envelope. Each account permits 120 management requests per minute.

Invalid fields return 400 invalid_request. Missing project selection returns 400 project_required. Unknown operations return 400 unknown_operation. Invalid sessions return 401 unauthorized, denied access returns 403 forbidden, and unavailable resources return 404 not_found.

Oversized requests return 413 payload_too_large. Unsupported content types return 415 unsupported_media_type. Request limits return 429 rate_limited with Retry-After: 60. Unexpected failures return 500 internal_error without database details.

Errors and retries#

Before a response stream starts, Sophy-generated failures use the OpenAI error envelope below with Cache-Control: no-store. Field validation errors may use code: null and identify the field in param.

{
  "error": {
    "message": "Invalid API key.",
    "type": "authentication_error",
    "param": null,
    "code": "invalid_api_key"
  }
}

Upstream client-input rejections (400/413/422) become 400 upstream_invalid_request with a safe, capped message. Upstream rate limits preserve 429 and forward Retry-After when present. A valid Sophy key whose project credential is missing, disconnected, invalid, billing-blocked, or undecryptable receives 503 project_gateway_unavailable; Sophy makes no fallback upstream call. Transient network and server failures are hidden behind 502 upstream_error.

Rate limits and monthly quota#

Chat, Responses, audio transcription, embeddings, image generation, and model evaluation enforce the key’s optional requests-per-minute limit and monthly cost cap before the paid model call. /files and /models do not consume those request counters.

• RPM uses a fixed 60-second window. Exceeding it returns 429 rate_limit_exceeded with Retry-After in seconds.
• Monthly quota uses the current UTC calendar month and counts client-attributable spend, including transcript processing. Reaching it returns 402 quota_exceeded.
• Champion-versus-challenger evaluation and knowledgebase spend is recorded under separate sources for visibility and does not consume a key’s proxy-traffic cap.

Console cost values are gateway estimates for operational decisions, not a billing-grade ledger.