# Botify Agent: AI Gateway **ID:** `ai_gateway` Generate structured output from text input Generate structured output from unstructured text input based on configurable item definitions. It can extract specific information types (strings, numbers, booleans) from text according to custom instructions. Supports both OpenAI models (gpt-5.6-luna, gpt-5.6-terra, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5.2, gpt-5.1, gpt-5-mini, gpt-5-nano) and Gemini models (gemini-3.1-pro-preview, gemini-3-flash-preview, gemini-3.1-flash-lite, gemini-3.5-flash). Can also use other agents as tools for complex reasoning tasks. ## Authentication All endpoints require a **Bearer token** in the `Authorization` header. ``` Authorization: Bearer ``` ## Calling this agent This agent supports the following processing modes: | Mode | Type | Description | |------|------|-------------| | **Process** | Synchronous | Single-item processing. Best for real-time requests with immediate response. | | **Batch Process** | Synchronous | Process multiple items in a single request for efficiency. | | **Async Process** | Asynchronous | Single-item processing for long-running tasks that exceed timeout limits. | | **Async Batch Process** | Asynchronous | Large-scale batch jobs with background processing. | # Synchronous Processing Synchronous calls block until the result is ready. Use these for quick operations where you need immediate results. ## Single Item Processing Process a single input and receive the result immediately. ``` POST https://agents.botify.com/{org}/{project}/ai_gateway/process ``` ### Request body ```json { "item": { "user_prompt": "" }, "config": { "structured_output_json_schema": {}, "model_settings": "" } } ``` ### Response Returns a single processed item (HTTP 200). ### cURL example ```bash curl -X POST "https://agents.botify.com/{org}/{project}/ai_gateway/process" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "item": { "user_prompt": "" }, "config": { "structured_output_json_schema": {}, "model_settings": "" } }' ``` ## Batch Processing Process multiple inputs in a single synchronous request. More efficient than making individual calls. Items are processed concurrently on the server. ``` POST https://agents.botify.com/{org}/{project}/ai_gateway/batch_process ``` ### Request body ```json { "items": [ { "user_prompt": "" } ], "config": { "structured_output_json_schema": {}, "model_settings": "" } } ``` ### Response Returns an array of results. Each element is either a successful processed item (`"status": "success"`) or an error (`"status": "error"`). ### cURL example ```bash curl -X POST "https://agents.botify.com/{org}/{project}/ai_gateway/batch_process" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "items": [ { "user_prompt": "" } ], "config": { "structured_output_json_schema": {}, "model_settings": "" } }' ``` ### When to use synchronous calls - Processing completes within timeout limits (typically 30 seconds) - You need immediate results for user-facing features - Small to medium batch sizes (up to ~100 items) # Asynchronous Processing Asynchronous calls return immediately with a batch ID. You then poll for results using the `async_batches/` endpoints. ### Async processing flow 1. **Submit job** -- `POST` to `async_process` or `async_batch_process` 2. **Receive batch ID** -- response contains `batch_id` 3. **Poll status** -- `HEAD` request to check readiness (lightweight, no body) 4. **Get results** -- `GET` request to retrieve processed results ## Async Single Item Submit a single item for background processing. Use the `/single` endpoint to retrieve the result. ``` POST https://agents.botify.com/{org}/{project}/ai_gateway/async_process ``` ### Request body ```json { "item": { "user_prompt": "" }, "config": { "structured_output_json_schema": {}, "model_settings": "" } } ``` ### cURL example ```bash # Step 1: Submit the job curl -X POST "https://agents.botify.com/{org}/{project}/ai_gateway/async_process" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "item": { "user_prompt": "" }, "config": { "structured_output_json_schema": {}, "model_settings": "" } }' # Response: { "batch_id": "abc123" } # Step 2: Check if batch is ready (HEAD request -- lightweight check) curl -I -X HEAD "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/abc123" \ -H "Authorization: Bearer $TOKEN" # Returns 200 if ready, 204 if still processing # Step 3: Get the result (for single-item async) curl -X GET "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/abc123/single" \ -H "Authorization: Bearer $TOKEN" # Returns 200 with result, 202 if still processing, 404 if batch not found ``` ## Async Batch Processing Submit multiple items for background processing. The request body uses **JSONL** (newline-delimited JSON) format. ``` POST https://agents.botify.com/{org}/{project}/ai_gateway/async_batch_process ``` ### Request body (JSONL, `Content-Type: text/plain`) The first line contains `config` and `batch_config`. Each subsequent line is an item with a unique `id`. ``` {"config": {"structured_output_json_schema": {}, "model_settings": ""}, "batch_config": {}} {"id": "item_1", "item": {"user_prompt": ""}} ``` ### Response Returns a JSON object with `batch_id`, `status`, and a `Location` header. ### cURL example ```bash # Step 1: Submit the batch job curl -X POST "https://agents.botify.com/{org}/{project}/ai_gateway/async_batch_process" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: text/plain" \ -d '{"config": {"structured_output_json_schema": {}, "model_settings": ""}, "batch_config": {}}\n{"id": "item_1", "item": {"user_prompt": ""}}' # Response: { "batch_id": "xyz789" } # Step 2: Check if batch is ready (HEAD request) curl -I -X HEAD "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/xyz789" \ -H "Authorization: Bearer $TOKEN" # Returns 200 if ready, 204 if still processing # Step 3: Get all results curl -X GET "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/xyz789" \ -H "Authorization: Bearer $TOKEN" ``` ## Checking Batch Status Use a lightweight `HEAD` request to check if your batch is ready without transferring data. ``` HEAD https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/{batch_id} ``` ```bash curl -I -X HEAD "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/{batch_id}" \ -H "Authorization: Bearer $TOKEN" # Response headers indicate status: # HTTP/1.1 200 OK -> Batch is complete, results ready # HTTP/1.1 204 No Content -> Still processing # HTTP/1.1 404 Not Found -> Batch does not exist ``` ## Retrieving Results Once the batch is ready, fetch results using the appropriate endpoint. ```bash # For async_process (single item) -- use /single endpoint curl -X GET "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/{batch_id}/single" \ -H "Authorization: Bearer $TOKEN" # For async_batch_process (multiple items) -- use base endpoint curl -X GET "https://agents.botify.com/{org}/{project}/ai_gateway/async_batches/{batch_id}" \ -H "Authorization: Bearer $TOKEN" # Returns streaming JSONL where each line is a processed item or error ``` ### Async response codes | Code | Endpoint | Meaning | Action | |------|----------|---------|--------| | 200 | `HEAD` / `GET` | Batch complete, results ready | Read results from response body | | 204 | `HEAD` | Still processing | Continue polling | | 202 | `GET /single` | Still processing | Continue polling | | 400 | `GET /single` | All items failed (client error) | Check error in response body | | 404 | `HEAD` / `GET` | Batch does not exist | Verify `batch_id` is correct | > **HEAD vs GET:** Use `HEAD` requests for lightweight status checks (no response body). Use `GET` only when ready to retrieve results to minimize bandwidth. ### When to use asynchronous calls - Processing takes longer than 30 seconds - Large batch sizes (100+ items) - Background processing where immediate results aren't required - Integration with job queues or workflow systems ## Python Example A complete Python example showing both synchronous and asynchronous patterns. ```python import requests import time API_TOKEN = "your_api_token" BASE_URL = "https://agents.botify.com/{organization}/{project}" AGENT = "ai_gateway" headers = { "Authorization": f"Bearer {API_TOKEN}", "Content-Type": "application/json" } single_payload = { "item": { "user_prompt": "" }, "config": { "structured_output_json_schema": {}, "model_settings": "" } } def process_sync(payload: dict) -> dict: """Synchronous single-item processing.""" response = requests.post( f"{BASE_URL}/{AGENT}/process", headers=headers, json=payload ) response.raise_for_status() return response.json() def process_async( payload: dict, poll_interval: int = 5, max_retries: int = 60 ) -> dict: """Asynchronous single-item processing with polling.""" # Submit job response = requests.post( f"{BASE_URL}/{AGENT}/async_process", headers=headers, json=payload ) batch_id = response.json()["batch_id"] # Poll for results using HEAD (lightweight check) for _ in range(max_retries): status = requests.head( f"{BASE_URL}/{AGENT}/async_batches/{batch_id}", headers=headers ) if status.status_code == 200: result = requests.get( f"{BASE_URL}/{AGENT}/async_batches/{batch_id}/single", headers=headers ) return result.json() elif status.status_code == 204: time.sleep(poll_interval) else: raise Exception(f"Unexpected status: {status.status_code}") raise Exception("Max retries exceeded") def process_async_batch( items: list, config: dict | None = None, poll_interval: int = 5 ) -> dict: """Asynchronous batch processing with polling.""" import json as _json # Build JSONL payload: first line is config, subsequent lines are items lines = [_json.dumps({"config": config or {}, "batch_config": {}})] for i, item in enumerate(items): lines.append(_json.dumps({"id": f"item_{i}", "item": item})) body = "\n".join(lines) # Submit batch (JSONL, text/plain) response = requests.post( f"{BASE_URL}/{AGENT}/async_batch_process", headers={**headers, "Content-Type": "text/plain"}, data=body ) batch_id = response.json()["batch_id"] # Poll until ready while True: status = requests.head( f"{BASE_URL}/{AGENT}/async_batches/{batch_id}", headers=headers ) if status.status_code == 200: results = requests.get( f"{BASE_URL}/{AGENT}/async_batches/{batch_id}", headers=headers ) return results.json() time.sleep(poll_interval) # Example usage # result = process_sync(single_payload) # result = process_async(single_payload) # results = process_async_batch([{"input": "item1"}, {"input": "item2"}]) ``` ## Billing Fixed cost per call. Token rates follow the model you pick -- a cheaper model is cheaper per token, a more capable one several times more -- and scale with the prompt you send and the answer written, with no cap on either. Attaching web search adds one AI web search (OpenAI or Gemini) each time the model searches, at the rate of the model you pick. Tools you attach are billed on top, at their own rates. These usage SKUs can be charged on a call. | SKU | Credits | Description | | --- | ------- | ----------- | | Tool call | 1 per request | Charged once per successful item, on top of any usage below. | | AI web search (Gemini) | 14 per query | Gemini's built-in web search. Charged only when web_search is listed in agents, and only when the model actually searches. Billed on top of the model's own usage. | | AI web search (OpenAI) | 10 per query | OpenAI's built-in web search. Charged only when web_search is listed in agents, and only when the model actually searches. Billed on top of the model's own usage. | | GPT-5.6 Luna (flex), input | 200 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.6 Luna (flex), output | 900 per million tokens | Tokens the model writes in its answer. | | GPT-5.6 Luna (flex), prompt caching | 250 per million tokens | Tokens written into the prompt cache so later calls can reread them cheaper. | | GPT-5.6 Luna (flex), read from cache | 20 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.6 Luna (standard), input | 400 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.6 Luna (standard), output | 1,800 per million tokens | Tokens the model writes in its answer. | | GPT-5.6 Luna (standard), prompt caching | 500 per million tokens | Tokens written into the prompt cache so later calls can reread them cheaper. | | GPT-5.6 Luna (standard), read from cache | 40 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.6 Terra (flex), input | 2,000 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.6 Terra (flex), output | 9,000 per million tokens | Tokens the model writes in its answer. | | GPT-5.6 Terra (flex), prompt caching | 2,500 per million tokens | Tokens written into the prompt cache so later calls can reread them cheaper. | | GPT-5.6 Terra (flex), read from cache | 200 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.6 Terra (standard), input | 4,000 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.6 Terra (standard), output | 18,000 per million tokens | Tokens the model writes in its answer. | | GPT-5.6 Terra (standard), prompt caching | 5,000 per million tokens | Tokens written into the prompt cache so later calls can reread them cheaper. | | GPT-5.6 Terra (standard), read from cache | 400 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | ### Other SKUs | SKU | Credits | Description | | --- | ------- | ----------- | | GPT-5 mini (flex), input | 125 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5 mini (flex), output | 1,000 per million tokens | Tokens the model writes in its answer. | | GPT-5 mini (flex), read from cache | 12.5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5 mini (standard), input | 250 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5 mini (standard), output | 2,000 per million tokens | Tokens the model writes in its answer. | | GPT-5 mini (standard), read from cache | 25 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5 nano (flex), input | 25 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5 nano (flex), output | 200 per million tokens | Tokens the model writes in its answer. | | GPT-5 nano (flex), read from cache | 2.5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5 nano (standard), input | 50 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5 nano (standard), output | 400 per million tokens | Tokens the model writes in its answer. | | GPT-5 nano (standard), read from cache | 5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.2 (flex), input | 875 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.2 (flex), output | 7,000 per million tokens | Tokens the model writes in its answer. | | GPT-5.2 (flex), read from cache | 87.5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.2 (standard), input | 1,750 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.2 (standard), output | 14,000 per million tokens | Tokens the model writes in its answer. | | GPT-5.2 (standard), read from cache | 175 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.4 (flex), input | 2,500 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.4 (flex), output | 11,250 per million tokens | Tokens the model writes in its answer. | | GPT-5.4 (flex), read from cache | 250 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.4 (standard), input | 5,000 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.4 (standard), output | 22,500 per million tokens | Tokens the model writes in its answer. | | GPT-5.4 (standard), read from cache | 500 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.4 mini (flex), input | 375 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.4 mini (flex), output | 2,250 per million tokens | Tokens the model writes in its answer. | | GPT-5.4 mini (flex), read from cache | 37.5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.4 mini (standard), input | 750 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.4 mini (standard), output | 4,500 per million tokens | Tokens the model writes in its answer. | | GPT-5.4 mini (standard), read from cache | 75 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.4 nano (flex), input | 100 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.4 nano (flex), output | 625 per million tokens | Tokens the model writes in its answer. | | GPT-5.4 nano (flex), read from cache | 10 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.4 nano (standard), input | 200 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.4 nano (standard), output | 1,250 per million tokens | Tokens the model writes in its answer. | | GPT-5.4 nano (standard), read from cache | 20 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.5 (flex), input | 5,000 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.5 (flex), output | 22,500 per million tokens | Tokens the model writes in its answer. | | GPT-5.5 (flex), read from cache | 500 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | GPT-5.5 (standard), input | 10,000 per million tokens | Tokens the model reads from the prompt you send. | | GPT-5.5 (standard), output | 45,000 per million tokens | Tokens the model writes in its answer. | | GPT-5.5 (standard), read from cache | 1,000 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3 Flash (batch), input | 250 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3 Flash (batch), output | 1,500 per million tokens | Tokens the model writes in its answer. | | Gemini 3 Flash (batch), read from cache | 50 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3 Flash (flex), input | 250 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3 Flash (flex), output | 1,500 per million tokens | Tokens the model writes in its answer. | | Gemini 3 Flash (flex), read from cache | 50 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3 Flash (standard), input | 500 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3 Flash (standard), output | 3,000 per million tokens | Tokens the model writes in its answer. | | Gemini 3 Flash (standard), read from cache | 50 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.1 Flash Lite (batch), input | 125 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.1 Flash Lite (batch), output | 750 per million tokens | Tokens the model writes in its answer. | | Gemini 3.1 Flash Lite (batch), read from cache | 12.5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.1 Flash Lite (flex), input | 125 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.1 Flash Lite (flex), output | 750 per million tokens | Tokens the model writes in its answer. | | Gemini 3.1 Flash Lite (flex), read from cache | 12.5 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.1 Flash Lite (standard), input | 250 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.1 Flash Lite (standard), output | 1,500 per million tokens | Tokens the model writes in its answer. | | Gemini 3.1 Flash Lite (standard), read from cache | 25 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.1 Pro (flex), input | 2,000 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.1 Pro (flex), output | 9,000 per million tokens | Tokens the model writes in its answer. | | Gemini 3.1 Pro (flex), read from cache | 400 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.1 Pro (standard), input | 4,000 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.1 Pro (standard), output | 18,000 per million tokens | Tokens the model writes in its answer. | | Gemini 3.1 Pro (standard), read from cache | 400 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.5 Flash (batch), input | 750 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.5 Flash (batch), output | 4,500 per million tokens | Tokens the model writes in its answer. | | Gemini 3.5 Flash (batch), read from cache | 75 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.5 Flash (flex), input | 750 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.5 Flash (flex), output | 4,500 per million tokens | Tokens the model writes in its answer. | | Gemini 3.5 Flash (flex), read from cache | 80 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | | Gemini 3.5 Flash (standard), input | 1,500 per million tokens | Tokens the model reads from the prompt you send. | | Gemini 3.5 Flash (standard), output | 9,000 per million tokens | Tokens the model writes in its answer. | | Gemini 3.5 Flash (standard), read from cache | 150 per million tokens | Tokens the model reads from a cached prompt. Cheaper than a fresh read. | ## Schemas ### Item ```json { "type": "object", "properties": { "prefeed": { "default": [], "description": "List of agent pre-calls whose results are injected into the user prompt before the LLM runs.", "items": { "$ref": "#/$defs/PrefeedCall" }, "title": "Prefeed", "type": "array" }, "user_prompt": { "description": "User prompt for the LLM", "title": "User Prompt", "type": "string" } }, "required": [ "user_prompt" ], "$defs": { "PrefeedCall": { "description": "A pre-call to an agent whose result is injected into the user prompt before the LLM runs.", "properties": { "description": { "description": "Description of the prefeed result (used as context label in the prompt)", "title": "Description", "type": "string" }, "agent": { "description": "Agent ID to call", "title": "Agent", "type": "string" }, "item": { "additionalProperties": true, "description": "Item payload for the agent", "title": "Item", "type": "object" }, "config": { "additionalProperties": true, "default": {}, "description": "Optional config payload for the agent", "title": "Config", "type": "object" } }, "required": [ "description", "agent", "item" ], "title": "PrefeedCall", "type": "object" } } } ``` ### Config ```json { "type": "object", "properties": { "prefeed": { "default": [], "description": "List of agent pre-calls whose results are injected into the user prompt before the LLM runs.", "items": { "$ref": "#/$defs/PrefeedCall" }, "title": "Prefeed", "type": "array" }, "system_prompt": { "default": "", "description": "System prompt used to generate the items", "title": "System Prompt", "type": "string" }, "structured_output_json_schema": { "additionalProperties": true, "description": "JSON Schema for the structured output", "title": "Structured Output Json Schema", "type": "object" }, "model_settings": { "description": "The settings for the AI model", "discriminator": { "mapping": { "gemini-3-flash-preview": "#/$defs/Gemini3FlashSettings", "gemini-3.1-flash-lite": "#/$defs/Gemini31FlashSettings", "gemini-3.1-pro-preview": "#/$defs/Gemini31Settings", "gemini-3.5-flash": "#/$defs/Gemini35FlashSettings", "gpt-5-mini": "#/$defs/OpenAISettings", "gpt-5-nano": "#/$defs/OpenAISettings", "gpt-5.1": "#/$defs/OpenAISettings", "gpt-5.2": "#/$defs/OpenAI52Settings", "gpt-5.4": "#/$defs/OpenAI54Settings", "gpt-5.4-mini": "#/$defs/OpenAI54Settings", "gpt-5.4-nano": "#/$defs/OpenAI54Settings", "gpt-5.5": "#/$defs/OpenAI55Settings", "gpt-5.6-luna": "#/$defs/OpenAI56Settings", "gpt-5.6-terra": "#/$defs/OpenAI56Settings" }, "propertyName": "model" }, "oneOf": [ { "$ref": "#/$defs/Gemini3FlashSettings" }, { "$ref": "#/$defs/Gemini31FlashSettings" }, { "$ref": "#/$defs/Gemini31Settings" }, { "$ref": "#/$defs/Gemini35FlashSettings" }, { "$ref": "#/$defs/OpenAISettings" }, { "$ref": "#/$defs/OpenAI52Settings" }, { "$ref": "#/$defs/OpenAI54Settings" }, { "$ref": "#/$defs/OpenAI55Settings" }, { "$ref": "#/$defs/OpenAI56Settings" } ], "title": "Model Settings" }, "agents": { "description": "List of agent IDs to be used as tools.", "items": { "enum": [ "action_board", "agentic_catalog_sources", "botify_config", "get_credit_usage", "google_trends", "html_code_executor", "html_code_executor_batch", "html_fetch", "html_grep", "html_grep_batch", "html_question", "knowledge", "list_annotations", "perplexity", "quality_control", "screenshot", "tables_create", "tables_insert", "tables_list", "tables_load", "tables_query", "tables_schema", "text_hash", "tool_documentation", "topics_extractor", "web_search" ] }, "title": "Agents", "type": "array" }, "is_gzip": { "default": false, "description": "If True, the user_prompt is expected to be base64-encoded and gzip-compressed. The agent will decode base64, then decompress gzip, and use the resulting UTF-8 string as the prompt. Enable this if your prompt is transmitted in this format, regardless of the processing mode (batch or not). This can be used to reduce payload size in any context.", "title": "Is Gzip", "type": "boolean" }, "toon_tool_results": { "default": false, "description": "Serialize agent tool results as TOON instead of JSON", "title": "Toon Tool Results", "type": "boolean" } }, "required": [ "structured_output_json_schema", "model_settings" ], "$defs": { "Gemini31FlashSettings": { "description": "Settings specific to Gemini 3.1 Flash model", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "model": { "const": "gemini-3.1-flash-lite", "description": "The model to use for the Gemini model", "title": "Model", "type": "string" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "flex_only" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for Google Models: 'default', 'flex', or 'flex_only'.", "title": "Service Tier" }, "thinking_level": { "anyOf": [ { "enum": [ "minimal", "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Thinking level: 'minimal', 'low', 'medium', or 'high'. Controls the depth of reasoning.", "title": "Thinking Level" } }, "required": [ "model" ], "title": "Gemini31FlashSettings", "type": "object" }, "Gemini31Settings": { "description": "Settings specific to Gemini 3.1 models", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "model": { "const": "gemini-3.1-pro-preview", "description": "The model to use for the Gemini model", "title": "Model", "type": "string" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "flex_only" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for Google Models: 'default', 'flex', or 'flex_only'.", "title": "Service Tier" }, "thinking_level": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Thinking level: 'low', 'medium', or 'high'. Controls the depth of reasoning.", "title": "Thinking Level" } }, "required": [ "model" ], "title": "Gemini31Settings", "type": "object" }, "Gemini35FlashSettings": { "description": "Settings specific to Gemini 3.5 Flash model", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "model": { "const": "gemini-3.5-flash", "description": "The model to use for the Gemini model", "title": "Model", "type": "string" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "flex_only" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for Google Models: 'default', 'flex', or 'flex_only'.", "title": "Service Tier" }, "thinking_level": { "anyOf": [ { "enum": [ "minimal", "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Thinking level: 'minimal', 'low', 'medium', or 'high'. Controls the depth of reasoning.", "title": "Thinking Level" } }, "required": [ "model" ], "title": "Gemini35FlashSettings", "type": "object" }, "Gemini3FlashSettings": { "description": "Settings specific to Gemini 3 Flash model", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "model": { "const": "gemini-3-flash-preview", "description": "The model to use for the Gemini model", "title": "Model", "type": "string" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "flex_only" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for Google Models: 'default', 'flex', or 'flex_only'.", "title": "Service Tier" }, "thinking_level": { "anyOf": [ { "enum": [ "minimal", "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Thinking level: 'minimal', 'low', 'medium', or 'high'. Controls the depth of reasoning.", "title": "Thinking Level" } }, "required": [ "model" ], "title": "Gemini3FlashSettings", "type": "object" }, "OpenAI52Settings": { "description": "Settings specific to OpenAI 5.2 models", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "verbosity": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Text verbosity level: 'low', 'medium', or 'high'. Controls how concise or verbose the model's text response is.", "title": "Verbosity" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "priority" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for OpenAI API: 'default', 'flex', or 'priority'.", "title": "Service Tier" }, "model": { "const": "gpt-5.2", "description": "The model to use for the OpenAI model", "title": "Model", "type": "string" }, "reasoning_effort": { "anyOf": [ { "enum": [ "low", "medium", "high", "xhigh" ], "type": "string" }, { "type": "null" } ], "default": "high", "description": "Reasoning effort level: 'low', 'medium', 'high', or 'xhigh'. Controls the depth of reasoning for reasoning models.", "title": "Reasoning Effort" } }, "required": [ "model" ], "title": "OpenAI52Settings", "type": "object" }, "OpenAI54Settings": { "description": "Settings specific to OpenAI 5.4 models", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "verbosity": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Text verbosity level: 'low', 'medium', or 'high'. Controls how concise or verbose the model's text response is.", "title": "Verbosity" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "priority" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for OpenAI API: 'default', 'flex', or 'priority'.", "title": "Service Tier" }, "model": { "description": "The model to use for the OpenAI model", "enum": [ "gpt-5.4", "gpt-5.4-mini", "gpt-5.4-nano" ], "title": "Model", "type": "string" }, "reasoning_effort": { "anyOf": [ { "enum": [ "none", "low", "medium", "high", "xhigh" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Reasoning effort level: 'none', 'low', 'medium', 'high', or 'xhigh'. Controls the depth of reasoning for GPT-5.4 models.", "title": "Reasoning Effort" } }, "required": [ "model" ], "title": "OpenAI54Settings", "type": "object" }, "OpenAI55Settings": { "description": "Settings specific to OpenAI 5.5 models", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "verbosity": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Text verbosity level: 'low', 'medium', or 'high'. Controls how concise or verbose the model's text response is.", "title": "Verbosity" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "priority" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for OpenAI API: 'default', 'flex', or 'priority'.", "title": "Service Tier" }, "model": { "const": "gpt-5.5", "description": "The model to use for the OpenAI model", "title": "Model", "type": "string" }, "reasoning_effort": { "anyOf": [ { "enum": [ "none", "low", "medium", "high", "xhigh" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Reasoning effort level: 'none', 'low', 'medium', 'high', or 'xhigh'. Controls the depth of reasoning for GPT-5.5 models.", "title": "Reasoning Effort" } }, "required": [ "model" ], "title": "OpenAI55Settings", "type": "object" }, "OpenAI56Settings": { "description": "Settings specific to OpenAI 5.6 models", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "verbosity": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Text verbosity level: 'low', 'medium', or 'high'. Controls how concise or verbose the model's text response is.", "title": "Verbosity" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "priority" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for OpenAI API: 'default', 'flex', or 'priority'.", "title": "Service Tier" }, "model": { "description": "The model to use for the OpenAI model", "enum": [ "gpt-5.6-luna", "gpt-5.6-terra" ], "title": "Model", "type": "string" }, "reasoning_effort": { "anyOf": [ { "enum": [ "none", "low", "medium", "high", "xhigh" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Reasoning effort level: 'none', 'low', 'medium', 'high', or 'xhigh'. Controls the depth of reasoning for GPT-5.6 models.", "title": "Reasoning Effort" } }, "required": [ "model" ], "title": "OpenAI56Settings", "type": "object" }, "OpenAISettings": { "description": "Settings specific to OpenAI models", "properties": { "temperature": { "default": 0.3, "description": "The temperature for the AI model", "maximum": 2.0, "minimum": 0.0, "title": "Temperature", "type": "number" }, "max_tokens": { "anyOf": [ { "minimum": 0, "type": "integer" }, { "type": "null" } ], "default": null, "description": "The maximum number of tokens for the AI model", "title": "Max Tokens" }, "service_tier_flex_fallback_to_default": { "default": false, "description": "If True and service_tier is 'flex'-like, fallback to 'default' tier on errors (429 rate limit, unsupported flex tier).", "title": "Service Tier Flex Fallback To Default", "type": "boolean" }, "verbosity": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Text verbosity level: 'low', 'medium', or 'high'. Controls how concise or verbose the model's text response is.", "title": "Verbosity" }, "service_tier": { "anyOf": [ { "enum": [ "default", "flex", "priority" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Service tier for OpenAI API: 'default', 'flex', or 'priority'.", "title": "Service Tier" }, "model": { "description": "The model to use for the OpenAI model", "enum": [ "gpt-5.1", "gpt-5-mini", "gpt-5-nano" ], "title": "Model", "type": "string" }, "reasoning_effort": { "anyOf": [ { "enum": [ "low", "medium", "high" ], "type": "string" }, { "type": "null" } ], "default": null, "description": "Reasoning effort level: 'low', 'medium', or 'high'. Controls the depth of reasoning for reasoning models.", "title": "Reasoning Effort" } }, "required": [ "model" ], "title": "OpenAISettings", "type": "object" }, "PrefeedCall": { "description": "A pre-call to an agent whose result is injected into the user prompt before the LLM runs.", "properties": { "description": { "description": "Description of the prefeed result (used as context label in the prompt)", "title": "Description", "type": "string" }, "agent": { "description": "Agent ID to call", "title": "Agent", "type": "string" }, "item": { "additionalProperties": true, "description": "Item payload for the agent", "title": "Item", "type": "object" }, "config": { "additionalProperties": true, "default": {}, "description": "Optional config payload for the agent", "title": "Config", "type": "object" } }, "required": [ "description", "agent", "item" ], "title": "PrefeedCall", "type": "object" } } } ``` ### Response ```json { "description": "Output from AI text generation", "properties": { "structured_output": { "additionalProperties": true, "description": "Structured output based on input configuration", "title": "Structured Output", "type": "object" } }, "required": [ "structured_output" ], "title": "LLMWrapperProcessedItem", "type": "object" } ``` ## Best Practices **Use exponential backoff for polling.** Start with short intervals (1-2 seconds) and increase the delay between polls to reduce API load. **Set reasonable timeouts.** For synchronous calls, configure your HTTP client with appropriate timeout values (30-60 seconds). **Handle rate limits gracefully.** Implement retry logic with backoff when you receive 429 (Too Many Requests) responses. **Batch when possible.** Use batch endpoints to reduce the number of API calls and improve throughput.