# Botify Agent: SpeedWorkers Page Snapshot

**ID:** `speedworkers_page_snapshot`

Get the HTML SpeedWorkers stored for a URL during a past indexation.


    Get what Botify SpeedWorkers stored during one past indexation of a page:
    the HTML received (and rendered) from the origin, the URL fetched, the HTTP
    status code and headers returned by the origin, the canonical URL, a summary
    of the JavaScript rendering, and the indexation job (cache behavior,
    rendering config, inventory sources and page properties).

    The indexation event is identified by the `event_id` returned by
    `speedworkers_page_history`. Old events may no longer be available. Use
    `speedworkers_page_preview` for the page currently delivered to bots.

    Header values that may hold credentials (cookies, authorization...) and
    credentials in URLs are masked. With `save_to_tmp_storage`, the HTML is saved
    to temporary storage and its URL is returned instead.

    The project must have SpeedWorkers enabled.
    

## Authentication

All endpoints require a **Bearer token** in the `Authorization` header.

```
Authorization: Bearer <your-token>
```

## Calling this agent

This agent supports the following processing modes:

| Mode | Type | Description |
|------|------|-------------|
| **Process** | Synchronous | Single-item processing. Best for real-time requests with immediate response. |
| **Batch Process** | Synchronous | Process multiple items in a single request for efficiency. |
| **Async Process** | Asynchronous | Single-item processing for long-running tasks that exceed timeout limits. |
| **Async Batch Process** | Asynchronous | Large-scale batch jobs with background processing. |

# Synchronous Processing

Synchronous calls block until the result is ready. Use these for quick operations where you need immediate results.

## Single Item Processing

Process a single input and receive the result immediately.

```
POST https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/process
```

### Request body

```json
{
  "item": {
    "event_id": "<event_id>"
  }
}
```

### Response

Returns a single processed item (HTTP 200).

### cURL example

```bash
curl -X POST "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/process" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "item": {
    "event_id": "<event_id>"
  }
}'
```

## Batch Processing

Process multiple inputs in a single synchronous request. More efficient than making individual calls. Items are processed concurrently on the server.

```
POST https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/batch_process
```

### Request body

```json
{
  "items": [
    {
      "event_id": "<event_id>"
    }
  ]
}
```

### Response

Returns an array of results. Each element is either a successful processed item (`"status": "success"`) or an error (`"status": "error"`).

### cURL example

```bash
curl -X POST "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/batch_process" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "items": [
    {
      "event_id": "<event_id>"
    }
  ]
}'
```

### When to use synchronous calls

- Processing completes within timeout limits (typically 30 seconds)
- You need immediate results for user-facing features
- Small to medium batch sizes (up to ~100 items)

# Asynchronous Processing

Asynchronous calls return immediately with a batch ID. You then poll for results using the `async_batches/` endpoints.

### Async processing flow

1. **Submit job** -- `POST` to `async_process` or `async_batch_process`
2. **Receive batch ID** -- response contains `batch_id`
3. **Poll status** -- `HEAD` request to check readiness (lightweight, no body)
4. **Get results** -- `GET` request to retrieve processed results

## Async Single Item

Submit a single item for background processing. Use the `/single` endpoint to retrieve the result.

```
POST https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_process
```

### Request body

```json
{
  "item": {
    "event_id": "<event_id>"
  }
}
```

### cURL example

```bash
# Step 1: Submit the job
curl -X POST "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_process" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "item": {
    "event_id": "<event_id>"
  }
}'

# Response: { "batch_id": "abc123" }

# Step 2: Check if batch is ready (HEAD request -- lightweight check)
curl -I -X HEAD "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/abc123" \
  -H "Authorization: Bearer $TOKEN"
# Returns 200 if ready, 204 if still processing

# Step 3: Get the result (for single-item async)
curl -X GET "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/abc123/single" \
  -H "Authorization: Bearer $TOKEN"
# Returns 200 with result, 202 if still processing, 404 if batch not found
```

## Async Batch Processing

Submit multiple items for background processing. The request body uses **JSONL** (newline-delimited JSON) format.

```
POST https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batch_process
```

### Request body (JSONL, `Content-Type: text/plain`)

The first line contains `config` and `batch_config`. Each subsequent line is an item with a unique `id`.

```
{"config": {}, "batch_config": {}}
{"id": "item_1", "item": {"event_id": "<event_id>"}}
```

### Response

Returns a JSON object with `batch_id`, `status`, and a `Location` header.

### cURL example

```bash
# Step 1: Submit the batch job
curl -X POST "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batch_process" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: text/plain" \
  -d '{"config": {}, "batch_config": {}}\n{"id": "item_1", "item": {"event_id": "<event_id>"}}'

# Response: { "batch_id": "xyz789" }

# Step 2: Check if batch is ready (HEAD request)
curl -I -X HEAD "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/xyz789" \
  -H "Authorization: Bearer $TOKEN"
# Returns 200 if ready, 204 if still processing

# Step 3: Get all results
curl -X GET "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/xyz789" \
  -H "Authorization: Bearer $TOKEN"
```

## Checking Batch Status

Use a lightweight `HEAD` request to check if your batch is ready without transferring data.

```
HEAD https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/{batch_id}
```

```bash
curl -I -X HEAD "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/{batch_id}" \
  -H "Authorization: Bearer $TOKEN"

# Response headers indicate status:
# HTTP/1.1 200 OK        -> Batch is complete, results ready
# HTTP/1.1 204 No Content -> Still processing
# HTTP/1.1 404 Not Found -> Batch does not exist
```

## Retrieving Results

Once the batch is ready, fetch results using the appropriate endpoint.

```bash
# For async_process (single item) -- use /single endpoint
curl -X GET "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/{batch_id}/single" \
  -H "Authorization: Bearer $TOKEN"

# For async_batch_process (multiple items) -- use base endpoint
curl -X GET "https://agents.botify.com/{org}/{project}/speedworkers_page_snapshot/async_batches/{batch_id}" \
  -H "Authorization: Bearer $TOKEN"
# Returns streaming JSONL where each line is a processed item or error
```

### Async response codes

| Code | Endpoint | Meaning | Action |
|------|----------|---------|--------|
| 200 | `HEAD` / `GET` | Batch complete, results ready | Read results from response body |
| 204 | `HEAD` | Still processing | Continue polling |
| 202 | `GET /single` | Still processing | Continue polling |
| 400 | `GET /single` | All items failed (client error) | Check error in response body |
| 404 | `HEAD` / `GET` | Batch does not exist | Verify `batch_id` is correct |

> **HEAD vs GET:** Use `HEAD` requests for lightweight status checks (no response body). Use `GET` only when ready to retrieve results to minimize bandwidth.

### When to use asynchronous calls

- Processing takes longer than 30 seconds
- Large batch sizes (100+ items)
- Background processing where immediate results aren't required
- Integration with job queues or workflow systems

## Python Example

A complete Python example showing both synchronous and asynchronous patterns.

```python
import requests
import time

API_TOKEN = "your_api_token"
BASE_URL = "https://agents.botify.com/{organization}/{project}"
AGENT = "speedworkers_page_snapshot"

headers = {
    "Authorization": f"Bearer {API_TOKEN}",
    "Content-Type": "application/json"
}

single_payload = {
  "item": {
    "event_id": "<event_id>"
  }
}


def process_sync(payload: dict) -> dict:
    """Synchronous single-item processing."""
    response = requests.post(
        f"{BASE_URL}/{AGENT}/process",
        headers=headers,
        json=payload
    )
    response.raise_for_status()
    return response.json()


def process_async(
    payload: dict,
    poll_interval: int = 5,
    max_retries: int = 60
) -> dict:
    """Asynchronous single-item processing with polling."""
    # Submit job
    response = requests.post(
        f"{BASE_URL}/{AGENT}/async_process",
        headers=headers,
        json=payload
    )
    batch_id = response.json()["batch_id"]

    # Poll for results using HEAD (lightweight check)
    for _ in range(max_retries):
        status = requests.head(
            f"{BASE_URL}/{AGENT}/async_batches/{batch_id}",
            headers=headers
        )

        if status.status_code == 200:
            result = requests.get(
                f"{BASE_URL}/{AGENT}/async_batches/{batch_id}/single",
                headers=headers
            )
            return result.json()
        elif status.status_code == 204:
            time.sleep(poll_interval)
        else:
            raise Exception(f"Unexpected status: {status.status_code}")

    raise Exception("Max retries exceeded")


def process_async_batch(
    items: list,
    config: dict | None = None,
    poll_interval: int = 5
) -> dict:
    """Asynchronous batch processing with polling."""
    import json as _json

    # Build JSONL payload: first line is config, subsequent lines are items
    lines = [_json.dumps({"config": config or {}, "batch_config": {}})]
    for i, item in enumerate(items):
        lines.append(_json.dumps({"id": f"item_{i}", "item": item}))
    body = "\n".join(lines)

    # Submit batch (JSONL, text/plain)
    response = requests.post(
        f"{BASE_URL}/{AGENT}/async_batch_process",
        headers={**headers, "Content-Type": "text/plain"},
        data=body
    )
    batch_id = response.json()["batch_id"]

    # Poll until ready
    while True:
        status = requests.head(
            f"{BASE_URL}/{AGENT}/async_batches/{batch_id}",
            headers=headers
        )

        if status.status_code == 200:
            results = requests.get(
                f"{BASE_URL}/{AGENT}/async_batches/{batch_id}",
                headers=headers
            )
            return results.json()

        time.sleep(poll_interval)


# Example usage
# result = process_sync(single_payload)
# result = process_async(single_payload)
# results = process_async_batch([{"input": "item1"}, {"input": "item2"}])
```

## Billing

Fixed cost per indexation event requested. This tool reads the page SpeedWorkers already stored for your project and calls no outside service, so nothing is added to the per-call price.

These usage SKUs can be charged on a call.

| SKU | Credits | Description |
| --- | ------- | ----------- |
| Tool call | 1 per request | Charged once per successful item. |


## Schemas

### Item

```json
{
  "type": "object",
  "properties": {
    "event_id": {
      "description": "The `event_id` of an indexation event, as returned by the `speedworkers_page_history` tool.",
      "title": "Event ID",
      "type": "string"
    }
  },
  "required": [
    "event_id"
  ]
}
```

### Config

```json
{
  "type": "object",
  "properties": {
    "save_to_tmp_storage": {
      "default": false,
      "description": "Save the HTML to Botify temporary storage and return the temporary URL instead of the HTML. Use this to save output tokens. The returned URL can be passed to other HTML-processing tools. Headers and details are still returned.",
      "title": "Save To Temporary Storage",
      "type": "boolean"
    }
  },
  "required": []
}
```

### Response

```json
{
  "$defs": {
    "SpeedWorkersCanonical": {
      "properties": {
        "has_canonical": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Whether the page declares a canonical URL",
          "title": "Has Canonical"
        },
        "canonical_url": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Canonical URL declared by the page",
          "title": "Canonical Url"
        },
        "canonical_equal": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Whether the canonical URL is the page URL",
          "title": "Canonical Equal"
        }
      },
      "title": "SpeedWorkersCanonical",
      "type": "object"
    },
    "SpeedWorkersIndexationJob": {
      "properties": {
        "section_id": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "ID of the matching cache behavior (section)",
          "title": "Section Id"
        },
        "config_id": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "ID of the rendering config used",
          "title": "Config Id"
        },
        "config_type": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Type of the rendering config (device)",
          "title": "Config Type"
        },
        "input_ids": {
          "default": [],
          "description": "IDs of the inventory sources of the page",
          "items": {
            "type": "string"
          },
          "title": "Input Ids",
          "type": "array"
        },
        "page_properties": {
          "default": [],
          "description": "Page properties at the time of the indexation",
          "items": {
            "$ref": "#/$defs/SpeedWorkersPageProperty"
          },
          "title": "Page Properties",
          "type": "array"
        },
        "status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Status code of the page known before the job",
          "title": "Status Code"
        },
        "previous_status_code": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Previous status code of the page",
          "title": "Previous Status Code"
        },
        "batch_date": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Date of the indexation batch",
          "title": "Batch Date"
        }
      },
      "title": "SpeedWorkersIndexationJob",
      "type": "object"
    },
    "SpeedWorkersJsRendering": {
      "properties": {
        "device": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Rendering device",
          "title": "Device"
        },
        "rendered": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Whether the page was rendered with JavaScript",
          "title": "Rendered"
        },
        "status": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Rendering status",
          "title": "Status"
        },
        "redirect_url": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Redirect target found during the rendering",
          "title": "Redirect Url"
        },
        "version_code": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Version of the rendering engine",
          "title": "Version Code"
        },
        "rendering_time_ms": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Rendering time, in ms",
          "title": "Rendering Time Ms"
        },
        "nb_resources": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Number of resources requested by the page",
          "title": "Nb Resources"
        },
        "nb_resources_error": {
          "anyOf": [
            {
              "type": "integer"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Number of resources that failed to load",
          "title": "Nb Resources Error"
        }
      },
      "title": "SpeedWorkersJsRendering",
      "type": "object"
    },
    "SpeedWorkersPageProperty": {
      "properties": {
        "input": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "ID of the inventory source of the property",
          "title": "Input"
        },
        "name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Property name",
          "title": "Name"
        },
        "value": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "default": null,
          "description": "Property value",
          "title": "Value"
        }
      },
      "title": "SpeedWorkersPageProperty",
      "type": "object"
    }
  },
  "properties": {
    "html": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "HTML of the page as stored by SpeedWorkers during this indexation. Omitted when `save_to_tmp_storage` is True; see `html_tmp_storage_url` instead.",
      "title": "Html"
    },
    "html_tmp_storage_url": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Temporary storage URL of the HTML page. Returned instead of `html` when `save_to_tmp_storage` is True. Can be passed to other HTML-processing tools.",
      "title": "Html Tmp Storage Url"
    },
    "url": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "URL fetched by SpeedWorkers from the origin",
      "title": "Url"
    },
    "indexed_at": {
      "description": "When the indexation happened",
      "format": "date-time",
      "title": "Indexed At",
      "type": "string"
    },
    "status_code": {
      "anyOf": [
        {
          "type": "integer"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "HTTP status code returned by the origin",
      "title": "Status Code"
    },
    "headers": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "string"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "HTTP headers returned by the origin; values that may hold credentials (cookies, authorization...) are masked with `**`",
      "title": "Headers"
    },
    "canonical": {
      "anyOf": [
        {
          "$ref": "#/$defs/SpeedWorkersCanonical"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Canonical URL found in the page"
    },
    "js_rendering": {
      "anyOf": [
        {
          "$ref": "#/$defs/SpeedWorkersJsRendering"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Summary of the JavaScript rendering"
    },
    "job": {
      "anyOf": [
        {
          "$ref": "#/$defs/SpeedWorkersIndexationJob"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Indexation job: cache behavior, rendering config, inventory sources and page properties"
    },
    "reason": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Why no HTML was returned, if any",
      "title": "Reason"
    }
  },
  "required": [
    "indexed_at"
  ],
  "title": "SpeedWorkersPageSnapshotProcessedItem",
  "type": "object"
}
```

## Best Practices

**Use exponential backoff for polling.** Start with short intervals (1-2 seconds) and increase the delay between polls to reduce API load.

**Set reasonable timeouts.** For synchronous calls, configure your HTTP client with appropriate timeout values (30-60 seconds).

**Handle rate limits gracefully.** Implement retry logic with backoff when you receive 429 (Too Many Requests) responses.

**Batch when possible.** Use batch endpoints to reduce the number of API calls and improve throughput.