Agentic Platform
Documentation API reference Tools Skills

Tools

HTML Structured Data

html_structured_data MCP Url data

Extract structured data (JSON-LD) from HTML pages.

Extract structured data from HTML pages in JSON-LD format.

Playground

POST
item
url required

The URL of the page to extract structured data from

config
country_code

The country code to use for the real-time fetch. 2 letters lowercase

types

The types of structured data to extract. Will extract all if not provided.

fallback_real_time_fetch

Fallback to real_time_fetch_url if the URL is not found in SiteCrawler

Credits

Fixed cost per URL requested. Structured data is parsed inside Botify. Pages in your SiteCrawler crawl add nothing; a live page fetch is billed only if you turn on the real-time fallback, which is off by default.

These usage SKUs can be charged on a call, including SKUs from tools this one may call.

SKU Credits Description Used by
Tool call 1 per request Charged once per successful item, on top of any usage below.
  • This tool
Live page fetch 5 per 10 page Fetches a page from the live web as an ordinary visitor would.
Live page fetch, hard-blocked page 75 per 10 page A page that refused every gentler attempt. Charged instead of them: the attempts it refused cost nothing.
Live page fetch, protected page 25 per 10 page A page that refused an ordinary fetch and had to be retried. Charged instead of the ordinary fetch, not on top of it.

Schemas

item is what you send, config how the tool behaves, and the response is what you get back.

Item

object
Properties
url string required

The URL of the page to extract structured data from

Example
{
  "url": "string"
}

Configuration

object
Properties
country_code string

The country code to use for the real-time fetch. 2 letters lowercase

Default: "us"

types array

The types of structured data to extract. Will extract all if not provided.

fallback_real_time_fetch boolean

Fallback to real_time_fetch_url if the URL is not found in SiteCrawler

Default: false

Example
{
  "country_code": "us",
  "types": [
    "string"
  ],
  "fallback_real_time_fetch": false
}

Response

object
Properties
origin OriginEnum

The origin of the HTML (site_crawler, real_time_fetch_url, or rendering_farm)

Type: OriginEnum

Default: null

status_code integer | null

HTTP status code of the fetched page

Default: null

url_found boolean

Whether the URL was found in the crawl

Default: true

objects array required

List of extracted structured data objects

Items: HTMLStructuredData

has_objects boolean required

Whether any structured data objects were found

has_errors boolean required

Whether any errors were reported in `errors` (currently JSON-LD extraction/parsing; may include other kinds later)

errors array required

List of errors encountered during extraction

Items: HTMLStructuredDataParsingError

title string required

Title indicating the source URL

Example
{
  "origin": null,
  "status_code": null,
  "url_found": true,
  "objects": [
    {
      "type": "string",
      "data": {}
    }
  ],
  "has_objects": true,
  "has_errors": true,
  "errors": [
    {
      "error": "string",
      "raw_json": "string"
    }
  ],
  "title": "string"
}

Type Definitions

3 types

These are the available sub-object types that can be used in the properties above.

HTMLStructuredData object
#

Represents a structured data object found in a page.

type string | array required

JSON-LD @type: a single type name or a list of types (e.g. Book and Product).

data object required

The structured data in JSON-LD format

HTMLStructuredDataParsingError object
#

Represents an error that occurred while parsing structured data.

error string required

The error message

raw_json string required

The raw JSON that caused the error

OriginEnum string
#

Values: site_crawler real_time_fetch_url rendering_farm temporary_upload

Endpoints

POST http://agents.botify.com/{organization}/{project}/html_structured_data/process
POST http://agents.botify.com/{organization}/{project}/html_structured_data/batch_process
POST http://agents.botify.com/{organization}/{project}/html_structured_data/async_process
POST http://agents.botify.com/{organization}/{project}/html_structured_data/async_batch_process

See running a long job in the background for the polling flow.

cURL

curl -X POST "http://agents.botify.com/{organization}/{project}/html_structured_data/process" \
  -H "Authorization: Bearer $BOTIFY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "item": {
    "url": "<url>"
  }
}'

Over MCP

This tool is exposed as the MCP tool html_structured_data. An MCP client passes organization, project and item as tool arguments. Set up MCP →

Raw metadata