Agentic Platform
Documentation API reference Tools Skills

Tools

HTML Grep

html_grep MCP Content

Search expressions in a page HTML and return surrounding context.

Fetch the HTML of a given URL and search for one or more expressions in it. For each occurrence found, the agent returns the matching text along with a configurable number of characters of context on each side. Each expression can be a literal substring (default) or a Python regular expression, and is configured independently (case sensitivity, max matches, context size).

Playground

POST
item
url required

The URL to fetch the HTML from. Temporary uploaded URLs on https://app.botify.com/:organization/:project/o/storage/tmp are allowed.

config
save_to_tmp_storage

Save the fetched HTML to Botify temporary storage and return the temporary URL instead of the HTML. Use this to save output tokens. The returned URL can be passed to `tmp_storage_read`, `tmp_storage_grep`, `tmp_storage_jq`, and other HTML-processing tools.

queries required

List of search queries to run against the fetched HTML. Each query is processed independently and shared by all items in a batch.

Credits

Fixed cost per URL requested. The search itself runs inside Botify. What you pay beyond the call is whatever fetching the page costs: nothing if it is in your SiteCrawler crawl, and a live page fetch otherwise, since this tool always allows the fallback.

These usage SKUs can be charged on a call, including SKUs from tools this one may call.

SKU Credits Description Used by
Tool call 1 per request Charged once per successful item, on top of any usage below.
  • This tool
Live page fetch 5 per 10 page Fetches a page from the live web as an ordinary visitor would.
Live page fetch, hard-blocked page 75 per 10 page A page that refused every gentler attempt. Charged instead of them: the attempts it refused cost nothing.
Live page fetch, protected page 25 per 10 page A page that refused an ordinary fetch and had to be retried. Charged instead of the ordinary fetch, not on top of it.

Schemas

item is what you send, config how the tool behaves, and the response is what you get back.

Item

object
Properties
url string required

The URL to fetch the HTML from. Temporary uploaded URLs on https://app.botify.com/:organization/:project/o/storage/tmp are allowed.

Example
{
  "url": "string"
}

Configuration

object
Properties
save_to_tmp_storage boolean

Save the fetched HTML to Botify temporary storage and return the temporary URL instead of the HTML. Use this to save output tokens. The returned URL can be passed to `tmp_storage_read`, `tmp_storage_grep`, `tmp_storage_jq`, and other HTML-processing tools.

Default: false

queries array required

List of search queries to run against the fetched HTML. Each query is processed independently and shared by all items in a batch.

Items: HtmlGrepQuery

min items 1

Example
{
  "save_to_tmp_storage": false,
  "queries": [
    {
      "expression": "string",
      "n_chars": 100,
      "is_regex": false,
      "case_sensitive": true,
      "max_matches": 10
    }
  ]
}

Type Definitions

1 type

These are the available sub-object types that can be used in the properties above.

HtmlGrepQuery object
#
expression string required

Expression to search in the HTML. Interpreted as a literal substring unless `is_regex` is True.

min length 1

n_chars integer

Number of characters of context to keep on each side of the match.

Default: 100

min 0

is_regex boolean

If True, `expression` is interpreted as a Python regular expression. Otherwise, it is treated as a literal substring.

Default: false

case_sensitive boolean

If True, the search is case-sensitive.

Default: true

max_matches integer

Maximum number of matches returned for this query. Use 0 (or a negative value) to return all matches.

Default: 10

Response

object
Properties
crawl string | null

If fetched from SiteCrawler, the crawl slug used

Default: null

results array

One result per input query, in the same order as `queries`.

Items: HtmlGrepQueryResult

Default: []

reason string | null

Reason for an empty result, if any

Default: null

Example
{
  "crawl": null,
  "results": [],
  "reason": null
}

Type Definitions

3 types

These are the available sub-object types that can be used in the properties above.

HtmlGrepMatch object
#
match string required

The exact substring that matched

context string required

The match with up to `n_chars` characters around it

start integer required

Start offset of the match in the HTML

end integer required

End offset of the match in the HTML

HtmlGrepQuery object
#
expression string required

Expression to search in the HTML. Interpreted as a literal substring unless `is_regex` is True.

min length 1

n_chars integer

Number of characters of context to keep on each side of the match.

Default: 100

min 0

is_regex boolean

If True, `expression` is interpreted as a Python regular expression. Otherwise, it is treated as a literal substring.

Default: false

case_sensitive boolean

If True, the search is case-sensitive.

Default: true

max_matches integer

Maximum number of matches returned for this query. Use 0 (or a negative value) to return all matches.

Default: 10

HtmlGrepQueryResult object
#
query HtmlGrepQuery required

The query this result corresponds to (echoed back)

Type: HtmlGrepQuery

matches array

Matches found for this query, up to `max_matches`

Items: HtmlGrepMatch

Default: []

total_matches integer

Total number of matches found in the HTML for this query (may be greater than `len(matches)` when truncated).

Default: 0

truncated boolean

True if results were truncated by `max_matches`.

Default: false

Endpoints

POST http://agents.botify.com/{organization}/{project}/html_grep/process
POST http://agents.botify.com/{organization}/{project}/html_grep/batch_process
POST http://agents.botify.com/{organization}/{project}/html_grep/async_process
POST http://agents.botify.com/{organization}/{project}/html_grep/async_batch_process

See running a long job in the background for the polling flow.

cURL

curl -X POST "http://agents.botify.com/{organization}/{project}/html_grep/process" \
  -H "Authorization: Bearer $BOTIFY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "item": {
    "url": "<url>"
  },
  "config": {
    "queries": []
  }
}'

Over MCP

This tool is exposed as the MCP tool html_grep. An MCP client passes organization, project and item as tool arguments. Set up MCP →

Raw metadata