# /webparser/parse

`POST /api/webparser/parse`

Price: 1 credit

Parse and clean HTML from web page

## How to use it

Parse and extract clean content from any web page URL. Supports minimal extraction and tag filtering. Does NOT support JavaScript-rendered SPAs (e.g. platform.openai.com, Vercel/Next.js apps) — these return empty content. Works best with static HTML pages, blogs, docs. For JavaScript-heavy pages use /webparser/render instead.

## Parameters

- `access-token` (string, required)

## Request body

- `timeout` (integer) — Max scrapping execution timeout (in seconds) (default: 300; min: 20; max: 1500)
- `url` (string, required) — URL of the page to parse (format: uri; examples: "https://www.example.com", "https://blog.example.com/article"; minLength: 10; maxLength: 2083)
- `include_tags` (array, nullable) — CSS selectors of elements to include (keep only these) (examples: ["article",".content","#main-content"])
- `exclude_tags` (array, nullable) — CSS selectors or wildcard masks of elements to exclude. Examples: '.sidebar', '.advertisement', '*promo*', '*banner*' (examples: [".sidebar",".advertisement","*promo*"])
- `only_main_content` (boolean) — Extract only main content of the page (heuristic algorithm) (default: false)
- `remove_comments` (boolean) — Remove HTML comments (default: true)
- `resolve_srcset` (boolean) — Convert image srcset to src (selects the largest image) (default: true)
- `return_full_html` (boolean) — Return full HTML document (True) or only body content (False) (default: false)
- `min_text_block` (integer) — Minimum text block size for main content detection (in characters) (default: 200; min: 0)
- `remove_base64_images` (boolean) — Remove base64-encoded images (reduces output size) (default: true)
- `strip_all_tags` (boolean) — Remove all HTML tags and return plain text only (default: false)
- `extract_contacts` (boolean) — Extract links, emails, and phone numbers from the page (default: false)
- `same_origin_links` (boolean) — Only extract links from the same domain (used with extract_contacts) (default: false)
- `social_links_only` (boolean) — Only extract social media links (LinkedIn, Twitter/X, Facebook, Instagram, etc.) (default: false)
- `extract_minimal` (boolean) — Use minimal extraction (only links, title, emails, phones if set) (default: false)

## Response

### 200 — Successful Response

- `@type` (string) (default: "WebParserResult")
- `cleaned_html` (string, required) — Cleaned HTML
- `url` (string, required) — URL of the original page
- `meta_description` (string, nullable) — Meta description
- `metadata` (object) — Additional metadata
- `links` (array, nullable) — Extracted URLs from the page
- `emails` (array, nullable) — Extracted email addresses
- `phones` (array, nullable) — Extracted phone numbers

## Errors

### 422 — Validation Error

The request body did not validate

What to do: Check the fields against this schema. A URN with the wrong prefix is the most common cause.

- `detail` (array)
  - `loc` (array, required)
  - `msg` (string, required)
  - `type` (string, required)
  - `input` (any)
  - `ctx` (object)

### 408

The request ran past its time limit

What to do: Raise `timeout` in the request body, up to the maximum this endpoint documents. Lowering `count` or turning off the `with_*` flags also helps, because less work finishes sooner.

### 412

Failed to fetch URL or page not found.

What to do: Retrying will not help: either the entity does not exist, or the input points at a different one.

### 415

The URL serves a document rather than a web page (PDF, Office file, image, CSV, archive).

### 429

Too many requests: a rate limit or a usage window is exhausted

What to do: When the response carries an X-Retry-After header, wait that many seconds and retry: the same number is in the body as `detail.retry_after`, and the limit clears once that window passes. The message in the body names the limit that was hit.

### 500

Something broke on our side

What to do: Retrying will not help. If it keeps happening, send us the X-Request-ID from the response headers.

### 529

Rate limit reached, or the endpoint is overloaded

What to do: Wait at least 30 seconds, then retry.

## Response envelope

Success: Array of objects (may be empty if no results)

Error: Error may coexist with partial results if it occurs mid-execution. Check X-Error header and status code.

Every response carries these headers:

- `X-Error` — Error message text (present only on error)
- `X-Request-ID` — Unique request identifier
- `X-Execution-Time` — Execution time in seconds
- `X-Result-Count` — How many records the body carries. 0 means an empty result, which is a normal answer and not by itself an error. A non-zero count can come back together with X-Error when the failure happened partway through — read this header and X-Error independently.
- `X-Total-Available-Results` — How many records exist for this query, when the endpoint can say. On a `dry_run` request this is the answer and the body is empty. It saturates: the endpoint's documented maximum means 'at least that many', any smaller number is exact.
- `X-Warning` — Present when the request body carried keys this endpoint does not document. They were ignored, so any filter you meant to apply through them did not apply. Check the spelling against this schema and retry.
- `X-Retry-After` — Seconds to wait before retrying. Present only on 429.