/webparser/render

POST /api/webparser/render

Price: 10 credits

Render a web page in a real browser and parse clean HTML (supports JavaScript / SPA)

How to use it

Render and extract clean content from JavaScript-heavy pages and SPAs (e.g. platform.openai.com, Vercel/Next.js apps) that /webparser/parse cannot read, because it executes the page in a real browser. Supports the same minimal extraction and tag filtering options. Use /webparser/parse for static HTML pages, blogs and docs, which need no rendering.

Parameters

• access-token string required

Request body

• timeout integer — Max scrapping execution timeout (in seconds) (default: 300; min: 20; max: 1500)
• url string required — URL of the page to parse (format: uri; examples: "https://www.example.com", "https://blog.example.com/article"; minLength: 10; maxLength: 2083)
• include_tags array nullable — CSS selectors of elements to include (keep only these) (examples: ["article",".content","#main-content"])
• exclude_tags array nullable — CSS selectors or wildcard masks of elements to exclude. Examples: '.sidebar', '.advertisement', '*promo*', '*banner*' (examples: [".sidebar",".advertisement","*promo*"])
• only_main_content boolean — Extract only main content of the page (heuristic algorithm) (default: false)
• remove_comments boolean — Remove HTML comments (default: true)
• resolve_srcset boolean — Convert image srcset to src (selects the largest image) (default: true)
• return_full_html boolean — Return full HTML document (True) or only body content (False) (default: false)
• min_text_block integer — Minimum text block size for main content detection (in characters) (default: 200; min: 0)
• remove_base64_images boolean — Remove base64-encoded images (reduces output size) (default: true)
• strip_all_tags boolean — Remove all HTML tags and return plain text only (default: false)
• extract_contacts boolean — Extract links, emails, and phone numbers from the page (default: false)
• same_origin_links boolean — Only extract links from the same domain (used with extract_contacts) (default: false)
• social_links_only boolean — Only extract social media links (LinkedIn, Twitter/X, Facebook, Instagram, etc.) (default: false)
• extract_minimal boolean — Use minimal extraction (only links, title, emails, phones if set) (default: false)

Response

• @type string (default: "WebParserResult")
• cleaned_html string required — Cleaned HTML
• url string required — URL of the original page
• meta_description string nullable — Meta description
• metadata object — Additional metadata
• links array nullable — Extracted URLs from the page
• emails array nullable — Extracted email addresses
• phones array nullable — Extracted phone numbers

Errors

• 422 — The URL is rejected as unsafe to fetch, or the rendered page exceeds the size limit. Check the fields against this schema. A URN with the wrong prefix is the most common cause.
• 408 — The request ran past its time limit Raise `timeout` in the request body, up to the maximum this endpoint documents. Lowering `count` or turning off the `with_*` flags also helps, because less work finishes sooner.
• 412 — Page not found, unreachable, or protected by an anti-bot defense. Retrying will not help: either the entity does not exist, or the input points at a different one.
• 429 — Too many requests: a rate limit or a usage window is exhausted When the response carries an X-Retry-After header, wait that many seconds and retry: the same number is in the body as `detail.retry_after`, and the limit clears once that window passes. The message in the body names the limit that was hit.
• 500 — Something broke on our side Retrying will not help. If it keeps happening, send us the X-Request-ID from the response headers.
• 529 — Rate limit reached, or the endpoint is overloaded Wait at least 30 seconds, then retry.

Response headers

• X-Error — Error message text (present only on error)
• X-Request-ID — Unique request identifier
• X-Execution-Time — Execution time in seconds
• X-Result-Count — How many records the body carries. 0 means an empty result, which is a normal answer and not by itself an error. A non-zero count can come back together with X-Error when the failure happened partway through — read this header and X-Error independently.
• X-Total-Available-Results — How many records exist for this query, when the endpoint can say. On a `dry_run` request this is the answer and the body is empty. It saturates: the endpoint's documented maximum means 'at least that many', any smaller number is exact.
• X-Warning — Present when the request body carried keys this endpoint does not document. They were ignored, so any filter you meant to apply through them did not apply. Check the spelling against this schema and retry.
• X-Retry-After — Seconds to wait before retrying. Present only on 429.