Web Scraping

Web Scraping Restrictions

The generic web endpoint is intentionally restricted so platform-specific traffic uses dedicated scrapers.

#Request Example

curl -X POST /v1/web \
  -H "Authorization: Bearer your-token" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com"
  }'

#Blocked Domains

  • Google
  • Upwork
  • Facebook
  • Twitter/X
  • Instagram
  • LinkedIn
  • YouTube
  • TikTok
  • Pinterest
  • Reddit

Requests to these domains return 403 with Domain not supported.

#Captcha Bypass Rules

If bypass_captcha=true, you cannot also send:

  • custom non-GET methods
  • custom headers
  • request body
  • proxy_country

#Geo Targeting

When proxy_country is supplied, the API rewrites the configured proxy username to include the selected country and charges the geo tier.