Web Scraping

Web Scraping Request Format

POST /v1/web

#Request Example

curl -X POST /v1/web \
  -H "Authorization: Bearer your-token" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/api",
    "method": "POST",
    "request_headers": {
      "Content-Type": "application/json",
      "Accept": "application/json"
    },
    "request_body": {
      "hello": "world"
    }
  }'

#Request Body

{
  "url": "https://example.com/api",
  "method": "POST",
  "request_headers": {
    "Content-Type": "application/json",
    "Accept": "application/json"
  },
  "request_body": {
    "hello": "world"
  }
}

#Supported Fields

  • url required
  • method optional
  • request_method optional legacy alias for method
  • request_headers optional object
  • request_body optional object
  • proxy_country optional geo-targeting control
  • bypass_captcha optional string flag

#Validation Rules

  • URL must start with http:// or https://
  • method must be one of GET, POST, PUT, PATCH, DELETE
  • POST requests require request_body
  • headers and body must be JSON objects

#Response

The endpoint returns the raw response body inside response.