Google Scraping

Get Google Search Results

GET /v1/google-search/{id}

Google searches run in the background. Submit one with POST /v1/google-search, then poll this endpoint with the id you got back.

#Step 1: Submit

curl -X POST https://scrapingapi.qoest.com/v1/google-search \
  -H "Authorization: Bearer your-token" \
  -H "Content-Type: application/json" \
  -d '{"query": "best scraping api", "country": "us"}'
{"id": "019a0000-0000-7000-8000-000000000000"}

#Step 2: Poll

curl https://scrapingapi.qoest.com/v1/google-search/019a0000-0000-7000-8000-000000000000 \
  -H "Authorization: Bearer your-token"

While it runs:

{"id": "019a0000-0000-7000-8000-000000000000", "status": "processing", "response": null}

When it is done:

{
  "id": "019a0000-0000-7000-8000-000000000000",
  "status": "completed",
  "response": [
    {
      "title": "Example result",
      "description": "Snippet text",
      "meta_data": "3 days ago",
      "url": "https://example.com/article"
    }
  ]
}

#Status Values

status Meaning
queued Waiting to start
processing Running, including automatic retries
completed response holds the results
failed No results after retries; the 2 credits are refunded

While the search is not finished, the response carries a Retry-After: 2 header. Polling is free and works even at 0 credits.

#Example Loop

ID=$(curl -s -X POST https://scrapingapi.qoest.com/v1/google-search \
  -H "Authorization: Bearer $QOEST_TOKEN" -H "Content-Type: application/json" \
  -d '{"query": "best scraping api"}' | jq -r .id)

until STATUS=$(curl -s https://scrapingapi.qoest.com/v1/google-search/$ID \
  -H "Authorization: Bearer $QOEST_TOKEN" | tee result.json | jq -r .status); \
  [ "$STATUS" = completed ] || [ "$STATUS" = failed ]; do sleep 2; done

jq .response result.json

#Errors

Status Body
404 {"message": "Not found"}: unknown id, or an id from another account