Google Scraping
Get Google Search Results
GET /v1/google-search/{id}
Google searches run in the background. Submit one with POST /v1/google-search, then poll this endpoint with the id you got back.
#Step 1: Submit
curl -X POST https://scrapingapi.qoest.com/v1/google-search \
-H "Authorization: Bearer your-token" \
-H "Content-Type: application/json" \
-d '{"query": "best scraping api", "country": "us"}'
{"id": "019a0000-0000-7000-8000-000000000000"}
#Step 2: Poll
curl https://scrapingapi.qoest.com/v1/google-search/019a0000-0000-7000-8000-000000000000 \
-H "Authorization: Bearer your-token"
While it runs:
{"id": "019a0000-0000-7000-8000-000000000000", "status": "processing", "response": null}
When it is done:
{
"id": "019a0000-0000-7000-8000-000000000000",
"status": "completed",
"response": [
{
"title": "Example result",
"description": "Snippet text",
"meta_data": "3 days ago",
"url": "https://example.com/article"
}
]
}
#Status Values
status |
Meaning |
|---|---|
queued |
Waiting to start |
processing |
Running, including automatic retries |
completed |
response holds the results |
failed |
No results after retries; the 2 credits are refunded |
While the search is not finished, the response carries a Retry-After: 2 header. Polling is free and works even at 0 credits.
#Example Loop
ID=$(curl -s -X POST https://scrapingapi.qoest.com/v1/google-search \
-H "Authorization: Bearer $QOEST_TOKEN" -H "Content-Type: application/json" \
-d '{"query": "best scraping api"}' | jq -r .id)
until STATUS=$(curl -s https://scrapingapi.qoest.com/v1/google-search/$ID \
-H "Authorization: Bearer $QOEST_TOKEN" | tee result.json | jq -r .status); \
[ "$STATUS" = completed ] || [ "$STATUS" = failed ]; do sleep 2; done
jq .response result.json
#Errors
| Status | Body |
|---|---|
404 |
{"message": "Not found"}: unknown id, or an id from another account |