How Google Programmable Search Engine Works

When you call the Custom Search JSON API, you send a single GET request to Google’s servers, and Google returns a ranked, structured JSON list of results in milliseconds. Under the hood, Google’s web crawler has already visited, indexed, and scored billions of pages — your request simply queries that pre-built index within the scope you define. This page walks you through every stage of that process so you can build confidently on top of it.

The request/response lifecycle

Every search interaction follows the same five-step flow:

Your app sends a GET request

Your application issues an HTTP GET request to the Custom Search JSON API endpoint. The three required parameters — your API key (key), your Search Engine ID (cx), and your query string (q) — travel as URL query parameters. No request body is needed.

GET https://www.googleapis.com/customsearch/v1
  ?key=YOUR_API_KEY
  &cx=YOUR_SEARCH_ENGINE_ID
  &q=climate+change+solutions

Google authenticates your request

Google validates your key parameter against the Google Cloud project that issued it. If the key is missing, invalid, or over quota, the API returns an error immediately before any search takes place. See Error Codes below for the full list.

The Search Engine ID scopes the query

The cx value tells Google which Programmable Search Engine to use. That engine’s configuration — created in the Programmable Search Engine Control Panel — determines whether results come from the full public web or a curated set of specific sites. Google applies that scope before ranking begins.

Google queries its index and ranks results

Google searches its pre-built index for documents that match your query within the configured scope. It applies its ranking algorithm — considering relevance signals such as keyword match, link authority, freshness, and user location — and assembles a sorted list of results. Any additional API parameters you supplied (date filters, language, safe search, etc.) are applied at this stage to further filter the candidate set.

Google returns a JSON response

A successful request receives an HTTP 200 OK response whose body is a JSON object containing search metadata, engine metadata, and an items array of up to 10 result objects. Your app parses this response and displays results however your UX requires.

Crawling and indexing — what you need to know

Google’s web crawler (Googlebot) continuously discovers and revisits pages across the web, building a searchable index. From a developer perspective, there are three practical implications:

You don't control crawling

The Custom Search API queries Google’s existing index. You cannot trigger a crawl or guarantee that a newly published page appears immediately. Freshness depends on Googlebot’s natural crawl schedule.

Public pages only

Only publicly accessible, non-blocked pages appear in results. Pages behind authentication, disallowed by robots.txt, or marked noindex are excluded.

Index coverage varies

Google does not index the entire web equally. High-authority, frequently updated sites tend to be crawled more often. The totalResults field in the response is an estimate and may not reflect every existing page.

What the Search Engine ID (`cx`) scopes

The cx parameter is the unique identifier for a Programmable Search Engine you configure in the Control Panel. It is not just an authentication token — it is a search scope definition.

Configuration option	Effect
Search the entire web	No site restrictions; Google searches all indexed pages.
Search specific sites only	Results are limited to the domains and URL patterns you add (e.g., `docs.example.com`, `.example.com/blog/`).
Mix of included and excluded sites	You can combine allowlisted sites with blocklisted ones using the `siteSearchFilter` API parameter.

The Search Engine ID (cx) format may look like 8ac1ab64606d234f1 (newer format) or 017576662512468239146:omuauf_lfve (legacy format with colon). Both formats are valid. Copy the value exactly from your Control Panel — do not modify it.

You can further narrow or widen scope at query time using the siteSearch parameter, which overrides the engine’s configured sites for a single request.

What the response contains

A successful 200 OK response is a single JSON object. Here is an annotated overview of its top-level fields:

{
  "kind": "customsearch#search",
  "url": { ... },
  "queries": {
    "request":      [{ ... }],
    "nextPage":     [{ ... }],
    "previousPage": [{ ... }]
  },
  "context": { "title": "My Search Engine" },
  "searchInformation": {
    "searchTime": 0.248,
    "formattedSearchTime": "0.25",
    "totalResults": "1430000",
    "formattedTotalResults": "1,430,000"
  },
  "items": [ { ... }, { ... } ]
}

Top-level response fields

Field	Type	Description
`kind`	`string`	Always `"customsearch#search"`. Identifies the response type.
`url`	`object`	OpenSearch URL template metadata for this API.
`queries`	`object`	Contains `request`, `nextPage`, and `previousPage` sub-objects, each an array with one element. Use `nextPage` to build pagination.
`context`	`object`	Metadata about the Programmable Search Engine — its name and any facet labels you configured.
`searchInformation`	`object`	Timing and estimated result count. Note: `totalResults` is an estimate.
`items`	`array`	The ranked list of search results, up to 10 per page.

Fields inside each items[] object

Field	Type	Description
`kind`	`string`	Always `"customsearch#result"`.
`title`	`string`	The plain-text page title.
`htmlTitle`	`string`	The page title with query terms bolded in `<b>` tags.
`link`	`string`	The canonical URL of the result.
`displayLink`	`string`	The display-friendly version of the URL (e.g., `www.example.com`).
`snippet`	`string`	A plain-text excerpt from the page relevant to the query.
`htmlSnippet`	`string`	The same excerpt with query terms bolded.
`cacheId`	`string`	Google’s cache identifier for this page (if available).
`formattedUrl`	`string`	The URL as displayed in search results.
`htmlFormattedUrl`	`string`	The formatted URL with query terms highlighted.
`pagemap`	`object`	Structured data extracted from the page (metadata, OpenGraph, JSON-LD, images, etc.).
`image`	`object`	Image-specific metadata (present only when `searchType=image`).

Pagination

The API returns at most 10 results per request. To retrieve subsequent pages, increment the start parameter. The first result is at index 1; each page advances by num (default 10).

# Page 1 (results 1–10)
GET https://www.googleapis.com/customsearch/v1?key=KEY&cx=CX&q=python+tutorials&start=1

# Page 2 (results 11–20)
GET https://www.googleapis.com/customsearch/v1?key=KEY&cx=CX&q=python+tutorials&start=11

# Page 3 (results 21–30)
GET https://www.googleapis.com/customsearch/v1?key=KEY&cx=CX&q=python+tutorials&start=21

The API can return at most the first 100 results for any query (start max is 91 with num=10). If queries.nextPage is absent from the response, you have reached the last available page.

Error codes

HTTP Status	Meaning	Common cause
`400 Bad Request`	Invalid parameters	Missing `q`, malformed `cx`, unsupported `num` value
`401 Unauthorized`	Invalid API key	Key doesn’t exist, was deleted, or the API isn’t enabled
`403 Forbidden`	Access denied or quota exceeded	Daily free-tier quota (100 queries) exhausted, or billing not enabled
`404 Not Found`	Resource not found	Incorrect endpoint URL
`429 Too Many Requests`	Rate limited	Too many requests per second
`500 Internal Server Error`	Google-side error	Transient; retry with exponential back-off

Always check response.items before iterating — the field is absent (not an empty array) when a query returns zero results. A missing items key is not an error; it simply means no results matched within your engine’s scope.

Rate limits and quotas

Tier	Queries per day
Free	100
Paid (billed per 1,000 queries)	Up to 10,000 per project

You can request a quota increase through the Google Cloud Console. All quotas are enforced per Google Cloud project, not per Search Engine ID.

Get Started

Core Concepts

Search Features

Guides

Help

How Google Programmable Search Engine Works

The request/response lifecycle

Crawling and indexing — what you need to know

You don't control crawling

Public pages only

Index coverage varies

What the Search Engine ID (`cx`) scopes

What the response contains

Error codes

Rate limits and quotas

​The request/response lifecycle

​Crawling and indexing — what you need to know

You don't control crawling

Public pages only

Index coverage varies

​What the Search Engine ID (cx) scopes

​What the response contains

​Pagination

​Error codes

​Rate limits and quotas

The request/response lifecycle

Crawling and indexing — what you need to know

What the Search Engine ID (`cx`) scopes

What the response contains

Pagination

Error codes

Rate limits and quotas