Google quietly broke the search-scraping stack

Google Search now rewrites organic result links to google.com/goto?url=... instead of putting the destination URL in the HTML. Click a result and Google redirects you to the real page. The url parameter carries a custom, Google-specific encoding rather than a plain base64 of the target, so it reads as an opaque reference to Google’s index record for that page. You cannot decode it offline.

As of late August 2026, this shows up consistently for logged-out and private-mode sessions, according to Autom, a SERP-scraping vendor tracking the rollout. Result URLs under those conditions are effectively all goto. It may still be an experiment, but it has moved well past a small slice of results.

Google has wrapped result links before. The older format is google.com/url?q=[URL-encoded destination], where the target sits readable in the query string, and a scraper parsed it straight out of the HTML. The goto format closes that off: the href points at /goto, the url= blob is undecodable without Google, and the real destination arrives only in the Location header on the /goto response.

The destination is in that Location header. Request the goto URL, read the header, and stop there. Do not follow the redirect to the target page, which doubles your traffic and hits the destination site directly. Google still needs the real URL to render the SERP itself (domain label, favicon, attribution), so copies of it remain elsewhere on the page. That is a separate extraction path from reading Location.

The mechanism is cheap for Google and expensive for anyone building an index from SERP data at scale. With plaintext links, one fetch of the results page yielded thousands of destination URLs with no further contact. With goto, every result you want to resolve is a second request back to Google. A client that resolves hundreds of links in sequence is slower, noisier, and easy to tell apart from a person clicking a single result.

This lines up with Google’s other recent moves against bulk harvesting: dropping the &num=100 parameter that returned a hundred results per query, and tightening its BotGuard/SearchGuard checks. Each change raises the per-URL cost of naive scraping rather than blocking it outright.

If you run a pipeline that reads Google result URLs out of HTML, it is already returning goto links for logged-out traffic, and anything downstream expecting a destination URL is getting an opaque Google redirect instead. The fix is to resolve each goto by reading Location with a HEAD or non-following GET, then emit the final URL in the fields you already populate. Budget for the extra request per result and the rate signal it creates, and watch for the encoding to shift again. An opaque, Google-controlled blob is one they can rotate whenever they choose.