Mason Trinca/Getty Images
- Google made a big concession this week.
- The tech giant said it will let websites “specifically opt out of Search generative AI features.”
- Cloudflare CEO Matthew Prince says more is needed to force Google to separate AI and search crawls.
Google said its Search engine could break if the company is forced to implement strict new controls to protect and nurture web content in the AI era.
The warning came after UK antitrust regulators proposed new rules for Google Search that would give publishers more control over how their content is used in AI features such as Google’s AI Overviews and AI Mode.
In response, Google said it is working on new ways to give websites more control over how AI chatbots and AI-powered answer engines access and use online content. The company faces mounting pressure to give content owners ways to opt out of having their data crawled for AI, while still allowing traditional search engines to index this valuable data.
“We’re now exploring updates to our controls to let sites specifically opt out of Search generative AI features,” Google said on Wednesday in a blog post.
That’s a major concession from Google, which has been quietly but firmly pushing back against such demands.
Yet, the tech giant also warned that strict new controls could threaten its prized Search engine, which generates most of the company’s profits.
“Any new controls need to avoid breaking Search in a way that leads to a fragmented or confusing experience for people,” the company said, arguing that search and AI are now deeply intertwined.
Google says AI has been core to how Search works for more than a decade, helping rank results and surface relevant links. Creating sharp opt-outs for generative AI features, Google suggests, could undermine the basic mechanics that allow people to find information quickly and allow websites to be discovered at scale.
At stake is a deeper question about what Search should be in the AI era. Publishers increasingly argue that AI summaries substitute for their content rather than pointing users to it, undermining the grand bargain that has underpinned the web for decades. Google counters that drawing hard lines between search and AI risks unintended consequences, including degraded results and a worse user experience.
Cloudflare CEO Matthew Prince said the UK’s proposal is “progress,” but didn’t go far enough. His company helps run about 20% of the web and has been pushing for new standards to level the AI playing field.
“The CMA’s recommendation today doesn’t go far enough because it doesn’t force Google to split search crawl from AI crawl,” Prince told Business Insider, referring to the UK’s Competition and Markets Authority. “Instead, it requires us all to trust that Google will not be evil when they build their unauditable black AI box.”
“If the CMA wants to encourage innovation and competition in AI, the best thing they should do is force Google to play by the same rules as everyone else and split crawl for AI from the crawl for search,” he added. “Every company other than Google would support that because it fosters a healthy market. It’s a no-brainer, so it’s disappointing the CMA didn’t go far enough today.”
A CMA consultation runs until February 25. Whether regulators can tighten the rules without, as Google warns, breaking Search may help determine not just the future of Google in the UK, but the shape of the open web itself.
Sign up for BI’s Tech Memo newsletter here. Reach out to me via email at abarr@businessinsider.com.
Â
