This month Reddit, long the single most-cited source across AI answers, nearly vanished from ChatGPT's citations. Its share fell from 3.8% to 0.5% in a matter of days. Most of the coverage framed it as a mystery: the tracking data, they said, showed when it happened but not why.
The why isn't a mystery. It's written directly in Reddit's robots.txt file, and it teaches the clearest strategic lesson we've seen all year.
What actually happened
Two things lined up. First, on August 8, ChatGPT changed how it searches the web behind your questions. It started running roughly twice as many background searches per answer and, crucially, began heavily using the `site:` operator - the command that scopes a search to a specific domain. According to Promptwatch, the share of ChatGPT's background searches using `site:` jumped from under 1% to about 17% in a single day.
Second, and this is where we can drop the "mystery" framing, Reddit blocks the crawlers of search engines that won't pay for its content. Reddit's robots.txt disallows LLM bots that won't commit to Reddit's terms, and Reddit has an exclusive licensing arrangement with Google (reported at roughly $60 million a year). The practical result: Google indexes Reddit, and other engines largely can't. Try it for yourself - run a `site:reddit.com` search on Bing and you get almost nothing.
Let's connect the two. ChatGPT Search retrieves from Bing's index. Reddit is largely absent from Bing's index because Reddit's robots.txt shuts Bing out. So the moment ChatGPT leaned into domain-scoped `site:` searching, its already-thin ability to surface Reddit collapsed. Meanwhile Google, which holds the license and indexes Reddit fully, kept citing it, showing only a gradual decline over the same window (see search Engine Land).
This was never ChatGPT "deciding" Reddit was low quality. It was an access problem created by a licensing deal. The engine can only cite what its index contains, and Reddit's own robots.txt determined which indexes contain it.
Why this is the most important AEO lesson of the year
Think about what this means from the perspective of a business that had built visibility through Reddit. Helpful threads, genuine recommendations, an active community presence. All of it evaporated from the largest AI platform in the world overnight. And not one of those businesses did anything wrong. Not one of them could have prevented it. The cause was a contract negotiated between Reddit and Google, and a robots.txt file they had no say in.
That is the nature of the external environment. You cannot control it.
- You cannot control whether Reddit, or any platform, signs an exclusive deal that cuts off other engines.
- You cannot control when ChatGPT changes how it searches, or which domains it starts to favour.
- You cannot control Google's next core update, or which sources Perplexity decides to weight this quarter.
Offsite presence is genuinely important. Being talked about on review sites, in communities, on LinkedIn, on X, is a real part of how AI engines corroborate and recommend a business. We are not telling anyone to abandon it. But you have to be clear-eyed about one thing: you are renting that visibility, and the landlord can change the locks without telling you.
What you actually own
There is one asset in this entire system that you control completely: your own entity. Your website. The facts you publish about who you are, what you do, and where you do it. Your structured data. The consistency of your name, category, and claims across the properties you operate.
That is the foundation, and it's the foundation precisely because no platform can revoke it. When an AI engine goes looking for who to recommend, the anchor it returns to is your own authoritative record of yourself. Everything offsite is a corroboration layered on top of that anchor. If the anchor is weak, the corroboration has nothing to attach to. If the anchor is strong, you can survive any single external source disappearing, because you were never dependent on it alone.
So the order of operations is this. First, make your own entity unmistakable and verifiable: a clear, plain-language statement of what you are and who you serve, correctly typed structured data that matches your visible content, consistent facts everywhere you control. Then build offsite presence on top of it, and broadly, across many sources, so no single one is load-bearing.
The businesses that got hurt this month were the ones who had outsourced their visibility to a platform they didn't own. The lesson isn't "Reddit is dead." The lesson is older and more durable than any algorithm: control what you can control. Strengthen your own house first, because it's the only one they can't take from you.
---
Clarity Digital Advisory helps companies get recommended, not just mentioned, by the AI platforms buyers now use to shortlist vendors.