Ask a chatbot where to shop, sometimes get a fake store
Perplexity pointed shoppers to a risky website in 15% of its answers. ChatGPT did it 8% of the time, Gemini 4%. That's from a study by Podqi, shared exclusively with The Verge on Thursday: https://www.theverge.com/ai-artificial-intelligence/1032020/gemini-chatgpt-perplexity-shopping-test-risky-fake-websites-citations
The setup was simple. Prompts like "top websites for buying [brand]" for 151 brands, run across two test periods in September, 6,795 answers in total. "Risky" covered three things: impersonator and knockoff stores, old brand domains that someone else had re-registered, and dead domains anyone could buy. Smaller brands got hit about twice as often as big ones. ChatGPT was the best at flagging a risky site when it showed one, and did so 24% of the time. Google and OpenAI both gave statements to The Verge.
Two caveats, and they matter. Podqi sells brand and IP protection, and one of the brands in the story, Madhappy, is its client. So the company measuring the problem also sells the fix. And the prompts were constructed for the test, not pulled from real user logs, so the percentages describe these queries, not what the average shopper sees.
Still, the attack surface is real and it gets worse with agents. A person reading a chatbot answer might notice a strange domain. An agent holding your card and told to "buy it from the official store" will just follow the link. A dead domain that a model remembers as a brand's shop is cheap to buy and hard to spot. As ChatGPT and Gemini move into checkout, citation hygiene stops being a quality problem and becomes a fraud problem.
← Back to all articles
The setup was simple. Prompts like "top websites for buying [brand]" for 151 brands, run across two test periods in September, 6,795 answers in total. "Risky" covered three things: impersonator and knockoff stores, old brand domains that someone else had re-registered, and dead domains anyone could buy. Smaller brands got hit about twice as often as big ones. ChatGPT was the best at flagging a risky site when it showed one, and did so 24% of the time. Google and OpenAI both gave statements to The Verge.
Two caveats, and they matter. Podqi sells brand and IP protection, and one of the brands in the story, Madhappy, is its client. So the company measuring the problem also sells the fix. And the prompts were constructed for the test, not pulled from real user logs, so the percentages describe these queries, not what the average shopper sees.
Still, the attack surface is real and it gets worse with agents. A person reading a chatbot answer might notice a strange domain. An agent holding your card and told to "buy it from the official store" will just follow the link. A dead domain that a model remembers as a brand's shop is cheap to buy and hard to spot. As ChatGPT and Gemini move into checkout, citation hygiene stops being a quality problem and becomes a fraud problem.
Comments