Growth Memo, 02/25: Amazon, eBay, and Walmart dominate in Google Search just as much as in AI chatbots.
My Take
There is a big question of how important backlinks are for AI chatbot visibility. I think there is a trap to think they have a direct impact. The way I understand the data is that they help with Google/Bing visibility, which passively translates to AI chatbot visibility. They might also help with LLM crawler discoverability. So, they’re still important but not as much as the content itself.
The biggest lever seems to be citable content on and off of Google: Industry reports with exclusive research and data, original surveys and case studies, and thought leadership content from recognized experts.
I wouldn’t restrict myself from optimizing for AI chatbot visibility as a small business with little to no visibility on classic search engines.
Ecommerce is an outlier because the journey is so much more transactional than for B2B or media. On one hand, the strong visibility of big ecommerce platforms like Amazon provides a direct path for AI chatbot visibility for merchants. On the other hand, integrating with programs like Perplexity’s Buy With Pro seems worth trying out.
How Are People Searching On AI Chatbots?
Consumers use AI chatbots differently than Google unless they turn on search features.
Semrush, 02/25: 70% of ChatGPT queries represent entirely new types of intent that don’t fit traditional search categories (navigational, informational, commercial, transactional).
Users are asking longer, more complex questions, with non-search-enabled ChatGPT prompts averaging 23 words compared to 4.2 words when search is enabled.
Higher Visibility, 02/25: People use different AI chatbots for different user intents, e.g., Google for initial product research, ChatGPT for product comparison, and Instagram for discovering new products. However, almost 80% stick to traditional search engines for informational searches.
Growth Memo, 02/25: AI chatbots send significantly more traffic to homepages (22% on average) compared to Google (10%) yet still maintain higher engagement metrics. This trend suggests that AI chatbots are effectively preparing users for brand interactions.
My Take
It’s fascinating to see that when people turn on Search in ChatGPT, they use shorter queries and emulate their behavior on Google. I wonder if this behavior sticks over the long term or not. If so, we can assume a stronger carryover from players who dominate classic search engines today to AI chatbots. If not, it might open the field to new players.
I’ve long been dissatisfied with our broad classification of user intents (information, navigational, etc.). We had this wrong for a long time. It’s too coarse. 70% of use cases are likely task-related and don’t fit our model for classic search engines. AI chatbots are more than search engines but solve the same problems, just with different means. That’s also where I see Google lagging behind: Consumers already associate AI chatbots with tasks rather than finding information.
What Challenges Are Associated With AI Chatbots?
AI chatbots make for a compelling marketing channel but put marketers in front of tracking and bias problems.
Tracking
Source
We can track the referral source for almost all AI chatbots, but some traffic can still fall into the direct traffic bucket.
Citations in ChatGPT typically include a “utm_source=chatgpt.com” parameter, but links in search results don’t have the parameter.2
Ahrefs, 02/25: AI traffic is likely underreported because AI chatbots like Copilot get clustered into direct while they’re actually referrals.
Brand Bias
Semrush, 12/24: Consumers and users are skeptical about AI output. 50% say they trust it more when it’s been reviewed by a human.
In the paper “Global is Good, Local is Bad?” Kamruzzaman et al. conducted experiments with fill-in-the-blank questions across four product categories and 15 countries (English only). The researchers studied the effect of:
Brand attribute bias: global vs. local brands.
Socio-economic bias: luxury vs non-luxury brands.
Geo bias: local brands when the domestic country is specified.
Results:
LLMs across multiple models (GPT-4o, Llama-3, Gemma-7B, Mistral-7B) consistently associate global brands with positive and local brands with negative attributes.
LLMs tend to recommend luxury brands to people from high-income countries. In contrast, non-luxury brands are more commonly suggested for people from low-income countries, even when models were given the flexibility to suggest the same brands for both groups.
The underlying reasons are that local brand names are underrepresented in LLM training data, and large companies can afford larger marketing campaigns and, therefore, create more bias.
In the paper “Generative AI Search Engines as Arbiters of Public Knowledge: An Audit of Bias and Authority” by Li et al., researchers tested how ChatGPT, Bing Chat, and Perplexity answer questions about four major topics: climate change, vaccination, alternative energy, and trust in media. They wanted to see if the AI showed bias in its answers and how it tried to appear trustworthy.
The results:
The AI tends to match the emotion of the question. If you ask a negative question, you get a negative answer.
Different topics got different emotional treatment, e.g., vaccination and alternative energy got more positive responses than climate change and media trust.
Bing Chat and Perplexity heavily cite news media and businesses.
Heavy reliance on U.S. sources (65% of sources), even when used in other countries.
Too many commercial/business sources, especially for topics like alternative energy.
Some models mix unreliable sources with good ones.
Answers often include uncertain language and hedging to avoid taking strong positions.
My Take
We’re used to significant tracking gaps from Google and Bing, so unless AI chatbots try to persuade site owners with more data, we’ll have to continue to operate with aggregate data, as I mentioned in Death of the Keyword.
AI chatbot bias is serious. User trust is key to winning, so I assume AI developers are aware and try to solve the problem. Until then, we have to factor bias in with our optimization strategies and do our best to clearly indicate the target audience for our product in our content.
Conclusion: Where It’s All Going
The data we have today shows that AI chatbots are developing into a significant customer acquisition channel with many familiar mechanics.
However, their task-based nature, bias, and demographics suggest we should be cautious when using the same approach as classic search engines.
Don’t forget – Search is just a means to an end. Ultimately, people search to solve problems, i.e., do tasks.
The fact that AI chatbots can skip the search part and do tasks on the spot means they’re superior to classic search engines. For this reason, I expect Google to add more agentic capabilities to AI Overviews or launch a new Gemini-based product in Search.
The underlying technology allows AI chatbots to fork off search engine ranks and develop their own signals. And it evolves rapidly.
The evolution so far went from machine learning in the pre-2022 era to early LLMs and now inference models (think: reasoning).
Better reasoning allows LLMs to recognize user intent even better than classic search engines, making it easier to train models on better sources to mention or cite.
This brings me to the question of whether Google/Bing incumbents will also dominate AI chatbots down the road. Right now, the answer is yes. But for how long?
Generational preferences could be the biggest driver of new platforms. The easiest way for Google to become irrelevant is to lose young people.
Semrush, 02/25: Searchers over 35 years use Google more often than ChatGPT. People between 18 and 24 use ChatGPT 46.7% of the time, compared to Google with 24.7%.
Higher Visibility, 02/25: 82% of Gen Z occasionally use AI chatbots, compared to 42% of Baby Boomers.
There is a chance that multimodality will quickly play a more prominent role in AI chatbot adoption. So far, text interfaces dominate.
But Google already reports 10 billion searches with Google Lens, and Meta’s Ray Ban smartglasses are very successful. Other than Google Search, the LLM answer format is easy to transport to other devices and modalities, which could transform AI.3
1 ChatGPT now has 400 million weekly users — and a lot of competition
2 Deep Dive: Tracking How ChatGPT + Search & Others Send Users To Your Site
3 Google Lens Reaches 10 Billion Monthly Searches
Featured Image: Paulo Bobita/Search Engine Journal