How search query length is shifting in the LLM era: Insights for brands

The advent of large language models (LLM) marks the most significant change in the search experience since the rise of mobile technology.

Google SERPs are evolving beyond the traditional 10 blue links, introducing AI previews and other enhanced features. The full impact on brands and consumers is still being felt

This article explores the hypothesis that integrating LLMs into everyday life will fundamentally change the way people conduct research.

With LLMs capable of providing detailed answers, consumers are likely to move away from simple queries (e.g. “car insurance”, “running shoes”) towards more complex conversational searches (e.g. “how much does l ‘car insurance for a 2022 agreement’).

Hypothesis: Change in Consumer Search Behavior

Since the introduction of LLM patterns, consumers have begun to change their search patterns for items with longer queries.

Methodology: Analyze Google Ads data

Since search data from various LLMs, including Google’s Gemini and AI overviews, is not yet available, I used search query data from Google Ads.

I extracted all search queries from January 1, 2022 to May 31, 2024, resulting in 41 million search queries and over 83 million impressions.

I divided the data into three categories:

• Pre-ChatGPT (January 1, 2022 – November 30, 2022): There are no consumer LLMs yet. Consumers would still search “traditionally”.
• 12 months after ChatGPT (December 1, 2022 – December 31, 2023): During this data window, consumers are now exposed to LLMs and may have begun to change the way they interact with search.
• 2024 cumulative (January 1, 2024 – May 31, 2024): Gemini launched on December 6, 2023. This bucket helps keep the months clean, but now that Google’s product is fully launched, this would be the most “advanced” they could get offered. the date range.

Results: How search term length and statistics changed

After examining 41 million search terms and corresponding data on impressions, clicks and costs over the last 29 months, what can we conclude?

Search term length has increased since ChatGPT launched, but it has been slow. CPCs are up and CTR is down. LLMs and AI previews appear to drive fewer clicks to traditional ads because consumers searching for longer queries can find what they are looking for through AI previews.

Let’s start by looking at how the data changes over time based on query length. Looking at the data from these three datasets, you can see some small changes over time.

Most notably, there was a mismatch between 5-6 word keywords and 7-8 word keywords. Search terms containing 7-8 words have almost doubled since ChatGPT launched and have taken that volume out of the 5-6 search term bracket. While the others remained fairly constant, only changing <15 in either direction.

What could this mean? I would argue that consumers who previously searched for slightly more complex terms have started to get even more specific by adding additional conversational modifiers to their queries.

Overall, we are still only talking about 10% of keywords containing more than seven words. The majority still contain fewer than four search terms in the query.

Now let’s look at some flow metrics and how they change over time. Starting with CPC, we can see a sharp increase in CPC costs over the last six months.

All search term groups felt this CPC increase, with the largest increases seen in groups of 3-6 keyword lengths. CPCs are up 60% after being relatively stable over the past 18 months.

There’s definitely something going on here, especially when you look at click-through rate on the same data set.

The assumption about click-through rate with AI previews would be that ads generate fewer clicks. So far, that seems to be true, with click-through rates dropping across the board. Again, this is especially noticeable in the 5-7 search term length brackets.

This would fit the idea that the more detail a person enters into the query, the more likely it is that Google will present an AI preview due to a more defined query.

The search volume is there because impressions haven’t gone down. However, it’s important to note that while CPCs have increased, CTRs have decreased over the past six months, coinciding with the rollout of AI insights.

---

Actions to take: adapting to the new research environment

Monitoring search term data

Start by tracking your search term data. This allows you to understand how your specific products and keyword sets are influenced and identify any shifts or trends that might indicate changes in consumer behavior.

Analyze consumer behavior

If you notice a change or evolution in the way consumers are searching, it’s time to take it a step further. Analyze data to uncover insights that can influence the types of landing pages, ad copy, and content you create.

As consumers move from traditional search queries to more conversational searches, they provide more direct information about their intent, which can significantly improve conversion rates.

LLM monitoring

We started tracking various LLMs (such as ChatGPT, Perplexity, and Gemini) to observe how often they show certain marks and how those results fluctuate daily.

For example, over a week, the presence score (the frequency with which a brand appears in different LLMs) can vary by up to 20% per day.

Understanding and monitoring this data at these early stages can provide a competitive advantage as the pace of evolution of search technologies continues to accelerate.

Random discovery

I didn’t know that consumers searched using emojis alone. This is not something I had considered doing myself.

However, after going through the search query results, I found many searches containing only emoji.

The example above shows consumers searching for the movie “Teenage Mutant Ninja Turtles.” Google is smart enough to interpret and map these searches correctly.

Depending on your industry and brand, this could be an untapped opportunity worth exploring.