GenAI for Complex Questions, Search for Critical Facts
Users choose AI to explore and synthesize information; but they rely on traditional search when accuracy and trust are critical.
---
GenAI for Complex Questions, Search for Critical Facts
Maria Rosala and
Josh Brown
February 27, 2026
Email article
Summary:
Users choose AI to explore and synthesize information; but they rely on traditional search when accuracy and trust are critical.
Users are increasingly turning to generative AI (genAI) for information-seeking tasks — but not uniformly. In a recent study, we observed how people chose between AI chatbots and traditional search for real tasks. Users turned to AI when they started with vague ideas, juggled multiple requirements, or combined information from many sources. They used traditional search when they needed accuracy, control, and trusted sources.
###
In This Article:
Why Users Turn to GenAI
Why Users Still Turn to Search
About the Study
Conclusion
Why Users Turn to GenAI
#### AI as a Starting Point when the Next Step Is Unclear
Some participants began with genAI chatbots when they weren't yet sure what they were looking for or how to describe it. Unlike traditional search tools, which require specific search keywords and, thus, some knowledge about the topic, chatbots offer greater flexibility when the search space is unfamiliar.
Participants used vague or exploratory prompts and let the AI suggest possible directions.
“It's made starting research or looking into something a little bit easier because it can kind of give you the best options to start.”
"It’s (...) one of the most helpful places to start. (...) It can really help direct me into a more narrow type of search."
If AI is fundamentally changing search, what else is it changing? Discover these new AI user patterns and leverage them for your next design in our live course, Designing AI Experiences.
One participant, who had recently purchased a Kindle, wanted to find free books to read on it. He wasn’t sure what options existed, so he turned to ChatGPT with a broad prompt:
Prompt: I just got a Kindle, and I am curious if there are free books that I can use on Kindle, or are all books going to be purchases?
He received six suggestions, which included Libby, public domain sites, and Prime Reading on Amazon. Reading through them helped him narrow in on the options that felt most relevant and achievable. Reflecting on his approach, he explained:
“I'm starting wide with, like, this big umbrella idea. Using an AI model of some kind helps get me more specific, to then be like, what do I actually want? Or what am I actually thinking?”
This also allowed users to bypass a common frustration in traditional search: keyword foraging — the trial-and-error process of guessing which search terms yield useful results. By offloading that cognitive burden to AI, users could focus more on refining their goals, rather than reverse-engineering the right query.
#### AI Handles Multiple Constraints Better
Many of the information-seeking tasks conducted by the participants involved constraints — such as a budget, timeline, or location. These constraints are often difficult to include in traditional search queries, especially when several need to be considered simultaneously. One participant planned a golfing-themed bachelor trip for a group of four friends. He had multiple requirements:
- A 3-4 night trip with 3-4 rounds of golf
- Multiple golf courses in the area
- Budget under $1,000 per person
- Preferably close to his home state
Rather than beginning with a search engine, he opened ChatGPT. He began with a broad prompt.
Prompt: Best destinations for four-person golf trips under $1,000
He then refined his query, asking ChatGPT to exclude destinations east of the Mississippi.
“Whenever I have multiple constraints (...) ChatGPT is usually my go-to because ChatGPT seems to be able to handle three or four constraints better than Google can.”
(Although Google now has AI mode, this feature was relatively new in our study and underutilized due to its low discoverability.)
#### AI Saves Interaction Cost
Several participants mentioned that using AI saved them clicks that they would have had to spend visiting multiple websites to acquire all that information, and thus shortcut the task of information foraging.
“You get so much more information (...) I definitely am not getting that on Google without having to do multiple clicks. And I mean, I guess that's the value of ChatGPT or Gemini or Grok or any of those things (...) it just saves the user clicks.”
GenAI chatbots aggregate a lot of information and present only what’s relevant to the user’s query, saving users the interaction cost and cognitive load involved in doing the same work by themselves.
“I do prefer (...) an AI model answering this type of question… so much more research would have to be done if all I’m doing is clicking through these [Google search results] and none of these are actually giving me the type of specific response I would get in an AI.”
AI was especially helpful when scouring product reviews. Multiple participants asked for summaries of reviews in their information-seeking process. For example, one participant, who wanted to find out what people were saying about a particular car model she was considering buying, didn’t want to get caught up reading many individual reviews, which sometimes got too technical.
“Sometimes, it's just too much information (…) some people go in [to] so much deep details (…) I'm just like, ‘get to the point, is it worth keeping or not?’ So, I would probably [lean] (…) more towards the AI tool for [that].”
Instead of reading many reviews on Reddit or other sites, she preferred having Gemini go through all the reviews and give her an understanding of how people find the car. Another participant mentioned that this was why he uses Rufus on Amazon.
“I do feel like [Rufus] helps me aggregate information and not have to read through every single review of a product.”
The ability of AI to summarize many reviews and product information makes it a useful companion. One participant likened AI to a salesperson, but without the selling pressure.
“It was almost like talking to a knowledgeable salesperson that isn't trying to direct the conversation, that's letting me kind of direct the conversation and really dig in and explore what I want to learn about.”
#### AI Reduces Working-Memory Load
However, users frequently compare options across different websites or sources. To do so, traditionally, they would either keep all the candidate choices in their working memory or use some form of external memory, such as page parking or notes.
AI provides a convenient solution. We observed participants frequently using AI to consolidate information and make comparisons easier.
Information Aggregation Without AI
A genAI novice participant researching day hikes near Tucson, Arizona started on Google and worked through two websites she was familiar with: the National Park Service website and AllTrails.com.
She wrote down hikes from the National Park Service website that looked suitable for her group, along with their distances. On AllTrails.com, she checked ratings and descriptions, looked for any hikes she may have missed, and starred hikes she’d already noted if they had good ratings or looked particularly appealing. This work took about 20 minutes, and the user needed a memory aid (pen and paper) to avoid overtaxing her working memory.
Information Aggregation with AI
The facilitator then asked her to use Gemini for the same task. Instead of listing all hikes in the national park, Gemini gave her hikes only on the east side and, without her asking, helpfully sorted them by difficulty. After completing the task, she reflected on the difference:
“Gemini (...) seemed to give it to me quicker, more summarized, laid out a little better (...) if I'd gone to Gemini first, it might have been quicker.”
Another advantage of AI is that users can ask it to display information in tables or another format supporting comparisons. Users with higher AI literacy utilized this strategy in their prompts.
For example, one participant planning a day trip to Montauk, Long Island asked ChatGPT for beach recommendations and received a list of beaches with descriptions. He followed up by asking ChatGPT to present the information in a table with pros and cons.
One participant asked ChatGPT to generate a table with pros and cons for the beaches it had recommended. The table format made it easier to scan multiple options at once.
Another expert user of genAI who was researching car models asked for tables in two of his prompts. As he prompted for a table, he said:
“[I] appreciate that it can create those tables that help me see a lot of information at a glance.”
Why Users Still Turn to Search
AI didn't replace traditional search entirely. Instead, participants made deliberate choices about when to use each tool, often treating them as complementary rather than competing. Even though Google and other search engines blur the lines between AI search and traditional search, our study participants still drew clear distinctions between them: they gravitated towards traditional search when they wanted control over the search process and confidence in the results.
Many participants (6 out of 9) went back and forth between search and an AI chatbot. For example, one participant planning a day trip to Montauk started on Google Maps, to get a sense of what was around the area. He then asked ChatGPT for recommendations and compared the results he received to what he'd previously seen on Google Maps.
#### Search as a Validation Mechanism
Search was commonly used as a validation mechanism and to get specific, trustworthy details. For example, our golf-trip researcher ping-ponged between genAI and Google search: he read the ChatGPT’s summary about each resort, then searched for each promising resort’s name to find the resort’s website and confirm what it offered.
This behavior reflected users’ mental model of genAI as helpful but fallible. We saw AI make mistakes multiple times during the sessions. For example, when our participant asked ChatGPT for golf resorts in the Midwest, it listed a resort in New York. The participant noticed the mistake and remarked:
“This is where ChatGPT can begin to lose its charm.”
#### Search for Pricing or Key Facts
Several participants said that they wouldn’t fully trust pricing information provided by AI. Even when they expected AI to be directionally right, they still wanted to confirm exact numbers through traditional search.
“Before just writing them [the price ranges] down, I would go to Google and search them (...) it wouldn't surprise me if that number wasn't close at all (....) I wouldn't trust it wholesale.”
“This is where I get skeptical (…) when we're talking, like, actual numbers, specific data. (...) This is actually going to affect me if this is wrong. And with AI in general, that's when I'll pretty much always go back to a more firm source.”
One participant believed the prices provided by AI might be in the right ballpark, but he worried that AI could be drawing from outdated sources or misleading ranges (like starting from price information).
#### Search when the Stakes Are High
Users were more likely to ditch AI or verify its results with search when they researched something important to them. When errors are costly, people tend to maximize: they want higher confidence than what they felt AI can reliably provide.
“It’s not that big of a deal if I spend 20 bucks on the wrong smart-home product. But I want to really make sure I know what I'm getting with the car that I'm spending a lot of money on. So, that's also why I would be interested in going to some other websites to sort of compare (…) [and] see what matches up.”
This pattern extends beyond purchases. For example, a participant gravitated towards reputable sources when researching the health implications of omega-3 supplements.
“I don’t really want to ask Google AI (…) I like to look at the source directly, especially, like, if it's a (...) university or some sort of government health website.”
Citations Don’t Fully Solve the Trust Problem
Even when genAI tools provided citations or source links, some participants struggled to understand which claims were supported by a named source and which parts were uncited synthesis.
“You can click like these links [citations] or whatever to see the source, but it's a little bit harder to tell what source is providing these conclusions versus another source (....) There's all this big chunk of information here that doesn't have a source. So is that all still from this source above (...) where did that come from?”
“Feels like I don't control the type of information or sources (…) it is just randomly sourcing. I don't know whether this, the citations they use, is really something that's trustful [sic].”
[Image: Google AI Mode is shown. After several sections of text, there is just one citation link. In the right rail, there is a list of websites and sources.]
Unlike search results, where each link is a distinct source, AI aggregates information in ways that obscure which claims come from which citations.
In these situations, participants preferred to start with Google and go directly to trusted sources.
#### Search for Specialized Topics
Finally, participants avoided AI for more academic or specialized research. One participant, a hobby historian interested in US military and presidential history, said he might use AI to generate research ideas, but avoided using it to conduct the research. Instead, he preferred to go to Google Scholar to find specific papers or books.
Similarly, another participant seeking new research on brain microstructures avoided ChatGPT because it commonly hallucinates papers and authors. He also noted that ChatGPT lacks access to paywalled articles and preferred Notebook.LM for conducting academic literature research.
About the Study
In a remote observational study, we asked participants with varying levels of AI experience and tech-savviness to bring real-life things they wanted to research online. We wanted to learn how users use generative AI in the information-seeking process.
How and where users chose to use AI in the information-seeking process depended on the nature of the task. Users brought and performed two main types of tasks:
- Understand tasks involved looking for information to understand or learn something. Some were exploratory: participants sought to deepen their knowledge of a topic. Others were more practical, focused on solving a specific problem.
- Choose tasks involved evaluating multiple options to make a decision. Some were straightforward product comparisons, while others — like trip planning — involved multiple, interdependent decisions.
The table below outlines some of the real tasks participants brought to the study.
| UNDERSTAND TASKS
| Learn about a topic
- Learn about James Madison’s presidency
- Find new research on brain microstructures
- Learn the risks and benefits of Omega-3 supplements
| Learn how to do something
- Learn how to remove odors from a sofa
- Learn how to get stains out of towels
- Learn how to fix a leaky drain pipe
- Learn how to get free books on a Kindle
| CHOOSE TASKS
| Trip planning
- Which hikes to do in Arizona
- Which beaches to visit on Montauk Island
- Where to go for a group golfing trip
- Where to visit in Puerto Rico
| Purchase something
- Choose a car model
- Choose a smart refrigerator
- Choose a portable soccer goal
Conclusion
GenAI and traditional search serve complementary roles in modern information seeking. Users reach for AI when they need to explore possibilities, juggle multiple constraints, or synthesize information across sources. But when the stakes are high — whether evaluating a major purchase or researching health information — they return to traditional search and trusted sources.
Related Courses
Designing AI Experiences
Interaction
Accelerating Research with AI
The AI-assisted user research workflow
Research
AI for Design Workflows
Leverage AI to enhance creativity, efficiency, and impact in everyday design work
Interaction
Related Topics
Artificial Intelligence
---
[Original source](https://www.nngroup.com/articles/ai-search-infoseeking/?utm_source=rss&utm_medium=feed&utm_campaign=rss-syndication)