I have seen AI generate an interview guide, a set of personas, and a tidy list of “user insights” in less time than it takes to recruit one participant. It is easy to see the appeal. Research takes time, and a convincing answer is already on the screen.
But who did we learn something about? Our users, or the model’s idea of them?
That question matters because research is not just a way to produce quotes for a slide. It is how we find out where our assumptions about people are wrong.
A simulated user can sound surprisingly real
Give an AI tool a persona and ask it to react to a product. It may tell a coherent story about what someone wants, what they find confusing, and why they would leave. Some of it will sound familiar. That does not make it an observation.
A CHI 2025 study asked qualitative researchers to work with AI-simulated participants. The responses initially resembled human stories, but the researchers noticed important differences as the conversations continued. They also raised questions about whose perspective a simulation represents and whose voice is missing. In a separate study of first clicks, AI-generated choices often differed from those of real participants. These studies do not say that AI has no place in research. They do show why plausible output needs a different label from evidence gathered with people.
If a model says that a customer would trust a new feature, we have a hypothesis. If a customer shows us where they hesitate, what they misunderstand, and what they do next, we have an observation.
Before the session: use AI to get ready
There is plenty of useful work around a study that does not require pretending the tool is a participant:
- Challenge the plan. Ask what assumptions the study relies on, which users might be missing, and what the proposed method will not tell you.
- Improve the questions. Use AI to spot leading language, unclear tasks, or an interview guide that asks for opinions instead of recent behaviour.
- Prepare alternatives. Generate a few ways to explain a task, recruit participants, or respond when a session takes an unexpected turn.
- Practise moderating. Let the tool play a difficult interviewee so you can rehearse follow-up questions. Treat the rehearsal as practice, not as data.
I would still review every suggestion. A well-written interview guide can ask the wrong question very politely.
During the session: pay attention to the person
Real people surprise us. Someone may ignore the prominent button we were sure they would click. They may understand the words on a screen but mistrust the offer. They may complete a task in a way we never considered, or stop because a constraint outside the interface makes the whole service impractical.
Those moments are the point of being there. We can ask what happened, observe the context, and notice the difference between what someone says and what they do. AI can help take notes or suggest a follow-up, but it should not pull our attention away from the person in front of us. Building trust in a usability test still matters when AI is in the room.
After the session: make the evidence easier to work with
Transcription, organising notes, finding repeated themes, and preparing a first summary can take a lot of time. AI can help here too. I would use it as a second pair of eyes: ask it to group observations, find contradictions, and show which parts of the material support a proposed finding.
Then go back to the source. Did the participant actually say that? Was the comment about the product, or about a situation the summary left out? Did one unusual but important experience disappear when the model grouped everything into a neat theme?
Before putting recordings or transcripts into any tool, I would also check what participants agreed to and how the organisation handles that material. Faster analysis is not a reason to lose their trust.
Where synthetic users fit
I would use a synthetic user to explore possibilities: What question have we forgotten? What objection might we hear? Which edge case deserves a closer look? That can be a useful way to prepare a study or challenge a team’s confidence.
I would not present the answers as “users said” or use them to replace a round of research with the people affected by the product. When the decision depends on whether people can complete a task, understand an offer, or trust a service, we need to see what those people actually do.
AI can make parts of research faster. The best outcome is that we spend more of the time we save learning from people, and less time formatting what we already know.