This autumn I spoke (/speak / am speaking) on different conferences on the topic of AI and UX research. For those of you that did not manage to see it live in Stavanger, Trondheim or Stockholm, I thought I write a little blogpost!

Prefer a video over reading? 2 of the conferences actually published the videos on youtube (together will all the other amazing talks).

The one at TDC Trondheim is 20 minutes, very concise.

The one at Hello Stavanger is 30 minutes, a bit more elaborate.

But if you prefer reading… Here is a short summary!

Why do we need UX research?

We can build anything, the sky is the limit! With services like claude code, Codex, or a CoPilot, any genius can implement the most complex solution. And with services like Lovable, bolt.new your neighbour can build a new website, app or game in seconds (OK minutes).

And that is AWESOME.

Finally coding and creating is not limited anymore to having a strong engineering background. If you have a great idea, you can create it. For yourself, for your community, or for everyone. And even though that sounds great…

It is not so great that everyone can build everything for anyone. If you want to run a profitable business you need to deliver value for your customers, that are then willing to pay for the product. If you don't deliver value, your product won't be used (or be profitable), and your tokens will not help you get out of there.

In a world where it is so simple to build everything for no one, it is more important than ever to understand your users (by conducting user research). Tokens are cheap. False assumptions are expensive.

Can we use artificial users?

OK cool Kim, we need to do UX research! But we do not have a UX researcher, and its also really hard to find our users, can we just ask Claude what our target group needs? (spoiler: no.)

Welcome to the world of artificial users! There are different ways to replace your users with AI users, and they all come with pros and cons. Lets look at how the market looks like now:

Persona prompting

Ask an AI to answer as “a 30-year-old woman with a university degree and a middle income” and you get an answer that sounds right. Researchers tested exactly this, across 19 experiments with 164 outcomes. Their AI got only 14 facts about each person, like age, education and income. It scored 75% accurate. Great, right? Well... Random guessing already scores 63%. And an AI that knows nothing about the person scores 73%. So those 14 facts barely helped.

We should look at the correlation: do the AI’s answers go up and down like the real people’s do? 0.145. That is a weak link. The AI gets the average of the group about right, but it can’t tell the people in that group apart… So. this is not so useful.

Source: Peng et al. (2025/2026), "Mega-Study of Digital Twins"

Fine Tuned models

Fine-tuning means you keep training an AI on real answers from real people, instead of just telling it to play a persona. Does that help? Only when the AI has seen the question before… (Well I also know the answers to a test if I have seen them before?). Researchers trained an AI on 50 years of survey answers from almost 69,000 Americans. For questions that had been asked in other years, it got about 77% of individual answers right, and its estimate of what the country thinks was almost perfect (correlation above 0.98). But for a question it had never seen, only 17.5% of its national estimates landed within 5 points of the truth, and the correlation dropped to 0.68…

We run research to uncover the unknown answers to unasked questions, so these numbers do not look to good to me.

Source : Kim & Lee, "AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction

Peng et al. (2025/2026), "Mega-Study of Digital Twins"

Digital twins

A digital twin is the next step up from a persona. Instead of a few facts, Stanford researchers gave an AI a full two-hour interview with a real person and asked it to answer as that person. They did this for 1,052 Americans. The AI matched people’s survey answers 85% as well as people match their own answers when they retake the survey two weeks later. In normal english, that is about 69% of answers correct.

But when it came to real choices with money (people could win up to $10), the match dropped to 0.66, while people themselves were nearly perfectly consistent (0.99).

That is why we still need to talk to real users.

Source Park et al., "Generative Agent Simulations of 1,000 People"

Synthetic panels

What if you don’t ask one persona, but a whole crowd? That is a synthetic panel. Stanford researchers asked GPT-4 to play thousands of Americans (each described by just age, gender, race, education and political leaning) and let them “take part” in 70 real experiments. They were trying to answer the question: could the AI predict which version of a message would work better? And... yes, it could! The predicted differences followed the real ones with a correlation of 0.85, as good as ordinary people guessing, or better. So if you want to know which option wins on average, a synthetic panel is surprisingly good.

But take it out of the survey and into real life, and it gets a lot shakier. In field experiments the score dropped to 0.27, which is, well very very low… So synthetic panels are fine for a first guess, but not for the real thing… Meaning it can not replace the power of UX research.

Source: Hewitt et al. (2024), "Predicting Results of Social Science Experiments Using LLMs"

Basically, what ever system you are using, the higher the human input, the higher the quality of the outcome is. So I would obivously advice you to run real interviews on real users.

How can I use AI to speed up my research processes?

Now that we have established that you still need to talk to your users (which you can always find btw, more on that in another blogpost), you might think that "UX research takes quite a lot of time". True, good work takes good time. But with the help of AI we can make it faster, but you can not take shortcuts because then the result will be useless.

I have been in situations where my clients have asked: can you just throw it in ChatGPT, I just need a summary. Sometimes it is quicker to show the result than to take the discussion, so while chatGPT creates the summary I started actually processing and synthesising the result. I forwarded the summary to the client, which agreed, that it was useless. (It came with general insights like: people do not like to pay *yeah, I can provide you with that insight on any service)

Time has passed and YES, you CAN use an LLM to help you with the processing… But it is not as simple as throwing out a prompt. You have to actually teach the skill of conducting UX research, and I like to devide that in (at least) 8 skills.

Skills needed

More on skils and LLMs another time - after NDC AI Oslo.

Conclusion

Can LLMS help with UX research? Yes! But you do need to understand what is actually happening, what the numbers mean, and hold the LLMs hand. You have to keep in mind that feeling confident is not the same think as being correct, and should avoid the gap of "quickly doing some UXR" and let you build a product on false hopes.
If you use synthetic users for a quick sense check, or a simple task as organising products, yes, go ahead.
If you teach an LLM your skills on how to synthesise and process data, and make it tripple check that it is providing insights related to your data and your work, go ahead.

If you have never done UX research, have no understanding of what you are doing, dont fall in the gap of getting some quick results. Book a chat with me to see how I can help you get insights that help your project forward.