What I Learned Testing AI for Shopping (And Follow Up Conversation with a Major LLM's PM)
- Dennis Yu
- Sep 21
- 3 min read
A week ago, I ran a personal online shopping experiment, posted on LinkedIn, got a pretty good amount of engagement, and eventually led to a call with Perplexity's Product Manager for Shopping.
Here is the backstory:
I used three of the major LLM platforms and an AI Shopping startup: ChatGPT, Claude, Perplexity, and Daydream AI, to help me shop for a very specific pair of shoes, and my experience differed quite drastically.
I wanted to unpack that experience further. Here's what I learned about the current state of AI-powered shopping, where the real UX gaps lie, and how platforms like Perplexity are thinking about the future of what they call "Gen-Commerce."
The Setup: Same Query, Four Different Platforms
The query was simple: I wanted to find a specific Nike Flyknit sneaker that is lightweight, breathable, affordable, and suitable for someone with foot issues. I gave each AI assistant the same prompt and evaluated them across three dimensions:
Product Discovery: Did it surface the right products quickly?
Experience Quality: Did it feel natural, helpful, human?
Trust Factor: Would I buy through this flow?
The Results: High Potential, Mixed Execution
Perplexity: Strong on product discovery. The engine provided detailed answers, relevant alternatives, and surfaced Amazon options I hadn't considered. It even highlighted materials like Flyknit—a key factor for me. But the flow still felt transactional, and the experience lacked warmth or conversational depth.
Claude: The most natural conversation. Claude felt like a helpful friend who asked the right follow-ups, but its product recall wasn't strong. It struggled to surface options and required me to dig deeper or reframe prompts.
ChatGPT: The most flexible. With the right custom instructions and GPTs, I could train it to become my shopping assistant. But out-of-the-box, it didn’t integrate directly with live commerce data—so the results were generalized.
Daydream: The most polished UI. Built as a shopping-first LLM, Daydream offered grid-style product displays, clear photos, and quick refinement prompts. It felt like an AI Nordstrom rep. But sometimes, the search results didn’t align with my intent—e.g., showing luxury sneakers when I asked for Nike Flyknits.
Expanded Takeaway: Context + Memory = Commerce Superpower
Let’s dig deeper into what this means from a product and UX perspective, especially after my 1:1 with Perplexity’s Shopping product lead.
1. From Keyword Search to Contextual Understanding
Today’s AI systems are rapidly evolving from static Q&A bots into contextual companions. He confirmed that Perplexity is actively investing in persistent memory and contextual search refinement. That means:
If I’ve shown interest in lightweight sneakers with arch support before, future queries should reflect that.
If I’ve searched for health-related foot issues, the assistant should automatically favor comfort-first shoes.
Think of it like Spotify Discover, but for your shopping preferences: inferred, remembered, and refined over time.
2. Memory as a Differentiator
Perplexity is building memory that extends beyond a single session, capable of tracking your preferences across time and verticals (e.g., travel and shopping). This unlocks:
Smarter recommendations based on real intent signals.
Seamless handoff between past research and future actions.
A deeply personalized commerce experience, without re-explaining yourself every time.
This memory layer is what sets “Gen-Commerce” apart from traditional search, and why it’s not just about better LLMs, but better product thinking.
3. Why This Changes Conversion
I told the PM candidly: I don’t need another list of links. I need confidence in the result, contextually tailored, visually clear, and frictionless to checkout.
The idea that I could:
Search conversationally,
Get curated results based on my prior needs,
And purchase instantly via Stripe (without hopping to external sites)…
...that’s when AI stops being a tool and becomes an experience.
4. "Ask Me a Better Question" Is UX Gold
Perhaps the most overlooked design opportunity is this: most shoppers don’t know how to phrase the perfect prompt. But a great assistant should. Imagine:
“You mentioned Flyknit and plantar fasciitis, do you want to see more options with arch support and soft midsoles under $120?”
That’s not just helpful. That’s commerce as conversation. And it's what most LLMs are still missing.
Where It Goes Next: The Future of AI Shopping
The next evolution of shopping won’t look like Google results or Amazon filters. It will look like a back-and-forth dialogue with a trusted, intelligent agent that knows you, your context, and your preferences, and serves up the best product, quickly and confidently.
Platforms like Perplexity are already in motion, balancing the challenge of serving horizontal queries with the opportunity of crafting vertical, deeply personalized commerce journeys.
It’s still early days, but the signal is clear: the real AI commerce race won’t be won on recall.
It will be won on strong relationships.
Comments