# Four AI Agents Said an In-Stock Product Was Backordered

Four AI agents said an in-stock product was backordered. Fixing the facts stopped the wrong answers, but it did not make the product recommended. A conversation with ecommerce practitioner Leo Nguyen on contested data, hallucinated policies, and the gap between being readable and being chosen.

Canonical URL: https://nile.app/blog/what-ai-agents-actually-see/
Author: Alice Cui
Created: 2026-08-17T11:40:54.747Z
First published: 2026-08-17T12:53:31.476Z
Last modified: 2026-08-17T12:53:31.476Z
Language: en-US

## Key takeaways

- Shopify's Agentic Storefronts are on by default: most merchants are already in AI commerce without a strategy.
- The common product-data failure is not missing markup. It is data that is present but contested across ERP, PIM, feed, and storefront.
- Fixing Home Elegance's data took the false backorder claim from 4 of 4 agents to 0, yet category ranking stayed flat. Correctness is necessary, not sufficient.
- Legibility gets a product read; corroboration and context get it recommended.

*A conversation with ecommerce practitioner Leo Nguyen on contested data, hallucinated policies, and why being readable is not the same as being chosen.*

In June, a Home Elegance USA product page told shoppers an item was in stock. ChatGPT, Gemini, Perplexity, and Claude all said it was backordered. Nothing looked broken. The storefront template had hardcoded a backorder message into its HTML and let JavaScript paint the correct status over it for human visitors. People saw the right answer. Machines kept the wrong one.

Leo Nguyen found and fixed that failure. He is the founder of LUMA-E, an ecommerce infrastructure firm, with more than 50 projects behind him, concentrated on Shopify Plus and Magento 2, working with catalog-heavy brands (jewelry, knitwear, furniture, beauty, publishing) across the US, UK, EU, Australia, and Vietnam. We spoke with him about what AI shopping systems actually receive from merchant stores, and what he has learned watching four different agents read the same catalog.

*This conversation has been edited for length and clarity. The Home Elegance case discussed below was a client-approved engagement conducted and documented by LUMA-E, not independent research.*

***

### Your clients are mostly established brands, not AI enthusiasts. Where are they on AI commerce adoption?

Almost none are true early adopters. A few are experimenting deliberately, but most are pragmatists who are already in it because Shopify flipped Agentic Storefronts to default, not because they had a strategy.

The interesting group isn't the people asking "how do I get into AI shopping." It's the ones already in it who don't know what the agent is saying about them.

*Editor's note: Shopify's Agentic Storefronts are active by default for eligible stores, exposing products to AI channels including ChatGPT and Microsoft Copilot, with Google AI Mode and Gemini in early access. The traffic is still small next to organic search, by Shopify's own account, but growing fast: AI-chatbot referral sessions grew more than 8x year over year in Q1 2026, AI-referred orders grew nearly 13x, and AI-referred visitors converted on product pages at nearly 50% higher rates than organic-search visitors.*

![Bar chart: Shopify Q1 2026, AI referral sessions up more than 8x and AI-referred orders up about 13x year over year.](https://nile-content-publisher-staging-igsqnij3dq-uw.a.run.app/blog-media/media/leo_growth_02e59f9fa9/leo_growth_02e59f9fa9.png)

### Mirakl benchmarked 427 product pages across 35 countries and scored them 48 out of 100 on machine readability. Does that match what you see in the field?

Yes, 48/100 matches what I see, and the tail is worse than the average suggests. Most of the merchants I work with would land around a 4 or 5 out of 10. The score is dragged down less by missing markup than by data that's present but contested: the same field carrying different values in the ERP, the PIM, the feed and the storefront, with no rule for which one wins.

The biggest perception-versus-reality gap: merchants equate "looks right to a human" and "we have schema" with "we're ready."

![Mirakl GEO Readiness Analyzer: 48/100 average machine-readability across 427 product pages in 35 countries.](https://nile-content-publisher-staging-igsqnij3dq-uw.a.run.app/blog-media/media/leo_readiness_v2_5ab561d04a/leo_readiness_v2_5ab561d04a.png)

### What does "contested" look like in practice?

Price and availability drift is the classic. Variant and spec mismatches. B2B contract or tier pricing living in the ERP while the storefront exposes retail values. Different systems independently recomputing the same field.

The way I put it is "lock the field, then carry it." Decide once, at ingest, what the true value of a field is and who owns it, then make sure every surface downstream carries that same value instead of recomputing its own.

A wrong description is embarrassing; a wrong price an agent commits to in a conversation is a lost sale or a promise you can't keep.

### Walk me through the Home Elegance case. What did the agents actually get wrong?

Home Elegance carries 30 brands across a 30,855-product Shopify catalog. Between June 27 and July 9 we ran five buyer questions across ChatGPT, Gemini, Perplexity, and Claude, scored on a 20-cell rubric.

The headline failure: all four agents described an in-stock product as backordered. The theme served a hardcoded backorder statement in the HTML and inserted the real inventory status later with JavaScript. Browsers executed the JavaScript. The machine-facing HTML kept the wrong answer.

We made the backorder statement conditional, server-rendered the inventory status, and reran the same queries. Four out of four agents claiming backordered went to zero out of four, and three of the four explicitly confirmed the product in stock. Claude still had a retrieval coverage issue. These systems differ in how they crawl and index; you cannot assume one fix lands everywhere.

### And beyond the stock status?

Three things, each instructive. Gemini misidentified the retailer as one of the wholesale brands it stocks, because the site had competing JSON-LD graphs: different components generating overlapping Product, Organization, and Breadcrumb nodes, one of them missing the correct brand relationship. Perplexity reported a 25% restocking fee where the real policy was 20%, and the number 25% appeared nowhere on the site. And ChatGPT looked at the conflicting signals, concluded the site was broken, and recommended competitors instead.

### After the fixes, did visibility improve?

Correctness improved. Ranking did not. Category visibility stayed roughly flat, about 4 out of 12 before and 3 to 4 out of 12 after. These were correctness fixes, not ranking fixes.

*Editor's note: This distinction matters. The fixes eliminated incorrect answers, but they did not materially improve category ranking. Correctness is necessary for reliable representation, but it is not sufficient for recommendation.*

### So what separates a product an agent can read from a product an agent recommends?

Legibility gets you read; corroboration gets you recommended.

Once the facts are consistent and machine-readable, the hardest step is rarely technical anymore. It's contextual and organizational. Agents comparing dozens of alternatives get more confident when they see consistent facts, clear category-level brand answers, consistent narratives across sources, and corroboration beyond the merchant's own site. Parseable isn't the same as chosen.

### You said organizational. Where does readiness stop being an engineering problem?

Which system owns this field is a decision no single team wants to make. Merchandising, ops, the ERP team and the agency all touch the data and none of them owns the truth.

More markup doesn't fix that. Someone has to decide which source owns each product fact, what happens when systems disagree, how updates propagate, who approves exceptions.

### Merchants are being sold citation dashboards right now. Should they buy the metric?

Citations are a leading indicator, not proof. They tell you the agent can find you, not that it's making you money. I'm deliberately skeptical of citation-count dashboards for the same reason I distrust impression counts: they move without the business moving.

What I trust more, in order: qualified traffic, assisted conversion, revenue. And the honest version today is that measurement is still immature. Anyone claiming clean direct-revenue attribution from AI shopping right now is ahead of what the data supports.

*Editor's note: that last point is Leo's read of the current state, not settled consensus. The directional data that does exist (Shopify's referral and conversion figures above) supports taking the channel seriously while treating precise attribution claims with care.*

### Last one. What assumption about AI commerce ages worst?

The assumption I think turns out wrong is that AI visibility is a marketing problem. It's an engineering and data-ownership problem.

Two or three years out I think we stop talking about "AI visibility" as a discipline at all, the same way we stopped talking about "mobile optimization." The brands that win the agent won't be the ones who "optimized for AI"; they'll be the ones who fixed which system owns each field and made every surface carry that truth.

***

## What we take from this

Leo's evidence covers the foundation: product facts that are accurate, current, consistent, and owned. The Home Elegance case shows how much can break at that layer, but it also shows its limit. After the facts were fixed, the agents stopped giving incorrect answers, while the product's category ranking barely changed. Correctness helps a product be understood accurately. It does not, by itself, make the product more likely to be chosen.

Choosing requires answers a clean catalog does not carry. Who is this product for? When is it the right call, and when is it not? What trade-offs matter, and what do reviews consistently say? That is [product context](https://nile.app/blog/ultimate-guide-agentic-commerce/), and it sits above correctness in what we think of as the new merchant stack:

1. **Correctness.** Accurate, consistent, owned product facts. Leo's layer, and the one the other three are only ever as good as.
2. **Context.** Audience, use case, trade-offs, differentiation, intent. What turns parseable into choosable.
3. **Distribution.** Getting that product intelligence into ChatGPT, Gemini, Copilot, Meta, Google, and whatever ships next, in the formats each requires.
4. **Performance.** Seeing what gets retrieved, recommended, clicked, and purchased, and feeding it back into better context.

![Nile's four-layer merchant stack: correctness, context, distribution, performance.](https://nile-content-publisher-staging-igsqnij3dq-uw.a.run.app/blog-media/media/leo_framework_v3_07e49a4cc4/leo_framework_v3_07e49a4cc4.png)

Nile [operates as a channel](https://nile.app/blog/agentic-commerce-channel-not-storefront/) across the top three layers, on the assumption Leo's prediction comes true: correctness becomes the constraint the other three inherit. Get it wrong, and context, distribution, and performance just amplify the wrong answer faster.

The first era of AI commerce was about showing up. The next is about being understood well enough to be chosen.

## Sources

* Shopify, [AI-referred shoppers convert better and spend more](https://www.shopify.com/enterprise/blog/ai-search-insights) (2026): AI referral sessions, orders, and conversion data cited in this article.
* Shopify Help Center, [Shopify agentic storefronts](https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts): default-on behavior for eligible stores.
* Mirakl, [The State of eCommerce GEO](https://ecommercegeoinsights.mirakl.com/): the 48/100 readiness benchmark across 427 product pages in 35 countries.
* The Home Elegance USA case is a client-approved engagement conducted and documented by LUMA-E, June 27 to July 9, 2026.

*Related reading: [Don't Think of Agentic Commerce as a Storefront. Think of It as a Channel.](https://nile.app/blog/agentic-commerce-channel-not-storefront/) and [From Zero to Hero: The Ultimate Guide to Agentic Commerce for Merchants](https://nile.app/blog/ultimate-guide-agentic-commerce/)*

**See what agents see for your own catalog: [preview your brand agent](https://nile.app/join).**

## Frequently asked questions

### Do AI shopping agents see the same product information as human shoppers?

Not always. AI systems may consume machine-facing representations that differ from the rendered page. In the Home Elegance case, the served HTML carried a hardcoded backorder message that JavaScript replaced for human visitors, so all four agents tested reported an in-stock product as backordered.

### Does fixing product data improve AI rankings?

It removes wrong answers rather than beating competitors. After the fixes, the false backorder claim went from four agents out of four to zero, but category visibility stayed roughly flat. Correctness is necessary for reliable representation; recommendation also needs context and corroboration.

### How should merchants measure AI visibility?

Treat citations as a leading indicator, not proof. The hierarchy that matters runs citation, then qualified traffic, then assisted conversion, then revenue. Attribution is still immature, so treat precise direct-revenue claims with care.

## Sources

1. [AI-referred shoppers convert better and spend more](https://www.shopify.com/enterprise/blog/ai-search-insights) — Shopify
2. [Shopify agentic storefronts](https://help.shopify.com/en/manual/online-sales-channels/agentic-storefronts) — Shopify Help Center
3. [The State of eCommerce GEO](https://ecommercegeoinsights.mirakl.com/) — Mirakl