Why Marketplaces Cannot Ban Every Bot

Short answer: Because the traffic that looks automated is no longer all hostile. Search crawlers, price comparison services, and AI shopping agents now read listings on behalf of real buyers, and a block rule that cannot tell them apart from a scraper will turn away paying customers. The fix is not a bigger block list. It is a policy that changes by page: welcome on a product page, challenged at checkout.

The false positive math nobody wants to quote

Forrester put it bluntly in a post this month: retailers spent years getting better at spotting bots, and now some of those bots are customers. The numbers behind that shift are uncomfortable. Automated requests made up 51 percent of all web traffic in 2024, and the malicious share climbed to 37 percent. That leaves a large middle: automated traffic that is not attacking anything. A block rule tuned to catch the 37 percent will also catch part of the rest, and the part it catches is the part that buys.

The cost of guessing wrong is not theoretical. On the average ecommerce platform, between 2 and 10 percent of orders rejected for suspected fraud are placed by real customers. Those are sales lost after the business already paid to acquire the visitor. Globally, false declines cost retailers roughly $443 billion a year, about nine times the losses from the fraud the declines were meant to stop. A marketplace that bans bots aggressively is not choosing safety over revenue. It is trading one loss for a larger one, and the larger one never shows up on a security dashboard.

The bots you actually want on your listings

Three kinds of automated visitor now show up with a legitimate reason to read a listing. The first is the oldest: search crawlers and link preview fetchers, the bots that put your products in search results and build the card when someone shares a link. Block Googlebot and your listings stop appearing in search. The second is price comparison and marketplace aggregation, including Amazon gathering data for its Buy for Me feature. The third is new: AI shopping agents acting for a consumer, and answer engines like ChatGPT and Gemini fetching product content so they can surface it in a synthesized answer.

The scale is no longer marginal. Akamai measured commerce at 47.9 percent of all AI bot traffic on its network from July to December 2025. HUMAN's benchmark found 77 percent of agentic traffic lands on product and search pages, and only 2.3 percent reaches checkout. That pattern is the tell: these agents browse like shoppers and rarely buy directly, because the purchase still usually happens through the person. A marketplace that treats all of that as scraping is blocking the discovery channel its own sellers depend on.

Same visitor, different verdict depending on the page

The useful way to read automated traffic is by two things at once: what kind of traffic it is, and which path it lands on. The same AI agent is welcome on a product page and suspicious when it hammers login. Fraud bots belong at login, gift cards, checkout, and payment, where the controls should be strictest. A verified identity is a starting point, not a verdict: a legitimate agent can still be misused, and a spoofed agent at checkout is a different problem from a crawler reading a listing.

This is why path-based policy beats a site-wide block. Allow the crawler on product and search pages, throttle it on internal search and faceted filters where it burns crawl budget, and challenge anything that touches an account or a payment. The failure mode to watch is controls disagreeing with each other: the bot tool allows a verified agent, then a CDN rate limit blocks it two hops later, and no single team sees the full path. When controls are tuned together on real traffic, false positives in booking and checkout flows have dropped by more than 80 percent in deployments Huskeys describes.

What a seller should watch instead of the block list

Sellers cannot set marketplace policy, but they can read their own traffic. Segment it. If sessions are up and orders are flat, look at what the extra sessions do: scrapers read sold listings and move on, while real buyers add to cart. Watch the payout destination the way you would watch a bank account, because the payout edit is still the theft, and watch listing velocity on new accounts in your category. A dozen identical listings appearing overnight is not a competitor. It is a farm.

And when a marketplace asks you to verify, verify fast. The hold that follows a bot attack lifts on delivery confirmation and clean account history, not on argument. The sellers who get their payouts back fastest are the ones whose tracking is already uploaded and whose messages are already answered. None of that requires banning a single bot.

Keep reading: Why Bots Hit Marketplaces and Resellers First, Why Marketplaces Freeze Payouts After a Bot Attack, Seller Account Takeover: How Bots Drain a Marketplace Payout.

Frequently asked questions

Why do marketplaces let any bots in at all?

Because a large share of automated traffic does useful work: search crawlers index listings, link preview bots build share cards, and AI shopping agents fetch products for real buyers. Blocking all of it would remove listings from search results and from the AI answers shoppers now read before they buy. The marketplace's job is sorting the useful automation from the hostile kind, not banning the category.

How can I tell if the traffic on my listings is a bot or a buyer?

Look at what the session does, not just that it exists. A scraper reads sold listings and leaves; a buyer adds to cart, asks a question, or watches a listing. In analytics, segment by behavior: high page views with no cart adds and no messages is the scraper pattern. A spike in sessions with flat orders is the same signal at store level.

Will blocking all bots hurt my search ranking?

Yes, if the block catches search crawlers. Googlebot and Bingbot need to read your listings to index them, and a firewall rule that blocks them removes the pages from search results. Use robots.txt to steer well-behaved crawlers away from low-value paths like internal search, and use rate limits and firewall rules for the traffic that ignores robots.txt. Robots.txt is a request, not a lock.

What should I do if my listings get scraped?

First, check whether anything was actually lost: scraped prices feed counterfeit listings, and sold-listing data tells counterfeiters what to copy. Report the scraping to the marketplace with the pattern, times, and affected listings, tighten nothing that would also block search crawlers, and watch your payout destination and account settings. Scraping alone rarely drains money; the payout edit that follows it does.

Get a free bot-traffic audit