Listings · Intermediate · 5 min read · 8 steps
How to A/B test your Amazon listing with Manage Your Experiments
Every listing is a guess. A good guess, usually, made by someone who knows the product, but still a guess about which photo or which first bullet makes a stranger click Add to Cart. Manage Your Experiments swaps the guess for a vote. Shoppers who land on your detail page are randomly split, half see your current content and half see the new version, and Amazon counts who buys.
Amazon's tool page says optimized content can help increase sales by up to 20%, and it tells the story of a brand that changed one word and saw an 8% increase in sales. Your results will be your own. What follows is how to get access, what to test first, how long to wait and how to read the scorecard without fooling yourself.
Part 01
Step by step
8 steps, in the order Seller Central asks for them.
- 01Confirm you can use the tool
You need a Professional selling account and a Brand Representative role for a brand enrolled in Amazon Brand Registry. If you work for an enrolled brand, ask the Rights Owner or another Administrator to assign you the Brand Representative selling role.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
- 02Open Manage Your Experiments
In Seller Central, hover over Brands in the main menu and select Manage Experiments.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 03Pick the experiment type and an eligible ASIN
Click Create a New Experiment and choose the type: product images, title, bullet points, description, or A+ Content including Brand Story. Amazon then lists your eligible ASINs, meaning products from your enrolled brand with enough traffic in recent weeks to produce valid results. A+ Content and Brand Story tests need that content already published.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 04Write a hypothesis and name the test
Add the experiment details, including a name and a hypothesis. One sentence is plenty, for example that a lifestyle main image will beat a plain studio shot. It keeps you honest when the results come in.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
- 05Build a Version B that is clearly different
Version A is your current published content and Version B is the new one. Type new titles, bullets and descriptions straight into the tool, upload new images, or for A+ Content choose Start by duplicating Version A to edit a copy in the A+ Content manager. Amazon will tell you if the two versions are too similar.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 06Keep the default settings and schedule it
By default the test starts as soon as validation is complete, runs to significance, which means until there's enough data to declare a winner, and auto-publishes the winning content. You can change these under the View Settings toggle. Then click Schedule Experiment.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 07Leave it alone until it ends
Results update once a week. A to significance test can finish in as little as four weeks. If you set your own duration, Amazon recommends 8 to 10 weeks. Early leads can be misleading, so let it run to the end.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 08Read the results and publish
Click View Details on the Manage Your Experiments page. You'll see the probability that one version is better, plus units sold, sales, conversion rate, units sold per unique visitor, sample size and the projected one-year impact of the winner. Publish it, or let auto-publish do it if Version B performed significantly better.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Part 02 Compare
Compare the options: What should you test first?
Tap a content type to see what Amazon suggests comparing, and how you build Version B for it.
Main image
- Try comparing
- Lifestyle against product photography
- Or
- Minimalist photos against information-rich ones
- Build Version B
- Upload the image, then pick it in the file picker
- Good for
- Getting noticed in search results
Title
- Try comparing
- A longer title with features against a shorter one
- Or
- With and without your brand name
- Build Version B
- Type it straight into the tool
Bullet points
- Try comparing
- Features against benefits, size or other details
- Or
- Different order, length and keywords
- Build Version B
- Type them straight into the tool
- Good for
- Shoppers who scan before they buy
Description
- Try comparing
- With and without details like product dimensions
- Build Version B
- Type it straight into the tool
- Good for
- Products where specs decide the sale
A+ Content
- Needs
- A+ Content or Brand Story already published
- Try comparing
- Different visuals, videos or comparison charts
- Build Version B
- Start by duplicating Version A
- Edited in
- The A+ Content manager
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Part 03
The picture
The same facts, drawn. Each chart links to the Amazon page it comes from.
The path from Amazon's Manage Your Experiments guide on the Sell on Amazon blog.
Source: Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Data desk by XENA IntelligenceFour weeks is the earliest Amazon says a to significance test can sometimes finish. 8 to 10 weeks is its recommendation when you pick the duration yourself.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
Data desk by XENA Intelligence- Professional selling plan
- Brand Representative role in Brand Registry
- An ASIN from your enrolled brand
- Enough traffic in recent weeks
- A+ Content or Brand Story published, for those test types
- A Version B that is clearly different
Access and eligibility rules from Amazon's Manage Your Experiments page.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
Data desk by XENA IntelligencePasses
- Make Version B significantly different
- Use the pre-selected settings
- Run the test to completion
- Test title, images and bullets together when you want a big swing
Bounces
- Change one comma and expect a verdict
- Stop early because week one looks great
- Test an ASIN with little traffic
- Forget what you were trying to learn
Drawn from Amazon's tips for an effective experiment.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
Data desk by XENA IntelligencePart 04
Tricks the help page buries
The lines in Amazon's own documentation that most sellers scroll past.
- 1Big differences, clear answers
Amazon's own advice is to make Version B significantly different from the current content. The greater the difference, the better the chance the result is meaningful and not random. Save the comma debates for later.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 2Week one is a rumor
Results update once a week, and Amazon warns that promising early results can be misleading. Put a reminder in your calendar for week four and look then.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
- 3Test several things at once when you need a big move
Multi-attribute experiments let you test the title, images and bullet points in one experiment. Use them when the listing needs an overhaul, and single attribute tests when you want to know which change did the work.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
- 4Ask the tool for ideas
For title and A+ Content experiments, Manage Your Experiments offers machine-learning suggestions built from thousands of previous experiments on similar products. Treat them as a starting point, then add what you know about your customer.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
- 5It's free, the plan isn't
The tool costs nothing extra. The Professional selling plan it requires is $39.99 a month plus selling fees, so if you already pay for it, an untested listing is money left on the counter.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Part 05
Questions sellers ask
Tap a question to open it.
Can you A/B test listings on Amazon?
Yes. Manage Your Experiments lets brand owners test two versions of product images, titles, bullet points, descriptions and A+ Content, including Brand Story. Shoppers are randomly split between Version A and Version B, and Amazon reports which one sold more.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Who can use Manage Your Experiments?
Sellers with a Professional selling account who are a Brand Representative for a brand enrolled in Amazon Brand Registry. The ASIN must belong to that brand and have enough traffic in recent weeks to produce valid results.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
How long does an Amazon A/B test take?
With the default to significance setting, the test runs until there's enough data to declare a winner, which can be as soon as four weeks. If you set your own duration, Amazon recommends 8 to 10 weeks.
Source: Sell on Amazon, Test listing content with Manage Your Experiments
Does Manage Your Experiments cost money?
No. Amazon describes it as a free tool. You do need the Professional selling plan, which is $39.99 a month plus selling fees.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
What metrics does Manage Your Experiments show?
The probability that one version is better, units sold, sales, conversion rate, units sold per unique visitor, sample size and the projected one-year impact of the winning content on your sales.
Sources: Sell on Amazon, Test listing content with Manage Your Experiments, Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Does Amazon publish the winning version automatically?
It can. Auto-publish is on by default, and Amazon publishes Version B automatically if it performed significantly better than your existing content. You can switch it off under View Settings and publish the winner yourself.
Source: Sell on Amazon blog, Optimize product content and increase sales with A/B testing
Sources
Every fact above comes from these Amazon pages, opened on . Amazon updates them; if a step looks different in your account, trust Seller Central.