Setting up your first product page A/B test on Shopify takes about an hour, and most of that is building the variant, the plumbing is surprisingly straightforward once you know where the pieces live. This tutorial walks through the whole thing: duplicating a template in the theme editor, making a change worth testing, QA-ing it so the test doesn’t silently break, and knowing when to call a winner. If you want the strategic background first, metrics, statistics, roadmap, start with the complete guide to A/B testing on Shopify and come back.

What you’ll need

  • A Shopify store on an Online Store 2.0 theme (any recent theme, Dawn and its descendants, or a current paid theme).
  • Access to the theme editor (Online Store → Themes → Customize).
  • An A/B testing app that supports template-based testing, to split traffic and track results, Shopify has no built-in split-testing.
  • One product page with decent traffic. Your best seller is usually the right choice: more traffic means a faster, more trustworthy result.

No code required for any of this.

Step 1: Duplicate your product template in the theme editor

Shopify themes can hold multiple templates for the same page type, distinguished by a suffix. That’s the foundation of the whole technique, the mechanics are explained fully in Shopify template testing explained, but here’s the doing part:

  1. Go to Online Store → Themes and click Customize on your live theme.
  2. In the template selector at the top of the editor, choose Products → Default product (or whichever template your test product currently uses).
  3. Open the same dropdown again and click Create template.
  4. Name it something you’ll recognise in six months. The name becomes the template suffix, so variant-b creates product.variant-b. Better still, name it after the change: sticky-atc, lifestyle-first, short-copy.
  5. Choose to base it on your existing product template, and create.

You now have an exact copy of your product template that you can edit freely without touching the version your customers currently see. Two things worth knowing before you edit:

  • Don’t assign the new template to the product in Shopify admin. In a normal (non-test) workflow you’d open the product and change its template in the “Theme template” dropdown, but that would send everyone to the variant. For a test, the product stays on its default template, and your testing app routes the variant half of traffic to the alternate one.
  • Sections vs blocks: in the JSON template you’re choosing which sections appear and in what order, and configuring their blocks. Changes to a section’s settings in one template don’t affect other templates, but edits to the underlying section code (if a developer changes the Liquid) affect every template using it. The distinction is unpacked in sections vs templates.

Step 2: Make a change worth testing

The most common first-test mistake isn’t technical, it’s testing something too timid to detect. A slightly different shade of button on a store with 5,000 monthly product page sessions will take months to reach a verdict, if it ever does. Your first variant should be a change a customer would actually notice.

Good first-test candidates, roughly in order of typical impact:

  • Restructure the mobile-first layout. Move the buy button and price above the fold on mobile; add a sticky add-to-cart bar; cut the number of gallery images before the description.
  • Change the image strategy. Lead with a lifestyle photo instead of a studio cutout, or vice versa. Reorder the gallery so the most persuasive shot is first.
  • Rewrite the description structure. Swap a wall of prose for scannable benefit-led bullets with a short expandable “details” section, or the reverse if your product needs a story.
  • Move or add trust elements. Put review stars directly under the title, surface shipping/returns messaging next to the button instead of in a footer tab.
  • Cut clutter. Remove low-value sections (size-chart banners nobody clicks, redundant feature grids) so the path to the button shortens.

Pick one of these. If you change five things and the variant wins, you won’t know which change did the work. Write the hypothesis down in one sentence before you build: “Because most of our traffic is mobile and the buy button is three screens down, a sticky add-to-cart bar will raise add-to-cart rate.” If nothing here fits your store, the product page test ideas library has plenty more.

Make the edits on your new template in the theme editor, and save.

Step 3: Set up the test in your testing app

The exact clicks vary by app, but a template-based test always comes down to the same four decisions:

  1. Which product(s) are in the test. Start with one high-traffic product. (Pooling several products onto the same pair of templates is a legitimate way to speed things up, a good move for test two.)
  2. Which template is the variant. Point the test at the suffix you created (product.sticky-atc). Visitors assigned to B get the alternate template via Shopify’s native ?view= mechanism; visitors assigned to A see the page exactly as before.
  3. The traffic split. Use 50/50 unless you have a specific reason not to, equal groups reach a verdict fastest.
  4. The primary metric. For product page tests, that’s add-to-cart rate. It’s the outcome the page directly controls and it accrues data several times faster than purchases; the reasoning is laid out in add-to-cart vs conversion rate.

In Atchoo! this is the whole setup: pick the product, pick the variant template, confirm the split, and the app handles deterministic assignment (a first-party cookie so each visitor always sees the same version) and real-time view/add-to-cart tracking. Whatever app you use, confirm it assigns visitors deterministically, a shopper who sees a different page on every visit produces garbage data.

Before launching, decide how long the test will run. Use a sample size calculator with your baseline add-to-cart rate and the smallest lift you’d care about, and commit to that number in advance. The why behind this is covered in sample size and duration.

Step 4: QA checklist, do not skip this

A broken variant doesn’t just lose the test; it quietly feeds you false data for weeks. Fifteen minutes of QA prevents that.

Preview the variant directly. Append the view parameter to your product URL:

https://yourstore.com/products/your-product?view=sticky-atc

Then work through:

  • Use your app’s test/preview mode to view the page as an assigned variant visitor, this confirms the assignment and template-switching actually work end to end, not just the raw template.
  • Mobile, on a real phone. Not just the desktop browser shrunk down. Most of your traffic is mobile; check the layout, tap targets, and that nothing important is pushed off-screen. (If mobile is where your test lives, mobile product page optimization covers the common traps.)
  • Add to cart works on the variant, including selecting a size/colour variant first, quantity changes, and whatever your theme does after adding (drawer, redirect, notification).
  • Product options render correctly. Variant pickers, swatches, and price updates when options change.
  • Tracking fires. Confirm in your testing app that a product view and a test add-to-cart from the variant template are both recorded.
  • No flicker. Load the page as a variant visitor several times (including on a throttled mobile connection). You should never see version A flash before B appears, and any anti-flicker treatment should only ever touch the tested product page, never the rest of the store.
  • Third-party apps still work. Review widgets, upsell blocks, and bundles apps sometimes need to be re-added or re-configured on a new template.

If anything fails, fix it before launch, never “launch now, patch tomorrow,” because the patched test is no longer one experiment.

Step 5: Launch, then leave it alone

Turn the test on and resist the urge to fiddle. Three rules for the running period:

  1. Don’t edit either template mid-test. Any change, even a typo fix, ideally, means the “before” and “after” data describe different pages. If you must fix something broken, note the date and mentally discount the earlier data.
  2. Check daily for breakage, not for winners. Glance at the numbers to confirm both arms are receiving roughly equal traffic and events are flowing. Do not stop the test the first morning it shows a significant-looking lead, early leads are usually noise, and stopping on them is the single most common way merchants fool themselves. The full explanation of why peeking-and-stopping inflates false positives is in the complete guide.
  3. Run whole weeks. Weekend shoppers behave differently from Tuesday-lunchtime shoppers. Two full weeks is the practical minimum; stop on the same weekday you started.

Step 6: When to call it

Call the test when both of these are true:

  • You’ve reached the sample size you committed to before launch (and at least two full weeks have passed), and
  • Your analysis gives a clear verdict, in Bayesian terms, one version has a comfortably high probability of being better (95%+ is a sensible bar); in frequentist terms, the result is statistically significant at your planned sample size.

Then act on the outcome:

  • Variant wins: roll it out by assigning the winning template to the product in Shopify admin (or making its changes the new default template), archive the test, and record the result. Then test the next idea on top of the winner.
  • Control wins: just as valuable, you avoided shipping a worse page to 100% of visitors. Record what you learned about what your customers don’t respond to.
  • Inconclusive at full sample size: the change probably doesn’t matter much either way. Keep whichever version you prefer operationally, and spend your next test on a different lever.

Segment the results by device and traffic source for hypotheses (maybe the sticky bar helped mobile but did nothing on desktop), but don’t declare segment-level winners from a test that wasn’t sized for it. There’s a full walkthrough of reading results honestly in how to analyze A/B test results, and you can double-check any result with the significance calculator.

Write down the hypothesis, both screenshots, the dates, the numbers, and the decision. Test two starts from a better place because test one is documented.

Frequently asked questions

Do I need a developer to do this?

No. Creating and editing alternate templates happens entirely in the standard theme editor, and a template-based testing app handles the traffic splitting. You’d only need a developer if your variant requires custom section code the theme doesn’t already offer.

Will visitors notice they’re in a test?

No. Each visitor sees one consistent version across all their visits (deterministic assignment via a first-party cookie). There’s no banner, no flash between versions if flicker is handled properly, and the URL customers see and share is the normal product URL.

Can I test more than one product at a time?

Yes, and for lower-traffic stores it’s often the right move: apply the same template change across several products and pool the data. What you learn is “this layout change works for our catalogue,” which is usually the question you actually care about.

How long will my first test take?

It depends on your traffic and how bold the change is. As a rough shape: a high-traffic product with a bold change might conclude in two to three weeks; a modest change on a mid-traffic page can take six or more. Run your numbers through the sample size calculator before you launch so the answer isn’t a surprise.

What if I edit my theme while a test is running?

Theme-wide changes (like a developer editing a section’s Liquid) affect both templates equally, which keeps the comparison fair but changes what you’re measuring mid-stream, avoid it if you can. Editing one of the two templates mid-test is the real sin: it invalidates the comparison. Save the redesign energy for the next test.