You’ve probably heard the warning: “A/B testing tools kill your page speed.” Like most warnings, it’s rooted in something real, plenty of testing tools genuinely do slow stores down, and some do it in ways that are worse than a slow load. But it’s not a law of physics. It’s a consequence of how a tool is built, and you can check for it yourself in about ten minutes.

This article explains where the speed cost actually comes from, why the standard “fix” (anti-flicker snippets) often makes things worse, what a well-built implementation does differently, and exactly how to audit any testing app before, or after, you install it.

The short answer

Yes, badly implemented A/B testing slows your store down, sometimes dramatically. Well-implemented A/B testing has a cost so small most merchants can’t detect it in real-world metrics. The difference comes down to three design decisions:

  1. Where the variant swap happens (in the browser after the page loads, or before the page renders)
  2. What gets hidden while the tool decides (the whole store, or just the element being tested)
  3. What happens when things go wrong (a frozen white screen, or a failsafe that shows the page anyway)

Let’s take these in order.

Why testing tools cause flicker in the first place

Most general-purpose A/B testing tools work by client-side modification. Your Shopify theme sends the original page to the visitor’s browser, and then a JavaScript snippet rewrites parts of it: swapping a headline, hiding a section, changing a button colour.

The problem is timing. The browser starts painting the page as soon as it has HTML to paint. The testing script, meanwhile, has to load, run, decide which variant this visitor belongs to, and apply the changes. If the page paints before the script finishes, the visitor sees the original content for a moment before it snaps to the variant.

That visible snap is called flicker, or more formally FOOC, a flash of original content. On a fast connection it might last 100 milliseconds. On a mid-range phone over 4G it can last a second or more. Either way, the visitor sees your page rearrange itself in front of them, which reads as “broken”, and visitors in the variant group get a measurably different (worse) experience than the control group, which quietly contaminates your test results.

Why flicker is worse than it sounds

Flicker isn’t just cosmetic:

  • It biases the test. If variant B loads with a visible jolt and variant A doesn’t, you’re no longer testing your headline, you’re testing “headline plus jank.”
  • It hurts Core Web Vitals. Content shifting after first paint contributes to Cumulative Layout Shift (CLS), one of Google’s page experience metrics.
  • It compounds on mobile, where CPUs and connections are slower and where most Shopify traffic lives. If you care about mobile product page performance, flicker should be near the top of your worry list.

The industry’s “fix”: anti-flicker snippets, and their dark side

Since flicker looks bad, most client-side tools ship an anti-flicker snippet: a small piece of code placed high in the page that hides content (usually by setting the whole <body> to invisible) until the testing script has loaded and applied its changes. No flicker, because the visitor sees nothing at all.

Read that again. The standard fix for a flash of wrong content is a deliberate white screen.

Anti-flicker snippets in their common form have three problems:

1. They hide the entire store

The typical snippet applies opacity: 0 or visibility: hidden to the whole page. That means your homepage, collection pages, blog, cart, pages with no active test on them at all, can be blanked while the testing library loads, on every visit. You pay the speed tax storewide to run a test on one page.

2. They wait for a heavyweight script

The page stays hidden until the testing library has downloaded, parsed, executed, and made a decision. Some testing libraries are hundreds of kilobytes of JavaScript. If that script comes from a slow third-party CDN, or the visitor is on a weak connection, the white screen stretches accordingly.

3. Their timeouts are long, or missing

Most snippets include a timeout: “if the script hasn’t decided within X seconds, show the page anyway.” Common defaults are in the 2–4 second range, which is an eternity, many visitors will have hit the back button by then. And if the snippet is misconfigured or the timeout logic itself fails, the page can stay hidden indefinitely.

The net effect: a tool installed to increase conversions can add seconds of blank screen to every page view, storewide, before your first test has even collected a data point.

What a good implementation looks like

None of the above is inevitable. There’s a checklist any well-designed testing tool should satisfy, and you should hold every app you evaluate to it:

Decide the variant server-side or at the routing level where possible. The less rewriting the browser has to do, the less there is to hide. On Shopify, the cleanest mechanism is the platform’s native alternate templates: the variant is a real, complete page rendered by Shopify itself, not a page mutated by JavaScript after the fact. (We’ve written a full explainer on how template-based testing works.)

Scope any hiding to tested pages only. If the test runs on one product page, no other page should ever be touched. Your homepage should not know the testing tool exists.

Hide the minimum, for the minimum time. Reveal the page the moment the variant decision is made, not when the full library, analytics, and everything else has finished loading. The decision itself (a cookie lookup, essentially) can be nearly instant.

Have a hard failsafe timeout. If anything goes wrong, script blocked, network hiccup, bug, the page must show after a short, non-negotiable cutoff. A blank storefront is never an acceptable failure mode.

Load asynchronously and stay small. The tracking script should be lightweight, load async or defer, and never block HTML parsing.

How Atchoo handles this, concretely

This checklist isn’t hypothetical, it’s how Atchoo! is built, so it’s worth being specific:

  • Atchoo tests product page templates using Shopify’s native view mechanism. The variant is a complete alternate template rendered by Shopify, not a JavaScript rewrite. Each visitor is assigned deterministically via a first-party cookie, so returning visitors go straight to their version.
  • Anti-flicker is scoped exclusively to product pages with an active test. Your homepage, collections, cart, and every product without a running test are never hidden, ever.
  • On a tested page, content is revealed at decision time, the moment the visitor’s assignment is known, not after a full page load.
  • There is a hard failsafe timeout. If the decision can’t be made quickly for any reason, the page shows anyway. The worst-case failure mode is “visitor sees the control version,” not “visitor sees a white screen.”

That’s the honest trade-off: on a page with an active test, there is a brief, bounded hold before reveal. On every other page of your store, the impact is essentially nil.

How to audit any testing app yourself

Don’t take any vendor’s word for it, including ours. Here’s a practical audit you can run in ten minutes:

1. Baseline with PageSpeed Insights

Before installing anything, run your key product page and homepage through PageSpeed Insights. Note the mobile Performance score, Largest Contentful Paint (LCP), and CLS. Screenshots help. After installing and launching a test, run the same pages again and compare. Field data (the Chrome UX Report section) takes weeks to update, so rely on the lab numbers for a quick before/after.

2. Watch it load in DevTools

Open Chrome DevTools (right-click → Inspect):

  • Network tab: reload a tested page and look for the app’s script. Check its size, whether it blocks other requests, and where it loads from. Then reload your homepage, a well-scoped tool should be doing little or nothing there.
  • Performance tab with throttling: set CPU throttling to 4x and network to “Fast 4G,” then record a page load. This approximates a mid-range phone. Watch the filmstrip: do you see a blank frame? For how long? Does content visibly jump after first paint?
  • The blocking test: in DevTools, block the testing app’s script URL (Network tab → right-click the request → Block request URL) and reload. If the page renders fine, the tool fails safely. If you get a blank page, that’s the failure mode your visitors will hit whenever the script is slow or blocked.

3. Read the snippet

If the app asks you to paste a snippet into theme.liquid, actually read it. Does it hide body unconditionally? What’s the timeout value? Is there one at all? You don’t need to be a developer to spot visibility:hidden applied to the whole document with a 4-second timeout.

4. Test the real experience

Load a tested product page on an actual phone, on mobile data, in an incognito window. Trust your eyes over any score.

Keep the trade-off in perspective

A final honest note: the goal isn’t zero cost, it’s a cost small enough that the value of testing dwarfs it. A test that finds a genuinely better product page layout is worth a few milliseconds of decision time on that page. A tool that blanks your whole store for two seconds per view is not. The audit above tells you which kind you have.

If you want the broader picture of running tests properly, sample sizes, duration, metrics, start with our complete guide to Shopify A/B testing. And if you want to see a scoped, failsafe implementation on your own store, Atchoo’s Pro plan has a 14-day free trial, install it, run the DevTools audit on it, and judge for yourself.

Frequently asked questions

Does A/B testing hurt Core Web Vitals?

It can. Client-side tools that rewrite pages after first paint hurt CLS, and storewide anti-flicker snippets hurt LCP by delaying when meaningful content appears. Template-based approaches with scoped, short-lived hiding have a much smaller footprint. Measure your own store with PageSpeed Insights before and after.

What is FOOC (flash of original content)?

FOOC is the visible moment where a visitor sees the original page before a testing script swaps in the variant. It happens when the browser paints faster than the testing JavaScript can run, common with client-side tools, especially on mobile.

Are anti-flicker snippets always bad?

No, brief, scoped hiding is a reasonable tool. The problems are scope (hiding the entire store instead of just tested pages), duration (waiting for a full script load instead of just the variant decision), and missing failsafes. A snippet that hides one tested page for a bounded moment with a hard timeout is a very different thing from one that blanks every page for seconds.

Will removing my A/B testing app speed up my store?

If the app uses a storewide anti-flicker snippet and a heavy client-side library, quite possibly yes, rerun PageSpeed Insights after removal and check. Also check that the app cleaned up after itself: orphaned snippets left in theme.liquid are common. If the app is scoped and lightweight, the difference will likely be within normal measurement noise.