Shopify checkout A/B testing: every option in 2026, and where each one stops Shopify checkout A/B testing: every option in 2026, and where each one stops

The verdict

Anyone still telling you the Shopify checkout can't be A/B tested is working from a page written before 5 June 2026. It can. The real questions are how deep the test goes, what it reports back, and whether you can still measure it after 26 August 2026.

The four levels of checkout testing

1. 1 Everything upstream Shoplift · Intelligems · ABConvert · Visually · Mtrix
2. 2 Content inside Shopify's checkout Intelligems · ABConvert · Visually
3. 3 One whole checkout configuration against another Shopify Rollouts
4. 4 A different checkout entirely, tested against yours Mtrix

The four levels of Shopify checkout testing

Level	What you're actually changing	Who reaches it	Shopify plan
1. Everything upstream	PDP, cart, add-to-cart — what decides whether a shopper reaches checkout at all	Shoplift, Intelligems, ABConvert, Visually, Mtrix	Any
2. Content inside Shopify's checkout	Trust badges, callouts, banners, timers, upsell blocks, payment icons	Intelligems, ABConvert, Visually	Plus only
3. One whole checkout configuration against another	Swap configuration A for configuration B and split traffic	Shopify Rollouts	Grow or higher
4. A different checkout entirely, tested against yours	The checkout itself — steps, structure, flow	Mtrix, on a store built on Mtrix	—

Level 2 has a hard ceiling, and Intelligems is the most explicit about it. What its checkout tests cannot do, verbatim from docs.intelligems.io, read 26 July 2026:

"One page vs three page checkout" · "Reordering the steps in checkout" · "Changing the structure of the Shopify checkout flow"

Every tool, side by side

Shopify checkout testing tools compared

Tool	What it tests in checkout	Checkout plan gate the pivot	Published price	Rating	Best for
Shopify Rollouts	One whole checkout & accounts configuration against another	Rollouts: Basic+. Experiments: Grow+	Included with Shopify	—	Brands testing the whole checkout occasionally rather than continuously
Intelligems	Content blocks, prices, compare-at prices, shipping rates	Blocks: Plus + Unlimited or the Checkout & Post-Purchase module. Price tests: any plan. Shipping: Advanced or Plus	From $69/mo; Unlimited card $349/mo; own calculator at 10,000 orders/mo $2,314 list, $1,279 discounted, 3-month minimum	4.8 (154)	Price and margin tests read as profit per visitor
ABConvert	Content blocks: messaging, trust badges, images, payment icons, banners	Plus; included from the entry plan	$99 / $199 / $399 / $599 per mo, metered on test orders, +$59 / $39 / $29 / $24 per extra 1,000	4.7 (72)	The cheapest way onto checkout content if you're already on Plus
Visually.io	Content blocks — information page only on multi-step checkout	Plus	$15 / $25 / $80 to 600 orders/mo; calculator at 7,500–10,000 orders $720 / $1,200 / $3,600	4.7 (120)	Brands with no CRO headcount who want tests executed for them
Shoplift	Nothing in checkout. Themes, templates, PDP, cart, price — the funnel that feeds it	—	App Store $99 / $399 / $999 per mo; the same three plans from $74 / $299 / $699 on shoplift.ai, annual, banded by visitors	4.9 (117)	Theme and template tests upstream of checkout
Shopify Plus (the platform)	"Full" checkout customization, against "Limited" on Basic, Grow and Advanced	—	$2,300/mo on a 3-year term, $2,500/mo on 1 year	—	The tollgate every checkout app sits behind
Mtrix	A native Mtrix checkout or your existing one — and an A/B test between them	—	Custom — talk to us	Not a Shopify app	Brands willing to move the store to test the checkout itself

Every figure was read from the vendor's own pricing page or App Store listing on 26 July 2026, with the billing basis named — these ladders meter on orders or visitors, so a bare number means nothing.

How we ranked: depth of change, what the tool reports back, true cost including the Shopify plan, and what survives 26 August 2026. Not review count — that measures install base, not fit.

Before you test anything: 26 August 2026

A checkout test is only as good as the conversion data around it, and the pages carrying most of that data change on 26 August 2026 for every store that isn't on Plus — 31 days from this review. Shopify's help centre, read 26 July 2026:

"If you're on a non-Plus plan, then you have until August 26, 2026 to upgrade and replace your Thank you and Order status pages."

Miss it and they're upgraded for you: "your Thank you and Order status pages will be auto-upgraded to the new pages." App ScriptTags on the Order status page are turned off the same day for non-Plus merchants, and anything pasted into Additional Scripts — pixels, GTM containers, affiliate tags — stops reporting with them. Two things most pages on this topic get wrong: checkout.liquid is not part of this date (always Plus-only, deadline 28 August 2025), and Additional Scripts have been read-only on every plan since 28 August 2025 — "the additional scripts section in the Checkout settings is view-only" — so the box you were planning to edit has been frozen for eleven months.

Do this before you evaluate a single tool. Open Settings → Checkout → Configurations. If you see Upgrade Thank you and Order status pages by August 26, 2026, you're on the clock, and Shopify has already generated a personalised report of the customizations it found on those two pages. Credit where it's owed — Shopify's migration tooling here is good, and sandboxed, consent-aware pixels instead of arbitrary JavaScript in a payment flow is a defensible trade.

This page covers only the part that breaks your test. The upgrade itself — the steps, what replaces what, what happens to your GTM container — is in the 26 August 2026 checkout extensibility deadline guide.

Levels 1 and 2 — what a Shopify app can reach

Shoplift — the funnel that feeds the checkout

$99 / $399 / $999 per month on their App Store listing · banded by monthly visitors · 4.9 (117 reviews)

Shoplift is the flat "no" in this set, and it says so in its own docs, read 26 July 2026:

"Shoplift does not currently support testing Shopify's checkout pages directly. Shopify restricts third-party apps from modifying the checkout flow. However, you can test elements that influence checkout behavior — like product page layouts, pricing, add-to-cart buttons, and cart pages."

Their listing carries a Works with: Checkout badge. That means order tracking — "Shoplift natively supports all major one-click checkout solutions" — not checkout testing.

Choose Shoplift if the number that's broken is your reached-checkout rate — which, for most brands, it is. See the abandonment breakdown further down before you assume the leak is in checkout.

Two limits to price in. The Core card caps at "Up to 50k monthly store visitors," so a growing brand moves to Advanced at $399/mo; cross your band mid-test and Shoplift pauses your tests "until you upgrade your plan, or your current plan rolls over to the next billing cycle and the usage resets." It clears itself — but a test paused mid-flight is still a test you can't read.

And their anti-flicker works by blanking the page. Their pricing-page FAQ, in full: "our anti-flicker script momentarily blanks out your pages to ensure shoppers don't see new experiences, but this is typically resolved within 150 milliseconds (and as low as 90 milliseconds)." Credit them for publishing it in milliseconds — almost nobody does. It is still, by design, blank screen time, which is the thing server-side bucketing removes rather than shortens.

Intelligems, ABConvert and Visually — content blocks, and all three need Plus

In their own words, and note that they blame the platform, not each other:

• Intelligems: "Checkout Personalizations and Checkout Testing is only available for Shopify Plus merchants."
• ABConvert: "Checkout UI testing is available on Shopify Plus plans only."
• Visually: "Only Shopify Plus accounts have the ability to edit the checkout section due to limitations with Shopify."

Shopify's own availability table agrees: apps built with UI extensions for the information, shipping and payment pages are marked ✘ for every plan except Plus.

Intelligems is the deepest of the three. Content blocks plus real price and shipping-rate tests, read against your own COGS as profit per visitor. Nothing else on this page reports margin.

Scope its Plus gate correctly, because it doesn't cover all of that. Checkout blocks need Shopify Plus and Intelligems' Unlimited plan or its Checkout & Post-Purchase module. Price tests need no Plus at all — they run on Shopify Cart Transform Functions, on any plan. Shipping tests need Advanced or Plus, because that's where Shopify turns on carrier-calculated rates. A Grow-plan brand can buy Intelligems and test price this week; the same brand cannot test a trust badge inside checkout. The App Store's "$69/mo" is the floor of the cheapest module at the lowest order band — their own calculator at 10,000 orders/mo prices the Unlimited bundle at $2,314 list, $1,279 discounted, on a 3-month minimum. Two more limits worth knowing: on Shopify's one-page checkout "running content tests on the One-page checkout isn't possible" (price and shipping tests still run), and unlike Rollouts, "checkout modifications work on Headless stores!"

ABConvert is the cheapest route in. Checkout UI testing sits in the $99/mo entry plan, no vendor upgrade needed, only Shopify Plus. Their stated limits: "one block in a test group," no full-checkout-page preview before launch, and Shopify's "three-block maximum per row." Their metering is the fairest design here — you pay for test orders, not for visitors who never saw the test. Six thousand test orders on the $199 Growth plan is $238/mo; 20,000 on Scale is $544/mo. The bill rises with the success of the test. Price testing, though, starts at Growth, not at the $99 entry plan.

Visually.io reaches one page of the checkout. Their docs: "For multi-step checkout — currently we support adding blocks and customizing only the information page." Everything else is roadmap. The App Store headline says "Free plan available"; that plan is development stores only, and the paid ladder stops at 600 orders/mo. Their calculator at 7,500–10,000 orders returns $720, $1,200 or $3,600 per month, and the upper tiers buy services as much as software — executed tests, engineering hours, storefront audits. Mtrix leaves the programme to your team. For a brand with no CRO headcount, that's a legitimate reason to pick Visually over anything else here.

Level 3 — Shopify Rollouts, and exactly where it stops

Included with Shopify · rollouts on Basic or higher · experiments on Grow or higher

On 5 June 2026 Shopify's changelog extended Rollouts to checkout: "Run an A/B test between two entirely different themes or checkout setups to find the winner." Native, free, no third-party script on your storefront. For a brand running an occasional whole-checkout test that is straightforwardly a good deal, and this page would be dishonest not to say so.

The boundary, from Shopify's own Rollouts help pages, read 26 July 2026:

• A checkout experiment reports one metric. The "Checkout and accounts experiment" table has a single row: checkout conversion rate — "online store sessions with completed purchases relative to the visits that reached checkout." No revenue, no AOV, no orders, no revenue per visitor. The launch version of the same rollout reports gross sales, AOV, orders and total sales; the experiment doesn't. And "you can't customize the available metrics for your rollout."
• No statistics on the page. Significance, confidence, probability and sample size appear nowhere on Shopify's four Rollouts help pages. The framing, in the changelog announcing the feature, is "find the winner" — you look at the number and decide.
• Whole configurations only. Three change types, one of each per rollout. No element-level variation and no behavioural audience — targeting is market-level, and "market-specific targeting is available for merchants on Advanced and Plus plans" per Shopify's 31 March 2026 changelog.
• "Rollouts support only online store checkouts. Headless and custom storefront checkouts aren't supported and can't be tested with rollouts."

Two things Rollouts gets right that cost money or nerve elsewhere. Multiple live experiments on the same resource run in mutually exclusive traffic segments, free: "Each experiment gets its own mutually exclusive segment of visitor traffic." And the safe default is the right one — when a rollout ends, "by default, the rollout reverts. No changes are applied permanently to the affected resource." You opt into permanence; you don't have to remember to opt out of it. Plenty of paid tools get that backwards.

Shopify can swap your whole checkout for another one and tell you whether the conversion rate moved. For a lot of brands that's enough. When the decision is worth more than one unqualified percentage, it isn't.

Level 4 — a different checkout, tested against the one you have

Mtrix is not a Shopify app — no App Store listing, no app pixel, no script tag in your checkout. Two ways to run it, and the difference decides what you can test:

1. On the store you already have — an npm package or CDN script on your Shopify storefront. You get experiments, session replay, analytics, error tracking and real-user performance monitoring on your storefront pages. You do not get the Mtrix checkout, the visual builder, or price testing; Shopify's hosted checkout stays Shopify's.
2. On Mtrix — the store is built and hosted on Mtrix, and the checkout becomes something you can test.

On the second path both checkouts are supported: a native Mtrix checkout, or the one you already run — and you can A/B test one against the other. Payment providers on the native checkout: Stripe, Braintree/PayPal, Shopify Payments, iyzico, Mollie and Klarna. Across Mtrix's own clients' checkout tests, the Mtrix checkout converted 14% better than Shopify's checkout.*

That's a migration rather than an install, so price it as one: Mtrix imports from your GitHub repo, scrapes the live site if there's no repo, or builds from Figma, then finishes the setup with you. The most complex stores are fully live within a week.

On the test itself. Four test types — A/B, multivariate, split URL and mask URL — plus a multi-armed bandit that shifts traffic toward the winner mid-flight. Assignment is made in Mtrix's backend and stored against your visitor ID on a 30-day server-side lookback, not in a browser cookie. Metrics are revenue per visitor, conversion rate and add-to-cart rate, reported as probability-to-be-best. Auto-complete on a visitor count, a date or a revenue threshold, and run overlapping tests mutually exclusive. No cap on experiments — not concurrent, not monthly, not by plan.

What survives 26 August, on either path. Purchase attribution on a Shopify store runs on a server-side Shopify Admin webhook — orders/paid, HMAC-verified — not a checkout script and not a ScriptTag, so nothing in this page's timeline switches it off. One piece of wiring — a _mtrixSession note attribute on the cart — and orders, line items and refunds land in Purchases with no pixel in the path.

On cost: flat across three tiers, not metered on sessions, orders or profiles. No per-1,000 overage, no visitor band that pauses your tests, no Mtrix transaction fee. One month free, no credit card, monthly or annual.

The honest limit. On a store that stays on Shopify, Mtrix's script runs on your storefront, and Shopify's hosted checkout is not a surface a third-party storefront script reaches. Replay and analytics cover the journey up to the checkout handoff, not inside it.

What checkout testing actually costs on Shopify

Every figure below was read from the vendor's own page on 26 July 2026, all monthly.

What checkout testing costs on Shopify, per month

Route	Shopify plan	Testing tool	Monthly total
Cheapest that touches checkout content	Plus, $2,300 (3-year term)	ABConvert Starter $99	$2,399
Mid-market	Plus, $2,500 (1-year term)	Intelligems Unlimited $1,279 (44%-discounted, 3-month minimum, at 10,000 orders/mo)	$3,779
Configuration testing only	Grow or higher — what you already pay	Shopify Rollouts, included	No new spend

The app is the small number: Shopify is 96% of the bill on the cheap route. Plus also charges 0.20% per transaction if you process through anyone other than Shopify Payments. What that buys, in Shopify's own one-word summary on its pricing page, is "Full" checkout customization against "Limited" on Basic, Grow and Advanced — all four cells re-read on shopify.com/pricing, 26 July 2026, with the toggle on annual.

Mtrix isn't in that table on purpose: it's a platform subscription, not a Shopify plan plus an app, and a qualitative tier among real dollars would mislead you.

Who should not run a checkout test this quarter

You don't have the traffic. A checkout test only exposes shoppers who already reached checkout, and the metric is checkout conversion rate, which starts high. That helps: at a 40% baseline, detecting a 10% relative lift at 95% confidence and 80% power needs 2,387 checkout-reaching visitors per variation. The catch is upstream. Say 12 in 100 sessions add to cart and two in three of those reach checkout — 8 in 100 — and that's roughly 30,000 sessions per variation, 60,000 for a two-arm test. (Computed 26 July 2026 — two-proportion z-test, unpooled variance, 40% baseline, 44% target, α = 0.05 two-sided, 80% power.) Under that, you'll get a number and the number won't mean anything.

You haven't located the leak yet. Baymard's top abandonment reason is intent, not checkout — 42% were "just browsing." The next four are checkout-surfaced: extra costs 40%, forced account creation 18%, a checkout that's too long or complicated 17%, site errors 17% (baymard.com, read 26 July 2026). Read your own funnel before you pick a level. If the broken number is your reached-checkout rate, test the PDP and the cart. If the drop is inside checkout, test checkout.

You're on Grow or Advanced and you've never run one. Use Rollouts. It's free, it's already in your admin, and one whole-configuration test will teach you more than a procurement cycle will.

You depend on niche Shopify apps. Subscriptions, loyalty, 3PL, a bundle builder. Shopify's app ecosystem runs thousands deep and Mtrix has no equivalent. That's a real reason to stay, and no consolidation argument outweighs it.

You have no CRO headcount. Mtrix does onboarding and support; it does not run your experimentation programme. Visually sells that on its pro and scale plans. Buy it there.

What the vendor pages leave out — ours included

Consolidation is a trade. Intelligems reads a price test as profit per visitor net of your COGS; nothing else here reports margin at all. Shoplift's workflow is Shopify-native end to end — duplicate a theme, split traffic, read the result without leaving the admin — and it holds the highest rating here. If one of those is your whole optimisation programme, buy that tool and skip the platform conversation.

Mtrix has no published customer stories. No logos and no case-study numbers, because there are none to publish. Intelligems and Visually both publish named DTC brands and conversion figures on their own sites. Ask us for a reference call instead.

Mtrix is web only — it records mobile web, not native apps, and session replay is retained for one month against three years for events and purchases. If you need a longer replay window, raise it before you commit.

FAQ

Test the checkout, not just the button on it

* Measured across 350,000+ sessions, tested side by side against Shopify's checkout in the same period, over two years of tests. Mtrix's own client results, not a guarantee of what a new store will see.