Glossary

Explore Meshline

Products Pricing Blog Support Log In

Start with your goals.

Book a call
Autonomous Operations

Should You A/B Test That Flow Email in Klaviyo? Decide, Then Set It Up

Should you A/B test that underperforming Klaviyo flow email? See when a test is worth running and how to set the winning metric before you publish.

A flat illustration of an envelope and a clock positioned side by side against a large circular background.

Your welcome, post-purchase or win-back flow email is not pulling its weight, and you are wondering whether an A/B test is the right move or just another task on the list.

The honest answer: a test is worth running when you can name the one element you want to change and the metric that should move because of it.

If you cannot name both, a winner alone gives you little guidance on what to change.

This article walks through that decision first, then the documented setup steps in Klaviyo's flow builder.

The goal is that the winning metric, variation weights and winner-selection settings match what the email is supposed to change before you publish.

When a flow email A/B test is worth running

Klaviyo's flow email A/B testing lets you configure two or more variations of a single flow email.

You can adjust the subject line or core content for each variation.

If the statistical-significance setting is active, Klaviyo can choose the winner automatically.

You can also choose one manually using open rate or click rate.

That machinery is only useful when the question behind it is specific.

Ask yourself three questions before you open the flow builder:

  • Can I name one variable? Klaviyo's campaign guidance says A/B tests should compare one factor at a time — if you edit the subject line, change nothing else. If your draft idea is 'new subject line, new layout and a different discount,' pick the one you most suspect; testing several at once makes it harder to tell what drove the result.
  • Do I know which metric should move? A subject-line change should move opens. A content or call-to-action change should move clicks. If the element and the metric do not line up, the test cannot answer your question.
  • Is the email still in draft, or has it been sending for a while? The email’s status affects how its analytics are tracked.

If you answered yes to the first two, a test is a reasonable investment.

If not, spend the time defining the hypothesis instead of running a test that cannot settle it.

Check the email's status before you build the test

Flow email tests have documented constraints that affect when you should start one:

  • Variations can only be cloned or deleted while the test is in Draft status. Deleting variations is disabled for live A/B tests, and a test must keep at least two variations.
  • Publishing a draft email test also changes the email's status from Draft to Live.
  • If the flow email is live and has already been sending, analytics for each variation copy are tracked separately once the test is published, and Klaviyo does not continue tracking analytics for the original email once you have added a variation.

The practical consequence: if the email has accumulated sending history you want to keep comparing as a single record, adding a test variation ends that continuous tracking.

For a brand-new email, set the test up before it goes live.

For an email already in production, decide whether losing unified original-email analytics is acceptable before you publish the test.

After a test has completed for an email, you can create another test for the same email in the future if needed.

One test does not lock you out of testing again later.

Set up the test in the flow builder

Once you have decided to proceed, the documented steps are:

  1. Click the email within the flow builder and configure its content if you have not already done so.
  2. Click Create A/B test in the Email details panel. Two new copies of the email are created.
  3. In the A/B test tab, select each variation to edit its name, subject line, preview text, sender name and sender email address. To edit the email content itself, click Edit Email above the preview.
  4. To add a third variation, clone an existing one while the test is in draft. You cannot clone variations after the test is live.

Name each variation descriptively so the test results are readable later.

Klaviyo's campaign guidance suggests names that reference what you are testing, such as which button style or subject-line approach a variation uses.

Match the winning metric to the element you are testing

This is the step where a default setting can quietly derail your test.

For flow emails, only Open Rate and Click Rate are available as winning metrics, and tests default to automatically choosing a winner based on click rate.

Klaviyo recommends click rate for flow emails because factors such as Apple Mail Privacy Protection (MPP) can cause inflated open rates.

When recipients turn on MPP in iOS 15 and newer releases, the tracking pixel is prefetched.

This changes the open-rate data Klaviyo receives, so open-rate winners may reflect automatic opens as well as human engagement.

But if you are testing a subject line, preview text or sender name, opens are the metric that should respond to your change.

Klaviyo's campaign testing guidance recommends open rate for those elements and click rate for content changes like button size or color.

Leaving a subject-line test on the click-rate default means the winner is chosen on a metric your variation was not designed to move.

To change it, click the Configure link under Winner Selection in the Test Settings section at the bottom of the A/B test tab.

Then set the Winning Metric to Open Rate if that matches your element.

Then click Save Settings.

One more consideration if you lean toward open-rate tests: Klaviyo notes a caveat for regular A/B testers.

If you regularly run A/B tests and more than ~45% of opens come from Apple Mail, Klaviyo suggests a custom report that includes an MPP property.

Check your own open-source distribution before relying on open-rate winners.

Set variation weights and winner selection

In the same Test Settings section, you can edit the weight of each variation, which establishes the percentage of recipients who receive each version.

Klaviyo generally recommends weighing each variation equally, but you can change the percentages to any amount you like.

You can also select automatic distribution.

This starts with an equal split and, over time, shifts more recipients toward the better-performing variation while the test is still running.

For winner selection, two options are available in the Configure modal:

  • End test by statistical significance: selected by default. Clear it if you would rather end the test manually or use only a date limit.
  • Set a date limit: the test ends automatically on the date you select. If you select both options, the test ends when whichever condition occurs first.

For a flow email that sends continuously to new contacts, a date limit gives you a defined evaluation window.

Use a date limit if the test needs to end by a specific date even when the statistical threshold has not been reached.

Publish and read the results

From the A/B test tab, click Publish Test in the upper right.

Review the confirmation carefully: publishing a draft email test also changes the email's status from Draft to Live.

To delete a draft test instead, click the arrow next to Publish Test and select Delete.

For a published test, use Choose Winner to end it rather than looking for that draft-only menu.

To view results, select the email in the flow builder.

For a running test, click View test in the email details panel; for a completed test, click View A/B test in the A/B test section.

The results page shows recipients, opens, clicks and the selected conversion metric with the number of recipients who converted.

A worked example: the win-back email

Here is an illustrative setup, not a real customer result.

Say your win-back flow email gets opens but few clicks, and you suspect the call-to-action text is too vague.

  • Element: CTA text only. Subject line stays identical across variations.
  • Winning metric: Click Rate, which matches the content change and happens to be the default.
  • Weights: equal split across two variations, per the general recommendation.
  • Winner selection: statistical significance plus a date limit, so a quiet segment does not keep the test running forever.
  • Status check: the email is already live, so you accept that original-email analytics stop once the test publishes and each variation is tracked separately.

Contrast that with a welcome email still in draft where you want to test a personalized subject line against a plain one.

There you would switch the winning metric to Open Rate and keep everything else in draft until the test is configured.

Publish once, knowing the publish action itself takes the email live.

Quick decision checklist

  • Can you name exactly one variable to change? If no, define the hypothesis first; testing multiple variables at once makes attribution harder, per Klaviyo's one-factor-at-a-time guidance.
  • Does the element map to an available flow metric? Subject, preview and sender changes point to open rate; content and CTA changes point to click rate. If no metric matches, reconsider the test.
  • Is the email live with history you want preserved as one record? If yes, confirm you accept that original-email analytics stop when you add a variation.
  • Is the default click-rate winner wrong for your element? If yes, change it under Winner Selection before publishing.
  • Do a large share of opens come from Apple Mail? If yes, treat open-rate winners cautiously and consider an MPP-aware report.

Run through those five questions and you will either have a test worth publishing or a clear reason to hold off.

Either outcome is better than a winner chosen on the wrong metric.

If you are still shaping the flow around this email, it may help to step back one level and decide how the flow branches in the first place, or which trigger should start it.

And if you are weighing whether to build the whole flow from a template, that decision pairs naturally with this one.

Sources: help.klaviyo.com, help.klaviyo.com

How Meshline can help. Connect automation, Organic Marketing (demand generation), and customer lifecycle management (Revenue Intelligence).

Bring topic planning, content publishing and performance feedback into the conversation about your workflow. Book a Meshline demo.

Revenue Intel

Ask us about this workflow.

Tell us what you want to fix or automate. We'll reply with the most useful next step.

See how we can help

Implementation decisions

Put this into practice

Before choosing a solution, define the problem, the available data and who will review the outcome.