A/B Testing Tools for Email: How to Choose, Test and Actually Improve Performance
Table of Contents
A/B testing tools let you send two versions of an email to different parts of your list, then keep whatever wins. The right tool for most UK and Irish SMEs is not the most powerful one, it is the one your team will actually use every week. Built-in testing inside platforms like Mailchimp or Kit is enough to start. Larger teams that want testing joined up with a CRM lean towards HubSpot or ActiveCampaign. The tool matters less than the method: a clear hypothesis, one variable at a time, and a list big enough to trust the result.
Three things to get right before you pick a tool
- Test one variable at a time. Change the subject line or the call to action, not both, or you will not know which move worked.
- Mind your sample size. Small lists produce noisy results. If your list is under a few hundred active contacts, run the same test across several sends before trusting it.
- Decide what a win looks like first. Open rate, click rate or conversion. Pick the metric that ties to revenue before you press send.
What is A/B testing for email?
A/B testing, also called split testing, compares two versions of an email to see which performs better. You send version A to one part of your audience and version B to another, then measure how each does against a metric you chose in advance: open rate, click-through rate or conversions. It replaces guesswork with evidence. Instead of arguing about which subject line sounds better, you let a slice of your own audience settle it.
The discipline is the same whether you run a corner shop newsletter or a national campaign. What changes is scale. A business with a list of 400 needs a different testing rhythm to one sending 40,000 emails a week, and we will come back to that under sample size.
What should you A/B test in an email?
Focus on the elements that move the numbers, not everything at once. The usual candidates, roughly in order of impact:
- Subject lines. The first thing anyone sees, and the biggest lever on open rate. Test tone, length and whether a question beats a statement.
- Preheader text. The preview snippet after the subject line. It can reinforce the subject or add a second hook.
- Sender name. A person, the company, or both. This quietly affects trust and open rate.
- Body copy and calls to action. Formal against conversational, long against short, and the wording, colour or placement of the button.
- Design and layout. Single column against multi-column, image-led against text-led, font sizes.
- Send times. Different days and times to find when your specific audience is paying attention.
One point worth making early: testing subject lines tells you what your audience responds to, and that insight belongs in your wider content marketing too, not just in email. The phrasing that lifts open rates often lifts headlines and ad copy for the same reason.
How do you choose the right A/B testing tool?
Start with your team, your budget and your existing stack, not with a ranked list of software. Every serious email platform now includes some form of A/B testing, so the real question is which approach fits how you work. There are four broad approaches, and most SMEs only need one of them.
| Approach | Best suited to | What you gain | What to watch |
| Built-in email tool testing (Mailchimp, Kit, AWeber, Benchmark) | Small teams and sole traders running regular campaigns | Simple subject line and content splits with no extra software | Limited to two variants on lower tiers; shallow reporting |
| All-in-one marketing platforms (HubSpot, ActiveCampaign, Campaign Monitor) | Growing SMEs that want testing tied to a CRM | Multivariate testing, segmentation, automation in one place | Higher monthly cost; more setup before value appears |
| Delivery and developer platforms (SendGrid, Moosend) | Higher volume senders and technical teams | Scale, deliverability control and API access | Needs technical resource; less hand-holding |
| Website and landing page testing (paired with the above) | Any business testing what happens after the click | Tests the full journey, not just the open | Requires a plan for traffic volume and significance |
Named tools are examples of each approach, not endorsements. Features and pricing change often, so check current plans before committing.
If you are a small team or sole trader
Use the testing already built into your email platform. Tools such as Mailchimp, Kit (formerly ConvertKit), AWeber, Benchmark Email and Moosend all let you split subject lines and content without buying anything extra. For a Belfast cafe testing whether “Saturday brunch is back” beats “Book your weekend table” as a subject line, that is all you need. The trap is buying enterprise software for a job a starter plan does handle.
If you are a growing SME that wants testing joined to your CRM
Look at all-in-one platforms such as HubSpot, ActiveCampaign or Campaign Monitor. These tie test results to customer records, so you can see not just which email won, but which version brought in the contacts who later became clients. That matters more for a professional services firm with a long sales cycle than for a fast retail list.
If you send at high volume or have developer support
Delivery-focused platforms such as SendGrid and Moosend offer scale, deliverability control and API access. They suit e-commerce operations and SaaS businesses sending large volumes, provided you have someone technical to set them up. If you do not, the flexibility becomes a cost rather than a benefit.
Do not forget the page behind the click
An email test tells you what makes people open and click. It says nothing about what happens next. If the landing page is slow, confusing or badly built, a better subject line just sends more people to a page that loses them. Testing the email and testing the landing page are two halves of the same job. This is where email A/B testing meets web design and development, and where a lot of otherwise sound campaigns quietly leak results.
Which tools offer AI-powered A/B testing?
A growing number of email platforms now use AI to suggest subject lines, predict the best send time and pick winners faster on smaller lists. HubSpot, ActiveCampaign and Mailchimp all offer some version of this, generating subject line variations or optimising send times automatically. Used well, it shortens the guesswork at the start of a test.
Treat AI features as a starting point, not an autopilot. A model can draft ten subject lines in seconds, but it does not know your customers, your tone or your compliance obligations. The judgement still has to be human. For businesses weighing up how far to take this, our AI training and implementation work with SMEs focuses on exactly that line: where automation genuinely helps, and where it quietly does harm.
Ciaran Connolly, founder of ProfileTree, puts it simply: “The tool that generates the test is never the hard part. The hard part is deciding what is worth testing, and being honest about whether your list is big enough to trust the answer.”
Is email A/B testing compliant with UK GDPR?
Yes, provided you have a lawful basis to email people and you handle their data properly. A/B testing itself does not breach UK GDPR or the Privacy and Electronic Communications Regulations (PECR). What matters is that the people on your list agreed to hear from you, and that any tracking you use is covered by consent where the law requires it.
Two points catch UK and Irish businesses out. First, where your data is stored. Many popular tools are US-based, so check whether contact data sits in a UK or EU data centre, and whether the provider has valid transfer safeguards in place. Second, tracking pixels and open-rate measurement can count as processing personal data, so they belong in your privacy notice.
The Information Commissioner’s Office publishes clear guidance on direct marketing and the difference between consent and legitimate interest. It is worth reading before you scale up any testing programme.
Read the ICO guidance on direct marketing.
Best practice for A/B testing emails
A few rules separate tests that teach you something from tests that just look busy.
Test one variable at a time
Change one thing per test. If you swap the subject line and redesign the layout in the same send, a better open rate could be down to either, and you will never know which. Isolate the variable and the result means something.
How many subscribers do you need for a valid test?
Enough that the result is not down to luck. As a rough guide, aim for at least a few hundred recipients per variation. Smaller lists can still test, but you should run the same test across several campaigns and look at the pattern rather than reacting to one send. A significance calculator, built into many tools, tells you whether the gap between A and B is real or noise.
Set a clear objective, then act on it
Every test needs one goal. Chasing opens? Test subject lines and sender names. Chasing clicks? Test the call to action and layout. The value only arrives when you feed the winner back into the next campaign. A test you do not act on is just data you paid for and ignored.
Common pitfalls that ruin results
- Calling a winner too early, before enough people have opened or clicked.
- Testing trivial changes like a single word, where any difference is meaningless.
- Ignoring the season or the send context, so a Black Friday result gets treated as a permanent rule.
- Testing the email but never the page it links to, which caps how much any result can help.
Frequently asked questions
How many subscribers do I need for a valid A/B test?
Aim for at least a few hundred recipients per variation so the result is not down to chance. If your list is smaller, run the same test across several campaigns and look for a consistent pattern rather than trusting a single send. Most email tools include a significance indicator that tells you whether the difference between the two versions is real.
What should I A/B test first in an email?
Start with the subject line. It has the biggest single effect on whether the email gets opened at all, and everything else depends on that open happening. Once you have a subject line that performs, move on to the call to action and then the layout.
Can I test more than two versions at once?
Yes. Testing three or more variants at the same time is called multivariate testing, and several platforms support it. It needs a much larger list, though, because you are splitting your audience more ways, so it suits high-volume senders rather than small businesses.
How long should an A/B test run?
Long enough to gather a meaningful number of opens and clicks, which usually means several hours to a full day depending on how active your audience is. Ending a test after the first hour, when only your keenest subscribers have engaged, gives a misleading picture.
Does A/B testing affect email deliverability?
Generally no, as long as both versions are good quality and you are sending to people who opted in. Testing does not itself harm deliverability. Poor list hygiene and irrelevant content do, so keep the fundamentals right whatever you are testing.
Are there free email A/B testing tools?
Yes. Several platforms, including Mailchimp, Benchmark Email and Moosend, offer free plans with basic testing for smaller lists. They are a sensible way to build the habit before paying for more advanced features such as multivariate testing or automation.
Where to go from here
Pick the smallest tool that does the job, run one clean test with a single variable, and act on what it tells you. The businesses that get value from A/B testing are rarely the ones with the most expensive software. They are the ones with a steady habit: test, learn, apply, repeat. If you want testing built into a wider plan across email, your website and your content, our team works with businesses across Northern Ireland, Ireland and the UK to do exactly that. See our digital marketing services for how that fits together.