What to A/B Test First: A Framework for Prioritizing Your Tests
Most A/B tests waste time testing the wrong thing. Here's a practical framework to choose which elements to test first — and stop guessing.
Introduction
Most teams pick what to A/B test randomly.
They run a test, get a result, move on — and wonder why their conversion rate barely moves.
The problem usually isn't the test setup. It's the element they chose to test.
Testing button color when your headline is confusing is like painting the walls when the roof is leaking. You're optimizing something that doesn't matter yet.
This guide gives you a practical framework to decide what to test first — so you stop wasting traffic on low-impact experiments.
Why Element Selection Matters More Than Test Setup
A well-run test on the wrong element produces nothing useful.
Here's a real pattern: a team spends three weeks testing two versions of a CTA button — "Get Started" vs "Start Free Trial." They reach statistical significance. The winner lifts clicks by 8%.
Meanwhile, their headline is vague, their value proposition is buried below the fold, and visitors can't tell what the product actually does in the first five seconds.
The 8% button improvement gets wiped out by a 40% bounce rate that nobody addressed.
Element choice is the most under-discussed part of conversion rate optimization. Before you run any test, you need a clear answer to: what's most likely broken right now?
The 4-Tier Testing Hierarchy
Not all elements are equal. Some changes move revenue. Others move metrics that don't matter.
Here's how to think about testing priority:
Tier 1: Headline and Value Proposition
Your headline is the first thing a visitor reads. It answers three questions within seconds: what is this, who is it for, and why should I care?
If visitors can't answer those three questions immediately, nothing else on the page matters. They're already gone.
Test your headline first. Every time. Before anything else.
Signs your headline needs testing:
- High bounce rate despite relevant traffic
- Low scroll depth on session recordings
- Visitors spending less than 10 seconds on the page
Tier 2: CTA — Text, Visibility, and Placement
Once visitors understand what you're offering, they need to know what to do next.
The CTA has three separate problems worth testing, and they're not the same test:
- Visibility: Can visitors even see the CTA at a glance? This is a structural problem.
- Placement: Is the CTA above the fold? Is it where the eye naturally lands?
- Text: Does the button text match what the visitor wants to accomplish?
A quick way to check CTA visibility before you run any test: use a website visual analysis tool to see if your CTA stands out when the page is blurred. If it disappears into the background at blur, real visitors' peripheral vision will miss it too. This is the squint test — a method CRO specialists use to evaluate visual hierarchy before running formal experiments. You can learn more about how it works here.
Tier 3: Social Proof — Placement, Format, and Specificity
Social proof works. But placement and format matter more than most teams realize.
A testimonial buried at the bottom of a long page does almost nothing. The same testimonial placed next to the CTA — right when the visitor is deciding — can significantly lift conversions.
Test these in order:
- Placement (where on the page)
- Format (star ratings vs. quotes vs. logos)
- Specificity (generic praise vs. specific results)
Tier 4: Visual Layout and Above-the-Fold Structure
Layout changes are high-effort, high-risk tests. They should come after you've extracted the gains from Tiers 1–3.
That said, if your above-the-fold section is dominated by a hero image instead of a headline and CTA, you have a structural problem that copy changes won't fix.
How to Identify Your #1 Test Candidate
The hierarchy gives you a framework. But you still need to know which specific element within each tier to address first.
Use these signals:
Scroll depth maps
If 60% of visitors never scroll past the hero section, something above the fold is failing. Focus there first — likely the headline or CTA visibility.
Click maps
Are visitors clicking on things that aren't links? That's a sign they expect something to be clickable that isn't. Are they ignoring your CTA entirely? That's a visibility or relevance problem.
Session recordings
Look for hesitation patterns — visitors who move the mouse to the CTA and then scroll away. That's usually a trust or clarity issue right at the decision point.
The squint test
Blur your page and look at it for three seconds. What stands out? What disappears? If your CTA blends into the background and a decorative image dominates, that's your test candidate — not the button color. Run a free squint test on your landing page to see your visual hierarchy before making assumptions.
Time on page vs. conversion rate
High time on page but low conversions usually means visitors are reading but not being convinced. That points to the value proposition or social proof. Low time on page points to the headline.
A Simple Scoring Framework: PIE
Once you have a shortlist of test candidates, score them with PIE:
| Factor | Question | Score (1–10) |
|---|---|---|
| Potential | How much room for improvement is there? | |
| Importance | How much traffic hits this element? | |
| Ease | How hard is this to implement and test? |
Multiply the three scores. Test the highest total first.
Example:
| Element | Potential | Importance | Ease | Total |
|---|---|---|---|---|
| Headline | 8 | 10 | 7 | 560 |
| CTA text | 5 | 10 | 9 | 450 |
| Hero image | 6 | 10 | 4 | 240 |
| Button color | 3 | 10 | 10 | 300 |
In this example, the headline wins. Not the button color — even though it's the easiest test to run.
PIE removes the bias toward easy tests and forces you to think about impact first.
What Not to Test (Yet)
Some tests waste your traffic budget and produce misleading results.
Don't test button color before you've tested the headline.
A red vs. green button test on a page with a confusing value proposition tells you which color looks better on a broken page. That's not useful information.
Don't test font sizes before layout structure.
If your page structure is wrong — CTA below the fold, headline competing with the hero image — font adjustments won't compensate.
Don't test micro-copy before core value proposition.
"Submit" vs. "Get My Free Report" is a valid test. But only after you've confirmed that visitors understand what the free report is and why they want it.
The rule: fix the bigger problem first. Don't run precise tests on fundamentally broken elements.
One More Thing: Make Sure You Have Enough Traffic
A/B testing requires statistical significance. Running a test on 200 visitors per month means waiting months for a result — and that result will still be unreliable.
As a rough guide:
- Under 1,000 monthly visitors: focus on qualitative research (recordings, user interviews, heuristic analysis) before running formal tests
- 1,000–5,000 visitors: test one element at a time, accept longer test durations
- 5,000+ visitors: run proper A/B tests, move faster
If you're in the low-traffic category, the squint test and session recordings will give you more useful information faster than a formal test. See how visual hierarchy analysis works as an alternative to waiting for statistical significance.
Quick-Start Checklist
Before your next A/B test, run through this:
- What's my current bounce rate and where do people drop off?
- Have I reviewed session recordings in the last 30 days?
- Does my headline clearly answer: what is this, who is it for, why should I care?
- Is my CTA visible above the fold and does it stand out visually?
- Have I run a squint test to check visual hierarchy?
- Am I testing the highest-impact element, or the most convenient one?
- Do I have enough traffic for a valid test?
If you can't say yes to the first six, you're not ready to run a formal A/B test yet. You're ready to do diagnostic work.
Summary
Choosing what to test is more important than how you run the test.
Follow the hierarchy: headline first, CTA second, social proof third, layout last. Use scroll depth, click maps, recordings, and the squint test to identify your specific test candidate. Score candidates with PIE to remove the bias toward easy-but-low-impact tests.
Most teams that struggle with A/B testing aren't failing at statistics. They're failing at element selection.
Fix that first.
Want to check your landing page's visual hierarchy before your next test? Run a free squint test on BlurTest — see what stands out, what disappears, and where your eye actually goes.