A/B Testing for Nonprofits Explained

Want to get more website visitors, donors and volunteers?

If you’re serious about improving your nonprofit’s online conversion rates, there’s one simple tool you shouldn’t neglect: A/B testing.

While you’ve probably heard of A/B testing, there are a lot of misconceptions about what it really is, and what you should actually spend time on testing.

Even marketers get this wrong, and end up wasting a lot of time without improving their results in any significant way–or learning anything actionable.

If you’re with a nonprofit, you don’t have hours of time to waste on poor A/B tests that don’t improve your bottom line.

Here’s what you need to know.

1. Definition: An A/B Test is a way to figure out what your most powerful appeal is.

In an A/B test, you compare two things that are almost the same–except for one important element. For example, if you send a fundraising email, here’s how you can easily A/B test your email subject line:

1. Write an email.

2. Write two subject lines that are different.

3. Cut your email list into half– now it’s 2 lists. Send the email to both lists, but make the subject line different for each list.

4. Measure. Did more people open one email than the other? Did more people click through a link on one email than the other?

5. Use the better email subject line in the future.

With a simple test, you can easily figure out what messages your audience most connects to.

Testing is essential, because no audience is the same. You need to figure out what works with YOUR constituents.

2. So what are the best elements for your nonprofit to A/B test?

Here we come to the dark side of A/B testing: there is no limit to the amount of things you can test.

If you wanted, you could send an email with a difference of only one punctuation point–say, switching out a single period for an exclamation point. You could test what time of the day your tweets get the best engagement on a minute-by-minute basis. You could test your email signature.

No change is too small to test. But just because you CAN test something doesn’t mean you SHOULD test it.

This is where misconceptions about A/B testing happen. The most common idea of A/B testing is doing something like changing the color of your Call to Action buttons to see which shade of red is the most effective at drawing your visitors in.

Sure, you can make small improvements in your marketing and website by testing these things–but improvements like this probably won’t make enough of an improvement in the long run to be worth their time investment.

For some organizations, minute optimizations make sense. If McDonalds can save $0.0001 per burger by not putting sesame seeds on their buns, they can save thousands of dollars since that minute amount of optimization is spread over millions of burgers sold.

But small and medium-sized nonprofits don’t operate on that level of scale.

You might increase your conversions by 5% with a better email subject. But that’s nothing if you can increase your conversions by 400% by making a better pitch, telling a better story or making a clearer case for your mission.

3. Here are a few great things for your nonprofit to test:

1. Your Donor Pages: How many forms do donors need to complete to donate? Can you reduce those? What different kinds of questions can you ask? Are more forms better, or fewer? Improving your donor page conversions mean more money for you = worth testing!

2. Your Appeals: This is probably the most important thing to test, and leads to the biggest rewards. What’s your message? Can you frame your solution more clearly? Can you better explain the problem? Is there anything in your message that doesn’t need to be there–any dead weight? Are you making it about the donor and their contribution, or about your nonprofit’s importance? (hint: the former usually results in more donations) Spend the most time figuring out your message and your storytelling.

3. Your Titles: Titles are easy to test and often lead to big rewards. Do shorter titles work better? Question titles vs. statement titles? How do different appeals work better or worse than others? Are negative titles or positive titles more effective? Spend a lot of time making your titles more effective.

4. Your Donate Button: Test every minute shade of color. Just kidding! Don’t get too into the minute details like the donate button color, especially if you haven’t optimized points 1-3. If you think your donate button is performing poorly, the best things to test are the actually Call to Action text and the appeal around it. Is it placed on an area of the page such that the visitor isn’t drawn to it? Is the benefit of clicking the button made clear?

Finally, the most important law of testing: don’t assume anything!

You might have read a report that says your constituents don’t use Facebook. Common wisdom says that the 65+ demographic tends to be the biggest donors. These are all assumptions, but are they correct for YOU?

But until you test these insights, you don’t know if they actually apply to your unique demographic.

So get out there – test and make your appeal more remarkable.


Marc Koenig

Marc Koenig is a regular contributor of Nonprofit Hub. Marc believes smart, ethical marketing can make the world a better place, and strives to create content that helps nonprofits tell better stories, push their organizations to excel and do work that matters. You'll find him writing Nonprofit Hub featured posts, brainstorming infographics and tweeting up a storm at @npmarc - follow him and say hi!

February 24, 2013

You May Also Enjoy

Partnerships Director

Partnerships Director

This position is a great opportunity to develop relationships with a wide-variety of nonprofits while learning the interior operations of a rapidly expanding organization. The position is accountable to the Executive Director.

Become a Member

Whether you’re with a large team or a solo entrepreneur looking to start the next great cause, we have a membership package that will help you grow your network and your cause.