A/B Testing and Optimization

A/B Testing and Optimization

Why A/B Testing?

The usage of A/B testing in marketing has been around for many years. It's basic premise is that you create multiple versions of creatives with different art work and/or variations on your call to action. Then you run them all for an initial testing period (i.e. a few days) and compare the results afterwards to see which one generated the most response (i.e. clicks). From there you can adjust the delivery of the creatives to show the better performing one(s) more often. Of course, too much of a good thing can sometimes become a bad thing, right? Creative burnout may occur so it's often necessary to throttle back a well-performing creative to avoid that.

Automation is Key

That sure sounds like a lot of work to keep checking up on your creatives and manually adjusting their delivery doesn't it? It's definitely tedious and prone to human error. The solution is to automate the whole process so it's constantly working to get the best results without requiring any effort on your part.

A/B Testing Requirements

For A/B testing to be possible you will need at least two same-sized media to be assigned to a campaign. You can certainly have more than two. In fact, there is no limit on how many media can be A/B tested simultaneously. Some A/B testing purists will tell you that you should only have two, but I think that's really up to your own judgement.

One thing A/B testing doesn't do is compare different-sized media in the same campaign. If you have a single leaderboard and a single skyscraper in a campaign they will not be A/B tested. This is to avoid problems with impression shortage that might occur if either media were downgraded, so that is more of a feature than a limitation.

Configuring A/B Testing

Unless you're a new user, we haven't enabled A/B testing for you automatically. You will need to go to Settings > Basic > A/B Testing and turn it on. You can also adjust the Initial Testing Duration if the default of 2 days doesn't suit you. The sensitivity of the optimization can also be adjusted by playing with the Minimum Percentage of Improvement but we think the default of 1.5% is pretty good in most cases.

A/B Testing Options

At this point, that's all you really need to do. Just ensure your campaigns have at least two samed-sized media and A/B testing will take care of the rest. It really doesn't get much easier than that!

Opting out of A/B Testing

There may be some cases where you want to opt a campaign out of A/B testing and you can certainly do that. When creating a campaign you will see a drop down that defaults to Enable A/B Testing Priority Optimization beneath the media assignments box. That same drop down appears if you edit the media assignments of an existing campaign. Simply change that drop down to disable A/B testing if you wish to opt the campaign out of it.

Opting out of A/B Testing

This may not be something you ever need to do as A/B testing is pretty useful, but it's certainly nice to know the option is there.

Closing Thoughts

Bringing the power of A/B testing into AdvertServe is definitely a significant improvement. It can be a game changer if you avoided A/B testing before due to the amount of work it required to implement manually. Hopefully it helps you drive more clicks to your advertisers, but remember it only works if you have multiple same-sized creatives to A/B test so make sure your advertisers are aware of that!

← AdvertServe Q1/2015 Update
Revision/Blame Tracking →