“It’s all about data these days. Leaders don’t want to make decisions unless they have evidence. That’s a good thing”. This sentiment from the Harvard Business Review illustrates why A/B testing is so popular in advertising and marketing. A/B testing is the more efficient, accessible and easy-to-measure form of experiment. A/B testing is actually over 100 years old and is actually a randomized controlled experiment. Sounds fancy, but it is actually simple to understand, execute and master it. We’ll be looking into what is A/B testing, why it is important and how to A/B test today.
What are Creatives?
- Creatives are the images, photos, videos, audio and other creative and tangible fixture. Creative elements include images, colors, styling, value propositions, text, CTAs, logos, length of video, landing pages, etc.
Why Testing Creatives is Important?
Ad quality has dramatically enhanced. Today, those who engage customers are those with the greatest creatives (greatest ads). Gaming companies are the most prolific users of A/B testing, however, this method is useful for all categories.
Trends can rapidly become the new norm in mobile, and effective advertisements from past years may not meet today’s performance objectives. Having intuition is nice, but you have to check it to see if it’s right.
Additionally, it can also be used to evidence decisions made. Making a gut choice that goes wrong could perhaps lead to repercussions. Making an informed decision made on empirical data that goes wrong is less reprehensible.
How Can You Test Creatives?
A/B testing is a randomized experiment with two variants, A and B. Thus, multiple elements can be compared with A/B testing, such as the accompanying text to an ad, time of day the ad is displayed, the audience selected, or even which CTA is used. A/B testing will provide you with information to back up important company choices rather than relying on your subjective feelings. You can do the testing every two or three weeks as a rule of thumb.
The only limit to A/B test is the time and ability of the tester to plan, execute and monitor the results of the A/B testing. This process can be slightly tedious and time-consuming, thus your testing efforts should be planned wisely. If you want to learn more about A/B testing check this article: Mobile A/B Testing: Best Practices And Tools.
Protip – set a good sample size: it is important to set a reasonable sample size, so the data is not skewed by any outliers. For example, A/B test two creatives with an audience size of 10, this data is more likely to be skewed by outliers. The larger the sample size, the less likely this is to occur.
Specific Variables to Test for Static Images
Here are some things, that are easily A/B testable, as they are often few elements, thus it is easier to pinpoint which variable has a higher correlation to your desired goal. Of course, you can A/B test different creatives that are totally different, however, it would be difficult to pinpoint exactly which variable (part of the creative) caused this difference in results.
- Size of image (e.g. square vs banner)
- Text or no text
- Different text within the image
- Colour of text. Red or blue? Which suits you?
- Different font
- Overall color of the image (perhaps you have the protagonist in a blue or green shirt)
- Change 1 or two items in the image (you might have an image where the focus is holding one or two items. You might think about changing this. It could be an eCommerce app, where the main person in the image is holding credit cards or holding shopping bags, however, all other elements remain the same).
Specific Variables to A/B Test for Animated Ads
All of the same variables that can be A/B testing in static images can also be tested in video images. There are also some additional variables that can be A/B tested, which include, but are not limited to:
- Different effects (e.g. one might have a bold transition between different images, the other might have a faded transition).
- Different environment. Perhaps you can test different backgrounds. Your awesome new alcoholic beverage sitting on a beach in Mexico, versus by the pool in NYC.
- Different Music.
- Music. A recent trend in advertisements has been silent advertisements on Youtube. This is a tactic to cut through via disrupting the audiences expected loud video ad. Thus, you can test music, versus no music. Or audio versus no audio.
- Different time. Try the same video or audio at different times. Good times to test are 15 seconds and 30 seconds. Or 30 seconds and 1 minute.
- Captions. Sometimes captions are advised. For example, when videos are displayed in many social feeds, often the audio is automatically muted unless clicked on. Thus, captions can be very useful in these situations. Thus, A/B testing automatic captions for your videos can be an excellent point of comparison.
- Voice Over. You can A/B test different voice overs. Perhaps test male vs female. Dramatic versus casual sounding narration. A mature versus youthful sounding voice over. \
- Seasonality. Does your creative convert more on a winter background, versus a summer background. A/B test this!
Final Thoughts
A/B testing is probably one of the easiest methods of testing, and if executed correctly can significantly improve the performance of your creatives. You can either A/B test entirely different creatives, and then adjust funds accordingly, or you can A/B test similar creatives with few variables changed, to get a clearer picture of what elements worked and what elements did not. This information and learning can also be stored over time, and patterns may emerge. Remember, each business’ results will be unique. What might work for one strategy game, might not work for the next? Contrastingly, what might work for one peer-to-peer eCommerce app, might actually also work for a dating app too.
Why? Because there are multiple factors to consider: the brand/company, the intended message/agenda (is it a branding or conversion campaign), the campaign type, product’s audience and so forth.
With all of these multiple variables in-play, affecting the success of your creatives, and ultimately the success of your ad campaigns, A/B testing is certainly advisable.
If you’d like to get started, App Samurai, with our easy-to-use dashboard can connect you with multiple ad networks easily. This can make A/B testing seamless. Register for free today to get started.