1.
AB Testing White Paper
What is AB Testing?
AB testing, Split Testing or Bucket Testing is a process for testing two different versions of an element (A
& B) and a metric which exactly defines success. To determine which version is batter you have to
perform experimentation simultaneously, at the end of experiment you measure which version was
more successful and select that version for real world use.
The Picture above Explains that two versions of web page have been made, version a consists of Blue
title with different placement of Navigation Bar, News Block, Signup Form & Content body while version
b consists of green title with entirely different placement of navigation bar, page title, signup form &
content body. Visitors are distributed randomly for accessing the page. Stats shows that more visitors
land on version b comparable to version a.
You Split your website traffic between two versions of entirely different design, navigation and measure
their performance using matrices like Bounce Rate, Click through Rate, Sales etc. At the end of test you
select the batter performed version.
Why AB Testing?
According to various researches a typical website converts only 2% of its visitors into customers, which is
very low amount of conversion of sales into leads. Visitors cannot explain why they left your websites!
Either designs issues, navigation issues, color scheme is not attractive or anything else! The major

2.
AB Testing White Paper
objective of A/B Testing is to select the most appropriate theme with complete navigation and user
friendly interface. Small changes can make a huge impact on conversion rates which could be beneficial
for webmaster in the long run.
After performing successful AB Testing you can enhance your site conversion rate up to 40%, Google
also believes A/B testing Work wells you can use Google Website Optimizer for further details.
How Does A/B Testing Works?
First of All you have to start with the original page or the page which is under your control! Then you
have to determine that goal has been reached. For Instance if you want to see how many people make
purchase, in this scenario Thank You Page is the key indicator for observing the conversion rate.
A Simple test would present one version of the page to one visitor, and another version to the next. Or
you can swap both of these test pages randomly, or could be based on other demographic or
parameters, it’s all depends upon the testing program and what you are trying to achieve!
One Important aspect in A/B testing is time domain; you have to prepare yourself for running the test
for a month or more to acquire the desired results. Online calculator is useful tool to determine the test
duration, bounce rate and expected conversion rate.
AB Testing Processes (Step by Step):
The Purpose of AB Split Testing is to work with one of these at a time or possibly several remedies at a
time in different environment, controlled, variations to find out the best possible combination of various
on page Factors.
These Processes are quite simple and easy to perform:
Measure
Test
Evaluate
Optimize
This Process needs to continue as far as many iterations as possible, ideally it is continuous process
which never ends, However practical real world issues limits it at some extant where testing stops and
marketing begins.
Measuring:
Measuring is one of the most important aspect of A/B Testing, Nothing can be tested properly unless it
have been measured, There are lots of factors which can be measured but we have to focus on key
factors first which are:

3.
AB Testing White Paper
Hits or Delivered visitors
CTR : Click-Through-Ratio (number of times the prospect has clicked through to the sales page)
Sales
Conversion vs. CTR
Conversion vs. Hits
The above must be tabulated with reference to the Medium (email, web page visits etc.). The initial
measurement is taken against to the control. In the first iteration, the control will be the existing landing
page or email promotion.
In subsequent iterations, it will be the best performing variation of the promotional piece. This will be a
combination of the best performing A/B Variations that are established as a result of the Testing
process.
Testing:
Testing is the procedure of creating two variation (Named A and B), and measuring their success against
one constant. This constant is usually equivalent to the number of page visits – it is the number of times
whenever promotional page being viewed.
By Conversion, the control page is denoted as A. Going to its First Iteration, there will only two
variations, both of them labeled as the control; each of them has equal chance of becoming the control.
In Order to track the exact page from conversion comes through, Link Tagging should be used. Basically
Link Tagging is the process of naming target link –A or –B so that they can be easily identified in the
statistics pages used to measure vectors.

4.
AB Testing White Paper
In each iteration, Variations of the control page are introduced, and the link tagged with –B suffix. The
variations should not be too subtle; otherwise the testing process could not be produce the worthwhile
results.
During the testing process measurement should be taken, however the variations should be taken once
when delivery quota has been achieved on Both A and B Variations.
Evolution:
The Evaluation process compares the vectors that have been measured. The control is then set to the
best performing variation (A or B). This then becomes the 'new A'. The exact definition of 'best'
performance will depend on the goals of the marketing process.
Optimization:
The optimization process takes the best performing A/B splits by category (Headline, CTA, etc.) and
combines them to produce the best overall content to test.
It is quite important aspect to continue the test, even after the optimization is completed; the whole
process is not over because small changes can make huge difference for converting leads to variance.
After creating a well optimized page, it should be established as a new control for Future A/B testing.
Benefits of A/B Testing:
Compared with other methods, A/B testing has four huge benefits.
1). It measures the actual behavior for your site in real world conditions. You can easily conclude the end
result by performing A/B testing, For Instance; if version B performs batter comparable to version A,
then version B Design you should be shown to all users in near future.
2). The Second Big advantage of A/B testing to provide you the clear picture regarding conversion and
generating leads into sales, you can easily find out the best suitable approach by performing A/B testing
under your own circumstances.
3). It has ability to measure very small differences with high statistical significance because you can
throw massive amount of traffic at each design.
4). It is comparatively cheap, Once you created two design alternatives, you simply upload both of them
on the server & perform monitoring which page perform better, there is no need for expensive usability
specialist to monitor each user behavior or analyze complicated interaction design questions. The only
thing you have to do is wait for statistics and the go with the design which have batter numbers.
Limitation of A/B Testing

5.
AB Testing White Paper
With these clear benefits why we don’t use A/B testing for each process because it has some limitation
as well. Let me provides you some details regarding A/B Testing Limitations.
A/B testing can only be used for project which has clear & measurable goals; furthermore these goals
must be measurable by computers. For many sites ultimate goal cannot be measurable through user
action on the server, Goals like improving brand repetition or supporting the company public relations
efforts cannot be measured by A/B testing.
Another downside of A/B testing is that it only works with fully implemented designs. It is quite easy to
test design when it is on running condition but implementation of specific design can be taken long
time, before presenting it in front of new customer on your live website, you must fully debug an
experimental design, Thus A/B testing is suitable for only very small number of ideas.
Results and Deliverables of A/B Testing
One of the most vital aspects of A/B testing is to measure the results and deliverables of A/B testing;
Conversion Rate Optimization is normally used to measure the results;
Conversion Rate Optimization
Conversion rate optimization also termed as CRO is a critical process for determine the results in A/B
testing. The main objective of CRO is to enhance your marketing efforts, no matters how effective at
drawing visitors, will struggle endlessly if your website is not effective in converting visitors to your
desired goals.
Tools for A/B Testing:

6.
AB Testing White Paper
Getting Traffic for website through search engine optimization is simply the First step, next step is to
convert your sales into lead, all you want the visitors you get to stay on your site and performed the
desired action.
Over the Past few years A/B Split testing has been booming, there are plenty of more user friendly tools
out there by now. Here is the list of some useful tools that is beneficial for A/B Testing:
Google Website Optimizer
Split Test (A/B Test) Calculator
A/B Theme Testing Plugin for WordPress
Visual Website Optimizer
Liveball Platfrom
Vertster – Multivariate Testing and A/B Split Test Software