4. Introduction to testing
Testing is the technique used to know the
actual intent of the customers who are visiting
a web sites
Though a website has a major goal
like, content production or e-commence or
engagement. There are micro elements that
are also part of this. Ex: Looking for careers
link in Amazon web site.
So its important to know how each group of
users are responding to different group of
people.
5. Types of tests
There are many types of testing techniques
that are used. Most popular are,
A/B testing or Split testing
MVT testing or Multivariate testing or bucking
testing
Any testing technique requires to have the
hypothesis ready and the have a clear
definition of test elements
By testing, you cant derive new learning’s
other than the elements that we have
hypothesis for.
6. A/B testing – Details
AB testing compares the effectiveness of two
versions of a web page, marketing email, creative
images, landing pages – in order to discover
which has better response rates, better conversion
rates, etc.
AB testing will test two samples of the test –
Control and test groups, to see which single
variable is most effective in increasing a response
rate or other desired outcome
To be effective, the test must reach an audience of
sufficient size that there is a reasonable chance of
detecting a meaningful difference between the
7. A/B testing – Tools
Example: A company want to test two banners
of a creative reaching out to 5000 people.
First 2500 people got a offer with CTA “Buy now”
Next 2500 people got a offer with CTA “Learn
more”
All other elements of the creative remain the
same
Record CTR for each banner and that will be the
winning banner
Tools that provide that reports are, Google
Analytics content
experiments, Optimizely, Visual website
optimizer, Unbounce, Omniture
8. MVT testing – Details
Multivariate technique for testing hypotheses on
complex multi-variable systems, especially used
in testing market perceptions
Multivariate testing is usually employed in order to
ascertain which content or creative variation
produces the best improvement in the defined
goals of a website, whether that be user
registrations or successful completion of a
checkout process
Testing can be carried out on a dynamically
generated website by setting up the server to
display the different variations of content in equal
proportions to incoming visitors
9. MVT testing – Details
Dynamic creative optimization is the
application of MVT testing. An automatic
system will show the high performing creative
to audience by testing the algorithm for
targeted profiles
In a nutshell, multivariate testing can be seen
as allowing website visitors to vote with their
clicks for which content they prefer and will
stand the most chance of their proceeding to a
defined goal
This is one of the high growth area in digital
marketing in the current digital marketing. All
10. Re-targeting – Details
Re-targeting is the technique of identifying the
group of audience which fits the segment
definition and giving them a relevant message
for their next visit
This targeting can be applied on specific web-
site or on an ad-network or with specific
publisher website
Retargeting can happen based on many
action. Ex: Click on the banner, drop out from
the cart, visit to the site etc.
11. Re-targeting – Tools
Retargeting can be done four segment bases:
demographic, geographic, behavioral, psychogr
aphic
While using this technique, you can use the
same message or different messages fine
tuned
Currently all ad serving platforms are
supporting re-targeting for media
Web analytics tools like – Omniture and Google
Analytics has also that capacity
13. Surveys – Introduction
Analytics tools provide the data what has
happened in the past. It does not really help us
understand why people are doing what they are
doing.
Data by itself does not give the “purchase intent”
With customer actions, we try to figure our this
indirectly
The direct way of figuring this out is – Surveys
Surveys can be done to address a specific need
or can be conducted as a general feedback
mechanism , i.e at site level or at a session/page
14. Surveys – Details
It helps to understand – what are the macro
and micro conversions that are happened on
the site
Surveys can happened at any point of the visit
of the user. However its been identified to have
the survey happened at the end of the visit or
entry of the visit.
With technology you can have surveys or
feedback taken even on the display banners
You can conduct these to all users or only
specific group of people selected based on
segmentation
15. Surveys – Details
With a simple survey, you can get the data related
to
Why a user has visited the site
Were they able to complete the task
If not, why not?
Guidelines for conducting surveys:
Don’t make surveys an obstruction to the user
experience
Don’t collect personal level information of users
Always provide an option of opting out of the surveys
Don’t make surveys long. Limit them to a max. of 10
questions
Use industry benchmarks to design and evaluate your
16. Surveys – Details
Companies who can help you do the surveys –
Iprospects – Paid
Surveymonkey – Free
Forsee results – Paid
The data can be in the form
of, desirability, preference, Discovery, was it
helpful etc. depending on what vendor you are
using
Surveys can collect the both positive and
negative feedbacks
18. Email marketing – Introduction
Email marketing also know as permission
marketing is one of the old forms of marketing.
There are two types of email marketing.
Explicit permission marketing
Implicit permission marketing
Both are powerful marketing forms, but there is a
good probability people will consider them spam if
the messages are not relevant over the time
Using the tracking parameters, we can collect
different metrics, however these are very different
from normal metrics that we collect via websites
19. Email marketing – Metrics
The metrics that are used for email marketing
success are,
No. of emails sent
No of successful emails sent
No of emails opened
Open rate
No of emails read
CTR from emails
No of responses
Response rate
20. Email marketing – Details
Email marketing is very effective if maintained
as an integral communication channel of
campaigns
Its little costly compared to the remaining
forms of marketing and need to be maintained
for valid customers
Its powerful channel to increase the loyal
users and keep them for up-selling and cross-
selling
This is also very good medium to conduct
surveys
22. Competitive intelligence –
Introduction
Competitive Intelligence is the process of
defining, gathering, analyzing &
distributing intelligence about
products, customers, competitors and any aspect of
the environment needed to support executives and
managers in making strategic decisions for an
organization
Key points of CI are,
Benchmark the effectiveness of your existing customer
acquisition strategies
Determine what is driving competitors’ success
Use historical consumer data to forecast future trends
23. Competitive intelligence –
Details
The data can we collected from,
Toolbar data: Toolbars are add-ons on web
browsers, which are used to collect limited information
about the browsing behaviour of the customers who use
them, including the pages visited, the search terms
used, perhaps even time spent on each page, and so forth.
Typically, data collected is anonymous and not personally
identifiable information. Ex: Alexa
Panel data: A company may recruit participants to be in a
panel and each panel member installs a piece of
monitoring software. The software collects all the panel’s
browsing behaviour and reports it to the company running
the panel. Additionally the person is also required to self
report demographic, salary, household
members, hobbies, education level and other such detailed
information. Ex: Comscore, Neilsen
24. Competitive intelligence –
Details
ISP (Network) data: Network provides collect
data from server logfiles. The data collected by
the ISP consists of elements that get passed
around in URLs, such as sites, page
names, keywords searched etc.. The ISP servers
can also capture information such as browser
types and operating systems. Ex: Hitwise
Search Engine Data: Search engines, such as
Bing, Google, Yahoo! and Baidu, are logged by
those search engines, along with basic
connectivity information such as IP address and
browser version.
25. Competitive intelligence –
Details
Benchmarks from Web Analytics vendors:
Web analytics vendors have lots of
customers, which means they have lots of data.
Many vendors now aggregate this real customer
data and present it in the form of benchmarks that
you can use to index your own performance. Ex:
Google Analytics, Omniture, Coremetrics
Self-reported data: Some websites can report
their data intentionally to panel data collectors to
attract more advertising
26. Competitive intelligence –
Details
Hybrid data: Any mix of above collected data can
be used for the analysis. Google Trends for
search is one of the freely available tools which
helps to provide the search query analysis based
on keywords
External voice of data: You can upload data that
is collected offline to support the online CI
Any of these methods has its own
advantages, short-comings and privacy related
issues. So its advisable to use a collection of
data sources to have the best solution
possible
28. RSS – Introduction
Rich Site Summary is an aggregator that used
to publish frequently updated works such
as blog entries, news headlines, audio, and
video
RSS Tracking is a methodology for
tracking RSS feeds
This tracking is tricky when measuring the user
usage
We discuss these methods in the following
slides with its advantages and limitations
29. RSS – Tracking Method
Method 1:
Transparent 1×1 pixel images - These images
can be embedded within the content of the RSS
feed by linking to the image which should be held
on the web server. The number of requests made
can be measured by using the web server log
files. This will give a rough estimate as to how
many times the RSS feed has been viewed.
The problem with this method is that not all RSS
feed aggregators will display images and
parse HTML
30. RSS – Tracking Method
Method 2
Third-party services - There are services
available on the Internet that will syndicate RSS
feed and then track all requests made to their
syndication of RSS feed. These services come
in both free and paid forms.
The problem with this method is that all
analytical data about the feeds are controlled by
the service provider and so not easily accessible
or transferable.
31. RSS – Tracking Method
Method 3
Unique URL per feed - This method requires
heavy web server programming to auto generate
a different RSS feed URL for each visitor to the
website. The visitor's RSS feed activity can then
be tracked accurately using standard web
analytics applications.
The problem with this method is that if the feed is
syndicated by a search engine for instance then
this will defeat the purpose of the unique URLs as
many people could potentially view the RSS feed
via a single URL
32. RSS – Tracking Methods
Method 4
Estimating number of subscribers from the log files.
Some aggregators (For ex: Bloglines and Google
Reader) include a number of unique users on whose
behalf the feed is being downloaded in the HTTP
request. Other readers, such as web browsers, can
be counted by noting the number of unique IP
addresses that retrieve the file in a given period.
This provides an estimate of actual
readership, although it is probably higher than the real
number because people may sign up for accounts
with multiple aggregators and never delete their
subscriptions and because they may read the same
feeds at different computers, or the same computer
may have a different IP address at different times.