This first study focuses on the personalized positioning of ads on content pages. It demonstrates a correlation between the level of attention that users devote to a content, and the position of the most noticeable graphic element on the side bar. In particular, we find that the graphic element most noticed by a user is roughly to the side of the point in the article where the user's attention waned. We argue that this finding lays the foundation for increasing display advertising effectiveness by tailoring ad positions on each article page impression to the user viewing it.

1. User Attention and
Engagement
on Content Pages
Oleg Rokhlenko
Yahoo! Labs, Haifa
Yandex, March 24, 2014

2. - 2 -
Yahoo Labs, Haifa

3. - 3 -
Yahoo Labs Beach200m

4. - 4 -
ENGAGEMENT-BASED USER
ATTENTION DISTRIBUTION ON
WEB ARTICLE PAGES
Joint work with Nadav Golbandi, Ronny Lempel, and Limor Leibovich

5. - 5 -
Motivation
• Online display advertising is a multi-billion
dollar business
• Ads are often sold by impression in a model
known as CPI (cost-per-impression) or CPM
(cost-per-mille)
• Advertiser return of investment (ROI) depends
on how much attention is paid to their ads by
users
– Till now the surrogate to that was thought to
be ad location

6. - 6 -
• Varies widely
• Benway and Lane studied “Banner blindness”
– Showed that users didn't find specific information
that they were asked to and if it was embedded in a
banner
– Moreover, ads located higher on a page are missed
more often than ads located lower on the page
• On the other hand, in a different online advertising
scenario --- sponsored search --- the most effective
(and expensive) ad spots on a page are the top ones
• We argue that users are neither blind to display
advertisements nor consume it only before/after the
main content
Main content
Ads
Users’ attention distribution

7. - 7 -
Our scope
• Hypothesis:
– For a given user reading a specific article page, the
best ad location is to the side of the point in the story
where the user's interest tapers off
• Envisioned application:
– Many media sites employ recommender systems
and other machine learned algorithms that predict
users' tendencies to click on stories
– Those algorithms will be enhanced to also predict -
given a user„s click on a story - where the user's
attention will wane
– The highest-cost ad position will then be to the side
of the predicted point of waning attention

8. - 8 -
Measuring ads effectiveness
• Recognition: is the awareness of having
previously experienced the stimuli
– Simply asking the respondent if he/she has
seen a specific ad would qualify as a
recognition measure of exposure
• Recall: is the mental reproduction of some
target item experienced or learned earlier
– To qualify as a recall measure of exposure,
the consumer would need to accurately
describe some aspect of the ad
R. Bagozzi and A. Silk. Recall, recognition, and the measurement of memory for print
advertisements. Marketing Science, 2:95-134, 1983

9. - 9 -
Experimental setup
Amazon Mechanical Turk
Primary
content
Instruction set Content page with ad slots Cool down page Questionnaire
1 2 3 4

10. - 10 -
Content page
“I‟m done reading”
button
Primary content
(from OMG!
Website)
Ad #1 slot
A total of 4
paragraphs
4 images acting
as ad surrogates

11. - 11 -
Images choice
• To measure recall in a valid manner, we used images that can
not be recognized easily by a superficial glance on the image
• Our images consisted of a photo of a famous person, whose
name should be familiar to most people but whose face is
much less known, along with a quote of the person with his
name at the end
• Only users who paid enough attention to the image and read
its text would be able to answer who was displayed in it
Instead of embedding ads in the
page, we used images as
surrogates.
We deemed it easier to control
recall aspects by using these
images.

12. - 12 -
Questionnaire page
• Contained questions on both the text and the images:
– Users were not told in advance the type of questions
they will be asked
– Furthermore, they were not able to return to the
article page once they started answering the
questions
• The purpose of the questions on the text is to allow a
confident estimation of the location in the text where the
user stopped reading the article
• The purpose of the questions on images is to measure
advertisements recall

13. - 13 -
Questionnaire page (cont.)
• The questionnaire contained 16 questions:
– 1 question on the overall level of interest they found
in the article
– 12 multiple choice questions, each with 6 possible
answers (one of them is “don't know”)
• 8 were on article‟s text (2 on every paragraph)
• 4 were on the images (1 recall question per image)
– 3 additional questions were to assess the
awareness of users
• E.g. “What was the main topic of the article”?
• Only users that answered correctly at least 2 of these 3
questions were included in the ensuing analysis

14. - 14 -
Classification of reading profiles
• The text shown to users was comprised of 4
paragraphs
• Users were asked 2 questions on every paragraph, and
their answers were coded as ternary response vectors
indicating the number of correct answers (0, 1 or 2) for
every paragraph
– Example: 0 2 1 0
Answered correctly both
questions of the second
paragraph
Answered correctly one
question of the third
paragraph

15. - 15 -
Classification of reading profiles (cont.)
• There are 81 possible response vectors
• Mapped into 5 reading profiles, indicating the number of
paragraphs read by the user -- ranging from 0 to 4
paragraphs
– Assumption: users read linearly (not necessarily
every sentence), from top to bottom
• Ideally, these 5 reading profiles correspond to the
response vectors 0000, 2000, 2200, 2220 and 2222
– In practice, only 21% of the response vectors
matched one of those 5 patterns
• We mapped the rest of the response vectors to one of
the 5 possible reading profiles based on probabilistic
considerations
– For more details please refer to the paper

16. - 16 -
Main result
• We report:
1. Correlation between workers‟ answers
about the level of their interest in the article
and the predicted reading profile (intuitive)
2. Correlation between the user's reading
profile and the recall measure on ad-
surrogate images (not intuitive)

17. - 17 -
Main result (cont.)
• 75% of the users were banner blind (consistent
with findings of Benway and Lane)
• We divided the 204 non-image blind worker
assignments into 3 reading profiles:
– very little reading (0-1 paragraphs)
– moderate reading (2-3 paragraphs)
– thorough reading (4 paragraphs)
• We next examined the average image recall
rate as a function of image position

18. - 18 -
Main result (cont.)
Baseline
T-TEST
significantly more recognized than the
corresponding points in little reading category
p-value=0.0121
p-value=0.0003
p-value=0.0053

19. - 19 -
Summary
• First step toward personalized placement of ads based
on attention considerations
• We argue that for a given user reading a specific article
page, the best ad location is to the side of the point in
the story where the user„s interest tapers off, which in
turn can be predicted from the expected user interest in
the content
• These findings lay the foundation for increasing display
advertising effectiveness by tailoring ad positions on
each article page impression to the user viewing it