Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneurs and startuppers. Annually it takes place at the beginning of October in Lviv at Arena Lviv stadium. In 2016 the conference gathered more than 1800 participants and over 100 speakers from companies like Microsoft, Philips, Twitter, UBER and IBM. More details about the conference at itarena.lviv.ua.
3. Marktplaats.nl
• Largest classifieds site in the Netherlands
• One of the most visited websites in NL
• Founded in 1999, acquired by eBay in 2004
• Now headquarters to eBay Classifieds Group: 12
brands in 17 countries
@schuilr
3
4. Facts & Figures
• 1.3 million visitors / day
– desktop: 34%, mobile: 49%, tablet: 18%
• 9 million live listings
– 350,000 new items / day
• 6 million unique search requests / day
– 70 searches per second (average)
@schuilr
4
15. “Nieuw & populair”
@schuilr
15
• “Nieuw & populair” = trending products
• Pay-per-click advertising model
• Advertisers bid for clicks, similar to Google Adwords
• Metric to optimize:
Revenue Per Mille (RPM) = CTR * bid * 1,000
16. First (minimal) version
• Find top 100 “trending” keywords using Spark
• Randomly pick one of those keywords
• Display top 4 results for the selected keyword
@schuilr
16
17. Can we do better?
• CTR and bid varies per keyword. Random selection
gives average performance.
• Doesn’t consider the user’s personal preferences
@schuilr
17
19. One armed bandit = slot machine
Problem:
How to pick between slot machines
so that you maximize profit?
@schuilr
19
20. Exploration – Exploitation
• Explore (learn)"
Try out different candidates to learn how they
perform over time
• Exploit (earn)"
Take advantage of what you’ve learned to maximize
payoff (your current best guess)
@schuilr
20
21. Many different approaches
• Epsilon First
• Epsilon Greedy
• Upper Confidence Bound
• Thompson Sampling
• LinUCB
@schuilr
21
23. Epsilon First
• Simple and intuitive
• Lots of tools available (VWO, Optimizely, …)
• Average reward until exploration is finished
• What if the best candidate is no longer the best?
@schuilr
23
25. Epsilon Greedy
• Very simple to implement and surprisingly effective
• Can deal with nonstationary problems
• How to determine the optimal value for ε?
@schuilr
25
26. Upper Confidence Bound
Basic idea:
• Calculate mean and a measure of uncertainty
(variance) for each candidate
• Pick current best performer based on mean +
uncertainty bonus
@schuilr
26
28. More data = less uncertainty
95% certain that
true mean ≤ 0.63
Uncertainty bonus:
0.13
@schuilr
28
29. Mean + uncertainty bonus
Upper Confidence Bound
A
B
C
Es)mated reward
Pick “A”!
@schuilr
29
30. Upper Confidence Bound
• Selecting “A” reduces uncertainty
• Candidate “C” now has the highest score
A
B
C
Es)mated reward
Pick “C”!
@schuilr
30
31. Upper Confidence Bound
• Uses variance measure to automatically balance
exploration with exploitation
• Deterministic; requires online learning (not suited for
small-batch mode)
@schuilr
31
32. Thompson Sampling
Basic idea:
• The number of pulls for a given lever should match
its actual probability of being the optimal lever
• Sample from the posterior for the mean of each
lever:
p(λ|X) = Gamma(conv + prior_conv, impr + prior_impr)
@schuilr
32
35. Many conversions
Candidate
Conversions
Impressions
Chance of being
winner
A (3.9%)
892
22,882
97%
B (3.3%)
174
5,261
2%
C (2.8%)
66
2,343
1%
@schuilr
35
36. Lots of conversions
Candidate
Conversions
Impressions
Chance of being
winner
A (3.9%)
5,621
144,132
> 99%
B (3.3%)
256
7,761
< 1%
C (2.8%)
101
3,593
< 1%
@schuilr
36
37. Thompson Sampling
• Weighted random sampling
• Works well in small-batch mode
• Doesn’t consider context (e.g. user’s personal
preferences)
@schuilr
37
39. LinUCB
Basic idea:
• Define a “context” of information of the user
• Fit a per-candidate logistic regression model
• Applies the concept of Upper Confidence Bound
(UCB)
– mean + uncertainty bonus
@schuilr
39
40. Context
• Gender
• Recently viewed categories
• Current date
• Weather forecast
• …
Principal Component
Analysis (PCA) to reduce
sparseness and
computation complexity
@schuilr
40