1. Big Data, Big Changes:
Data-Driven Product
Development at Etsy
Strata London, 2012
Dr. Jason Davis
Director of Search & Data
jdavis@etsy.com / @jasondavis
Monday, October 1, 12

2. (The change)
Monday, October 1, 12

3. 6 months from inception to launch
3 engineering teams
3 product launches
Dozens of experiments
Lots and lots of data analysis
Monday, October 1, 12

4. The switch to relevancy
Search, advertising, analytics, and more
Building big data products at Etsy
Invest in tooling
Understand tradeoffs
Appreciate complexity
Monday, October 1, 12

5. Monday, October 1, 12

6. By the Numbers
1.4B pageviews/month
$525MM sales in 2011
40MM unique visitors/month
800k shops / 150 countries
Monday, October 1, 12

7. SELECT * FROM listings
WHERE title LIKE
'%silver%'
ORDER BY creation_tsz
DESC;
Monday, October 1, 12

8. Solving the recency
problem
Monday, October 1, 12

9. Better buyer
experience
Improved search
relevancy
Monday, October 1, 12

10. Improved advertising
Search Ads
Monday, October 1, 12

11. More search insight
for sellers
Shop Stats
Monday, October 1, 12

12. Building big data
products
Monday, October 1, 12

13. Build lots of tools
Monday, October 1, 12

14. Go to the source
Raw data
visit logs
raw query logs
Descriptive statistics
search depth
query conversion rates
Monday, October 1, 12

15. Invest in tooling
Lunr
Monday, October 1, 12

16. Side by side testing
Monday, October 1, 12

17. A/B Analyzer
Monday, October 1, 12

18. Shop Stats with Referrals
Monday, October 1, 12

19. Understand tradeoffs
(most tradeoffs don’t look like this)
Monday, October 1, 12

20. The bike is 20% lighter!
Monday, October 1, 12

21. Queries a, b, & c are better. Queries x & y are not.
Monday, October 1, 12

22. ¡Hola! வணக்கம்!
Guten Tag 你好 !"#$% ‫أ*)'م‬
ّ
Hello नम#$ 今日は
Monday, October 1, 12

23. Thinking short-term vs long-term
How will the change affect behavior?
Affect expectations?
User incentives?
Improved data?
Monday, October 1, 12

24. Appreciate complexity
Monday, October 1, 12

25. Vs
Algorithmic Complexity
Monday, October 1, 12

26. Success / Complexity
Tradeoff
4 months
30 experiments
11 wins
Monday, October 1, 12

27. UI
June 2005
Complexity
November 2006
May 2008
Today
Monday, October 1, 12

28. Communications & Support
Complexity
Monday, October 1, 12

29. (The change)
Monday, October 1, 12

30. Etsy - Strata NYC
Continous Experimentation with Continuous
Deplyoment
Steve Mardenfeld
Scala + Cascading = Scalding
Avi Bryant
Case Study: What’s a Customer Worth?
Roberto Medri
October 23-25
Monday, October 1, 12

31. Resources
Code as Craft blog
http://codeascraft.etsy.com/
Slides (including these)
http://www.slideshare.net/etsy
Github
http://github.com/etsy/
Join in the fun
http://www.etsy.com/careers/
Monday, October 1, 12

32. Monday, October 1, 12