Transforming Search in the Digital Marketplace
•Transferir como KEY, PDF•
3 gostaram•1,185 visualizações
Etsy transformed its search functionality over several years through multiple launches and an acquisition. The initial search was based on recency but this did not provide a good buyer experience. Etsy launched new search relevancy and ads features through several beta tests and launches over 6 months in 2011. The new search prioritized relevancy over recency to improve the buyer experience and seller visibility. This involved testing changes, communicating updates, and analyzing results before making relevancy the default search setting.
Recomendados
Recomendados
Mais conteúdo relacionado
Último
Último (20)
Destaque
Destaque (20)
Transforming Search in the Digital Marketplace
- 1. Transforming Search in the Digital Marketplace Dr. Jason Davis Director of Search & Data jdavis@etsy.com / @jasondavis
- 2. (the change)
- 3. 6 months from inception to launch 3 engineering teams 3 product launches 5 different functions 1 startup acquisition 4 months to realize success
- 4. About Etsy History of Etsy search The pitfalls of recency Relevancy: strategy and goals Takeaways: what we learned
- 6. By the Numbers 1.5B pageviews/month $525MM sales in 2011 40MM unique visitors/month 800k shops / 150 countries
- 7. The beginnings June 2005 the birth of Etsy 2005 2006 2007 2008 2009 2010 2011 2012
- 10. Text December 2008 Solr / Lucene 2005 2006 2007 2008 2009 2010 2011 2012
- 11. Text December 2009 Jason starts at Etsy 2005 2006 2007 2008 2009 2010 2011 2012
- 12. Relevancy “For the next three Thursdays, we will change the default sorting method for all searches on Etsy, from recency to relevancy.” Text January 2010 2005 2006 2007 2008 2009 2010 2011 2012
- 14. Buyer experience I was looking for a pink bonnet with blue polka dots. I searched for "polka dot bonnet" but all I got were vintage polka records. Then I wanted a classic rock album and searched for "rock and roll" and got nothing but circular stones. http://www.etsy.com/blog/news/2007/search-what-you-wanted-and-what-you-got/
- 15. Predictability
- 16. Visibility
- 17. Strategy
- 18. Better buyer experience Improved search relevancy
- 19. Better advertising for Search Ads
- 20. More search insight for Shop Stats
- 21. Sequencing
- 22. Search Ads development April May June July August September October
- 23. Core system and architecture Keyword pricing & analysis Ad buying experience Ad quality Integration with search
- 24. Search improvements April May June July August September October
- 28. July 21, 2011 Shop Stats with referrals April May June July August September October
- 29. “We plan to move search by relevancy to the default search setting once we feel this is the strongest site experience.” Natalie on the Blog http://www.etsy.com/blog/en/2011/putting-your-shop-on-top-etsy%E2%80%99s-most-relevant- search/ July 28, 2011 Announcing the switch April May June July August September October
- 30. “As we continue to develop search sorted by relevancy, our goal is to make this the default site Frank in the Forums experience.”http://www.etsy.com/teams/7718/site-help/discuss/8755739/ July 28, 2011 Announcing the switch April May June July August September October
- 31. “With more buyers coming to the site over the holidays, we know it’s critical that we improve the search experience on Etsy. We want to make those changes well in advance of the holiday season to give you plenty of time to prepare, so we’ve begun communicating what changes you may need to make to your Notes from Chad listings” http://www.etsy.com/blog/en/2011/keeping-it-relevant-default-search-is-now-sort-by-relevancy/ August 1, 2011 A note from our CEO April May June July August September October
- 32. August 9, 2011 Switch flipped to relevancy April May June July August September October
- 33. September 1, 2011 Surprise! April May June July August September October
- 35. September 20, 2011 Search ads available for purchase April May June July August September October
- 36. September 28, 2011 Search ads go live April May June July August September October
- 37. Takeaways
- 39. Understand your customers
- 40. Communicate Set
- 42. Resources Code as Craft blog http://codeascraft.etsy.com/ Slides (including these) http://www.slideshare.net/etsy Github http://github.com/etsy/ Join in the fun http://www.etsy.com/careers/
Notas do Editor
- \n
- Last summer, Etsy changed its default sort ranking from recency sorting to relevancy sorting. In many levels, the change was seemingly innocuous, consisting of a few changed pixels and a couple of lines of code.\n
- Behind the scenes, the change involved a substantial effort, including collaboration between engineering, product, marketing, support, and finance.\nA big part of the change was the launch of Etsy’s first performance-based keyword ad system called search ads. Search ads inventory today is sold out.\n
- An overview of what I’ll be covering today.\n
- Etsy is the world’s handmade marketplace\n
- \n
- Etsy was founded in June of 2005. The site offered basic functionality to buy, sell, and search.\n
- Etsy’s initial search technology wasn’t too much different from this. While simple, this form of searching could easily be gamed by sellers by simply “renewing” their item, which updated their creation_tsz timestamp.\n
- An entire culture and cottage industry grew around keeping your creation_tsz as recent as possible.\n
- As Etsy grew, we encountered scaling issues. Our search was modernized in late 2008. However, seller behavior around renewing and expectations around sorting by recency were set at this point. The new technology launched largely behind the scenes; the sorting behavior from a user’s perspective was unchanged.\n
- \n
- Relevancy Thursdays was a fun science experiment. We learned a lot, and not all sellers found it to be very fun.\n
- \n
- Sorting by recency provides a sub-optimal buyer experience.\n
- Renewing a listing costs $0.20, and the effect of renewing was highly unpredictable. It could vary based on category, time of day, and season.\n
- Other than search on etsy.com, sellers had very little visibility into their shop’s performance.\n
- I’m here today to talk about strategy....\n
- Improving search relevancy would lead to a better buyer experience. Moving towards black-box search sorting enabled us to take control of search and set direction.\n
- We wanted to build an advertising product that was performance based, simple to understand, and measurable.\n
- Understanding shop performance from an analytics point of view is critical\n
- So, we launched a new ad product, new seller analytics, and switched the default sort order of search. What order did we do this in? \n
- Search ads was a large undertaking and took many months of development effort. Components include keyword selection, keyword pricing, statistical problems around scheduling, billing, and the buying console\n
- An overview of what I’ll be covering today.\n
- Before making the switch to relevancy, we made several key improvements to our search ranking algorithms. \n
- We used human raters to judge improvements in search quality via a side by side testing tool that we developed in house.\n
- \n
- We ran dozens of 1% search experiments on Etsy.com and carefully measured changes in response\n
- Our first major release was shop stats. Before making any changes to search or ad campaigns, we felt it was critically important for sellers to understand the data behind their shop.\n
- When we announced our intentions to make the switch, our communications spanned several sources, including the blog and the forums. This was an important announcement and we wanted to make sure it was heard.\n
- When we announced our intentions to make the switch, our communications spanned several sources, including the blog and the forums. This was an important announcement and we wanted to make sure it was heard.\n
- Affirmation from Chad was an important part of our communications strategy. It stressed the importance of this initiative to our sellers.\n\n
- We made the switch to relevancy before releasing an advertising alternative. The lack of clear search advertising options for sellers also represented lost revenue for Etsy. We felt that this reinforced our message that this change was best for the marketplace.\nPriori to the switch, over 90% of all searches were sorted by recency. After the switch, over 90% of searches were sorted by relevance.\n
- \n
- \n
- \n
- \n
- What did we learn?\n
- Doing this wasn’t easy. It was disruptive to the marketplace. But we felt it was necessary.\n
- At Etsy, we have two primary customers.\nSellers are much more outspoken. We listened to them very carefully, and thought very hard. Some sellers were very unhappy with the change.\nBuyers are much quieter. They spoke through quantitative improvements we saw with our search changes.\n
- We communicated with our sellers early and often. We set direction both through our standard channels (blogs, forums, etc.), but also through our actions.\n
- Without great products, this would have all failed.\n
- \n
- \n