This document summarizes Liz Graefe and Matt Nicole's presentation on selling taxonomy as a product at Etsy. They discuss their goals of adding filters to Etsy like other ecommerce sites, but finding their initial gatekeepers to be Etsy sellers who did not categorize or attribute listings well. They then worked to better communicate with and support sellers. While making progress, they continue to face challenges around data and analytics. Their lessons are to focus on user needs, start small, and prioritize understanding product data.

1. 1
Liz Graefe & Matt Nicole
March 16, 2019
Know Your Gatekeepers
How to Sell Taxonomy
as a Product
Agenda
1 - Intros
2 - What were we trying to do?
3 - Finding our gatekeepers
4 - Appeasing our gatekeepers
5 - Where are we now?
6 - What would we do different?

2. 2
MattLiz

3. Etsy is a marketplace
Etsy is home to a universe of special, extraordinary items,
from unique handcrafted pieces to vintage treasures.
Our mission is to Keep Commerce Human. We believe that
our sellers are the experts about their wares, and we
empower them with control over their content.
Translation: Anyone can list anything* and they can call it
whatever they want.
3
*Almost. Contrary to that Facebook meme, you can’t actually list
your children.

4. Anything & Everything
4

5. 5
What were we
trying to do?
● Add filters to Etsy
● Just like every other ecommerce website

6. Filters
● Filters are built on Attributes
● Attributes are built on Categories
● Categories & Attributes are populated by
Listings
● Listings are created by Sellers

7. 7
Our First Gatekeepers:
Etsy Sellers

8. 8
How We
Taxonomy
at Etsy
We don’t enforce categorization.
We provide a hierarchy for the seller to classify
their products, but classification can happen at
almost any level, and accuracy is not enforced.
Until recently, it was even possible to classify a
product at the top level.
We don’t require attributes.
We provide category-specific structured attributes,
but we rarely require sellers to fill them out.
We allow sellers to vary by anything.
While we do offer structured variation attributes, sellers
have the ability to build their own variation attribute
using their own terms.

9. Family Llama T-shirt by
Good4theGoose
If our structured metadata isn’t meeting a seller’s
needs, in many cases they have the power to create
their own attributes and values.
Sellers will find a way.

10. 10
We can’t turn on filters if sellers don’t fill out
attributes.
But sellers won’t fill out attributes if they don’t see
what we’re using them for.
Chicken, meet Egg.
Any plan to launch filters was going to need
communication to sellers so they would understand
the intent behind our actions.

11. 11
Change
Communication
It all begins with data.
In order for any taxonomy change to have a chance
at success, the data has to be there to support it.
1. Products have to be classified into
categories.
2. Categories need attributes that meet the
specifications of the products and the
needs of the users
3. Attributes have to be filled out correctly.
If the data powering your taxonomy comes from
other people, make sure they know how to use it
correctly. If something is changing, make sure
they’re prepared.
Changes need to be
communicated.

12. 12
Our team ● Product Manager
● Dedicated Designer
● Fully Staffed Engineering Team
● Dedicated Analyst
● Taxonomist

13. 13
Our Second Gatekeeper:
Staffing

14. 14
Our team ● Product Manager
● The role of designer
● The role of Engineering Team
● The role of Analyst
● TaxonomistActual

15. Nerd Sniping
xkcd.com/356

16. Enumerate the
Roles
I am not an astronaut.

17. 17
Be the product
manager ● There are Different kinds
○ 6 types
○ 3 different types
○ 7 other kinds
● Product management is about
building the right things for the right
people.
○ Lower risk, effort, friction
● Many myths

18. 18
Where are we
going?
● Maps
● Navigating

19. Maps
Define the frontier

20. Maps
Find the dragons

21. Maps
Slay the first dragon first.

22. Maps
The perfect is the enemy of the good.

23. Maps
Tell people where you can’t see the
trees through the forest.

24. 24
Our Third Gatekeeper:
Engineering Complexity

25. Navigating
The map is not the terrain

26. Navigating
Find the critical path things.

27. 27
How are we
going to get
there?
● Who is “we”?
● We work together

28. “We”
Gatekeepers are people too.

29. Better “We”
Spreadsheet to the promised land

30. 30
Our Fourth Gatekeeper:
Organizational
Complexity

31. “We”
Humans are messy
"Regardless of what we discover, we understand
and truly believe that everyone did the best job
they could, given what they knew at the time,
their skills and abilities, the resources available,
and the situation at hand."
Norm Kerth, Project Retrospectives: A Handbook for
Team Review

32. Scarcity
What is value?
I’m in your cart.

33. 33
Selling your product
manager
● Talk about the Product Triangle
○ Users (personas)
○ Business (canvas)
○ Technology (lots of links here)
● Talk in impact
○ “We can move this metric” vs
○ “I want to build this thing”
○ Some starter metrics
● Know what a good one looks like
○ SVPG
● Create specific user stories
○ As a taxonomist (or other
persona)
○ I’d like to be able to (some
capability)
○ In order to (make some impact)

34. Shared “We”
GMS
Gross Merchandise Sales

35. 35
Culture of
Experiments
● Does anyone want this?
● Will this contribute to our shared goals?
KNOW YOUR GATEKEEPERS

36. 36
Our Fifth Gatekeeper:
Reality

37. Listen to the users.
“I come to Etsy when I have
time to kill.”
“I will literally click through
250 pages of results in order
to find what I’m looking for.”

38. “I want shoes in the most
popular size.”
Listen to the users.

39. No one has ever said this. “I want shoes in the most
popular size.”

40. 40
Issues with
Past
Experiments
Traffic
In order to achieve significance in a timely fashion,
we created filters that were broad and applied to all
categories. And when that didn’t work, we tried
adding filters to top-level categories.

41. Holidays &
Occasions

42. 42
Issues with
Past
Experiments
Traffic
In order to achieve significance in a timely fashion,
we created filters that were broad and applied to all
categories. And when that didn’t work, we tried
adding filters to top-level categories.
Ordering
Values displayed in order of highest inventory.
Filters displayed in the order that they were created
in. We had no control over how our data was being
displayed.
Listing Adoption
In order for filters to succeed, the values need to meet a
user’s needs, but so does the assortment that those
values return. If we build a new attribute, it can take
months for sellers to update existing listings or create
new ones that fill them out.

43. 43
If our overall goal was just “proving that filters are a win”
or just “getting filters defined everywhere regardless of
what they are,” then we’d created a vanity metric that
would lead us to develop products that aren’t useful.
Chasing a “win” was only ever going to lead us to failure.

44. 44
An Experiment
that Worked
The Right Filters Everywhere
At this point we’d been creating attributes for a few
years. Some of our highest-trafficked verticals had
category-specific filters with good coverage at all
levels of the taxonomy.
Filters vs. No Filters
This time we didn’t try to test the experience on a
single vertical. We went full A/B. Our users would
either see all of our filters, or they wouldn’t see any.
Old Filters vs. New Filters
The experiment closely followed the release of a
redesign of the Clothing vertical that included a lot of
brand new attributes. We took the opportunity to
include these filters, knowing that they might not have
the best adoption.

45. 45
Analytics ● What’s going on in there?
● Now we know.

46. 46
Our Sixth Gatekeeper:
Entropy

47. In Need of
Analytics
● How many of our products use the value April Fools’?
● How many active listings are in Unisex T-shirts?
● How many people use the filters on Wall Art?
● What are the top search queries for Slime?
● How many people browse vs. search?
In the past, we had analytics from
experiments, which weren’t that
helpful. If we wanted other insights
we had to ask an analyst and wait
until they had time to send us a one-
day snapshot of the data we needed.

48. We Built
Ourselves Some
Tools

49. 49
Where are we now?
Questions we still can’t answer:
● When users click on filters, which values
are they choosing most often?
● Is there any impact to filter engagement
or seller attribute adoption if we reorder
our attributes?
● How do our listings change over time?
● How often are our listings recategorized?
Problems we still face:
● The code that classifies search
queries to our taxonomy still
doesn’t see all categories
● We don’t have the ability to
migrate listings, which means we
can’t delete obsolete categories.
● Browse sessions are often logged
as search sessions, giving us an
incomplete view of how our
Buyers are looking for products.

50. 50
Things we wished
we’d done sooner. ● Securing access to Analytics
● Manually ordering attributes and
values
● Being able to QA changes on the site
before they go live.
● Being able to QA changes in the tools
before launching them to the site.

51. 51
We’re done.
Thanks.
Liz Graefe - @sistermischief
Matt NIcole - @d44pan