Stitch fix NLP internship project (+Business Development Experience Report on commercialization of my project)

2. AIM of the project: Effective human in the (ML) loop:
• Providing stylists with the best style insights for them to do the best
recommendations to clients, on top of the recommendation algorithm
How?
• Leveraging the feedback of other clients and the advice of other stylists to
help stylists write helpful and insightful notes by providing insights on every
item in the inventory

3. Project 1:
A sentence-to-garment-ID
classifier algorithm and a
Folksonomy of styles
(as a module for a note
writing assistant tool)
3

4. Results Project 1
•A sentence2garmentID classifier supervised algorithm
•A crowdsourcing task to gather a ground truth dataset of sentences
to validate the former (***)
•A folksonomy(*) of styles (totally unsupervised approach) with very
high precision and coverage:
•Positively labelled garment ids: 94.869%
•Positively labelled shipments: 88.417% (**)
•Positively labelled sentences: 24.529%
(*) Definition: Folksonomy: a (less structured than an ontology) collection of entities (garments) and
everything ever said about them
(**) +88% of the garment ids have comments, i.e., something was said about them, e.g., how to wear/style
such garment
(***) 100% coverage

5. Project 2 (this presentation):
Subjective Influence Networks
to extend the
Fashion
Knowledge
Graph
5

10. 10
State of the art on fashion ontologies

11. What existing ontologies about fashion look like
11

12. What existing ontologies about fashion look like
12

13. 13
Subjective Influence Networks
• A way to model influence in time and subjectivity for machine learning hypothesis
testing
• A mathematical framework to better frame fuzzy and subjective, aesthetics hypothesis

15. 15
Subjective Influence Networks

16. Example of
addressable ML
problems
16

20. Proprietary and confidential

21. Proprietary and confidential

22. Proprietary and confidential

24. Blending image clouds…
24

25. …with word clouds
(as FoxType does)
25

26. Subjective Influence Networks:
Evaluation and
Modelling Obligations
26

27. 27
Modelling obligations
For the model to be useful, an ontology must include:
• Enumerating at least some of the members in G, N and M
(Garment space, Styles and Mechanisms)
• Characterising a relevant set of subjective judge functions s() s.t.
s(g) = x, is a region in our theoretical set of all clothing styles
•where x is a distinct style
• Calculating, estimating or assuming values for the quads (t,i,m,a)
for edges between the nodes x in N
• Consideration of cycles: e.g. 70s’ disco fashion has come back
multiple times: explicit modelling despite being a heavily influenced
new style is important

28. 28
Modelling obligations
Types of influence mechanisms:
•Explicit: The creator of a style explicitly declares its
source of influence
•Calculated: algorithmic or other mechanical means
may estimate mechanism and strength (based on
garment features or causal cultural models)
•Extrinsic: caused by parallel cultural influences
(music, sports, religion).

29. 29
Task specific ontology expressiveness
How well the ontology maps the user’s mental model?
Statistical measures of “expressiveness” in IR:
E.g.: Given a query:
“Natural fabric buttons dress from Banana Republic”

30. 30
Task specific ontology expressiveness
• We derive a mapping between concepts that appeared in
the query and concepts encoded in the ontology.
• We ask expert judges to identify important classes ci in
the query.
• For each class ci, we ask them to map it to the most
similar label in the ontology. Then we compute the recall
of concepts in the query as
Query concept recall:

31. 31
Evaluation: Task-specific ontology expressiveness
Search result ranking:
• We use rankings of search results for a given query as a proxy of
“golden standard”
• Normalized Discounted Cumulative Gain (NDCG) metric can be
adapted to measure the ontology’s relevancy to search quality.
• The gain is quantified by the ontology’s recall of concepts in
search results. We examine the top K returned search results. For
each of them Dp at position p, we compute the ontology’s
document concept recall, which is then discounted by its
logarithmic rank log(p):

32. 32
Evaluation quality metrics to assess the ontology’s quality
How well the ontology approximates empirical data in
the fashion domain?
•Categorical precision: how many styles encoded in
the influence network are real-world recognisable
styles?
•Temporal bias (time-invariance of features): the
length of time span before the divergence is the
representativeness timescale of the network, and the
scope indicates its robustness

33. 33
Evaluation quality metrics to assess the ontology’s quality
• Semantic similarity: This measurement provides a distance metric between a traditional
ontology augmented with an influence network and the text data in fashion domain in terms of
“meanings" they express. We project both labels (class names Oc and property names Op) in
the ontology and the tokens in the document corpus D in the same vector space (e.g., using
word2vec). Then we compute the overall similarity based on all labels' distances weighted by
their importance within the network:
The importance score θ(ci) is a normalized score [0, 1], can be defined depending on the
context and usage (e.g. θ(ci) refers to how knowledgeable a network is w.r.t. class ci, which can
be approximated by the cumulative distribution function of its number of attributes ui:

35. Business Development
Experience:
Can this technology be
monetized?
35

36. 36
Consumption of Subjective Influence Networks by ML systems
General ML problems:
• NLP as a service: e.g., NER, POS, KG.
• Fashion data retrieval: integrations with schema.org,
Freebase or other publishers to build:
• Crowd-sourcing systems
• Search engine indexes
• Recommender system:
• Taxonomy building as a service: to learn KG,
bottom-up approach
• Deal with sparsity and cold start problem.

37. Business model proposals:
• Real-time advice giving from stylists with a snapchat-
like UI Bot/Human assistant [The rise of the social bots,
ACM, 16]
• Scaling a very domain specific writing assistant
• Bot style checker for a look in a fix (box)
Business Modelling
37

38. Business Modelling
38

39. Business Modelling
39

40. 40

41. Some survey
answers…

42. Future life
42

43. • getAsteria.com
• Learning Deep Learning
• through practical or fun projects (involving music, parasites and stars)
Current sabbatical time & futuristic projects
43

44. Thank you!
diaz.rodriguez.natalia@gmail.com
MORE DETAILS:
• Paper
• Poster
• Stitch Fix Technology Blog: http://multithreaded.stitchfix.com/
44