Asking if two items of clothing have a “similar style” is a subjective and difficult question to answer. One can attempt to use implicit signals such as purchases, clicks, and views in order to learn product similarities, like in the ubiquitous “People who bought this also bought that” section of any ecommerce product page. Such techniques prove difficult in the world of fashion where clothing can have a short shelf-life. One must then resort to using knowledge of the clothing itself in order learn these similarities. Sometimes this knowledge takes the form of images, and sometimes techniques get deep. I will discuss this journey from implicit signals to an image-based approach that we embarked on at Dia&Co, a fashion retailer. I will first start with the multiple, non-deep models that failed to produce satisfactory results. These will build the foundation and rationale for resorting to admittedly more complicated, deep learning-based methods. This represents a tradeoff between performance and difficulty. While deep learning may be extremely popular, the need for more complex infrastructure (i.e. GPUs), rapidly changing frameworks, and an arguably reduced level of interpretability are significant costs in a production system. The final, successful technique was written from scratch in PyTorch, and results from training this model on our database of product images will be presented. Lastly, details of training and serving model results via an API will be discussed. https://pydata.org/nyc2017/schedule/presentation/18/

1. PyData NYC 2017
Diving into the Deep End of
Clothing Styles
Ethan Rosenthal
Dia&Co
PyData NYC 2017 eprosenthal
EthanRosenthal
blog.ethanrosenthal.com

2. PyData NYC 2017
Nearly 70% of women in
the U.S. are plus-size,
but they represent only
16% of apparel spend.

3. PyData NYC 2017
Dia&Co is the
leading digital-first
retailer dedicated
to serving women
sizes 14+.

4. PyData NYC 2017

5. PyData NYC 2017
User Item

6. PyData NYC 2017
Styling Algorithms:
Power to the Stylists!

7. PyData NYC 2017
Item Item

8. PyData NYC 2017
Similarity is
Subjective!

9. PyData NYC 2017

10. PyData NYC 2017
Matrix
Factorization

11. PyData NYC 2017
https://github.com/benfred/implicit
Python
Easy to install
Fast AF
Industry-standard Algo

12. PyData NYC 2017
Problem - Temporal Separation

13. PyData NYC 2017
Problem - Temporal Separation

14. PyData NYC 2017
Solution? - Past <> Present
sneakers=True
sneaker_color=White
pants_fit=Relaxed
etc...

15. PyData NYC 2017
Python
Easy to install
Fast AF
MF + user and product metadata
https://github.com/lyst/lightfm

16. PyData NYC 2017

17. PyData NYC 2017
Product Metadata - Purchases + Samplings?
Product Metadata + Purchases + Samplings?
Product Metadata + Purchases + User Metadata?
Product Metadata + Purchases + Samplings
+ User Metadata?

18. PyData NYC 2017
Generating
High Quality
Metadata is Hard

19. PyData NYC 2017
What
is
Metadata?

20. PyData NYC 2017

21. PyData NYC 2017
Standing on the Shoulders of Giants
Lyst - PyData 2016 London
https://www.slideshare.net/CalvinGiles/finding-needles-in-haystacks-with-deep-n
eural-networks
ThreadGenius - NYC Machine Learning Meetup Feb 2017
https://www.slideshare.net/AhmadQamar3/using-deep-neural-networks-for-fashi
on-applications

22. PyData NYC 2017
FaceNet - Verification
?
Schroff, CVPR (2015)

23. PyData NYC 2017
FaceNet - Recognition
?
George
Jerry
Kramer
Elaine
Schroff, CVPR (2015)

24. PyData NYC 2017
FaceNet - Clustering
Schroff, CVPR (2015)

25. PyData NYC 2017
ProductNet - Clustering

26. PyData NYC 2017
Pretrained
Neural
Network
Anchor
Anchor
Vector

27. PyData NYC 2017
Pretrained
Neural
Network
Anchor
Positive
Positive
vector
Anchor
Vector

28. PyData NYC 2017
Pretrained
Neural
Network
Anchor
Positive
Negative
Positive
vector
Negative
vector
Anchor
Vector

29. PyData NYC 2017

30. PyData NYC 2017

31. PyData NYC 2017

32. PyData NYC 2017
Minimize
Maximize
Triplet Loss

33. PyData NYC 2017
Requirements
Easy to Debug
Simple Sampling
Dynamic Decision-Making
Pretrained Models

34. PyData NYC 2017

35. PyData NYC 2017
Simple to Debug

36. PyData NYC 2017
Flexible Sampling

37. PyData NYC 2017
Easily Make Dynamic Decisions

38. PyData NYC 2017
Pretrained Models

39. PyData NYC 2017
Overfit

40. PyData NYC 2017

41. Normalized
Embedding

42. PyData NYC 2017
Validate

43. PyData NYC 2017
True Positive
False Positive

44. PyData NYC 2017
Loss
Now What?

45. PyData NYC 2017
Explore

46. PyData NYC 2017
Explore
http://0.0.0.0:8777/product_style?psid=8100
www.rec-a-sketch.science (demo)
https://github.com/EthanRosenthal/rec-a-sketch (code)

47. PyData NYC 2017
Build

48. PyData NYC 2017 http://book.paddlepaddle.org/03.image_classification/
When fine-tuning, cache what you can!

49. PyData NYC 2017
Use Real Data
Don’t Mock the IO
Regression Tests
It’s hard!

50. PyData NYC 2017
Serialize

51. PyData NYC 2017
tensor.numpy()
Pretrained
Neural
Network
Learned Vector
Learned Vector
Mean
S3

52. PyData NYC 2017
Deploy

53. PyData NYC 2017
sanic

54. PyData NYC 2017
Deep Learning

55. PyData NYC 2017
Thank you
Ethan Rosenthal
Dia&Co
www.dia.com eprosenthal
EthanRosenthal
blog.ethanrosenthal.com