Web-Scale Language-Independent Cataloging of Noisy Product Listings for E-Commerce
•
1 gostou•153 visualizações
Slides corresponding to our poster at EACL 2017.
Recomendados
Recomendados
Mais conteúdo relacionado
Semelhante a Web-Scale Language-Independent Cataloging of Noisy Product Listings for E-Commerce
Semelhante a Web-Scale Language-Independent Cataloging of Noisy Product Listings for E-Commerce (20)
Último
Último (20)
Web-Scale Language-Independent Cataloging of Noisy Product Listings for E-Commerce
- 1. Web-Scale Language-Independent Cataloging of Noisy Product Listings for E-Commerce Pradipto Das Yandi Xia Aaron Levine Giuseppe Di Fabbrizio Ankur Datta Rakuten Institute of Technology – Boston | Rakuten USA Boston, MA USA
- 2. The opinions expressed are solely my own and do not express the views or opinions of my employer In the beginning… Our neighboring data vendor
- 4. The opinions expressed are solely my own and do not express the views or opinions of my employer When the vendor is sending its data, it is not sending its best… It is sending data that has lots of problems…
- 5. The opinions expressed are solely my own and do not express the views or opinions of my employer It is bringing noisy extractions http://www.neimanmarcus.com/Gucci-GG-Pattern-Stole/prod149350458/p.prod Gucci GG Pattern Stole Gucci GG Pattern Stole Brand Title
- 6. The opinions expressed are solely my own and do not express the views or opinions of my employer It is bringing noisy extractions http://www1.bloomingdales.com/shop/product/trish-mcevoy-24-hour-eye-shadow-liner?ID=485600#1178340 Trish McEvoy 24-Hour Eye Shadow & Liner Smokey Quartz Hinkley LightingBrand Title
- 7. The opinions expressed are solely my own and do not express the views or opinions of my employer It is bringing incorrect labels http://www.neimanmarcus.com/Gucci-GG-Pattern-Stole/prod149350458/p.prod Not available from product web page Clothing & Accessories > Novelty & Special Use > Work Wear & Uniforms > Military > Men > Outerwear
- 8. The opinions expressed are solely my own and do not express the views or opinions of my employer It is bringing incorrect labels Beauty Tools & Home Improvement > Lighting & Ceiling Fans > Outdoor Lighting > Post Lights http://www1.bloomingdales.com/shop/product/trish-mcevoy-24-hour-eye-shadow-liner?ID=485600#1178340
- 9. The opinions expressed are solely my own and do not express the views or opinions of my employer It is severely imbalanced Toys Home Furniture & Patio Jewelry & Watches Bag Handbags & Accessories Health Beauty & Fragrance Shoes Electronics & Computers Office Sports & Fitness Automotive Industrial Baby Products Food Baby & Kids Clothes Men's Clothing Women's Clothing Other Clothing 3.42 32.54 4.41 2.20 6.34 3.48 11.14 4.73 3.19 8.20 6.09 0.36 1.57 0.82 1.71 2.84 0.14 Number of *deduplicated* instances in raw feeds (Millions)
- 10. The opinions expressed are solely my own and do not express the views or opinions of my employer But some, we assume are good data Gradual Tan Plus Firming Four-in-One Tan Tone Firm & Moisturize 5oz Health & Personal Care > Skin Care > Sun > Self-tanners & Bronzers Brand Title http://shop.nordstrom.com/s/st-tropez-gradual-tan-plus-firming-four-in-one-tan-tone-firm-moisturize/3865243#29533668 Home > Women > Beauty & Fragrance > Spa Nordstrom > Shop Spa Products > Sunless Tanning
- 11. The opinions expressed are solely my own and do not express the views or opinions of my employer But some, we assume are good data http://shop.nordstrom.com/s/trotters-lauren-pump/3193165#24499313 'Lauren' Pump DarkGrey Brand Title Shoes > Women > Pumps Home > Women > Shoes > Comfort > Pumps
- 12. The opinions expressed are solely my own and do not express the views or opinions of my employer • Merchant cataloging errors Noise is unavoidable – bigly!
- 13. The opinions expressed are solely my own and do not express the views or opinions of my employer Noise is unavoidable – bigly! (X)Learn: Y= Women’s Shoes Comfort Shoes Pumps Sneakers Flats … Machine Learning Model 100MM Runway Felt Mule Sandals Lips Too Women's 'Too Sliver' Patent Casual Shoes Size 6 10 Crosby Women's Ynez Pump 1883 by Wolverine Women's Maisie Oxford Tan/Taupe Leather/Suede 1803 Women's 'Nome' Crocodile Dress Shoes Size 9 … x1 x2 x3 x4 x5 y1 y2 y3 y4 yL xN • We also have no control over the external vendor’s categorization accuracy
- 14. The opinions expressed are solely my own and do not express the views or opinions of my employer • Labeling noise propagates directly through manual taxonomy mapping Noise is unavoidable – bigly! Vendor BURakuten.com Books Self-help Personal growth/Gen eral Shoes Women Sandals Women’s Shoes Sandals Shoes Noise in vendor specific taxonomic organization Noise in BU taxonomy mapping ~600 leaf nodes ~10K leaf nodes
- 15. The opinions expressed are solely my own and do not express the views or opinions of my employer What classifier handles noise better? source: Elements of Statistical Learning – II, Hastie et al. Predictive power Learn lots of weaker decision trees and the optimal combination of such trees using additive stage- wise gradient boosting
- 16. The opinions expressed are solely my own and do not express the views or opinions of my employer State-of-the-art linear model 40 45 50 55 60 65 70 75 80 85 90 95 100 Baby and Kids Clothes Toys Furniture Jewelry and Watches Bags, Handbags and … Health, Beauty and … Shoes Electronics and … Office Sports and Fitness Automotive Industrial Baby Products Men's Clothing Women's Clothing ROOT Level (Level 0) MACRO/MICROPRECISION/F1 LOGISTIC REGRESSION WITH L1 REGULARIZATION ON 10% TEST SET FROM DEC 2015 DATA SNAPSHOT FROM BU2 (BUSINESS UNIT 2) MACRO F1 MICRO F1
- 17. The opinions expressed are solely my own and do not express the views or opinions of my employer Shoes • Too many listings to inspect manually – more than 3M • QA based rules: • Look for specific keywords in available breadcrumbs e.g. books, notebooks, vhs, dvd • Specific words in titles e.g. paperback, hardcover, necklace
- 18. The opinions expressed are solely my own and do not express the views or opinions of my employer Shoes – Rule based filtering Blacklisted Titles 1 Chronicles Thru Job (Hardcover) "She Was a Sister Sailor" (Hardcover) 1 & 2 Thessalonians: A Critical and Exegetical Commentary (Hardcover) 1. Fachtagung Uber Programmiersprachen: Munchen, 9.?11. Marz 1971 (Paperback) ...Keeping Clarke: One Son. One Day. One Mom's Personal Journey Through Grief (Paperback) "Messiah" from Scratch Tenor Edition (Paperback) "Lucky Streak" (Compact Disc) 10 Hymns & Gospel Songs for Medium High Voice (Compact Disc) 2030 (Compact Disc) (Un) Armed And Dangerous, Vol. 2 (VHS) 100 Years of Irish Dancing 1897-1997 (VHS) 101 Classic Goals of the World Cup (VHS) 10 Movie Mega Pack, Vol. 4 (DVD) 101 Greatest Moments From 1992 World Cup (DVD) 30-Day Guitar Workout - Book Included (DVD) 1.00 CTW Emerald Three Stone Pendant Necklace 14K Gold yellow
- 20. The opinions expressed are solely my own and do not express the views or opinions of my employer Shoes More Blacklisted Titles ! El Gato Con Sombrero Viene De Nuevo "12"".0118WIRE SIZE 2""AH STANDARD TWIST WIREWHEEL" "6"" Trampoline Springs heavy-duty galvanized (Set of 15)" (2) COLEMAN Premium Quickbed Double High Queen Airbed Mattresses w/Built-In Pump 2006 Insulin Pump Planner 2006 Ribbon HIV Calendar red 2007 Honda Civic Nighthawk Pearl B-92P Touch Up Paint Spray Can Kit - Original Factory OEM Automotive Paint black 2008 - 2010 ZX10 R Carbon Heel Plates 2013 Comprehensive Accreditation Manual for Ambulatory Care (CAMAC) (Loose-leaf) 2011 Eddie Vedder - Long Beach Concert Poster by Chuck Sperry 1 Chronicles: God's Faithfulness to the People of Judah 1.5"X30 Yards ivory 1/3 Carat Diamond Wedding Band in 10K Gold white ( Package Of 3) JO Volt 12v Single Use (12/Box) "Unleash Your Beast" Size Up, Male Enhancement Formula (3 Bottles, 180 Tablets)
- 21. The opinions expressed are solely my own and do not express the views or opinions of my employer Impossible to inspect 3.4M label, instance pairs Women’s Shoes Comfort Pumps Sneakers Sandals … 100MM Runway Felt Mule Sandals Lips Too Women's 'Too Sliver' Patent Casual Shoes Size 6 10 Crosby Women's Ynez Pump 1883 by Wolverine Women's Maisie Oxford Tan/Taupe Leather/Suede God’s Big Idea: Reclaiming God’s Original Purpose for Your Life (CompactDisc) … x1 x2 x3 x4 x5 y1 y2 y3 y4 yL xN Men’s Shoes Girl’s Shoes… Shoes
- 22. The opinions expressed are solely my own and do not express the views or opinions of my employer Take a step back and meditate! 100MM Runway Felt Mule Sandals Lips Too Women's 'Too Sliver' Patent Casual Shoes Size 6 10 Crosby Women's Ynez Pump 1883 by Wolverine Women's Maisie Oxford Tan/Taupe Leather/Suede God’s Big Idea: Reclaiming God’s Original Purpose for Your Life (CompactDisc) … x1 x2 x3 x4 x5 xN
- 23. The opinions expressed are solely my own and do not express the views or opinions of my employer Problem: Describe the modes of the latent topics, 𝜌" Generate K (<< 3.4M) labels instead! … 100MM Runway Felt Mule Sandals Lips Too Women's 'Too Sliver' Patent Casual Shoes Size 6 10 Crosby Women's Ynez Pump 1883 by Wolverine Women's Maisie Oxford Tan/Taupe Leather/Suede God’s Big Idea: Reclaiming God’s Original Purpose for Your Life (CompactDisc) … x1 x2 x3 x4 x5 xN 𝑊$ % 𝑊& % 𝑊$ ' 𝑊%( ' 𝑊$ ) 𝑊& ) 𝑊$ * 𝑊+ * 𝑊$ & 𝑊%% & 𝑍$ % 𝑄& % 𝜃%Unif(5) 𝑍$ ' 𝑄%( ' Unif(10) 𝑍$ ) 𝑄& ) 𝜃)Unif(5)𝜃' 𝑍$ * 𝑄+ * 𝜃*Unif(8) 𝑍$ & 𝑄%% & 𝜃&Unif(11) 𝜌/ Generated Labels
- 24. The opinions expressed are solely my own and do not express the views or opinions of my employer Modes of suspicious latent topics, 𝝆 𝒌 hardcover guide design handbook international health business social law science english vhs world series time king war ball house trek christmas space dog circle license standard symantec system service cisco support year essential years package upgrade heart diamond pearl 40 ring sterling chain charm 39 14k 47 bracelet watch stainless pendant i c love day e night u single lady child woman uk good deluxe park dvd jazz media mixed country artists vol play music product pop ed import john complete club paperback book history god home und soul stories journey bible der die code people set 20 100 24 case kit oz 30 hand drive body wall digital ft 48 spray paper liquid mount gift de calendar la american disc el 2013 art 2009 ray 2012 compact america great 2010 Shoes – suspicious latent topics
- 25. The opinions expressed are solely my own and do not express the views or opinions of my employer Describing modes of latent topics Modes of latent topics Taxonomy labeling of the modes oxford burgundy plain espresso wing madison [sorel bungee tip cord buffalo brogue florsheim] Oxfords > Men's Shoes > Shoes Flats > Women's Shoes > Shoes pr boots work mens blk composite [brn ee wrk comp stl prwork 8inh] Boots > Men's Shoes > Shoes Sneakers & Athletic Shoes > Men's Shoes > Shoes sneaker low top perforated marc neck [roll sapphire tank robin ruffle petite profile] Boots > Girl's Shoes > Kids & Baby Shoes > Shoes Sneakers & Athletic Shoes > Men's Shoes > Shoes tan polyurethane life saddle stride bed [lux brandy women rustic galaxy teak stu] Flats > Women's Shoes > Shoes Boots > Women's Shoes > Shoes 13 reaction york ak driving steven [rust polished ny wallet belt key unlisted] Sneakers & Athletic Shoes > Men's Shoes > Shoes Sandals > Women's Shoes > Shoes original rose bone mark lizard copper [company tobacco croc luggage classics] Climbing > Men's Shoes > Shoes Loafers > Men's Shoes > Shoes Ambiguous Maybe
- 26. The opinions expressed are solely my own and do not express the views or opinions of my employer The problem must be in the fields
- 27. The opinions expressed are solely my own and do not express the views or opinions of my employer What is the minimal relevant field of information that causes errors? Category Title Brandname Storename Breadcrumbs List price Image Sneakers & Athletic Shoes > Men’s Shoes > Shoes Walk Ultra IV DMX Max White Other brands Reebok 74.99 Slippers > Men’s Shoes > Shoes Tumi Alpha Anthracite Multi Wndow Card Case Tumi ABT Electronics Home > Watches & Fashion > Luggage & Accessories > Wallets > Men’s Wallets 65.00 Signals: Highly-Relevant Relevant Neutral
- 28. The opinions expressed are solely my own and do not express the views or opinions of my employer BU2 – Feature Engineering 82 84 86 88 90 92 94 title words title words title words [remove rare & stop words] title words with breadcrumb leaves [remove rare & stop words] title words with list prices [remove rare & stop words] title words with breadcrumb leaves & list prices [remove rare & stop words] Micro F1 Dec 2015 Feb 2016
- 31. The opinions expressed are solely my own and do not express the views or opinions of my employer Amazon (McAuley et al. 2015) 0 10 20 30 40 50 60 70 80 90 100 appliances arts, crafts & sewing automotive baby products beauty books cds & vinyl cell phones & accessories clothing, shoes & jewelry collectibles & fine art electronics grocery & gourmet food health & personal care home & kitchen industrial & scientific movies & tv musical & instruments office products patio, lawn & garden pet supplies software sports & outdoors tools & home improvement toys & games video games NB LogReg ElasticNet (OvA) LogReg L1 (OvO) GBT CNN w/ pretraining 49.013 69.393 66.647 67.171 72.659 Average Micro precision on 10% test sets for 25 top level categories Micro precision on 10% test set Total training instances: 7.4M Total test instances : 0.9M Average Level 1 KL(empirical | uniform) = 1.654 Amazon dataset from: Julian McAuley, Rahul Pandey, and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, pages 785–794, New York, NY, USA
- 32. The opinions expressed are solely my own and do not express the views or opinions of my employer Rakuten Ichiba 50.00 55.00 60.00 65.00 70.00 75.00 80.00 85.00 90.00 95.00 100.00 デジタルコンテンツ(Digital Content) 光回線・モバイル通信(Fiber Optic … インテリア・寝具・収納(Home … メンズファッション(! 'Men''s Clothing ') インナー・下着・ナイトウエア… 腕時計(Watches) ペット・ペットグッズ(Pet Food & … スポーツ・アウトドア(Sports & … 旅行・出張・チケット(Travel / … キッチン用品・食器・調理器具… キッズ・ベビー・マタニティ… ジュエリー・アクセサリー… 食品(Food) TV・オーディオ・カメラ(TV / … 車・バイク(Cars / Motorbikes) 車用品・バイク用品(Car & … 本・雑誌・コミック(Books) 美容・コスメ・香水(Beauty, … ダイエット・健康(Health & Wellness) スイーツ・お菓子(Sweets / Snacks) 医薬品・コンタクト・介護… スマートフォン・タブレット… 靴(Shoes) 日用品雑貨・文房具・手芸… おもちゃ・ホビー・ゲーム(Toys, … バッグ・小物・ブランド雑貨… レディースファッション(! … 学び・サービス・保険(Learning / … 家電(Home Appliances & Small Electrics) ビール・洋酒(Alcoholic Beverages) 花・ガーデン・DIY (Gardening & … 日本酒・焼酎(Sake) CD・DVD・楽器(Music & Video) パソコン・周辺機器(Computers & … 水・ソフトドリンク(Beverages) Micro-F1 KNN+DBN GBT • Average KL at level one – 7.88 • Average KL at level two – 3.79 Average micro-F1 at level one • KNN+DBN – 73.85 • GBT – 76.89* KNN+DBN from: Ali Cevahir and Koji Murakami. 2016. Large-scale multi-class and hierarchical product categorization for an e-commerce giant. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 525–535, Osaka, Japan, December
- 33. The opinions expressed are solely my own and do not express the views or opinions of my employer • Navigational and data organizational taxonomies should be separate – the latter should be more compact • Navigational breadcrumbs and list prices are important metadata for product classification in the wild • An ideal classification system can be constructed by mixing CNNs and GBTs to favor reduced runtimes and interpretability Main take-home messages