O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Predicting the “Stars of Tomorrow” on Social Media

1.671 visualizações

Publicada em

People are interested in predicting the future. For example, which films will bomb or who will win the upcoming Grammy awards? Making predictions about the future is not only fun matters but can bring real value to those who correctly predict the course of world events, such as which stocks are the best purchases for short-term gains. Predictive analytics is thus a field that has attracted major attention in both academia and the industry. As social media has become an inseparable part of modern life, there has been increasing interest in research of leveraging and exploiting social media as an information source for inferring rich social facts and knowledge. In this talk, we will address an interesting and challenging problem in social media research, i.e., predicting social media popularity. We aim to discover which image posts on social media are the “stars of tomorrow”, those will be the most engaging for social media audiences, e.g., receiving the most likes. Sociological finding and our novel solutions to effectively develop a structured modeling for popularity dynamics will be presented.

Publicada em: Tecnologia
  • Hi there! Get Your Professional Job-Winning Resume Here - Check our website! http://bit.ly/resumpro
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Seja a primeira pessoa a gostar disto

Predicting the “Stars of Tomorrow” on Social Media

  1. 1. Predicting the “Stars of Tomorrow” on Social Media Wen-Huang Cheng (鄭文皇) Multimedia Computing Lab (MCLab) Research Center for Information Technology Innovation (CITI), Academia Sinica, Taipei, Taiwan whcheng@citi.sinica.edu.tw Presented at on 10 May 2017
  2. 2. 2
  3. 3. Academia Sinica (中央研究院) • The highest national research institute in Taiwan – with about 1,000 professors (60 in EE/CS) 3 located in Nangang, Taipei
  4. 4. Multimedia Computing Lab (MCLab) 4 http://mclab.citi.sinica.edu.tw
  5. 5. We are social… 5 Real World Digital World
  6. 6. Nanit Baby Monitor
  7. 7. 9 Social Signals
  8. 8. Leading Social Networks 13 [Ref] http://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/
  9. 9. 14
  10. 10. Sociology and Human Interaction • With the huge number of people who are involved nowadays with social networks, it is very interesting to note how they are influenced by each other in many different ways. – e.g., identity in the age of social media 15 [Ref] http://edition.cnn.com/2015/10/05/health/being-13-teens-social-media-study/index.html
  11. 11. 100 years after 100 years ago
  12. 12. Social Popularity Prediction • General Popularity Prediction: Predicting the popularity score of a new social media post by combining post content (photo, text or video) and user cues 17 Score: 4.9 ? Model A new post Predicted Popularity Training Images 5.6 2.3 5.1 2.8 7.8 3.1 History data
  13. 13. Why is it important? • wide applications and high business value – e.g., predicting the “Stars of Tomorrow” (top popular models) within the fashion Industry using social media 18 [Ref] “Style in the Age of Instagram: Predicting Success within the Fashion Industry using Social Media,” CSCW 2016. Fashion Model Directory (FMD) profile page Can you tell who will be the “top”?
  14. 14. People are desired for knowing the future… 19 [Ref] https://www.oreilly.com/ideas/inside-the-washington-posts-popularity-prediction-experiment
  15. 15. 天池大數據競賽 20 https://tianchi.shuju.aliyun.com/competition/index.htm
  16. 16. 21
  17. 17. 22 經過7年的發展與沉澱,目前阿里音樂擁有數百萬的曲庫資源,每天千萬的用戶活躍在平台上,擁有數億人次的用戶試聽、收 藏等行為。在原創藝人和作品方面,更是擁有數萬的獨立音樂人,每月上傳上萬個原創作品,形成超過幾十萬首曲目的原創 作品庫,如此龐大的數據資源庫對於音樂流行趨勢的把握有著極為重要的指引作用。本次大賽以阿裡音樂用戶的歷史播放數 據為基礎,期望參賽者可以通過對阿裡音樂平台上每個階段藝人的試聽量的預測,挖掘出即將成為潮流的藝人,從而實現對 一個時間段內音樂流行趨勢的准確把控。
  18. 18. 競賽數據 23
  19. 19. Our Related Publications • “Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks,” IJCAI 2017. • “Time Matters: Multi-scale Temporalization of Social Media Popularity,” ACM Multimedia 2016 (full paper). • “Unfolding Temporal Dynamics: Predicting Social Media Popularity Using Multi-scale Temporal Decomposition,” AAAI 2016. • “SocialCRC: Enabling Socially-Consensual Rendezvous Coordination by Mobile Phones,” Pervasive and Mobile Computing, 2016. 24
  20. 20. What Makes A Post Popular? 25 [Ref]“What Makes an Image Popular?” WWW, 2014.
  21. 21. What Makes A Post Popular? • Features for prediction – Post content • e.g., visual sentiment features (color and texture) 26 [Ref]“Analyzing and predicting sentiment of images on the social web,” ACM Multimedia 2010.
  22. 22. What Makes A Post Popular? • Features for prediction – User cues • e.g., followers (a user’s follower count), friends (how many users a user follows), statuses (a user’s current total post count), user time (a user’s account creation time), etc. 27 A friend graph: 6 1
  23. 23. What Makes A Post Popular? • Features for prediction – User cues (topological features) • e.g., closeness centrality, the average length of the shortest path between the node and all other nodes in the graph 28 1 2 3 4 5 6 7
  24. 24. Closeness Centrality 29 1 2 3 4 5 6 7 1 2 3 4 5 6 7 0.5 0.67 0.75 0.46 0.75 0.46 0.46
  25. 25. Latent Factor Models • The popularity prediction task is formulated as a matrix completion problem of filling in the missing entries of a partially observed matrix. 30 known popularity to be estimated
  26. 26. Our Observations: Time Matters 31 [Ref] http://www.adweek.com/socialtimes/best-time-to-post-social-media/504222
  27. 27. Temporal Modeling for Popularity • To incorporate the temporal evolving structures in popularity prediction 32
  28. 28. • The popularity evolving at multi-granularities with different patterns 33 Challenge 1: Temporal Evolving Multi-granularities Characteristics of Popularity Dynamics
  29. 29. Challenge 2: Data Noise • Popularity patterns are covered in very noisy behavior data or information Popularity distribution on time series
  30. 30. Our Solution#1 [AAAI’16]: Incorporating Multi-Scale Temporal Decomposition 35 popularity matrix time scales Solver: Multiple Update Rule (D.Lee and Sebastian.Seung 1999)
  31. 31. Datasets • Data Sets – Over 1.8M photos – Over 70K users – Views, User profile, Photo stream – Metadata, Images, Annotations • Settings 36 User-specific Dataset (UsD) Users 400 Images 600K Photo-mix Dataset (PmD) Users 70K Images 1200K
  32. 32. Experiments • Metric: Spearman Correlation • Time scales: – period, week, month, season • period: “morning (8:00am-12:00am)”, “lunch time (12:00am-14:00pm)”, “afternoon (14:00am-17:00pm)”, “dinner time (17:00am- 20:00pm)”, “evening (20:00am-24:00pm)” and “sleeping (0:00am-8:00am)” 37
  33. 33. Our Solution#2 [MM’16]: A Multi-scale Temporalization (MT) Framework 38
  34. 34. Algorithm: Multi-scale Temporalization (MT) 39 Optimization Updating Steps Optimization Updating Steps
  35. 35. Experiments 40
  36. 36. Our Solution#3 [IJCAI’17]: Deep Temporal Context Networks (DTCN) • We address the problem as a sequential prediction task, where the input is a user-photo sequence (with time order) while the output is the popularity of a “future” photo (a photo before its publication on social media) 41
  37. 37. Experiments • Prediction performances on TPIC17-100K, 200K, and 400K datasets – Metric: Spearman Ranking Correlation 42
  38. 38. More Influential Factors: Cultures • A voting survey of the 2014 TripAdvisor's Top 10 Attractions in Japan by visitors from different countries shows how much the favorites for attractions can vary among people from different regions, i.e., different cultures. 43 [Ref] 2014 TripAdvisor’s Top 20 Attractions in Japan: http://www.tripadvisor.com/pages/- HotSpotsJapan.html.
  39. 39. Foursquare Dataset https://sites.google.com/site/yangdingqi/home/foursquare-dataset • Individual “check-ins” data of the more than 10 million users on Foursquare 44
  40. 40. A Pilot Study: Understanding Foursquare Venue Popularity in Taiwan 45 Performed by Mr. Mrinal Kanti Baowaly in 2016
  41. 41. Taiwan vs. USA – Venue Distribution of Top 10 Categories 46 Taiwan USA
  42. 42. More Influential Factors: Personalization • What Your Facial Features Say About Your Personality (MM13) 47 personality report facial image
  43. 43. Learning Relevance by Neighbor Voting 48 [Ref] X. Li, C.G.M. Snoek, M. Worring, “Learning tag relevance by neighbor voting for social image retrieval,” Proc. ACM Intl. Conf. Multimedia Information Retrieval (MIR), 2008.
  44. 44. More Influential Factors: Personal Fashion Flavor 49 [Ref] “Fashion Analysis: Current Techniques and Future Directions,” IEEE Multimedia, 2014.
  45. 45. Urban Tribes: Analyzing Group Photos from a Social Perspective [CVPR’12] 50 Urban tribe: the term to describe subcultures of people who share common interests and tend to have similar styles of dress, to behave similarly, and to congregate together. (coined by French sociologist Michel Maffesoli in 1985) Which groups of people would more likely choose to interact socially? (a) and (b) or (a) and (c)?
  46. 46. Clothing Fashion Analysis 51  "i-Stylist: Finding the Right Dress Through Your Social Networks," MMM 2017.  "A Framework of Enlarging Face Datasets Used for Makeup Face Analysis," BigMM 2016.  "What are the Fashion Trends in New York?" MM 2014. (Grand Challenge Prize)  "Clothing Genre Classification by Exploiting the Style Elements," MM 2012.
  47. 47. Clothing fashion is a reflection of the society of a period • The global fashion apparel market today has surpassed 1 trillion US dollars since 2013, and accounts for nearly 2 percent of the world's Gross Domestic Product (GDP) 52
  48. 48. Trend Analysis for the Clothing Fashion Our work received “Multimedia Grand Challenge Award” in 2014 ACM Multimedia Conference.
  49. 49. Applications: “Fashion is becoming mobile first with apps that help track down must-have clothes, accessories and shoes” - theguardian.com LIKEtoKNOW.it The Netbook Snap Fashion The Hunt
  50. 50. http://www.fashiontv.com/videos/fashion-weeks Construct a fashion show dataset Source: New York Fashion Weeks
  51. 51. Color Cut Pattern Head decoration major elements for fashion style investigation key factors for discovering fashion trends:  coherence (frequently occur within a fashion week)  uniqueness (occur much more often in a fashion week than in other fashion weeks)
  52. 52. http://www.fashiontv.com/videos/fashion-weeks Detect the presence of catwalk models over all video frames . . . . . . . . . . . . . . . . . . http://www.fashiontv.com/videos/fashion-weeks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identify distinct catwalk models
  53. 53. . . . . . . . . . . . . . . . . . . Identify distinct catwalk modelsExtract model location and the full-body image
  54. 54. Collect full-body image of catwalk models Catwalk Models e.g. NYFW Autumn/Winter 2014 Positive set  Negative set  e.g. all catwalk models at NYFW  except for Autumn/Winter 2014 Divide the collection of full-body images into two sets Distributional clustering technique W.‐H. Cheng et al., "Learning and Recognition of On‐Premise Signs (OPSs) from Weakly  Labeled Street View Images," IEEE Tran. on Image Processing (TIP), 2014.
  55. 55. Query Image Query Image Color Analysis Texture Analysis Color + Texture Analysis Query ImageQuery Image Color Analysis Texture Analysis Color + Texture Analysis Query Image Query Image Query Image Color Analysis Texture Analysis Color + Texture Analysis Query Image Query Image Query Image Color Analysis Texture Analysis Color + Texture Analysis Query Image Query Image Query Image Spring/Summer 2011 Spring/Summer 2013 Spring/Summer 2013 Spring/Summer 2011 Spring/Summer 2013 Spring/Summer 2013
  56. 56. Predicting Occupation via Human Clothing and Contexts [ICCV’11] • Diving into the recognition of high-level semantic categories of human such as occupations 61
  57. 57. Recognizing City Identity via Attribute Analysis of Geo-tagged Images [ECCV’14] • A set of 7 high-level attributes is used to describe the spatial form of a city (amount of vertical buildings, type of architecture, water coverage, and green space coverage) and its social functionality (transportation network, athletic activity, and social activity). 62
  58. 58. From Scene Attributes to City Attributes • 102 scene attributes are defined. • Each of the city attribute classifier is modeled as an ensemble of SVMs. 63
  59. 59. Spatial Analysis of City Attributes • The city perception map visualizes the spatial distribution of the 7 city attributes in different colors and exhibits the visitors’ and inhabitants’ own experience and perception of the cities, while it reflects the spatial popularity of places in the city across attributes. 64
  60. 60. Attribute-Based City Identity Recognition 65
  61. 61. 66 Sociological understanding of humans and human interactions is fun but still a long way to go!
  62. 62. 67 ACM Multimedia 2017 http://www.acmmm.org/2017/ Grand Challenge Social Media Prediction (SMP): Predicting the “Stars of Tomorrow” on Social Media https://social-media-prediction.github.io/MM17PredictionChallenge/ Organizers Wen-Huang Cheng Academia Sinica Bo Wu Chinese Academy of Sciences Yongdong Zhang Chinese Academy of Sciences Tao Mei Microsoft Research Asis
  63. 63. 68
  64. 64. Yahoo! Dataset http://webscope.sandbox.yahoo.com/ 69
  65. 65. YFCC100M • This dataset contains 100 million media objects and explain the rationale behind its creation. This list is compiled from data available on Yahoo! Flickr. 70 Two photos of real world scenes from photographers in the YFCC100M dataset.
  66. 66. YFCC100M • Global coverage of a sample of one million photos from the YFCC100M dataset. 71
  67. 67. Yelp Dataset Challenge https://www.yelp.com/dataset_challenge 72
  68. 68. Visual Genome Dataset https://visualgenome.org/ 73
  69. 69. Instagram Dataset http://www.emilio.ferrara.name/datasets/ 74
  70. 70. ICWSM-16 Dataset International Conference on Web and Social Media • http://www.icwsm.org/2016/datasets/datasets/ 75
  71. 71. 76 General Chairs Program Chairs Wan-Chi Siu Hong Kong Polytechnic University Chia-Wen Lin National Tsinghua University Wen-Huang Cheng Academia Sinica Gene Cheung National Institute of Informatics vcip2018.org
  72. 72. Let’s exchange ideas! 77 whcheng@citi.sinica.edu.tw Wen-Huang Cheng wenhuangcheng