16. Recommenders:
Tools to help identify worthwhile stuff
推薦システム:
どれに価値があるかを特定するのを助ける道具
J. A. Konstan and J. Riedl. Recommender systems: Collaborating in commerce and communities. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, Tutorial, 2003.
レコメンドとは
49. Spark processing overview
Load
Tokenize
Vectorize
Cluster
Loads text data and assign class
(assigned classes are maintained during processing)
Tokenize text and produce:
• plain tokens from Kuromoji
• tokens with keyphrases combined
Run word2vec on tokens with keyphrases
Also do PCA with 128 components
Runs t-SNE clustering on word2vec data
• project to 3D, set iterations, PCA components
(Implemented in C++ and used from Scala with Java JNI)
Visualize Explore cluster space in 3D (web browser)
• user WebGL for visualization