Real-time Machine Learning with Hopsworks

1. Real-time Machine Learning with Hopsworks An integrated Feature Store and Model Serving platform Jim Dowling - CEO

2. ML Operational Capabilities Business Value Online predictions Batch updates Offline predictions Batch updates Traditional Analytics Training/Test Data Analytical ML Operational ML Real-Time Machine Learning Where business value is generated in AI Online inference Batch features Offline inference Batch features Model Serving Online Feature Store Batch jobs Offline Feature Store Model Serving Online Feature Store Online inference Streaming features Online predictions Real-time updates

3. Data warehouse Applications - Services Search, Versioning, Statistics, Code Lineage, Provenance Feature Views Model Registry Feature Groups Online Applications & Services KServe Feature Store Models Where Feature Stores and Model Serving meet

4. Feature Groups Feature Views Batch (DataFrames) Read Feature Vectors Online API Read Files/DataFrames Oﬄine API Streaming (Data Instances) Models Feature Store Transformer Prediction Service Predictor Model Artifact Online Predictions REST API Model Registry Deploy Inference logs (Data Instances) Model Serving Code Model ﬁles Model Server Inference Logger From Raw Data to Online Predictions Search, Versioning, Statistics, Transformations Lineage, Provenance Versioning, Experiments, Metrics, Code Canary, A/B Testing

5. Keeping Your Pipelines on Track Model Registry Batch Apps Online Apps Feature Groups Feature Views Vector DB Training Pipelines Inference Pipelines Online Oﬄine Model artifact Index Creation Encoder schema transformation functions versioning versioning versioning experiments versioning schema schema schema ✓ Versioning → ■ code : feature eng., transformation functions, model training, model serving scripts ■ assets: model files, model artifacts, experiments ■ configuration: experiment settings, deployments, indexes ✓ Schema management → columns, data types // fg, fv, models, deployments ✓ Transformation functions → avoid training / serving skew ✓ Provenance and Lineage → track predictions down to the ingested features Provenance versioning Data warehouse (historical data) Applications, Service (context, trends) Feature Pipelines Batch Streaming versioning

6. A Closer Look to Inference Pipelines Data warehouse (historical data) Model Registry Batch Apps Online Apps Feature Groups Feature Views Applications, Service (context, trends) Feature Pipelines Vector DB Batch Streaming Training Pipelines Oﬄine Index Creation Encoder Model artifact Batch Inference Jobs Prediction Service Transformer Predictor Model artifact Online Recent features Embeddings Online predictions Inference logs Inference logger Batch data Batch predictions

7. Feature Store Inference Request Streaming Feature Pipeline Feature Group FG 1 FG 2 FG 4 FG 3 Feature View FV 1 FV 2 FG 5 FV 3 Features Feature 1 Feature 2 Feature 4 (pk) Feature 3 Feature 5 Feature 6 Feature 7 (pk) Feature 9 Feature 8 Model Serving Transformer Feature 4 (pk) Feature Vector Vector DB Embedding Embedding Embedding Embeddings Predictor Embeddings Model Input Inference Response Prediction Prediction Embedding space Online Apps Similarity search Feedback Lookups Inference logs Model A Deeper Look to Real-time Inference Pipelines mapping

8. Real-Time, Personalized Recommendation Systems Candidate Retrieval and Ranking

9. Embedding User-Query Encoder Features Embeddings compress high dimensional data, retaining semantic relationships current user search user session data user purchases user profile

10. What about Multi-Modal Similarity Search? Can a “user query” ﬁnd “items” with similarity search? Yes, by mapping the “user query” embedding into the “item” embedding space with a two-tower model. Representation learning for retrieval usually involves supervised learning with labeled or pseudo-labeled data from user-item interactions.

11. Training data for our Two-Tower Model will be User-Item Interactions Log user-item interactions as training data for our two-tower model and ranking model. Retail Website Search Item 1 Item 2 Item 3 Item 4 Purchase 3 Click 2 Click 3 Score: 0 Item 1 Score: 1 Item 2 Score: 5 Item 3 Score: 0 Item 4 Features Features Features Features

12. Training the Two-Tower Embedding Modoel User Query embeddings User Query encoder Item embeddings Item encoder Item category, price, popularity, etc User features, preferences, history Dot product (Loss fn) 0 → Non-interaction LOSS 1 → highest interaction User-Item Interactions Training Data

13. Model Training for Embedding Models and Ranking Model Feature Views items user queries Feature Store Training Data retrieval.csv ranking.csv Ranking User/Query Embedding Item Embedding Hopsworks Model Registry Train Models Train Models Models item user clicks

14. Build the ANN Index on Items. Similarity Search with user queries on it. OpenSearch k-NN (ANN Index) items.csv Job computes embeddings for all Items Encode all items Insert all pairs (item-ID, embedding)

15. Two-Tower Network with a Vector Database for ANN Search Source: https://cloud.google.com/blog/products/ai-machine-learning/vertex-matching-engine-blazing-fast-and-massively-scalable-nearest-neighbor-search

16. Retrieval and Ranking for Personalized Real-time Recommendation Systems User-Query Embedding User-Query Encoder Features Candidate Retrieval Ranking Model Ranked items Hopsworks Feature Store OpenSearch k-NN (items) Candidate Items Trends, Feedback Search Get Features for items Features

17. Real-time Recommendation Systems Query Model Retrieve closest candidates using similarity search Enrich with features for candidates Ranked candidates Recommended candidates Ranking Model Candidate 1 Candidate 2 Candidate N Recommendation request Enrich with item/user features

18. Real-time Recommendation Systems with Hopsworks User Query Model Retrieve closest candidates with similarity search Enrich with features for candidates Recommendation request Recommended candidates Enrich with item/user features Ranking Model Ranked candidates Candidate 1 Candidate 2 Candidate N Hopsworks Feature Store Predictor Predictor KServe Deployment OpenSearch K-NN KServe Deployment Transformer Transformer

19. Extended Retrieval and Ranking Architecture Embeddings, Retrieval, Filtering, Ranking Jointly train with two-tower model: User/query embedding Item embedding models Built Approx Nearest Neighbor (ANN) Index with items and item embedding model. User/Query & Item Embeddings With a ranking model, score all the candidate items with both user and item features, ensuring, candidate diversity. Ranking Remove candidate items for various reasons: • underage user • item sold out • item bought before • item not available in user’s region Filtering Retrieve candidate items based on the user embedding from the ANN Index - similarity search Retrieval

Real-time Machine Learning with Hopsworks

Esté a ler uma amostra.

Confira estes a seguir

Real-time Machine Learning with Hopsworks

Mais Conteúdo rRelacionado

Semelhante a Real-time Machine Learning with Hopsworks (20)

Mais recentes (20)