Published on Nov 2, 2018 This talk was recorded in London on October 30th, 2018 and can be viewed here: https://youtu.be/erHt-1yBuUw Session: Travelport is a leading travel commerce platform that has truly huge data and many complex needs in terms of processing, performance and latency. This talk will demonstrate how we were able to harness big data technologies, H2O and cloud integration to deploy AI at scale and at low latency. The talk to cover practical advice taken from our AI journey; you will learn the successful strategies and the pitfalls of near real-time retraining ML models with streaming data and using all opensource technologies. Bio: As principal data scientist at Travelport, Levi Brackman leads a team of data scientists that are putting ML model into production. Prior to Travelport, Levi spent most of his career in the start-up world. He founded and led an organization that created innovative educational software applications and solutions used by high schools and youth organizations in the USA and Australia. Levi earned a PhD in the quantitative social sciences under the supervision of one the world's leading educational psychologists. He earned master’s degree from University College London and is author of a business book published in eight languages that was a bestseller in multiple countries. A native of North London (UK) Levi is married and has five children and now lives in Broomfield, Colorado.

1. Near Realtime AI
Deployment with Huge
Data & Super Low Latency
Levi Brackman
Principal Data Scientist
Travelport
@levibrackman
linkedin.com/in/levibrackman/

2. Motivation, Challenges and Solutions
1. Model Decay
2. Volume of training data
3. Time series data
4. Compute necessary
5. Scoring latency
6. DevOps

3. • Cloud or OnPrem?
• Real time streaming data or batch?
• Scoring mechanism?
• REST API or DLL/Jar?
Infrastructure Considerations

4. • You need an entire cast of talent
• Big data Engineers
• Software Engineers
• Buy-in from other business applications
Talent – Not Just Data Scientists

5. • Can you get enterprise support for the software?
• How easy is it to deploy?
• What is the prediction latency?
• This is where H2O really shines
• Do you want to deploy this on-prem or in the cloud
• Can your applications that work in your data center intergrate with
cloud ML deployments?
Technology Decisions

6. • The goal is to operationalize, automate and push an ML and AI
solution to production
• Grid search therefore needs to be automated and run periodically and
the results automatically incorporated into the model building
process
• Monitoring the performance of models is also vital
• Determine your own model success metrics based on the business
case
• Model type and feature engineering must take productionalization
into consideration
Keep Productionalization in Mind Throughout

7. • H2O cluster stand alone or with Sparkling Water?
• ETL at scale
• How to transfer data (streaming or batch)?
• What TPS will you need for scoring?
• How will scoring API scale?
Productionalization cont.

8. • Source control
• Automated Build and Deploy (Jenkins)
• CI/CD (TBD)
• Monitoring
Dev-ops for Data Science

9. Near Real-time retraining of ML models