Flink Forward San Francisco 2022. To improve Amazon Alexa experiences and support machine learning inference at scale, we built an automated end-to-end solution for incremental model building or fine-tuning machine learning models through continuous learning, continual learning, and/or semi-supervised active learning. Customer privacy is our top concern at Alexa, and as we build solutions, we face unique challenges when operating at scale such as supporting multiple applications with tens of thousands of transactions per second with several dependencies including near-real time inference endpoints at low latencies. Apache Flink helps us transform and discover metrics in near-real time in our solution. In this talk, we will cover the challenges that we faced, how we scale the infrastructure to meet the needs of ML teams across Alexa, and go into how we enable specific use cases that use Apache Flink on Amazon Kinesis Data Analytics to improve Alexa experiences to delight our customers while preserving their privacy. by Aansh Shah

1. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
© 2022, Amazon Web Services, Inc. or its affiliates.
“Alexa, be quiet!”: End-to-end
near-real time model building and
evaluation in Amazon Alexa
Aansh Shah
Software Development Engineer
Secure AI Foundations (Alexa AI), Amazon

2. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
Overview
Continual Learning
Platform Overview
Kinesis Data Analytics and Flink
Use Case
Challenges
Roadmap
2

3. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
What is not Continual Learning?
3

4. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
What is Continual Learning?
4

5. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
Why?
• Adaptability
• Cost
5

6. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
How does the platform work?
6

7. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
Why Flink and Kinesis Data Analytics?
• Generate windows on streaming data
• Easily transform raw data stream into other representations
• Focus on use case logic and not infrastructure
7

8. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
Feedback Detection
8

9. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
What challenges do we face today?
• Abstraction and simplification
• Dataflow transparency
9

10. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
What’s next?
• Doubling down on our success with Kinesis Data Analytics and Flink
to replace components within our data storage
10

11. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates. 11

12. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates. 12

13. “ALEXA, BE QUIET!”: END-TO-END NEAR-REAL TIME MODEL BUILDING AND EVALUATION IN AMAZON ALEXA
© 2022, Amazon Web Services, Inc. or its affiliates.
Thank you!
© 2022, Amazon Web Services, Inc. or its affiliates.