Presented at FITC Toronto 2017 More info at http://fitc.ca/event/to17/ Presented by Kevin Hughes, Shopify Overview TensorKart is an end-to-end deep learning demonstration about training an AI to play MarioKart 64 with TensorFlow. Using the same technique as Google’s self-driving cars this talk will go through the process step by step and explain it in an approachable way. By the end of the session you’ll see an AI playing MarioKart 64 and understand how it works regardless of previous experience in deep learning or programming. Objective to bring deep learning to a wider audience in an approachable way Target Audience Anyone interested in artificial intelligence or self driving cars Assumed Audience Knowledge Some basic programming knowledge or at least interest is all that is required to enjoy this talk Three Things Audience Members Will Learn What TensorFlow and deep learning are all about How to approach an AI problem using deep learning A deeper appreciation for how real self driving cars might work

1. TENSORKART
SELF-DRIVING MARIO KART WITH TENSORFLOW

2. HELLO!
▸ Senior Developer at Shopify
▸ Previously an AI researcher at Queen’s University

3. WHAT IS DEEP
LEARNING?

4. MACHINE
LEARNING..?

5. source: https://upxacademy.com/spam-alert-machine-learning-is-filtering-your-emails/

6. LAZY

7. source: https://www.kaggle.com/benhamner/nips-2015-papers/discussion/22778

9. source: http://www.butleranalytics.com/neural-network-data-mining-explained/

12. Source: http://synapsebristol.blogspot.ca/2013/02/the-ai-lab-brain-computer-interfaces.html

13. TENSORKART

14. THE PROJECT
▸ Use Deep Learning to train an AI to play Mario Kart 64
▸ Document the entire process

15. STEP 1 - TRAINING DATA
▸ Save images of an N64 emulator window
▸ Record the joystick input from a user at the same time

16. STEP 2 - BUILDING THE MODEL
▸ I used TensorFlow - the latest Deep Learning Framework
from Google
Nvidia Autopilot Model

17. STEP 3 - TRAINING
▸ Collect all the training data and feed it to the model
▸ Its an optimization problem - TensorFlow solves for the
smallest error across the dataset
TensorFlow training

18. STEP 4 - PLAYING
▸ How to connect the AI back to Mario Kart?
▸ Fake joystick

19. DOWN THE RABBIT
HOLE

21. n

22. GO TIME

24. BACK TO THE DATA
▸ Remove corrupted screenshots
▸ Drive smoother
▸ Play more Mario Kart

26. TAKE 2

31. WHY?

32. THANKS!
▸ Read the full post at kevinhughes.ca/blog/tensor-kart
▸ Checkout the source on Github kevinhughes27/TensorKart
▸ Follow me on Twitter @kevinhughes27