Talk given at Friday Hacks organized by NUS Hackers

1. Programs that Play better than Us
Melvin Zhang
melvin@melvinzhang.net
@melvinzhangzy

2. https://en.wikipedia.org/wiki/File:ST Battle Chess.png

3. https://en.wikipedia.org/wiki/Deep Blue (chess computer)
Deep Blue (IBM, 1996)

4. http://afflictor.com/2012/09/11/chess-programs-regularly-play-at-good-amateur-level/

5. Game tree

6. Optimal play
Terminal
min player
max player

7. Optimal play
1 01 1 1Terminal
min player
max player

8. Optimal play
1 01 1 1
0
Terminal
min player
max player

9. Optimal play
1 01 1 1
0 1
Terminal
min player
max player

10. Optimal play
1 01 1 1
0 1
1
Terminal
min player
max player

11. Chess has about 1046
states!

12. Minimax algorithm
Cut-off
min player
max player

13. Minimax algorithm
.7 .1 .6 .9Cut-off
min player
max player

14. Minimax algorithm
.7 .1 .6 .9
.1
Cut-off
min player
max player

15. Minimax algorithm
.7 .1 .6 .9
.1 .6
Cut-off
min player
max player

16. Minimax algorithm
.7 .1 .6 .9
.1 .6
.6
Cut-off
min player
max player

17. https://stockfishchess.org/
Stockfish

18. https://tests.stockfishchess.org/
Testing AI changes is crucial

19. Value functions are hard!

20. http://mathworld.wolfram.com/Go.html

21. http://www.remi-coulom.fr/CrazyStone/
R´emi Coulom

22. http://www.wired.com/2014/05/the-world-of-computer-go/

23. Monte Carlo evaluations
Cut-off
min player
max player

24. Monte Carlo evaluations
Cut-off
min player
max player

25. Monte Carlo evaluations
Cut-off
min player
max player

26. Monte Carlo evaluations
Cut-off
min player
max player

27. Monte Carlo evaluations
Cut-off
min player
max player
.7

28. Monte Carlo Tree Search (MCTS)

29. by Google Deepmind
https://deepmind.com/research/alphago/

30. https://gogameguru.com/alphago-races-ahead-2-0-lee-sedol/

31. MCTS + Policy and value networks

32. http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html
MCTS + Policy and value networks

33. Some games have hidden information!

34. http://magic.wizards.com/en/events/coverage/gpsin15/father-son-2015-06-27

35. https://magarena.github.io

37. Determinization: choose a random instance of the
hidden information during simulation

38. Comparison of Minimax and MCTS
At 1s thinking time:
Minimax MCTS
1 0.88

39. Comparison of Minimax and MCTS
At 1s thinking time:
Minimax MCTS
1 0.88
At 4s thinking time:
Minimax MCTS
1 1.71

40. Open problems

41. MCTS is bad at tight tactical play.

42. MCTS plays badly when it is behind in the game.

43. Further readings

44. Further readings