1. Thomas G. Dietterich Department of Computer Science Oregon State University Corvallis, Oregon 97331 http://www.cs.orst.edu/~tgd Machine Learning: Making Computer Science Scientific

4. Scenario 1: Reading Checks Find and read “courtesy amount” on checks:

19. Supervised Learning Training Examples Learning Algorithm Classifier New Examples 8 8 3 6 0 1

20. AT&T/NCR Check Reading System Recognition transformer is a neural network trained on 500,000 examples of characters The entire system is trained given entire checks as input and dollar amounts as output LeCun, Bottou, Bengio & Haffner (1998) Gradient-Based Learning Applied to Document Recognition

23. Density Estimation Training Examples Learning Algorithm Density Estimator P(chip i is bad) = 0.42 Partially-tested wafer

26. On-Wafer Chip Test Results 3.8% increase in profit

28. Reinforcement Learning Environment state s reward r action a Agent’s goal: Choose actions to maximize total reward Action Selection Rule is called a “policy”: a =  (s) agent

31. Experimental Results: % trials robot reaches goal Busquets, Lopez de Mantaras, Sierra, Dietterich (2002)

41. Triple Tradeoff (2) Number of training examples N Accuracy Hypothesis Complexity H 1 H 2 H 3

44. Corollary 1: Fixed H will underfit Number of training examples N Accuracy H 1 H 2 underfit

45. Corollary 2: Variable-sized H will overfit Hypothesis Space Complexity Accuracy N = 100 overfit

46. Ideal Learning Algorithm: Adapt complexity to data Hypothesis Space Complexity Accuracy N = 10 N = 100 N = 1000

53. The Data Explosion (3) http://www.cs.columbia.edu/~hgs/internet/traffic.html

54. Old Computer Science Conception of Data Store Retrieve

55. New Computer Science Conception of Data Store Build Models Solve Problems Problems Solutions

58. Hardware Branch Prediction Source: Jim é nez & Lin (2000) Perceptron Learning for Predicting the Behavior of Conditional Branches

62. Video Textures Arno Schödl, Richard Szeliski, David H. Salesin, Irfan Essa (SIGGRAPH 2000) You can find this video at Virtual Fish Tank Movie

63. Graphics: Image Analogies : :: : ? Hertzmann, Jacobs, Oliver, Curless, Salesin (2000) SIGGRAPH

64. Learning to Predict Textures Find p to minimize Euclidean distance between and B’(q) := A’(p) A(p) A’(p) B(q) B’(q)

65. Image Analogies : :: :

66. A video can be found at Image Analogies Movie