Gradient descent is the algorithm at the heart of many machine learning problems. In this talk, I’ll introduce the algorithm and code it up from scratch to apply it to a toy linear regression problem on the relationship between videogame metacritic scores and sales.

4. gradient descent

5. const makeModel = (data, learningRate, iterations) !=> {
let m = 0;
let b = 0;
for (let i = 0; i < iterations; i!++) {
const mGrad =
data.reduce(
(acc, { criticScore, globalSales }) !=>
acc + (m * criticScore + b - globalSales) * criticScore,
0
) / data.length;
const bGrad =
data.reduce(
(acc, { criticScore, globalSales }) !=>
acc + (m * criticScore + b - globalSales),
0
) / data.length;
m -= mGrad * learningRate;
b -= bGrad * learningRate;
}
return { m, b, predict: (criticScore) !=> m * criticScore + b };
};

8. 6.5 million

9. Create a function that predicts
global sales given Metacritic score.

10. Code…

13. y = m · x + b

14. Sales = m · CriticScore + b

15. m = +0.0253
b = -1.1406

16. Code…

17. m = +0.0253
b = -1.1406

18. m = ???????
b = ???????

19. m = 0
b = 0

21. mean(
)

23. mean(
)

24. Cost =
n
∑
i=1
((mxi + b) − yi)2
n

25. Cost =
n
∑
i=1
((mxi + b) − yi)2
n

26. Cost(m, b) =
n
∑
i=1
((mxi + b) − yi)2
n

27. Code…

28. Cost(m, b) =
n
∑
i=1
((mxi + b) − yi)2
n

29. Cost(m, b) =
n
∑
i=1
((mxi + b) − yi)2
n
Cost(m) =
n
∑
i=1
(mxi − yi)2
n

30. m = 0
hint: why might this algo be called “gradient descent?”
m′ > 0

31. m = m − m′

32. m = -1
m′ < 0
m = m − −m′

33. Cost =
n
∑
i=1
((mxi + b) − yi)2
n

34. J =
n
∑
i=1
((mxi + b) − yi)2
n

35. J =
n
∑
i=1
((mxi + b) − yi)2
n
∂J
∂m
≈
n
∑
i=1
((mxi + b) − yi)x
n

36. Questions?

37. Code…

38. m = 0
m′ > 0

39. b = 0
b′ > 0

40. J =
n
∑
i=1
((mxi + b) − yi)2
n

41. J =
n
∑
i=1
((mxi + b) − yi)2
n
∂J
∂b
≈
n
∑
i=1
((mxi + b) − yi)
n

42. Code…

44. 1000000

45. Code…

47. “Learning Rate” = 1

48. “Learning Rate” = .1

49. .0003

50. Code…

51. const makeModel = (data, learningRate, iterations) !=> {
let m = 0;
let b = 0;
for (let i = 0; i < iterations; i!++) {
const mGrad =
data.reduce(
(acc, { criticScore, globalSales }) !=>
acc + (m * criticScore + b - globalSales) * criticScore,
0
) / data.length;
const bGrad =
data.reduce(
(acc, { criticScore, globalSales }) !=>
acc + (m * criticScore + b - globalSales),
0
) / data.length;
m -= mGrad * learningRate;
b -= bGrad * learningRate;
}
return { m, b, predict: (criticScore) !=> m * criticScore + b };
};