Recomendados
Recomendados
Mais conteúdo relacionado
Semelhante a How Fast is AI in Pharo? Benchmarking Linear Regression
Semelhante a How Fast is AI in Pharo? Benchmarking Linear Regression (20)
Mais de Oleksandr Zaitsev
Mais de Oleksandr Zaitsev (16)
Último
Último (20)
How Fast is AI in Pharo? Benchmarking Linear Regression
- 1. How Fast is AI in Pharo? Benchmarking Linear Regression 1Arolla, Paris 2Inria, Univ. Lille, CNRS, Centrale Lille, UMR 9189 - CRIStAL Oleksandr ZAITSEV1,2, Sebastian JORDAN MONTAÑO2, Stéphane DUCASSE 2 IWST22 — International Workshop on Smalltalk Technologies Novi Sad, Serbia oleksandr.zaitsev@arolla.fr
- 2. 2 Fast Very slow! Why? pharo-ai Scikit-learn (Python) Training machine learning models
- 3. 3 Training machine learning models Can we do the same in Pharo?
- 4. 4
- 5. 5 LAPACK SciPy scikit-learn pharo-ai PolyMath LAPACK Pharo Python Fast Algebra Math Machine Learning
- 6. 6 We want to provide a proof of concept that math & AI algorithms in Pharo can be boosted by delegating time- consuming operations to highly-optimised external libraries. To do that, we build a prototype implementation of linear regression based on the DGELSD routine of LAPACK
- 7. Linear regression and how it is implemented
- 11. 11 Linear Regression Implementations Implementation Gradient Descent Least Squares - Iterative - Approximate - One iteration - Exact solution
- 12. 12 Implementation 1. Gradient Descent
- 13. 13 Need to minimise the errors ei = ̂ yi − yi Cost function - mean squared errors J(w) = 1 m m ∑ i=1 e2 i Implementation 1. Gradient Descent
- 14. 14 J(w) = 1 m m ∑ i=1 ( ̂ yi − yi)2 Implementation 1. Gradient Descent
- 15. 15 J(w) = 1 m m ∑ i=1 ( ̂ yi − yi)2 Implementation 1. Gradient Descent
- 16. 16 J(w) = 1 m m ∑ i=1 ( ̂ yi − yi)2 θ(new) = θ(old) − α ∂ ∂θi J(θ) Implementation 1. Gradient Descent
- 17. 17 Implementation 2. Least Squares ̂ θ = argmin||y − Xθ||2 Orthogonal projection
- 18. 18 Implementation 2. Least Squares Implemented as DGELSD routine In LAPACK We can call it through FFI
- 19. 19 Demo will come in the end Wait for it!
- 20. Experiment
- 21. 21 Datasets
- 22. 22 Research Questions - RQ.1 - Measuring LAPACK speedup. How much time improvement can we achieve by calling LAPACK from Pharo? - RQ.2 - Comparing to scikit-learn. How does Pharo & LAPACK implementation compare to the one provided by scikit-learn? - RQ.3 - Comparing pure Pharo with Python. How does pure Pharo implementation of linear regression compare to equivalent pure Python implementation?
- 23. 23 RQ.1 - Measuring LAPACK speedup
- 24. 24 Time in seconds (less is better) Small Medium Pure Pharo vs Pharo & LAPACK 1820 times faster 2103 times faster Pharo Pharo & LAPACK 19min40sec 1h44min 0.6sec 2.9sec Help RQ.1 - Measuring LAPACK speedup
- 25. 25 RQ.2 - Comparing to scikit-learn.
- 26. 26 RQ.2 - Comparing to scikit-learn.
- 27. 27 RQ.3 - Comparing pure Pharo with Python
- 28. 28 RQ.3 - Comparing pure Pharo with Python
- 29. Summary ‣ We propose a prototype implementation of Linear Regression based on LAPACK ‣ We show that LAPACK & Pharo is up to 2103 times faster than pure Pharo ‣ We also show that scikit-learn is 8-5 times faster than our prototype, depending on the size of the data. ‣ Finally, we demonstrate that pure Pharo is up to 15 times faster than the equivalent implementation in pure Python ‣ Those findings can lay the foundation for the future work in building fast numerical libraries for Pharo and further using them in higher-level libraries such as pharo-ai.