Seu SlideShare está sendo baixado. ×

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

DataScienceConferenc1

•

30 de Nov de 2022

• •

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Anúncio

Sketch of the proof:

Sketch of the proof:

Sketch of the proof:

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Over-parametrization is necessary for robustness!

Over-parametrization is necessary for robustness!

Over-parametrization is necessary for robustness!

Curse of dimensionality!

This is huge over-parametrization!

This is huge over-parametrization!

This is huge over-parametrization!

Source: nilesjohnson.net

•Thank you!

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

Próximo SlideShare

Próximos SlideShares

[DSC Europe 22] The nuts and bolts of music source separation - Stipe Kabic

[DSC Europe 22] The nuts and bolts of music source separation - Stipe Kabic

Carregando em…3

×

Confira estes a seguir

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

[DSC Europe 22] Engineers guide for shepherding models in to production - Mar...

DataScienceConferenc1

[DSC Europe 22] Anti-Money Laundering ML Modeling approach - Gizem Akar

DataScienceConferenc1

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

30 de Nov de 2022

• •

Baixar para ler offline

Dados e análise

We will present some recent fundamental developments in understanding how neural networks work, in particular why the overparametrized models with the billions or even trilions of parameters perform so well even in cases when the number of parameters outnumbers the number of training samples, contrary to classical machine learning paradigm. We will present the BLN Conjecture (2020) and demonstrate few extreme cases of Bubeck-Sellke Theorem (2021). In order to illustrate highly technical theorem and it's proof we will use an original pictorial approach to represent the behaviour of neural networks from the point of view of the higher-dimensional geometry.

DataScienceConferenc1

We will present some recent fundamental developments in understanding how neural networks work, in particular why the overparametrized models with the billions or even trilions of parameters perform so well even in cases when the number of parameters outnumbers the number of training samples, contrary to classical machine learning paradigm. We will present the BLN Conjecture (2020) and demonstrate few extreme cases of Bubeck-Sellke Theorem (2021). In order to illustrate highly technical theorem and it's proof we will use an original pictorial approach to represent the behaviour of neural networks from the point of view of the higher-dimensional geometry.

Dados e análise

Mais Conteúdo rRelacionado

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

7 visualizações

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

11 visualizações

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

10 visualizações

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

6 visualizações

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

4 visualizações

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

6 visualizações

[DSC Europe 22] Engineers guide for shepherding models in to production - Mar...

DataScienceConferenc1

5 visualizações

[DSC Europe 22] Anti-Money Laundering ML Modeling approach - Gizem Akar

DataScienceConferenc1

7 visualizações

[DSC Europe 22] The ways AI will change your workplace - Iliada Kothra

DataScienceConferenc1

3 visualizações

[DSC Europe 22] Similarity search: Take your search engine to the next level ...

DataScienceConferenc1

7 visualizações

[DSC Europe 22] Equality alert in newsrooms: Providing transparency about the...

DataScienceConferenc1

4 visualizações

[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar

DataScienceConferenc1

6 visualizações

[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...

DataScienceConferenc1

35 visualizações

[DSC Europe 22] AIOPS – How can machine learning help in IT operations - Dam...

DataScienceConferenc1

3 visualizações

[DSC Europe 22] AI Ethics and AI Quality By Design - Muthu Ramachandran

DataScienceConferenc1

12 visualizações

[DSC Europe 22] Machine learning algorithms as tools for student success pred...

DataScienceConferenc1

6 visualizações

[DSC Europe 22] Parakeet - a parrot that can repeat words very well - Tomisla...

DataScienceConferenc1

4 visualizações

[DSC Europe 22] Disrupting Supply Chain with Data Science - Arthur Jansen

DataScienceConferenc1

3 visualizações

[DSC Europe 22] How to enable enterprises for DevSecOps without fighting the ...

DataScienceConferenc1

8 visualizações

[DSC Europe 22] Starting deep learning projects without sufficient amount of ...

DataScienceConferenc1

5 visualizações

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

7 visualizações

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

11 visualizações

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

10 visualizações

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

6 visualizações

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

4 visualizações

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

6 visualizações

SecurityAndComplianceCenter-QuickReference-1.01.pdf

2 visualizações

8 visualizações

Canon Project – Developing the first cover.pptx

0 visualizações

c programming practical filepdf.pdf

DhirajKumarCOE011

0 visualizações

First Draft Flat Plan.pptx

0 visualizações

Canon Project – Developing the second cover.pptx

0 visualizações

Converters.pptx

4 visualizações

seltpresentation.ppt

0 visualizações

10 Punjab in 1965 war

2 visualizações

CANAL CON MEMBRETE-Model.pdf

4 visualizações

Recommender Systems.pptx

0 visualizações

Re-Empower the Public with Data Visualization and Game Design

6 visualizações

RIL AR 2014 -15 Key Performance Indicators.pdf

2 visualizações

Presentation2.pptx

RishithaGadiparthi1

0 visualizações

Evaluation.pptx

0 visualizações

Post Production.pptx

0 visualizações

Unit 1 OMT.pptx

2 visualizações

Explore Old Dhaka.pptx

Md Farukur Rahman

2 visualizações

SecurityAndComplianceCenter-QuickReference-1.01.pdf

2 visualizações

8 visualizações

Canon Project – Developing the first cover.pptx

0 visualizações

c programming practical filepdf.pdf

DhirajKumarCOE011

0 visualizações

First Draft Flat Plan.pptx

0 visualizações

Canon Project – Developing the second cover.pptx

0 visualizações

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

1. Sketch of the proof:
2. Sketch of the proof:
3. Sketch of the proof:
4. Sketch of the proof (d=2):
5. Sketch of the proof (d=2):
6. Sketch of the proof (d=2):
7. Sketch of the proof (d=2):
8. Sketch of the proof (d=2):
9. Sketch of the proof (d=2):
10. Sketch of the proof (d=2):
11. Over-parametrization is necessary for robustness!
12. Over-parametrization is necessary for robustness!
13. Over-parametrization is necessary for robustness!
14. Curse of dimensionality!
15. This is huge over-parametrization!
16. This is huge over-parametrization!
17. This is huge over-parametrization!
18. Source: nilesjohnson.net
19. •Thank you!