O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

Deep Learning Deep Change: Mapping the evolution of the Artificial Intelligence GPT

Carregando em…3

Confira estes a seguir

1 de 33 Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (15)

Semelhante a Deep Learning Deep Change: Mapping the evolution of the Artificial Intelligence GPT (20)


Mais de Juan Mateos-Garcia (17)

Mais recentes (20)


Deep Learning Deep Change: Mapping the evolution of the Artificial Intelligence GPT

  1. 1. Deep Learning, Deep Change Mapping the development of the Artificial Intelligence General Purpose Technology Joel Klinger, Juan Mateos-Garcia and Konstantinos Stathoulopoulos SPRU Friday Seminar 16 November 2018
  2. 2. The news this week: more than Brexit Artificial Intelligence is a powerful technology with widespread applicability… but is it a General Purpose Technology?
  3. 3. The global picture Countries across the world are putting in place national strategies to develop their AI sectors… but what is its geography?
  4. 4. The local picture Geographical proximity could help coordinate the development of complex AI technologies... What local factors matter?
  5. 5. Structure 1. Theory 2. Defining AI 3. Method and data 4. Findings a. Is AI a GPT? b. How is it geography changing? c. What drives these changes? 5. Conclusions and next steps
  6. 6. Theory. General Purpose Technologies as engines of growth Technologies ‘characterized by the potential for pervasive use in a wide range of sectors and by their technological dynamism’ [Bresnahan and Trajtenberg, 1995] ● Novelty ● Disruption ● Require complementary investments ● Generate externalities (risk of coordination failures)
  7. 7. Literature. Geography GPT → change in drivers of advantage → New geographies of production and innovation [Rosenberg and Trajtenberg, 2004] https://fineartamerica.com/featured/3- corliss-steam-engine-1876-granger.html GPT discontinuity → early product life-cycle → windows of opportunity Followed by maturity and consolidation. [Abernathy and Utterback, 1977, Anderson and Tushman 1990, Klepper, 1996, Scott and Storper 2003]
  8. 8. Literature. Relatedness National (regional, local) economies develop by diversifying into related sectors / disciplines. [Delgado et al, 2018] Co-location with relevant sectors could be important/necessary for complex GPTs that require coordination in deployment. [Balland and Rigby, 2017] https://phys.org/news/2007-08-nation-position-product-space- economic.html
  9. 9. Defining AI. History https://hackernoon.com/ai-in-medicine-a-beginners-guide-a3b34b1dd5d7 A long dream of machines that ‘think’ And a long journey of disappointing implementations
  10. 10. Defining AI. A modern approach http://cdn.aiindex.org/2017-report.pdf https://www.quora.com/How-do-I-learn-Neural-Network-by-myself More data More processing power Software innovations Deep learning
  11. 11. Defining AI. Policy questions AI is being described as the latest GPT [Cockburn et al, 2017, Agrawal et al, 2018] But is it so? Brittle, narrow, opaque. Maybe not so general. [Marcus, 2018] https://www.theatlantic.com/technology/archive/201 8/03/can-you-sue-a-robocar/556007/ http://lifeinvestasset.com/noticias/reporte-lifeinvest- 15-06-18/ A global race in AI is afoot with dozens of national strategies being launched. But what is their economic rationale? [Stix, 2018, Goldfarb et al, 2018]
  12. 12. Method and data. Research questions GPT Geographical change Driven by presence of complements Q1. Does AI behave like a GPT? Q2. Is its geography being transformed? Q3. What is the role of regional complements?
  13. 13. Method and data. Data pipeline arXIv CrunchBase Deep Learning papers Filtering Deep Learning clusters Geocoding Related research clusters Related industry clustersSemantic proximity to DL Co-occurrence with DL Q2 Q3 Topic modelling Q1
  14. 14. Method and data. arXiv Popular pre-prints website in physics, engineering and computer science. 1.3m papers. Very popular in the AI community
  15. 15. Method and data. arXiv processing arXIv Deep Learning papers Filtering Deep Learning clusters Geocoding Related research clusters Similarity with DL Topic modelling We fuzzy-match ArXiv with Microsoft Academic Graph to retrieve citations (for QA) and institutions (90% match rate) We fuzzy-match institutions with Global Research Identifier (GRID) to extract institution locations. We then bin those into countries / regions using a point on polygon approach and Natural Earth shapefiles We use CorEx, a topic modelling algorithm to identify 2 topics related to Deep Learning. CorEX looks for clusters of words in the data that maximise correlations in the data We measure cosine distance between DL and other computer science sub-disciplines based on co-occurrence in papers
  16. 16. Method and data. Anchored correlation explanation (CorEX) Identify topics in corpora of text. Looks for topics that maximally explains dependencies between words in documents. Does not require selection of k (number of topics) like LDA. Can be seeded with anchor words [Gallagher et al 2017]
  17. 17. Method and data. Computer science subject proximities with DL Disciplines such as computer vision, learning, machine learning, neural nets and AI appear closer to DL
  18. 18. Method and data. arXiv content We end with 130k unique papers and 250k paper-institution pairs. We identify 15k papers as Deep learning. This is what the data look like
  19. 19. Method and data. CrunchBase Global startup-up directory with ~ 450,000 entries (257,000 organisations). Includes sectors and descriptions. Increasingly used in economics and management research. [Dalle et al, 2017, Menon et al 2018] CrunchBase Related industry clustersSemantic proximity to DL Train a machine learning model on the CrunchBase data and out-of-sample predict to what sectors do arXiv papers relate.
  20. 20. Method and data. Sector relatedness with DL Sectors such as data, AI, software, hardware, science and engineering, education and ICT are more semantically related to DL
  21. 21. Findings. Q1: Is DL a GPT? Test 1: Dynamism DL is rapidly gaining importance in absolute and relative terms in the arXiv corpus
  22. 22. Findings. Q1: Is DL a GPT? Test 2: Generality [We compare pre and post 2012 because 2012 is generally considered a watershed moment for DL with the publication of Krizhevsky et al, 2012.] DL is being increasingly applied in more computer science disciplines
  23. 23. Findings. Q1: Is DL a GPT? Test 3: spin out impact. DL papers are over-represented in the high citation groups in most CS sub-disciplines.
  24. 24. Findings. Q2: How is the geography of DL evolving? National specialisation [Based on Revealed Comparative Indices, focusing on high activity places and higher quality papers] Change is afoot. China, Singapore and Canada ascending, US keeps up, Europe (excepting Switzerland and UK) fall behind.
  25. 25. Findings. Q2: How is the geography of DL evolving? Regional picture We see some leading digital/creative/ defence clusters amongst the most DL specialised regions.
  26. 26. Findings. Q2: How is the geography of DL evolving? Volatility and concentration After an initial period of volatility (fat tailed distribution of specialisation), we see an apparent shake-up, increasing concentration in a small number of regions (nations).
  27. 27. Findings. Q3: What are the drivers of regional specialisation? [All totals logged, all variables normalised, focusing on the top quartile of locations by arXiv activity] Persistence + Research specialisation ~ Industry specialisation ~ Complementarities ++ China ++
  28. 28. Findings. Q3: What are the drivers of regional specialisation? Complementarities are more important for DL / other data CS disciplines.
  29. 29. Conclusions. Wrap up of findings 1. DL looks like a General Purpose Technology ■ Widespread exploration of opportunities 2. Evidence of discontinuity in its geography ■ But things seem to be settling down 3. Presence of relevant industries linked to DL cluster development. ■ Although also some evidence suggestive of unexpected spillovers
  30. 30. Conclusions. Policy implications 1. Further evidence of localised knowledge spillovers from AI research activity: might justify activist DL policies. 2. But is the window of opportunity for DL closed now? 3. DL spreading widely despite some concerns about its narrow / brittle nature. What is the role of R&I funders in diversifying the AI science/technology trajectory? 4. And what about getting DL applied in less related sectors (in missions)?
  31. 31. Conclusions. Limitations and next steps 1. Data caveats: Triangulate with other data (traditional publications, patents) 2. Causality: a. Look for natural experiments / exogenous shocks b. Explore mechanisms for research - industry complementarities (networks, labour flows?) 3. Directionality a. Further analyse text descriptions to understand sub- branches of modern DL research and their drivers.
  32. 32. Appendix. GitHub code and data https://github.com/nestauk/arxiv_ai
  33. 33. nesta.org.uk @nesta_uk juan.mateos-garcia@nesta.org.uk