When robots open their eyes

ETSETB(Barcelona)/ETSEIAAT(Terrassa)
UniversitatPolitècnicadeCatalunya(UPC)
Xavier Giró i Nieto, “When robots open their eyes...”. UPC Outreach Programme (February 2016)
When robots open
their eyes...
Xavier Giró-i-Nieto
@DocXavi
Versió 2016/2

2
Introduction

3
Densely linked slides

4
Acknowledgements

Why studying engineering ?
5
Source: University degrees with higher employment rate in Span (El País, 22/12/2015)

6
Source: University degrees with higher employment rate in Span (El País, 22/12/2015)
Why studying engineering ?

7
Source: 25 Best jobs in America (Glassdoor)
IT degrees (telecommunication and computer
science) are the closest ones to the best job: Data
scientist.
Why studying IT ?

Where are the robots ?
Nexi, del MIT Media Lab (Foto: Spencer Lowel)

Li Fei-Fei, “How we’re teaching computers to understand
pictures” TEDTalks 2014.

Font: Andrej Karpathy, “What I learned from competing against a computer on ImageNet” (2014)

Source: Andrej Karpathy, “What I learned from competing against a computer on ImageNet” (2014)

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document
recognition. Proceedings of the IEEE, 86(11), 2278-2324.
Machine learning
(Deep learning)

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional
neural networks." In Advances in neural information processing systems, pp. 1097-1105. 2012

Object detection
Girshick, Ross, Jeff Donahue, Trevor Darrell, and Jitendra Malik. "Region-based convolutional networks for
accurate object detection and segmentation." Pattern Analysis and Machine Intelligence, IEEE Transactions
on 38, no. 1 (2016): 142-158.

Object segmentation
Source: Pascal Visual Object Challenge

Face detection and recognition
Farfade, Sachin Sudhakar, Mohammad Saberian, and Li-Jia Li. "Multi-view Face
Detection Using Deep Convolutional Neural Networks." ICMR (2015).

Activity Recognition
Tran, Du, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. "Learning
spatiotemporal features with 3D convolutional networks." In Proceedings of the IEEE International
Conference on Computer Vision, pp. 4489-4497. 2015

Learn more with Nat & Lo 20% Google Project:

Only open their eyes ?
Nexi, del MIT Media Lab (Foto: Spencer Lowel)

Big data
Internet of things - IoT

Personal data
Big data

Atlas, de Boston Dynamics
Robust manipulation and motion

Games (reinforcement learning)
(Google) DeepMind

Autonomous Driving
Google Self-driving car

Visual arts
Google Research, “Going deeper into neural networks” - DeepDream (2015)

Google Research, “Going deeper into neural networks” - DeepDream (2015)
Visual arts

http://turing.deepart.io/
Visual arts

Music composition
Manuel Araoz, “Training a Recurrent Neural Network to Compose Music” (2016).

Poetry
Ross Goodwin, Neuralsnap (2016).

“Scripts” (!?)
Darknet
JON
He leaned close and onions, barefoot from his shoulder. "I am not a purple
girl," he said as he stood over him. "The sight of you sell your father with you a
little choice."
"I say to swear up his sea or a boy of stone and heart, down," Lord Tywin
said. "I love your word or her to me."

Public Health
Announcement of Google DeepMind Health (24/02/2016)

Nacho Hernandez, “Why artificial intelligence will democratize
healthcare” (TEDx Talk, 2014)
Public health

Nancy Lublin, “The heartbreaking text that inspired a crisis
helpline” (TED Talk 2015)
Mental health

“Google’s chairman (Eric Schmidth) thinks artificial intelligence
will let scientists solve some of the world’s "hard problems," like
population growth, climate change, human development,
and education.” (Bloomberg Business, 11/01/2016)
[+info @ MIT Technology Review]

Només poden obrir els ulls ?
Jeremy Howard, “The wonderful and terrifying implications of
computers that can learn”, TEDTalks 2014.

Elon Musk (Tesla), one of OpenAI promoters

Neil Lawrence, OpenAI won’t benefit humanity
without open data sharing (The Guardian,
14/12/2015)
Phd Comics: Who owns your data ? (Hint: it is not you)

39
Xavier Sala-i-Martin (Columbia University),
“Les conclusions del Fòrum de Davos”
(TV3, 03/02/2016) - in Catalan
Carles Boix (Princeton University),
“La quarta revolució industrial”
(Diari Ara, 08/02/2016) - in Catalan

Psychological support and counseling ?

41
Affective computing
Rana el Kalioubi, “This app know how you feel, from the look on
your face”, TEDTalks 2015.

42
Nexi Project,
from MIT Media Lab
(Photos: Spencer Lowel)
[video]
Affective computing

43
When robots open their eyes...
...is because they have learned to see.

44
Where to learn from ?
In school,
you are also (mainly) learning to learn !

45
?

46
Antonio Torralba,
Professor at MIT
Cristian Canton
Microsoft Research
Joan Bruna
Professor at Berkeley
These experts (and MANY MORE who are not in this page)...
Oriol Vinyals
Google DeepMind
...studied at

47
Source: La UPC als llistat de rànquins internacionals
Evolution of the position of the UPC in the
QS World University Rankings and the QS WUR by Faculty
Ranking 2015 edition 2014 edition
Catalonia Technical
university
in Spain
Spain Europe World Catalonia Technical
university
in Spain
Spain Europe World
QS World
University
Ranking,
general
classification
4 1 7 134 299 4 1 7 152 337
QS WUR by
Faculty:
Engineering
and
Technology
1 1 1 22 82 1 1 1 17 60

More details on the
website by
Professor Jordi Torres.

Object retrieval from videos
V. - T. Nguyen, -Dinh-Le, D., Salvador, A., -Zhu, C., Nguyen, D. - L., Tran,
M. - T., Duc, T. Ngo, Duong, D. Anh, Satoh, S. 'ichi, and Giró-i-Nieto, X.,
“NII-HITACHI-UIT at TRECVID 2015 Instance Search”, in TRECVID 2015
Workshop, Gaithersburg, MD, USA, 2015.

EEG-based object segmentation
E. Mohedano, Healy, G., McGuinness, K., Giró-i-Nieto, X., O'Connor, N.,
and Smeaton, A. F., “Object segmentation in images using EEG
signals”, in ACM Multimedia, Orlando, Florida (USA), 2014.

Visual saliency prediction
J. Pan and Giró-i-Nieto, X., “End-to-end Convolutional Network for
Saliency Prediction”, in Large-scale Scene Understanding Challenge
(LSUN) at CVPR Workshops , Boston, MA (USA), 2015. [Slides]
LSUN Challenge

54
A. Salvador, Zeppelzauer, M., Manchon-Vizuete, D., Calafell-Orós,
A., and Giró-i-Nieto, X., “Cultural Event Recognition with Visual
ConvNets and Temporal Models”, in CVPR ChaLearn Looking at
People Workshop 2015, 2015. [slides]
ChaLearn Worshop
Image classification

Sentiment analysis
V. Campos, Salvador, A., Jou, B., and Giró-i-Nieto, X., “Diving Deep into
Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment
Prediction”, in 1st International Workshop on Affect and Sentiment in
Multimedia, Brisbane, Australia, 2015.

Conclusions
56

Thanks a lot !
57
Slides available at:
https://imatge.upc.edu/web/people/xavier-giro
@DocXavi
/ProfessorXavi

When robots open their eyes

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Similar to When robots open their eyes

Similar to When robots open their eyes (20)

More from Universitat Politècnica de Catalunya

More from Universitat Politècnica de Catalunya (20)

Recently uploaded

Recently uploaded (20)

When robots open their eyes