Talk by Ganes Kesari in AnacondaCON 2019 (https://anacondacon.io)

1. Counting the World with AI Models
Ganes Kesari,
Gramener

2. Introduction
2
Co-founder & Head
of Analytics
Insights as Stories
100+ Clients
Help apply & adopt
Analytics
“Simplify Data
Science for all”
Our data science platform,
Gramex is now open-sourced

3. What was the count really?!
3
https://www.theguardian.com/world/2018/sep/06/donald-trump-inauguration-crowd-size-photos-edited#img-1

4. Machine Learning 101
4
New Input
Desired
Outcome
Machine learning
how to do the job
Known Input
Known
Outcome
“Programs that solve
the problem”
“Programs that learn
to solve the problem”
vs

5. Why Deep Learning?
5
Input Output
Identify features
to teach model
Traditional Machine Learning
Deep Learning
Person
Name
Input Output
Model automatically identifies
features to learn
Person
Name
https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf

6. Not very different from how we learn
Training ..Versatile Detection!

7. 7
Pic: https://theglobepost.com/2017/11/24/worst-black-fridays/
Counting Crowds

8. Approaches to Crowd Counting
8
Occlusion
Density Difference
Perspective Distortion
Camera angle

9. Counting using Density-based Estimations
9
Preserve spatial information
Localize count
Handle scale variations
No longer looking for a head!
https://blog.dimroc.com/2017/11/19/counting-crowds-and-lines/

10. Let’s save the Penguins
10
Penguin populations are
dwindling in Antarctica
and we must act now
Source: Giphy (https://giphy.com/gifs/push-bwLowbhUWm2lO)

11. Tracking the Penguins
11
https://www.zooniverse.org/projects/penguintom79/penguin-watch/about/research
~100 cameras in 16 locations,
and hourly images over years
Crowd-sourced annotations
to identify the penguins

12. Crowd-sourced Annotations
12
https://www.zooniverse.org/projects/penguintom79/penguin-watch/classify

13. Challenges
13
https://www.robots.ox.ac.uk/~vgg/data/penguins/
Cleaned dataset:
• Training: 18k
• Validation: 3k
• Test: 9k
Hurdles
• Camera angles,
• Occlusion,
• Perspective distortion,
• Density difference,
• Weather conditions

14. Approach
14
https://arxiv.org/pdf/1707.09605.pdf
• High-level prior to classify image into buckets
• Density estimation to create the density map
Input an image Density Map Estimate the countSplit into 9 patches
• NC6 v3 virtual machine with V100 GPU card
• Trained for 200 epochs, MAE for the model : ~10.5
Work done by Gramener in partnership with Microsoft AI for Earth

15. Walkthrough
15
Work done by Gramener in partnership with Microsoft AI for Earth

16. Sizing up human crowds
16
Getting estimates of
people can be of use in
several use cases such
as security, retail and
marketing
Source: Giphy (https://giphy.com/gifs/nbc-thanksgiving-xUOxeRSgzGd7rbAqWs)

17. Results
17
https://crowd-counting-app.azurewebsites.net/

18. Counting Biological cells
18
Counting of biological
cells is a painful, manual
process that takes time
of trained pathologists
Source: Giphy

19. Counting Biological cells
19
Key differences:
• Data volume
• Labels
• Fuzzy entities
• Boundaries
• Counts aren’t actionable
• Business workflow
• Low accuracies

20. Approach
20
Clean
Data
Label
Images
Identify
Boundaries
Count
Shapes
Find
Coordinates
Build
UI

21. Business workflow for counting
21

22. Business workflow for counting
22

23. Learnings: Enterprise Application
23
• Label your own data
• Look out for practical data challenges
• Don’t stop at counts - go for actionability
• Build it into the business workflow
• Sensitize users on low accuracy
• Plan for model refresh

24. Best time to buy a burger?!
24
https://blog.dimroc.com/2017/11/19/counting-crowds-and-lines/

25. Additional References
• Single-Image Crowd Counting via Multi-Column Convolutional Neural Network, 2016 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 589-597; Zhang, Yingying Zhou, Desen Chen, Siqin Gao,
Shenghua Ma, Yi,
• CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting,
Vishwanath A. Sindagi, Vishal M. Patel
• Counting in the Wild, Carlos Arteta, Victor Lempitsky, Andrew Zisserman, University of Oxford, UK; Skoltech,
Russia (https://www.robots.ox.ac.uk/~vgg/data/penguins/)
• CrowdNet: A Deep Convolutional Network for Dense Crowd Counting, 2016; Boominathan, Lokesh
Kruthiventi, Srinivas S S Babu, R. Venkatesh
• Cross-scene crowd counting via deep convolutional neural networks, Proceedings of the IEEE Computer
Society Conference on Computer Vision and Pattern Recognition, 2015, 833-841; Zhang, Cong Li, Hongsheng
Wang, Xiaogang Yang, Xiaokang.
• Switching convolutional neural network for crowd counting, Sam, Deepak Babu Surya, Shiv Babu, R.
Venkatesh, Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017,
4031-4039
25

26. Thank you.
gkesari.com/anaconda
@kesari@kesaritweetsgramener.com
Presentation deck with references