This document summarizes recent progress and opportunities in analyzing data from global network cameras. It discusses the CAM2 system, a general-purpose computing platform for analyzing large amounts of image data from thousands of cameras worldwide. CAM2 has demonstrated the ability to analyze billions of images per day using cloud computing resources. It aims to provide abundant real-world image data and computing power for computer vision and machine learning applications. The document also outlines several challenges in managing and analyzing data from networked cameras at a large scale.

1. Opportunities and Challenges
in Global Network Cameras
全球網路攝影機帶來的機會與挑戰
Yung-Hsiang Lu 陸永祥
Purdue University
Acknowledgments: National Science Foundation ACI-1535108, IIP-1530914, OISE-1427808, and
CNS-0958487, Lynn CSE Fellowship, Amazon, Microsoft, and the owners of the data. Any
opinions, findings, and conclusions or recommendations expressed in this material are those of
the author and do not necessarily reflect the views of the sponsors.
1

2. Purpose of Today's Seminar
Share our recent progress
Discuss new ideas
Recruit users and collaborators
Please feel free to interrupt and share your
comments / questions / suggestions.
2

3. 3
臺北市交通控制中心

4. 4
2016-07-04 08:58:25

5. 5
高雄市政府交通局
07/04/2016 10:27:25

6. 6
交通部臺灣區國道高速公路局即時路況資訊
10:58:33
07/04/2016

7. Demonstration:
Image-Based Navigation
Purdue University
7

8. Demonstration:
即時圖像導航
真理大學, 資訊工程學系
蘇維宗 教授
8

9. 9
Image capture: Mar 2009

10. Why is real-time data
important?
為什麼即時的數據很重要？
10

11. Emergency Responses
災難救助
11

12. Houston Flooding 2016/04/18
12
04/18/2016 14:40:03

13. Parade Route Traffic Cameras
13
公共安全
2014 Thanksgiving Parade in NYC
[International Conference on Cloud Computing and Big Data 2015]

14. Parade Scenes
14

15. Object Tracking
Network cameras provide abundant real-world
data for vision programs.
15

16. 16

17. One Image = One Thousands Words
17

18. One Image = One Thousands Words
18
resting
groups
talking
alone
female
choosing

19. What do you see?
19

20. What do you see?
20
standing
child
nobody
resting

21. CAM2: Continuous Analysis of Many CAMeras
http://cam2.ecn.purdue.edu
21

22. CAM2: Continuous Analysis of Many CAMeras
http://cam2.ecn.purdue.edu
22
CAM2: general-purpose
computing platform for analyzing
large amounts of data.

23. 23
Temporal
Spatial
real-time
recent
obsolete
single few many worldwide
stationary
network
camera
personal
photographs
organizations'
network
cameras
CAM2
photographs
on the Internet
street view

24. Who Can Use CAM2?
誰可以使用CAM2 ？
Big Data 大數據
Computer Vision 計算機視覺
Cloud Computing 雲計算
Mobile Computing 移動計算
Programming Language 程序設計語言
Architecture 計算機結構
Network 網路
Human Interface 人機界面
You 你！
24

25. CAM2 has demonstrated the ability to
• analyze 200 million images (7TB) in 24 hours
• (200 M images = 1 image/sec for 8.8 years)
• from 16,000 cameras worldwide
• one live (real-time) image every 5 seconds
• 17 Amazon high-performance instances
• detect motion (background subtraction)
working on analyzing 1B images/day now
25

26. Background Subtraction
26

27. Moving Object and Human Detection
27

28. Demonstration
Object Tracking + Speed
28

29. 29
user Web
Portal
user
database
camera
database
data sources
cloud
resource
manager
CAM2
visual data
Visual data do not go
through CAM2.

30. CAM2 has more than 80,000 cameras now,
5 seconds/camera  14 days (8 hours / day)
30
100,000
17

31. Big Data?
• 100,000 cameras
• one image/minute-camera  140M images/day
• one image/second-camera  8B images/day
• Each image ~ 100KB  14 TB ~ 800 TB/day
31

32. Examples of Dataset
32

33. Network Cameras for Public Safety
33
ChicagoPurdue
[IEEE Technologies for Homeland Security 2016]

34. Use Network Cameras for Public Safety
34

35. Privacy Protection
We do not identify any individual.
Users agree not to identify any individual.
35

36. 36

37. 37

38. 38
CCTV: close circuit television
not connected to the Internet

39. Data for Machine Learning
39

40. [Cloud Computing and Big Data 2015]
[Cloud Computing Technology and Science (Cloudcom) 2015]
40
cameras +
locations +
resolutions
desired frame rate
visual content
analysis program
cloud instances:
• types (# cores, memory)
• locations
• numbers
Resource Management

41. Cloud Computing Locations
41
Amazon EC2's Locations
Microsoft Azure's Locations

42. Cloud Pricing ($/hour)
AWS m3.2xlarge (8 vCPU + 30GB memory)
42
0.532 0.585
0.632
0.784
0.616
0.761
0.77
784
532
= 1.474

43. Bring alldata to the cheapest instance?
43
0.532 0.585
0.632
0.784
0.616
0.761
0.77

44. Round-Trip Time (RTT) and Frame Rates
[IEEE Cloud Computing Magazine September/October 2015]
44
0
5
10
15
20
25
30
35
0 50 100 150 200 250 300
FramesperSecond(fps)
Round-Trip Time (RTT) in ms
MJPEG measured
MJPEG using
netem to inject
delays
If high frame rates are required,
data must be retrieved by
a cloud instance with small RTT

45. Nonlinear Frame Rate and Utilization
(Amazon m3.xlarge)
IA: Image Archival
ME: Motion Estimation
MOD: Moving Object Detection
HD: Human Detection
45
[Cloud Computing and Big Data 2015 (Best Paper Award)]
(a) 0.2 frame/s (b) 10 frame/s
0.02% 0.31% 0.20% 0.03%
0.15% 0.21% 0.40% 2.65%
0.1% 0.4% 0.32%
5.78% 8.34% 14.48%

46. [Transactions on Cloud Computing (submitted for review)] 46
cameras +
locations +
resolutions
desired frame rate
visual content
analysis program
cloud instances:
• types (# cores, memory)
• locations
• numbers
Variable-Size Bin Packing
stream 1 stream 5
stream 2 stream 7
stream 3 stream 8
stream 4 stream 6
VM1 VM2 VM3

47. Cost Per Million Frames
47
(a) 0.2 frame/s (b) 10 frame/s
[Cloud Computing and Big Data 2015]
Choosing the right cloud
instance can reduce cost
by more than 50%
Larger differences at higher frame rates

48. Resource Management
48
Scenario Program
Frame
Rate
Cameras Intensive
Scenario 1
(CPU Intensive)
FT 15.00 25 CPU
HD 0.50 250 CPU
Scenario 2
(Memory Intensive)
BS 0.10 5000 Memory
MOD 0.05 3000 Memory
Scenario 3
(Mixed)
BS 0.20 4000 Memory
MOD 0.20 1000 CPU
FT 10.00 10 CPU
HD 0.20 300 CPU
FT: Feature Tracking (optical flow)
HD: Human Detection (HOG)
BS: Background Subtraction
MOD: BS + erosion + dilation + contour
Abbr. Resource Allocation Strategy
ST1 Always use m4.xlarge
ST2 Always use c4.xlarge
ST3 Always use r3.xlarge
ST4 Use the most cost-effective
instance for each program without
sharing instances between
programs
ST5 Enhanced Manager: Reduce the
overall cost with sharing instances
between programs
Model and solve the problem using
multi-dimensional bin packing
The experiments ran for 24 hours as many as 120 cores in AWS.

49. Evaluation of Resource Allocation
49
61% Savings
$326/Day
$128/Day
25% Savings37% Savings
$336/Day
$211/Day
CPU Intensive Memory Intensive Mixed
$248/Day
$185/Day

50. Analysis for 24 Hours
50
Lectures End

51. Analyze Archive using Spot Instances
• Three types of pricing models:
• Spot instances' costs depend on the market.
• A spot instance may be terminated when the
market price exceed the bidding price.
51
Pricing Model Pay Analogue
On-Demand Hourly Hotel Room
Long-Term Yearly Apartment Lease
Spot Bidding Priceline.com

52. Offline Analysis of Archival Data
• Spot instances can be a cost-effective solution
for analyzing archival data (i.e., not real-time).
• Using periodic check-pointing, analyses may
resume after terminations.
• Setting bidding prices strategically can reduce
cost (as much as 85%) with less than 5%
performance degradation.
[Electronic Imaging 2016]
52

53. Computation Offloading
Integrating Mobile and Cloud
53

54. Lessons Learned (many)
• Data management must be planned in advance
• Treat the data as "non-persistent": only one
chance to touch the data
• Metadata must be generated in advance or
during data acquisition, not after
• When in doubt, save the data and (more
important) metadata
• Metadata must be machine readable
• Encode (some) metadata in file names
• Supervised learning (with truth) is impossible
54

55. Future of CAM2
• Computing platform for analyzing "big data"
(TB/h), real-time or archival
• Integration of many different sources of data
(weather, earthquake, tweets, traffic ...)
• Repository of "real-world" visual data
• Test bed for system research
• Opportunities for collaboration
55

56. Many Challenges
56
• Create metadata for searching the sources
• Develop standards to retrieve data
• Find locations of the cameras
• Design vision solutions to understand the world
• Allocate resources to analyze and store data
• .... many more

57. 57
Acknowledgments

58. Former Members Started Perceive Inc.
and receives $225,000 from NSF
SBIR IIP- 1622082
(已經募集七百萬新台幣 )
58
Yung-Hsiang Lu is a co-founder and the Scientific Adviser of Perceive Inc.

59. Conclusion
• Network cameras provide many opportunities for
understand this world.
• CAM2 is a system for large-scale analysis.
• It is a platform for vision program at large scales
as well as cloud resource management.
• Please register as users cam2.ecn.purdue.edu.
• Source code is available upon request.
59

60. 60