Data mining and analysis has been dominated by the big looking at the small. Businesses, institutions and governments examine our habits with an eye to commercial opportunities, welfare, and security.  However, big data is migrating analysis into the arena of networking and association to enhance services: advertising, ‘pre-selling,’ healthcare, security and tax avoidance reduction. But this leaves the critical arena of Small Data unaddressed - the small looking at the small - individuals and things examining and exploiting their own data. Here we consider a future of ubiquitous tagging, sensors, measuring and networked monitoring powered by the IoT. Key conclusions see many devices talking to each other at close range with little (or no) need of internet connection, and more network connections generated between things than those on the net.

1. cochrane.org.uk
Peter Cochrane
BIG
S m a l l D a t a
I M P A C T
Ahmed Elmagarmid
Mourad Ouzzani
Stochastic complexity on a massive scale
exhibiting fractal behaviours from a local
to a national, international and global scale

2. ManAgement Struggle
Generally confounded by complexity
Instinct and experience increasingly dangerous
Knee jerk management is always dangerous
Dynamic connectivity
and multi-species
non-linearity

3. L i m i t e d :
- A s s u m p t i o n s
- M e a s u r e s
- T h i n k i n g
- M o d e l s
- D a t a
E v o l v e d f o r a n s l o w c h a n g i n g
a n a l o g u e w o r l d - l i n e a r i t y
r u l e d & o u r m a t h w o r k e d
- b u t i t a l l c r a s h e s a b o v e
o r d e r ( 5 ) - c o m p l e x i t y
n o w r u l e s w i t h 1 0 s t o
1 0 0 0 s o f v a r i a b l e s -
a n d p e r h a p s w o r s e
w e c o p e w i t h < 5 - 7
s i m u l t a n e o u s i d e a s
“There are lots of simple
solutions to complex problems
and they are all wrong”
tyranny of THE SIMPLE
We are linear thinkers but the world is complex

4. p o l i t i c s
M a n a g e r s
E C O N O M I CS
“People love simple - it is so
obvious and doesn’t involve thinking”

5. Technology
Ecology
d e p e n d e n cy J I GS AW
Economics
Society
Industry
Care
Markets
Trade
Education
Policing
Defence
Commerce
Security
Logistics
Finance
Government
Companies
W e h a v e n o m o d e l o r u n d e r s t a n d i n g
“Sometimes WE might observe or infer some
limited relationships along with typical
characteristics - however, AI can
produce a more
comprehensive
analysis and a
much fuller
picture”
The BIG DATA DOMAIN links everything and records relationships and
patterns of behaviour - in doing so it creates vast amounts of data vital
to the modelling of the real world…
Law & Order
Defence
Education
Health
Institutions

6. A X I O M S
Ve r y o b v i o u s ?

7. A X I O M S
Ve r y o b v i o u s ?
I n c re a s e d e f f i c i e n c y l e a d s
t o i n c re a s e d b r i t t l e n e s s
a n d m o re f a i l u re s … .
…more data is a must for
a c c u ra t e b u s i n e s s m o d e l s
a n d wa r g a m e s …
….BIG and small DATA are
key to a more predictable
t h e f u t u re
Stop Over Optim ising
if you wa nt your
c o m p a n y t o s u r v i ve a n d
p ro s p e r
Avoid erroneous KPIs
a n d b o n u s d r i ve n
c u l t u re s
Embrace a da ta an d
business model driven
a p p ro a c h

9. B I G D A T A
M A Y N O T B E E N O U G H

10. - P o l i s h i n g w h a t w e h a v e w i l l n o t r e a l i s e s u s t a i n a b i l i t y
- N e w m a t e r i a l s & m a n u f a c t u r i n g m e t h o d s a r e r e q u i r e d
- N e w d i s t r i b u t i o n , s u p p o r t a n d m a i n t e n a n c e r e g i m e s
- E n e r g y a n d m a t e r i a l u s a g e h a s t o b e m i n i m i s e d
- R e u s i n g , R e p u r p o s i n g , R e c y c l i n g , h a s t o b e r e a l
- E c o n o m i c s y s t e m s h a v e t o b e f i t f o r p u r p o s e
A X I O M S
Not so obvious?

12. “ To b u i l d a n e r v o u s s y s t e m f o r t h e p l a n e t w i t h m o s t
p e o p l e a n d t h i n g s c o n n e c t i n g ( a t w i l l ) w i t h
d i s t r i b u t e d s t o r a g e o f d a t a a n d i n f o r m a t i o n ”
MISSION
or evolution ?
“ To s t o p p r o d u c i n g m o r e a n d m o r e f o r t h e f e w a n d
s t a r t p r o v i d i n g s u f f i c i e n t f o r t h e m a n y ”

13. DATA Big & SMALL
Small Data
Focussed
Detail
Big Data
Macro-View
Relationships
Limited
C o n t a i n e d
C o n s t r a i n e d
Expanding
Te n d i n g t o
The Infinite
Both important & dynamic

14. THE AUDIT TRAIL
Raw materials to roll out to user
Raw material extraction, refinement
Raw material recover from scrap
Shipping logistics & supply chain
Processing: forming, machining
Testing and Q A assessment
Assembly machine & people
Software loading & test
Assembly testing
Packaging
Shipping
Supply
Sale
Seller testing
Sale process/prep
Ownership Registration
User habits & maintenance
User feedback to design team
Onward sale (repeat all above)
Disposal - cannibalisation for parts
Recycling/recovery of raw materials

15. H EA LT H P ro g n os is
Hospital
Prime Centre
of Expertise
& Capability
Medical Centre
Prime Point of Local
Capability
Home Based
Measurements
Monitoring
Medication
Technology
Push
- Patient Pull
M i g r a t i n g c e n t r e o f g r a v i t y

16. D e v i c e to d e v i c e
Monitoring the in:out & effect give a complete picture
Self contained home net with no internet connection
- or an aggregated and sporadic net connect

17. M y to ot h b r u s h
Will not want to connect to the internet
….nor will it use 5G !!

18. The coffee machine
ANECDOTE
Purchased
Failed in month 12
Repaired under warranty
Returned with a user report
8000 coffees consumed in one year
8200 hours in one year ~ 1 coffee an hour
Health implications
Product implications

19. The coffee machine
ANECDOTE
Purchased
Failed in month 12
Repaired under warranty
Returned with a user report
8000 coffees consumed in one year
8200 hours in one year ~ 1 coffee an hour
Health implications
Product implications
What will our appliances
have to tell us ??
Fridge
Freezer
Microwave
Dishwasher
Washing Machine
Tumbler dryer
Oven/Hob
Security
Air-Con
Shower
+++Designer Feedback

20. Forward looking radar
and car-to-car comms
Rearward looking radar
and car-to-car comms
3,4,5G Network
Connection
WiFi and WiMax Net Connection to Gas Station,
Repair Shop for OS, APP, Info upgrades, failure
reporting and real time remote diagnostics
Self contained data storage and analysis Small Data Domain
VEHICLE to vehicle Comms

21. VEHICLE Comms
Networks without infrastructure
BIG DATA = Vehicle type, location, speed,
owner, insurance, taxation…
small DATA = Production and repair history,
performance data, driver habits,
apps,…
3,4,5G nets
cannot support
the demands of
future traffic and
logistic networks…
but vehicle to vehicle
systems can…by providing
the most efficient and relevant
linking of users reporting their status
and sharing local/pertinent information

22. distribution
Automated & intelligent

23. BIG DATA
Where has the package
come from and where is it
going ? Where is it now and
what does it contain ?
small DATA
The detailed product
history of how it was
created; where it has been
s t o re d a n d d e t a i l e d
handling history including
humidity, temperature, G-
force, damage +++
Item
Box
Pallet
Warehouse
Container
Supplier
Customer

24. Ship to
shore
Container
to Ship
H I D D E N V A L U E
In every container operation globally
BIG DATA = Where is my container,
schedule and contents ?
Small DATA = What is the condition of
the container contents ?

25. No single solution
can deal with all
forms of attack....
Security
N o o n e s o l u t i o n i s
s u f f i c i e n t : t h e re a re
n o g o l d e n b u l l e t s …
I o T a n d D a t a
B I G & s m a l l DATA re q u i re d
t o m o d e l & s y n t h e s i s e re a c t i v e
a n t i d o t e s a n d a n t i c i p a t o r y p ro t e c t i o n
… a n a u t o i m m u n e s y s t e m
i s re q u i re d t o c o p e w i t h
t h e I o T a n d s t o c h a s t i c
c o m p l e x i t y / a t t a c k s !

27. N E E D T O S H A R E
T R U M P S N E E D T O K N O W

28. We m i g h t s u p p o s e t h a t
t h i s wo u l d t a ke p l a c e i n
s e c o n d s o r m i n u t e s a n d n o t
h o u r s a n d d a y s …

30. How Many Devices ?
I m p l a n t e d , w o r n , c a r r i e d b y p e o p l e
To d a y I w e a r a n d c a r r y 6 s m a r t
c o m m u n i c a t i n g t h i n g s … b u t i n t h e
f u t u r e i t i s l i k e l y t o b e > > 1 0 ! !
H o w m a n y h a v e y o u g o t … . ? ?
T h e y a r e a l l a p a r t o f t h e I o T -
a n d m a y / c a n / d o c r e a t e B I G a n d
s m a l l d a t a …

31. Forbes
Projection
Based on an
Industry Average

32. Forbes
Projection
Based on an
Industry Average
A N I o T O F 5 0 B n T H I N G S
I S C O N S E R V A T I V E !

33. Stored information growth
1986
I can’t draw a
dot small
enough!!
.
1996
0.001ZB
2006
0.13ZB
2016
13ZB
IoT will accelerate this trend to promote ever more on device storage
ZetaByte = 1021 ExaB = 1018 PetaB = 1015 TeraB = 1012
Prognosis: IoT small data distributed
storage may exceed the
internet

34. 2014 2016 2018 2020 2022 2024
2
6
10
14
16
18
20
22
% Data Stored on Things
Year
Projection
Actual
Data extracted from
numerous reports,
papers & presentations
by IDC, EMC, ICS,
Stored ON THINGS

35. Thank You
cochrane.org.uk
ca-global.org