In our current world, we need to manage very big systems with a huge number of assets. In this context, SOGETI developed by leveraging IBM software and IBM Cloud solutions (Bluemix, Watson) a cockpit to control and command those systems of systems. This session will present one of our first implementations of Cockpit for Big Systems (CBS) and Cockpit for Big IoT Systems (CBIoTS) for precision farming with Drotek—a new solution for better analysis and control of crop production with improved efficiency and reduced environmental impact.
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Cockpit for Big Systems and Big IoT Systems Leveraging IBM Bluemix and Watson
1. Local Touch - Global Reach.
Intelligent System for Precision Farming
A partnership project between Drotek & Sogeti
Leveraging IBM Bluemix and Watson Technologies
World of Watson – Octrober 24-27 – Las Vegas
SOGETI France - 2016
IBM World of Watson 2016
Conference
Session ID: IOT-2052
2. Partnership Project – IBM World of Watson 2016 Conference
2 The Speakers
Patrick started programming on a Casio FX-602P and after on HP-41C, a
long time ago! After more than a quarter century working on machines
for people and large organizations, as a developer, as an architect, as a
project manager or consultant he is still fascinated by the beauty and by
the monstrous and elusive nature of complex systems, and in particular
socio-technical systems. In 2012, as an innovation leader at Sogeti and
members of the SogetiLabs, he launched five R&D programs using big
data, software system testing, multi-agent systems, cloud computing,
and project management as fields of experiments. His conviction is that
instead of undergoing complex systems we must try to understand
them, better control them, innovate, and do business with that.
Kevin is Drotek’s CTO & Associate. Technology aficionado, late night
explorer, language learning addict, he is a real Swiss army knife for the
start up. He loves it when someone comes up with a new « out of the
blue » idea (if he doesn’t himself) and he has to design a prototype for it
from scratch. Fast. Innovate. Iterate. That’s his motto. Skydiver, pilot,
there seems to be no limit for him. He strongly believes that without risk
there can be no success. Most important thing about his personality is
that he is relentlessly eager to learn. He has proved his ability to solve
complex problems in record time, as an intern he designed, developed
and started to sell a disruptive product in less than 6 months.
3. Partnership Project – IBM World of Watson 2016 Conference
3 Once upon a time …
At different scale …
But still complex systems !
With unpredictable behavior
and evolution!
Complexity !
4. Partnership Project – IBM World of Watson 2016 Conference
4
ConstituentEcosystem
Why Complexity? Which issues?
System
(of interest)
Constituent
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
applies toapplies to
applies toapplies to
aggregation of
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
Behavior
Evolution
Plurality
Variety
Hierarchy
Feedback
Speed of
changes
Emergence
of Behavior
FACTS
Representation
& Modelization
Calculability:
Optimization
& Prediction
ISSUES
5. Partnership Project – IBM World of Watson 2016 Conference
5
ConstituentEcosystem
System
(ofinterest)
Constituent
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
applies toapplies to
applies toapplies to
aggregation of
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
System Constituant
Behavior
Evolution
Behavior
Evolution
The Agent and Multi-Agent Based System Paradigm
Associate System-Agent with Software-Agent
Wherever and Whenever possible Models based Agents
6. Partnership Project – IBM World of Watson 2016 Conference
6 Control and Command of Complex Systems
Control
from
Sensors
Command
to
Actuators
Intelligence
7. Partnership Project – IBM World of Watson 2016 Conference
7
Historically,CBSstartedwiththeneedtocontrol-commandandmanageVeryLargeSystemsof
Systems(~100Kmachines)intheITarea,withamuchlargernumberofassets (~20managedassets
permachine)andUltraLargeScaleSystemasfinaltarget(morethan1millionmachines~20
millionassets).Ourconcernswere:
• Efficient functioning with the same quality at different scales
• Inherent heterogeneity of management of assets and diversity of modes to communicate with each others
• Adjustment to the fine granularity of information in order to follow in real time the business needs
• Orchestration control of the systems of systems
Webuiltagenericinfrastructuresystem,adaptabletobusinessneeds,andtousecases.
CBSisOKwithVLSS,butwearestillrunningattheULSS.
Weneedmorescalability,elasticityandfastapplicationdevelopmentforexperiments.
WeneededsomethinglikeBluemix!
Cockpit for Big System (CBS): The First Step
8. Partnership Project – IBM World of Watson 2016 Conference
8
• Currently,eachinstanceofCBScanmanage~30000machines(morethanhalfmillionassetsmanaged)
• Monitoring information are retrieved each 15 minutes, then consolidated in a daily inventory
3Go of data consolidated daily
4To of data on a 18 months window
• +250000assetsinventoriedinlessthan5minutesonacommonLAN/WAN
• ~200uniqueinventorykeys,allowingtorepertory,e.g.,morethan700applications,severalOS,CPU,GPU,network
interfacecontroller,…Basedonagenericinventorykeyssystem,newsensorsoractuatorsinthenewversionof CBS
calledCBIoTScanbeaddedinsomeminutes.
• +100KSLOC ofJavascript,Perl,C,sh
• +10yearsofmaturityoftheagentsoftware,whichmemoryfootprintis~500KoonLinuxand~1MoonWindows
• +20environments/OShandled(Unix,Linux,Windows,Android,RaspberryPI,Yocto).SoononIOSandOSX
• Allcommunications(agent-agent,agent-platform)usekeyauthentication(AES)andcommunicationsencrypted(RSA)
Cockpit for Big System: Some Metrics
9. Partnership Project – IBM World of Watson 2016 Conference
9 Cockpit for Big Systems: Architecture Overview
CBS: Web Server Portal
• Apache
• Js/HTML
• Widgets
• Perl
CBS: Databases Cluster
• SQL
• NoSQL
• Graph
CBS: Consolidation Servers
• Consolidation
• Operations automation
CBS: Data Collection Servers
• Data Collection
• Data Integration
• Listener
External Repositories
• LDAP
• CMDB
• …
CBS: Secure Gateway
• End-to-end connection
• Encrypted tunnel
• Encrypted communication
CBS real time compatible Agent
• UNIX (Oracle, HP , IRIX, …)
• LINUX (Red Hat, Cent OS,
Debian, Raspbian, …)
• MS WINDOWS (all)
• Yocto (Intel Edison)
• Andoid
• Soon on IoS and OSX
IBM Bluemix
• Docker containers
• DB Server
• Services
10. Partnership Project – IBM World of Watson 2016 Conference
10
TheIoTExperiments:reachtheULSS
andtakeotherkindofsocio-technicalSoSup
Cockpit for Big IoT Systems: The New Stage of CBS
• Instantiation of the CBS platform for very big IoT systems
• Processing of Big Data
• Retrieve of data and function determination given the
sensors
• Action on the environment using actuators
• A sensor (or actuator) = a function
Functional composition and triggered actions
CBIoTS
Engine
CBSAgent
Sensors/Actuators
upto20/unit
DevicewithUnix,
Linux,Windowsor
AndroidOS
CBSAgent
Sensors/Actuators
upto20/unit
DevicewithUnix,
Linux,Windowsor
AndroidOS
CBSAgent
Sensors/Actuators
upto20/unit
DevicewithUnix,
Linux,Windowsor
AndroidOS
CBSAgent
Sensors/Actuators
up
to
20/unit
Device
with
Unix,
Linux,W
indowsor
Android
OS
CBSAgent
Sensors/Actuators
up
to
20/unit
Device
with
Unix,
Linux,W
indowsor
Android
OS
CBSAgent
Sensors/Actuators
up
to
20/unit
Device
with
Unix,
Linux,W
indowsor
Android
OS
CBSAgent
Sensors/Actuators
up
to
20/unit
Device
with
Unix,
Linux,W
indowsor
Android
OS
Environment
Environment
Environment
11. Partnership Project – IBM World of Watson 2016 Conference
11
Business
Application
Business
Application
Cockpit for Big IoT Systems: Architecture Overview
Up to 30 000 units
encrypted
encrypted encrypted
encrypted
CBIoTS Platform
Sensors or Actuators
(up to 20/unit)
Device OS
CBS Agent
IBM Watson
Business
Application
Data (cloudified (IBM Bluemix) or on-premise)
High Data Streaming on IBM Bluemix
CBIoTS Engine on IBM Bluemix
Feedback Loop Feedback Loop
Feedback Loop
Up to half million
sensors/ actuators under
control/ command per
CBioTS Engine instance
Device OS
CBS Agent
The IoT Experiments:
reach the ULSS and take
other kind of socio-technical
SoS up
Use of Watson as a Global
Feedback Loop
Use CBIoTS on Bluemix &
Watson as an Infrastructure
to quickly develop intelligent
IoT business applications
12. Partnership Project – IBM World of Watson 2016 Conference
12
• Very LargeIT Systems monitoring (doneandrunningon-premise)
• DevOps (doneandrunningonBluemix,seeNADOproject)
• Smart Farming (inprogress,seepartnershipwithDrotek)
• Smart Cities (ongoingprospection)
• Smart Villages(inprogress,someservicesareinpartnershipwithDroteck)
• Home automation
• Transport (ongoingprospection)
• Betterunderstanding of System of Systems and Complex Systems (Currentlinkswith
researchlaboratoriesinToulouse)
• …
CBS/CBIoTS: Use Cases
13. Partnership Project – IBM World of Watson 2016 Conference
13 CBS/CBIoTS: Partnership with IBM
Special thanks to the IBM Architect Team in India
Which are helping us in the porting of the CBS/CBIoTS platform on Bluemix
Many thanks to all the people involved in our partnership with IBM
19. Partnership Project – IBM World of Watson 2016 Conference
19
IBM Bluemix
ISPF Portal
Drotek embedded system
&
CBIoTS
A fully distributed Architecture
based on Embedded Systems, Multi-
Agent and Cloud Technologies
TCP, LoRa, Sigfox
ISPF: Architectureoverview
21. Partnership Project – IBM World of Watson 2016 Conference
21 ISPF Measurable Metrics
22. Partnership Project – IBM World of Watson 2016 Conference
22 ISPF: Main script
+
+
Network
Network A
A
Network+1
2
3
1 Data collection 2
Automation with:
• Context awareness
• Smart behavior
3
Analytics & Forecasting for
• Context awareness
• Smart content
23. Partnership Project – IBM World of Watson 2016 Conference
23 ISPF: Functional Overview
24. Partnership Project – IBM World of Watson 2016 Conference
24 Intelligent System for Precision Farming
25. Partnership Project – IBM World of Watson 2016 Conference
25 Thank you
Thank you
For more details contact:
Patrick Marquet
Practice Leader Enterprise Solutions Consulting - Sogeti France
patrick.marquet@sogeti.com