Philippe Coval is a software engineer working on open source projects around Web of Things and virtual reality. He is exploring blending IoT and extended reality by connecting physical devices to virtual worlds. He discusses the A-Frame and WebThings frameworks for building VR/AR experiences and programming internet-connected devices. Coval demonstrates how to implement a WebThing and control a servo motor from a VR scene using these technologies.
Testing tools and AI - ideas what to try with some tool examples
digital-twins-webthings-iotjs-20190512rzr
1. Samsung Open Source Group
1
https://social.samsunginter.net/@rzr
WebThings in VR
…towards ”DigitalTwins”
#GrafikLabor, Rennes, France <2019-05-12>
https://afgral.org/grafiklabor-2019
Philippe Coval
Samsung Open Source Group / SRUK
pcoval@samsung.com
2. Samsung Open Source Group
2
https://social.samsunginter.net/@rzr
$ who is Philippe Coval
●
Software engineer for Samsung OSG
– Belongs to SRUK team, based in Rennes, France
– Interest: Web of Things with “Privacy by Design”
●
Some past 3D experiences (OpenGL, VRML, ArtoolKit+, etc)
●
After FOSDEM 2019, I was inspired to blend IoT and XR
●
Join the fediverse: https://social.samsunginter.net/@rzr
3. Samsung Open Source Group
3
https://social.samsunginter.net/@rzr
Immersive Web
4. Samsung Open Source Group
4
https://social.samsunginter.net/@rzr
Immersive Web
●
A successful evolution: 3D, VR, AR, XR
– HTML5, WebGL, Three.js, WebVR, OpenVR, OpenXR and WebXR !
●
Several FLOSS frameworks:
– Three.js: High-level library on top on WebGL with WebVR support (for devices)
– A-Frame: Originally from Mozilla, HTML for VR (built on Three.js)
– Babylon.js: from MS and more ...
●
Support: most desktop Web Browsers and VR browsers (for headsets)
– I use GearVR2017 (with controller)
●
Runs SamungInternet for VR (chromium based)
5. Samsung Open Source Group
5
https://social.samsunginter.net/@rzr
A-Frame’s static scene in DOM
<script src="https://aframe.io/releases/0.9.0/aframe.min.js">
</script>
<a-scene>
<a-box
color='blue' rotation="0 42 0">
</a-box>
<a-camera position="1 1 3">
</a-camera>
</a-scene>
6. Samsung Open Source Group
6
https://social.samsunginter.net/@rzr
A-Frame’s static scene in DOM
<script src="https://aframe.io/releases/0.9.0/aframe.min.js">
</script>
<a-scene>
<a-box id='foo’
color='blue' rotation="0 42 0">
</a-box>
<a-camera position="1 1 3">
</a-camera>
</a-scene>
<script>
var el = document.getElementById("foo");
el.setAttribute("color", "red"); // via DOM API
el.object3D.rotateY(43); // via Three.js API
</script>
9. Samsung Open Source Group
9
https://social.samsunginter.net/@rzr
Mozilla WebThing Project
●
An open platform for monitoring and controlling devices over the web.
– With “Privacy by Design” (data stay home)
●
Inspired by Web of Things works (Web Tech for IoT)
– RESTful API, Schema, Data model (JSON)
– Proposed to W3C
●
Implemented in Mozilla’s WebThings gateway
– To connect and connect webthings with “Privacy by Design” (data stay home)
– To be controlled from Web UI
10. Samsung Open Source Group
10
https://social.samsunginter.net/@rzr
Webthings are standalone HTTP servers
●
Exposing Mozilla Things API (REST, JSON)
– Self described using Things schema
●
by properties which contains values:
– Example: a MotorThing has an AngleProperty of value 42
●
Can be implemented for many runtimes (C, Python, Java, JS…):
– Physical objects
●
Internet access is not required (uses home network/LAN)
– Or services? can be deployed anywhere
●
on gateway as adapters: privacy matters!
●
on cloud as micro services?
– Eg: for development purpose try glitch.com
11. Samsung Open Source Group
11
https://social.samsunginter.net/@rzr
Angle/Motor example simulator
●
curl https://rzr-webthing-example.glitch.me/
{"name":"Angle" …
"properties":{"angle":{"title":"Angle","type":"number"…
"links":[{"rel":"property","href":"/properties/angle"…
●
curl https://rzr-webthing-example.glitch.me/properties
{"angle":0}
●
curl -X PUT -d '{"angle": 42}' https://rzr-webthing-example.glitch.me/properties/angle
{"angle": 42}
12. Samsung Open Source Group
12
https://social.samsunginter.net/@rzr
●
var webthing = require('webthing-iotjs');
●
function AngleProperty(thing){
webthing.Property.call(this, thing, 'angle',
new webthing.Value(0), { type: 'number' });
}
●
var thing = new webthing.Thing('Motor');
thing.addProperty(new AngleProperty(thing));
●
var server = new webthing.WebThingServer(new webthing.SingleThing(thing),
8888);
●
server.start();
Implementing a WebThing Actuator
13. Samsung Open Source Group
13
https://social.samsunginter.net/@rzr
Deploy to micro controllers using IoT.js
●
IoT.js an alternative runtime inspired by Node.js:
– Powered by JerryScript engine optimized for micro-controllers
– Supporting JS modules: mastodon-lite, generic-sensors-lite
●
WebThings can be built using webthing-iotjs module:
– Supporting IoT.js runtime:
●
OS: RT, NuttX, GNU/Linux …
●
MCU: ARTIK05X, STM32*
– Or use Node.js runtime:
●
for development/simulation (glitch)
– Note: webthing-iotjs is based on webthing-node
14. Samsung Open Source Group
14
https://social.samsunginter.net/@rzr
●
SG90: 9g Micro Servo
– Angle: [-90°, +90°]
– I/O:
●
Power: 4.8V, GND
●
PWM Signal
Servo motors and PWM
●
Pulse Width Modulation (PWM):
– Constant frequency: 1/.02
– Variable duty cycle:
●
[1 ms = -90°, 2 ms = +90°]
●
(angle + 90 / 180) + 1
SG90
15. Samsung Open Source Group
15
https://social.samsunginter.net/@rzr
●
Nucleo-F767ZI
– Nucleo-144 I/O
●
32.768 Khz
●
Ethernet
●
4 PWMs *
– STM32F7
●
ARM Cortex M7
●
RAM: 320KB
●
Supporting
– NuttX
●
* nuttx-7.29-115-gf06a2cabb5+
●
Origin of TizenRT
– TizenRT possible
– Arduino API
Micro Controller
PWM1.CH1_1 @PA8
(CN12: 12th
from top on right)
PWM2.CH1_1 @PA0
(CN10: 3th
from bottom on left)
PWM3.CH1_1 @PA6
(CN7: 6th
from top on right)
GND
PWM4.CH1_1 @PB6
(CN12: 9th
from top on left)
16. Samsung Open Source Group
16
https://social.samsunginter.net/@rzr
Generating a PWM using IoT.js
●
Example:
●
Note: dutyCycle [0, 1] adapt Row 1 Row 2 Row 3 Row 4
0
2
4
6
8
10
12
Column 1
Column 2
Column 3
var pwm = require('pwm');
var config =
{ pin: 0,
period: 20,
dutyCycle: 0.5}
var port = pwm.open(config,
function(err) {
port.setDutyCycle(0.42);
});
17. Samsung Open Source Group
17
https://social.samsunginter.net/@rzr
From webthings…
to “Digital Twins”
18. Samsung Open Source Group
18
https://social.samsunginter.net/@rzr
Updating XR view from webthing
●
function update(id, type, url) {
fetch(url)
.then( response => { return response.json(); } )
.then( json => {
for (var property of Object.keys(json)) {
document.getElementById(id).setAttribute(type, property, json[property]);
}})
.catch(function() { console.log('error: ') })
}
●
var url = 'https://rzr-webthing-example.glitch.me/'; // For simulator
// url = ‘http://webthinghost:8888’; // For device
update('foo’, 'motor', url + ‘/properties’);
●
curl -X PUT -d '{ "angle": 5}' $url/properties/angle
<a-scene>
<a-entity id="foo" motor="angle: 45"></a-entity>
<a-camera position="1 1 3"></a-camera>
</a-scene>
19. Samsung Open Source Group
19
https://social.samsunginter.net/@rzr
Digital Twins with WebThing-IoTjs (on MCU)
https://purl.org/rzr/digitaltwins-webthings-iotjs-20190512rzr
20. Samsung Open Source Group
20
https://social.samsunginter.net/@rzr
Live control in 3D using A-Frame on GearVR:
https://youtu.be/s3r8pQtzhAU#wotxr-20190320rzr
22. Samsung Open Source Group
22
https://social.samsunginter.net/@rzr
Controlling real devices / consuming OpenData
https://purl.org/rzr/webthing-iotjs-opendata-20190202rzr