The document discusses the W3C Device APIs working group and various device APIs they are standardizing. It provides an overview of APIs for media capture, camera/microphone access, battery status, vibration, network service discovery, and picking media. It discusses the status and support for each API as well as examples. It also covers topics like Web Intents, sensor APIs, network information, and the new SysApps working group. The goal is to standardize browser-based JavaScript APIs to access device hardware and services.
W3C Device APIs: What are they, how to use them and where are they going
1. W3C Device APIs
What are they, how to use them and where are they going
Diana Cheng
TakeOff Conference
Lille, January 2013
2. Diana Cheng – Vodafone Group, R&D
diana.cheng@vodafone.com
daianacheng
W3C DAP and SysApps member, formerly
also Geolocation
January 2013 Take Off Conf – Device APIs 2
3. Contents
• Introduction: DAP Working Group and scope
• DAP specs and APIs (with code and demos)
• HTML Media Capture
• getUserMedia
• Battery Status
• Vibration
• Network Service Discovery
• Pick Media Intent
• Web Intents
• Other Intents-based APIs
• Network Information
• Conclusions: What now and future work
• Questions?
January 2013 Take Off Conf – Device APIs 3
4. W3C DAP Working Group
• Device APIs working group
• Scope of the group and its specifications has changed over time: recharted
in 2011
• Edits a number of specs with different degrees of maturity
• Focuses on APIs that run in the browser security context and are therefore
sandboxed
• Works also on „Task Forces“ together with the Web Intents and WebRTC
WGs
January 2013 Take Off Conf – Device APIs 4
5. So what are Device APIs?
• Browser-based Javascript APIs
• Provide access to the device hardware and services
• Allows us to build rich Web applications Making the Web THE platform!
• A lot of people seem to consider them part of HTML5
Such functionality was previously only available to native apps
PhoneGap, now Apache Cordova, was started to fill this void in mobile devices
January 2013 Take Off Conf – Device APIs 5
6. Device APIs
Specification Status Desktop Mobile
Implementations Implementations
HTML Media Capture LC Draft ✔✔✔ ✔✔✔
getUserMedia (WebRTC) PW Draft ✔✔✔
Battery API CR ✔ ✔
Vibration API CR ✔
Network Service Discovery PW Draft ✔
Contacts: Pick Contacts Intent PW Draft ?
“Gallery”: Pick Media Intent PW Draft ?
Proximity API LC Draft ✔
Ambient Lights API LC Draft ✔
Network Information Pretty ?
bad?
PW Draft
January 2013 Take Off Conf – Device APIs 6
7. HTML Media Capture (1/2)
<input type="file" accept="image/*;capture=camera” multiple>
• extends the HTML Input Element with a capture attribute
• capture is a hint for the source of the media.
• Possible values: camera, camcorder, microphone, filesystem
• Media type could be: image, video, audio
• multiple won‟t work on mobile devices due to the camera interface
January 2013 Take Off Conf – Device APIs 7
8. HTML Media Capture (2/2)
Support:
Desktop Safari, Firefox, Chrome, Opera latest versions
Mobile Chrome for Android, Firefox for Android, Android 4.0 stock
browser, Blackberry browser
My own experience:
• Desktop versions tested on Mac
• Opera Mobile seems to provide no real access to camera, only to device
Gallery
• Demo won‟t work in Opera Desktop (FileReader problem)*
* Opera for Mac 12.1 has a bug in the FileReader‟s getDataAsURL and the demo for this API won‟t work
there.
January 2013 Take Off Conf – Device APIs 8
9. getUserMedia (WebRTC) (1/3)
navigator.getUserMedia (constraints, successCallback, errorCallback);
• Access to the device‟s camera and microphone on the Web today!
• Farewell to plugins!
• Call will prompt the user for permission and invoke the errorCallback if
denied
• Can request a LocalMediaStream with audio, video or both
constraints = {video: true, audio: true}
• Still prefixed in several implementations
January 2013 Take Off Conf – Device APIs 9
11. getUserMedia (WebRTC) (3/3)
My own experience:
• Support in Opera Mobile 12.10?
• Making it work cross-browser requires considering the prefixes and careful
testing
• spec keeps on changing: e.g. MediaStreamRecorder out? currently no
standard way to record a device‟s live stream:
recorder = localMediaStream.recorder();
localMediaStream.stop();
recorder.getRecordedData(function(blob) { //do smth…});
(see recorder demo for a workaround)
January 2013 Take Off Conf – Device APIs 11
12. Battery Status API (1/2)
navigator.battery
• Programmatic access to the device‟s battery level and events!
• Avoid tasks in our Web apps that are not suitable for a low battery level:
Power-efficient Web apps!
• Example use cases:
• avoid network operations like syncing with the server, or do them with a lower
frequency
• Avoid power-consuming, processor-intensive animations
January 2013 Take Off Conf – Device APIs 12
13. Battery Status API (2/2)
• Attributes: level (]0 - 1.0]), charging, chargingtime*, dischargingtime*
chargingtime = Infinity (battery is discharging)
chargingtime = 0 (battery is full)
dischargingtime = Infinity (battery is charging)
• Events: levelchange, chargingchange, chargingtimechange,
dischargingtimechange
• How often the update takes place (events fire) is left to the implementation
• Support: Firefox and Firefox for Android
• My own experience:
• Couldn‟t be simpler to use! But it‟s not too easy to test
* in seconds
January 2013 Take Off Conf – Device APIs 13
14. Vibration API (1/1)
navigator.vibrate(time);
navigator.vibrate(pattern);
• Programmatically trigger feedback in the form of vibration
• Particularly nice for mobile games; not aimed for generic notification
• time is in milliseconds
• pattern is an array of consecutive vibration durations, e.g. [1000, 500, 2000]
• navigator.vibrate(0) or navigator.vibrate([ ]) stop the vibration
January 2013 Take Off Conf – Device APIs 14
15. Network Service Discovery API (1/2)
navigator.getNetworkServices(type, success, error);
• Asynchronous discovery of local HTTP-based services in Web Apps via a
single API call!
• Common discovery protocols: SSDP (UPnP), DNS-SD (ZeroConf)
• Use cases:
• Remotely controlling local network appliances from Web Apps, e.g. Web-based
TV remote control
• Browse content in a local network and select the appropriate renderer among
those found
• type:
• „upnp:urn:schemas-upnp-org:service:ContentDirectory:1‟,
urn:schemas-upnp-org:service:RenderingControl:1
• zeroconf:_boxee-jsonrpc._tcp
January 2013 Take Off Conf – Device APIs 15
16. Network Service Discovery API (2/2)
• After discovery, the URL of a requested service is whitelisted by the browser
for the given context; this allows to do cross-site requests via AJAX, Web
Sockets, etc.
• Events for changes in service reachability: onserviceonline, onserviceoffline
• Only implementation at the moment is an experimental build by Opera
supporting only UPnP and with no UI for user consent yet
http://dev.opera.com/articles/view/network-service-discovery-api-support-in-
opera/
(two very cool demos there too)
• My own experience:
• The Opera build works as expected
• Familiarity with the networking protocols (and XML/SOAP for UPnP)
required
January 2013 Take Off Conf – Device APIs 16
17. Pick Media Intent
“The Pick Media Intent defines a Web Intent …”
Web Intents?
January 2013 Take Off Conf – Device APIs 17
19. Web Intents
The Problem in the Web today
Sharing
1001 APIs to deal with…
January 2013 Take Off Conf – Device APIs 19
19
20. Web Intents fixes that
• Applications register their ability to handle an Intent (action). They express a
certain capability
<intent action="http://webintents.org/share" type="image/*" />
• A given app can delegate functionality it doesn‟t support to any of those apps
that have registered it
var intent = new Intent();
intent.action = 'http://webintents.org/share';
intent.type = 'image/*';
intent.data = [imgData];
window.navigator.startActivity(intent);
January 2013 Take Off Conf – Device APIs 20
20
21. Web Intents fixes that
• Users select which app to use
• The original site will then send the payload over to the handling app, and will
get the result back in a callback
window.intent.postResult([ /*... some data ...*/ ]);
January 2013 Take Off Conf – Device APIs 21
21
22. Pick Media Intent (1/2)
var intent = new Intent({ action: "http://webintents.org/pick",
type: "http://w3.org/type/media",
extras: { search: "olympic",
filters: ["title", "description", "author”],
limit: 100 }});
• Defines a new Web Intent in order to access the user‟s media on the device
or in the cloud
• Intent action: pick
• Intent type: media
• Extras: hints for the Intents Service
• Why: users store media on a variety of cloud services and Web apps should
not require prior knowledge of them. Local or remote, the sources (services)
just need to be registered with the browser
January 2013 Take Off Conf – Device APIs 22
23. Pick Media Intent (2/2)
• The spec also defines a Media dictionary with the relevant Metadata
• Other DAP specs which are based on Web Intents:
• Pick Contacts Intent
• Web Intents Addendum for Local Services: Local Services (UPnP and mDNS)
register as Service Intents, i.e. their ability to perform e.g. media playback
January 2013 Take Off Conf – Device APIs 23
24. Sensor APIs
• Proximity Events API
• Ambient Lights API
• More coming soon: Temperature? Atmospheric Pressure?
January 2013 Take Off Conf – Device APIs 24
25. And few issues with ….
January 2013 Take Off Conf – Device APIs 25
27. Network Information API (2/4)
Premise: bandwidth on mobile is still a constraint
• Provide an API that gives information about the speed of the connection the
device is currently on
• Use cases:
• help us serve the right images to the device according to the available bandwidth
(as it is done according to screen size and resolution)
• avoid initiating network-hungry requests if they are unlikely to be completed
• Interesting research by PPK:
http://www.quirksmode.org/blog/archives/2012/11/principles_of_m.html
January 2013 Take Off Conf – Device APIs 27
28. Network Information API (3/4)
What information to provide:
1. Connection type vs. bandwidth (actual speed)
Connection type Bandwidth
• can be obtained from the • tricky to measure
underlying OS • power and data consuming
• can be a poor indicator. e.g : a • highly dependent on both the
crowded conference Wifi might client and the server
offer slower connectivity than 3G • can change quickly as the user
moves in/out/inside buildings, on
a fast train, etc.
• a single “sample” might not be
representative so as to make a
decision
January 2013 Take Off Conf – Device APIs 28
29. Network Information API (4/4)
2. roaming vs. metered
roaming metered
• user agent could obtain this from • in other words: is the user on a
the underlying OS limited data plan? e.g: 1GB/month
• definite indicator that networking • useful but this information is hard
should be limited? the user might to obtain
not care about costs: e.g. I have a • the user could be prompted but
European flat rate from my they can provide wrong
employer, other people might information
have a lot of money
• Should developers be making this kind of decisions?
• Should we rely on the browser to do batched requests (if indicated by the
developer) in order to preserve a low-power radio state?
…..SUGGESTIONS?
January 2013 Take Off Conf – Device APIs 29
30. And whatever happened to…
• Messaging
• Calendar
• System Information
• Gallery
• Telephony …?
January 2013 Take Off Conf – Device APIs 30
31. SysApps Working Group
http://www.w3.org/2012/sysapps/
• A new scope: a different runtime environment and security model and APIs
to build system-level Web Apps
• Standardize and converge specifications inspired by projects such as
Mozilla‟s Firefox OS and Tizen
January 2013 Take Off Conf – Device APIs 31
32. What can you do?
Provide your feedback!
• DAP Mailing list: http://lists.w3.org/Archives/Public/public-device-apis/
• SysApps Mailing list: http://lists.w3.org/Archives/Public/public-sysapps/
or join?
January 2013 Take Off Conf – Device APIs 32
Devices in this context include desktop computers, laptop computers, cellular phones, TVs, camera, etc.richer and more context-aware Web applications.
Joint work with the WebRTC Working Group. DAP not concerned with other APIs part of WebRTC such as PeerConnection or DataChannel API
window.requestAnimationFrame
window.requestAnimationFrame
My experience: the only mobile browser that supports this is FF for Android. Even though it “landed” on WebKit Chrome for Android doesn’t include it.
-If the user declines, the error callback is invoked…
-Browse buttons in the demos could be disabled if the service gets disconnect When requesting a service: onserviceavailable;onserviceunavailable;When comunicating with a onserviceonline;onserviceoffline;
Started as Inter-app communication mechanism; now “everything” can be done with Web Intents Has been strongly pushed by Google. They currently face a few issues with UI and UXNative support therefore removed from Chrome for now (o_o)Mozilla’s counterproposal: Web Activities; will feature in Firefox OS R1
This approach moved away from the original idea of providing read/write access to the device’s gallery.
Also: User contacts can be store in a variety of sources, such as cloud not just in the deviceThe scope for Web Intents have changed from a simple inter-app communication mechanism to doing a bit of everything, including seamless access to local and remote sensors, as well as local network services
What happens when you measure a bandwidth against the host of the web app but an image will actually be retrieved from a CDN…
What happens when you measure a bandwidth against the host of the web app but an image will actually be retrieved from a CDN…Metered is only really known by the provider and “some users”. Metered is also not a great name for the attribute…
What happens when you measure a bandwidth against the host of the web app but an image will actually be retrieved from a CDN…Metered is only really known by the provider and “some users”. Metered is also not a great name for the attribute…
Stronger integration with the host platform Typically platform-certified appsDAP: Web apps that run in the browser runtime, i.e. can cope with users visiting untrusted sites-> cautious approach narrows the scope of what this Apps/APIs can doEx: contacs picker vs. contacts manager