Le Streaming d'API : pourquoi et comment transformer vos APIs statiques en données temps réel ?

#DevoxxFR @Audrey_Neveu
The End of Polling
Why and how to transform a REST API
into a Data Streaming API

ABOUT ME
Developer Relations Co-Leader France
@Audrey_Neveu

MODERN TIMES

OUR GOAL

ANIMATION IS THE KEY... BUT WHY?

BECAUSE OF EVOLUTION

USER INTERFACE

KEEP CALM ...

KEEP CALM ... AND LEAVE.

REFRESH BUTTON IS EVIL

REAL-TIME USER EXPERIENCE

F.O.M.O (FEAR OF MISSING OUT)

API STREAMING

SOLUTIONS FOR REAL-TIME APPLICATIONS
✓ WebHooks
✓ (Long) Polling
✓ Web-Sockets
✓ Pub/Sub
✓ Server-Sent Events

IT’S ALL ABOUT FREQUENCY ...

… AND USAGE

(Long) Polling

POLLING

98.5% of polls are
wasted
Source:
http://resthooks.org/
POLLING MADNESS ™

98.5% of polls are
wasted
Source:
http://resthooks.org/
POLLING IS NOT A SOLUTIONMADNESS ™
(and neither is long polling)

WebHooks

✓ webhooks.org
✓ aka User Defined HTTP Callback
✓ concept / method
WEBHOOKS
✓

HTTP/1.1 201 Created
Link:<http://subscription-serve
r.com/subscription>;
rel="subscription"
@StreamdataIOstreamdataio
WEBHOOKS
POST /eventsource HTTP/1.1
Host: subscription-server.com
Pragma: subscribe
Callback:
<http://example.com/callback>;
method="POST"
rel="subscriber"
Consumer API Provider

HTTP/1.1 201 Created
Link:<http://subscription-serve
r.com/subscription>;
rel="subscription"
@StreamdataIOstreamdataio
WEBHOOKS
POST /eventsource HTTP/1.1
Host: subscription-server.com
Pragma: subscribe
Callback:
<http://example.com/callback>;
method="POST"
rel="subscriber"
POST /callback HTTP/1.1
Host: example.com
Link:
<http://subscription-server.com
/subscription>;
rel="subscription"
Content-HMAC: sha1
C+7Hteo/D9vJXQ3UfzxbwnXaijM=
Content-Length: 21
Content-Type:
application/x-www-form-urlencod
ed
payload=Hello%20world

✓ Define a callback URL
TODO LIST
✓ Implement your webhook queue
- inline HTTP Requests
- SQL-based queue
- AMQP broker
- batch

✓ Define a callback URL ✓ Create a subscription endpoint
- GET /webhook
- POST /webhook
- GET /webhook/{id}
- PUT /webhook/{id}
- DELETE /webhook/{id}
TODO LIST

CONSUMER SETUP

EVENT RECEIVED
POST /callback HTTP/1.1
Host: www.example.com
X-Github-Delivery:
72d3162e-cc78-11e3-81ab-4c9367
dc0958
User-Agent:
GitHub-Hookshot/044aadd
Content-Type: application/json
Content-Length: 6615
X-GitHub-Event: issues
Payload=
{
"action": "opened",
"issue": {
"url": "https://api.github.com/repos/octocat/Hello-World/issues/1347",
"number": 1347,
...
},
"repository" : {
"id": 1296269,
"full_name": "octocat/Hello-World",
"owner": {
"login": "octocat",
"id": 1,
...
},
...
},
"sender": {
"login": "octocat",
"id": 1,
...
}
}

PROS AND CONS
✓ Real-Time updates
✓ Easily consumed
✓ Without dedicated resources
■ Poor user experience
■ Does not work with all clients
■ Manual setup✓ Easily integrated
■ Debugging

PROS AND CONS
✓ Easily consumed
■ Debugging
✓ (almost) Real-Time updates

KNOWN ISSUES
■ DDoS Attack
■ Missed notification
■ DDoS Attack
■ Deduplication

CHECK LIST
✓ Implement authentication
✓ Expected answer?
✓ Monitor payload size
✓ Handle request number
✓ Handle duplicates
✓ One callback per webhook

DYNAMIC SUBSCRIPTIONS
✓ Restful Webhooks
✓ REST Hooks
✓ PubSubHubbub

PubSubHubbub
aka PubSub
aka PuSH

RSS / Atom feeds
pubsubhubbub/
Open Protocol
Based on Publish / Subscribe Pattern …
PUBSUB
… and on topics

SubscribersHub
PUBSUB
Publishers

Discovery
SUBSCRIBE - PUBLISHERS
Link: <https://my-hub.com/>; rel="hub"
Link: <https://my.resource.com>; rel="self"
<link rel="self" href="https://my-resource.com/">
<link rel="hub" href="https://my-hub.com/">
Publishers

SUBSCRIBE - SUBSCRIBERS 1/2
POST https://my-hub.com/
…
hub.mode="subscribe"
hub.topic="https://my-resource.com/"
hub.callback="http://example.com/callback"
hub.secret="my-token"
Subscribers Send subscription request to the Hub

SUBSCRIBE - HUB
GET http://example.com/callback
…
hub.mode="subscribe"
hub.challenge="a random string"
hub.lease_seconds=86400
Hub Verify intent of the subscribers

SUBSCRIBE - SUBSCRIBERS 2/2
HTTP/1.1 200 OK
Body:{
hub.challenge: "a random string"
}
Subscribers Answer verification request

PUBLISH - PUBLISHERS
…
hub.mode=publish
hub.url=https://my.updated-resource.com
Publishers Ping the Hub

PUBLISH - HUB
POST http://example.com/callback
Link: <https://my.updated-resource.com>; rel="self"
Hub Content Distribution

PUBLISH - HUB
POST http://example.com/callback
Link: <https://my.updated-resource.com>; rel="self"
X-Hub-Signature="my-sha1-signature"
Hub (Authenticated) Content Distribution

PUBLISH - SUBSCRIBERS
GET https://my.updated-resource.com
Subscribers Pull the updated resource

{
"object":"page",
"entry":[
{
"id":"51044240199134611",
"time":1447342027,
"changes":[
{
"field":"leadgen",
"value":{
"adgroup_id":0,
"ad_id":0,
"created_time":1447342026,
"leadgen_id":55459717045641545,
"page_id":516540199134611,
"form_id":551111744595541
}
}
]
}
]
}
EVENT RECEIVED

UNSUBSCRIBE - SUBSCRIBERS 1/2
…
hub.mode="unsubscribe"
hub.callback="http://example.com/callback"
Subscribers Send unsubscription request to the Hub

UNSUBSCRIBE - HUB
GET http://example.com/callback
…
hub.mode="unsubscribe"
hub.challenge="a random string"
Hub Verify intent of the subscribers

UNSUBSCRIBE - SUBSCRIBERS 2/2
HTTP/1.1 200 OK
Body:{
hub.challenge: "a random string"
}
Subscribers Answer verification request

WEBHOOKS PROS AND CONS
✓ Easily consumed
■ Debugging

WEBHOOKS PROS AND CONS
✓ Easily consumed
✓ Easily integrated
■ Debugging
■ Need another call to get data

WEBHOOK VS PUBSUB

Push Technologies

PUSH TECHNOLOGIES
Server-Sent Events
2008 2006
W3C Specification
Web-Sockets

PUSH TECHNOLOGIES
Text + Binary Text
Server-Sent EventsWeb-Sockets

PROTOCOLS
GET /chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Protocol: chat
Sec-WebSocket-Version: 13
GET /stream HTTP/1.1 1
Host: example.com
Accept: text/event-stream
Web-Sockets
(RFC-6455)
Server-Sent Events
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
Sec-WebSocket-Protocol: chat
Sec-WebSocket-Version: 13

CONFIGURATION

MESSAGES FORMAT
var msg = {
type: "message",
text: "Hello Devoxx
France!",
id: 12345,
date: Date.now()
};
data: Hello Devoxx France!
data: with two lines
data: {"time": "16:34:36", "text": "Hello Devoxx
France!"}
id: 12345
event: foo
retry: 10000

IMPLEMENTATION
var websocket =
new WebSocket
('ws://websocketserver/echo');
var eventSource =
new EventSource
('http://sseserver/echo');
websocket.onOpen = function(){
...
};
eventSource.onopen = function(){
...
};
websocket.onMessage = function(e){
var data = e.data;
var message = data.msg;
...
};
eventSource.onMessage = function(e){
var message = e.data;
...
};
websocket.onError = function(e){
...
};
eventSource.onError = function(e){
...
};

IMPLEMENTATION
var websocket =
new WebSocket
('ws://websocketserver/echo');
var eventSource =
new EventSource
('http://sseserver/echo');
websocket.onOpen = function(){
...
};
eventSource.onopen = function(){
...
};
websocket.onMessage = function(e){
var data = e.data;
var message = data.msg;
...
};
eventSource.onMessage = function(e){
var message = e.data;
...
};
websocket.onError = function(e){
...
};
eventSource.onError = function(e){
...
};
...
eventSource
.addEventListener('foo',function(e){
// do something
}, false);
eventSource
.addEventListener('bar',function(e){
// do something else
}, false);

LOST IN CONNECTION

BROWSER SUPPORT
57. 52. 10. 44. 11. 14. 57. 52. 10. 44. 11. UC.
source : http://caniuse.com/

MOBILE BROWSER SUPPORT
source : http://caniuse.com/
57. 51. 10.3. all. 56. 57. 51. 10.3. all. 56.

PERFORMANCES
8s 5s x1.6
8s 6s x1.3
16s 7s x.2.2
source: http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/
SSEWeb-Sockets Diff

CHOOSE WISELY

Proxy-as-a-Service
✓ works with any JSON API
✓ streaming based on Server-Sent Events
✓ dynamic cache
✓ incremental updates
STREAMDATA.IO

JSON-PATCH (RFC-6902)
[{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
{"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
{"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
[{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
{"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
[{"op":"replace","path":"/2/price","value":5},
{"op":"replace","path":"/3/param2","value":"32"}]

VOTE SERVER-SENT EVENTS!

THANKS!
Q&A

Le Streaming d'API : pourquoi et comment transformer vos APIs statiques en données temps réel ?

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (19)

Destaque

Destaque (7)

Semelhante a Le Streaming d'API : pourquoi et comment transformer vos APIs statiques en données temps réel ?

Semelhante a Le Streaming d'API : pourquoi et comment transformer vos APIs statiques en données temps réel ? (20)

Mais de Audrey Neveu

Mais de Audrey Neveu (6)

Último

Último (20)

Le Streaming d'API : pourquoi et comment transformer vos APIs statiques en données temps réel ?