O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese an...
Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the sprea...

 Tesla

 mid 2000s





 LINQ to Events
Nikola Tesla
www.knowledgeoftoday.org

 Tier-splitting
 metadata







 asynchronous
www.Wikipedia.org



 Events are not first-class objects

[RunOnClient]
public event EventHandler<MouseEventArgs> MouseMoved;
// Runs i...

 delegates

Action a = new Action(Foo); // explicit creation of delegate instance
Action b = Foo; // method group conv...

 duality enumerator

interface IObservable<out T>
{
IDisposable Subscribe(IObserver<T> observer);
}
interface IObserve...

 Composition
 Composition
interface IScheduler
{
IDisposable Schedule(Action work);
…
}
static class Observable
{
stat...
 Function composition
www.Wikipedia.org
public static class Observable
{
public static IObservable<T> Merge<T>(this IObse...
 layering


 asynchrony and time


 virtual time historical time
public static IObservable<T> Return<T>(T value, IS...

 push pull

Category theory
www.Wikipedia.org
interface IObservable<out T>
{
IDisposable Subscribe(IObserver<T> observ...


 Await Stephen Kleene
Kleene star (closure)
www.Wikipedia.org
Func<T> f
var x = wait f();
IEnumerable<T> xs
foreach (...

 Code-as-data


Alan Kay
Homoiconicity, 1969
www.Wikipedia.org
IEnumerable<T> xs
from x in xs where f(x) …
IQueryable...
 Events




 scalable abstraction



Satya Nadella
“Cloud-first, mobile-first”
www.businessinsider.com

 Bing


 Cortana


 Cloud and device


 Optimization



 Reliability






Richard Feynman
Westview Press
from w in weather
where w.City == “Seattle”
select w
flights
.Where(f => f.Code == “BA49”)
.DelaySubscription(departure – ...
 IReactiveProcessing



Bing cloud
IRP
Partner1 cloud
IRP
Partner2 cloud
IRP
Tablet
IRP
Phone
IRP
Node
IRP
Node
IRP No...
 Rx



 distributed







 Proxies
 Definitions
 Metadata


 Hyper-cube





Sync Async
IntrinsicExtrinsic
Higher dimensions
Calabi-Yau...

 Proxies
 Definitions
 Metadata


 Hyper-cube
 Synchronous versus asynchronous
 Extrinsic versus intrinsic ident...
 context




 proxies

var conn = new ReactiveServiceConnection(endpoint);
var ctx = new ClientContext(conn);
var t...


 extrinsic identifiers
 Async


var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);
var http = ctx.GetObserv...
 SubscribeAsync
 expression tree



 serialized



 IRP
var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);
var http = ctx.GetObserver<HttpData>(httpUri);
var subscription ...
 definition of artifacts



var traffic = ctx.GetObservable<TrafficInfo>(trafficUri);
// Define
await ctx.DefineObserv...

 parameterized observables



var xs = ctx.GetObservable<int>(xsUri);
var iv = ctx.GetObserver<int>(ivUri);
await xs...
 [KnownResource] attributes can be used everywhere
 Provide shorthand syntax for Get* operations
 Enable code generatio...
 properties


 tooling


 delegation




 recovery
 density
 reliability
 Scalable

 checkpointing
 Acknowledge / replay


 Coordinator engines

Event Processing
Execution Engine Node1
Rx
IRP
Event Processing
Execution Engine Node2
Rx
IRP
Chec...
 traverse the operator tree






 ISubscribable<T>

interface ISubscribable<out T>
{
ISubscription Subscribe(ISu...




 Physical schedulers


 logical schedulers

 Unit of pausing


 onion




 GitHub




 bartde@microsoft.com
Service
Host
Data model
Query engine
Operators
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/reactive-
cloud-scale
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Reactive Programming at Cloud-Scale and Beyond
Próximos SlideShares
Carregando em…5
×

Reactive Programming at Cloud-Scale and Beyond

678 visualizações

Publicada em

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1NAoRFg.

Bart de Smet discusses how the reactive programming paradigm fits the requirements of rich querying capabilities for event streams and can be applied from small devices all the way to cloud-scale infrastructures. Filmed at qconnewyork.com.

Bart de Smet is a Principal Software Development Engineer working on large scale stream processing systems in the Applications and Services Division at Microsoft, a course and book author. In his current role at Microsoft, he’s overseeing the architecture, design, and implementation of event processing systems that run at massive scale in data centers, as well as on cloud-connected devices.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Reactive Programming at Cloud-Scale and Beyond

  1. 1. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /reactive-cloud-scale
  2. 2. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  3. 3.   Tesla   mid 2000s       LINQ to Events Nikola Tesla www.knowledgeoftoday.org
  4. 4.   Tier-splitting  metadata         asynchronous www.Wikipedia.org
  5. 5.     Events are not first-class objects  [RunOnClient] public event EventHandler<MouseEventArgs> MouseMoved; // Runs in cloud public void CloudCanvas() { MouseMoved += (o, e) => { /* do stuff */ }; } An electric eel…
  6. 6.   delegates  Action a = new Action(Foo); // explicit creation of delegate instance Action b = Foo; // method group conversion Action c = () => { … }; // creates anonymous method void Foo() { … } event Action Bar // metadata that refers to … { add { … } // add accessor remove { … } // remove accessor } http://en.wikipedia.org/wiki/First-class_citizen
  7. 7.   duality enumerator  interface IObservable<out T> { IDisposable Subscribe(IObserver<T> observer); } interface IObserver<in T> { void OnNext(T value); void OnError(Exception error); void OnCompleted(); } Notification grammar OnNext* (OnError | OnCompleted)? “Gang of four” book Addison-Wesley
  8. 8.   Composition  Composition interface IScheduler { IDisposable Schedule(Action work); … } static class Observable { static IObservable<T> Where<T>(this IObservable<T> source, Func<T, bool> f); static IObservable<R> Select<T, R>(this IObservable<T> source, Func<T, R> p); … } Function composition www.Wikipedia.org
  9. 9.  Function composition www.Wikipedia.org public static class Observable { public static IObservable<T> Merge<T>(this IObservable<T> xs, IObservable<T> ys) { return Create<T>(observer => { var gate = new object(); return new CompositeDisposable { xs.Subscribe(x => { lock (gate) { observer.OnNext(x); } }, …), ys.Subscribe(y => { lock (gate) { observer.OnNext(y); } }, …), }; }); } } First-class: can build extension methods Composition of resource management
  10. 10.  layering    asynchrony and time    virtual time historical time public static IObservable<T> Return<T>(T value, IScheduler scheduler) { return Create<T>(obs => scheduler.Schedule(() => { obs.OnNext(value); obs.OnCompleted(); })); } Space-time www.Wikipedia.org
  11. 11.   push pull  Category theory www.Wikipedia.org interface IObservable<out T> { IDisposable Subscribe(IObserver<T> observer); } interface IObserver<in T> { void OnNext(T value); void OnError(Exception error); void OnCompleted(); } interface IEnumerable<out T> { IEnumerator<T> GetEnumerator(); } interface IEnumerator<out T> : IDisposable { bool MoveNext() throws Exception; T Current { get; } void Reset(); }
  12. 12.    Await Stephen Kleene Kleene star (closure) www.Wikipedia.org Func<T> f var x = wait f(); IEnumerable<T> xs foreach (var x in xs) { f(x); } Task<T> t var x = await t; IObservable<T> xs xs.Subscribe(x => { f(x); }); OneMany Synchronous Asynchronous
  13. 13.   Code-as-data   Alan Kay Homoiconicity, 1969 www.Wikipedia.org IEnumerable<T> xs from x in xs where f(x) … IQueryable<T> xs from x in xs where f(x) … IObservable<T> xs from x in xs where f(x) … IQbservable<T> xs from x in xs where f(x) … CodeData Pull Push
  14. 14.  Events      scalable abstraction    Satya Nadella “Cloud-first, mobile-first” www.businessinsider.com
  15. 15.   Bing    Cortana    Cloud and device  
  16. 16.  Optimization     Reliability       Richard Feynman Westview Press
  17. 17. from w in weather where w.City == “Seattle” select w flights .Where(f => f.Code == “BA49”) .DelaySubscription(departure – 24 * H) .TakeUntil(arrival + 24 * H) .Select(f => f.Status) userLocation .Sample(1 * M) .DistinctUntilChanged() .Select(here => traffic(here, meetingLocation) .TakeUntil(meeting – minimumTime) .Select(t => Timer(meeting – t.EstimatedTime)) .StartWith(Timer(meeting – estimatedTime)) .Switch()) .Switch() .Select(_ => “Leave now for ” + subject) .DelaySubscription(meeting – estimatedTime) Temporal query operators Device-side event stream // Insert cloud-side observable sequence // of time-to-leave timer here Cloud-side event stream Higher-order query operators Remember tier-splitting?
  18. 18.  IReactiveProcessing    Bing cloud IRP Partner1 cloud IRP Partner2 cloud IRP Tablet IRP Phone IRP Node IRP Node IRP Node IRP
  19. 19.  Rx     distributed      
  20. 20.   Proxies  Definitions  Metadata    Hyper-cube      Sync Async IntrinsicExtrinsic Higher dimensions Calabi-Yau manifold www.Wikipedia.org
  21. 21.   Proxies  Definitions  Metadata    Hyper-cube  Synchronous versus asynchronous  Extrinsic versus intrinsic identities  Code-as-data versus code  Reliable versus non-reliable  Etc. Higher dimensions Calabi-Yau manifold www.Wikipedia.org Async Extrinsic E.g. this is the client-side API Need string theory 
  22. 22.  context      proxies  var conn = new ReactiveServiceConnection(endpoint); var ctx = new ClientContext(conn); var traffic = ctx.GetObservable<TrafficInfo>(trafficUri); var http = ctx.GetObserver<HttpData>(httpUri); Explicit identifiers provided for artifacts
  23. 23.    extrinsic identifiers  Async   var traffic = ctx.GetObservable<TrafficInfo>(trafficUri); var http = ctx.GetObserver<HttpData>(httpUri); var subscription = await traffic.Where(t => t.Road == “I-90”) .Select(t => new HttpData { Uri = myService, Body = t.ToString() }) .SubscribeAsync(http, mySubUri); Explicit identifiers provided for artifacts
  24. 24.  SubscribeAsync  expression tree     serialized     IRP
  25. 25. var traffic = ctx.GetObservable<TrafficInfo>(trafficUri); var http = ctx.GetObserver<HttpData>(httpUri); var subscription = await traffic.Where(t => t.Road == “I-90”) .Select(t => new HttpData { Uri = …, Body = … }) .SubscribeAsync(http, mySubUri); Invoke rx://subscribe Invoke rx://select Invoke rx://where trafficUri httpUri λ t => new { bing://http/uri = …, bing://http/body = … } λ t => t.Road == “I-90” Unbound parameters processed by binder Structural typing to reduce coupling
  26. 26.  definition of artifacts    var traffic = ctx.GetObservable<TrafficInfo>(trafficUri); // Define await ctx.DefineObservableAsync<string, TrafficInfo>( trafficByRoadUri, road => traffic.Where(t => t.Road == road)); // Proxies var trafficByRoad = ctx.GetObservable<string, TrafficInfo>(trafficByRoadUri); trafficByRoad(“I-90”).…
  27. 27.   parameterized observables    var xs = ctx.GetObservable<int>(xsUri); var iv = ctx.GetObserver<int>(ivUri); await xs.Where(x => x > 0).Select(x => x * x).SubscribeAsync(iv, …); var xs = ctx.GetObservable<int>(xsUri); var iv = ctx.GetObserver<int>(ivUri); var where = ctx.GetObservable<IARQ<int>, Expr<Func<int, bool>>, int>(whereUri); var select = ctx.GetObservable<IARQ<int>, Expr<Func<int, int>>, int>(selectUri); await select(where(xs, x => x > 0), x => x * x).SubscribeAsync(iv, …);
  28. 28.  [KnownResource] attributes can be used everywhere  Provide shorthand syntax for Get* operations  Enable code generation using the Metadata facilities  What’s IAsyncReactiveQbservable<T>?  Async because it’s client-side (cf. SubscribeAsync)  Reactive because we had to disambiguate with Rx concepts  Qbservable because it’s expression tree based static class AsyncReactiveQbservable { [KnownResource(whereUri)] static IAsyncReactiveQbservable<T> Where<T>(this IAsyncReactiveQbservable<T> xs, Expression<Func<T, bool>> filter) {…} }
  29. 29.  properties    tooling    delegation 
  30. 30.     recovery  density  reliability  Scalable   checkpointing  Acknowledge / replay  
  31. 31.  Coordinator engines  Event Processing Execution Engine Node1 Rx IRP Event Processing Execution Engine Node2 Rx IRP Checkpoint storage Save Load Checkpoint storage Save Load Reliable messaging OnNext Replay Ack
  32. 32.  traverse the operator tree        ISubscribable<T>  interface ISubscribable<out T> { ISubscription Subscribe(ISubscriber<T> subscriber); } interface ISubscription : IDisposable { void Accept(ISubscriptionVisitor v); }
  33. 33.      Physical schedulers    logical schedulers   Unit of pausing  
  34. 34.  onion      GitHub      bartde@microsoft.com Service Host Data model Query engine Operators
  35. 35. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/reactive- cloud-scale

×