This is my presentation for week 3 of CSIS 638 presentations to be presented on 7/20/2011

1. Scalable Distributed Stream
Processing
Matt Garren

2. Aurora
● Tuple Processor
● Uses "Boxes and Arrows" paradigm for
dataflow
● Queries are continuous

3. Aurora: Operators
● Filter
● Union
● WSort
● Tumble
● Map
● XSection
● Slide
● Join
● Resample

4. Aurora: Runtime
● Scheduler
● Storage Manager
● Load Shedder
● QoS

5. Distributed Stream Processing
● Benefits of a Node Federation
○ Sharing/Multiplexing
○ Resiliency
○ Programmatically the same

6. Aurora*: Distributed Aurora
● Federation of nodes running Aurora
● Single Admin domain
● Queries allowed on all nodes
● Boxes partitioned to nodes
● Overloaded nodes offload tasks
● Decentralized reconfiguration

7. Medusa: Distributed Aurora
● Multiple Administrative domains
● "Participants"
● Agoric system
● "Contracts" with senders/receivers
● Receiver "pays" sender
● Goal: Stable "Economy"

8. Infrastructure Requirements
● Naming and Discovery
○ Global namespace
○ Catalogs (Inter/Intra participant)
■ Operators
■ Schemas
■ Queries
■ Streams
■ Contacts
■ Event locations

9. Infrastructure Requirements
● Routing
○ Before producing: register schema
○ Label events w/ stream name
○ Intermediary nodes forward (Intra)
○ Explicit connections to cross boundaries

10. Infrastructure Requirements
● Message Transport
○ Individual TCP connections bad idea
○ Multiplex messages
■ Single TCP connection
■ Message scheduler
● Remote Definition
■ Users only see what they care about

11. Load Sharing
● Load Share Daemon per node
● Box Sliding
● Box Splitting

12. Load Sharing
● Considerations
○ How often to load share?
○ When is it not ok to offload?
○ What should be split?

13. Availability
● Primary/Back up servers
● Heartbeats to neighbors
● Additional work at failover

14. QoS
● Intermediate/Output
● Precision

15. Questions?