16. Architecting Infinitely Scalable Systems A seminal paper on the architecture of elastic applications by Pat Helland(Tandem Computing, Amazon.com, Microsoft) “Life Beyond Distributed Transactions: an Apostate’s Opinion” http://www.cidrdb.org/cidr2007/papers/cidr07p15.pdf http://blogs.msdn.com/b/pathelland/ Application architectures need to change to achieve infinite scalability and elasticity without using large hardware
17. Scale - Layered Code Common layered architecture in largest scale applications Top layer Scale Agnostic Code Programming Abstraction Abstraction layer Scale Aware Code Bottom layer understands application is distributed
22. Reliability No data loss No data corruption Consistency Race condition Synchronous vs Asynchronous
23. No Data Loss, No Data corruption, Consistency Distributed semaphore - lightweight Primary copy Distributed transaction(s) – heavy weight MVCC – Acronyms are annoying
24. Race Conditions, Consistency Eventually Consistent Stateless ? Application Data Tier Stateful Controllably Consistent
33. Latency & Reliability - Memory-based Performance Memory on a peer machine to make data updates durable, Writes return 10x to 100x faster than disk, 10s to 100s of Microseconds vs 10s to 100s Milliseconds Perform Customers Orders Product Keep redundant copies of data Update thru primary 0 data loss Optionally write updates to disk, Optional write todata warehouse asynchronously and reliably. Protect
38. Data Distribution Distribute Keep clusters synchronized in real-time Operate reliably Disconnected, Intermittent and Low-Bandwidth network environments.
40. Cloud Ready Soar Load Balancer Web Tier Application Tier GemFire Jar 11MB (or less) Optional reliable, asynchronous feed to Data Warehouse or Archival Database
Explosion- According to Siemans AG (automated systems will soon generate more data than the total created by humans) Sources customer systems, wire feeds, social mediaThe OODA loop is a decision making model developed by US Air Force Colonel John Boyd. Some examples of time that matters Quick description of todays DW mech Bet 365 – milliseconds Websites that suggest – Sense and respond for power companies - seconds Ability to fill cargo hold before ship sails to make its SLA -hours