SlideShare a Scribd company logo
1 of 39
Download to read offline
Australia’s #1 horse racing website
Our Brands

PuntersParadise.com.au

BestOdds.com.au

+ over 110 individual content websites

rleague.com
Stupid shit we do
More stupid shit we do
Why AWS?
Normal 30 days
12 month traffic
PuntersParadise.com.au

Melbourne Cup Day 2012
•  13,350	
  concurrent	
  users	
  
peak	
  load	
  
	
  
•  550,000	
  uniques	
  in	
  a	
  few	
  
hours	
  
	
  
•  Almost	
  3	
  x	
  more	
  visitors	
  
than	
  Flemington	
  
racecourse	
  
•  Residents	
  of	
  Darwin	
  loose	
  
internet	
  	
  
PuntersParadise.com.au

Our year on year growth

Visits	
  
2009	
  

572,000	
  

2010	
  

796,000	
  

2011	
  

1,731,000	
  

2012	
  

3,008,000	
  

2013	
  

3M	
  +	
  so	
  far	
  
Melbourne Cup Uptime History

2008
2009
2010
2011
2012

Migrated to AWS
about here
The early days.
Immediate wins

We killed our pets and started producing cattle
Automated builds
Performance issues were obvious
Tech was there

Survived 1st Melbourne Cup!
Our Basic Stack

ELB
Cache Servers
ELB
Web Servers

ElastiCache
RDS
Odds Comparison
Our Basic Stack

ELB
Cache Servers
ELB
Web Servers

ElastiCache
RDS

Feed Server
On any given day..

8-10 races per meeting
On any given day..

Average of 12 race meetings per day
On any given day..

+ 45 sporting games
On any given day..

+ 8 greyhound meetings x 10 races
On any given day..

x2 bet types (win/place or sport margin/outrights)
On any given day..

x12 bookmakers!
On any given day..

Odds can change every few seconds
Two problems here.

Processing

Delivery
Our first attempt

while(true)
{
//some magic in here
$market = getMarketInQueue();
//web requests to XML/JSON feeds
$market->hitUpBookmakers();
//save to DB
$market->saveOddsToMySQL();
}
How it works

//this sucks. Doesn’t feel right.
$market->saveOddsToMySQL();
Push that puppy

while(true)
{
//some magic in here
$market = getMarketInQueue();
//web requests to XML/JSON feeds
$market->hitUpBookmakers();
//ahhh. That feels better
$market->saveOddsToDynamoDB();
//like a boss
$market->pushOddsToPusher();
}
Push that puppy
"2081715-Average-FixedWin": {
"2081715-Average-FixedWin": {
"sid": 2081715,
"sid": 2081715,
"bt": "2081715-Average-FixedWin": {
"FixedWin",
"bt": "FixedWin",
"b": "Average",
"sid": 2081715,
"2081715-Average-FixedWin": {
"f": [ "b": "Average", "FixedWin",
"bt":
"sid": 2081715,
"f": [
[
"b": "Average","FixedWin",
"bt":
[
1381882761,
"f": [
"b": "Average",
1381882761,
1
[
"f": [
1
]
1381882761,
[
]
],
1
1381882761,
],
"p": null,
]
1
"p": null,
"ps": null
],
]
"ps": null
"p": null,
],
"ps": null
"p": null,
"ps": null

From
DynamoDB

Pushed
Message

Template
language
Our Full Stack

ELB
Cache Servers
ELB
Web Servers

DynamoDB

Pusher.com

ElastiCache
RDS

Feed Servers
Whats next?

ELB
Cache Servers
ELB

Autoscale!

Web Servers

DynamoDB

Pusher.com

ElastiCache

Autoscale!
RDS

Feed Servers
AWS Reference Architecture

Too easy. Let’s just do that..
Autoscaling the feedserver
Controller

Feed Server

Feed Server
Autoscaling the feedserver
Controller

Feed Server

Feed Server
Autoscaling the feedserver

Feed Server

Feed Server

Feed Server
Autoscaling the feedserver

Feed Server 1

Feed Server 2

Feed Server 3
Autoscaling the feedserver
Autoscaling the feedserver
Autoscaling the feedserver
Our advice

Let everything get hit by a bus
Empower the worker, not the boss
Utilise the AWS API as much as possible
It’s easy. We can all do it.
One last cool thing…

More Related Content

Similar to Architecting high traffic websites with rapid changes in data on AWS

#NewMeetup Performance
#NewMeetup Performance#NewMeetup Performance
#NewMeetup PerformanceJustin Cataldo
 
WebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan WintermeyerWebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan WintermeyerElixir Club
 
MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks
 
Tweaking performance on high-load projects
Tweaking performance on high-load projectsTweaking performance on high-load projects
Tweaking performance on high-load projectsDmitriy Dumanskiy
 
Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up Abhishek Singh
 
TPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and FluxTPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and FluxJirat Kijlerdpornpailoj
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsSerge Smetana
 
Developing node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBDeveloping node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBRob Tweed
 
Using Apache Solr
Using Apache SolrUsing Apache Solr
Using Apache Solrpittaya
 
Web Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP MeetupWeb Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP MeetupJonathan Klein
 
Immutability
ImmutabilityImmutability
Immutabilitybyanjati
 
Socket applications
Socket applicationsSocket applications
Socket applicationsJoão Moura
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud
 
Scale, baby, scale
Scale, baby, scaleScale, baby, scale
Scale, baby, scaleJulien SIMON
 
"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien SimonTheFamily
 
Innovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringInnovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringCary Millsap
 
Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"Fwdays
 

Similar to Architecting high traffic websites with rapid changes in data on AWS (20)

#NewMeetup Performance
#NewMeetup Performance#NewMeetup Performance
#NewMeetup Performance
 
WebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan WintermeyerWebPerformance: Why and How? – Stefan Wintermeyer
WebPerformance: Why and How? – Stefan Wintermeyer
 
MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...MeasureWorks - Why people hate to wait for your website to load (and how to f...
MeasureWorks - Why people hate to wait for your website to load (and how to f...
 
Tweaking performance on high-load projects
Tweaking performance on high-load projectsTweaking performance on high-load projects
Tweaking performance on high-load projects
 
ApacheCon 2005
ApacheCon 2005ApacheCon 2005
ApacheCon 2005
 
Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up Scaling a Rails Application from the Bottom Up
Scaling a Rails Application from the Bottom Up
 
TPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and FluxTPSE Thailand 2015 - Rethinking Web with React and Flux
TPSE Thailand 2015 - Rethinking Web with React and Flux
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails Applications
 
Developing node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBDeveloping node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDB
 
NoSQL and CouchDB
NoSQL and CouchDBNoSQL and CouchDB
NoSQL and CouchDB
 
Using Apache Solr
Using Apache SolrUsing Apache Solr
Using Apache Solr
 
Web Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP MeetupWeb Performance, Scalability, and Testing Techniques - Boston PHP Meetup
Web Performance, Scalability, and Testing Techniques - Boston PHP Meetup
 
Immutability
ImmutabilityImmutability
Immutability
 
Figaro
FigaroFigaro
Figaro
 
Socket applications
Socket applicationsSocket applications
Socket applications
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
 
Scale, baby, scale
Scale, baby, scaleScale, baby, scale
Scale, baby, scale
 
"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon"How to optimize the architecture of your platform" by Julien Simon
"How to optimize the architecture of your platform" by Julien Simon
 
Innovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and MonitoringInnovative Specifications for Better Performance Logging and Monitoring
Innovative Specifications for Better Performance Logging and Monitoring
 
Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"Martin Splitt "A short history of the web"
Martin Splitt "A short history of the web"
 

Recently uploaded

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024SynarionITSolutions
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsRoshan Dwivedi
 

Recently uploaded (20)

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 

Architecting high traffic websites with rapid changes in data on AWS