SlideShare a Scribd company logo
1 of 26
Download to read offline
Realtime Webpack
Pushing on-demand bundling to the limits
March 2017
@oliverwoodings
We help our clients deliver highly
persuasive personalization at scale, to
their best customers, at the right time.
What does that mean?
Collect data
Surface insights
(machine learning etc)
Define segments,
design personalisations
Serve personalisations
/segments back to end users
How do we deliver web assets
Build Pipeline
Qubit Platform
CDN
Browser
End UserQubit Customer
Prod Bundle
Personalisation
& Segments
Preview Bundle
Event stream
How do we deliver web assets
Build Pipeline
Qubit Platform
CDN
Browser
End UserQubit Customer
Prod Bundle
Personalisation
& Segments
Preview Bundle
Event stream
Filesystem
- index.js
- app/
- actions/
- components/
- views/
- reducers/
- App.js
- node_modules/
Yarn
Webpack Dev Server Browser
A typical webpack setup
Why doesn't this work for us?
Each client is different!
Why doesn't this work for us?
Filesystem
- index.js
- app/
- actions/
- components/
- views/
- reducers/
- App.js
- node_modules/
Yarn
Webpack Dev Server Browser
Let’s fix it up…
1. Get rid of the fs
Webpack Dev Server Browser
MemoryFS
- entry.js
- .cache
- config.js
- experiences-2499/
- activation.js
- variation-1.js
- variation-1.css
MySQL
Redis package
cache
npm registry
2. Replace webpack dev server
Redis package
cache
npm registry
Dynamic compiler
service
Browser
Redis bundle
cache
My SQL
MemoryFS
- entry.js
- .cache
- config.js
- experiences-2499/
- activation.js
- variation-1.js
- variation-1.css
Redis package
cache
3. Scale
Worker PoolMySQL
Dynamic
compiler service
MemoryFS
Loadbalancer
MemoryFS
MemoryFS
Dynamic
compiler service
Dynamic
compiler service
Redis bundle
cache
npm registry
Browser
Redis package
cache
Worker PoolMySQL
Dynamic
compiler service
MemoryFS
Loadbalancer
MemoryFS
MemoryFS
Dynamic
compiler service
Dynamic
compiler service
Redis bundle
cache
npm registry
Browser
4. Queue and autoscale
Autoscaler
Job Queue
CDN
The result
11k
builds/day on
average
2.6
seconds average
build duration
4
preview builds/second
with 6 preview workers
Performance under load
Performance under load
• 1000 webpack builds
Performance under load
• 1000 webpack builds
• 100 workers
Performance under load
• 1000 webpack builds
• 100 workers
• Completed in 2 minutes
Performance under load
• 1000 webpack builds
• 100 workers
• Completed in 2 minutes
• Peak throughput of 9 builds
per second
So what?
So what?
• Make it snappy - HappyPack
So what?
• Make it snappy - HappyPack
• Keep it simple - Preact and Buble
So what?
• Make it snappy - HappyPack
• Keep it simple - Preact and Buble
• Tidy up after yourself - clean webpack config
So what?
• Make it snappy - HappyPack
• Keep it simple - Preact and Buble
• Tidy up after yourself - clean webpack config
• Be realistic - tree shaking
So what?
• Make it snappy - HappyPack
• Keep it simple - Preact and Buble
• Tidy up after yourself - clean webpack config
• Be realistic - tree shaking
• Try it flat - Rollup
@oliverwoodings
oli@qubit.com
Thanks!

More Related Content

What's hot

How to Speed Up Your Joomla! Site
How to Speed Up Your Joomla! SiteHow to Speed Up Your Joomla! Site
How to Speed Up Your Joomla! Site
Daniel Kanchev
 
Single Page Application Development with backbone.js and Simple.Web
Single Page Application Development with backbone.js and Simple.WebSingle Page Application Development with backbone.js and Simple.Web
Single Page Application Development with backbone.js and Simple.Web
Chris Canal
 

What's hot (20)

Building services for apps on a shoestring budget
Building services for apps on a shoestring budgetBuilding services for apps on a shoestring budget
Building services for apps on a shoestring budget
 
The WordPress REST API as a Springboard for Website Greatness
The WordPress REST API as a Springboard for Website GreatnessThe WordPress REST API as a Springboard for Website Greatness
The WordPress REST API as a Springboard for Website Greatness
 
WebAssembly in Houdini CSS, is it possible?
WebAssembly in Houdini CSS, is it possible?WebAssembly in Houdini CSS, is it possible?
WebAssembly in Houdini CSS, is it possible?
 
Jumpstart Azure
Jumpstart AzureJumpstart Azure
Jumpstart Azure
 
Railsconf 2014 - Deploying Rails is Easier Thank It Looks
Railsconf 2014 - Deploying Rails is Easier Thank It LooksRailsconf 2014 - Deploying Rails is Easier Thank It Looks
Railsconf 2014 - Deploying Rails is Easier Thank It Looks
 
Office 365 vs. Google Apps
Office 365 vs. Google AppsOffice 365 vs. Google Apps
Office 365 vs. Google Apps
 
Why Javascript matters
Why Javascript mattersWhy Javascript matters
Why Javascript matters
 
Modern Front End Tools & Workflow
Modern Front End Tools & WorkflowModern Front End Tools & Workflow
Modern Front End Tools & Workflow
 
The Often Useful Somewhat Annoying World of WordPress Multisite
The Often Useful Somewhat Annoying World of WordPress MultisiteThe Often Useful Somewhat Annoying World of WordPress Multisite
The Often Useful Somewhat Annoying World of WordPress Multisite
 
Production optimization with React and Webpack
Production optimization with React and WebpackProduction optimization with React and Webpack
Production optimization with React and Webpack
 
Automated perf optimization - jQuery Conference
Automated perf optimization - jQuery ConferenceAutomated perf optimization - jQuery Conference
Automated perf optimization - jQuery Conference
 
WebAssembly vs JavaScript: What is faster?
WebAssembly vs JavaScript: What is faster?WebAssembly vs JavaScript: What is faster?
WebAssembly vs JavaScript: What is faster?
 
Building fast aspnet websites
Building fast aspnet websitesBuilding fast aspnet websites
Building fast aspnet websites
 
Cloudcamp Chicago Nov 2104 Fintech - John Downey's "A Hypothetical Public Clo...
Cloudcamp Chicago Nov 2104 Fintech - John Downey's "A Hypothetical Public Clo...Cloudcamp Chicago Nov 2104 Fintech - John Downey's "A Hypothetical Public Clo...
Cloudcamp Chicago Nov 2104 Fintech - John Downey's "A Hypothetical Public Clo...
 
Session dotNed Saturday 28 januari 2017
Session dotNed Saturday 28 januari 2017Session dotNed Saturday 28 januari 2017
Session dotNed Saturday 28 januari 2017
 
Mciro Services & Zookeeper
Mciro Services & ZookeeperMciro Services & Zookeeper
Mciro Services & Zookeeper
 
How do we work with Angular.js
How do we work with Angular.jsHow do we work with Angular.js
How do we work with Angular.js
 
How to Speed Up Your Joomla! Site
How to Speed Up Your Joomla! SiteHow to Speed Up Your Joomla! Site
How to Speed Up Your Joomla! Site
 
Single Page Application Development with backbone.js and Simple.Web
Single Page Application Development with backbone.js and Simple.WebSingle Page Application Development with backbone.js and Simple.Web
Single Page Application Development with backbone.js and Simple.Web
 
2010 11 pubcon_hendison_wordpress
2010 11 pubcon_hendison_wordpress2010 11 pubcon_hendison_wordpress
2010 11 pubcon_hendison_wordpress
 

Similar to Realtime Webpack - Pushing on-demand bundling to the limits by Oliver Woodings

ASP.NET Scalability - VBUG London
ASP.NET Scalability - VBUG LondonASP.NET Scalability - VBUG London
ASP.NET Scalability - VBUG London
Phil Pursglove
 
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
DataWorks Summit
 
ASP.NET Scalability - WebDD
ASP.NET Scalability - WebDDASP.NET Scalability - WebDD
ASP.NET Scalability - WebDD
Phil Pursglove
 

Similar to Realtime Webpack - Pushing on-demand bundling to the limits by Oliver Woodings (20)

Performance stack
Performance stackPerformance stack
Performance stack
 
A faster web
A faster webA faster web
A faster web
 
Ryan Armstrong - Monitoring More Than 6000 Devices in Zabbix | ZabConf2016
Ryan Armstrong - Monitoring More Than 6000 Devices in Zabbix | ZabConf2016Ryan Armstrong - Monitoring More Than 6000 Devices in Zabbix | ZabConf2016
Ryan Armstrong - Monitoring More Than 6000 Devices in Zabbix | ZabConf2016
 
Docker and Cloud - Enables for DevOps - by ACA-IT
Docker and Cloud - Enables for DevOps - by ACA-ITDocker and Cloud - Enables for DevOps - by ACA-IT
Docker and Cloud - Enables for DevOps - by ACA-IT
 
'DOCKER' & CLOUD: ENABLERS For DEVOPS
'DOCKER' & CLOUD:  ENABLERS For DEVOPS'DOCKER' & CLOUD:  ENABLERS For DEVOPS
'DOCKER' & CLOUD: ENABLERS For DEVOPS
 
Stress Test as a Culture
Stress Test as a CultureStress Test as a Culture
Stress Test as a Culture
 
Simpler, faster, cheaper Enterprise Apps using only Spring Boot on GCP
Simpler, faster, cheaper Enterprise Apps using only Spring Boot on GCPSimpler, faster, cheaper Enterprise Apps using only Spring Boot on GCP
Simpler, faster, cheaper Enterprise Apps using only Spring Boot on GCP
 
20 tips for website performance
20 tips for website performance20 tips for website performance
20 tips for website performance
 
Edge 2014: Maintaining the Balance: Getting the Most of Your CDN with IKEA
Edge 2014: Maintaining the Balance: Getting the Most of Your CDN with IKEAEdge 2014: Maintaining the Balance: Getting the Most of Your CDN with IKEA
Edge 2014: Maintaining the Balance: Getting the Most of Your CDN with IKEA
 
Standardizing and Managing Your Infrastructure - MOSC 2011
Standardizing and Managing Your Infrastructure - MOSC 2011Standardizing and Managing Your Infrastructure - MOSC 2011
Standardizing and Managing Your Infrastructure - MOSC 2011
 
AWS re:Invent 2016: Born in the Cloud; Built Like a Startup (ARC205)
AWS re:Invent 2016: Born in the Cloud; Built Like a Startup (ARC205)AWS re:Invent 2016: Born in the Cloud; Built Like a Startup (ARC205)
AWS re:Invent 2016: Born in the Cloud; Built Like a Startup (ARC205)
 
ASP.NET Scalability - VBUG London
ASP.NET Scalability - VBUG LondonASP.NET Scalability - VBUG London
ASP.NET Scalability - VBUG London
 
Picnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable applicationPicnic Software - Developing a flexible and scalable application
Picnic Software - Developing a flexible and scalable application
 
Matt Franklin - Apache Software (Geekfest)
Matt Franklin - Apache Software (Geekfest)Matt Franklin - Apache Software (Geekfest)
Matt Franklin - Apache Software (Geekfest)
 
Mainframe, the fast PHP framework
Mainframe, the fast PHP frameworkMainframe, the fast PHP framework
Mainframe, the fast PHP framework
 
Anton Boyko, "The evolution of microservices platform or marketing gibberish"
Anton Boyko, "The evolution of microservices platform or marketing gibberish"Anton Boyko, "The evolution of microservices platform or marketing gibberish"
Anton Boyko, "The evolution of microservices platform or marketing gibberish"
 
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
 
Get the most out OpenStack block storage with SolidFire
Get the most out OpenStack block storage with SolidFireGet the most out OpenStack block storage with SolidFire
Get the most out OpenStack block storage with SolidFire
 
ASP.NET Scalability - WebDD
ASP.NET Scalability - WebDDASP.NET Scalability - WebDD
ASP.NET Scalability - WebDD
 
Speed Up WordPress Websites - Part 1 - WordPress Cairo Meetup
Speed Up WordPress Websites - Part 1 - WordPress Cairo MeetupSpeed Up WordPress Websites - Part 1 - WordPress Cairo Meetup
Speed Up WordPress Websites - Part 1 - WordPress Cairo Meetup
 

More from React London 2017

More from React London 2017 (9)

A Tiny Fiber Renderer by Dustan Kasten
A Tiny Fiber Renderer by Dustan Kasten A Tiny Fiber Renderer by Dustan Kasten
A Tiny Fiber Renderer by Dustan Kasten
 
Snapshot testing by Anna Doubkova
Snapshot testing by Anna Doubkova Snapshot testing by Anna Doubkova
Snapshot testing by Anna Doubkova
 
Next.js in production by Jasdeep Lalli
Next.js in production by Jasdeep Lalli Next.js in production by Jasdeep Lalli
Next.js in production by Jasdeep Lalli
 
The road to <> styled-components: CSS in component-based systems by Max S...
The road to <> styled-components: CSS in component-based systems by Max S...The road to <> styled-components: CSS in component-based systems by Max S...
The road to <> styled-components: CSS in component-based systems by Max S...
 
Logux, a new approach to client-server communication by Andrey Sitnik
Logux, a new approach to client-server communication by Andrey SitnikLogux, a new approach to client-server communication by Andrey Sitnik
Logux, a new approach to client-server communication by Andrey Sitnik
 
Weapons grade React by Ken Wheeler
Weapons grade React by Ken Wheeler Weapons grade React by Ken Wheeler
Weapons grade React by Ken Wheeler
 
Offline For The Greater Good by Jani Eväkallio
Offline For The Greater Good by Jani EväkallioOffline For The Greater Good by Jani Eväkallio
Offline For The Greater Good by Jani Eväkallio
 
What's in a language? By Cheng Lou
What's in a language? By Cheng Lou What's in a language? By Cheng Lou
What's in a language? By Cheng Lou
 
JavaScript Code Formatting With Prettier by Christopher Chedeau
JavaScript Code Formatting With Prettier by Christopher ChedeauJavaScript Code Formatting With Prettier by Christopher Chedeau
JavaScript Code Formatting With Prettier by Christopher Chedeau
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 

Realtime Webpack - Pushing on-demand bundling to the limits by Oliver Woodings

  • 1. Realtime Webpack Pushing on-demand bundling to the limits March 2017 @oliverwoodings
  • 2. We help our clients deliver highly persuasive personalization at scale, to their best customers, at the right time.
  • 3. What does that mean? Collect data Surface insights (machine learning etc) Define segments, design personalisations Serve personalisations /segments back to end users
  • 4. How do we deliver web assets Build Pipeline Qubit Platform CDN Browser End UserQubit Customer Prod Bundle Personalisation & Segments Preview Bundle Event stream
  • 5. How do we deliver web assets Build Pipeline Qubit Platform CDN Browser End UserQubit Customer Prod Bundle Personalisation & Segments Preview Bundle Event stream
  • 6. Filesystem - index.js - app/ - actions/ - components/ - views/ - reducers/ - App.js - node_modules/ Yarn Webpack Dev Server Browser A typical webpack setup
  • 7. Why doesn't this work for us?
  • 8. Each client is different! Why doesn't this work for us?
  • 9. Filesystem - index.js - app/ - actions/ - components/ - views/ - reducers/ - App.js - node_modules/ Yarn Webpack Dev Server Browser Let’s fix it up…
  • 10. 1. Get rid of the fs Webpack Dev Server Browser MemoryFS - entry.js - .cache - config.js - experiences-2499/ - activation.js - variation-1.js - variation-1.css MySQL Redis package cache npm registry
  • 11. 2. Replace webpack dev server Redis package cache npm registry Dynamic compiler service Browser Redis bundle cache My SQL MemoryFS - entry.js - .cache - config.js - experiences-2499/ - activation.js - variation-1.js - variation-1.css
  • 12. Redis package cache 3. Scale Worker PoolMySQL Dynamic compiler service MemoryFS Loadbalancer MemoryFS MemoryFS Dynamic compiler service Dynamic compiler service Redis bundle cache npm registry Browser
  • 13. Redis package cache Worker PoolMySQL Dynamic compiler service MemoryFS Loadbalancer MemoryFS MemoryFS Dynamic compiler service Dynamic compiler service Redis bundle cache npm registry Browser 4. Queue and autoscale Autoscaler Job Queue CDN
  • 14. The result 11k builds/day on average 2.6 seconds average build duration 4 preview builds/second with 6 preview workers
  • 16. Performance under load • 1000 webpack builds
  • 17. Performance under load • 1000 webpack builds • 100 workers
  • 18. Performance under load • 1000 webpack builds • 100 workers • Completed in 2 minutes
  • 19. Performance under load • 1000 webpack builds • 100 workers • Completed in 2 minutes • Peak throughput of 9 builds per second
  • 21. So what? • Make it snappy - HappyPack
  • 22. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble
  • 23. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config
  • 24. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config • Be realistic - tree shaking
  • 25. So what? • Make it snappy - HappyPack • Keep it simple - Preact and Buble • Tidy up after yourself - clean webpack config • Be realistic - tree shaking • Try it flat - Rollup