SlideShare uma empresa Scribd logo
1 de 63
Baixar para ler offline
Hosting and the Woes

       by Jamie van Dyke
  Engine Yard Developer (UK)
Jamie van Dyke


My name is Jamie van Dyke, and in case you can see me from the back, I look ...
like this fella here. I used to be the ...
support


...Application Support Manager for Europe, but now I’ve transitioned over to ...
develop cool stuff


the Automation Development team over in ...
England


...the UK. ( So we do 24 hour development as well as support now ;-)
...just in case you forgot about us, we’re this little island here...we don’t have as many nukes
as you...but we’ve got Gordon Ramsay who’s even scarier! Enough about me, here’s...
Tom Mornini


...Tom Mornini, who looks like...
...this. Study the face well and feel free to grab him at any time during the conference to talk
shop, or just question about anything. Tom is our...
Rails Hosting and the Woes
The Chief


...CTO, Tom has experience in...
jack


every technology at Engine Yard...and can put many to shame...
of all trades


...in most of these areas. Moving on, we have...
Taylor Weibley


...Taylor Weibley. Taylor looks...
...like this. Taylor is our...
king


...Application Support Director. He’s the man responsible for keeping everyone happy, so if
you’re a customer and you’re not (is that even possible?), he’s the man to speak to. He has
tonnes of skills in many...
Rails ++


...areas, of course Rails is one of them, but we consider him to be...
...one of our Supermen in the App Support Team. Finally, we have...
Edward Muller


...Edward Muller. Edward looks like...
...this, and he’s one of those incredibly clever people you meet that...
fluent in binary


...is able to talk to a computer instead of typing on the keyboard. Ed was and to some
degree still is our...
cluster ninja


...Cluster Engineer. After Jayson Vantuyl conceived and built ey00, our first cluster, Edward
went on to build all the others and make improvements along the way. He’s now my ...
automation manager


... Manager, in the automation department. We basically improve the lives of the Engine Yard
staff, by automating the processes they do every day. For example, deploys can take a few
hours manually, but with a sprinkling of automation we can pull that down to a few minutes.
</intro>


Enough about us, let’s talk about some of the problems that our customers have bumped
into and how to solve them.
Problems


Through hundreds of customers and hours of debugging, we’ve found a recurring set of
problems that crop up with Rails development, and we’d like to make you aware of them if
you aren’t already. No surprise, top of the list...
active_record


...is Active Record. Too many times...
...I’ve seen a find(:all) that goes crazy and chews up memory like there’s unlimited banks of
it. You all know the solution to this, in views you use pagination and in processing loops you
use batches. The main cause of this going unnoticed before hitting production, is that...
development


...on a development machine you have a small set...
20 records


...of records. Your tests run against them and everything works fine. Your machine has 2Gb
of RAM and you don’t sit watching the memory usage as you test. However...
production


...in production your site might be really popular and you’re running...
millions of records


...millions of users and assets for each of those. Running a find(:all) on that would probably
cause a problem. :-/ So paginate, and make sure you have...
indexes


...properly indexed your tables. At Engine Yard we use a team of DBA’s and the MySQL
Performance Blog Experts to help our customers get the best speed out of their databases.
plugins


There are a handful of technologies and plugins that cause problems for a lot of our
customers too. For example...
ferret


...the popular ferret indexing engine along with numerous different plugins that provide
interfaces to it, is constantly corrupting indexes. Initially you’d think that it’s multiple
processes accessing the same index...but we always use ferret server, so it’s not that. Our
solution is simple and effective...
sphinx


...switch over to sphinx, which has a lot less problems (if any) and does the same job. We’ve
seen ferret indexes at 5Gb that are reduced to 1Gb when moved to sphinx. In conjunction
with Evan Weaver’s ultrasphinx plugin, this is the ideal solution.
image science


image_science is great but there are a couple of gotchas for when you shift to using it in
production, with monit...Image Science requires environment variables (INLINE_DIR or HOME)
set up to work, monit clears these variables...so either use a wrapper script or set them in
your app manually.
hodel 3000


...hodel compliant logging allows you to use a specific log format that reporting tools can
handle. However, out of the few plugins that are out there for it, they neglect to inform you
that you *need* to set logging to :info, otherwise your logs will (default to debug), be
absolutely huge, and they don’t need to be if you want your reporting tools to work.
...ever wonder what kind of traffic you can expect from different media coverages? okay, let’s
have a look at a few...
can haz trafficz?


...ever wonder what kind of traffic you can expect from different media coverages? okay, let’s
have a look at a few...
Digg


so you get featured on digg’s homepage, how much traffic? Well, you’re looking...
10,000+ visitors


...at tens of thousands of visitors...but they’re all just checking out what’s new and featured,
because...
signups --


...you’ll rarely get many signups to your service. It’s all about seeing what’s new. Which is
exactly what we can say...
TechCrunch


...about TechCrunch. The only difference being the level of traffic, which means more likely...
1000+ visitors


...thousands of visitors, you might hit tens of thousands but it depends on what your site is
all about. A Ruby specific site...
Ruby Inside/Flow


...like Ruby Inside or Ruby Flow, will send you between...
500-1000 visitors


...five hundred and a thousand visitors a day, and you’ll get that for a few days to a week,
you’ll also get virtually...
signups --


...no signups from that. Which if you’re a ruby/rails specific site, will be higher than if you’re
not. It’s all whether your site is relevant to the market with these smaller sites, when I say
smaller I mean not on the scale of ...
The Today Show


...the Today Show, which is a great traffic provider, if you’re hosted with a provider who can
scale you quickly and provide you with a team of Rails experts that will help you through it,
by both scaling up your service, as well as providing quick advice on how to increase the
efficiency of your pages. Someone like us ;-) You’re looking at...
100,000+ visitors


...a huge number of visitors which is dependent on your site content, of course. One of our
customers called Catalog Choice got featured on the Today show and the traffic spike was
enormous, as well as obtaining...
10,000 signups


...them ten thousand signups in the first hour alone. So this is definitely one of the media
spots you want to get featured on. Another, not surprisingly, high traffic provider ...
Fox News
                 Business Show

...that could do you well, is Fox News Business Show. From just showing a site url on the tv,
you can expect...
2000+ connections


...a whole chunk of visitors. 2000-3000 simultaneous visitors come rushing your way. Pray
to the Gods that you’re ready to scale up for this one. ;-)
$10,000,000


The ten million dollar question is always, how many users can one of our slices handle. Well,
how long is a piece of string?
can you code?


Of course it’s all dependent on what your code quality is like. We can say that an average
Rails application running over 3 mongrels, can trivially handle millions of unique visitors a
month.
queries per page


So, take a look at how many queries are running per page...
cache cache cache


... cache wherever possible...
file io


...keep file io down to a minimum...
separate services


...and if you can, move your services on to different servers. Like assets in one place,
application in another, database in another, etc.
goodies


Let’s wrap up the talk with a mention of some open source goodies we have that you can
download...then we’ll move on to the Q and A section.
eycap
                   (gems.engineyard.com)




This is a gem we developed that extends capistrano, it allows logging of deploys, has tons of
tasks for sphinx / ferret / memcached / etc. If you use nginx, or monit, or sphinx
or....whatever, then this gem will be handy for you, download it using ...
gem source -a http://gems.engineyard.com
           gem install eycap --no-rdoc --no-ri
              (in deploy.rb) require ‘eycap’




...these commands, with sudo if you’re on a mac...and you get lots of capistrano
goodies...like
...these.
ask us anything


We filled the rest of the presentation with a q & a section, which went extremely well. Thank
you to everyone who came and participated.

Mais conteúdo relacionado

Semelhante a Rails Hosting and the Woes

3stages Wdn08 V3
3stages Wdn08 V33stages Wdn08 V3
3stages Wdn08 V3Boris Mann
 
When e-commerce meets Symfony
When e-commerce meets SymfonyWhen e-commerce meets Symfony
When e-commerce meets SymfonyMarc Morera
 
What every C++ programmer should know about modern compilers (w/ comments, AC...
What every C++ programmer should know about modern compilers (w/ comments, AC...What every C++ programmer should know about modern compilers (w/ comments, AC...
What every C++ programmer should know about modern compilers (w/ comments, AC...Sławomir Zborowski
 
Python enterprise vento di liberta
Python enterprise vento di libertaPython enterprise vento di liberta
Python enterprise vento di libertaSimone Federici
 
Frontend at Scale - The Tumblr Story
Frontend at Scale - The Tumblr StoryFrontend at Scale - The Tumblr Story
Frontend at Scale - The Tumblr StoryChris Miller
 
2020 Top Web Development Trends
2020 Top Web Development Trends2020 Top Web Development Trends
2020 Top Web Development TrendsPencil Agency
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdminsPuppet
 
How to serve 2500 Ad requests per second
How to serve 2500 Ad requests per secondHow to serve 2500 Ad requests per second
How to serve 2500 Ad requests per secondMiguel Sousa Filipe
 
One Path to a Successful Implementation of NaturalONE
One Path to a Successful Implementation of NaturalONEOne Path to a Successful Implementation of NaturalONE
One Path to a Successful Implementation of NaturalONESoftware AG
 
A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023
A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023
A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023Pedro Vicente
 
Reactive Microservice Architecture with Groovy and Grails
Reactive Microservice Architecture with Groovy and GrailsReactive Microservice Architecture with Groovy and Grails
Reactive Microservice Architecture with Groovy and GrailsSteve Pember
 
Boyd Hemphill (Tsunami) Geekfest
Boyd Hemphill (Tsunami) GeekfestBoyd Hemphill (Tsunami) Geekfest
Boyd Hemphill (Tsunami) GeekfestW2O Group
 
Javaland 2017: "You´ll do microservices now". Now what?
Javaland 2017: "You´ll do microservices now". Now what?Javaland 2017: "You´ll do microservices now". Now what?
Javaland 2017: "You´ll do microservices now". Now what?André Goliath
 
From DevOps to NoOps
From DevOps to NoOpsFrom DevOps to NoOps
From DevOps to NoOpsCapgemini
 
Finding Frank - Spotify API.pdf
Finding Frank - Spotify API.pdfFinding Frank - Spotify API.pdf
Finding Frank - Spotify API.pdfaspleenic
 
Scaling on DigitalOcean
Scaling on DigitalOceanScaling on DigitalOcean
Scaling on DigitalOceandavid_e_worth
 
Serverless is more findev than devops
Serverless is more findev than devopsServerless is more findev than devops
Serverless is more findev than devopsYan Cui
 
OSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian Heigl
OSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian HeiglOSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian Heigl
OSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian HeiglNETWAYS
 
OSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyone
OSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyoneOSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyone
OSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyoneNETWAYS
 

Semelhante a Rails Hosting and the Woes (20)

3stages Wdn08 V3
3stages Wdn08 V33stages Wdn08 V3
3stages Wdn08 V3
 
When e-commerce meets Symfony
When e-commerce meets SymfonyWhen e-commerce meets Symfony
When e-commerce meets Symfony
 
What every C++ programmer should know about modern compilers (w/ comments, AC...
What every C++ programmer should know about modern compilers (w/ comments, AC...What every C++ programmer should know about modern compilers (w/ comments, AC...
What every C++ programmer should know about modern compilers (w/ comments, AC...
 
Python enterprise vento di liberta
Python enterprise vento di libertaPython enterprise vento di liberta
Python enterprise vento di liberta
 
Frontend at Scale - The Tumblr Story
Frontend at Scale - The Tumblr StoryFrontend at Scale - The Tumblr Story
Frontend at Scale - The Tumblr Story
 
2020 Top Web Development Trends
2020 Top Web Development Trends2020 Top Web Development Trends
2020 Top Web Development Trends
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
 
How to serve 2500 Ad requests per second
How to serve 2500 Ad requests per secondHow to serve 2500 Ad requests per second
How to serve 2500 Ad requests per second
 
One Path to a Successful Implementation of NaturalONE
One Path to a Successful Implementation of NaturalONEOne Path to a Successful Implementation of NaturalONE
One Path to a Successful Implementation of NaturalONE
 
A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023
A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023
A Multiplatform, Multi-Tenant Challenge - Droidcon Lisbon 2023
 
Reactive Microservice Architecture with Groovy and Grails
Reactive Microservice Architecture with Groovy and GrailsReactive Microservice Architecture with Groovy and Grails
Reactive Microservice Architecture with Groovy and Grails
 
Boyd Hemphill (Tsunami) Geekfest
Boyd Hemphill (Tsunami) GeekfestBoyd Hemphill (Tsunami) Geekfest
Boyd Hemphill (Tsunami) Geekfest
 
Javaland 2017: "You´ll do microservices now". Now what?
Javaland 2017: "You´ll do microservices now". Now what?Javaland 2017: "You´ll do microservices now". Now what?
Javaland 2017: "You´ll do microservices now". Now what?
 
From DevOps to NoOps
From DevOps to NoOpsFrom DevOps to NoOps
From DevOps to NoOps
 
Finding Frank - Spotify API.pdf
Finding Frank - Spotify API.pdfFinding Frank - Spotify API.pdf
Finding Frank - Spotify API.pdf
 
Scaling on DigitalOcean
Scaling on DigitalOceanScaling on DigitalOcean
Scaling on DigitalOcean
 
Serverless is more findev than devops
Serverless is more findev than devopsServerless is more findev than devops
Serverless is more findev than devops
 
OSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian Heigl
OSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian HeiglOSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian Heigl
OSDC 2017 | Experiences with Rudder, is it really for everyone? by Florian Heigl
 
OSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyone
OSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyoneOSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyone
OSDC 2017 - Florian Heigl - Experiences with rudder, is it really for everyone
 
Ditlev bredahl on app
Ditlev bredahl on appDitlev bredahl on app
Ditlev bredahl on app
 

Mais de Engine Yard

6 tips for improving ruby performance
6 tips for improving ruby performance6 tips for improving ruby performance
6 tips for improving ruby performanceEngine Yard
 
Simplifying PCI on a PaaS Environment
Simplifying PCI on a PaaS EnvironmentSimplifying PCI on a PaaS Environment
Simplifying PCI on a PaaS EnvironmentEngine Yard
 
The Tao of Documentation
The Tao of DocumentationThe Tao of Documentation
The Tao of DocumentationEngine Yard
 
Innovate Faster in the Cloud with a Platform as a Service
Innovate Faster in the Cloud with a Platform as a ServiceInnovate Faster in the Cloud with a Platform as a Service
Innovate Faster in the Cloud with a Platform as a ServiceEngine Yard
 
JRuby: Enhancing Java Developers Lives
JRuby: Enhancing Java Developers LivesJRuby: Enhancing Java Developers Lives
JRuby: Enhancing Java Developers LivesEngine Yard
 
High Performance Ruby: Evented vs. Threaded
High Performance Ruby: Evented vs. ThreadedHigh Performance Ruby: Evented vs. Threaded
High Performance Ruby: Evented vs. ThreadedEngine Yard
 
Release Early & Release Often: Reducing Deployment Friction
Release Early & Release Often: Reducing Deployment FrictionRelease Early & Release Often: Reducing Deployment Friction
Release Early & Release Often: Reducing Deployment FrictionEngine Yard
 
Rails Antipatterns | Open Session with Chad Pytel
Rails Antipatterns | Open Session with Chad Pytel Rails Antipatterns | Open Session with Chad Pytel
Rails Antipatterns | Open Session with Chad Pytel Engine Yard
 
JRuby: Apples and Oranges
JRuby: Apples and OrangesJRuby: Apples and Oranges
JRuby: Apples and OrangesEngine Yard
 
Debugging Ruby Systems
Debugging Ruby SystemsDebugging Ruby Systems
Debugging Ruby SystemsEngine Yard
 
Everything Rubinius
Everything RubiniusEverything Rubinius
Everything RubiniusEngine Yard
 

Mais de Engine Yard (12)

6 tips for improving ruby performance
6 tips for improving ruby performance6 tips for improving ruby performance
6 tips for improving ruby performance
 
Simplifying PCI on a PaaS Environment
Simplifying PCI on a PaaS EnvironmentSimplifying PCI on a PaaS Environment
Simplifying PCI on a PaaS Environment
 
The Tao of Documentation
The Tao of DocumentationThe Tao of Documentation
The Tao of Documentation
 
Innovate Faster in the Cloud with a Platform as a Service
Innovate Faster in the Cloud with a Platform as a ServiceInnovate Faster in the Cloud with a Platform as a Service
Innovate Faster in the Cloud with a Platform as a Service
 
JRuby: Enhancing Java Developers Lives
JRuby: Enhancing Java Developers LivesJRuby: Enhancing Java Developers Lives
JRuby: Enhancing Java Developers Lives
 
High Performance Ruby: Evented vs. Threaded
High Performance Ruby: Evented vs. ThreadedHigh Performance Ruby: Evented vs. Threaded
High Performance Ruby: Evented vs. Threaded
 
Release Early & Release Often: Reducing Deployment Friction
Release Early & Release Often: Reducing Deployment FrictionRelease Early & Release Often: Reducing Deployment Friction
Release Early & Release Often: Reducing Deployment Friction
 
Rails Antipatterns | Open Session with Chad Pytel
Rails Antipatterns | Open Session with Chad Pytel Rails Antipatterns | Open Session with Chad Pytel
Rails Antipatterns | Open Session with Chad Pytel
 
JRuby: Apples and Oranges
JRuby: Apples and OrangesJRuby: Apples and Oranges
JRuby: Apples and Oranges
 
Debugging Ruby Systems
Debugging Ruby SystemsDebugging Ruby Systems
Debugging Ruby Systems
 
Geemus
GeemusGeemus
Geemus
 
Everything Rubinius
Everything RubiniusEverything Rubinius
Everything Rubinius
 

Último

The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 

Último (20)

The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 

Rails Hosting and the Woes

  • 1. Hosting and the Woes by Jamie van Dyke Engine Yard Developer (UK)
  • 2. Jamie van Dyke My name is Jamie van Dyke, and in case you can see me from the back, I look ...
  • 3. like this fella here. I used to be the ...
  • 4. support ...Application Support Manager for Europe, but now I’ve transitioned over to ...
  • 5. develop cool stuff the Automation Development team over in ...
  • 6. England ...the UK. ( So we do 24 hour development as well as support now ;-)
  • 7. ...just in case you forgot about us, we’re this little island here...we don’t have as many nukes as you...but we’ve got Gordon Ramsay who’s even scarier! Enough about me, here’s...
  • 8. Tom Mornini ...Tom Mornini, who looks like...
  • 9. ...this. Study the face well and feel free to grab him at any time during the conference to talk shop, or just question about anything. Tom is our...
  • 11. The Chief ...CTO, Tom has experience in...
  • 12. jack every technology at Engine Yard...and can put many to shame...
  • 13. of all trades ...in most of these areas. Moving on, we have...
  • 15. ...like this. Taylor is our...
  • 16. king ...Application Support Director. He’s the man responsible for keeping everyone happy, so if you’re a customer and you’re not (is that even possible?), he’s the man to speak to. He has tonnes of skills in many...
  • 17. Rails ++ ...areas, of course Rails is one of them, but we consider him to be...
  • 18. ...one of our Supermen in the App Support Team. Finally, we have...
  • 19. Edward Muller ...Edward Muller. Edward looks like...
  • 20. ...this, and he’s one of those incredibly clever people you meet that...
  • 21. fluent in binary ...is able to talk to a computer instead of typing on the keyboard. Ed was and to some degree still is our...
  • 22. cluster ninja ...Cluster Engineer. After Jayson Vantuyl conceived and built ey00, our first cluster, Edward went on to build all the others and make improvements along the way. He’s now my ...
  • 23. automation manager ... Manager, in the automation department. We basically improve the lives of the Engine Yard staff, by automating the processes they do every day. For example, deploys can take a few hours manually, but with a sprinkling of automation we can pull that down to a few minutes.
  • 24. </intro> Enough about us, let’s talk about some of the problems that our customers have bumped into and how to solve them.
  • 25. Problems Through hundreds of customers and hours of debugging, we’ve found a recurring set of problems that crop up with Rails development, and we’d like to make you aware of them if you aren’t already. No surprise, top of the list...
  • 27. ...I’ve seen a find(:all) that goes crazy and chews up memory like there’s unlimited banks of it. You all know the solution to this, in views you use pagination and in processing loops you use batches. The main cause of this going unnoticed before hitting production, is that...
  • 28. development ...on a development machine you have a small set...
  • 29. 20 records ...of records. Your tests run against them and everything works fine. Your machine has 2Gb of RAM and you don’t sit watching the memory usage as you test. However...
  • 30. production ...in production your site might be really popular and you’re running...
  • 31. millions of records ...millions of users and assets for each of those. Running a find(:all) on that would probably cause a problem. :-/ So paginate, and make sure you have...
  • 32. indexes ...properly indexed your tables. At Engine Yard we use a team of DBA’s and the MySQL Performance Blog Experts to help our customers get the best speed out of their databases.
  • 33. plugins There are a handful of technologies and plugins that cause problems for a lot of our customers too. For example...
  • 34. ferret ...the popular ferret indexing engine along with numerous different plugins that provide interfaces to it, is constantly corrupting indexes. Initially you’d think that it’s multiple processes accessing the same index...but we always use ferret server, so it’s not that. Our solution is simple and effective...
  • 35. sphinx ...switch over to sphinx, which has a lot less problems (if any) and does the same job. We’ve seen ferret indexes at 5Gb that are reduced to 1Gb when moved to sphinx. In conjunction with Evan Weaver’s ultrasphinx plugin, this is the ideal solution.
  • 36. image science image_science is great but there are a couple of gotchas for when you shift to using it in production, with monit...Image Science requires environment variables (INLINE_DIR or HOME) set up to work, monit clears these variables...so either use a wrapper script or set them in your app manually.
  • 37. hodel 3000 ...hodel compliant logging allows you to use a specific log format that reporting tools can handle. However, out of the few plugins that are out there for it, they neglect to inform you that you *need* to set logging to :info, otherwise your logs will (default to debug), be absolutely huge, and they don’t need to be if you want your reporting tools to work.
  • 38. ...ever wonder what kind of traffic you can expect from different media coverages? okay, let’s have a look at a few...
  • 39. can haz trafficz? ...ever wonder what kind of traffic you can expect from different media coverages? okay, let’s have a look at a few...
  • 40. Digg so you get featured on digg’s homepage, how much traffic? Well, you’re looking...
  • 41. 10,000+ visitors ...at tens of thousands of visitors...but they’re all just checking out what’s new and featured, because...
  • 42. signups -- ...you’ll rarely get many signups to your service. It’s all about seeing what’s new. Which is exactly what we can say...
  • 43. TechCrunch ...about TechCrunch. The only difference being the level of traffic, which means more likely...
  • 44. 1000+ visitors ...thousands of visitors, you might hit tens of thousands but it depends on what your site is all about. A Ruby specific site...
  • 45. Ruby Inside/Flow ...like Ruby Inside or Ruby Flow, will send you between...
  • 46. 500-1000 visitors ...five hundred and a thousand visitors a day, and you’ll get that for a few days to a week, you’ll also get virtually...
  • 47. signups -- ...no signups from that. Which if you’re a ruby/rails specific site, will be higher than if you’re not. It’s all whether your site is relevant to the market with these smaller sites, when I say smaller I mean not on the scale of ...
  • 48. The Today Show ...the Today Show, which is a great traffic provider, if you’re hosted with a provider who can scale you quickly and provide you with a team of Rails experts that will help you through it, by both scaling up your service, as well as providing quick advice on how to increase the efficiency of your pages. Someone like us ;-) You’re looking at...
  • 49. 100,000+ visitors ...a huge number of visitors which is dependent on your site content, of course. One of our customers called Catalog Choice got featured on the Today show and the traffic spike was enormous, as well as obtaining...
  • 50. 10,000 signups ...them ten thousand signups in the first hour alone. So this is definitely one of the media spots you want to get featured on. Another, not surprisingly, high traffic provider ...
  • 51. Fox News Business Show ...that could do you well, is Fox News Business Show. From just showing a site url on the tv, you can expect...
  • 52. 2000+ connections ...a whole chunk of visitors. 2000-3000 simultaneous visitors come rushing your way. Pray to the Gods that you’re ready to scale up for this one. ;-)
  • 53. $10,000,000 The ten million dollar question is always, how many users can one of our slices handle. Well, how long is a piece of string?
  • 54. can you code? Of course it’s all dependent on what your code quality is like. We can say that an average Rails application running over 3 mongrels, can trivially handle millions of unique visitors a month.
  • 55. queries per page So, take a look at how many queries are running per page...
  • 56. cache cache cache ... cache wherever possible...
  • 57. file io ...keep file io down to a minimum...
  • 58. separate services ...and if you can, move your services on to different servers. Like assets in one place, application in another, database in another, etc.
  • 59. goodies Let’s wrap up the talk with a mention of some open source goodies we have that you can download...then we’ll move on to the Q and A section.
  • 60. eycap (gems.engineyard.com) This is a gem we developed that extends capistrano, it allows logging of deploys, has tons of tasks for sphinx / ferret / memcached / etc. If you use nginx, or monit, or sphinx or....whatever, then this gem will be handy for you, download it using ...
  • 61. gem source -a http://gems.engineyard.com gem install eycap --no-rdoc --no-ri (in deploy.rb) require ‘eycap’ ...these commands, with sudo if you’re on a mac...and you get lots of capistrano goodies...like
  • 63. ask us anything We filled the rest of the presentation with a q & a section, which went extremely well. Thank you to everyone who came and participated.