The community platform at The New York Times is a PHP-based, multi-tiered application that utilizes MySQL, Apache and Memcached. It was originally hosted on our own internal servers. We recently moved the platform to the cloud in order to increase its flexibility and to better handle high-load events (like links from the Yahoo home page). Moving an existing application to the cloud presented both organizational and technical challenges. We discuss our adventures in the cloud so far. Topics will include cloud management, auto-scaling and deployment on the cloud.

1. Moving to the Cloud
with NYTimes.com
Ben Gerst
Vadim Jelezniakov
OSCON 2010
Friday, August 13, 2010

2. Vanilla AWS - Do
Everything Yourself
Choose / Build Images
‘Slice’ Your Resources
Store Meta Data
Define Process
etc.
Friday, August 13, 2010

3. Great For Start-Ups
Nothing to Lose,
Everything to Gain
Friday, August 13, 2010

4. Difficult for
Grown-Ups
Friday, August 13, 2010

5. Difficult for
Grown-Ups
Like a 150 y.o.
Newspaper with a
14 y.o. Web Site
and a 3 y.o. building
that the guy on the right
climbed in 2008
Friday, August 13, 2010

6. But First...
Why Move?
Ben talks about
Community
Calls at 6PM on Friday
and Beer
Friday, August 13, 2010

7. What is UGC
@nytimes?
Friday, August 13, 2010

8. Comments on Articles and Blogs
Friday, August 13, 2010

9. Rate and Review for Movies,
Theater, Dining and Travel
Friday, August 13, 2010

10. once upon a time...
Friday, August 13, 2010

11. Friday, August 13, 2010

12. hmm...
• need to be able
to scale quickly
on demand
• we don’t like
calls from
systems at 6pm
on Fridays
Friday, August 13, 2010

13. buy more hardware
You've been down there, Neo. You already know
that road. You know exactly where it ends.
Friday, August 13, 2010

14. move to the cloud
Friday, August 13, 2010

15. traffic spikes - add capacity
Friday, August 13, 2010

16. traffic spikes - add capacity
Friday, August 13, 2010

17. nytimes.com community
architecture circa. 2007
Friday, August 13, 2010

18. ugc architecture
Friday, August 13, 2010

19. gap analysis
• load balancing?
• how do we manage communication
between instances? what about talking
back to the datacenter?
• how do we scale up and back?
• how do we secure the instances?
Friday, August 13, 2010

20. nginx
• elastic ip points to nginx which handles all
of our traffic
• nginx has the rules which determine where
to send requests
Friday, August 13, 2010

21. communication &
scaling
• host files
• monit
Friday, August 13, 2010

22. security groups
production
cmty
cmty- cmty-
cmty-fe cmty-api
cache mysql
Friday, August 13, 2010

23. monitoring
Friday, August 13, 2010

24. development instances!
Friday, August 13, 2010

25. Grown-Ups:
Existing Organization
Software Developers
QA Specialists
Management
Infrastructure Engineers
System Administrators
Friday, August 13, 2010

26. Grown-Ups:
Existing Process
Development
QA
Staging
Production
Security / Compliance
Friday, August 13, 2010

27. Grown-Ups:
Existing
Infrastructure
Authentication
Source Control
Monitoring
Network Security
Friday, August 13, 2010

28. Other Cloud
Managers
No Auth Hooks
No Slices (Subaccounts)*
No Monitoring Hooks
Expensive*
Friday, August 13, 2010

29. Nimbul
Light Cloud Manager
http://github.com/nimbul/nimbul
Emissary
Fast AMQP Messaging
http://github.com/nimbul/emissary
CloudSource
Simple SVN Deployment
http://github.com/nimbul/cloudsource
based on ServerMattic developed by WordPress
Friday, August 13, 2010

30. Nimbul
Meta Data Store
Configuration Management
Access Control
Publishers
Sane Auto-Scaling UI
F2WW
Friday, August 13, 2010

31. Nimbul Cloud
Providers
( EC2 )
Provider Accounts
( Dev, Staging, Production )
Clusters (“Slices”)
( UGC Staging, WWW Production )
Server Profiles
( UGC FrontEnd, UGC MySQL Master )
Instances
Friday, August 13, 2010

32. Nimbul Users
Nimbul Admins
( Full Access, can’t read keys )
Before Nimbul
Provider Account Admins
( Control Users, Resources, Env Vars, Startup Scripts, etc )
Cluster (“Slice”) Admins
( Control Users, Resources, Env Vars, Startup Scripts, etc )
SSH Users
( Can be granted SSH access to any running instance )
After Nimbul
Friday, August 13, 2010

33. Extending Existing Infrastructure
to the Cloud
Friday, August 13, 2010

34. Friday, August 13, 2010

35. Friday, August 13, 2010

36. Friday, August 13, 2010

37. Friday, August 13, 2010

38. Friday, August 13, 2010

39. Friday, August 13, 2010

40. Friday, August 13, 2010

41. CloudSource
SVN + bash
Role in SVN:
files/
etc/
my.cnf
<svn:external opt/nyt/app>
scripts/
install.sh
Deploy:
role.sh apply role1,role2,role3
Update:
role.sh update role
Friday, August 13, 2010

42. Friday, August 13, 2010

43. Nimbul
Light Cloud Manager
http://github.com/nimbul/nimbul
Emissary
Fast AMQP Messaging
http://github.com/nimbul/emissary
CloudSource
Simple SVN Deployment
http://github.com/nimbul/cloudsource
based on ServerMattic developed by WordPress
Friday, August 13, 2010