The document summarizes how caibangzi.com, a personal investment management tool and community, was built and evolved over time to handle increasing traffic demands. It started on a single server in 2007 but faced challenges with performance and bandwidth as daily pageviews reached 1 million. Key steps taken included upgrading hardware, separating application and database servers, adding a load balancer, implementing distributed computation solutions, and upgrading to Rails 2.0. Even at peak loads of 1 million daily pageviews and 160 requests per second, CPU usage never exceeded 30% due to these architectural improvements.

1. How we built
Caibangzi.com
Robin Lu (caibangzi.com)

2. Introduction
Personal Investment Management. Tool
+ Community
Founders: We are engineers!
Technology:
Ruby On Rails
MySQL，Linux，Lighttpd ...

3. Topics
Problems we are facing
Architecture evolution of caibangzi
Maintenance
Q&A

4. Peaks!
High Peak
Heavy Computation
Visiting Peak, Computation Peak and Cache
expiration Peak are in the same time. :...(

5. Evolution - Start
Launched at March, 2007
Hardware: 1 x 1U server
1xAMD Opteron 250, 2G Memory,
250G SATA, RAID 1
Bandwidth
Shared
less than 3Mbps

6. Evolution - Start
Apache 2 + mod_proxy_balancer
mongrel x 8
All in
memcached caibangzi
one
box
rails 1.1.6
MySQL 5

7. First Challenge
May, 2007
Extremely time consuming actions
blocked the site.
Performance tuning
Optimize Rails cache

8. Second Challenge
August, 2007
We were running out of bandwidth
Solution:
Money
upgrade to 5Mbps

9. Server upgrading
High growth rate after August, 2007
Hardware upgrading in Sept. 2007
still 1 x 1U
2 x AMD 2214 2.2GHz
4G RAM

10. Until Sept. 2007
Apache 2 + mod_proxy_balancer
mongrel x 20
Still in
memcached caibangzi
one
box
rails 1.1.6
MySQL 5

11. Something happened
in Oct. 2007
Stock Index reached 6000
Daily pv reached 1 million
Bigger challenge came

12. Several Changes We
Made
Upgrade bandwidth to 10Mbps
Separate APP server and DB server
Load Balancer
Distributed Computation

13. 2 Boxes
Web Server
event mongrel x 30
AP
memcached caibangzi P
rails
MySQL 5
Tianyuanji
a

14. Load balancer
Run with
2 mongrels
behind
balancer

15. Load balancer
Slow actions block the others
with balancer:
by request Apach
e
by traffic HAproxy
round-robin
Nginx
Lighttpd
1.4
Note: Nginx fixed it with
‘fair’ balancer later

16. Caibangzi Choice of
Balancer
Lighttpd 1.5 beta
proxy-core.balancer = "sqf"
http://redmine.lighttpd.
net/issues/show/1362
For Rails application, choice of balancer
is sometimes more important than
choice of web server

17. More on Web Server
disable keep-alive
serve static files with web server instead
of mongrel
tune configurations according to your
web server documentations

18. Distributed solution
BackgrounDRB
based on Distributed Ruby (DRB)
use ruby thread
high overhead during context
switching
very slow when concurrent
connections higher than 10

19. Distributed solution
FastBackground
our own distributed computation
solution
based on Eventmachine
simple, but fast and works for us

20. Rails upgrading
Early of 2008, we upgraded to Rails 2.0
expired several plugins
refactory to be more RESTful
cookie session
multi view => wap.caibangzi.com

21. Open API
API Based on OAuth
integrate caibangzi into your own sites
Other APPs support several F8 clones
xiaonei.com, hainei.com, UCHome

22. Maintenance
Munin
easy to use and extend
Monit
Crontab

23. With all that....
we handled:
pv/day 1M
rails req/day 1.8M
peak rails req/sec 160
total req/day 5M
peak req/sec 430
It’s the fact. Not the capacity.
Our CPU usage never goes higher
than 30%

24. Q&A