Tools, Tips, and Techniques for Approaching Scalability in Plone Systems

1. unloading plone
Tools, Tips, and Techniques for Approaching Scalability
Elizabeth Leddy
@eleddy

2. am I in the right room?
Symptoms Include
inability to grow with traditional caching solutions,
icky system performance, crashitis, log-o-phobia,
hardware anemia, mashup mayhem, diminishing
confidence in your abilities as plone integrator or
administrator, where do I start migraines, all the rest of the
talks look boring, this room has better internet connectivity, I
have no clue what I’m doing here

3. CONFUCIUS
SAY
A responsive, scalable Plone setup has little to do with
optimizing code

5. Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Act

6. AUTO-RESTART
GANGLIA ZOPE HEALTH WATCHER
MUNIN MONIT CUSTOM PLUGINS TIMEOUTS
ALERTS TIME BASED GRAPHS AFFECTION
ABNORMALITIES
I quit...
Monitoring Tools

7. Watching Bad Performances
Whale watchers are
triggered by system
abnormalities, and
grab relevant
information to help
you diagnose what
happened.

8. Simple(r) Error Monitoring
Sludge Through Logs
Escalate with Email
Google Analytics for
Timeouts, 4XX & 5XX

9. 4 Stages of System Grief
Denial Anger Bargaining Acceptance
LB
HTTPD HTTPD HTTPD HTTPD^N
CACHE CACHES
PROXY PROXY PROXY
ZOPE ZOPE++ ZOPE++++ ZOPE^N
DB CACHE DB CACHES
LB
ZEO ZEO SHARD ZEO SHARD ZEO & CO.

10. H OW MUCH
A CPU DISK
R HAPROXY
D POUND ZEO
W HTTPD SQUID
A NGINX
R ZOPE
VARNISH
E ZEO CACHE?
MEMCACHE ZODB CACHE
RAM

11. Two’s Company, Three’s a Crowd
ZOPE++
DB CACHE

12. How many zopes should I have?
50% avg util
2 zopes/cpu
api/async instances?
ZOPE++

13. Complex by Default

14. The CacheFu
Conundrum
The browser is the cheapest, easiest caching tool
at your disposal, Daniel-son. It’s time to show the
world you are the developer I know you can be...
Oh, CacheFu you have been a great
leader and I wish I would have taken the
time to figure out how caching really
works. I’m afraid I will fail you, never fully
understanding the 304. Shame!

15. Serve STATIC content from a STATIC web server
to unload plone, provide faster
response time to initial requests, 5200
and enable simpler caching
strategies
2700
2600
2100
1800
1500
httpd
plone
user.gif link_icon.gif poo.gif
HTTPD RewriteRule ^/(.*)portal_skins/mysite_images/(.*) /mysite_images/$2 [L]

16. CSS & Sprites
30 requests, ~200 kb
.contenttype-folder{
background-image: url(folder.gif);
}
1 request, ~18 kb
.contenttype-folder{
background-image: url(all-my-images.png);
background-position: 75px 25px;
}
UNLOAD
HTTPD

17. Picking a Proxy or:
How I learned to stop worrying and love HAProxy
graceful reloads
backend health
distribution algorithms
warmup time
preserve keepalives
web based stats
PROXY

18. Playing Nice with Others
if python < 2.6:
socket.settimeout(2)
else:
urllib2.urlopen(url, timeout=2)
ZOPE++
ZEO & CO.

19. Pickle
Drop
Pickup
Process
ZOPE++ Callback
Pickle Me Some Asynchronicity

20. EXTERNAL DATA ST.
ZEO & CO.
tal:on-error

21. ACCEPTANCE
ZEO is perfect for object
oriented access, but it needs
a little help with other data
access paradigms
ZEO & CO.

22. RESTARTing zope is NOT a SIN
sometimes its just easier...
a PACKED zodb is a HAPPY zodb
beware of backup side effects
CRON is
deploy with 1-step
HOMEBOY
fabrics’ rolling blackouts
FACTS
ZOPE
ZEO & CO.

23. ON BACKUPS
use repozo
chunked rsync
watch the disk
ZEO & CO.

24. Plone != Drupal Scalability = Talent + Hardware

25. Living in the Cloud
The Oath:
I will automate a modular
installation of everything
I will not do anything manually
that could be done
automatically, which is everything
I can trigger unscheduled
processes anytime, from
anywhere
I appreciate that scaling is hard:
the cloud offers options, not
solutions

26. ‣ Always setup system and error monitoring first
‣ Choose the right software for YOUR hardware
‣ Use as many zopes as possible but no more
‣ Do it like a flickr engineer
‣ Don’t make zope handle unnecessary requests
‣ Never underestimate the importance of a
proper disk-RAM partnership
‣ Design processes to be able to run from
anywhere, at anytime, on any machine
SUMMARY

27. All this and more at
http://scalingplone.pbwiki.com
Thank You!