1. 1
Plone TuneUp challenges
in search of enterprise intranet solution
Andrew Mleczko (redturtle.net)
European Plone Symposium 2009
Sorrento, Italy

2. 2
Poznan Ferrara

3. 3
Co-founder of ACC Poland Ltd., the first Polish
●
Plone company (past)
Plone consultant at RedTurtle Technology -
●
based in Ferrara (present)
Plone-based and Lotus-based web applications
●

4. 4
RedTurtle Technology
“Sharing knowledge
about knowledge sharing”

5. 5
So what can you expect...
Plone is just a CMS?
●
Case study – Intranet Provincia di Ferrara
●
Loadtests environment
●
Plone TuneUp challenges
●

6. 6
How we can use
Plone?

7. 7
Plone variety
Community portals
●
Document management system (DMS)
●
Groupware - Collaboration
●
Subsite management system
●
Enterprise intranets
●
...
●
Combination of all mantioned above
●

8. 8
Plone as CMS
vs.
Plone as framework

9. 9
“Plone is NOT a general web development
framework.“
/Carlos de la Guardia/
Why?

10. 10
Plone can be SLOW*
*when using the wrong way

11. 11
Framework comparision
Used hardware:
●
CPU: AMD OpteronT Processor 146 (2 GHz)
–
Memory: 2 GB
–
Testing with Siege 2.68
●
Read scenario
●
Results for 50, 200, 300 Concurent Users (CU)
●
More details:
●
www.alrond.com/en/2007/jan/25/performance-test-of-6-leading-frameworks

12. 12
Comparision
Response time, secs.
2,5
2
1,5
50 CU
200 CU
300 CU
1
0,5
0
Django RubyOnRails TurboGears

13. 13
Comparision
Longest transaction, secs.
35
30
25
20
50 CU
200 CU
15 300 CU
10
5
0
Django RubyOnRails TurboGears

14. 14
So where is Plone?

15. 15
Far, far away
Response time, secs. Longest transaction, secs.
18 60
16
50
14
12 40
10
50 CU 50 CU
30 200 CU
200 CU
300 CU
300 CU
8
6 20
4
10
2
0 0
Django RubyOnRails TurboGears Plone Django RubyOnRails TurboGears Plone

16. 16
But comparing with Zope2
Response time, secs.
18
16
14
12
10
50 CU
200 CU
8
300 CU
6
4
2
0
Django + Psyco RubyOnRails 1.2.1 TurboGears Plone Zope2

17. 17
What can we do...
Do not treat Plone as a framework
●
For some use cases:
●
use ContentMirror + fast framework (django,
pylons, repoze)
But what about enterprise intranet solutions?
●

18. 18
Plone for enterprise intranet
Large number of users (more than 500) and
●
documents (more than 100 000)
A lot of simultaneous write requests (>20 CU/s)
●
Using of caching limited
●
Slowest acceptable resp. time <10 seconds
●
the limit for keeping the user's attention focused
[Miller 1968; Card et al. 1991]

19. 19
Intranet Provincia di Ferrara
y
d
tu
s
e
s
a
C

20. 20
Prerequisites
800 users stored in Domino Server (by PAS)
●
170.000 documents
●
Yearly growth: ca 25.000 documents
●
Peak traffic: 20 CU/s - write transactions
●
(based on Domino logs)

21. 21
Objective
User response time (mixed read/write):
●
Slowest (5-6 seconds)
–
Recommended (2-4 seconds) 90% of all request
–
Fast (<2 seconds)
–

22. 22
Test environment
Hardware (current):
●
4 dual-core Intel Xeon 2.8 Ghz
–
32 GB RAM
–
Amazon EC2 (future)
●

23. 23
Test environment
collective.loadtesting
●
funkload
–
collective.funkload
–
PageTemplate Profiler
●
mr.bent
●
Munin + redturtle.munin
●

24. 24
collective.loadtesting
class Readonly(testcase.FLTestCase):
def setUp(self):
self.server_url = 'http://localhost:8080/plone'
def test_ReadOnly(self):
server_url = self.server_url
self.get(server_url + quot;/plone/login_formquot;,
description=quot;Get /plone/login_formquot;)
self.post(server_url + quot;/plone/login_formquot;, params=[
['came_from', server_url + '/plone'],
['form.submitted', '1'],
['cookies_enabled', ''],
['login_name', ''],
['pwd_empty', '0'],
['__ac_name', 'admin'],
['__ac_password', 'admin'],
['submit', 'Accedi']],
description=quot;Post /plone/login_formquot;)

25. 25
mr.bent
>>> from mr.bent.wrapper import mkwrapper
>>> from mr.bent.tests.test_bent import callcounter
>>> from Products.CMFPlone.CatalogTool import
CatalogTool
>>> mkwrapper(CatalogTool.searchResults,
callcounter, quot;catalogsearchesquot;)

26. 26
Test scenario
Log in
●
View random folder
●
Create document (2 requests)
●
View newly created document
●
Publish it
●
View home page
●
Logout
●

27. 27
Default clean Plone
Response time, secs.
60
50
40
5CU
30 20CU
60CU
20
10
0
Log in Folder view Create document Save document View Publish Home page Logout

28. 28
Default clean Plone
% Errors (ConflictErrors)
100
90
80
70
60
5CU
50
20CU
60CU
40
30
20
10
0
Log in Folder view Create document Save document View Publish Home page Logout

29. 29
Let's start TuneUp and install
the basics:
Varnish + Pound (with sticky session) +
+ 6 ZEO Clients (18 threads) + ZEO Server

30. 30
Apache – proxy/rewriting urls
●
Varnish – cache resources, add http headers
●
Pound – balancing with sticky sessison
●
6 ZEO clients
●
ZEO Server
●

31. 31
+ Varnish + Pound + ZEO
Response time, secs.
40
35
30
25
5CU
20
20CU
60CU
15
10
5
0
Log in Folder view Create document Save document View Publish Home page Logout

32. 32
+ Varnish + Pound + ZEO
% Errors (ConflictErrors)
100
90
80
70
60
5CU
50
20CU
60CU
40
30
20
10
0
Log in Folder view Create document Save document View Publish Home page Logout

33. 33
Comparing to default Plone

34. 34
Comparision
Response time, secs., 20 CU
40
35
30
25
20 Default Plone
Improvment
15
10
5
0
Log in Folder view Create document Save document View Publish Home page Logout

35. 35
Where are the conflict errors
comming from?
mainly portal_catalog

36. 36
Let's try:
mount portal_catalog to different ZODB
and increase zodb cache on it

37. 37
Install redturtle.catalogmount
●
Add zope-conf-additional:
●
<zodb_db catalog>
mount-point /plone/portal_catalog
container-class Products.CMFPlone.CatalogTool.CatalogTool
cache-size 300000
<zeoclient>
server ${zeoserver:zeo-address}
storage 2
name catalogstorage
var ${buildout:parts-directory}/instance1/var
cache-size 400MB
</zeoclient>
</zodb_db>

38. 38
And add zeo-conf-additional:
●
<filestorage 2>
path ${buildout:directory}/var/filestorage/CatalogData.fs
</filestorage>

39. 39
+ redturtle.catalogmount
Response time, secs.
25
20
15
5CU
20CU
60CU
10
5
0
Log in Folder view Create document Save document View Publish Home page Logout

40. 40
+ redturtle.catalogmount
% Errors (ConflictErrors)
100
90
80
70
60
5CU
50
20CU
60CU
40
30
20
10
0
Log in Folder view Create document Save document View Publish Home page Logout

41. 41
Maybe there is a more
stable solution:
collective.indexing
experimental.catalogqueryplan
archetypes.schematuning

42. 42
+ collective.indexing + archetypes.schematuning
Response time, secs.
16
14
12
10
5CU
8
20CU
60CU
6
4
2
0
Log in Folder view Create document Save document View Publish Home page Logout

43. 43
+ collective.indexing + archetypes.schematuning
% Errors (ConflictErrors)
100
90
80
70
60
5CU
50
20CU
60CU
40
30
20
10
0
Log in Folder view Create document Save document View Publish Home page Logout

44. 44
So what is the improvment?

45. 45
Improvment
Response time, secs., 20 CU
40
35
30
25
20 fresh Plone
after TuneUp
15
10
5
0
Log in Folder view Create document Save document View Publish Home page Logout

46. 46
Let's try to scale it...

47. 47
ZEO scalability problem
Response time, secs. 20 CU
45
40
35
30
25
1:10
4:20
20
6:18
10:30
15
10
5
0
Log in Folder view Create document Save document View Publish Home page Logout

48. 48
ZEO scalability problem
% Errors (ConflictErrors), 20 CU
60
50
40
1:10
30
4:20
6:18
10:30
20
10
0
Log in Folder view Create document Save document View Publish Home page Logout

49. 49
Plone intranet limits
ZODB is slow when handling concurrent writes
●
transactions
Scalability problem using flat ZEO
●
Missing 'out of the box' performance
●
(comparing to other framworks, CMS)

50. 50
How we can solve the problem?

51. 51
Ideas
RelStorage for scalability problem
●
collective.solr
●
others ?
●
share knowledge at plone-enterprise mailing list
●