Webinar by Kaliop

1. 1401 Avenue du Mondial 98
Immeuble Oxygène
34000 Montpellier
04 99 13 68 60
1 KALIOP.COM
How to deploy &
optimize eZ Publish
high-traffic / bigdata context

2. Le
High traffic ? Huge volume ?
2
High-traffic ?
Exemples
> 5 M pages / vues / mois
Peak loads ?
Big Data ?
Transactions ?
(import – export - REST)
User profiling ?
(auth – settings – peruser UI)
500K + user/day500K + user/day
+200%
1M+ datas
100k transact./day
50%+ authenticated
(portal features)
+
++
+
++
+++
Critical issues : Critical mass : Complexity to solve :
Exemples
> 5 M pages / vues / mois
Replicated ?
(PRA – Worldwide cloud)
2+ ++++

3. Le
Keys of success
3
Exemples
> 5 M pages / vues / mois
Project management
approach
Technical expertise Tools
Do not focus solely on technical
expertise !
Web Architect /
SCRUM Master
Web Architect /
SCRUM Master

4. Le
Keys of success / PM approach
4
Exemples
> 5 M pages / vues / mois
Project management
approach
Role of the Web Architect : Never stop to (re)design. Arbitrate & find
workarounds for all the contradictory issues : UX, front-end, performances, dev,
hosting, delay, cost, stability, SEO, marketing operations...
Role of the project manager (or SCRUM Master) : Protect the team.
Understand, explain & make the POC possible , code refactoring, continuous
integration & devOps
Role of the project owner : Build the team & be clear on the KPIs
Exemple : possible to challenge the UX ? (VS performances)
Exemple : ready to work without a Gantt ? (VS kanban)

5. Le
Keys of success / PM approach
5
Tools
Monitoring : complexity to monitor all the stack
(in most cases, only on low level services, not on PHP / slow queries)
Metrics : complexity to have metrics on all the the stack
(in most cases, only on low level services, or end-users trafic)
Deployment tools : the easier it is, the more consuming it is
(in most cases, copy the files & purge the cache...)
Testing : complexity to automate unit testing / user testing
(in most cases... nothing)
Capistrano

6. Le
Keys of success / Technical expertise
6
Technical expertise
(not only eZ)
FPM
Fast
CGI
APC
And more...

7. Le
Keys of success / Technical expertise
7
eZ FS2 StaleCache
INI
Templates
Médias
Viewcache
Template-block
Médias (copy)
HTTP Caches
ESI
++++
Easy to host
Save ressources
Easy to setup/switch
StaleCache
Simplify the deployment process
----
High NFS activity
Require top-level ($$$) NFS
Require an high eZ quality dev
Require frequent purges
APC APC APC
Stale Cache
( no flock )

8. Le
Keys of success / Technical expertise
8
eZ DFS Cluster
INI
Templates
Cache (copy)
INI
Templates
Cache (copy)
INI
Templates
Cache (copy)
Médias (copy)
HTTP Caches
ESI
APC APC APC
EZ DFS
Médias
Viewcache
Template-block
...
META DATA
++++
Compatible with most of platforms
Low NFS activity
Easy to setup/switch
StaleCache
----
Waste of ressources
ClusterDB bottleneck
Complex deployment process
Require an high eZ quality dev
Require frequent purges

9. Le
Keys of success / Technical expertise
9
DevOps mode
Do not expect the hosting architecture (sizing) to make up
for the application issues...
Do not expect the development (hack, trick) to make up
for the hosting architecture issues...

10. Le
Keys of success / Technical expertise
10
DevOps mode
(expertise in the middle)
APC
apc.filters = "-cache.*.php"
(exlude compiled templates)
MySql
tmpdir → tmpfs (avoid « on disk » storage)
Add missing indexes (remote_id)
Solr
Split read/write operations
Varnish
Synchronize with eZ (x-node pruge)
APC

11. Le
Keys of success / Technical expertise
11
eZ PublishOptimize your code
(what you can deduce)
Reduce the
Cache-block activity
& volume
Optimize the
viewcache
activity
Remove
Useless SQL
Use small range keys
Set an expiry value
Use ezpagedata() instead
Replace with ESI
No more than 1K files !
Avoid too many view parameters
Optimize smartviewcache
Synchronize with Varnish
Replace with Solr
Use PHP cache
Avoid datamap loadOptimize
Solr
split read / write activities
Delay « commit » (not only index)
Schedule your
background
tasks

12. Le
Keys of success / Technical expertise
12
eZ PublishOptimize your code
(read tutos)

13. Le
Keys of success / Technical expertise
13
eZ PublishOptimize your code
(Learn how it works)
Basic optimization :
DelayedIndexing = true (→ will not delay hard commits for all actions ! )
OptimizeOnCommit = disabled
CRON // ezfoptimizeindex (3 minutes)
CRON // ezfindexcontent (1 day)
Medium optimization :
OptimizeOnCommit = disabled
DisableDeleteCommits = true (+ CRON to commit)
DisableDirectCommits=true (+ CRON to commit)
CRON // bin/php/ezsolrcommit.php (3 minutes)
CRON // ezfoptimizeindex (1 day)
Other great optimizations :
- Replicate Solr, and split the Read (search..) / Write (commit, optimize), to avoid concurrencies
- Avoid hard commits (soft commits to avoid fsync ops, commitWithin to delay pending docs within Solr)
addDoc Hard Commit (rebuild indexes + fsync)
Soft Commit (rebuild indexes)
CommitWithin (commit all pending docs within X secs)

14. Le
Keys of success / Technical expertise
14
Optimize your code
(Get metrics)
(what you can not guess)
APC
EZ DFS
META DATA
GET
METRICS

15. Le
Keys of success / Technical expertise
15
Optimize your code
(Get metrics)
(what you can not guess)
Why session_start
lock ???
eZ Bug on the
checkValidity=false setting...
will always create a
session_start on the index.php

16. Le
Keys of success / Technical expertise
16
Optimize your code
(Get metrics)
(what you can not guess)
Why a unique table
insert needs 9,6
sec ???
Server side tuning requested
(memory, table optimize, host
the clusterDB on another
server...)

17. Le
Keys of success / Technical expertise
17
Optimize your code
(Get metrics)
(what you can not guess)
Why my code
provides 1988
cURL calls ???
Mmhh...
How to say...
NEXT

18. Le
Enough to resolve the following context ?
18
High-traffic ?
Exemples
> 5 M pages / vues / mois
Peak loads ?
Big Data ?
Transactions ?
(import – export - REST)
User profiling ?
(auth – settings – peruser UI)
500K + user/day500K + user/day
+200%
1M+ datas
100k transact./day
50%+ authentifié
(portal features)
+
++
+
++
+++
Critical issues : Critical mass : Complexity to solve :
Exemples
> 5 M pages / vues / mois
Replicated ?
(PRA – Worldwide cloud)
2+ ++++

19. Le
Enough to resolve the following context ?
19
Externalize the user database (if too big)
Build a full ESI front-end (pure HTTP cache logical)
Replace the DB cluster with Redis or memcached
Externalize medias on CDN
Use a custom PHP Cache (exemple)
...
NO ! Be creative !

20. Le
Enough to resolve the following context ?
20
Be creative !
Object.datamap → PHP array
PHP cache file (eZPHPCreator)
On publish
Store all the attributes
Pre-generate related values
On read
Read object
datamap
Fetch, filter,
facet, sort..
HTML

21. Le
Enough to resolve the following context ?
21
Ready for eZP5...
Native HTTP Cache
Native Storage abstraction
Native pure MVC
Native ESI support
…
Full Symfony stack