3. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
4. How to design and develop
big and challenging sites with
Drupal?
19. 55 people, over 120 customers,
over 3 MEUR revenue 2011, profitable
20. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
22. Platform
Since Drupal is a PHP software, all platform
enhancements regarding PHP in general will
usually work for Drupal as well
23. Platform
Always use a opcode cache (e.g. APC) with
optimized settings
Nginx/FPM uses less memory than Apache/
mod_php and results in better performance
especially with a lot of concurrent users
Nginx is also way more powerful in serving binary
files
Drupal uses MySQL always, so make sure your
MySQL settings are optimal (they vary with different
data)
25. Drupal settings
Drupal has some internal settings concerning
performance
In Configuration/Performance you can set the
Drupal internal anonymous page cache on
You can also enable JS and CSS aggregation
In Configuration/Logging you can also tune
logging settings
26. Drupal settings
Always use Drupal page cache unless you’re
using an external anonymous cache (or you
have something special preventing it)
JS/CSS aggregation will make the browsing
experience better, but won’t affect your backend
load level
Logging, in some cases, may cause serious
load to the database, so keep your logs short
and sweet
28. Drupal performance
modules
Drupal has a lot of performance –related modules
This doesn’t mean Drupal’s especially slow, it’s
comparable to other CMSs, but that Drupal is used
a lot on really high volume sites
I’ll go through some of the key modules next
There’s a lot of different modules available, for more
info follow the High Performance group on
Drupal.org (http://groups.drupal.org/high-
performance)
29. Replacing Drupal cache
backend
Drupal uses it’s only storage engine MySQL for also
storing cache, sessions and locks
Using different modules you can make it use other
available storage backends
Memcache (http://drupal.org/project/memcache) is
a module that will make Drupal use Memcached as
the cache, sessions and/or locks backend
It makes Drupal’s internal cache way more effective
and lessens the burden on the MySQL
There are similar modules for many different
backends, such and APC, MongoDB and Redis
30. Optimizing Drupal for
anonymous users
To gain maximum performance for anonymous
users you can use an external cache
If you can’t use an external caching software,
there are still solutions for using something other
than Drupal for caching
If you can use an external software then Varnish
is the best choice
Varnish is an external http caching software that
can serve 10 000 requests per second
31. Using an external cache
With no extra software the right way is to use Boost
(http://drupal.org/project/boost) which stores all
ready-made html pages to the disk and serves
them from there
If you can install Varnish, that would be even faster
To use Varnish, you need to use some module
enabling you to tune the cache headers on Drupal’s
end, Varnish (http://drupal.org/project/varnish) and
Cache Control (
http://drupal.org/project/cache_control) should both
do the trick
32. Optimizing Drupal for
logged in users
After you’ve optimized your site for anonymous
users, it’s time to look into logged in users
If your site doesn’t have logged in users, that’s
great and makes things easy
If your site is only accessible through a login,
then this should get really interesting for you
33. Optimizing Drupal for
logged in users
When a user logs in to Drupal, the Drupal
internal page caching turns off
That’s to enable users to have personalized
content
It also makes the site significantly slower and
burdens the back-end
There are solutions for making things better
34. Cache Control
Already mentioned Cache Control module is a
module that enables you to cache most of a page
being served to a logged in user
The user will get the same cache stored page from
Varnish that the anonymous users get but it will be
enriched with personalized content using AJAX
Cache Control can give a significant boost to logged
in user peformance
There’s also the ESI module, which uses the Edge
Side Includes functionality, a technology that has it’s
downsides compared to pure Javascript
35. Front themer
Front themer (http://drupal.org/project/front_themer)
is a module for moving theming partially to the front-
end
Drupal’s own theming engine user a lot of CPU
cycles when theming small snippets of personalized
content for users
Front themer can be used to use those CPU cycles
on the user’s computer instead of the server
36. Certain case-specific
optimizations
When using Drupal for a big site, you might
bump into a special problem, that will only occur
with large amounts of data
For example, if your menus grow into tens of
thousands of items, Drupal’s node edit page
becomes unusable
Or, using Views with a large database can
cause problems in searching the rows to be
displayed
37. Menus
To optimize very large menus, you should use
Menuperformance (
http://drupal.org/project/menuperformance)
It splits the Drupal internal menu selection
dialogues to smaller bits and only queries the
database part by part
It makes the node edit page actually work with
large menus
38. Views backend
When a large node table is searched for a View,
it might cause problems with the MySQL
Search API (http://drupal.org/project/search_api)
enables you to switch the backend from MySQL
With submodules, you can switch to backend to
Apache SOLR or MongoDB
This will make small Views calls to large
databases faster
39. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
41. It’s a place where people come to
Discuss and chat about stuff
Find new online friends
Date online and offline
Read and send emails
Check news and tv programming
Buy and sell stuff
Read comics and horoscopes
Etc...
June 1, 2012 | 41
42. And it’s relatively big place
Over 12 million pages
Hundreds of gigabytes of DB’s
Around 30 TB storage
Millions of lines of code
And so on...
June 1, 2012 | 42
43. Traffic and usage patterns
In addition to plain requests consider
- Static or dynamic content
- Personalized features for authenticated users
- Amount of simultanous users
- Read or write intesive features
- API traffic
- 3rd party systems
- SSL
June 1, 2012 | 43
44. Key aspects for mastering the performance dilemma
1. Know your current traffic and usage patterns
2. Know the solutions and hacks that give you the current performance
3. Know your performance bottlenecks
4. Understand if a change or a new feature affects the balance
between 1, 2 and 3
5. Refactor, optimize, repeat
6. Test performance during development cycle (CI, acceptance, release)
7. Have a scaling plan ready for the infrastructure
8. Make sure that your team and subcontractors is aware of all the
above
June 1, 2012 | 44
45. How to keep Drupal performing
1. The basic stuff
- Choose the right caching solutions matching your traffic and usage patterns
- Run Pressflow and use slave reads
- Offload applicable data to Solr or NoSQL database
- Keep the codebase clean and don’t use excess or heavy modules
2. Look at your product roadmap
- Is there something that will change the load or usage patters?
- Try to estimate beforehand when do you need more capacity or refactor
performance related solutions
3. Look at your site usage and data growth
- Should we remove stuff from the DB?
- How much growth we can take with these features and services?
June 1, 2012 | 45
46. Tips for scaling the infrastructure
• Make sure you have nicely clustered and virtualized infrastructure
• and that all the clusters can be scaled independently
• and maybe you can scale to the cloud?
• You can also automate scaling!
• Network is hardest part to scale. Make sure both internal and internet
bandwidth have plenty of room to grow.
• Consider fluid storage if you have a lot of data
June 1, 2012 | 46
48. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
50. Continous availability
Continous availability means that our servers
have to be allways on and serving the content
Traditionally this means…
multiple servers,
backups and disaster plans,
software version upgrades with detailed plans
But it is also linked to changes in content
52. Changes in linked content
This page is being This page is being
This page is being
edited and has edited and has
edited and has
links to other links to other
links to other
pages. pages.
pages.
- Link 1 - - Link to aapage
Link to page
- Link 2 - - Link to aapage
Link to page
- Link 3 - - New link to…
New link to…
53. Atomic change
All has changed or nothing has changed
Prevent inconsistency during the change by
changing everything on one go
Content version upgrade with detailed plans
Editing as draft, publishing with schedule
59. Huge amounts of changes
1 Lorem ipsum dolor sit
amet, consectetur
2 adipisicing elit, sed do
eiusmod tempor incididunt
3 ut labore et dolore
magna aliqua. Ut enim ad
4 minim veniam, quis
nostrud exercitation
5 ullamco laboris nisi ut
aliquip ex ea commodo
6 consequat. Duis aute
irure dolor in reprehenderit
7 in voluptate velit esse
System X
cillum dolore eu fugiat
8 nulla pariatur.
Excepteur sint occaecat
9 cupidatat non proident,
sunt in culpa qui officia
10 deserunt mollit anim id
est laborum. Lorem ipsum
11 dolor sit amet,
consectetur adipisicing
12 elit, sed do eiusmod
tempor incididunt ut labore
13 et dolore magna aliqua.
Ut enim ad minim veniam,
Server
14 quis nostrud
exercitation ullamco l
15 aboris nisi ut aliquip ex
ea commodo consequat.
16 Duis aute irure dolor in
reprehenderit in voluptate
17 velit esse cillum dolore
eu fugiat nulla pariatur.
18 Excepteur sint
Server
occaecat cupidatat non
19 proident, sunt in culpa
qui officia deserunt mollit
20 anim id est laborum.
21 …
22 …
Server
60. Challenges
Updating a node takes only a little while.
Updating thousands of nodes takes a lot longer.
When we have thousands of nodes, which are
in continous change, this can be a problem for
continous availability.
61. Caching as facade
Keep old version in cache during the big change
Clear (and reload) the cache after the change
62. What kind of availability?
Does everyone need to get the newest content?
Are some changes linked to each other?
Plan the order of priority
63. Know your use case
Plan your content change
Embrace the change
64. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
67. Sanoma – strong European media company
Aikakauslehdet
Televisio
Sanomalehdet
Oppimateriaalit
Norja Suomi
Verkkoliiketoiminta
Ruotsi
Viro Jakelu
Liettua
Venäjä
Englanti
Hollanti
Puola
Belgia
Tsekki Ukraina
Unkari
Slovenia
Romania
Kroatia
Serbia
Bulgaria
Lisäksi Esmerk toimii Suomessa, Englannissa, Ruotsissa, Saksassa, Ranskassa, Venäjällä ja Malesiassa.
AAC Global toimii Suomessa, Ruotsissa, Tanskassa, Norjassa, Englannissa, Venäjällä ja Kiinassa.
67
68. Nelonen Media is a diverse and strongly developing
broadcasting group reaching 3,7 million Finns weekly
3 ad-funded ja ruutu.fi –web-tv +
2 radio stations
5 pay-tv channels channels websites
69. TV is not dead
http://www.flickr.com/photos/schmilblick/
82. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
83. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
85. Designing and developing
multilanguage websites
The internet is a global tool
Keep this in mind from the very beginning
A single content management solution can host
multiple websites (or a group of websites)
They do not come off the assembly line
Common components can be utilized
Differences in the same product across regions
86. Opportunities
Save time and money
Keep a consistent feel
Efficient reuse of technical elements, ideas,
campaigns, content…
Developing internal competence
Improving Search Engine visibility through cross
linking
87. Challenges
Design by committee
Communication
Avoid wasted efforts
Ongoing development
One size does not fit all
The solution needs to be modular and maintainable
For some functionalities (e.g. forums) use a separate
product
Who is in control internally?
88. Pre-project phase
Sufficient planning before the RFQ
Reduces surprises
Use consultants if you need to
Define and communicate the project goal internally
We are making a public site
Bulgaria has this password protected section where…
Map legacy features
Use your local resources to map the current features
Prioritize language versions and features
89. Design phase
Defining the content structure
Less is more
Which data is common to all languages?
Features
Which features do we have and which sites will use
them?
What are the priorities?
Domain strategy
How will the sites be structured?
Prepare early for country domains
90. Implementation phase
Technical development
Make features independent
Assume everything should be translatable
Content production & entry
Prioritize the main language
Translations
Sourcing this internally or externally?
Provide worst cases early on
Prepare for life after initial deployment
91. Maintenance phase
Once the project is complete, the work begins
Minor changes
Subprojects
Keeping things under control and everyone happy
Change management is key
Deploying updates
Who is responsible?
Who can order changes?
Surprising costs
92. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion
94. Case CSR-kompassi
• What s CSR?
• Why do we bother?
• What s the CSR-kompassi?
• Why does the site need to be bilingual?
• Offering a bilingual service by using Drupal done a) the hard way or
b) the easy way
• Thanks!
95. Agenda
9.00 Opening words Janne Kalliola, Exove
9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break
10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion