SlideShare uma empresa Scribd logo
1 de 14
Baixar para ler offline
Making ZOPE faster
●   Without Squid
●   Without CacheFu
●   Without Varnish
The Problem
                (my problem)
●   The website uses LinguaPlone
    (what is that?!?)
    ... It uses Cookies for
    language selection

●   Aaaaagh !
Apache + Dynamic Rewrite Map
RewriteMap rpx prg:/home/admin/rpx.py
RewriteRule ^(.*)$ ${rpx:%{HTTP_COOKIE}<>$1<>%{QUERY_STRING}} [P,L]

RewriteMap rpx prg:/home/admin/rpx.py

RewriteRule ^(.*)$
   ${rpx:%{HTTP_COOKIE}
   <>$1
   <>%{QUERY_STRING}}
   [P,L]
The Dynamic Map Script

#!/usr/bin/python -u
import sys, os, time, urllib2, urllib, re

cacheroot = "/var/www/cache"
zoperoot =
  "http://slowzope:10000/VirtualHostBas
  e/http/fastrewriter:80/VirtualHostRoot"
logpath = "/home/admin/rpx.log"
The Real Dynamic Map Script
cookies, uri, qstring = request.split('<>')
lang = re.findall('I18N_LANGUAGE="([A_Za-z]+)"', cookies)
lang = lang and lang[0] or ''
locuri = uri + "/<>" + lang
if qstring:
    log("PASS %sn"%locuri)
    sys.stdout.write(zoperoot+uri+"n")
elif os.path.isfile(cacheroot + locuri):
    log("HIT %sn"%locuri)
    sys.stdout.write("http://faststatic/cache/%sn"%locuri)
else:
    log("MISS %sn"%locuri)
    sys.stdout.write(zoperoot+uri+"n")
The Cache Feeder
if line.startswith("MISS "):
     filepath = line[5:].strip()
     uri, extra = filepath.split("/<>")
     req = urllib2.Request(zoperoot+urllib.quote(uri),
              None, {'Cookie': 'I18N_LANGUAGE="%s"'%extra})
     o = urllib2.urlopen(req)
     data = o.read()
     datadir = cacheroot + "/" + uri
     if not os.path.isdir(datadir):os.makedirs(datadir)
     f = open(datadir + "/.htaccess", "w")
     f.write("ForceType %sn"%o.info().type)
     f.close()
     f = open(cacheroot + "/" + filepath,"w")
     f.write(data)
     f.close()
     log("FETCH %s <> %sn"%(filepath,extra))
The Cache Itself
admin@cache:~/cache$ find -type f | head
./services/locations-and-contacts/.htaccess
./services/locations-and-contacts/<>
./services/locations-and-contacts/<>fr
./services/locations-and-contacts/<>de
./capabilities-1/navigation-services-1/.htaccess
./capabilities-1/navigation-services-1/<>
./services/index_html/rss.gif/.htaccess
./services/index_html/rss.gif/<>fr
I swear the code is simple !

admin@cache:~$ wc -l rpx.py
74 rpx.py
Results




Watch to the lovely pictures !
In the unlikely event you want to know
      more about this really dirty hack ...
●   Jérôme Petazzoni
    <jerome.petazzoni@pilotsystems.net>

Mais conteúdo relacionado

Mais procurados

Javascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJSJavascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJS
Ladislav Prskavec
 

Mais procurados (19)

Raspberry pi Part 4
Raspberry pi Part 4Raspberry pi Part 4
Raspberry pi Part 4
 
serverstats
serverstatsserverstats
serverstats
 
Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)
 
File::CleanupTask
File::CleanupTaskFile::CleanupTask
File::CleanupTask
 
Cpsh sh
Cpsh shCpsh sh
Cpsh sh
 
Profiling with Xhprof
Profiling with XhprofProfiling with Xhprof
Profiling with Xhprof
 
Find and zip files
Find and zip filesFind and zip files
Find and zip files
 
Express Presentation
Express PresentationExpress Presentation
Express Presentation
 
mongodb-introduction
mongodb-introductionmongodb-introduction
mongodb-introduction
 
Compile kernel
Compile kernelCompile kernel
Compile kernel
 
Bash Scripting
Bash ScriptingBash Scripting
Bash Scripting
 
Ch3(working with file)
Ch3(working with file)Ch3(working with file)
Ch3(working with file)
 
App-o-Lockalypse now!
App-o-Lockalypse now!App-o-Lockalypse now!
App-o-Lockalypse now!
 
Gaelyk
GaelykGaelyk
Gaelyk
 
working with files
working with filesworking with files
working with files
 
Web Scrapping with Python
Web Scrapping with PythonWeb Scrapping with Python
Web Scrapping with Python
 
S3 & Glacier - The only backup solution you'll ever need
S3 & Glacier - The only backup solution you'll ever needS3 & Glacier - The only backup solution you'll ever need
S3 & Glacier - The only backup solution you'll ever need
 
Abusing Erlang compilation pipeline for Fun and Profit
Abusing Erlang compilation pipeline for Fun and ProfitAbusing Erlang compilation pipeline for Fun and Profit
Abusing Erlang compilation pipeline for Fun and Profit
 
Javascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJSJavascript Continues Integration in Jenkins with AngularJS
Javascript Continues Integration in Jenkins with AngularJS
 

Destaque (6)

Neoppod, discloses first open source transactional NoSQL Object database for ...
Neoppod, discloses first open source transactional NoSQL Object database for ...Neoppod, discloses first open source transactional NoSQL Object database for ...
Neoppod, discloses first open source transactional NoSQL Object database for ...
 
DoSomething + Mobile
DoSomething + MobileDoSomething + Mobile
DoSomething + Mobile
 
Deutsche EuroShop "Acquisition of A10 Center Wildau"
Deutsche EuroShop "Acquisition of A10 Center Wildau"Deutsche EuroShop "Acquisition of A10 Center Wildau"
Deutsche EuroShop "Acquisition of A10 Center Wildau"
 
Computer Beginners
Computer BeginnersComputer Beginners
Computer Beginners
 
How Animals Move - Chapter 30
How Animals Move - Chapter 30How Animals Move - Chapter 30
How Animals Move - Chapter 30
 
Transformation digitale
Transformation digitaleTransformation digitale
Transformation digitale
 

Semelhante a Plone Conference 2008 Lightning Talk Static Zope Rpx

On secure application of PHP wrappers
On secure application  of PHP wrappersOn secure application  of PHP wrappers
On secure application of PHP wrappers
Positive Hack Days
 
4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook
guoqing75
 
Facebook的缓存系统
Facebook的缓存系统Facebook的缓存系统
Facebook的缓存系统
yiditushe
 

Semelhante a Plone Conference 2008 Lightning Talk Static Zope Rpx (20)

ELK: a log management framework
ELK: a log management frameworkELK: a log management framework
ELK: a log management framework
 
On secure application of PHP wrappers
On secure application  of PHP wrappersOn secure application  of PHP wrappers
On secure application of PHP wrappers
 
4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook4069180 Caching Performance Lessons From Facebook
4069180 Caching Performance Lessons From Facebook
 
Facebook的缓存系统
Facebook的缓存系统Facebook的缓存系统
Facebook的缓存系统
 
Debugging: Rules & Tools
Debugging: Rules & ToolsDebugging: Rules & Tools
Debugging: Rules & Tools
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
 
Debugging: Rules And Tools - PHPTek 11 Version
Debugging: Rules And Tools - PHPTek 11 VersionDebugging: Rules And Tools - PHPTek 11 Version
Debugging: Rules And Tools - PHPTek 11 Version
 
Heavy Web Optimization: Backend
Heavy Web Optimization: BackendHeavy Web Optimization: Backend
Heavy Web Optimization: Backend
 
Key features PHP 5.3 - 5.6
Key features PHP 5.3 - 5.6Key features PHP 5.3 - 5.6
Key features PHP 5.3 - 5.6
 
RESTing with the new Yandex.Disk API, Clemens Аuer
RESTing with the new Yandex.Disk API, Clemens АuerRESTing with the new Yandex.Disk API, Clemens Аuer
RESTing with the new Yandex.Disk API, Clemens Аuer
 
Using and scaling Rack and Rack-based middleware
Using and scaling Rack and Rack-based middlewareUsing and scaling Rack and Rack-based middleware
Using and scaling Rack and Rack-based middleware
 
Hacking with hhvm
Hacking with hhvmHacking with hhvm
Hacking with hhvm
 
GDG DevFest 2013 - PHP Web Apps on Google Cloud
GDG DevFest 2013 - PHP Web Apps on Google CloudGDG DevFest 2013 - PHP Web Apps on Google Cloud
GDG DevFest 2013 - PHP Web Apps on Google Cloud
 
AWS Hadoop and PIG and overview
AWS Hadoop and PIG and overviewAWS Hadoop and PIG and overview
AWS Hadoop and PIG and overview
 
PHP Backdoor: The rise of the vuln
PHP Backdoor: The rise of the vulnPHP Backdoor: The rise of the vuln
PHP Backdoor: The rise of the vuln
 
PHP CLI: A Cinderella Story
PHP CLI: A Cinderella StoryPHP CLI: A Cinderella Story
PHP CLI: A Cinderella Story
 
Puppet Camp 2012
Puppet Camp 2012Puppet Camp 2012
Puppet Camp 2012
 
PHP5.5 is Here
PHP5.5 is HerePHP5.5 is Here
PHP5.5 is Here
 
Twas the night before Malware...
Twas the night before Malware...Twas the night before Malware...
Twas the night before Malware...
 
Configuration Surgery with Augeas
Configuration Surgery with AugeasConfiguration Surgery with Augeas
Configuration Surgery with Augeas
 

Mais de Paris, France

Django Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristesDjango Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristes
Paris, France
 
SeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQLSeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQL
Paris, France
 
Neopod: une base de données objet distribuée
Neopod: une base de données objet distribuéeNeopod: une base de données objet distribuée
Neopod: une base de données objet distribuée
Paris, France
 

Mais de Paris, France (20)

Django Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristesDjango Hacking : Comment dresser les serpents guitaristes
Django Hacking : Comment dresser les serpents guitaristes
 
SeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQLSeSQL : un moteur de recherche en Python et PostgreSQL
SeSQL : un moteur de recherche en Python et PostgreSQL
 
Neopod: une base de données objet distribuée
Neopod: une base de données objet distribuéeNeopod: une base de données objet distribuée
Neopod: une base de données objet distribuée
 
Conférence Sesql - DjangoCong 2011
Conférence Sesql - DjangoCong 2011Conférence Sesql - DjangoCong 2011
Conférence Sesql - DjangoCong 2011
 
API Cockpit : faites communiquer vos applications web
API Cockpit : faites communiquer vos applications webAPI Cockpit : faites communiquer vos applications web
API Cockpit : faites communiquer vos applications web
 
Indexation d'une base documentaire pour Liberation
Indexation d'une base documentaire pour LiberationIndexation d'une base documentaire pour Liberation
Indexation d'une base documentaire pour Liberation
 
Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...
Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...
Systematic/NEOPPOD dévoilent la première base de données objets NoSQL Open So...
 
Cockpit, gestion de contacts et d'emailing
Cockpit, gestion de contacts et d'emailingCockpit, gestion de contacts et d'emailing
Cockpit, gestion de contacts et d'emailing
 
Plone, du site web à la gestion documentaire
Plone, du site web à la gestion documentairePlone, du site web à la gestion documentaire
Plone, du site web à la gestion documentaire
 
Les critères de choix d'un intranet
Les critères de choix d'un intranetLes critères de choix d'un intranet
Les critères de choix d'un intranet
 
Virtualisation Cloud Computing Saas Open Source
Virtualisation Cloud Computing Saas Open SourceVirtualisation Cloud Computing Saas Open Source
Virtualisation Cloud Computing Saas Open Source
 
Partenariat Exo Platform/Pilot Systems
Partenariat Exo Platform/Pilot SystemsPartenariat Exo Platform/Pilot Systems
Partenariat Exo Platform/Pilot Systems
 
Open World Forum 2009 Synthesis
Open World Forum 2009 SynthesisOpen World Forum 2009 Synthesis
Open World Forum 2009 Synthesis
 
Open World Forum 2009 Presentation
Open World Forum 2009 PresentationOpen World Forum 2009 Presentation
Open World Forum 2009 Presentation
 
Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...
Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...
Deliverance, la puissance de la gestion de contenu Plone appliquee aux themes...
 
Deploiement de Plone et Deliverance avec Ubuntu VM Builder
Deploiement de Plone et Deliverance avec Ubuntu VM BuilderDeploiement de Plone et Deliverance avec Ubuntu VM Builder
Deploiement de Plone et Deliverance avec Ubuntu VM Builder
 
Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...
Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...
Deliverance, la puissance de la gestion de contenu Plone appliquée aux thèmes...
 
Construire un module enfichable (pluggable) et dynamique pour Plone
Construire un module enfichable (pluggable) et dynamique pour PloneConstruire un module enfichable (pluggable) et dynamique pour Plone
Construire un module enfichable (pluggable) et dynamique pour Plone
 
La virtualisation dans le Datacenter
La virtualisation dans le DatacenterLa virtualisation dans le Datacenter
La virtualisation dans le Datacenter
 
Guide pratique des logiciels libres dans les administrations
Guide pratique des logiciels libres dans les administrationsGuide pratique des logiciels libres dans les administrations
Guide pratique des logiciels libres dans les administrations
 

Último

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 

Plone Conference 2008 Lightning Talk Static Zope Rpx

  • 1. Making ZOPE faster ● Without Squid ● Without CacheFu ● Without Varnish
  • 2. The Problem (my problem) ● The website uses LinguaPlone (what is that?!?) ... It uses Cookies for language selection ● Aaaaagh !
  • 3. Apache + Dynamic Rewrite Map RewriteMap rpx prg:/home/admin/rpx.py RewriteRule ^(.*)$ ${rpx:%{HTTP_COOKIE}<>$1<>%{QUERY_STRING}} [P,L] RewriteMap rpx prg:/home/admin/rpx.py RewriteRule ^(.*)$ ${rpx:%{HTTP_COOKIE} <>$1 <>%{QUERY_STRING}} [P,L]
  • 4. The Dynamic Map Script #!/usr/bin/python -u import sys, os, time, urllib2, urllib, re cacheroot = "/var/www/cache" zoperoot = "http://slowzope:10000/VirtualHostBas e/http/fastrewriter:80/VirtualHostRoot" logpath = "/home/admin/rpx.log"
  • 5. The Real Dynamic Map Script cookies, uri, qstring = request.split('<>') lang = re.findall('I18N_LANGUAGE="([A_Za-z]+)"', cookies) lang = lang and lang[0] or '' locuri = uri + "/<>" + lang if qstring: log("PASS %sn"%locuri) sys.stdout.write(zoperoot+uri+"n") elif os.path.isfile(cacheroot + locuri): log("HIT %sn"%locuri) sys.stdout.write("http://faststatic/cache/%sn"%locuri) else: log("MISS %sn"%locuri) sys.stdout.write(zoperoot+uri+"n")
  • 6. The Cache Feeder if line.startswith("MISS "): filepath = line[5:].strip() uri, extra = filepath.split("/<>") req = urllib2.Request(zoperoot+urllib.quote(uri), None, {'Cookie': 'I18N_LANGUAGE="%s"'%extra}) o = urllib2.urlopen(req) data = o.read() datadir = cacheroot + "/" + uri if not os.path.isdir(datadir):os.makedirs(datadir) f = open(datadir + "/.htaccess", "w") f.write("ForceType %sn"%o.info().type) f.close() f = open(cacheroot + "/" + filepath,"w") f.write(data) f.close() log("FETCH %s <> %sn"%(filepath,extra))
  • 7. The Cache Itself admin@cache:~/cache$ find -type f | head ./services/locations-and-contacts/.htaccess ./services/locations-and-contacts/<> ./services/locations-and-contacts/<>fr ./services/locations-and-contacts/<>de ./capabilities-1/navigation-services-1/.htaccess ./capabilities-1/navigation-services-1/<> ./services/index_html/rss.gif/.htaccess ./services/index_html/rss.gif/<>fr
  • 8. I swear the code is simple ! admin@cache:~$ wc -l rpx.py 74 rpx.py
  • 9. Results Watch to the lovely pictures !
  • 10.
  • 11.
  • 12.
  • 13.
  • 14. In the unlikely event you want to know more about this really dirty hack ... ● Jérôme Petazzoni <jerome.petazzoni@pilotsystems.net>