SlideShare uma empresa Scribd logo
1 de 52
Baixar para ler offline
Tricks & challenges
developing a large
Django application

    Simon Willison
    EuroPython 2011, 22nd June
    http://lanyrd.com/sfwcq
Lanyrd.com
Lanyrd.com




  Definitive database
 of Professional Events
      and Speakers
Lanyrd.com




  Definitive database       Social event recommendation
 of Professional Events        Event networking tools
      and Speakers        Archive of slides, notes and video
Part 1: Tricks
Trick #1: Signing
Lanyrd.com




Pass data through an
untrusted source with
confidence that it hasn't
been tampered with
Lanyrd.com



Signing uses
 Unsubscribe" links in emails
lanyrd.com/un/ImN6VyI.ii0Hwm7p71DEcGfaVzziQaxeuu
 Signed cookies
 "You are logged in as simonw" without
hitting the database
Lanyrd.com



Signing in Django 1.4
 from django.core import signing
 signing.dumps({"foo": "bar"})
 signing.loads(signed_string)


 response.set_signed_cookie(key, value...)
 request.get_signed_cookie(key)
Trick #2: cache_version
add a conference     you are signed in as simonw, do you want to sign out?




                                                                       calendar   conferences           coverage         profile

                                                                                                                        search




Django conferences
                                                                                                 Django
     Django events looking for participants                                                      coverage
     1 Django event is looking for participants
                                                                                                          52 videos
                                                                                                          Most recent added 3
                                                                                                          weeks ago


ON NOW                 EuroPython 2011                                                                    52 slide decks
                 19                                                                                       Most recent added 4
                          Italy / Florence                                                                hours ago
                       19th–26th June 2011
                        Django   Plone    Pyramid   Python   Twisted                                      3 audio clips
                                                                                                          Most recent added 1
                                                                                                          week ago

                                                                                                          27 write-ups
SEPTEMBER              DjangoCon US 2011
                  6                                                                                       Most recent added 1

2011                      United States / Portland
                       6th–8th September 2011
                                                                                                          week ago

                                                                                                          11 handouts
                        Django   Open Source    Python
                                                                                                          Most recent added 18
                                                                                                          hours ago

                 17    PyCON FR 2011                                                                      3 notes
                          France / Rennes                                                                 Most recent added 10
                       17th–18th September 2011                                                           hours ago

                        Django   Python



                                                                                                 By country
OCTOBER                PyCon DE 2011                                                             Ireland   1
                  4
Lanyrd.com

class Conference(models.Model):
    ...
    cache_version = models.IntegerField(default = 0)

    def save(self, *args, **kwargs):
        self.cache_version += 1
        super(Conference, self).save(*args, **kwargs)

    def touch(self):
        Conference.objects.filter(
            pk = self.pk
        ).update(
            cache_version = F('cache_version') + 1
        )
Lanyrd.com




{%   cache 36000 conf-topics conference.pk conference.cache_version %}
	     <ul class="tags inline-tags meta">
	     	   {% for topic in conference.topics.all %}
	     	   	   <li><a href="{{ topic.get_absolute_url }}">{{ topic }}</a></li>
	     	   {% endfor %}
	     </ul>
{%   endcache %}
Lanyrd.com



Bulk invalidation
from django.models import F

topic.conferences.all().update(
    cache_version = F('cache_version') + 1
)
Trick #3: NoSQL for denormalisation
Lanyrd.com




NoSQL = Not Only SQL
Lanyrd.com



But we like joins!
 Slides from a session on Lanyrd show up...
 Under "coverage of this conference"
 Under "slides by Simon Willison"
 Under "slides about Django"
Lanyrd.com




Solution: denormalise
data to Redis and Solr
Lanyrd.com
                    Commands         Clients     Documentation      Community      Download     Issues




 Redis is an open source, advanced key-value store. It is often                                          What people are saying
 referred to as a data structure server since keys can contain                                               Comparison of CouchDB, Redis,
                                                                                                             MongoDB, Casandra, Neo4J &
 strings, hashes, lists, sets and sorted sets.
 strings hashes lists                    sets                                                                others http://j.mp/l32SqM via
                                                                                                             @DZone
 Learn more →
                                                                                                             @__NeverGiveup Oh YAY, oui tu
                                                                                                             me redis ! *-* Hm, on s'rejoint à
 Try it                                                     Download it                                      14h au bahut ? :o

 Ready for a test drive? Check this interactive             Redis 2.2.10 is the latest stable version.       JE L REDIS JE FOLLOW BACK
                                                                                                             SUR @Fuckement_TL
 tutorial that will walk you through the most               Interested in legacy or unstable versions?
 important features of Redis.                               Check the downloads page.                        une question : "How to use
                                                                                                             ServiceStack Redis in a web
                                                                                                             application to take advantage of
                                                                                                             pub / sub paradigm"
                                                                                                             http://t.co/EOgyLU1 #redis #web
                                                                                                             Nice - Cassandra vs MongoDB vs
                                                                                                             CouchDB vs Redis vs Riak vs
                                                                                                             HBase vs Membase vs Neo4j
                                                                                                             comparison http://bit.ly/l32SqM
                                                                                                             from @kkovacs
                                                                                                                                          More...




                                                                                                                                 Sponsored by
    This website is open source software developed by Citrusbyte.
    The Redis logo was designed by Carlos Prioglio.
Lanyrd.com



Our redis sets...
 simonw-follows:{144,21345,12328...}
 europython-attendees:{344,21345,787...}


 redis.sinter(
   'simonw-follows',
    'europython-attendees'
 )
add a conference   you are signed in as simonw, do you want to sign out?


Lanyrd.com
                                                              calendar          conferences       coverage         profile

                                                                                                                  search




 EuroPython 2011                                                                            You're
                                                                                            speaking
 The European Python Conference                                                             AT THIS EVENT




 19 –26              JUNE
                     2011
                                                   Florence
                                                   in Italy


                                                                                           97 attending
    http://ep2011.europython.eu/               @europython                                            PEOPLE

    View the schedule on Lanyrd                #europython

                                                                                           80 tracking
                                                                                                      PEOPLE
    Save to iCal / iPhone / Outlook /          lanyrd.com/ccdpc   (short URL)
 GCal
                                                                                                    TELL YOUR FRIENDS!
                                                                                                    Tweet about this
                                                                                                    event
 119 speakers
         Andreas                   Alan                           Anna
         Schreiber                 Franzoni                       Ravenscroft
                                                                                           Topics
         @onyame                   @franzeur
                                                                                            Django
         Andrew                    Alessandro                     Anselm Kruis
         Godwin                    Dentella                                                 Plone
         @andrewgodwin                                                                      Pyramid
         Andrii                    Alex Martelli                  Antonio Cuni
                                                                  @antocuni
                                                                                            Python
         Mishkovskyi
         @mishok13                                                                          Twisted
                                   Ali Afshar                     Armin Rigo
         Armin                                                                                 Edit topics
Lanyrd.com
apache > lucene > solr




                                                                                                         Search the site with Solr   Search
  Main      Wiki                                                                                                 Powered by Lucid Imagination
                                                                                                  Last Published: Sat, 04 Jun 2011 12:23:42 GMT
   About
    Welcome
    Who We Are
                         Welcome to Solr
   Documentation
                                                                                                                                         PDF
   Resources
                             What Is Solr?
   Related Projects
                             Get Started
                             News
                                 May 2011 - Solr 3.2 Released
                                 March 2011 - Solr 3.1 Released
                                 25 June 2010 - Solr 1.4.1 Released
                                 7 May 2010 - Apache Lucene Eurocon 2010 Coming to Prague May 18-21
                                 10 November 2009 - Solr 1.4 Released
                                 20 August 2009 - Solr's first book is published!
                                 18 August 2009 - Lucene at US ApacheCon
                                 09 February 2009 - Lucene at ApacheCon Europe 2009 in Amsterdam
                                 19 December 2008 - Solr Logo Contest Results
                                 03 October 2008 - Solr Logo Contest
                                 15 September 2008 - Solr 1.3.0 Available
                                 28 August 2008 - Lucene/Solr at ApacheCon New Orleans
                                 03 September 2007 - Lucene at ApacheCon Atlanta
                                 06 June 2007: Release 1.2 available
                                 17 January 2007: Solr graduates from Incubator
                                 22 December 2006: Release 1.1.0 available
                                 15 August 2006: Solr at ApacheCon US
                                 21 April 2006: Solr at ApacheCon
Lanyrd.com
                                  add a conference    you are signed in as simonw, do you want to sign out?




                                           calendar       conferences           coverage              profile

                                                                                                     search




     Search
 We found 467 results for “django”

                                                                         FILTER BY
    django                                           Search              type
                                                                             Sessions         209

     Topic: Django                                                           Coverage          190
     ON NOW      1 conference
     FUTURE      5 conferences                                               Conferences             60
     PAST        51 conferences
                                                                             Books    5

     Topic: GeoDjango                                                        Topics    2
     PAST        1 conference
                                                                             People       1

     Practical Django Projects
     By James Bennett
     PUBLISHED   June 2009                                               FILTER BY
                                                                         past or future
     Django Geek                                                         Past events 55
     @djangeek
                                                                         Future events 5
     SLIDES
Lanyrd.com
                                                           add a conference   you are signed in as simonw, do you want to sign out?




                                                                    calendar      conferences           coverage         profile

                                                                                                                        search




 Your contacts' calendar
   yours   24   contacts   182

                                                                                                             Simon
  We've found 182 conferences your Twitter contacts are                                                      Willison
  interested in.                                                                                             Your profile
                                                                                                             page
  TODAY                     Café Scientifique: Exploring                          Attend
                     21
                            the dark side of star                                   Track
                            formation with the Herschel                                          From our blog
                            Space Observatory                                                    Welcoming Sophie
                               United Kingdom / Brighton                                         Barrett to team
                            21st June 2011                                                       Lanyrd
                             Astronomy    Science                                                Today we have a very special
                                                                                                 announcement (and for once,
                            4 contacts tracking
                                                                                                 it's not a new feature!) We
                                                                                                 would like to welcome the
                                                                                                 super-wonderful Sophie Barrett
                                                                                                 to the Lanyrd team.


                     21     Usability Professionals'                              Attend         Session schedules in
                            Association – International                             Track        your calendar
                            Conference                                                           You can now subscribe to event
                                                                                                 schedules in your calendar of
                                 United States / Atlanta
CSS    Maintainable CSS         Maintainability
     @Natbat on Twitter
                                                                                               1 write-up
                               Design Process       Internet Explorer      Debugging
Lanyrd.com                                                                                     Added 7 months ago


  16
          CONFERENCES
     spoken at                 Show and tell      Personal Projects                            2 links
                                                                                               Most recent added 7


  14
          CONFERENCES          Web Standards        Crowdsourcing                              months ago
          involved with
                                                                                               SEE ALL COVERAGE
                               WildlifeNearYou       RSI     Community
                                                                                               11 items in total
                               Event Planning       Geek Night      User Experience
  Browse by year               Usability    Usability Testing       JavaScript
                                                                                               FAVOURITE COVERAGE
                                                                                               See 6 items that
                                                                                               Natalie Downe has
  2011 ! 2010 ! 2009 !         Progressive Enhancement                                         favourited
  2008 ! 2007 ! 2006 !
  2005                         Unobtrusive JavaScript          Flexible design

                               Planning Artifacts      Social Software                     Book credits
  Appears with                                                                                 TECHNICAL REVIEWER
                               Motivation     CSS3     CSS Layout
                                                                                               The Ultimate CSS
      Simon Willison                                                                           Reference
                               Project Management            Dev Fort
      9 times                                                                                  TECHNICAL EDITOR
                               Rapid Application Development                                   CSS Mastery
      Nigel Crawley
      5 times

      Reinier Zwitserloot         Full speaking history for Natalie Downe
      4 times
                              PAST CONFERENCES                FUTURE CONFERENCES
      Neil Crosby   4 times   spoken at                       speaking at
      Cristiano Betta         Five Pound App #22              London Insites Tour
      4 times
                                  Brighton, United Kingdom    Evening
      Matt Westcott           30th March 2010                     London, United Kingdom
      4 times                                                 19th July 2011
                              CSS3 Wizardry
      Gareth Rushgrove        Workshop
      4 times                                                 FUTURE CONFERENCES
                                  Brighton, United Kingdom
                              29th January 2010               attending
Trick #4: Hashed static asset
   filenames in CloudFront
Lanyrd.com




cdn.lanyrd.net/js/global.ed81d119.js

global.js

global.ed81d119.js
Lanyrd.com



Benefits
 Far futures expiry headers
 Cache-Control: max-age=315360000
 Expires: Fri, 18 Jun 2021 06:45:00 -0000 GMT

 Guaranteed updated CSS in IE
 Deploy new assets in advance of
 application
 Old versions stick around for rollbacks
Lanyrd.com



./manage.py push_static
 Minifies JavaScript and CSS
 Renames files to include sha1(contents)[:8]
 Pushes all assets to S3
Part 2: Challenges
Challenge #1: HTTP requests
Lanyrd.com



Talking to an API?
 What if it fails to load?
 What if it takes 30 seconds to load?
Lanyrd.com



A user gives you a URL...
urllib.urlopen(url).read() ?
 what if it's a huge file?
 what if it's a slow loading tarpit?
 what if it's the URL to your private,
 firewalled Solr instance?
 http://10.0.1.1/solr/select/?q=secrets
Lanyrd.com



Safe URL consumption
 Connection timeouts
 Logging and Profiling
 Host validation
 HTTP caching / if-none-match / etc
 django.httpclient ?
Challenge #2: Profiling and
debugging production systems
Lanyrd.com


Debugging in
development rocks!
 django_debug_toolbar = awesome
 assert False = instant useful 500 page
 import pdb; pdb.set_trace() adds a
 command line debugger to your running
 application!
Lanyrd.com




DEBUG = False
DEBUG = "BLIND"
Lanyrd.com




from django.views.debug import technical_500_response
import sys

class UserBasedExceptionMiddleware(object):
    def process_exception(self, request, exception):
        if request.user.is_superuser:
            return technical_500_response(request, *sys.exc_info())
Lanyrd.com



mysql-proxy
 Very handy lua-customisable proxy for all
 of your MySQL traffic
 Worst documented software ever
 log.lua - logs out ALL queries
 https://gist.github.com/1039751
Lanyrd.com



django_instrumented
 (Unreleased) code I wrote for Lanyrd
 Collects various runtime stats about the
 current request, stashes a profile JSON in
 memcached
 Writes out the profile UUID as part of the
 HTML
 A bookmarklet to view the profile
Lanyrd.com
Lanyrd.com



Django improvements
 DEBUG as a global setting is an anti-
 pattern
 More low-level hooks for measuring,
 well, pretty much everything
 Live profiling tools (as a third-party
 product)
MVC Mini Profiler




http://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
Challenge #3: Zero downtime deploys
Lanyrd.com




Challenge:
deploy upgrades,
including database upgrades,
without downtime
Lanyrd.com


Deploy DB changes
separately from code changes
 Make backwards compatible schema
 changes
 We deploy code to a none-web server
 just so we can run "./manage.py migrate"
 (you are using South, right?)
 Use symlinks for instant rollback to
 previous code
Lanyrd.com



Read-only mode
If you can flip your site in to read-only mode,
upgrades become a lot easier
 Flip to read only mode
 Replicate DB to a new machine
 Run migrations, then test
 Switch app servers to new database
 (On EC2, fire up an entire new web cluster)

Feature flags: finely grained version of this
Lanyrd.com




   The lesson we
   keep on learning
Lanyrd.com



Global settings are BAD
 Database settings => multi-db (1.2)
 Cache settings => multi cache backends (1.3)
 Haystack backends => multi backends (v2.0)
Lanyrd.com



Global settings are BAD
 Database settings => multi-db (1.2)
 Cache settings => multi cache backends (1.3)
 Haystack backends => multi backends (v2.0)


 DEBUG
 TIME_ZONE
 Middleware?
Lanyrd.com




Settings should
be modifiable
at run-time
Thank you!
http://lanyrd.com/sfwcq



[                                     [
We're hiring a developer/web operations
 person in London, come and talk to us!

Mais conteúdo relacionado

Destaque

Sample email submission
Sample email submissionSample email submission
Sample email submissionDavid Sommer
 
Putting Out Fires with Content Strategy (InfoDevDC meetup)
Putting Out Fires with Content Strategy (InfoDevDC meetup)Putting Out Fires with Content Strategy (InfoDevDC meetup)
Putting Out Fires with Content Strategy (InfoDevDC meetup)John Collins
 
Designing for Multiple Mobile Platforms
Designing for Multiple Mobile PlatformsDesigning for Multiple Mobile Platforms
Designing for Multiple Mobile PlatformsRobert Douglas
 
Bank Account Of Life
Bank Account Of LifeBank Account Of Life
Bank Account Of LifeNafass
 
Linguistic Potluck: Crowdsourcing localization with Rails
Linguistic Potluck: Crowdsourcing localization with RailsLinguistic Potluck: Crowdsourcing localization with Rails
Linguistic Potluck: Crowdsourcing localization with RailsHeatherRivers
 
Internationalization in Rails 2.2
Internationalization in Rails 2.2Internationalization in Rails 2.2
Internationalization in Rails 2.2Nicolas Jacobeus
 
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)John Collins
 
The ruby on rails i18n core api-Neeraj Kumar
The ruby on rails i18n core api-Neeraj KumarThe ruby on rails i18n core api-Neeraj Kumar
The ruby on rails i18n core api-Neeraj KumarThoughtWorks
 
Putting Out Fires with Content Strategy (STC Academic SIG)
Putting Out Fires with Content Strategy (STC Academic SIG)Putting Out Fires with Content Strategy (STC Academic SIG)
Putting Out Fires with Content Strategy (STC Academic SIG)John Collins
 
My Valentine Gift - YOU Decide
My Valentine Gift - YOU DecideMy Valentine Gift - YOU Decide
My Valentine Gift - YOU DecideSizzlynRose
 
Open Software Platforms for Mobile Digital Broadcasting
Open Software Platforms for Mobile Digital BroadcastingOpen Software Platforms for Mobile Digital Broadcasting
Open Software Platforms for Mobile Digital BroadcastingFrancois Lefebvre
 
Building Quality Experiences for Users in Any Language
Building Quality Experiences for Users in Any LanguageBuilding Quality Experiences for Users in Any Language
Building Quality Experiences for Users in Any LanguageJohn Collins
 
Sample of instructions
Sample of instructionsSample of instructions
Sample of instructionsDavid Sommer
 
Pycon 2012 Apache Cassandra
Pycon 2012 Apache CassandraPycon 2012 Apache Cassandra
Pycon 2012 Apache Cassandrajeremiahdjordan
 
Building a Localization Kit
Building a Localization KitBuilding a Localization Kit
Building a Localization KitLuigi Muzii
 
Rails I18n From The Trenches
Rails I18n From The TrenchesRails I18n From The Trenches
Rails I18n From The Trenchesclemensk
 

Destaque (19)

Sample email submission
Sample email submissionSample email submission
Sample email submission
 
Putting Out Fires with Content Strategy (InfoDevDC meetup)
Putting Out Fires with Content Strategy (InfoDevDC meetup)Putting Out Fires with Content Strategy (InfoDevDC meetup)
Putting Out Fires with Content Strategy (InfoDevDC meetup)
 
Designing for Multiple Mobile Platforms
Designing for Multiple Mobile PlatformsDesigning for Multiple Mobile Platforms
Designing for Multiple Mobile Platforms
 
Bank Account Of Life
Bank Account Of LifeBank Account Of Life
Bank Account Of Life
 
Linguistic Potluck: Crowdsourcing localization with Rails
Linguistic Potluck: Crowdsourcing localization with RailsLinguistic Potluck: Crowdsourcing localization with Rails
Linguistic Potluck: Crowdsourcing localization with Rails
 
Silmeyiniz
SilmeyinizSilmeyiniz
Silmeyiniz
 
Internationalization in Rails 2.2
Internationalization in Rails 2.2Internationalization in Rails 2.2
Internationalization in Rails 2.2
 
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)
Strategies for Friendly English and Successful Localization (InfoDevWorld 2014)
 
The ruby on rails i18n core api-Neeraj Kumar
The ruby on rails i18n core api-Neeraj KumarThe ruby on rails i18n core api-Neeraj Kumar
The ruby on rails i18n core api-Neeraj Kumar
 
Putting Out Fires with Content Strategy (STC Academic SIG)
Putting Out Fires with Content Strategy (STC Academic SIG)Putting Out Fires with Content Strategy (STC Academic SIG)
Putting Out Fires with Content Strategy (STC Academic SIG)
 
My Valentine Gift - YOU Decide
My Valentine Gift - YOU DecideMy Valentine Gift - YOU Decide
My Valentine Gift - YOU Decide
 
Open Software Platforms for Mobile Digital Broadcasting
Open Software Platforms for Mobile Digital BroadcastingOpen Software Platforms for Mobile Digital Broadcasting
Open Software Platforms for Mobile Digital Broadcasting
 
Building Quality Experiences for Users in Any Language
Building Quality Experiences for Users in Any LanguageBuilding Quality Experiences for Users in Any Language
Building Quality Experiences for Users in Any Language
 
Shrunken Head
 Shrunken Head  Shrunken Head
Shrunken Head
 
Sample of instructions
Sample of instructionsSample of instructions
Sample of instructions
 
Glossary
GlossaryGlossary
Glossary
 
Pycon 2012 Apache Cassandra
Pycon 2012 Apache CassandraPycon 2012 Apache Cassandra
Pycon 2012 Apache Cassandra
 
Building a Localization Kit
Building a Localization KitBuilding a Localization Kit
Building a Localization Kit
 
Rails I18n From The Trenches
Rails I18n From The TrenchesRails I18n From The Trenches
Rails I18n From The Trenches
 

Semelhante a Tricks & challenges developing a large Django application

Djenie way for developers
Djenie way for developersDjenie way for developers
Djenie way for developersdjenie
 
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017Codemotion
 
Jenkins Shared Libraries Workshop
Jenkins Shared Libraries WorkshopJenkins Shared Libraries Workshop
Jenkins Shared Libraries WorkshopJulien Pivotto
 
LingPy : A Python Library for Historical Linguistics
LingPy : A Python Library for Historical LinguisticsLingPy : A Python Library for Historical Linguistics
LingPy : A Python Library for Historical LinguisticsDr. Amit Kumar Jha
 
The Butler and the Snake - Continuous Integration for Python
The Butler and the Snake - Continuous Integration for PythonThe Butler and the Snake - Continuous Integration for Python
The Butler and the Snake - Continuous Integration for PythonTimo Stollenwerk
 
Scientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of dataScientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of dataGael Varoquaux
 
Intro to Pinax: Kickstarting Your Django Apps
Intro to Pinax: Kickstarting Your Django AppsIntro to Pinax: Kickstarting Your Django Apps
Intro to Pinax: Kickstarting Your Django AppsRoger Barnes
 
總之來講講 Search
總之來講講 Search總之來講講 Search
總之來講講 SearchAndy Dai
 
Software Heritage: Archiving the Free Software Commons for Fun & Profit
Software Heritage: Archiving the Free Software Commons for Fun & ProfitSoftware Heritage: Archiving the Free Software Commons for Fun & Profit
Software Heritage: Archiving the Free Software Commons for Fun & ProfitSpeck&Tech
 
Dependency Management with Composer
Dependency Management with ComposerDependency Management with Composer
Dependency Management with ComposerJordi Boggiano
 
What makes Python So Awesome
What makes Python So AwesomeWhat makes Python So Awesome
What makes Python So AwesomeVikash Agrawal
 
OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...
OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...
OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...eNovance
 

Semelhante a Tricks & challenges developing a large Django application (14)

Djenie way for developers
Djenie way for developersDjenie way for developers
Djenie way for developers
 
State Of Django
State Of DjangoState Of Django
State Of Django
 
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017
 
Jenkins Shared Libraries Workshop
Jenkins Shared Libraries WorkshopJenkins Shared Libraries Workshop
Jenkins Shared Libraries Workshop
 
LingPy : A Python Library for Historical Linguistics
LingPy : A Python Library for Historical LinguisticsLingPy : A Python Library for Historical Linguistics
LingPy : A Python Library for Historical Linguistics
 
The Butler and the Snake - Continuous Integration for Python
The Butler and the Snake - Continuous Integration for PythonThe Butler and the Snake - Continuous Integration for Python
The Butler and the Snake - Continuous Integration for Python
 
Scientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of dataScientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of data
 
Intro to Pinax: Kickstarting Your Django Apps
Intro to Pinax: Kickstarting Your Django AppsIntro to Pinax: Kickstarting Your Django Apps
Intro to Pinax: Kickstarting Your Django Apps
 
總之來講講 Search
總之來講講 Search總之來講講 Search
總之來講講 Search
 
Software Heritage: Archiving the Free Software Commons for Fun & Profit
Software Heritage: Archiving the Free Software Commons for Fun & ProfitSoftware Heritage: Archiving the Free Software Commons for Fun & Profit
Software Heritage: Archiving the Free Software Commons for Fun & Profit
 
Dependency Management with Composer
Dependency Management with ComposerDependency Management with Composer
Dependency Management with Composer
 
What makes Python So Awesome
What makes Python So AwesomeWhat makes Python So Awesome
What makes Python So Awesome
 
OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...
OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...
OpenStack in Action 4! Doug hellman - Intersection of OpenStack and python co...
 
Plone in news media
Plone in news mediaPlone in news media
Plone in news media
 

Mais de Simon Willison

Cheap tricks for startups
Cheap tricks for startupsCheap tricks for startups
Cheap tricks for startupsSimon Willison
 
The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)Simon Willison
 
How we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graphHow we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graphSimon Willison
 
Web Services for Fun and Profit
Web Services for Fun and ProfitWeb Services for Fun and Profit
Web Services for Fun and ProfitSimon Willison
 
Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & FabricAdvanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & FabricSimon Willison
 
How Lanyrd uses Twitter
How Lanyrd uses TwitterHow Lanyrd uses Twitter
How Lanyrd uses TwitterSimon Willison
 
Building Things Fast - and getting approval
Building Things Fast - and getting approvalBuilding Things Fast - and getting approval
Building Things Fast - and getting approvalSimon Willison
 
Rediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The LibrariesRediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The LibrariesSimon Willison
 
Building crowdsourcing applications
Building crowdsourcing applicationsBuilding crowdsourcing applications
Building crowdsourcing applicationsSimon Willison
 
Evented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunniesEvented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunniesSimon Willison
 
Cowboy development with Django
Cowboy development with DjangoCowboy development with Django
Cowboy development with DjangoSimon Willison
 
Crowdsourcing with Django
Crowdsourcing with DjangoCrowdsourcing with Django
Crowdsourcing with DjangoSimon Willison
 
Class-based views with Django
Class-based views with DjangoClass-based views with Django
Class-based views with DjangoSimon Willison
 
Web App Security Horror Stories
Web App Security Horror StoriesWeb App Security Horror Stories
Web App Security Horror StoriesSimon Willison
 
Web Security Horror Stories
Web Security Horror StoriesWeb Security Horror Stories
Web Security Horror StoriesSimon Willison
 
When Zeppelins Ruled The Earth
When Zeppelins Ruled The EarthWhen Zeppelins Ruled The Earth
When Zeppelins Ruled The EarthSimon Willison
 

Mais de Simon Willison (20)

How Lanyrd does Geo
How Lanyrd does GeoHow Lanyrd does Geo
How Lanyrd does Geo
 
Cheap tricks for startups
Cheap tricks for startupsCheap tricks for startups
Cheap tricks for startups
 
The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)The Django Web Framework (EuroPython 2006)
The Django Web Framework (EuroPython 2006)
 
Building Lanyrd
Building LanyrdBuilding Lanyrd
Building Lanyrd
 
How we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graphHow we bootstrapped Lanyrd using Twitter's social graph
How we bootstrapped Lanyrd using Twitter's social graph
 
Web Services for Fun and Profit
Web Services for Fun and ProfitWeb Services for Fun and Profit
Web Services for Fun and Profit
 
Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & FabricAdvanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
Advanced Aspects of the Django Ecosystem: Haystack, Celery & Fabric
 
How Lanyrd uses Twitter
How Lanyrd uses TwitterHow Lanyrd uses Twitter
How Lanyrd uses Twitter
 
ScaleFail
ScaleFailScaleFail
ScaleFail
 
Building Things Fast - and getting approval
Building Things Fast - and getting approvalBuilding Things Fast - and getting approval
Building Things Fast - and getting approval
 
Rediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The LibrariesRediscovering JavaScript: The Language Behind The Libraries
Rediscovering JavaScript: The Language Behind The Libraries
 
Building crowdsourcing applications
Building crowdsourcing applicationsBuilding crowdsourcing applications
Building crowdsourcing applications
 
Evented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunniesEvented I/O based web servers, explained using bunnies
Evented I/O based web servers, explained using bunnies
 
Cowboy development with Django
Cowboy development with DjangoCowboy development with Django
Cowboy development with Django
 
Crowdsourcing with Django
Crowdsourcing with DjangoCrowdsourcing with Django
Crowdsourcing with Django
 
Django Heresies
Django HeresiesDjango Heresies
Django Heresies
 
Class-based views with Django
Class-based views with DjangoClass-based views with Django
Class-based views with Django
 
Web App Security Horror Stories
Web App Security Horror StoriesWeb App Security Horror Stories
Web App Security Horror Stories
 
Web Security Horror Stories
Web Security Horror StoriesWeb Security Horror Stories
Web Security Horror Stories
 
When Zeppelins Ruled The Earth
When Zeppelins Ruled The EarthWhen Zeppelins Ruled The Earth
When Zeppelins Ruled The Earth
 

Último

ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 

Último (20)

ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 

Tricks & challenges developing a large Django application

  • 1. Tricks & challenges developing a large Django application Simon Willison EuroPython 2011, 22nd June http://lanyrd.com/sfwcq
  • 3. Lanyrd.com Definitive database of Professional Events and Speakers
  • 4. Lanyrd.com Definitive database Social event recommendation of Professional Events Event networking tools and Speakers Archive of slides, notes and video
  • 7. Lanyrd.com Pass data through an untrusted source with confidence that it hasn't been tampered with
  • 8. Lanyrd.com Signing uses Unsubscribe" links in emails lanyrd.com/un/ImN6VyI.ii0Hwm7p71DEcGfaVzziQaxeuu Signed cookies "You are logged in as simonw" without hitting the database
  • 9. Lanyrd.com Signing in Django 1.4 from django.core import signing signing.dumps({"foo": "bar"}) signing.loads(signed_string) response.set_signed_cookie(key, value...) request.get_signed_cookie(key)
  • 11. add a conference you are signed in as simonw, do you want to sign out? calendar conferences coverage profile search Django conferences Django Django events looking for participants coverage 1 Django event is looking for participants 52 videos Most recent added 3 weeks ago ON NOW EuroPython 2011 52 slide decks 19 Most recent added 4 Italy / Florence hours ago 19th–26th June 2011 Django Plone Pyramid Python Twisted 3 audio clips Most recent added 1 week ago 27 write-ups SEPTEMBER DjangoCon US 2011 6 Most recent added 1 2011 United States / Portland 6th–8th September 2011 week ago 11 handouts Django Open Source Python Most recent added 18 hours ago 17 PyCON FR 2011 3 notes France / Rennes Most recent added 10 17th–18th September 2011 hours ago Django Python By country OCTOBER PyCon DE 2011 Ireland 1 4
  • 12. Lanyrd.com class Conference(models.Model): ... cache_version = models.IntegerField(default = 0) def save(self, *args, **kwargs): self.cache_version += 1 super(Conference, self).save(*args, **kwargs) def touch(self): Conference.objects.filter( pk = self.pk ).update( cache_version = F('cache_version') + 1 )
  • 13. Lanyrd.com {% cache 36000 conf-topics conference.pk conference.cache_version %} <ul class="tags inline-tags meta"> {% for topic in conference.topics.all %} <li><a href="{{ topic.get_absolute_url }}">{{ topic }}</a></li> {% endfor %} </ul> {% endcache %}
  • 14. Lanyrd.com Bulk invalidation from django.models import F topic.conferences.all().update( cache_version = F('cache_version') + 1 )
  • 15. Trick #3: NoSQL for denormalisation
  • 17. Lanyrd.com But we like joins! Slides from a session on Lanyrd show up... Under "coverage of this conference" Under "slides by Simon Willison" Under "slides about Django"
  • 19. Lanyrd.com Commands Clients Documentation Community Download Issues Redis is an open source, advanced key-value store. It is often What people are saying referred to as a data structure server since keys can contain Comparison of CouchDB, Redis, MongoDB, Casandra, Neo4J & strings, hashes, lists, sets and sorted sets. strings hashes lists sets others http://j.mp/l32SqM via @DZone Learn more → @__NeverGiveup Oh YAY, oui tu me redis ! *-* Hm, on s'rejoint à Try it Download it 14h au bahut ? :o Ready for a test drive? Check this interactive Redis 2.2.10 is the latest stable version. JE L REDIS JE FOLLOW BACK SUR @Fuckement_TL tutorial that will walk you through the most Interested in legacy or unstable versions? important features of Redis. Check the downloads page. une question : "How to use ServiceStack Redis in a web application to take advantage of pub / sub paradigm" http://t.co/EOgyLU1 #redis #web Nice - Cassandra vs MongoDB vs CouchDB vs Redis vs Riak vs HBase vs Membase vs Neo4j comparison http://bit.ly/l32SqM from @kkovacs More... Sponsored by This website is open source software developed by Citrusbyte. The Redis logo was designed by Carlos Prioglio.
  • 20. Lanyrd.com Our redis sets... simonw-follows:{144,21345,12328...} europython-attendees:{344,21345,787...} redis.sinter( 'simonw-follows', 'europython-attendees' )
  • 21. add a conference you are signed in as simonw, do you want to sign out? Lanyrd.com calendar conferences coverage profile search EuroPython 2011 You're speaking The European Python Conference AT THIS EVENT 19 –26 JUNE 2011 Florence in Italy 97 attending http://ep2011.europython.eu/ @europython PEOPLE View the schedule on Lanyrd #europython 80 tracking PEOPLE Save to iCal / iPhone / Outlook / lanyrd.com/ccdpc (short URL) GCal TELL YOUR FRIENDS! Tweet about this event 119 speakers Andreas Alan Anna Schreiber Franzoni Ravenscroft Topics @onyame @franzeur Django Andrew Alessandro Anselm Kruis Godwin Dentella Plone @andrewgodwin Pyramid Andrii Alex Martelli Antonio Cuni @antocuni Python Mishkovskyi @mishok13 Twisted Ali Afshar Armin Rigo Armin Edit topics
  • 22. Lanyrd.com apache > lucene > solr Search the site with Solr Search Main Wiki Powered by Lucid Imagination Last Published: Sat, 04 Jun 2011 12:23:42 GMT About Welcome Who We Are Welcome to Solr Documentation PDF Resources What Is Solr? Related Projects Get Started News May 2011 - Solr 3.2 Released March 2011 - Solr 3.1 Released 25 June 2010 - Solr 1.4.1 Released 7 May 2010 - Apache Lucene Eurocon 2010 Coming to Prague May 18-21 10 November 2009 - Solr 1.4 Released 20 August 2009 - Solr's first book is published! 18 August 2009 - Lucene at US ApacheCon 09 February 2009 - Lucene at ApacheCon Europe 2009 in Amsterdam 19 December 2008 - Solr Logo Contest Results 03 October 2008 - Solr Logo Contest 15 September 2008 - Solr 1.3.0 Available 28 August 2008 - Lucene/Solr at ApacheCon New Orleans 03 September 2007 - Lucene at ApacheCon Atlanta 06 June 2007: Release 1.2 available 17 January 2007: Solr graduates from Incubator 22 December 2006: Release 1.1.0 available 15 August 2006: Solr at ApacheCon US 21 April 2006: Solr at ApacheCon
  • 23. Lanyrd.com add a conference you are signed in as simonw, do you want to sign out? calendar conferences coverage profile search Search We found 467 results for “django” FILTER BY django Search type Sessions 209 Topic: Django Coverage 190 ON NOW 1 conference FUTURE 5 conferences Conferences 60 PAST 51 conferences Books 5 Topic: GeoDjango Topics 2 PAST 1 conference People 1 Practical Django Projects By James Bennett PUBLISHED June 2009 FILTER BY past or future Django Geek Past events 55 @djangeek Future events 5 SLIDES
  • 24. Lanyrd.com add a conference you are signed in as simonw, do you want to sign out? calendar conferences coverage profile search Your contacts' calendar yours 24 contacts 182 Simon We've found 182 conferences your Twitter contacts are Willison interested in. Your profile page TODAY Café Scientifique: Exploring Attend 21 the dark side of star Track formation with the Herschel From our blog Space Observatory Welcoming Sophie United Kingdom / Brighton Barrett to team 21st June 2011 Lanyrd Astronomy Science Today we have a very special announcement (and for once, 4 contacts tracking it's not a new feature!) We would like to welcome the super-wonderful Sophie Barrett to the Lanyrd team. 21 Usability Professionals' Attend Session schedules in Association – International Track your calendar Conference You can now subscribe to event schedules in your calendar of United States / Atlanta
  • 25. CSS Maintainable CSS Maintainability @Natbat on Twitter 1 write-up Design Process Internet Explorer Debugging Lanyrd.com Added 7 months ago 16 CONFERENCES spoken at Show and tell Personal Projects 2 links Most recent added 7 14 CONFERENCES Web Standards Crowdsourcing months ago involved with SEE ALL COVERAGE WildlifeNearYou RSI Community 11 items in total Event Planning Geek Night User Experience Browse by year Usability Usability Testing JavaScript FAVOURITE COVERAGE See 6 items that Natalie Downe has 2011 ! 2010 ! 2009 ! Progressive Enhancement favourited 2008 ! 2007 ! 2006 ! 2005 Unobtrusive JavaScript Flexible design Planning Artifacts Social Software Book credits Appears with TECHNICAL REVIEWER Motivation CSS3 CSS Layout The Ultimate CSS Simon Willison Reference Project Management Dev Fort 9 times TECHNICAL EDITOR Rapid Application Development CSS Mastery Nigel Crawley 5 times Reinier Zwitserloot Full speaking history for Natalie Downe 4 times PAST CONFERENCES FUTURE CONFERENCES Neil Crosby 4 times spoken at speaking at Cristiano Betta Five Pound App #22 London Insites Tour 4 times Brighton, United Kingdom Evening Matt Westcott 30th March 2010 London, United Kingdom 4 times 19th July 2011 CSS3 Wizardry Gareth Rushgrove Workshop 4 times FUTURE CONFERENCES Brighton, United Kingdom 29th January 2010 attending
  • 26. Trick #4: Hashed static asset filenames in CloudFront
  • 28. Lanyrd.com Benefits Far futures expiry headers Cache-Control: max-age=315360000 Expires: Fri, 18 Jun 2021 06:45:00 -0000 GMT Guaranteed updated CSS in IE Deploy new assets in advance of application Old versions stick around for rollbacks
  • 29. Lanyrd.com ./manage.py push_static Minifies JavaScript and CSS Renames files to include sha1(contents)[:8] Pushes all assets to S3
  • 31. Challenge #1: HTTP requests
  • 32. Lanyrd.com Talking to an API? What if it fails to load? What if it takes 30 seconds to load?
  • 33. Lanyrd.com A user gives you a URL... urllib.urlopen(url).read() ? what if it's a huge file? what if it's a slow loading tarpit? what if it's the URL to your private, firewalled Solr instance? http://10.0.1.1/solr/select/?q=secrets
  • 34. Lanyrd.com Safe URL consumption Connection timeouts Logging and Profiling Host validation HTTP caching / if-none-match / etc django.httpclient ?
  • 35. Challenge #2: Profiling and debugging production systems
  • 36. Lanyrd.com Debugging in development rocks! django_debug_toolbar = awesome assert False = instant useful 500 page import pdb; pdb.set_trace() adds a command line debugger to your running application!
  • 38. Lanyrd.com from django.views.debug import technical_500_response import sys class UserBasedExceptionMiddleware(object): def process_exception(self, request, exception): if request.user.is_superuser: return technical_500_response(request, *sys.exc_info())
  • 39. Lanyrd.com mysql-proxy Very handy lua-customisable proxy for all of your MySQL traffic Worst documented software ever log.lua - logs out ALL queries https://gist.github.com/1039751
  • 40. Lanyrd.com django_instrumented (Unreleased) code I wrote for Lanyrd Collects various runtime stats about the current request, stashes a profile JSON in memcached Writes out the profile UUID as part of the HTML A bookmarklet to view the profile
  • 42. Lanyrd.com Django improvements DEBUG as a global setting is an anti- pattern More low-level hooks for measuring, well, pretty much everything Live profiling tools (as a third-party product)
  • 44. Challenge #3: Zero downtime deploys
  • 46. Lanyrd.com Deploy DB changes separately from code changes Make backwards compatible schema changes We deploy code to a none-web server just so we can run "./manage.py migrate" (you are using South, right?) Use symlinks for instant rollback to previous code
  • 47. Lanyrd.com Read-only mode If you can flip your site in to read-only mode, upgrades become a lot easier Flip to read only mode Replicate DB to a new machine Run migrations, then test Switch app servers to new database (On EC2, fire up an entire new web cluster) Feature flags: finely grained version of this
  • 48. Lanyrd.com The lesson we keep on learning
  • 49. Lanyrd.com Global settings are BAD Database settings => multi-db (1.2) Cache settings => multi cache backends (1.3) Haystack backends => multi backends (v2.0)
  • 50. Lanyrd.com Global settings are BAD Database settings => multi-db (1.2) Cache settings => multi cache backends (1.3) Haystack backends => multi backends (v2.0) DEBUG TIME_ZONE Middleware?
  • 52. Thank you! http://lanyrd.com/sfwcq [ [ We're hiring a developer/web operations person in London, come and talk to us!