2. Trainer Andreas Jung Python developersince 1993 Python, Zope & Plonedevelopment Specialized in Electronic Publishing DirectoroftheZopeFoundation Authorofdozensadd-onsfor Python, ZopeandPlone Co-Founderofthe German Zope User Group (DZUG) Member ofthePloneFoundation usingMongoDBsince 2009
7. Let‘sagree on thefollowingorleave... MongoDBis cool MongoDBis not the multi-purpose-one-size-fits-all database MongoDBisanotheradditionaltoolforthesoftwaredeveloper MongoDBis not a replacementfor RDBMS in general Usetherighttoolforeachtask
9. Oh, SQL – let‘shavesomefunfirst A SQL statementwalksinto a bar andseestwotables. He walksandsays: „Hello, may I joinyou“ A SQL injectionwalksinto a bar andstartstoquotesomething but suddenlystops, drops a tableanddashes out.
10. The historyofMongoDB 10gen founded in 2007 Startedascloud-alternative GAE App-engineed Database p Javascriptasimplementationlanguage 2008: focusing on thedatabasepart: MongoDB 2009: firstMongoDBrelease 2011: MongoDB 1.8: Major deployments A fast growingcommunity Fast adoptationfor large projects 10gen growing
19. Durability Default: fire-and-forget (usesafe-mode) Changesarekept in RAM (!) Fsynctodiskevery 60 seconds (default) Deploymentoptions: Standaloneinstallation: usejournaling (V 1.8+) Replicated: usereplicasets(s)
20. Differences from Typical RDBMS Memory mapped data All data in memory (if it fits), synced to disk periodically No joins Reads have greater data locality No joins between servers No transactions Improves performance of various operations No transactions between servers
21. Replica Sets Cluster of N servers Only one node is ‘primary’ at a time This is equivalent to master The node where writes go Primary is elected by concensus Automatic failover Automatic recovery of failed nodes
22. Replica Sets - Writes A write is only ‘committed’ once it has been replicated to a majority of nodes in the set Before this happens, reads to the set may or may not see the write On failover, data which is not ‘committed’ may be dropped (but not necessarily) If dropped, it will be rolled back from all servers which wrote it For improved durability, use getLastError/w Other criteria – block writes when nodes go down or slaves get too far behind Or, to reduce latency, reduce getLastError/w
23. Replica Sets - Nodes Nodes monitor each other’s heartbeats If primary can’t see a majority of nodes, it relinquishes primary status If a majority of nodes notice there is no primary, they elect a primary using criteria Node priority Node data’s freshness
25. Replica Sets - Nodes {a:1} Member 1 SECONDARY {a:1} {b:2} Member 2 SECONDARY {a:1} {b:2} {c:3} Member 3 PRIMARY
26. Replica Sets - Nodes {a:1} Member 1 SECONDARY {a:1} {b:2} Member 2 PRIMARY {a:1} {b:2} {c:3} Member 3 DOWN
27. Replica Sets - Nodes {a:1} {b:2} Member 1 SECONDARY {a:1} {b:2} Member 2 PRIMARY {a:1} {b:2} {c:3} Member 3 RECOVERING
28. Replica Sets - Nodes {a:1} {b:2} Member 1 SECONDARY {a:1} {b:2} Member 2 PRIMARY {a:1} {b:2} Member 3 SECONDARY
29. Replica Sets – Node Types Standard – can be primary or secondary Passive – will be secondary but never primary Arbiter – will vote on primary, but won’t replicate data
32. Shard A replica set Manages a well defined range of shard keys
33. Shard Distribute data across machines Reduce data per machine Better able to fit in RAM Distribute write load across shards Distribute read load across shards, and across nodes within shards
34. Shard Key { user_id: 1 } { lastname: 1, firstname: 1 } { tag: 1, timestamp: -1 } { _id: 1 } This is the default
36. Differences from Typical RDBMS Memory mapped data All data in memory (if it fits), synced to disk periodically No joins Reads have greater data locality No joins between servers No transactions Improves performance of various operations No transactions between servers A weak authentication and authorization model
37. Part 2/4 UsingMongoDB StartingMongoDB Usingtheinteractive Mongo console Basic databaseoperations
62. MongoAlchemy (1/2) MongoAlchemyis a layer on top ofthe Python MongoDBdriverwhichadds client-sideschemadefinitions, an easiertoworkwithandprogrammaticquerylanguage, and a Document-Objectmapperwhichallowspythonobjectstobesavedandloadedintothedatabase in a type-safe way. An explicit goalofthisprojectistobeabletoperformasmanyoperationsaspossiblewithouthavingtoperform a load/save cyclesincedoing so isbothsignificantlyslowerandmorelikelytocausedataloss. http://mongoalchemy.org/
63. MongoAlchemy(2/2) frommongoalchemy.documentimportDocument, DocumentField frommongoalchemy.fieldsimport * fromdatetimeimportdatetime frompprintimportpprint class Event(Document): name = StringField() children = ListField(DocumentField('Event')) begin = DateTimeField() end = DateTimeField() def __init__(self, name, parent=None): Document.__init__(self, name=name) self.children = [] ifparent != None: parent.children.append(self)