SlideShare uma empresa Scribd logo
1 de 29
Mongo Admin’ing
             Jesse Wolgamott
twitter: @jwo slides: http://github.com/jwo
Quick Install

• Dev: brew install mongodb
• Server: apt-get install mongodb
  • http://www.mongodb.org/display/
    DOCS/Ubuntu+and+Debian+packages
Don’t Fear the
Command Line
   Mongoid / Mongo Mapper


         Ruby Driver


Command Line Javascript (mongo)
Quick Server Status

db
db.system.namespaces.find()
db.serverStatus()
db.currentOp()
Command Line Basics
use dba
db.collection.find({json params})
db.collection.findOne({json params})
db.collection.mapReduce(map,reduce)
db.collection.save({json})
Why Bother?
• Mongo queries enhance Mongoid queries
• Drop to command line when needed
• Game.where(:outcome=>{‘$in’=>[‘winning’
  ,‘tie’]})
• Game.collection.update({:outcome=>‘win
  ning’, ‘$set’=>{:winning=>true}})
All Admin is
     Command Line

• Setup Replica Sets
• Check on Status
Replica / Slave /
       Journaling

“If you aren’t running a replica set, you don’t
            care about your data”
                            -- Mongo DB dude
Master / Slave
  What You’d Expect


           M


  Slave   Slave   Slave
Master / Slave
  What You’d Expect


           M              Writes Go Here



  Slave   Slave   Slave
Master / Slave
  What You’d Expect


            M              Writes Go Here



  Slave    Slave   Slave


          Reads CAN Go
              Here
Replica Sets
    The new Hotness (and Recommended)


Primary   Secondary   Secondary   Secondary



                                   Arbiter
Replica Sets
       The new Hotness (and Recommended)


   Primary   Secondary   Secondary   Secondary



Writes Go Here                        Arbiter
Replica Sets
       The new Hotness (and Recommended)


   Primary   Secondary   Secondary   Secondary



Writes Go Here      CAN Read          Arbiter
Replica Sets
       The new Hotness (and Recommended)


   Primary   Secondary   Secondary   Secondary



Writes Go Here      CAN Read          Arbiter


                               Nothing Goes
                                  Here
Arbiter?

• Arbiters exist when even number of hosts
• They cast votes on primary elections
• Can exist on your app server
Journaling

• Added in 1.8
• mongod -- journal
• Will write out changes to a journal
  directory
• You can restore from journal
Replica Set Fun
config = {
   _id: 'houstonrb',
   members: [
     {_id: 0, host: 'localhost:20000'},
     {_id: 1, host: 'localhost:20001'},
     {_id: 2, host: 'localhost:20002', arbiterOnly: true}]
}

// Start it up
rs.initiate(config);

// Check it Out
rs.status()

// Who’s your daddy?
rs.isMaster()
Non-Master Reads
error: { "$err" : "not master", "code" : 10107 }

               OK, so we’ll do:

         db.getMongo().setSlaveOk()

                 // Awesome
Backup / Restore
mongodump --db houstonrb

mongorestore --db houstonrb -c
collections --drop --indexesLast
DirectoryName

* Have all your .bson files in a
directory named your database
Repairing

• The Data Gobbler -- mongodb likes hard-
  drive space and won’t give it back

• Use mongod    --repair to reclaim your
  space!
Nightly Plan
service mongodb stop && mongod --repair --dbpath /var/
lib/mongodb && service mongodb start

rm -f /data/mongodump-backup.tar.bz2 && mongodump -
o /data/backups && tar -jcf /data/mongodump-
backup.tar.bz2 /data/backups && rm -rf /data/backups/
Catch Up!
• When a Replica Set gets ‘stale’
• rs.status will remain “5” (Recovering)
• On Master, will say “Error RS102 too stale
  to catch up”
• stop mongo on all servers, copy over all
  data files, including local.*, start all
Table Scans == Devil
• When you get to 500,000 records, your
  DB may slow to crawl.
• /var/log/mongodb/mongodb.log for logged
  queries (table scans)

• db.collection.getIndexKeys()
• Mongoid: rake
  db:mongoid:create_indexes
No, Really

• I’ve been surprised LOTS at how often a
  table scan happens.
• Test it out: mongod --notablescan
• Stuff will break.
Sharding

• Don’t Shard Early
• Helps in map-reduce
• You define a “shard-key”, like lastname
  from A-H on this shard
Scaling (Secret Sauce)

• Many Shard Masters, even more slaves
• Add Shards for write and data size scaling
• Add Slaves for read scaling (map-reduce)
  and redundancy
• Sharding is Hard
Neat Snippets

            Replication Setup
             Sharding Setup

https://github.com/mongodb/mongo-snippets
EC2 Notes

• Reformat drive to EXT4 or XFS
• Don’t use EBS for long term backups
• Can use EBS for setting up replica sets

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

#OktoCampus - Workshop : An introduction to Ansible
#OktoCampus - Workshop : An introduction to Ansible#OktoCampus - Workshop : An introduction to Ansible
#OktoCampus - Workshop : An introduction to Ansible
 
Ansible, Simplicity, and the Zen of Python
Ansible, Simplicity, and the Zen of PythonAnsible, Simplicity, and the Zen of Python
Ansible, Simplicity, and the Zen of Python
 
Linux basic3
Linux basic3Linux basic3
Linux basic3
 
puppet @techlifecookpad
puppet @techlifecookpadpuppet @techlifecookpad
puppet @techlifecookpad
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
 
Microarmy - by J2 Labs
Microarmy - by J2 LabsMicroarmy - by J2 Labs
Microarmy - by J2 Labs
 
Ansible - Swiss Army Knife Orchestration
Ansible - Swiss Army Knife OrchestrationAnsible - Swiss Army Knife Orchestration
Ansible - Swiss Army Knife Orchestration
 
Introducing Ansible
Introducing AnsibleIntroducing Ansible
Introducing Ansible
 
Introduction to Ansible
Introduction to AnsibleIntroduction to Ansible
Introduction to Ansible
 
Jenkins and ansible reference
Jenkins and ansible referenceJenkins and ansible reference
Jenkins and ansible reference
 
Ansible for beginners ...?
Ansible for beginners ...?Ansible for beginners ...?
Ansible for beginners ...?
 
Automating everything with PowerShell, Terraform, and AWS
Automating everything with PowerShell, Terraform, and AWSAutomating everything with PowerShell, Terraform, and AWS
Automating everything with PowerShell, Terraform, and AWS
 
Basics of Ansible - Sahil Davawala
Basics of Ansible - Sahil DavawalaBasics of Ansible - Sahil Davawala
Basics of Ansible - Sahil Davawala
 
Building Windows Images with Packer
Building Windows Images with PackerBuilding Windows Images with Packer
Building Windows Images with Packer
 
Ansible intro
Ansible introAnsible intro
Ansible intro
 
Monitoring and tuning your chef server - chef conf talk
Monitoring and tuning your chef server - chef conf talk Monitoring and tuning your chef server - chef conf talk
Monitoring and tuning your chef server - chef conf talk
 
Redis persistence in practice
Redis persistence in practiceRedis persistence in practice
Redis persistence in practice
 
Apache HBase - Lab Assignment
Apache HBase - Lab AssignmentApache HBase - Lab Assignment
Apache HBase - Lab Assignment
 
Ansible Crash Course
Ansible Crash CourseAnsible Crash Course
Ansible Crash Course
 
Automation with ansible
Automation with ansibleAutomation with ansible
Automation with ansible
 

Semelhante a Mongo admin'ing

MongoDB Best Practices in AWS
MongoDB Best Practices in AWS MongoDB Best Practices in AWS
MongoDB Best Practices in AWS
Chris Harris
 
MongoDB and AWS Best Practices
MongoDB and AWS Best PracticesMongoDB and AWS Best Practices
MongoDB and AWS Best Practices
MongoDB
 

Semelhante a Mongo admin'ing (20)

Replication, Durability, and Disaster Recovery
Replication, Durability, and Disaster RecoveryReplication, Durability, and Disaster Recovery
Replication, Durability, and Disaster Recovery
 
MongoDB Best Practices in AWS
MongoDB Best Practices in AWS MongoDB Best Practices in AWS
MongoDB Best Practices in AWS
 
Mongodb replication
Mongodb replicationMongodb replication
Mongodb replication
 
Functional Hostnames and Why they are Bad
Functional Hostnames and Why they are BadFunctional Hostnames and Why they are Bad
Functional Hostnames and Why they are Bad
 
Replica Sets (NYC NoSQL Meetup)
Replica Sets (NYC NoSQL Meetup)Replica Sets (NYC NoSQL Meetup)
Replica Sets (NYC NoSQL Meetup)
 
MongoDB Replica Sets
MongoDB Replica SetsMongoDB Replica Sets
MongoDB Replica Sets
 
Thoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency ModelsThoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency Models
 
Scaling with MongoDB
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDB
 
Mongodb workshop
Mongodb workshopMongodb workshop
Mongodb workshop
 
MongoDB and AWS Best Practices
MongoDB and AWS Best PracticesMongoDB and AWS Best Practices
MongoDB and AWS Best Practices
 
NoSQL Infrastructure - Late 2013
NoSQL Infrastructure - Late 2013NoSQL Infrastructure - Late 2013
NoSQL Infrastructure - Late 2013
 
Python mongo db-training-europython-2011
Python mongo db-training-europython-2011Python mongo db-training-europython-2011
Python mongo db-training-europython-2011
 
MongoDB: Optimising for Performance, Scale & Analytics
MongoDB: Optimising for Performance, Scale & AnalyticsMongoDB: Optimising for Performance, Scale & Analytics
MongoDB: Optimising for Performance, Scale & Analytics
 
Mongo db japan
Mongo db japanMongo db japan
Mongo db japan
 
OSDC 2012 | Scaling with MongoDB by Ross Lawley
OSDC 2012 | Scaling with MongoDB by Ross LawleyOSDC 2012 | Scaling with MongoDB by Ross Lawley
OSDC 2012 | Scaling with MongoDB by Ross Lawley
 
Postgres Vienna DB Meetup 2014
Postgres Vienna DB Meetup 2014Postgres Vienna DB Meetup 2014
Postgres Vienna DB Meetup 2014
 
MapReduce with Hadoop and Ruby
MapReduce with Hadoop and RubyMapReduce with Hadoop and Ruby
MapReduce with Hadoop and Ruby
 
Modeling Tricks My Relational Database Never Taught Me
Modeling Tricks My Relational Database Never Taught MeModeling Tricks My Relational Database Never Taught Me
Modeling Tricks My Relational Database Never Taught Me
 
Free The Enterprise With Ruby & Master Your Own Domain
Free The Enterprise With Ruby & Master Your Own DomainFree The Enterprise With Ruby & Master Your Own Domain
Free The Enterprise With Ruby & Master Your Own Domain
 
Configuring MongoDB HA Replica Set on AWS EC2
Configuring MongoDB HA Replica Set on AWS EC2Configuring MongoDB HA Replica Set on AWS EC2
Configuring MongoDB HA Replica Set on AWS EC2
 

Último

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Último (20)

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

Mongo admin'ing

  • 1. Mongo Admin’ing Jesse Wolgamott twitter: @jwo slides: http://github.com/jwo
  • 2. Quick Install • Dev: brew install mongodb • Server: apt-get install mongodb • http://www.mongodb.org/display/ DOCS/Ubuntu+and+Debian+packages
  • 3. Don’t Fear the Command Line Mongoid / Mongo Mapper Ruby Driver Command Line Javascript (mongo)
  • 5. Command Line Basics use dba db.collection.find({json params}) db.collection.findOne({json params}) db.collection.mapReduce(map,reduce) db.collection.save({json})
  • 6. Why Bother? • Mongo queries enhance Mongoid queries • Drop to command line when needed • Game.where(:outcome=>{‘$in’=>[‘winning’ ,‘tie’]}) • Game.collection.update({:outcome=>‘win ning’, ‘$set’=>{:winning=>true}})
  • 7. All Admin is Command Line • Setup Replica Sets • Check on Status
  • 8. Replica / Slave / Journaling “If you aren’t running a replica set, you don’t care about your data” -- Mongo DB dude
  • 9. Master / Slave What You’d Expect M Slave Slave Slave
  • 10. Master / Slave What You’d Expect M Writes Go Here Slave Slave Slave
  • 11. Master / Slave What You’d Expect M Writes Go Here Slave Slave Slave Reads CAN Go Here
  • 12. Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Arbiter
  • 13. Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Writes Go Here Arbiter
  • 14. Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Writes Go Here CAN Read Arbiter
  • 15. Replica Sets The new Hotness (and Recommended) Primary Secondary Secondary Secondary Writes Go Here CAN Read Arbiter Nothing Goes Here
  • 16. Arbiter? • Arbiters exist when even number of hosts • They cast votes on primary elections • Can exist on your app server
  • 17. Journaling • Added in 1.8 • mongod -- journal • Will write out changes to a journal directory • You can restore from journal
  • 18. Replica Set Fun config = { _id: 'houstonrb', members: [ {_id: 0, host: 'localhost:20000'}, {_id: 1, host: 'localhost:20001'}, {_id: 2, host: 'localhost:20002', arbiterOnly: true}] } // Start it up rs.initiate(config); // Check it Out rs.status() // Who’s your daddy? rs.isMaster()
  • 19. Non-Master Reads error: { "$err" : "not master", "code" : 10107 } OK, so we’ll do: db.getMongo().setSlaveOk() // Awesome
  • 20. Backup / Restore mongodump --db houstonrb mongorestore --db houstonrb -c collections --drop --indexesLast DirectoryName * Have all your .bson files in a directory named your database
  • 21. Repairing • The Data Gobbler -- mongodb likes hard- drive space and won’t give it back • Use mongod --repair to reclaim your space!
  • 22. Nightly Plan service mongodb stop && mongod --repair --dbpath /var/ lib/mongodb && service mongodb start rm -f /data/mongodump-backup.tar.bz2 && mongodump - o /data/backups && tar -jcf /data/mongodump- backup.tar.bz2 /data/backups && rm -rf /data/backups/
  • 23. Catch Up! • When a Replica Set gets ‘stale’ • rs.status will remain “5” (Recovering) • On Master, will say “Error RS102 too stale to catch up” • stop mongo on all servers, copy over all data files, including local.*, start all
  • 24. Table Scans == Devil • When you get to 500,000 records, your DB may slow to crawl. • /var/log/mongodb/mongodb.log for logged queries (table scans) • db.collection.getIndexKeys() • Mongoid: rake db:mongoid:create_indexes
  • 25. No, Really • I’ve been surprised LOTS at how often a table scan happens. • Test it out: mongod --notablescan • Stuff will break.
  • 26. Sharding • Don’t Shard Early • Helps in map-reduce • You define a “shard-key”, like lastname from A-H on this shard
  • 27. Scaling (Secret Sauce) • Many Shard Masters, even more slaves • Add Shards for write and data size scaling • Add Slaves for read scaling (map-reduce) and redundancy • Sharding is Hard
  • 28. Neat Snippets Replication Setup Sharding Setup https://github.com/mongodb/mongo-snippets
  • 29. EC2 Notes • Reformat drive to EXT4 or XFS • Don’t use EBS for long term backups • Can use EBS for setting up replica sets

Notas do Editor

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n