Webinar: Keeping Your MongoDB Data Safe
•Transferir como PPTX, PDF•
5 gostaram•3,303 visualizações
A backup and recovery strategy is necessary to protect your mission critical data against the risk of catastrophic failure or human error. In this session, we'll discuss the different strategies to backing up and restoring your MongoDB clusters in case of a disaster scenario. We'll review the benefits and drawbacks of various approaches, including taking filesystem snapshots, using mongodump, or using MongoDB Management Service.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (19)
Semelhante a Webinar: Keeping Your MongoDB Data Safe
Semelhante a Webinar: Keeping Your MongoDB Data Safe (20)
Mais de MongoDB
Mais de MongoDB (20)
Último
Último (20)
Webinar: Keeping Your MongoDB Data Safe
- 1. Keeping Your Data Safe Andrew Erlichson (andrew@mongodb.com) VP, Education & Cloud Services
- 3. Sometimes they are our fault
- 4. 4 • Lose very little to zero data in a failure • Down for a very short interval (availability is high) • BUT: programmer errors, deliberate data loss is often replicated nearly instantly. Replication
- 5. 5 • Much slower to restore (availability suffers) • Out of date to some extent • Fairly cheap • Well isolated • Handles all risks Backup
- 6. 6 • Checkpointing through Snapshots • Isolation Backup Defined
- 7. 7 • mongodump/mongorestore • Storage-level solutions • MMS Backup (new!) Backup Approaches
- 9. 9 • Can be run in live or offline mode • Oplog-aware for point-in-time operations • Filter can be applied in both directions • Considerations – Working set – Sharding mongodump / mongorestore
- 10. 10 Backing up mongodump --host foo.bar.com --oplog --out /data/backup Restoring To a running mongodb mongorestore –host foo.bar.com --oplogReplay /data/backup Directly to the filesystem mongorestore –dbpath /data/backup Using Mongodump
- 11. 11 • Copy files in your data directory (e.g. /data/db) • Filesystem or block storage snapshot • Fastest way to backup/restore • Considerations – Consistency – Backup granularity – Ops expertise Storage-level Backups
- 12. 12 • Don’t turn off journaling – Safe to take a snapshot of a running system if journaling journaling. • No Journaling – either shutdown a node or use fsyncLock() If you have Snapshotting
- 13. 13 • Use a volume manager that can handle taking incremental snapshots. • Turn on journaling • Backup from a secondary node • Take a snapshot ever six hours (or whatever frequency you need) • Use features with the volume manager to take incremental snapshots. Single Replica Set – Best Practices
- 15. 15 • Option 1: Run mongodump against the mongos router – But can’t use --oplog option – impractical for large system • Option 2: Backup each shard individually – Stop the balancer – Shutdown a replica set secondary on each shard – Shut down a node of the config server – Run mongodump or take snapshot of each shard – Run mongodump against the config server – Restart the config server, restart the balancer, Sharded System
- 16. 16 What is MongoDB Management Service (MMS)?
- 19. 19 MMS Backup: Big Picture Point in time restores • Restore replica to any moment in last 24 hours. Reliable • Developed and monitored by MongoDB • Redundant data centers • Fully managed solution Handles Sharded Systems • Consistent Snapshots Frees up Engineering to do Engineering • Are backups your core competency?
- 20. 20 Unlimited free restores Seed dev, test, or new environments
- 21. 21 • From the initial sync, we rebuild your data in our datacenters and take a snapshot • We take snapshots every 6 hours • Oplog is stored for 24 hours MMS Backup (under the hood)
- 22. 22 Sharded Clusters Backup Agent Shard0 Shard1 Shard2 OPLOG MongoDB Datcenter Encrypted Oplog Detect NOP and create snapshot for each shard NOPs Snapshot Snapshot Snapshot NOP NOP NOP
- 23. 23 Mongodump File system MMS Backup Initial complexity Medium High Low Confidence in Backups Medium Medium High Point in time recovery of replica set Snapshot Moments only Snapshot Moments only 24H System Overhead High Varies Low Scalable No With work Yes Consistent Snapshot of Sharded System Difficult Difficult Yes Recovery Approaches
- 24. 24 1. Create an account at mms.mongodb.com 2. Install MMS Monitoring Agent on your deployment 3. Install MMS Backup Agent on your deployment 4. Start initial sync 5. Rest easy! Getting Started with MMS Backup
- 25. 25 MongoDB World New York City, June 23-25 http://world.mongodb.com Save $200 with discount code MODERNAPPS #MongoDBWorld See how Bosch, UK Government Digital Service, Carfax, Stripe and others are engineering the next generation of data with MongoDB
Notas do Editor
- Welcome. I’m Andrew Erichson, VP of education and cloud services at MongoDB. Today I will be talking to you about Backup. Backup is not the sexiest topic, but its important in any production system.
- Backup is important because sometimes you lose the entire database or datacenter due to an unforseen disaster. However, disasters that result in the total loss of a database server are not the reason you will most often need to restore from backup. These scenarios can also be addressed by replication and and fault tolerance. If you lose the west coast data center, you could fail over to the east coast datacenter.
- The disasters that is difficult to address without backup are the ones where humans are involved. If we do a bad release for instance, and data becomes corrupted, or if someone successfully hacks into the system. The reason why these disasters are the most difficult to address without backup is that any corruption of data is typically propagated out to all our replica sets. For example, if someone breaks in and deletes data, that deletion will be replicated out to the other members of the replica set within seconds.
- Replication is never a complete substitute for backup, but it’s still a good idea. Replication makes us fault tolerant and we will lose a close to zero data in failure of a single component. The main shortcoming is that replication can’t address problems that affect all connected parts of the systems.
- The first defining element of all backup systems is they can take a snapshot of the system and can therefore perform checkpointing. This is both their best and worse feature. Because the snapshots represent a moment in time in the past, you can turn back the clock and bring the system to a point where everything was operating normally. If your system is attacked maliciously, you may lose confidence in the integrity of the state of the system and want to go back to a point way back in time. You might not even choose to restore this snapshot, but you might want to inspect it and compare it to the current system. The second defining element of Backup is that the the snapshots are isolated enough from the running system that a loss of the systems not correlated with a loss of the backups. So for example, you don’t want to leave your backups on your database server because the loss of the server will take the backups with it, leaving you nothing to go back to.
- Today we are going to talk about three approaches to backup. The first is using the mongodump and mongorestore commands to backup and restore your database. The second is to use a storage-level solution, essentially copying the database files, and the last is to use MMS Backup, an online backup service designed specifically to backup MongoDB.
- Before we dive into the first backup solution, I want to review the basic architecture for replication within Mongo. A replica set has multiple nodes that are all kept in sync. At any given time, there is a single primary that takes writes. In a typical configuration, you might have a single primary and two secondaries. It’s not unusual to take the backups from a secondary node to reduce the burden that backup puts on the system.
- Mongodump and mongorestorearre utilities that are distributed with Mongo. They are entirely free. Mongodump can connect to a running database server and read all the data out of it, essentially by querying it. It then writes BSON files to disk with the data. It’s a flexible program that can target a particular collection or database. And it even produce a point in time backup of a running replica set node by also copying the oplog. The big drawback of mongodump and its counterpart mongorestore is that they tend to be slow. As Mongodump runs, it will bring every document into memory. For a large database, it may simply not be feasible to run mongodump. It also copies the entire database each time, even though only a small amount of data may have changed since the last snapshot. Nevertheless, Mongodump is the way many folks get started because it’s the easiest solution to use. It also has an offline mode that can read the underlying database files directly, but to use that mode, you must shutdown the server, which is probably only feasible if you are backing up from a secondary node of a replica set.
- Using mongodump is straight forward. You run it from the command line, targeting a particular server on the network. In this case, I am showing with the –oplog flag to capture a point in time backup of running mongodb. Without that flag, there are not guarantees that your snapshot represents a consistent state that ever existed on the running system. When you restore, you can go through the running server. --oplog works even on a one node system, if you run it as a single node replica set. If you have multiple nodes to your replica set, an alternative approach is to either shutdown a secondary or call fsyncLock to stop writes to it before performing the mongodump Either way, you need to have a sufficiently large oplog that the allocated that the secondary does not fall out of sync with the primary.
- Rather than using mongodump and mongorestore, you can backup the underlying files that mongodump uses to store data. Here there are two basic approaches. You can just copy the underlying database files, but to do that, you need to stop writes to the server. Hence, that is only appropriate on a secondary node and you can do that by either shutting it down or running fsynclock.
- If you have mongodb running on top of a file systems that supports taking consistent snapshots of the entire file systems, then you should use that facility as it will be faster and lighter weight. Volume managers often support this, for example LVM. And EBS supports this on Amazon – but only on a single volume. Note that in order to snapshot a running instance of mongodb, you must have journaling enabled. Otherwise in flight operations may be captured in a state where they can be be properly recovered from the backup. You could wind up with a corrupted snapshot.
- Take this slide with a grain of salt. This is my opinion for the best way to properly backup a large replica set. If you have a small dataset, using mongodump is a fine solution. But if you have a very large running system, this solution will be the lowest overhead and will scale to very large datasets. First, use a volume manager that supports taking shapshos. Something like LVM or if you are on AWS, EBS, or LVM on top of EBS (provisioned iops I hope). If you are running on ephemeral SSD drives on amazon, then this si not an option. Turn on journaling. Don’t back up from your primary node. Use a secondary. Take a snapshot every six hours and use any features that will allow you to take a fast incremental snapshot.
- Ok, let’s talk about how you backup a sharded systems. This is not a talk on sharding, but to make this understandable we will need to spend just a minute talking about what sharding is. Sharding is the way you scale out in mongodb, and partitions each collection by a shard key across a number of different servers. Rather than your application making requests directly to MongoDB, you make them to a mongos router that sends the requests to the appropriate mongod servers. Each shard in this system is typically itself a replica set. And on the left, you see three purple config servers that keep meta data about the mapping of data between the shards. First, backing up this system to a single point in time is going to be hard because it’s a large distributed system with multiple entry paths. Our approach to backing up it up will be to try to get close to a single point in time, but it won’t be a single point in time.
- The first option is to run mongodump against a mongos router. But if you do this, you can’t use the oplog option and restores have to go back through a mongos. Any notion of which shard the data was stored on is lost. This is really not a practical option in most cases, but it does do something, just not something that is all that useful. It will take a long time to run on a large sharded system. Option 2 is to backup each shard individually. To try to get a close to a single moment in time, we will follow this procedure. First, turn off the balancer that redistributes data across the different shards. Second, shutdown a replica set member on each shard, Shut down a single config server. That will make sure that no changes occur to config server meta data. Now run mongodump or take a file systems snapshot against each secondary that is in the shutdown state. Once that completes, restart the config server and restart the balancer. If ths sounds like a lot o fwork, it is. And even then you don’t have a consistent moment in time of your whole running system. Now I am going to introduce an alternative to taking the backups youself that is much easier.
- MongoDB Management Service (MMS) is a cloud-based suite of services for managing MongoDB deployments. MMS provides monitoring and backup capabilities, helping users optimize clusters, automate tasks and mitigate operational risk. For customers that prefer to manage MongoDB on their own infrastructure, 10gen also offers MMS MonitoringOn-Prem as part of MongoDB Enterprise and the backup features will be part of that suite.
- To run MMS backup you signnup at mms.mongodb.com and install a backup agent in your datacenter. That backup agent continuously reads your incremental oplog data and ships it encrypted to MongoDB, Inc. From there, we maintain an image of your database, periodically applying the oplog to your data image. You can retrieve backup snapshots on demand.
- MMS monitoring is a free service that you must use in order to use MMS backup. The dependency exists because it’s the monitoring agent that collets the information about your system that the backup agent uses to locate running mongodb instances to backup.
- MMS backup is a reliable way to backup your data and it provides better backups than you could likely build yourself, especially if you have a sharded configuration. In particular, it offers point in time restores of each replica set for up to 24 hours. We achieve this by keeping your oplog data for 24 hours. By default, MMS backup takes a snapshot every six hours. but if you want to restore to a time between snapshots, we can roll forward from an existing snapshot to the exact moment tin time you want to restore. This ability to restore to a moment in time makes the MMS backup solution more useful than simple snapshots. We also handle sharded systems in a way that is more sophisticated than you can probably achieve yourself.
- MMS Backup is a premiujm part of the MMS suite and it costs money. You pay as you go, muck like AWS, however restores are free. This can be useful if you want to seed a new replica set with production data or just check to make sure that your backups are correct.
- When you configure MMS backup, we initially take a full snapshot of your data. We call this the initial sync. It’s achieved by copying the underlying backup files from your server, compressing and encrypting them and sending them to MongoDB. At the end of the process, we apply any intervening operations that occurred because we also record the oplog during the initial sync. Once the sync is complete, we take snapshots on OUR side every six hours. I want to emphasize that on your side, once we are synced, we only read oplog data, which is going to be much lighter weight than taking full snapshots. 6-hour interval snapshots for 2 days, Daily snapshots stored for 1 week, Weekly snapshots stored for 1 month,Monthly snapshots stored for 1 year.
- MMS Backup supports sharded clusters and provides a consistent moment In time. It’s not a point in time, but the snapshots we take are consistent. Here is how we do it. We pause the balancer every six hours on your side, we insert a no-op token into the mongos on your side. That token is replicated out to all shards. That nop token appears in the oplog we receive from each of your replica sets. The token acts as a sentinel on our side to create a snapshot that represents a consistent moment in time. Note that due to minor variances in how long each shard takes to process the token, there is no guarantee that the various snapshots represent the exact same moment, but they do represent a moment of consistency as seen by an observer who enters via a single mongos.
- Ok, lets recap the backup solutions that we covered today and compare their properties. Using mongodump is fairly easy to get started with, although I am not sure how confident you will be in your backups unless you are testing them. You can’t get point in time recovery using mongodump. You will be limited to your snapshot moments. System overhead with mongodump is high, and this is the biggest problem with the solution and the reason it does not scale well. Taking a consistent snapshot of a sharded system is difficult. We showed a way today to get approximately there, but not exactly there. As a variation to using mongodump, you can backup at the filesystem level. The problem there is that the initial complexity is higher System overhead can be quite low with this solution if your filesystem supports taking incremental snapshots and if you backup fromssecondaires. Creaing a consistent snapshot of a sharded system at a filesystem level is hard, but we showed you a way to get most of the way there.The final solution is MMS backup, which arguably provides the best of all worlds. It is easy to setup, provides a high degree of confidene, offers point in time recover for 24 hours and does not affect system performance much if ta all because it simply reads the oplog. It’s very scalabe and perhaps most interstingly, can provide a consistent snapshot of a sharded system.
- If you want to get started with MMS backup online, here are the steps. Create an account at mms.mongodb.com. Install the monitoring agent, enable some hosts. Now go back to MMS and click on “backup.” that will provide you an estimate of the costs of running MMS backup. If you like what you see, register with a credit card. You can now download and install the backup agent and then enable backup from within MMS. Note that we are lowering prices by about 50% in two weeks, so keep that in mind as you inspect the pricing.