For the first time this year, 10gen will be offering a track completely dedicated to Operations at MongoSV, 10gen's annual MongoDB user conference on December 4. Learn more at MongoSV.com
Come learn about the different ways to back up your single servers, replica sets, and sharded clusters
11. mongodump
• Snapshot of each collection
– Does NOT represent a point in time, even for a single
collection
• Can NOT be combined with fsyncLock
– Remember, you can't read…
• You CAN dump directly from data files to get a
point in time backup
– mongodump –dbpath
• Can be costlier than archiving as FS level
14. Journaling
• Write-ahead log
• Guarantees a consistent view even after a hard
crash
• Default behavior as of 2.0
• Journal stored in –dbpath /journal folder
• --journalCommitInterval* (2ms - 300ms)
15. Journaling implications for
backup
• Logical Volume Manager (LVM)
• LVM snapshots to the rescue
– lvcreate –size 100M –snapshot –name mdb-snap01 /dev/vg0/mongodb
• No shutdown or fsyncLock needed
• True point in time backup for a single instance
17. Backing up a replica set
• Back up a (hidden) secondary
– kill mongod
– fsyncLock
– mongodump
– LVM snapshot
18. Mongodump for replica sets
• True point in time
– mongodump –oplog
– mongorestore –-oplogreplay
• Snapshot query of each collection, then replay
the oplog at the end
– Similar to how a new secondary does an initial sync
Make the point that while you can turn journaling off, you shouldn't.Without journaling, the approach is quite straightforward, there is a one-to-one mapping of data files to memory and when either the OS or an explicit fsync happens, your data is now safe on disk.With journaling we do some tricks.Write ahead log, that is, we write the data to the journal before we update the data itself.Each file is mapped twice, once to a private view which is marked copy-on-write, and once to the shared view – shared in the context that the disk has access to this memory.Every time we do a write, we keep a list of the region of memory that was written to.Batches into group commits, compresses and appends in a group commit to disk by appending to a special journal fileOnce that data has been written to disk, we then do a remapping phase which copies the changes into the shared view, at which point those changes can then be synced to disk.Once that data is synced to disk then it’s safe (barring hardware failure). If there is a failure before the shared/storage view is written to disk, we simply need to apply all the changes in order to the data files since the last time it was synced and we get back to a consistent view of the data
LVM Logical volume manager. LVM is a program that abstracts disk images from physical devices, and provides a number of raw disk manipulation and snapshot capabilities useful for system management.LvcreateThis command creates an LVM snapshot (with the --snapshot option) named mdb-snap01 of the mongodbvolume in the vg0 volume group.This example creates a snapshot named mdb-snap01 located at /dev/vg0/mdb-snap01. The location and paths to your systems volume groups and devices may vary slightly depending on your operating system’sLVM configuration.The snapshot has a cap of at 100 megabytes, because of the parameter --size 100M. This size does not reflect the total amount of the data on the disk, but rather the quantity of differences between the current state of /dev/vg0/mongodb and the creation of the snapshot (i.e. /dev/vg0/mdb-snap01.) Make sure you size this big enough.EBS:If your deployment depends on Amazon’s Elastic Block Storage (EBS) with RAID configured within your instance, it is impossible to get a consistent state across all disks using the platform’s snapshot tool. As a result you may: 1. Flush all writes to disk and create a write lock to ensure consistent state during the backup process. If you choose this option see Backup Without Journaling. 2. Configure LVM to run and hold your MongoDB data files on top of the RAID within your system.If you choose this option, perform the LVM backup operation described in Create Snapshot
If the secondary is hidden, then options are more varied. Killing and locking are valid options, so long as there is enough spare capacity in the system to catch up after the backup is complete
Ok if you have enough space to store all the data on all the shards