Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a 视觉中国的MongoDB应用实践(QConBeijing2011)
Semelhante a 视觉中国的MongoDB应用实践(QConBeijing2011) (20)
Último
Último (20)
视觉中国的MongoDB应用实践(QConBeijing2011)
- 1. QCon beijing 2011 MongoDB : http://czone.chinavisual.com/art/4b4501548f47e8ef73699a0c.html
- 2. About.me • ( nightsailer) • @nightsailer //twitter,sina,linkedin, github ... • nightsailer # gmail.com • http://nightsailer.com/
- 3. MongoDB NoSQL ? Auto-shard ? • No! 08 K/V MySQL
- 4. App MMM • MySQL Percona Mysql(Master-Master) Master-Master-Slaves • HA:MMM Mysql(M1) Mysql(M2) Slave Slave
- 5. • • schema • • •
- 6. MySQL * JSON • schema • * • schema • query
- 7. App MMM/vdb11 KV KV1 KV2 Mysql(M1) Mysql(M2) Slave Slave
- 8. Memcached • Memcached • KV
- 9. • PHP/Perl • Memcached • • •
- 10. • Flare • Repcached • Redis • TC/TT
- 11. Flare • cluster , • Memcached
- 12. 1 • • • • • ;-(
- 13. • Cassandra • • CouchDB •
- 14. MongoDB • Redis • Document • Redis • MySQL , • MySQL
- 15. MySQL MongoDB • MySQL MongoDB • MySQL <=> MongoDB •
- 17. • •1 1 90% • 35 table => 10 collection • happy!
- 18. MongoDB, GridFS • • • MogileFS
- 19. MongoDB, • SourceForge • • 10gen mailing-list • NoSQL MongoDB
- 20. 1 • comments:{ _id:ObjectId(‘xxx’), art_id:2, content: ‘ ...’, replied_on: 12233 created_on: 12222 replies: [{ _id: ObjectId(‘xxx’), content: ‘ , ...’, replies:[] }]}
- 21. 2 / : • • • / / ...) •
- 22. 2 • db.activity_stream.feed ( • db.activity_stream.user ) • db.activity_stream(
- 23. 2 feed • • follower collection • embed list
- 24. 3: 1: Regex • 2: Sphinx • xml •
- 25. 3: 3: Array/List ($all) • • PHP-SCWS • : • oplog
- 26. MongoDB • Scons, Python • boost(CentOS) • static link mongod • tcmalloc
- 27. • MongoDB( ICC COMMON_CXXFLAGS='-fp-model source -unroll2 -axSSE4.1,SSE4.2 -xSSE3 - static-intel -fpic -fno-strict-aliasing' CXXFLAGS="-O3 -ipo -static-libgcc $COMMON_CXXFLAGS" scons --release --static --extrapath=/opt/local --cxx=$CXX --icc -- extralib='tcmalloc_minimal' --icc-cxxflags="$CXXFLAGS" --icc- cppflags="$CPPFLAGS" -c $BIN_SERVER scons -j4 --release --static --extrapath=/opt/local --cxx=$CXX --icc --extralib='tcmalloc_minimal' --icc-cxxflags="$CXXFLAGS" --icc- cppflags="$CPPFLAGS" $BIN_SERVER
- 28. • •
- 29. • Raid10 • • XFS • Ext4 (?)
- 30. PHP-FPM 2009/6, 0.9/1.0 • 1 Master + 2 slaves Nginx Mongod Master) Gearman-workers • 1m • 20g Mongod Slave) Gearman-workers • Dell 2850/4g(Master) 2*Dell 2950/4g(
- 31. * Slave • * lvm snapshot • • fsync & lock db * mongodump •
- 32. • mongostat / vmstat / iostat • collectd • : json-rest+perl plugin • Munion / Nagios ...
- 33. MongoDB • CPU • 4G • IO :http://czone.chinavisual.com/art/4c7918b74979590970b80000.html
- 35. 0: * MongoDB • Out of Memory! • v1.3 * cursor • Perl driver bug,
- 36. 0: 1: • • 2 2: • driver • driver
- 37. 1:50x 502 Bad Gateway * GridFS • Perl Plack Starman/Plack Nginx Proxy store MongoDB Disk Cache * GridFS • Nginx proxy_store •
- 38. 1:50x 504 Gateway timeout • • • Perl nginx_errorlog 5 mongodb.log • client
- 39. 1:50x 3 • XFS pre-allocation FS • for i in {1..50} do echo $i head -c 2146435072 /dev/zero > $db.$i done
- 40. 2: • Mongod crash • Why? • Map/Reduce / •
- 41. • • 10 • MongoDB • Repair 5 • • 10 : http://czone.chinavisual.com/art/4cad0e08497959d621a10000.html
- 42. • MongoDB • • cluster : slave •
- 43. 1 • --syndelay • 60s(default) => 15~30s • IO • fsync •
- 44. 2 • 1.6.3, Master-Slaves RelicaSets • 1 Primary + 2 Secondary • 4g-8g • w=2 •
- 45. 5 • • • •
- 47. 4: RS fail-over • MongoDB • Primary kill 2 secondary • secondary
- 48. 4: RS fail-over • 2 Arbitor • $ mongod --bind_ip 127.0.0.1,192.168.8.10 -- replSet rs10 --oplogSize 1 ... • > rs.addArb(‘192.168.8.10:27020’) • ...
- 49. 4: RS fail-over 6: ReplicaSet • 1 Primary + 2 Secondary + n Arbitor • •
- 50. •6 “ ” • ;-( • • ReplicaSet fail-over secondary • Primary , •
- 51. 1.8 • 1.8 • journaling file • mongod -dur • crash repairDatabase •
- 52. • GridFS • 10mb-500mb •
- 53. • GridFS • Nginx proxy_store • MongoDB • ,Plack app • prefork
- 54. • GridFS • • slaveOK • Plack app Twiggy AnyEvent
- 55. Starman workers/PSGI Starman RS02 Nginx Gearman 127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003 127.0.0.1:9004 Twiggy/PSGI RS01 (slaveOK) Sencondary
- 56. • • •
- 57. Primary • Slave only IDC Secondary1 MongoD Secondary2 ReplicaSet1 • priority 0 VPN • VPN 2 IDC Slave1
- 58. • >300ms • MongoDB
- 59. • MongoDB • snapshot • • local.oplog.rs ( tailable cursor) • • replay oplog
- 60. • 2 • GridFS • VPN
- 61. • VPN • GridFS • • BSON HTTP • mongod
- 62. 7 • • • • unix : http://czone.chinavisual.com/art/4c05e7be4979596b7e570000.html
- 63. Auto-sharding • 1.6GA Auto-sharding • •2 1 • •
- 64. • Shard_key • shard key chunk • 4sq • shard_key • counting • chunk • balancer
- 65. • • shard • shard • • 1.8
- 66. • • MongoDB auto-shard • GA • 2.0+ ?
- 67. MongoDB • MySQL Web • Schema free • Geo • MySQL • GridFS • sharding • Auto-sharding shared_key balancing 1.8/2.0
- 68. • mongostat • idx missing, faults,global locked • db.serverStatus & rs.status
- 69. • Replication • oplogsize // slave clone • --fastsync + snapshot • --maxConns • --replSet=<set_name>/<seed list> • seed list 1 arbitor
- 70. • Some collections • local.oplog.rs/replset.minvalid • $cmd • system.indexes
- 71. • GridFS • :Write once, Read many. • fs.files _id ETag • file content hash • gc • * Node.js
- 72. • Update/Delete • ,Schema padding • In-place update • Mark delete remove • • remove() , drop
- 73. • Group • 4Mb MR • db.eval & Javascript • db • scope •
- 74. • DBRef • • MR • • MR • v8
- 75. Question? :http://czone.chinavisual.com/art/4b45015461be3def730e6351.html