CHT TMS training and MongoDB introduction.

1. TMS system and MongoDB introduction
RICH LEE
CHT TMS Developer
rich.lee@thinkpower.com.tw

2. TMS System Architecture
MongoDB Introduction
TMS MongoDB
Q&A
2014/8/26 CHT TMS Training 2

3. TMS Module
 CDR Parser
 CDR Consolidator
▪ Data, Voice, Mvpn
 CSV Generator
 Report
▪ Usage, Lumsum, Frozen
 Performance Report
 SNMP Alarm
 API
2014/8/26 CHT TMS Training 3

4. TMS Module Diagram
2014/8/26 CHT TMS Training 4

5. CDR process flow
2014/8/26 CHT TMS Training 5

6. Parser multi-thread concepts
• Three stages of the CDR Parser:
 1st: Decoder (Decode worker and MongoBO)
 2nd: Queues (Mongo Queue)
 3rd: Insert into DB (Mongo worker)
• Three parser instances:
 Parser (CCN)
 Parser (SDP)
 Parser (AIR)
2014/8/26 CHT TMS Training 6

7. What is MongoDB?
 MongoDB is an open source, document-oriented
database designed with both scalability and developer
agility in mind.
 Instead of storing your data in tables and rows as you
would with a relational database, in MongoDB you store
JSON-like documents with dynamic schemas(schema-free,
schemaless).
2014/8/26 CHT TMS Training 7

8. MongoDB Feature
 Schema-less
 Document-oriented (Bson)
 Scale-out
▪ Replica set
▪ Sharding
 Rich functionality
▪ Index, Aggregation, Backup
2014/8/26 CHT TMS Training 8

9. Why use MongoDB
 bridge the gap between key-value stores (which are fast and
scalable) and relational databases (which have rich functionality).
2014/8/26 CHT TMS Training 9

10. MongoDB vs RDBMS
2014/8/26 CHT TMS Training 10

11. Data model
 Using BSON (binary JSON), developers can easily map
to modern object-oriented languages without a
complicated ORM layer.
2014/8/26 CHT TMS Training 11

12. Schema design
2014/8/26 CHT TMS Training 12

13. Schema design
2014/8/26 CHT TMS Training 13

14. Basic operator
2014/8/26 CHT TMS Training 14

15. Read Command in MySQL
Read Command in MongoDB
2014/8/26 CHT TMS Training 15

16. Replica set – scale out
1. Availability
2. Fault tolerance
2014/8/26 CHT TMS Training 16

17. Replica set - election
2014/8/26 CHT TMS Training 17

18. Replica set consistency
 Only one server is active for writes (the primary) at a given time –
this is to allow strong consistent (atomic) operations.
 One can optionally send read operations to the secondaries when
eventual consistency semantics are acceptable.
2014/8/26 CHT TMS Training 18

19. DB list
2014/8/26 CHT TMS Training 19

20. User role
Account DB Role Description
tmsdb ROOT Super admin
cht_app Read Write Can CRUD
cht_report Read Only query data
cht_cluster Cluster admin View replica status
Login TMS MongoDB
 mongo --host 10.106.60.65:4001 --username tmsdb —password xxxx --
authenticationDatabase admin
Use DB and count
 show dbs (only root uesr)
 use TMS_CCN_1_DB
 db.TMS_CCN_1_140506.count()
2014/8/26 CHT TMS Training 20

21. Field mapping
2014/8/26 CHT TMS Training 21

22. Query definition
Equal Query
 db.TMS_CCN_2_140715.find({"MSISDN":"886988413631"}})
Regex Query
 db.TMS_CCN_2_140715.find({"MSISDN":{$regex : ".*988413631"}})
And Query
 db.TMS_CCN_4_140804.find({ $and: [ {"MSISDN":"886988413631”},
{”BP":"886905001256”} ] })
Project (Select feild)
 db.TMS_CCN_4_140724.find( {MSISDN:"886905004094”},
{MSISDN : 1, BP : 1, TT :1, DT :1})
2014/8/26 CHT TMS Training 22

23. Query by date range
 GMT not local timezone
 db.TMS_CCN_1_140609.find({ $and: [ {"MSISDN": "886905001250"}, {"TT" :
{ $gte : new ISODate("2014-06-16 01:00:50.652Z") }}, {"DT" : { $lte : new
ISODate("2014-06-16 01:59:50.652Z") }} ] })
Distinct Query
 db.TMS_CCN_1_140822.distinct( "MSISDN",
{ TT: { $gt: new ISODate("2014-08-22 05:00:00.294Z") } })
Sort Query
 db.TMS_CCN_4_140722.find( {"ISCSV":false} ).sort( { OCCR8: 1 } )
2014/8/26 CHT TMS Training 23

24. Check Replica set status
 rs.status()
View Replica set configuration
 rs.conf()
Set secondary node can read
 rs.slaveOk()
2014/8/26 CHT TMS Training 24

25. Http Rest interface
 http://host:port ，port is mongod port +1000
2014/8/26 CHT TMS Training 25

26. MongoDB monitor
 mongostat --host localhost:27017 --username account—password
pass --authenticationDatabase admin
Check MongoDB log
 Mongo.log (/opt/mongo/data/log/)
2014/8/26 CHT TMS Training 26

27. 2014/8/26 CHT TMS Training 27