Query performance can either be a constant headache or the unsung hero of an application. MongoDB provides extremely powerful querying capabilities when used properly. As a member of the support team I will share common mistakes observed as well as tips and tricks to avoiding them.
5. The Power of Query Optimization
Query tuning results in:
Improved performance
Reduced resource utilization
This may lead to:
Improved stability and predictability
A smaller hardware footprint
Not uncommon to observe efficiency improvements greater than 99%
6. About Me
Technical Services Engineer (Support)
2.5 year tenure
Member of the Technical Experts program
Focus: Queries and Indexing
Previously: Data Warehouse workload optimization
7. About Me
Technical Services Engineer (Support)
2.5 year tenure
Member of the Technical Experts program
Focus: Queries and Indexing
Previously: Data Warehouse workload optimization
8. Meet Asya
DBA at Acme Game, Inc.
MongoDB Champion
Meet Stakeholders
Others at Acme, Inc.
Developers
Leadership
RDBMS Historically
12. Stakeholder Concerns
Game nearly complete
Developers have learned a lot from Asya
Indexes support the efficient
execution of queries in MongoDB
13. Stakeholder Concerns
Game nearly complete
Developers have learned a lot from Asya
Indexes support the efficient
execution of queries in MongoDB
Ace Sue
… …Bob
15. Stakeholder Concerns
Game nearly complete
Developers have learned a lot from Asya
App being stress tested
Concerns over current performance
16. Stakeholder Concern #1
Developers created index
db.games.createIndex({ gamerTag: 1 })
This query takes several seconds to execute:
db.games.find( { gamerTag: "Ace" } ).sort({score:-1})
Adding the index on score does not help!
db.games.createIndex({ score: -1 })
17. Developers created index
db.games.createIndex({ gamerTag: 1 })
This query takes several seconds to execute:
db.games.find( { gamerTag: "Ace" } ).sort({score:-1})
Adding the index on score does not help!
db.games.createIndex({ score: -1 })
“Clearly MongoDB
is not webscale!”
Stakeholder Concern #1
19. Blocking Operation
Formally:
“An operation which must process all input before it can begin to produce any output.”
Opposite of the often desirable “fully pipelined” plan which can stream results back as
soon as they are found.
Commonly observed when a sort is added to a query
49. Working with blocking stages
For sorting:
Add a supporting index
Worth the overhead in almost all circumstances
For other stages:
Do you need the blocking stage?
Offload to secondary member
52. Stakeholder Concern #1
Performance of
db.games.find( { gamerTag: "Ace" } ).sort({score:-1})
db.games.createIndex({ gamerTag: 1, score:-1 })
"That’ll work great!”
53. Stakeholder Concern #2
The $and version of a query
returns quickly:
db.games.find({
$and : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
But the $or version is slow:
db.games.find({
$or : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
54. Stakeholder Concern #2
The $and version of a query
returns quickly:
db.games.find({
$and : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
But the $or version is slow:
db.games.find({
$or : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
We just created an index with both
those fields… Can it be used?
105. Recommendations
We already have the {gamerTag:1, score:-1}
index, do we need both of these new ones?
Use multiple indexes!
db.data.createIndex({gamerTag: 1})
db.data.createIndex({score: 1})
106. Recommendations
We already have the {gamerTag:1, score:-1}
index, do we need both of these new ones?
Use multiple indexes!
db.data.createIndex({gamerTag: 1})
db.data.createIndex({score: 1})
110. db.games.find({
$or : [ { gamerTag: "Ace" }, { score: {$gt: 9000} } ]
})
Having the right index is critical
"Super!!”
Stakeholder Concern #2
111. “Wait wait wait, we can’t even FIND the gamers!”
A basic search on gamerTag takes several seconds already:
db.games.find({gamerTag: /^Ace$/i})
“This query is SLOWER with the index than it is without it!”
Stakeholder Concern #3
122. Recommendations
Case insensitive index!
Collations available since 3.4
db.games.createIndex( { gamerTag: 1},
{ collation: { locale: 'en', strength: 2 } } )
Store a transformed (eg toLower()) copy of the string
127. Work Smarter Not Harder
Understand the business logic
Index appropriately
Is it the right index to support the query?
Be aware of:
Blocking Stages
Usage of $or
Case sensitivity
Leverage the Performance Advisor
128. Work Smarter Not Harder
Understand the business logic
Index appropriately
Is it the right index to support the query?
Be aware of:
Blocking Stages
Usage of $or
Case sensitivity
Leverage the Performance Advisor