Fireside fusion talk - message queues
•
0 gostou•538 visualizações
A presentation about creating message queues in relational database. Presented at SQL BITS X
Recomendados
Mais conteúdo relacionado
Semelhante a Fireside fusion talk - message queues
Semelhante a Fireside fusion talk - message queues (6)
Último
Último (20)
Fireside fusion talk - message queues
- 1. Fireside Grade of the Steel Thomas Kejser
- 2. Grade of the Steel 1. Pick a simple workload 2. Do the naïve thing 3. Tune the … out of it 4. Find the limit 5. Get smart (optional)
- 3. Grade of the Steel Examples • 16M BULK Rows/sec • 50M rows SELECT (4 x 10Gbit) • Backup at 6GB/sec • 1.1M INSERT/sec
- 4. Step 1 – Theory and Baseline Commit SQLIO.EXE BufferOffset –t1 –o1 –b60 ... Slot 1 Slot –kW ... n -fsequential Writer COREINFO.EXE > info.txt Queue Log Writer
- 5. Step 2 – Design Workload File 1 FG 1 Table 1 File 2 FG 2 Table 2 SQLCMD Stripe File 3 FG 3 Threads Table 3 File X FG X Table X
- 6. Step 3 – Tune • Add Files / Tables Theory Measure • Thread Sweet spot • Affinitize I/O • Placement of Log Change
- 7. Messages Queues Part 1 – Naïve
- 8. Problem Statement Push Pop Queue Structure 300B Msg Msg Msg Msg Msg msg Ordered
- 9. The Naïve Approach Push Pop • Seek first value page • Seek last value page • INSERT a row at page • DELETE/Output key on page Min Max Min Max Msg Msg Msg
- 10. Why this doesn’t Scale! Root Min Min Min Min Min Min Min Min Min Min Max
- 11. Messages Queues Part 2 – Getting smart
- 12. Summarising the Problem • Hot stuff – Root – Min page – Max page – Intermediate pages • Alloc/Dealloc • BUT: We Must have order!
- 13. Cooling it down
- 14. Allocations - What if?... Push Pop • Seek first value page • Seek last value page • UPDATE Reference Count • UPDATE Reference Count Min Max Min Max Msg++ Msg--
- 15. Dissipate the Heat Last Digit = 0 Last Digit = 1 Last Digit = 2 Min Max Min Max Min Max Msg-- Msg++ Msg-- Msg++ Msg-- Msg++
- 16. Eliminating Thread Contention VERY fast! PopSequence++ PushSequence++ Queue Structure 4 5 6 7 8 Ordered
- 17. Ring Buffering PushSequence+=2 PopSequence++ PushSequence+=1 Mod 100 Mod 100 Mod 100 MyQ Slot:0 Slot:42 Slot: 97 Slot: 98 Slot: 99 Msg: 200 Msg: 142 Msg: 197 Msg: 198 Msg: 199 Ordered
- 18. Summing it up • UPDATE instead of INSERT/DELETE • Multi partitions = multiple trees • Ring buffer using modulo • Sweet spot concurrency