WalB kernel driver の内部の簡単な紹介スライド (LT)

1. WalB Driver
Internals
星野 喬@サイボウズ・ラボ
2017-09-25
Cybozu Meetup #8 SRE
1

2. 自己紹介
• 星野 喬 (@starpoz)
• サイボウズ・ラボ (2009〜)
• 興味
• データベース/ストレージ/分散システム
• 仕事
• WalB の開発と cybozu.com への導入
• トランザクションシステムの探求
2

3. WalB デバイス
3
WalB device
Data device Log device
Read Write
Any application (File system, DBMS, etc.)
Linear mapped Ring buffer

4. WAL (Write-Ahead Logging)
• 書き込み履歴 (wlog) を先に書く
• データの一貫性を守るため
• WalB では wlog をバックアップのために利用
Data storageLog storage
Write at 0
Write at 2
Read at 2
Write at 2
0
0 2
0 2
0 2 2
Time
4

5. Log Device Layout
5
Checksum
2nd
written data
Ring buffer
Log pack
Logpack
header
block
1st
written data …
The oldest logpackThe latest logpack
1st log record
IO address
IO size
...
Logpack lsid
Log pack
header
block
Num of records
Total IO size
2nd log record
IO address
IO size
...
...

6. WalB は redo log のみ記録
• Undo log を作るためには追加の read が必要
• Undo log を作ると性能低下は避けられない
Data Log
(1) write IOs
submitted
(2) read current
data
(3) write
undo logs
Data Log
WalB
(1) write IOs
submitted
(2) write
redo logs
Generating redo logs Generating undo logs
6

7. WalB の IO 処理
• Wlog 書くだけで IO complete
Submitted
Time
Completed
Packed
Log submitted Log completed
Wait for log flushed and overlapped IOs done
Data submitted Data completed
Log IO response Data IO response
WalB write IO response
Pdata inserted Pdata deleted
Write
Submitted
Time
Completed
(Data submitted) (Data completed)
Read
Pdata copied
Data IO response
7