1. Windows Kernel Debugging
Sisimon Soman

2. Agenda
• Introduction to BSOD
• Common approaches to analyze Windows
crash.
• Story 1 – Memory corruption.
• Story 2 – Stack Overflow
• Story 3 – Deadlock
• Story 4 - Memory Leak
• Q&A – Bring your dumps to discuss here.

3. Why BSOD
• Cause by buggy drivers.
• Drivers share address space with Kernel.
• Alternate stack to write dumps.
• KeBugcheckEx()
– Turn off interrupts.
– SMB box say all processor I am dying..
– Paint blue screen and write dump to pagefile.
• Winlogon role in writing crash dump.
• Type of crash dumps.

4. Common approaches to Kernel issue
• What bugcheck code says
• Which instruction cause crash
• How the stack looks at the time of crash
• Is third party drivers involved in stack
• The IRP which cause the crash.
• Is crash or system hung.

5. Story 1– Memory Corruption
• Driver Verifier
– Heap Corruption
– Special pool check (Guard Page)
• WinDBG commands to demo.
– !analyze –v
– kv
–u
– !pool

6. Pool corruption continued..
Make sure the pool chaining is correct
kd> !pool bc00248c
– Pool page bc00248c region is Paged session pool
– bc002000 size: 90 previous size: 0 (Allocated) Gla@
– bc002090 size: 10 previous size: 90 (Allocated) Glnk
– bc0020a0 size: 20 previous size: 10 (Allocated) Vtfd
– bc0020c0 size: 8 previous size: 20 (Free) Gtmp
– bc0020c8 size: 38 previous size: 8 (Free ) Usqm
– bc002100 size: 28 previous size: 38 (Allocated) Gldv
– bc002128 size: 58 previous size: 28 (Allocated) GFil
– bc002180 size: 198 previous size: 58 (Allocated) Uspi Process: 856b2a58
– bc002318 size: 18 previous size: 198 (Allocated) Uspi Process: 856b4528
– bc002330 size: 90 previous size: 18 (Allocated) Gla@
– bc0023c0 size: c0 previous size: 90 (Allocated) Gla4
– *bc002480 size: 80 previous size: c0 (Allocated) *Usms P

7. Stack Frames in Windows
• The purpose of EBP register – Frame Pointer.
• Function parameter, local variables in stack
and, it’s the same thread stack (one stack per
thread).
• EBP + offset points to function parameter
• EBP – offset points to local variables.
• X64 Stack frames.

8. Stack Frames continued..
• 0:000> u Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1
• Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1 [d:sisimonworking
areapresentationswindbgsimple app crash3simple app crash3simple app crash3dlg.cpp @
154]:
• 00401580 55 push ebp - First Push EBP to stack
• 00401581 8bec mov ebp,esp  Set new ebp as current esp
• 00401583 83ec14 sub esp,14h  make enough room for local variable
• 00401586 894dec mov dword ptr [ebp-14h],ecx <- Copy the this pointer to stack first
element for the current stack frame
• 00401589 c745f400000000 mov dword ptr [ebp-0Ch],0  one of the local var to 0
• 00401590 c745fc0a000000 mov dword ptr [ebp-4],0Ah  Another local var to 10
• 00401597 c745f80f000000 mov dword ptr [ebp-8],0Fh  Another local var to 15
• 0040159e cc int 3  Our hard coded break point

9. • 0:000> r
• eax=00000000 ebx=00000111 ecx=0013fe74 edx=00000000 esi=00403444 edi=00000001
• eip=0040159e esp=0013f860 ebp=0013f874 iopl=0 nv up ei pl nz na pe nc
• cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206
• Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1+0x1e:
• 0040159e cc int 3
• 0:000> dds 0013f860
• 0013f860 0013fe74
• 0013f864 7e41b788 USER32!GetParent+0x59
• 0013f868 00000000
• 0013f86c 0000000f
• 0013f870 0000000a
• 0013f874 0013f884  push current ebp to stack, set new ebp as esp Stack Frame - 1
• 0013f878 7c1746cc MFC71!_AfxDispatchCmdMsg+0x3d [f:vs70builds3077vcmfcatlshipatlmfcsrcmfccmdtarg.cpp @ 89]
• 0013f87c 00403444 Simple_App_Crash3!CSimpleAppCrash3Dlg::messageMap
• 0013f880 0013fe74
• 0013f884 0013f8b4  push current ebp to stack, set new ebp as esp Stack Frame - 2
• 0013f888 7c171915 MFC71!CCmdTarget::OnCmdMsg+0x10a [f:vs70builds3077vcmfcatlshipatlmfcsrcmfccmdtarg.cpp @ 396]
• 0013f88c 0013fe74
• 0013f890 000003e8
• 0013f894 00000000
• 0013f898 00401580 Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1 [d:sisimonworking areapresentationswindbgsimple app
crash3simple app crash3simple app crash3dlg.cpp @ 154]
• 0013f89c 00000000
• 0013f8a0 00000035
• 0013f8a4 00000000
• 0013f8a8 0013fe74
• 0013f8ac 000003e8
• 0013f8b0 00000000
• 0013f8b4 0013f8d8  push current ebp to stack, set new ebp as esp, Stack Frame - 3
• 0013f8b8 7c14db36 MFC71!CDialog::OnCmdMsg+0x1b [f:vs70builds3077vcmfcatlshipatlmfcsrcmfcdlgcore.cpp @ 88]
• 0013f8bc 000003e8
• 0013f8c0 00000000
• 0013f8c4 00000000
• 0013f8c8 00000000
• 0013f8cc 000003e8
• 0013f8d0 0013fe74
• 0013f8d4 00000000
• 0013f8d8 0013f928
• 0013f8dc 7c175cd8 MFC71!CWnd::OnCommand+0x51 [f:vs70builds3077vcmfcatlshipatlmfcsrcmfcwincore.cpp @ 2549]

10. Story 2 – Stack Overflow
• WinDBG commands to demo.
– dds
– dbs
– !thread
– !irp
– lmv
– dt
– du
– .trap
– .frame n
– dv

11. Story 3 – Kernel mode Deadlock
• Windbg commands to demo
– !locks
– !thread
– !handle
– kb
– !irql
– !irpfind
– ~ <processor>
– !devobj
– !stacks

12. Story 4 – Kernel Pool leak
• What is pool tags
• WinDBG commands to demo.
– !poolused 2
– !poolused 4
– !vm

13. Q&A