Recently the interest in concurrent programming has grown dramatically. Unfortunately, parallel programs do not always have reproducible behavior. Even when they are run with the same inputs, their results can be radically different. In this talk I’ll show how to debug concurrency programs in Go.
I’ll start from showing how you can debug your gorotines using delve and gdb debuggers. Then I’ll try to visualize goroutines using different scenarios, sometimes it helps to better understand how things work. Next part of the topic will be about dumping a goroutine stack trace of your application while it’s running and inspect what each goroutine is doing. And I’ll demonstrate how to debug leaking goroutines by tracing the process of how the scheduler runs goroutines on logical processors which are bound to a physical processor via the operating system thread that is attached.
As a bonus i’ll cover debugging tips on how to find deadlocks and how to avoid race conditions in your application.
5. @a_soldatenko
Debugging sequential
programs
dlv test -- -test.run TestFibonacciBig
(dlv) b main_test.go:6
Breakpoint 1 set at 0x115887f for github.com/andriisoldatenko/
debug_test.TestFibonacciBig() ./main_test.go:6
(dlv) c
> github.com/andriisoldatenko/debug_test.TestFibonacciBig() ./
main_test.go:6 (hits goroutine(17):1 total:1) (PC: 0x115887f)
1: package main
2:
3: import "testing"
4:
5: func TestFibonacciBig(t *testing.T) {
=> 6: var want int64 = 55
7: got := FibonacciBig(10)
8: if got.Int64() != want {
9: t.Errorf("Invalid Fibonacci value for N: %d, got: %d,
want: %d", 10, got.Int64(), want)
10: }
11: }
(dlv)
6. @a_soldatenko
Debugging concurrent
programs
package main
import (
"fmt"
"time"
)
func say(s string) {
for i := 0; i < 5; i++ {
time.Sleep(100 * time.Millisecond)
fmt.Println(s)
}
}
func main() {
go say("world")
say("hello")
}
gowayfest git:(master) ✗ go run main.go
hello
world
world
hello
hello
world
world
hello
world
hello
gowayfest git:(master) ✗ go run main.go
world
hello
hello
world
hello
world
world
hello
hello
world
7.
8. What is a goroutine?
https://tpaschalis.github.io/goroutines-size/
12. visualize goroutines🐞 ?
func main() {
c := coloredgoroutine.Colors(os.Stdout)
fmt.Fprintln(c, "Hi, I am go routine", goid.ID(), "from main routine")
count := 10
var wg sync.WaitGroup
wg.Add(count)
for i := 0; i < count; i++ {
i := i
go func() {
fmt.Fprintln(c, "Hi, I am go routine", goid.ID(), "from
loop i =", i)
wg.Done()
}()
}
wg.Wait()
}
31. @a_soldatenko
How to detect deadlocks
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/Users/andrii/workspace/src/github.com/andriisoldatenko/
gowayfest/main_deadlock.go:5 +0x50
exit status 2
34. @a_soldatenko
7 simple rules for debugging
concurrency applications
- Never assume a particular order of execution
- Implement concurrency at the highest level possible
- Don’t forget Go only detects when the program as a
whole freezes, not when a subset of goroutines get
stuck.
- STRACE
35. @a_soldatenko
7 simple rules for debugging
concurrency applications
- conditional breakpoints your best friend
- DEBUG=schedtrace=5000
- go-deadlock