Talk from Sysdig Camp-Con-World-Fest-Summit 2016

1. Adam Leventhal, @ahl
A Brief History of
System Calls

2. Who Am I?
• Co-inventor of DTrace
• Founder ZFS Storage Appliance team
• CTO at Delphix
• EIR at Sutter Hill Ventures
• Recent founder

3. What’s a System Call?
• User call into the kernel to do some work
• Typically implemented with a trap
• Mostly standardized in terms of function
• Fundamental to multi-user systems

4. USENIX Summer 1986

5. Time Spent in Syscalls Matters

6. Syscalls Tell You About The System

7. ULTRIX Trace
/dev/trace
…
If (tracing)
...
/* do the syscall */
If (tracing)
...
Syscall Handler
call exec(...)
return exec(...)
call open(...)
return open(...)
call read(...)
return read(...)
Buffer

8. Aside: How Great Is This?

9. Syscall Tracing For All
• Truss 1988
– “If your program doesn't work, put it in a truss.”
– Roger Faulker
– Used then-new structured /proc
– SunOS / UNIX
• Strace 1991
– “an alternative syscall tracer”
– Paul Kranenburg on comp.sources.sun
– SunOS then, famously, Linux in 1992
– Non-Linux code removed in 2012

10. Breakpoint Tracing
process
invoke syscall
execute syscall
proceed
scheduler
stop process
wake tracer
stop tracer
wake process
stop process
wake tracer
stop tracer
wake process
tracer
record arguments
resume process
record return value
signal process

11. Slow
# Slow the target command and print details for each syscall:
strace command
# Slow the target PID and print details for each syscall:
strace -p PID
# Slow the target PID and any newly created child process, pri
strace -fp PID
# Slow the target PID and record syscalls, printing a summary:
strace -cp PID
# Slow the target PID and trace open() syscalls only:
strace -eopen -p PID
Brendan Gregg

12. Do No Harm
• Unsafe
• Chase away the
problem

13. DTrace
• Dynamic tracing framework
– Solaris, macOS, FreeBSD, some Linux distros
• Safe for production by design
• Concise answers to arbitrary questions
• Systemic in scope
• Syscalls are a common place to start

14. Syscalls with DTrace

15. Aside: Java
“Write once, run anywhere”

16. Java Java Java

17. Servers in1996

18. Write Once Run Anywhere + 20yrs

19. Primacy of Syscalls
• Highly significant events
• 30 years later, still the right granularity
• Well-understood domain for tracers
• Ubiquitous
• De facto standard in Linux

20. Thank You!