Embedded meetup of Open Tech Week by Sigma Software, Odesa 2018

1. 1

2. 2
Mesa and its debugging
Vadym Shovkoplias
Andrii Khulap
December 11, 2018

3. 3
Agenda
1. About myself
2. Mesa library
3. Debugging tools
3.1. Mesa environment variables
3.2. Basic debugging tools
3.3. GDB
3.4. Apitrace and Frameretrace
3.5. Steam specifics

4. 44
About myself

5. 5
About my self
Vadym and Andrii have more than 10 years of development experience. The main
specialty of Andrii is the built-in devices and software for them and main
speciality of Vadim is the development of software applications for embedded
systems based on Linux and Android.

6. 850+2000+
1000+
100+
TOP 4
IT-employer of
Ukraine
4 000+
Technology
professionals
350+
Ongoing
projects
25 +
Years in the
Ukrainian market

7. GlobalLogic
Kharkiv
● GlobalLogic Kharkiv — fastest growing
engineering center in Ukraine
● 30% of our consultants celebrated 5+ years
with Kharkiv GlobalLogic
● 30 active accounts / 80 ongoing projects
Key Domains:
- Medical
- Automotive
- Finance
- Industrial
Horizontal practices:
- Big Data
- Embedded
- Cloud (MS Azure, GCS, AWS)
- Data Science

8. 88
About Mesa

9. 9
Mesa
• Open source project with OpenGL and
Vulkan implementations
• Multi platform support by several
graphics vendors
• Used in Linux, Windows and MacOS

10. 10
Hardware support
• Intel
• AMD
• Nvidia
• Broadcom
• Qualcomm
• VMWare
• Vivante

11. 11
Software drivers
• llvmpipe
Uses LLVM for x86 JIT code generation
and is multi-threaded
• Softpipe
A reference Gallium driver
• Swrast
The legacy/original Mesa software
rasterizer

12. 12
https://mesamatrix.net/
# Driver Extentions OpenGL OpenGL ES
1 mesa (92.6%) 226 4.5 3.2
2 radeonsi (90.2%) 220 4.5 3.1
3 i965 (89.8%) 219 4.5 3.2
4 nvc0 (87.7%) 214 4.5 3.1
5 r600 (81.1%) 198 4.4 3.1
6 softpipe (70.9%) 173 3.3 N/A
7 llvmpipe (64.8%) 158 3.3 N/A
8 nv50 (62.3%) 152 3.3 N/A
9 freedreno (62.3%) 152 3.1 N/A
10 swr (57.8%) 141 3.3 N/A
OpenGL support

13. 13
Development workflow
https://bugs.freedesktop.org/descri
becomponents.cgi?product=Mesa
Bugzilla
https://www.mesa3d.org/lists.html
Mailing lists and IRC
https://www.mesa3d.org/submitting
patches.html
Submitting patches

14. 14
Project statistics
Patches:
Pushed: 67
On review: 51
Rejected: 26
Bugs:
Total: 156
- Debugged/analyzed: 86
- Fixed: 70

15. 1515
Debugging tools

16. 16
Mesa environment variables
Normally, no environment variables need to be set. Most of the environment
variables used by Mesa are for debugging purposes, but they can sometimes
be useful for debugging end-user issues.
Example:
export LIBGL_ALWAYS_SOFTWARE=1 (if set, always use software rendering)

17. 17
Mesa environment variables: Shader dump

18. 18
The basic debugging tool is console output
• Use printf() or fprintf() to output any valuable information
• It will output to the terminal where tested application is started

19. 19
Use static analyzers to find possible causes
• Some bugs or issues can be caused by simple mistakes that can be found by
automated tools.
• Like smatch or cppcheck
For example, smatch warning:
[src/intel/vulkan/anv_nir_apply_pipeline_layout.c:457]: (error)
Possible null pointer dereference: null_data
And the library code itself:
struct anv_push_constants *null_data = NULL;
const struct brw_image_param *image_param = null_data->images;

20. 20
GDB
Building mesa with GDB support
# export CFLAGS='-O0 -ggdb3'
# export CXXFLAGS='-O0 -ggdb3'
# ./autogen.sh --with-gallium-drivers="" --with-dri-drivers=i965
# make -j4

21. 21
You able to debug Mesa even with
comercial apps and games without
debug symbols.
Run the application in GDB:
GDB

22. 22
Run the program with r and receive
a backtrace with bt:
GDB

23. 23
Apitrace is a tool to capture and replay OpenGL calls

24. 24
Frameretrace - view and analyze each render for any frame

25. 25
Renderdoc

26. 26
Piglit is a collection of automated tests for OpenGL
implementations.
• At this moment, there are around 60k tests total. There are performance
(rendering time) test as well as compatibility tests. Test results can be used for
regression testing.
• Useful to check if patch has some side effects on other areas.
• Piglit tests can be run either directly or as a profiles (test sets). Running quick
profile will take around 20 minutes. Profiles like “all” will run for several hours.
• The results will be in json format. But the html (good looking) summaries can also
be created.

27. 27
Steam games specifics - using GDB
• Usually steam games doesn’t contain any debug information. But we still can see
useful backtraces and mesa calls.
• You need to determine game’s PID. Not steam or some launcher/helper.
Use ps -aux for example.
• Then you may easily attach gdb to that process
(gdb) attach 2057
(gdb) continue

28. 28
Steam games specifics - customizing environment
You can configure steam GUI to launch the script with “preparations”:
• Create the script and place it in the /home/${USER}/bin
E.g steam_dbg.sh:
#!/bin/bash
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/andriy/mesa/lib
"$@" > /home/andriy/steam_game.log
• Go to library, right click on a game and choose “Properties”
• On General tab click “Set launch options”
• Specify the run options: steam_dbg.sh %command%

29. 29
Useful Links
• https://cgit.freedesktop.org/mesa/mesa/
• https://lists.freedesktop.org/mailman/listinfo/mesa-dev
• https://lists.freedesktop.org/mailman/listinfo/piglit
• https://bugs.freedesktop.org/describecomponents.cgi?product=Mesa
• https://fosdem.org/2018/schedule/event/apitrace/
• https://github.com/janesma/apitrace/wiki/frameretrace-branch

30. 30
Thank you

31. 31