C++ compilation boost

1. www.luxoft.com
C++ compilation boost
Last update 12 Oct 2016

2. www.luxoft.com
Compilation is verry long in C++

3. www.luxoft.com
Stages of problem understanding
 Negation ( C++ compilation is no slow)
 Reasons (compilation is slow because runtime is very fast)
 Transfer responcibility on C (backward capability)
 Adoption and finding ways to change the situation.
 Time is money

4. www.luxoft.com
Project
 Open source
 C++ code base
 About 20 developers team
 Cross platform (Linux, QNX, Windows, Qt)
 SmartDeviceLink https://github.com/smartdevicelink/sdl_core/
SmartDeviceLink (SDL) is a standard set of protocols and messages that connect applications on a
smartphone to a vehicle head unit.

5. www.luxoft.com
Why long compilation is bad?
 Spent developers time
 Diffusion of developers attention
 Destruction of nerves
Situation 1:
9:00 pm. Release, hotfix is ready, but processes does not allow to provide it
before build server won’t compile code, run unit tests, collect coverage, analyse
with static code analysys tools.
Situation 2:
Developer know that project builds ~ 1h. He trigger build and go to the
meating/dinner/coffe… But due to compilation errors it stops in 10 minutes. After
1 hour developer see it and upset.

6. www.luxoft.com
How c++ make executable from source files?
File_1.cc
File_2.cc
File_N.cc
main.cc
…
Magic Executable

7. www.luxoft.com
How c++ make executable from source files?
File_1.cc
File_2.cc
File_N.cc
main.cc
…
Executable
File_1.cc
File_2.cc
File_N.cc
…
File_1.o
File_2.o
File_N.o
…
main.cc main.o
Linker

8. www.luxoft.com
Components segmentation
File_1.cc
File_2.cc
File_N.cc
main.cc
…
Executable
File_1.cc
File_2.cc
File_N.cc
…
main.cc
Component_1 Component_2 Component_3
Linkage
Linkage

9. www.luxoft.com
Parallel build
Time
Cores
Build file1.cc Build file5.cc
Build file2.cc
Build file3.cc
Build file4.cc
Link component 1
Link component2
Link component 3 Link Executable

10. www.luxoft.com
Parallel build
Problem + Multithreading = Multiproblems
 Components dependencies
 Code generation dependencies
 Optimal count of jobs
 Parsing compiler error’s output
 Multithreaded build blocks work on slow worstations
 Unable to run multithreaded build on public build servers (Travis)

11. www.luxoft.com
Build is still to slow
Small tips for developers for fast build and good project health
 Do not store any logic in headers
 Avoid abuse of templates and metaprogramming.
 Follow SOLID principles
 Lookup that each *.cc file compiles only ones.
 Keep build dirrectory on RAM disk (tmpfs)
 Build project in a background in low priority during editing
 Avoid a lot of #ifdefs (use strategy pattern)
 Avoid creating special methods that are available in UT mode.

12. www.luxoft.com
Good ideas are around us

13. www.luxoft.com
Distributed build
WS1WS1 WS1WS2 WS1WS3 WS1WS4
Send file1.cc
Send file2.cc
Send file3.cc
Compile Compile Compile Compile
Send file1.o
Send file2.o
Send file3.o
Link

14. www.luxoft.com
Distributed build
Problem + distributed multithreading
=
distributed multiproblems
 Network latency
 Unification of environment
 Hold and sync list of workstations withat are able to take part in distributed build
 Performance issues on all workstations during build

15. www.luxoft.com
Statistics with distcc
statistics

16. www.luxoft.com
Can we do better?
CCache
 Cache build results. do not need to rebuild already builded binaries
 Easy to use and integrate in project
Usage in the wild:
$ ccache g++ file.cc -o output
Usage with cmake:
$ cmake ../ -DUSE_CCACHE

17. www.luxoft.com
Statistics with ccache
statistics

18. www.luxoft.com
Some final statistics
statistics
Project : https://github.com/smartdevicelink/sdl_core/

19. www.luxoft.com
THANK YOU
Alexander Kutsan
Email : Akutsan@luxoft.com
Github work: LuxoftAKutsan
Github personal : alexkutsan