Evaluate - a swiss knife to revise tool default behavior. One popular case is using `evaluate` to revise default compiler options. Because building systems, such as nijia and make files, are intricated and difficult to modify and study. So Skymizer provides `evaluate` to help you change the default option of compiler easier.

1. evaluate: revise tool default behavior
Luba Tang

2. replayer
Evaluate - Swiss knife to revise tool
behavior
 Intercept commands to the tool and revise it
 replayer (burglar) - intercept the control to original tool. Modify
the options to the original tool
 evaluate (hobbit) - controller of replayer
original
tool
evaluate
control
revised command
intercept
original
tool
command

3. Compiler Sensitivity - evaluate compiler optimization
 high watermark/low watermark
– Trivial upper bound/lower bound for a compilation
 Compiler sensitivity - The performance distance between
watermarks
compiler sensitivity
high watermark
low watermark

4. Set up high/low watermark
-mark-high [tool path]
• set target compiler to high watermark
-mark-low [tool path]
• set target compiler to low watermark
[local: ~]$ sudo evaluate -mark-high /usr/bin/gcc
[local: ~]$ gcc ./main.c -o ./a.out # build your project as usual
[local: ~]$ time ./a.out # evaluate performance as usual
real 0m0.002s
user 0m0.000s
sys 0m0.000s
[local: ~] sudo evaluate -mark-low /usr/bin/gcc
[local: ~] gcc ./main.c ./b.out
[local: ~] time ./b.out
real 0m0.004s
user 0m0.000s
sys 0n0.001s

5. clean watermarks
-unmark [tool path]
• clean up the watermarks on the target compiler
[local: ~]$ sudo evaluate -mark-high /usr/bin/gcc
[local: ~]$ gcc ./main.c -o ./a.out # build your project as usual
[local: ~]$ time ./a.out # evaluate performance as usual
real 0m0.002s
user 0m0.000s
sys 0m0.000s
[local: ~] sudo evaluate -unmark /usr/bin/gcc

6. List all watermarks
-l, -list
• list all watermarks
[local: ~]$ evaluate -l
/// *///
......... /// ....... ... ... .... .... ... ......... ......... ... ...
///////// /// //// /// *//////*////,//// */// ///////// ///////// ///*///
/// /// //// /// */// /// /// /// */// //// /// /// ////
///////// /////// ///./// /// /// /// */// ///// ///////// ///
/// /// ///. ////// /// /// /// */// .//// /// ///
///////// /// ///* //// /// /// /// */// ///////// ///////// ///
/////
Copyright (c) 2017 Skymizer Taiwan, Inc.
------------------------------------------------------------------------------
• low-mark -O0
locate in file://home/luba/skymizer-dev/build-trunk-dbg/echo.sh
• high-mark -O3
locate in file://usr/bin/gcc
------------------------------------------------------------------------------

7. Set customized watermarks
-tool [path of tool] -mark [strings]
– -tool and -mark must be used together
– All strings after -mark will be treated as new watermarks
 After settings customized watermarks, you can revise the behavior of target tool
[local: ~]$ evaluate -tool `which gcc` -mark -flto -O3 -finline

8. Show more details: -v
-verbose=N
set up verbose level, from zero to five
-v -v -v …
The number of -v represents the verbose level as well
[local: ~]$ evaluate -v -v -v # set up verbose level to 3
[local: ~]$ evaluate -verbose=2 # set up verbose level to 2
[local: ~]$ evaluate -quiet # set up verbose level to 0

9. Download
File format URL OS
static tar ball https://dl.skymizer.com/evaluate-1.0-
3.tar.gz
Ubuntu, CentOS
static rpm https://dl.skymizer.com/evaluate-1.0-
3.rpm
CentOS and can be
installed in all RPM system
static deb https://dl.skymizer.com/evaluate-1.0-
3.deb
ubuntu 14 LTS