Most of this code is not templated methods. Rather, it implements over several types to call templated functions, which creates quite a bit of code. Rather than have all code using a reader recompile the code, just compile it once and put it in a library. |
||
---|---|---|
.. | ||
BenchmarkArrayTransfer.cxx | ||
BenchmarkAtomicArray.cxx | ||
BenchmarkCopySpeeds.cxx | ||
BenchmarkDeviceAdapter.cxx | ||
Benchmarker.h | ||
BenchmarkFieldAlgorithms.cxx | ||
BenchmarkFilters.cxx | ||
BenchmarkRayTracing.cxx | ||
BenchmarkTopologyAlgorithms.cxx | ||
CMakeLists.txt | ||
README.md |
BENCHMARKING VTK-m
TL;DR
When configuring VTM-m with CMake pass the flag -DVTKm_ENABLE_BENCHMARKS=1
. In the build directory you will see the following binaries:
$ ls bin/Benchmark*
bin/BenchmarkArrayTransfer* bin/BenchmarkCopySpeeds* bin/BenchmarkFieldAlgorithms*
bin/BenchmarkRayTracing* bin/BenchmarkAtomicArray* bin/BenchmarkDeviceAdapter*
bin/BenchmarkFilters* bin/BenchmarkTopologyAlgorithms*
Taking as an example BenchmarkArrayTransfer
, we can run it as:
$ bin/BenchmarkArrayTransfer -d Any
Parts of this Documents
Choosing devices
Taking as an example BenchmarkArrayTransfer
, we can determine in which
device we can run it by simply:
$ bin/BenchmarkArrayTransfer
...
Valid devices: "Any" "Serial"
...
Upon the Valid devices you can chose in which device to run the benchmark by:
$ bin/BenchmarkArrayTransfer -d Serial
Run a subset of your benchmarks
VTK-m benchmarks uses Google Benchmarks which allows you to choose a subset
of benchmaks by using the flag --benchmark_filter=REGEX
For instance, if you want to run all the benchmarks that writes something you would run:
$ bin/BenchmarkArrayTransfer -d Serial --benchmark_filter='Write'
Note you can list all of the available benchmarks with the option:
--benchmark_list_tests
.
Compare with baseline
VTM-m ships with a helper script based in Google Benchmarks compare.py
named compare-benchmarks.py
which lets you compare benchmarks using different
devices, filters, and binaries. After building VTM-m
it must appear on the
bin
directory within your build
directory.
When running compare-benchmarks.py
:
- You can specify the baseline benchmark binary path and its arguments in
--benchmark1=
- The contender benchmark binary path and its arguments in
--benchmark2=
- Extra options to be passed to
compare.py
must come after--
Compare between filters
When comparing filters, we only can use one benchmark binary with a single device as shown in the following example:
$ ./compare-benchmarks.py --benchmark1='./BenchmarkArrayTransfer -d Any
--benchmark_filter=1024' --filter1='Read' --filter2=Write -- filters
# It will output something like this:
Benchmark Time CPU Time Old Time New CPU Old CPU New
---------------------------------------------------------------------------------------------------------------------------------------------------------------
BenchContToExec[Read vs. Write]<F32>/Bytes:1024/manual_time +0.2694 +0.2655 18521 23511 18766 23749
BenchExecToCont[Read vs. Write]<F32>/Bytes:1024/manual_time +0.0212 +0.0209 25910 26460 26152 26698
Compare between devices
When comparing two benchmarks using two devices use the option benchmark
after --
and call ./compare-benchmarks.py
as follows:
$ ./compare-benchmarks.py --benchmark1='./BenchmarkArrayTransfer -d Serial
--benchmark_filter=1024' --benchmark2='./BenchmarkArrayTransfer -d Cuda
--benchmark_filter=1024' -- benchmarks
# It will output something like this:
Benchmark Time CPU Time Old Time New CPU Old CPU New
---------------------------------------------------------------------------------------------------------------------------------------------------
BenchContToExecRead<F32>/Bytes:1024/manual_time +0.0127 +0.0120 18388 18622 18632 18856
BenchContToExecWrite<F32>/Bytes:1024/manual_time +0.0010 +0.0006 23471 23496 23712 23726
BenchContToExecReadWrite<F32>/Bytes:1024/manual_time -0.0034 -0.0041 26363 26274 26611 26502
BenchRoundTripRead<F32>/Bytes:1024/manual_time +0.0055 +0.0056 20635 20748 21172 21291
BenchRoundTripReadWrite<F32>/Bytes:1024/manual_time +0.0084 +0.0082 29288 29535 29662 29905
BenchExecToContRead<F32>/Bytes:1024/manual_time +0.0025 +0.0021 25883 25947 26122 26178
BenchExecToContWrite<F32>/Bytes:1024/manual_time -0.0027 -0.0038 26375 26305 26622 26522
BenchExecToContReadWrite<F32>/Bytes:1024/manual_time +0.0041 +0.0039 25639 25745 25871 25972
Installing compare-benchmarks.py
compare-benchmarks.py
relies on compare.py
from Google Benchmarks which also
relies in SciPy
, you can find instructions here regarding its
installation.