Sandia National Laboratories recently changed management from the
Sandia Corporation to the National Technology & Engineering Solutions
of Sandia, LLC (NTESS). The copyright statements need to be updated
accordingly.
We previously included windows.h in numerous locations using different
techniques to guard against bringing in parts of the file that are bad
(min/max macros, etc). This solves the problem by consistently using
vtkm/internal/Windows.h to setup everything.
Change the VTKM_CONT_EXPORT to VTKM_CONT. (Likewise for EXEC and
EXEC_CONT.) Remove the inline from these macros so that they can be
applied to everything, including implementations in a library.
Because inline is not declared in these modifies, you have to add the
keyword to functions and methods where the implementation is not inlined
in the class.
The benchmarking header files were not listed. Not a huge deal since
these files do not need to be installed, but they should be listed
anyway. Changed the vtkm_save_benchmarks CMake macro to be able to list
headers.
Also moved everything from BenchmarkDeviceAdapter.h to
BenchmarkDeviceAdapter.cxx. Since this code shouldn't need to be
included by anything except this benchmark, there is no need to have it
in a header file. Plus, the build changes would mean that any change in
the header (where most of the source was) could cause all code in this
directory to recompile. I do not want to set that precedent.
- A warm up run is done and not timed to allow for any allocation of
room for output data without accounting for it in the run times.
Previously this time spent allocating memory would be included in the
time we measured for the benchmark.
- Benchmarks are run multiple times and we then compute some statistics
about the run time of the benchmark to give a better picture of the
expected run time of the function. To this end we run the benchmark
either 500 times or for 1.5s, whichever comes sooner (though these are
easily changeable). We then perform outlier limiting by Winsorising the
data (similar to how Rust's benchmarking library works) and print out
the median, mean, min and max run times along with the median absolute
deviation and standard deviation.
- Because benchmarks are run many times they can now perform some
initial setup in the constructor, eg. to fill some test input data
array with values to let the main benchmark loop run faster.
- To allow for benchmarks to have members of the data type being
benchmarked the struct must now be templated on this type, leading to
a bit of awkwardness. I've worked around this by adding the
`VTKM_MAKE_BENCHMARK` and `VTKM_RUN_BENCHMARK` macros, the make
benchmark macro generates a struct that has an `operator()` templated on
the value type which will construct and return the benchmark functor
templated on that type. The run macro will then use this generated
struct to run the benchmark functor on the type list passed. You can
also pass arguments to the benchmark functor's constructor through the
make macro however this makes things more awkward because the name of
the MakeBench struct must be different for each variation of constructor
arguments (for example see `BenchLowerBounds`).
- Added a short comment on how to add benchmarks in
`vtkm/benchmarking/Benchmarker.h` as the new system is a bit different
from how the tests work.
- You can now pass an extra argument when running the benchmark suite to
only benchmark specific functions, eg. `Benchmarks_TBB
BenchmarkDeviceAdapter ScanInclusive Sort` will only benchmark
ScanInclusive and Sort. Running without any extra arguments will run all
the benchmarks as before.
