`vtkm::VecFlat` is a wrapper around a `Vec`-like class that may be a
nested series of vectors. For example, if you run a gradient operation
on a vector field, you are probably going to get a `Vec` of `Vec`s that
looks something like `vtkm::Vec<vtkm::Vec<vtkm::Float32, 3>, 3>`. That
is fine, but what if you want to treat the result simply as a `Vec` of
size 9?
The `VecFlat` wrapper class allows you to do this. Simply place the
nested `Vec` as an argument to `VecFlat` and it will behave as a flat
`Vec` class. (In fact, `VecFlat` is a subclass of `Vec`.) The `VecFlat`
class can be copied to and from the nested `Vec` it is wrapping.
There is a `vtkm::make_VecFlat` convenience function that takes an
object and returns a `vtkm::VecFlat` wrapped around it.
The changelog is not quite accurate because it is claiming that all
virtual methods are removed when that is not quite the case.
Hopefully soon the changelog text will be accurate.
Previously, all atomic functions were stored in classes named
`AtomicInterfaceControl` and `AtomicInterfaceExecution`, which required
you to know at compile time which device was using the methods. That in
turn means that anything using an atomic needed to be templated on the
device it is running on.
That can be a big hassle (and is problematic for some code structure).
Instead, these methods are moved to free functions in the `vtkm`
namespace. These functions operate like those in `Math.h`. Using
compiler directives, an appropriate version of the function is compiled
for the current device the compiler is using.
We have made several improvements to adding data into an `ArrayHandle`.
## Moving data from an `std::vector`
For numerous reasons, it is convenient to define data in a `std::vector`
and then wrap that into an `ArrayHandle`. It is often the case that an
`std::vector` is filled and then becomes unused once it is converted to an
`ArrayHandle`. In this case, what we really want is to pass the data off to
the `ArrayHandle` so that the `ArrayHandle` is now managing the data and
not the `std::vector`.
C++11 has a mechanism to do this: move semantics. You can now pass
variables to functions as an "rvalue" (right-hand value). When something is
passed as an rvalue, it can pull state out of that variable and move it
somewhere else. `std::vector` implements this movement so that an rvalue
can be moved to another `std::vector` without actually copying the data.
`make_ArrayHandle` now also takes advantage of this feature to move rvalue
`std::vector`s.
There is a special form of `make_ArrayHandle` named `make_ArrayHandleMove`
that takes an rvalue. There is also a special overload of
`make_ArrayHandle` itself that handles an rvalue `vector`. (However, using
the explicit move version is better if you want to make sure the data is
actually moved.)
## Make `ArrayHandle` from initalizer list
A common use case for using `std::vector` (particularly in our unit tests)
is to quickly add an initalizer list into an `ArrayHandle`. Now you can
by simply passing an initializer list to `make_ArrayHandle`.
## Deprecated `make_ArrayHandle` with default shallow copy
For historical reasons, passing an `std::vector` or a pointer to
`make_ArrayHandle` does a shallow copy (i.e. `CopyFlag` defaults to `Off`).
Although more efficient, this mode is inherintly unsafe, and making it the
default is asking for trouble.
To combat this, calling `make_ArrayHandle` without a copy flag is
deprecated. In this way, if you wish to do the faster but more unsafe
creation of an `ArrayHandle` you should explicitly express that.
This requried quite a few changes through the VTK-m source (particularly in
the tests).
## Similar changes to `Field`
`vtkm::cont::Field` has a `make_Field` helper function that is similar to
`make_ArrayHandle`. It also features the ability to create fields from
`std::vector`s and C arrays. It also likewise had the same unsafe behavior
by default of not copying from the source of the arrays.
That behavior has similarly been depreciated. You now have to specify a
copy flag.
The ability to construct a `Field` from an initializer list of values has
also been added.
As a programming convenience, all `vtkm::cont::DataSet` written by
`vtkm::io::VTKDataSetWriter` were written as a structured grid. Although
technically correct, it changed the structure of the data. This meant that
if you wanted to capture data to run elsewhere, it would run as a different
data type. This was particularly frustrating if the data of that structure
was causing problems and you wanted to debug it.
Now, `VTKDataSetWriter` checks the type of the `CoordinateSystem` to
determine whether the data should be written out as `STRUCTURED_POINTS`
(i.e. a uniform grid), `RECTILINEAR_GRID`, or `STRUCTURED_GRID`
(curvilinear).
`assert` is supported on recent CUDA cards, but compiling it appears to be
very slow. By default, the `VTKM_ASSERT` macro has been disabled whenever
compiling for a CUDA device (i.e. when `__CUDA_ARCH__` is defined).
Asserts for CUDA devices can be turned back on by turning the
`VTKm_NO_ASSERT_CUDA` CMake variable off. Turning this CMake variable off
will enable assertions in CUDA kernels unless there is another reason
turning off all asserts (such as a release build).
ECP is no longer offering CI via NMC, so we can now remove the
infrastructure we used for that.
See merge request 2115 for how we are adding gitlab-ci at OLCF
de3bda373 Use deque instead of list for ArrayHandle queue
498d44548 Pass Token::Reference by value
c32c9e8e8 Fix deadlock when changing device during read
99e14ab8a Add proper enqueuing of Tokens for ArrayHandle
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2130
c0dee7402 make it explicit that we are using 64-bit unsigned integer in bit op
34f350588 Added changelog
e9f584a91 ArrayHandleRandomUniformReal
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Kenneth Moreland <kmorel@sandia.gov>
Merge-request: !2116
An issue that was identified for the thread safety of `ArrayHandle` is
that if several threads are waiting to use an `ArrayHandle`, there might
be an expectation of the order in which the operations happen. For
example, if one thread is modifying the contents of an `ArrayHandle` and
another is reading those results, we would need the first one to start
before the second one.
To solve this, a queue is added to `ArrayHandle` such that when waiting
to read or write an `ArrayHandle` the `Token` has to be at the top of
the queue in addition to other requirements being met.
Additionally, an `Enqueue` method is added to add a `Token` to the queue
without blocking. This allows a control thread to queue the access and
then spawn a thread where the actual work will be done. As long as
everything is enqueued on the main thread, the operations will happen in
the expected order.
Initially, the probe filter would simply not set a value if a sample was
outside the input `DataSet`. This is not great as the memory could be
left uninitalized and lead to unpredictable results. The testing
compared these invalid results to 0, which seemed to work but is
probably unstable.
This was partially fixed by a previous change that consolidated to
mapping of cell data with a general routine that permuted data. However,
the fix did not extend to point data in the input, and it was not
possible to specify a particular invalid value.
This change specifically updates the probe filter so that invalid values
are set to a user-specified value.
544a078cd Remove use of deprecated policies in examples
06f5119c2 Fix deprecation warning
f29a4712b Correct field types for ComputeMoments filter
a20ec03d0 Disable proxies in filter benchmark
72cd0107e Deprecate Execute with policy
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2093
There have been several new features that were merged without
appropriate documentation in the changelogs. This adds some
new changelogs for some of these new features.
The version of `Filter::Execute` that takes a policy as an argument is now
deprecated. Filters are now able to specify their own fields and types,
which is often why you want to customize the policy for an execution. The
other reason is that you are compiling VTK-m into some other source that
uses a particular types of storage. However, there is now a mechanism in
the CMake configuration to allow you to provide a header that customizes
the "default" types used in filters. This is a much more convenient way to
compile filters for specific types.
One thing that filters were not able to do was to customize what cell sets
they allowed using. This allows filters to self-select what types of cell
sets they support (beyond simply just structured or unstructured). To
support this, the lists `SupportedCellSets`, `SupportedStructuredCellSets`,
and `SupportedUnstructuredCellSets` have been added to `Filter`. When you
apply a policy to a cell set, you now have to also provide the filter.
To simplify reproducing docker based CI workers locally, VTK-m has python program that handles all the
work automatically for you.
The program is located in `[Utilities/CI/reproduce_ci_env.py ]` and requires python3 and pyyaml.
To use the program is really easy! The following two commands will create the `build:rhel8` gitlab-ci
worker as a docker image and setup a container just as how gitlab-ci would be before the actual
compilation of VTK-m. Instead of doing the compilation, instead you will be given an interactive shell.
```
./reproduce_ci_env.py create rhel8
./reproduce_ci_env.py run rhel8
```
To compile VTK-m from the the interactive shell you would do the following:
```
> src]# cd build/
> build]# cmake --build .
```
Previously, the PolicyDefault used to compile all the filters was hard-
coded. The problem was that if any external project that depends on VTK-
m needs a different policy, it had to recompile everything in its own
translation units with a custom policy.
This change allows an external project provide a simple header file that
changes the type lists used in the default policy. That allows VTK-m to
compile the filters exactly as specified by the external project.
Currently, VTK-m is using C++11. However, it is often useful to use
features in the `std` namespace that are defined for C++14 or later. We
can provide our own versions (sometimes), but it is preferable to use
the version provided by the compiler if available.
There were already some examples of defining portable versions of C++14
and C++17 classes in a `vtkmstd` namespace, but these were sprinkled
around the source code.
There is now a top level `vtkmstd` directory and in it are header files
that provide portable versions of these future C++ classes. In each
case, preprocessor macros are used to select which version of the class
to use.
Previously, when a ReadPortal or a WritePortal was returned from an
ArrayHandle, it had wrapped in it a Token that was attached to the
ArrayHandle. This Token would prevent other reads and writes from the
ArrayHandle.
This added safety in the form of making sure that the ArrayPortal was
always valid. Unfortunately, it also made deadlocks very easy. They
happened when an ArrayPortal did not leave scope immediately after use
(which is not all that uncommon).
Now, the ArrayPortal no longer locks up the ArrayHandle. Instead, when
an access happens on the ArrayPortal, it checks to make sure that
nothing has happened to the data being accessed. If it has, a fatal
error is reported to the log.
This commit also:
- Removes a corner case not longer used at ArrayPortalGroupVecVariable::get
- Changes doc regarding the number of offset elements in the input
array handler of ConvertNumComponentsToOffsets.
- Updates invokation of make_ArrayGroupVectVariable in multiple files
- Adds its corresponding changelog entry
To get a portal to access ArrayHandle values in the control
environment, you now use the ReadPortal and WritePortal methods.
The portals returned are wrapped in an ArrayPortalToken object
so that the data between the portal and the ArrayHandle are
guaranteed to be consistent.
Also discovered that many C++ compilers have trouble giving warnings
for partial specialization of classes marked as deprecated. Fix
the problem by instead deprecating the items in the class.
The convenience functions `ArrayPortalToIteratorBegin()` and
`ArrayPortalToIteratorEnd()` wouldn't detect specializations of
`ArrayPortalToIterators<PortalType>` since the specializations aren't
visible when the `Begin`/`End` functions are declared.
Since the CUDA iterators rely on a specialization, the convenience
functions would not compile on CUDA.
Now, instead of specializing `ArrayPortalToIterators` to provide custom
iterators for a particular portal, the portal may advertise custom
iterators by defining `IteratorType`, `GetIteratorBegin()`, and
`GetIteratorEnd()`. `ArrayPortalToIterators` will detect such portals
and automatically switch to using the specialized portals.
This eliminates the need for the specializations to be visible to the
convenience functions and allows them to be usable on CUDA.
Previously we relied on CMake's compiler detection module to build the
macros for using the deprecated attribute. However, CMake created macros
for pre-C++14 versions of the feature, which do not work in all cases.
Also, we have the need to be able to suppress deprecation warnings when
we are implementing a deprecated thing. Since we have to query compilers
ourself, we might as well figure out if the deprecated attribute we want
is supported.
Worst case is that we won't support deprecation warnings everywhere we
could. That will not create incorrect code and we can always add that
later.
The `VTKM_DEPRECATED` macro allows us to remove (and usually replace)
features from VTK-m in minor releases while still following the conventions
of semantic versioning. The idea is that when we want to remove or replace
a feature, we first mark the old feature as deprecated. The old feature
will continue to work, but compilers that support it will start to issue a
warning that the use is deprecated and should stop being used. The
deprecated features should remain viable until at least the next major
version. At the next major version, deprecated features from the previous
version may be removed.
If a worklet doesn't explicitly state an ExecutionSignature, VTK-m
assumes the worklet has no return value, and each ControlSignature
argument is passed to the worklet in the same order.
For example if we had this worklet:
```cxx
struct DotProduct : public vtkm::worklet::WorkletMapField
{
using ControlSignature = void(FieldIn, FieldIn, FieldOut);
using ExecutionSignature = void(_1, _2, _3);
template <typename T, vtkm::IdComponent Size>
VTKM_EXEC void operator()(const vtkm::Vec<T, Size>& v1,
const vtkm::Vec<T, Size>& v2,
T& outValue) const
{
outValue = vtkm::Dot(v1, v2);
}
};
```
It can be simplified to be:
```cxx
struct DotProduct : public vtkm::worklet::WorkletMapField
{
using ControlSignature = void(FieldIn, FieldIn, FieldOut);
template <typename T, vtkm::IdComponent Size>
VTKM_EXEC void operator()(const vtkm::Vec<T, Size>& v1,
const vtkm::Vec<T, Size>& v2,
T& outValue) const
{
outValue = vtkm::Dot(v1, v2);
}
};
VTK-m now provides the following filters with the default policy
as part of the vtkm_filter library:
- CellAverage
- CleanGrid
- ClipWithField
- ClipWithImplicitFunction
- Contour
- ExternalFaces
- ExtractStructured
- PointAverage
- Threshold
- VectorMagnitude
By building these as a library we hope to provide faster compile
times for consumers of VTK-m when using common configurations.
05b679250 Merge branch 'master' into tangle_source
a6c044df9 added changelog
0d818701c put VTKM_SOURCE_EXPORT in .cxx
e0aae7d86 remove EXPORT from .cxx
4d7de67ee use VTKM_SOURCE_EXPORT
cd49136d5 add copyright notice, add installation of header
0c094a568 used the Tangle source
aeb8877a9 add newline at EOF, minor change based review
...
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !1804
Consumers of VTK-m when enabling of dropping of unused functions
will see VTK-m functions dropped. Previously this didn't happen
as VTK-m didn't build object files with the correct flags for this.
By allowing the linker to remove unused symbols we see a significant
saving the file size of VTK-m tests, examples, and benchmarks.
An OpenMP build of the tests and benchmarks goes from 168MB to
141MB which is roughly a 16% filesize reduction.
Initially I had presumed that these changes would increase link times.
But in measurements the total wall time for compilation of VTK-m has
stayed about the same ( seeing a decrease of 1.5% ). Presumably the
increased computation is offset by the reduction in file writing.
Instead of having to be called for each target you can now
pass multiple targets at once. Do note that if you pass multiple
targets you will need to pass all the sources from those targets
that need to be 'device' compiled.
Rather than do a CastAndCall on all possible field types when calling a
worklet with two fields (where they all typically get cast to the same
type as the primary field), use the new mechanism with
ArrayHandleMultiplexer to create one code path.
Also update the ApplyPolicy to accept the Field type, which is used to
determine any additional storage types to support.
This is done through a new version of ApplyPolicy. This version takes
a type of the array as its first template argument, which must be
specified.
This requires having a list of potential storage to try. It will use
that to construct an ArrayHandleMultiplexer containing all potential
types. This list of storages comes from the policy. A StorageList
item was added to the policy.
Types are automatically converted. So if you ask for a vtkm::Float64 and
field contains a vtkm::Float32, it will the array wrapped in an
ArrayHandleCast to give the expected type.
Added a BaseComponentType to VecTraits that recursively finds the base
(non-Vec) type of a Vec. This is useful when dealing with potentially
nested Vec's (e.g. Vec<Vec<T, M>, N>) and you need to keep the structure
but know the base type.
Also added a couple of templates for keeping the structure but changing
the type. These are ReplaceComponentType and ReplaceBaseComponentType.
These allow you to create new Vec's with the same structure as the query
Vec but with differen component types.
This behaves just like `ScanExclusive`, but rather than returning the
total sum, it is appended to the end of the output array.
This is in preparation for the CellSetExplicit refactoring described in
issue #408.
The `MultiBlock` class has been renamed to `PartitionedDataSet`, and its API
has been refactored to refer to "partitions", rather than "blocks".
Additionally, the `AddBlocks` method has been changed to `AppendPartitions` to
more accurately reflect the operation performed. The associated
`AssignerMultiBlock` class has also been renamed to
`AssignerPartitionedDataSet`.
This change is motivated towards unifying VTK-m's data model with VTK. VTK has
started to move away from `vtkMultiBlockDataSet`, which is a hierarchical tree
of nested datasets, to `vtkPartitionedDataSet`, which is always a flat vector
of datasets used to assist geometry distribution in multi-process environments.
This simplifies traversal during processing and clarifies the intent of the
container: The component datasets are partitions for distribution, not
organizational groupings (e.g. materials).
Ref #405
By removing the ability to have multiple CellSets in a DataSet
we can simplify the following things:
- Cell Fields now don't require a CellSet name when being constructed
- Filters don't need to manage what the active cellset is
For polygon cell shapes (that are not triangles or quadrilaterals),
interpolations are done by finding the center point and creating a
triangle fan around that point. Previously, the gradient was computed in
the same way as interpolation: identifying the correct triangle and
computing the gradient for that triangle.
The problem with that approach is that makes the gradient discontinuous
at the boundaries of this implicit triangle fan. To make things worse,
this discontinuity happens right at each vertex where gradient
calculations happen frequently. This means that when you ask for the
gradient at the vertex, you might get wildly different answers based on
floating point imprecision.
Get around this problem by creating a small triangle around the point in
question, interpolating values to that triangle, and use that for the
gradient. This makes for a smoother gradient transition around these
internal boundaries.
