Previously, `ArrayHandleSOA` worked with any value type that supported
`VecTraits`. That means that `ArrayHandleSOA` worked with scalar types
like `Float32`. However, for scalar types, the behavior is essentially
the same as `ArrayHandleBasic`, but with lots of extra templating and
code generation.
Although there is nothing _wrong_ with allowing `ArrayHandleSOA` holding
a scalar, there is no real reason to either (other than likely template
convenience). Generally, there is nothing wrong with supporting it, but
if you want to support `ArrayHandleSOA` in places where types are not
known (e.g. `Field`), then templating tends to iterate over the cross of
all supported types with all supported storage. That means such code
will automatically generate a bunch of code for `ArrayHandleSOA` with
scalars even if there is no reason for those code paths.
So, we can just disable the use of `ArrayHandleSOA` with scalars to
allow us to use `ArrayHandleSOA` as a default storage without creating
all these useless code paths.
`ArrayHandleRecombineVec` is used when calling
`ExtractArrayFromComponents` from `UnkownArrayHandle`. It needs special
handling with the `Fetch` for an output array.
This method allows you to extract an `ArrayHandle` from
`UnknownArrayHandle` when you only know the base component type.
Also removed the `Read/WritePortalForBaseComponentType` method
from `UnknownArrayHandle`. This functionality is subsumed by
`ExtractArrayFromComponents`.
The primary purpose of `ArrayHandleRecombineVec` is to take arrays
returned from `ArrayExtractComponent` and recombine them again into a
single `ArrayHandle` that has `Vec` values.
The previous implementation of test_equal_ArrayHandles was several
templates that had to be resolved by any test that used them, which
could be costly for unknown array types. Simplify this a bit by moving
the implementation of testing unknown arrays into a library.
Another advantage of the new implementation is that is handles more
cases. Thus, you should not need to `ResetTypes` on the unknown/
uncertain arrays.
Some of the `test_equal` functions return a `TestEqualResult`
instead of a `bool` to capture more information about what
the error was. Unfortunately, using this was awkward because
you couldn't just call the `test_equal_*` inside of a
`VTKM_TEST_ASSERT`. Rather, you would have to do the comparison
and then check it.
This change adds an overload to `VTKM_TEST_ASSERT` that specifically
takes a `TestEqualResult`, checks its condition, and prints out
the contained messages. Thus, your command can just look like
`VTKM_TEST_ASSERT(test_equal_ArrayHandles(...));` and it will
provide the additional information.
These helper functions were in vtkm::cont::testing, but that made them
hard to discover (and I personally kept forgetting about them). Move
them to the top namespace so that IDE of test_equal will helpfully
remind us of these other test functions.
Recent merge requests !2354 and !2356 both edited ArrayHandleView. Git
successfully merged the changes, but the changes were still incompatible
with each other, causing an unexpected compile error on master. This
fixes the issue.
Previously, the `MapFieldMergeAverage` and `MapFieldPermutation` helper
function had to iterate over every possible type and create a separate
code path. This change uses the new extract component functionality to
create separate code paths only for different component types. This both
requires less code (the common filter library dropped from 66MB to 42MB
on my Mac) and covers more cases (such as `Vec`s larger than 4
components).
To make the implementation easier, `UnknownArrayHandle` now can create a
new `UnknownArrayHandle` of the same `ValueType` but with the basic
storage (so you can work with read-only storage) and the ability to
allocate the unknown array.
This allows you to handle just about every type of array with about 10
basic types. It allows you to ignore both the size of `Vec`s and the
actual storage of the data.
The typical use case of `ArrayHandleStride` is to flexibly point into
another array, often looking at a single component in an array. It is
typical that multiple things will be accessing the same array, and bad
things could happen as they all try to resize. There was some code to
try to figure out what the size of the original array was, but it was
fragile.
It is safer for now to disable the behavior altogether. If a use case
pops up, we can reintroduce the code.
`ArrayExtractComponent` allows you to get a component of an array.
Unlike `ArrayHandleExtractComponent`, the type you get is always the
same: an `ArrayHandleStride`. This way, you can get an array that
contains the data of an extracted component with less templating and
potentially dramatically reduce the amount of code generated (although
some runtime integer arithmetic is added).
7475c318b VTK-m now uses CMake's future HIP lang for Kokkos+HIP
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Sujin Philip <sujin.philip@kitware.com>
Merge-request: !2351
7811cc4b1 Add standard support for read-only storage
a6b9d5c49 Add tests for ReleaseResources of fancy arrays
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2345
Many of the fancy `ArrayHandle`s are read-only and therefore connot
really create write portals. Likewise, many `ArrayHandle`s (both read-
only and read/write) have no way to resize themselves. In this case,
implementing the `CreateWritePortal` and `ResizeBuffers` methods in the
`Storage` class was troublesome. Mostly they just threw an exception,
but they also sometimes had to deal with cases where the behavior was
allowed.
To simplify code for developers, this introduces a pair of macros:
`VTKM_STORAGE_NO_RESIZE` and `VTKM_STORAGE_NO_WRITE_PORTAL`. These can
be declared in a `Storage` implementation when the storage has no viable
way to resize itself and create a write portal, respectively.
Having boilerplate code for these methods also helps work around
expected behavior for `ResizeBuffers`. `ResizeBuffers` should silently
work when resizing to the same size. Also `ResizeBuffers` should behave
well when resizing to 0 as that is what `ReleaseResources` does.
The `ReleaseResources` method should work for all arrays (even if it
effectively does not do anything). However, the implementation of
`ReleaseResources` is generally to call `Allocate` with 0. Several fancy
arrays balk at this because it is resizing a read-only array. There
should be an exception for this.
This class was used indirectly by the old `ArrayHandle`, through
`ArrayHandleTransfer`, to move data to and from a device. This
functionality has been replaced in the new `ArrayHandle`s through the
`Buffer` class (which can be compiled into libraries rather than make
every translation unit compile their own template).
This commit removes `ArrayManagerExecution` and all the implementations
that the device adapters were required to make. None of this code was in
any use anymore.
`ArrayTransfer` is used with the old `ArrayHandle` style to move data
between host and device. The new version of `ArrayHandle` does not use
`ArrayTransfer` at all because this functionality is wrapped in `Buffer`
(where it can exist in a precompiled library).
Once all the old `ArrayHandle` classes are gone, this class will be
removed completely. Although all the remaining `ArrayHandle` classes
provide their own versions of `ArrayTransfer`, they still need the
prototype to be defined to specialize. Thus, the guts of the default
`ArrayTransfer` are removed and replaced with a compile error if you try
to compile it.
78aa463da Fix compile error when template parameter shadows superclass
905b5a02b Call PrepareForControl to get transform for undefined device
650e416cc Convert ArrayHandleTransform to new buffer-style-array
76880dd8c Enable ExecutionObjectBase::PrepareForExecution for DeviceAdapterId
b0146b1e4 Fix issue with CreateBuffers function
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2341
I'm too lazy to look up the C++ spec, but it seems like template
parameter names would shadow the same names from a superclass.
Apparently that is not the case for Visual Studio. The `Storage` for the
cast array inherits from the `Storage` of the transform array. The
latter declares a private type named `SourceStorage`, and that is being
used instead of the former's `SourceStorage` template parameter. You
then get an error for accessing a private member that you did not want
in the first place.
Fix the problem by changing the name of the template parameter.
A relatively new convention is to use `DeviceAdapterTagUndefined` when
referring to the host. When getting the transform functor for the
control, detect when the device is undefined and call
`PrepareForControl` instead of `PrepareForExecution` with the undefined
device.
Previously, the implementation of `ExecutionObjectBase` required you to
call `PrepareForExecution` with a specific `DeviceAdapterTag`. If you
passed in a `DeviceAdapterId`, which is the superclass of all device
adapter tags, you would get a compile error.
As we move to the new `ArrayHandle` style, it is more common to handle a
straight `DeviceAdapterId` without templating. Perhaps there will be a
time when the templated version is removed.
1fb4c0499 Rename ArrayHandle::FakeBuffer to BufferAsStorageWrapper
05fea4d17 Bridge old-style ArrayHandle to new-style
62c509520 Save host portal for ArrayHandleVirtual in array instead of portal
bc3a7d8e2 Restore composite of transform in UnitTestParticleDensity.cxx
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2339
The new-style `ArrayHandle` uses `Buffer` objects to manage data. Thus,
when one is decorating the other, it expects to find the `Buffer`
objects, which the old-style `ArrayHandle`s do not have. To make the two
work together, fake buffers in the old-style arrays.
The buffers in old-style arrays are empty, but have metadata that points
back to the `ArrayHandle.
Previously, when you got a host/control portal from
`ArrayHandleVirtual`, you got a version of an `ArrayPortal` that manged
its own reference to the virtual structure that was implementing the
portal. This was not done for device/execution portals because those
objects could not do the appropriate resource management from the
execution environment.
Rather than releasing the host object to the portal, keep the host
object managed by `StorageVirtual`. This allows the control and
execution portals to be the same type, which we will need to be friendly
with new array objects.
1. The code now works without CUDA_LAUNCH_BLOCKING set by using explicit
synchronizations where required.
2. The code has also been modified to use thread specific memory spaces,
which for Kokkos' Cuda backend means per thread streams.
fb130d42d Remove test for ArrayHandleCompositeVector with bad lengths
7cbf40d5b Back out of using a composite of transform arrays
7ff1a690d Support ArrayHandleCompositeVector with 1 component
5391e353a Convert ArrayHandleExtractComponent to new buffer-style array
22fac15b3 Convert ArrayHandleCompositeVector to new buffer-style array
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2330
As we update the `ArrayHandle`s to the new buffer style, simplify
`ArrayHandleSwizzle` by implementing it on top of
`ArrayHandleTransform`. Also, introduce some C++11 template concepts to
handle variable arguments better.
In this conversion, some features have been removed. Previously if the
swizzle dropped some components, you were still allowed to write to the
array. This would be an in place write that would only update the passed
components. This feature has been removed and if any components are
dropped, the array becomes read only. If all components are swizzled,
then the array becomes writable (through templating).
Another lost feature is that the swizzle map is no longer checked for
correctness. The array handle just assumes that the indices are correct.
It should be noted that it does not look like `ArrayHandleSwizzle` is
actually used anywhere. If it is not used by any customers, we may
want to consider deprecating the class.
The old version of `ArrayHandleCompositeVector` would check (at
construction) to make sure that all the arrays had the same length. This
check has been removed because (1) it did not always catch all problems
(such as if the input arrays change), (2) debug builds check for out-of-
range array accesses anyway, and (3) it required a lot of extra template
expansion.
Because that check has been removed, also remove the part of the
`ArrayHandleCompositeVector` test that checks this check.
When `ArrayHandleCompositeVector` has only 1 component, it is supposed
to have a special template that uses the base value type rather than a
`Vec` of that type. However, the `Storage` with the value type was
missing. I'm not sure how we weren't getting compile errors before, but
moving to the new buffer arrays seems to bring about the expected error.
The `StorageImplicit.h` header is no longer used. However, just removing
it causes compiles to fail if they include it. To make the transition
better, re-add this file, but cause a deprecation warning if it is
included.
It should be noted that a break in backward compatibility is introduced.
The implementation class passed to `ArrayHandleDecorator` changed the
specification of `AllocateSourceArrays`. Thus, decorators that were
previously allocatable now no longer will be until that method is
updated to the new form.
A large portion of the VTK-m filters are now compiled into the
vtkm_filter library. These pre-built filters now don't include
the respective hxx file to remove the amount of template
instantiation users do.
To verify that this removal reduces compiler memory ( and maybe time)
I profiled the compiler while it building filter tests in debug mode.
Here is a selection of results:
```
CleanGrid 10.25s => 9.01s, 901MB => 795MB
ExternalFaces 13.40s => 5.96s, 1122MB => 744MB
ExtractStructured 4.69s => 4.75s, 492MB => 492MB
GradientExplicit 22.97s => 5.88s, 1296MB => 740MB
```
The array given to `ArrayHandleGroupVec` should really have a number of
values that divides evenly among the `Vec`s. Previously, if this was not
the case, an error exception was thrown. These changes allow this
condition. Extra components are dropped and a warning is logged.
Most `ArrayHandle` implementations must create an
`std::vector<vtkm::cont::internal::Buffer>` object. It is most helpful
to create it in the constructor when initializing the superclass (so the
superclass does not have to create it's own). Added a `CreateBuffers`
convenience function to make it easy to build these vectors.
MR !2311 fixed an issue where some base C types were not recognized by
`SerializableTypeString`. However, the fix was such that different types
with the same layout had the same string. For example `char` and
`signed char` both were given the string `I8`. That meant that the
serialization/deseralization would work, but the deserialization could
change the type. That could cause problems if two arrays were expected
to have the same type but did not.
This change undoes much of MR !2311 and redoes it so that the types are
correct.
C++ template considers some types different even though they have the
exact same format. For example `int`, `long`, and `long long` all match
different types even though they all represent either signed 32-bit ints
or signed 64-bit ints.
List all these possible types in
`TypeListScalarAll`.
Previously, `SerializableTypeString` was specialized only for the VTK-m
defined int types (`Int8`, `UInt8`, `Int16`, etc.). You would think this
would cover all the base int types, but in fact C++ defines multiple int
types that have the exact same bit layout. These duplicates will not
match each other when matching template specializations. For example,
`char` is either the same as `signed char` or `unsigned char`, but will
match neither. Likewise, `long` is the same as either `int` or `long
long`, but all these types will match different template
specializations.
Now all integer types are covered.
The specialization for SerializableTypeString for ArrayHandleCast tried
to get the serialization for the base type and the storage tag. This is
wrong since the storage tag itself is not seralizable nor has a type
string. Instead, just record the type name for the sub ArrayHandle
itself.
Previously you had to create a special virtual object to place in the
`Buffer`'s metadata. This added a lot of difficulty that is probably
unnecessary.
Instead, just have `Buffer` hold an arbitrary object and some simple
functions to copy and delete it. There may be issues with type safety
across translation units, but we'll deal with that when/if we run into
it.
ArrayRangeCompute.hxx was removed, but other code may be including it.
The file was replaced. It now just includes its replacement
(ArrayRangeComputeTemplate.h) and forces a deprecation warning.
e3dfa4891 Do not attempt to move non-trivial objects in Variant
8e11b3ecd Remove ArrayPortalCheck
21db210a7 Make separate exec and cont versions of Variant
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2274
This portal only works on the control environment, which means it cannot
work with the new `ArrayHandle` type. Recent changes to
`ArrayHandleMultiplexer` also do not allow this, so just remove it
rather than try to fix it.
The `Variant` class is templated to hold objects of other types.
Depending on whether those objects of are meant to be used in the
control or execution side, the methods on `Variant` might need to be
declared with (or without) special modifiers. We can sometimes try to
compile the `Variant` methods for both host and device and ask the
device compiler to ignore incompatibilities, but that does not always
work.
To get around that, create two different implementations of `Variant`.
Their API and implementation is exactly the same except one declares its
methods with `VTKM_CONT` and the other its methods `VTKM_EXEC`.
Instead, always use precompiled versions of range computing. This means
you won't be able to specify the type. Currently, types are limited to
scalars vecs up to size 4.
The main motivation for this change is to allow you to include Field.h
with a non-device compiler. This is an important feature for our
customers.
I plan in the future to implement a mechanism to pull out a component of
most ArrayHandle's as a single array. This would enable us to support a
precompiled version that can compute the range of arbitrarily sized
Vecs.
When `DeviceAdapterAlgorithm::ScheduleTask` was called directly (i.e.
not through `Schedule`), nothing was added to the log. Adding
`VTKM_LOG_SCOPE` to these methods so that all scheduling is added to the
performance log.
The old atomic compare and swap operations (`vtkm::AtomicCompareAndSwap`
and `vtkm::exec::AtomicArrayExecutionObject::CompareAndSwap`) had an
order of arguments that was confusing. The order of the arguments was
shared pointer (or index), desired value, expected value. Most people
probably assume expected value comes before desired value. And this
order conflicts with the order in the `std` methods, GCC atomics, and
Kokkos.
Change the interface of atomic operations to be patterned off the
`std::atomic_compare_exchange` and `std::atomic<T>::compare_exchange`
methods. First, these methods have a more intuitive order of parameters
(shared pointer, expected, desired). Second, rather than take a value
for the expected and return the actual old value, they take a pointer to
the expected value (or reference in `AtomicArrayExecutionObject`) and
modify this value in the case that it does not match the actual value.
This makes it harder to mix up the expected and desired parameters.
Also, because the methods return a bool indicating whether the value was
changed, there is an additional benefit that compare-exchange loops are
implemented easier.
For example, consider you want to apply the function `MyOp` on a
`sharedValue` atomically. With the old interface, you would have to do
something like this.
```cpp
T oldValue;
T newValue;
do
{
oldValue = *sharedValue;
newValue = MyOp(oldValue);
} while (vtkm::AtomicCompareAndSwap(sharedValue, newValue, oldValue) != oldValue);
```
With the new interface, this is simplfied to this.
```cpp
T oldValue = *sharedValue;
while (!vtkm::AtomicCompareExchange(sharedValue, &oldValue, MyOp(oldValue));
```
The `ReleaseResourcesExecution` method makes changes by booting data off
of the execution environment. But logically the array does not change.
It remains the same size with the same contents. Thus, it makes sense
for this to be a const method.
Also modified some deprecated methods a bit to remove some unnecessary
templates.
f5c5b6188 Log the computation of ranges in fields and arrays
bef55f8ee Log the conversion of num components to offsets
420b9d397 Log the building of arrays for scatters and masks
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2283
The new name reflects better what the underlying algorithm does. It also
helps prevent confusion about what types of data the locator is good
for. The old name suggested it only worked for structured grids, which
is not the case.
We would really like to be able to include `vtkm::cont::ColorTable` in
such a way that you don't have to compile device code (unless you are
actually compiling functions for the device). Thus, the `Map` functions
of `ColorTable` were in a special `ColorTable.hxx` that contains the
"implementation" for `ColorTable`.
That is confusing to many users. It is more clear to simply have `.h`
headers that do a specific thing. To achieve these two goals, the `Map`
functionality of `ColorTable` is separated out into its own header file.
So you don't need to be using a device compiler just to use `ColorTable`
(including `ColorTable.h`), but you do need to use a device compiler if
mapping values to colors (including `ColorTableMap.h`).
6a3ba4291 Fix warning about unused function
c6a4f9b79 Fix warning about return value
9465d2611 Adjust TransferFunction worklet to get color table as ExecObj argument
11996f133 Remove virtual methods from ColorTable
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2261
Previously, the `TransferFunction` worklet required you to call
`PrepareForExecution` on the `ColorTable` and give that to the
constructor. Bad things can happen if the `ColorTable` used in the
constructor does not match the device the worklet actually gets invoked
on. Change this to make the `ColorTable` a worklet argument of type
`ExecObj`. The dispatcher will automatically call
`ColorTable::PrepareForInput` and guarantee that the devices match.
Virtual methods are being deprecated, so remove their use from the
ColorTable classes. Instead of using a virtual method to look up a value
in the ColorTable, we essentially use a switch statement. This change
also simplified the code quite a bit.
The execution object used to use pointers to handle the virtual objects.
That is no longer necessary, so a simple `vtkm::exec::ColorTable` is
returned for execution objects. (Note that this `ColorTable` contains
pointers that are specific for the particular device.) This is a non-
backward compabible change. However, the only place (outside of the
`ColorTable` implementation itself) was a single worklet for converting
scalars to colors (`vtkm::worklet::colorconversion::TransferFunction`).
This is unlikely to affect anyone.
I also "fixed" some names in enum structs. There has been some
inconsistencies in VTK-m on whether items in an enum struct are
capitolized or camel case. We seem to moving toward camel case, so
deprecate some old names.
482266b44 Particle: explicitly make the copy ctor and assignment op
1037aa756 AssignerPartitionedDataSet: mark dtor as override
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2259
One of the features of `UnknownArrayHandle` is that it allows you to
query how many `Vec` components each value has without resolve the type
of the array. The functionality to implement this failed if you tried to
store an `ArrayHandle` that stored `Vec`-like objects with `Vec` sizes
that varied from value to value (i.e. an `ArrayHandleGroupVecVariable`).
Storing such an array in `UnknownArrayHandle` might not be the best
idea, but it should probably work. This change allows you to store such
an array. If you try to query the number of components, you will get 0.
e706880d7 Fix unnecessary deprecation warnings in visual studio
f7cc03107 Fix deprecated warnings
b27e4c7ea Ignore files for deprecated virtual classes for SourceInInstall test
284774cf4 Turn on "no virtual" option on some CI builds
cd08fd499 Add changelog for removing virtual methods
63ef84ed7 Optionally remove all use of ArrayHandleVirtual
e11f612ad Deprecate ArrayHandleVirtual
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2256
As we remove more and more virtual methods from VTK-m, I expect several
users will be interested in completely removing them from the build for
several reasons.
1. They may be compiling for hardware that does not support virtual
methods.
2. They may need to compile for CUDA but need shared libraries.
3. It should go a bit faster.
To enable this, a CMake option named `VTKm_NO_DEPRECATED_VIRTUAL` is
added. It defaults to `OFF`. But when it is `ON`, none of the code that
both uses virtuals and is deprecated will be built.
Currently, only `ArrayHandleVirtual` is deprecated, so the rest of the
virtual classes will still be built. As we move forward, more will be
removed until all virtual method functionality is removed.
`UnknownArrayHandle` has special behavior when putting in or getting out
an `ArrayHandleMultiplexer` or an `ArrayHandleCast`. When putting in
either of these, `UnknownArrayHandle` gets the actual array stored in
the multiplexer and cast so that you can later retrieve the base array.
Likewise, when you get the array out with `AsArrayHandle`, you can give
it an `ArrayHandleCast` or `ArrayHandleMultiplexer`, and you will get
the base array placed inside of it.
The Visual Studio compiler has an annoying habit where if you use a
templated class or method with a deprecated class as a template
parameter, you will get a deprecation warning where that class is used
in the templated thing. Thus, if you want to suppress the warning, you
have to supress every instance of the template, not just where the
template is declared.
This is annoying behavior that is thankfully not replicated in other
compilers.
The implementation of `VariantArrayHandle` has been changed to be a
relatively trivial subclass of `UnknownArrayHandle`.
The advantage of this change is twofold. First, it removes
`VariantArrayHandle`'s dependence on `ArrayHandleVirtual`, which gets us
much closer to deprecating that class. Second, it ensures that
`UnknownArrayHandle` is a reasonable replacement for
`VariantArrayHandle`, so we can move forward with replacing that.
The new method `ArrayHandle::DeepCopy` had the pattern such that the
data in the `this` array was pushed to the provided destination array.
However, this is the opposite pattern used in the equivalent method in
VTK, which takes the data from the provided array and copies it to
`this` array.
So, swap the pattern to match that of VTK. Also, make the method name
more descriptive by renaming it to `DeepCopyFrom`. Hopefully, users will
read that to mean given the `ArrayHandle`, you copy data from the other
provided `ArrayHandle`.
A recent modification to `ArrayCopy` created a fast path for when
copying arrays of the same type. This fast path just deep copies the
buffers. The issue was that the buffer copy was creating new buffers
instead of updating the existing buffers. The passed in `ArrayHandle`
would get updated to the new buffers, but any other `ArrayHandle`s
sharing those buffers would still have the old versions. That would lead
to unexpected errors in code like this.
```cpp
vtkm::cont::ArrayHandle<T> outArray1;
vtkm::cont::ArrayHandle<T> outArray2 = outArray1;
vtkm::cont::ArrayCopy(inArray, outArray2);
```
If `inArray` was a different type than `outArray2`, then the data for
both `outArray1` and `outArray2` would be updated (which is the expected
behavior for something considered a "pointer"). However, if `inArray`
was the same type as `outArray2`, then the fast path would change
`outArray2` but leave `outArray1` the same.
This behavior has been corrected so that, in this case, the data of
`outArray1` always follows that of `outArray2`.
C++ was not resolving the overloads of `ArrayCopyImpl` as expected,
which was causing `ArrayCopy` to sometimes use a less efficient method
for copying.
Also fix an issue with `Buffer::DeepCopy` that caused a deadlock when
copying a buffer that was not actually allocated anywhere (as well as
failing to copy the metadata, which was probably the whole point).
The old style `ArrayHandle` stored most of its state, including the
data, in the `vtkm::cont::internal::Storage` object (templated to the
type of array). The new style of `ArrayHandle` stores the data itself in
`Buffer` objects, and recent changes to `Buffer` allow metadata to be
stored there, too.
These changes make it pretty unnecessary to hold any state at all in the
`Storage` object. This is good since the sharing of state from one type
of `ArrayHandle` to another (such as by transforming the data), can be
done by just sharing the `Buffer` objects.
To reinforce this behavior, the `Storage` object has been changed to
make it completely stateless. All the methods of `Storage` must be
marked as `static`.
While in the transition between two types of `ArrayHandle`
implementations, we need to declare when an `ArrayHandle` is implemented
with the new style. To consolidate, create a
`VTKM_ARRAY_HANDLE_NEW_STYLE` to override the default `ArrayHandle`
implementation with the `ArrayHandleNewStyle` implementation.
The testing helper class provided a method named `GetTestDataBasePath`
that returned the base path to all the data files stored in the VTK-m
repo. This is fine, but it was a little cumbersome to build filenames.
To make things easier, there is now a new method named `DataPath` that
takes a string of the filename (or, rather, subpath) to the file in that
directory and automatically builds the path to it.
4345fe26b Store the number of bits of a BitField in the Buffer's metadata
da0403be7 Add metadata to Buffer object.
a84891cd3 Update ArrayHandleBitField to new array style with Buffer
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2218
The number of bits in a `BitField` cannot be directly implied from the
size of the buffer (because the buffer gets padded to the nearest sized
word). Thus, the `BitField stored the number of bits in its own
internals.
Unfortunately, that caused issues when passing the `BitField` data
between it and an `ArrayHandleBitField`. If the `ArrayHandleBitField`
resized itself, the `BitField` would not see the new size because it
ignored the new buffer size.
To get around this problem, `BitField` now declares its own
`BufferMetaData` that stores the number of bits. Now, since the number
of bits is stored in the `Buffer` object, it is sufficient to just share
the `Buffer` to synchronize all of the state.
One of the goals of the `Buffer` object is to allow sharing of data
among objects that will interpret the data differently or give a
different interface over the data. However, when sharing only the array,
important metadata can become lost.
Provide a field that can store some custom metadata in the buffer object
so that the rest of the state can follow the buffer object around.
Now that we have the functions in `vtkm/Atomic.h`, we can deprecate (and
eventually remove) the more cumbersome classes `AtomicInterfaceControl`
and `AtomicInterfaceExecution`.
Also reversed the order of the `expected` and `desired` parameters of
`vtkm::AtomicCompareAndSwap`. I think the former order makes more sense
and matches more other implementations (such as `std::atomic` and the
GCC `__atomic` built ins). However, there are still some non-deprecated
classes with similar methods that cannot easily be switched. Thus, it's
better to be inconsistent with most other libraries and consistent with
ourself than to be inconsitent with ourself.
Now that we have atomic free functions (e.g. `vtkm::AtomicAdd()`), we no
longer need special implementations for control and each execution
device. (Well, technically we do have special implementations for each,
but they are handled with compiler directives in the free functions.)
Convert the old atomic interface classes (`AtomicInterfaceControl` and
`AtomicInterfaceExecution`) to use the new atomic free functions. This
will allow us to test the new atomic functions everywhere that atomics
are used in VTK-m.
Once verified, we can deprecate the old atomic interface classes.
ed41874cc Consolidate tests for base vtkm code that is device-specific
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2219
Some of the code in the base `vtkm` namespace is device specific. For
example, the functions in `Math.h` are customized for specific devices.
Thus, we want this code to be specially compiled and run on these
devices.
Previously, we made a header file and then added separate tests to each
device package. That was created before we had ways of running on any
device. Now, it is much easier to compile the test a single time for all
devices and use the `ALL_BACKENDS` feature of `vtkm_unit_tests` CMake
function to automatically create the test for all devices.
8983154e9 Add DeepCopy to ArrayHandle
694ba7e92 Change ArrayCopy to deep copy Buffer objects where possible
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2212
2d1b609b3 Use Ubuntu instead of rhel8 for cuda+kokkos
769248583 Make sure we use c++14 when using CUDA 11+
64efa6401 Kokkos: make sure we don't pass multiple rdc flags
b2f4c8e5e Switch -O3 to -O2 on Linux with Cuda 10
db57ed26a Fix warnings
452f61e29 Add Kokkos backend
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2164
`ArrayHandle::DeepCopy` creates a new `ArrayHandle` of the same type and
deep copies the data into it.
This functionality is similar to `ArrayCopy`. However, it can be used
without having to compile for the device on which the copy happens.
Now that the data in an `ArrayHandle` is stored in `Buffer` objects, we
now have a more efficient way of doing deep copies of memory. Rather
than call `Algorithm::Copy`, which iterates over the array and copies
each item, `ArrayCopy` now uses the `Buffer` interface to do direct
device-to-device (or host-to-host) mem copies. This should be more
efficent and take less time to compile.
Note that this direct `Buffer` copy only works if the two `ArrayHandle`s
are of the same type. If they are different, `ArrayCopy` still has to
fall back to using `Algorithm::Copy`.
Also note that not all `ArrayHandle`s are using the new `ArrayHandle`
interface (and therefore not using `Buffer` objects). Thus, a fallback
is still available for old `ArrayHandle` types.
This has no real change in the operation, but it will simplify code as
we convert `ArrayHandle`s to the new type. We will be able to write
simple runtime code rather than complex metaprogramming to determine the
number of buffers to use.
For `make_ArrayHandle` and `make_Field` when it is determined that the
data can be safely moved, just silently move instead of copy instead of
printing a log message saying the copy flag will be ignored.
Also fix an issue with `make_ArrayHandle` when the data was not moved
when it could have been.
The recent version of ArrayHandleBasic allocates typeless arrays without
any initialization. This can cause issues with types that require a
constructor. The UnitTestVariantArrayHandle was trying to create an
ArrayHandle with an std::string, and the uninitialized strings were
causing crashes on some platforms.
We have made several improvements to adding data into an `ArrayHandle`.
## Moving data from an `std::vector`
For numerous reasons, it is convenient to define data in a `std::vector`
and then wrap that into an `ArrayHandle`. It is often the case that an
`std::vector` is filled and then becomes unused once it is converted to an
`ArrayHandle`. In this case, what we really want is to pass the data off to
the `ArrayHandle` so that the `ArrayHandle` is now managing the data and
not the `std::vector`.
C++11 has a mechanism to do this: move semantics. You can now pass
variables to functions as an "rvalue" (right-hand value). When something is
passed as an rvalue, it can pull state out of that variable and move it
somewhere else. `std::vector` implements this movement so that an rvalue
can be moved to another `std::vector` without actually copying the data.
`make_ArrayHandle` now also takes advantage of this feature to move rvalue
`std::vector`s.
There is a special form of `make_ArrayHandle` named `make_ArrayHandleMove`
that takes an rvalue. There is also a special overload of
`make_ArrayHandle` itself that handles an rvalue `vector`. (However, using
the explicit move version is better if you want to make sure the data is
actually moved.)
## Make `ArrayHandle` from initalizer list
A common use case for using `std::vector` (particularly in our unit tests)
is to quickly add an initalizer list into an `ArrayHandle`. Now you can
by simply passing an initializer list to `make_ArrayHandle`.
## Deprecated `make_ArrayHandle` with default shallow copy
For historical reasons, passing an `std::vector` or a pointer to
`make_ArrayHandle` does a shallow copy (i.e. `CopyFlag` defaults to `Off`).
Although more efficient, this mode is inherintly unsafe, and making it the
default is asking for trouble.
To combat this, calling `make_ArrayHandle` without a copy flag is
deprecated. In this way, if you wish to do the faster but more unsafe
creation of an `ArrayHandle` you should explicitly express that.
This requried quite a few changes through the VTK-m source (particularly in
the tests).
## Similar changes to `Field`
`vtkm::cont::Field` has a `make_Field` helper function that is similar to
`make_ArrayHandle`. It also features the ability to create fields from
`std::vector`s and C arrays. It also likewise had the same unsafe behavior
by default of not copying from the source of the arrays.
That behavior has similarly been depreciated. You now have to specify a
copy flag.
The ability to construct a `Field` from an initializer list of values has
also been added.
While compiling UnitTestVariantArrayHandle, some versions of gcc
(between 6 and 8, I think) gave a warning like the following:
```
../vtkm/cont/StorageVirtual.h:227:12: warning: 'vtkm::Id vtkm::cont::internal::detail::StorageVirtualImpl<T, S>::GetNumberOfValues() const [with T = std::__cxx11::basic_string<char>; S = vtkm::cont::StorageTagImplicit<{anonymous}::UnusualPortal<std::__cxx11::basic_string<char> > >]' declared 'static' but never defined [-Wunused-function]
```
This warning makes no sense because it is refering to a method that is
not declared static. (In fact, it overrides a virtual method.)
I believe this is an obscure bug in these versions of gcc. I found a
[stackoverflow post] that seems to have the same problem, but no
workaround was found.
The warning originated from code that had little effect. It was part of
a test with a custom ArrayHandle storage type that was already disabled
for other reasons. Just removed the code.
[stackoverflow post]: https://stackoverflow.com/questions/56615695/how-to-fix-declared-static-but-never-defined-on-member-function
c689a68c5 Suppress bad deprecation warnings in MSVC
a3f23a03b Do not cast to ArrayHandleVirtual in VariantArrayHandle::CastAndCall
f6b13df51 Support coordinates of both float32 and float64
453e31404 Deprecate ArrayHandleVirtualCoordinates
be7f06bbe CoordinateSystem data is VariantArrayHandle
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2177
The Microsoft compiler has this annoying and stupid behavior where if
you have a generic templated method/function and that method is
instantiated with a deprecated class, then the compiler will issue a
C4996 warning even if the calling code is suppressing that warning
(because, for example, you are implementing other deprecated code and
the use is correct). There is no way around this other than suppressing
the warnings for all uses of the templated method.
We are moving to deprecate `ArrayHandleVirtual`, so we are removing the
feature where `VariantArrayHandle::CastAndCall` automatically casts to
an `ArrayHandleVirtual` if possible.
The big reason to make this change now (as opposed to later when
`ArrayHandleVirtual` is deprecated) is to improve compile times.
This prevents us from having to compile an extra code path using
`ArrayHandleVirtual`.
`CoordinateSystem` differed from `Field` in that its `GetData`
method returned an `ArrayHandleVirtualCoordinates` instead of
a `VariantArrayHandle`. This is probably confusing since
`CoordianteSystem` inherits `Field` and has a pretty dramatic
difference in this behavior.
In preparation to deprecate `ArrayHandleVirtualCoordinates`, this
changes `CoordiantSystem` to be much more like `Field`. (In the
future, we may change the `CoordinateSystem` to point to a `Field`
rather than be a special `Field`.)
A method named `GetDataAsMultiplexer` has been added to
`CoordinateSystem`. This method allows you to get data from
`CoordinateSystem` as a single array type without worrying
about creating functors to handle different types and without
needing virtual methods.
18b5be92d Fix issue with CUDA and ArrayHandleMultiplexer
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2168
`ArrayHandle::PrepareForOutput` often has to reallocate the array to the
specified size. Previously, this allocation was not happening with the
`Token` that is passed to `PrepareForOutput`. If the `ArrayHandle` is
already attached or enqueued for that `Token`, then the allocation would
deadlock.
You can now pass a `Token` object to `Allocate`, which is what
`PrepareForOutput` does.
When you try to call the `Reduce` operation in the CUDA device adapter
with a sufficently complex interator type, you get a compile error
that says `error: cannot pass an argument with a user-provided
copy-constructor to a device-side kernel launch`.
This appears to be a bug in either nvcc or Thrust. I believe it is
related to the following reported issues:
* https://github.com/thrust/thrust/issues/928
* https://github.com/thrust/thrust/issues/1044
Work around this problem by making a special condition for calling
`Reduce` with an `ArrayHandleMultiplexer` that calls the generic
algorithm in `DeviceAdapterAlgorithmGeneral` instead of the algorithm in
Thrust.
Often when a user gives memory to an `ArrayHandle`, she wants data to be
written into the memory given to be used elsewhere. Previously, the
`Buffer` objects would delete the given buffer as soon as a write buffer
was created elsewhere. That was a problem if a user wants VTK-m to write
results right into a given buffer.
Instead, when a user provides memory, "pin" that memory so that the
`ArrayHandle` never deletes it.
The buffer class encapsulates the movement of raw C arrays between
host and devices.
The `Buffer` class itself is not associated with any device. Instead,
`Buffer` is used in conjunction with a new templated class named
`DeviceAdapterMemoryManager` that can allocate data on a given
device and transfer data as necessary. `DeviceAdapterMemoryManager`
will eventually replace the more complicated device adapter classes
that manage data on a device.
The code in `DeviceAdapterMemoryManager` is actually enclosed in
virtual methods. This allows us to limit the number of classes that
need to be compiled for a device. Rather, the implementation of
`DeviceAdapterMemoryManager` is compiled once with whatever compiler
is necessary, and then the `RuntimeDeviceInformation` is used to
get the correct object instance.
143e3d39a remove unused type alias
01a448663 Merge branch 'master' into uniform_real
c67e5bb12 fixe warnings about implicit type conversion
1e4294392 Add deterministic seed to avoid potential spurious failure
5b0e309b9 the random source is still 64 bits
cc3061bab Avoid calling ReadPortal() all the time
9bf6dea22 remove inline initialization of seed
e69308047 Add statistics base testing, add Flot32 RNG
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2148
If a test throws any unexpected exception, the test is supposed to
detect that and fail. For the STL exceptions, the test failed to return
an error code. Fix that.
Many of the operations of `VariantArrayHandleBase` are not dependent on
the TypeList parameter of the class. Still others can operate just as
well by providing a type list to a method. Thus, it is convenient to
create a superclass that is not templated. That allows us to pass around
a `VariantArrayHandle` when the type list does not matter.
This superclass is called `VariantArrayHandleCommon` because "base" was
already taken.
This makes it clear that it returns true for an invalid array handle.
The previous name implied that it was looking for an ArrayHandle in some
"valid" set, which is the opposite.
The ArrayPortalMultiplexer always called Set on the delegate portals.
However, sometimes the portals were not supported. Instead, only call
when the delegate is supported.
Probably the "right" thing to do would be to check all posible portals
for support, but that sounds like it would slow down the compile.
Previously, the VariantArrayHandle::AsMultiplexer operation silently
failed and returned an invalid ArrayHandleMultiplexer. This is now
changed to throw an exception if the desired ArrayHandleMultiplexer
cannot hold the VariantArrayHandle's array.
de3bda373 Use deque instead of list for ArrayHandle queue
498d44548 Pass Token::Reference by value
c32c9e8e8 Fix deadlock when changing device during read
99e14ab8a Add proper enqueuing of Tokens for ArrayHandle
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2130
c0dee7402 make it explicit that we are using 64-bit unsigned integer in bit op
34f350588 Added changelog
e9f584a91 ArrayHandleRandomUniformReal
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Kenneth Moreland <kmorel@sandia.gov>
Merge-request: !2116
934f085e0 Build diy as a library
f0a37ac6a Merge branch 'upstream-diy' into diy-mpi-nompi
7687aabf8 diy 2020-06-05 (b62915aa)
6ca2b9f87 Point to new version of Diy
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Robert Maynard <robert.maynard@kitware.com>
Merge-request: !2123
Because ArrayHandle currently only supports one device at a time, it is
possible that a `PrepareForInput` might actually need to wait for write
access so that it could move the data between devices. However, we don't
want the `Token` to be attached for writing because that could block
other read operations.
To get around this, add a hack to WaitToWrite to allow it to attach for
reading instead of writing.
An issue that was identified for the thread safety of `ArrayHandle` is
that if several threads are waiting to use an `ArrayHandle`, there might
be an expectation of the order in which the operations happen. For
example, if one thread is modifying the contents of an `ArrayHandle` and
another is reading those results, we would need the first one to start
before the second one.
To solve this, a queue is added to `ArrayHandle` such that when waiting
to read or write an `ArrayHandle` the `Token` has to be at the top of
the queue in addition to other requirements being met.
Additionally, an `Enqueue` method is added to add a `Token` to the queue
without blocking. This allows a control thread to queue the access and
then spawn a thread where the actual work will be done. As long as
everything is enqueued on the main thread, the operations will happen in
the expected order.
Initially, the probe filter would simply not set a value if a sample was
outside the input `DataSet`. This is not great as the memory could be
left uninitalized and lead to unpredictable results. The testing
compared these invalid results to 0, which seemed to work but is
probably unstable.
This was partially fixed by a previous change that consolidated to
mapping of cell data with a general routine that permuted data. However,
the fix did not extend to point data in the input, and it was not
possible to specify a particular invalid value.
This change specifically updates the probe filter so that invalid values
are set to a user-specified value.
251bd82b8 Significantly improve FlyingEdges performance across all devices
fa9373801 Rework FlyingEdges::Pass1 to handle NUMA and CUDA requirements.
769a10b47 FlyingEdge Normal and Point generation occurs in Pass4
93d87e06f Optimize StructuredPointGradient for non boundary points.
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2080
We now use SumYAxis when executing with CUDA for better memory patterns.
Instead of using the heavy Pass4/Pass4WithNormals, CUDA now uses a
2 pass approach with the second pass outputting the normals and
coordinates using with significantly less warp divergence
Generally, fields that have a WHOLE_MESH association might be valid even
if the structure of the mesh changes. Thus, it makes sense for filters
to pass this data pretty much all the time.
Also cleaned up some code and comments to make the relationship between
`MapFieldOntoOutput` and `DoMapField` more clear.
The only reason Keys has a template is so that it can hold a UniqueKeys
array and provide the key for each group. If that is not needed and you
want to implement a library function that takes a keys object, you can
now grab the Keys superclass KeysBase. KeysBase is not templated, so you
can pass it to a standard method in a library.
The version of `Filter::Execute` that takes a policy as an argument is now
deprecated. Filters are now able to specify their own fields and types,
which is often why you want to customize the policy for an execution. The
other reason is that you are compiling VTK-m into some other source that
uses a particular types of storage. However, there is now a mechanism in
the CMake configuration to allow you to provide a header that customizes
the "default" types used in filters. This is a much more convenient way to
compile filters for specific types.
One thing that filters were not able to do was to customize what cell sets
they allowed using. This allows filters to self-select what types of cell
sets they support (beyond simply just structured or unstructured). To
support this, the lists `SupportedCellSets`, `SupportedStructuredCellSets`,
and `SupportedUnstructuredCellSets` have been added to `Filter`. When you
apply a policy to a cell set, you now have to also provide the filter.
24d022b02 Implement and test ImageReader and ImageWriter capabilities in the io library
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Kenneth Moreland <kmorel@sandia.gov>
Merge-request: !1967
