To simplify reproducing docker based CI workers locally, VTK-m has python program that handles all the
work automatically for you.
The program is located in `[Utilities/CI/reproduce_ci_env.py ]` and requires python3 and pyyaml.
To use the program is really easy! The following two commands will create the `build:rhel8` gitlab-ci
worker as a docker image and setup a container just as how gitlab-ci would be before the actual
compilation of VTK-m. Instead of doing the compilation, instead you will be given an interactive shell.
```
./reproduce_ci_env.py create rhel8
./reproduce_ci_env.py run rhel8
```
To compile VTK-m from the the interactive shell you would do the following:
```
> src]# cd build/
> build]# cmake --build .
```
Previously, the PolicyDefault used to compile all the filters was hard-
coded. The problem was that if any external project that depends on VTK-
m needs a different policy, it had to recompile everything in its own
translation units with a custom policy.
This change allows an external project provide a simple header file that
changes the type lists used in the default policy. That allows VTK-m to
compile the filters exactly as specified by the external project.
Currently, VTK-m is using C++11. However, it is often useful to use
features in the `std` namespace that are defined for C++14 or later. We
can provide our own versions (sometimes), but it is preferable to use
the version provided by the compiler if available.
There were already some examples of defining portable versions of C++14
and C++17 classes in a `vtkmstd` namespace, but these were sprinkled
around the source code.
There is now a top level `vtkmstd` directory and in it are header files
that provide portable versions of these future C++ classes. In each
case, preprocessor macros are used to select which version of the class
to use.
Previously, when a ReadPortal or a WritePortal was returned from an
ArrayHandle, it had wrapped in it a Token that was attached to the
ArrayHandle. This Token would prevent other reads and writes from the
ArrayHandle.
This added safety in the form of making sure that the ArrayPortal was
always valid. Unfortunately, it also made deadlocks very easy. They
happened when an ArrayPortal did not leave scope immediately after use
(which is not all that uncommon).
Now, the ArrayPortal no longer locks up the ArrayHandle. Instead, when
an access happens on the ArrayPortal, it checks to make sure that
nothing has happened to the data being accessed. If it has, a fatal
error is reported to the log.
This commit also:
- Removes a corner case not longer used at ArrayPortalGroupVecVariable::get
- Changes doc regarding the number of offset elements in the input
array handler of ConvertNumComponentsToOffsets.
- Updates invokation of make_ArrayGroupVectVariable in multiple files
- Adds its corresponding changelog entry
To get a portal to access ArrayHandle values in the control
environment, you now use the ReadPortal and WritePortal methods.
The portals returned are wrapped in an ArrayPortalToken object
so that the data between the portal and the ArrayHandle are
guaranteed to be consistent.
Also discovered that many C++ compilers have trouble giving warnings
for partial specialization of classes marked as deprecated. Fix
the problem by instead deprecating the items in the class.
The convenience functions `ArrayPortalToIteratorBegin()` and
`ArrayPortalToIteratorEnd()` wouldn't detect specializations of
`ArrayPortalToIterators<PortalType>` since the specializations aren't
visible when the `Begin`/`End` functions are declared.
Since the CUDA iterators rely on a specialization, the convenience
functions would not compile on CUDA.
Now, instead of specializing `ArrayPortalToIterators` to provide custom
iterators for a particular portal, the portal may advertise custom
iterators by defining `IteratorType`, `GetIteratorBegin()`, and
`GetIteratorEnd()`. `ArrayPortalToIterators` will detect such portals
and automatically switch to using the specialized portals.
This eliminates the need for the specializations to be visible to the
convenience functions and allows them to be usable on CUDA.
Previously we relied on CMake's compiler detection module to build the
macros for using the deprecated attribute. However, CMake created macros
for pre-C++14 versions of the feature, which do not work in all cases.
Also, we have the need to be able to suppress deprecation warnings when
we are implementing a deprecated thing. Since we have to query compilers
ourself, we might as well figure out if the deprecated attribute we want
is supported.
Worst case is that we won't support deprecation warnings everywhere we
could. That will not create incorrect code and we can always add that
later.
The `VTKM_DEPRECATED` macro allows us to remove (and usually replace)
features from VTK-m in minor releases while still following the conventions
of semantic versioning. The idea is that when we want to remove or replace
a feature, we first mark the old feature as deprecated. The old feature
will continue to work, but compilers that support it will start to issue a
warning that the use is deprecated and should stop being used. The
deprecated features should remain viable until at least the next major
version. At the next major version, deprecated features from the previous
version may be removed.
If a worklet doesn't explicitly state an ExecutionSignature, VTK-m
assumes the worklet has no return value, and each ControlSignature
argument is passed to the worklet in the same order.
For example if we had this worklet:
```cxx
struct DotProduct : public vtkm::worklet::WorkletMapField
{
using ControlSignature = void(FieldIn, FieldIn, FieldOut);
using ExecutionSignature = void(_1, _2, _3);
template <typename T, vtkm::IdComponent Size>
VTKM_EXEC void operator()(const vtkm::Vec<T, Size>& v1,
const vtkm::Vec<T, Size>& v2,
T& outValue) const
{
outValue = vtkm::Dot(v1, v2);
}
};
```
It can be simplified to be:
```cxx
struct DotProduct : public vtkm::worklet::WorkletMapField
{
using ControlSignature = void(FieldIn, FieldIn, FieldOut);
template <typename T, vtkm::IdComponent Size>
VTKM_EXEC void operator()(const vtkm::Vec<T, Size>& v1,
const vtkm::Vec<T, Size>& v2,
T& outValue) const
{
outValue = vtkm::Dot(v1, v2);
}
};
VTK-m now provides the following filters with the default policy
as part of the vtkm_filter library:
- CellAverage
- CleanGrid
- ClipWithField
- ClipWithImplicitFunction
- Contour
- ExternalFaces
- ExtractStructured
- PointAverage
- Threshold
- VectorMagnitude
By building these as a library we hope to provide faster compile
times for consumers of VTK-m when using common configurations.
05b679250 Merge branch 'master' into tangle_source
a6c044df9 added changelog
0d818701c put VTKM_SOURCE_EXPORT in .cxx
e0aae7d86 remove EXPORT from .cxx
4d7de67ee use VTKM_SOURCE_EXPORT
cd49136d5 add copyright notice, add installation of header
0c094a568 used the Tangle source
aeb8877a9 add newline at EOF, minor change based review
...
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !1804
Consumers of VTK-m when enabling of dropping of unused functions
will see VTK-m functions dropped. Previously this didn't happen
as VTK-m didn't build object files with the correct flags for this.
By allowing the linker to remove unused symbols we see a significant
saving the file size of VTK-m tests, examples, and benchmarks.
An OpenMP build of the tests and benchmarks goes from 168MB to
141MB which is roughly a 16% filesize reduction.
Initially I had presumed that these changes would increase link times.
But in measurements the total wall time for compilation of VTK-m has
stayed about the same ( seeing a decrease of 1.5% ). Presumably the
increased computation is offset by the reduction in file writing.
Instead of having to be called for each target you can now
pass multiple targets at once. Do note that if you pass multiple
targets you will need to pass all the sources from those targets
that need to be 'device' compiled.
Rather than do a CastAndCall on all possible field types when calling a
worklet with two fields (where they all typically get cast to the same
type as the primary field), use the new mechanism with
ArrayHandleMultiplexer to create one code path.
Also update the ApplyPolicy to accept the Field type, which is used to
determine any additional storage types to support.
This is done through a new version of ApplyPolicy. This version takes
a type of the array as its first template argument, which must be
specified.
This requires having a list of potential storage to try. It will use
that to construct an ArrayHandleMultiplexer containing all potential
types. This list of storages comes from the policy. A StorageList
item was added to the policy.
Types are automatically converted. So if you ask for a vtkm::Float64 and
field contains a vtkm::Float32, it will the array wrapped in an
ArrayHandleCast to give the expected type.
Added a BaseComponentType to VecTraits that recursively finds the base
(non-Vec) type of a Vec. This is useful when dealing with potentially
nested Vec's (e.g. Vec<Vec<T, M>, N>) and you need to keep the structure
but know the base type.
Also added a couple of templates for keeping the structure but changing
the type. These are ReplaceComponentType and ReplaceBaseComponentType.
These allow you to create new Vec's with the same structure as the query
Vec but with differen component types.
This behaves just like `ScanExclusive`, but rather than returning the
total sum, it is appended to the end of the output array.
This is in preparation for the CellSetExplicit refactoring described in
issue #408.
The `MultiBlock` class has been renamed to `PartitionedDataSet`, and its API
has been refactored to refer to "partitions", rather than "blocks".
Additionally, the `AddBlocks` method has been changed to `AppendPartitions` to
more accurately reflect the operation performed. The associated
`AssignerMultiBlock` class has also been renamed to
`AssignerPartitionedDataSet`.
This change is motivated towards unifying VTK-m's data model with VTK. VTK has
started to move away from `vtkMultiBlockDataSet`, which is a hierarchical tree
of nested datasets, to `vtkPartitionedDataSet`, which is always a flat vector
of datasets used to assist geometry distribution in multi-process environments.
This simplifies traversal during processing and clarifies the intent of the
container: The component datasets are partitions for distribution, not
organizational groupings (e.g. materials).
Ref #405
By removing the ability to have multiple CellSets in a DataSet
we can simplify the following things:
- Cell Fields now don't require a CellSet name when being constructed
- Filters don't need to manage what the active cellset is
For polygon cell shapes (that are not triangles or quadrilaterals),
interpolations are done by finding the center point and creating a
triangle fan around that point. Previously, the gradient was computed in
the same way as interpolation: identifying the correct triangle and
computing the gradient for that triangle.
The problem with that approach is that makes the gradient discontinuous
at the boundaries of this implicit triangle fan. To make things worse,
this discontinuity happens right at each vertex where gradient
calculations happen frequently. This means that when you ask for the
gradient at the vertex, you might get wildly different answers based on
floating point imprecision.
Get around this problem by creating a small triangle around the point in
question, interpolating values to that triangle, and use that for the
gradient. This makes for a smoother gradient transition around these
internal boundaries.
This allows for developers to do things such as the following
as constexpr's:
```cxx
constexpr vtkm::Id2 dims(16,16);
constexpr vtkm::Float64 dx = vtkm::Float64(4.0 * vtkm::Pi()) / vtkm::Float64(dims[0] - 1);
```
Perhaps a better title for this change would be "Make the Threshold
filter not totally useless."
A long standing issue with the Threshold filter is that its output
CellSet was stored in a CellSetPermutation. This made Threshold hyper-
efficient because it required hardly any data movement to implement.
However, the problem was that any other unit that had to use the CellSet
failed. To have VTK-m handle that output correctly in other filters and
writers, they all would have to check for the existance of
CellSetPermutation. And CellSetPermutation is templated on the CellSet
type it is permuting, so all units would have to compile special cases
for all these combinations. This is not likely to be feasible in any
static solution.
The simple solution, implemented here, is to deep copy the cells to a
CellSetExplicit, which is a known type that is already used everywhere
in VTK-m. The solution is a bit disappointing since it requires more
memory and time to build. But it is on par with solutions in other
libraries (like VTK). And it really does not matter how efficient the
old solution was if it was useless.
3a7d3cb5f VTK-m filters now launch all worklets via a vtkm::cont::Invoker
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Kenneth Moreland <kmorel@sandia.gov>
Merge-request: !1760
The `From` and `To` nomenclature for topology mapping has been confusing for
both users and developers, especially at lower levels where the intention of
mapping attributes from one element to another is easily conflated with the
concept of mapping indices (which maps in the exact opposite direction).
These identifiers have been renamed to `VisitTopology` and `IncidentTopology`
to clarify the direction of the mapping. The order in which these template
parameters are specified for `WorkletMapTopology` have also been reversed,
since eventually there may be more than one `IncidentTopology`, and having
`IncidentTopology` at the end will allow us to replace it with a variadic
template parameter pack in the future.
Other implementation details supporting these worklets, include `Fetch` tags,
`Connectivity` classes, and methods on the various `CellSet` classes (such as
`PrepareForInput` have also reversed their template arguments. These will need
to be cautiously updated.
The convenience implementations of `WorkletMapTopology` have been renamed for
clarity as follows:
```
WorkletMapPointToCell --> WorkletVisitCellsWithPoints
WorkletMapCellToPoint --> WorkletVisitPointsWithCells
```
The `ControlSignature` tags have been renamed as follows:
```
FieldInTo --> FieldInVisit
FieldInFrom --> FieldInMap
FromCount --> IncidentElementCount
FromIndices --> IncidentElementIndices
```
The Invoker is a control side object that handles the construction
of the relevant worklet dispatcher. Moving it to control makes it
obvious that it isn't an algorithm itself but a way to launch
worklets.
There was a special case for ArrayHandleMultiplexer where if you gave it
just one type it would treat that as a value type rather than an array
to support and instead provide a default list of types. However, GCC 4.8
is having trouble compiling the code to create the default list, the
semantics are confusing, and the more I think about it the less likely I
think we will need this functionality. So, just getting rid of that.
6592e5288 Fix IsWritableArrayHandle for portals that exist but cannot be written
0e15a1116 Enable writing to ArrayHandleCast
6d37ce945 Remove invalid PortalType
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !1731
Previously, `ArrayHandleCast` was considered a read-only array handle.
However, it is trivial to reverse the cast (now that `ArrayHandleTransform`
supports an inverse transform). So now you can write to a cast array
(assuming the underlying array is writable).
One trivial consequence of this change is that you can no longer make a
cast that cannot be reversed. For example, it was possible to cast a simple
scalar to a `Vec` even though it is not possible to convert a `Vec` to a
scalar value. This was of dubious correctness (it is more of a construction
than a cast) and is easy to recreate with `ArrayHandleTransform`.
The ScopedRuntimeDeviceTracker now can force, enable, or disable
devices. Additionally the ScopedRuntimeDeviceTracker and the
RuntimeDeviceTracker handle the DeviceAdapterTagAny robustly
across all methods.
