Shuenhoy/vtk-m
1
0
Fork 0
mirror of https://gitlab.kitware.com/vtk/vtk-m synced 2024-09-08 13:23:51 +00:00
Code Issues 2 Actions Packages Projects Releases Wiki Activity

Add ArrayPortalToken object and implement Read/WritePortal

Browse Source 
To get a portal to access ArrayHandle values in the control
environment, you now use the ReadPortal and WritePortal methods.
The portals returned are wrapped in an ArrayPortalToken object
so that the data between the portal and the ArrayHandle are
guaranteed to be consistent.
This commit is contained in:
Kenneth Moreland 2020-01-23 18:01:30 -07:00
parent e43770888f
commit 6b089be03e
11 changed files with 350 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

94
docs/changelog/scoping-tokens.md
View File

@ -11,16 +11,19 @@ The original `PrepareFor*` methods of `ArrayHandle` returned an object to
be used in the execution environment on a particular device that pointed to
data in the array. The pointer to the data was contingent on the state of
the `ArrayHandle` not changing. The assumption was that the calling code
would next use the returned execution environment object and would not
further change the `ArrayHandle` until done with the execution environment
object.
would immediately use the returned execution environment object and would
not further change the `ArrayHandle` until done with the execution
environment object.
This assumption is broken if multiple threads are running in the control
environment. After one thread has called `PrepareFor*` and is in the
process of using the resulting execution object and another thread attempts
to write to or otherwise change the same array. Perhaps a well designed
program should not share `ArrayHandle`s in this way, but if a mistake is
made that would lead to a very difficult to diagnose intermittent error.
environment. For example, if one thread has called `PrepareForInput` to get
an execution array portal, the portal or its data could become invalid if
another thread calls `PrepareForOutput` on the same array. Initially one
would think that a well designed program should not share `ArrayHandle`s in
this way, but there are good reasons to need to do so. For example, when
using `vtkm::cont::PartitionedDataSet` where multiple partitions share a
coordinate system (very common), it becomes unsafe to work on multiple
blocks in parallel on different devices.
What we really want is the code to be able to specify more explicitly when
the execution object is in use. Ideally, the execution object itself would
@ -96,6 +99,52 @@ It actually still works to use the old style of `PrepareForExecution`.
However, you will get a deprecation warning (on supported compilers) when
you try to use it.
## Invoke and Dispatcher
The `Dispatcher` classes now internally define a `Token` object during the
call to `Invoke`. (Likewise, `Invoker` will have a `Token` defined during
its invoke.) This internal `Token` is used when preparing `ArrayHandle`s
and `ExecutionObject`s for the execution environment. (Details in the next
section on how that works.)
Because the invoke uses a `Token` to protect its arguments, it will block
the execution of other worklets attempting to access arrays in a way that
could cause read-write hazards. In the following example, the second
worklet will not be able to execute until the first worklet finishes.
``` cpp
vtkm::cont::Invoker invoke;
invoke(Worklet1{}, input, intermediate);
invoke(Worklet2{}, intermediate, output); // Will not execute until Worklet1 finishes.
```
That said, invocations _can_ share arrays if their use will not cause
read-write hazards. In particular, two invocations can both use the same
array if they are both strictly reading from it. In the following example,
both worklets can potentially execute at the same time.
``` cpp
vtkm::cont::Invoker invoke;
invoke(Worklet1{}, input, output1);
invoke(Worklet2{}, input, output2); // Will not block
```
The same `Token` is used for all arguments to the `Worklet`. This deatil is
important to prevent deadlocks if the same object is used in more than one
`Worklet` parameter. As a simple example, if a `Worklet` has a control
signature like
``` cpp
using ControlSignature = void(FieldIn, FieldOut);
```
it should continue to work to use the same array as both fields.
``` cpp
vtkm::cont::Invoker invoke;
invoke(Worklet1{}, array, array);
```
## Transport
The dispatch mechanism of worklets internally uses
@ -106,15 +155,18 @@ the covers for most users.
## Control Portals
The calling signatures of `GetPortalControl` and `GetPortalConstControl`
have not changed. That is, they do not require a `Token` object. This is
because these are control-only objects and so the `Token` is embedded
within the return portal object.
The `GetPortalConstControl` and `GetPortalControl` methods have been
deprecated. Instead, the methods `ReadPortal` and `WritePortal` should be
used. The calling signature is the same as their predecessors, but the
returned portal contains a `Token` as part of its state and prevents and
changes to the `ArrayHandle` it comes from. The `WritePortal` also prevents
other reads from the array.
The advantage is that the returned portal will always be valid. However, it
is now the case that a control portal can prevent something else from
running. This means that control portals should drop scope as soon as
possible.
possible. It is because of this behavior change that new methods were
created instead of altering the old ones.
## Deadlocks
@ -127,7 +179,7 @@ Care should be taken to ensure that a single thread does not attempt to use
an `ArrayHandle` two ways at the same time.
``` cpp
auto portal = array.GetPortalControl();
auto portal = array.WritePortal();
for (vtkm::Id index = 0; index < portal.GetNumberOfValues(); ++index)
{
portal.Set(index, /* An interesting value */);
@ -152,3 +204,17 @@ Instead, `portal` should be properly scoped.
vtkm::cont::Invoker invoke;
invoke(MyWorklet, array); // Runs fine because portal left scope
```
Alternately, you can call `Detach` on the portal, which will invalidate the
portal and unlock the `ArrayHandle`.
``` cpp
auto portal = array.WritePortal();
for (vtkm::Id index = 0; index < portal.GetNumberOfValues(); ++index)
{
portal.Set(index, /* An interesting value */);
}
portal.Detach();
vtkm::cont::Invoker invoke;
invoke(MyWorklet, array); // Runs fine because portal detached
```

65
vtkm/cont/ArrayHandle.h
View File

@ -36,6 +36,7 @@
#include <vtkm/cont/internal/ArrayHandleExecutionManager.h>
#include <vtkm/cont/internal/ArrayPortalFromIterators.h>
#include <vtkm/cont/internal/ArrayPortalToken.h>
namespace vtkm
{
@ -261,8 +262,9 @@ public:
using StorageType = vtkm::cont::internal::Storage<T, StorageTag_>;
using ValueType = T;
using StorageTag = StorageTag_;
using PortalControl = typename StorageType::PortalType;
using PortalConstControl = typename StorageType::PortalConstType;
using WritePortalType = vtkm::cont::internal::ArrayPortalToken<typename StorageType::PortalType>;
using ReadPortalType =
vtkm::cont::internal::ArrayPortalToken<typename StorageType::PortalConstType>;
template <typename DeviceAdapterTag>
struct ExecutionTypes
{
@ -271,6 +273,11 @@ public:
typename ExecutionManagerType::template ExecutionTypes<DeviceAdapterTag>::PortalConst;
};
using PortalControl VTKM_DEPRECATED(1.6, "Use ArrayHandle::WritePortalType instead.") =
typename StorageType::PortalType;
using PortalConstControl VTKM_DEPRECATED(1.6, "Use ArrayHandle::ReadPortalType instead.") =
typename StorageType::PortalConstType;
/// Constructs an empty ArrayHandle. Typically used for output or
/// intermediate arrays that will be filled by a VTKm algorithm.
///
@ -367,13 +374,61 @@ public:
/// Since worklet invocations are asynchronous and this routine is a synchronization point,
/// exceptions maybe thrown for errors from previously executed worklets.
///
VTKM_CONT PortalControl GetPortalControl();
/// \deprecated Use `WritePortal` instead. Note that the portal returned from `WritePortal`
/// will disallow any other reads or writes to the array while it is in scope.
///
VTKM_CONT
VTKM_DEPRECATED(1.6,
"Use ArrayHandle::WritePortal() instead. "
"Note that the returned portal will lock the array while it is in scope.")
typename StorageType::PortalType GetPortalControl();
/// Get the array portal of the control array.
/// Since worklet invocations are asynchronous and this routine is a synchronization point,
/// exceptions maybe thrown for errors from previously executed worklets.
///
VTKM_CONT PortalConstControl GetPortalConstControl() const;
/// \deprecated Use `ReadPortal` instead. Note that the portal returned from `ReadPortal`
/// will disallow any writes to the array while it is in scope.
///
VTKM_CONT
VTKM_DEPRECATED(1.6,
"Use ArrayHandle::ReadPortal() instead. "
"Note that the returned portal will lock the array while it is in scope.")
typename StorageType::PortalConstType GetPortalConstControl() const;
/// \brief Get an array portal that can be used in the control environment.
///
/// The returned array can be used in the control environment to read values from the array. (It
/// is not possible to write to the returned portal. That is `Get` will work on the portal, but
/// `Set` will not.)
///
/// **Note:** The returned portal will prevent any writes or modifications to the array. To
/// ensure that the data pointed to by the portal is valid, this `ArrayHandle` will be locked to
/// any modifications while the portal remains in scope. (You can call `Detach` on the returned
/// portal to unlock the array. However, this will invalidate the portal.)
///
/// **Note:** The returned portal cannot be used in the execution environment. This is because
/// the portal will not work on some devices like GPUs. To get a portal that will work in the
/// execution environment, use `PrepareForInput`.
///
VTKM_CONT ReadPortalType ReadPortal() const;
/// \brief Get an array portal that can be used in the control environment.
///
/// The returned array can be used in the control environment to reand and write values to the
/// array.
///
/// **Note:** The returned portal will prevent any reads, writes, or modifications to the array.
/// To ensure that the data pointed to by the portal is valid, this `ArrayHandle` will be locked
/// to any modifications while the portal remains in scope. Also, to make sure that no reads get
/// out of sync, reads other than the returned portal are also blocked. (You can call `Detach` on
/// the returned portal to unlock the array. However, this will invalidate the portal.)
///
/// **Note:** The returned portal cannot be used in the execution environment. This is because
/// the portal will not work on some devices like GPUs. To get a portal that will work in the
/// execution environment, use `PrepareForInput`.
///
VTKM_CONT WritePortalType WritePortal() const;
/// Returns the number of entries in the array.
///
@ -617,7 +672,7 @@ protected:
vtkm::Id GetNumberOfValues(LockType& lock) const;
VTKM_CONT
void ReleaseResourcesExecutionInternal(LockType& lock)
void ReleaseResourcesExecutionInternal(LockType& lock) const
{
if (this->Internals->IsExecutionArrayValid(lock))
{

52
vtkm/cont/ArrayHandle.hxx
View File

@ -118,7 +118,7 @@ const typename ArrayHandle<T, S>::StorageType& ArrayHandle<T, S>::GetStorage() c
}
template <typename T, typename S>
typename ArrayHandle<T, S>::PortalControl ArrayHandle<T, S>::GetPortalControl()
typename ArrayHandle<T, S>::StorageType::PortalType ArrayHandle<T, S>::GetPortalControl()
{
LockType lock = this->GetLock();
@ -139,7 +139,8 @@ typename ArrayHandle<T, S>::PortalControl ArrayHandle<T, S>::GetPortalControl()
}
template <typename T, typename S>
typename ArrayHandle<T, S>::PortalConstControl ArrayHandle<T, S>::GetPortalConstControl() const
typename ArrayHandle<T, S>::StorageType::PortalConstType ArrayHandle<T, S>::GetPortalConstControl()
const
{
LockType lock = this->GetLock();
@ -155,6 +156,53 @@ typename ArrayHandle<T, S>::PortalConstControl ArrayHandle<T, S>::GetPortalConst
}
}
template <typename T, typename S>
typename ArrayHandle<T, S>::ReadPortalType ArrayHandle<T, S>::ReadPortal() const
{
LockType lock = this->GetLock();
vtkm::cont::Token token;
this->WaitToRead(lock, token);
this->SyncControlArray(lock);
if (this->Internals->IsControlArrayValid(lock))
{
token.Attach(
*this, this->Internals->GetReadCount(lock), lock, &this->Internals->ConditionVariable);
return ReadPortalType(std::move(token),
this->Internals->GetControlArray(lock)->GetPortalConst());
}
else
{
throw vtkm::cont::ErrorInternal(
"ArrayHandle::SyncControlArray did not make control array valid.");
}
}
template <typename T, typename S>
typename ArrayHandle<T, S>::WritePortalType ArrayHandle<T, S>::WritePortal() const
{
LockType lock = this->GetLock();
vtkm::cont::Token token;
this->WaitToWrite(lock, token);
this->SyncControlArray(lock);
if (this->Internals->IsControlArrayValid(lock))
{
// If the user writes into the iterator we return, then the execution
// array will become invalid. Play it safe and release the execution
// resources. (Use the const version to preserve the execution array.)
this->ReleaseResourcesExecutionInternal(lock);
token.Attach(
*this, this->Internals->GetWriteCount(lock), lock, &this->Internals->ConditionVariable);
return WritePortalType(std::move(token), this->Internals->GetControlArray(lock)->GetPortal());
}
else
{
throw vtkm::cont::ErrorInternal(
"ArrayHandle::SyncControlArray did not make control array valid.");
}
}
template <typename T, typename S>
vtkm::Id ArrayHandle<T, S>::GetNumberOfValues(LockType& lock) const
{

5
vtkm/cont/Token.cxx
View File

@ -56,6 +56,11 @@ vtkm::cont::Token::Token()
{
}
vtkm::cont::Token::Token(Token&& rhs)
: Internals(std::move(rhs.Internals))
{
}
vtkm::cont::Token::~Token()
{
this->DetachFromAll();

1
vtkm/cont/Token.h
View File

@ -67,6 +67,7 @@ class VTKM_CONT_EXPORT Token final
public:
VTKM_CONT Token();
VTKM_CONT Token(Token&& rhs);
VTKM_CONT ~Token();
/// Use this type to represent counts of how many tokens are holding a resource.

28
vtkm/cont/internal/ArrayHandleBasicImpl.h
View File

@ -347,8 +347,14 @@ public:
using StorageTag = ::vtkm::cont::StorageTagBasic;
using StorageType = vtkm::cont::internal::Storage<T, StorageTag>;
using ValueType = T;
using PortalControl = typename StorageType::PortalType;
using PortalConstControl = typename StorageType::PortalConstType;
using WritePortalType = vtkm::cont::internal::ArrayPortalToken<typename StorageType::PortalType>;
using ReadPortalType =
vtkm::cont::internal::ArrayPortalToken<typename StorageType::PortalConstType>;
using PortalControl VTKM_DEPRECATED(1.6, "Use ArrayHandle::WritePortalType instead.") =
typename StorageType::PortalType;
using PortalConstControl VTKM_DEPRECATED(1.6, "Use ArrayHandle::ReadPortalType instead.") =
typename StorageType::PortalConstType;
template <typename DeviceTag>
struct ExecutionTypes
@ -380,10 +386,22 @@ public:
VTKM_CONT StorageType& GetStorage();
VTKM_CONT const StorageType& GetStorage() const;
VTKM_CONT PortalControl GetPortalControl();
VTKM_CONT PortalConstControl GetPortalConstControl() const;
VTKM_CONT vtkm::Id GetNumberOfValues() const;
VTKM_CONT
VTKM_DEPRECATED(1.6,
"Use ArrayHandle::WritePortal() instead. "
"Note that the returned portal will lock the array while it is in scope.")
typename StorageType::PortalType GetPortalControl();
VTKM_CONT
VTKM_DEPRECATED(1.6,
"Use ArrayHandle::ReadPortal() instead. "
"Note that the returned portal will lock the array while it is in scope.")
typename StorageType::PortalConstType GetPortalConstControl() const;
VTKM_CONT ReadPortalType ReadPortal() const;
VTKM_CONT WritePortalType WritePortal() const;
VTKM_CONT void Allocate(vtkm::Id numberOfValues);
VTKM_CONT void Shrink(vtkm::Id numberOfValues);
VTKM_CONT void ReleaseResourcesExecution();
@ -438,7 +456,7 @@ public:
private:
VTKM_CONT void SyncControlArray(LockType& lock) const;
VTKM_CONT void ReleaseResourcesExecutionInternal(LockType& lock);
VTKM_CONT void ReleaseResourcesExecutionInternal(LockType& lock) const;
/// Acquires a lock on the internals of this `ArrayHandle`. The calling
/// function should keep the returned lock and let it go out of scope

56
vtkm/cont/internal/ArrayHandleBasicImpl.hxx
View File

@ -119,7 +119,7 @@ ArrayHandle<T, StorageTagBasic>::GetStorage() const
}
template <typename T>
typename ArrayHandle<T, StorageTagBasic>::PortalControl
typename ArrayHandle<T, StorageTagBasic>::StorageType::PortalType
ArrayHandle<T, StorageTagBasic>::GetPortalControl()
{
LockType lock = this->GetLock();
@ -128,7 +128,6 @@ ArrayHandle<T, StorageTagBasic>::GetPortalControl()
//CheckControlArrayValid will throw an exception if this->Internals->ControlArrayValid
//is not valid
// If the user writes into the iterator we return, then the execution
// array will become invalid. Play it safe and release the execution
// resources. (Use the const version to preserve the execution array.)
@ -138,9 +137,8 @@ ArrayHandle<T, StorageTagBasic>::GetPortalControl()
return privStorage->GetPortal();
}
template <typename T>
typename ArrayHandle<T, StorageTagBasic>::PortalConstControl
typename ArrayHandle<T, StorageTagBasic>::StorageType::PortalConstType
ArrayHandle<T, StorageTagBasic>::GetPortalConstControl() const
{
LockType lock = this->GetLock();
@ -154,6 +152,54 @@ ArrayHandle<T, StorageTagBasic>::GetPortalConstControl() const
return privStorage->GetPortalConst();
}
template <typename T>
typename ArrayHandle<T, StorageTagBasic>::ReadPortalType
ArrayHandle<T, StorageTagBasic>::ReadPortal() const
{
LockType lock = this->GetLock();
vtkm::cont::Token token;
this->Internals->WaitToRead(lock, token);
this->SyncControlArray(lock);
this->Internals->CheckControlArrayValid(lock);
//CheckControlArrayValid will throw an exception if this->Internals->ControlArrayValid
//is not valid
StorageType* privStorage =
static_cast<StorageType*>(this->Internals->Internals->GetControlArray(lock));
token.Attach(this->Internals,
this->Internals->Internals->GetReadCount(lock),
lock,
&this->Internals->Internals->ConditionVariable);
return ArrayHandle<T, StorageTagBasic>::ReadPortalType(std::move(token),
privStorage->GetPortalConst());
}
template <typename T>
typename ArrayHandle<T, StorageTagBasic>::WritePortalType
ArrayHandle<T, StorageTagBasic>::WritePortal() const
{
LockType lock = this->GetLock();
vtkm::cont::Token token;
this->Internals->WaitToWrite(lock, token);
this->SyncControlArray(lock);
this->Internals->CheckControlArrayValid(lock);
//CheckControlArrayValid will throw an exception if this->Internals->ControlArrayValid
//is not valid
// If the user writes into the iterator we return, then the execution
// array will become invalid. Play it safe and release the execution
// resources. (Use the const version to preserve the execution array.)
this->ReleaseResourcesExecutionInternal(lock);
StorageType* privStorage =
static_cast<StorageType*>(this->Internals->Internals->GetControlArray(lock));
token.Attach(this->Internals,
this->Internals->Internals->GetWriteCount(lock),
lock,
&this->Internals->Internals->ConditionVariable);
return ArrayHandle<T, StorageTagBasic>::WritePortalType(std::move(token),
privStorage->GetPortal());
}
template <typename T>
vtkm::Id ArrayHandle<T, StorageTagBasic>::GetNumberOfValues() const
{
@ -277,7 +323,7 @@ void ArrayHandle<T, StorageTagBasic>::SyncControlArray(LockType& lock) const
}
template <typename T>
void ArrayHandle<T, StorageTagBasic>::ReleaseResourcesExecutionInternal(LockType& lock)
void ArrayHandle<T, StorageTagBasic>::ReleaseResourcesExecutionInternal(LockType& lock) const
{
this->Internals->ReleaseResourcesExecutionInternal(lock);
}

64
vtkm/cont/internal/ArrayPortalToken.h Normal file
View File

@ -0,0 +1,64 @@
//============================================================================
// Copyright (c) Kitware, Inc.
// All rights reserved.
// See LICENSE.txt for details.
//
// This software is distributed WITHOUT ANY WARRANTY; without even
// the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
// PURPOSE. See the above copyright notice for more information.
//============================================================================
#ifndef vtk_m_cont_internal_ArrayPortalToken_h
#define vtk_m_cont_internal_ArrayPortalToken_h
#include <vtkm/cont/ArrayPortal.h>
#include <vtkm/cont/Token.h>
#include <vtkm/internal/ArrayPortalHelpers.h>
namespace vtkm
{
namespace cont
{
namespace internal
{
/// \brief A wrapper around an `ArrayPortal` that also holds its state with a token.
///
/// Usually when you get an `ArrayPortal`, you have to associate it with a `Token`
/// object to define its scope. This class wraps around an `ArrayPortal` and also
/// holds its own `Token` that should be attached appropriately to the source
/// `ArrayHandle`. When this class goes out of scope, so will its `Token`.
///
/// Because `Token`s only work in the control environment, so it is for this class.
///
template <typename PortalType>
class VTKM_ALWAYS_EXPORT ArrayPortalToken : public PortalType
{
std::shared_ptr<vtkm::cont::Token> Token;
public:
template <typename... PortalArgs>
VTKM_CONT ArrayPortalToken(vtkm::cont::Token&& token, PortalArgs&... args)
: PortalType(std::forward<PortalArgs>(args)...)
, Token(new vtkm::cont::Token(std::move(token)))
{
}
template <typename... PortalArgs>
VTKM_CONT ArrayPortalToken(PortalArgs&... args)
: PortalType(std::forward<PortalArgs>(args)...)
, Token(new vtkm::cont::Token)
{
}
VTKM_CONT void Detach() const
{
this->Token.DetachFromAll();
this->Portal = PortalType();
}
};
}
}
} // namespace vtkm::cont::internal
#endif //vtk_m_cont_internal_ArrayPortalToken_h

1
vtkm/cont/internal/CMakeLists.txt
View File

@ -16,6 +16,7 @@ set(headers
ArrayManagerExecution.h
ArrayManagerExecutionShareWithControl.h
ArrayPortalFromIterators.h
ArrayPortalToken.h
ArrayTransfer.h
AtomicInterfaceControl.h
AtomicInterfaceExecution.h

2
vtkm/cont/testing/TestingDeviceAdapter.h
View File

@ -542,7 +542,7 @@ private:
"Shrink did not set size of array handle correctly.");
// Get the array back and check its values.
StorageType::PortalConstType checkPortal = handle.GetPortalConstControl();
auto checkPortal = handle.GetPortalConstControl();
VTKM_TEST_ASSERT(checkPortal.GetNumberOfValues() == ARRAY_SIZE, "Storage portal wrong size.");
for (vtkm::Id index = 0; index < ARRAY_SIZE; index++)

14
vtkm/internal/ArrayPortalHelpers.h
View File

@ -44,6 +44,16 @@ struct PortalSupportsSetsImpl
using type = decltype(has<PortalType>(0));
};
template <typename PortalType>
struct PortalSupportsIteratorsImpl
{
template <typename U, typename S = decltype(std::declval<U>().GetIteratorBegin())>
static std::true_type has(int);
template <typename U>
static std::false_type has(...);
using type = decltype(has<PortalType>(0));
};
} // namespace detail
template <typename PortalType>
@ -53,6 +63,10 @@ using PortalSupportsGets =
template <typename PortalType>
using PortalSupportsSets =
typename detail::PortalSupportsSetsImpl<typename std::decay<PortalType>::type>::type;
template <typename PortalType>
using PortalSupportsIterators =
typename detail::PortalSupportsIteratorsImpl<typename std::decay<PortalType>::type>::type;
}
} // namespace vtkm::internal