This is related to T76659.
This just renames data type names to `CD_PROP_STRING`, `CD_PROP_FLOAT`
and `CD_PROP_INT32`. It makes them a bit more specific and removes
unnecessary abbreviations.
Reviewers: brecht
Differential Revision: https://developer.blender.org/D7980
Previously, this function would expect a callback function as parameter.
This behavior is now in Map.lookup_or_add_cb. The new version just
takes the key and value directly.
This also renames `MutableArrayRef` to `MutableSpan`.
The name "Span" works better, because `std::span` will provide
similar functionality in C++20. Furthermore, a shorter, more
concise name for a common data structure is nice.
The main focus here was to improve the docs significantly. Furthermore,
I reimplemented `Set`, `Map` and `VectorSet`. They are now (usually)
faster, simpler and more customizable. I also rewrote `Stack` to make
it more efficient by avoiding unnecessary copies.
Thanks to everyone who helped with constructive feedback.
Approved by brecht and sybren.
Differential Revision: https://developer.blender.org/D7931
This adds a new `CPPType` that encapsulates information about how to handle
instances of a specific data type. This is necessary for the function evaluation
system, which will be used to evaluate most of the particle node trees.
Furthermore, this adds an `IndexMask` class which offers a surprisingly useful
abstraction over an array containing unsigned integers. It makes two assumptions
about the underlying integer array:
* The integers are in ascending order.
* There are no duplicates.
`IndexMask` will be used to "select" certain particles that will be
processed in a data-oriented way. Sometimes, operations don't have to
be applied to all particles, but only some, those that are in the indexed by
the `IndexMask`. The two limitations imposed by an `IndexMask` allow for
better performance.
Reviewers: brecht
Differential Revision: https://developer.blender.org/D7957
Add TBB::flow graph scheduling to BLI_task.
Using flow graphs, a graph of nodes (tasks) and links can be defined.
Work can flow though the graph. During this process the execution of the nodes will be
scheduled among the available threads.
We are planning to use this to improve the threading in the draw manager.
The implemented API is still limited it only supports sequential flows. Joins and buffers
are not supported. We could eventually support them as part of an CPP API. These features
from uses compile time templates and are hard to make a clean C-API for this.
Reviewed By: Sergey Sharybin, Brecht van Lommel
Differential Revision: https://developer.blender.org/D7578
The long-term goal is to move code out of `abc_util.{h,cc}` into either
files with better, more concrete names, or simply into the one file
where they are used.
No functional changes.
Unfortunately there is no `EXPECT_NOT_LT`; as the `HierarchyContext` only
has an `operator<()` function, testing for `(A < B) == false` is different
than simply testing `(A >= B)`.
No functional changes.
This introduces unittests for FCurve evaluation.
No functional changes to actual Blender code.
Differential Revision: https://developer.blender.org/D6778
This patch enables TBB as the default task scheduler. TBB stands for Threading Building Blocks and is developed by Intel. The library contains several threading patters. This patch maps blenders BLI_task_* function to their counterpart. After this patch we can add more patterns. A promising one is TBB:graph that can be used for depsgraph, draw manager and compositor.
Performance changes depends on the actual hardware. It was tested on different hardwares from laptops to workstations and we didn't detected any downgrade of the performance.
* Linux Xeon E5-2699 v4 got FPS boost from 12 to 17 using Spring's 04_010_A.anim.blend.
* AMD Ryzen Threadripper 2990WX 32-Core Animation playback goes from 9.5-10.5 FPS to 13.0-14.0 FPS on Agent 327 , 10_03_B.anim.blend.
Reviewed By: brecht, sergey
Differential Revision: https://developer.blender.org/D7475
* Rename template parameter N to InlineBufferCapacity
* Expose InlineBufferCapacity parameter for Set and Map
* Add some comments
* Fixed an error that I introduced recently
This is not currently used and will take some work to support with TBB, so
remove it until we have a new implementation based on TBB.
Fixes T76005, parallel range pool tests failing.
Ref D7475
Instead of building on top of `BLI::Vector`, just use a raw array
and handle the growing in `BLI::VectorSet`.
After this change, the existing `EdgeSet` can be reimplemented using
`BLI::VectorSet` without performance regressions.
This was used to investigate T73840. Since the armature math is far from
simple, I thought it would be a good idea to start writing some unit
tests for it.
No functional changes in Blender itself.
In preparation of TBB we need to split the finalize function into reduce
and free. Reduce is used to combine results and free for freeing any
allocated memory.
The reduce function is called to join user data chunk into another, to reduce the
result to the original userdata_chunk memory. These functions should have no side
effects so that they can be run on any thread.
The free functions should free data created during execution (TaskParallelRangeFunc).
Original patch by Brecht van Lommel
{rB61f49db843cf5095203112226ae386f301be1e1a}.
Reviewed By: Brecht van Lommel, Bastien Montagne
Differential Revision: https://developer.blender.org/D7394
Tasks: move priority from task to task pool {rBf7c18df4f599fe39ffc914e645e504fcdbee8636}
Tasks: split task.c into task_pool.cc and task_iterator.c {rB4ada1d267749931ca934a74b14a82479bcaa92e0}
Differential Revision: https://developer.blender.org/D7385
The test failure in T75491 only showed up in debug builds because
`BLI_assert()` is a no-op in release builds. This is now replaced by a
proper GTests call to `ADD_FAILURE()`, ensuring that the test fails
regardless of build mode.
Test specs are read from strings, and there was a comma instead
of a decimal point, and then an extra decimal point in the Quad0 test.
This test has been flaky on Windows buildbot. Perhaps this is why.
This moves the `alembic`, `avi`, `collada`, and `usd` modules into a common
`io` directory.
This also cleans up some `#include "../../{somedir}/{somefile}.h"` by
adding `../../io/{somedir}` to `CMakeLists.txt` and then just using
`#include "{somefile}.h"`.
No functional changes.
Introduce new IDTypeInfo structure.
Each ID type will have its own, with some minimal basic common info,
and ID management callbacks.
This patch only does it for Object type, for demo/testing purpose.
Moving all existing IDs is a goal of next "cleanup Friday".
Note that BKE_idcode features should then be merged back into BKE_idtype -
but this will have to be done later, once all ID types have been properly
converted to the new system.
Another later TODO might be to try and add callbacks for file read/write,
and lib_query ID usages looper.
This is part of T73719.
Thanks to @brecht for initial idea, and reviewing the patch.
Differential Revision: https://developer.blender.org/D6966
As explained in T65568 by @LucaRood, the self collision system should exclude triangles that are connected by sewing springs.
Differential Revision: https://developer.blender.org/D6911
The 'random' unit tests and some examples from the new boolean code
triggered asserts and crashes. This fixes those.
There is a new flag in the input that optionally disables a pass
over input to snap segment edges to other segments.
The changes come from the `functions` branch, where I'm using
these structures a lot.
This also includes a new `BLI::Optional<T>` type, which is similar
to `std::Optional<T>` which can be used when Blender starts using
C++17.
A big rework of the code now uses exact predicates for orientation
and incircle. Also switched the main algorithm to use a faster
divide and conquer algorithm, which is possible with the exact
predicates.
The dependency graph has to know whether a driver must be re-evaluated
every frame due to a dependency on the current frame number. For python
drivers it was using a heuristic based on searching for certain sub-
strings in the expression, notably including '('.
When the expression is actually evaluated using Python, this can't be
easily improved; however if the Simple Expression evaluator is used,
this check can be done precisely by accessing the parsed data.
Differential Revision: https://developer.blender.org/D6624
This is a more correct fix to the issue Brecht was fixing in D6600.
While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).
Top-level idea: leave it to CMake to keep track of dependency graph.
The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.
It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.
For example, this order will likely fail:
libbf_blenfont.a libfreetype6.a libbf_blenfont.a
This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.
General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.
The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.
Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.
The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:
- Have every library prefixed as "optimized" or "debug" if
separation is needed (non-prefixed libraries will be considered
"generic").
- Loop through libraries passed to function and do simple parsing
which will look for "optimized" and "debug" words and specify
following library to corresponding category.
This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.
Tested the following configurations on Linux, macOS and Windows:
- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer
NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.
Differential Revision: https://developer.blender.org/D6642
`posix_memalign` requires the `alignment` to be at least `sizeof(void *)`.
Previously, `MEM_mallocN_aligned` would simply return `NULL` if a too small
`alignment` was used. This was an OS specific issue.
The solution is to use a minimal alignment of `8` for all aligned allocations.
The unit tests have been extended to test more possible alignments (some
of which were broken before).
Reviewers: brecht
Differential Revision: https://developer.blender.org/D6660
This add a basic sanity check that validates
the features we use from ffmpeg are actually
available
Differential Revision: https://developer.blender.org/D5999
Reviewed By: sybren
E.g. "Cube" would be placed after "Cube.001", which is not what you'd
expect. 2.80 handled this correctly.
Loosely based on D6525 by @radcapricorn, but found a bug in that and
prefered to do some further adjustments.
Also activates test for this case.
Adds tests covering a good amount of common cases and corner cases for
`BLI_strcasecmp_natural()`.
Could of course always add more tests for more cases, but don't want to
spend too much time on this.
This commit introduces the first version of an exporter to Pixar's
Universal Scene Description (USD) format.
Reviewed By: sergey, LazyDodo
Differential Revision: https://developer.blender.org/D6287
- The USD libraries are built by `make deps`, but not yet built by
install_deps.sh.
- Only experimental support for instancing; by default all duplicated
objects are made real in the USD file. This is fine for exporting a
linked-in posed character, not so much for thousands of pebbles etc.
- The way materials and UV coordinates and Normals are exported is going
to change soon.
- This patch contains LazyDodo's fixes for building on Windows in D5359.
== Meshes ==
USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.
Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.
Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.
The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.
The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.
A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.
== Animation ==
Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.
The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.
== Support for simple preview materials ==
Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.
When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.
The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.
Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.
Materials are exported as a flat list under a top-level '/_materials'
namespace. This inhibits instancing of the objects using those
materials, so this is subject to change.
== Hair ==
Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.
== Camera ==
Only perspective cameras are supported for now.
== Particles ==
Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).
Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.
== Instancing/referencing ==
This exporter has experimental support for instancing/referencing.
Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.
I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.
== Lights ==
USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.
== Fluid vertex velocities ==
Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.
== The Building Process ==
- USD is built as monolithic library, instead of 25 smaller libraries.
We were linking all of them as 'whole archive' anyway, so this doesn't
affect the final file size. It does, however, make life easier with
respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into datafiles/usd; they
are required on every platform. Set the `PXR_PATH_DEBUG` to any value
to have the USD library print the paths it uses to find those files.
- USD is patched so that it finds the aforementioned JSON files in a path
that we pass to it from Blender.
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
building the tools in its `bin` directory. This is sent as a pull
request at https://github.com/PixarAnimationStudios/USD/pull/1048
This new test class minimally sets up Blender so that it can load blend
files and construct a depsgraph without crashing.
Note that it hasn't been tested on very complex blend files, so it may
still crash when the loaded blend file references/requires uninitialised
data structures.
The test will certainly crash with Blend files created with Blender
older than 2.80, as the versioning code requires space types to be
registered. This is normally done by initialising the window manager,
which is not done in this test. The WM requires Python to run, which in
turn requires that Blender finds the release directory in the same
directory that contains the running executable, which is not the case
for GTest tests (they are written to `bin/tests/executablename`.
Reviewed By: sergey, mont29
Differential Revision: https://developer.blender.org/D6246
This code allows to push a set of different operations all based on
iterations over a range of indices, and then process them all at once
over multiple threads.
This commit also adds unit tests for both old un-pooled, and new pooled
task_parallel_range family of functions, as well as some basic
performances tests.
This is mainly interesting for relatively low amount of individual
tasks, as expected.
E.g. performance tests on a 32 threads machine, for a set of 10
different tasks, shows following improvements when using pooled version
instead of ten sequential calls to BLI_task_parallel_range():
| Num Items | Sequential | Pooled | Speed-up |
| --------- | ---------- | ------- | -------- |
| 10K | 365 us | 138 us | 2.5 x |
| 100K | 877 us | 530 us | 1.66 x |
| 1000K | 5521 us | 4625 us | 1.25 x |
Differential Revision: https://developer.blender.org/D6189
Note: Compared to previous commit yesterday, this reworks atomic handling in
parallel iter code, and fixes a dummy double-free bug.
Now we should only use the two critical values for synchronization from
atomic calls results, which is the proper way to do things.
Reading a value after an atomic operation does not guarantee you will
get the latest value in all cases (especially on Windows release builds
it seems).
This reverts commit f9028a3be1f77c01edca44a68894e2ba9d9cfb14.
This is giving weird heisenbug crash on only Windows release builds...
Reverting until we understand to issue.
This code allows to push a set of different operations all based on
iterations over a range of indices, and then process them all at once
over multiple threads.
This commit also adds unit tests for both old un-pooled, and new pooled
`task_parallel_range` family of functions, as well as some basic
performances tests.
This is mainly interesting for relatively low amount of individual
tasks, as expected.
E.g. performance tests on a 32 threads machine, for a set of 10
different tasks, shows following improvements when using pooled version
instead of ten sequential calls to `BLI_task_parallel_range()`:
| Num Items | Sequential | Pooled | Speed-up |
| --------- | ---------- | ------- | -------- |
| 10K | 365 us | 138 us | 2.5 x |
| 100K | 877 us | 530 us | 1.66 x |
| 1000K | 5521 us | 4625 us | 1.25 x |
Differential Revision: https://developer.blender.org/D6189
This new function is part of the 'parallel for loops' functions. It
takes an iterator callback to generate items to be processed, in
addition to the usual 'process' func callback.
This allows to use common code from BLI_task for a wide range of custom
iteratiors, whithout having to re-invent the wheel of the whole tasks &
data chuncks handling.
This supports all settings features from `BLI_task_parallel_range()`,
including dynamic and static (if total number of items is knwon)
scheduling, TLS data and its finalize callback, etc.
One question here is whether we should provide usercode with a spinlock
by default, or enforce it to always handle its own sync mechanism.
I kept it, since imho it will be needed very often, and generating one
is pretty cheap even if unused...
----------
Additionaly, this commit converts (currently unused)
`BLI_task_parallel_listbase()` to use that generic code. This was done
mostly as proof of concept, but performance-wise it shows some
interesting data, roughly:
- Very light processing (that should not be threaded anyway) is several
times slower, which is expected due to more overhead in loop management
code.
- Heavier processing can be up to 10% quicker (probably thanks to the
switch from dynamic to static scheduling, which reduces a lot locking
to fill-in the per-tasks chunks of data). Similar speed-up in
non-threaded case comes as a surprise though, not sure what can
explain that.
While this conversion is not really needed, imho we should keep it
(instead of existing code for that function), it's easier to have
complex handling logic in as few places as possible, for maintaining and
for improving it.
Note: That work was initially done to allow for D5372 to be possible... Unfortunately that one proved to be not better than orig code on performances point of view.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D5371
The function now allows custom return types defined
by the callbacks. This can be useful when a user of the
data structure has to implement some custom behavior.
This commit adds some new hashing based data structures to blenlib.
All of them use open addressing with probing currently.
Furthermore, they support small object optimization, but it is not
customizable yet. I'll add support for this when necessary.
The following main data structures are included:
**Set**
A collection of values, where every value must exist at most once.
This is similar to a Python `set`.
**SetVector**
A combination of a Set and a Vector. It supports fast search for
elements and maintains insertion order when there are no deletes.
All elements are stored in a continuous array. So they can be
iterated over using a normal `ArrayRef`.
**Map**
A set of key-value-pairs, where every key must exist at most once.
This is similar to a Python `dict`.
**StringMap**
A special map for the case when the keys are strings. This case is
fairly common and allows for some optimizations. Most importantly,
many unnecessary allocations can be avoided by storing strings in
a single buffer. Furthermore, the interface of this class uses
`StringRef` to avoid unnecessary conversions.
This commit is a continuation of rB369d5e8ad2bb7.
These two data structures reference strings somewhere in memory.
They do not own the referenced string. The string is considered
const.
A string referenced by StringRefNull can be expected to be
null-terminated. That is not the case for StringRef.
This commit is a continuation of rB369d5e8ad2bb7c2.
