Enabling viewport denoising causes Cycles to use a multi-device, which always returned NULL when
asked for OSL memory and would subsequently crash. This fixes that by returning the correct OSL
memory pointer from the CPU device in the special viewport denoising multi-device.
This feature takes some inspiration from
"RenderMan: An Advanced Path Tracing Architecture for Movie Rendering" and
"A Hierarchical Automatic Stopping Condition for Monte Carlo Global Illumination"
The basic principle is as follows:
While samples are being added to a pixel, the adaptive sampler writes half
of the samples to a separate buffer. This gives it two separate estimates
of the same pixel, and by comparing their difference it estimates convergence.
Once convergence drops below a given threshold, the pixel is considered done.
When a pixel has not converged yet and needs more samples than the minimum,
its immediate neighbors are also set to take more samples. This is done in order
to more reliably detect sharp features such as caustics. A 3x3 box filter that
is run periodically over the tile buffer is used for that purpose.
After a tile has finished rendering, the values of all passes are scaled as if
they were rendered with the full number of samples. This way, any code operating
on these buffers, for example the denoiser, does not need to be changed for
per-pixel sample counts.
Reviewed By: brecht, #cycles
Differential Revision: https://developer.blender.org/D4686
This fixes denoising being delayed until after all rendering has finished. Instead, tile-based
denoising is now part of the "RENDER" task again, so that it is all in one task and does not
cause issues with dedicated task pools where tasks are serialized.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6940
This makes the memory allocation for the denoiser state use the memory allocator in Cycles, which
will evict textures to host memory when there is not enough space on the device. This means the
allocation for the denoiser state won't just fail if there is no more space and instead more space is
made for it to work. Also simplifies code somewhat.
This modifies the common CUDA implementation for adaptive kernel compilation slightly to support both CUBIN and PTX output (the latter which is then used in the OptiX device). It also fixes adaptive kernel compilation on Windows.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6851
MSVC does not have -march=native, so the kernel gets built without AVX2 and
BVH8 support. The code assumed it to be available and crashed
Differential Revision: https://developer.blender.org/D6082
This reduces code duplication between the CUDA and OptiX device implementations: The CUDA device
class is now split into declaration and definition (similar to the OpenCL device) and the OptiX device
class implements that and only overrides the functions it actually has to change, while using the CUDA
implementation for everything else.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6814
The OptiX denoiser can be a great help when rendering in the viewport, since it is really fast
and needs few samples to produce convincing results. This patch therefore adds support for
using any Cycles denoiser in the viewport also (but only the OptiX one is selectable because
the NLM one is too slow to be usable currently). It also adds support for denoising on a
different device than rendering (so one can e.g. render with the CPU but denoise with OptiX).
Reviewed By: #cycles, brecht
Differential Revision: https://developer.blender.org/D6554
Not sure when this happened but apparently the lower bar is now windows 7 [1]
This patch bumps to API version to 0x0601 (Win7) and cleans up any uses that
worked around the globally set API version.
[1] https://www.blender.org/download/requirements/
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6758
This is a more correct fix to the issue Brecht was fixing in D6600.
While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).
Top-level idea: leave it to CMake to keep track of dependency graph.
The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.
It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.
For example, this order will likely fail:
libbf_blenfont.a libfreetype6.a libbf_blenfont.a
This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.
General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.
The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.
Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.
The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:
- Have every library prefixed as "optimized" or "debug" if
separation is needed (non-prefixed libraries will be considered
"generic").
- Loop through libraries passed to function and do simple parsing
which will look for "optimized" and "debug" words and specify
following library to corresponding category.
This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.
Tested the following configurations on Linux, macOS and Windows:
- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer
NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.
Differential Revision: https://developer.blender.org/D6642
The OptiX SRT motion expects a motion defined by translation,
rotation, shear and scale, but the matrix decomposition code in
Cycles was not able to extract shear information and instead
produced a stretch matrix with the information baked in. This
caused conflicting transforms between traversal and shading
and lead to render artifacts.
This patch changes the matrix decomposition to produce factors
inline with what OptiX expects to fix that.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6605
Commit baeb11826b9fe5525db6dd05ba5271949079fc1e switched memory
allocation for the motion transform to use CUDA directly, instead of going
through abstractions. But no CUDA context was set active before those
were called, so the calls failed. This fixes that by binding a context beforehand.
The `optixAccelBuild` API throws an error when the property to get compacted size is passed in
without the `OPTIX_BUILD_FLAG_ALLOW_COMPACTION` flag set. This is not currently hit
because `background` is always true (set in `mem_alloc`), but would become an issue once that
is sorted out, so fixing it now to be safe.
This patch adds support for the OptiX denoiser as an alternative to the existing NLM denoiser in Cycles. It's re-using the same denoising architecture based on tiles and therefore implicitly also works with multiple GPUs.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D6395
In the current OpenCL implementation we have a work-around for platforms
that didn't support NULL pointers. We used to replace all NULLs and
empty arrays with a pointer to a single byte on the OpenCL Device.
During investigation of {T65924} it was asked to remove this work-around
for testing. This change improves the render times.
SCENE | BEFORE | AFTER
--------------------+--------+-------
bmw27 | 108 | 89
barbershop_interior | 867 | 673
classroom | 270 | 173
fishy_cat | 244 | 196
koro | 249 | 207
pavillon_barcelona | 582 | 414
Note that this change does not fix T65924 it just improves the
rendering performance for OpenCL. We haven't tested this patch on all
platforms so we should keep an eye out on the tracker.
Reviewed By: sergey
Differential Revision: https://developer.blender.org/D6391
This adds compaction support for OptiX acceleration structures, which reduces the device memory footprint in a post step after building. Depending on the scene this can reduce the amount of used device memory quite a bit and even improve performance (smaller acceleration structure improves cache usage). It's only enabled for background renders to make acceleration structure builds fast in viewport.
Also fixes a bug in the memory management for OptiX acceleration structures: These were held in a dynamic vector of 'device_memory' instances and used the mem_alloc/mem_free functions. However, those keep track of memory instances in the 'cuda_mem_map' via pointers to 'device_memory' (which works fine everywhere else since those are never copied/moved). But in the case of the vector, it may decide to reallocate at some point, which invalidates those pointers and would result in some nasty accesses to invalid memory. So it is not actually safe to move a 'device_memory' object and therefore this removes the move operator overloads again.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D6369
When encountering an error during context creation, the "OptiXDevice" constructor aborts early.
This means the "cuda_stream" vector is never resized and the destructor iterated over non-existent data.
The acceleration structure built by OptiX may be different between GPUs, so cannot assume the memory size is the same for all.
This fixes that by moving the memory management for all OptiX acceleration structures into the responsibility of each device (was already the case for BLAS previously, now for TLAS too).
Calling "OptiXDevice::load_kernels" multiple times would call "optixPipelineDestroy" on a pipeline
pointer that may have already been deleted previously (since the PIP_SHADER_EVAL pipeline is only
created conditionally).
This change also avoids a CUDA kernel reload every time this is called. The CUDA kernels are
precompiled and don't change, so there is no need to reload them every time.
The multi device code did not correctly handle cases where some GPUs store a
resource in device memory and others store it in host mapped memory.
Differential Revision: https://developer.blender.org/D6126
Was caused by D6068, which did not handle "MEM_PIXELS" memory
when not in background mode. Before that it always fell back to using
generic device memory, so restoring that behavior. In future this
should be changes to use OpenGL interop for optimal performance.
The OptiX implementation wasn't trying to allocate memory on the host if device allocation failed, while the CUDA implementation did. This copies the implementation over to OptiX to remedy that.
Differential Revision: https://developer.blender.org/D6068
This is intended for developers on Windows primarily:
Now, CUDA architectures of type compute_xx are supported. This allows for quicker builds,
at the expense of the CUDA driver running ptxas the first time a kernel is loaded.
Differential Revision: https://developer.blender.org/D5953
Rendering would produce invalid results or crash if the Vector pass was active but motion blur was inactive. This caused the OptiX BVH to be built with motion (because objects reported motion available), but the pipeline to be built without motion support (since with disabled motion blur this is not in the list of requested features). The two are not compatible and therefore caused issues. This patch fixes that by not building the BVH with motion if motion blur is not active (which makes sense).
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5968
Curves with motion blur produced wrong results with OptiX (T69801). This is because the AABBs for the motion steps were calculated from incorrect attribute data because the offset into the attribute data array was incorrect.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5961
The "optix_devices" array was not freed on exit, which caused a memory leak (see T69801).
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5944
OpenCL Parallel compilation only works inside Blender. When using cycles in a different setup (standaline or other software) it failed compiling kernels as they don't have the appropriate Python API and command line arguments.
This change introduces a `running_inside_blender` debug flag, that triggers out of process compilation of the kernels. Compilation still happens in subthread that enabled the preview kernels and compilation of the kernels during BVH building
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5439
Before there were two options: Paste to original layer called "Paste" and Paste to active layer called "Paste & Merge"
Now, by default the paste is in active layer and the "Paste & Merge" has been renamed "Paste".
For old "Paste", now is called "Paste by Layer" and it's not the default value anymore.
Note: Minor edits to add icons not present in Differential revision.
Differential Revision: https://developer.blender.org/D5591
x64 builds with WITH_CYCLES_OPTIMIZED_KERNEL_SSE2 not defined
since SSE2 is the lower bar for x64 cpus. Turning the architecture
logging related if into the last if in the architecture detection
chain, which will never execute unless you turn off all kernels
in de debug flags.
Reviewers: brecht
Differential Revision: https://developer.blender.org/D5579
The issue was caused by un-initialized local storage for volume
intersection hits which are supposed to be stored in per-thread
KernelGlobals.
Fix is to make thread_shader() be the same as thread_render() in
respect of KernelGlobals.
Reviewers: brecht
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5230
We want users to go to the current version for their current version
when possible if not point to latest.
/dev should really only be for development related work. End users
should not be browsing /dev unless they are reading about upcoming
features ahead of time.
When compute preemption is available we schedule more work which is more
efficient. However the CUDA driver appears to be incorrectly reporting this as
unavailable, even though it should be supported starting with Windows 10 1803
and Pascal and Turing (10x0 and 20x0) graphics cards.
This reduces render time by about a 25% difference on our benchmark scenes. On
Linux compute preemption appears to be reported correctly.
Previously, bright edges (e.g. caused by rim lighting) would sometimes get
halos around them after denoising.
This change introduces a log(1+x) highlight compression step that is performed
before denoising and reversed afterwards. That way, the denoising algorithm
itself operates in the compressed space and therefore bright edges cause less
numerical issues.
The kernel does not use AVX2 vectorization, and trying to use BVH8 was
leading to an empty scenes.
Fixes T64624: Ctest : Win32 + AVX2 fails virtually all cycles tests
This adds our own OSL texture handle, that has info for OIIO textures or our
own custom texture types. A filename to handle hash map is used for lookups.
This is efficient because it happens at OSL compile time, because the optimizer
can figure out constant strings and replace them with texture handles.
It's effectively always enabled, only not on some unsupported OpenCL devices.
For testing those it's not useful to disable these features. This is replaced
by the more fine grained feature toggles that we have now.
This version fixes various bugs, and there is no need anymore to use both
9.1 and 10.0 for different cards.
There is a bug related to WITH_CYCLES_CUBIN_COMPILER and bump mapping in the
regression tests, so that remains disabled same as it was for CUDA 10.0.
Fix T59286: CUDA bake failing on some cards.
Fix T56858: CUDA 9.2 and 10 issues.
The main goals of this change is faster starting when using foreground
rendering.
This patch will build kernels in parallel to the update process of
the scene. When these optimized kernels are not available (yet) an AO
kernel will be used.
These AO kernels are fast to compile (3-7 seconds) and can be
reused by all scenes. When the final kernels become available we
will switch to these kernels.
In background mode the AO kernels will not be used.
Some kernels are being used during Scene update (displace, background
light). When these kernels are being used the process can halt until
these become available.
Reviewed By: brecht, #cycles
Maniphest Tasks: T61752
Differential Revision: https://developer.blender.org/D4428
The functions that determine the program name + filename of kernels
were missing some base kernels like denoising and base. For completeness
I added those kernels so the function returns the correct results.
This patch will reduce the number of times that we need to
recompile kernels. It does this by (en/dis)abling features
by default. So when the user needs them that the kernels are
already available.
Other features are enabled by default for background and foreground
rendering. When in background rendering the user wants the best
render performance. When in foreground rendering the user wants
the least amount of recompilations.
Enabling volumetrics or subdivision evaluation will still trigger
a recompilation during foreground rendering.
Reviewed By: #cycles, brecht
Differential Revision: https://developer.blender.org/D4485
Part of the cleanup of the OpenCL codebase.
Single program is not effective when using OpenCL, it is slower
to compile and slower during rendering (when used in for example
`barbershop` or `victor`).
Reviewers: brecht, #cycles
Maniphest Tasks: T62267
Differential Revision: https://developer.blender.org/D4481
Displacement and Background kernels are selectively used, but always compiled. This patch will not compile these kernels when they are not needed.
Displacement kernel is only used for true displacement.
Background kernel is only used when there is a (Cycles)Light of type `LIGHT_BACKGROUND`.
Reviewed By: brecht, #cycles
Tags: #cycles
Maniphest Tasks: T61971
Differential Revision: https://developer.blender.org/D4412
The goal of this patch is to have limit the number of times
kernels needs to be compiled and are reused as kernels with
different compile directives can lead to identical same
binaries.
The implementation does this by stripping the compile directives.
and reshuffling kernels so the output is more likely to be the
same.
We focussed on the kernels where it was easy to detect and maintain
(bundle, bake, displace, do_volume and background). More optimizations
could be done but they are probably less obvious.
Merged the data_init and state_buffer_size kernels to split_bundle.
This patch will also remove empty kernels for do_volume and bake
when their features are not enabled.
When using the benchmark files there are less background, bake and
do_volume kernels compiled.
Fix: T61576, T61501, T61466
Reviewed By: brecht, #cycles
Differential Revision: https://developer.blender.org/D4390