Commit Graph

2504 Commits

Author SHA1 Message Date
Ray Molenkamp
98c74c6a6e Fix: Excessive (re)builds of subprojects
Recent refactor external dependencies handling (D6642)
improperly linked all library dependencies with public
linkage rather than interface linkage. Causing excessive
(re)builds of subprojects when not needed.

This patch restores the interface linkage.

Reviewed By: brecht sergey

Differential Revision: https://developer.blender.org/D6983
2020-03-02 12:42:01 -07:00
Phillip Thomas
6a6ccb26ec Build: show draco library under extern folder in Visual Studio
And other code tweaks to make this library more consistent with others.

Differential Revision: https://developer.blender.org/D6864
2020-03-01 16:19:37 +01:00
Brecht Van Lommel
1ec11363bd Build: don't include WITH_SYSTEM_GLEW as part of make lite
System GLEW often is not new enough, which gives error on startup. The build
correctness should not be affected by using lite vs. full, so better to leave
this out than save compiling one extra file.
2020-02-27 16:11:07 +01:00
Brecht Van Lommel
78391def6e Cleanup: deduplicate OpenVDB library definitions/include/libs logic
This will more important when we start using OpenVDB in more modules.
2020-02-26 18:13:14 +01:00
Brecht Van Lommel
22abc7f080 Build: add compatibility between precompiled libraries and new glibc
On Linux, precompiled libraries may be made with a glibc version that is
incompatible with the system libraries that Blender is built on. To solve
this we add a few -ffast-math symbols that can be missing.

Differential Revision: https://developer.blender.org/D6930
2020-02-26 18:13:14 +01:00
Nathan Craddock
30b3bf2b57 Build: ignore system paths when using precompiled libraries on Linux
Based on work by Nathan Craddock, with further changes to apply it to all
precompiled libraries.

Differential Revision: https://developer.blender.org/D6929
2020-02-26 18:12:18 +01:00
Bastien Montagne
6dd3e7d84d install_deps: fix several issues.
Lots of fixes and cleanups, mainly addressing:
* OpenEXR building was fully broken.
* Missing dependencies of Alembic to Boost and openEXR.
* OSL had changed its CMake parameters for custom OpenEXR install path.
* Dependencies between libs were not properly handles when switching a
lib from own build to system package.
2020-02-25 15:59:26 +01:00
d109ea5ee5 Fix make deps failing to build opencollada on Linux, due to line endings
The OpenCOLLADA package contains a mix of files with unix and dos line endings.
Now we mark the diff as a binary file so that the patch also contains a mix of
line endings that matches the package.
2020-02-24 19:23:25 +01:00
Sergey Sharybin
1f8f4c8cfe Codesign: Make file watcher robust for network errors 2020-02-21 11:00:11 +01:00
Sergey Sharybin
20b47d77e1 Codesign: Remove archive with files after they were copied 2020-02-21 09:31:55 +01:00
Ray Molenkamp
a9b184bbc4 windows: Add some more verbose logging to make.bat 2020-02-20 10:34:15 -07:00
Brecht Van Lommel
02f44049a8 Fix make deps OSL build on some systems with Qt libraries 2020-02-19 11:41:38 +01:00
Ray Molenkamp
d0c159ae97 build_deps: include venv on windows 2020-02-17 18:41:24 -07:00
Stefan Werner
f5740ec8cf Cycles: Enabled quaternion motion blur with Embree.
Bringing Embree's motion blur closer to Cycles' native blur.
This requries Embree 3.8.0 or newer.

Differential Revision: https://developer.blender.org/D6575
2020-02-17 23:44:12 +01:00
Sergey Sharybin
37cdfac6b9 Codesign: Harden check for archive being ready for sign
Seems like sometimes files are being only partially ready, which makes it so there
are unsigned files, failing to deliver fully signed bundle.

Now expected archive file size is stored into stamp file and is checked against
size of the archive file on another side.

There are some bare prints used for debugging, would need to switch it to a proper
logger (or to be removed).
2020-02-17 18:52:38 +01:00
Ray Molenkamp
8d60e6fa38 Fix: T73830 OSL not finding stdosl.h on linux
This extends FindOpenShadingLanguage.cmake to also look for the location of
stdosl.h and adds the path to the invocation of oslc to deal with the headers
being in different locations a little better.

Differential Revision: https://developer.blender.org/D6865

Reviewers: brecht
2020-02-17 10:28:12 -07:00
c498abb103 Fix macOS LLVM precompiled link error with older Xcode versions 2020-02-17 11:35:07 +01:00
Ray Molenkamp
70ff8f4a97 BPY/Windows: Do not bundle the CRT for a bpy build
- Doesn't work
- If it worked, having a different CRT than the rest of
  the process would not be a good thing.
2020-02-16 10:38:33 -07:00
50975026ff Cleanup: remove WITH_EMBREE make deps option
No other default enabled libraries have an option either.
2020-02-15 20:31:18 +01:00
39fb33f0b7 Fix macOS build warnings about Boost symbol visibility
The default in Boost changed from global to hidden, but other libraries do
not use hidden symbols which gives warnings at link time on macOS.
2020-02-15 13:44:20 +01:00
9531a8a10a Build: enable Embree by default for make deps
Ref T73819, T73778
2020-02-15 13:44:17 +01:00
Campbell Barton
1135c2cd17 Cleanup: CMake formatting 2020-02-15 10:40:41 +11:00
Stefan Werner
6608caef0e build_environment: Upgraded Embree to 3.8.0
The latest versions of Embree allow similar motion interpolation
as Cycles' own BVH.
2020-02-14 19:14:32 +01:00
Ray Molenkamp
75a5ea01c1 Cleanup/MSVC: Enable C++ conformance mode on compiler versions that support it.
MSVC has a conformance mode (/permissive-) where the C++ standard is more strictly
enforced. This mode is available on MSVC 15.5+ [1]

This patch enables this mode on compilers that support it and cleans up the few violations it threw up in the process.

- Mantaflow was using M_PI without requesting them using the _USE_MATH_DEFINES define to opt in to non default behaviour.
- Collada did not include the right header for std::cerr, this seemingly was fixed for other platforms already but put inside a platform guard.
- Ghost had some scoping issues regarding uninitialized variables and goto behaviour

Second landing of this patch, earlier commit was reverted due to some compiler configurations having slipped though testing

[1] https://docs.microsoft.com/en-us/cpp/build/reference/permissive-standards-conformance

Differential Revision: https://developer.blender.org/D6824

Reviewed By: brecht
2020-02-13 17:13:07 -07:00
Ray Molenkamp
6022cd015f Revert "Cleanup/MSVC: Enable C++ conformance mode on compiler versions that support it."
It is breaking compilation on some configurations, revert for now while
i see what is wrong.

This reverts commit 9fe469c110940af5d2525158305d5d365bd15276.
2020-02-13 07:04:30 -07:00
Ray Molenkamp
9fe469c110 Cleanup/MSVC: Enable C++ conformance mode on compiler versions that support it.
MSVC has a conformance mode (/permissive-) where the C++ standard is more strictly
enforced. This mode is available on MSVC 15.5+ [1]

This patch enables this mode on compilers that support it and cleans up the few violations it threw up in the process.

- Mantaflow was using M_PI without requesting them using the _USE_MATH_DEFINES define to opt in to non default behaviour.
- Collada did not include the right header for std::cerr, this seemingly was fixed for other platforms already but put inside a platform guard.
- Ghost had some scoping issues regarding uninitialized variables and goto behaviour

[1] https://docs.microsoft.com/en-us/cpp/build/reference/permissive-standards-conformance

Differential Revision: https://developer.blender.org/D6824

Reviewed By: brecht
2020-02-12 12:49:34 -07:00
Sergey Sharybin
5527cd4ba0 Codesign: Attempt to make macOS codesign more reliable
Is still sometimes .ready file appears prior to an actual archive.
2020-02-12 18:57:57 +01:00
Philipp Oeser
58cdab8b97 install_deps.sh: correct clang/llvm download URLs 2020-02-11 15:02:45 +01:00
Ray Molenkamp
3657bb5141 Fix: Build error on windows
USD "library" directory was not set, leading to an error
during the INSTALL phase.
2020-02-11 06:51:18 -07:00
757da61606 Fix T68243: Python sqlite module not working on macOS 2020-02-11 10:17:35 +01:00
Bastien Montagne
c6e28f6513 Updated install_deps to latest required OSL/LLVM. 2020-02-10 17:36:06 +01:00
ec116e3d49 USD: Install USD library via install_deps.sh
This commit adds the download, extract, patch, build, and install of the
Universal Scene Description (USD) library to the `install_deps.sh`
script.

Reviewed By: mont29, LazyDodo

Differential Revision: https://developer.blender.org/D6478
2020-02-10 15:07:56 +01:00
Ray Molenkamp
94c8bbec9c Windows: Improve rebuild.cmd
Previously the MSVC environment was setup every time using vcvarsall.bat
when you ran you ran rebuild.cmd, Generally not an issue but after many
rebuilds on the same console, it grows the path environment variable
beyond what is supported and building breaks.

This patch adds a check to see if the environment is setup already
and skips the call to vcvarsall.bat

Also cleans up the double build in the msbuild's version of rebuild.cmd
install.vcxproj will build all that is needed, so no need to do a regular
build first.
2020-02-08 17:14:47 -07:00
Ray Molenkamp
6a1414dc58 Build_environment: Fix unwanted exports from oidn on windows.
oidn puts dllexport on all its functions causing the
blender binary to export these symbols.

this patch fixes this unwanted behaviour.

Differential Revision: https://developer.blender.org/D6647

Reviewers: brecht , sergey
2020-02-08 15:32:19 -07:00
Ray Molenkamp
8c509f98e1 Build_environment: Fix unwanted exports from libxml on windows.
libxml puts dllexport on all its functions causing the
blender binary to export these symbols.

this patch fixes this unwanted behaviour.

Differential Revision: https://developer.blender.org/D6646

Reviewers: brecht , sergey
2020-02-08 15:04:09 -07:00
Ray Molenkamp
67d3613d1a Build_environment: Freetype 2.10.1
Freeetype 2.9.1 tags dllexport on most of its functions so these
are now exported from the blender binary. (Same issue as D6563
which fixed it for USD)

Issue has already been fixed upstream so a simple version bump
fixes it.

This patch bumps freetype to 2.10.1

Differential Revision: https://developer.blender.org/D6645

Reviewers: brecht , sergey
2020-02-08 14:23:28 -07:00
Ray Molenkamp
b9c9b00171 Build_environment: OSL 1.10.9 / llvm 9.0.1
OSL 1.10.9 fixes osl-bug 866 [1] which is long standing issue
on windows where paths get un-escaped and osl breaks when you
install it to for instance c:\blender-tests\new-boolean

This patch bumps osl to 1.10.9, and since osl is llvm's
only consumer, llvm/clang were bumped 9.0.1

Removed some of the patches that were no longer needed

Builds and passes all tests on windows and linux

[1] https://github.com/imageworks/OpenShadingLanguage/issues/866

Differential Revision: https://developer.blender.org/D6744

Reviewers: brecht
2020-02-08 13:01:31 -07:00
Ray Molenkamp
6477fcf40f BuildSystem/Cleanup: Fix warning behaviour regarding library dependencies
Adding USD to a lite build fails to build due to boost errors, when you turn
boost on and rebuild still boost errors, boost was silently turned off since
it was not deemed needed. Once boost was forced on, it still fails due to TBB
being off.

This patch fixes:

- The Silent disabling of boost
- Add a check that USD is is not on before doing that
- move the TBB checks to a central location rather than the individual platform files
- Add USD to the TBB checks.

Differential Revision: https://developer.blender.org/D6479

Reviewers: brecht, sybren
2020-02-07 07:27:40 -07:00
Ray Molenkamp
02226ef653 Code_Cleanup_Day/Windows: Clean-up windows API Level.
Not sure when this happened but apparently the lower bar is now windows 7 [1]

This patch bumps to API version to 0x0601 (Win7) and cleans up any uses that
worked around the globally set API version.

[1] https://www.blender.org/download/requirements/

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D6758
2020-02-05 20:03:06 -07:00
Campbell Barton
d2c0df2842 Merge branch 'blender-v2.82-release' 2020-02-06 01:33:23 +11:00
Campbell Barton
6d10c24062 Fix finding freetype on Linux not using pre-compiled libs
Finding X11 before platform libs caused freetype not to use
pre-compiled libraries.
2020-02-06 01:30:26 +11:00
Campbell Barton
be70fcd7f2 Merge branch 'blender-v2.82-release' 2020-02-06 01:24:49 +11:00
Campbell Barton
87c18fbbc5 CMake: remove non-standard package search paths
Reduce the number of possible locations used to find libraries,
to simplify troubleshooting.

Only keep '*_ROOT_DIR' and the path used by 'install_deps.sh'.
2020-02-06 01:23:22 +11:00
Julian Eisel
e4605cb155 Merge branch 'blender-v2.82-release' 2020-02-04 20:09:03 +01:00
Simon G
370a6175e0 Fix wrong glColor3usv array size check in clang array checker
Differential Revision: https://developer.blender.org/D6716
2020-02-04 17:54:51 +01:00
Sergey Sharybin
04ec64b251 Merge branch 'blender-v2.82-release' 2020-02-04 11:08:15 +01:00
Sergey Sharybin
2264590d8c CMake: Attempt to fix tests on buildbot
An educated guess to put ensure order of static libraries initialization.

A bit weird, since OpenImageDenoise should be depending on TBB, but that
is likely being ensured by bf_compositor.

Linking succeeded on my Intel machine, and blenloader_test was passing
when doing manual test on buildbot.
2020-02-04 11:05:57 +01:00
Sergey Sharybin
9dd6b0842a Merge branch 'blender-v2.82-release' 2020-02-04 10:25:34 +01:00
Sergey Sharybin
ae433393af Codesign: Fix script for Windows
Apparently, there is no os.sync() on Windows.
2020-02-04 10:24:59 +01:00
Sergey Sharybin
ea8975cb27 Merge branch 'blender-v2.82-release' 2020-02-04 09:50:53 +01:00
Sergey Sharybin
5dc1183580 Codesign: Possible fix for stamp appearing prior to archive
From looking into builder's logs it seems that stamp file is picked
up prior to actual archive: sometimes worker reports missing archive
file, from a code path which is only possible if there is a stamp file.

Could be something with IO scheduling where bigger file is sent to
Samba server after smaller file.

Hopefully with this change this will not happen anymore.
2020-02-04 09:47:59 +01:00
Sergey Sharybin
85776521f8 Merge branch 'blender-v2.82-release' 2020-02-03 17:53:28 +01:00
Sergey Sharybin
26ba7573e6 Codesign: Fix wrong logging message 2020-02-03 17:53:03 +01:00
Sergey Sharybin
7799890d8e Merge branch 'blender-v2.82-release' 2020-02-03 17:31:26 +01:00
Sergey Sharybin
3125cfceec Codesign: Add codesign for macOS worker
Works similarly to Windows configuration where buildbot worker and
codesign machines are communicating with each other using network
drive.
2020-02-03 17:03:51 +01:00
Sergey Sharybin
f037244e2c Merge branch 'blender-v2.82-release' 2020-02-03 14:07:48 +01:00
Sergey Sharybin
bc3d7faab7 Buildbot: Specify path to macOS codesign 2020-02-03 14:07:21 +01:00
53d805abcb Merge branch 'blender-v2.82-release' 2020-01-31 13:03:15 +01:00
c82b8c5944 Fix tests failing on AMD Ryzen, due TBB initialization order issue
Similar fix as the one we did for the blender executable, see T72015.
2020-01-31 12:56:20 +01:00
Sergey Sharybin
e207cba8cc Merge branch 'blender-v2.82-release' 2020-01-30 13:32:54 +01:00
Sergey Sharybin
8285a12f4e Buildbot: Make archive naming shorter and cleaner
It changes name to be blender-<version>-linux64.

Since CentOS is used as a base host for builds there is no real need
in specifying libc version. Is unlikely anything older could be used
anyway.

Also make bitness to be the same as windows. It is something what
users will read easier.
2020-01-30 13:32:22 +01:00
Sergey Sharybin
6dcb4c9b4f Buildbot: Allow building on systems without scl
Makes it easier to verify changes on local machine without scl
before committing changes to repo.
2020-01-30 13:32:22 +01:00
Sergey Sharybin
332f310b81 Merge branch 'blender-v2.82-release' 2020-01-30 10:25:36 +01:00
Sergey Sharybin
306edb7477 Buildbot: Enable 16 threads for macOS worker as well 2020-01-30 10:24:57 +01:00
Sergey Sharybin
657ff6ef5e Merge branch 'blender-v2.82-release' 2020-01-29 13:41:43 +01:00
Sergey Sharybin
b683fcb628 Buildbot: Increase number of compile thread for Linux Buildbot 2020-01-29 13:41:04 +01:00
5c75c8d8d0 Fix T63805: remove outdated and unused Linux package spec files
These are maintained by the various Linux distributions themselves instead.
2020-01-27 12:22:01 +01:00
Nathan Craddock
36f713e216 CMake: Fix precompiled Boost libraries on Linux
When building with precompiled libraries on Linux, CMake used boost libs
from the system outside the lib dir. This restricts CMake to use only the
libraries from the precompiled libraries.

Reviewed By: sergey

Differential Revision: https://developer.blender.org/D6659
2020-01-25 23:37:13 -07:00
Sergey Sharybin
b5652f3f02 Merge branch 'blender-v2.82-release' 2020-01-24 15:24:27 +01:00
Sergey Sharybin
cb6f9c2f19 Make deps: Force some boost dependencies
Boost could have picked up system-wide libbz2-dev installed and enable
this compression in iostreams. Nothing really wrong with this, but it
makes it so final Blender binary depends on bz2, which breaks default
linker flags.

This commit makes it so Boost is not using libraries which we don't
need, simplifying linking setup.

Differential Revision: https://developer.blender.org/D6668
2020-01-24 15:21:36 +01:00
Sergey Sharybin
6fff73e3f0 Merge branch 'blender-v2.82-release' 2020-01-23 16:59:50 +01:00
Sergey Sharybin
517870a4a1 CMake: Refactor external dependencies handling
This is a more correct fix to the issue Brecht was fixing in D6600.

While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).

Top-level idea: leave it to CMake to keep track of dependency graph.

The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.

It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.

For example, this order will likely fail:

  libbf_blenfont.a libfreetype6.a libbf_blenfont.a

This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.

General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.

The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.

Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.

The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:

- Have every library prefixed as "optimized" or "debug" if
  separation is needed (non-prefixed libraries will be considered
  "generic").

- Loop through libraries passed to function and do simple parsing
  which will look for "optimized" and "debug" words and specify
  following library to corresponding category.

This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.

Tested the following configurations on Linux, macOS and Windows:

- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer

NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.

Differential Revision: https://developer.blender.org/D6642
2020-01-23 16:59:18 +01:00
544ee7a4f2 Make deps: Fix zlib symbols defined twice
We compile zlib as own dependency, but are not informing BLOSC
to use it. This leads to zlib symbols defined twice when linking
Blender: one set comes from libz.a and another one from libblosc.a.

Tested on Linux Debian testing and CentOS 7.5.

It is possible that this change on its own will lead to linking
errors after libraries are re-compiled, This will be fixed as
a dedicated fix to Blender's build system.

Reviewed By: brecht, mont29, LazyDodo

Differential Revision: https://developer.blender.org/D6641
2020-01-23 16:59:18 +01:00
Campbell Barton
abce4833a6 install_deps.sh: strip trailing punctuation from version numbers
Encountered this issue when enabling more libraries.
2020-01-23 03:39:06 +11:00
Campbell Barton
fde6151641 install_deps.sh: fix versions such as 1.6_RC2 failing to compare
This is stripped off since it causes an error when evaluating
the strings as numbers.
2020-01-23 02:17:28 +11:00
Campbell Barton
0272acee0d install_deps.sh: fix IFS being incorrect when a version check fails
This caused quoting to fail later on.
2020-01-23 01:42:33 +11:00
Campbell Barton
fc0df1ffb4 Revert "install_deps.sh: fix ffmpeg package installation on Arch"
This reverts commit 3cb212602cacc186800c82febc86f5c68c1f92fb.

The root cause was IFS being set incorrectly.
2020-01-23 01:42:33 +11:00
Campbell Barton
3cb212602c install_deps.sh: fix ffmpeg package installation on Arch
A blank space at the start of the string caused the list to be quoted
as a single argument.
2020-01-23 00:50:33 +11:00
Campbell Barton
e790f4a49f install_deps.sh: define sections with overlines
Over-lines are used in other large files (keymaps for e.g),
using this add-hoc convention for sections make it easier to
configure editors to jump between them (as I have locally).
2020-01-23 00:45:34 +11:00
Campbell Barton
2b236294bb install_deps.sh: use jack2 on Arch 2020-01-23 00:41:26 +11:00
Campbell Barton
a90c2834a1 install_deps.sh: add debugging options
Add USE_DEBUG_TRAP, USE_DEBUG_LOG for trapping errors and logging
executed lines respectively.

Handy for troubleshooting issues in the script.
2020-01-23 00:41:26 +11:00
Campbell Barton
4099ad1984 install_deps.sh: quiet warning in install_deps.sh
The unquoted check gave a 'binary operator expected' warning.
2020-01-22 17:30:54 +11:00
Ray Molenkamp
4db9562246 libs/windows: Prevent USD exports from leaking into blender binary.
Even though we build USD as static, it still feels the need to mark its
symbols with declspec(dllexport) which means the blender binary now exports
these symbols.

this patch fixes that unwanted behaviour, however USD libs still need to
rebuild before this becomes visible in the blender binary

Differential Revision: https://developer.blender.org/D6563

Reviewed By: sybren
2020-01-21 09:47:10 -07:00
Ray Molenkamp
ce92e3d553 Fix: Building with clang on windows.
The USD landing broke building with clang on windows
due to a couple of reasons:

1) Some incompatibilities in their headers [1] only one
of them was important for us and is included in our patchset
now.

2) clangs lld wanted the full path to the libusd_b library
when using the whole archive link option, while msvc can
figure it out from just the library name.

Tested with clang/msvc and msbuild and ninja generators

[1] https://github.com/PixarAnimationStudios/USD/issues/1030
2020-01-21 09:46:53 -07:00
463941b6a1 Merge remote-tracking branch 'origin/blender-v2.82-release' 2020-01-21 11:13:47 +01:00
Ray Molenkamp
93a9fbb35f Windows: Update platform_win32.cmake to boost 1.70
to match the updated libs in svn.
2020-01-20 19:31:31 -07:00
a5a30e485f Merge branch 'blender-v2.82-release' 2020-01-20 09:50:09 +01:00
Brecht Van Lommel
02f6722350 Build: upgrade to OpenEXR 2.4.0, OpenVDB 7.0.0 and Boost 1.70.0
This aligns with the VFX reference platform 2020 along with the decision
to stick to Python 3.7, see T68774.

Blosc was downgraded to 1.5 as recommended by the OpenVDB documentation.

IlmBase and OpenEXR are now built together with CMake rather separately
using autoconf.

Differential Revision: https://developer.blender.org/D6593
2020-01-20 09:43:28 +01:00
4f4435001a Fix OSL build error on macOS, no need to use external pugixml 2020-01-20 09:39:54 +01:00
Ray Molenkamp
c2e62c1292 Cleanup/deps: Clean up remnants of old python packaging method 2020-01-18 12:00:42 -07:00
Ray Molenkamp
2272f380bd Merge remote-tracking branch 'origin/blender-v2.82-release' 2020-01-18 11:59:18 -07:00
Ray Molenkamp
3e11c4e63b Fix: T71159 missing python3.dll
File got forgotten during the last repack of python.
2020-01-18 11:58:38 -07:00
2aa497661f Merge branch 'blender-v2.82-release' 2020-01-17 09:26:51 +01:00
554f861ac1 Build: fix Linux linking errors with some combinations of build options
Differential Revision: https://developer.blender.org/D6600
2020-01-17 09:21:18 +01:00
Ray Molenkamp
86db35845a libs/windows: Prevent USD exports from leaking into blender binary.
Even though we build USD as static, it still feels the need to mark its
symbols with declspec(dllexport) which means the blender binary now exports
these symbols.

this patch fixes that unwanted behaviour, however USD libs still need to
rebuild before this becomes visible in the blender binary

Differential Revision: https://developer.blender.org/D6563

Reviewed By: sybren
2020-01-16 13:11:07 -07:00
Ray Molenkamp
8e6c6e2ba1 Fix: Building with clang on windows.
The USD landing broke building with clang on windows
due to a couple of reasons:

1) Some incompatibilities in their headers [1] only one
of them was important for us and is included in our patchset
now.

2) clangs lld wanted the full path to the libusd_b library
when using the whole archive link option, while msvc can
figure it out from just the library name.

Tested with clang/msvc and msbuild and ninja generators

[1] https://github.com/PixarAnimationStudios/USD/issues/1030
2020-01-14 11:12:14 -07:00
Campbell Barton
bbe7c278d2 CMake: enable FFTW for headless & bpy configurations
This caused the ocean modifier to be disabled.
2020-01-05 16:16:03 +11:00
4c295ad478 install_deps.sh: No longer forcing Alembic sources to be redownloaded
For no apparent reason, when building Alembic the script would always
re-download and re-extract the Alembic source code. This is no longer the
case, and it now only happens if the source directory is missing. Since the
source directory name contains the Alembic version, it will automatically
trigger a download+extract when the version changes.
2019-12-24 12:10:54 +01:00
d616938449 install_deps.sh: show which parameter is wrong
Previously, when an unknown parameter was passed to `install_deps.sh`,
the script would just show "Wrong parameter!" without any context. This
can make it hard to figure out what's exactly going wrong. Now it prints
which parameter it thinks is wrong.
2019-12-24 12:10:54 +01:00
Sebastián Barschkis
41fd60db16 Mantaflow [Part 4]: Adapted build config
Smaller changes in the build files to reflect the new Mantaflow macro.

Reviewed By: sergey

Maniphest Tasks: T59995

Differential Revision: https://developer.blender.org/D3853
2019-12-16 16:32:25 +01:00
Campbell Barton
5a97a74c69 Cleanup: whitespace 2019-12-16 13:49:11 +11:00
f8cdc5ac4c USD: on Apple disable USD if library cannot be found
When building with `WITH_USD=ON` on Apple but the USD library cannot be
found, the CMake script now just sets `WITH_USD=OFF`.
2019-12-14 12:53:37 +01:00
ec62413f80 USD: Introducing a simple USD Exporter
This commit introduces the first version of an exporter to Pixar's
Universal Scene Description (USD) format.

Reviewed By: sergey, LazyDodo

Differential Revision: https://developer.blender.org/D6287

- The USD libraries are built by `make deps`, but not yet built by
  install_deps.sh.
- Only experimental support for instancing; by default all duplicated
  objects are made real in the USD file. This is fine for exporting a
  linked-in posed character, not so much for thousands of pebbles etc.
- The way materials and UV coordinates and Normals are exported is going
  to change soon.
- This patch contains LazyDodo's fixes for building on Windows in D5359.

== Meshes ==

USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.

Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.

Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.

The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.

The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.

A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.

== Animation ==

Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.

The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.

== Support for simple preview materials ==

Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.

When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.

The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.

Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.

Materials are exported as a flat list under a top-level '/_materials'
namespace. This inhibits instancing of the objects using those
materials, so this is subject to change.

== Hair ==

Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.

== Camera ==

Only perspective cameras are supported for now.

== Particles ==

Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).

Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.

== Instancing/referencing ==

This exporter has experimental support for instancing/referencing.

Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.

I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.

== Lights ==

USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.

== Fluid vertex velocities ==

Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.

== The Building Process ==

- USD is built as monolithic library, instead of 25 smaller libraries.
  We were linking all of them as 'whole archive' anyway, so this doesn't
  affect the final file size. It does, however, make life easier with
  respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into datafiles/usd; they
  are required on every platform. Set the `PXR_PATH_DEBUG` to any value
  to have the USD library print the paths it uses to find those files.
- USD is patched so that it finds the aforementioned JSON files in a path
  that we pass to it from Blender.
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
  building the tools in its `bin` directory. This is sent as a pull
  request at https://github.com/PixarAnimationStudios/USD/pull/1048
2019-12-13 10:27:40 +01:00
Ray Molenkamp
5e96b860a3 Windows: Fix failing tests due to missing manifest.
Tests were missing a manifest, and were importing the
wrong version of Microsoft.Windows.Common-Controls
causing blenloader_test, bmesh_core_test and alembic_test
to fail due a loader error.
2019-12-06 11:37:14 -07:00
Ray Molenkamp
6df2ede341 Cleanup/Windows: Separate out the MS-CRT into a subfolder
In older versions the ms crt was only a few dlls, in recent versions
this jumped to over 40 leading to quite a bit of clutter in our
bin folder.

This change moves the CRT into its own folder.

For developers that generally already have the runtime globaly
available on their machine, there is a new cmake option
(WITH_WINDOWS_BUNDLE_CRT, default ON) that you can use to toggle
installing the runtime to the blender bin folder, and save some
time during the initial build, this option is off by default for
only the developer profile.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D6132
2019-12-06 10:12:03 -07:00
Sergey Sharybin
adb6be36e4 Buildbot: Correct timestamp argument for Windows codesign
The current authority we use RFC 3161 time stamp server,
so need to pass different command line argument.
2019-12-03 14:41:49 +01:00
b25bb2d7ec Archive build script: stop when creating archive fails
The `subprocess.call()` function doesn't check the exit status code of the
subprocess. Use `subprocess.check_call()` or `subprocess.run()` instead.
2019-11-29 10:37:43 +01:00
ae13bba24e Archive build script: fix compatibility with older tar on CentOS 7
On CentOS 7, `tar --use-compress-program='xz -9'` tries to run `xz -9` as
executable, rather than running `xz` with `-9` as argument. Passing the
`-9` option via the `XZ_OPT` environment variable, as suggested by
@campbellbarton in D6138, works fine.
2019-11-29 10:35:19 +01:00
Ray Molenkamp
77c7440540 CMake: Remove stray WITH_JACK in blender_release.cmake
Missed one in the previous commit.
2019-11-28 15:02:54 -07:00
Ray Molenkamp
7e9a827c58 CMake: Remove stray WITH_JACK in full configuration 2019-11-28 15:00:30 -07:00
bbd5f30ad6 Build: change CMake option defaults to match "make full"
Previously some important features like OpenSubdiv were disabled by default,
which caused confusion.

The purpose of disabling some of these features was to avoid potentiall build
errors on Linux. But with precompiled libraries, install_deps.sh and better
library availability checking this is hopefully not much of a problem anymore.

This makes "make full" obsolete, but it's kept to not break docs or shell
scripts that people may have, and the .cmake config file remains useful to
modify an existing build folder.

This also changes some option to only be available on platforms where they
are actually supported (WITH_JACK, WITH_TBB_MALLOC_PROXY and X11 options).

Fixes T69742

Differential Revision: https://developer.blender.org/D6306
2019-11-28 19:20:06 +01:00
f445f72eca Tests: Blendfile-loading test class
This new test class minimally sets up Blender so that it can load blend
files and construct a depsgraph without crashing.

Note that it hasn't been tested on very complex blend files, so it may
still crash when the loaded blend file references/requires uninitialised
data structures.

The test will certainly crash with Blend files created with Blender
older than 2.80, as the versioning code requires space types to be
registered. This is normally done by initialising the window manager,
which is not done in this test. The WM requires Python to run, which in
turn requires that Blender finds the release directory in the same
directory that contains the running executable, which is not the case
for GTest tests (they are written to `bin/tests/executablename`.

Reviewed By: sergey, mont29

Differential Revision: https://developer.blender.org/D6246
2019-11-28 17:41:32 +01:00
Jens
d4835b88b2 Buildbot: Migrate package archive format for Linux from tar.bz2 to tar.xz
xz compresses 25% better than bz2, reducing download times and server load.
The numbers:

blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz:    96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)

blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz:    133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)

xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).

Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.

Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.

Differential Revision: https://developer.blender.org/D6138
2019-11-28 11:41:25 +01:00
Ray Molenkamp
b213d82c19 Cleanup/CMake: Remove dormant windows codesign code
This was added years ago to prepare for code-signing the executable
but was never used, buildbots use a different mechanism now to sign
so no need to keep this around.
2019-11-26 15:13:22 -07:00
Sergey Sharybin
1fbca07634 Buildbot: Increaser codesign timelimit
It was possible that it would exceed when signing all the DLLs if the machine
is busy with some background tasks or when internet is slow.
2019-11-26 10:08:33 +01:00
Campbell Barton
b2d940250c Cleanup: remove unused CMake WITH_MOD_CLOTH_ELTOPO option 2019-11-25 14:52:44 +11:00
dfdbb237bb Alembic: Upgrade from 1.7.8 to 1.7.12
Alembic 1.7.12 introduces a 'DCC FPS' hint, allowing Blender to write
the scene frame rate to the Alembic file. This will make it possible for
importers and converters to properly deal with situations where 'frame
number' is the only reference to time.

Writing this new DCC FPS hint will be done in a separate commit. Here
only the Alembic library is upgraded from 1.7.8 to 1.7.12.
2019-11-20 10:19:00 +01:00
Campbell Barton
9d8af29267 Cleanup: remove WITH_RAYOPTIMIZATION
This is redundant as WITH_CPU_SSE adds these flags
when they're supported.
2019-11-20 14:54:50 +11:00
Sergey Sharybin
c2a251d655 Build deps: Actually tweak EOL style
Seems that `git am` will force native EOL.
2019-11-18 13:55:55 +01:00
blender
14882c4bed Build deps: Fix compilaiton of OpenCollada
Was caused by "wrong" EOL characters used in the patch: the file is
actuallyu saved using CRLF EOL style.

The patch was using CRLF as well for until recent change in the C
runtime.
2019-11-18 13:52:49 +01:00
Ray Molenkamp
6d9d24e3d4 msvc: Use debug versions of tbb malloc for debug builds
Using the release versions gave unpredictable results when
the msvc debugger was attached for some developers.
2019-11-17 14:26:29 -07:00
Sergey Sharybin
8ff9eb97fb Merge branch 'blender-v2.81-release' 2019-11-14 10:44:27 +01:00
Sergey Sharybin
c25a910e4e Buildbot: Explicitly disable code signer on Linux and macOS
The script requires Python 3.7 as a very minimum, and CentOS is
only 3.6.

On macOC there was an access to a None object, due to missing
implementation of code signer on this platform.
2019-11-14 10:40:28 +01:00
Sergey Sharybin
97d79ca844 Merge branch 'blender-v2.81-release' 2019-11-13 11:49:09 +01:00
Sergey Sharybin
c73a99ef90 Initial implementation of code signing routines
This changes integrates code signing steps into a buildbot worker
process.

The configuration requires having a separate machine running with
a shared folder access between the signing machine and worker machine.

Actual signing is happening as a "POST-INSTALL" script run by CMake,
which allows to sign any binary which ends up in the final bundle.
Additionally, such way allows to avoid signing binaries in the build
folder (if we were signing as a built process, which iwas another
alternative).
Such complexity is needed on platforms which are using CPack to
generate final bundle: CPack runs INSTALL target into its own location,
so it is useless to run signing on a folder which is considered INSTALL
by the buildbot worker.

There is a signing script which can be used as a standalone tool,
making it possible to hook up signing for macOS's bundler.

There is a dummy Linux signer implementation, which can be activated
by returning True from mock_codesign in linux_code_signer.py.
Main purpose of this signer is to give an ability to develop the
scripts on Linux environment, without going to Windows VM.

The code is based on D6036 from Nathan Letwory.

Differential Revision: https://developer.blender.org/D6216
2019-11-13 09:24:41 +01:00
Ray Molenkamp
d60a60f0cb Add support for the TBB allocator on windows.
The heap on windows is single threaded causing it to lag behind linux in performance in allocation heavy multithreaded scenarios, BVH building is a prime example.

See https://developer.blender.org/D6218 for benchmark results

for testing with the allocator enabled/disabled you can set the environment variable TBB_MALLOC_DISABLE_REPLACEMENT=1 to disable the TBB allocator.

Reviewed By: @sergey

Differential Revision: https://developer.blender.org/D6218
2019-11-12 20:55:39 -07:00
Sergey Sharybin
c69872bd2f Merge branch 'blender-v2.81-release' 2019-11-12 14:30:22 +01:00
Sergey Sharybin
1252577580 Safer fix for make_update.py on buildbot
Makes it so compilation doesn't fail when the SVN updating
stumbles upon checkout which doesn't have correspondence in
a tag, but which isn't so risky as previous change.
2019-11-12 14:28:39 +01:00
Sergey Sharybin
1e19ff8bfd Revert "Fix issues with make_update.py when run from release branch"
This reverts commit 8e9e58895b32afc38f856053335c9b27324c6f9e.

The change broke behavior when typing `make update` from the root of
the sources: tests folder wouldn't be updated anymore.

Getting quite close to release now, so will revert to a safer change.
2019-11-12 14:27:47 +01:00
Sergey Sharybin
5fcbec9fce Merge branch 'blender-v2.81-release' 2019-11-12 14:06:54 +01:00
Sergey Sharybin
8e9e58895b Fix issues with make_update.py when run from release branch
The issue was rooting to the fact that the script was iterating into
every directory inside of blender.git/../lib/ and attempted to switch
them to the desired path. This doesn't work in an environment where
both master and release branch are built (or any environment where
non-needed SVN directories are not automatically removed).

This change makes it so script explicitly generates a list of
directories which are required for the build. For example, the script
now stores an exact folder with ABI such as win64_vc14.

Only those explicitly listed directories will be updated.

This allows to:

- Solve compilation failure of 2.81 branch after checkout for
  win64_vc15 libraries has been created.

- Fail compilation if actually expected tag is missing (for example,
  when trying to build release branch prior to libraries tag).

Now, there was a confusing logic about possible .svn folder in
lib_dirpath (effectively, blender.git/../lib/.svn) which is not
something what is supposed to happen with the setup of buildbot we are
using for quite some time now. This logic has been removed now.

This change includes old-style string format(), mainly because it is
not know that the buidlbot scripts are run using python3 on CentOS
builder.

Differential Revision: https://developer.blender.org/D6230
2019-11-12 14:01:06 +01:00
Ray Molenkamp
4c5477a8dc Windows: Fix build errors during deps build on windows. 2019-11-10 15:45:30 -07:00
Ray Molenkamp
6f1237b2eb make_update.py: update windows library folder 2019-11-08 19:09:59 -07:00
Ray Molenkamp
a67aa11b12 Windows: Switch to the dynamic C runtime
This change switches windows to the dynamic C runtime
avoiding issues coming from mixing the static and dynamic
runtime like the ones outlined in [1]

[1] https://developer.blender.org/D5387#122165

Differential Revision: https://developer.blender.org/D6175

Reviewed by: @Sergey
2019-11-08 09:01:00 -07:00
Sergey Sharybin
a3d5758902 Merge branch 'blender-v2.81-release' 2019-11-06 13:54:41 +01:00
Sergey Sharybin
e71963a37e Buildbot: Ensure proper ABI is used
This wasn't an issue in the real buildbot environment since the
precompiled libraries are compiled with same ABI as the compiler
used for Blender build. But it was causing issues when building
Blender using buildbot scripts (for troubleshooting purposes)
on a machine with different default compiler ABI.

Usually ABI detection is happening in platform_unix.cmake when
detecting whether there are any precompiled libraries folder
available. This detection is not happening when library folder
is provided explicitly, expecting ABI to be setup explicitly
as well.
2019-11-06 13:50:43 +01:00
Ray Molenkamp
3c32c5c2dd make.bat: Warn user about missing svn.exe 2019-10-31 09:45:56 -06:00
Campbell Barton
5043003584 GNUmakefile: avoid using group/owner for source_archive
Thanks to @JRottm for pointing out this issue.
2019-10-31 00:51:15 +11:00
Campbell Barton
60d0446db3 GNUmakefile: use tar.xz instead of tar.gz
Also rename "make tgz" to "make source_archive" as it wasn't clear
this only archived the source, not binaries.

D6153 by @JRottm with minor edits
2019-10-31 00:19:05 +11:00
Campbell Barton
c050247edc CMake: update cmake_consistency_check
Support for listing files which are known not to exist
(needed by standalone cycles).
2019-10-29 03:46:50 +11:00
Campbell Barton
312075e688 CMake: add missing headers, use space before comments 2019-10-29 01:33:44 +11:00
Campbell Barton
4dc443908c CMake: disable OpenMP on macOS bpy_module config
This gives linking errors on build.
2019-10-23 14:50:42 +11:00
Campbell Barton
41ec25d27b Cleanup: style 2019-10-21 15:05:56 +11:00
Arto Kitula
95f020c853 macOS: add opus to FFMPEG_LIBRARIES 2019-10-12 18:33:34 +03:00
60d02b336e Build: also use release branch for source/tools module 2019-10-11 21:43:45 +02:00
b57a89f062 Buildbot: don't add branch prefix when building release branches 2019-10-11 13:46:24 +02:00
2b35ee3ea2 Fix issue in "make update" checking out submodule branch on buildbot 2019-10-11 13:11:10 +02:00
41d9cf225d Fix "make update" not using the right branch for source/tools 2019-10-11 12:22:28 +02:00
f3d9ea0a8a Fix build error on Windows after recent changes 2019-10-10 18:04:40 +02:00
86042b7ced Build: add WITH_TBB option, in preparation of sculpt using it
It should no longer be tied to OpenVDB and OpenImageDenoise then.

Differential Revision: https://developer.blender.org/D6029
2019-10-10 17:35:35 +02:00