Commit Graph

389 Commits

Author SHA1 Message Date
Sergey Sharybin
adb6be36e4 Buildbot: Correct timestamp argument for Windows codesign
The current authority we use RFC 3161 time stamp server,
so need to pass different command line argument.
2019-12-03 14:41:49 +01:00
Jens
d4835b88b2 Buildbot: Migrate package archive format for Linux from tar.bz2 to tar.xz
xz compresses 25% better than bz2, reducing download times and server load.
The numbers:

blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz:    96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)

blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz:    133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)

xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).

Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.

Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.

Differential Revision: https://developer.blender.org/D6138
2019-11-28 11:41:25 +01:00
Sergey Sharybin
1fbca07634 Buildbot: Increaser codesign timelimit
It was possible that it would exceed when signing all the DLLs if the machine
is busy with some background tasks or when internet is slow.
2019-11-26 10:08:33 +01:00
Sergey Sharybin
c25a910e4e Buildbot: Explicitly disable code signer on Linux and macOS
The script requires Python 3.7 as a very minimum, and CentOS is
only 3.6.

On macOC there was an access to a None object, due to missing
implementation of code signer on this platform.
2019-11-14 10:40:28 +01:00
Sergey Sharybin
c73a99ef90 Initial implementation of code signing routines
This changes integrates code signing steps into a buildbot worker
process.

The configuration requires having a separate machine running with
a shared folder access between the signing machine and worker machine.

Actual signing is happening as a "POST-INSTALL" script run by CMake,
which allows to sign any binary which ends up in the final bundle.
Additionally, such way allows to avoid signing binaries in the build
folder (if we were signing as a built process, which iwas another
alternative).
Such complexity is needed on platforms which are using CPack to
generate final bundle: CPack runs INSTALL target into its own location,
so it is useless to run signing on a folder which is considered INSTALL
by the buildbot worker.

There is a signing script which can be used as a standalone tool,
making it possible to hook up signing for macOS's bundler.

There is a dummy Linux signer implementation, which can be activated
by returning True from mock_codesign in linux_code_signer.py.
Main purpose of this signer is to give an ability to develop the
scripts on Linux environment, without going to Windows VM.

The code is based on D6036 from Nathan Letwory.

Differential Revision: https://developer.blender.org/D6216
2019-11-13 09:24:41 +01:00
Sergey Sharybin
e71963a37e Buildbot: Ensure proper ABI is used
This wasn't an issue in the real buildbot environment since the
precompiled libraries are compiled with same ABI as the compiler
used for Blender build. But it was causing issues when building
Blender using buildbot scripts (for troubleshooting purposes)
on a machine with different default compiler ABI.

Usually ABI detection is happening in platform_unix.cmake when
detecting whether there are any precompiled libraries folder
available. This detection is not happening when library folder
is provided explicitly, expecting ABI to be setup explicitly
as well.
2019-11-06 13:50:43 +01:00
b57a89f062 Buildbot: don't add branch prefix when building release branches 2019-10-11 13:46:24 +02:00
Sergey Sharybin
26b1216629 CMake: Move software-gl to generic install procedure
Removes custom logic from buildbot's packing step.

This also removes icons/ folder, but CMake was already copying the
icons to the root of the install folder.
2019-10-09 13:07:40 +02:00
Sergey Sharybin
2fd2bced56 Buildbot: Correction to previous commit
Off-by-one calculating parent folders.
2019-10-09 10:37:09 +02:00
Sergey Sharybin
06ff734d48 Buildbot: Attempt to point precompiled libraries to a correct folder
The CMAKE_SOURCES variable is not yet initialized when the buildbot
configuration is read. This is similar to the include of full release
configuration happening earlier in the file.
2019-10-09 10:34:12 +02:00
Sergey Sharybin
62a7e8d429 Buildbot: Point to pre-compiled CentOS 7 libraries 2019-10-09 09:50:58 +02:00
Sergey Sharybin
40a55be82c Buildbot: Checkout precomiled CentOS libraries 2019-10-09 09:42:42 +02:00
Patrick Mours
a2b52dc571 Cycles: add Optix device backend
This uses hardware-accelerated raytracing on NVIDIA RTX graphics cards.

It is still currently experimental. Most features are supported, but a few
are still missing like baking, branched path tracing and using CPU memory.
https://wiki.blender.org/wiki/Reference/Release_Notes/2.81/Cycles#NVIDIA_RTX

For building with Optix support, the Optix SDK must be installed. See here for
build instructions:
https://wiki.blender.org/wiki/Building_Blender/CUDA

Differential Revision: https://developer.blender.org/D5363
2019-09-13 11:50:11 +02:00
abeab4fcad Fix buildbot not reporting error when tests fail 2019-09-07 18:22:48 +02:00
2028302f44 Tests: run tests from install path
Blender can only be run correctly from the install path since it requires Python
scripts, dynamic libraries and other files to be present. By default the install
path is the same as the build path, so it works anyway. But on the buildbot it
isn't. There was a workaround but it failed on Windows and macOS.

Now tests run from the install path. Detecting that path for ctest is more
complicated than I would like, but I couldn't find a better solution.

Ref T69541.
2019-09-07 18:09:41 +02:00
708e81bdfa Fix buildbot error building release on macOS 2019-09-07 18:09:41 +02:00
b6e7e173ec Buildbot: enable tests to run again
These will not stop the build on failure yet, first step is to get them passing
more reliably.
2019-09-05 12:52:54 +02:00
4f92725283 Fix buildbot unsuccessfully trying to update Blender repository for branches 2019-09-03 18:01:57 +02:00
51d8facbc3 Fix buildbot Python executable not found on Windows 2019-09-03 14:35:57 +02:00
81849e7c9d Buildbot: support building releases, make non-releases more consistent
* Auto detect rc and release version cycle in BKE_blender_version.h.
* On Windows, generate zip and installer if a release is detected.
* On macOS, always generate a dmg instead of zip.
* Use standard package names without hash if a release is detected.
* Buildbot package names now match platform names in releases.

Ref T67056

Differential Revision: https://developer.blender.org/D5643
2019-09-03 12:23:39 +02:00
283d96de11 Buildbot: refactor all build slave code
* Move common code into buildbot_utils.py
* Remove legacy code from removed builders
* Split code into smaller functions

Differential Revision: https://developer.blender.org/D5642
2019-09-02 17:37:22 +02:00
fd48e17461 Buildbot: add script to update libraries and submodules
This is meant to replace the individual build steps for the various libraries
and submodules, so we can easily get the correct revisions associated with
different branches and releases.
2019-08-30 17:57:18 +02:00
Campbell Barton
760dbd1cbf Cleanup: misc spelling fixes
T68035 by @luzpaz
2019-08-01 14:02:41 +10:00
Sergey Sharybin
1f65606911 Buildbot: Disable test step for now
It causes some false indication of a failed build on soma platforms
where it times out.

Windows and macOS needs extra work to properly support python paths
and path to the bundle.

This can now happen without poking git every time by doing local
modifications on a builder prior to commit.
2019-07-02 15:39:47 +02:00
Sergey Sharybin
e7356bb011 Buildbot: Fix missing configuration for Win ctest 2019-07-02 15:31:06 +02:00
Sergey Sharybin
df330ab2bb Buildbot: Fix missing environment for schroot tests 2019-07-02 14:15:44 +02:00
Sergey Sharybin
5439f682fd Buildbot: Correct previous commit 2019-07-02 12:56:07 +02:00
Sergey Sharybin
3cc51cf28c Buildbot: Use correct path to scripts
The `scripts` is to be included into the path, not just the
folder where the scripts are.

Also specify path to datafiles.
2019-07-02 12:52:53 +02:00
Sergey Sharybin
6b17b51baa Buildbot: Attempt to make ctest use proper installed scripts 2019-07-02 12:36:07 +02:00
Sergey Sharybin
3ff984d746 Buildbot: Enable GTests and test step 2019-07-02 12:07:20 +02:00
Sergey Sharybin
a7e32dbcb0 Buildbot: Fix undefined architecture string for CentOS 2019-06-26 19:40:46 +02:00
Sergey Sharybin
b98428e046 Buildbot: Fix undefined bits for CentOS based builder 2019-06-26 19:37:17 +02:00
Sergey Sharybin
14de0d6a5d Buildbot: Fix pack stage for CentOS based builder 2019-06-26 19:35:10 +02:00
Sergey Sharybin
ae396912fb Buildbot: Fix scl command prefix
Now it should work correct with command passed as an individual
elements of a list.
2019-06-26 18:07:03 +02:00
Sergey Sharybin
81b11b795c CMake: Cleanup, remove explicit Python version
This is now coming from a common CMake configuration.
2019-06-26 16:19:49 +02:00
Sergey Sharybin
846b4f1360 CMake: Initial support of CentOS for buildbot configuration 2019-06-26 16:17:08 +02:00
Sergey Sharybin
8218fdd89f CMake: Cleanup, remove unused legacy hardcoded paths 2019-06-26 16:12:38 +02:00
Sergey Sharybin
93f1703900 Buildbot: Preliminary support for CentOS based builder 2019-06-26 15:42:46 +02:00
Sergey Sharybin
3471fde8cc Buildbot: Cleanup, remove unused option
It was originally needed for various migration needs, now CUDA
binaries are always to be built for 64 bit platforms and never
to be built on 32bit platforms.
2019-06-26 15:42:46 +02:00
Campbell Barton
dfe2ca26f7 Cleanup: style, indentation 2019-06-19 07:32:21 +10:00
Sergey Sharybin
3bc73c1b61 Merge branch 'blender2.7' 2019-04-18 11:57:44 +02:00
Sergey Sharybin
cc9528d3c8 Buildbot: Attempt to fix wrong branch in buildinfo
For some reason the buildinfo header was not re-generated. The root
reason is not really clear to me, so simply remove the header similar
to the CMake cache.
2019-04-18 11:52:34 +02:00
Campbell Barton
3076d95ba4 Cleanup: use 2 space indentation for CMake 2019-04-17 06:35:54 +02:00
e691929686 Merge branch 'blender2.7' 2019-03-17 12:54:19 +01:00
9d2397c710 Cleanup: remove unused buildbot code. 2019-03-17 12:01:19 +01:00
0676badb80 Fix Windows 32bit buildbot trying to use CUDA, should be disabled. 2019-03-17 12:01:19 +01:00
bc8bd87dff Merge branch 'blender2.7' 2019-03-15 18:31:48 +01:00
65d95879f7 Cycles: upgrade to CUDA 10.1 as the one officially supported version.
This version fixes various bugs, and there is no need anymore to use both
9.1 and 10.0 for different cards.

There is a bug related to WITH_CYCLES_CUBIN_COMPILER and bump mapping in the
regression tests, so that remains disabled same as it was for CUDA 10.0.

Fix T59286: CUDA bake failing on some cards.
Fix T56858: CUDA 9.2 and 10 issues.
2019-03-15 16:52:28 +01:00
c9df453ce7 Update for moving blender2.8 to master. 2018-12-21 12:15:21 +01:00
e72f989cf4 Update for moving master to blender2.7 branch. 2018-12-21 12:01:05 +01:00