autopep8_format_paths.py was never wired up in the windows tooling,
this PR wires it up, to get around a 8191 limitation for command
lines on windows an alternative "--no-subprocess" option has been
introduced to call the pep8 module directly from python.
TODO: A subprocess is still used for getting the version information
Contributors:
- @campbellbarton fixes to autopep8_format_paths.py to support
the -no-subprocess option.
Pull Request: https://projects.blender.org/blender/blender/pulls/120794
Support for building blender with clang on windows on x64 was added
years ago but given there are no active users support has crumbled a
bit.
This PR brings the build system back into working order but upstream
patches in openVDB are still required for a successful build see PR
#120317 for details.
Blender when build with clang the classroom scenes rendered on the cpu
with cycles is seeing a 5% reduction in render time on both an
AMD 7700x and an Intel 14900k.
* Only works on machines with a Qualcomm Snapdragon 8cx Gen3 or above.
Older generation devices are not and will not be supported due to
some driver issues
* Requires VS2022 for building.
* Uses new MSVC preprocessor for sse2neon compatibility.
* SIMD is not enabled, waiting on conversion of blenlib to C++.
Ref #119126
Pull Request: https://projects.blender.org/blender/blender/pulls/117036
First time builders get spooked by this warning and start
installing python trying to resolve it. This isn't needed
and we should only warn about python being missing if the
lib folder is cloned, but python is still somehow not found.
This was "fixed" a few times before, but it took a bit
to find the right filename to check for the new library
folder.
git on windows is giving no status updates while git lfs
is downloading the libs, causing people to assume its
stuck and interrupt the download. This change adds a
rather large visual message telling people not to do
that.
also it was not respecting people selecting n on the lib
download so some extra messaging has been added there
as well.
Git escaped a quote and used all of the cli arguments as the
folder name.
Also fixes:
- Progress on the initial clone so it doesn't appear stuck
- re-detect python after updating libs, so you don't have to run
make update twice on a fresh clone.
- Hide python not found warning unless the lib folder exists (this
seemingly got lost in one of the merges)
This change makes it so build system and update utilities for Blender builds
are using pre-compiled libraries and other resources attached as Git modules
instead of using checkout of SVN repositories in the parent folder.
The directory layout:
```
* release/datafiles/
* assets/ -> blender-assets.git
* publish/
* ...
* README.txt
* lib/
* darwin_x64/ -> lib-darwin_x64.git
* darwin_arm64/ -> lib-darwin_arm64.git
* linux_x64/ -> lib-linux_x64.git
* windows_x64/ -> lib-windows_x64.git
* tests/
* data/ -> blender-test-data.git
```
The changes about configuring the actual Git sub-modules are not included
into this patch, as those require repository to actually exist before it
can be used.
The assets submodule is enabled by default, and the rest of them are
disabled. This means that if someone runs `git submodule update --init`
they will not get heavy libraries. The platform-specific and tests
related submodules are enabled when using `make update` or `make test`.
All the submodules are tracked: this means that when new commits are
done to the submodule, the blender.git repository is to be updated to
point them to the new hash. This causes some extra manual work, but it
allows to more easily update Blender and its dependencies to known good
state when performing operations like bisect.
Ref #108978
Pull Request: https://projects.blender.org/blender/blender/pulls/117946
People running `make update` first time, will not have a lib
folder yet, and get a warning about python being missing, this
sends some of them off onto somewhat of a wild goose chase
installing python when this is really not needed since it's
included in our libraries.
This change, changes the warning to only emit when the lib
folder exists, but python is missing in the lib folder and
there actually is likely an issue with the lib folder.
Add silently fail option to GPU based render tests. This is a pre-requisite to enable
render tests on the buildbot. By default these render tests will pass silently.
* Test will pass when using the `--pass-silently` arguments.
* Only crashes will be reported as failed tests.
* To find out failing test, review the test reports.
`WITH_GPU_RENDER_TESTS_SILENT` compile option can be used to let tests pass (default)
or fail (default for developers).
Although some tests fail, they still passed. In the generated render report,
the silently passed failures are correctly reported to be failures.
Pull Request: https://projects.blender.org/blender/blender/pulls/117629
The build scripts are still referring to gpu tests as being opengl.
Although they can also use Metal or Vulkan. This PR only replaces
the work `opengl` with `gpu` for build options.
Special note is that the windows argument `with_opengl_tests` is
also replaced with `with_gpu_tests` for consistency.
Pull Request: https://projects.blender.org/blender/blender/pulls/116030
This broke when we moved the submodules around and no one ever noticed
This will now list:
-The current branch
-The current branch hash
-Addons branch
-Addons branch hash
-Libs url
-Libs revision
-Libs last-changed-date
So it'll be a little easier to diagnose what exactly the state of
things are when someone asks for help building.
the path for clang_format_paths.py changed when the submodules
moved but format.cmd was never updated for that.
the work previously done by check_submodules.cmd is now done by
make_update.py so this file can be removed.
This commit implements described in the #104573.
The goal is to fix the confusion of the submodule hashes change, which are not
ideal for any of the supported git-module configuration (they are either always
visible causing confusion, or silently staged and committed, also causing
confusion).
This commit replaces submodules with a checkout of addons and addons_contrib,
covered by the .gitignore, and locale and developer tools are moved to the
main repository.
This also changes the paths:
- /release/scripts are moved to the /scripts
- /source/tools are moved to the /tools
- /release/datafiles/locale is moved to /locale
This is done to avoid conflicts when using bisect, and also allow buildbot to
automatically "recover" wgen building older or newer branches/patches.
Running `make update` will initialize the local checkout to the changed
repository configuration.
Another aspect of the change is that the make update will support Github style
of remote organization (origin remote pointing to thy fork, upstream remote
pointing to the upstream blender/blender.git).
Pull Request #104755
The issue we ran into a lot is we have a python script that updates
git+SVN. Which works fine most of the time, except when we have a
python update in SVN, or worse a python version change. Python really
doesn't enjoy having its files being deleted or changed while it is
running and users generally end up with a corrupted lib folder.
This change updates the library folder using svn.exe first before
letting the python script run sidestepping the issue in most cases.
The python script will still run and do the more elaborate work
like updating git and switching SVN branches which could still
run into issues cause python still doesn't like being changed
while running but there's not a whole lot we can about that,
for *most* people however things will just work now.
Several sub commands tried on their own
to locate python, given I wanted to look
in several locations for a broader libdir
compatibility this is best done in a
central location.
Python 3.9 is still preferred, but if
3.10-3.12 are available that be accepted
as well.
note: this is about the python version
make.bat uses to run various python helper
scripts, this change has no influence on
the python version blender itself uses.
SVN seems to die randomly *a lot* during
large updates for some users, and I'm no
closer to finding out why that keeps happening.
"The internet" seems to imply some AV vendors
may be at fault here but nothing conclusive.
The solution however is repeatedly running
`svn cleanup`and `svn update` in the library
folder to repair the corruption and finish the
update.
This change adds a small convenience helper
to automate the repair.
This is done inside the make.bat code rather
than the shared python based update code, since
python lives in the library folder and may
or may not exist when this corruption occurs.
This adds support to compile the html python api docs from the command line by running `make doc_py` matching support between windows and unix.
This patch also makes it so the compiler is not needed if you set the `blender_bin` variable, this affects icon generation as well.
In the future, I want to move away from generating the build output in the build directory but that can come in a later change.
Reviewed By: LazyDodo
Differential Revision: https://developer.blender.org/D12144
This adds support for building the icons from make.bat
unlike bash there is no passing environment variables
on the command line.
The scripts go out of their way to locate both blender
and inkscape however if they are not found, the user is
given a helpful error message telling them how to set
the variables.
Although some extra help can be given there, if your
normal build is a 2019 full build running
`make 2019 full icons`
will help it find the blender executable as well.
finally if you know the name of your build folder
running
`make builddir build_windows_Lite_x64_vc16_Release icons`
will also work, if all fails you can point directly to
the blender executable by running
`set BLENDER_BIN=c:\where\blender\lives\blender.exe`
before running `make icons` or `make icons_geom`
The python scripts needed some small modifications since
without the PATHEXT, SystemRoot and SystemDrive
environment variables python will not initialize properly
on windows. (Not blender related, even mainline python
won't start without those)
This adds preliminary VS 2022 support, since
there currently is no CMake version that
supports the VS2022 IDE only ninja support
was tested.
IDE support should work without any additional
changes as soon as an updated CMake becomes
available.
As VS2022 appears to keep binary compatibility
with earlier MSVC versions, the current SVN
libraries will work for this version.
This enables ASAN support when used with VS 16.9
enable as usual in cmake with the WITH_COMPILER_ASAN
option, or when using make.bat just tag on `asan'
to the invocation, ie: `make lite 2019 asan`
MSVC: Asan support for 16.9
This enables ASAN support when used with VS 16.9
enable as usual in cmake with the WITH_COMPILER_ASAN
option, or when using make.bat just tag on `asan'
to the invocation, ie: `make lite 2019 asan`
Differential Revision: https://developer.blender.org/D7794
Reviewed By: brecht, sergey
The windows build leaves a convenience helper script
in the build folder called, rebuild.cmd. This change
passes any parameters you give rebuild.cmd to ninja
so you can easily pass it additional parameters without
having to edit the batch file manually.
sccache [1] is one of the few ccache like solutions that will
work on windows.
sccache support can be enabled with the `WITH_WINDOWS_SCCACHE`
cmake option however it will only will work with ninja as the
build system, msbuild is not supported currently.
Advanced option, developes are expected to obtain and configure
sccache on their own.
```
Full build no cache 1428.90s (100.00%)
Full build cached 434.34s ( 30.40%)
```
[1] https://github.com/mozilla/sccache
Reviewed By: nicholas_rishel, Brecht
Differential Revision: https://developer.blender.org/D7466
Previously the MSVC environment was setup every time using vcvarsall.bat
when you ran you ran rebuild.cmd, Generally not an issue but after many
rebuilds on the same console, it grows the path environment variable
beyond what is supported and building breaks.
This patch adds a check to see if the environment is setup already
and skips the call to vcvarsall.bat
Also cleans up the double build in the msbuild's version of rebuild.cmd
install.vcxproj will build all that is needed, so no need to do a regular
build first.
1) Clang was given the wrong VS version to emulate when used in
combination with VS2019 causing build issues.
2) The erroneous supplied parameter `-std::c++11`caused CMake to
fail running its compiler detection scripts.
When calling make.bat multiple times to rebuild blender
make.bat failed to rebuild if a custom build dir was set.
reported and fixed on chat by @dgsantana
