Changes to an extensions manifest weren't accounted for.
This was particularly a problem for "System" extensions which aren't
intended to be managed inside Blender however the problem existed for
any changes made outside of Blender.
Now enabled extensions are checked on startup to ensure:
- They are compatible with Blender.
- The Python wheels are synchronized.
Resolves#123645.
Details:
- Any extension incompatibilities prevent the add-on being enabled
with a message printing the reason for it being disabled.
- Incompatible add-ons are kept enabled in the preferences to avoid
loosing their own preferences and allow for an upgrade to restore
compatibility.
- To avoid slowing down Blender's startup:
- Checks are skipped when no extensions are enabled
(as is the case for `--factory-startup` & running tests).
- Compatibility data is cached so in common case,
the cache is loaded and all enabled extensions `stat` their
manifests to detect changes without having to parse them.
- The cache is re-generated if any extensions change or the
Blender/Python version changes.
- Compatibility data is updated:
- On startup (when needed).
- On an explicit "Refresh Local"
(mainly for developers who may edit the manifest).
- When refreshing extensions after install/uninstall etc.
since an incompatible extensions may become compatible
after an update.
- When reloading preferences.
- Additional info is shown when the `--debug-python` is enabled,
if there are ever issues with the extension compatibility cache
generation not working as expected.
- The behavior for Python wheels has changed so they are only setup
when the extension is enabled. This was done to simplify startup
checks and has the benefit that an installed but disabled extension
never runs code - as the ability to install wheels means it could
have been imported from other scripts. It also means users can disable
an extension to avoid wheel version conflicts.
This does add the complication however that enabling add-on which is
an extension must first ensure it's wheels are setup.
See `addon_utils.extensions_refresh(..)`.
See code-comments for further details.
Support for "System" extensions as an alternative to the current
"User" extensions repository.
The purpose of this change is to support bundling extensions for
offline work or in environments where users setting up thier own
extensions isn't desirable, see #122512.
Details:
The default "System" repository on Linux will for example use:
- `/usr/share/blender/4.2/extensions/{system}` For system installs.
- `./4.2/extensions/{system}` For portable installs.
- Blender's default startup now has a "System" repository
which users or administrators may populate.
- Repositories can select between User/System paths,
setting a custom path overrides overrides this setting.
- Add "BLENDER_SYSTEM_EXTENSIONS" (matching "BLENDER_LOCAL_EXTENSIONS").
Ref !122832
Originally this would replace scripts that come bundled with Blender,
but it's unclear how this is useful.
Searching for this online mainly leads to people asking how they can
use it to add scripts. For example in a studio environment you might
want to deploy add-ons and startup scripts for all users.
Even if you wanted to use it for replacement though, it wasn't really
doing that and inconsistent for different types of scripts:
* startup: ignored
* modules: replaces bundled scripts
* presets: adds to bundled scripts
* addons (in 4.1): ignored
* addons_core (in 4.2): ignored
* startup/bl_app_templates_system: replaces bundled scripts
This change makes it add scripts from this path for all. This is a
breaking change, though arguably this feature was just broken to
begin with and not used much in practice because of that.
The alternative would be add a new set of environment variables to
avoid breaking existing behavior. But that also means keeping around the
broken behavior or fixing it in another way.
Supporting multiple paths may be used too, but for now just support
a single one as doing this for all BLENDER_SYSTEM variables is
non-trivial. The main use case for that would be add-ons anyway, and
those will mainly be handled through upcoming
BLENDER_SYSTEM_EXTENSIONS instead.
Ref #122512
Pull Request: https://projects.blender.org/blender/blender/pulls/122689
This is the first commit of the several required to support
subprocess-based parallel compilation on OpenGL.
This provides the base API and implementation, and exposes the max
subprocesses setting on the UI, but it's not used by any code yet.
More information and the rest of the code can be found in #121925.
This one includes:
- A new `GPU_shader_batch` API that allows requesting the compilation
of multiple shaders at once, allowing GPU backed to compile them in
parallel and asynchronously without blocking the Blender UI.
- A virtual `ShaderCompiler` class that backends can use to add their
own implementation.
- A `ShaderCompilerGeneric` class that implements synchronous/blocking
compilation of batches for backends that don't have their own
implementation yet.
- A `GLShaderCompiler` that supports parallel compilation using
subprocesses.
- A new `BLI_subprocess` API, including IPC (required for the
`GLShaderCompiler` implementation).
- The implementation of the subprocess program in
`GPU_compilation_subprocess`.
- A new `Max Shader Compilation Subprocesses` option in
`Preferences > System > Memory & Limits` to enable parallel shader
compilation and the max number of subprocesses to allocate (each
subprocess has a relatively high memory footprint).
Implementation Overview:
There's a single `GLShaderCompiler` shared by all OpenGL contexts.
This class stores a pool of up to `GCaps.max_parallel_compilations`
subprocesses that can be used for compilation.
Each subprocess has a shared memory pool used for sending the shader
source code from the main Blender process and for receiving the already
compiled shader binary from the subprocess. This is synchronized using
a series of shared semaphores.
The subprocesses maintain a shader cache on disk inside a
`BLENDER_SHADER_CACHE` folder at the OS temporary folder.
Shaders that fail to compile are tried to be compiled again locally for
proper error reports.
Hanged subprocesses are currently detected using a timeout of 30s.
Pull Request: https://projects.blender.org/blender/blender/pulls/122232
With #120283 the Win32 codepath for registration creates an endless
loop. Command-line arguments are detected for registration that causes
another blender to launch with the same arguments, that repeats
forever. But the only time these arguments is used is when Blender has
already been specifically launched with the permissions needed to
register. So this means it should just do so and then exit.
Pull Request: https://projects.blender.org/blender/blender/pulls/122682
Add a preference to "Work Offline" system preference as well as command
line options `--offline-mode` & `--online-mode`
(which overrides the preference).
This option is displayed in the initial setup screen too.
This is currently respected by:
- Check for updates on startup
- Disables running an update when enabling extensions.
When Blender is launched with `--offline-mode` the option cannot be
enabled in the preferences. This is intended for environments
where internet access is intentionally disallowed.
Background: with Blender supporting access to online-repositories
as well as 3rd party extensions themselves potentially accessing the
internet. This setting provides a way for users to disable online
functionality.
This prevents error messages when online access fails in environments
without internet access as well as the ability for users who prefer
Blender doesn't access the internet to have one place to turn this off.
While it does not enforce limitations on add-ons, 3rd party scripts
are expected to respect this setting using `bpy.app.internet_offline`.
The details for this will be handled along with other policies scripts
are expected to follow.
Ref !121994
The extensions system allows to extend Blender with connectivity to the internet. Right now it means Blender can
discover and install add-ons and themes directly from the internet, and notify users about their updates.
By default this is disabled (opt-in), and users can enable it the first time they try to install an extension or visit
the Prefences > Extensions tab. If this is enabled, Blender will automatically check for updates for
extensions.blender.org upon startup.
When will Blender access the remote repositories:
* Every time you open the Preferences → Extensions: ALL the enabled repositories get checked for the latest info (json)
* Every time you try to install by dragging: ALL the enabled repositories get checked for the latest info (json).
* Every time you start Blender: selected repositories get checked for the latest info (json).
------------------
From the Blender code point of view, this means that most of the add-ons and themes originally bundled with Blender
will now be available from the online platform, instead of bundled with Blender. The exception are add-ons which are
deemed core functionality which just happened to be written as Python add-ons.
Links:
* Original Extenesions Platform Announcement: https://code.blender.org/2022/10/blender-extensions-platform/
* Extensions website: https://extensions.blender.org/
* User Manual: https://docs.blender.org/manual/en/4.2/extensions/index.html#extensions-index
* Technical specifications: https://developer.blender.org/docs/features/extensions/
* Changes on add-ons bundling: https://devtalk.blender.org/t/changes-to-add-on-bundling-4-2-onwards/34593
------------------
This PR does the following:
* Move extensions out of experimental.
* No longer install `scripts/addons` & `scripts/addons_contrib`.
* Add `scripts/addons_core` to blender's repository.
These add-ons will still be bundled with Blender and will be always enabled in the future, with their preferences
moved to be more closely integrated with the rest of Blender. This will happen during the remaining bcon2 period.
For more details, see #121830
From scripts/addons:
* copy_global_transform.py
* hydra_storm
* io_anim_bvh
* io_curve_svg
* io_mesh_uv_layout
* io_scene_fbx
* io_scene_gltf2
* pose_library
* ui_translate
* viewport_vr_preview
Extra: bl_pkg (scripts/addons_contrib)
Note: The STL (legacy) add-on is going to be moved to the extensions platform. There is already a C++ version on core
which is enabled by default.
All the other add-ons are already available at extensions.blender.org. To use them you need to:
* Go to User Preferences > Extensions
* You will be greated with an "Online Extensions" message, click on "Enable Repository".
* Search the add-on you are looking for (e.g, Import Images as Planes).
* Click on Install
Over time their maintaince will be transferred over to the community so their development can carry on. If you used to
help maintain a bundled add-on please read: https://devtalk.blender.org/t/changes-to-add-on-bundling-4-2-onwards/34593
Ref: !121825
Move all header file into namespace.
Unnecessary namespaces was removed from implementations file.
Part of forward declarations in header was moved in the top part
of file just to do not have a lot of separate namespaces.
Pull Request: https://projects.blender.org/blender/blender/pulls/121637
While additional context is typically useful to include,
this is such a corner-case that it's not expected script authors
would run into this during regular development.
Support for building blender with clang on windows on x64 was added
years ago but given there are no active users support has crumbled a
bit.
This PR brings the build system back into working order but upstream
patches in openVDB are still required for a successful build see PR
#120317 for details.
Blender when build with clang the classroom scenes rendered on the cpu
with cycles is seeing a 5% reduction in render time on both an
AMD 7700x and an Intel 14900k.
This PR adds a cmake option `WITH_WINDOWS_EXTERNAL_MANIFEST`
which is off by default which addresses the following 2 problems:
The CI env occasionally fails to link the manifest into blender.exe
with mt.exe getting file in use error. The solutions mentioned online
vary wildly between, just rebuild, turn off your AV, use this magic
switch. None of them actually point to a root cause we can address.
When building blender with clang and the visual studio generator
it also somehow doesn't embed the manifest.
If the bots stay problematic this option can be turned on for the CI
environment, and will be automatically turned on when it detects clang
and the visual studio generator being used.
Pull Request: https://projects.blender.org/blender/blender/pulls/111683
Support freedesktop file association on Linux/Unix via the command line
arguments: `--register{-allusers}` `--unregister{-allusers}` as well
registration actions from the user preferences.
Once registered, the "Blender" application is available from launchers
and `*.blend` files are assoisated with the blender binary used for
registration.
The following operations are performed:
- Setup the desktop file.
- Setup the file association & make it default.
- Copy the icon.
- Setup the thumbnailer (`--register-allusers` only).
Notes:
- Registering/unregistering for all users manipulates files under
`/usr/local` and requires running Blender as root.
From the command line this can be done using `sudo`, e.g.
`sudo ./blender --register-allusers`.
From the GUI, the `pkexec` command is used.
- Recent versions of GNOME execute the thumbnailer in a restricted
environment (`bwrap`) requiring `blender-thumbnailer` to be copied
into `/usr/local/bin` (synlinks don't work).
So thumbnailing copies the binary rather than linking and only works
when registering for all users.
Ref !120283
This cleans up our `TEST_SSE_SUPPORT` macro to only test
for SSE42 and passes the flags to the CMAKE_C/CXX_FLAGS
the cpu check module needed to move to its own folder since
the flags at the end of a CMakeLists.txt appear to be used
for all targets inside a CMakeLists file and cpu_check cannot
be build with sse42 flags.
This only affects Mac/Linux since MSVC has no buildflags
to target SSE42
Pull Request: https://projects.blender.org/blender/blender/pulls/118468
MoltenVK original intent was to let developers work on a mac system developing
for the vulkan eco-system. MoltenVK doesn't support all the features that we
require and would require additional workarounds to be actually supported.
It is not expected that we will release Blender with MoltenVK for this reason.
But it still has value for shader developers to validate shaders on metal and
vulkan on a single platform.
Pull Request: https://projects.blender.org/blender/blender/pulls/117940
While calling into scripts with an empty `Main` shouldn't be possible
under normal circumstances, I ran into this during development.
This situation is meant to print an error, add a null check so it works
as expected.
Making arguments call into Python was impractical because Python
is only initialized for ARG_PASS_FINAL.
Replace "--command" specific logic with a general method of arguments
requesting to be executed once all sub-systems have been initialized.
Without this, the `main()` function needs hard coded logic to support
any time an argument needs to use Python internally.
Replace: `{BLENDER_RESOURCE_PATH_USER}/extensions`
With: `{BLENDER_USER_EXTENSIONS}`
This follows BLENDER_USER_CONFIG & BLENDER_USER_SCRIPTS conventions.
Reading the environment variable and accessible via
`bpy.utils.user_resource('SCRIPTS')`
Add support for add-ons to define commands using the new argument
`-c` or `--command`.
Commands behave as follows:
- Passing in a command enables background mode without the need to pass
in `--background`.
- All arguments following the command are passed to the command
(without the need to use the `--` argument).
- Add-ons can define their own commands via
`bpy.utils.register_cli_command` (see examples in API docs).
- Passing in `--command help` lists all available commands.
Ref !119115
* Only works on machines with a Qualcomm Snapdragon 8cx Gen3 or above.
Older generation devices are not and will not be supported due to
some driver issues
* Requires VS2022 for building.
* Uses new MSVC preprocessor for sse2neon compatibility.
* SIMD is not enabled, waiting on conversion of blenlib to C++.
Ref #119126
Pull Request: https://projects.blender.org/blender/blender/pulls/117036
Other minor changes:
- Remove reference to `TMP` environment variable in the `--help` text
as this isn't a posix standard and is noted as something that may be
removed in the future.
- Show WIN32 `TEMP` when showing all help text so it will be included
in the user manual.
This was a debug feature for developers that allowed writing the last MemFile
undo step as a .blend file when Blender crashed. This was rarely/never used
nowadays and is now removed.
This is in preparation for #106903 which removes support for writing `MemFile`
undo steps as .blend files.
