Commit Graph

240 Commits

Author SHA1 Message Date
8e05024267 UPnP/SSDP: Worker, ping Manager URLs in parallel
Ping all Manager URLs at once, and wait until they've all responded (or
caused an error), instead of pinging them one by one sequentially.
2022-03-08 14:25:44 +01:00
9761786982 UPnP/SSDP: better logging when recovering from panic 2022-03-08 14:08:01 +01:00
eec4895c73 UPnP/SSDP client: avoid duplicate URLs 2022-03-08 14:06:35 +01:00
ae892ff712 Worker: test all discovered Manager URLs
Ping each discovered Manager URL to see whether it is reachable or not.
If there are multiple usable URLs, it'll just pick the first.
2022-03-08 13:56:45 +01:00
d6a60c73d0 UPnP/SSDP Server: allow advertising slice of URLs
This makes it possible for the Manager to expose multiple URLs. This way
the Worker can try them out and see which ones work.
2022-03-08 13:56:45 +01:00
13a74e61d4 UPnP/SSDP client: allow clean shutdowns
The SSDP library will log a warning on shutdown, as it doesn't expect the
UDP socket to be closed. Copy the hack on the Server implementation, which
avoids this warning getting logged, to the Client.
2022-03-08 13:56:45 +01:00
d0b677ffe3 Add API endpoint /api/version to get the Flamenco Manager version
This will be used in the auto-discovery, to check that a discovered URL can
actually be reached & points to a Flamenco Manager.
2022-03-08 13:56:43 +01:00
42407865eb Worker: change how configuration & credentials are loaded/saved
The Worker config/credential management was a bit of a mess. It's now
better structured, and also allows runtime overrides of the Manager URL,
without writing that override to the config file.
2022-03-08 13:51:03 +01:00
c0cd3ca5ad UPnP/SSDP: Cleaner, easier to test (and actually tested) code
Convert "get own URLs" code into nicer chunks, and test those.

This minimises the code that actually depends on the available network
interfaces, and increases test coverage. Found a few bugs too.
2022-03-08 12:11:47 +01:00
bb91c2e6d6 UPnP/SSDP: prefer link-local addresses if available 2022-03-08 11:03:48 +01:00
fca628b325 Merge remote-tracking branch 'origin/main' into upnp-ssdp 2022-03-07 15:27:12 +01:00
9f5e4cc0cc License: license all code under "GPL-3.0-or-later"
The add-on code was copy-pasted from other addons and used the GPL v2
license, whereas by accident the LICENSE text file had the GNU "Affero" GPL
license v3 (instead of regular GPL v3).

This is now all streamlined, and all code is licensed as "GPL v3 or later".

Furthermore, the code comments just show a SPDX License Identifier
instead of an entire license block.
2022-03-07 15:26:46 +01:00
43a9e1d439 Add some nice-to-have new features to FEATURES.md 2022-03-07 11:21:31 +01:00
4062f024b7 Cleanup: move code to a nicer place 2022-03-04 17:46:44 +01:00
d153db4280 Work in progress on using UPnP/SSDP to make the Worker find its Manager
Due to the way SSDP works, Flamenco Manager needs to know its own URL,
where the Workers can reach it. These URLs are now found, and since there
can be multiple (like IPv6 + IPv4) they are all sent in a SSDP
notification as ;-separated strings.
2022-03-04 17:44:04 +01:00
656a495652 Manager: clean shutdown on Ctrl+C 2022-03-04 14:34:40 +01:00
1e784452f3 README: no longer a "proof of concept" 2022-03-04 13:24:57 +01:00
183bb23d2a Tests: sleep test using bigger time steps
Since every mocked clock time step also waits for 1ms to give other
goroutines a chance to run, it took too much wallclock time to mock-sleep
for 47 seconds with 100ms increments.

Stepping the mocked clock with 1s increments makes the test 10x faster.
2022-03-04 12:35:01 +01:00
dbb9c71df8 Tests: more unified way to do database tests 2022-03-04 12:33:45 +01:00
f497ac8536 Cleanup: add and remove some comments 2022-03-04 12:19:19 +01:00
3bfd5a339f Fix database tests getting interrupted
The root cause was a 2nd `context.Context()` that was used in
`constructTestJob()`, which cancelled when that function returned.

The cancellation of the context caused an interrupt in the SQLite
driver, which got into a race condition and could cause an interrupt on
a subsequent database query.
2022-03-04 12:19:19 +01:00
de150567b0 Manager: avoid double error message 2022-03-04 11:37:29 +01:00
cd2fe8170e Errors: remove "error" prefix from message
Instead of returning an error "error doing X", just return "doing X". The
fact that it's returned as an error object says enough about that it's
an error.

This also makes it easier to chain error messages, without seeing the
word "error" in every part of the chain.
2022-03-04 11:30:31 +01:00
02ce95070f Add VSCode extension recommendations 2022-03-04 11:13:17 +01:00
3c6d7773ef Crosspath: add ToNative() function
Add `crosspath.ToNative(path)`, which returns the path with platform-
native path separators. This is meant for use in the Worker, to convert
paths before attempting to use them.
2022-03-04 11:10:10 +01:00
70f8140df5 Crosspath: extend tests with "long UNC" paths
See https://en.wikipedia.org/wiki/Path_(computing)#Universal_Naming_Convention
2022-03-04 11:09:14 +01:00
0a97d04741 Cleanup: rewrite crosspath.Stem unit test
Use the same style of testing as the other test functions in the file.

No functional changes.
2022-03-04 10:27:31 +01:00
580b3e5726 Go.mod: mark github.com/glebarez/sqlite as directly used 2022-03-03 15:26:48 +01:00
c91e7b1cac Manager: solve failing unittests by implementing some filepath functions
Both Go's standard `path` and `path/filepath` packages are too limiting to
work well for Flamenco. The former assumes Linux/POSIX paths, the latter
only works with platform-native paths. Neither can work with Windows paths
on Linux, or Linux paths on Windows.
2022-03-03 15:22:43 +01:00
b9609f8866 Cleanup: remove unused code 2022-03-03 13:52:57 +01:00
641ed7ace9 Manager: make Gorm use Zerolog for logging
A wrapper for Zerolog implements the Gorm logger interface. This gives
us coloured output on Windows, and uniform-looking logs in production.
2022-03-03 13:52:50 +01:00
8824489980 Manager: use in-memory SQLite database for testing
The on-disk database that was used before caused issues with tests running
in parallel. Not only is there the theoretical issue of tests seeing each
other's data (this didn't happen, but could), there was also the practical
issue of one test running while the other tried to erase the database file
(which fails on Windows due to file locking).
2022-03-03 13:51:55 +01:00
9b9c6bffff Replace self-hacked SQLite Gorm driver with 3rd party one
The new Gorm driver is made by the creators of the pure-Go SQLite library
we were already using.
2022-03-03 13:48:14 +01:00
2b04623e00 Manager: fix DB transaction isolation issue in task scheduler
The created transaction wasn't actually used for the should-be-in-the-
transaction queries. That's now resolved.
2022-03-03 13:46:27 +01:00
a563f36b98 README: document licenses better 2022-03-03 12:10:24 +01:00
9643bf768e Manager: Fix DB migration error of not-null columns
Where the PostgreSQL DB migration code could handle `NOT NULL` columns just
fine, SQLite has less table-altering functionality. As a result, migrations
have to copy entire database tables, which doesn't play well with
not-nullable columns.
2022-03-03 12:10:13 +01:00
bf4cc9b056 Add notice to our SQLite Gorm driver
The Gorm driver README now links to the original code, which was modified
to use a different SQLite library.
2022-03-03 11:52:43 +01:00
a7654394b4 Worker: update default task types to match what's currently in use
Flamenco v2 had a different classification of task types than Flamenco v3,
and the defaults still referenced the v2 classifications.
2022-03-03 11:51:53 +01:00
2610836861 Makefile: don't generate code when running make
Generated code is to be committed to Git anyway, so there is no need to
regenerate it on every build.

The code can be regenerated explicitly by running `make generate`.
2022-03-03 11:51:15 +01:00
efbbb8c7d8 Makefile: remove some unused parts
Some parts were still leftovers from other projects this Makefile was
copied from.
2022-03-03 11:50:03 +01:00
42daa87839 Makefile: convert generated files to native line-ends on Windows
Git wants to see native line-ends in source files, but the code generators
we use always write UNIX line-ends. `make generate` on Windows now passes
generated files through `unix2dos`. This allows regenerating files without
Git listing them as modified.
2022-03-03 11:49:26 +01:00
eaeea38e3f Worker: remove -manager URL CLI argument
This CLI arg isn't actually used by the subsequent code, so better to not
have it for now.
2022-03-03 11:33:22 +01:00
20fd3228ba Worker: better logging configuration
Always show startup notification at INFO level, regardless of configured
level. Replace `-verbose` with `-quiet`, add `-trace`.
2022-03-03 11:32:57 +01:00
954babd914 Manager: redirect web root to Swagger UI
Temporarily redirect the index page to the Swagger UI, so that at least
you can see something other than a "404 Not Found" error.
2022-03-01 20:49:11 +01:00
ddd0c7602b Change application name to "Flamenco 3"
"Flamenco NG PoC" was the name of the proof of concept, and it's no longer
in use.
2022-03-01 20:45:13 +01:00
47e36c927c Change package URL to the blender.org repository 2022-03-01 20:45:09 +01:00
e70a44a146 Manager: switch from PostgreSQL to SQLite
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
2022-03-01 18:50:31 +01:00
60d1710020 Addon: job type form seems to be working 2022-03-01 17:28:24 +01:00
5e7e2adef0 Addon: Fixed mypy errors 2022-03-01 17:28:24 +01:00
616784df0a Addon: start of framework for API communication 2022-03-01 17:28:24 +01:00