Commit Graph

34 Commits

Author SHA1 Message Date
2f76df437b T99415: Worker: change default location for writing local files
Change the location where the Worker writes its local files so that it
follows the XDG specification (instead of writing to the current working
directory).

- Linux:   `$HOME/.local/share/flamenco`
- Windows: `C:\Users\UserName\AppData\Local\Flamenco`
- macOS:   `$HOME/Library/Application Support/Flamenco`

NOTE: The old files will not be loaded any more. This means that if
nothing is done and the new worker is run as-is, it will reregister as a
brand new worker. Move `flamenco-worker-credentials.yaml` and
`flamenco-worker.sqlite` to the new location to avoid this.
2022-07-19 12:08:41 +02:00
d425bca5cf Upgrade GORM and SQLite
Upgrade just GORM and SQLite (and their dependencies), in a hope that it
will solve the "interrupted (9)" errors returned on our production Flamenco
Manager.
2022-07-19 10:39:19 +02:00
5fbf44ac30 Revert "Upgraded all Go dependencies"
This reverts commit b04889dd79986eca9547108c60fa61ac0ef4cc71. It caused
some errors that I'm not willing to debug this time of day ;-)
2022-07-18 19:44:24 +02:00
b04889dd79 Upgraded all Go dependencies
Ran `go get -t -u ./...` to get all the main + test dependencies upgraded.
2022-07-18 19:39:43 +02:00
aa9837b5f0 First incarnation of the first-time wizard
This adds a `-wizard` CLI option to the Manager, which opens a webbrowser
and shows the First-Time Wizard to aid in configuration of Flamenco.

This is work in progress. The wizard is just one page, and doesn't save
anything yet to the configuration.
2022-07-14 11:17:03 +02:00
e687c95e5d Manager: add "last rendered image" processing pipeline
Add a handler for the OpenAPI `taskOutputProduced` operation, and an
image thumbnailing goroutine.

The queue of images to process + the function to handle queued images
is managed by `last_rendered.LastRenderedProcessor`. This queue currently
simply allows 3 requests; this should be improved such that it keeps
track of the job IDs as well, as with the current approach a spammy job
can starve the updates from a more calm job.
2022-06-24 16:51:11 +02:00
6e12a2fb25 Manager: keep track of which worker failed which task
When a Worker indicates a task failed, mark it as `soft-failed` until
enough workers have tried & failed at the same task.

This is the first step in a blocklisting system, where tasks of an
often-failing worker will be requeued to be retried by others.

NOTE: currently the failure list of a task is NOT reset whenever it is
requeued! This will be implemented in a future commit, and is tracked in
`FEATURES.md`.
2022-06-13 18:41:38 +02:00
d153db4280 Work in progress on using UPnP/SSDP to make the Worker find its Manager
Due to the way SSDP works, Flamenco Manager needs to know its own URL,
where the Workers can reach it. These URLs are now found, and since there
can be multiple (like IPv6 + IPv4) they are all sent in a SSDP
notification as ;-separated strings.
2022-03-04 17:44:04 +01:00
9b9c6bffff Replace self-hacked SQLite Gorm driver with 3rd party one
The new Gorm driver is made by the creators of the pure-Go SQLite library
we were already using.
2022-03-03 13:48:14 +01:00
e70a44a146 Manager: switch from PostgreSQL to SQLite
This includes a modified copy of the Gorm SQLite backend, adjusted to use
https://modernc.org/sqlite instead.
2022-03-01 18:50:31 +01:00
2a1f5a0df4 Worker: buffer task updates when Manager offline
Queue task updates in an SQLite database when the Manager is
unreachable, periodically checking whether they can be flushed.
2022-02-22 19:05:33 +01:00
32c7a1fd84 Worker: Blender command, parse CLI arguments from 'exe' parameter 2022-02-22 12:52:16 +01:00
Sybren A. Stüvel
6bd1a86337 Start of DB initialisation functions
Not properly working yet
2022-02-21 15:25:56 +01:00
bb53cc1e4a Task log storage service 2022-02-18 18:19:35 +01:00
8424e690ee Bump Go version to 1.18 for generics support 2022-02-18 11:41:31 +01:00
c4df62d5d4 Start of sending task updates to Manager
This includes a mocking framework for unittests.
2022-02-15 15:58:24 +01:00
58619ea6db Cleanup: go mod tidy 2022-02-15 11:20:54 +01:00
96023932da SocketIO based chat client as PoC for backend/frontend communication
The chat client itself is just a throwaway project. The SocketIO system
will be used to send realtime updates about jobs, tasks, and workers to
the web frontend.
2022-02-11 14:47:26 +01:00
7c14b2648d Much more of the Worker life cycle implemented 2022-01-31 15:02:05 +01:00
c501899185 Ported lots of stuff from gitlab.com/dr.sybren/flamenco-worker-go
Much isn't working though.
2022-01-28 17:02:50 +01:00
3b1cb3c187 Remove sqlite library & run go mod tidy 2022-01-28 14:52:46 +01:00
2a4bf1c60f Toying around with GORM and PostgreSQL 2022-01-24 16:59:48 +01:00
0629728ce9 Add frame chunker and make unit test for simple blender render succeed 2022-01-13 16:32:08 +01:00
0c9c99806b Start of database interface
Contains a dummy migration because the migrate package will error out if
it cannot find any migration files.
2022-01-11 18:07:34 +01:00
Sybren A. Stüvel
3e771cb7b7 Basic HTTP auth for workers implemented
Still a dummy, but at least it works with the OpenAPI 3
auth specification.
2022-01-10 17:45:13 +01:00
Sybren A. Stüvel
a8ada28a41 Hook up Zerolog to Echo 2022-01-10 17:45:13 +01:00
Sybren A. Stüvel
949ff08b0e Switch from Gin to Echo framework 2022-01-10 17:45:13 +01:00
Sybren A. Stüvel
38a8943767 Add Flamenco Worker PoC 2022-01-10 17:45:13 +01:00
Sybren A. Stüvel
af4304565a Lots of changes to merge the Gin/OpenAPI PoC 2022-01-10 17:45:13 +01:00
Sybren A. Stüvel
2f4c22bb39 Use UUID for job ID 2022-01-10 17:45:12 +01:00
Sybren A. Stüvel
24db04455c Add creation of 'create-video' task + setting dependencies 2022-01-10 17:45:12 +01:00
Sybren A. Stüvel
96f2a6bb52 Allow importing stuff from JS 2022-01-03 19:51:22 +01:00
Sybren A. Stüvel
4e741d75f4 Correct logging colors on Windows 2022-01-03 19:51:22 +01:00
Sybren A. Stüvel
ae1ea0e2bc Initial checkin 2022-01-03 19:51:22 +01:00