Update Go from 1.22.2 to 1.22.3 (released 2024-05-07)
It includes security fixes to the go command and the net package, as
well as bug fixes to the compiler, the runtime, and the net/http
package.
Log any CLI command that's run in a way that can be easily copy-pasted
from the task log. This can help a lot in determining whether an issue
is caused by Flamenco or by the CLI program itself.
Replace GORM's auto-migration with Goose. The latter uses hand-written
SQL queries to apply database schema changes, which is safer and easier to
understand than what GORM is doing.
In addition to logging `GOOS` and `GOARCH`, also log more info about the
system:
- Windows: the Windows version and edition.
- Linux: distribution, distribution version, and kernel version.
- macOS: just "macOS", until we know more about getting info there too.
Vulnerability: GO-2023-2102
HTTP/2 rapid reset can cause excessive work in net/http
More info: https://pkg.go.dev/vuln/GO-2023-2102
Module: golang.org/x/net
Found in: golang.org/x/net@v0.7.0
Fixed in: golang.org/x/net@v0.17.0
Fix a vulnerability in the YAML parser library. It's in a part of the
library that Flamenco doesn't touch, which is why this isn't mentioned
in the CHANGELOG.md file.
------------
Found 1 vulnerability in packages that you import, but there are no call
stacks leading to the use of this vulnerability. You may not need to
take any action. See https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck
for details.
Vulnerability #1: GO-2022-0603
Panic in gopkg.in/yaml.v3
More info: https://pkg.go.dev/vuln/GO-2022-0603
Module: gopkg.in/yaml.v3
Found in: gopkg.in/yaml.v3@v3.0.0-20210107192922-496545a6307b
Fixed in: gopkg.in/yaml.v3@v3.0.0-20220521103104-8f96da9f5d5e
Fix a bug in the translation from 'human' frame ranges ('A-B,C-D') to
'Blender' frame ranges ('A..B,C..D'). It required calling
`string.replaceAll()` instead of `string.replace()` in the job compiler
script, which in turn required an upgrade of the JavaScript engine Goja.
Upgrade:
- `gorm.io/gorm` v1.23.8 → 1.25.2
- `github.com/glebarez/go-sqlite` v1.17.3 → v1.8.0
- `github.com/glebarez/sqlite` v1.4.6 → v1.8.0
and also some indirect dependencies.
This is in the hope that some weird cases at Blender Studio get resolved.
It appears that sometimes, for some unknown reason, when deleting a job,
its tasks get reassigned to another job (instead of also getting deleted).
Since there is no code in Flamenco itself to do this task deletion (it's
all depending on SQLite following the foreign keys and cascading to tasks),
I hope it was a bug in either GORM or SQLite that got fixed at some point.
Fix these vulnerabilities:
```
Vulnerability #1: GO-2023-1572
An attacker can craft a malformed TIFF image which will consume
a significant amount of memory when passed to DecodeConfig. This
could lead to a denial of service.
More info: https://pkg.go.dev/vuln/GO-2023-1572
Module: golang.org/x/image
Found in: golang.org/x/image@v0.0.0-20191009234506-e7c1f5e7dbb8
Fixed in: golang.org/x/image@v0.5.0
Vulnerability #2: GO-2023-1571
A maliciously crafted HTTP/2 stream could cause excessive CPU
consumption in the HPACK decoder, sufficient to cause a denial
of service from a small number of small requests.
More info: https://pkg.go.dev/vuln/GO-2023-1571
Module: golang.org/x/net
Found in: golang.org/x/net@v0.5.0
Fixed in: golang.org/x/net@v0.7.0
```
Upgrade Echo from v4.6.1 to 4.9.1. The security issue was fixed in 4.9.0,
so this upgrade picks the latest 4.9.x to upgrade to.
For more info on the security issue, see
https://pkg.go.dev/vuln/GO-2022-1031
Change the location where the Worker writes its local files so that it
follows the XDG specification (instead of writing to the current working
directory).
- Linux: `$HOME/.local/share/flamenco`
- Windows: `C:\Users\UserName\AppData\Local\Flamenco`
- macOS: `$HOME/Library/Application Support/Flamenco`
NOTE: The old files will not be loaded any more. This means that if
nothing is done and the new worker is run as-is, it will reregister as a
brand new worker. Move `flamenco-worker-credentials.yaml` and
`flamenco-worker.sqlite` to the new location to avoid this.
Upgrade just GORM and SQLite (and their dependencies), in a hope that it
will solve the "interrupted (9)" errors returned on our production Flamenco
Manager.
This adds a `-wizard` CLI option to the Manager, which opens a webbrowser
and shows the First-Time Wizard to aid in configuration of Flamenco.
This is work in progress. The wizard is just one page, and doesn't save
anything yet to the configuration.
Add a handler for the OpenAPI `taskOutputProduced` operation, and an
image thumbnailing goroutine.
The queue of images to process + the function to handle queued images
is managed by `last_rendered.LastRenderedProcessor`. This queue currently
simply allows 3 requests; this should be improved such that it keeps
track of the job IDs as well, as with the current approach a spammy job
can starve the updates from a more calm job.
When a Worker indicates a task failed, mark it as `soft-failed` until
enough workers have tried & failed at the same task.
This is the first step in a blocklisting system, where tasks of an
often-failing worker will be requeued to be retried by others.
NOTE: currently the failure list of a task is NOT reset whenever it is
requeued! This will be implemented in a future commit, and is tracked in
`FEATURES.md`.
Due to the way SSDP works, Flamenco Manager needs to know its own URL,
where the Workers can reach it. These URLs are now found, and since there
can be multiple (like IPv6 + IPv4) they are all sent in a SSDP
notification as ;-separated strings.
The chat client itself is just a throwaway project. The SocketIO system
will be used to send realtime updates about jobs, tasks, and workers to
the web frontend.
