Flamenco/FEATURES.md

Flamenco-NG Features

This file lists the features that ought to be implemented at some point in time. It will likely be expanded over time while forgotten features of Flamenco v2 are rediscovered.

Note that list is not in any specific order.

Manager

• OpenAPI implementation that actually runs as a server process

• Process lifecycle management, clean shutdown

• Task Scheduler

• Unify command definition (type/parameters in persistence layer, name/settings in API layer)

• Job status change handling

• Worker-assigned task re-queueing on sign-off

• Variable replacement

• Configuration file reading

• Shaman integration

• Worker API authentication

• Web frontend for Job & Task management

  • Jobs list
  • Job details
  • Job action buttons
  • Notification/status bar system
  • Tasks list
  • Task details
  • Task action buttons
  • Notification history pop-over
  • Task log viewer
  • Subscription system for "job detail+task" updates.
  • Subscription system for "task log" updates.
  • Filtering jobs by status
  • Filtering tasks by status

• Make tables respect the window height

  • Jobs
  • Tasks
  • Workers

• Web frontend for Worker management

  • "Tabs" system + Vue Router support.
  • API endpoint for worker status change requests
  • Web: Worker action buttons
  • Implementation of lazy vs. forced status change requests

• Port the old 'fail-requested' task status handling code to the new Manager

• At startup check & fix "stuck" jobs.

  • Jobs in transitional statuses cancel-requested, requeueing, etc.
  • Jobs with impossible to execute tasks. For example, consider the scenario where all but one worker were blocklisted for a certain task type, and the last worker that could run it, failed it. Now if that failure was that Worker's first one, it wouldn't get blocklisted and still counts as "can execute this task type on this job". However, since it failed the task, it won't be allowed to retry it, and thus the task will get stuck in soft-failed status.

• Task timeout monitoring

• Worker blocklisting & failed task requeueing

  • Keep track of which worker failed which task.
  • Clear task failure list when the task gets (re)queued.
  • Keep track of a blocklist as (worker ID, job ID, task type) tuple in the database.
  • Show blocklist in web interface, and allow removal of workers.

• Worker timeout monitoring

• Loading of job compiler scripts from disk

• Let Manager write to task log when it's assigned to a worker.

• Last rendered image display

  • Per-job last rendered image
  • Worker: Detect when Blender outputs an image, and send it as JPEG to the Manager
  • Manager: API endpoint for receiving images, and thumbnailing them.
  • SocketIO notifications that a job's last-rendered image has been updated.
  • API endpoint for getting the last-rendered image of a job.
  • Expand the processing queue from a single channel to a queue per job, so that a spammy job doesn't starve the other jobs from queueing images.

• Web interface: Support actions on multiple selected things

  • Workers
  • Jobs
  • Tasks

• Worker sleep schedule

• CLI option to write built-in job compiler scripts to disk

• Support pausing jobs.

• Support deleting jobs, including task logs, last-rendered images, etc.

• Support archiving jobs. Basically storing the job & task database entries as JSON, then ZIPping the whole lot before removing from the database & filesystem.

• Shaman/Job settings: Distinction between "can be auto-deleted" jobs and "must be kept around" jobs.

• Caching of job compilers. Currently every operation (including "fetch job type info", which is done by the web interface whenever job details are shown) will just parse the JS from disk again. It'll be better to only do this when the file on disk changed.

• API: only allow valid job & task status changes. Currently any change is allowed by the API.

• Frontend authentication

• User/Job Submission API authentication

• Auto-removal of old Workers

• Ensure "task state machine" can run in a single database transaction.

• Refactor internal/manager/task_logs so that it uses internal/manager/local_storage.

• Web interface: fetch the Flamenco Manager version on SocketIO reconnect.

Worker

• OpenAPI client that actually runs as a process
• Process lifecycle management, clean shutdown
• Task executor
• Command runner
• Log uploader
• Implement frames-to-video command.
• Implement move-directory command.
• Refactor CLI-running code by analyzing the current Blender and FFmpeg commands.
• Re-queueing of task at shutdown of the main context (i.e. when pressing Ctrl+C). Since the context is closed, everything shuts down, but then it's too late to cleanly communicate to the Manager that the current task should be requeued.
• CLI option to override configured manager URL.
• Filesystem readability/writability tests
  • Add check per task-type
  • Allow workers to request blocklisting for certain job, with "reason" string.
• POSIX signal handling (sleep/wake up)
• Remove doneChan field from Worker struct, and the handling of it. Its intended use overlaps with the global context.
• Change default location for writing files (sqlite database, credentials file) from "current working directory" to a platform-specific location in the user's home directory (~/.local/flamenco3/worker on Linux, %APPDATA%\Blender Foundation\Flamenco3 on Windows, etc.).
• Allow merging config files (load global from current working directory, merge with local from dir mentioned above).
• Keep retrying to register at Manager. Currently registration failure is a fatal error; it should be retried just like sign-on failures.

Both

• Worker registration & auto-reregistration
• Worker API authentication
• Worker sign on/off
• Task status change handling
• UPnP/SSDP for automatic Manager finding
• Worker status change management, including lazy status changes
• Worker test mode
• Worker registration secret (to protect the Manager from arbitrary Workers registering)
• Support the may-i-keep-running protocol thingy

Blender Add-on

• Communication with the Flamenco Manager API
• Job construction
• BAT-packing
• uploading to Shaman
• Editable job names (for rendering multiple jobs from the same filename)
• User authentication
• Add options={'HIDDEN'} to the Flamenco RNA properties.
• Add eTag-like checksum to the job types. The aim is to prevent artists from sending in a new job with an old job type. This can happen when the job type is changed on the Manager, but in Blender the "refresh" button wasn't pressed yet. This should only cover the job settings, not the rest of the compiler script.
• Frame Range: allow automatically syncing with scene frame start/end.
• Don't take output settings from the current scene, but have separate settings for Flamenco. To be fleshed out still.

Nice to have / new features

• Notification system to push "job done" messages to. Ideally would be in a form/shape that allows sending a message to Rocket.Chat, Matrix, Telegram, Discord, email, webbrowser, push URL-encoded/JSON/XML to some URL, stuff like that. Idea by Dan McLaughlin.
• Notification client inside Blender itself, so that you get a message when your job is done.
• Separate the OpenAPI definition of Shaman from the rest of Flamenco Manager. That way a part of BAT can also use the code generator. It also is the first step towards running Shaman as a standalone service.
• Web frontend for configuration management
• Configuration provisioning support
• Sending data upstream to Flamenco Overwatch
• Video preview of jobs, with black frames for unrendered frames.
• "Manual approval" task between low-quality/preview renders and rendering the final quality
• Grid view of jobs / tasks, showing the latest-rendered image/video.
• Split up OpenAPI-generated server code into an interface per tag. This'll help splitting up the rather large Flamenco struct into smaller pieces.
• Job "required specs", to match with certain workers.