It is now possible to display per-frame track reprojection
error in curve view of clip editor. Simply enable corresponding
option in filter buttons.
Currently displayed using blue color which might confuse with
average reprojection error, further color tweaks are possible
and easy.
Also changed icon track x/y curves. Better icons here are
really appreciated.
Summary:
Now it's possible to assign an image to plane tracks
in clip editor. This image is only used for display
in clip editor and this image is being warped into
the plane track rectangle.
Main purpose of this is to get early feedback about
how good image warping matches the footage, before
clip goes to the compositor.
Pretty much straightforward change: just compute
homography from undeformed normalized frame corner
coordinates (unity square) to plane marker corners
and apply this matrix to opengl stack.
Still could improve behavior when perspective
plane transform is degenerate, but that's not so
much critical for now i'd say.
Reviewers: brecht, campbellbarton
Reviewed By: brecht
CC: sebastian_k
Differential Revision: http://developer.blender.org/D57
Notes:
* Did not touch to addons, that's up to the authors. ;)
* Did not removed any "name" field below lists. We might want to do this in some cases (less UI clutter), but probably not always, so will let maintainers of the related areas decide here.
Added a weight slider to track which defines
how much particular track affects in a final
reconstruction. This weight is for sure
animateable.
Currently it affects on BA step only which in
most cases will work just fine.
The usecase of this slider is to have it set
to 1.0 most of the time where the track is
good, but blend it's weight down to 0 when
tracker looses the track. This will prevent
camera from jump.
Tutorial is to be done by Sebastian.
It was rather confusing from the user usage point
of view and didn't get so much improvement after
new bundle adjuster was added.
In the future we might want to switch resection
to PPnP algorithm, which could also might be a
nice alternative to fallback option.
- Do plane re-evaluation only when transform is actually done.
Before this re-evaluation happened on every mouse move.
- Added a flag "Auto Keyframe" for the plane track, which does:
* If Auto Keyframe is enabled, then every manual edit of the
plane will create a new keyframe at current frame and update
plane motion between current frame and previous/next keyframe.
This now also implies blending detected motion with neighbor
keyframes, so there's no jump happening.
No automatic update on manual point tracks edit will happen.
* If auto Keyframe is disabled, then no keyframes are adding
to the plane and every plane tweak will re-evaluate in on
the whole frame range.
In this case manual tweaks to point tracks and re-tracking
them implies plane re-evaluation.
Notes:
* Made those edits by full checking of py files, so I should have spoted most needed edits, yet it remains quite probable I missed a few ones, we'll fix if/when someone notice it...
* Also made some cleanup "on the road"!
- Collapse plane track panels by default
- Hide 3D markers when in mask edit mode
- Remove alpha from mask layers list
Discussed with Sebastian and Roman.
This commit includes all the changes made for plane tracker
in tomato branch.
Movie clip editor changes:
- Artist might create a plane track out of multiple point
tracks which belongs to the same track (minimum amount of
point tracks is 4, maximum is not actually limited).
When new plane track is added, it's getting "tracked"
across all point tracks, which makes it stick to the same
plane point tracks belong to.
- After plane track was added, it need to be manually adjusted
in a way it covers feature one might to mask/replace.
General transform tools (G, R, S) or sliding corners with
a mouse could be sued for this. Plane corner which
corresponds to left bottom image corner has got X/Y axis
on it (red is for X axis, green for Y).
- Re-adjusting plane corners makes plane to be "re-tracked"
for the frames sequence between current frame and next
and previous keyframes.
- Kayframes might be removed from the plane, using Shit-X
(Marker Delete) operator. However, currently manual
re-adjustment or "re-track" trigger is needed.
Compositor changes:
- Added new node called Plane Track Deform.
- User selects which plane track to use (for this he need
to select movie clip datablock, object and track names).
- Node gets an image input, which need to be warped into
the plane.
- Node outputs:
* Input image warped into the plane.
* Plane, rasterized to a mask.
Masking changes:
- Mask points might be parented to a plane track, which
makes this point deforming in a way as if it belongs
to the tracked plane.
Some video tutorials are available:
- Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4
- Artist video: https://vimeo.com/71727578
This is mine and Keir's holiday code project :)
Now button in the toolshelf behaves this way:
- User clicks on "Add Marker"
- Then he clicks where the marker should get placed
Patch by Marcos Couto (ocf) with own modifications.
Implements an automatic keyframe selection algorithm which uses
couple of approaches to find out best keyframes candidates:
- First, slightly modifier Pollefeys's criteria is used, which
limits correspondence ration from 80% to 100%. This allows to
reject keyframe candidate early without doing heavy math in
cases there're not much common features with first keyframe.
- Second step is based on Geometric Robust Information Criteria
(aka GRIC), which checks whether features motion between
candidate keyframes is better defined by homography or
fundamental matrices.
To be a good keyframe candidate, fundamental matrix need to
define motion better than homography (in this case F-GRIC will
be smaller than H-GRIC).
This two criteria are well described in this paper:
http://www.cs.ait.ac.th/~mdailey/papers/Tahir-KeyFrame.pdf
- Final step is based on estimating reconstruction error of
a full-scene solution using candidate keyframes. This part
is based on the following paper:
ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf
This step requires reconstruction using candidate keyframes
and obtaining covariance matrix of 3D points positions.
Reconstruction was done pretty much straightforward using
other simple pipeline routines, and for covariance estimation
pseudo-inverse of Hessian is used, which is in this case
(J^T * J)+, where + denotes pseudo-inverse.
Jacobian matrix is estimating using Ceres evaluate API.
This is also crucial to get rid of possible gauge ambiguity,
which is in our case made by zero-ing 7 (by gauge freedoms
number) eigen values in pseudo-inverse.
There're still room for improving and optimizing the code,
but we need some point to start with anyway :)
Thanks to Keir Mierle and Sameer Agarwal who assisted a lot
to make this feature working.
Use center of currently visible frame part instead of
center of the whole frame for position of marker which
is adding from toolbox.
Used separate operator for this to keep operators more
atomic and not confuse with lots of conflicting properties.
This operator will run a tracker from previous
keyframe to current frame for all selected markers.
Current markers positions are considering initial
position guess which could be updated by a tracker
for better match.
Useful in cases when feature disappears from the
frame and then appears again. Usage in this case
is the following:
- When feature point re-appeared on frame, manully
place marker on it.
- Use Refine Markers operation (which is in Track
panel) to allow tracker to find a better match.
Depending on direction of tracking use either
Forwards or Backwards refining. It's easy: if
trackign happens forwards, use Refine Frowards,
otherwise use Refine Backwards :)
This is an alternative to using camera to scale the
scene and it's expected to be better solution because
scaling camera leads to issues with z-buffer.
Found the whole scaling thing a bit confusing,
especially for object tracking, but cleaning this up
is a bit different topic.
Displays such information as current frame dimension,
frame number within image sequence/movie and in case
of image sequence input displays current file name of
a frame.
Not entirely happy with such approach, but was requested
a lot by artists.
Made it an operator instead of automatic prefetching.
Filling the whole memory with frames is not always
desired behavior.
Now prefetching is available via P-key, or from Clip
panel in toolbox or from Clip menu.
Also enabled prefetching for non-proxied movies.
Several major things are done in this commit:
- First of all, logic of modal solver was changed.
We do not rely on only minimizer to take care of
guessing rotation for frame, but we're using
analytical rotation computation for point clouds
to obtain initial rotation.
Then this rotation is being refined using Ceres
minimizer and now instead of minimizing average
distance between points of point of two clouds,
minimization of reprojection error of point
cloud onto frame happens.
This gives quite a bit of precision improvement.
- Second bigger improvement here is using bundle
adjustment for a result of first step when we're
only estimating rotation between neighbor images
and reprojecting markers.
This averages error across the image sequence
avoiding error accumulation. Also, this will
tweak bundles themselves a bit for better match.
- And last bigger improvement here is support of
camera intrinsics refirenment.
This allowed to significantly improve solution
for real-life footage and results after such
refining are much more usable than it were before.
Thanks to Keir for the help and code review.
Systematically adding some custom id to template_list using default UI_UL_list class, this one is commoly used more than once in an area, yielding collision issues if they do not have a custom id...
(Did not add those when I created that module, because I did not thought we would actually need them in usual UI code, but turned out I was wrong).
Also made some optimizations in those py gettext funcs, when i18n is disabled at build time, no need to do pyobject -> cstring -> pyobject conversions!.
It introduces a new (py-extendable and registrable) RNA type, UIList (roughly similar to Panel one), which currently contains only "standard" list's scroll pos and size (but may be expended to include e.g. some filtering data, etc.). This now makes lists completely independent from Panels!
This UIList has a draw_item callback which allows to customize items' drawing from python, that all addons can now use. Incidentally, this also greatly simplifies the C code of this widget, as we do not code any "special case" here anymore!
To make all this work, other changes were also necessary:
* Now all buttons (uiBut struct) have a 'custom_data' void pointer, used currently to store the uiList struct associated with a given uiLayoutListBox.
* DynamicPaintSurface now exposes a new bool, use_color_preview (readonly), saying whether that surface has some 3D view preview data or not.
* UILayout class has now four new (static) functions, to get the actual icon of any RNA object (important e.g. with materials or textures), and to get an enum item's UI name, description and icon.
* UILayout's label() func now takes an optional 'icon_value' integer parameter, which if not zero will override the 'icon' one (mandatory to use "custom" icons as generated for material/texture/... previews).
Note: not sure whether we should add that one to all UILayout's prop funcs?
Note: will update addons using template list asap.
This fixes some "regressions" introduced in rev50781 which lead to much
worse solution in some cases. Now it's possible to bring old behavior back.
Perhaps it's more like temporal solution for time being smarter solution is
found. But finding such a solution isn't so fast, so let's bring manual
control over reprojection usage.
But anyway, imo it's now nice to have a structure which could be used to
pass different settings to the solver.
- Fix for copy default settings from active track operator
- Add meaningful tracking presets
API changes:
- Added parameter exact to Marker.find_frame, so now it's
possible to get estimated marker
- Added Marker.pattern_bound_box to get pattern's bound box
* Code cleanup, removed unneeded code.
* Style cleanup, don't break lines to early
(unless marked as pep8-80 or pep8-120 compliant)
* Keep 1 line after layout declaration empty.
