- Fixed incorrect memory access on distoritons more than 128 pixels
- Do not use UNDO operators flags for delete proxy operator (files can't be restored form disk),
and also do not use UNDO for set as background operator (background images are storing in
3d viewport which isn't getting re-loaded on undo which can lead to incorrect users count
of movie clip user).
- Add support for refining the camera's intrinsic parameters
during a solve. Currently, refining supports only the following
combinations of intrinsic parameters:
f
f, cx, cy
f, cx, cy, k1, k2
f, k1
f, k1, k2
This is not the same as autocalibration, since the user must
still make a reasonable initial guess about the focal length and
other parameters, whereas true autocalibration would eliminate
the need for the user specify intrinsic parameters at all.
However, the solver works well with only rough guesses for the
focal length, so perhaps full autocalibation is not that
important.
Adding support for the last two combinations, (f, k1) and (f,
k1, k2) required changes to the library libmv depends on for
bundle adjustment, SSBA. These changes should get ported
upstream not just to libmv but to SSBA as well.
- Improved the region of convergence for bundle adjustment by
increasing the number of Levenberg-Marquardt iterations from 50
to 500. This way, the solver is able to crawl out of the bad
local minima it gets stuck in when changing from, for example,
bundling k1 and k2 to just k1 and resetting k2 to 0.
- Add several new region tracker implementations. A region tracker
is a libmv concept, which refers to tracking a template image
pattern through frames. The impact to end users is that tracking
should "just work better". I am reserving a more detailed
writeup, and maybe a paper, for later.
- Other libmv tweaks, such as detecting that a tracker is headed
outside of the image bounds.
This includes several changes made directly to the libmv extern
code rather expecting to get those changes through normal libmv
channels, because I, the libmv BDFL, decided it was faster to work
on libmv directly in Blender, then later reverse-port the libmv
changes from Blender back into libmv trunk. The interesting part
is that I added a full Levenberg-Marquardt loop to the region
tracking code, which should lead to a more stable solutions. I
also added a hacky implementation of "Efficient Second-Order
Minimization" for tracking, which works nicely. A more detailed
quantitative evaluation will follow.
Original patch by Keir, cleaned a bit by myself.
- Replace set of booleans with menu, so now you'll simply be unable to choose
unsupported refine combination
- Some internal code cleanup and minor refactor
- Add support for refining the camera's intrinsic parameters
during a solve. Currently, refining supports only the following
combinations of intrinsic parameters:
f
f, cx, cy
f, cx, cy, k1, k2
f, k1
f, k1, k2
This is not the same as autocalibration, since the user must
still make a reasonable initial guess about the focal length and
other parameters, whereas true autocalibration would eliminate
the need for the user specify intrinsic parameters at all.
However, the solver works well with only rough guesses for the
focal length, so perhaps full autocalibation is not that
important.
Adding support for the last two combinations, (f, k1) and (f,
k1, k2) required changes to the library libmv depends on for
bundle adjustment, SSBA. These changes should get ported
upstream not just to libmv but to SSBA as well.
- Improved the region of convergence for bundle adjustment by
increasing the number of Levenberg-Marquardt iterations from 50
to 500. This way, the solver is able to crawl out of the bad
local minima it gets stuck in when changing from, for example,
bundling k1 and k2 to just k1 and resetting k2 to 0.
- Add several new region tracker implementations. A region tracker
is a libmv concept, which refers to tracking a template image
pattern through frames. The impact to end users is that tracking
should "just work better". I am reserving a more detailed
writeup, and maybe a paper, for later.
- Other libmv tweaks, such as detecting that a tracker is headed
outside of the image bounds.
This includes several changes made directly to the libmv extern
code rather expecting to get those changes through normal libmv
channels, because I, the libmv BDFL, decided it was faster to work
on libmv directly in Blender, then later reverse-port the libmv
changes from Blender back into libmv trunk. The interesting part
is that I added a full Levenberg-Marquardt loop to the region
tracking code, which should lead to a more stable solutions. I
also added a hacky implementation of "Efficient Second-Order
Minimization" for tracking, which works nicely. A more detailed
quantitative evaluation will follow.
===========================
Commiting camera tracking integration gsoc project into trunk.
This commit includes:
- Bundled version of libmv library (with some changes against official repo,
re-sync with libmv repo a bit later)
- New datatype ID called MovieClip which is optimized to work with movie
clips (both of movie files and image sequences) and doing camera/motion
tracking operations.
- New editor called Clip Editor which is currently used for motion/tracking
stuff only, but which can be easily extended to work with masks too.
This editor supports:
* Loading movie files/image sequences
* Build proxies with different size for loaded movie clip, also supports
building undistorted proxies to increase speed of playback in
undistorted mode.
* Manual lens distortion mode calibration using grid and grease pencil
* Supervised 2D tracking using two different algorithms KLT and SAD.
* Basic algorithm for feature detection
* Camera motion solving. scene orientation
- New constraints to "link" scene objects with solved motions from clip:
* Follow Track (make object follow 2D motion of track with given name
or parent object to reconstructed 3D position of track)
* Camera Solver to make camera moving in the same way as reconstructed camera
This commit NOT includes changes from tomato branch:
- New nodes (they'll be commited as separated patch)
- Automatic image offset guessing for image input node and image editor
(need to do more tests and gather more feedback)
- Code cleanup in libmv-capi. It's not so critical cleanup, just increasing
readability and understanadability of code. Better to make this chaneg when
Keir will finish his current patch.
More details about this project can be found on this page:
http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011
Further development of small features would be done in trunk, bigger/experimental
features would first be implemented in tomato branch.
- Remove the overly-conservative use of libmv's re-track tracker. The re-track
tracker would take a normal tracker such as TRKLT or KLT or pyramid KLT, and
track from frame 1 to 2, then back from the position found in 2 back to 1.
Then, when the reverse-track doesn't match the original track with high
precision, the track is considered "failed". This is a good approach for
fully automatic reconstruction, but is too conservative for supervised
tracking.
The retrack-tracker will return when fully automatic tracking is added.
- Always solve for (dx, dy) in the TRKLT loop even if the linear system is
ill-conditioned. The client (Blender in this case) can still use the solved
position, even though it is less reliable.
- Expose the pyramid level setting to the tracking UI when in KLT tracking
mode. While it was tempting to hide this detail from the user, in reality it
does more harm than good, since the way tracking fails depends on the pyramid
level. For now, exposing the pyramid details is a good compromise.
- Move the settings for the tracking algorithm, including search window,
tracking algorithm, pyramid levels, etc, into the track object instead of a
global movie clip object.
===========================
DeBalie commit (main changes only):
- Code cleanup
- Selectable tracking graph curves and curve knots
- Graph curve and curve knots can be deleted now
- Fixed wrong write when linking libraries
(error is caused by hardcoded cound of linkable ID blocks)
- Fixed for KLT tracker when using keyframe adjustment
===========================
Initial implementation of graph view for movie tracking data.
Used the same UI-side approach as preview region for sequencer:
create region for graph-related information inside clip editor.
It's easier and nicer than trying to hack graph editor which is
currently designed to work with AnimData only. Trying to make it
more abstract to deal with any kind of data doesn't seem be real
benefit for now.
Currently supported displaying per-frame average error and
selected tracks' movement speed (pixels per frame).
Additional changes:
- Collect per-frame average error after solving.
- Split space clip drawing code into different files.
- Added per-frame average solving error.
===========================
Some improvements for feature detectors:
- Sort features by score when filtering features by
distance using FAST library.
- Added option to place markers only in areas outlined
by grease pencil.
===========================
Added option to choose which method to use for feature detector: FAST or Moravec
Moravec was supposed to be more "stable", but it collected worse feedback.
before returning back to FAST detector added option to easier toggle between
different detector to compare it's result and find out if we can leave only FAST.
===========================
Support for (un)distortion of images with overscan. Needed for
easier keeping this up-to-date until proper overscan support is
implemented in renderer. Not exposed into UI yet at all.
===========================
Fixed threading issues used to happen when compositor tree
and rendering happened in the same time. Problem was caused
by cached camera intrinsics stored in MovieTracking structure
and access to which wasn't thread-safe.
Now each movie distoriton mode stores it's own camera intrinsics
in it's storage which makes compositor (un)distoriton be totally
thread-safe and this also makes compositor tree faster when
(un)distortion happens on images with different resolution.
===========================
- Merge Movie Distort and Movie Undistort nodes into one node
called Movie Distortion where action (distort/undistort is specified).
- Implemented more proper distortion/undistortion for scaled images.
NOTE: Please, delete distortion nodes with current blender first
before opening files with blender versions from this commit/
===========================
Lookup grids used for image distortion/undistortion are now
caching and re-building if distortion settings are changing.
This speedups compositor node for cases distortion model isn't
changing, but first run and runs after changing distortion
coefficients is still slow.
===========================
- If render size is set to "Full Render" and undistorted render
is enabled, do on-fly undistortion so distortion coefficients
can be visually verifed before baking.
- Made slight change in libmv-side to survive with large distortion
coefficients. Hope this change is fine.
===========================
- Do not disable track when tracking frame-by-frame and tracking
threshold became bad.
- Show anchored image in track preview widget.
- Do not show search area for non-selected and disabled markers.
- Keep anchor constant position when sliding offset parameters in panel.
- Re-enabled occasionally disabled openmp for tracking.
- Renamed clearing operator so now buttons are more clear.
- Updated eigen to the very recent version.
NOTE: I had to enable static aligning again due to it gave crashes
when tracking on my new laptop. I'm not sure it'll work fine
on another systems.
===========================
Bundle new libmv to fix crash caused by some errors in detector.
This commit makes SAD tracking much slower because now it supports
afgine tracking. Not implemented in Blender yet to keep commits
more clear.
===========================
Stop SAD tracker when error becomes too high.
Maximal allowed error value is controlled in Tracking Settings panel.
I haven't been able to find value which will work for most of cases
so it's now quite stupid value of 0. Currently values of 2-6 gives
quite nice result, but it depends on footage.
Upcoming changes from libmv side related on returning normalized SAD
would help here. Until this, please set max_sad manually.
===========================
- Bundling new libmv, issues with undefined uint and C99 functions should gone.
- Changes to Detect operator to match new API.
===========================
- Bundle new version of libmv.
- Use image from nearest track's keyframe as reference for tracking.
This should reduce drifting.
- Added option to switch to new SAD tracker.
This tracker doesn't support pattern with variable size (yet?)
and always uses pattern 16x16px.
Tracker could be choosed in Tracking Settings panel.
===========================
- Distortion/undistortion were accidentally swapped in both of grease
pencil calibration and grid.
This was fixed but distortion coefficients should be re-adjusted.
There are some issues with high distortion coefficients but it
came from libmv and until it'll be fixed there nothing really
can be done here.
- Fixed displayed aspect ratio for muted footage when pixel/display
aspect ratio is differ from 1.0 or proxy size is not Full Render,
- Added c-api for image distortion/undistortion, Not used yet.
===========================
Bundling new version of libmv. It's repo is ready for
bundling again, some patches which were applied on our
side are in libmv repo now.
This new version of libmv also contains patch from John Carpenter
which makes reconstruciton a bit more "stable" -- some kind
of fallback algorithm.
===========================
- Added function to calculate per-track average error.
Made this in libmv c-api, probably should be ported
to libmv pipeline.
- MovieTrackingTrack now holds average reprojection error
which is displayed in Active Track panel and information
text in clip editor (if displaying of names are enabled).
- Make "Clean Tracks" take average error on account.
