Commit Graph

23 Commits

Author SHA1 Message Date
Sergey Sharybin
ed2ddc9f70 Support multiple distortion models, including a new division model
This commit makes it so CameraIntrinsics is no longer hardcoded
to use the traditional polynomial radial distortion model. Currently
the distortion code has generic logic which is shared between
different distortion models, but had no other models until now.

This moves everything specific to the polynomial radial distortion
to a subclass PolynomialDistortionCameraIntrinsics(), and adds a
new division distortion model suitable for cameras such as the
GoPro which have much stronger distortion due to their fisheye lens.

This also cleans up the internal API of CameraIntrinsics to make
it easier to understand and reduces old C-style code.

New distortion model is available in the Lens panel of MCE.

- Polynomial is the old well-known model
- Division is the new one which s intended to deal better with huge
  distortion.

Coefficients of this model works independent from each other
and for division model one probably want to have positive values
to have a barrel distortion.
2014-04-17 17:28:41 +06:00
Sergey Sharybin
df63e8fd93 Speedup track preview widget for byte images
This gives a huge speedup gain for cases when you've got
rather huge markers on a byte images.

Done by skipping IMB_float_from_rect()/IMB_rect_from_float()
for such cases. We can sample the buffers without color space
conversion.
2014-04-10 21:14:36 +06:00
Sergey Sharybin
dbf653acb7 Disable eager-refinement step of region tracking
This gives much worse results on mango footage (see 04_2e)
so disabling for now for until proper prediction model is landed.

The thing is, currently blender sends input coordinates as the guess to
region tracker and in case of fast motion such an early out ruins the track.
2014-02-25 16:55:24 +06:00
Sergey Sharybin
be6643b5ec Followup to the previous commit
Need to take weight into account when drawing per-frame track
reprojection curve and when computing per-track average error.
2014-01-28 18:02:22 +06:00
Sergey Sharybin
9178dc9d38 Rework detector API and implement Harris detector
Switch the detector API to a single function which accepts
a float image and detector options. This makes usage of
feature detection more unified across different algorithms.

Options structure is pretty much straightforward and contains
detector to be used and all the detector-specific settings.

Also implemented Harris feature detection algorithm which
is not as fast as FAST one but is expected to detect more
robust feature points. It is also likely that less features
are detected, but better quality than quantity.

Blender will now use Harris detector by default, later we'll
remove FAST detector.
2014-01-28 15:37:19 +06:00
Sergey Sharybin
5933b2455c Update Ceres to the latest upstream
Summary:
This brings up much easier termination type usage,
which for us means we might use:

  ceres::Summary::IsSolutionUsable()

instead of doing manual funky enum values check.

Reviewers: keir

Differential Revision: https://developer.blender.org/D153
2014-01-01 22:32:47 +06:00
Sergey Sharybin
1de23f6c0d Tweaks and improvements to fundamental/homography estimation
- A bit more reasonable name for the estimation option
  structure and estimation functions.

- Get rid of unclear function and parameter tolerance,
  it wasn't clear at all in which units they are.

  Now we've got expected_average_symmetric_distance as
  an early output check and as soon as average symmetric
  error goes below this threshold refining finishes.

  This distance is measured in the same units as input
  points are.

  It is arguable whether we need callback for this or not,
  but seems Ceres doesn't have some kind of absolute
  threshold for function value and function_tolerance
  behaves different from logic behind expected symmetric
  error.

- Added option to normalize correspondences before
  estimating the homography in order to increase estimation
  stability. See

    R. Hartley and A. Zisserman. Multiple View Geometry in Computer
    Vision. Cambridge University Press, second edition, 2003.

    https://www.cs.ubc.ca/grads/resources/thesis/May09/Dubrofsky_Elan.pdf
2013-11-20 15:30:28 +06:00
Keir Mierle
427844c28d Eagerly attempt to refine a track before doing a brute search
Before the refinement phase of tracking, a brute force SAD search
is run across the search area. This works well but is slow;
especially if the guess for the track's location is accurate.

This patch runs a refinement phase before running a brute force
search, hoping that the guessed position (in x2, y2) is close to
the best answer. If it is, then no brute search is done. If it is
not, then a normal brute force search followed by refinement is
done.

In some cases this may produce worse tracks than before; the
regressions will need investigation. The predictive motion model
(to be implemented) will reduce the probability of that happening.
2013-10-29 01:06:50 +00:00
Sergey Sharybin
9f32e83175 Weighted tracks
Added a weight slider to track which defines
how much particular track affects in a final
reconstruction. This weight is for sure
animateable.

Currently it affects on BA step only which in
most cases will work just fine.

The usecase of this slider is to have it set
to 1.0 most of the time where the track is
good, but blend it's weight down to 0 when
tracker looses the track. This will prevent
camera from jump.

Tutorial is to be done by Sebastian.
2013-10-26 13:22:38 +00:00
Sergey Sharybin
eb69cb7de3 Get rid of Allow Fallback option
It was rather confusing from the user usage point
of view and didn't get so much improvement after
new bundle adjuster was added.

In the future we might want to switch resection
to PPnP algorithm, which could also might be a
nice alternative to fallback option.
2013-10-15 15:21:41 +00:00
Sergey Sharybin
b5491291e4 Libmv: move platform-specific defines into own file 2013-10-09 20:02:02 +00:00
Sergey Sharybin
ccd2e4375a Fix compilation error after recent libmv change
- Tweaked typedefs in stdint so they match
  what we've got in BLI_sys_types (needed to
  explicitly tell sign to MSVC).

  Not so much harmful to be more explicit here,
  but we really better to have single stdint
  int blender.

- Tweaked allocations macros so MSVC is happy
  with structures allocation.
2013-10-09 19:49:09 +00:00
Sergey Sharybin
49bc310671 Move guarded objetc allocation to a guardedalloc header
Also made libmv-capi use guarded objetc allocation.
Run into some suspecious cases when it was not so
clear whether memory is being freed or not.

Now we'll know for sure whether there're leaks or not :)

Having this macros in a guardedalloc header helps
using them in other areas (for now it's OCIO and libmv,
but in the future it'll be more places).
2013-10-09 08:46:02 +00:00
Sergey Sharybin
2ddbb5d1e1 Fix for plane track jittering
Jittering was caused by homography not being estimated
accurate enough.

Before this, only algebraic estimation was used, which
is indeed not so much great, Now use algebraic estimation
followed with refinement step using Ceres minimizer.

The code was already there since keyframe selection patch,
made such estimation a generic function in multiview/ and
changed API for estimation in order to pass all additional
options via an options structure (the same way as it's
done fr Ceres).

This includes changes to both homography and fundamental
estimation.

TODO:
- Need to document Ceres functors better.
- Need to support homogeneous coordinates (currently
  only euclidean coords are supported).
2013-09-30 09:35:04 +00:00
Joseph Mansfield
9707bdfe75 Code cleanup: Minor function capitalization fix 2013-09-09 00:06:23 +00:00
Sergey Sharybin
24ce60cfe4 Merge plane track feature from tomato branch
This commit includes all the changes made for plane tracker
in tomato branch.

Movie clip editor changes:

- Artist might create a plane track out of multiple point
  tracks which belongs to the same track (minimum amount of
  point tracks is 4, maximum is not actually limited).

  When new plane track is added, it's getting "tracked"
  across all point tracks, which makes it stick to the same
  plane point tracks belong to.

- After plane track was added, it need to be manually adjusted
  in a way it covers feature one might to mask/replace.

  General transform tools (G, R, S) or sliding corners with
  a mouse could be sued for this. Plane corner which
  corresponds to left bottom image corner has got X/Y axis
  on it (red is for X axis, green for Y).

- Re-adjusting plane corners makes plane to be "re-tracked"
  for the frames sequence between current frame and next
  and previous keyframes.

- Kayframes might be removed from the plane, using Shit-X
  (Marker Delete) operator. However, currently manual
  re-adjustment or "re-track" trigger is needed.

Compositor changes:

- Added new node called Plane Track Deform.

- User selects which plane track to use (for this he need
  to select movie clip datablock, object and track names).

- Node gets an image input, which need to be warped into
  the plane.

- Node outputs:
  * Input image warped into the plane.
  * Plane, rasterized to a mask.

Masking changes:

- Mask points might be parented to a plane track, which
  makes this point deforming in a way as if it belongs
  to the tracked plane.

Some video tutorials are available:
- Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4
- Artist video: https://vimeo.com/71727578

This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
Sergey Sharybin
7eeaefb97c Potential fix for wrogn k3 value after solve 2013-08-01 16:34:55 +00:00
Joseph Mansfield
c15ae082bb Code cleanup: libmv C API
Clean up inconsistencies in the libmv C API:
- All type identifiers are libmv_TypeName
- All function identifiers libmv_functionName
- Prefer libmv_nounVerb function names (e.g. libmv_featuresDestroy)
- Match Blender code formatting rather than Google
- Spelling corrections

Code review: https://codereview.appspot.com/11494044/
2013-07-31 13:48:12 +00:00
Sergey Sharybin
cf5e979fb4 Motion tracking: automatic keyframe selection
Implements an automatic keyframe selection algorithm which uses
couple of approaches to find out best keyframes candidates:

- First, slightly modifier Pollefeys's criteria is used, which
  limits correspondence ration from 80% to 100%. This allows to
  reject keyframe candidate early without doing heavy math in
  cases there're not much common features with first keyframe.

- Second step is based on Geometric Robust Information Criteria
  (aka GRIC), which checks whether features motion between
  candidate keyframes is better defined by homography or
  fundamental matrices.

  To be a good keyframe candidate, fundamental matrix need to
  define motion better than homography (in this case F-GRIC will
  be smaller than H-GRIC).

  This two criteria are well described in this paper:
  http://www.cs.ait.ac.th/~mdailey/papers/Tahir-KeyFrame.pdf

- Final step is based on estimating reconstruction error of
  a full-scene solution using candidate keyframes. This part
  is based on the following paper:

  ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf

  This step requires reconstruction using candidate keyframes
  and obtaining covariance matrix of 3D points positions.
  Reconstruction was done pretty much straightforward using
  other simple pipeline routines, and for covariance estimation
  pseudo-inverse of Hessian is used, which is in this case
  (J^T * J)+, where + denotes pseudo-inverse.

  Jacobian matrix is estimating using Ceres evaluate API.

  This is also crucial to get rid of possible gauge ambiguity,
  which is in our case made by zero-ing 7 (by gauge freedoms
  number) eigen values in pseudo-inverse.

  There're still room for improving and optimizing the code,
  but we need some point to start with anyway :)

  Thanks to Keir Mierle and Sameer Agarwal who assisted a lot
  to make this feature working.
2013-05-30 09:03:49 +00:00
Sergey Sharybin
d4c6ac9a60 Cleanup and small improvements to libmv
- Added const modifiers where it makes sense and
  helps keep code safe.
- Reshuffled argument to match <inputs>,<outputs>
  convention on parameters.
- Pass values to ApplyRadialDistortionCameraIntrinsics
  by a constant reference.
  This will save lots of CPU ticks passing relatively
  heavy jet objects to this function when running
  bundle adjustment.
2013-05-13 14:39:06 +00:00
Sergey Sharybin
4e5541a9a6 Headers cleanup. 2013-05-13 09:51:36 +00:00
Sergey Sharybin
91b659d174 Forgot this in one of previous commits
Idea here is to be able to have all files opened in IDE
and not confuse it by the same function implemented in
multiple files.
2013-05-12 22:40:12 +00:00
Sergey Sharybin
32faec1a53 Remplace bunch of annoying ifdefs in tracking.c with a libmv-capi_stub.cc
Makes code in tracking.cc much easier to understand and modify,
without worring to breck compulation with Libmv disabled.

It is still possible compilation will break due to libmv-capi
changes, but that's not happening so much often.
2013-05-12 22:17:37 +00:00