sandey/blender
1
0
Fork 0
forked from bartvdbraak/blender
Code Pull Requests Activity
d2b55d1171
blender/extern/libmv/libmv-capi.h

174 lines
7.2 KiB
C
Raw Normal View History

Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
/*
* ***** BEGIN GPL LICENSE BLOCK *****
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software Foundation,
* Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* The Original Code is Copyright (C) 2011 Blender Foundation.
* All rights reserved.
*
* Contributor(s): Blender Foundation,
* Sergey Sharybin
*
* ***** END GPL LICENSE BLOCK *****
*/
#ifndef LIBMV_C_API_H
#define LIBMV_C_API_H
#ifdef __cplusplus
extern "C" {
#endif
struct libmv_Tracks;
struct libmv_Reconstruction;
struct libmv_Features;
struct libmv_CameraIntrinsics;
/* Logging */
void libmv_initLogging(const char *argv0);
void libmv_startDebugLogging(void);
void libmv_setLoggingVerbosity(int verbosity);
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
/* TrackRegion (new planar tracker) */
struct libmv_trackRegionOptions {
Masking support for motion tracks Added option to use Grease Pencil datablock as a mask for pattern when doing motion tracking. Option could be found in Tracking Settings panel. All strokes would be rasterized separately from each other and every stroke is treating as a closed spline. Also added option to apply a mask on track preview which is situated just after B/B/W channel button under track preview.
2012-06-12 11:13:53 +00:00
int motion_model;
int num_iterations;
int use_brute;
int use_normalization;
double minimum_correlation;
double sigma;
float *image1_mask;
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
};
Masking support for motion tracks Added option to use Grease Pencil datablock as a mask for pattern when doing motion tracking. Option could be found in Tracking Settings panel. All strokes would be rasterized separately from each other and every stroke is treating as a closed spline. Also added option to apply a mask on track preview which is situated just after B/B/W channel button under track preview.
2012-06-12 11:13:53 +00:00
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
struct libmv_trackRegionResult {
Masking support for motion tracks Added option to use Grease Pencil datablock as a mask for pattern when doing motion tracking. Option could be found in Tracking Settings panel. All strokes would be rasterized separately from each other and every stroke is treating as a closed spline. Also added option to apply a mask on track preview which is situated just after B/B/W channel button under track preview.
2012-06-12 11:13:53 +00:00
int termination;
const char *termination_reason;
double correlation;
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
};
Masking support for motion tracks Added option to use Grease Pencil datablock as a mask for pattern when doing motion tracking. Option could be found in Tracking Settings panel. All strokes would be rasterized separately from each other and every stroke is treating as a closed spline. Also added option to apply a mask on track preview which is situated just after B/B/W channel button under track preview.
2012-06-12 11:13:53 +00:00
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
int libmv_trackRegion(const struct libmv_trackRegionOptions *options,
Scale search area when doing planar tracking Helps keeping features tracked when there's large scale happens without need to manually re-adjust search area. Currently using factor of pattern's boundbox scale, but probably could be done in more accurate way?
2012-06-11 11:40:54 +00:00
const float *image1, int image1_width, int image1_height,
const float *image2, int image2_width, int image2_height,
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
const double *x1, const double *y1,
struct libmv_trackRegionResult *result,
double *x2, double *y2);
void libmv_samplePlanarPatch(const float *image, int width, int height,
int channels, const double *xs, const double *ys,
Masking support for motion tracks Added option to use Grease Pencil datablock as a mask for pattern when doing motion tracking. Option could be found in Tracking Settings panel. All strokes would be rasterized separately from each other and every stroke is treating as a closed spline. Also added option to apply a mask on track preview which is situated just after B/B/W channel button under track preview.
2012-06-12 11:13:53 +00:00
int num_samples_x, int num_samples_y,
const float *mask, float *patch,
Planar tracking support for motion tracking =========================================== Major list of changes done in tomato branch: - Add a planar tracking implementation to libmv This adds a new planar tracking implementation to libmv. The tracker is based on Ceres[1], the new nonlinear minimizer that myself and Sameer released from Google as open source. Since the motion model is more involved, the interface is different than the RegionTracker interface used previously in Blender. The start of a C API in libmv-capi.{cpp,h} is also included. - Migrate from pat_{min,max} for markers to 4 corners representation Convert markers in the movie clip editor / 2D tracker from using pat_min and pat_max notation to using the a more general, 4-corner representation. There is still considerable porting work to do; in particular sliding from preview widget does not work correct for rotated markers. All other areas should be ported to new representation: * Added support of sliding individual corners. LMB slide + Ctrl would scale the whole pattern * S would scale the whole marker, S-S would scale pattern only * Added support of marker's rotation which is currently rotates only patterns around their centers or all markers around median, Rotation or other non-translation/scaling transformation of search area doesn't make sense. * Track Preview widget would display transformed pattern which libmv actually operates with. - "Efficient Second-order Minimization" for the planar tracker This implements the "Efficient Second-order Minimization" scheme, as supported by the existing translation tracker. This increases the amount of per-iteration work, but decreases the number of iterations required to converge and also increases the size of the basin of attraction for the optimization. - Remove the use of the legacy RegionTracker API from Blender, and replaces it with the new TrackRegion API. This also adds several features to the planar tracker in libmv: * Do a brute-force initialization of tracking similar to "Hybrid" mode in the stable release, but using all floats. This is slower but more accurate. It is still necessary to evaluate if the performance loss is worth it. In particular, this change is necessary to support high bit depth imagery. * Add support for masks over the search window. This is a step towards supporting user-defined tracker masks. The tracker masks will make it easy for users to make a mask for e.g. a ball. Not exposed into interface yet/ * Add Pearson product moment correlation coefficient checking (aka "Correlation" in the UI. This causes tracking failure if the tracked patch is not linearly related to the template. * Add support for warping a few points in addition to the supplied points. This is useful because the tracking code deliberately does not expose the underlying warp representation. Instead, warps are specified in an aparametric way via the correspondences. - Replace the old style tracker configuration panel with the new planar tracking panel. From a users perspective, this means: * The old "tracking algorithm" picker is gone. There is only 1 algorithm now. We may revisit this later, but I would much prefer to have only 1 algorithm. So far no optimization work has been done so the speed is not there yet. * There is now a dropdown to select the motion model. Choices: * Translation * Translation, rotation * Translation, scale * Translation, rotation, scale * Affine * Perspective * The old "Hybrid" mode is gone; instead there is a toggle to enable or disable translation-only tracker initialization. This is the equivalent of the hyrbid mode before, but rewritten to work with the new planar tracking modes. * The pyramid levels setting is gone. At a future date, the planar tracker will decide to use pyramids or not automatically. The pyramid setting was ultimately a mistake; with the brute force initialization it is unnecessary. - Add light-normalized tracking Added the ability to normalize patterns by their average value while tracking, to make them invariant to global illumination changes. Additional details could be found at wiki page [2] [1] http://code.google.com/p/ceres-solver [2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
2012-06-10 15:28:19 +00:00
double *warped_position_x, double *warped_position_y);
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
/* Tracks */
struct libmv_Tracks *libmv_tracksNew(void);
void libmv_tracksInsert(struct libmv_Tracks *libmv_tracks, int image, int track, double x, double y);
void libmv_tracksDestroy(struct libmv_Tracks *libmv_tracks);
/* Reconstruction solver */
Camera tracking: moved camera solver into it's own job In some cases solving can take a while (especially when refining is used) and keeping interface locked is a bit annoying. Now camera solver is moved to job system and interface isn't locking. Reporting progress isn't really accurate, but trying to make it more linear can lead to spending more effort on it than having benefit. Also, changing status in the information line helps to understand that blender isn't hang up and solving is till working nicely. Main changes in code: - libmv_solveReconstruction now accepts additional parameters: * progress_update_callback - a function which is getting called from solver algorithm to report progress back to Blender. * callback_customdata - a user-defined context which is passing to progress_update_callback so progress can be updated in needed blender-side data structures. This parameters are optional. - Added structure MovieTrackingStats which is placed in MovieTracking structure. It's supposed to be used for displaying information about different operations (currently it's only camera solver, but can be easily used for something else in the future) in clip editor. This statistics structure is getting allocated for time operator is working and not saving into .blend file. - Clip Editor now displays statistics stored in MovieTrackingStats structure like it's done for rendering.
2011-11-28 13:49:42 +00:00
code cleanup: camera tracking - Moved keyframes and refirement flags into reconstruction options structure - Moved distortion coefficients and other camera intrinsics into own structure - Cleaned up reconstruction functions in libmv c-api
2012-12-10 16:38:13 +00:00
#define LIBMV_REFINE_FOCAL_LENGTH (1 << 0)
#define LIBMV_REFINE_PRINCIPAL_POINT (1 << 1)
#define LIBMV_REFINE_RADIAL_DISTORTION_K1 (1 << 2)
#define LIBMV_REFINE_RADIAL_DISTORTION_K2 (1 << 4)
typedef struct libmv_cameraIntrinsicsOptions {
double focal_length;
double principal_point_x, principal_point_y;
double k1, k2, k3;
double p1, p2;
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
int image_width, image_height;
code cleanup: camera tracking - Moved keyframes and refirement flags into reconstruction options structure - Moved distortion coefficients and other camera intrinsics into own structure - Cleaned up reconstruction functions in libmv c-api
2012-12-10 16:38:13 +00:00
} libmv_cameraIntrinsicsOptions;
typedef struct libmv_reconstructionOptions {
int keyframe1, keyframe2;
int refine_intrinsics;
Merging r50625 through r51896 from trunk into soc-2011-tomato Merging just in case we'll want to develop some experimental stuff
2012-11-05 19:42:27 +00:00
double success_threshold;
int use_fallback_reconstruction;
code cleanup: camera tracking - Moved keyframes and refirement flags into reconstruction options structure - Moved distortion coefficients and other camera intrinsics into own structure - Cleaned up reconstruction functions in libmv c-api
2012-12-10 16:38:13 +00:00
} libmv_reconstructionOptions;
Merging r50625 through r51896 from trunk into soc-2011-tomato Merging just in case we'll want to develop some experimental stuff
2012-11-05 19:42:27 +00:00
Camera tracking: moved camera solver into it's own job In some cases solving can take a while (especially when refining is used) and keeping interface locked is a bit annoying. Now camera solver is moved to job system and interface isn't locking. Reporting progress isn't really accurate, but trying to make it more linear can lead to spending more effort on it than having benefit. Also, changing status in the information line helps to understand that blender isn't hang up and solving is till working nicely. Main changes in code: - libmv_solveReconstruction now accepts additional parameters: * progress_update_callback - a function which is getting called from solver algorithm to report progress back to Blender. * callback_customdata - a user-defined context which is passing to progress_update_callback so progress can be updated in needed blender-side data structures. This parameters are optional. - Added structure MovieTrackingStats which is placed in MovieTracking structure. It's supposed to be used for displaying information about different operations (currently it's only camera solver, but can be easily used for something else in the future) in clip editor. This statistics structure is getting allocated for time operator is working and not saving into .blend file. - Clip Editor now displays statistics stored in MovieTrackingStats structure like it's done for rendering.
2011-11-28 13:49:42 +00:00
typedef void (*reconstruct_progress_update_cb) (void *customdata, double progress, const char *message);
Assorted camera tracker improvements - Add support for refining the camera's intrinsic parameters during a solve. Currently, refining supports only the following combinations of intrinsic parameters: f f, cx, cy f, cx, cy, k1, k2 f, k1 f, k1, k2 This is not the same as autocalibration, since the user must still make a reasonable initial guess about the focal length and other parameters, whereas true autocalibration would eliminate the need for the user specify intrinsic parameters at all. However, the solver works well with only rough guesses for the focal length, so perhaps full autocalibation is not that important. Adding support for the last two combinations, (f, k1) and (f, k1, k2) required changes to the library libmv depends on for bundle adjustment, SSBA. These changes should get ported upstream not just to libmv but to SSBA as well. - Improved the region of convergence for bundle adjustment by increasing the number of Levenberg-Marquardt iterations from 50 to 500. This way, the solver is able to crawl out of the bad local minima it gets stuck in when changing from, for example, bundling k1 and k2 to just k1 and resetting k2 to 0. - Add several new region tracker implementations. A region tracker is a libmv concept, which refers to tracking a template image pattern through frames. The impact to end users is that tracking should "just work better". I am reserving a more detailed writeup, and maybe a paper, for later. - Other libmv tweaks, such as detecting that a tracker is headed outside of the image bounds. This includes several changes made directly to the libmv extern code rather expecting to get those changes through normal libmv channels, because I, the libmv BDFL, decided it was faster to work on libmv directly in Blender, then later reverse-port the libmv changes from Blender back into libmv trunk. The interesting part is that I added a full Levenberg-Marquardt loop to the region tracking code, which should lead to a more stable solutions. I also added a hacky implementation of "Efficient Second-Order Minimization" for tracking, which works nicely. A more detailed quantitative evaluation will follow. Original patch by Keir, cleaned a bit by myself.
2011-11-14 06:41:23 +00:00
int libmv_refineParametersAreValid(int parameters);
code cleanup: camera tracking - Moved keyframes and refirement flags into reconstruction options structure - Moved distortion coefficients and other camera intrinsics into own structure - Cleaned up reconstruction functions in libmv c-api
2012-12-10 16:38:13 +00:00
struct libmv_Reconstruction *libmv_solveReconstruction(struct libmv_Tracks *libmv_tracks,
libmv_cameraIntrinsicsOptions *libmv_camera_intrinsics_options,
libmv_reconstructionOptions *libmv_reconstruction_options,
reconstruct_progress_update_cb progress_update_callback,
void *callback_customdata);
struct libmv_Reconstruction *libmv_solveModal(struct libmv_Tracks *libmv_tracks,
libmv_cameraIntrinsicsOptions *libmv_camera_intrinsics_options,
Modal (aka tripod) solver rework Several major things are done in this commit: - First of all, logic of modal solver was changed. We do not rely on only minimizer to take care of guessing rotation for frame, but we're using analytical rotation computation for point clouds to obtain initial rotation. Then this rotation is being refined using Ceres minimizer and now instead of minimizing average distance between points of point of two clouds, minimization of reprojection error of point cloud onto frame happens. This gives quite a bit of precision improvement. - Second bigger improvement here is using bundle adjustment for a result of first step when we're only estimating rotation between neighbor images and reprojecting markers. This averages error across the image sequence avoiding error accumulation. Also, this will tweak bundles themselves a bit for better match. - And last bigger improvement here is support of camera intrinsics refirenment. This allowed to significantly improve solution for real-life footage and results after such refining are much more usable than it were before. Thanks to Keir for the help and code review.
2013-02-28 14:24:42 +00:00
libmv_reconstructionOptions *libmv_reconstruction_options,
code cleanup: camera tracking - Moved keyframes and refirement flags into reconstruction options structure - Moved distortion coefficients and other camera intrinsics into own structure - Cleaned up reconstruction functions in libmv c-api
2012-12-10 16:38:13 +00:00
reconstruct_progress_update_cb progress_update_callback,
Merging r50625 through r51896 from trunk into soc-2011-tomato Merging just in case we'll want to develop some experimental stuff
2012-11-05 19:42:27 +00:00
void *callback_customdata);
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
int libmv_reporojectionPointForTrack(struct libmv_Reconstruction *libmv_reconstruction, int track, double pos[3]);
double libmv_reporojectionErrorForTrack(struct libmv_Reconstruction *libmv_reconstruction, int track);
double libmv_reporojectionErrorForImage(struct libmv_Reconstruction *libmv_reconstruction, int image);
int libmv_reporojectionCameraForImage(struct libmv_Reconstruction *libmv_reconstruction, int image, double mat[4][4]);
double libmv_reprojectionError(struct libmv_Reconstruction *libmv_reconstruction);
void libmv_destroyReconstruction(struct libmv_Reconstruction *libmv_reconstruction);
/* feature detector */
struct libmv_Features *libmv_detectFeaturesFAST(unsigned char *data, int width, int height, int stride,
int margin, int min_trackness, int min_distance);
struct libmv_Features *libmv_detectFeaturesMORAVEC(unsigned char *data, int width, int height, int stride,
int margin, int count, int min_distance);
int libmv_countFeatures(struct libmv_Features *libmv_features);
void libmv_getFeature(struct libmv_Features *libmv_features, int number, double *x, double *y, double *score, double *size);
void libmv_destroyFeatures(struct libmv_Features *libmv_features);
/* camera intrinsics */
Assorted camera tracker improvements - Add support for refining the camera's intrinsic parameters during a solve. Currently, refining supports only the following combinations of intrinsic parameters: f f, cx, cy f, cx, cy, k1, k2 f, k1 f, k1, k2 This is not the same as autocalibration, since the user must still make a reasonable initial guess about the focal length and other parameters, whereas true autocalibration would eliminate the need for the user specify intrinsic parameters at all. However, the solver works well with only rough guesses for the focal length, so perhaps full autocalibation is not that important. Adding support for the last two combinations, (f, k1) and (f, k1, k2) required changes to the library libmv depends on for bundle adjustment, SSBA. These changes should get ported upstream not just to libmv but to SSBA as well. - Improved the region of convergence for bundle adjustment by increasing the number of Levenberg-Marquardt iterations from 50 to 500. This way, the solver is able to crawl out of the bad local minima it gets stuck in when changing from, for example, bundling k1 and k2 to just k1 and resetting k2 to 0. - Add several new region tracker implementations. A region tracker is a libmv concept, which refers to tracking a template image pattern through frames. The impact to end users is that tracking should "just work better". I am reserving a more detailed writeup, and maybe a paper, for later. - Other libmv tweaks, such as detecting that a tracker is headed outside of the image bounds. This includes several changes made directly to the libmv extern code rather expecting to get those changes through normal libmv channels, because I, the libmv BDFL, decided it was faster to work on libmv directly in Blender, then later reverse-port the libmv changes from Blender back into libmv trunk. The interesting part is that I added a full Levenberg-Marquardt loop to the region tracking code, which should lead to a more stable solutions. I also added a hacky implementation of "Efficient Second-Order Minimization" for tracking, which works nicely. A more detailed quantitative evaluation will follow. Original patch by Keir, cleaned a bit by myself.
2011-11-14 06:41:23 +00:00
struct libmv_CameraIntrinsics *libmv_ReconstructionExtractIntrinsics(struct libmv_Reconstruction *libmv_Reconstruction);
Multi-threaded frame calculation for movie clip proxies This commit implements multi-threaded calculation of frames when building proxies. Both scaling and undistortion steps are now threaded. Frames and proxy resolution are still handled one-by-one, saving files after every single step. So if HDD is not so fast, this commit could have not so much benefit. Internal changes: - Added IMB_scaleImBuf_threaded which scales given image buffer in multiple threads and uses bilinear filtering. - libmv's camera intrinsics now have SetThreads() method which is used to specify how many OpenMP threads to use for buffer distortion/undistortion. And yeah, this code is using OpenMP for threading. - Reshuffled a bit libmv-capi calls and added function BKE_tracking_distortion_set_threads to specify number of threads used by intrinscis.
2013-03-15 11:59:46 +00:00
struct libmv_CameraIntrinsics *libmv_CameraIntrinsicsNewEmpty(void);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
struct libmv_CameraIntrinsics *libmv_CameraIntrinsicsNew(libmv_cameraIntrinsicsOptions *libmv_camera_intrinsics_options);
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
struct libmv_CameraIntrinsics *libmv_CameraIntrinsicsCopy(struct libmv_CameraIntrinsics *libmv_intrinsics);
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsDestroy(struct libmv_CameraIntrinsics *libmv_intrinsics);
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsUpdate(struct libmv_CameraIntrinsics *libmv_intrinsics,
libmv_cameraIntrinsicsOptions *libmv_camera_intrinsics_options);
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
Multi-threaded frame calculation for movie clip proxies This commit implements multi-threaded calculation of frames when building proxies. Both scaling and undistortion steps are now threaded. Frames and proxy resolution are still handled one-by-one, saving files after every single step. So if HDD is not so fast, this commit could have not so much benefit. Internal changes: - Added IMB_scaleImBuf_threaded which scales given image buffer in multiple threads and uses bilinear filtering. - libmv's camera intrinsics now have SetThreads() method which is used to specify how many OpenMP threads to use for buffer distortion/undistortion. And yeah, this code is using OpenMP for threading. - Reshuffled a bit libmv-capi calls and added function BKE_tracking_distortion_set_threads to specify number of threads used by intrinscis.
2013-03-15 11:59:46 +00:00
void libmv_CameraIntrinsicsSetThreads(struct libmv_CameraIntrinsics *libmv_intrinsics, int threads);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsExtract(struct libmv_CameraIntrinsics *libmv_intrinsics, double *focal_length,
Assorted camera tracker improvements - Add support for refining the camera's intrinsic parameters during a solve. Currently, refining supports only the following combinations of intrinsic parameters: f f, cx, cy f, cx, cy, k1, k2 f, k1 f, k1, k2 This is not the same as autocalibration, since the user must still make a reasonable initial guess about the focal length and other parameters, whereas true autocalibration would eliminate the need for the user specify intrinsic parameters at all. However, the solver works well with only rough guesses for the focal length, so perhaps full autocalibation is not that important. Adding support for the last two combinations, (f, k1) and (f, k1, k2) required changes to the library libmv depends on for bundle adjustment, SSBA. These changes should get ported upstream not just to libmv but to SSBA as well. - Improved the region of convergence for bundle adjustment by increasing the number of Levenberg-Marquardt iterations from 50 to 500. This way, the solver is able to crawl out of the bad local minima it gets stuck in when changing from, for example, bundling k1 and k2 to just k1 and resetting k2 to 0. - Add several new region tracker implementations. A region tracker is a libmv concept, which refers to tracking a template image pattern through frames. The impact to end users is that tracking should "just work better". I am reserving a more detailed writeup, and maybe a paper, for later. - Other libmv tweaks, such as detecting that a tracker is headed outside of the image bounds. This includes several changes made directly to the libmv extern code rather expecting to get those changes through normal libmv channels, because I, the libmv BDFL, decided it was faster to work on libmv directly in Blender, then later reverse-port the libmv changes from Blender back into libmv trunk. The interesting part is that I added a full Levenberg-Marquardt loop to the region tracking code, which should lead to a more stable solutions. I also added a hacky implementation of "Efficient Second-Order Minimization" for tracking, which works nicely. A more detailed quantitative evaluation will follow. Original patch by Keir, cleaned a bit by myself.
2011-11-14 06:41:23 +00:00
double *principal_x, double *principal_y, double *k1, double *k2, double *k3, int *width, int *height);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsUndistortByte(struct libmv_CameraIntrinsics *libmv_intrinsics,
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
unsigned char *src, unsigned char *dst, int width, int height, float overscan, int channels);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsUndistortFloat(struct libmv_CameraIntrinsics *libmv_intrinsics,
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
float *src, float *dst, int width, int height, float overscan, int channels);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsDistortByte(struct libmv_CameraIntrinsics *libmv_intrinsics,
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
unsigned char *src, unsigned char *dst, int width, int height, float overscan, int channels);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_CameraIntrinsicsDistortFloat(struct libmv_CameraIntrinsics *libmv_intrinsics,
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
float *src, float *dst, int width, int height, float overscan, int channels);
/* utils */
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_applyCameraIntrinsics(libmv_cameraIntrinsicsOptions *libmv_camera_intrinsics_options,
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
double x, double y, double *x1, double *y1);
Camera tracking: libmv distortion API now also uses camera intrinsics structure instead of passing all the parameters to every function. Makes it much easier to tweak distortion model.
2012-12-10 16:38:28 +00:00
void libmv_InvertIntrinsics(libmv_cameraIntrinsicsOptions *libmv_camera_intrinsics_options,
Camera tracking integration =========================== Commiting camera tracking integration gsoc project into trunk. This commit includes: - Bundled version of libmv library (with some changes against official repo, re-sync with libmv repo a bit later) - New datatype ID called MovieClip which is optimized to work with movie clips (both of movie files and image sequences) and doing camera/motion tracking operations. - New editor called Clip Editor which is currently used for motion/tracking stuff only, but which can be easily extended to work with masks too. This editor supports: * Loading movie files/image sequences * Build proxies with different size for loaded movie clip, also supports building undistorted proxies to increase speed of playback in undistorted mode. * Manual lens distortion mode calibration using grid and grease pencil * Supervised 2D tracking using two different algorithms KLT and SAD. * Basic algorithm for feature detection * Camera motion solving. scene orientation - New constraints to "link" scene objects with solved motions from clip: * Follow Track (make object follow 2D motion of track with given name or parent object to reconstructed 3D position of track) * Camera Solver to make camera moving in the same way as reconstructed camera This commit NOT includes changes from tomato branch: - New nodes (they'll be commited as separated patch) - Automatic image offset guessing for image input node and image editor (need to do more tests and gather more feedback) - Code cleanup in libmv-capi. It's not so critical cleanup, just increasing readability and understanadability of code. Better to make this chaneg when Keir will finish his current patch. More details about this project can be found on this page: http://wiki.blender.org/index.php/User:Nazg-gul/GSoC-2011 Further development of small features would be done in trunk, bigger/experimental features would first be implemented in tomato branch.
2011-11-07 12:55:18 +00:00
double x, double y, double *x1, double *y1);
#ifdef __cplusplus
}
#endif
#endif // LIBMV_C_API_H
Copy Permalink