commit da4607f010bca0b3532cd4444afbb10bc774fc32 Author: Sergey Sharybin Date: Tue Jan 28 18:32:39 2014 +0600 Implemented scoped_array and use it in detector scoped_array is pretty much the same as scoped_ptr with the only difference that it'll free memory using delete[] operator. It also gives some additional API functions to access array elements. Currently it only used to manage images denoted as byte arrays in detector. Reviewers: keir Reviewed By: keir Differential Revision: https://developer.blender.org/D266 commit cd7eb3eff2e69ce5e08570ead83ae6d35ee48857 Author: Sergey Sharybin Date: Tue Jan 28 17:23:47 2014 +0600 Improvements to weighted tracks behavior First thing changed by this commit is making it so Euclidean intersection takes track weight into account when solving minimization problem. This behaves the same exact way as it is for BA step. Second thing is related on how average reprojection error is being calculated. It didn't take track weight into account which could confuse users. Now average reprojection error will give the same result as intersection/BA uses during minimization which gives much more predictable behavior. Reviewers: keir Reviewed By: keir CC: sebastian_k Differential Revision: https://developer.blender.org/D265 commit 6559b36dc14369175bfa0830323146acd3426483 Author: Sergey Sharybin Date: Tue Jan 28 16:39:14 2014 +0600 Fixes for keyframe selection Using tracks with constant zero weight used to crash keyframe selection since it was trying to use missing parameter blocks for Jacobian evaluation, Also fixed possible issues with wrong camera block being marked as variable. This could technically happen when having zero weighted tracks. Made it so all camera blocks are marked as variable for now. commit 557d531b061aa69d114e89cbb325c5175389afec Author: Sergey Sharybin Date: Tue Jan 28 16:10:33 2014 +0600 Style cleanup: wrong indentation of wrapped line commit ca15262cf07a873268173965ee1fb84f9729b744 Author: Sergey Sharybin Date: Tue Jan 28 15:21:36 2014 +0600 Rework detector API and implement Harris detector Switch the detector API to a single function which accepts a float image and detector options. This makes usage of feature detection more unified across different algorithms. Options structure is pretty much straightforward and contains detector to be used and all the detector-specific settings. Also implemented Harris feature detection algorithm which is not as fast as FAST one but is expected to detect more robust feature points. Reviewers: keir Reviewed By: keir Differential Revision: https://developer.blender.org/D258 commit 6458915f64fceba108c5279b7320ca8c76e8a742 Author: Sergey Sharybin Date: Fri Jan 24 19:14:18 2014 +0600 Add arcanist configuration file commit 0a69fadadc5aacbd339f839ac5bd12c3571278b1 Author: Sergey Sharybin Date: Thu Jan 9 15:50:11 2014 +0600 Fix compilation error on NetBSD - NetBSD doesn't provide sincos(3) in libm, so don't try to use it - Use posix_memalign on NetBSD Original patch is by Jeorg Sonnenberger to Blender patch tracker, thanks! commit b0df3e291e6c85f791658be04334efafc41989f5 Author: Sergey Sharybin Date: Thu Jan 2 15:12:18 2014 +0600 Fix build configuration warnings Those warnings were mainly caused by installation configuration of Ceres. Made some tweaks to make CMake happy for now. But for sure bigger cleanup here is needed. commit b68de6acd20f3ffab92e0cd450198a700cd109ab Author: Sergey Sharybin Date: Thu Jan 2 15:04:05 2014 +0600 Code and style cleanup Mainly fixed some style warnings reported by cpplint. Also changed how camera (un)distortion happens internally by replacing number of channels as a template argument with number as channels passing as function argument. Makes code easier to follow by eliminating loads checks how much channels are used and which argument to pass to the template. commit b9e467e7c077b58199c4110f6967b7c18d1e7bf7 Author: Sergey Sharybin Date: Tue Dec 31 20:34:39 2013 +0600 Update Ceres to the latest upstream This brings up much easier termination type usage, which for us means we might use: ceres::Summary::IsSolutionUsable() instead of doing manual funky enum values check. commit 3aeb2367e50b52ca2b9d59d4f0f0b4bbfd6a05e8 Author: Sergey Sharybin Date: Tue Dec 31 20:43:24 2013 +0600 Update gtest to latest version 1.7.0 Also reshuffled CMakeLists in order to avoid conflicts happening between gflags bundled to Ceres and third_party. commit 30aaa9cd0b4a4eb0948a17824e7e7622d8ebcb41 Author: Sergey Sharybin Date: Tue Dec 31 20:10:06 2013 +0600 Changes for VC2013 Solves compilation error when using msvc2013 Original patch is by Alexandr Kuznetsov to blender.git commit b204c0d25065a2b149de256929ff37d8f00f45bb Author: Keir Mierle Date: Tue Dec 31 20:05:58 2013 +0600 Eagerly attempt to refine a track before doing a brute search Before the refinement phase of tracking, a brute force SAD search is run across the search area. This works well but is slow; especially if the guess for the track's location is accurate. This patch runs a refinement phase before running a brute force search, hoping that the guessed position (in x2, y2) is close to the best answer. If it is, then no brute search is done. If it is not, then a normal brute force search followed by refinement is done. In some cases this may produce worse tracks than before; the regressions will need investigation. The predictive motion model (to be implemented) will reduce the probability of that happening. commit 5361513f0328ff94b53125d29129561bb03132e8 Author: Keir Mierle Date: Tue Dec 31 20:04:46 2013 +0600 Fix bug where libmv tracking incorrectly succeeds on failure Before this patch, if Ceres returned USER_SUCCESS indicating that Ceres was only changing the tracked quad slightly between iterations (indicating convergence), no final correlation check was done. This leads to incorrectly returning that the tracking was successful, when it actually failed. commit ba9e63eed09e33a48bbcb081058f45ac16f8738e Author: Sergey Sharybin Date: Tue Dec 31 20:00:46 2013 +0600 Implementation of weighted tracks Added a weight field to Track structure which means how much affect this track will have on the final reconstruction. Currently it affects on BA step only which in most cases will work just fine. However, it worth looking into weight support for intersection/resection. commit 4600df8b685ca8c4daa22d6c3b0125fd42c3bc67 Author: Sergey Sharybin Date: Tue Dec 31 19:30:14 2013 +0600 Code cleanup: move function prototype to header file commit 0ce5b6efde774b3f042acf9e42c95674548f1c01 Author: Sergey Sharybin Date: Tue Dec 31 19:26:48 2013 +0600 Get rid of Allow Fallback option for euclidean resection It was rather confusing from the user usage point of view and didn't get so much improvement after new bundle adjuster was added. In the future we might want to switch resection to PPnP algorithm, which could also might be a nice alternative to fallback option. commit 5d063426f4809000c27f38ed798e4224bbd09a6d Author: Sergey Sharybin Date: Tue Dec 31 19:24:05 2013 +0600 Use explicit declaration of int types sign Mainly needs to make blender happy with this custom header which used to run into conflict with other int types headers. Wouldn't harm being more explicit here anyway. commit c5be59dee94f94de369006c544080282cfb245cc Author: Sergey Sharybin Date: Tue Dec 31 14:50:00 2013 +0600 Implement refinement of estimated homography/fundamental matrices It was only possible to perform algebraic estimation, which didn't give so much accurate results. Implemented a way to perform algebraic estimation followed with refinement step using Ceres minimizer. The code was actually mostly already there since keyframe selection patch. Made such estimation a generic function in multiview/ and hanged API for estimation in order to pass all additional options via an options structure (the same way as it's done fr Ceres). Most of the options are straight-forward to understand,but some deserves more description here: * expected_average_symmetric_distance is used as an early output check and as soon as average symmetric error goes below this threshold refining finishes. This distance is measured in the same units as input points are. It is arguable whether we need callback for this or not, but seems Ceres doesn't have some kind of absolute threshold for function value and function_tolerance behaves different from logic behind expected symmetric error. * There' an option to normalize correspondences before estimating the homography in order to increase estimation stability. See R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2003. https://www.cs.ubc.ca/grads/resources/thesis/May09/Dubrofsky_Elan.pdf commit 1cdad972c4a9005e78891524cbd6d65600ca7e99 Author: Sergey Sharybin Date: Wed Sep 25 16:12:29 2013 +0600 Code cleanup: Minor function capitalization fix Original patch by Joseph Mansfield to the Blender repository. commit 434316d084e8a41fd452f03610d7244d664948dc Author: Sergey Sharybin Date: Wed Sep 25 16:07:43 2013 +0600 Code cleanup: spelling correction Original patch by Joseph Mansfield to the Blender repository. commit 5cfe8465ac70407c0959c53bcd5206657a1322a2 Author: Sergey Sharybin Date: Wed Sep 25 16:02:48 2013 +0600 Fix for uninitialized covariance matrix Lead to unpredictable tracking termination criteria. commit fd86b77d413489649a989f075b061714ed9a72fc Author: Sergey Sharybin Date: Wed Sep 25 16:01:19 2013 +0600 Add Procrustes PNP ("PPnP") resection algorithm to libmv This adds a new Euclidean resection method, used to create the initial reconstruction in the motion tracker, to libmv. The method is based on the Procrustes PNP algorithm (aka "PPnP"). Currently the algorithm is not connected with the motion tracker, but it will be eventually since it supports initialization. Having an initial guess when doing resection is important for ambiguous cases where potentially the user could offer extra guidance to the solver, in the form of "this point is in front of that point". Original patch by Keir Mierle made to Blender repository. commit 0d98e9bbde5d57f6cd9515ce8ff2786d322f29ea Author: Sergey Sharybin Date: Tue Jun 18 19:24:07 2013 +0600 Compilation error when using clang Where couple of issues: - Overloaded assignment operator usage ambiguity around some of the Eigen matrix assignment. - Using -O4 flag here on linux ended up in lots of cryptic linker issues, even when using recently release clang 3.3. Disabled forcing optimization flag for now. We could end up with something smarter in the future, but for now i'm not as much fan of forcing compiler's flag. commit 2b90b3915671cb629f7aabed30a88f28450294f8 Author: Sergey Sharybin Date: Sat Jun 1 16:20:35 2013 +0600 Pass vectors by a reference Saves couple of time which used to waste on copying objects, also solves win32 compilation errors caused by alignment. commit 994c02d0ec55e9ae14a93a3ada6e5d4939247fc3 Author: Sergey Sharybin Date: Thu May 30 18:00:03 2013 +0600 Code cleanup - Made some arguments passing to detector const. - Remove unneeded include from track_region header. commit 7d0c5325a38e61929f44206761b8aa3565631af5 Author: Sergey Sharybin Date: Mon May 13 04:44:14 2013 +0600 Pass ApplyRadialDistortionCameraIntrinsics input arguments by reference This shall save some CPU time on calling copy constructor and give some boost of bundle adjuster (where jet could take some time to be passed by value). commit eb2e567df4567a54887d602aa95d6744aa154d8b Author: Sergey Sharybin Date: Mon May 13 04:22:05 2013 +0600 Minor code style cleanup. commit ad3dbaaef10ea721230694311a359df152c7a44a Author: Sergey Sharybin Date: Sun May 12 22:34:54 2013 +0600 Cleanup in simple pipeline's bundler - Better match Google's code style conventions. - Move evaluation part into own function, makes bundling itself easier to follow. - Made evaluation an optional parameter. - Removed note about unsupported camera intrinsics refining flags. Technically, all combinations are possible. commit 4432eb80f27e929f8750229aaada625d4f3ac5ee Author: Sergey Sharybin Date: Sun May 12 22:19:31 2013 +0600 Remove check for zero focal length in BA cost functor This check is actually redundant, because empty intrinsics will have focal length of 1.0, which means original comment about BundleIntrinsics was not truth. It is possible that external user will send focal length of zero to be refined, but blender prevents this from happening. commit 34a91c9b8acb0dba3382866fbd29bb9884edb98a Author: Sergey Sharybin Date: Sat May 11 20:33:54 2013 +0600 Fix compilation error with msvc2012 Using change from glog's upstream for this. commit 87be4f030d025e4b29d9243d12bc458b2bb6762a Author: Sergey Sharybin Date: Sat May 11 19:50:57 2013 +0600 Style cleanup, mainly pointed by Sameer in Ceres's codereview commit 7fa9c0b83d5e0fbd331add2952045076c2028d1b Author: Sergey Sharybin Date: Fri May 10 18:30:40 2013 +0600 Keyframe selection improvements Added additional criteria, which ignores keyframe pair if success intersection factor is lower than current candidate pair factor. This solves issue with keyframe pair at which most of the tracks are intersecting behind the camera is accepted (because variance in this case is really small), Also tweaked generalized inverse function, which now doesn't scale epsilon by maximal matrix element. This gave issues at really bad candidates with unstable covariance. In this case almost all eigen values getting zeroed on inverse leading to small expected error. commit 0477ef1aa8fc92848f03c45e32539210be583b80 Author: Sergey Sharybin Date: Fri May 10 17:59:40 2013 +0600 Keyframe selection code cleanup - Updated comments in code. - Removed currently unused functions. Actually, they'd better be moved to some generic logging module, but we don't have it now so was lazy to create one now. Could happen later using code from git history - Renamed function to match better to what's going on in it. commit fee2d7cc6003942f628c9a24b74008fd491b85b9 Author: Sergey Sharybin Date: Fri May 10 17:44:49 2013 +0600 Optimization for reconstruction variance Achieved by replacing SVD-based pseudo-inverse with an eigen solver pseudo inverse. New function works in the same way: it decomposes matrix to V*D*V^-1, then inverts diagonal of D and composes matrix back. The same way is used to deal with gauges - last 7 eigen values are getting zeroed. In own tests gives approx 2x boost, without visible affect on selected keyframe quality. commit b735649ead4d3d61f7896e46f35b1f7e0cecea3d Author: Sergey Sharybin Date: Thu Mar 14 14:53:42 2013 +0600 Initial commit of reconstruction variance criteria which is an addition for GRIC-based keyframe selection. Uses paper Keyframe Selection for Camera Motion and Structure Estimation from Multiple Views, ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf as a basis. Currently implemented camera positions reconstructions, bundle positions estimation and bundle adjustment step. Covariance matrix is estimating using generalized inverse with 7 (by the number of gauge freedoms) zeroed eigen values of J^T * J. Additional changes: - Added utility function FundamentalToEssential to extract E from F matrix, used by both final reconstruction pipeline and reconstruction variance code. - Refactored bundler a bit, so now it's possible to return different evaluation data, such as number of cameras and points being minimized and also jacobian. Jacobian currently contains only camera and points columns, no intrinsics there yet. It is also currently converting to an Eigen dense matrix. A bit weak, but speed is nice for tests. Columns in jacobian are ordered in the following way: first columns are cameras (3 cols for rotation and 3 cols for translation), then goes 3D point columns. - Switched F and H refining to normalized space. Apparently, refining F in pixel space squeezes it a lot making it wrong. - EuclideanIntersect will not add point to reconstruction if it happened to be behind the camera. - Cleaned style a bit. commit 8bee982972a0b5bded3555b60e0987af685f85bd Author: Sergey Sharybin Date: Fri May 10 13:27:21 2013 +0600 Left extra debugging print in reconstruction scale by accident. commit 7971967d24e3d8d505bd6e54523161ab5dd5b728 Author: Sergey Sharybin Date: Fri May 10 12:23:03 2013 +0600 Add check for points behind camera in euclidan BA cost functor In cases keyframes are no so good, algebraic two frames construction could produce result, for which more aggressive Ceres-based BA code will fall to a solution for which points goes behind the camera, which is not so nice. Seems in newer Ceres returning false from cost functor wouldn't abort solution, but will restrict solver from moving points behind the camera. Works fine in own tests, but requires more tests. commit f34e73f6bd0041a31f779a78e1c2324f9799a4a2 Author: Sergey Sharybin Date: Wed Apr 24 22:06:38 2013 +0600 Forgot to add reconstruction scale to CMakeLists commit de08cbaebe1b95673c4bc8f639aa0228414cf4a2 Author: Sergey Sharybin Date: Wed Apr 24 19:40:39 2013 +0600 Reconstructed scene scale ambiguity improvement Added a function EuclideanScaleToUnity() which is aimed to solve scale ambiguity by scaling solution in a way cameras centers variance in unity. Currently only available for euclidean pipeline, projective one is not finished anyway. commit 30ecb7dd075527c0e49220744bae65ec208dbdf5 Author: Sergey Sharybin Date: Tue Apr 23 01:41:29 2013 +0600 Use epsilon in modal solver test Default epsilon for isApprox was too small, leading to some false test failures. commit d03b303c171b0b0a33ed0b31c9a744c9676492a9 Author: Sergey Sharybin Date: Tue Apr 23 01:35:56 2013 +0600 Update Ceres to current HEAD Brings optimization for DENSE_NORMAL_CHOLESKY and also fixes threading issues with BLAS. commit 0a352686de0e87d59a2e37dbd0fddb351cbea2b7 Author: Sergey Sharybin Date: Mon Apr 15 05:35:33 2013 +0600 Fix for bundle adjusting with motion restricted Was a bug introduced in previous commit, which was trying to set parameterization for non-existing camera->t parameter block. Replaced with subset parameterization. Also added basic synthetic unit test for modal solver. commit 1742515b89ad5da53be95bfc574cbeffc51dc0e2 Author: Sergey Sharybin Date: Mon Apr 8 23:33:20 2013 +0600 Bundle adjustment improvements - Get rid of rotation matrix parameterization, use angle-axis instead. Also Joined rotation and translation into a single parameter block. This made minimization go significantly faster, like 1.3x times in average. - Fix first camera when bundling. This is to address orientation ambiguity. Reconstruction result could still vary in size, but that's another issue to be addressed later. Additional change: Split EuclideanBundleCommonIntrinsics into smaller functions, so it's now a bit easier to follow. commit 74bbeabf1c4570bfe04f3034afcde95134b3e161 Author: Sergey Sharybin Date: Mon Apr 8 23:31:57 2013 +0600 Update Ceres to current HEAD Brings up some noticeable speed improvements. In particular the automatic differentiation and bundle adjustment solvers. commit e47519b0442527533ee046dd9881212d169561da Author: Sergey Sharybin Date: Mon Apr 8 02:21:26 2013 +0600 Corrected path to gflags Currently tools/track.cc is not used, but let's keep things a bit more up-to-date :) commit f617741b1f2a0ac7981dd71144eba187521d9ac4 Author: Sergey Sharybin Date: Mon Apr 8 02:17:16 2013 +0600 Re-enable tests for multiview and image For as long code is in repo and used by some tools better to have it covered by tests. Some of them are failing tho, but that's completely different story to be addressed later. commit 7dbb5a37333fb6c5a729b8bafb317ea210735cbc Author: Sergey Sharybin Date: Mon Apr 8 02:10:07 2013 +0600 Do not modify cache's CMAKE_CXX_FLAGS_RELEASE when configuring Ceres Otherwise you'll have infinite appending of Ceres-specific flags on every saving of any CmakeLists.txt. commit b413d22280de2481a2e2acdcda8431f601d65448 Author: Sergey Sharybin Date: Sun Apr 7 21:53:23 2013 +0600 Fixed compilation with BUILD_TOOLS enabled This commit mainly reverts parts of following commits: 0eebc21db831211738acc938566bbc29d68d45db d8109b7a4fede1660e0dbd73735f1a9e3fd79eec e59595806c045916ab4ef15ef7047c1a728b2da9 2d6cd58ee1cd7c5073980f358c71b2898ad16b4c They declared lots of stuff deprecated, but in fact it's not deprecated just a bit different usage pipeline. Anyway, deprecation shall not happen spontaneously as a part of other changes. And for sure shall not break anything. commit c9eb9d36bb1ca5d3037ce66633d812a224f77958 Author: Sergey Sharybin Date: Sat Apr 6 20:49:05 2013 +0600 Revert "Change libmv's bilinear sampling to assume the same" Revert changes to bilinear sampler which were originally aimed to match blender's pixel center (where integer coord is a left-bottom corner, x.5 coords are centers. The reason of revert is changing this assumption in only sampler didn't work well and lead to wrong results of BlurredImageAndDerivativesChannels for example. Discovered when was doing unit-tests for brute region tracker. This reverts commit daa354c0735b954b0cd7725626e9a3d67416d46b.