Sunday, 4 October 2015

Filtering the Features

This is basically an image matching process, which looks for corresponding matches on points in the image set.

It's actually a two step process – an initial run through looks for likely matches (the “putative” matches) which uses one of a set of possible matching algorithms. Following this then the possible matches are run through a geometric filter which should discard outliers (unreliable) matches.

Once again this isn't something you really need to understand the behind the scenes detail to use – there are a couple of default values which determine the thresholds for flagging a match and determining the outliers, but otherwise the API is fairly opaque.

However the Region information must be used to determine which of the Scalar or Binary geometric matcheroptions are available.

First – Load the image information

Run through the view list, get the file names and push to a list of names and sizes

Next– The matchers

Then we create a matcher to process the image information. This is of the base type openMVG::Matcher_Regions_AllInMemory and which is one of the derived matcher types which matches the region we used (scalar or binary).

We pass in a distance ratio, used to determine the threshold for discarding spurious points: A higher value is stricter (from the Nearest Neighbour distance matching code; the "Ratio between best and second best matches must be superior to [the] given threshold")

Since we used the default AKAZE earlier we can simply assign an ANN_L2 matcher to it.
On the successful load then we use openMVG::exhaustivePairs() to generate all the pair values (type openMVG::Pair_Set), and then we run the Match function, and the output is set of matches we drop to a file using the PairedIndMatchToStream() call.

Next: Geometric Filter

The geometric filter process takes a maximum residual error value, the complete list of potential (putative) matches alongside a geometric model fitting function. It then generates an output set of matches that fit the model to within the residual error.

There are three available solvers, which are used to generate a target model to filter points against:

  • GeometricFilter_FMatrix_AC AContrario Fundamental matrix solver
  • GeometricFilter_EMatrix_AC AContrario Essential matrix solver
  • GeometricFilter_HMatrix_AC AContrario Homography matrix solver

So, for the Fundamental Matrix solver then we pass it the list of putative match pairs, and using these points then the geometric filter estimates the fundamental matrix and removes outliers from the set of matches. Following this we save the final matches to a file using PairedIndMatchToStream().

Although there's some fiddly detail hiding behind this model, we can simply pass in the filter reference and the maximum residual error that is used to discard outliers; a lower value here discards more points (but too high a value will include outliers and cause problems when bundle adjusting).

There's some more information on using fundamental matrix and a-contrario estimation at

A simple MWE using some default values from the sample code is:
void ImageList::computeMatches()
float fDistRatio = 0.6f;
openMVG::matching::PairWiseMatches map_PutativesMatches;
std::vector<std::string> vec_fileNames;
std::vector<std::pair<size_t, size_t> > vec_imagesSize;

  for (openMVG::sfm::Views::const_iterator iter = _sfm_data.GetViews().begin(); iter != _sfm_data.GetViews().end(); ++iter)
    const openMVG::sfm::View * v = iter->second.get();
    vec_fileNames.push_back(stlplus::create_filespec(_sfm_data.s_root_path, v->s_Img_path));
    vec_imagesSize.push_back( std::make_pair( v->ui_width, v->ui_height) );

  std::unique_ptr<openMVG::Matcher_Regions_AllInMemory> collectionMatcher;
  collectionMatcher.reset(new openMVG::Matcher_Regions_AllInMemory(fDistRatio, openMVG::ANN_L2));

  if (collectionMatcher->loadData(_regionsType, vec_fileNames, getDirectory()))
    openMVG::Pair_Set pairs;
    pairs = openMVG::exhaustivePairs(_sfm_data.GetViews().size());
    collectionMatcher->Match(vec_fileNames, pairs, map_PutativesMatches);
    std::ofstream file(_matches_full);
    if (file.is_open())
      PairedIndMatchToStream(map_PutativesMatches, file);

  std::shared_ptr<openMVG::sfm::Features_Provider> feats_provider = std::make_shared<openMVG::sfm::Features_Provider>();

  if (!feats_provider->load(_sfm_data, getDirectory(), _regionsType))

   openMVG::PairWiseMatches map_GeometricMatches;

  ImageCollectionGeometricFilter collectionGeomFilter(feats_provider.get());

  const double maxResidualError = 1.0;


  std::ofstream file (_matches_filtered);
  if (file.is_open())
    PairedIndMatchToStream(map_GeometricMatches, file);