Saturday, 26 September 2015

Loading the images

Overview

The core abstraction here is a view, represented by openMVG::sfm::View.

This represents a single image, and contains a reference to the image path as well as the camera information associated with it.

The top level SfM_Data object holds a list of View objects, openMVG::sfm::Views, which is a Hash Map indexed through the IndexT Id's of each View.

We also need to consider the camera intrinsic structure mentioned earlier, openMVG::cameras::IntrinsicBase. The SfM_Data object also holds aopenMVG::sfm::Intrinsics list - another hash map which contains an Id indexed map of camera intrinsics.

In theory we could share the intrinsics, since several of the images will come from a single camera (and the reference code does this), however it's simpler just to have an intrinsic structure per picture.

Using unique cameras per image does have a fairly major drawback though: During the bundle adjustment OpenMVG will refine the camera intrinsics for us, and dump these in the output. Obviously if we were sharing a camera model these distortion parameters should converge to a single model for a given camera, however since we're using separate intrinsics then the distortion parameters per view can be slightly different even if they should not be.

This problem isn't enough to break things completely, but it does mean our code will be sub-optimal here. For now lets run with it though, and fix it later.

The Image Load Process

Create a view, with an instance of openMVG::sfm::View

This simply takes the basic View constructor parameters, which include the image name, the view intrinsic and pose ID's associated with the view and the width and height of the image.  Note that Pose is the camera extrinsic – it's basically a 3D transform, but we don't need to get involved with it right now.

For the ID's we simply use the current view size to provide a unique integer value. We can use common ID's for the view, intrinsic & pose because they're stored in different lists. As a result we wind up with the view constructor:

openMVG::sfm::View v(which, views.size(), views.size(), views.size(), width, height);

We also create an Intrinsic, allocating and filling in a copy of openMVG::cameras::IntrinsicBase

For our example we're using the class: openMVG::cameras::Pinhole_Intrinsic_Radial_K3 and leaving the distortion parameters as 0.

We add the intrinsic & view to the top level hash map of SfM_Data, with the Id's from the view object that we allocated earlier using the array accessor notation:

  intrinsics[v.id_intrinsic] = intrinsic;
  views[v.id_view] = std::make_shared(v);

Putting this all together we get a final MWE code layout of:

void ImageList::loadImage(std::string which)
{
double width;
double height;
double image_focal;
double focal;
double ppx;
double ppy;

openMVG::sfm::Views& views = _sfm_data.views;
openMVG::sfm::Intrinsics& intrinsics = _sfm_data.intrinsics;
std::shared_ptr intrinsic (NULL);

  if (openMVG::image::GetFormat(which.c_str()) == openMVG::image::Unknown)
    return;


  std::unique_ptr exifReader(new openMVG::exif::Exif_IO_EasyExif());
  if (!exifReader->open(which))
    return;

  image_focal = static_cast(exifReader->getFocal());
  width = static_cast(exifReader->getWidth());
  height = static_cast(exifReader->getHeight());

  ppx = width / 2.0;
  ppy = height / 2.0;

  if (image_focal == 0)
    return;

  printf("Image %s: %f x %f, Focal Length %f\n", which.c_str(), width, height, image_focal);

  const double ccdw = 5.75;
  focal = std::max (width, height) * image_focal / ccdw;

  openMVG::sfm::View v(which, views.size(), views.size(), views.size(), width, height);

  intrinsic = std::make_shared (width, height, focal, ppx, ppy, 0.0, 0.0, 0.0);
  intrinsics[v.id_intrinsic] = intrinsic;
  views[v.id_view] = std::make_shared(v);
}