Saturday, 26 September 2015

Processing: Image Processing

At this stage we load the image file information, including the width and height as well as the focal information embedded in the camera images.

We make a decision as to how to handle the camera model, and compensate for image distortion introduced by the camera itself in this stage, and OpenMVG offers several different types of model to choose from.

Reading Image Parameters

OpenMVG provides a simple mechanism for extracting exif data, which can then be used to calculate focal length. Since we have to use the exif information to get a reliable focal length value then at this stage we may as well use the exif reader to extract the width and height information.

Therefore our minimum working example to extract information from the image file is:

  std::unique_ptr exifReader(new openMVG::exif::Exif_IO_EasyExif());
  if (!exifReader->open(which))
    return;
  image_focal = static_cast(exifReader->getFocal());
  width = static_cast(exifReader->getWidth());
  height = static_cast(exifReader->getHeight());

However the value of focal length we have retrieved is in mm, and the actual focal length value we want to use would be in pixels, so we calculate this value using information on our sensor sizes (in mm):

  const double ccdw = 5.75;
  focal = std::max (width, height) * image_focal / ccdw;

where ccdw is the CCD sensor width of the camera, which can be extracted from a camera information database, or hardcoded for a given camera.  It can also be retrieved from the exif image information in some cases.

The OpenMVG sample code uses the database lookup to pick up the information based on the camera name, but I'm hardcoding the numbers for my camera ccd here.

There is some more background on the derivation of the focal length here: http://phototour.cs.washington.edu/focal.html


Note that OpenMVG provides simple methods to read the image data directly, and we can extract the width & height information from that process instead, i.e.:

openMVG::image::Image
  if (openMVG::image::ReadImage(_name.c_str(), &_image_data))
  {
    width = _image_data.Width();
    height = _image_data.Height();
  }

The example code does this in a few places. However this load is redundant in our case, since we need to run the exif parsing for focal lengths. While we want to load the data eventually doing it now would simply incur a double load, and the load call itself is actually quite slow.

Filling in the camera details

The camera information associated with an image is held in a camera Intrinsics structure.

The intrinsic structure stores the optical model, covering such parameters as the camera focal length, the principal point (ideal image centre) and the associated image distortion parameters, etc.

There is also a camera Extrinsics model, referred to in OpenMVG as a Pose (of type openMVG::geometry::Pose3) ,which covers the camera position and rotation, however the details of this actually will be handled by OpenMVG itself.

The structure openMVG::cameras::IntrinsicBase is used as a base type when building lists, and this is ultimately derived into two common classes, openMVG::cameras::Pinhole_Intrinsic_Radial_K1 and openMVG::cameras::Pinhole_Intrinsic_Radial_K3.

The difference between these two classes is the number of parameters supplied (one for K1 and three for K3) which are used to remove image distortion.

The full documentation on this is held here:
http://openmvg.readthedocs.org/en/latest/openMVG/cameras/cameras/#pinhole-camera-model

To keep things simple we initially use a K3 model, but leave the distortion coefficients as 0.

So, to build the intrinsic model from an image then

  intrinsic = std::make_shared (
        width, height, focal,
        ppx, ppy,
        0.0, 0.0, 0.0);
and now we can pass the intrinsic into the SfM container objects.
How to do that next...