Sunday, 20 September 2015

Using the OpenMVG Library

The next few posts look at an Incremental Structure from Motion (SfM) pipeline, where we use the openMVG library to produce a point cloud from some input images, i.e. to generate a 3D model.

Using the openMVG library simplifies the construction process immensely, and hides some of the algorithm details from us.

As a library it's not perfect, and it can be annoyingly verbose at times, but it beats doing it from scratch. There's a few sample applications in the OpenMVG tree, which most of this work is based off, but no good overview on how to tie all the pieces together when using the library directly.

The Very Top Level

We're processing a set of 2D images, and the output will be a 3D model of the objects in the image, as a point cloud.

So we start with a set of photographs:


 And generate a point cloud:





Using an incremental pipeline means that the reconstruction will start with a model reconstruction based on two initial images, and then we add  add new images and 3d points to generate the complete model.

The process is described at: http://imagine.enpc.fr/~marletr/publi/ACCV-2012-Moulon-et-al.pdf.

This approach is based around feature detection within the images and Bundle Adjustment (https://en.wikipedia.org/wiki/Bundle_adjustment) to synthesise the 3D data. The “a contrario model estimation” means that we're going to use the input data itself to identify and remove outlying data samples, however this processing will be hidden within the library itself.

Using an incremental pipeline is slightly simpler, from the perspective of a library client. An alternative approach is Global SFM, which would process all the images simultaneously: Global SFM is potentially more accurate, since sequential processing can introduce errors, and the global version is more efficient and simpler to parallelise. But the library usage in the sequential case is easier, so lets start with that.