Creation and use of synthetic circular fisheye images

Foreword: you may read this page to know better about fisheye lens various projections

Q: Why one would bother to create synthetic circular fisheye images?

A: (Mainly) To put panoramic software to test.

Images of real world images that are carefully shot with good cameras fitted with fisheye lenses can of course be used to put panoramic software to tests.
However, many inevitable various intrinsic (and often unwanted) "features" of these images interact simultaneously and shall conjugate their (generally) negative effects to defeat the performance of these software. For instance: light radial fall-off, optical vignetting, sensor/lens misalignment, image softness near the edge, parallax effect, entrance pupil shift along and off the lens axis, textures or features similitude between overlaying images, poor angular (i.e. pixel density) resolution near the image boundery, digital noise, lack of (or excessive) contrast, stray light and colored ring around the images circle, etc., these features shall all challenge the software algorithm during detection, correction and alignment and therefore they may disrupt the performance of the algorithm.The user's perception and analyzing skill may then be fooled and he may wrongly attribute the cause of the software disorder.

Inputing "artificially controlled images" is therefore preferable for better diagnostic after testing. The actual precision of the fisheye lens modeling (that leads to efficient image correction) and the numerical stability (i.e. ability to avoid mathematical chaos) can for instance be more accurately assessed when using such "aberration less" synthetic images.

How to create sample of synthetic images of all circular fisheye projections?

While they are also intended for correction of regular rectilinear image, many Software are capable of outputting fisheye images of the four "classical " types (i. e., Stereographic, Equidistant, Equal Area and Orthographic).

They also accept several various fisheye image formats as input but a considerable difference of processing context is to be noted: either a generic(Equidistant) input format is inputable or a specific format can preferably set. The latter is mostly intended to possibly reduce the extent of the required image correction and in some case to strengthen the sofware stability: the input format is set at a type that is the closest to the known projection of the concerned source image. Anyhow, when the fisheye projection is unknown, the generic projection can be selected.

Such capable tools that are currently installed on my computer for this task are:

Flexify2 (Flaming Pear Software), AutoPano Giga (Kolor), PTGui (New House Internet Services), Hugin (Open Source software), Pano2VR (Garden Gnome Software) and the Panorama tools (Freeware by Helmut Dersh. The stand alone PTools are obsolete nowadays: Hugin is now the popular substitute). Please pardon me if I forgot to put other tools on this list...

Here follows an overview for (this present topic related) features of these tools. In the Input files format column, Equirectangular has been added as this ability will be essential in the process of synthetic image creation later on this page.

Having no "red" box, Hugin is the only software amongst the 5 considered tools that fulfills all requirements including Equi-solid angle (a.k.a. Equal Area) outputting.

Input files
Output files
Equirectangular
Rectilinear
Stereographic
Equidistant
Equi-solid Angle
Orthographic
Equirectangular
Rectilinear
Stereographic
Equidistant
Equi-solid Angle
Orthographic
Flexify2
Yes (and "more")
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
No
Yes
AutoPano Giga
No :(
Yes
Generic (FE Equidistant)
Yes
Generic (FE Equidistant)
Generic (FE Equidistant)
Yes
Yes
Yes ( Little planet)
No
No
Yes
PTGui
Yes
Yes
Generic (FE Equidistant)
Yes
Generic (FE Equidistant)
Generic (FE Equidistant)
Yes
Yes
Yes
Yes
No
No
Hugin
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Pano2VR
Yes
Yes
Generic (FE Equidistant)
Yes
No
No
Yes
Yes
Yes ( little planet)
Yes
No
No
Cubic Converter (Mac only)
Yes
No
No
No
No
No
Yes
No
No
No
No
No

What image to select as Hugin input... to create a simulated fisheye image?

An equirectangular image is certainly the best or at the least more convenient. 12000 x 6000 px (reduced from original native 20000 x 10000 px) is a good compromise as it allows creating individual images of a reasonnable dimension on a next step (e.g. 6000 x 6000 px)

What image dimensions to select for output from Hugin to input in the tested S/W?

Stereographic HFoV=200°
Equidistant HFoV=200°
Equi-solid angle HFoV= 200°
Orthographic FoV = Max (180°)

6000 x 6000 px seems to be a good compromise depending on the required FoV for the Hugin output image. I have selected not to crop this output(square) image. A circular crop may be done with the circular selection/crop tool in the panoramic software under test (when required).

Can a "measurement grid" be added?

A grid that shall be apparent on the circular images (and on the final reconstructed equirectangular image as well) is overlayed on the initial equirectangular image. The quality of the warping and of the stitch shall then be more easily assessed visually: the grid on the final panorama should ideally be exactly the same as on the original panorama, from top (zenit) to bottom (nadir).

The grid lines should be wide enough to be visible but tey should also be thin enough to not disturb CP detection by the software.

As a consequence, the subsequent source images then become more complex in appearence, but in fact the grid is also very useful for accurate manual CP creation. Here follows an extreme example with stereographic projection (gigantic FoV= 220°).

The same in fisheye equidistant projection (FoV = 220°)

In equisolid angle fisheye projection(FoV = 220°)

And now with the fisheye orthographic projection (that cannot be with a FoV >180°):

How many images should be created?

It all depends on the extent of the test that is intended to put the stitching software through. From 2 up to 32 seems a fair amount not counting possible extra "shots" required to test "View Point" correction tool in PTGui, for example...

Testing for 6 images around on the horizontal plane is probably the maximal requirement when the FoV of the images is, say, 200 degrees (but always limited to180 degrees for Orthographic projection of course).

Exhaustive testing for two-shots (back-to-back), three images around, four images around and six images around (and in any possible combination) would most probably require to create 32 images as shown on this table:

What are the polynomial image correction coefficients (PTGui and AutoPano Pro) for each fisheye projection?

Hugin
PTGui
AutoPano Pro or Giga (First attempt)
AutoPano Pro or Giga (Further attempts)
a
b
c
a
b
c
User's imposed FoV setting
k1
k2
k3
Estimated FoV
k1
k2
k3
Estimated FoV
Stereographic HFoV 200° circular Crop
TBD
TBD
TBD
0.3539
-0.2515
0.1922
200°
0.044/0.481/0.391/... (chaotic)
0.681/0.231/-0.285/... (chaotic)
0.000 (*)
220° (S/W -wrongly- limited)
0.232
0.129
0.000 (*)
200.219°
Equidistant HFoV 200 circular Crop
TBD
TBD
TBD
0.0004
0.0008
0.0007
200°
0.000
0.000
0.000 (*)
194.914°
0.000
0.000
0.000 (*)
194.914°
Equi-solid angle 200° circular Crop
TBD
TBD
TBD
0.0139
-0.1573
0.0049
200°
-0.126
0.004
0.000 (*)
200.019°
-0.126
0.004
0.000 (*)
200.019°
Orthographic 180° circular Crop
TBD
TBD
TBD
0.1756
-0.7902
0.0497
180°
-0.409/-0.242/(chaotic)
0.192/0.011/...(chaotic)
0.000 (*)
143.133°/151.704°....(wrong **)
-0.408
0.046
0.000 (*)
179.683°

Important warnings:

For many concurrent reasons, the optimized values for the polynomial coefficients will widely vary as a function of the diameter of the circular image cropping. While they could be a good starting point, the figures that are reported here above cannot be duplicated exactly for other source images having a widely different FoVfrom those above or if they were cropped to a much different diameter size in the stitching program before Optimization.

(*) The k3 coefficient value is always displayed at ZERO even when selected by the user. When asked about this (bug?) Kolor did not ever aknowledge this observation. Note that Kolor wrote a paragraph titled "Distortion model : Sets the distortion model of the lens(es)" about "k3 on page 13 of the APP user's manual: "3rd order : Further complex distortion. Few lenses still get this type of distortion (not very useful in practice)." Well it looks like K3 may be set as active by the user on the Optimizer Tab dialog but then its value seems to never been displayed different to nought (Grrrrrr!)

(**) Estimated FoV is given for APG because it is optimized (with all other coefficients and parameters) against my wil : it cannot be "deselected" for optimizationl. Furthermore:

Conclusion (concerning results of test attempting to stitch 6 or 8 images each with the same projection as the others):

AutoPano optimization algorithm is as good as is PTGui when Equidistance and Equi-Solid angle are concerned but it's not good when Stereographic or Orthographic projection are inputed. When using a normal and rather casual optimization workflow, the FoV is respectively grossly over-estimated or under-estimatedin these two opposite conditons:(

What happens when four images with each of a different projection from the others are stitched in a single panorama?

It works.With both PTGui and with APG! Lot of fun doing this....

 

Michel Thoby

 

3 September 2012

Revision: 17 November 2012 (using APG 3.0( with alternate optimization processes)