What's the effect of cropping Fisheye Images?

A real example.

Case study: my own "Energy" WWP panorama

Introduction: the blurred tower truss

I have submitted this panorama for the World Wide Panorama of September 2005 Equinox.

As one can see, most of the upper hemisphere of the view is the sky and some metallic truss structure on the background and foreground respectively.

Technically this could be described as Low Frequency Spatial details and High Frequency Spatial details. Almost no Medium Frequency.

I used Five shots to get the panorama: Four sides and a Zenith shot from a Sigma 8mm fisheye on a Canon EOS 20D camera.

I observed (to my surprise and my dismay) that I could not get a sharp and contrasty tower truss on the panorama. It kept in being "soft" and in any case really bad.

The original Zenith shot looked perfectly sharp and focused all right when I scrutinized it at 300% in Photoshop.

I was using a beta version of PTGui 5 and thought that either I lacked the expertise to use it or there was a bug in this GUI...

Cropping in Photoshop saved the panorama

I decided to patch the sharp Zenith onto the panorama in Photoshop. The result was far from my expectation. The softness of the background was very difficult to be covered by sharp details without visible transition :-(

I then cropped the five image. I simulated having shot with a 6x6 camera and got five 2336 x 2336 pixels TIF images that I ran into PTGui again: Bingo, at last a really crisp truss over a fine sky!

BTW: That can be done also with PTMac / XBlend.

Of course the JPEG photo compression of the final QTVR movie does not give that impression. Wait for more precise photos to be seen in the next paragraphs.

The cause of the failure, the technical investigation

Four photos with VFOV of 180 degrees shot at 90 degrees are sufficient to cover the full spherical space. The Zenith (and the Nadir) is in fact covered by at least two or three information layers provided by the overlap of the adjacent images.

The information supplement that is coming from the Zenith image just confuses the blending engine. The result is simply not predictable for two blenders (Enblend and Smartblend) and sometimes also not good for the third and more recent PTGui blending

I have made an extensive study of all the possible combinations that PTGui 5 offers now to produce an equirectangular panorama. The results are summarized in another page (click on SYNTHESIS button from the navigation bar at the top).

As soon you give the not cropped warped images to any of the three blend engines currently available (PTGui, Enblend and Smartblend) and if at the same time you have more than the horizontal row of portrait photos to be stitched and blended, some spill over between images shall happen. The Zenith image that is supposed to give sharper result may also spoil the lateral faces of the cube and vice versa!
Sometime, the difference (between the pano made from original not cropped images and the new one made from cropped images) is so subtle and then fortunately very hard to see. You have to look at the two panoramas side to side to spot it (Superposition of one on the other is not possible as the Optimization shall give slightly different subsequent shape and warping).

The effect of not cropping is unpredictable as it depends totally of the images contents (high frequency/low frequency details distribution, location etc).

Furthermore, in some cases the order in which the image are placed in the "Source Images" list has also strong influence on the final blending!

Cropping can be made right into PTGui 5 and this is easy to do!

First case for reference: without cropping (or with standard "180 degrees Circular" Crop):

The usual workflow for Circular Fisheye images is to crop (or not at all) along the circle of the "clipped" circular image. Then a ~180 degrees HFOV is determined by the Optimizer.

I ran this workflow with the five Images of my WWP "Energy" panorama. This is the PTGui project file.

I have obtained five different panoramas:

I have then extracted the Top face of the Spatial cube that shows the Zenith of each of the panoramas.

Click on the images to enlarge to see a bigger picture.

PTGui (Warp by PanoTools) PTGui (Warp by PanoTools) PTGui (Warp by PanoTools) PanoTools (PTStitcher) PanoTools (PTStitcher)
PTGui Enblend Smartblend Enblend PTStitcher

The ghost artifacts in the four last sub-cases are due to the not optimum control points setting in the Nadir area. As I assumed (wrongly) that the Zenith shall be covered in any case by the Specific Zenith image, I had prioritized the best fit in the horizontal plane rather than the top.

The PTGui project script is EXACTLY the same (but the rendering data ("Create Panorama" tab) for the five sub-cases.

This is the proof that the Zenith image is partly (or here TOTALLY) ignored in the blending process. The blenders (*) are fooled by the multiple overlap at the Zenith (Please compare with the next paragraph case). Note that the Zenith image has been placed LAST in the order of the "Source Images" list and this may be important also.

(*) Note that PTGui (warp by PanoramaTools) associated with PTgui 5 blending does here a very good job though.

I think that this problem is well known by the Enblend and Smartblend engine developers, but the solution has yet to come.

Q: Is there a difference when the image of the Zenith is placed first rather than last in the "Source Images" list?

Let's place the Zenith image first (Recall: It was the LAST in the list in all the other sub-cases of this page)

PTGui (Warp by PanoTools) PTGui (Warp by PanoTools) PTGui (Warp by PanoTools) PanoTools (PTStitcher) PanoTools (PTStitcher)
PTGui Enblend Smartblend Enblend PTStitcher

Enblend fails to give the expected result wether fed by PTGui (but warp by PanoTools) or by PTStitcher. This time Smartblend does all right. I have found other occasion where this did not happen. Placing the Zenith (and Nadir) in first place helps but this is not a panacea.

Second case: with only FOUR images

Q: Is there a real benefit by not cropping Five shots (4 + 1 Z) rather than using only Four shots around?

PTGui (Warp by PanoTools) PTGui (Warp by PanoTools) PTGui (Warp by PanoTools) PanoTools (PTStitcher) PanoTools (PTStitcher)
PTGui Enblend Smartblend Enblend PTStitcher

Here is the PTGui 5 project file.

As there is no specific Zenith image here, priority was given to the perfect match of Control Points of the upper part of lateral images. Emphasis was planned to be on the Zenith (it is the opening view of my panorama here) the Zenith is even better with only four shots than in the previous case!

Adding a Zenith shot is not always the winning process if NO CROP is done.

Best case: Square Cropping (109.6 x 109.6 degrees) of all the Source Images:

Cropping i.e. specifying shape (Lens Type") and dimensioning of the crop can be done two ways:
a) Physically and for real by saving a new file after cropping in a graphic application (e.g. Adobe Photoshop) and using this new file for source image in PTGui 5.
b) Digitally and virtually by setting the crop size directly in PTGui (Tab "Crop") possibly after proper adequate selection of "Lens Type" in the "Images Parameters" table.

I have taken the second way. This is the PTGui project file.

As the HFOV is smaller than 110 degrees (that is an internal limit set by the developer), PTGui 5 supports warping the Panorama

PTGui PTGui PTGui PanoTools (PTStitcher) PanoTools (PTStitcher)
PTGui Enblend Smartblend Enblend PTStitcher

All cases but, of course, the last sub-case (see on the left of the image) are just as expected: crisp and contrasty Zenith.

Circular Cropping Vs Square Cropping?

Circular cropping (Circular Fisheye "Lens Type") of 109.5 deg angular Diameter instead of Square cropping gives in fine exactly the same result.

In spite of this equivalent performance, I would recommend this other process ONLY if you use a near perfect registration for the four horizontal shots (90 Deg apart). The margin that assures the overlap at the upper corners of the Spatial Cube is so thin that it is much safer to go the "Square way".

Some conclusions drawn from this case study:

1) With only the "Standard" 180 degrees crop, only one panorama is satisfactory in the case: PTGui (Warp by PanoTools) blended by PTGui.

The other cases have two types of defects:

It is here a paradox that it would have been better to use only four images with a good CP setting at the Zenith instead of using Five shots and the "wrong" blender/warper/stitcher combination!

The result depends on the order in the Source Images list. When the image of the Zenith is placed at the top, only Enblend fails to produce the expected result.

Recall: only one Image is here a supplement to the for lateral sides of the spatial cube. If an image of the Nadir was also to be stitched and blended this result would change!

2) When square cropping (109 x 109 Deg) is applied, all four combinations that get a real blender help are perfect. The differences are almost not visible between the contenders. The case of PTStitcher without blending is only for reference as I have even set the feather size to ZERO (I keep now this setting as default to not forget it for good performance of Enblend when it is associated with PTStitcher).

Cropping is here (for this specific case study) essential, PTGui would be a good alternative, but I have found that this is not always the case: read the results of my complete investigation (click on SYNTHESIS button from the navigation bar at the top).

You want to repeat the above tests?

The images were downsized and formatted/compressed in JPEG from the TIF 16bits original and placed in this folder.

To put them in the Project Script file you may restore tem to their original size and format if you wish by upsampling.

Michel Thoby

11 November

Revised on 17 November 2005