Models for the various classical lens projections

... where fisheye lenses are considered on an equal footing with the others...

Foreword

For simplification and clarity of the present article, the photographic images are all assumed to be perfectly rotationally symmetric and produced by refractive optics. In practice and for the vast majority of photographic and panoramic (i.e. non-photogrammetric) purposes, this assumption is nowadays probably valid. Furthermore, the center of the rectangular image capture device (e.g. the digital sensor) is hereafter supposed to be on the lens optical axis. As a matter of fact this is not always true: a slight de-centering may exist even on high-end photographic gears. It is often left because this defect is of no real consequence in casual photography and is not considered really as a flaw. The possible decentering shall however be taken into account and possibly digitally corrected during the optimisation phase of panorama imaging (stitching) workflow.

In this article, is fundamentally assumed the existence of a single and fixed point linking the object space and the image side through which the incoming light rays converge (BTW it is called Perspective Center in the classical perpective projection). This simplification models (in the simplest possible manner) the complex path resulting from refraction in the lens. This important approximation is certainly valid when no aperture and no focus distance setting is involved and for instance when the object point of interest is located at a "certain" distance away from the lens. Shift of the entrance pupil depending on incoming light angle, multiple perspective points camera systems, non-central camera systems and slit-camera systems are for instance not considered here.

A qualitative and empirical description of the various classical lens projections was presented on this page.

Advanced projection modeling: this other page is a complement to the present article. Many ways to model the lenses have been proposed and developed since the early 1990's to fit the needs of 3D scene modeling,, etc,. In particular, parametric polynomial models (under many very various forms) are currently very efficiently used for instance, for photographic lenses evaluation, graphic computer vision, robotics engineering , panoramic imaging (e.g... panorama stitching), etc. These various ways have the advantage of allowing possible generalization of model description amongst at least all refractive lenses.

 

Plotting lens projections (with a bit of trigonometry)

The underlying principle

 

 

The classical lens projections

The 5-model classical taxonomy is without any doubt purely conventional (that is to say: arbitrary) but nevertheless it is a very bright simplification that has the advantage of nicely matching each of the models with various simple mathematical formulas. It was invented to clearly differentiate among themselves some isolated typical photographic lens "concepts". This has become a de-facto Standard.

Applying the principle illustrated above on five famous specific cases, we then get the following plots:

 

Using the same convention for the colors of the graph, the basic graph above is equivalent to those that follow :

1

Plotting the 5 various theoretical "classical" projections

These plots illustrate the distance r (unit = "f ") extending from the center of the image to the Image point (in the image plane) as a function of the angle θ measured from the lens longitudinal axis to the corresponding point of interest (in the object space).

  • The formula for the "Standard" (a.k.a. Central Perspective or Rectilinear) undistorted projection is r = f tan θ.

Most of the photographic lenses belong to this first type of projection that render straight lines in the object space as straight lines on the image plane... even though many are more or less affected of geometrical aberration such as distortion (i.e., barrel, pin-cushion or "mustache").

 

Fisheye lenses are often used for shooting multiple images that shall be stitched into a spherical panorama. There are various kinds of optical construction but the trigonometric formulas for fisheye image projections are most often arbitrarily defined by verbal expressions which correspond to math functions. Such classical categories are:

  • Stereographic projection: r = 2f tan (θ/2)
  • Equidistance (a.k.a. equiangular) projection: r = f θ
  • Equisolid angle (a.k.a. Equal Area) projection: r = 2f sin (θ/2)
  • Orthographic (a.k.a. Sine-law) projection: r = f sin (θ)

 

 

 

 

 

 

 

2

Some real lenses: to what projection type do they belong?

Remarks concerning the radial mapping of the fisheye lenses defined above:

Q: Are there any possible math radial mapping functions for fisheye lenses other than the classical models?

R: Of course, for instance the empty space between curves on the plot area of the charts can be filled with other curves that would describe intermediate "no-name" models!

We should note that while back in the 1960's one could find some very informative technical literature about special optical concepts, most of them were rarely actually constructed. In any case, fisheye lenses in general were seldom used to shot other things than scientific still photographs... until digital technologies came and changed both the playground and the rules of the game in the 1990's....The simplified taxonomy previously presented and comprising only five categories could naturally be completed: there is no reason to limit the whole graph plane surface [r; θ] to some discreet plots only. The plot area could be continuous or with smooth transitions. For example, three supplementary plots were added to the "classical five" on this following chart: we might call them (from left to right on the chart): hefty Pin-cushion, Barrel distortion and Sub-Stereographic:)

 

 

Some real lenses were experimentally measured. Here follow the r= f(θ) plots

These lenses are popular amongst the panorama photographers community.

Important notes:

  1. Along the vertical coordinate axis is the distance frome the center, but unlike for the others plots above, it's not divided by the focal length!
  2. The horizontal axis (θ) spans from negative to positive for simplification of the polar presentation: the angle of incidence should not have any sign as there is a rotational symmetry (instead, a supplementary polar angle Ψ allows to complete the whole object sphere)

Other ways to present the 5 classical projections

In addition we can use new ways to draw diagrams representing classical projections:

Radial distance r function of incident light angle θ
Radial distance deviation with respect to an arbitrary projection, along the image radius

This is the regular and most common presentation with r= function of θ.

This presentation is done the usual way: It's the most frequently found in the scientific literature for illustrating the "Variety of lens Projections". The five different classical non-parametric projections are presented hereabove as a function of the incoming angle of light rays entering the lens.

With a straight line (in black) at the middle of all the others, the Equidistant projection may look as being "middle register" among all the projections... but tradition, human vision and good sense all tell us that Rectilinear projection is de-facto the normal ruling Standard. Remember: Straight lines must be straight!

θ= π/2 (90° assymptotical vertical line) is the hard limit that separates the rectilinear world from the fisheye world, but unlike its siblings, the orthographic projection can't cross that border line: π/2 is indeed the ultimate possible incident angle limit. The horizontality of the slope of the curve near the end denotes a hefty compression affecting the image near the circular boundary ...

The radial deviation (divided by the focal length) from the ideal fisheye Equidistant projection (r= f *θ) is reported for each of the five classical projections along the radius of the image (and divided by the focal length):

One can clearly see here how the classical projection taxonomy tends to isolate a group ("Fisheye(s)") from the sole perspective projection.

This observation is important: the ideal projection that is aimed at for correcting images may need to be different for the two families:

  • perspective (distortion correction)
  • fisheye (image projection conversion)

A different correction model may have to be used when dealing with these two different occasions.

The radial deviation (divided by the focal length) from the ideal Rectilinear projection (rd= f *θ) is here reported for each of the five projections along the radius of the image (and divided by the focal length).

This illustrates the dominant way for optimizing the panoramic stitching of fisheye images.

This comes probably because the "sweet spot" (closed to the horizontal coordinate axis) for the performance (accuracy and stability) of panoramic software when having to stitch fisheye images to render a full spherical panorama....

... Fortunately most of the fisheyes lenses are of this kind (Equisolid angle). On the contrary, one may expect higher RMS Control Points distance errors when using fisheye lenses that are away from the "ideal" projection (Equidistant).

 

Remarks about focal length

Diagonal angle of view (on APS-C and Full Frame sensors) function of the focal length

These plots presents the theoretical angle of view that can ultimately be captured along the diagonal of the digital sensor. The image from a real-life camera is generally limited by optical vignette within the lens structure and the FoV doesn't go much over 180° to 195° depending on the projection type.

Other topics

Q: Is "Distortion" affecting fisheye?

R: In my humble opinion, No, but this can be debated. Distortion should not be a word applied without supplementary explanation to fisheye lenses eventhough this is done in the majority of the related literature (and I had to do it also a few time too, just to make sure to be understood). Distortion of fisheye has often annegative aesthetic meaning to the casual photographer, but we should explain him that this "distortion" is absolutely feature is absolutely done on purpose and is quite a powerfull tool.

Yet the word "Distortion" is officially defined for metering of a geometric aberration that negatively affects the "Standard" (i.e. rectilinear) lenses. The fisheye lenses are deliberately designed to produce images with bent lines, when these lines would have been straight with Standard lenses. Excepting for some non-photographic purpose there is nothing "wrong" with bent edges and lines on the planar image. And on the positive side they can yield very wide Angle of field that is impossible to get with Standard lenses. The fisheye projections deviate from the rectilinear projection (or vice-versa). Deviation is not distortion though.

In the official ISO 9039:2008(E) << Optics and photonics -Quality evaluation of optical systems -Determination of distortion >>, one can read in the introduction: Generally, the function of rotationally symmetric optical systems is to form an image that is geometrically similar to the object, except for some particular systems, such as fish-eye lenses and eyepieces, where this condition is deliberately not maintained. Ideally, this function is accomplished according to the geometry of perspective projection. Departures from the ideal image geometry are called disortion. (...).This text deliberately avoids to formally forbid or to discourage of calling "distortion" when fisheye lenses are concerned and this is IMO a pity: while it is obvious that using the official Standard definition of "distortion" for a fisheye lens is a non-sense, this has been (and still is) often done, including in many recent famous Patent documents. IMO "Distortion" should be explicitely restricted to Rectilinear lenses.

Well, when all this is said, the matter becomes sometimes not so simple. Having a firm and definitive opinion and judgement is not always feasible. Lets take an example for demonstration: is an image shot with a fisheye lens set at a long focal (say 28 mm with a smc Pentax 1/3.5-4.5; 17-28mm for instance) on a cropped camera sensor very different from an image from a 28 mm rectilinear lens affected with barrel distortion and mounted on a full frame camera? I must admit that under such conditions, the difference can be tenuous...

Q: Can a real fisheye lens be affected with intricate "deviation from Standard" too ?

R: Yes. Like Standard rectilinear lenses with "distortion", a fisheye lense might well be affected with intricate radial mapping strongly deviating from any simple "Classical" fisheye model.

Many Rectilinear lenses are affected with considerable distortion. This geometrical aberration can be of the barrel type (Wide Angle lenses are mostly concerned). It could alternatively be pin-cushion distortion (Tele-lenses are mostly concerned), but one can also frequently observe intricate mixed distortion ( i.e. "mustache" distortion) on images shot with many modern Very Wide Angle rectilinear lenses. There are no reason to assume that the same possible complexity that is observed on Standard lenses would not be existing for fisheye lenses too, especially with the recent introduction of aspherical lens elements in some designs.... Such an intricate mapping cannot be modeled by a "simple" classical trigonometric function and high degree polynomial functions may thus better fit. But that is another story.

BTW and again about Distortion "Standards": they in fact are all applying a simplistic linear approximation to an unique trio of pixels near the edge of the Distorted image, these current "Standard" methods cannot even provide a correct information in the case of "mustache" a.k.a "wavy" distortion for rectilinear lenses.

 

References and Links:

Perhaps the most provocative yet educated point of view about lens projections :" Perspective Projection: The Wrong Imaging Model" was written in 1995 by Margaret M. Fleck Technical report 95-01.

The wonderful page also about Nikkor fisheye lenses by Pierre Toscani.

A very good study about Fisheye lenses by James Kumler and Martin Bauer.

An other chart (PDF) by this author: Radial mapping of some fisheye lenses.