a2) United States Patent
`US 6,844,990 B2
`(10) Patent No.:
`Jan. 18, 2005
`(45) Date of Patent:
`Artonneet al.
`
`US006844990B2
`
`(54) METHOD FOR CAPTURING AND
`DISPLAYING A VARIABLE RESOLUTION
`DIGITAL PANORAMIC IMAGE
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4/1976 Fisheretal.
`3,953,111 A
`3/1999 Ishii etal.
`5,880,896 A
`2/2000 Inoue
`6,031,670 A
`6,333,826 B1 * 12/2001 Charles... 359/725
`6,449,103 Bl *
`9/2002 Charles ........ ee 359/725
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`WO
`
`0 695 085 Al
`1 004 915 Al
`WO 00/42470 Al
`
`1/1996
`5/2000
`7/2000
`
`* cited by examiner
`
`Primary Examiner—Scott J. Sugarman
`(74) Attorney, Agent, or Firm—Akin GumpStrauss Hauer
`& Feld, LLP
`
`(57)
`
`ABSTRACT
`
`A method for capturing a digital panoramic image includes
`projecting a panorama onto an image sensor by meansof a
`panoramic objective lens. The panoramic objective lens has
`a distribution function of the image points that is not linear
`relative to the field angle of the object points of the
`panorama,suchthat at least one zone of the image obtained
`is expanded while at least another zone of the image is
`compressed. When a panoramic image obtained is then
`displayed, correcting the non-linearity of the initial imageis
`required and is performed by meansofa reciprocal function
`of the non-linear distribution function of the objective lens
`or by means of the non-linear distribution function.
`
`26 Claims, 11 Drawing Sheets
`
`(75)
`
`Inventors: Jean-Claude Artonne, Montreal (CA);
`Christophe Moustier, Marseilles (FR);
`Benjamin Blanc, Montreal (CA)
`
`(73) Assignee: 6115187 CanadaInc., Saint Laurent
`(CA)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`US.C. 154(b) by 0 days.
`
`(21) Appl. No.: 10/706,513
`
`(22)
`
`(65)
`
`63)
`
`(30)
`
`Filed:
`
`Nov. 12, 2003
`
`Prior Publication Data
`US 2004/0136092 Al Jul. 15, 2004
`
`Related U.S. Application Data
`
`PP
`Continuation of application No. PCT/FR02/01588, filed on
`May 10, 2002.
`
`Foreign Application Priority Data
`
`May 11, 2001
`
`(FR)
`
`cesesscsscsssessessecsstsssesssesseeseseees O1 06261
`
`Int. C1? oe G02B 13/06; GO2B 13/18
`(51)
`(52) U.S. Cd. cc ecccecetetecceeeseeceeeneeenes 359/725; 359/718
`(58) Field of Search oe 359/718, 719,
`359/725, 728
`
`
`
`APPLE 1001
`
`APPLE 1001
`
`1
`
`

`

`Jan. 18, 2005
`
`Sheet 1 of 11
`
`US 6,844,990 B2
`
`U.S. Patent
`
`
`
`
`
`2
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 2 of 11
`
`US 6,844,990 B2
`
`a
`
` 0
`
`10 20 30 40 50 60 70 80 90 Fig. 4B
`
`3
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 3 of 11
`
`US 6,844,990 B2
`
`Fig.5
`
`PRIOR ART
`
`se*a”
`
`“““7
`
`
`
`™\ pdl(dr=0.22)
`
`
`
`4
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 4 of 11
`
`US 6,844,990 B2
`
`Pdl(dr=0.777)
`
`Fd2
`
`Pd(dr=0.3)
`
`dr
`
`0.3
`
`
`
`70°
`
`90°
`
`
`
`
`
`30°
`
`70°
`
`90°
`
`5
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 5 of 11
`
`US 6,844,990 B2
`
`
`
`6
`
`

`

`U.S. Patent
`
`Sheet 6 of 11
`
`Jan. 18, 2005
`
`US 6,844,990 B2
`
`7
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 7 of 11
`
`US 6,844,990 B2
`
`$1 - Acquisition
`
`- Taking a panoramic image by meansofa still digital camera
`or a digital video camera equipped with a panoramic lens
`having a non-linear distribution function Fd
`
`S2 — Transfer of the imagefile into a computer
`
`(image
`
`disk)
`
`into
`
`a
`
`shrinking/expanding the image sector displayed)
`
`- Transfer of
`microcomputer
`- Storage in the auxiliary storage (optional)
`
`the
`
`image
`
`file
`
`S3 -Linearisation of the image disk
`
`- Transfer of the image points of the initial image disk into a
`second virtual image disk comprising more image points than
`the initial image disk, by meansofthe function Fd
`Obtaininga linear image disk
`
`S4 — Digitisation
`
`- Transfer of the image points of the second image disk into a
`system of axes OXYZ in spherical coordinates Obtaining a
`panoramic image in a hemisphere
`
`35 ~ Interactive display
`
`- Determination of the image points of an image sector to be
`displayed
`- Display of the image sector on a display window
`- Detection of the user’s actions on a screen pointer or any
`other control means,
`- Detection of the user’s actions on keys
`enlargement,
`- Modification of the sector displayed (sliding the image
`sector displayed on the surface of the hemisphere and/or
`
`image
`
`for
`
`8
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 8 of 11
`
`US 6,844,990 B2
`
`
`
`p(pu, pv)
`
`P(px, Py, P2)
`
`(i,j)
`
`
`
`DW
`
`HS
`
`9
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 9 of 11
`
`US 6,844,990 B2
`
`Fig. 14
`
`S1 — Acquisition
`
`- Taking a panoramic image by meansofa still digital camera
`or a digital video camera equipped with a panoramic lens
`having a non-lineardistribution function Fd
`
`S2 — Transfer of the imagefile into a computer
`
`image
`
`file
`
`(image
`
`disk)
`
`into
`
`a
`
`and/or shrinking/expanding the image sector)
`
`- Transfer of
`microcomputer
`- Storage in the auxiliary storage (optional)
`
`the
`
`S3' — Interactive display with implicit correction of the
`non-linearity of the initial image
`
`A - Determination of the colour of the points E(i, j) of an
`image sector to be displayed using the points p(pu, pv) of the
`image disk:
`
`l- determination of the coordinates Ex, Ey, Ez in the
`coordinate system OXYZ ofeach point E(i, j) of the sector to
`be displayed,
`2- determination of the coordinates Px, Py, Pz of points P of
`the hemisphere correspondingto the points EQ,j),
`3- calculation of the coordinates, in the coordinate system
`O'UVofthe imagedisk, of the points p(pu, pv) corresponding
`to the points P of the hemisphere, by means of the function
`Fd,
`
`B - Presentation of the imagesector in a display window,
`C - Detection of the user’s actions on a screen pointer or any
`other control means,
`D - Detection ofthe user’s actions on enlargement keys,
`E - Modification of the image sector displayed (moving
`
`10
`
`10
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 10 of 11
`
`US 6,844,990 B2
`
`SS
`TAN
`
`
`WEEP
`
`
`N
`
`—
`OY WS
`
`
`
`11
`
`

`

`U.S. Patent
`
`Jan. 18, 2005
`
`Sheet 11 of 11
`
`US 6,844,990 B2
`
`40
`
`?
`
`Mi
`
`Ly
`
`Le
`
`L3
`
`43
`
`Fig. 18
`
`M2
`
`12
`
`12
`
`

`

`US 6,844,990 B2
`
`1
`METHOD FOR CAPTURING AND
`DISPLAYING A VARIABLE RESOLUTION
`DIGITAL PANORAMIC IMAGE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of International Appli-
`cation No. PCT/FR02/01588, filed May 10, 2002 the dis-
`closure of which is incorporated herein by reference.
`BACKGROUND OF THE INVENTION
`
`10
`
`15
`
`20
`
`30
`
`35
`
`2
`central point O' of the system of axes O'X"Y'Z, which defines
`with the center O" of the image sector 12, a direction O'O"
`called “viewing direction”.
`In order to avoid the image sector displayed 12 having
`geometrical distortions unpleasant
`for the observer,
`the
`classical panoramic objective lenses must have a distribution
`function of the image points according to the field angle of
`the object points of a panoramathat is as linear as possible.
`Therefore,if two points A’, B', situated on the same meridian
`of the hemisphere 11, and the corresponding points A, B on
`the image disk 10 are considered,
`the ratio between the
`angles (A'O'Z) and (B'O'Z) must be equal
`to the ratio
`The present invention relates to obtaining digital pan-
`between the distances OA and OB onthe image disk.
`oramic images and displaying panoramic images on com-
`puter screens.
`Dueto this property of linearity of a classical panoramic
`objective lens, image points corresponding to object points
`FIG. 1 represents a classical device allowing a digital
`having an identical field angle form concentric circles C10,
`panoramic image to be produced and presented on a com-
`C20... . C90 on the image disk 10, as represented in FIG.
`puter screen. The device comprises a digital camera 1
`4A.Classically, “field angle of an object point” means the
`equipped with a panoramic objective lens 2 of the “fish-eye”
`angle of an incident light ray passing through the object
`type, having an angular aperture on the order of 180°. The
`point considered and through the center of the panorama
`camera 1 is connected to a computer 5, such as a micro-
`photographed, relative to the optical axis of the objective
`computer for example, equipped with a screen 6. The
`lens. The field angle of an object point can be between 0 and
`connection to the microcomputer 5 may be permanent,
`90° for an objective lens having an aperture of 180°.
`when, for example, the camera 1 is a digital video camera,
`Therefore,
`the circle C10 is formed by the image points
`25
`or temporary, when, for example, the camera 1 isastill
`corresponding to object points having a field angle of 10°,
`digital camera equipped with an image memory, the con-
`the circle C20 is formed by image points corresponding to
`nection then being carried out at the time the imagefiles are
`object points having a field angle of 20°, etc., the circle C90
`to be transferred into the microcomputer.
`being formed by the image points having a field angle of
`FIG. 2 schematically represents the appearance of a
`90°.
`panoramic image 3 obtained by means of the panoramic
`FIG. 4B represents the shape of the distribution function
`objective lens 2. The round appearance of the image is
`Fde of a classical panoramic objective lens, which deter-
`characteristic of the axial symmetry of panoramic objective
`minesthe relative distance dr of an image point in relation
`lenses and the imagehas dark edges 4 that will subsequently
`to the center of the image disk according to the field angle
`be removed. This digital panoramic image is delivered by
`ax of the corresponding object point. The relative distance dr
`the camera 1 in the form of a computerfile containing image
`is between 0 and 1 andis equal to the distance of the image
`points coded RGBAarranged in a two-dimensional table,
`point in relation to the center of the image divided by the
`“R” being the red pixel of an image point, “G” the green
`radius of the image disk. The ideal form of the function Fde
`pixel, “B” the blue pixel, and “A” the Alpha parameter or
`is a straight line of gradient K:
`transparency. The parameters R, G, B, A are generally being
`coded on8bits.
`40
`dr=Fdc(a)=Ka
`
`The image file is transferred into the microcomputer 5
`which transformsthe initial image into a three-dimensional
`digital image, then presents the user with a sector of the
`three-dimensional image in a display window 7 occupying
`all or part of the screen 6.
`FIG. 3 schematically showsclassical steps of transform-
`ing the two-dimensional panoramic image into a panoramic
`imageoffering a realistic perspective effect. After removing
`the black edges of the image, the microcomputerhas a set of
`image points forming an image disk 10 of center O and axes
`OX and OY. The image points of the image disk are
`transferred into a three-dimensional space defined by an
`orthogonal coordinate system of axes O'X'Y'Z, the axis O'Z
`being perpendicular to the plane of the image disk. The
`transfer is performed by a mathematical function imple-
`mented by an algorithm executed by the microcomputer, and
`leads to obtaining a set of image points referenced in the
`coordinate system O'X'Y'Z. These image points are for
`example codedin spherical coordinates RGBA(9,8), » being
`the latitude and 0 the longitude of an imagepoint. The angles
`» and @ are coded in 4 to 8 bytes (IEEE standard). These
`image points form a hemisphere 11 when the panoramic
`objective lens used has an aperture of 180°, otherwise a
`portion of a hemisphere. The microcomputer thus has a
`virtual image in the shape of a hemisphere onesector 12 of
`which, corresponding to the display window 7,is presented
`on the screen (FIG. 1) considering that the observer is on the
`
`45
`
`50
`
`55
`
`60
`
`65
`
`in which the constant K is equal to 0.111 degree? (1/90°).
`This technique of displaying a digital panoramic image
`sector on a computer screen has various advantages, par-
`ticularly the possibility of “exploring” the panoramic image
`by sliding the image sector presented on the screen to the
`left, the right, upwards or downwards,until the limits of the
`panoramic image are reached. This technique also allows
`complete rotations of the image to be carried out when two
`complementary digital images have been taken and supplied
`to the microcomputer, the latter thus reconstituting a com-
`plete panoramic sphere by assembling two hemispheres.
`Another advantage provided by presenting a panoramic
`image on screen is to enable the observer to make enlarge-
`ments or zooms on parts of the image. The zooms are
`performeddigitally, by shrinking the image sector displayed
`and expanding the distribution of the image points on the
`pixels of the screen.
`Various examplesof interactive panoramic images can be
`found on the Web. Reference could be made in particular to
`the central site “http://www.panoguide.com”(“The Guide to
`Panoramas and Panoramic Photography’) which gives a
`full overview of all the products available to the public to
`produce these images. Software programs allowing digital
`panoramic photographs to be transformed into interactive
`panoramic imagesare offered to the public in the form of
`downloadable programs or CD-ROMsavailable in stores.
`
`13
`
`13
`
`

`

`US 6,844,990 B2
`
`3
`Despite the various advantages that this technique for
`displaying digital images offers,
`the digital enlargements
`have the disadvantage of being limited by the resolution of
`the image sensor used whentaking the initial image and the
`resolution of an image sensor is generally much lower than
`that of a classical photograph. Therefore, when the enlarge-
`mentincreases, the granulosity of the image appears as the
`limits of the resolution of the image sensor are being
`reached.
`is well known to
`it
`To overcome this disadvantage,
`proceed with pixel interpolations so as to delay the appari-
`tion of the blocks of color which betray the limits of the
`resolution of the sensor. However,
`this method only
`improves the appearance of the enlarged image sector and
`does not in any wayincrease the definition. Another obvious
`solution is to provide an image sensorwith a high resolution,
`higher than the resolution required to present an image
`sector without enlargement, so that there is a remaining
`margin of definition for zooms. However, this solution is
`expensive as the cost price of an image sensorrapidly rises
`with the number of pixels per unit of area.
`Some attempts have been made to improve the quality of
`the enlargements, by changing the optical properties of the
`panoramic objective lenses themselves. ‘hus, U.S. Pat. No.
`5,710,661 teaches capturing a panoramic image with two
`overlocking objective lenses using a set of mirrors. A first set
`of mirrors provides an overall view, and a mobile central
`mirror provides a detailed view on a determined zoneof the
`panorama. However, this solution does not offer the same
`flexibility as digital zooms, particularly when the image is
`not displayed in real time, as the observer no longer has the
`possibility of choosing the image portion that he wants to
`enlarge once the photograph has been taken.
`BRIEF SUMMARY OF THE INVENTION
`
`invention comprises a method
`the present
`Therefore,
`allowing the physical limits of image sensors to be circum-
`vented and the definition offered by digital enlargements
`concerning certain parts of a digital panoramic image to be
`improved, without the need to increase the numberof pixels
`per unit of area of an image sensor or to provide an
`overlooking optical enlargement system in a panoramic
`objective lens.
`The present invention is based on the observation that, in
`several applications, only certain zones of a panoramic
`image are of a practical
`interest and are likely to be
`expandedby the observer by meansof a digital zoom. Thus,
`in applications such as video surveillance,
`videoconferencing, visio-conferencing, a panoramic camera
`can be installed against a wall or on the ceiling and there is
`generally no reason to make enlargements on the zones of
`the panoramic image corresponding to the wall or
`the
`ceiling. Similarly, as part of a videoconference performed by
`means of a panoramic camera, the most interesting zone is
`generally situated at a specific place situated towards the
`center of the image (in the case of individual use) or on the
`edges of the image (in the case of collective use or visio-
`conferencing). Furthermore, when used for recreation and
`leisure, most panoramic images comprise parts that are less
`interesting than others, such as the parts representing the sky
`or a ceiling for example, the most useful part generally being
`in the vicinity of the center of the image.
`Therefore, the present invention is based on the premise
`that a panoramic image has some zones that are not very
`useful and that can tolerate a reasonable definition to the
`benefit of other zones of the image.
`On the basis of this premise,
`the idea of the present
`invention is to produce panoramic photographs by meansof
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`a panoramic objective lens that is not linear, which expands
`certain zones of the image and compressesother zonesof the
`image. The technical effect obtained is that the expanded
`zones of the image cover a numberof pixels of the image
`sensorthat is higher than if they were not expanded, and thus
`benefit from a better definition. By choosing an objective
`lens that expands the most useful zones of an image (which
`depend on the intended application), the definition is excel-
`lent in these zones and the definition is mediocre in the zones
`
`of lesser importance.
`Thus, the present invention proposes a method for cap-
`turing a digital panoramic image, by projecting a panorama
`onto an image sensor by means of a panoramic objective
`lens, in which the panoramic objective lens has an image
`point distribution function that is not linear relative to the
`field angle of object points of the panorama,the distribution
`function having a maximum divergence of at least +10%
`compared to a linear distribution function, such that the
`panoramic image obtained has at least one substantially
`expanded zone and at least one substantially compressed
`zone.
`
`According to one embodiment, the objective lens has a
`non-linear distribution function that is symmetrical relative
`to the optical axis of the objective lens, the position of an
`image point relative to the center of the image varying
`according to the field angle of the corresponding object
`point.
`According to one embodiment, the objective lens expands
`the center of the image and compresses the edges of the
`image.
`According to one embodiment, the objective lens expands
`the edges of the image and compresses the center of the
`image.
`According to one embodiment, the objective lens com-
`presses the center of the image and the edges of the image,
`and expands an intermediate zone of the image located
`between the center and the edges of the image.
`According to one embodiment, the objective lens com-
`prises a set of lenses forming an apodizer.
`According to one embodiment, the set of lenses forming
`an apodizer comprises at least one aspherical lens.
`According to one embodiment, the set of lenses forming
`an apodizer comprises at least one diffractive lens.
`According to one embodiment, the objective lens com-
`prises a set of mirrors comprising at least one distorting
`mirror.
`
`The present invention also relates to a method for dis-
`playing an initial panoramic image obtained in accordance
`with the method described above, comprising a step of
`correcting the non-linearity of the initial image, performed
`by means of a reciprocal function of the non-linear distri-
`bution function of the objective lens or by means of the
`non-linear distribution function.
`
`the step of correcting
`According to one embodiment,
`comprises a step of transforming the initial image into a
`corrected digital
`image comprising a number of image
`points higher than the numberofpixels that the image sensor
`comprises.
`According to one embodiment, the method comprises a
`step of calculating the size of the corrected image, by means
`of the reciprocal function of the distribution function, so that
`the resolution of the corrected image is equivalent to the
`most expanded zone of the initial image, and a step of
`scanning each image point of the corrected image, searching
`for the position of a twin point of the image point on the
`
`14
`
`14
`
`

`

`US 6,844,990 B2
`
`5
`initial image and allocating the color of the twin point to the
`image point of the corrected image.
`According to one embodiment, the initial image and the
`corrected image comprise an image disk.
`According to one embodiment, the method comprises a
`step of transferring the image points of the corrected image
`into a three-dimensional space and a step of presenting one
`sector of the three-dimensional image obtained on a display
`means.
`
`According to one embodiment, the method comprises a
`step of determining the color of image points of a display
`window, by projecting the image points of the display
`window onto the initial image by means of the non-linear
`distribution function, and allocating to each image point of
`the display window the color of an image pointthat is the
`closest on the initial image.
`the projection of the
`According to one embodiment,
`image points of the display window onto the initial image
`comprises a step of projecting the image points of the
`display window onto a sphere or a sphere portion, a step of
`determining the angle in relation to the center of the sphere
`or the sphere portion of each projected image point, and a
`step of projecting onto the initial image each imagepoint
`projected onto the sphere or the sphere portion, the projec-
`tion being performed by meansofthe non-linear distribution
`function considering the field angle that each point to be
`projected has in relation to the center of the sphere or the
`sphere portion.
`‘The present invention also relates to a panoramic objec-
`tive lens comprising optical means for projecting a pan-
`orama into an image plane of the objective lens, the pan-
`oramic objective lens having an image point distribution
`function that is not linearrelativeto the field angle of object
`points of the panorama,the distribution function having a
`maximum divergenceof at least +10% comparedto a linear
`distribution function, such that a panoramic image obtained
`by means of the objective lens comprises at
`least one
`substantially expanded zone and at least one substantially
`compressed zone.
`According to one embodiment, the panoramic objective
`lens has a non-linear distribution function thal is symmetri-
`cal relative to the optical axis of the objective lens,
`the
`position of an imagepoint relative to the center of an image
`obtained varying according to the field angle of the corre-
`sponding object point.
`According to one embodiment, the panoramic objective
`lens expands the center of an image and compresses the
`edges of the image.
`According to one embodiment, the panoramic objective
`lens expands the edges of an image and compresses the
`center of the image.
`According to one embodiment, the panoramic objective
`lens compresses the center of an image and the edges of the
`image, and expands an intermediate zone of the image
`located between the center and the edges of the image.
`According to one embodiment, the panoramic objective
`lens comprises a set of lenses forming an apodizer.
`According to one embodiment, the set of lenses forming
`an apodizer comprises at least one asphericallens.
`According to one embodiment, the set of lenses forming
`an apodizer comprises at least one diffractive lens.
`According to one embodiment, the panoramic objective
`lens comprises polymethacrylate lenses.
`According to one embodiment, the panoramic objective
`lens comprises a set of mirrors comprising at least one
`distorting mirror.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`6
`BRIEF DESCRIPTION OF THE SEVERAL
`VIEWS OF THE DRAWINGS
`
`The foregoing summary, as well as the following detailed
`description of preferred embodiments of the invention, will
`be better understood when read in conjunction with the
`appended drawings. For the purpose of illustrating the
`invention, there are shown in the drawings embodiments
`which are presently preferred.
`It should be understood,
`however, that the invention is not limited to the precise
`arrangements and instrumentalities shown.
`In the drawings:
`FIG. 1 described above represents a system for displaying
`a digital panoramic image on a screen;
`FIG. 2 described above represents a panoramic image
`before it is processed by a computer;
`FIG. 3 described above shows a classical method for
`transforming a two-dimensional panoramic image into a
`three-dimensional digital panoramic image;
`FIGS. 4A and 4B described above showthe linearity of a
`classical panoramic objective lens;
`FIGS. 5 and 6 show one aspect of the method according
`to the present invention and respectively represent a distri-
`bution of image points obtained with a classical panoramic
`objective lens and a distribution of image points obtained
`with a non-linear panoramic objective lens according to the
`present invention;
`FIGS. 7A and 7B showa first example of non-linearity of
`a panoramic objective lens according to the present inven-
`tion;
`FIG. 8 shows a second example of non-linearity of a
`panoramic objective lens according to the present invention;
`FIG. 9 shows a third example of non-linearity of a
`panoramic objective lens according to the present invention;
`FIG. 10 represents a system for displaying a digital
`panoramic image by meansof which a methodfor correcting
`the panoramic image according to the present invention is
`implemented;
`FIG. 11 schematically shows a first embodiment of the
`correction method according to the present invention;
`FIG. 12 is a flow chart describing a method for displaying
`a panoramic image incorporating the first correction method
`according to the present invention;
`FIG. 13 schematically shows a second embodimentofthe
`correction method according to the present invention;
`FIG. 14is a flow chart describing a method for displaying
`a panoramic image incorporating the second correction
`method according to the present invention;
`FIG. 15 is a cross-section of a first embodiment of a
`
`non-linear panoramic objective lens according to the present
`invention;
`FIG. 16 is an exploded cross-section of a system of lenses
`present in the panoramic objective lens in FIG. 15;
`FIG. 17 is a side view ofa lens present in the panoramic
`objective lens in FIG. 15; and
`FIG. 18 is the diagram of a second embodiment of a
`non-linear panoramic objective lens according to the present
`invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`A—Compression/Expansion of an Initial Image
`FIG. 5 schematically represents a classical system for taking
`panoramic shots, comprising a panoramic objective lens 15
`
`15
`
`15
`
`

`

`US 6,844,990 B2
`
`8
`and compressing another part of the image, the part to be
`expanded and the part to be compressed can be chosen
`according to the intended application, by producing several
`types of non-linear objective lenses and by choosing an
`objective lens suited to the intended application. Depending
`on the intended application,
`the most useful part of a
`panoramic image may belocated in the center of the image,
`on the edge of the image, in an intermediate zone situated
`between the center and the edge of the image, etc.
`FIGS. 7A-7B, 8 and 9 show three examples of non-linear
`distribution functions according to the present invention.
`The distribution function shown in FIGS. 7A and 7B
`
`10
`
`7
`of optical axis OZ and a digital image sensor 17 arranged in
`the image plane of the objective lens 15. Here, four object
`points a, b, c, d will be considered that belong to a panorama
`PM located opposite the objective lens and respectively
`having angles of incidence a1, a2, -a2, -a1. As explained
`in the preamble, the field angle of an objectpointis the angle
`that an incident light ray passing through the object point
`considered and through the center of the panorama PM,
`marked by a point “p” on FIG. 5, hasrelative to the optical
`axis OZ of the objective lens. In this example, the angle a1
`is equal to two times the angle «2. On the image sensor 17,
`image points a’, b', c', d' corresponding to the object points
`a, b, c, d are located at distances from the center of the image
`respectively equal to d1, d2, -d2, -d1. As the distribution of
`the image points according to the field angle of the object
`points is linear with a classical panoramic objective lens, the
`distances d1 and d2 are linked by the followingrelation:
`
`dlfol=d2/a2
`
`corresponds to the example in FIG. 6, that is a panoramic
`objective lens that expands the imagein the center. FIG. 7A
`represents equidistant concentric circles C10, C20, ..., C90
`present on an imagedisk, each circle being formed by image
`points corresponding to object points having the samefield
`angle. The circle C10 is formed by the image points corre-
`sponding to object points having a field angle of 10°, the
`circle C20 is formed by image points corresponding to
`As the angle a1 is here equal to 202, it follows that:
`object points havingafield angle of 20°, etc. By comparing
`di=2d2
`FIG. 7A with FIG. 4A described in the preamble, it appears
`that the circles C10 and C20 are further from the center of
`
`15
`
`20
`
`As is well known by those skilled in the art, the term
`“linearity” here refers to a ratio of proportionality between
`the distance of an image point measured relative to the
`center of the image andthefield angle of the corresponding
`object point. The notion of “linearity” in the field of pan-
`oramic objective lenses is therefore different from that
`prevailing in the field of paraxial optics (in the vicinity of the
`optical axis) when the conditions of Gauss are met.
`function, expressed by Fdc=Ka andin the form ofa straight
`line of gradient K,is also represented as a guide mark (the
`FIG. 6 represents a system for taking shots of the same
`constant K being equal to %o for an objective lens having an
`type as above, but in whichthe classical panoramic objective
`aperture of 180°, i.e., a gradient of 0.111 per degree). The
`lens 15 is replaced by an objective lens 18 according to the
`field angle a of the object points is represented on the X-axis
`present invention, the image sensor 17 being arranged in the
`and is between 0 and 90°. The relative distance dr of an
`imageplane of the objective lens 15. The projection onto the
`image point in relation to the center of the image disk is
`image sensor 17 of the object points a, b, c, d having angles
`of incidence a1, a2, -a2 and -a.1 relative to the axis OZ of
`represented on the Y-axis and is between 0 and 1. The curve
`of the function Fd1 has a higher gradient than the straight
`the objective lens and to the center “p” of the panoramaare
`line Fde for angles a of between 0 and 20°, then a lesser
`considered again. On the image sensor17, the corresponding
`gradient after 20° and up to 90°. A high gradient means an
`image points a", b", c", d" are located at distances from the
`expansion of the image and a low gradient means a com-
`center of the image respectively equal to d1', d2', -d2', -d1'.
`pression of the image.
`According to the present invention, the objective lens 18
`As demonstrated in this example, the curve Fdl has a
`has a distribution function of the image points that is not
`linear. The ratio of the distances d1', d2', -d2', -d1' are not
`point of maximum divergence Pd at the angle a=20°. “Point
`of maximum divergence”refers to the image point Pd(a) at
`equal to the ratio of the angles of incidence a1, a2, -a2,
`which the greatest gap in relative distance dr in relation to
`-a1. In the example represented, the distance d2' is clearly
`a corresponding point Pdl(a) on the linear distribution
`greater than d1'/2, such that the centralpart of the panoramic
`straight line Ka can be observed. In this example, the point
`image projected onto the image sensor 17, which corre-
`50
`Pd(a=20°) hasarelative distance dr equal to 0.5 relative to
`spondsto a solid angle 22 centered on the optical axis OZ,
`occupies a greater area on the image sensor 17 than the area
`the center of the image while the corresponding point
`Pdl(a=20°) onthe linear curve Fdc hasarelative distance dr
`it occupies in FIG. 5 with the classical panoramic objective
`lens (hatched zone). This central part of the panoramic
`of 0.222. The maximum divergence DIVmaxof the distri-
`image is therefore projected onto the image sensor with
`bution function Fd1 according to the present invention can
`expansion ofits area, in relation to the area the central part
`be calculated by a formula of the type:
`would occupyif the objective lens were linear. Theresult is
`that the numberofpixels of the image sensor covered bythis
`part of the image is greater than in previous practices and
`that the definition obtained is improved. Onthe other hand,
`the part of the image delimited by two circles respectively
`passing through the points a", d" and through the points b",
`c" is compressed relative to the corresponding part in FIG.
`5, and the definition on the edges of the imageis less than
`that obtained with a classical linear objective lens, to the
`benefit of the central part of