(19)
`
`'
`
`(12)
`
`Europiisches Patentarnt
`
`European Patent Office
`
`Office europeen des brevets
`
`I IIIIIII IIIIII Ill lllll lllll lllllllll lllll lllll lllll lllll llllll llll llll llll
`EP 1 028 389 A2
`
`(11>
`
`EUROPEAN PATENT APPLICATION
`
`(43} Date of publication:
`16.08.2000 Bulletin 2000/33
`
`(21) Application number: 00100376.3
`
`(22) Date of filing: 07.01.2000
`
`(51) Int. Cl.7: G06T 3/00
`
`(84} Designated Contracting States:
`ATBECHCYDE DKES R FRGBGRIEITLI LU
`MCNL PTSE
`Designated Extension States:
`ALLTLVMKROSI
`
`(30) Priority: 12.02.1999 JP 3477099
`
`(71 } Applicants:
`• Advanet, Inc.
`Okayama-shi, Okayama-Pref. 700-0971 (JP)
`• Rios Corporation
`Okayama-shi, Okayama-Pref. 700-0942 (JP)
`
`• Fit Corporation
`Suwa-Gun, Nagano-Pref. 393-0023 (JP)
`
`(72) Inventors:
`• Shiota, Fuminori
`Okayama-ken 701-1343 (JP)
`• Matsui, Tetsuji
`Okayama-ken 703-8275 (JP)
`
`(74) Representative:
`Sajda, Wolf E., Dipl.-Phys. et al
`MEISSNER, BOLTE & PARTNER
`Postfach860624
`81633 Munchen (DE)
`
`(54)
`
`Arithmetic unit for image transformation
`
`An arithmetic unit for image transformation is
`(57)
`disclosed for transforming a fisheye image obtained by
`using a fisheye lens (2) into a plane image for display,
`comprising: a first coordinate calculating unit (35) for
`obtaining first projection coordinates derived by project(cid:173)
`ing coordinates on the plane image onto a fisheye
`Fl G. 4
`
`image face as an imaginary object face; and a second
`coordinate calculating unit (36) for obtaining second
`projection coordinates derived by projecting the first
`projection coordinates obtained by the first coordinate
`calculating unit (35) onto the fisheye image face.
`
`Frame
`Camera
`t - - - - - -1 I /F I-----<"" memory
`
`Interpolation
`CClll)Uting
`uait
`
`2
`
`1
`
`JO
`
`31
`
`36
`r--- ---- ------- -
`
`32 35 33
`______ ,,
`
`- I i
`
`--40
`
`3
`
`C\I
`c(
`O') co
`C?
`co
`C\I
`0
`T"'"
`
`CL w
`
`r'~'::-~-~t-:-t~-~-i-...... 1
`ca
`<21~-
`~.-
`ting wrl.t
`ting unit
`unit
`,
`' -~~ - - - ' : - . c - - - - r -~ ~ - . - -~ - - ' ,
`34
`!
`:
`i
`!
`------=
`LUT#2
`Int:eErlal. :.., ___ ------- ------
`bus
`
`I
`
`'
`
`------
`
`38
`
`37
`
`Printod by Xorox (UK) Business Services
`2.16.7 (HRS)/3.6
`
`APPLE 1012
`
`1
`
`

`

`EP 1 028 389 A2
`
`2
`
`Description
`
`BACKGROUND OF THE INVENTION
`
`1 . Field of the Invention
`
`The present invention relates to an arithme(cid:173)
`[0001]
`tic unit for image transformation and a monitoring sys(cid:173)
`tem. More particularly, the invention relates to an
`arithmetic unit for image transformation for transforming
`an image obtained by using a fisheye lens into a plane
`image for display and a monitoring system having the
`arithmetic unit.
`
`2. Description of the Related Art
`
`5
`
`camera is determined at the time of designing, by exe(cid:173)
`cuting arithmetic computations according to mathemat(cid:173)
`ical expressions by hardware, the
`image can be
`transformed to a plane image.
`As disclosed
`in WO92/21208, however,
`[0006]
`when calculations upon transformation are expressed
`by mathematical equations and executed according to
`the mathematical equations by hardware, it is neces(cid:173)
`sary to realize many calculations of not only addition,
`10 subtraction, multiplication, and division, but also square
`root, trigonometric function, and the like by the hard(cid:173)
`ware. The unit consequently cannot help becoming
`expensive inevitably.
`The invention has been achieved in consid-
`[0007]
`15 eration of the actual condition of the conventional tech(cid:173)
`nique and it is an object of the invention to provide an
`arithmetic unit for image transformation capable of pro(cid:173)
`viding inexpensive hardware for transforming a fisheye
`image obtained by using a fisheye lens into a plane
`image for display.
`
`20
`
`An arithmetic unit for image transformation is
`[0002]
`used in, for example, a monitoring system using a mon(cid:173)
`itoring camera. The operator monitors the state in a
`space (tor example, shop) in which the monitoring cam-
`era is installed, by watching images from the monitoring
`camera displayed on a monitor provided for the monitor-
`ing system. When a lens attached to the camera is an
`ordinary standard lens, the space can be monitored
`only within the range of the angle of view of the standard
`lens. In order to monitor the entire space in which the
`camera is installed, it is necessary to provide a mecha(cid:173)
`nism tor properly changing the orientation of the cam-
`era. In case of providing such a mechanism, the cost
`increases and the camera has to be remote controlled.
`It consequently becomes hard for the operator to handle
`it.
`There is an idea such that a fisheye lens
`[0003]
`having the wide angle of view is attached to the camera
`and monitoring is performed by using the fisheye lens.
`An image produced by the fisheye lens is, however, dis(cid:173)
`torted as compared with an image obtained by using the
`standard lens and is very hard for the operator to watch
`it. A technique for transforming an image produced by
`the fisheye lens into a plane image is disclosed as a 40
`camera orientation system in WO92/21208.
`The system transforms a circular image
`[0004]
`obtained by using the fisheye lens into an image pro(cid:173)
`duced by a normal image pickup lens (for example,
`standard lens) by an arithmetic process and a plane 45
`image seen from an arbitrary view point can be
`obtained. In this case, when a high speed arithmetic unit
`is used, plane image data can be obtained at real time
`rates only by software. In most of the cases, it takes
`long time for a communicating process, operation as a
`human interface, and the like in a terminal device con-
`nected to a network. It is therefore preferable to realize
`a part in which the same process is repeated in an
`image transforming process, by hardware.
`Since the projecting method (stereoscopic
`[0005]
`projection, equidistant projection, orthogonal projection,
`or the like) of projecting a fisheye lens image onto an
`image pickup device (such as CCD) provided for the
`
`SUMMARY OF THE INVENTION
`
`In order to achieve the object, an arithmetic
`[0008]
`25 unit for image transformation according to the invention,
`for transforming a fisheye image obtained by using a
`fisheye lens into a plane image for display, comprises:
`
`30
`
`35
`
`a first coordinate calculating unit for obtaining first
`projection coordinates on a fisheye image face as
`an imaginary object face derived by projecting coor(cid:173)
`dinates on the plane image; and
`a second coordinate calculating unit for obtaining
`second projection coordinates derived by projecting
`the first projection coordinates obtained by the first
`coordinate calculating unit onto the fisheye image
`face.
`
`[0009]
`
`The action of the configuration is as follows:
`
`Step 1. The first projection coordinates on the fish(cid:173)
`eye image face as an
`imaginary object face
`obtained by projecting coordinates on the plane
`image are derived by the first coordinate calculating
`unit; and
`Step 2. The second projection coordinates obtained
`by projecting the first coordinates onto the fisheye
`image face are derived.
`
`50
`
`That is, when an image is transformed, the
`[001 OJ
`coordinate calculations are not performed by mathe(cid:173)
`matical equations at once but are executed by stages.
`Consequently, the calculating unit can be constructed
`by combining simple arithmetic circuits and the hard-
`55 ware part in the arithmetic unit for image transformation
`can be realized at low cost.
`As a preferred embodiment of the invention,
`[0011]
`a logic circuit for arithmetic operation has a pipelined
`
`2
`
`2
`
`

`

`3
`
`EP 1 028 389 A2
`
`4
`
`ficient k1;
`Fig. 3 is a diagram for explaining a procedure of
`transforming coordinates by hardware; and
`Fig. 4 is a block diagram of circuits mounted on a
`PCI bus substrate.
`
`DETAILED DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`[0019]
`A preferred embodiment of an arithmetic unit
`for image transformation according to the invention will
`be described in detail with reference to the drawings.
`
`{ Positional relation of coordinate system l
`
`5
`
`10
`
`15
`
`[0020]
`Referring to Fig. 1, the positional relation of
`coordinates to be transformed will be described. The
`coordinate system will be set as follows. As a space for
`showing the position of an object, an (X, Y, Z) coordi(cid:173)
`nate space in which the position of a fisheye lens is the
`origin and the direction of the optical axis is the Z axis is
`set. The azimuth (q,) and the zenithal angle (8) are set
`as parameters indicating the position of the object seen
`from the origin.
`[0021]
`Since the position on an image pickup
`device (such as CCD), of an object whose image is
`obtained through the fisheye lens is determined at an
`angle seen through the fisheye lens, it is assumed that
`the object is positioned on the surface of a hemisphere
`30 of radius of 1. The hemispherical face is called an imag(cid:173)
`inary object face.
`[0022]
`A plane image to be obtained is shown by an
`(u, v) coordinate system in Fig. 1. It is assumed that the
`center (origin) of the (u, v) coordinate system is in the
`35 position apart from the origin of the (X, Y, Z) coordinate
`system by distance 1 and is in contact with the hemi(cid:173)
`spherical face as the imaginary object face. The plane
`expressed by the (u, v) coordinate system is made cor(cid:173)
`respond to pixels of a display image on a monitor. The
`40 angle between the u axis of the coordinate system and
`the (X, Y) plane, that is, the angle formed by an inter(cid:173)
`secting line of a plane which passes the origin of the (u,
`v) coordinate system and is in parallel with the {X, Y)
`plane and the (u, v) plane and the u axis is set as (a).
`[0023]
`A plane image (fisheye image) obtained
`through the fisheye lens is expressed by a (p, q) coordi-
`nate system as shown in Fig. 1. It is assumed that the
`(p, q) coordinate system is parallel to the (X, Y) plane
`and has the origin on the Z axis. In a position on an
`image pickup face {for example, position of a pixel on a
`CCD image pickup device), the image circle diameter
`differs according to the size of the image pickup device
`and the focal distance of the fisheye lens. Consequently,
`it is assumed that a fisheye image is projected in a circle
`55 of radius 1 of an image of an object positioning at 90
`degrees (Z = 0) from the front of the lens. At the time of
`actually use, magnification adjustment is performed.
`[0024]
`An image is transformed as follows. The pro-
`
`architecture in each of the first and second coordinate
`calculating units.
`[0012]
`By making the logic circuit have the pipelined
`architecture, the coordinate transformation can be
`sequentially performed by stages and the circuit config-
`uration can be partially simplified. As a result, the hard(cid:173)
`ware part in the arithmetic unit for image transformation
`can be realized at low cost.
`[0013]
`As another preferred embodiment of the
`invention, the logic circuit for arithmetic operation is lim-
`ited to calculations of addition, subtraction, multiplica(cid:173)
`tion, and square root, and division and other function
`calculations are handled by referring to a lookup table.
`[0014]
`In case of executing the calculations accord-
`ing to the equations by the hardware, it is necessary to
`realize many arithmetic operations such as addition,
`subtraction, multiplication, and division, and in addition,
`square root, trigonometric function, and the like by hard(cid:173)
`ware. It is consequently necessary to use a large-scale
`circuit part. Especially, depending on the projecting
`method of the fisheye lens, there may be a case such
`that approximation occurs by trigonometric function or
`infinite polynomial. The logic circuit for arithmetic opera-
`tion is consequently limited to addition, subtraction, mul(cid:173)
`tiplication, and square root. With respect to a part 25
`expressed by division or other function, a lookup table is
`referred to. Consequently, the hardware can be simpli-
`fied and it can contribute to reduce the cost of the arith(cid:173)
`metic unit for image transformation.
`[0015]
`According
`to
`further another preferred
`embodiment of the invention, the second coordinate
`calculating unit is used to obtain the second projection
`coordinates by multiplying the first projection coordi(cid:173)
`nates by a predetermined coefficient and the predeter(cid:173)
`mined coefficient is obtained from the lookup table.
`[0016] When the projecting method of the fisheye
`lens differs, the coefficient becomes different. Only by
`changing data in the lookup table, the invention can deal
`with a different projecting method and it is unnecessary
`to replace the hardware part. The cost of the hardware
`part in the arithmetic unit for image transformation can
`be therefore reduced also with respect to the point.
`[0017]
`A monitoring system according to the inven-
`tion is characterized by comprising the arithmetic unit
`for image transformation. According to the arithmetic
`unit for image transformation, the arithmetic unit can be
`constructed by combining simple arithmetic circuits and
`the cost of the hardware part in the arithmetic unit for
`image transformation can be reduced, so that an inex(cid:173)
`pensive monitoring system can be provided.
`
`20
`
`45
`
`50
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0018]
`
`Fig. 1 is a diagram showing setting of a coordinate
`system;
`Fig. 2 is a diagram for explaining a correction coef-
`
`3
`
`3
`
`

`

`5
`
`EP 1 028 389 A2
`
`6
`
`jecting position on the image pickup face {p, q coordi(cid:173)
`nates) of a point {u, v coordinates) on a plane image is
`obtained by arithmetic operation. By referring to lumi(cid:173)
`nance information at the point, data of the plane image
`to be obtained can be generated. Information of the
`view points (1p, 0, a) and the scale factor (zoom ratio) as
`the size of the plane image is information inputted by
`the operator through a keyboard, pointing device, or the
`like and is obtained and set in advance as data for cal(cid:173)
`culation by a higher-order arithmetic processing unit.
`[0025]
`Necessary parameters are, as shown in
`Figs. 1 and 2, (X0 , Y0 , Z0) indicative of the center (ori-
`gin) of a plane image and change amounts 81.lx, fJvx,
`81.ly, Bvy, 81.lz, fNz in the respective axes of the (X, Y, Z)
`coordinates when a point is moved in the respective
`directions on the (u, v) coordinate system by an amount
`of one pixel (corresponding to one pixel on the monitor
`screen).
`[0026]
`The parameters can be easily obtained from
`the information of the angle information (1p, e, a) of the 20
`view point and the magnification of the image.
`
`5
`
`10
`
`15
`
`[0032] When the distance L is determined, k1 is
`constant. Consequently, a table of k1 with respect to the
`distance L is formed as a lookup table and multiplication
`by k1 obtained from the lookup table is executed,
`thereby enabling (X2 , Y2 , Z2) to be obtained as follows
`(refer to Fig. 3).
`
`In this manner, the first projection coordinates on the
`hemispherical face are determined.
`
`Step2
`
`< Calculating procedure for image transformation)
`
`A calculating procedure for image transfor(cid:173)
`[0027]
`mation will now be described with reference to Fig. 3.
`The calculating procedure is performed by, broadly, two
`steps of first and second steps.
`
`film..1
`
`[0028]
`First, the coordinates of a point P' (first pro(cid:173)
`jection coordinates) on the hemispherical face as an
`imaginary object face, which is a projection of a point P
`on a plane image (the u, v coordinates) are obtained.
`[0029] When it is assumed that the (X, Y, Z) coordi(cid:173)
`nates of the point Pare (X1, Y1, Z1) and the (u, v) coor(cid:173)
`dinates are (u 1, v1), the following are given.
`
`Y 1 = Yo + u 1 • Buy + v 1 • fJvy
`
`Z 1 = Z O + u 1 • fJuz + v 1 • fJvz
`
`[0030]
`As clearly shown in Fig. 2, the point P' is on
`the line connecting O and P. When a coefficient is set to
`k1, the following relation is satisfied.
`
`[0031]
`The distance between the origin of the (X, Y,
`Z) coordinate system and the point P' expressed by the
`coordinates (X2, Y 2 , Z2) is 1. The distance between the
`origin (Xo, Y 0 , Z0) of the (u, v) coordinate system and
`the point P' expressed by the coordinates (X2 , Y 2 , Z2' is
`1. The distance L from the origin of the (u, v) coordinate
`system to the point Pis obtained as follows.
`
`25
`
`[0033]
`As a second step of the calculation, a proce(cid:173)
`dure of obtaining second projection coordinates w(p1,
`q1) on a fisheye image face from the first projection
`coordinates (X2, Y 2 , Z2) determined (refer to Fig. 1 with
`respect tow) will be explained.
`[0034]
`As described above, since the point P' is on
`the surface of the hemisphere of radius of 1, the zenithal
`angle (01) is unconditionally determined from the value
`of Z2 (refer to Fig. 1 with respect to 01). The following
`30 equation (1) is therefore derived.
`
`(1)
`
`[0035]
`Since the azimuth of the point w on the
`image pickup face and that of the point P' on the hemi(cid:173)
`spherical face are the same, the following equation (2)
`is satisfied.
`
`(2)
`
`35
`
`40
`
`[0036]
`Since the height (h) from the origin on the
`fisheye image face (origin of the p, q coordinate system)
`to the point w is expressed as a function of 01 according
`to the fisheye image projecting method, the following
`45 equation (3) is satisfied.
`
`(3)
`
`[0037]
`Some examples of specific functions of F(0 1)
`50 according to the projecting method will be shown. When
`it is assumed that the focal distance of the fisheye lens
`is f, the following relations are established.
`
`equidistant projection: h = f • 0
`
`stereoscopic projection: h = 21 • tan(G/2)
`
`[0038] When the equation (1) is substituted for the
`
`55
`
`4
`
`4
`
`

`

`7
`
`EP 1 028 389 A2
`
`8
`
`equation (3), the following equation (4) is given. h can
`be expressed as a function determined by Z2.
`
`(4)
`
`[0039] With respect to the distance r from the origin
`of the (X, Y, Z) coordinate system to the point Q
`obtained by projecting the point P' onto the (X, Y) plane,
`since the point P' is a point on the surface of the hemi(cid:173)
`sphere of radius of 1, the following equation (5) is satis(cid:173)
`fied.
`
`r = (1 - Z 2 2) o.5
`
`(5)
`
`[0040]
`The following equation (6) is therefore given
`from the equations (4) and (5).
`
`k 2 = h/r
`= F(cos- 1(Z 2))/(1-Z 2
`
`2
`
`)
`
`0
`
`5 (6)
`·
`
`[0041]
`That is, the coefficient k2 of the equation (2)
`can be derived as a function of Z2. With respect to the
`coefficient k2, a lookup table for obtaining the coefficient
`k2 from the value of Z2 in accordance with the equation
`(3) is generated. By using the value, second projection
`coordinates (p1, q 1) on the image pickup face are
`obtained as follows.
`
`q 1 = k2 • y 2
`
`[0042]
`As described above, in the first step for
`image transformation, calculation is limited to addition,
`subtraction, multiplication, and square root, and the
`other functions are obtained from the lookup table. The
`calculation in the second step is limited to multiplication
`and the other calculation of functions and the like is han(cid:173)
`dled by referring to the lookup table. By executing the
`calculations in accordance with the steps, a simple
`arithmetic circuit (logic circuit such as an adder) can
`have a pipelined architecture. The circuit scale of the
`part of the complicated function calculation is sup(cid:173)
`pressed by using the lookup table. Further, by changing
`the table for obtaining k2 in the second step, a fisheye
`lens of a different projecting method can be dealt with
`easily, moreover, by the same arithmetic circuit.
`
`< Example of circuit configuration)
`
`[0043]
`A specific circuit block configuration will now
`be described with reference to Fig. 4. The configuration
`relates to an example of a substrate which is configured
`for a computer system having a PCI (Peripheral Compo(cid:173)
`nent Interconnect) bus.
`[0044]
`A fisheye lens 2 is attached to a CCD cam(cid:173)
`era 1 and image information obtained by the CCD cam-
`
`5
`
`10
`
`15
`
`20
`
`era 1 is sent to a PCI bus substrate 3 and subjected to
`processes for image transformation. The PCI bus sub(cid:173)
`strate 3 comprises: a camera interface 30 for obtaining
`fisheye image data from a CCD provided for the CCD
`camera 1; a frame memory 31 for storing fisheye image
`data of one frame; an interpolation computing unit 32 for
`executing an interpolating computation on the basis of
`the calculation result of an operation part 40; an FIFO
`memory 33; a capture control unit 34 for controlling cap-
`ture of the fisheye image data to the frame memory 31;
`the operation part 40 (part surrounded by a broken line)
`having the configuration which characterizes the inven(cid:173)
`tion; and a PCI bus interface 39 for sending plane image
`data obtained by the coordinate transformation.
`[0045]
`The operation part 40 comprises a first coor(cid:173)
`dinate calculating unit 35, a second coordinate calculat(cid:173)
`ing unit 36, a first lookup table 37 connected to the first
`coordinate calculating unit 35, and a second lookup
`table 38 connected to the second coordinate calculating
`unit 36. Further description will be given in relation to
`the coordinate transforming procedure. The first coordi(cid:173)
`nate calculating unit 35 is a part of executing the calcu(cid:173)
`lation of the first step shown in Fig. 3 and can obtain the
`first projection coordinates (X2, Y 2 , Z2) on the hemi-
`25 spherical face from the (u, v) coordinates in the plane
`image. The first lookup table 37 is a table for obtaining
`the correction coefficient k1 from the distance L.
`[0046]
`The second coordinate calculating unit 36 is
`a part of executing the calculation of the second step in
`Fig. 3 and can obtain the second projection coordinates
`(p1, q1) on the fisheye image face from the first projec(cid:173)
`tion coordinates (X2, Y 2 , Z2' derived by the first coordi(cid:173)
`nate calculating unit 35. The second lookup table 38 is
`a table for obtaining the correction coefficient k2.
`
`30
`
`35
`
`< Description of circuit operation l
`
`40
`
`[0047]
`The operation of the circuit shown in Fig. 4
`will now be described.
`[0048]
`Fisheye image data obtained by the CCD
`camera 1 is written into the frame memory 31 via the
`camera interface 30. Since the coordinates (p, q) on the
`image pickup face corresponding to the coordinates (u,
`v) on the display screen are obtained in the operation
`45 part 40, image data corresponding to the coordinates
`(p, q) sequentially designated by the operation part 40 is
`read from the frame memory 31 and is sent to the inter(cid:173)
`polation computing unit 32.
`[0049]
`Strictly, the position of a pixel on the fisheye
`image does not coincide with a pixel on the plane image
`to be displayed on the monitor. Plural data of pixels
`nearby is consequently read from the frame memory 31
`and interpolation is performed by obtaining a weighted
`mean of the data, thereby enabling a natural plane
`image to be obtained.
`[0050]
`The interpolated image data is subjected to
`speed adjustment by the FIFO memory and resultant
`data is transferred to a memory on the host computer
`
`50
`
`55
`
`5
`
`5
`
`

`

`9
`
`EP 1 028 389 A2
`
`10
`
`side via the PCI bus interlace 39. In case of the example
`of the configuration of Fig. 4, since the processes such
`as coordinate calculation and interpolating calculation
`can be executed by a middle-scale FPGA (or gate
`array), the hardware can be constructed relatively
`cheap.
`
`5
`
`Claims
`
`projecting coordinates on the plane image onto
`a fisheye image face as an imaginary object
`face; and
`a second coordinate calculating unit (36) for
`obtaining
`second projection
`coordinates
`derived by projecting the first projection coordi(cid:173)
`nates obtained by the first coordinate calculat(cid:173)
`ing unit (35) onto the fisheye image face.
`
`1. An arithmetic unit for image transformation for
`transforming a fisheye image obtained by using a
`fisheye lens (2) into a plane image for display, char(cid:173)
`acterized by comprising:
`
`10 6. The monitoring system according to claim 5, char(cid:173)
`acterized in that a logic circuit for arithmetic opera(cid:173)
`tion has a pipelined architecture in each of the first
`and second coordinate calculating units (35, 36).
`
`a first coordinate calculating unit (35) for
`obtaining first projection coordinates derived by
`projecting coordinates on the plane image onto
`a fisheye image face as an imaginary object
`face; and
`a second coordinate calculating unit (36) for
`obtaining
`second projection
`coordinates
`derived by projecting the first projection coordi(cid:173)
`nates obtained by the first coordinate calculat-
`ing unit (35) onto the fisheye image face.
`
`2. The arithmetic unit according to claim 1, character(cid:173)
`ized in that a logic circuit for arithmetic operation
`has a pipelined architecture in each of the first and
`second coordinate calculating units (35, 36).
`
`3. The arithmetic unit according to claim 2, character-
`ized in that the logic circuit for arithmetic operation
`is limited to calculations of addition, subtraction,
`multiplication, and square root, and division and
`other function calculations are handled by referring
`to a lookup table (37, 38).
`
`4. The arithmetic unit according to claim 3, character(cid:173)
`ized in that the second coordinate calculating unit
`(36) is used to obtain the second projection coordi-
`nates by multiplying the first projection coordinates
`by a predetermined coefficient (k1) and the prede(cid:173)
`termined coefficient (k1) is obtained from a lookup
`table (38).
`
`5. A monitoring system comprising
`
`a CCD camera (1) to which a fisheye lens (2) is
`attached and
`a Peripheral Component Interconnect (PCI)
`bus substrate (3) having an arithmetic unit for
`image transformation for transforming a fisheye
`image obtained by using the fisheye lens (2)
`into a plane image for display,
`characterized in that the arithmetic unit for
`image transformation comprises:
`a first coordinate calculating unit (35) for
`obtaining first projection coordinates derived by
`
`15 7. The monitoring system according to claim 6, char(cid:173)
`acterized in that the logic circuit for arithmetic oper(cid:173)
`ation
`is
`limited
`to calculations of addition,
`subtraction, multiplication, and square root, and
`division and other function calculations are per(cid:173)
`formed by referring to a lookup table (37, 38).
`
`20
`
`8. The monitoring system according to claim 7, char(cid:173)
`acterized in that the second coordinate calculating
`unit (36) is used to obtain the second projection
`coordinates by multiplying the first projection coor(cid:173)
`dinates by a predetermined coefficient (k1) and the
`predetermined coefficient (k1) is obtained from a
`lookup table (38).
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`6
`
`6
`
`

`

`g 389 A2
`EP 1 028 389 A2
`
`F \ G.1
`
`
`
`X
`
`l
`\
`\
`I
`I
`I
`I
`
`I . I
`\ . \
`
`I
`1
`
`p
`
`7
`
`7
`
`

`

`EP 1 028 389 A2
`EP 1 028 389 A2
`
`Fl G._2
`FIG.2
`
`
`—P(u44) YF¥E
`P( u1,v1)
`v'u~+vf
`
`(X v Zo)
`Q,TQ,
`
`( U, V)
`
`(x0¥9 Zp) (u,v)
`
`1
`
`0
`
`8
`
`8
`
`

`

`FIG. 3
`
`(Stepl)
`---- Projected
`- - - - - -
`. .
`point
`Position on Surface x,=Xo+U•aux+V•avx
`X2=k,·X,
`(U, V)--. Y1=Yo+U·ou ytV·ovy --(Xi, Y,,Z1)-- Y2=k1·Y1 -
`Z1=Zo+U•auz+V•avz
`Z2=k1·Z,
`
`(X,.Y,,Z1)
`on hemisphere
`
`m
`
`,, -
`
`(Step 1 )
`Projected point
`(X2. Yi, 22)
`on hemisphere
`
`.. ~--(h)----.. P,=k 2-X 2
`r Q,=k2·Y2 -
`
`• (X 2• Y z)
`
`Position
`(P,. q,)
`on image
`pickup face
`
`9
`
`

`

`Fl G. 4
`
`( --i'r~ ___ cam
`/erFa i-----ai Frame
`If
`- memory
`1
`
`Inteqx>lation
`t--...,.. canputing
`- unit
`
`_..,_ F I F 0
`---- memory
`
`2 1
`
`J
`30
`
`.i.
`0
`
`I
`
`i
`
`31
`
`,...._ _ _ ..._,
`
`Capture
`control
`unit
`
`I
`
`I
`I
`
`f
`\
`32 35 33
`36
`r-· ----f------· --------·--~---..... ··1
`: Second coordi- ~irst coordi-
`:
`: nate calcu-
`nate calcu-
`,
`~-- ... /40
`: lating unit
`lating unit
`:
`:
`------ :
`:
`.------....,
`ILUT#2l
`:
`ILUT#ll :
`Internal 1,, ________________________ ... ----~
`bus
`
`' - , , - - -y - -~ I
`I
`
`)
`
`34
`
`-
`
`I
`
`38
`
`'
`37
`
`PC I bus
`I /F ~39
`
`10
`
`