1305049
`
`VERIFICATION OF TRANSLATION
`
`1,
`
`Karen McGillicuddy
`
`of
`
`1950 Roland Clarke Place
`
`Reston, VA 20191
`
`declare that I am well acquainted with both the Japanese and English languages, and that
`
`the attached is an accurate translation,
`
`to the best of my knowledge and ability, of
`
`Japanese Patent Laid—open Publication No. 2000—242773, published September 8, 2000.
`
`
`Signature ‘
`
`
`
`
`aren McGillicuddy
`
`{1305049 02000032DOC}
`
`Panasonic Exhibit 1010 Page 1 of 18
`
`

`

`(12) Laid—Open Patent Publication (A)
`(19) Japan Patent Office (JP)
`(11) Patent Application Publication No.
`
`2000-242773
`
`(P2000—242773A)
`(43) Published September 8, 2000
`w._.6—______m
`(51)
`Int. Cl.
`ID No.
`F1
`Theme Code (Ref)
`G06T 3/00
`G06F15/66
`5B057
`5C022
`//H04N 5/225
`H04N 5/225
`
`360
`
`C
`
`(21) Application No. H1 1—41002
`
`
`Request for Examination Requested
`No. of Claims: 7 OL (Total of8 page(s))
`(71) Applicant 594066648
`Yugen Kaisha FIT
`6750 Shimosuwamachi
`
`(22) Application Date- February 19, 1999
`
`
`
`Shimosuwa-gun, Nagano-ken
`
`(71) Applicant 596010382
`Rios Corporation
`2—7—16, Hosei
`
`Okayama-shi, Okayama-ken
`
`(71) Applicant 594066648
`Advanet Co., Ltd.
`3—20—8, Noda
`
`Okayama-shi, Okayama—ken
`cont ’d on last page
`
`(54) [TITLE OF INVENTION]
`IMAGE DATA CONVERSION DEVICE
`
`(57) [ABSTRACT]
`[PROBLEM] To convert image capture data of a full
`surrounding area obtained by a fish—eye lens such that
`distortion can be corrected and the image capture data
`can be displayed as a seamless single image.
`[MEANS FOR SOLVING THE PROBLEM} With
`
`respect to a region for a portion of circular image data
`obtained by image capture using a fish—eye lens, a
`point (g(®)’cosw, g(®)-sinw) in the region on a
`planar Cartesian coordinate system whose origin
`point is a center ofthe circular image (where O is a
`parameter fulfilling 0 < O < Tc/2; g(®) is a function
`fulfilling g(0) = 0 and monotonically increasing in the
`range of O; and \l/ is an angle formed by a line
`segment and a coordinate axis on the planar Cartesian
`coordinate system. the line segment linking the origin
`point of the planar Cartesian coordinate system and
`the point on the circular image) is converted into a
`point (R, w, R/tanO) on a cylindrical coordinate
`system where R is a constant.
`
`
`
`AID
`converter
`
`
`First
`image
`memory
`
`3
`
`
`
`
`DEA
`com’erter
`
`{1305049 02051422.DOC}1
`
`Panasonic Exhibit 1010 Page 2 of 18
`
`

`

`[SCOPE OF THE CLAIMS]
`
`[CLAIM 1] An image data conversion device wherein, with respect to a region for a
`
`portion of circular image data obtained by image capture using a fish—eye lens, a point
`
`(g(®)’cosw, g(®)'sin\[1) in the region on a planar Cartesian coordinate system whose origin
`
`point is a center of the circular image (where (9 is a parameter fulfilling 0 < (9 < 1t/2; g(®) is a
`
`function fulfilling g(0) = 0 and monotonically increasing in the range of G); and \[l is an angle
`
`formed by a line segment and a coordinate axis on the planar Cartesian coordinate system, the
`
`line segment linking the origin point of the planar Cartesian coordinate system and the point
`
`on the circular image) is converted into a point (R, w, R/tanG) on a cylindrical coordinate
`
`system where R is a constant.
`
`[CLAIM 2] The image data conversion device according to claim 1, wherein the fish—eye
`
`lens has a property in which h = g(9) (h being an image height and 6 being a field angle).
`
`[CLAIM 3] The image data conversion device according to claim 2, wherein the function
`
`g(6) is g(9) = 2f-tan(6/2) (f being a focal distance).
`
`[CLAIM 4] The image data conversion device according to claim 2, wherein the function
`
`g(6) is g(8) = f-G (f being a focal distance).
`
`[CLAIM 5] A computer-readable storage medium storing a program executing operations
`
`ofthe image data conversion device according to any one of claims 1 to 4 on the computer.
`
`[CLAIM 6] An image capture system comprising:
`
`the fish—eye lens;
`
`a conversion means converting an image obtained by the fish-eye lens into the image
`
`data;
`
`the image data conversion device according to any one of claims 1 to 4; and
`
`a display means displaying the image data on the cylindrical coordinate plane by
`
`projecting the image data onto a plane, the image data on the cylindrical coordinate plane
`
`having been converted by the image data conversion device,
`
`[CLAIM 7] An image data conversion method wherein, with respect to a region for a
`
`portion of circular image data obtained by image capture using a fish—eye lens, a point
`
`g(®)'cosw, g(®)-sin\y) in the region on a planar Cartesian coordinate system whose origin
`
`point is a center ofthe circular image (where G is a parameter fulfilling 0 < (9 < 7t/2; g(®) is a
`
`function fulfilling g(0) = O and monotonically increasing in the range of G); and \[1 is an angle
`
`formed by a line segment and a coordinate axis on the planar Cartesian coordinate system, the
`
`line segment linking the origin point of the planar Cartesian coordinate system and the point
`
`{J305049 02051422.DOC}2
`
`Panasonic Exhibit 1010 Page 3 of 18
`
`

`

`on the circular image) is converted into a point (R, \p, R/tanG) on a cylindrical coordinate
`
`system where R is a constant.
`
`[DETAILED DESCRIPTION OF THE INVENTION]
`
`[0001]
`
`[Technical Field of the Invention] The present invention relates to an image data
`
`conversion device and, specifically, relates to a device converting image data having a
`
`circular shape and obtained by image capture with a fish-eye lens into planar image data in
`
`which distortion is corrected and wide area display is possible.
`
`[0002]
`
`[Prior Art] Conventionally, in an in-store surveillance camera, a traffic regulating camera,
`
`and the like, in order to monitor a status of a surrounding area, a camera having a field of
`
`View that is limited to a forward view is typically rotated by a drive device such as a motor.
`
`However, when such a camera having a limited field of view is rotated, a rotation driver
`
`becomes necessary, and thus the device becomes complex.
`
`In addition, the camera must be
`
`rotated when a specific target is to be captured and so when the target is moving, it may not
`
`be possible to capture the target due to limitations of rotation speed. Furthermore, a plurality
`
`of directions cannot be imaged simultaneously, constantly resulting in blind spots in a
`
`monitored area, and therefore such a camera cannot be said to be sufficient for a monitoring
`
`function and the like.
`
`[0003]
`
`Japanese Patent Publication No. H06~501585 suggests, as a technology resolving
`
`such problems, converting image data having a circular shape into a planar image, the image
`
`data being obtained by capturing an image of all orientations forward of a fish-eye lens. A
`
`simple description of the technology follows. The circular image data obtained by the fish-
`
`eye lens can include an image of all orientations, but the image becomes more distorted
`
`further toward an outer periphery. This distortion can be largely removed when the circular
`
`image data is translated onto a hemisphere surface, but image data translated to the
`
`hemisphere surface cannot be displayed on a planar monitor. In the above technology, the
`
`portion of the image data translated to the hemisphere surface is further projected onto a
`
`planar surface, thereby enabling the image data to be displayed by conversion into a plane.
`
`By using such a method, a device can be simplified by making a rotation drive mechanism
`
`unnecessary, and switching a display direction simply becomes a matter of control during
`
`data processing, and thus can also be sped up remarkably.
`
`[0004]
`
`{3305049 02051422.DOC}3
`
`Panasonic Exhibit 1010 Page 4 of 18
`
`

`

`[Problem to Be Solved by the Invention] However, even when circular image data
`
`obtained in this way by the fish—eye lens is configured to be converted directly into planar
`
`image data, an entire hemisphere surface cannot be projected onto a single planar surface
`
`with adequate distortion correction, and therefore the image obtained relates only to a specific
`
`direction. Accordingly, blind spots occur on a display screen even though the full
`
`surrounding area is captured by the fish-eye lens. In addition, as a method to eliminate blind
`
`spots, a method performing planar image conversion for a plurality of directions
`
`simultaneously and performing simultaneous display can be considered. However, in such a
`
`case, seams between images are not continuous and a plurality of different image conversions
`
`must be performed.
`
`[0005]
`
`In order to resolve these problems, the present invention has as an object to convert
`
`image capture data of a full surrounding area obtained by a fish-eye lens such that distortion
`
`can be corrected and the image capture data can be displayed as a seamless single image.
`
`[0006]
`
`[Means for Solving the Problems]
`
`In order to resolve the above-noted problems, with
`
`respect to a region for a portion of circular image data obtained by image capture using a
`
`fish-eye lens, an image data conversion device according to the present invention converts a
`
`point (g(®)°cosw, g(®)-sin\[t) in the region on a planar Cartesian coordinate system whose
`
`origin point is a center ofthe circular image (where G) is a parameter fulfilling 0 < 6) < 1t/2;
`
`g(®) is a function fulfilling g(0) = O and monotonically increasing in the range of G; and w is
`
`an angle formed by a line segment and a coordinate axis on the planar Cartesian coordinate
`
`system, the line segment linking the origin point of the planar Cartesian coordinate system
`
`and the point on the circular image) into a point (R, w, R/tan®) on a cylindrical coordinate
`
`system where R is a constant. Moreover, what is referred to as a fish—eye lens in the present
`
`invention includes not only what is typically called a fish-eye lens, having a viewing angle of
`
`substantially 180°, but also includes what is typically called a wide—angle lens, having a
`
`narrower viewing angle.
`
`[0007] Moreover, the conversion is sufficient when a conversion is performed that, as a
`
`result, satisfies the above-noted relationship and the process of the conversion is not
`
`discussed. In other words, the above-noted formula need not be applied directly, and cases
`
`are also included where conversion is performed using a table showing correspondence
`
`between pre—conversion pixels and post—conversion pixels.
`
`In addition, the planar Cartesian
`
`coordinate system and the cylindrical coordinate system are mutually independent and
`
`directions of the coordinates can be defined as desired.
`
`In addition, the cylindrical coordinate
`
`[1305049 02051422.DOC}4
`
`Panasonic Exhibit 1010 Page 5 of 18
`
`

`

`system where R is a constant forms a cylindrical surface. However, herein, cases are also
`
`included where the above—described conversion is performed by projecting the image as a
`
`plane.
`
`[0008] When configured in this way, first, when w is fixed and (9 is increased from a small
`
`value to a large value within a range where the point (g(®)-cosw, g(®)°sin\y) is in the region
`
`for the portion of the circular image data, a line segment oriented in a radial direction from
`
`the center of the circular image data is converted into the cylindrical coordinate system where
`
`R is a constant (i.e., as a line segment oriented from up to down on a cylindrical surface of
`
`radius R). This conversion is further performed for W in a maximum range of 360° such that
`
`the above—noted point is in the region for the portion of the circular image data. Therefore,
`
`the portion of the circular image data is smoothly converted into a cylindrical surface in a
`
`form where distortion is corrected, the portion of the circular image data including image data
`
`for a full surrounding area centered on the fish—eye lens and captured by the fish—eye lens.
`
`The image converted into a cylindrical surface can be readily made into a planar image by
`
`projecting the cylindrical surface. Therefore, distortion of the image capture data of the full
`
`surrounding area obtained by the fish—eye lens can be corrected and the image capture data
`
`can be displayed as a seamless single image.
`
`[0009]
`
`In addition, when the fish—eye lens has a property in which h = g(6) (h being an
`
`image height and 0 being a field angle), an amount that g(®) increases accompanying an
`
`increase of (9 is the same as the amount that the image height h increases. Accordingly, in
`
`other words, changing ® is the same as changing the field angle 9. In such a case, the
`
`circular image is converted into an ideal, undistorted hemisphere surface and, moreover, an
`
`image can be obtained in which a converted hemisphere surface image is projected with
`
`respect to the cylindrical surface using light projected from a center point of the hemisphere
`
`surface and oriented in a radial direction, the cylindrical surface being positioned such that a
`
`center axis matches the center axis of the hemisphere surface. Accordingly, an image capture
`
`region spanning the entire area on the sides of the fish—eye lens (i.e., an image on an outer
`
`peripheral side of the circular image) is converted into an image with almost no distortion.
`
`[0010]
`
`In addition, when the function g(9) indicating a property of the fish—eye lens is g(0)
`
`= 2f'tan(9/2) (where f is a focal distance), an amount of data for the image on the outer
`
`peripheral side of the circular image data is greater. Therefore, reproducibility during
`
`conversion into a cylindrical surface of an image capture area spanning the entire area on the
`
`sides of the fish—eye lens can be improved.
`
`{J305049 O2051422.DOC}5
`
`Panasonic Exhibit 1010 Page 6 of 18
`
`

`

`[0011] Moreover, when the function g(0) indicating a property of the fish—eye lens is g(9) =
`
`f°0 (where f is a focal distance), the property is that of the most typically used fish—eye lens.
`
`Therefore, a low—cost, mass—produced fish-eye lens can be used and cost of the device can be
`
`reduced.
`
`[0012] The image data conversion device can also be achieved by installing a program
`
`executing the operations of the image data conversion device onto a computer, and the
`
`program can be stored on a storage medium readable by the computer.
`
`[0013] Moreover, in order to resolve the above problems, an image capture system
`
`according to the present invention includes the fish—eye lens; a conversion means converting
`
`an image obtained by the fish-eye lens into the image data; the image data conversion device
`
`according to any one of claims 1 to 4; and a display means displaying the image data on the
`
`cylindrical coordinate plane as a plane by projecting the cylindrical coordinate plane, the
`
`image data on the cylindrical coordinate plane having been converted by the image data
`
`conversion device.
`
`[0014] According to such a configuration, the image capture system, as described above,
`
`displays the image on the cylindrical surface by projecting the image as a plane, the image
`
`data conversion device having corrected distortion of the image and converted the image to
`
`remove seams. Therefore, a situation of a surrounding area captured by the fish—eye lens can
`
`be entirely shown on one screen. Moreover, the projected plane also includes cases where
`
`the projected plane is processed as appropriate, e.g., a case where the projected plane is cut at
`
`a predetermined position and each cut section is arrayed as appropriate for display.
`
`[0015] Also, in order to resolve the above-noted problems, with respect to a region for a
`
`portion of circular image data obtained by image capture using a fish-eye lens, an image data
`
`conversion method according to the present invention converts a point (g(®)'cosw,
`
`g(®)°sin\y) in the region on a planar Cartesian coordinate system whose origin point is a
`
`center of the circular image (where (9 is a parameter fulfilling O < 9 < 7t/2; g(®) is a function
`
`fulfilling g(0) = 0 and monotonically increasing in the range of G); and \y is an angle formed
`
`by a line segment and a coordinate axis on the planar Cartesian coordinate system, the line
`
`segment linking the origin point of the planar Cartesian coordinate system and the point on
`
`the circular image) into a point (R, \p, R/tanG) on a cylindrical coordinate system where R is
`
`a constant. The image data conversion method also smoothly converts the portion of the
`
`circular image data into a cylindrical surface in a form where distortion is corrected, the
`
`portion of the circular image data including image data for a full surrounding area centered
`
`on the fish—eye lens and captured by the fish—eye lens. Therefore, distortion can be corrected
`
`(1305049 02051422.DOC}6
`
`Panasonic Exhibit 1010 Page 7 of 18
`
`

`

`and the image capture data for the full surrounding area obtained by the fish—eye lens can be
`
`displayed seamlessly.
`
`[0016]
`
`[Mode for Carrying out the Invention] Hereafter, a description of an embodiment of the
`
`present invention is given with reference to the drawings. Fig.
`
`1 shows a block diagram
`
`illustrating a configuration of an image capture system according to the embodiment of the
`
`present invention. The image capture system is configured with a camera portion 1, a data
`
`converter 2, a first image memory 3, a second image memory 4, an A/D converter 5, a D/A
`
`converter 6, and a monitor 7.
`
`[0017] The camera portion 1 uses a fish-eye lens 1 1 as a lens and is capable of image
`
`capture at a field angle of 90° or more at least with respect to an optical axis of the lens.
`
`Herein, a fish—eye lens is used having a property wherein
`
`h = Zf'tan(0/2)
`
`Q)
`
`When a lens having such a property is used, an amount that an image height h increases
`
`relative to an increase in the field angle 0 becomes greater in comparison to a typical lens
`
`where h = f-G. Therefore, an amount of information can be greater for a range having a large
`
`field angle 9 (i.e., a periphery of a circular image obtained by the fish~eye lens). The circular
`
`image obtained by the fish—eye lens 11 having this property is converted into an electronic
`
`signal by a CCD 12 for each RGB value of each pixel and is output to the A/D converter 5.
`
`Herein, the CCD is, for example, a CCD using 1280x1024 pixels.
`
`[0018] The A/D converter 5 converts an analog signal transmitted from the C C D 12 into a
`
`digital signal and outputs the signal to the first image memory 3. The first image memory 3
`
`is configured with a RAM and stores RGB values for each pixel based on the digital signal
`
`representing the transmitted circular image. The data converter 2 is equivalent to an image
`
`data conversion device according to the present invention and is configured with an
`
`arithmetic circuit performing calculations described below. The data converter 2 converts the
`
`circular image data stored in the first image memory 3 into cylindrical image data using a
`
`conversion described in detail hereafter, then outputs the image data to the second image
`
`memory 4.
`
`[0019] The second image memory 4 is again configured with a RAM, and the RGB values
`
`for each pixel of the output image data are stored as a plane, described hereafter, onto which
`
`the cylindrical surface is projected.
`
`In addition, the image onto which the cylindrical surface
`
`is projected is long in a width direction. Therefore, in order to obtain the display data
`
`{1305049 02051422.DOC}7
`
`Panasonic Exhibit 1010 Page 8 of 18
`
`

`

`described below, when storing image data, the second image memory 4 stores the projected
`
`image such that the image is divided into halves and placed at a predetermined vertical
`
`interval. The D/A converter 6 reads the image data having digital values stored in the second
`
`image memory 4 and converts the image data into an analog signal, then transmits the analog
`
`signal as an image signal in order of scan lines of the monitor 7. The monitor 7 displays the
`
`image stored in the second image memory 4 based on the image signal transmitted by the
`
`D/A converter 6.
`
`[0020] Moreover, portions of the present image capture system except the camera portion 1
`
`can be achieved by installing a program to accomplish the above—noted functions and the
`
`below—noted operations on a typical computer. The program can be stored on a storage
`
`medium such as a floppy disk that is readable by the computer. In addition, the program can
`
`be installed on the computer through a communication means such as the Internet.
`
`[0021] At this point, a conversion method of the data converter 2 is described in detail. The
`
`data converter 2 converts the image data on the circular surface onto a cylindrical surface. A
`
`correspondence relationship between points on the circular surface and points on the
`
`cylindrical surface is described with reference to Figs. 2 and 3. Fig. 2 is a view showing a
`
`relationship between a desired point P0 on a hemisphere surface 0, which supposes an ideal
`
`full circumferential surface captured by the fish—eye lens 1 1, and a point P2 on the cylindrical
`
`surface C, which is the converted surface. In Fig. 2, in an xyz coordinate system, the
`
`hemisphere surface 0 has a base circle (a circle configuring an edge of the hemisphere
`
`surface) positioned on an xy plane and centered on the origin point. A z axis coincides with
`
`the optical axis of the fish-eye lens 11. Furthermore, the cylindrical surface C is a cylinder of
`
`radius R axially centered on the z axis. In other words, the cylindrical surface C can be
`
`represented as a curved surface in an riJ/z cylindrical coordinate system where R is constant.
`
`In addition, herein, points on the cylindrical surface C are represented by (R, w, 2) as points
`
`on the rwz cylindrical coordinate system. However, \i! is an angle formed by a line segment
`
`and an x axis, the line segment being created by the origin point and a point of intersection
`
`between a vertical line down to the xy plane and the xy plane. Also, P2 is a point where a
`
`straight line extension of a line segment linking the origin point with the desired point P0 on
`
`the hemisphere surface 0 intersects with the cylindrical surface C. Moreover, an angle
`
`formed by the z axis and the line segment linking the origin point with the point P0 has a
`
`field angle 0 with respect to the point P0.
`
`[0022] The desired point P0 on the hemisphere surface 0, which supposes the above—noted
`
`ideally captured full circumferential surface, indicates a position of a point P] on a captured
`
`{J305049 02051422.DOC}8
`
`Panasonic Exhibit 1010 Page 9 of 18
`
`

`

`image surface F captured by the fish~eye lens ll in Fig. 3. A circular region on the captured
`
`image surface F in Fig. 3 is a region on the CCD l2 bombarded by light through the fish—eye
`
`lens 1 l, and forms a circular surface S displaying converted image data.
`
`In Fig. 3, an origin
`
`point on an XY coordinate system is occupied by the center of the circular surface S.
`
`Moreover, directions of the X axis and the Y axis are configured to be opposite from the x
`
`axis and the y axis of the xyz coordinate system of Fig. 2. This is in order to handle P0 also
`
`being shown in a position inverted top to bottom and left to right because the image captured
`
`by the camera portion 1 is inverted top to bottom and left to right. However, the cylindrical
`
`surface is arbitrary, and so in an actual conversion there is also liberty in how coordinates are
`
`handled.
`
`In addition, the position of P1 is a position at an image height h distance from the
`
`origin point where an angle formed by the X axis and a line segment linking the origin point
`
`with Pl is w. h is expressed by the above noted formula 1 and is defined by the field angle 8.
`
`[0023] The data converter 2 converts Pl shown in Fig. 3 into P2 shown in Fig. 2. Hereafter,
`
`a description is given of a concrete calculation. First, a case is considered where parameters
`
`are set to 9 and w.
`
`In this case, the coordinates of P1 on the XY plane are (h'cosw, h°sinw)
`
`and, when h is substituted out using the above—noted formula I, are represented by
`
`(2f-tan(9/2)'costy, 2f-tan(6/2)'sinw). Meanwhile, for the coordinates of P2, R is a constant,
`
`and 2 has a relationship according to Fig. 2 in which 2 = R / tanQ, and thus the coordinates of
`
`P2 are represented by (R, w, R / tanG). Accordingly, the parameters 8 and w can be moved to
`
`specify Pl on the circular image on the XY plane, and a point on the rwz cylindrical
`
`coordinate system (that is, the point on the cylindrical surface C) can also be specified with
`
`respect thereto. Therefore, the image data can be converted according to a formula that
`
`indicates Pl and P2. However, in a case where 6 = 0, the z coordinate of P2 becomes
`
`infinitely large, and therefore in reality a value of 6 is used which is a certain value or more.
`
`In the present embodiment, a range of 1r/4 S 6 < 7t/2 is used for 9.
`
`[0024]
`
`In an actual conversion, calculation is facilitated by using data directly representing
`
`a position of each pixel in the captured CCD 12 or a post-conversion position of each pixel
`
`on the converted cylindrical surface C as the parameters. Use of x and w as parameters
`
`directly representing the post-conversion position of each pixel on the cylindrical surface C is
`
`considered.
`
`In such a case, when the point P2 on the cylindrical surface C is represented with
`
`the parameters 2 and w, the coordinates are (R, w, z). The converted cylindrical surface C is
`
`projected and displayed on the monitor 7, and each pixel is aligned at equal intervals of a
`
`pixel pitch d in a longitudinal direction and a lateral direction. Fig. 4 illustrates a View
`
`representing a relationship between the projected cylindrical surface C and the circular
`
`{.1305049 02051422.DOC}9
`
`Panasonic Exhibit 1010 Page 10 of 18
`
`

`

`surface S. The longitudinal direction of the cylindrical surface C coincides with the
`
`parameter 2. Therefore, when a change is made to the parameter 2 in units of the pixel pitch
`
`d, the changes coincide with the longitudinal direction pixel positions of the cylindrical
`
`surface C. An amount of change accompanying an amount of change in w in the lateral
`
`direction of the projected cylindrical surface can be represented by Rwy. Therefore, when the
`
`amount of change of the parameter \y corresponding to the pixel pitch d of the cylindrical
`
`surface C is Aw, the amount of change can be represented as Aw = d / R. Accordingly, when
`
`a change is made to the parameter w in units of d / R, the change coincides with the lateral
`
`direction pixel positions of the cylindrical surface C. Moreover, a range where the circular
`
`surface S has hatching in the drawings is a range of the circular surface S corresponding to
`
`7t/4 S 6 < 7t/2 and indicates the range converted into the cylindrical surface C .
`
`[0025] Meanwhile, representing the point P] on the circular image using the parameters 2
`
`and w is considered. First, when transforming the above-noted formula ® using a double—
`
`angle formula for a trigonometric function,
`
`h = 2f°tan(0/2) = 2f-{sin9 / (l + cos9)}
`
`@’
`
`results. Fig. 2 shows that sine = R / (R2 + 251/2, c050 = z / (R2 + 22)“, and therefore when
`
`these are substituted into the formula 1 and simplified, h = 2f-R / {(R2 + 22)”2 + 2)} results.
`
`Therefore, as a result, P] is (2f°R'cosw / {(R2 + 22)”2 + 2)}, 2f-R'sinw / {(R2 + 251/2 + z)}).
`
`Accordingly, as noted above, changes are made to z in units of d, and changes are made to \p
`
`in units of d / R, then the pixel positions on the circular surface S, in which each pixel
`
`position on the cylindrical surface C is converted by the formula, can be found.
`
`In such a
`
`case, the pixel positions obtained by calculation and the pixel positions actually captured by
`
`the CCD 12 fail to coincide completely, and therefore the nearest pixel is extracted.
`
`Specifically, obtained X coordinate values and Y coordinate values are, for example, divided
`
`by a pixel pitch ds of the CCD l2 and those values at or below a decimal point, for example,
`
`are discarded. Thereby, the obtained integer value can be extracted as a number for the pixel.
`
`Moreover, in such a case, where the origin point of the XY coordinates is (O, 0), each pixel of
`
`the CCD 12 is respectively defined with a positive or negative in accordance with each
`
`quadrant of the XY coordinates.
`
`[0026] Using the above method, the data converter 2 calculates the circular surface S pixel
`
`position corresponding to each pixel on the post-conversion cylindrical surface C, then
`
`converts the pixel data of the pixel positions as respective pixel data on the cylindrical
`
`surface C . As a result, as shown in Fig. 4, the pixel data on the circular surface S is converted
`
`{.1305049 02051422.DOC}10
`
`Panasonic Exhibit 1010 Page 11 of 18
`
`

`

`as respective pixel data on the cylindrical surface C. Moreover, an unambiguous
`
`determination is made as to which pixel position of the cylindrical surface each pixel of the
`
`circular image is transformed into. Therefore, the data converter 2 can be provided with a
`
`table showing a correspondence between the pixels of the circular image and the pixel
`
`positions on the cylindrical surface without performing calculations, and the data conversion
`
`can be performed based on this.
`
`[0027] Next, a description is given of operations of the image capture system having the
`
`above-noted configuration. Moreover, the camera portion 1 is mounted inside a room,
`
`oriented downward from a center of a ceiling. Further, a range of z of the converted
`
`cylindrical surface is 0 < z S R.
`
`In other words, the range OH) is defined as 7t/2 > 0 2 7t/4.
`
`Fig. 5 is a flowchart illustrating operations of the image capture system. First, a panoramic
`
`View of the room is captured by the camera portion 1 as a circular image through the fish—eye
`
`lens 1 1. The circular image is converted to an electrical signal by the CCD 12, then is further
`
`converted to digital values for each pixel by the A/D converter 5. Then, this data is stored in
`
`the first image memory 3 (3101). Furthermore, O is input for each of the parameters w and 2
`
`as an initial value (3102).
`
`[0028] Next, using the data converter 2, a parameter M is increased in units of d / R until i
`
`= l~n (n = ZnR / d), and a parameter zj is increased in units ofd untilj = l~m (m = R / d).
`
`Then, for each of the values,
`
`Xi = [2f-R-coswi/ {(123 + 2f)” + zj}-ds]
`
`Yj : [2f-R-sinwi / {(R2 + 2f)"2 + zj}-ds]
`
`is calculated and, based on the results, (Xi, Yj) order pixel data stored in the first image
`
`memory 3 is extracted and stored in the second image memory 4 as pixel data of the (i, 3')
`
`order pixel in the cylindrical surface (le3 — $108). Moreover, in the second image memory 4,
`
`in order to display the image as shown in Fig. 6 on the monitor 7, the pixels (i,j) on the
`
`cylindrical surface are divided into two parts, one part in which i = l~n/2 and j = 1~m and
`
`one part in which i : (rt/2) + l~n and j = ]~m, and are stored so as to obtain an image in
`
`which the latter part is positioned so as to be a predetermined interval below the former part.
`
`Moreover, text data indicating the angle, shown in Fig. 6, is pre~stored in the second image
`
`memory 4.
`
`[0029] The image data stored in the second image memory 4 is output to the monitor 7 as
`
`an image signal by the D/A converter 6 (5109) and is displayed by the monitor 7 (Si 10). The
`
`image data is divided into two parts and formed so as to be positioned vertically, as noted
`
`above, and thus a screen shown in Fig. 6 is displayed. In this way, in the image capture
`
`{1305049 02051422.DOC}11
`
`Panasonic Exhibit 1010 Page 12 of 18
`
`

`

`system according to the present embodiment, footage of an environment in 360° captured by
`
`the fish—eye lens can be fully and seamlessly displayed, and a status of the environment of the
`
`lens can be fully monitored without blind spots.
`
`[0030]
`
`In the above-described embodiment, a fish-eye lens was used having a property
`
`represented by the above formula 1. However, the fish—eye lens may also have a property not
`
`represented in formula 1. Specifically, when the property of the fish-eye lens is generalized
`
`to h = g(0), P2 remains as described above and P] can be represented with (g(0)-cosw,
`
`g(0)'sin\y). Therefore, in a case where the parameters are 0 and w, it is sufficient to use the
`
`same. Moreover, in a case where the parameters are 2 and w, as described above, it is
`
`possible to use the expression 0 = Tan"l (R / 2), per Fig. 2. Therefore, P2 remains as
`
`described above and P1 can be represented with (G(Tan‘l (R / 2)) )mosw, G(Tan'I (R /
`
`z))-sinw). Therefore, similar to the above, corresponding P2 and P1 pixel positions can be
`
`found while changing the values ofz and \y.
`
`[0031]
`
`For example, in a case where a commonly used fish»eye lens having a property in
`
`which h = f '0 is used as the fish-eye lens, it is sufficient to make similar conversions as
`
`above in which, when the parameters are 9 and \il, Pl is (f-G'cosw, f- 0°sinw), and when the
`
`parameters are 2 and \V, Pl is (f'G(Tan’l (R / 2)) )'cosw, f-G(Tan"1 (R / z))~sin