`
`
`
`TRANSLATOR CERTIFICATION
`
`County of New York
`State of New York
`
`
`
`To whom it may concern:
`
`Frank McGee
`I, __________________________, a translator fluent in the Japanese and English languages, on
`behalf of Morningside Translations, do solemnly and sincerely declare that the following is, to
`the best of my knowledge and belief, an accurate translation of the accompanying Japanese
`JP02117935U
`March 10, 1989
`Patent Application No. _______________ filed on _______________, titled
`Visibility Assisting Device
`_____________________________________.
`
`
`
`
`
`
`
` Frank McGee
`
`Signature
`
`April 16, 2015
`
`
`
`Date
`
`
`VALEO EXHIBIT 1005
`Valeo v. Magna
`IPR2015-____
`
`VALEO EX. 1005_001
`
`

`

`(19) Japan Patent Office (JP)
`(12) Gazette of Unexamined Utility Model Publications (U)
`
`
`(11) Unexamined Utility Model Publication Number: 2-117935
`(43) Unexamined Utility Model Publication Date: September 21, 1990
`
`(51) Int. Cl.5
`
`Identification Code
`
`
`
`Internal File Nos.
`
`
`
`
`
`
`
`
`7812-3D
`7033-5C
`
`(Total of … Pages)
`
`
`
` B
`
` 1/00
` 7/18
`
`
`
`
`
`
`
`
`J
`
` 60 R
`H 04 N
`
`Request for Examination: Not Yet Received
`Number of Claims: 1
`
`
`
`
`
`1-27739
`
`
`
`
`
`March 10, 1989
`
`
`(54) Title of Utility Model: Visibility Assisting Device For Vehicle Occupant
`
`(21) Application Number:
`
`(22) Application Date:
`
`(72) Creator:
`
`
`
`(71) Applicant:
`
`
`(74) Agent:
`
`
`
`
`
`
`
`
`
`
`Hiroshi FUJII
`Mitsubishi Motors Corporation
`5-33-8, Shiba, Minato-ku, Tokyo
`
`Mitsubishi Motors Corporation
`5-33-8, Shiba, Minato-ku, Tokyo
`
`
`Tamotsu SANADA, Patent Attorney
`
`
`
`
`
`
`
`
`
`1
`
`VALEO EX. 1005_002
`
`
`
`
`
`

`

`Specification
`
`
`
`1. Title of the Utility Model
`
`Visibility Assisting Device For Vehicle Occupant
`
`
`
`2. Claims
`
` A
`
` visibility assisting device for a vehicle occupant comprising an image
`synthesizing means for creating a series of synthesized images from video
`output from a plurality of imaging devices installed in a vehicle for imaging
`the situation outside of the vehicle including the field of view of the
`occupant, and a display means for displaying the synthesized image
`information from the image synthesizing means at a location visible to the
`occupant inside the cabin.
`
`
`
`3. Detailed Description of the Utility Model
`
`(Field of Industrial Applicability)
`
`The present utility model relates to a visibility assisting device for a vehicle
`occupant which synthesizes and displays inside the cabin video from a
`plurality of imaging devices installed in the vehicle.
`
`(Prior Art)
`
` A
`
` single video camera is sometimes used to record a portion outside of the
`field of vision of the vehicle occupant (driver) and display the video at a
`location visible to the occupant inside the cabin. This is used, for example, to
`allow the driver of a large bus to view the situation behind the vehicle in
`order to prevent collisions when the vehicle is backing up (traveling in
`reverse).
`
`(Problem to be Solved by the Utility Model)
`
`However, the device would be even more effective and useful if the situation
`outside of the vehicle viewed by the driver from the driver’s seat included
`not only a localized area captured by a single camera, but the entire
`situation outside of the vehicle including a wide area beyond the field of view
`of the driver from the driver’s seat in front of or behind the vehicle.
`
`
`2
`
`VALEO EX. 1005_003
`
`

`

`It is an object of the present utility model to solve this problem by providing
`a visibility assisting device for a vehicle occupant enabling a vehicle
`occupant from an occupant’s seat to view a series of synthesized images
`synthesized from video images captured by a plurality of imaging devices
`installed facing the front or rear of the vehicle.
`
`(Means for Solving the Problem)
`
`The present utility model achieves this object by providing a visibility
`assisting device for a vehicle occupant comprising an image synthesizing
`means for creating a series of synthesized images from video output from a
`plurality of imaging devices installed in a vehicle for imaging the situation
`outside of the vehicle including the field of view of the occupant, and a
`display means for displaying the synthesized image information from the
`image synthesizing means at a location visible to the occupant inside the
`cabin.
`
`(Operation)
`
`In the visibility assisting device for a vehicle occupant of the present utility
`model, when video output from the plurality of imaging devices is inputted
`to the image synthesizing means, the image synthesizing means synthesizes
`the video output while correcting for overlap in the video output to create a
`series of images, and the series of images is displayed on a display means at
`a location visible from an occupant’s seat inside the vehicle.
`
`(Examples)
`
`The following is an explanation of a visibility assisting device for a vehicle
`occupant in an example of the present utility model. FIG. 1 is a top view of a
`vehicle showing an example of imaging device installation for the device,
`FIG. 2 is a block diagram showing the overall configuration of the device,
`and FIG. 3 is a diagram showing the synthesized areas in the displayed
`image. In these drawings, identical components are denoted using the same
`reference numbers.
`
`First, in the present example, as shown in FIG. 1, three cameras 1, 2, 3 are
`installed along the edge of the roof of the vehicle A as imaging devices for
`capturing video in front of the vehicle A. The cameras are oriented
`differently so as to sufficiently cover the entire field of view of the occupant.
`The cameras are also arranged so that adjacent portions of each captured
`image partially overlap. Note that cameras 1-3 are not staggered vertically
`when attached.
`
`3
`
`VALEO EX. 1005_004
`
`

`

`As shown in FIG. 1, the cameras 1-3 are installed in a radial pattern with the
`cameras 1-3 centered on a single axis. However, the following arrangements
`may also be considered.
`
`(1) The cameras 1-3 may be installed in a single row and arranged at a
`predetermined interval.
`
`(2) The cameras 1-3 may be installed in a central location behind the
`windshield.
`
`(3) The cameras 1-3 may be installed behind the windshield and along the
`edge of the roof directly in front of the driver (shifted towards the driver
`from the central location).
`
`The arrangement of the cameras may be altered depending on the type of
`vehicle and the location of the driver’s seat so that the field of vision is not
`obstructed by such objects as the front pillar.
`
`In order to capture images of the situation behind the vehicle, multiple
`cameras, for example, three cameras 1’, 2’, 3’, may be installed in the rear
`portion of the vehicle A so as to face the rear of the vehicle A in a manner
`similar to the cameras facing forward.
`
`In this situation, the following arrangements may be considered for the
`cameras 1’-3’.
`
`(1) The cameras 1’-3’ may be installed in a pattern radiating from a single
`point.
`
`(2) The cameras 1’-3’ may be installed in a single row and arranged at a
`predetermined interval.
`
`(3) The cameras 1’-3’ may be installed in a central location along the edge
`of the roof (see the dotted lines in FIG. 1).
`
`(4) The cameras 1’-3’ may be installed in a central location along the outer
`edge of the rear window (see the solid lines in FIG. 1).
`
`(5) The cameras 1’-3’ may be installed in above or below a central location
`inside the rear window.
`
`The arrangement of the cameras may be altered depending on the type of
`vehicle so that the entire field of vision behind the vehicle is covered.
`
`
`4
`
`VALEO EX. 1005_005
`
`

`

`However, as shown in FIG. 2, the digital video output from each camera
`(imaging device) is sent to the image processing circuit 4 described below,
`digital video processing is performed by the image processing circuit 4, and
`a series of synthesized images are outputted to an image display unit
`(display means) 7 such as a cathode ray tube or liquid crystal panel via a
`display output unit (the drive circuit functioning as the output interface) 6.
`
`This image display unit 7 is located in or near the instrument panel
`(dashboard) where an occupant such as the driver can easily view the unit
`from his or her seat.
`
`The image processing circuit 4 functions as an image synthesizing means
`which synthesizes a series of synthesized images from the digital video
`output of the cameras 1-3 by digitally processing the digital video output
`from the cameras 1-3. In order to function in this manner, the image
`processing circuit 4 includes image input units 8-10 serving as input
`interfaces for inputting images from the cameras 1-3, and an arithmetic
`processing unit 5 for creating the desired synthesized image from the
`inputted video signals on the basis of the inputted video signals from the
`image input units 8-10. The arithmetic processing unit 5 includes a
`comparison extraction unit 11 for extracting patterns in order to compare
`images, a comparison determining unit 12 to compare images and
`determine whether the images include the same pattern, and an image
`correcting unit 13 for correcting images to be synthesized.
`
`The following operations are performed by an image processing circuit 4
`including an arithmetic processing unit 5, comparison extraction unit 11,
`comparison determining unit 12, and an image correcting unit 13.
`
`First, as shown in the upper portion of FIG. 3, the comparison extraction unit
`11 extracts three image patterns (for example, PTA, PTB, PTC) including a x
`b pixels from P1 image 14 in the overlapping portions c, d of P1 image 14
`from the image input unit 8 for camera 1 and P2 image 15 from the image
`input unit 9 for camera 2.
`
`Next, the comparison determining unit 12 performs a comparison to
`determine whether or not the patterns match in overlap portion d of the P2
`image. For example, PTA data from P1 image 14 is inputted to the
`comparison determining unit 12, and is successively compared to pixel
`patterns with the same number of pixels beginning in the upper left corner
`of the P2 image 15. In this way, PTA’ data can be searched for data with the
`same pattern as PTA data. Other pattern data PTB, PTC can be searched in
`the same manner.
`
`
`5
`
`VALEO EX. 1005_006
`
`

`

`The image correcting unit 13 corrects the images in order to join
`(synthesize) them by performing a coordinate conversion on data with the
`same patterns (for example, PTA and PTA’) retrieved from overlapping
`portions c, d in P1 image 14 and P2 image 15. For example, when the
`coordinates in the upper left corner of PTA image in P1 image 14 are (a1, b1)
`and the coordinates in the upper left corner of PTA’ image in P2 image 15
`are (x1, y1), the image correcting unit 13 performs a coordinate conversion
`by moving P1 image 14 horizontally by x1-a1 and vertically by y1-b1 until the
`images in the overlapping portions c, d of P1 image 14 and P2 image 15
`match.
`
`The same processing is performed on the other patterns (PTB, PTC) and the
`processing results are handled in a comprehensive manner to join and
`synthesize P1 image 14 and P2 image 15.
`
`The overlapping regions of P2 image 15 and P3 image 16 are corrected in
`the same way to join and synthesize P2 image 15 and P3 image 16.
`
`In this way, P1 image 14, P2 image 15, and P3 image 16 are synthesized as
`shown in the lower portion of FIG. 3.
`
`This image synthesis processing is performed by the arithmetic processing
`unit 5.
`
`Afterwards, the synthesized images are displayed via the display output unit
`6 on the image display unit 7.
`
`In FIG. 2, the overlapping portions of the synthesized image displayed on
`the image display unit 7 are denoted by cross-hatching. No cross-hatching is
`displayed on the image display unit 7 during actual use.
`
`Because the display areas 1a, 2a, 3a in images P1, P2 and P3 cover the
`entire field of view from cameras 1, 2 and 3, the desired portions (denoted
`by 7’) are included in the series of images that are displayed on the image
`display screen 7 by the display output unit 6. In other words, the area
`denoted by 7’ in FIG. 3 is displayed on the image display unit 7.
`
`Image processing is performed in the same manner on the digital video
`output of the camera 1’-3’ installed to face the rear to obtain synthesized
`images.
`
`When the height of the cameras differs from the eye line of the occupant,
`the distance between the lines of sight can be interpolated to adjust the
`displayed images to the occupant’s eye line.
`
`6
`
`VALEO EX. 1005_007
`
`

`

`
`In the preceding explanation, the cameras were arranged horizontally.
`However, the same processing can be performed on images from cameras
`installed in other directions if necessary. In this case, the cameras must be
`installed so as not to diverge from each other in the vertical direction.
`
`In the visibility assisting device for a vehicle occupant, as described above, a
`plurality of cameras (imaging devices) are installed facing the front or rear
`of the vehicle, images from these cameras are received by an image
`processing circuit 4, the relationship between overlapping portions of images
`extracted by the image processing circuit 4 is compared and determined,
`corrected images are selected to create a series of synthesized images, and
`these are displayed on the image display unit 7 via the display output unit 6
`to show the situation in front of or behind the vehicle within the angle of
`vision of the occupant or within an even larger range.
`
`Currently, when a vehicle is backing up but the seat of the occupant is facing
`the front windshield, the occupant has a good view of conditions through the
`front windshield but a narrow view of conditions via the rear window. This
`can cause problems. However, when the visibility assisting device for a
`vehicle of the present utility model is used, the occupant has a clear and
`accurate view of conditions outside of the vehicle in front and to the rear of
`the vehicle from the occupant’s seat in an area at least as wide as the
`occupant’s field of view.
`
`The imaging devices can be any type of imaging device able to output video
`images, such as video cameras, ITVs, and CCDs.
`
`(Effect of the Utility Model)
`
`As explained above, the visibility assisting device for a vehicle of the present
`utility model includes a plurality of imaging devices installed so as to monitor
`the situation outside of the vehicle including the field of view of the occupant
`of the vehicle. The video output from these imaging devices is adjusted and
`corrected to obtain a series of synthesized images which can be viewed from
`the occupant’s seat. By installing imaging devices in predetermined locations
`in front of or in the rear of a vehicle, the occupant can be given a clear,
`unobstructed view of conditions outside of the vehicle including within the
`field of view of the occupant. This makes operating a vehicle easier and
`safer.
`
`
`
`4. Brief Explanation of the Drawings
`
`
`7
`
`VALEO EX. 1005_008
`
`

`

`FIG. 1 through FIG. 3 show a visibility assisting device for vehicle occupants
`in an example of the present utility model. FIG. 1 is a top view of a vehicle
`showing an example of imaging device installation for the device, FIG. 2 is a
`block diagram showing the overall configuration of the device, and FIG. 3 is a
`diagram showing the synthesized areas in the displayed image.
`
`
`
`1, 2, 3, 1’, 2’, 3’: cameras (imaging devices)
`1a, 2a, 3a: display areas
`4: image processing circuit (image synthesizing means)
`5: arithmetic processing unit
`6: display output unit
`7: image display unit (display means)
`8, 9, 10: camera image input unit
`11: comparison extraction unit
`12: comparison determining unit
`13: image correcting unit
`14: P1 image
`15: P2 image
`16: P3 image
`A: vehicle
`c, d: overlapping regions
`
`
`
`Agent
`
`
`
`Tamotsu SANADA, Patent Attorney
`
`
`
`
`
`8
`
`VALEO EX. 1005_009
`
`

`

`
`
`FIG. 1
`
`
`
`
`
`FIG. 2
`
`
`Tamotsu SANADA, Patent Attorney
`
`Agent
`Unexamined Utility Model Publication Number: 2-117935
`
`
`
`9
`
`VALEO EX. 1005_010
`
`

`

`FIG. 3
`
`
`
`Tamotsu SANADA, Patent Attorney
`
`Agent
`Unexamined Utility Model Publication Number: 2-117935
`
`
`
`
`10
`
`VALEO EX. 1005_011
`
`