`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`RWS Group Ltd, of Europa House, Chiltern Park, Chiltern Hill, Chalfont St Peter,
`
`Buckinghamshire, United Kingdom, hereby declares that, to the best of its knowledge and
`
`belief, the following document, prepared by one of its translators competent in the art and
`
`conversant with the English and Japanese languages, is a true and correct translation of the
`
`accompanying Japanese Patent Application No. H3-99952 filed on 25 April 1991.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Signed this 10th day of October 2013
`
`
`
`
`
`
`
`C. E. SITCH
`
`Managing Director – UK Translation Division
`
`For and on behalf of RWS Group Ltd
`
`VALEO EX. 1004_001
`
`
`
`(19) Japanese Patent Office
`(JP)
`
`
`
`(11) Patent application
`publication number: H3-99952
`(12) Official Gazette for Unexamined Patents (A)
`(43) Application
`publication date: April
`25, 1991
`Internal File Nos.
`7812-3D
`7626-3D
`7033-5C
`
`(51) Int. Cl.5
`B 60 R 1/00
`21/00
`H 04 N 7/18
`
`Identification
`Nos.
`C
`J
`
`Number of Claims: 1
`
`(Total 5 Japanese
`pages)
`
`Vehicle Surroundings Monitor
`
`Request for
`Examination:
`Not filed
`(54) Title of
`the Invention
`
`
`
`
`(72) Inventor
`
`(21) Application
`No.
`(22) Application
`Date
`Kazunori Noso
`
`Patent Application No.
`H1-234612
`September 12, 1989
`
`Nissan Motor Co., Ltd.
`2, Takara-cho, Kanagawa-
`ku, Yokohama, Kanagawa
`Prefecture
`2, Takara-cho, Kanagawa-
`ku, Yokohama, Kanagawa
`Prefecture
`Hidekazu Miyoshi, Patent Attorney, and one
`other
`
`(71) Applicant Nissan Motor Co.,
`Ltd.
`
`(74) Agent
`
`VALEO EX. 1004_002
`
`
`
`
`
`
`
`
`
`
`Specifications
`1. Title of the Invention
`Vehicle Surroundings Monitor
`2. Patent Claim
`A vehicle surroundings monitor characterized in having
`one or a plurality of cameras installed in a vehicle,
`means for converting images input by said cameras to
`other coordinates by a perspective conversion, means
`for combining said converted images into one image
`related to an image of the vehicle itself, and a
`display for displaying said image to the vehicle
`occupants.
`3. Brief Description of the Invention
`Objectives of the Invention
`Industrial Field of the Invention
`The present invention relates to a device for
`displaying the positions of other vehicles, the states
`of obstructions, and the center line, etc. as
`environment information of the vehicle surroundings to
`the driver by using television (TV) cameras (if an
`object that acquires images is simply referred to as a
`camera, then the type of camera does not matter in
`these Specifications).
`Prior Art
`A conventional vehicle monitor is as shown, for
`example, in Figure 5. This system has a camera 21 set
`up in the back of the vehicle directed towards the rear
`(Figure 5(a)), reverses the obtained image for left-
`right conversion of the image in an image signal
`processor 22 (Figure 5(b)), displays the image on a
`monitor TV 23, and obtains the so-called rearview image
`as in Figure 5(c) for monitoring. The driver can
`determine how to back up the vehicle by looking at this
`image.
`Problems to Be Solved by the Invention
`However, this type of conventional vehicle monitor is a
`device for simply displaying the rearview image as
`shown in Figure 5(c). One camera is inadequate because
`it is difficult to get a feel of the distance between
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`VALEO EX. 1004_003
`
`
`
`
`
`
`
`- 3 -
`
`the vehicle and the obstructions and to understand the
`environment surrounding the vehicle. Even if the number
`of cameras is increased, and the number of monitors is
`increased, if there is no awareness about the relative
`positions of the cameras, it is difficult to
`sufficiently understand the environment. Therefore, a
`problem was that suitable driving measures could not be
`obtained. The present invention has the objective of
`providing a vehicle surroundings monitor that enables
`adequate awareness of the relationship between the
`vehicle and the surrounding environment and taking
`suitable driving measures to solve said problems.
`Structure of the Invention
`Means for Solving the Problems
`To achieve the objective, the present invention has one
`or a plurality of cameras installed in a vehicle, means
`for converting images input by said cameras to other
`coordinates by a perspective conversion, means for
`combining said converted images into one image related
`to an image of the vehicle itself, and a display for
`displaying said image to the occupants.
`Operation
`For example, a camera image is converted to an image,
`in plane coordinates and is displayed so that the
`vehicle is at the origin in these coordinates. The
`driver of the vehicle understand the relationship
`between the traveling direction of the vehicle and the
`state of surroundings by looking at the display, and
`can take appropriate measures.
`Working Examples
`A working example of the present invention is described
`below with reference to Figures 1 to 4.
`
`Figure 1 is a plane view of a vehicle equipped with
`cameras of the present invention. First, the structure
`is explained. A plurality of cameras 1-6 is set up so
`that the respective imaging ranges (field of view) 1a-
`6a cover the environment surrounding the vehicle as
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`VALEO EX. 1004_004
`
`
`
`
`
`
`
`- 4 -
`
`much as possible. Specifically, two cameras are
`embedded in each of the front and rear bumpers, and one
`in each front directional indicator. Alternatively, the
`cameras may be set in the rear combination lamp instead
`of the rear bumper. If a mirror is placed in front of
`the lens in the camera and a reversed image is input,
`the camera becomes easy to install, and the degrees of
`freedom in modeling of the image increases. The number
`of cameras may be one or any number. Alternately, the
`cameras are positioned to give importance to the road
`surface and are oriented downward somewhat.
`
`Figure 2 is a block diagram showing the structure of a
`processor of the images captured by the cameras in
`Figure 1. In an image converter 7, the images from
`cameras 1 to N are input and converted to other
`coordinates by a perspective conversion, are combined
`into one image by an image display unit 8, then
`displayed on a TV monitor 9 positioned at the driver’s
`seat. Preferably, the image converter 7 has an internal
`high-speed processor capable of parallel processing of
`the images of only N cameras. In the image display unit
`8, the vehicle’s position is simultaneously drawn in a
`diagram. The displayed position of the vehicle is
`offset from the center of the image screen depending on
`signals corresponding to the gear position, the vehicle
`speed, and the operation of the directional indicator.
`The region of the vehicle environment to be viewed is
`widened. Figure 3 is an example of display results of
`these screens, and each is displayed in plane (road
`surface) coordinates. Figure 3(a) shows backing up
`(direction of the arrow) and is a fixed display with
`the vehicle 10 in the upper part in the center. The
`other neighboring vehicle 11 is displayed in a side
`view. Figure 3(b) is the operation of the right
`directional indicator. A lane change (arrow) to the
`right is assumed, and another vehicle 11 behind in the
`same direction can be displayed.
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`VALEO EX. 1004_005
`
`
`
`
`
`- 5 -
`
`
`
`
`
`Figure 3(c) is an example when stopped before an
`intersection. At the intersection, there is a wall 12
`and the view is obstructed, therefore, the vehicle
`should stop because there is another vehicle 11.
`Another vehicle 11’ that is behind is displayed.
`
`Figure 4 is a flow chart of the process of the image
`converter 7 in Figure 2. The process of this converter
`is described.
`
`The images of cameras 1-N are input simultaneously to
`the image converter 7, and the images are set to
`Ai(x,y), where i indicates the camera number. Next, the
`perspective conversions (coordinate conversion) of the
`N images are performed in parallel. If the images after
`conversion are set to Bi(x,y) and are represented by
`
`
`where ai, bi, ci, di, ei, fi, gi, hi, and ki are the
`parameters determined by the focal lengths of the
`cameras and the set-up positions of the cameras (angle
`with respect to the vehicle, angle with the road
`surface, etc.). Once the parameters are set, they are
`completely determined and can be determined in advance
`and stored. The perspective conversion converts to the
`road surface (plane) coordinates (center of vehicle is
`the origin, left and right sides of the traveling
`vehicle are along the X axis, the traveling direction
`of the vehicle is the Y axis) from the screen
`coordinates of the camera.
`
`Next, in the image display unit 8, each image converted
`to the coordinates described above is constructed.
`Thus, it is determined in what environment to display
`the vehicle in response to the directional indicator
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`VALEO EX. 1004_006
`
`
`
`
`
`
`
`- 6 -
`
`operation, the gear position, and the vehicle speed.
`That coordinate is (Xo, Yo). Next, the monitor image
`C(X,Y) is cleared, and a figure of the vehicle is drawn
`with (Xo,Yo) at the center. Next, a camera conversion
`image Bi(X,Y) (i – 1 to N: number of cameras) is sent
`to C(X,Y). In this region overlapping the field of
`view, the average value of the density is set.
`Specifically, the following is acceptable.
`C(X,Y) =
`
`Bi(X-Xo, Y-Yo)...when C(X,Y) = 0
`
`C(X,Y)...when B(X-Xo, Y-Yo) = 0
`
`1/2(C(X,Y) + Bi(X-Xo, Y-Yo))... otherwise
`where i = 1 – N
`X = 1 – Mx (number of pixels in the horizontal
`direction of the display image)
`Y = 1 – My (number of pixels in the vertical direction
`of the display image)
`
`From the above, the converted image of each camera in
`image C(X,Y) and the drawn vehicle figure can be
`displayed together.
`
`By looking at the display device described above, the
`driver (and other occupants) can behave appropriately
`in the future when backing up as in Figure 3(a), or
`moving to the right lane as in (b), or when passing
`through an intersection with an obstructed view as in
`(c). The conversion coordinates may have an inclined
`axis on the road surface as the reference. The
`coordinates are not limited to the road surface and may
`be, for example, on a plane at some height above the
`road surface.
`
`Furthermore, depending on the gear position or the
`operating state of the directional indicator, the
`region of the surrounding state desired for viewing can
`be widened and displayed by offsetting the display
`position of the vehicle on the TV monitor from the
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`VALEO EX. 1004_007
`
`
`
`
`
`
`
`- 7 -
`
`center. In addition, at fast speeds, the image on the
`TV monitor is compressed and displayed, and the display
`region is widened and displayed, thus, more effective
`images can be obtained.
`
`In addition, if a color TV is used to display in color,
`objects can be easily recognized.
`
`In addition, if infrared cameras are used at night, the
`recognition of obstructions becomes easier even if the
`illumination is inadequate.
`Effects of the Invention
`As in the above description, according to the present
`invention, this structure has one or a plurality of
`cameras installed in a vehicle, means for converting
`the images input by the cameras to other coordinates by
`a perspective conversion, means for combining the
`converted images into one image in relation to an image
`of the vehicle, and a display for displaying the image
`to the occupants. Therefore, the relative positions of
`the obstructions, the other vehicles surrounding the
`vehicle, and the center line to the vehicle itself can
`be easily recognized. Thus, the effect obtained is that
`the driver can take appropriate measures.
`
`4. Brief Descriptions of the Drawings
`Figure 1 is a plane view of a vehicle showing an
`example of the camera installation of the present
`invention. Figure 2 is a structural block diagram of
`the device of the present invention. Figure 3 is an
`example of the display results of the TV monitor of the
`present invention. (a) shows the image when backing up;
`(b), when operating the right directional indicator;
`and (c), when stopping. Figure 4 is a flow chart of the
`image processing of the block diagram in Figure 2.
`Figure 5 is a drawing of a conventional example, where
`(a) is an example of an installation diagram of the TV
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`VALEO EX. 1004_008
`
`
`
`
`
`
`
`- 8 -
`
`camera; (b), an example of the process structure
`diagram; and (c), an example of an image.
`
`cameras
`1,2,3,4,5,6
`7
`image converter
`8
`image display
`9
`TV monitor
`
`Hidekazu Miyoshi, Patent Attorney
`
`5
`
`10
`
`VALEO EX. 1004_009
`
`
`
`
`
`Figure 1
`
`- 9 -
`
`
`
`
`AA Forward
`1,2,3,4,5,6
`
`Figure 2
`
`5
`
`
`
`cameras
`
`
`
`Camera
`1,2,N
`7
`Image converter
`8
`Image display unit
`9
`TV monitor
`AA
`Gear position
`BB
`Speed
`CC
`Directional indicator operation
`DD
`Vehicle
`
`10
`
`15
`
`VALEO EX. 1004_010
`
`
`
`
`
`Figure 3(a)
`
`- 10 -
`
`
`
`
`Figure 3(b)
`
`
`
`5
`
`Figure 3(c)
`
`
`
`
`
`VALEO EX. 1004_011
`
`
`
`
`
`Figure 4
`
`- 11 -
`
`
`
`
`5
`
`10
`
`15
`
`20
`
`
`
`Input image -> Ai(x,y)
`AA
`Input image -> Ai+1(x,y)
`BB
`Coordinate conversion
`CC
`Image synthesis
`DD
`Determine Xo and Yo from directional indicator and
`EE
`gear
`FF
`Clear the image screen C(X,Y), and draw the
`vehicle at (Xo,Yo)
`GG
`i = 1-N (number of cameras) loop
`HH
`Image scanning x = 1 – Mx } 2 overlapping loops
`
`
`
` y = 1 – My
`II
`C(X,Y) =
`
`Bi(X-Xo, Y-Yo)...when C(X,Y) = 0
`
`C(X,Y)...when Bi(X-Xo, Y-Yo) = 0
`
`(C(X,Y) + Bi(X-Xo, Y-Yo))... otherwise
`JJ
`Does x = Mx and y = My?
`KK
`Is i = N?
`LL
`Display the image C(X,Y).
`
`Figure 5(a)
`
`(Driver’s seat)
`22,23
`AA
`Forward
`
`25
`
`
`
`VALEO EX. 1004_012
`
`
`
`
`
`Figure 5(b)
`
`- 12 -
`
`
`
`
`Camera
`21
`Image signal processor
`22
`Monitor TV
`23
`Figure 5(c)
`
`5
`
`
`
`
`
`
`
`VALEO EX. 1004_013
`
`
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
