US 8,854,432 B2
`(10) Patent No.:
`(12) Unlted States Patent
`
`Orimoto
`(45) Date of Patent:
`Oct. 7, 2014
`
`USOO8854432B2
`
`(54) MULTI-LENS CAMERA AND CONTROL
`METHOD
`
`.
`.
`.
`.
`Inventor. Masaakl Orlmoto, Sa1tama (JP)
`
`(75)
`
`(73) Assignee: FUJIFILM Corporation, Tokyo (JP)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 15403) by 1052 days.
`
`JP
`JP
`JP
`JP
`Jp
`JP
`JP
`JP
`JP
`
`FOREIGN PATENT DOCUMENTS
`06-054349 A
`2/1994
`11-355624 A
`12/1995
`11027703 A
`“1999
`2003-052058 A
`2/2003
`2004354257 A
`12/2004
`2006121229 A
`5/2006
`2006162991 A
`6/2006
`2006165894 A
`6/2006
`2007052060 A
`3/2007
`OTHER PUBLICATIONS
`
`Notification of Reasons for Refusal, dated Nov. 28, 2012, issued in
`corresponding JP Application No. 2009-218771, 6 pages in English
`and Japanese.
`
`(21) Appl.No.: 12/887,134
`
`(22)
`
`Filed:
`
`Sep. 21, 2010
`
`Prior Publication Data
`
`(65)
`
`(30)
`
`US 2011/0069151 A1
`
`Mar. 24, 2011
`
`Foreign Application Priority Data
`
`Primary Examiner 4 Young Lee
`(74) Attorney, Agent, or Firm 7 Sughrue Mion, PLLC
`
`Sep. 24, 2009
`
`(JP) ................................. 2009-218771
`
`(57)
`
`ABSTRACT
`
`(51)
`
`Int. Cl.
`H04N13/02
`G03B 35/00
`(52) US. Cl.
`CPC
`
`(2006.01)
`(2006.01)
`
`H04N 13/0239 (2013 01). G03B 35/00
`(2013 01). H04N 1.3/02346 (2013 01)
`iiiiiiiiiiii
`USPC ........................ ......,............................. 348/47
`(58) Field of Classification Search
`USPC ...................................................... 348/42, 47
`IPC ....................................................... H04N 13/02
`
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`US. PATENT DOCUMENTS
`
`7,102,686 B1
`7,667,768 B2
`2006/0268159 A1
`
`9/2006 Orimoto et a1.
`2/2010 Orimoto et a1.
`11/2006 Orimoto et a1.
`
`A dual lens camera for producing a three-dimensional image
`includes plural lens systems and a zoom mechanism. Initial
`correction data is constituted by a displacement vector of an
`amount and a direction of misalignment between plural
`images according to a superimposed state thereof for each of
`zoom positions of the lens systems. A vector detector, if a
`calibration mode is set, obtains a current displacement vector
`related to one first zoom position. A data processor outputs
`current correction data by adjusting the initial correction data
`according to the initial correction data and current displace-
`ment vector. If the current correction data is stored, a dis-
`placement vector is obtained from the current correction data
`according to a zoom position of the lens systems upon form-
`ing the plural images, to carry out image registration between
`the images according to the obtained displacement vector for
`producing the three-dimensional image.
`
`11 Claims, 10 Drawing Sheets
`
`
`/ 2
`
`
`
`
`STORAGE
`MEDIUM
`
`—— SHUTTER BUTTON I‘ll
`
`—— MODE DESIGNATION WHEEL I'\v13
`
`POWER sw |—»12
`
`ZOOM BUTTON [~14
`CROSS SHAPED KEY
`17
`
`
`
`
`
`
`
`
`
`813
`
`
`
`
`
`
`-— MENU BUTTON I45
`CONFIRMATION
`l8
`
`BUTTON
`
`EEPROM
`FOCAL LENGTH INFO
`
`81
`
`APPL-1014/Page 1 of 19
`Apple v. Corephotonics
`
`APPL-1014 / Page 1 of 19
`Apple v. Corephotonics
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 1 of 10
`
`US 8,854,432 B2
`
`
`
`APPL-1014 / Page 2 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 2 of 10
`
`US 8,854,432 B2
`
`FIG. 2
`
`
`
`
`
`
`
`
`
`APPL-1014 / Page 3 of 19
`
`APPL-1014 / Page 3 of 19
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 3 of 10
`
`US 8,854,432 B2
`
`mu<m09m
`
`355m:
`
`om
`
`22sz
`
`mmnSOELZOQ
`Eémm
`
`
`
`
`
`EOEm—z<F~<QZOEUmmmOU
`
`
`
`vm<63ZOFummmooHzmmmzu
`
`
`
`<55ZOEQEMOD12E;
`
`Dmo
`
`
`
`ZOHHDmmmHHDIm
`
`1Q1
`
`MagmaR:R:$sz
`
`mug
`
`
`
`
`
`____
`
`__
`
`
`2onEm53E
`\\.mmUHm
`I'58:
`
`nHé9mmHm
`
` _mmmm_
`
` ammlgxmm$.03wwJ|x|:|:|.....gm;
`
`I.3DEm3m8nmw$0-!0:fl.%mma1%we;mo{
`
`2m5
`
`E75:3208
`
`aEN532
`
`
`2
`
`
`
`15ng29529memac}
`
`5Examm/EmmmOmo
`
`
`
`
`
`m:zoribm32m:
`
`maz©F<§mEzoo
`
`zorrSm
`
`_a
`
`mm
`
`
`
` moo
`
`
`
`
`
`Omzfi5.0231200””
`
`
`filli--|--|:mmflm.g-|_
`
`APPL-1014 / Page 4 of 19
`
`APPL-1014 / Page 4 of 19
`
`
`
`
`
`
`
`
`
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 4 of 10
`
`US 8,854,432 B2
`
`PIG.4
`
`,,/90R,90L
`/_ _
`- _
`
`91L
`
`91R
`
`I
`1
`
`PIG.5
`
`CORRECTION DATA
`
`W END
`
`DISPLACEMENT VECTOR MW
`
`ZOOM POSITION I DISPLACEMENT VECTOR MI
`
`ZOOM POSITION 2 DISPLACEMENT VECTOR M2
`
`ZOOM POSITION 3
`
`D SPLACEMENT VECTOR M3
`
`
`DISPLACEMENT VECTOR Mt
`
`ZOOM POSITION n
`
`DISPLACEMENT VECTOR Mn
`
`T END
`
`APPL-1014 / Page 5 of 19
`
`APPL-1014 / Page 5 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 5 of 10
`
`US 8,854,432 B2
`
`FIG.6
`
`START
`
`READ R & L EYE IMAGES
`
`DETERMINE ZOOM POSITION OP LENS SYSTEMS
`
`OBTAIN DISPLACEMENT
`
`IS CURRENT CORRECTION DATA STORED?
`YES
`
`
`
`
`OBTAIN DISPLACEMENT
`
`
`
`INITIAL CORRECTION DATA
`CORRECTION DATA
`
`
`
`VECTOR M PROM
`
`VECTOR M PROM CURRENT
`
`DETERMINE OVERLAP REGION
`
`AS PER VECTOR M
`
`RETRIEVE OVERLAP REGION PROM IMAGE DATA
`
`PRODUCE 3D IMAGE DATA
`
`AS PER OVERLAP REGION
`
`APPL-1014 / Page 6 of 19
`
`APPL-1014 / Page 6 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 6 of 10
`
`US 8,854,432 B2
`
`F1 G . 7 A
`
`90L
`/90R
`
`_
`_
`W
`U
`_
`__
`_
`/_-_
`,/
`
`_____________#______-:[L__;____
`I
`.
`H
`l‘
`
`l
`:
`‘
`
`l
`:
`\
`:
`!
`
`\
`i
`I
`
`H
`1
`
`‘
`
`.
`
`l
`
`__
`
`
`
`
`
`L—-~
`
`l
`I:
`\
`1
`:w
`
`‘
`
`I
`
`I
`
`i
`I
`I
`'1
`F\—92R,92L
`:
`_______g
`___ _
`,
`_._
`# _______________________ J
`
`FIG.7B
`
`91R,91L
`
`“‘92R,92L
`
`APPL-1014 / Page 7 of 19
`
`APPL-1014 / Page 7 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 7 of 10
`
`US 8,854,432 B2
`
`PIG.8
`
`START
`
`SHIFT L EYE IMAGE TO REGISTER REP POINTS
`
`
`
`YES
`
`OBTAIN DISPLACEMENT VECTOR Mt RELATED TO T END
`
`MOVE LENS SYSTEMS TO W END
`
`PICK UP R 81 L EYE IMAGES
`
`SUPERIMPOSE & DISPLAY IMAGES
`
`
`
`SHIFT L EYE IMAGE TO REGISTER REP POINTS
`
`
`
`
`YES
`
`OBTAIN DISPLACEMENT VECTOR MW RELATED TO W END
`
`DETERMINE VECTORS Mn RELATED TO ZOOM POSITIONS
`
`STORE VECTORS Mt, Mw & Mn AS INITIAL CORRECTION DATA
`
`END
`
`APPL-1014 / Page 8 of 19
`
`APPL-1014 / Page 8 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 8 of 10
`
`US 8,854,432 B2
`
`FIG.9
`
`SET CALIBRATION MODE
`
`
`
`
`
`SUPERIMPOSE & DISPLAY IMAGES
`
`
`SHIFT L EYE IMAGE TO REGISTER OBJECT
`
`CONFIRMED?
`
`YES
`
`
`
`OBTAIN DISPLACEMENT VECTOR Mc’ RELATED TO
`
`ZOOM POSITION C UPON DEPRESSING BUTTON
`
`DETERMINE DIFFERENCE VECTOR nd
`
`RELATED TO ZOOM POSITION C
`
`
`
`
`
`
`
`
`
`
`
`DETERMINE DIFFERENCE VECTOR dMn
`RELATED TO GIVEN ZOOM POSITION
`
`
`
`
`DETERMINE DISPLACEMENT VECTOR Mn’
`
`RELATED TO GIVEN ZOOM POSITION
`
`
`
`ARE DISPLACEMENT VECTORS Mn’
`DETERMINED FOR ALL ZOOM POSITIONS?
`
`
`
`STORE DISPLACEMENT VECTORS Mc' & Mn’
`
`FOR CURRENT CORRECTION DATA
`
`
`
`APPL-1014 / Page 9 of 19
`
`APPL-1014 / Page 9 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 9 of 10
`
`US 8,854,432 B2
`
`
`
`APPL—1014 / Page 10 of 19
`
`APPL-1014 / Page 10 of 19
`
`

`

`US. Patent
`
`Oct. 7, 2014
`
`Sheet 10 of 10
`
`US 8,854,432 B2
`
`FIG.11
`
`START
`
`SET CALIBRATION MODE
`
`MOVE LENS SYSTEMS TO T END
`
`SUPERIMPOSE & DISPLAY IMAGES
`
`SHIFT L EYE IMAGE TO REGISTER OBJECT
`
`
`
`CONFIRMED?
`
`OBTAIN DISPLACEMENT VECTOR Mt’ RELATED TO T END
`
`YES
`
`DETERMINE DIFFERENCE VECTOR th RELATED TO T END
`
`
`
`DETERMINE DIFFERENCE VECTOR dMn
`
`RELATED TO GIVEN ZOOM POSITION
`
`
`DETERMINE DISPLACEMENT VECTOR Mn’
`RELATED TO GIVEN ZOOM POSITION
`
`
`
`ARE DISPLACEMENT VECTORS Mn’
`
`DETERMINED FOR ALL ZOOM POSITIONS?
`
`YES
`
`STORE DISPLACEMENT VECTORS Mt’ & Mn’
`
`FOR CURRENT CORRECTION DATA
`
`END
`
`APPL—1014 / Page 11 of 19
`
`APPL-1014 / Page 11 of 19
`
`

`

`US 8,854,432 B2
`
`1
`MULTI-LENS CAMERA AND CONTROL
`METHOD
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to a multi-lens camera and a
`control method. More particularly,
`the present
`invention
`relates to a multi-lens camera of which plural images are
`created with parallax, and misalignment between plural lens
`systems can be eliminated easily and exactly, and a control
`method for control of the same.
`
`2. Description Related to the Prior Art
`A dual lens camera as multi-lens camera for producing a
`three-dimensional image by combining plural images with
`parallax is known, and includes plural lens systems. To obtain
`the three-dimensional image of good quality in the dual lens
`camera, misalignment other than suitable parallax between
`the images should be suppressed in the lens systems. How-
`ever, skew may occur with optical axes of the lens systems
`due to various reasons such as errors in dimensions of parts,
`errors in assembly and the like. The misalignment will occur
`unacceptably due to the skew of the optical axes. There have
`been a number of suggestions in the dual lens camera for
`preventing the misalignment of the plural images in occur-
`rence of skew of the optical axes of the lens systems.
`In JP-A 2003-052058, image registration for the images is
`disclosed in which positions of reading the images are
`adjusted according to a displacement amount between a cen-
`ter of an image pickup device and a center of the optical axes
`of the lens systems, to read only positions corresponding
`between the images. In JP-A 6-054349, a mechanism for
`shifting the optical axes is disclosed to adjust a direction of
`the optical axes in the lens systems in relation to an elevation
`angle direction, azimuthal angle direction, and horizontal
`direction. Image registration is carried out for the images by
`regulating the optical axes of the lens systems by the shifting
`mechanism. In U.S. Pat. Nos. 7,102,686 and 7,667,768 (cor-
`responding to JP-A 11-355624), a cross shaped key is used to
`input information of a direction and amount of shift finely to
`adjust the position and angle of the images. Limited portions
`are retrieved from image frames of the images according to
`the direction and amount of the shift, to remove the misalign-
`ment for the image registration.
`A zoom mechanism is additionally incorporated in each of
`the lens systems. An amount ofcorrecting the misalignment is
`different according to a zoom position (focal length) of the
`lens systems. It is necessary to carry out image registration by
`adjustment for each of the zoom positions of the lens systems
`in the structure with the zoom mechanism in the lens systems,
`to complicate the operation excessively.
`Generally, a manufacturer of the dual lens camera carries
`out the image registration in the manufacture to suppress the
`misalignment of plural images. Among numerous users or
`operators purchasing the dual lens camera, somebody may be
`unfamiliar to manual handling of mechanical parts, and will
`not carry out the image registration successfully by himselfor
`herself. However, the image registration on the side of the
`manufacturer is suitable for the user ofthe dual lens camera to
`
`use the dual lens camera easily and sufficiently to obtain the
`three-dimensional image without noticing the misalignment
`of the plural images.
`However, an accident may occur to give extraordinary
`shock to the dual lens camera typically when the user drops or
`strikes the dual lens camera. The misalignment is likely to
`occur with images as the optical axes ofthe lens systems may
`skew no matter how exactly the dual lens camera has been
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`conditioned and adjusted by the manufacturer. If a user or
`operator uses the dual lens camera during a trip or under other
`unfamiliar conditions, it is very difficult to have the manufac-
`turer to repair the dual lens camera. It is preferable practically
`for the user to correct the misalignment of the images by
`readjustment for image registration. The image registration
`with difficulties is not acceptable so that he or she may give up
`in continuous use of the dual lens camera. Also, difficult
`operation for the image registration will result in failure to
`cause the misalignment to remain after the readjustment.
`There is no suggestion of correcting the misalignment of the
`images by readjustment easily to end users who may be
`unskilled in mechanical operation.
`
`SUMMARY OF THE INVENTION
`
`In view of the foregoing problems, an object of the present
`invention is to provide a multi-lens camera of which plural
`images are created with parallax, and misalignment between
`plural lens systems can be eliminated easily and exactly, and
`a control method for control of the same.
`
`In order to achieve the above and other objects and advan-
`tages of this invention, a multi-lens camera for producing a
`three-dimensional image from plural images with parallax is
`provided, and includes plural lens systems for receiving entry
`of obj ect light to form the plural images. A zoom mechanism
`changes a magnification of each of the lens systems within a
`zooming range. A first correction data memory stores initial
`correction data constituted by a displacement vector of an
`amount and a direction of misalignment between the plural
`images according to a superimposed state thereof for each of
`zoom positions of the lens systems. A mode designation
`device sets a calibration mode for adjusting the initial correc-
`tion data. A vector detector, if the calibration mode is set,
`obtains a current displacement vector between the plural
`images in relation to one first zoom position. A data processor
`outputs current correction data by adjusting the initial correc-
`tion data according to the initial correction data and the cur-
`rent displacement vector. A second correction data memory
`stores the current correction data. An image registration pro-
`cessor, if the current correction data is absent in the second
`correction data memory, obtains a displacement vector from
`the initial correction data according to a zoom position of the
`lens systems upon forming the plural images, and if the cur-
`rent correction data is stored in the second correction data
`
`memory, obtains a displacement vector from the current cor-
`rection data according to a zoom position of the lens systems
`upon forming the plural images, to carry out image registra-
`tion between the plural images according to the obtained
`displacement vector for producing the three-dimensional
`image.
`The data processor obtains the displacement vector in asso-
`ciation with the first zoom position from the initial correction
`data, to constitute the initial displacement vector, determines
`a first difference amount between the initial displacement
`vector and the current displacement vector, determines a sec-
`ond difference amount in association with one second zoom
`
`position according to the first difference amount and a ratio
`between focal lengths in relation to respectively the first and
`second zoom positions, obtains a second initial displacement
`vector in association with the second zoom position from the
`initial correction data, and determines a sum of the second
`difference amount and the second initial displacement vector,
`to constitute the current correction data in relation to the
`
`second zoom position.
`
`APPL—1014 / Page 12 of 19
`
`APPL-1014 / Page 12 of 19
`
`

`

`US 8,854,432 B2
`
`3
`Furthermore, a zoom control device drives the zoom
`mechanism in response to setting of the calibration mode, to
`set the lens systems in a telephoto end position for a zoom
`position.
`The image registration processor determines an overlap
`region where the plural
`images overlap on one another
`according to the obtained displacement vector, for the image
`registration by retrieving the overlap region from the plural
`images.
`Furthermore, a display panel displays the plural images in
`the superimposed state. The vector detector is externally
`operable to input a signal for shifting one of the plural images
`in one direction. An amount and direction of shift of the
`
`shifted image are determined for the current displacement
`vector upon registering a common object in the plural images
`on the display panel.
`The second correction data memory is accessed to rewrite
`the current correction data determined newly at each time of
`setting the calibration mode.
`Also, a control method of controlling a multi-lens camera
`is provided, the multi-lens camera including plural lens sys-
`tems for receiving entry of object light to form plural images
`with parallax, and a zoom mechanism for changing a magni-
`fication of each of the lens systems within a zooming range,
`so as to produce a three-dimensional image from the plural
`images. In the control method, a displacement vector of an
`amount and a direction of misalignment between the plural
`images according to a superimposed state thereof is obtained
`for each of zoom positions of the lens systems, to store initial
`correction data. If a calibration mode is set for adjusting the
`initial correction data, a current displacement vector in rela-
`tion to one first zoom position is obtained. Current correction
`data is output by adjusting the initial correction data accord-
`ing to the initial correction data and the current displacement
`vector. The current correction data is written to a correction
`
`data memory. If the current correction data is absent in the
`correction data memory, a displacement vector is obtained
`from the initial correction data according to a zoom position
`ofthe lens systems upon forming the plural images, and if the
`current correction data is stored in the correction data
`
`memory, a displacement vector is obtained from the current
`correction data according to a zoom position of the lens
`systems upon forming the plural images, to carry out image
`registration between the plural
`images according to the
`obtained displacement vector for producing the three-dimen-
`sional image.
`The step of outputting the current correction data includes
`obtaining the displacement vector in association with the first
`zoom position from the initial correction data, to constitute
`the initial displacement vector. A first difference amount
`between the initial displacement vector and the current dis-
`placement vector is determined. A second difference amount
`in association with one second zoom position is determined
`according to the first difference amount and a ratio between
`focal lengths in relation to respectively the first and second
`zoom positions. A second initial displacement vector in asso-
`ciation with the second zoom position is obtained from the
`initial correction data. A sum ofthe second difference amount
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`and the second initial displacement vector is determined, to
`constitute the current correction data in relation to the second
`
`60
`
`zoom position.
`Furthermore, the zoom mechanism is driven in response to
`setting of the calibration mode, to set the lens systems in a
`telephoto end position for a zoom position
`The step of the image registration includes determining an
`overlap region where the plural
`images overlap on one
`another according to the obtained displacement vector. The
`
`65
`
`4
`
`image registration is carried out by retrieving the overlap
`region from the plural images.
`The step of initially obtaining the displacement vector
`includes obtaining displacement vectors of the lens systems
`when the lens systems are set in a telephoto end position and
`a wide-angle end position. A displacement vector in relation
`to one intermediate zoom position is obtained according to
`the displacement vectors in relation to the telephoto and wide-
`angle end positions and ratios between focal lengths of the
`optical systems in the telephoto and wide-angle end positions
`and the intermediate zoom position thereof.
`Also, a computer-executable program for controlling a
`multi-lens camera is provided, the multi-lens camera includ-
`ing plural lens systems for receiving entry of object light to
`form plural images with parallax, and a zoom mechanism for
`changing a magnification of each of the lens systems within a
`zooming range, so as to produce a three-dimensional image
`from the plural images. The computer-executable program
`includes an obtaining program code for obtaining a displace-
`ment vector of an amount and a direction of misalignment
`between the plural images according to a superimposed state
`thereof for each ofzoom positions ofthe lens systems, to store
`initial correction data. An obtaining program code is for, if a
`calibration mode is set for adjusting the initial correction data,
`obtaining a current displacement vector between the plural
`images in relation to one first zoom position. An outputting
`program code is for outputting current correction data by
`adjusting the initial correction data according to the initial
`correction data and the current displacement vector. A writing
`program code is for writing the current correction data to a
`correction data memory. An obtaining program code is for, if
`the current correction data is absent in the correction data
`
`memory, obtaining a displacement vector from the initial
`correction data according to a zoom position of the lens
`systems upon forming the plural images, and if the current
`correction data is stored in the correction data memory,
`obtaining a displacement vector from the current correction
`data according to a zoom position of the lens systems upon
`forming the plural images, to carryout image registration
`between the plural images according to the obtained displace-
`ment vector for producing the three-dimensional image.
`Therefore, misalignment between the plural lens systems
`can be eliminated easily and exactly, because a step of image
`registration is determined selectively according to setting of a
`calibration mode.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The above objects and advantages of the present invention
`will become more apparent from the following detailed
`description when read in connection with the accompanying
`drawings, in which:
`FIG. 1 is a perspective view illustrating a dual lens camera;
`FIG. 2 is a rear elevation illustrating the dual lens camera;
`FIG. 3 is a block diagram schematically illustrating circuit
`elements in the dual lens camera;
`FIG. 4 is an explanatory view illustrating a vector;
`FIG. 5 is a table illustrating correction data;
`FIG. 6 is a flow chart illustrating a sequence of producing
`a three-dimensional image;
`FIG. 7A is an explanatory view illustrating a state of
`images with misalignment before image registration;
`FIG. 7B is an explanatory view illustrating the same as
`FIG. 7A but after image registration;
`FIG. 8 is a flow chart illustrating a sequence of obtaining
`initial correction data;
`
`APPL—1014 / Page 13 of 19
`
`APPL-1014 / Page 13 of 19
`
`

`

`US 8,854,432 B2
`
`5
`FIG. 9 is a flow chart illustrating a sequence of obtaining
`current correction data;
`FIG. 10 is an explanatory view illustrating vectors at the
`time of obtaining the current correction data; and
`FIG. 11 is a flow chart illustrating a sequence ofmoving the
`lens systems to a telephoto end position upon setting of a
`calibration mode.
`
`DETAILED DESCRIPTION OF THE PREFERRED
`
`EMBODIMENT(S) OF THE PRESENT
`INVENTION
`
`In FIG. 1, a dual lens camera 2 or multi-lens camera by way
`of stereoscopic camera or three-dimensional camera is illus-
`trated, and includes a camera body 211. A set of right and left
`camera units 3 and 4 are disposed on the camera body 211. The
`right and left camera units 3 and 4 are so arranged that their
`front portions protrude from the front of the camera body 2a
`and their optical axes extend in parallel with one another. The
`dual lens camera 2 forms a pair of images with parallax by
`image pickup through the right and left camera units 3 and 4.
`The images are combined to produce a three-dimensional
`image.
`The right camera unit 3 includes a first lens barrel 6, in
`which a first lens system 5 is incorporated. The left camera
`unit 4 includes a second lens barrel 8, in which a second lens
`system 7 is incorporated. The lens barrels 6 and 8, when the
`power source is turned off or at the time for reproducing an
`image, are moved back to a closed position contained in the
`camera body 2a as indicated by the phantom line, and at the
`time for image pickup, are moved forwards to a forward
`position protruding from the camera body 211. A flash light
`source 10 is disposed in a front face ofthe camera body 2a for
`applying flash light to an object to illuminate.
`Plural elements are disposed on an upper surface of the
`camera body 211, including a shutter button 11, a power switch
`12, and a mode designation wheel 13 as mode designation
`device. The shutter button 11 is depressed for recording an
`image. The power switch 12 is operable for turning on and off
`a power source. The mode designation wheel 13 is operable
`for setting a selected one of plural operation modes including
`a calibration mode. The modes include a still image mode, a
`moving image mode, a playback mode and the calibration
`mode. In the still image mode, the dual lens camera 2 forms a
`still image as a three-dimensional image. In the moving
`image mode, the dual lens camera 2 forms a moving image as
`a three-dimensional image. In the playback mode, the three-
`dimensional image is reproduced and displayed. In the cali-
`bration mode, the dual lens camera 2 obtains correction data
`for image registration of images from the right and left cam-
`era units 3 and 4 by image processing. The mode designation
`wheel 13 is rotationally movable for the mode selection.
`In the still image mode, the shutter button 11 is depressed
`to record a three-dimensional image of an object in a still
`form.
`
`the shutter button 11 is
`In the moving image mode,
`depressed to start recording a moving image, and is depressed
`again to stop the recording, to obtain the three-dimensional
`moving image.
`In FIG. 2, various elements are disposed on a rear surface
`of the camera body 211, including a zoom button 14, a display
`panel 15, a menu button 16, and a cross shaped key 17 as a
`vector detector. The zoom button 14 operates for zooming the
`lens systems 5 and 7 toward the telephoto and wide-angle end
`positions. The display panel 15 displays a three-dimensional
`image, a live image in a standby state, various menu screens
`and the like. The menu button 16 operates for display of the
`
`6
`menu screens. The cross shaped key 17 operates for selecting
`a term or button inside the menu screen. A confirmation
`
`button 18 is disposed at the center of the cross shaped key 17
`for confirming selected information or input information or
`signal.
`The display panel 15 includes a lenticular lens on a front
`side, and is a three-dimensional display of an autostereo-
`scopic type. A user or operator can view a three-dimensional
`image on the display panel 15 in the dual lens camera 2 only
`with his or her eyes without specialized eyewear.
`In FIG. 3, the right camera unit 3 contains the first lens
`barrel 6, and also includes a first lens motor 31, a first focus
`motor 32 or driving motor, a first motor driver 33, a first CCD
`35, a first timing generator 36 or TG, a first CD8 37 or
`correlated double sampling device, a first amplifier 38, and a
`first A/D converter 39.
`
`The first lens system 5 includes a magnification lens 5a for
`zooming, a focus lens 5b and an aperture stop device Sc inside
`the first lens barrel 6. The first lens motor 31 rotates for
`
`moving the first lens barrel 6 forward to the forward position
`and moving the first lens barrel 6 backwards to the closed
`position. The first focus motor 32 rotates to move the magni-
`fication lens 5a and the focus lens 5b on the optical axis back
`and forth. The motors 31 and 32 are connected with the first
`
`motor driver 33. A CPU 70 as data processor and zoom
`control device sends a controls signal to the first motor driver
`33, and drives the motors 31 and 32 to control the entirety of
`the dual lens camera 2.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`The first CCD 35 is disposed behind the first lens system 5,
`which focuses object light from an object on an image pickup
`surface of the first CCD 35. The first CCD 35 is connected
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`with the first timing generator 36. The first timing generator
`36 is connected with the CPU 70, and caused by the CPU 70
`to generate a timing signal or clock pulse to the first CCD 35.
`In response to the timing signal, the first CCD 35 picks up an
`object image of the object focused on the image pickup sur-
`face, and outputs an image signal of the image.
`An image signal output by the first CCD 35 is input to the
`first CD8 37. In response, the first CD8 37 outputs image data
`of B, G and R colors exactly corresponding to a charge
`amount of stored charge in each of the cells in the first CCD
`35. The image data output by the first CD8 37 is amplified by
`the first amplifier 38, and converted by the first A/D converter
`39 into digital image data. The digital image data is output by
`the first A/D converter 39 as right eye image data. An input
`controller 71 is supplied by the image data.
`In a manner similar to the right camera unit 3, the left
`camera unit 4 has the second lens barrel 8, and includes a
`second lens motor 51, a second focus motor 52 or driving
`motor, a second motor driver 53, a second CCD 55, a second
`timing generator 56 or TG, a second CD8 57 or correlated
`double sampling device, a second amplifier 58, and a second
`A/D converter 59. Those operate in the same manner as the
`elements in the right camera unit 3. When an image is picked
`up by the second CCD 55, an image signal is sent to the
`second CD8 57 and the second amplifier 58, and is converted
`into digital image data by the second A/D converter 59. The
`image data is output by the second A/D converter 59 to the
`input controller 71 by way of left eye image data.
`There is a data bus 72 with which the input controller 71 is
`connected to the CPU 70. An SDRAM 73 is accessed by the
`input controller 71. The CPU 70 controls the input controller
`71 to write image data from the right and left camera units 3
`and 4 to the SDRAM 73. An image signal processor 74 reads
`image data from the SDRAM 73, and processes image data in
`various functions of the image processing, such as gradation
`conversion, white balance correction, and gamma correction.
`APPL—1014 / Page 14 of 19
`
`APPL-1014 / Page 14 of 19
`
`

`

`US 8,854,432 B2
`
`7
`The processed image data are written by the image signal
`processor 74 to the SDRAM 73 again.
`An image synthesizer 75 or image registration processor or
`correction device reads image data from the SDRAM 73 after
`processing in the image signal processor 74. The image syn-
`thesizer 75 splits the image ofthe image data into a number of
`strip regions extending vertically, and synthesizes those by
`alternative arrangement, to obtain three-dimensional image
`data of a lenticular lens type for the display panel 15. The
`three-dimensional image data is written to the SDRAM 73.
`An LCD driver 76 reads three-dimensional image data
`from the SDRAM 73, converts the same into an analog com-
`posite signal, and outputs the composite signal to the display
`panel 15. A three-dimensional image is displayed by the
`display panel 15 as a live image visible with eyes without
`specialized eyewear.
`A compressor/expander 77 compresses the three-dimen-
`sional image data in a predetermined format of compression,
`for example, TIFF and JPEG. A removable storage medium
`80 is set in a medium slot in a removable manner. A medium
`
`controller 78 accesses the storage medium 80, and reads the
`data from the storage medium 80 or writes the data to the
`storage medium 80 after the compression.
`An EEPROM 81 is connected to the CPU 70. The
`
`EEPROM 81 stores various programs and data for controlling
`the dual lens camera 2. The CPU 70 runs programs read from
`the EEPROM 81, and controls circuit elements in the dual
`lens camera 2. The data stored in the EEPROM 81 include
`
`focal length information 8111 in a form of a data table in which
`zoom positions of the magnification lens 5a and a magnifi-
`cation lens 7a for zooming are stored, and focal lengths
`corresponding to the zoom positions are stored. The CPU 70
`refers to the focal length information 81a when each of the
`magnification lenses 5a and 7a is positioned in any of the
`zoom positions, and recognizes a focal length determined
`according to the zoom position.
`The various elements are connected to the CPU 70, includ-
`ing the shutter button 11, the power switch 12, the mode
`designation wheel 13, the zoom button 14, the menu button
`16, the cross shaped key 17 and the confirmation button 18. A
`user or operator can manually operate any of those, which
`inputs a control signal ofthe CPU 70. The shutter button 11 is
`a two step switch. When the shutter button 11 is depressed
`halfway, the CPU 70 operates for focus adjustment ofthe lens
`systems 5 and 7, exposure control, and other sequences for
`image pickup. When the shutter button 11 is depressed fully
`with a greater depth, an image signal of one frame ofthe right
`and left camera units 3 and 4 is converted into image signal.
`The power switch 12 is a sliding type of switch. See FIG. 1.
`When the power switch 12 is moved to its ON position, a
`battery (not shown) supplies circuit elements with power, to
`start up the dual lens camera 2. When the power switch 12 is
`moved to its OFF position, supply of the power is discontin-
`ued to turn off the dual lens camera 2. The CPU 70, upon
`detecting a shift of the power switch 12 or the mode designa-
`tion wheel 13, drives the fi