(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2012/0019704 A1
`
`
` Levey (43) Pub. Date: Jan. 26, 2012
`
`US 20120019704A1
`
`(54) AUTOMATIC DIGITAL CAMERA
`PHOTOGRAPHY MODE SELECTION
`
`(76)
`
`Inventor:
`
`.
`Charles 1- Levey, West Henrietta,
`NY (US)
`
`(21) APP1~ N05
`
`12/843341
`
`(22)
`
`Filed:
`
`Jul. 26, 2010
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`H04N 5/225
`
`(2006.01)
`
`(52) US. Cl. ................................. 348/335; 348/E05.024
`(57)
`ABSTRACT
`A digital camera having a plurality of photography modes,
`comprising: an imaging sensor; an optical system for imaging
`a scene onto the imaging sensor; an image capture control for
`initiating an image capture operation; a photography mode
`user interface for selecting between a plurality of photogra-
`phy modes, the photography modes having associated image
`capture and image processing settings; a power control for
`turning the digital camera on or off, wherein when the camera
`is in an off state and the power control is activated with a first
`activation pattern the digital camera is turned on and set to
`operate in a default photography mode and when the power
`control is activated with a second activation pattern the digital
`camera is turned on and set to operate in a previously selected
`photography mode.
`
`175
`100
`r _________ /________1185
`i
`90 5
`CAMERA
`5
`5
`SETTINGS
`.
`:
`i
`5
`i
`110 i
`5
`i
`5
`
`COLOR SENSOR
`DATA
`
`WHITE BALANCE
`
`95
`
`105
`
`NOISE REDUCTION
`
`DEMOSAICING
`
`00g32320N
`
`115
`
`125
`
`135
`
`WHITE
`BALANCE
`SETTING
`
`ISO SETTING
`
`720
`5
`g RESOLUTION
`.
`MODE
`I:
`SETTING
`5
`i
`.
`5
`E
`
`COLOR
`MODE
`SETTING
`
`:
`g
`130 5
`:
`5
`5
`140 '
`
`145
`
`IMAGE
`SHARPEMNG
`
`155
`
`IMAGE
`COMPRESSION
`
`165
`
`175
`, ___________d_________ ‘
`E
`150
`2
`E
`SHARPENING
`E
`:
`SEWING
`
`760
`E COMPRESSION
`:
`MODE
`5
`SETTING
`i
`
`
`
`:
`E
`'
`
`FILE FORM/lTTING
`
`METADATA
`
`17o
`
`TONE SCALE
`CORRECTION
`
`E
`:
`
`CONTRAST
`SETTING
`
`5
`g
`
`DIGITAL IMAGE
`FILE
`
`180
`
`APPL-1015/ Page 1 of 19
`Apple v. Corephotonics
`
`APPL-1015 / Page 1 of 19
`Apple v. Corephotonics
`
`

`

`v� SENSOR
`�6 _ _[ 14
`4
`r---
`-------------------------------------------------------------------·
`
`I+--+
`
`[20
`
`CONVERTER MEMORY
`ASP AND AID __ BUFFER , • ,
`
`
`18
`
`16
`
`�
`
`40 .=J.
`!RECHARGER
`INTERFACE
`WIRED I INTERFACE/
`......__....,:
`38 i 48
`� 10
`
`MPUTER
`
`CO
`
`MEMORY
`FIRMWARE
`
`r2s
`
`;' ?
`. (') � ......... 0
`
`> ....
`.i;...
`0
`--.J
`
`0 .... 1,0
`0 ....N ---0
`c rJJ
`
`N
`
`0 ....
`('D ..... ....
`
`1,0
`
`=­('D
`
`rJJ
`
`
`
`0 .... N
`
`N
`�Cl's
`N
`
`=
`
`O"
`=
`""O
`=
`
`-...
`
`. (') � ......... 0
` "e
`
`-...
`=.....
` t
`
`""O � .....
`
`('D
`
`INTERNET
`
`PROVIDER
`SERVICE
`PHOTO
`[72
`
`o
`r7
`
`PROCESSOR
`
`i-- i- i-
`
`I
`
`DIO l+-----------i
`
`AU
`
` • I
`
`12 :j TIMING
`
`GEN.
`
`I•
`
`I
`
`f
`
`FIG. 1
`
`NETWORK
`WIRELESS
`
`•
`
`i I
`
`[58
`CONTROLS DISPLAY MODEM : -l- ..
`
`DISPLAY
`46 � VIDEO
`44 ----------- 35 ----------
`--------32 ---------- ------_: 52
`� -------------------
`INTERFACE CONTROL
`J l
`VIDEO POWER USER IMAGE WIRELESS
`22
`
`l [ 50
`MEMORY
`DISPLAY
`[36
`MEMORY
`IMAGE
`\30
`
`-11
`
`t
`
`r.===�.1-
`
`EAKER
`MIC H CODEC
`I FLASH I·
`B S1 MOTOR DRIVERS
`ZOOM&FOCUS
`
`26-=:J
`
`SP
`
`24
`
`2
`
`APPL-1015 / Page 2 of 19
`
`

`

`Patent Application Publication
`
`US 2012/0019704 A1
`
`.m625%m
`WGZEKMMmwh‘Ixs" mm:
`m323%.”mozfiwmHEB
`E0400
`
`IOmzmm.
`
`3..{Q
`
`{ENETo
`
`8“m9:m2
`
`Hszmm“mz‘zmmmwimmn
`
`
`
` 9muMu"actommmoomE<35:32E«StemmummmmodoomhHmm...om:mm“mmozEmmmmezEmmmmmm8:_zo‘mwmwmhwnuomwoos.”wz‘oEmoEmQm,mzo‘mmmmméoommzotaqommm"mmo9mBm8hmm:
`
`atm9:m92
`
`
`cm;was):456.6m3%:onmméommack
`
`fitmszmmmzotommmoo
`
`APPL-1015/ Page 3 of 19
`
`APPL-1015 / Page 3 of 19
`
`
`

`

`Patent Application Publication
`
`nJa
`
`2n.
`
`hS
`
`9
`
`U
`
`910m
`
`1A
`
`
`
`
`
`.mwas:>Im<moohorm8mmwA
`
`
`mmi:mmmmmmn
`
`2zotbmmmiommzfimmhmo
`
`6,8m
`
`"toEmEE2&3«20Egon
`
`mm>
`
`mmmmo.
`
`oz
`
`20Emfiqo2:3
`
`
`
`
`
`mNm.3muoSmEEmmm:m2:mmmmnmn%mmm.
`
`4ImmGE
`
`
`2.5:Ema5mmS8m
`mmeE>Im<mwokoi
`
`02
`
`
`
`
`
`ZOtDmENE—OmHOMEMQ
`
`com.
`
`Em
`
`APPL-1015/ Page 4 of 19
`
`APPL-1015 / Page 4 of 19
`
`
`
`

`

`Patent Application Publication
`
`Jan. 26, 2012 Sheet 4 of 9
`
`US 2012/0019704 A1
`
`
`
`“EDEMETOZEDH
`
`«20ENEOm
`
`ohm.
`
`mm}
`
`mam
`
`zotbmEms;Om#0:”..ka
`
`mmmmo.
`
`02
`
`
`
`20<tm=é<0REEF
`
`
`
`ZotbmENE—00.MEEIMHMQ
`
`NEEmmMInEQ
`
`
`
`Hangman.hmm
`
`MODS.>Im«‘EGOKOIQ
`
`0mm.
`
`
`
`mm>MS:mmmmmmq
`
`mNm
`
`02
`
`mbmmNA
`
`09m.
`
`mhm.
`
`V.GE
`
`thSmEQHmm
`
`
`
`MODEExt«$605011
`
`mmm
`
`APPL-1015/ Page 5 of 19
`
`APPL-1015 / Page 5 of 19
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 26, 2012 Sheet 5 of 9
`
`US 2012/0019704 A1
`
`nEOSum:<0ZEDE
`
`«“20mintsOm
`
`ohm.
`
`mm;
`
`mom
`
`mmmml
`
`02
`
`
`
`20<EME<O223k
`
`
`
`mDOSMImHWm.
`
`
`
`NQOE>IQ<EOQFOIQ
`
`mam.
`
`InEMDOQ
`
`mGE
`
`
`
`MUZMDOmmmmmmlmn
`
`MAGZE.
`
`IOmqwgm
`
`
`
`mmmmmlMdeOQ
`
`mmm
`
`
`
`.531.kahmm
`
`
`
`No.03.>IQE¢GOHOIQ
`
`
`
`
`
`ZOtDmENE—OmNZ‘EINHMQ
`
`
`
`
`
`ZOtDmEms—OmHUNHNQ
`
`00m
`
`mhm.
`
`0mm.
`
`0mm
`
`APPL-1015/ Page 6 of 19
`
`APPL-1015 / Page 6 of 19
`
`
`
`
`
`

`

`Patent Application Publication
`
`n.
`
`nJa
`
`hS
`
`9f0
`
`US 2012/0019704 A1
`
`
`
`mmm(mhub—“EMShmmMIEDOQt0Ina-.625%0mm
`
`
`
`
`
`
`
`mMOZMngmmmmmlmn
`
`
`
`
`
`2:ZOtDm$9501m2x§mmhmnM0mm.
`
`02
`
`
`
`
`
`20{EMEfivZIDH.
`
`MQOE>11EGOHOIQ
`
`
`
`«mmmmo.meDOD.
`
`bGE
`
`MHOZB
`
`WDOSNIQHmm
`
`
`
`No.05.>Il<IGOHOIQ
`
`
`
`”to{EME<U2th....«”20EMEOQ
`
`
`
`
`
`mm;
`
`
`
`ohm.new
`
`mmmml
`
`
`
`
`
`2.0thEMS—OD.RDMHMQ
`
`Dom.
`
`mhm.
`
`APPL-1015/ Page 7 of 19
`
`APPL-1015 / Page 7 of 19
`
`
`
`
`
`
`

`

`emP
`
`mon
`
`aonM
`
`n.
`
`“J4
`
`hS
`
`9
`
`U
`
`w0m
`
`1A
`
`
`m“to<mm3<o2&3«20mm:onm:
`
`
`.m0hm.MOW
`
`mmmmmmAzotnmEgon
`Emmam.can
`
`.m02
`
`nm8
`
`20<mm§<u2%:
`
`
`
`mmmnmmmmn.mmmmqmo
`
`
`
`2,zotamEgonmzfimmtmo.%can
`
`Mmoo:
`
`
`
`imqmcohorn.n:5EmuEmEu:moEm:mownnew
`
`M.7kGE
`
`
`
`mmodE>tm<mookormmmnosmmmEm
`
`\SE:
`
`APPL-1015/ Page 8 of 19
`
`APPL-1015 / Page 8 of 19
`
`
`

`

`Patent Application Publication
`
`Jan. 26, 2012 Sheet 8 of 9
`
`US 2012/0019704 A1
`
`
`
`"ECEMEGG213....
`
`
`
`«“20Ems—On.
`
`Em.
`
`mm;
`
`mom.
`
`
`
`ZOtDm.Ems:OmkOmFMQ
`
`mmmmn.
`
`
`
`mbOSMIQ.56
`
`
`
`No.03>IQ§GOHOIl
`
`mmm
`5103
`
`02
`
`
`
`20{mmEEO2E5...
`
`
`
`ZOtbmEMEOQNZ‘EENHMQ
`
`
`
`wtbmmmmo.mmmmmmo.
`
`.iE:
`
`ommbmmmml\S«m.
`
`to12.5.5
`
`mmm.
`
`.55FIEQEm
`
`
`
`MODE>IQ<EGOHOIQ
`
`can
`
`m5
`
`090
`
`0mm
`
`mGE
`
`APPL-1015/ Page 9 of 19
`
`APPL-1015 / Page 9 of 19
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 26, 2012 Sheet 9 of 9
`
`US 2012/0019704 A1
`
`mDOSmmlEm
`
`
`
`No.92>IQ<EGOHOIQ
`
`
`
`m“vMAQDOQ
`
`IONdw‘ém
`
`
`
`mmmmmlM15300.
`
`
`
`QOEHZOOmmm:MZEEMRMQ
`
`
`
`MDZMDOmmmmmmlmn
`
`m...02..m
`
`>IQ<EOOHOIQwith(KNEE
`
`
`
`
`
`mmmuoml20.2.0qu[”503
`
`mGE
`
`
`
`QMFDMqu.hMm.
`
`MQOE>11EGOFOIQ
`
`APPL—1015/ Page 10 of 19
`
`09»
`
`mow
`
`QNV
`
`mmv
`
`
`
`
`
`
`
`mmmmlthHZOUEmmakomhmn
`
`APPL-1015 / Page 10 of 19
`
`
`
`
`
`
`
`
`
`
`

`

`US 2012/0019704 A1
`
`Jan. 26, 2012
`
`AUTOMATIC DIGITAL CAMERA
`PHOTOGRAPHY MODE SELECTION
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] Reference is made to commonly assigned, co-pend-
`ing U.S. patent application Ser. No.
`(96374), entitled:
`“Automatic Digital Camera Photography Mode Selection”,
`by Frank Razavi et al., which is incorporated herein by ref-
`erence.
`
`FIELD OF THE INVENTION
`
`[0002] This invention pertains to the field ofdigital cameras
`and more particularly to the automatic selection of a photog-
`raphy mode.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Digital cameras with a multitude of operational fea-
`tures including but not limited to exposure control, white
`balance, auto focus, etc. have been a consumer staple for
`decades. As camera complexity has increased, required
`actions by the user to operate digital cameras have increased.
`[0004] Most digital cameras have a variety of photography
`modes that can be selected by the user to control various
`elements ofthe image capture process and the image process-
`ing chain. Examples of typical photography modes include
`smart capture, portrait, sport, landscape, close-up, sunset,
`backlight, children, bright, self-portrait, night portrait, night
`landscape, high-ISO and panorama. Various methods have
`been described to select an appropriate photography mode
`and control other digital camera functions. Some digital cam-
`eras include mechanical dials that can be used to select the
`
`photography mode. However, this adds expense to the digital
`camera and limits the number of choices that can be offered.
`
`In other digital camera models, the photography
`[0005]
`mode can be selected by navigating a series of menus on a
`soft-copy display on the back of the digital camera. Often it
`can be a lengthy process for the user to navigate through
`several levels of menus to choose the desired photography
`mode. Then, if the user turns the camera off and back on
`again, it is generally necessary to repeat those time-consum-
`ing steps again in order to return to the same photography
`mode. This can be a frustrating experience for the user.
`[0006] U.S. Pat. No. 6,571,066 to Tsai, entitled “Camera
`with multimode power button,” describes a method for using
`the power button to either turn on the camera or select camera
`operation mode. This scheme requires multiple pushes of the
`power button to first turn on the camera then select the oper-
`ating mode.
`[0007] U.S. Pat. No. 7,721,227 to Ronkainen, entitled
`“Method for describing alternative actions caused by pushing
`a single button,” describes a user interface for making a
`choice between two different actions dependent on how long
`a button is depressed. If the button is released immediately, a
`first action is taken. If the button is held for a longer period of
`time a message is displayed indicating that if the button is
`continued to be held a second action will be taken. If the
`
`button is not released within a specified time interval, the
`second action is then taken.
`
`[0008] U.S. Pat. No. 6,727,830 to Lui, entitled “Time based
`hardware button for application launch,” teaches the use of a
`time dependent press of an “application” button to select
`
`between various functions. Multiple button presses can also
`be used to select different functions.
`
`[0009] U.S. Pat. No. 6,976,215 to Roderick et al., entitled
`“Pushbutton user interface with functionality preview,”
`teaches a user-interface that uses a pressure-sensitive multi-
`state button. If the button is pushed with a first pressure a
`preview is provided of the effect that will be produced if the
`button is pressed using a higher pressure.
`[0010] There remains a need for a simple user interface that
`enables a user to conveniently return to a previously-selected
`photography mode without adding the cost of additional but-
`tons or the inconvenience of needing to navigate complex
`user-interface menus.
`
`SUMMARY OF THE INVENTION
`
`[0011] The present invention represents a digital camera
`having a plurality of photography modes, comprising:
`[0012]
`an imaging sensor;
`[0013]
`an optical system for imaging a scene onto the imag-
`ing sensor;
`[0014]
`an image capture control for initiating an image
`capture operation;
`[0015]
`a photography mode user interface for selecting
`between a plurality of photography modes, the photography
`modes having associated image capture and image process-
`ing settings;
`[0016]
`a power control for turning the digital camera on or
`off, wherein when the camera is in an off state and the power
`control is activated with a first activation pattern the digital
`camera is turned on and set to operate in a default photogra-
`phy mode and when the power control is activated with a
`second activation pattern the digital camera is turned on and
`set to operate in a previously selected photography mode.
`[0017] The present invention has the advantage that it sim-
`plifies camera operation by allowing a user to conveniently
`return to a previously selected photography mode when the
`digital camera is powered on. In this way a single button
`activation can be used to perform two different tasks: turning
`on the camera and selecting between the previously selected
`photography mode and a default photography mode.
`[0018]
`It has the additional advantage that the user can
`select between different photography modes with a single
`button activation without needing to interact with a multi-step
`menu selection process. This reduces the required number of
`user inputs required to select the photography mode.
`[0019]
`It has the further advantage that no additional user
`interface controls are required to provide the added function-
`ality.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a high-level diagram showing the compo-
`[0020]
`nents of a system for classifying digital image according to an
`embodiment of the present invention;
`[0021]
`FIG. 2 is a flow diagram depicting typical image
`processing operations used to process digital images in a
`digital camera;
`[0022]
`FIGS. 3 and 4 are flowcharts illustrating a method
`for selecting a photography mode in a digital camera using a
`short or long button press according to an embodiment of the
`present invention;
`[0023]
`FIGS. 5 and 6 are flowcharts illustrating a method
`for selecting a photography mode in a digital camera using a
`APPL—1015/ Page 11 of 19
`
`APPL-1015 / Page 11 of 19
`
`

`

`US 2012/0019704 A1
`
`Jan. 26, 2012
`
`single or double button press according to an alternate
`embodiment of the present invention;
`[0024]
`FIGS. 7 and 8 are flowcharts illustrating a method
`for selecting a photography mode in a digital camera using a
`light or heavy button pressure according to an alternate
`embodiment of the present invention; and
`[0025]
`FIG. 9 is a flowchart illustrating a method for select-
`ing a photography mode in a digital camera using different
`activation patterns for a photography mode user interface
`control according to an alternate embodiment of the present
`invention.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`In the following description, a preferred embodi-
`[0026]
`ment of the present invention will be described in terms that
`would ordinarily be implemented as a software program.
`Those skilled in the art will readily recognize that the equiva-
`lent of such software can also be constructed in hardware.
`
`Because image manipulation algorithms and systems are well
`known, the present description will be directed in particular to
`algorithms and systems forming part of, or cooperating more
`directly with, the system and method in accordance with the
`present invention. Other aspects of such algorithms and sys-
`tems, and hardware or software for producing and otherwise
`processing the image signals involved therewith, not specifi-
`cally shown or described herein, can be selected from such
`systems, algorithms, components and elements known in the
`art. Given the system as described according to the invention
`in the following materials, software not specifically shown,
`suggested or described herein that is useful for implementa-
`tion of the invention is conventional and within the ordinary
`skill in such arts.
`
`Still further, as used herein, a computer program for
`[0027]
`performing the method of the present invention can be stored
`in a computer readable storage medium, which can include,
`for example; magnetic storage media such as a magnetic disk
`(such as a hard drive or a floppy disk) or magnetic tape;
`optical storage media such as an optical disc, optical tape, or
`machine readable bar code; solid state electronic storage
`devices such as random access memory (RAM), or read only
`memory (ROM); or any other physical device or medium
`employed to store a computer program having instructions for
`controlling one or more computers to practice the method
`according to the present invention.
`[0028] The invention is inclusive of combinations of the
`embodiments described herein. References to “a particular
`embodiment” and the like refer to features that are present in
`at least one embodiment ofthe invention. Separate references
`to “an embodiment” or “particular embodiments” or the like
`do not necessarily refer to the same embodiment or embodi-
`ments; however, such embodiments are not mutually exclu-
`sive, unless so indicated or as are readily apparent to one of
`skill in the art. The use of singular or plural in referring to the
`“method” or “methods” and the like is not limiting. It should
`be noted that, unless otherwise explicitly noted or required by
`context, the word “or” is used in this disclosure in a non-
`exclusive sense.
`
`employing imaging
`cameras
`[0029] Because digital
`devices and related circuitry for signal capture and process-
`ing, and display are well known, the present description will
`be directed in particular to elements forming part of, or coop-
`erating more directly with, the method and apparatus in
`accordance with the present invention. Elements not specifi-
`cally shown or described herein are selected from those
`
`known in the art. Certain aspects of the embodiments to be
`described are provided in software. Given the system as
`shown and described according to the invention in the follow-
`ing materials, software not specifically shown, described or
`suggested herein that is useful for implementation of the
`invention is conventional and within the ordinary skill in such
`arts
`
`[0030] The following description ofa digital camera will be
`familiar to one skilled in the art. It will be obvious that there
`
`are many variations of this embodiment that are possible and
`are selected to reduce the cost, add features or improve the
`performance of the camera.
`[0031]
`FIG. 1 depicts a block diagram of a digital photog-
`raphy system, including a digital camera 10 in accordance
`with the present invention. Preferably, the digital camera 10 is
`a portable battery operated device, small enough to be easily
`handheld by a user when capturing and reviewing images.
`The digital camera 10 produces digital images that are stored
`as digital image files using image memory 30. The phrase
`“digital image” or “digital image file”, as used herein, refers
`to any digital image file, such as a digital still image or a
`digital video file.
`[0032]
`In some embodiments, the digital camera 10 cap-
`tures both motion video images and still images. The digital
`camera 10 can also include other functions, including, but not
`limited to, the functions ofa digital music player (e. g. an MP3
`player), a mobile telephone, a GPS receiver, or a program-
`mable digital assistant (PDA).
`[0033] The digital camera 10 includes a lens 4 having an
`adjustable aperture and adjustable shutter 6. In a preferred
`embodiment, the lens 4 is a zoom lens and is controlled by
`zoom and focus motor drives 8. The lens 4 focuses light from
`a scene (not shown) onto an image sensor 14, for example, a
`single-chip color CCD or CMOS image sensor. The lens 4 is
`one type optical system for forming an image of the scene on
`the image sensor 14. In other embodiments, the optical sys-
`tem may use a fixed focal length lens with either variable or
`fixed focus.
`
`[0034] The output of the image sensor 14 is converted to
`digital form by Analog Signal Processor (ASP) and Analog-
`to-Digital (A/D) converter 16, and temporarily stored in
`buffer memory 18. The image data stored in buffer memory
`18 is subsequently manipulated by a processor 20, using
`embedded software programs (e.g. firmware) stored in firm-
`ware memory 28. In some embodiments, the software pro-
`gram is permanently stored in firmware memory 28 using a
`read only memory (ROM). In other embodiments, the firm-
`ware memory 28 can be modified by using, for example,
`Flash EPROM memory. In such embodiments, an external
`device can update the software programs stored in firmware
`memory 28 using the wired interface 38 or the wireless
`modem 50. In such embodiments, the firmware memory 28
`can also be used to store image sensor calibration data, user
`setting selections and other data which must be preserved
`when the camera is turned off. In some embodiments, the
`processor 20 includes a program memory (not shown), and
`the software programs stored in the firmware memory 28 are
`copied into the program memory before being executed by
`the processor 20.
`[0035]
`It will be understood that the functions of processor
`20 can be provided using a single programmable processor or
`by using multiple programmable processors, including one or
`more digital signal processor (DSP) devices. Alternatively,
`the processor 20 can be provided by custom circuitry (e. g., by
`APPL—1015/ Page 12 of 19
`
`APPL-1015 / Page 12 of 19
`
`

`

`US 2012/0019704 A1
`
`Jan. 26, 2012
`
`one or more custom integrated circuits (ICs) designed spe-
`cifically for use in digital cameras), or by a combination of
`programmable processor(s) and custom circuits. It will be
`understood that connectors between the processor 20 from
`some or all ofthe various components shown in FIG. 1 can be
`made using a common data bus. For example,
`in some
`embodiments the connection between the processor 20, the
`buffer memory 18, the image memory 30, and the firmware
`memory 28 can be made using a common data bus.
`[0036] The processed images are then stored using the
`image memory 30. It is understood that the image memory 30
`can be any form of memory known to those skilled in the art
`including, but not limited to, a removable Flash memory card,
`internal Flash memory chips, magnetic memory, or optical
`memory. In some embodiments, the image memory 30 can
`include both internal Flash memory chips and a standard
`interface to a removable Flash memory card, such as a Secure
`Digital (SD) card. Alternatively, a different memory card
`format can be used, such as a micro SD card, Compact Flash
`(CF) card, MultiMedia Card (MMC), xD card or Memory
`Stick.
`
`[0037] The image sensor 14 is controlled by a timing gen-
`erator 12, which produces various clocking signals to select
`rows and pixels and synchronizes the operation of the ASP
`and A/D converter 16. The image sensor 14 can have, for
`example, 12.4 megapixels (4088x3040 pixels) in order to
`provide a still image file of approximately 4000x3000 pixels.
`To provide a color image, the image sensor is generally over-
`laid with a color filter array, which provides an image sensor
`having an array ofpixels that include different colored pixels.
`The different color pixels can be arranged in many different
`patterns. As one example, the different color pixels can be
`arranged using the well-known Bayer color filter array, as
`described in commonly assigned US. Pat. No. 3,971,065,
`“Color imaging array” to Bayer, the disclosure of which is
`incorporated herein by reference. As a second example, the
`different color pixels can be arranged as described in com-
`monly assigned US. Patent Application Publication 2007/
`0024931 to Compton and Hamilton, entitled “Image sensor
`with improved light sensitivity”, the disclosure of which is
`incorporated herein by reference. These examples are not
`limiting, and many other color patterns may be used.
`[0038]
`It will be understood that the image sensor 14, tim-
`ing generator 12, and ASP and A/D converter 16 can be
`separately fabricated integrated circuits, or they can be fab-
`ricated as a single integrated circuit as is commonly done with
`CMOS image sensors. In some embodiments, this single
`integrated circuit can perform some of the other functions
`shown in FIG. 1, including some ofthe functions provided by
`processor 20.
`[0039] The image sensor 14 is effective when actuated in a
`first mode by timing generator 12 for providing a motion
`sequence oflower resolution sensor image data, which is used
`when capturing video images and also when previewing a still
`image to be captured, in order to compose the image. This
`preview mode sensor image data can be provided as HD
`resolution image data, for example, with 1280x720 pixels, or
`as VGA resolution image data, for example, with 640x480
`pixels, or using other resolutions which have significantly
`fewer columns and rows of data, compared to the resolution
`of the image sensor.
`[0040] The preview mode sensor image data can be pro-
`vided by combining values of adj acent pixels having the same
`color, or by eliminating some of the pixels values, or by
`
`combining some color pixels values while eliminating other
`color pixel values. The preview mode image data can be
`processed as described in commonly assigned US. Pat. No.
`6,292,218 to Parulski, et al., entitled “Electronic camera for
`initiating capture of still images while previewing motion
`images,” which is incorporated herein by reference.
`[0041] The image sensor 14 is also effective when actuated
`in a second mode by timing generator 12 for providing high
`resolution still image data. This final mode sensor image data
`is provided as high resolution output image data, which for
`scenes having a high illumination level includes all of the
`pixels of the image sensor, and can be, for example, a 12
`megapixel final image data having 4000x3000 pixels. At
`lower illumination levels, the final sensor image data can be
`provided by “binning” some number of like-colored pixels on
`the image sensor, in order to increase the signal level and thus
`the “ISO speed” of the sensor.
`[0042] The zoom and focus motor drivers 8 are controlled
`by control signals supplied by the processor 20, to provide the
`appropriate focal length setting and to focus the scene onto
`the image sensor 14. The exposure level of the image sensor
`14 is controlled by controlling the f/number and exposure
`time of the adjustable aperture and adjustable shutter 6, the
`exposure period of the image sensor 14 via the timing gen-
`erator 12, and the gain (i.e., ISO speed) setting ofthe ASP and
`A/D converter 16. The processor 20 also controls a flash 2
`which can illuminate the scene.
`
`[0043] The lens 4 ofthe digital camera 10 can be focused in
`the first mode by using “through-the-lens” autofocus, as
`described in commonly-assigned US. Pat. No. 5,668,597,
`entitled “Electronic Camera with Rapid Automatic Focus of
`an Image upon a Progressive Scan Image Sensor” to Parulski
`et al., which is incorporated herein by reference. This is
`accomplished by using the zoom and focus motor drivers 8 to
`adjust the focus position ofthe lens 4 to a number ofpositions
`ranging between a near focus position to an infinity focus
`position, while the processor 20 determines the closest focus
`position which provides a peak sharpness value for a central
`portion of the image captured by the image sensor 14. The
`focus distance which corresponds to the closest focus posi-
`tion can then be utilized for several purposes, such as auto-
`matically setting an appropriate scene mode, and can be
`stored as metadata in the image file, along with other lens and
`camera settings.
`[0044] The processor 20 produces menus and low resolu-
`tion color images that are temporarily stored in display
`memory 36 and are displayed on the image display 32. The
`image display 32 is typically an active matrix color liquid
`crystal display (LCD), although other types of displays, such
`as organic light emitting diode (OLED) displays, can be used.
`A video interface 44 provides a video output signal from the
`digital camera 10 to a video display 46, such as a flat panel
`HDTV display. In preview mode, or video mode, the digital
`image data from buffer memory 18 is manipulated by proces-
`sor 20 to form a series of motion preview images that are
`displayed, typically as color images, on the image display 32.
`In review mode, the images displayed on the image display 32
`are produced using the image data from the digital image files
`stored in image memory 30.
`[0045] The graphical user interface displayed on the image
`display 32 is controlled in response to user input provided by
`user controls 34. The user controls 34 are used to select
`
`various camera modes, such as video capture mode, still
`capture mode, and review mode, and to initiate capture of still
`APPL—1015/ Page 13 of 19
`
`APPL-1015 / Page 13 of 19
`
`

`

`US 2012/0019704 A1
`
`Jan. 26, 2012
`
`images and recording of motion images. User controls 34
`typically include some combination of buttons,
`rocker
`switches, joysticks, or rotary dials. In some embodiments,
`some of the user controls 34 are provided by using a touch
`screen overlay on the image display 32. In other embodi-
`ments, additional status displays or images displays can be
`used. In a preferred embodiment, the user controls 34 include
`a power control 35 (e.g., a power button) which is used to turn
`on or off the camera, as well as other controls such as a lens
`controls to control the lens 4, and an image capture control
`(e. g., a shutter button) to initiate an image capture operation.
`In some embodiments, the still preview mode is initiated
`when the user partially depresses the shutter button, and the
`still image capture mode is initiated when the user fully
`depresses the shutter button.
`[0046] The camera modes that can be selected using the
`user controls 34 include a “timer” mode. When the “timer”
`
`mode is selected, a short delay (e.g., 10 seconds) occurs after
`the user fully presses the shutter button, before the processor
`20 initiates the capture of a still image.
`[0047] An audio codec 22 connected to the processor 20
`receives an audio signal from a microphone 24 and provides
`an audio signal to a speaker 26. These components can be to
`record and playback an audio track, along with a video
`sequence or still image. If the digital camera 10 is a multi-
`function device such as a combination camera and mobile
`
`phone, the microphone 24 and the speaker 26 can be used for
`telephone conversation.
`[0048]
`In some embodiments, the speaker 26 can be used as
`part of the user interface, for example to provide various
`audible signals which indicate that a user control has been
`depressed, or that a particular mode has been selected. In
`some embodiments, the microphone 24, the audio codec 22,
`and the processor 20 can be used to provide voice recognition,
`so that the user can provide a user input to the processor 20 by
`using voice commands, rather than user controls 34. The
`speaker 26 can also be used to inform the user of an incoming
`phone call. This can be done using a standard ring tone stored
`in firmware memory 28, or by using a custom ring-tone
`downloaded from a wireless network 58 and stored in the
`
`image memory 3 0. In addition, a vibration device (not shown)
`can be used to provide a silent (e. g., non audible) notification
`of an incoming phone call.
`[0049] The processor 20 also provides additional process-
`ing of the image data from the image sensor 14, in order to
`produce rendered sRGB image data which is compressed and
`stored within a “finished” image file, such as a well-known
`Exif—JPEG image file, in the image memory 30.
`[0050] The digital camera 10 can be connected via the
`wired interface 38 to an interface/recharger 48, which is con-
`nected to a computer 40, which can be a desktop computer or
`portable computer located in a home or office. The wired
`interface 38 can conform to, for example, the well-known
`USB 2.0 interface specification. The interface/recharger 48
`can provide power via the wired interface 38 to a set of
`rechargeable batteries (not shown) in the digital camera 10.
`[0051] The digital camera 10 can include a wireless modem
`50, which interfaces over a radio frequency band 52 with the
`wireless network 58. The wireless modem 50 can use various
`
`wireless interface protocols, such as the well-known Blue-
`tooth wireless interface or the well-known 802.11 wireless
`
`interface. The computer 40 can upload images via the Internet
`70 to a photo service provider 72, such as the Kodak Easy-
`
`Share Gallery. Other devices (not shown) can access the
`images stored by the photo service provider 72.
`[0052]
`In alternative embodiments, the wireless modem 50
`communicates over a radio frequency (e.g. wireless) link with
`a mobile phone network (not shown), such as a 3GSM net-
`work, which connects with the Internet 70 in order to upload
`digital image files from the digital camera 10. These digital
`image files can be provided to the computer 40 or the photo
`service provider 72.
`[0053]
`FIG. 2 is a flow diagram depicting image processing
`operations that can be performed by the processor 20 in the
`digital camera 10 (FIG. 1) in order to process color sensor
`data 100 from the image sensor 14 output by the ASP andA/D
`converter 16. In some embodiments, the processing param-
`eters used by the processor 20 to manipulate the color sensor
`data 100 for a particular digital image are determined by
`various user settings 175, which can be selected via the user
`controls 34 in response to menus displayed on the image
`display 32.
`[0054] The color sensor data 100 which has been digitally
`converted by the ASP andA/D converter 16 is manipulated by
`a white balance step 95. In some embodiments, this process-
`ing can be performed using the methods described in com-
`monly-assigned US. Pat. No. 7,542,077 to Miki, entitled
`“White balance adjustment device and color identification
`device”, the disclosure of which is herein incorporated by
`reference. The white balance can be adjusted in response to a
`white balance setting 90, which can be manually set by a user,
`or which can be automatically set by the camera.
`[0055] The color image data is then manipulated by a noise
`reduction step 105 in order to reduce noise from the image
`sensor 14. In some embodiments, this processing can be
`performed using the methods described in commonly-as-
`signed U.S. Pat. No. 6,934,056 to Gindele et al