I
`Umted States Patent
`
`19
`
`US006l l8480A
`
`11
`
`Patent Number:
`
`6 118 480
`9
`9
`
`Anderson et al.
`
`[45] Date of Patent:
`
`*Sep. 12, 2000
`
`395/173
`395194
`.... .. 380/40
`348/231
`395/821
`386/107
`N 348],“
`343/233
`348/207
`
`
`
`5./1997 (jasper el al.
`5,030,017
`4.tl998 Meyer ct al.
`5.742,504
`5./1998 Kubo
`5,748,831
`8./1998 Matsumoto ct al
`5.796,428
`5,819,103 ll)./1998 Endoh et al.
`5.822,492 ll’)./1998 Wakui ct al.
`$845,100 12‘/1998 Feuegara at al.
`5,861,918
`1.11999 Anderson et al.
`5,940,121
`8./1999 Mcintyre et al.
`
`[54] Ml*1'l‘H0l) AND APPARATUS FOR
`INTEGRATING A [)[(}[TAL CAMERA USER
`INTERFACE ACROSS MULTIPLE
`
`-
`[75]
`
`OPERATING MODES
`_
`.
`.
`Inventors. Eric C. Anderson, San Jose, Steve
`5“Y'°"~ Morgan H1119 A“‘““‘‘‘! R-
`Manda? P310 A110: 311 Of Callfi
`
`[,1 Notice:
`
`[73] Assignee: FIashP0int Teclmology, Inc., San Jose,
`C“1‘f'
`This palm .55“-Cd on a continued pms_
`ecution application filed under 37 CFR
`153(d), and is SUbjt‘«Cl 10 ‘ht? ‘Wally year
`P310111
`tt-‘«l‘t'l‘l PF0Vt$i0flS Of 35 U»5‘C~
`154
`2 .
`(ax )
`
`FOREIGN PATENT DOCUMENTS
`3223524
`811006
`Japan ..........................._. 11n4N 5,225
`9220186
`1111992 WIPO ............................ H04N 5/202
`P,-[nmry Em,,,;,w,._wendy Garber
`Assistant Exam1'ner—Aung Moe
`Attorney, Agent, or Firm—Sawycr Law Group LLP
`[57]
`ABSTRACT
`
`A method and apparatus for integrating a user interface
`across multtple operating modes of a dtgital camera 1nclud-
`mg a dtsplay. When the digital camera is placed mto each
`one of the multiple operating modes, mode-specific items
`corresponding to that mode are displayed on the display. The
`digital camera includes It first button and a second button for
`interacting with the multiple operating modes, where the
`first button has a first orientation, and the second button has
`a second orientation. The method and apparatus includes
`mapping an aligned set of mode-specific items in the display
`to the orientation of the first button. After the mode-specific
`items are displayed, the user scrolls from one mode-specific
`item to the next in the aligned Set by Pt€S9itlg the first httttttn
`and the display indicates which of the mode-specific items
`is a currently active item. When a mode-specific item
`becomesthe active item, additional information correspond-
`ing to the currently active item is displayed in the display in
`a location offset from the active item in a direction of
`-
`-
`-
`‘
`orientation corresponding to that of the second button.
`
`31 Clalms, 11 Drawing Sheets
`
`lzll APPL N0-5 08/8512667
`[22]
`Filed,
`May 6’ 1997
`
`Int. Cl.7 ................................................... .. H04N 5/222
`[51]
`[52] US. Cl.
`............................................................ .. 348/207
`[58] Field of Search ................................... .. 348/333, 334,
`3431373, 375, 376, 552, 233, 233, 211,
`231, 33; 207, 239; 358/9116, 9()9_1
`
`[56]
`
`References Cited
`
`U5" PATENT DOCUMENTS
`o/1990 Hayashi ct nl.
`....................... .. 358/S27
`1/1991 Kurahashi et al.
`386171,)
`8/1992 Egawa .............. ..
`358/009.1
`9/1992 1S0guchi ct al.
`.. 358/909.1
`8/1994 Tflnflka C‘ “L -
`""""""""""""
`1:: :{““gl‘”°‘
`.
`.... ..
`.
`.
`oronagzt c
`''_'._'.
`_'. _H 395/148
`4/19% Mills el al.
`S/1996 Mathcny et al.
`.... .. 395/156
`9/1990 Hirano et al.
`. . . . . .
`. . . . .. 395/139
`
`
`
`4,935,809
`4,982,291
`5,138,460
`5,146,353
`5»335>072
`-,
`,
`5.513.306
`5,517,606
`5.553.277
`
`1_tQ
`
` Image #314
`
`10:26AM
`
`12/4/96
`
`4103
`
`0001
`0001
`
`Apple/Twitter
`Apple/Twitter
`Ex. 1020
`Ex. 1020
`IPR1 of U.S. Pat. No. 7,765,482
`IPR1 of U.S. Pat. No. 7,765,482
`
`

`
`U.S. Patent
`
`Sep. 12, 2000
`
`Sheet 1 of 11
`
`6,118,480
`
`FIG.1
`
`:_
`Q3-0-
`
`3Q EO
`
`(3
`
`112
`
`0002
`
`0002
`
`

`
`
`
`111319cI'S'[1
`
`U:
`G
`‘F
`I-I
`
`‘('9
`
`Nccc
`
`IIJ0Z199llS
`
`08I7‘8II‘9
`
`A/D
`
`Convertet
`
`Analog
`.
`
`P 3'9"“
`
`rocessor
`
`Timing
`Generator
`
`0003
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IIJ09193118000z‘zI°das111e1e(['S'f1
`
`
`
`
`
`
`
`
`
`
`
`08I7‘8II‘9
`
`
`
`
`
`
`
`
`
`
`
`
`408
`
`
`
`406
`
`
`
`LCD Screen
`
`
`
`
`
`Bun.°nS
`
`
`& Dials
`
`
`
`Status LCD
`
`
`
`
`
`Removable
`
`
`Memory
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Imaging Device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LCD
`
`
`Controller
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Non-volatile
`
`
`Memory
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`0004
`
`0004
`
`

`
`6,118,480
`
`WORKING MEMORY
`
`FRAME BUFFERS
`
`INPUT BUFFEFIS
`
`SYSTEM AREA
`
`0005
`
`

`
`U.S. Patent
`
`Sep. 12,2000
`
`Sheet 5 of 11
`
`6,118,480
`
`Input Buffer A
`
`Frame Buffer A
`
`Processing
`
`Input Buffer B
`
`Frame Buffer B
`
`Processing
`
`0006
`
`

`
`
`
`6,118,4806,118,480
`
`0007
`
`

`
`U.S. Patent
`
`Sep. 12,2000
`
`Sheet 7 of 11
`
`6,118,480
`
`600
`
`Provide the digital camera with
`at least two navigation buttons
`
`Map an aligned a set of mode-specific
`items in the display to the orientation
`of the first navigation button
`
`Press the mapped navigation
`button to scroll from one mode-
`
`specific item to the next
`
`Indicate which of the mode-specific
`items is the currently active item
`
`Display additional information corresponding
`to the currently active item in a location
`in the display that is offset lrorn the
`active item in a direction mapped
`to the orientation of the second button
`
`0008
`
`

`
`
`
`6,118,4806,118,480
`
`0009
`
`

`
`
`
`
`
`IIJ06199|lS000Z‘ZI°d9S1II31Bd‘S‘['lIIJ06199|lS000Z‘ZI°d9S1II31Bd‘S‘['l
`
`
`
`
`
`
`
`
`
`12/4/9612/4/96
`
`
`
`10:26AM10:26AM
`
`
`
`FIG. 9FIG. 9
`
`
`
`08i7‘8II‘908i7‘8II‘9
`
`0010
`
`

`
`U.S. Patent
`
`Sep. 12,2000
`
`Sheet 10 of 11
`
`6,118,480
`
`0011
`
`

`
`U.S. Patent
`
`Sheet 11 of 11
`
`6,118,480
`
`FIG. 11B
`
`0012
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`6,118,480
`
`
`
`
`
`
`
`
`1
`METHOD AND APPARATUS FOR
`
`
`
`
`INTEGRATING A DIGITAL CAMERA USER
`
`
`
`
`INTERFACE ACROSS MULTIPLE
`
`
`
`OPERATING MODES
`
`
`FIELD OF THE INVENTION
`
`
`
`
`The present invention relates generally to digital cameras,
`
`
`
`
`
`
`
`and more particularly to a method and apparatus for inte-
`
`
`
`
`
`
`
`
`
`grating a digital camera user interface across multiple oper-
`
`
`
`
`
`
`
`
`ating modes.
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`
`Most digital cameras today are similar in size to and
`
`
`
`
`
`
`
`
`
`
`behave like conventional point-and-shoot cameras. Unlike
`
`
`
`
`
`
`conventional cameras, however, most digital cameras store
`
`
`
`
`
`
`
`digital images in an internal flash memory or on external
`
`
`
`
`
`
`
`
`
`
`memory cards, and some are equipped with a liquid-crystal
`
`
`
`
`
`
`
`
`
`display (LCD) screen on the back of the camera. Through
`
`
`
`
`
`
`
`
`
`
`the use of the LCD, most digital cameras operate in two
`
`
`
`
`
`
`
`
`
`
`
`modes, record and play, although some only have a record
`
`
`
`
`
`
`
`
`
`
`mode. In record mode, the LCD is used as a viewfinder in
`
`
`
`
`
`
`
`
`
`
`
`
`which the user may view an object or scene before taking a
`
`
`
`
`
`
`
`
`
`
`
`
`picture. In play mode, the LCD is used as a playback screen
`
`
`
`
`
`
`
`
`
`
`
`
`for allowing the user to review previously captured images
`
`
`
`
`
`
`
`
`
`either individually or in arrays of four, nine, or sixteen
`
`
`
`
`
`
`
`
`
`
`images.
`
`Digital camera user interfaces typically include a number
`
`
`
`
`
`
`
`
`of buttons or switches for setting the camera into one of the
`
`
`
`
`
`
`
`
`
`
`
`
`two modes and for navigating between images in play mode.
`
`
`
`
`
`
`
`
`
`
`One type of camera, for instance, includes two navigation
`
`
`
`
`
`
`
`
`
`buttons labeled “—” and “+”, a mode button, a display
`
`
`
`
`
`
`
`
`
`
`button, a zoom button and a delete button. Play mode for this
`
`
`
`
`
`
`
`
`
`
`camera begins with a default screen displaying a full-sized
`
`
`
`
`
`
`
`
`
`individual image. Other images stored in the camera may
`
`
`
`
`
`
`
`
`
`then be displayed in a backward or forward sequence by
`
`
`
`
`
`
`
`
`
`
`pressing the “—” and “+” navigation buttons, respectively.
`
`
`
`
`
`
`
`
`Pressing the mode button during play mode causes four
`
`
`
`
`
`
`
`
`
`images to be displayed in a 2x2 array, and pressing the mode
`
`
`
`
`
`
`
`
`
`
`
`button again causes nine images to be displayed in a 3x3
`
`
`
`
`
`
`
`
`
`
`
`array. The user can then “page” through screens of image
`
`
`
`
`
`
`
`
`
`
`arrays by pressing the navigation buttons, or the user can
`
`
`
`
`
`
`
`
`
`
`move from image to image in the arrays by first pressing the
`
`
`
`
`
`
`
`
`
`
`
`
`display button and then traversing across the images in the
`
`
`
`
`
`
`
`
`
`
`rows of the arrays using the navigation buttons. The user
`
`
`
`
`
`
`
`
`
`
`may have the full-sized image displayed of a chosen image
`
`
`
`
`
`
`
`
`
`by pressing the zoom button or can delete the image by
`
`
`
`
`
`
`
`
`
`
`
`pressing the delete button.
`
`
`
`
`Although digital cameras that have both a record mode
`
`
`
`
`
`
`
`
`
`and a play mode are more versatile than digital cameras
`
`
`
`
`
`
`
`
`
`
`having only the record mode,
`two mode digital cameras
`
`
`
`
`
`
`
`
`
`suffer from several disadvantages associated with the cam-
`
`
`
`
`
`
`
`era’s user interface. One disadvantage is that having only
`
`
`
`
`
`
`
`
`
`two modes means that either the camera only has a limited
`
`
`
`
`
`
`
`
`
`
`
`number of functions, or that several functions must be
`
`
`
`
`
`
`
`
`
`accessed in play mode since the record mode only has one
`
`
`
`
`
`
`
`
`
`
`
`function, capturing images. The disadvantages of having
`
`
`
`
`
`
`
`several functions in one mode is that the functions may have
`
`
`
`
`
`
`
`
`
`
`
`to be accessed through multiple levels of navigation screens,
`
`
`
`
`
`
`
`
`
`which complicates the operability of the camera.
`
`
`
`
`
`
`
`Another disadvantage of conventional cameras is that the
`
`
`
`
`
`
`
`
`operation of user interface is non-intuitive, especially for the
`
`
`
`
`
`
`
`
`
`novice user. The user interface is non-intuitive because the
`
`
`
`
`
`
`
`
`
`operation of the user interface across different modes and/or
`
`
`
`
`
`
`
`
`
`navigation screens is inconsistent. Accessing most features
`
`
`
`
`
`
`
`in the two mode camera described above, for instance,
`
`
`
`
`
`
`
`
`
`requires that the user press the keys of the interface in a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`
`
`15
`
`
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`0013
`
`
`2
`certain sequence. Each of these key sequences may be
`
`
`
`
`
`
`
`
`
`different depending on which play-mode navigation screen
`
`
`
`
`
`
`
`is displayed,
`the navigation screen showing individual
`
`
`
`
`
`
`
`images or the navigation screen showing arrays of images.
`
`
`
`
`
`
`
`
`
`For example,
`the function of the display button changes
`
`
`
`
`
`
`
`
`
`when the navigation screens change, and in some situations
`
`
`
`
`
`
`
`
`
`where the display button has been depressed,
`the mode
`
`
`
`
`
`
`
`
`
`button either becomes inoperable or the functionality of
`
`
`
`
`
`
`
`
`mode button becomes mutually exclusive with the function-
`
`
`
`
`
`
`
`ality of the zoom button. Furthermore, because each navi-
`
`
`
`
`
`
`
`
`gation screen has a different key sequence, it is not obvious
`
`
`
`
`
`
`
`
`
`
`to the user how to exit that screen or how to choose a
`
`
`
`
`
`
`
`
`
`
`
`
`particular function. Thus, this type of user interface requires
`
`
`
`
`
`
`
`
`that the user memorize a different key sequence for each
`
`
`
`
`
`
`
`
`
`navigation screen before being able to effectively operate the
`
`
`
`
`
`
`
`
`camera.
`
`A further disadvantage of conventional digital-camera
`
`
`
`
`
`user-interfaces is that the camera is capable of displaying
`
`
`
`
`
`
`
`
`only the images themselves, or a combination of an image
`
`
`
`
`
`
`
`
`
`and its image number. The user interface is either incapable
`
`
`
`
`
`
`
`
`
`of delivering further
`information regarding displayed
`
`
`
`
`
`images and the camera features, or accessing such informa-
`
`
`
`
`
`
`
`
`tion requires the user to enter another non-intuitive and
`
`
`
`
`
`
`
`
`complicated key sequence.
`
`
`
`Accordingly, what is needed is an improved user interface
`
`
`
`
`
`
`
`
`for a multi-mode digital camera. The present
`invention
`
`
`
`
`
`
`
`addresses such a need.
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`
`The present invention provides a method and system for
`
`
`
`
`
`
`
`
`
`integrating a user interface across multiple operating modes
`
`
`
`
`
`
`
`
`of a digital camera wherein mode-specific items are dis-
`
`
`
`
`
`
`
`
`played on a display when the digital camera is placed into a
`
`
`
`
`
`
`
`
`
`
`
`
`particular operating mode. The digital camera includes a first
`
`
`
`
`
`
`
`
`and a second navigation button for interacting with the
`
`
`
`
`
`
`
`
`
`operating modes, where the first navigation button has a first
`
`
`
`
`
`
`
`
`
`orientation and the second navigation button has a second
`
`
`
`
`
`
`
`
`
`orientation. The method includes the step of mapping an
`
`
`
`
`
`
`
`
`
`aligned set of mode-specific items in the display to the
`
`
`
`
`
`
`
`
`
`
`orientation of the first navigation button. After the mode-
`
`
`
`
`
`
`
`
`specific items are displayed,
`the user scrolls from one
`
`
`
`
`
`
`
`
`
`mode-specific item to the next in the aligned set by pressing
`
`
`
`
`
`
`
`
`
`
`
`the first navigation button, and the display indicates which
`
`
`
`
`
`
`
`
`
`of the mode-specific items is a currently active item. After
`
`
`
`
`
`
`
`
`
`
`a mode-specific item becomes the active item, additional
`
`
`
`
`
`
`
`
`information is displayed corresponding to the currently
`
`
`
`
`
`
`
`active item in the display in a location that is offset from the
`
`
`
`
`
`
`
`
`
`
`
`
`
`active item in a direction of orientation corresponding to that
`
`
`
`
`
`
`
`
`
`of the second button.
`In certain modes,
`the additional
`
`
`
`
`
`
`
`
`
`information includes a list of information items that
`is
`
`
`
`
`
`
`
`
`
`displayed in an alignment corresponding to the orientation
`
`
`
`
`
`
`
`
`of the second button, wherein the user can scroll through the
`
`
`
`
`
`
`
`
`
`
`
`list of information items using the second navigation button.
`
`
`
`
`
`
`
`
`
`According to the method and apparatus disclosed herein,
`
`
`
`
`
`
`
`
`the digital camera is provided with more than two modes
`
`
`
`
`
`
`
`
`
`
`wherein the user can navigate, manipulate, and view camera
`
`
`
`
`
`
`
`
`
`contents using a consistent and intuitive spatial navigation
`
`
`
`
`
`
`
`
`technique. Providing more than one mode in which the user
`
`
`
`
`
`
`
`
`
`
`can view images and camera contents reduces the complex-
`
`
`
`
`
`
`
`
`ity of the user interface, and the spatial navigation frees the
`
`
`
`
`
`
`
`
`
`
`
`user from entering long key sequences. The user interface
`
`
`
`
`
`
`
`
`
`also automatically displays context sensitive information
`
`
`
`
`
`
`regarding the active item, which reduces the input required
`
`
`
`
`
`
`
`
`
`from the user and thereby increases the ease of use and
`
`
`
`
`
`
`
`
`
`
`
`operation of the digital camera.
`
`
`
`
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`
`
`FIG. 1 is a block diagram of a digital camera that operates
`
`
`
`
`
`
`
`
`
`in accordance with the present invention.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`0013
`
`

`
`6,118,480
`
`
`
`
`
`
`
`
`
`
`
`3
`FIG. 2 is a block diagram of the preferred embodiment for
`
`
`
`
`
`
`
`
`
`
`the imaging device of FIG. 1.
`
`
`
`
`
`
`FIG. 3 is a block diagram of the preferred embodiment for
`
`
`
`
`
`
`
`
`
`
`the computer of FIG. 1.
`
`
`
`
`
`FIG. 4A is a memory map showing the preferred embodi-
`
`
`
`
`
`
`
`ment of the Dynamic Random-Access-Memory (DRAM).
`
`
`
`
`
`
`FIG. 4B is a diagram illustrating the input buffers and
`
`
`
`
`
`
`
`
`
`frame buffers.
`
`
`FIGS. 5A and 5B are diagrams depicting the back and top
`
`
`
`
`
`
`
`
`
`view, respectively, of a digital camera.
`
`
`
`
`
`FIG. 6 is a flow chart illustrating the process of integrating
`
`
`
`
`
`
`
`
`
`a user interface across multiple operating modes of a digital
`
`
`
`
`
`
`
`
`camera according to the present invention.
`
`
`
`
`
`
`FIG. 7 is a block diagram illustrating a preferred embodi-
`
`
`
`
`
`
`
`
`ment of the present invention in which the alignment of the
`
`
`
`
`
`
`
`
`
`
`
`mode-specific items are mapped to the orientation of the
`
`
`
`
`
`
`
`
`
`horizontal navigation buttons.
`
`
`
`FIG. 8 is a block diagram of the user interface illustrating
`
`
`
`
`
`
`
`
`
`that when the mode-specific items are displayed
`
`
`
`
`
`
`horizontally, information corresponding to an active item is
`
`
`
`
`
`
`
`displayed vertically offset from the row of mode-specific
`
`
`
`
`
`
`
`items.
`
`FIG. 9 is a diagram illustrating the operation and appear-
`
`
`
`
`
`
`
`
`
`ance of the integrated user interface during review mode in
`
`
`
`
`
`
`
`
`
`accordance with a preferred embodiment of the present
`
`
`
`
`
`
`
`invention.
`
`FIGS. 10A and 10B are diagrams illustrating the opera-
`
`
`
`
`
`
`
`
`tion and appearance of the integrated user interface during
`
`
`
`
`
`
`
`
`
`menu mode in accordance with a preferred embodiment of
`
`
`
`
`
`
`
`
`
`the present invention.
`
`
`
`FIGS. 11A and 11B are diagrams illustrating the operation
`
`
`
`
`
`
`
`
`and appearance of the integrated user interface during cap-
`
`
`
`
`
`
`
`
`ture mode in accordance with a preferred embodiment of the
`
`
`
`
`
`
`
`
`
`present invention.
`
`
`DETAILED DESCRIPTION OF THE
`
`
`
`INVENTION
`
`4
`
`incorporates the co-pending application by reference, and
`
`
`
`
`
`
`
`reproduces portions of that application herein with reference
`
`
`
`
`
`
`
`
`to FIGS. 1-3 for convenience.
`
`
`
`
`
`Referring now to FIG. 1, a block diagram of a digital
`
`
`
`
`
`
`
`
`
`
`
`camera 110 is shown according to the present invention.
`
`
`
`
`
`
`
`
`
`Camera 110 preferably comprises an imaging device 114, a
`
`
`
`
`
`
`
`
`
`system bus 116 and a computer 118. Imaging device 114 is
`
`
`
`
`
`
`
`
`
`
`
`optically coupled to an object 112 and electrically coupled
`
`
`
`
`
`
`
`
`
`via system bus 116 to computer 118. Once a photographer
`
`
`
`
`
`
`
`
`
`
`has focused imaging device 114 on object 112 and, using a
`
`
`
`
`
`
`
`
`
`
`
`capture button or some other means, instructed camera 110
`
`
`
`
`
`
`
`
`
`to capture an image of object 112, computer 118 commands
`
`
`
`
`
`
`
`
`
`
`imaging device 114 via system bus 116 to capture raw image
`
`
`
`
`
`
`
`
`
`
`
`data representing object 112. The captured raw image data is
`
`
`
`
`
`
`
`
`
`
`transferred over system bus 116 to computer 118 which
`
`
`
`
`
`
`
`
`
`performs various image processing functions on the image
`
`
`
`
`
`
`
`
`data before storing it in its internal memory. System bus 116
`
`
`
`
`
`
`
`
`
`
`
`also passes various status and control signals between imag-
`
`
`
`
`
`
`
`
`ing device 114 and computer 118.
`
`
`
`
`
`
`Referring now to FIG. 2, a block diagram of the preferred
`
`
`
`
`
`
`
`
`
`
`embodiment of imaging device 114 is shown. Imaging
`
`
`
`
`
`
`
`
`device 114 preferably comprises a lens 220 having an iris, a
`
`
`
`
`
`
`
`
`
`
`filter 222, an image sensor 224, a timing generator 226, an
`
`
`
`
`
`
`
`
`
`
`
`analog signal processor (ASP) 228, an analog-to-digital
`
`
`
`
`
`
`
`(A/D) converter 230, an interface 232, and one or more
`
`
`
`
`
`
`
`
`
`
`motors 234.
`
`
`U.S. Pat. No. 5,496,106, entitled “A System and Method
`
`
`
`
`
`
`
`
`
`For Generating a Contrast Overlay as a Focus Assist for an
`
`
`
`
`
`
`
`
`
`
`
`Imaging Device,” is incorporated herein by reference and
`
`
`
`
`
`
`
`
`provides a detailed discussion of the preferred elements of
`
`
`
`
`
`
`
`
`
`imaging device 114. Briefly, imaging device 114 captures an
`
`
`
`
`
`
`
`
`
`image of object 112 via reflected light impacting image
`
`
`
`
`
`
`
`
`
`sensor 224 along optical path 236. Image sensor 224, which
`
`
`
`
`
`
`
`
`
`
`is preferably a charged coupled device (CCD), responsively
`
`
`
`
`
`
`
`generates a set of raw image data in CCD format represent-
`
`
`
`
`
`
`
`
`
`ing the captured image 112. The raw image data is then
`
`
`
`
`
`
`
`
`
`
`routed through ASP 228, A/D converter 230 and interface
`
`
`
`
`
`
`
`
`232. Interface 232 has outputs for controlling ASP 228,
`
`
`
`
`
`
`
`
`motors 234 and timing generator 226. From interface 232,
`
`
`
`
`
`
`
`
`the raw image data passes over system bus 116 to computer
`
`
`
`
`
`
`
`
`
`
`118.
`
`Referring now to FIG. 3, a block diagram of the preferred
`
`
`
`
`
`
`
`
`
`
`embodiment for computer 118 is shown. System bus 116
`
`
`
`
`
`
`
`
`
`provides connection paths between imaging device 114, an
`
`
`
`
`
`
`
`
`optional power manager 342, central processing unit (CPU)
`
`
`
`
`
`
`
`
`344, dynamic random-access memory (DRAM) 346, input/
`
`
`
`
`
`
`
`
`output interface (I/O) 348, non-volatile memory 350, and
`
`
`
`
`
`
`
`buffers/connector 352. Removable memory 354 connects to
`
`
`
`
`
`
`
`system bus 116 via buffers/connector 352. Alternately, cam-
`
`
`
`
`
`
`
`era 110 may be implemented without removable memory
`
`
`
`
`
`
`
`354 or buffers/connector 352.
`
`
`
`
`Power manager 342 communicates via line 366 with
`
`
`
`
`
`
`
`power supply 356 and coordinates power management
`
`
`
`
`
`
`operations for camera 110. CPU 344 typically includes a
`
`
`
`
`
`
`
`
`conventional processor device for controlling the operation
`
`
`
`
`
`
`of camera 110. In the preferred embodiment, CPU 344 is
`
`
`
`
`
`
`
`
`
`capable of concurrently running multiple software routines
`
`
`
`
`
`
`to control the various processes of camera 110 within a
`
`
`
`
`
`
`
`
`
`multi-threading environment. DRAM 346 is a contiguous
`
`
`
`
`
`
`block of dynamic memory which may be selectively allo-
`
`
`
`
`
`
`
`
`cated to various storage functions. LCD controller 390
`
`
`
`
`
`
`
`
`accesses DRAM 346 and transfers processed image data to
`
`
`
`
`
`
`
`
`
`LCD screen 402 for display.
`
`
`
`
`
`I/O 348 is an interface device allowing communications
`
`
`
`
`
`
`
`
`to and from computer 118. For example, I/O 348 permits an
`
`
`
`
`
`
`
`
`
`
`
`external host computer (not shown) to connect
`to and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`
`
`15
`
`
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The present invention relates to an improvement in user
`
`
`
`
`
`
`
`
`interfaces of digital imaging devices, including digital cam-
`
`
`
`
`
`
`
`eras. The following description is presented to enable one of
`
`
`
`
`
`
`
`
`
`
`ordinary skill in the art to make and use the invention and is
`
`
`
`
`
`
`
`
`
`
`
`
`
`provided in the context of a patent application and its
`
`
`
`
`
`
`
`
`
`
`requirements. Various modifications to the preferred
`
`
`
`
`
`
`embodiment will be readily apparent to those skilled in the
`
`
`
`
`
`
`
`
`
`
`art and the generic principles herein may be applied to other
`
`
`
`
`
`
`
`
`
`
`
`embodiments. Thus, the present invention is not intended to
`
`
`
`
`
`
`
`
`
`be limited to the embodiment shown but is to be accorded
`
`
`
`
`
`
`
`
`
`
`
`the widest scope consistent with the principles and features
`
`
`
`
`
`
`
`
`
`described herein.
`
`
`The present
`invention is a method and apparatus for
`
`
`
`
`
`
`
`
`
`integrating a digital camera user interface across multiple
`
`
`
`
`
`
`
`
`operating modes. According to the present
`invention, a
`
`
`
`
`
`
`
`
`method and system is provided for providing a digital
`
`
`
`
`
`
`
`
`
`camera with more than two modes and for providing a
`
`
`
`
`
`
`
`
`
`
`consistent and intuitive user interface across the multiple
`
`
`
`
`
`
`
`
`modes. The operation of the user interface across the mul-
`
`
`
`
`
`
`
`
`
`tiple modes includes a user controlled horizontal interaction
`
`
`
`
`
`
`
`following by a reply from the camera of a vertical display of
`
`
`
`
`
`
`
`
`
`
`additional information in response to the user controlled
`
`
`
`
`
`
`
`
`horizontal interaction.
`
`
`A digital camera architecture has been disclosed in
`
`
`
`
`
`
`
`
`co-pending U.S. patent application Ser. No. 08/666,241,
`
`
`
`
`
`
`
`entitled “A System And Method For Using A Unified
`
`
`
`
`
`
`
`
`
`Memory Architecture To Implement A Digital Camera
`
`
`
`
`
`
`
`Device,” filed on Jun. 20, 1996. The Applicant hereby
`
`
`
`
`
`
`
`
`
`
`0014
`
`0014
`
`

`
`6
`
`6,118,480
`
`
`
`5
`
`communicate with computer 118. I/O 348 also interfaces
`
`
`
`
`
`
`
`with a plurality of buttons and/or dials 404, and an optional
`
`
`
`
`
`
`
`
`
`
`status LCD 406, which in addition to the LCD screen 402,
`
`
`
`
`
`
`
`
`
`
`are the hardware elements of the camera’s user interface
`
`
`
`
`
`
`
`
`408.
`
`
`
`
`
`
`
`Non-volatile memory 350, which may typically comprise
`
`
`
`
`
`
`
`a conventional read-only memory or flash memory, stores a
`
`
`
`
`
`
`
`
`
`set of computer-readable program instructions to control the
`
`
`
`
`
`
`
`
`operation of camera 110. Removable memory 354 serves as
`
`
`
`
`
`
`
`
`
`an additional image data storage area and is preferably a
`
`
`
`
`
`
`
`
`
`
`non-volatile device, readily removable and replaceable by a
`
`
`
`
`
`
`
`camera 110 user via buffers/connector 352. Thus, a user who
`
`
`
`
`
`
`
`
`
`possesses several removable memories 354 may replace a
`
`
`
`
`
`
`
`
`full removable memory 354 with an empty removable
`
`
`
`
`
`
`
`
`memory 354 to effectively expand the picture-taking capac-
`
`
`
`
`
`
`
`ity of camera 110. In the preferred embodiment of the
`
`
`
`
`
`
`
`
`
`
`present
`invention,
`removable memory 354 is typically
`
`
`
`
`
`
`
`implemented using a flash disk.
`
`
`
`
`
`Power supply 356 supplies operating power to the various
`
`
`
`
`
`
`
`
`
`components of camera 110. In the preferred embodiment,
`
`
`
`
`
`
`
`
`power supply 356 provides operating power to a main power
`
`
`
`
`
`
`
`
`
`
`bus 362 and also to a secondary power bus 364. The main
`
`
`
`
`
`
`
`
`
`
`
`
`power bus 362 provides power to imaging device 114, I/O
`
`
`
`
`
`
`
`
`
`
`348, non-volatile memory 350 and removable memory 354.
`
`
`
`
`
`
`
`
`The secondary power bus 364 provides power to power
`
`
`
`
`
`
`
`
`
`manager 342, CPU 344 and DRAM 346.
`
`
`
`
`
`
`
`Power supply 356 is connected to main batteries 358 and
`
`
`
`
`
`
`
`
`
`
`also to backup batteries 360. In the preferred embodiment,
`
`
`
`
`
`
`
`
`
`a camera 110 user may also connect power supply 356 to an
`
`
`
`
`
`
`
`
`
`
`
`
`external power source. During normal operation of power
`
`
`
`
`
`
`
`
`supply 356, the main batteries 358 provide operating power
`
`
`
`
`
`
`
`
`
`to power supply 356 which then provides the operating
`
`
`
`
`
`
`
`
`
`power to camera 110 via both main power bus 362 and
`
`
`
`
`
`
`
`
`
`
`
`secondary power bus 364. During a power failure mode in
`
`
`
`
`
`
`
`
`
`
`which the main batteries 358 have failed (when their output
`
`
`
`
`
`
`
`
`
`
`voltage has fallen below a minimum operational voltage
`
`
`
`
`
`
`
`
`level) the backup batteries 360 provide operating power to
`
`
`
`
`
`
`
`
`
`power supply 356 which then provides the operating power
`
`
`
`
`
`
`
`
`
`only to the secondary power bus 364 of camera 110.
`
`
`
`
`
`
`
`
`
`
`Referring now to FIG. 4A, a memory map showing the
`
`
`
`
`
`
`
`
`
`
`preferred embodiment of dynamic random-access-memory
`
`
`
`
`
`(DRAM) 346 is shown.
`In the preferred embodiment,
`
`
`
`
`
`
`
`
`DRAM 346 includes RAM disk 532, a system area 534, and
`
`
`
`
`
`
`
`
`
`
`
`working memory 530.
`
`
`
`RAM disk 532 is a memory area used for storing raw and
`
`
`
`
`
`
`
`
`
`
`
`compressed image data and typically is organized in a
`
`
`
`
`
`
`
`
`
`“sectored” format similar to that of conventional hard disk
`
`
`
`
`
`
`
`
`
`drives. In the preferred embodiment, RAM disk 532 uses a
`
`
`
`
`
`
`
`
`
`
`well-known and standardized file system to permit external
`
`
`
`
`
`
`
`
`host computer systems, via I/O 348, to readily recognize and
`
`
`
`
`
`
`
`
`
`
`access the data stored on RAM disk 532. System area 534
`
`
`
`
`
`
`
`
`
`
`
`typically stores data regarding system errors (for example,
`
`
`
`
`
`
`
`
`why a system shutdown occurred) for use by CPU 344 upon
`
`
`
`
`
`
`
`
`
`
`a restart of computer 118.
`
`
`
`
`
`Working memory 530 includes various stacks, data struc-
`
`
`
`
`
`
`
`tures and variables used by CPU 344 while executing the
`
`
`
`
`
`
`
`
`
`
`software routines used within computer 118. Working
`
`
`
`
`
`
`
`memory 530 also includes input buffers 538 for initially
`
`
`
`
`
`
`
`
`
`storing sets of raw image data received from imaging device
`
`
`
`
`
`
`
`
`
`
`114 for image conversion, and frame buffers 536 for storing
`
`
`
`
`
`
`
`
`
`
`data for display on the LCD screen 402.
`
`
`
`
`
`
`
`
`In a preferred embodiment,
`the conversion process is
`
`
`
`
`
`
`
`performed by a live view generation program, which is
`
`
`
`
`
`
`
`
`stored in non-volatile memory 350 and executed on CPU
`
`
`
`
`
`
`
`
`344. However, the conversion process can also be imple-
`
`
`
`
`
`
`
`
`mented using hardware. Referring again to FIG. 3, during
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`
`
`15
`
`
`
`20
`
`
`
`25
`
`
`30
`
`
`
`35
`
`
`
`40
`
`
`
`45
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`0015
`
`
`the execution of the live view generation program (not
`
`
`
`
`
`
`
`
`
`shown), the CPU 344 takes the raw image data from the
`
`
`
`
`
`
`
`
`
`
`
`input buffers 538 in CCD format and performs color space
`
`
`
`
`
`
`
`
`
`
`conversion on the data. The conversions process performs
`
`
`
`
`
`
`
`
`gamma correction and converts the raw CCD data into either
`
`
`
`
`
`
`
`
`
`
`a RGB or YCC color format which is compatible with the
`
`
`
`
`
`
`
`
`
`
`
`LCD screen 402. After the conversion, CPU 344 stores the
`
`
`
`
`
`
`
`
`
`
`image data in the frame buffers 536. The LCD controller 390
`
`
`
`
`
`
`
`
`
`
`
`then transfers the processed image data from the frame
`
`
`
`
`
`
`
`
`
`buffers to the LCD screen 402 (via an optional analog
`
`
`
`
`
`
`
`
`
`
`converter) for display.
`
`
`
`Referring now to FIG. 4B, the contents of input buffers
`
`
`
`
`
`
`
`
`
`
`538 and the frame buffers 536 are shown. In a preferred
`
`
`
`
`
`
`
`
`
`
`
`embodiment, both the input buffers 538 and the frame
`
`
`
`
`
`
`
`
`
`buffers 536 utilize two separate buffers, called ping-pong
`
`
`
`
`
`
`
`
`
`buffers, to improve the display speed of the digital camera
`
`
`
`
`
`
`
`
`
`
`and to prevent the tearing of the image in the display 402.
`
`
`
`
`
`
`
`
`
`
`
`
`As shown, input buffers 538 include an input buffer A and an
`
`
`
`
`
`
`
`
`
`
`
`
`input buffer B, and frame buffers 536 include a frame buffer
`
`
`
`
`
`
`
`
`
`
`
`A and a frame buffer B.
`
`
`
`
`
`
`The input buffersAand B alternate between an input cycle
`
`
`
`
`
`
`
`
`
`
`
`and a processing cycle. During the input cycle, the input
`
`
`
`
`
`
`
`
`
`
`buffers 538 are filled with raw image data from the image
`
`
`
`
`
`
`
`
`
`
`
`device 114, and during the processing cycle, CPU 344
`
`
`
`
`
`
`
`
`
`processes the raw data and transmits the processed data to
`
`
`
`
`
`
`
`
`
`
`the frame buffers 536. More specifically, while input buffer
`
`
`
`
`
`
`
`
`
`A is filling with image data, the data from input buffer B is
`
`
`
`
`
`
`
`
`
`
`
`
`
`processed and transmitted to frame buffer B. At the same
`
`
`
`
`
`
`
`
`
`
`time, previously processed data in frame buffer A is output
`
`
`
`
`
`
`
`
`
`
`to the LCD screen 402 for display. While input buffer B is
`
`
`
`
`
`
`
`
`
`
`
`
`filling with image data,
`the data from input buffer A is
`
`
`
`
`
`
`
`
`
`
`
`processed and transmitted to frame buffer A. At the same
`
`
`
`
`
`
`
`
`
`
`time, previously processed data in frame buffer B is output
`
`
`
`
`
`
`
`
`
`
`to the LCD screen 402 for display.
`
`
`
`
`
`
`
`According to the present invention, the flexible architec-
`
`
`
`
`
`
`
`ture of the digital camera is used to provide and integrated
`
`
`
`
`
`
`
`
`
`
`camera user interface. More specifically, the present inven-
`
`
`
`
`
`
`
`tion provides a method and system for integrating a digital-
`
`
`
`
`
`
`
`
`
`camera user-interface across multiple operating modes of
`
`
`
`
`
`
`the digital camera.
`
`
`
`FIGS. 5A and 5B are diagrams depicting the hardware
`
`
`
`
`
`
`
`
`
`components of the camera’s 110 user interface 408. FIG. 5A
`
`
`
`
`
`
`
`
`
`
`is back view of the camera 110 showing the LCD screen 402,
`
`
`
`
`
`
`
`
`
`
`
`
`a four-way navigation control button 409, an overlay button
`
`
`
`
`
`
`
`
`
`412, a menu button 414, and a set of programmable soft keys
`
`
`
`
`
`
`
`
`
`
`416. FIG. 5B is a top view of the camera 110 showing a
`
`
`
`
`
`
`
`
`
`
`
`
`
`shutter button 418, and a mode dial 420. The camera may
`
`
`
`
`
`
`
`
`
`
`
`optionally include status LCD 406, status LCD scroll and
`
`
`
`
`
`
`
`
`
`select buttons 422 and 424, a sound record button 426, and
`
`
`
`
`
`
`
`
`
`
`
`zoom-in, zoom-out buttons 426a and 426b.
`
`
`
`
`
`
`In one aspect of the present invention, the user interface
`
`
`
`
`
`
`
`
`
`408 includes several different operating modes for support-
`
`
`
`
`
`
`
`ing various camera functions. However, the modes relevant
`
`
`
`
`
`
`
`
`to this description are review mode, menu mode, and capture
`
`
`
`
`
`
`
`
`
`
`(record) mode. In review mode, the camera 100 supports the
`
`
`
`
`
`
`
`
`
`
`actions of reviewing camera contents, editi