United States Patent [19]
`Kagle
`
`US006148149A
`[11] Patent Number:
`[45] Date of Patent:
`
`6,148,149
`Nov. 14, 2000
`
`[54] AUTOMATIC IMAGE ROTATION IN
`DIGITAL CAMERAS
`
`5,576,759 11/1996 Kawamura et a1. .................. .. 348/231
`5,900,909
`5/1999 Parulski e181. ....................... .. 348/232
`
`[75] Inventor: Jonathan Kagle, Redmond, Wash.
`
`[73] Assignee: xhgllicgosoft Corporation, Redmond,
`
`as .
`
`[21] Appl. N0.: 09/085,144
`_
`May 26’ 1998
`[22] Flled:
`[51]
`Int. Cl.7 ................................................... .. G03B 17/00
`[52] US. Cl. ........................ .. 396/50; 358/909.1; 348/231
`[58] Field of Search ............................ .. 396/50, 311, 321,
`396/374> 543> 13; 358/906> 9091; 348/333>
`239’ 240’ 231’ 232
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`Primary Examiner—W. B. Perkey
`Attorney) Agent) 0,, Firm_Lee & Hayes, PLLC
`
`[57]
`
`ABSTRACT
`
`A digital camera comprises a controller and an image sensor
`that digitally captures an image. The camera also includes an
`orientation sensor, such as a mercury sWitch, that that
`indicates orientation of the image sensor Whenever an image
`is captured. When capturing an image, the controller is
`con?gured to format the image in a Way that it Will be
`correctly oriented When displayed. In one embodiment, the
`camera automatically rotates the digital image to correct for
`different orientations of the camera.
`
`5,262,867 11/1993 Kojima ........................... .. 358/9091 X
`
`15 Claims, 4 Drawing Sheets
`
`10
`
`km
`”
`
`K11
`1:15”
`
`@012
`
`Canon Exhibit 1026, Page 1
`
`

`

`U.S. Patent
`
`Nov. 14,2000
`
`Sheet 1 of4
`
`6,148,149
`
`COLUMN 3
`COLUMN 2
`COLUMN 1
`
`Canon Exhibit 1026, Page 2
`
`

`

`U.S. Patent
`
`Nov. 14,2000
`
`Sheet 2 of4
`
`6,148,149
`
`(ROW1
`
`l ROW 2
`l ROW 3
`
`§
`§
`§
`
`§§§§§§§
`§§§§§§§
`
`/
`
`11
`
`14
`
`COLUMN 1
`COLUMN 2
`COLUMN 3
`
`llllllll
`
`Canon Exhibit 1026, Page 3
`
`

`

`U.S. Patent
`
`Nov. 14,2000
`
`Sheet 3 of4
`
`6,148,149
`
`f0
`
`Image Sensor
`Array
`
`Lens
`
`Processor
`
`Removable
`Memory
`
`Non-Volatile
`Memory
`
`Vo|at||e Memory
`
`30
`
`Orientation
`Sensor
`
`Canon Exhibit 1026, Page 4
`
`

`

`U.S. Patent
`
`Nov. 14, 2000
`
`Sheet 4 of4
`
`6,148,149
`
`100
`
`102
`
`104
`
`106
`
`i
`i
`
`Trigger Sensor and
`Transfer image to
`Memory
`
`Detect Camera
`Orientation
`
`i
`V
`
` n hm “t We m em
`
`mm
`
` mm M... .U m m rn we r O
`
`
`
`i
`
`Save Image
`
`
`
`a?!‘ 03.380
`
`Q Q Q Q Q Q Q Q Q Q
`
`Canon Exhibit 1026, Page 5
`
`

`

`1
`AUTOMATIC IMAGE ROTATION IN
`DIGITAL CAMERAS
`
`TECHNICAL FIELD
`
`This invention relates to digital cameras and to methods
`of correcting for camera rotation When acquiring digital
`images With such cameras.
`
`BACKGROUND OF THE INVENTION
`
`Digital cameras are becoming more and more popular as
`their price decreases and the quality of their images
`increases. Consumers like digital cameras because images
`taken With such cameras can be quickly vieWed on a
`computer. In addition, a computer can be used to correct and
`alter digital images. This alloWs the photographer to eXercise
`a high degree of control and creativity, Without compleX
`chemical procedures.
`Thus, the involvement of a computer in the picture-taking
`process alloWs functions that Were previously obtainable
`only in conjunction With a photo processing lab. On the
`other hand, hoWever, the computer adds a step that Was
`previously not required for vieWing snapshots. With ?lm
`based technology, a consumer could simply drop a ?lm
`cartridge With a photo developer and pick up the resulting
`prints. Although this did not provide the creative opportu
`nities that are possible With digital images, it Was simpler
`and required less effort.
`One speci?c inconvenience When using digital cameras,
`Which this invention addresses, relates to correcting images
`that are taken When the digital camera is rotated 90° from its
`normal orientation. When held normally, a camera captures
`an image having a larger Width dimension than height
`dimension. The resulting image is said to be in a “landscape”
`orientation. In many instances, hoWever, a user Wishes to
`create an image having a larger height dimension as com
`pared to the Width dimension. The resulting image is said to
`be in a “portrait” orientation. To capture an image With a
`portrait orientation, the user simply rotates the camera 90°.
`With ?lm-based cameras this presents no particular dif
`?culties. Landscape and portrait images are oriented differ
`ently on the eXposed ?lm. HoWever, prints can be made from
`the ?lm Without any knoWledge of the actual orientation.
`When a user broWses through the prints, he or she simply
`rotates them, by hand, to the correct vieWing orientation.
`With a digital camera, hoWever, images are often dis
`played on a computer monitor. Any images taken in the
`portrait orientation Will be displayed at 90° from their
`desired orientation. Since a computer monitor cannot be
`easily rotated to correct for pictures taken at different
`orientations, it is usually desired to rotate the pictures
`digitally before vieWing.
`Although this is a task that can be readily performed by
`the user’s computer, the task also requires user intervention.
`Thus, When a user doWnloads pictures from a digital camera,
`each picture needs to be vieWed and checked for correct
`orientation. User intervention is required to rotate any pic
`tures that Were taken in the portrait orientation. Although
`this process is not dif?cult, it requires time and can become
`tedious When doWnloading and correcting doZens of pic
`tures.
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`SUMMARY OF THE INVENTION
`
`In accordance With the invention, a digital camera is
`equipped With a sensor that indicates orientation of the
`camera at the time an image is captured. The camera notes
`
`65
`
`6,148,149
`
`2
`the orientation and creates an image object in a prede?ned
`image format that indicates correct orientation of the image
`based on the orientation of the camera When the image Was
`captured. In one embodiment, the camera simply sets a ?ag
`that indicates Whether the image needs to be rotated for
`vieWing. In another embodiment, the camera automatically
`reformats the digital version of the image to re?ect the
`proper orientation of the image, before sending the digital
`picture to a personal computer or other vieWing device. This
`eliminates the need to previeW and rotate pictures When
`doWnloading them to the computer.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a front perspective vieW of a digital camera in
`accordance With the invention.
`FIG. 2 is a diagrammatic vieW shoWing the position and
`orientation of an image sensor array Within the digital
`camera shoWn in FIG. 1, With the digital camera in a
`landscape orientation.
`FIG. 3 is a diagrammatic vieW shoWing the position and
`orientation of the image sensor array Within the digital
`camera shoWn in FIG. 1, With the digital camera in a portrait
`orientation.
`FIG. 4 is a block diagram shoWing internal components of
`the digital camera of FIG. 1.
`FIG. 5 is a ?oWchart shoWing preferred methodological
`steps in accordance With the invention.
`FIG. 6 is a conceptual vieW shoWing formatting of an
`image Without rotation.
`FIG. 7 is a conceptual vieW shoWing formatting of an
`image With rotation.
`
`DETAILED DESCRIPTION
`
`FIG. 1 shoWs a digital camera 10 in accordance With the
`invention. Camera 10 comprises a case or housing 11 and a
`lens 12 mounted thereon that focuses on an image that is to
`be digitally captured. Camera 10 includes a vieW?nder 13
`and a button 14 that triggers or initiates image capture.
`Camera 10 is shoWn in its default, landscape orientation.
`FIG. 2 illustrates the position and orientation of an image
`sensor array 15 Within camera 10. The image sensor com
`prises a rectangular array of individual piXel sensors 16,
`arranged in roWs and columns. In most cameras, there are
`more columns than roWs, so that a captured image Will have
`conventional landscape dimensions (in Which the Width is
`greater than the height). When capturing an image, the roWs
`and columns of piXel sensors produce corresponding roWs
`and columns of piXel data. When the camera is in its default
`orientation as shoWn, the roWs are oriented horiZontally and
`the columns are oriented vertically. RoWs 1—3 and columns
`1—3 are labeled in FIG. 2.
`Note that only a limited number of piXel sensors are
`shoWn in FIG. 2 for purposes of illustration, as compared to
`the much greater number of piXel sensors in most commer
`cial image sensor arrays.
`FIG. 3 is the same as FIG. 2, eXcept it shoWs the
`orientation of image sensor array 14 When the camera is in
`its non-default, rotated, portrait orientation. This orientation
`is rotated 90° from the default, landscape orientation. In this
`orientation, the roWs of image sensor array 15 are oriented
`vertically, and the columns are oriented horiZontally.
`FIG. 4 shoWs high-level components of digital camera 10,
`including lens 12, sensor array 15, a microcontroller or
`microprocessor 16, non-volatile memory 18, volatile
`
`Canon Exhibit 1026, Page 6
`
`

`

`3
`memory 20, and removable memory 22. Lens 12 is posi
`tioned to focus an image on sensor array 15. Microcontroller
`16 is con?gured to control lens 12 and sensor array 15 in
`order to capture a digital image in response to a user
`command (such as by pressing trigger button 14). The digital
`image at this stage is simply a plurality of piXel values.
`Digital images are usually stored as objects or ?les in
`some type of nonvolatile memory, according to some pre
`de?ned and standardiZed format such as “J PEG,” “GIF,” etc.
`Within each ?le, piXels values are typically arranged in a
`speci?c order, such as from left-to-right and from top-to
`bottom. Thus, the organiZation and order of piXel values
`Within the image object de?nes the orientation at Which a
`computer or computer-like device Will attempt to display the
`image. An image object in accordance With the prede?ned
`format contains piXel roWs that eXtend horiZontally relative
`to the orientation of the image When the image is eventually
`displayed.
`During or after capturing an image, the camera transfers
`the piXel values from sensor array 15 to volatile memory 20
`for processing. This processing involves arranging or for
`matting the piXel values into an image object conforming to
`a pre-de?ned standard format, such as one of the “JPEG” or
`“GIF” standards mentioned above. The processing often
`includes data compression. Once the image has been for
`matted and compressed, it is transferred to removable
`memory 22. The removable memory can be removed from
`the camera and plugged into a computer or other device for
`vieWing or doWnloading the images stored thereon.
`In addition to the components above, camera 10 includes
`an orientation sensor 30 Within the body of the camera to
`indicate the orientation of the image sensor When an image
`is captured. The orientation sensor indicates Whether the
`camera is being held in its default, landscape orientation, as
`shoWn in FIGS. 1 and 2, or in a rotated, portrait orientation
`as shoWn in FIG. 3, that is 90° from the landscape orienta
`tion. The sensor is connected so that it can be read by
`microcontroller 16. The sensor enables automatic digital
`rotation of captured images to correct for different orienta
`tions of the image sensor.
`To capture an image, microcontroller 16 is con?gured or
`programmed to perform the steps indicated in FIG. 5. A?rst
`step 100 comprises activating the image sensor 15 and
`transferring an image from the image sensor to volatile
`memory 20. The image comprises roWs and columns of
`piXel values corresponding to the roWs and columns of
`individual piXel sensors. Generally, the piXel values are
`Written to volatile memory 20 in the same order in Which
`they are read from sensor array 15.
`Astep 102, actually performed either immediately before,
`immediately after, or concurrently With step 100, comprising
`detecting and noting the orientation of the camera and image
`sensor during image capture. Speci?cally, this step com
`prises detecting and noting Whether the camera is in its
`default orientation or its rotated orientation.
`A step 104 is then performed of formatting the image in
`the prede?ned digital format. The image is formatted in such
`a Way that it indicates correct orientation of the image based
`on the orientation of the camera and image sensor When the
`image Was captured, as indicated by the orientation sensor.
`This can be done in at least tWo different Ways. One Way is
`to embed a ?ag Within the formatted image object that
`speci?es the correct orientation by specifying the orientation
`of the camera When the image Was captured. Actual rotation
`of the digital image can then be performed outside the
`camera, by a computer or other processing device, prior to
`viewing.
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6,148,149
`
`4
`Another Way is for the microcontroller of the camera itself
`to automatically rotate the image to correct for camera
`rotation—to re-order piXel values from the image sensor and
`to thereby orient the image Within the prede?ned digital
`format depending on the orientation of the image sensor
`When the image Was captured. In this embodiment, the
`camera automatically rearranges piXel values Within the
`prede?ned digital format of the image object to represent the
`corrected orientation of the image. If the digital camera Was
`in the default orientation When the image Was captured, the
`controller Writes the roWs of piXel data to corresponding
`piXel roWs of the image object. HoWever, if the digital
`camera Was not in the default orientation When capturing the
`image, the controller Writes the columns of piXel values from
`the image sensors to corresponding piXel roWs of the image
`object. This effectively rotates the image Within the image
`object, so that it Will be displayed correctly Without further
`manipulations When transferred to a vieWing device such as
`a personal computer.
`
`Such digital rotation of an image is illustrated in FIGS. 6
`and 7. On the left, each ?gure shoWs an array of piXels as
`they are read from the image sensor array. On the right, each
`?gure shoWs the same piXels as they are arranged in an
`actual image object. In FIG. 6, a landscape image is shoWn.
`No rotation is performed because the sensor indicated that
`the camera Was in the landscape orientation When the image
`Was captures. Image sensor roWs are transferred directly to
`image object roWs. In FIG. 7, hoWever, the sensor indicated
`a portrait orientation. Accordingly, roWs are reformatted as
`columns to thereby rotate the image by 90. FIG. 7 illustrates
`a clockwise rotation of an image.
`Step 106 comprises saving the image in the prede?ned
`format, in non-volatile memory 18. This step comprises
`Writing the roWs and columns of piXel values to an image
`object having the prede?ned format. From there, the image
`can be transferred to a device other than the camera for
`vieWing.
`Note that various types of automatic orientation sensors
`can be used, such as mercury sWitches, self-orienting
`sWitches, optical sensors, and gyroscopic devices.
`Furthermore, the orientation sensors can conceivably infer
`orientation from some factor relating to the user’s handling
`of the camera, such as by Which of multiple activation
`buttons the user activates to capture a picture. As another
`alternative, a sensor or sensors could detect the position of
`a user’s hands and ?ngers on the camera to detect camera
`orientation.
`
`The invention eliminates the time-consuming step of
`previeWing each picture as it is doWnloaded to a personal
`computer. In one embodiment, orientation information
`supplements actual piXel data, alloWing the personal com
`puter to automatically rotate pictures that Were taken With
`the camera in a non-default orientation. In the other
`embodiment, the camera itself automatically rotates the
`images before saving them or transferring them to a personal
`computer or other storage device.
`Although the invention has been described in language
`speci?c to structural features and/or methodological steps, it
`is to be understood that the invention de?ned in the
`appended claims is not necessarily limited to the speci?c
`features or steps described. Rather, the speci?c features and
`steps are disclosed as preferred forms of implementing the
`claimed invention.
`
`Canon Exhibit 1026, Page 7
`
`

`

`6,148,149
`
`5
`
`What is claimed is:
`1. A digital camera comprising:
`an image sensor that digitally captures images;
`an optical sensor that differentiates betWeen landscape
`and portrait orientations of the image sensor When
`capturing images;
`the digital camera inserting an indicator in a captured
`image to indicate Whether the image has been captured
`in a landscape orientation or a portrait orientation as
`differentiated by the optical sensor.
`2. A digital camera comprising:
`an image sensor that digitally captures images;
`a gyroscopic device that differentiates betWeen landscape
`and portrait orientations of the image sensor When
`capturing images;
`the digital camera inserting an indicator in a captured
`image to indicate Whether the image has been captured
`in a landscape orientation or a portrait orientation as
`differentiated by the gyroscopic device.
`3. A method of capturing a digital image using a digital
`camera, comprising:
`capturing an image from an image sensor;
`detecting orientation of the image sensor When capturing
`the image based on a user’s contact With the digital
`camera;
`saving the image in a prede?ned format that includes an
`indicator of Whether the image has been captured in a
`landscape orientation or a portrait orientation.
`4. A method of capturing a digital image comprising:
`capturing an image from an image sensor;
`detecting orientation of the image sensor When capturing
`the image With an optical sensor;
`saving the image in a prede?ned format that includes an
`indicator of Whether the image has been captured in a
`landscape orientation or a portrait orientation as
`detected by the optical sensor.
`5. A method of capturing a digital image comprising:
`capturing an image from an image sensor;
`detecting orientation of the image sensor When capturing
`the image With a gyroscopic device;
`saving the image in a prede?ned format that includes an
`indicator of Whether the image has been captured in a
`landscape orientation or a portrait orientation as
`detected by the gyroscopic device.
`
`10
`
`25
`
`35
`
`45
`
`6
`6. A digital camera comprising:
`an image sensor that digitally captures images;
`an orientation sensor that differentiates betWeen land
`scape and portrait orientations of the image sensor
`based on a user’s contact With the digital camera When
`capturing images.
`7. A digital camera as recited in claim 6, Wherein the
`digital camera inserts an indicator in a captured image to
`indicate Whether the image has been captured in a landscape
`orientation or a portrait orientation as differentiated by the
`orientation sensor.
`8. A digital camera as recited in claim 6, Wherein the
`orientation sensor indicates the differentiated orientation of
`the image sensor to enable subsequent digital rotation of the
`captured images.
`9. A digital camera as recited in claim 6, further compris
`ing a processor that formats captured images in a prede?ned
`image format, the prede?ned image format including an
`indicator to indicate the differentiated orientation.
`10. A digital camera as recited in claim 6, further com
`prising a controller con?gured to transfer captured images
`from the image sensor and to format the captured images in
`a prede?ned digital format, the prede?ned digital format
`including an indicator to indicate the differentiated orienta
`tion.
`11. A digital camera comprising:
`an image sensor that digitally captures images;
`a non-gravity-based orientation sensor that differentiates
`betWeen landscape and portrait orientations of the
`image sensor.
`12. A digital camera as recited in claim 11 Wherein the
`orientation sensor is an optical sensor.
`13. A digital camera as recited in claim 11 Wherein the
`orientation sensor is a gyroscopic device.
`14. A digital camera as recited in claim 11 Wherein the
`orientation sensor senses a user’s contact With the camera to
`differentiate betWeen landscape and portrait orientations.
`15. A digital camera as recited in claim 11, Wherein the
`digital camera inserts an indicator in a captured image to
`indicate Whether the image has been captured in a landscape
`orientation or a portrait orientation as differentiated by the
`orientation sensor.
`
`Canon Exhibit 1026, Page 8
`
`