W0 0]/37209
`
`PCT/US00/30825
`
`-17-
`
`However, in yet other alternate embodiments, each image file could be organized in “quality
`level support order” with the data for each analysis array being arranged so that each
`successive data structure 172 stores the image information needed to increase image quality by
`one predefined image quality level. Thus, the information in some data structures 172 might
`represent two, three or more bit planes» of information.
`In this embodiment, an image can be
`reduced by one quality level by deleting the last data structure 172 from every analysis array
`data structure 172 in the image file.
`
`l0
`
`15
`
`20
`
`25'
`
`30
`
`Digital Camera State Machines
`
`For the purposes of this explanation, it will be assumed that the digital camera 100 has four
`
`predefined image quality levels: High, Very Good +, Very Good -, and Good.
`
`It will be further
`
`assumed that image files stored at High quality typically occupy about twice as much space as
`image files stored at Good quality. In other embodiments, more or fewer image quality levels
`could be used, and the ratio ofimage file sizes from highest to lowest quality could be larger or
`smaller than 2:1. For instance, ifthe camera is capable oftaking very high resolution images,
`such as 2000 x 2000 pixels or even 4000 x 4000 pixels, and at very high fidelity, then it would
`
`make sense to provide a large number of quality levels with a ratio ofimage file sizes from
`highest to lowest quality of perhaps as high as 6421.
`
`It is noted that an image file’s quality cannot be increased once it has been lowered, unless the
`original image file or an alternate source thereof remains available, because the information
`
`needed to restore the image’s quality has been lost.
`
`Referring back to Fig. 1, the digital camera 100 preferably includes data processing circuitry
`106 for performing a predefined set of primitive operations, such as performing the multiply
`and addition operations required to apply a transform to a certain amount ofimage data, as
`well as a set of state machines 200-212 for controlling the data processingcircuitry so as to
`perform a set of predefined image handling operations.
`In one embodiment, the state machines
`in the digital camera are as follows.
`
`One or morestate machines 200 for transforming, compressing and storing an image
`°
`received from the camera’s image capture mechanism. This image is sometimes called the
`“viewfinder” image, since the image being processed is generally the one seen on the camera's
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4001
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4001
`
`

`
`wo 01/37209
`
`PCT/US00/30825
`
`_ 13 -
`
`image viewer 114. This set of state machines 200 are the ones that initially generate each
`
`image file stored in the nonvolatile image memory 108. Prior to taking the picture. the user
`
`specifies the quality level ofthe image to be stored, using the camera’s buttons 112.
`
`It should
`
`noted that in most digital cameras the viewfinder is capable of displaying only a very small and
`
`low fidelity version of the captured image, and thus the image displayed in the camera’s
`
`viewfinder is typically a much lower quality rendition ofthe captured image than the quality of
`
`the “viewfinder” image stored in the image file.
`
`-
`
`One or more state machines 202 for decompressing, inverse transfonning and
`
`displaying a stored image file on the camera’s image viewer. The reconstructed image
`
`generated by decompressing, inverse transforming and dequantizing the image data is stored in
`
`camera’s framebuffer 1 18 so that it can be viewed on the image viewer 114.
`
`-
`
`One or more state machines 204 for updating and displaying a count of the number of
`
`images stored in the nonvolatile image memory 108. The image count is preferably displayed
`
`on the user interface display 1 16. This set of state machines 204 will also typically indicate
`
`what percentage of the nonvolatile image memory 108 remains unoccupied by image files. or
`
`some other indication of the camera’s ability to store additional images. If the camera does not
`
`have a separate interface display 116, this memory status information maybe shown on the
`
`image viewer 1 14, for instance superimposed on the image shown in the image viewer 1 14 or
`
`shown in a region ofthe viewer 114 separate from the main viewer image.
`
`°
`
`One or more state machines 206 for implementing a “viewfinder" mode for the camera
`
`in which the image currently “seen" by the image capture mechanism 102 is displayed on the
`
`image viewer 1 14 to that the user can see the image that would be stored if the image capture
`
`button is pressed. These state machines transfer the image received from the image capture
`
`device 102, possibly after appropriate remedial processing steps are pcrformed to improve the
`
`raw image data, to the camera’s framebuffer 1 18.
`
`10
`
`15
`
`20
`
`-
`
`One or more state machines 208 for downloading images from the nonvolatile image
`
`memory 108 to an external device, such as a general purpose computer.
`
`-
`
`One or more state machines 210 for uploading images from an extemal device, such as
`
`a general purpose computer, into the nonvolatile image memory 108. This enables the camera
`
`30
`
`to be used as an image viewing device, and also as a mechanism for transferring image files on
`memory cards.
`
`°
`
`One or more state machines 212 for reducing the size of image files in the nonvolatile
`
`image memory 108. This will be described in more detail next.
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4002
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4002
`
`

`
`wo 01/37209
`
`PCT/US00/30825
`
`-]4-
`
`In the context of the present invention, an image file’s quality level can be reduced in one of
`
`two ways: 1) by deleting from the image file all the analysis arrays associated with one or more
`
`transform layers, or 2) by deleting from the image file one or more bit planes of data.
`
`In either
`
`method, the state machines 212 extract the data structures of the image file that corTespond to
`
`the new, lower image quality level selected by the user. and then replaces the original image
`
`file with one that stores the extracted data structures. Altemately, the original image file is
`
`updated by deleting a portion of its contents, therebyfreeing some of the memory previously
`
`‘ occupied by the file. A feature of the present invention is that the image quality level of an
`
`image file can be lowered without having to reconstruct the image and then re-encode it, which
`
`would be costly in terms of the computational resources used. Rather, the data structures
`
`within the image file are pre-arranged so that the image data in the file does not need to be
`read, analyzed or reconstructed. The image quality level of an image file is lowered simply by
`
`keeping an easily determined subset of the data in the image file and deleting the remainder of
`
`the data in the image file, or equivalently by extracting and storing in a new image file a
`
`determined subset ofthe data in the image file and deleting the original image file._
`
`For the purposes of this document, it should be noted that the term “deleting” when applied to
`a data structure in an image file does not necessarily mean that the information in the data
`structure is replaced with null values. Rather, what this means is that the image file is replaced
`with another image filepthat does not contain the “deleted" data structure. Thus, various data
`
`structures in an image file may be deleted simply by copying all the other data structures in the
`
`image file into a new image file, and updating all required bookkeeping information in the
`
`image file and the image directory for the modified file. The “deleted” data structures may
`
`actually remain in memory unchanged until they are overwritten with new information.
`Altemately, data in an image file may in some implementations be deleted solely by updating
`the bookkeeping information for the file, without moving any ofthe image data.
`
`In one embodiment of the present invention, the digital camera lowers the image quality of an
`
`image fi'om High quality to “Very Good +" by deleting the two lowest bit planes ofthe image.
`Similarly, lowering the image’s quality to “Very Good -" is accomplished by deleting two I
`more bit planes of the image, and then lowering the image’s quality to Good is accomplished
`by deleting yet another two bit planes of the image. More generally, each quality level
`
`10
`
`15
`
`20
`
`25
`
`30
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4003
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4003
`
`

`
`W0 0]/37209
`
`PCT/US00/30825
`
`-15-
`
`transition is represented by deleting a certain percentage of the bit planes of the highest quality
`
`_
`
`level representation of the image.
`
`UI
`
`10
`
`15
`
`20
`
`25
`
`30
`
`In an alternate embodiment, the transition from High quality to the next highest quality level is
`
`accomplished by deleting the analysis arrays for a first transform layer (e.g., the analysis arrays
`
`for the HL1, HH1 and LH1 regions ofthe transfonned image in Fig. 3). Subsequent quality
`
`level transitions to lower quality levels are accomplished by deleting appropriate numbers of
`
`bit planes.
`
`In one embodiment of the present invention the digital camera provides two image file size
`
`reduction modes.
`
`In a first size reduction mode, the user selects one image (or a specified
`
`group of images), uses the camera’s buttons to indicate what lower quality level the image is to
`
`be stored at, and then the state machine 212 generates a smaller image file and stores the
`
`resulting image file in the camera’s nonvolatile image memory 108.
`
`In the second size
`
`reduction mode, the user commands the camera to reduce the size of all image files that are
`
`currently stored at quality level A to quality level B. For instance, in this second size reduction
`
`mode the user might command the camera to convert all “Hi gh" quality image files to “Very
`
`Good +" image quality files. This latter size reduction mode is particularly useful for “clearing
`
`space” in memory 108 to enable additional pictures to be stored in the memory 108.
`
`In another embodiment, the camera or other device may include one or more automatic image
`
`file size reduction modes. For instance, in one such mode the camera could be set to record all
`
`pictures at a particular quality level. When the camera’s memory is sufficiently filed with
`
`images files so that there is insufficient room to store one more image at the current quality
`
`level setting, the camera automatically reduces the size of enough of the stored image files so
`
`as to create room for one more image at the current qualitylevel.
`
`In some embodiments, the
`
`quality level setting of the device for future images might be automatically reduced to match
`
`the quality level of the highest quality image stored in the camera’s memory. In this way, the
`
`camera takes and stores the maximum quality images for the space available, and this
`
`maximization will occur flexibly and “on-the—fly.”
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4004
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4004
`
`

`
`wo 01/37209
`
`PCT/US00/30825
`
`_ 16 -
`
`Camera Operation and
`
`Image File Size Reduction
`
`Referring to Fig. 5, the status of a digital camera is represented by status information displayed
`
`on the camera’s user interface display 1 16. For example, before the camera’s image memory
`
`108 is filled, the camera might indicate to the user thatit is currently storing twenty-one
`
`pictures at High quality and has enough memory to store three more pictures. The indication
`
`ofhow many more pictures can be stored in the camera’s image memory 108 (Fig. 1) depends
`
`on the camera's current picture quality setting, which determines the quality of the next picture
`to be taken.
`
`10
`
`After the camera has stored three more pictures, the camera’s image memory 108 is full (i.e., it
`
`has insufficient room to store another picture at the camera’s current picture quality setting),
`
`and the camera indicates to the user that it is currently storing twenty-four pictures at High
`
`15
`
`quality and has enough memory to store zero more pictures. For the purposes of this example,
`
`we will assume that the user wants to take at least ten more pictures, despite the fact that he/she
`
`has no more memory cards. To make this possible, the user utilizes the image size reduction
`feature of the camera.
`
`In this example, the user commands the camera to reduce all “High“ quality image files down
`
`one quality level to the “Very Good +" quality level. The camera accomplishes this by running
`
`the size reduction state machine 212 and then updating the status information displayed on the
`
`camera’s user interface display 1 16.
`
`In this example, the twenty-four images are now shown to
`
`be stored in image files having the “Very Good -‘r" quality level, and the camera has room for
`
`seven new images at the High quality image level.
`
`In this example. the user next commands the camera to perform a second size reduction so as
`
`to compress all “Very Good +” quality image files down one quality level to the “Very
`
`Good -” quality level. The camera accomplishes this by running the size reduction state
`
`machine 212 and then updating the status information displayed on the camera’s user interface
`
`display 116. In thisexample, the twenty-four images are now shown to be stored in image
`files having the “Very Good -" quality level, and the camera has room for twelve new images
`at the High quality image level.
`
`25
`
`30
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4005
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4005
`
`

`
`WO 01/37209
`
`PCT/US00/30825
`
`_ 17 -
`
`Altemately, if the user had, before capturing the images. switched the quality level for new
`
`images to “Very Good +” quality. a single image size reduction step might have been sufficient
`
`to create room for at least ten additional pictures.
`
`In another example, the digital camera may be configured to have an automatic image file size
`
`reduction mode that is activated only when the camera’s memory is full and the user
`
`nevertheless presses the image capture button on the camera.
`
`In this mode of operation. the
`
`camera’s image processing circuitry reduces the size of previously stored image files as little as
`
`possible so as to make room for an additional image file. Ifthe user continues to take more
`
`pictures in this mode, the quality of the stored images will eventually degradeto some user
`defined or predefined setting for the lowest allowed quality level, at which point the camera
`
`will not store any additional image files until the permitted quality level is lowered further or at
`
`least some of the previously stored image files are transferred to another device or otherwise
`deleted.
`
`The image management system and method of the present invention can also be implemented
`
`in computer systems and computer controlled systems, using a data processor that executes
`
`procedures for carrying of the image processing steps discussed above. The present invention
`
`can also be implemented as a computer program product (e.g., a CD-ROM or data signal
`
`conveyed on a carrier signal) containing image processing procedures suitable for use by a
`computer system.
`
`Video Image Management System
`
`Referring to Fig. 6, there isshown a conceptual data flow diagram for a video image
`
`management system for storing video images at a plurality of image quality levels. The basic
`
`structure ofthe video image management system is the same as shown in Fig. 1. However,
`
`when the camera is a digital video camera, successive images F, are automatically generated at
`
`10
`
`15
`
`20
`
`25
`
`30
`
`a predefined rate, such as eight, sixteen, twenty—four or thirty frames per second. In a preferred
`embodiment, the sequence of video images is processed N frames at a time, where N is an
`
`integer greater than three, and is preferably equal to four, eight or sixteen; generally N will be
`determined by the availability ofmemory and computational resources in the particular system
`in which the invention is being implemented. That is, each set ofN (e.g., sixteen) successive
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4006
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4006
`
`

`
`wo 0]/37209
`
`PCT/US00/30825
`
`-18-
`
`images (i.e., frames) are processed as a set, as follows. For each set of sixteen frames Fm to
`F,,,,,,,, all the frames except the first one are replaced with differential frames. Thus. when
`
`N=l6, fifteen differential frames FM-Fl. are generated. Then, following the data processing
`
`method shown in Fig. 3 and discussed above, the first frame and the fifteen differential frames
`
`are each divided into analysis arrays, a wavelet-like or other transform is applied to the
`
`analysis arrays, and then the resulting transform coefficients are encoded.
`
`In alternate embodiments, other methodologies could be used for initially transforming and
`
`encoding each set of success frames. For instance, a frame by frame decision might be made.
`
`IO
`
`based on a measurement of frame similarity or dissimilarity, as to whether or not to replace the
`
`frame with a differential frame before applying the transform and encoding steps.
`
`In all embodiments, the image file (or files) representing the set of frames is stored so as to
`
`15
`
`In
`facilitate the generation of smaller image files with minimal computational resources.
`particular, the data in the image file(s) is preferably stored using distinct data structures for
`
`each bit plane (see Fig. 4B). Furthennore, as explained above with reference to Fig. 4A. the
`
`analysis arrays may be adjusted prior to the transform step so that the boundaries between
`
`analysis arrays correspond to the boundaries between transfomi layer coefficients. By so
`
`arranging the data stored in the video image files, the generation of smaller, lower quality level
`
`20
`
`video image files is made much easier.
`
`25
`
`30
`
`Continuing to refer to Fig.6, after the video image files for a video frame sequence have been
`
`generated at a particular initial quality level, the user ofthe device (or the device operating in a
`particular automatic mode) may decide to reduce the size ofthe video image files while
`retaining as much image quality as possible. By way of example, in a first video image file
`size reduction step, the HH1 transform coefficients for the last eight frames of each sixteen
`frame sequence are deleted.
`In a second reduction step, the HI-Il transfonn coefficients are
`
`deleted for all frames other than the first frame of each sixteen frame sequence. In a third
`
`reduction step, the Z (e.g., four) least significant bit planes ofthe video image files are deleted.
`In a fourth reduction step, the HLI and LH1 coefficients are deleted for the last eight frames of
`each sixteen frame sequence. In a fifth reduction step, the HLI and LHI coefficients are
`
`deleted for all frames other than the first frame of each sixteen frame sequence. These
`reduction steps are only examples of the type of file size reduction steps that could be
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4007
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4007
`
`

`
`WO 01/37209
`
`PCT/U500/30825
`
`- 19 _
`
`perfomied. For instance, in other embodiments, bit planes might be deleted in earlier reduction
`
`steps and transform layers (or portions of transform layers) deleted only in later reduction
`
`steps. In general, each video image size reduction causes a corresponding decrease in image
`quality.
`
`Referring to Figs. 7, 8 and 9, in another embodiment, video image sequences are compressed
`and encoded by performing a time domain, one dimensional wavelet transfomiation on a set of
`
`video frames. In particular, the video frames are divided into groups of N frames. where N is
`
`an integer greater than 3, and for every x,y pixel position in the video image, a one dimensional
`
`K level wavelet transform is performed on the pixels for a sequence of N+l frames. For
`
`instance, the K level wavelet transform is performed on the 1,1 pixels for the last frame of the
`
`previous group and the current group ofN frames, as well as the 1, 2 pixels, the 1,3 pixels and
`$0 on.
`
`In order to avoid artifacts from the separate encoding of each group of N frames, the frame
`
`immediately preceding the current group is included in K level wavelet transfomi. The
`wavelet transform uses a short transform filter that extends only one position to the left
`(backwards in time) of the position for which a coefficient is being generated and extends in
`the right hand (forward in time) direction only to the right hand edge of the set of N frames.
`
`10
`
`15
`
`20
`
`Furthermore, as shown in Fig. 8, the “right edge" coefficients are saved for each of the first
`
`through K-1 level wavelet transforms for use when processing the next group of N frames.
`a preferred embodiment, only the rightmost edge coefficient is saved for each of the first
`
`In
`
`25
`
`through K-1 level transforms; in other embodiments two or more right edge coefficients may
`be saved and used when processing the next block of N frames. When the second level
`
`30
`
`transform is performed on a block of N frames, the saved layer 1 right edge coefficients for the
`previous set ofN frames are used (i.e., included in the computation ofthe leftmost computed
`coefficient(s) for layer 2). By saving the rightmost edge coefficients for each of the] through
`K-1 layers, artifacts that would caused (during regeneration of the video image sequence) by
`the discontinuities between the last frame of one block and the first frame of the nextiblock are
`avoided,_ resulting in a smoother and more visually pleasing reconstructed video image
`sequence. The wavelet-like transformation and data compression of a video sequence is shown
`in pseudocode fomi in Table 1.
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4008
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4008
`
`

`
`wo 01/37209
`
`PCT/US00/30825
`
`- T
`
`able i
`
`-
`
`Pseudocode for Wavelet-Like Transform and
`Compression of One Block of Video Frames
`
`Repeat for each block of video frames:
`
`{ F
`
`or each row y (of the images)
`
`{ F
`
`or each column x (of the images)
`
`{ S
`
`ave rightmost edge value for use when processing next block of video frames;
`
`Apply level 1 wavelet-like transform to time-ordered sequence of pixel values
`at position x,y. including saved edge value from prior block to generate level 1
`L and H coefficients;
`'
`
`Save rightmost edge L coefficient for use when processing next block of video
`frames;
`
`Apply level 2 wavelet-like transform to level 1 L coefficients for position x,y,
`including saved level 1 edge value from prior block to generate level 2 L and H
`coefficients;
`
`Save rightmost edge level 2 L coefficient for use when processing next block of
`video frames;
`
`Apply level k-l wavelet-liketransfonn to level k-2 L coefficients for position
`x,y, including saved level k-2 edge value from prior block to generate level k—1
`L and_H coefficients;
`'
`
`. Save rightmost edge level k-l L coefficient for use when processing next block
`of video frames;
`
`Apply level k wavelet-like transfonn to level k—1 L coefficients for position x,y,
`including saved level k-1 edge value from prior block to generate level k L and
`H coefficients;
`}
`
`}
`Quantize coefficients
`Encode coefficients
`Store coefficients in image data structure(s), creating image file for current block of video
`frames
`
`}
`
`.
`
`10
`
`.15
`
`20
`
`25
`
`'30
`
`35
`
`40
`
`45
`
`Petitioner Apple Inc. — Exhibit 1024, p; 4009
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4009
`
`

`
`W0 01/37209
`
`PCT/US00/30825
`
`-2]-
`
`A more detailed explanation of saving edge coefficient from one block ofimage data for use
`
`while performing a wavelet or wavelet like transforms on a neighboring block ofimage data is
`
`provided in U.S. patent application serial no. 09/358,876, filed 07-22-99, “Memory Saving
`
`Wavelet-Like Image Transfomi System and Method for Digital Camera And Other Memory
`
`Conservative Applications,” which is hereby incorporated by reference as background
`infomiation.
`
`Once the wavelet-like transform of each block of video data has been completed, all other
`
`aspects of processing the transformed video data are as described above. That is, the
`
`transformed data is quantized, stored in image data structures and subject to reductions in
`image quality, using the same techniques as those applied to still images and video image
`sequences as described above.
`
`The video image management system and method of the present invention can also be
`
`implemented in computer systems and computer controlled systems, using a data processor
`
`that executes procedures for carrying of the video frame processing steps discussed above. The
`
`present invention can also be implemented as a computer program product (e.g., a CD-ROM or
`
`data signal conveyed on a carrier signal) containing image and/or video frame processing
`
`procedures suitable for use by a computer system.
`
`10
`
`15
`
`20
`
`Alternate Embodiments
`
`The state machines of the embodiments described above can be replaced by software
`
`procedures that are executed by a general purpose (programmable) data processor or a
`
`programmable image data processor, especially if speed of operation is not a concern.
`
`Numerous other aspects of the described embodiments may change over time as technology
`improvements are used to upgrade various parts of the digital camera. For instance, the
`
`30
`
`memory technology used to store image files might change from flash memory to another type
`of memory, or a camera might respond to voice commands, enabling the use of fewer buttons.
`
`Referring to Fig. 10, the present invention can also be used in a variety of image processing
`systems other than digital cameras and digital video cameras, including cable television set top
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4010
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4010
`
`

`
`W0 0] /37209
`
`PCT/US00/30825
`
`-22-
`
`boxes, computer systems and devices used to warehouse libraries of images. computer systems
`and devices used to store and distribute image files, and so on. For example. an Internet server
`300 can store images and/or video sequences in the wavelet transform compressed data
`
`structures of the present invention. Copies of those compressed data structures are transferred
`
`to the memory 306 of client computers or other client devices 302, using HTTP or any other
`suitable protocol via the Internet 304 or other communications network. When appropriate. an
`image or video sequence is reduced in size so as to fit in the memory available in the client
`
`computer or other device (client device). Furthermore, once an image or video sequence has
`
`been stored in the memory 306 of a client device, the techniques of the present invention can
`
`be used to manage the storage of the image, for instance through gradual reduction of image
`quality so as to make room for the storage of additional images or video sequences.
`In the
`
`embodiment shown in Fig. 10, the memory 306 ofthe client computer will have stored therein:
`
`-
`
`-
`-
`
`an.operating system 3 l0;
`
`a browser or other image viewer application 312 for viewing documents and images;
`image files 314;
`I
`
`image transform procedures 316, such as wavelet or wavelet—like transform procedures
`-
`for converting a raw image array into wavelet transfonn coefficients, procedures for
`
`compressing and encoding the wavelet transform coefficients, as well as other transfomt
`
`procedures for handling images received in other image formats, such IPEG transform
`
`procedures for converting JPEG files into reconstructed image data that is then used as the raw
`
`— image data by a wavelet or wavelet-like transform procedure;
`-
`an image compression, quality reduction procedure 318 for implementing the image
`data structure size and quality reduction features of the present invention; and
`
`image reconstruction procedures 320 for decompressing and reverse transforming
`-
`image files so as to generate reconstructed image data arrays that are suitable for viewing on
`the monitor of the client workstation, or for printing or otheruse.
`
`The client workstation memory 306 will typically include both high speed random access
`memory and slower non-volatile memory such as a hard disk and/or read—only memory. The
`client workstation’s central processing unit(s) 308 execute operating system procedures and
`image handling procedures, as well as other applications, thereby performing image processing
`functions similar to those performed by dedicated circuitry in other embodiments of the present
`invention.
`
`IO
`
`15
`
`20
`
`25
`
`30
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4011
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4011
`
`

`
`wo 01/37209
`
`PCT/US00/30825
`
`_ 23 _
`
`As indicated above, when the present invention is used in conjunction with. or as part of, a
`
`browser application, for management of image storage, some images may be initially received
`
`in formats other than “raw” image arrays. For instance, some images may be initially received
`
`as JPEG files, or in other proprietary or industry standard formats. To make full use of the
`
`capabilities of the present invention, such images are preferably decoded so as to generate
`
`reconstructed “raw” image arrays, and then those raw image arrays are wavelet or wavelet-like
`
`transformed so as to put the images in a form that enables use of the image quality level
`
`management features of the present invention.
`
`10
`
`While the present invention has been described with reference to a few specific embodiments,
`
`the description is illustrative of the invention and is not to be construed as limiting the
`
`invention. Various modifications may occur to those skilled in the art without departing from
`
`the true spirit and scope of the invention as defined by the appended claims.
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4012
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4012
`
`

`
`W0 01/37209
`
`PCT/U S00/30825
`
`- 24 _
`
`WHAT IS CLAIMED IS:
`
`©\DOO\)O'\LII-bl.»JI\J
`
`ll
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`26
`
`IO
`
`U.)
`
`1.
`
`Image processing apparatus, for use in conjunction with an image capture mechanism.
`
`the image processing apparatus comprising:
`
`a memory device for storing a plurality of image data structures that each represent a
`
`respective image, each image data structure having an associated image quality level
`
`corresponding to a quality level at which the corresponding image has been encoded in the
`
`image data structure; the image quality level of each image data structure being a member of
`
`predefined range of image quality levels that range from a highest quality level to a lowest
`
`quality level and that include at least two distinct quality levels;
`
`image management logic, including data processing circuitry and state machines for
`
`storing and processing image data received from the image capture mechanism, the data
`
`processing circuitry and state machines including:
`
`image processing circuitry for applying a predefined transfonn to image data
`
`received from the image capture mechanism to generate transfonn image data and for applying '
`
`a data compression method to the transform image data so as to generate a new image data
`
`structure having an associated image quality level selected from the predefined range of image
`
`quality levels; the new image data structure being stored in the memory device;
`
`image size reduction circuitry for extracting a subset of the data in a first
`
`specified one of the image data structures stored in the memory device, and fonning a lower
`
`quality version ofthe first specified image data structure that occupies less space in the
`
`memory device than was previously occupied by the first specified image data structure; and
`
`image reconstruction circuitry for successively applying a data decompression
`
`method and an inverse transform to any specified one of the image data structures so as to
`
`generate a reconstructed image suitable for display on an image viewer;
`
`wherein the amount of space occupied by images stored in the form ofimage data
`
`structures in the memory device can be reduced so as to make room for the storage of
`
`additional image data structures in the memory device.
`
`2.
`
`The image processing apparatus of claim 1, wherein
`
`each image data structure contains image transfonn data organized on a bit plane basis
`such that image transform data for at least one bit plane is stored in distinct portions of the
`image data structure from image transform data for other bit planes; and
`
`Petitioner Apple Inc. — Exhibit 1024, p. 4013
`
`Petitioner Apple Inc. - Exhibit 1024, p. 4013
`
`

`
`W0 0|/37209
`
`PCT/US00/30825
`
`- 2 5 -
`
`the image size reduction circuitry and one or more state machines includes logic for
`
`extracting a portion of an image data structure that excludes the image transfonn data for at
`
`least one bit plane and for replacing the image data structure with an image data structure
`
`containing the extracted portion.
`
`3.
`
`The image processing apparatus of