United States Patent (19)
`Narayan et al.
`
`54
`
`(75)
`
`SUB-FRAME DECODER WITH AREA
`DEPENDENT UPDATE RATE FOR DIGITAL
`CAMCORDER TRANSMISSION STANDARD
`
`Inventors: Ajai Narayan, Dallas, Tex.; Manoj
`Aggarwal, Champaign, Ill., Bruce E.
`Flinchbaugh, Dallas, Tex.
`Assignee: Texas Instruments Incorporated,
`Dallas, TeX.
`
`Appl. No.: 08/967,559
`Filed:
`Nov. 12, 1997
`Related U.S. Application Data
`Provisional application No. 60/030,995, Nov. 15, 1996.
`Int. Cl." ................................................ H04N 7/26
`U.S. Cl. ........................... 348/420; 348/390; 348/845
`Field of Search ..................................... 348/420, 421,
`348/390, 384, 845; H04N 7/26
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,222,076 9/1980 Knowlton ................................ 348/384
`5,150,432 9/1992 Ueno ........
`... 348/438
`5,508,743 4/1996 Iizuka ...
`... 348/420
`5,614,952 3/1997 Boyce ...
`... 348/392
`5,907,361
`5/1999 Okada ......
`... 348/399
`5,930,526
`7/1999 Iverson .....
`... 348/385
`5,951,140 8/1990 Ueno ....................................... 348/413
`
`
`
`US006008853A
`Patent Number:
`11
`(45) Date of Patent:
`
`6,008,853
`Dec. 28, 1999
`
`Primary Examiner-Howard Britton
`Attorney, Agent, or Firm-Robert D. Marshall, Jr.; Gerald
`E. Laws; Richard L. Donaldson
`57
`ABSTRACT
`This invention is a method of decoding a Stream of Video
`image data transmitted as independent image frames con
`Sisting of plural marcoblockS transmitted in a nonsequential
`order. The method defines a Sub-frame corresponding to a
`proper subset of the full frame. The method determines if a
`currently received macroblock is within the Sub-frame. The
`method decodes the Sub-frame. The sub-frame may be
`decoded at less than or equal to the frame rate of the Video
`image data. A table has one entry for each macroblock that
`stores a transmission order within the video frame for the
`corresponding macroblock. The method determine if a cur
`rent macroblock is within the Sub-frame by reading the table.
`Each macroblock consists of a plurality of contiguous blockS
`and includes luminance data for any included blocks and
`chrominance data for the macroblock as a whole. The
`method optionally decodes the luminance data for each
`included block and ignores the chrominance data. The
`method decodes the Sub-frame employing only data prior to
`an end of data marker or the end of a data group allocated
`to that block, and ignores data following an end of data
`marker. The method may also decode a full frame of video
`image data at a full frame decode rate less than the Sub
`frame decode rate. The method preferably employs a digital
`camcorder to generate the Stream of Video image data and a
`notebook computer for decoding and display.
`30 Claims, 3 Drawing Sheets
`
`150
`
`165
`
`Petitioners' Exhibit 1005
`Page 0001
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 1 of 3
`
`6,008,853
`
`100
`\
`
`FIC.. 2
`(PRIOR ART)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DETERMINE NEXT MACROBLOCK
`
`
`
`
`
`
`
`101
`
`102
`
`103
`
`104.
`
`105
`
`106
`
`
`
`YES O7
`
`ASSEMBLE DATA BLOCK
`
`
`
`108
`
`TRANSMIT DATA BLOCK
`
`109
`
`110
`
`Petitioners' Exhibit 1005
`Page 0002
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 2 of 3
`
`6,008,853
`
`150
`
`
`
`210
`
`
`
`14
`
`14
`
`213
`Y1
`
`Y
`12
`
`217
`Y13
`
`FIG. 4
`(PRIOR ART)
`14
`BYTES | BYTES -- BYTES -- BYTES - arts
`as
`as
`as
`
`14
`
`Y
`14
`
`10
`
`U
`
`10
`
`arts
`
`W
`
`211
`
`Y21
`
`Y22
`
`Y23
`
`Y24
`
`U2
`
`V2
`
`Y31
`
`Y32
`
`Y33
`
`Y34
`
`U3
`
`V3
`
`Y41
`
`Y42
`
`Y43
`
`Y44
`
`U4
`
`V4
`
`Y51
`
`Y52
`
`Y53
`
`Y54
`
`U5
`
`V5
`
`Petitioners' Exhibit 1005
`Page 0003
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 3 of 3
`
`6,008,853
`
`
`
`FIG. 6
`300
`
`
`
`FIC. 7
`
`150
`
`Petitioners' Exhibit 1005
`Page 0004
`
`

`

`1
`SUB-FRAME DECODER WITH AREA
`DEPENDENT UPDATE RATE FOR DIGITAL
`CAMCORDER TRANSMISSION STANDARD
`
`This application claims priority under 35 USC 119(e)(1)
`of the provisional application No. 60/030,995, filed Nov. 15,
`1996.
`
`TECHNICAL FIELD OF THE INVENTION
`
`The technical field of this invention is encoded video
`image decoding and especially Software only decoding on a
`personal computer.
`
`BACKGROUND OF THE INVENTION
`Digital Video camcorders are now available as commer
`cial products. These digital camcorders produce digital
`Video imageS which may be digitally manipulated and
`displayed. Personal computers are built for manipulating
`and Storing digital data. Current Standards for Video cam
`corder data transmission require Substantial amounts of
`hardware for image decompression when used in conjunc
`tion with a general purpose personal computer. The expense
`of this required hardware limits the number of applications
`where coupling a digital camcorder to a personal computer
`is economically feasible. Therefore there is a need in the art
`for a technique enabling Software only decoding of encoded
`digital images is useful.
`
`15
`
`25
`
`SUMMARY OF THE INVENTION
`This invention is a method of decoding a stream of Video
`image data transmitted as independent image frames con
`sisting of plural marcoblocks transmitted in a nonsequential
`order. The method defines a Sub-frame corresponding to a
`proper Subset of the full frame. The method determines if a
`currently received macroblock is within the sub-frame. The
`method decodes the Sub-frame. The sub-frame may be
`decoded at less than or equal to the frame rate of the Video
`image data.
`In the preferred embodiment the Video image data is
`transmitted in plural macroblocks and the Sub-frame consists
`of whole macroblocks. A table has one entry for each
`macroblock. The table stores a transmission order within the
`Video frame for the corresponding macroblock. The Sub
`frame is defined by Selecting a plurality of contiguous
`macroblocks. The method determine if a current macroblock
`is within the Sub-frame by reading the table to see if the
`currently received macroblock has a transmission order
`entry corresponding to the Sub-frame.
`In the preferred embodiment each macroblock consists of
`a plurality of contiguous blockS. Each macroblock includes
`luminance data for any included blockS and chrominance
`data for the macroblock as a whole. The method optionally
`decodes the luminance data for each included block and
`ignores the chrominance data. Each macroblock includes
`plural data groups. The blocks are represented by data of
`differing lengths. If the data corresponding to a block does
`not fill a data group allocated to that block, the data is
`marked by an end of data marker. The method decodes the
`Sub-frame employing only data prior to and end of data
`marker or an end of the data group, and ignores data
`following an end of data marker. The method may also
`decode a full frame of Video image data at a full frame
`decode rate less than the Sub-frame decode rate.
`The method preferably employs a digital camcorder to
`generate the Stream of Video image data. This is transmitted
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6,008,853
`
`2
`to a personal computer, preferably a notebook computer, for
`decoding and display.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`These and other aspects of this invention are illustrated in
`the drawings, in which:
`FIG. 1 illustrates a camcorder and a notebook computer
`connected in the manner of this invention;
`FIG. 2 illustrates in flow chart form the encoding of video
`data in the prior art SD format;
`FIG. 3 illustrates an example of five Superblocks of a
`Single Video frame from which five macroblocks are Selected
`in the encoding Sequence of the prior art SD format;
`FIG. 4 illustrates the encoding of the five macroblocks
`selected from the five Superblocks illustrated in FIG. 3;
`FIG. 5 illustrates in flow chart form the decoding of video
`data in accordance with this invention;
`FIG. 6 illustrates in flow chart form the steps of the data
`decoding blocks of FIG. 5; and
`FIG. 7 illustrates a video frame showing the areas
`decoded at differing rates.
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`FIG. 1 illustrates the system contemplated in a preferred
`embodiment of this invention. Note that FIG. 1 illustrates a
`notebook computer according to the preferred embodiment,
`but this invention may be practiced with any personal
`computer. A digital camcorder 11 forms a digital image. This
`digital image is compressed according to a digital compres
`Sion Standard known as SD format. This compressed digital
`image is recorded on a digital tape in a digital tape drive
`internal to digital camcorder 11. The compressed digital
`image is also transmitted via cable 12 in an electrical
`interface known as IEEE 1394. Cable 12 connects to an
`IEEE 1394 interface card 14 preferably formed as a PCM
`CIA card in computer 13. This IEEE 1394 serves as only a
`convenient electrical interface and other types of data links
`may be used. In accordance with the preferred embodiment,
`computer 13 employs Software to decode the compressed
`digital image data transmitted via cable 12 from digital
`camcorder 11. Computer 13 may display the decoded digital
`image data on viewing Screen 15. This image display may
`occupy the entire viewing Screen 15 or may occupy only a
`window within viewing screen 15.
`The digital data transmitted by digital camcorder 11 is
`encoded in a format known as SD format. In the SD format
`each video frame is divided into an array of 720 horizontal
`pixels and 480 vertical pixels. This array of 720 by 480
`pixels is grouped into blockS consisting of 8 by 8 arrays of
`pixels. The frame is thus 90 blocks wide in the horizontal
`and 60 blocks high in the vertical. These blocks are grouped
`into macroblockS. Each macroblock consists of four adja
`cent blocks. Most macroblocks consist of four horizontally
`adjacent blockS. Some of the macroblockS consist of four
`blocks disposed in a Square. Each horizontal line of the
`frame encompasses 18 4-by-1 macroblocks 9 2-by-2 mac
`roblocks. Most vertical stripes encompass 60 4-by-1 mac
`roblockS. Some vertical Stripes encompass 20 2-by-2 mac
`roblocks. These macroblocks are in turn grouped into
`Superblocks. Each Superblock includes 27 macroblocks. The
`Superblocks have three different shapes, which mesh to fill
`the frame. There are 50 Superblocks in each frame.
`FIG. 2 illustrates flow chart 100 of the encoding sequence
`according to the prior art SD format. This encoding Serves
`
`Petitioners' Exhibit 1005
`Page 0005
`
`

`

`6,008,853
`
`15
`
`35
`
`40
`
`25
`
`3
`a data compression function by reducing the amount of data
`to be transmitted. This data compression function results in
`a minimal loSS of image quality. The encoding proceSS is
`described in terms of discrete Steps for ease of description.
`Flow chart 100 begins with the video image captured by the
`camera part of camcorder 11 Stored in a frame buffer in a
`known order. This image data is in the form of a data word
`corresponding to each of the 720 by 480 pixels of the image.
`The encoding process first determines which macroblock of
`data is next to be transmitted (block 101). The SD format
`transmits each frame individually without interframe inter
`polation or motion estimation. Thus each transmitted frame
`includes all the data needed to view that frame and any
`Sub-frame. The SD format transmits data of macroblocks in
`a fixed Sequence of nonadjacent macroblocks. A part of this
`sequence is illustrated in FIG. 3. FIG. 3 illustrates video
`image 150. The data corresponding to a first macroblock of
`a first Superblock 151 is transmitted, then data correspond
`ing to a first macroblock in a Second Superblock 152, then
`data corresponding to a first macroblock in a third Super
`block 153, then data corresponding to a first macroblock in
`a fourth Superblock 154, followed by data corresponding to
`a first macroblock in a fifth Superblock 155. This sequence
`then repeats with respect to a Second macroblock of each of
`the first to fifth superblocks. Next the sequence transmits
`data corresponding to a third macroblock of each of the first
`to fifth Superblocks. This continues until data corresponding
`to all 27 macroblocks of the first to fifth superblocks have
`been transmitted. The process then repeats with respect to
`the sixth to tenth Superblocks, the eleventh to fifteenth
`Superblocks and so forth, until data of all the macroblocks of
`all 50 Superblocks have been transmitted. This transmission
`protocol is aimed at providing better quality in noisy trans
`mission channels. Any noise will be more or less evenly
`distributed throughout the frame. Thus noise will not result
`in large degradations of a Small region but a Small degra
`dation of the entire picture. Though the order of transmission
`of macroblockS is nonsequential, it is a fixed order. Thus
`each macroblock has a known macroblock next in the
`transmission Sequence.
`The encoding process next recalls from the frame buffer
`the image data corresponding to the next macroblock (block
`102). This image data is Subjected to a discrete cosine
`transform (block 103). As known in the art, the discrete
`cosine transform converts the data from the Spatial domain
`into the Spatial frequency domain. The discrete cosine
`transform is applied to the Video image in blockS. Next, the
`data is quantized by coding the data into predetermined
`quantized bins (block 104). The quantized data is then
`Huffman encoded (block 105). Huffman encodes most fre
`quently occurring data with shorter data words. LeSS fre
`quently occurring data words are encoded with longer data
`words. Because of this Huffman encoding, the amount of
`data transmitted for Similar sized data blockS vary depending
`on the nature of the image of that block.
`55
`Following Huffman encoding, a macroblock counter MB
`is incremented (block 106). The encoding process deter
`mines whether the macroblock counter MB is greater than 5
`(decision block 107). If this macroblock counter MB is
`greater than 5, then the encoding proceSS assembles a data
`block (block 108). Each data block is as illustrated in FIG.
`4. Data for five macroblockS is transmitted as a group. One
`of these five macroblocks comes from each of the five
`Superblocks as described above. Each macroblock is trans
`mitted in a predetermined data Sequence illustrated in FIG.
`4. There are four 14 byte data groups corresponding to
`luminance data Y from the four blocks of the macroblock.
`
`45
`
`50
`
`60
`
`65
`
`4
`Following the four luminance data groups are two chromi
`nance data groups U and V of 10 bytes each. Each chromi
`nance data group includes data for all four blocks of the
`macroblock. This limitation of the amount of chrominance
`data encoded takes advantage of the known perception
`character that people are leSS Sensitive to changes in chromi
`nance that changes in luminance. This Same Set of four 14
`byte luminance data groups followed by two 10 byte
`chrominance data groups repeats for each macroblock in the
`set of five Superblocks.
`Since the data is Huffman encoded, the data lengths are
`not known. The 14 bytes allocated to each luminance data
`group and the 10 bytes allocated to each chrominance data
`group is not guaranteed to be enough in all cases. Blocks that
`are busy, Such as Stripes or checkerboard boxes, will require
`more data than plain or unfeatured blocks. The SD format
`includes an end of data marker 215 that marks the end of
`data of the corresponding block. Data corresponding to
`Some of the blocks within the set of 5 superblocks will
`underflow the data bytes allocated. In those data blocks
`which underflow their allocated 14 bytes, the end of data is
`marked by an end of data marker 215 equal to “0110”. In
`those data groups where the data overflows the allocated
`data Size, this overflow data is placed in the first data group
`having unused Space. AS illustrated in FIG. 4, luminance
`data Y overflows into area 211 of the 14 bytes allocates to
`luminance data Y. If this data ends before the end of the
`data block, this data end is marked by another end of data
`marker 215. The unused space within the first data group
`having unused Space may be insufficient for the overflow.
`This is illustrated in FIG. 4 where overflow data from
`luminance data Y fills area 213 of luminance data Y. If
`this occurs, then the remaining overflow data is placed in the
`next data group having unused space. In this case the
`remaining data of luminance data Y is placed in area 217
`of luminance data Y. This process is used by all overflow
`ing data blocks. Note that FIG. 4 only illustrates data
`overflow within data 210 allocated to the first macroblock.
`Illustration of shifting overflow data into data 220, 230, 240
`and 250 allocated to the other four macroblocks is omitted
`for clarity of illustration. However, overflow data may be
`placed within unused data Space at any location within data
`210, 220, 230, 240 and 250. If there is no remaining unused
`Space for Overflow data from a particular data group, this
`data is truncated. Such truncation will be unlikely because
`the macroblocks are distributed about the frame. Therefore
`it is unlikely that overflow will occur for all of the macrob
`locks within the same set of five macroblocks. Thus, no
`overflow data is relocated outside the set of 5 macroblocks.
`Note that a particular data group may include no end of data
`marker, one end of data marker or more than one end of data
`marker.
`Following assembly, the data block is transmitted (block
`109). In the preferred embodiment the data is transmitted via
`IEEE 1394 cable 12. The SD format is also applicable to
`data Storage, Such as Storage in the digital tape in the internal
`digital tape drive within digital camcorder 11. Note that FIG.
`2 does not imply any delay between data block transmis
`Sions. It is assumed that data blocks as illustrated in FIG. 4
`are transmitted continuously without interruption. Suitable
`buffering as known in the prior art may be used for this
`purpose. Following data transmission, the macroblock
`counter MB is reset to zero (block 110). This resets the
`process to accumulating data for the next data block as
`illustrated in FIG. 4.
`Regardless of whether the newly encoded macroblock is
`assembled and transmitted, the encoding proceSS returns to
`
`Petitioners' Exhibit 1005
`Page 0006
`
`

`

`S
`block 101. This determines the next macroblock to be
`encoded. Note that frames are continuous, therefore the next
`macroblock to be encoded following completion of a Video
`frame is the first macroblock of the next frame. The process
`of changing to the next frame may require Suitable buffering
`as known in the art.
`The goal of Viewing the picture from digital camcorder on
`a personal computer has a Serious problem. The data pro
`cessing capacity of a personal computer having a micropro
`cessor Such as a Pentium operating at 133 MHz is insuffi
`cient to decode that digital data Stream for the entire frame
`in real time. The personal computer may operate at Some
`reduced frame rate for the entire frame. The data processing
`capability of a 133 MHZ Pentium microprocessor without
`hardware assistance can decode only about one entire frame
`in three seconds. This rate may be too slow to be useful in
`Some applications. The Second Solution is to reduce the
`amount of data to be decoded by limiting the decoding to a
`sub-frame of the video image. The size of the Sub-frame
`determines amount of data to be decoded.
`There is a problem with this second solution. It should be
`immediately apparent that in the SD format data of adjacent
`macroblockS is not transmitted Sequentially. The data for the
`macroblocks corresponding any Sub-frame are disbursed
`throughout the frame data. The SD format does not contem
`plate decoding Sub-frames. Though it may be difficult, Since
`the order of transmission of data corresponding to the
`macroblocks within the frame is fixed, it should be possible
`to identify only the data of macroblocks within an arbitrary
`Sub-frame. Even after Solving this problem, decoding of
`only a Sub-frame may not give the user enough information
`about the complete frame. Based upon the capability of a
`personal computer with a 133 MHZ Pentium
`microprocessor, it is believed possible to decode: a 120 by
`96 pixel Sub-frame at 10 frames per second; and a 180 by
`120 pixel Sub-frame at 1 frame per second. The sub-frame of
`the first option may be too small for the user to judge the
`activity of the Video image. Likewise, the decoded frame
`rate of the Sub-frame of the Second option decoding of the
`Second option may be too slow for the user to judge the
`activity of the Video image.
`A third option is implemented in the preferred embodi
`ment of this invention. A selected Sub-frame of each video
`frame is decoded at a high rate and the remainder of the
`frame is decoded at a low rate. Thus the user may have a
`better understanding of the entire frame while following a
`more frequently changing Sub-frame. This technique is
`advantageous for Video conferencing. In Video conferencing
`applications there is generally a unchanging background and
`a more rapidly changing foreground around the video con
`ferencing user. Proper Selection of the decoding rates and
`Sub-frame dimensions would enable a match to the maxi
`mum decoding rate of the particular personal computer.
`The decoding process of the preferred embodiment is
`illustrated as process 300 in FIG. 5. The decoding process is
`described in terms of discrete Steps for ease of description.
`The proceSS begins with receiving the next transmitted
`macroblock (block 301). The decoding process then incre
`ments a macroblock counter MB (block 302). The decoding
`process then checks to determine if the macroblock counter
`exceeds 1350 (decision block 303). Because the number of
`macroblocks per frame is 1350, this counter serves to detect
`frame boundaries. If the macroblock counter MB exceeds
`1350, then the macroblock counter MB is reset to Zero and
`a frame counter FR is incremented (block 304). The decod
`ing proceSS next tests to determine if the frame counter FR
`exceeds a predetermined number M (decision block 305). In
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6,008,853
`
`6
`accordance with the preferred embodiment a full frame
`decode takes place every M frames. If the frame counter FR
`exceeds M, then the frame counter is reset to Zero (block
`306).
`The decoding process next determines if the frame
`counter FR equals the predetermined number M (decision
`block 307). As noted above, the preferred performs a full
`frame decode every M frames. If this is the case, then the
`image data is decoded (block 308). This process, which is
`essentially the reverse of the encoding process described in
`conjunction with FIG. 2, is illustrated in FIG. 6. After
`starting at start block 321, the data is Huffman decoded
`(block 322). This involves converting the data from the
`coded form in which most used data is encoded as shorter
`data word into their normal format. There are a number of
`options in this process which may be employed to reduce
`data processing. These options will be discussed below.
`Next the data is inverse quantized (block323). This converts
`each value into a central value for the particular quantity bit.
`The data is then Subjected to an inverse discrete cosine
`transform (block 324) which is a known process. Following
`the inverse discrete cosine transform, the decoding process
`is complete at end block 325. The decoded data is then
`Stored in a frame buffer in an appropriate location corre
`sponding to the decoded macroblock (block 309). As noted
`above, the macroblocks are transmitted in a fixed and
`Scrambled order. The current state of the macroblock counter
`MB unambiguously indicates the position of the macroblock
`within the video frame. Following this replacement in the
`Video frame, the decoding proceSS returns to receive the next
`macroblock (block 301).
`If the frame counter FR does not equal the predetermined
`frame count M, then a full frame decode is not performed.
`The decoding proceSS next determines if the macroblock is
`within the designated sub-frame (decision block 310). As
`previously mentioned, the macroblock counter MB unam
`biguously indicates the position of the macroblock within
`the frame. Thus the macroblock counter may be used to
`determine if the current macroblock is within the designated
`sub-frame. In the preferred embodiment the decoding soft
`ware includes a table having one entry corresponding to
`each macroblock. Each entry in this table holds the mac
`roblock counter number of the transmission order of that
`macroblock within the video frame. The Sub-frame is
`defined in terms of included macroblocks. By limiting the
`Sub-frame to macroblock boundaries, this table enables
`determination of whether the currently received macroblock
`is within the sub-frame. As illustrated in FIG. 7, a portion
`160 of the video frame 150 is decoded at a higher rate than
`the remaining portion 160 of the video frame 150. The
`sub-frame 160 may be identified by any convenient means,
`Such as designation of the pixel boundaries. If the macrob
`lock is not within the Sub-frame, the current macroblock is
`ignored, and the decoding process returns to receive the next
`macroblock (block 301). If the macroblock is within the
`sub-frame, then the macroblock data is decoded (block 311).
`This process may be identical to the decoding of block 308
`illustrated in detail in FIG. 6. The decoded data is then stored
`in the frame buffer at a location corresponding to the
`macroblock counter MB (block 312). Following this
`replacement in the Video frame, the decoding process returns
`to receive the next macroblock (block 301). Note that some
`parts of process 300, Such as the image data decoding of
`blocks 308 and 311, must take place almost continuously to
`take advantage of the processing capacity of computer 13.
`Thus computer 13 would not be idle when the incoming data
`is neither within a fully decoded frame nor within the
`
`Petitioners' Exhibit 1005
`Page 0007
`
`

`

`7
`Sub-frame. Those skilled in the art of real time processing
`would understand how to employ interrupts or polling to
`achieve this purpose.
`AS mentioned above, there are various decoding compro
`mises which can be made to reduce the computational
`requirements. Process 300 contemplated that the Sub-frame
`boundaries would be on macroblock boundaries. It is tech
`nically feasible to decode less than all the blocks within a
`macroblock Such as illustrated in FIG. 4. This process would
`involve using the luminance data corresponding to the
`blocks within the sub-frame together with the chrominance
`data U and V for the whole macroblock. Luminance data
`corresponding to other blocks within the macroblock would
`not be used. However, the additional data processing com
`plexity required to decode partial macroblockS may be too
`great to reach the desired decoding rate goals. Accordingly,
`15
`decoding for the Sub-frame may be limited to whole mac
`roblocks.
`It would be advantageous to Save computational resources
`by ignoring the overflow data chaining illustrated in FIG. 4.
`In this case data within each data group following an end of
`data marker 215 would be ignored. Thus Huffman decoding
`would stop up reaching an end of data marker 215 or at the
`end of the data Size allocated (14 bytes for luminance
`components and 10 bytes for chrominance components),
`whichever occurs first. Because the data is encoded in an
`order from the least Spatial frequencies to the greatest Spatial
`frequencies, this would result is Some degradation of the fine
`detail of the image. However, this degradation is believed
`acceptable in this application. Note that ignoring the over
`flow data chaining makes the four data groups 210, 220, 230,
`240 and 250 independent. This permits the decoding process
`to ignore data groups not originally allocated to the desired
`macroblock. Note that this technique may be used for both
`the full frame decoding and for the Sub-frame decoding.
`A further reduction in computational requirements may be
`achieved by ignoring the chrominance components. Using
`only the luminance data groups would result in decoding a
`gray Scale image. This would reduce the amount of data to
`be decoded by about 25%. In some applications this reduc
`tion in data processing requirements in exchange for a gray
`Scale image may be advantageous. AS in the case of ignoring
`the data overflow chaining, this technique may be used for
`both the full frame decoding and for the Sub-frame decoding.
`It is also possible to employ this technique for one portion
`of the frame and not the other. The full frame decoding may
`ignore the chrominance data and produce a gray Scale image
`while the Sub-frame decoding may produce a full color
`image using the corresponding chrominance data.
`The rate of decoding the sub-frame data need not be the
`frame rate of the transmitted video image. AS hinted at
`above, the computational capacity of a personal computer
`having a 133 MHZ Pentium microprocessor may be insuf
`ficient to decode a Sub-frame of useful size at the full frame
`rate. A Sub-frame decode rate of greater than one Sub-frame
`per second, or more preferably between 5 and 10 sub-frames
`per Second, would still be useful in Video conferencing
`applications. A reasonable compromise using this hardware
`would be a full frame every three Seconds or more and a
`sub-frame of 120 by 96 pixels (or 4 by 8 macroblocks) at 5
`to 10 per Second. This slower rate of Sub-frame decoding can
`be enabled by checking the frame counter FR prior to
`detecting whether the current macroblock is within the
`designated Sub-frame. The Sub-frame rate may be Synchro
`nous with the full frame rate or asynchronous with the full
`frame rate.
`A yet additional manner to reduce the computational
`requirements is to omit decoding the full frame even at a low
`
`65
`
`45
`
`50
`
`55
`
`60
`
`6,008,853
`
`25
`
`35
`
`40
`
`8
`rate. This further reduction in the computation requirements
`may enable the personal computer to decode the Sub-frame
`at a higher rate, perhaps up to the full frame rate.
`Alternatively, omitting decoding the full frame may be used
`to enable the personal computer to decode a larger Sub-frame
`than otherwise possible.
`What is claimed is:
`1. A method of decoding a stream of Video image data
`transmitted as independent image frames at a predetermined
`rate, the Stream of Video image data consisting of plural
`marcoblockS transmitted in a nonsequential order, compris
`ing the Steps of:
`defining a Sub-frame corresponding to a proper Subset of
`said full frame;
`determining if a currently received macroblock is within
`Said Sub-frame; and
`decoding Said Sub-frame.
`2. The method of claim 1, wherein:
`Said Step of decoding Said Sub-frame takes place at a
`Sub-frame decode rate equal to Said predetermined rate.
`3. The method of claim 1, wherein:
`Said Step of decoding Said Sub-frame takes place at a
`Sub-frame decode rate less than Said predetermined
`rate.
`4. The method of claim 1, wherein:
`Said Sub-frame consists of whole macroblockS.
`5. The method of claim 4, further comprising:
`forming a table having one entry corresponding to each
`macroblock and Storing within Said each entry a trans
`mission order within the video frame for said corre
`Sponding macroblock,
`Said Step of defining Said Sub-frame includes Selecting a
`plurality of contiguous macroblocks, and
`Said Step of determining if Said currently received mac
`roblock is within Said Sub-frame includes determining
`if Said currently received macroblock has a transmis
`Sion order entry in Said table corresponding to one of
`Said contiguous macroblockS.
`6. The method of claim 1, wherein each macroblock
`consists of a plurality of contiguous blockS and wherein:
`Said Sub-frame consists of whole macroblockS.
`7. The method of claim 6, wherein each macroblock
`includes luminance data for each included block and
`chrominance data for the macroblock as a whole, the method
`wherein:
`Said Step of decoding Said Sub-frame includes decoding
`Said luminance data for each included block of each
`macroblock and ignoring Said chrominance data.
`8. The method of claim 6, wherein each macroblock
`includes plural data groups at least one corresponding to
`each block, Said blockS represented by data of differing
`lengths whereby if the data corresponding to a block does
`not fill a data group allocated to that block the data is marked
`by an end of data marker Separating that data from free data
`Space until the end of the data group and if the data
`corresponding to a block overflows a data group allocated to
`that block the Overflow data is placed in