1
`
`
`
`(11) Publication No.: 0 626 653 A1
`
`
`EUROPEAN PATENT APPLICATION
`
`(51)
`
`Int. Cl.5: G06F 15/64
`
`Filing No.: 94410036.1
`
`Filing Date: May 20, 1994
`
`Inventor: Artieri, Alain
`7, Allée des Eyminées
`F-38240 Meylan (FR)
`
`Agent: de Beaumont, Michel
`1bis, rue Champollion
`F-38000 Grenoble (FR)
`
`(72)
`
`
`
`(74)
`
`
`
`
`
`Priority: October 29, 1993 FR 9313293
`
`May 27, 1993 FR 9306612
`
`Publication Date of the Application:
`November 30, 1994 Bulletin 94/48
`
`Designated Contracting States:
`DE FR GB IT
`
`Applicant: SGS-THOMSON
`MICROELECTRONICS S.A.
`7, Avenue Galliéni
`F-94250 Gentilly (FR)
`
`
`
`(12)
`
`
`(21)
`
`(22)
`
`
`(30)
`
`
`(43)
`
`
`(84)
`
`
`(71)
`
`
`
`
`(54) System for processing images.
`
`The present invention relates to a system for
`(57)
`processing compressed image data that arrive in packets,
`these packets being separated by headers including
`parameters. A memory bus (MBUS) is managed by a
`memory controller (24) in order to exchange data
`between processing elements and an image memory
`(15). A pipeline circuit (11, 12, 13) includes several
`processing elements, and a parameter bus (VLDBUS) is
`used to provide packets to be processed to the pipeline
`circuit as well as parameters to elements of the pipeline
`circuit. This parameter bus is managed by a VLD circuit
`that receives compressed data of the memory bus and
`that includes a header detector in order to provide the
`parameters to the elements of the pipeline circuit and to
`other elements of the system that require them.
`
`
`
`
`______________________
`
`Jouve, 18, rue Saint-Denis, 75001 PARIS
`
`
`
`Apple Exhibit 1007
`Page 1 of 85
`
`

`

`EP 0 626 653 A1
`
`2
`
`The present invention relates to systems for processing images, and, in particular, to a
`
`
`
`
`
`
`
`system for decoding an image encoded according to an MPEG standard.
`
`
`
`Figure 1 represents the main elements of an MPEG decoder. Any MPEG decoder, in
`
`particular for standard MPEG 2, generally includes a variable length word decoder 10 (VLD), a
`
`sequence-of-zeros decoder 11 (RLD), an inverse quantization circuit 12 (Q-1), an inverse discrete
`
`cosine transform circuit 13 (DCT-1), a half-pixel filter 14, and a memory 15. The encoded data
`
`are input via a bus CDin, and the decoded data are output via a bus VIDout. Between the input
`
`and the output, the data pass through the processing circuits 10 to 13 in the order indicated
`
`above, which is illustrated by arrows in dotted lines. The output of the decoder is provided by an
`
`adder 16 that sums the outputs of the filter 14 and of the cosine transform circuit 13. The filter 14
`
`needs a previously decoded image portion stored in memory 15.
`
`
`
`Figure 2A illustrates a step of decoding an image portion IM1 in the process of
`
`reconstruction. The decoding of an image is carried out macroblock by macroblock, where a
`
`macroblock generally corresponds to a 16 x 16 pixel image block.
`
`
`
`Figure 2B illustrates a format example, noted 4:2:0, of a macroblock MB. This
`
`macroblock includes a luminance block formed by four blocks Y1 to Y4 of 8 x 8 pixels and a
`
`chrominance block formed by two blocks U and V of 8 x 8 pixels. Another possible format is the
`
`format noted 4:2:2, wherein the chrominance block includes two blocks of 8 x 16 pixels.
`
`
`
`In the current image IM1 of Figure 2A, a current macroblock MBc is decoded, previously
`
`decoded macroblocks being represented by crosshatching. In the general case, the macroblock
`
`MBc is reconstructed from a predictor macroblock MBp obtained in a previously decoded image
`
`Apple Exhibit 1007
`Page 2 of 85
`
`

`

`
`IMO. To find this predictor macroblock MBp, the data used for decoding the macroblock MBc
`
`3
`
`provide a movement compensation vector V that defines the position of the predictor macroblock
`
`MBp with respect to the position P of the macroblock MBc in the image.
`
`
`
`The predictor macroblock MBp is obtained in the memory 15 that stores the previously
`
`decoded image IMO and is provided to the filter 14, while the cosine transform circuit 13
`
`processes data corresponding to the macroblock MBc.
`
`
`
`The decoding that has just been described is a so-called predicted-type decoding. The
`
`decoder macroblock is also referred to as being of the predicted type. According to the MPEG
`
`standards, three main types of macroblocks exist, referred to as "intra," "predicted" and
`
`"bidirectional."
`
`
`
`An intra macroblock corresponds directly to an image block, that is to say that it is not
`
`combined with a predictor macroblock at its output from the cosine transform circuit 13.
`
`
`
`A predicted macroblock, described above, is combined with a macroblock of a previously
`
`decoded image, which, in the order of display, occurs before the image in the process of
`
`reconstruction.
`
`
`
`A bidirectional macroblock is combined with two predictor macroblocks of two
`
`previously decoded images, respectively. These two images are respectively before (forward
`
`image) and subsequent (backward image), respectively, in the display order, with respect to the
`
`image in the process of reconstruction. Thus, the encoded images arrive in an order different
`
`from the display order.
`
`
`
`Moreover, each of the predicted or bidirectional macroblocks is of progressive or
`
`interlaced type. When the macroblock is progressive, the circuit DCT-1 provides the lines of the
`
`macroblock in consecutive order. When the macroblock is interlaced, the circuit DCT-1 first
`
`Apple Exhibit 1007
`Page 3 of 85
`
`

`

`
`provides the even lines of the macroblock, then the odd lines. In addition, a predictor block that
`
`4
`
`serves to decode a predicted or bidirectional macroblock is also of progressive or interlaced type.
`
`When the predictor macroblock is interlaced, it is divided into two half-macroblocks, one
`
`corresponding to even lines and the other to odd lines, each predictor half-macroblock being
`
`obtained at different places of the same previously decoded image.
`
`
`
`An image is also of intra, predicted or bidirectional type. An image of intra type contains
`
`only intra macroblocks, an image of predicted type contains intra or predicted macroblocks, and
`
`an image of bidirectional type contains intra, predicted or bidirectional macroblocks.
`
`
`
`In order to provide the different decoding parameters to the different circuits of the
`
`decoder, in particular the vectors V and the macroblock types, the flow of encoded data includes
`
`headers. There are several headers, namely:
`
`
`
`- an image sequence header that includes, in particular, two quantization tables to be
`
`provided to the inverse quantization circuit 12, one being used for the macroblocks of intra type
`
`of the sequence and the other for macroblocks of the predicted or bidirectional type;
`
`
`
`- an image group header of a sequence, which includes no information used for the
`
`decoding;
`
`
`
`- an image header that includes the type (predicted, intra, bidirectional) of the image and
`
`information on the use of the movement compensation vectors;
`
`
`
`
`
`- an image slice header including information used for correcting errors; and
`
`- a macroblock header including the type of the macroblock, a quantization scale
`
`coefficient to be provided to the inverse quantization circuit 12, and the components of the
`
`movement compensation vectors. Up to four vectors can be used in the case of the processing of
`
`an interlaced bidirectional macroblock.
`
`Apple Exhibit 1007
`Page 4 of 85
`
`

`

`
`
`
`The high hierarchy headers (image, group, sequence) can moreover include private data
`
`5
`
`used, for example, for displaying text at the same time as an image. These private data can also
`
`be used by elements external to the decoder.
`
`
`
`The various processing circuits of an MPEG decoder are often arranged in a series
`
`architecture, commonly referred to as "pipeline" architecture, which makes it possible to process
`
`high data flow rates, but which is very complex and rigid, that is to say that it is difficult to adapt
`
`it to variants of the standards and it is poorly suited for the use of display and private data.
`
`
`
`The simplest and least expensive solution is to connect the various processing circuits to
`
`the memory via a common bus managed by a multi-task processor.
`
`
`
`The European Patent Application 0 503 956 (C Cube) describes such a system, which
`
`includes a processor managing the transfers on the bus and three coprocessors performing the
`
`processing operations corresponding to the circuits 10 to 14. Each different transfer type to be
`
`performed on the bus corresponds to a task performed by the processor. All the tasks are
`
`concurrent and executed upon interruptions of the processor, which are generated by the
`
`coprocessors. The coprocessors exchange the data to be processed via the bus and they receive
`
`instructions provided by the processor via the bus.
`
`
`
`This system is simple, but it has been found that it does not enable the processing of the
`
`currently needed data flow.
`
`
`
`An object of the present invention is to provide a particularly fast image decompression
`
`system that has a relatively simple structure.
`
`
`
`Another object of the present invention is to provide such a decompression system that
`
`can easily be placed in parallel with identical decompression systems for processing particularly
`
`high compressed data flow rates.
`
`Apple Exhibit 1007
`Page 5 of 85
`
`

`

`
`
`
`To achieve these objects, the present invention provides a mixed architecture decoder,
`
`6
`
`that is to say that some of the processing elements are connected to one another and to an image
`
`memory via a first bus and that others are connected in a pipeline architecture. These other
`
`processing elements are referred to hereafter as pipeline circuit. A second bus is provided in
`
`order to provide the data to be processed to the first element of the pipeline circuit, and decoding
`
`parameters to the elements of the system that require such decoding parameters.
`
`
`
`With this structure, the pipeline circuit processes the data in series without any need to
`
`exchange them with the memory via the first bus. In addition, the first bus is relieved of the role
`
`of transmission of the decoding parameters, the latter being transmitted via the second bus. Thus,
`
`the number of exchanges on the first bus corresponding to a given decoding step is substantially
`
`reduced, which makes it possible to increase the performances of the system. The system keeps
`
`great flexibility associated with the use of an exchange system via bus. This flexibility is
`
`increased by an optimal selection of the elements to be included in the pipeline circuit.
`
`
`
`The present invention relates more particularly to a system for processing compressed
`
`data arriving in packets corresponding to image blocks, these packets being separated by headers
`
`containing parameters for decoding the data of the packets, the system including several
`
`processing elements that use said parameters in order to decode the data in packets, and a
`
`memory bus managed by a memory controller in order to exchange data between the processing
`
`elements at rates adapted to the processing speeds of these elements and in order to store in an
`
`image memory data awaiting processing or to be reused. The system includes a pipeline circuit
`
`including several processing elements for processing data packets in series, and a parameter bus
`
`in order to provide packets to be processed to the pipeline circuit, and in order to provide
`
`decoding parameters to elements of the pipeline circuit, this parameter bus being managed by a
`
`Apple Exhibit 1007
`Page 6 of 85
`
`

`

`
`master processing element that receives compressed data from the memory bus and that includes
`
`7
`
`a header detector in order to provide said decoding parameters to the elements of the pipeline
`
`circuit and to other elements of the system that require them.
`
`
`
`According to an embodiment of the present invention, each packet of compressed data is
`
`preceded by a block header, and the packets are collected in successive groups, each group of
`
`packets being preceded by a group header containing parameters for decoding the packets of the
`
`group as well as optionally private and on-screen display information. The system moreover
`
`includes a processor bus managed by a microprocessor in order to supply the group decoding
`
`parameters and the private and display information to the elements of the system that require
`
`them; a buffer memory accessible by the processor bus, which receives the compressed data via
`
`the memory bus; and a group header detector cooperating with this buffer memory in order to
`
`generate an interruption of the microprocessor.
`
`
`
`According to an embodiment of the present invention, a data transfer between two
`
`elements connected to the memory bus corresponds to a specific task that is to be initiated or
`
`continued when one of the two elements issues a request to supply or receive data, all the
`
`possible tasks being concurrent tasks carried out by the memory controller by request priority
`
`management.
`
`
`
`According to an embodiment of the present invention, elements that have to exchange
`
`data with the image memory are connected to the memory bus by respective write or read buffer
`
`memories via the memory bus. A write buffer memory is emptied by the associated element and
`
`issues a request to receive data via the memory bus when its content reaches a lower limit. A
`
`read buffer memory is filled by the associated element and issues a request to supply data on the
`
`memory bus when its content reaches an upper limit.
`
`Apple Exhibit 1007
`Page 7 of 85
`
`

`

`
`
`
`According to an embodiment of the present invention, the system includes: a variable
`
`8
`
`length word decoder (VLD) forming said master processing element; a sequence-of-zeros
`
`decoder (RLD) forming a first element of the pipeline circuit and receiving via the parameter bus
`
`the data processed by the variable length word decoder; an inverse quantization circuit forming a
`
`second element of the pipeline circuit and receiving quantization coefficients via the parameter
`
`bus; an inverse cosine transform circuit forming a third element of the pipeline circuit; the
`
`memory controller receiving movement compensation vectors via the parameter bus; a filter
`
`receiving a parameter of image block type via the parameter bus, this filter issuing distinct
`
`requests, according to the block type parameter, in order to receive corresponding data provided
`
`by the bus memory as a function of the vectors that the memory controller receives; and an adder
`
`in order to provide on the memory bus the sum of the outputs of the filter and of the cosine
`
`transform circuit.
`
`
`
`According to an embodiment of the present invention, the group header detector is
`
`provided in order to generate an interruption of the microprocessor, when the associated buffer
`
`memory contains an image sequence header or an image header, the microprocessor being
`
`provided in order to respond to this interruption by reading in the buffer memory associated with
`
`the group header detector quantization tables that it provides to the inverse quantization circuit,
`
`information of image type and of movement compensation vector amplitude, which it provides to
`
`the variable length word decoder (VLD), and display adjustment information that it provides to a
`
`display controller that receives decoded data via the memory bus.
`
`
`
`According to an embodiment of the present invention, the memory controller includes: an
`
`instruction memory independent of said memory bus, in which the program instructions
`
`corresponding respectively to the transfer tasks on the memory bus are stored; a command
`
`Apple Exhibit 1007
`Page 8 of 85
`
`

`

`
`processing unit (ALU) coupled to the instruction memory in order to receive from it the
`
`successive instructions to be executed, and connected in order to act as a function of these
`
`9
`
`instructions on the memory bus; a plurality of instruction pointers associated respectively with
`
`the possible tasks and each containing the address of the current instruction to be executed of the
`
`associated task, only one of these pointers being capable of being validated at a time in order to
`
`supply its content as instruction address to the instruction memory; a priority level decoder
`
`assigning a predetermined priority level to each request and validating the instruction pointer
`
`associated with the active request of highest priority level; and means for incrementing the
`
`content of the validated instruction pointer and for reinitializing it at the start address of the
`
`associated program when its content reaches the end address of the associated program.
`
`
`
`According to an embodiment of the present invention, each instruction includes a
`
`command field provided to the command processing unit (ALU) and an instruction type field
`
`provided to a prefix decoder, including: means for authorizing a new instruction pointer
`
`validation by the priority level decoder if the type field of the current instruction is at a first
`
`predetermined value, and means for initializing the content of the validated instruction pointer at
`
`the start address of the current program if the type field of the current instruction is at a second
`
`predetermined value.
`
`
`
`According to an embodiment of the present invention, the prefix decoder includes means
`
`for inhibiting the incrementation of the validated instruction pointer if the type field is at a third
`
`predetermined value, so that the current instruction is executed several times successively, the
`
`number of executions being determined by this third value.
`
`
`
`According to an embodiment of the present invention, each instruction includes a
`
`command field that is provided to the command processing unit (ALU), and an acknowledgment
`
`Apple Exhibit 1007
`Page 9 of 85
`
`

`

`10
`
`
`field that is provided to means for activating at least one buffer memory connected to the
`
`memory bus, when the instruction is in the process of being executed.
`
`
`
`According to an embodiment of the present invention, the processing unit (ALU)
`
`includes a plurality of hard wired functions for calculating addresses, each function being
`
`selected by a field of a read or write instruction in the process of being executed.
`
`
`
`According to an embodiment of the present invention, an address register connected to
`
`the memory bus is associated with each hard wired function, the hard wired function being
`
`provided in order to modify the content of its address register appropriately, at each execution of
`
`an instruction in the processing unit (ALU).
`
`
`
`The present invention also provides a system for processing compressed data
`
`corresponding to images, including decoding means that provide decoded image data to an image
`
`memory, these means requiring, in order to decode a current block of an image in the process of
`
`reconstruction, a predictor block of a previously decoded image. In fact, the system includes
`
`multiple decoders associated with respective image memories each storing a specific slice of
`
`blocks corresponding to several images, as well as at least one margin in which a predictor block,
`
`which is used to reconstruct a block of the specific slice, is likely to be found.
`
`
`
`According to an embodiment of the present invention, each decoder in consideration
`
`includes means for storing in its image memory, as a margin, a boundary area of at least one
`
`other specific slice, and for providing to at least one other decoder, as a margin, a boundary area
`
`of the specific slice associated with the decoder in consideration.
`
`
`
`According to an embodiment of the present invention, each decoder in consideration
`
`includes: a first buffer memory receiving image blocks of the specific slice; at least one second
`
`buffer memory receiving image blocks from a boundary area of another specific slice; a final
`
`Apple Exhibit 1007
`Page 10 of 85
`
`

`

`
`processing circuit providing the blocks of the specific slice to the first buffer memory of the
`
`11
`
`decoder in consideration and to the second memory of another decoder; and a memory controller
`
`in order to read the blocks in the first buffer memory and to write them in the image memory at
`
`addresses corresponding to the specific slice, and in order to read the blocks in the second buffer
`
`memory and to write them at addresses corresponding to a margin.
`
`
`
`According to an embodiment of the present invention, each second buffer memory is
`
`preceded by a barrier circuit in order to store in the second buffer memory only the data
`
`corresponding to the desired margin.
`
`
`
`According to an embodiment of the present invention, the images to be processed are
`
`high-definition television images cut into horizontal slices of equal height.
`
`
`
`These objects, characteristics and advantages as well as others of the present invention
`
`will be explained in detail in the following description of particular embodiments made on a non-
`
`limiting basis with the help of the attached figures, among which:
`
`
`
`Figure 1, described above, represents the main elements of an MPEG decompression
`
`system;
`
`
`
`
`
`
`
`Figure 2A illustrates a step of decoding an image macroblock;
`
`Figure 2B represents an example of a constitution of a macroblock;
`
`Figure 3 represents an embodiment of an architecture of a decompression system, or
`
`MPEG decoder, according to the present invention;
`
`
`
`Figure 4 represents a chronogram illustrating the operation of the decompression system
`
`of Figure 3;
`
`
`
`Figure 5 represents an advantageous embodiment of a memory controller according to the
`
`present invention;
`
`Apple Exhibit 1007
`Page 11 of 85
`
`

`

`
`
`
`Figure 6 represents another embodiment of an architecture of a decompression system
`
`12
`
`according to the invention;
`
`
`
`Figure 7 illustrates a high-definition television image that is to be processed in slices by
`
`several parallel decompression systems;
`
`
`
`Figure 8 represents multiple decompression systems connected in parallel in order to
`
`process a high-definition image; and
`
`
`
`Figure 9 partially represents an embodiment of an internal structure of a decoder
`
`according to the invention that makes possible a particularly easy parallel connection.
`
`
`
`General architecture of MPEG decoder
`
`
`
`In Figure 3, the elements already included in Figure 1 are designated by the same
`
`references.
`
`
`
`A bus, hereafter memory bus MBUS, connects the image memory 15 to the compressed
`
`data input bus CDin, to the input of the variable length word decoder (VLD) 10, to the input of
`
`the half-pixel filter 14, and to the input of a display controller 18. The bus CDin, the decoder 10,
`
`and the display controller 18 are connected to the memory bus MBUS by means of respective
`
`buffer memories (FIFO) 20, 21 and 22. The half-pixel filter 14 includes two internal FIFO
`
`memories connected to the memory bus MBUS. The exchanges on the memory bus MBUS are
`
`managed by a memory controller (MCU) 24, which is tasked with carrying out, upon request of
`
`the FIFO memories, transfer operations between these FIFO memories and the image memory.
`
`For this purpose, the memory controller 24 receives a plurality of requests RQ and provides
`
`corresponding acknowledgments ACK. This memory controller can be of the type described in
`
`Apple Exhibit 1007
`Page 12 of 85
`
`

`

`
`European Patent Application 0 503 956 mentioned above. However, a more advantageous
`
`embodiment of this memory controller will be described below.
`
`13
`
`
`
`According to the invention, the sequence-of-zeros decoder (RLD) 11, the inverse
`
`quantization circuit (Q-1) 12, and the inverse discrete cosine transform circuit (DCT-1) 13 are
`
`connected according to a pipeline architecture, that is to say that these circuits 11 to 13 process
`
`the data to be decoded one after the other without these data transiting temporarily in a memory
`
`(15). Together the circuits 11 to 13 are referred to as the pipeline circuit below. The output of the
`
`half-pixel filter 14 is summed at the output of the circuit DCT-1 13 by an adder 16 that is
`
`connected to the bus MBUS by means of a FIFO memory 26 managed by the memory controller
`
`24. Communication management links HS1 and HS2 link the adder 16 to the VLD circuit and to
`
`the circuit DCT-1, respectively.
`
`
`
`According to an aspect of the invention, the VLD circuit 10 manages a bus VLDBUS
`
`used to provide to the RLD circuit 11 data to be processed by the pipeline circuit 11-13, and to
`
`provide parameters to the half-pixel filter 14, to the inverse quantization circuit 12, to the display
`
`controller 18, and to the memory controller 24. A VLD circuit generally decodes the headers of
`
`the compressed data that it receives. As mentioned above, these headers include decoding
`
`parameters to be provided to various elements of the system.
`
`
`
`A macroblock header includes a quantization scale coefficient to be provided to the
`
`inverse quantization circuit 12, a parameter of macroblock type, and the components of
`
`movement compensation vectors. These decoding parameters are decoded by the VLD circuit
`
`and respectively written in dedicated registers of the inverse quantization circuit 12, of the half-
`
`pixel filter 14, and of the memory controller 24.
`
`Apple Exhibit 1007
`Page 13 of 85
`
`

`

`
`
`
`An image header includes, as mentioned above, a parameter of the image type and
`
`14
`
`information on the use of the movement compensation vectors. These parameters are used by the
`
`VLD circuit itself for decoding the vectors and the macroblock data.
`
`
`
`A sequence header includes two quantization tables that the VLD circuit extracts and
`
`provides to two respective registers of the inverse quantization circuit 12. The image headers
`
`include scaling or truncating parameters of the displayed image, which the VLD circuit decodes
`
`and provides to the display decoder 18.
`
`
`
`The VLD circuit executes write operations on the bus VLDBUS as this circuit decodes
`
`the headers. The write operations of the VLD circuit on the bus VLDBUS can be stopped by the
`
`RLD circuit 11 when the latter can no longer receive data to be processed. This is represented by
`
`an HS3 link.
`
`
`
`A sequencer 28 provides a validation signal EN of the VLD circuit. This sequencer
`
`receives display synchronization signals (horizontal, vertical) H/VSYNC through the display
`
`controller 18, a macroblock synchronization signal MBS from the half-pixel filter 14, and an end
`
`of image signal EOP from the VLD circuit 10. The sequencer 28 provides to the memory
`
`controller 24 an image synchronization signal ISYNC that is active when the end of image signal
`
`EOP and the vertical synchronization signal VSYNC are active. The role of the sequencer 28
`
`will be clarified below.
`
`
`
`As indicated above, in order to reconstruct an image, it is often necessary to use image
`
`portions of two previously decoded images. For this purpose, the memory 15 must include three
`
`image areas IM1, IM2 and IM3 for storing the image in the process of reconstruction and two
`
`previously decoded images. The memory 15 includes, in addition, an area CD for storing
`
`compressed data arriving on the bus CDin, while awaiting their processing.
`
`Apple Exhibit 1007
`Page 14 of 85
`
`

`

`
`
`
`15
`
`Management of the areas of the image memory
`
`
`
`To know in which memory areas IM1 to IM3 the memory controller 24 must write, the
`
`latter uses four image pointers ImP+ provided by the VLD circuit. This VLD circuit includes a
`
`unit for the calculation of the image pointers based on parameters of the image type provided by
`
`the image headers. Below, an example of a succession of images and the method for calculating
`
`the image pointers is described.
`
`
`
`
`
`Consider the following succession of compressed images arriving on the bus CDin:
`
`I0, P1, B2, B3, P4, B5, B6
`
`where the letters I, P and B designate an intra image, a predicted image, and a bidirectional
`
`image, respectively. According to the MPEG standards, a bidirectional image cannot be used to
`
`calculate another image. Thus, the reconstruction of the image P1 requires the image I0, the
`
`reconstruction of the images B2 and B3 requires the images I0 and P1, the reconstruction of the
`
`image P4 requires the image P1, and the reconstruction of the images B5 and B6 requires the
`
`images P4 and P1.
`
`
`
`
`
`The order of the display of these images is the following:
`
`I0, B2, B3, P1, B5, P4, B6
`
`since a predicted image P is reconstituted from a previous image in the order of display and since
`
`a bidirectional image B is reconstructed from two images, one subsequent and the other previous
`
`in the display order.
`
`
`
`In order to determine the memory area IM1 to IM3 that the memory controller 24 must
`
`access, four image pointers RP, FP, BP and DP are used, which indicate respectively the
`
`locations of the image in process of reconstruction, of the subsequent image, of the previous
`
`Apple Exhibit 1007
`Page 15 of 85
`
`

`

`
`image, and of the image being displayed. The following table summarizes the values of the
`
`image pointers during the decoding of the images of the above-mentioned succession.
`
`16
`
`
`
`Decoding
`
`Display
`
`RP
`
`FP
`
`BP
`
`DP
`
`
`
`I0
`
`-
`
`IM1
`
`-
`
`-
`
`-
`
`P1
`
`I0
`
`IM2
`
`IM1
`
`-
`
`IM1
`
`B2
`
`B2
`
`IM3
`
`IM1
`
`IM2
`
`IM3
`
`B3
`
`B3
`
`IM3
`
`IM1
`
`IM2
`
`IM3
`
`P4
`
`P1
`
`IM1
`
`IM2
`
`-
`
`IM2
`
`B5
`
`B5
`
`IM3
`
`IM2
`
`IM1
`
`IM3
`
`When the first image I0 is decoded, there is as yet no image to be displayed. The
`
`reconstituted image pointer RP indicates an empty area, for example, the area IM1, for storing
`
`the image I0.
`
`
`
`When the image P1 is decoded, the image I0 is the one that must be displayed. The
`
`reconstructed image point RP indicates, for example, the area IM2, and the displayed image
`
`pointer DP indicates the area IM1 in which the image I0 is located. Since the predicted image P1
`
`uses, in its reconstruction, the image I0, previous in the order of display, the previous image
`
`pointer FP also indicates the area IM1.
`
`
`
`When the bidirectional image B2 is decoded, this image B2 is also the image to be
`
`displayed. The reconstructed image pointer RP and the displayed image pointer DP indicate the
`
`area IM3 that is still free. The image B2, for its decoding, uses the previous image I0 and the
`
`subsequent image P1; the previous and subsequent image pointers FP, BP indicate the areas IM1
`
`and IM2, respectively.
`
`Apple Exhibit 1007
`Page 16 of 85
`
`

`

`
`
`
`To be able to display an image in the process of being decoded, the actual display is
`
`17
`
`generally carried out with a delay of approximately one half of an image; the area IM3 is
`
`sufficiently filled when the image B2 starts to be displayed.
`
`
`
`When the image B3 is decoded, it is also used as image to be displayed. Since the image
`
`B3 also uses for its decoding the images I0 and P1, these images I0 and P1 remain stored in the
`
`areas IM1 and IM2 that are still indicated by the previous image pointer FP and the subsequent
`
`image pointer BP. The image B3 can be stored only in the area IM3 indicated by the
`
`reconstructed image pointer RP and the displayed image pointer DP.
`
`
`
`However, at the time when the reconstruction of the image B3 is started in the area IM3,
`
`the image B2, also stored in the area IM3, is in the process of being displayed. If the image in the
`
`process of being displayed B2 risks being overwritten by the image in the process of
`
`reconstruction B3, the VLD circuit that is providing the image data B3 is stopped. The role of the
`
`sequencer 28 mentioned above is also precisely to stop the VLD circuit by deactivating the
`
`validation signal EN when the number of decoded macroblocks risks corresponding to an image
`
`fraction larger than the displayed image fraction. The size of this fraction is determined by
`
`counting the number of horizontal synchronization pulses HSYNC, and the number of
`
`macroblocks decoded is determined by counting the macroblock synchronization pulses MBS.
`
`
`
`When the image P4 is being decoded, the image P1 must be displayed. The image P4 is
`
`stored in the area IM1 that is now free; the reconstructed image pointer RP indicates the area
`
`IM1. The displayed image pointer DP indicates the area IM2 where the image P1 is stored. The
`
`image P4, for its decoding, uses the previous image P1; the previous image pointer FP indicates
`
`the area IM2.
`
`Apple Exhibit 1007
`Page 17 of 85
`
`

`

`When the image B5 is decoded, it must also be displayed. The image B5 is stored in the
`
`18
`
`
`
`
`area IM3 that is freed; the reconstructed image pointer RP and the displayed ima