(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2010/0086025 A1
`(43) Pub. Date:
`Apr. 8, 2010
`Chen et al.
`
`US 2010.0086025A1
`
`(54)
`
`(75)
`
`(73)
`
`(21)
`(22)
`
`(60)
`
`
`
`QUANTIZATION PARAMETER SELECTIONS
`FORENCOOING OF CHROMA AND LUMA
`VIDEO BLOCKS
`
`Publication Classification
`
`(51) Int. Cl.
`H04N 7/26
`
`(2006.01)
`
`Inventors:
`
`Peisong Chen, San Diego, CA
`(US); Marta Karczewicz, San
`Diego, CA (US)
`Correspondence Address:
`QUALCOMMINCORPORATED
`S775 MOREHOUSE DR.
`SAN DIEGO, CA 92121 (US)
`Assignee:
`QUALCOMM Incorporated, San
`Diego, CA (US)
`12/390,325
`
`Appl. No.:
`
`Filed:
`
`Feb. 20, 2009
`Related U.S. Application Data
`Provisional application No. 61/102,622, filed on Oct.
`3, 2008.
`
`........SSEEC9NS)...................
`
`aP
`SELECTION(S)
`
`PREDICTION UNIT
`32
`
`(52) U.S. Cl. .............................. 375/240.03; 375/E07.14
`
`ABSTRACT
`(57)
`This disclosure describes rules that may be applied during
`block-based video coding to ensure that quantization param
`eter selections for luma blocks will not adversely affect the
`quality of chroma blocks. In accordance with this disclosure,
`rate-controlled video encoding occurs in which quantization
`parameter changes in luma blocks are pre-evaluated to deter
`mine whether Such quantization parameter changes in luma
`blocks will also cause quantization changes for chroma
`blocks. If quantization parameter changes in the luma blocks
`will also cause quantization changes for chroma blocks, then
`that quantization parameter change for luma blocks may be
`skipped and not evaluated. In this way, secondary effects of
`quantization parameter changes in the luma blocks (with
`respect to the chroma blocks) can be avoided.
`
`TRANSFORM
`UNIT
`
`OUANTIZATION
`UNIT
`
`UANTIZED
`RESIDUAL
`COEFFICIENTS
`
`MEMORY
`34
`
`INVERSE
`TRANSFORM
`UNIT
`44
`
`INVERSE
`QUANTIZATION
`UNIT
`42
`
`NTROPY
`ENCOOING
`UNIT
`46
`
`RECONSTRUCTE
`WIDEO BLOCK
`
`WIDEO ENCODER
`50
`
`BITSTREAM
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 1
`
`

`

`Patent Application Publication
`
`Apr. 8, 2010 Sheet 1 of 3
`
`US 2010/0086025 A1
`
`
`
`SOURCE DEVICE
`12
`
`VIDEO SOURCE
`20
`
`-1 10
`
`DESTINATION
`DEVICE
`16
`
`DISPLAY DEVICE
`30
`
`VIDEO
`ENCODER
`22
`
`VIDEO
`DECODER
`28
`
`TRANSMITTER
`24
`
`RECEIVER
`26
`
`FIG. 1
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 2
`
`

`

`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 3
`
`

`

`Patent Application Publication
`
`Apr. 8, 2010 Sheet 3 of 3
`
`US 2010/0086025 A1
`
`TEST ORIGINAL
`QUANTIZATION PARAMETER FOR
`RATE-DISTORTION PERFORMANCE
`
`ADJUST THE QUANTIZATION PARAMETER
`
`ADJUSTED OUANTIZATIO
`PARAMETER CAUSE UNDESRABLE
`QUANTIZATION CHANGE FOR CHROMA
`BLOCKS2
`
`NO
`
`YES
`
`301
`
`302
`
`303
`
`TEST ADJUSTED OUANTIZATION
`PARAMETER
`
`SKIP TESTING OF ADJUSTED
`QUANTIZATION PARAMETER
`
`ANOTHER QUANTIZATION
`PARAMETER TO CONSIDER2
`
`NO
`
`ENCODE VIDEO DATA BASED ON ONE OF THE
`TESTED OUANTIZATION PARAMETERS THAT
`ACHIEVES DESIRABLE RATE-DISTORTION
`PERFORMANCE
`
`306
`
`307
`
`
`
`YES
`
`
`
`
`
`FIG. 3
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 4
`
`

`

`US 2010/0O86025 A1
`
`Apr. 8, 2010
`
`QUANTIZATION PARAMETER SELECTIONS
`FORENCOOING OF CHROMA AND LUMA
`VIDEO BLOCKS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`0001. This application claims the benefit of U.S. Provi
`sional Application 61/102,622 filed Oct. 3, 2008, the entire
`content of which is incorporated herein by reference.
`
`TECHNICAL FIELD
`
`0002. This disclosure relates to block-based video encod
`ing used to compress video data, and quantization techniques
`associated with block-based video encoding.
`
`BACKGROUND
`
`0003 Digital video capabilities can be incorporated into a
`wide range of devices, including digital televisions, digital
`direct broadcast systems, wireless communication devices
`Such as radio telephone handsets, wireless broadcast systems,
`personal digital assistants (PDAs), laptop or desktop comput
`ers, digital cameras, digital recording devices, video gaming
`devices, video game consoles, and the like. Digital video
`devices implement video compression techniques, such as
`MPEG-2, MPEG-4, or H.264/MPEG-4, Part 10, Advanced
`Video Coding (AVC), to transmit and receive digital video
`more efficiently. Video compression techniques perform spa
`tial and temporal prediction to reduce or remove redundancy
`inherent in video sequences.
`0004 Block-based video compression techniques may
`perform spatial prediction and/or temporal prediction. Intra
`coding relies on spatial prediction to reduce or remove spatial
`redundancy between video blocks within a given coded unit,
`which may comprise a video frame, a slice of a video frame,
`or the like. In contrast, inter-coding relies on temporal pre
`diction to reduce or remove temporal redundancy between
`Video blocks of Successive coded units of a video sequence.
`For intra-coding, a video encoder performs spatial prediction
`to compress databased on other data within the same coded
`unit. For inter-coding, the video encoder performs motion
`estimation and motion compensation to encode Video infor
`mation based the movement of corresponding video blocks of
`two or more adjacent coded units.
`0005 Video blocks may include luminance (luma) blocks
`and chrominance (chroma) blocks. A 16-by-16 block of pix
`els, for example, may be represented by four 8-by-8 luma
`blocks and two sub-sampled 8-by-8 chroma blocks. Block
`based coding may occur with respect to each of these different
`video blocks. In video coding, the YCbCr color space is
`commonly used, in which Y represents the luma component
`and Cb and Cr represent two different chroma components of
`a block of pixels. Given a 16-by-16 block of pixels, four
`8-by-8Y blocks, one sub-sampled 8-by 8-Cb block, and one
`sub-sampled 8-by-8 Cr block may be used to represent the
`16-by-16 block of pixels, and block based coding may occur
`for each of these video blocks. The term "macroblock” is
`sometimes used to refer to a set of four 8-by 8-Y blocks, one
`sub-sampled 8-by-8 Cb block, and one sub-sampled 8-by-8
`Crblock that collectively define a 16-by-16 block of pixels. In
`Some formats, macroblocks can be partitioned into otherluma
`and chroma block sizes, and may define even finer block
`
`partitions such as 2-by-2 blocks, 2-by-4 blocks, 4-by-2
`blocks, 4-by-4 blocks, 4-by-8 blocks, 8-by-4 blocks, and so
`forth.
`0006. A coded video block may be represented by predic
`tion information that can be used to create or identify a pre
`dictive block, and a residual block of data indicative of dif
`ferences between the block being coded and the predictive
`block. In the case of inter-coding, one or more motion vectors
`are used to identify the predictive block of data (typically
`from a previous or Subsequent video frame of a video
`sequence), while in the case of intra-coding, the prediction
`mode may define how the predictive block is generated based
`on data within the same frame or other coded unit. Both
`intra-coding and inter-coding may define several different
`prediction modes, which may define different block sizes
`and/or prediction techniques used in the coding. Additional
`types of syntax elements may also be included as part of
`encoded video data in order to control or define the coding
`techniques or parameters used in the coding process.
`0007. After block-based prediction, the video encoder
`may apply transform, quantization and entropy coding pro
`cesses to further reduce the bit rate associated with commu
`nication of a residual block. Transform techniques may com
`prise discrete cosine transforms or conceptually similar
`processes, wavelet transforms, integer transforms, or other
`types of transforms. In a discrete cosine transform (DCT)
`process, as an example, the transform process converts a set of
`pixel values into transform coefficients, which may represent
`the energy of the pixel values in the frequency domain. Quan
`tization is applied to the transform coefficients, and generally
`involves a process that limits the number of bits associated
`with any given transform coefficient. Entropy coding com
`prises one or more processes that collectively compress a
`sequence of quantized transform coefficients.
`
`SUMMARY
`0008. In general, this disclosure describes quantization
`parameter selection techniques that may be used during
`block-based video encoding. This disclosure recognizes and
`accounts for secondary effects that quantization parameter
`selections for luma blocks can have on the quantization of
`chroma blocks. This disclosure proposes rules that may be
`applied during block-based video encoding to ensure that
`quantization parameter selections for luma blocks will not
`adversely affect the quality of chroma blocks.
`0009. In one example, this disclosure describes a method
`of encoding video data. The method comprises testing a first
`quantization parameter with respect to a set of video blocks
`for rate-distortion performance in video encoding, wherein
`the set of video blocks include luma blocks and chroma
`blocks, adjusting the first quantization parameter to a second
`quantization parameter, and determining whether the second
`quantization parameter causes an undesirable quantization
`change for the chroma blocks relative to the first quantization
`parameter. The method also comprises testing the second
`quantization parameter with respect to the set of video blocks
`for rate-distortion performance in the video encoding if the
`second quantization parameter does not cause the undesirable
`quantization change for the chroma blocks. In addition, the
`method comprises skipping testing of the second quantization
`parameter with respect to the set of video blocks for rate
`distortion performance in the video encoding if the second
`quantization parameter does cause the undesirable quantiza
`tion change for the chroma blocks. The method also com
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 5
`
`

`

`US 2010/0O86025 A1
`
`Apr. 8, 2010
`
`prises encoding the set of video blocks based on a tested one
`of the quantization parameters.
`0010. In another example, this disclosure provides an
`apparatus that encodes video data, the apparatus comprising a
`Video encoder that tests a first quantization parameter with
`respect to a set of video blocks for rate-distortion perfor
`mance in video encoding, wherein the set of video blocks
`include luma blocks and chroma blocks, adjusts the first
`quantization parameter to a second quantization parameter,
`and determines whether the second quantization parameter
`causes an undesirable quantization change for the chroma
`blocks relative to the first quantization parameter. The video
`encoder tests the second quantization parameter with respect
`to the set of video blocks for rate-distortion performance in
`the video encoding if the second quantization parameter does
`not cause the undesirable quantization change for the chroma
`blocks, skips testing of the second quantization parameter
`with respect to the set of video blocks for rate-distortion
`performance in the video encoding if the second quantization
`parameter does cause the undesirable quantization change for
`the chroma blocks, and encodes the set of video blocks based
`on a tested one of the quantization parameters.
`0011. In another example, this disclosure describes a
`device that encodes video data, the device comprising means
`for testing a first quantization parameter with respect to a set
`of video blocks for rate-distortion performance in video
`encoding, wherein the set of video blocks include luma
`blocks and chroma blocks, means for adjusting the first quan
`tization parameter to a second quantization parameter, means
`for determining whether the second quantization parameter
`causes an undesirable quantization change for the chroma
`blocks relative to the first quantization parameter, means for
`testing the second quantization parameter with respect to the
`set of video blocks for rate-distortion performance in the
`Video encoding if the second quantization parameter does not
`cause the undesirable quantization change for the chroma
`blocks, means for skipping testing of the second quantization
`parameter with respect to the set of video blocks for rate
`distortion performance in the video encoding if the second
`quantization parameter does cause the undesirable quantiza
`tion change for the chroma blocks, and means for encoding
`the set of video blocks based on a tested one of the quantiza
`tion parameters.
`0012. In another example, this disclosure describes a
`device comprising video encoder that tests a first quantization
`parameter with respect to a set of video blocks for rate
`distortion performance in video encoding, wherein the set of
`Video blocks include luma blocks and chroma blocks, adjusts
`the first quantization parameter to a second quantization
`parameter, determines whether the second quantization
`parameter causes an undesirable quantization change for the
`chroma blocks relative to the first quantization parameter,
`tests the second quantization parameter with respect to the set
`of video blocks for rate-distortion performance in the video
`encoding if the second quantization parameter does not cause
`the undesirable quantization change for the chroma blocks,
`skips testing of the second quantization parameter with
`respect to the set of video blocks for rate-distortion perfor
`mance in the video encoding if the second quantization
`parameter does cause the undesirable quantization change for
`the chroma blocks, and encodes the set of video blocks based
`on a tested one of the quantization parameters. The device
`also comprises a wireless transmitter that sends the encoded
`set of video blocks to another device.
`
`0013 The techniques described in this disclosure may be
`implemented in hardware, software, firmware, or any combi
`nation thereof. If implemented inhardware, an apparatus may
`be realized as an integrated circuit, a processor, discrete logic,
`or any combination thereof. If implemented in software, the
`Software may be executed in one or more processors, such as
`a microprocessor, application specific integrated circuit
`(ASIC), field programmable gate array (FPGA), or digital
`signal processor (DSP). The software that executes the tech
`niques may be initially stored in a computer-readable
`medium and loaded and executed in the processor.
`0014. Accordingly, this disclosure also contemplates a
`computer-readable medium comprising instructions that
`upon execution in a video encoding device cause the device to
`encode video data, wherein the instructions cause the device
`to test a first quantization parameter with respect to a set of
`video blocks for rate-distortion performance in video encod
`ing, wherein the set of video blocks include luma blocks and
`chroma blocks, adjust the first quantization parameter to a
`second quantization parameter, determine whether the sec
`ond quantization parameter causes an undesirable quantiza
`tion change for the chroma blocks relative to the first quanti
`Zation parameter, test the second quantization parameter with
`respect to the set of video blocks for rate-distortion perfor
`mance in the video encoding if the second quantization
`parameter does not cause the undesirable quantization change
`for the chroma blocks, skip testing of the second quantization
`parameter with respect to the set of video blocks for rate
`distortion performance in the video encoding if the second
`quantization parameter does cause the undesirable quantiza
`tion change for the chroma blocks, and encode the set of video
`blocks based on a tested one of the quantization parameters.
`0015 The details of one or more aspects of the disclosure
`are set forth in the accompanying drawings and the descrip
`tion below. Other features, objects, and advantages of the
`techniques described in this disclosure will be apparent from
`the description and drawings, and from the claims.
`
`BRIEF DESCRIPTION OF DRAWINGS
`0016 FIG. 1 is an exemplary block diagram illustrating a
`Video encoding and decoding system.
`0017 FIG. 2 is a block diagram illustrating an exemplary
`video encoder consistent with this disclosure.
`0018 FIG. 3 is a flow diagram illustrating an encoding
`technique consistent with this disclosure.
`
`DETAILED DESCRIPTION
`0019. This disclosure describes quantization parameter
`selection techniques that can be used during block-based
`Video encoding. This disclosure recognizes and accounts for
`undesirable secondary effects that quantization parameter
`selections for luma blocks can have on the quantization of
`chroma blocks. Luma block quantization parameters map to
`corresponding chroma block quantization parameters. How
`ever, changes in luma block quantization parameters do not
`necessarily cause changes in chroma block quantization
`parameters. This disclosure proposes rules that may be
`applied during block-based video encoding to ensure that
`quantization parameter selections for luma blocks will not
`adversely affect the quality of chroma blocks.
`0020. The techniques of this disclosure may apply with
`respect to a rate-distortion algorithm that may be executed
`during video encoding. Rate-distortion testing refers to video
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 6
`
`

`

`US 2010/0O86025 A1
`
`Apr. 8, 2010
`
`encoding analysis of coded video blocks in order to define or
`balance the number of bits required for the encoding (i.e., the
`coding “rate”) relative to the quality that is achieved (i.e., the
`level of “distortion”). Rate-distortion algorithms may balance
`these competing goals to achieve a desired balance of rate
`relative to distortion.
`0021 One way to balance rate and distortion is to adjust
`the so-called quantization parameter (sometime abbreviated
`“QP), which is used to quantize data during the video encod
`ing. Higher values for the quantization parameter typically
`result in more quantization, and thus less quality and a lower
`bit rate. In contrast, lower values for the quantization param
`eter typically result in less quantization, and thus higher qual
`ity and a higher bit rate. The manner and direction in which
`quantization parameter changes affect the quantization may
`vary in different standards. In this disclosure, it is assumed
`that higher values of quantization parameters correspond to
`lower levels of quality, although video coding standards or
`techniques could be defined such that higher values of quan
`tization parameters correspond to higher levels of quality. In
`any case, in rate-controlled video coding, a video encoder
`may encode a set of video blocks using a first quantization
`parameter, adjust the quantization parameter to define a sec
`ond quantization parameter, and the re-encode the set of video
`blocks using a second quantization parameter. The encoder
`may then analyze the level of rate and distortion associated
`with the different quantization parameters, and may select the
`quantization parameter that achieves a desirable balance of
`rate and distortion. The selected quantization parameter can
`define the actual encoding for that set of video blocks.
`0022 Sets of video blocks may refer to a set of luma and
`chroma video blocks that define a 16-by-16 area of pixels.
`These sets of video blocks are sometimes referred to as mac
`roblocks. In particular, a macroblock may refer to a set of four
`8-by-8Y blocks, one sub-sampled 8-by-8 Cb block, and one
`sub-sampled 8-by-8 Crblock that collectively define a 16-by
`16 block of pixels. In some formats, macroblocks can be
`partitioned into other luma and chroma block sizes, and may
`define even finer block-partitions such as 2-by-2 blocks,
`2-by-4 blocks, 4-by-2 blocks, 4-by-4 blocks, 4-by-8 blocks,
`8-by-4 blocks, and so forth. The techniques of this disclosure
`may apply with respect to a set of video blocks that collec
`tively define a macroblock comprising a 16-by-16 area of
`pixels.
`0023. Rate-controlled video encoding by selecting and
`adjusting the quantization parameter typically considers luma
`blocks, but may ignore secondary effects on the coding of
`chroma blocks. This is due to the fact that quantization param
`eters for luma blocks typically map to corresponding quanti
`Zation parameters for chroma blocks. Since chroma blocks
`are typically Sub-Sampled and do not have as much signifi
`cance to the encoding quality as luma blocks, quantization
`changes in chroma blocks may be less significant to video
`quality than quantization changes in chroma blocks. For this
`reason, rate-controlled coding often adjusts quantization
`parameters for luma blocks without considering the second
`ary effects that Such changes can have on chroma blocks.
`Indeed, the effects of quantization changes in chroma blocks
`due to changes in quantization parameters for luma blocks
`may be noticeable. In some cases, a quantization change in
`luma blocks often does not result in any quantization change
`for chroma blocks. In other cases, however, a quantization
`change for chroma blocks does result as a consequence of a
`quantization change in luma blocks.
`
`0024. In accordance with this disclosure, rate-controlled
`Video encoding occurs in which quantization parameter
`changes in luma blocks are pre-evaluated to determine
`whether Such quantization parameter changes in luma blocks
`will also cause quantization changes for chroma blocks. If
`quantization parameter changes in the luma blocks will also
`cause undesirable quantization changes for chroma blocks,
`then that quantization parameter change for the set of video
`blocks may be skipped and not evaluated. In this way, sec
`ondary effects of quantization parameter changes in the luma
`blocks (with respect to the chroma blocks) can be avoided.
`Moreover, the need to execute block-based coding with
`respect to a set of blocks in which the quantization parameter
`changes in the luma blocks will also cause quantization
`changes for chroma blocks may be avoided. A quantization
`changes for chroma blocks may be considered undesirable
`anytime the quantization changes for chroma blocks results in
`more quantization, and thus, lower quality in the encoding.
`Any quantization changes that result in less quantization, and
`thus, higher quality in the encoding may be allowed as these
`changes are desirable to improve the video quality.
`0025. There are at least two cases where a change from a
`first luma quantization parameter to a second luma quantiza
`tion parameter will not cause quantization changes for
`chroma blocks. In these two cases, the second luma quanti
`Zation parameter (i.e., the changed quantization parameter)
`can be evaluated for possible use in rate-controlled video
`coding. In other cases, however, where luma quantization
`parameter changes adversely effect quantization of chroma
`blocks, evaluation of the luma quantization parameter
`changes may be skipped during rate-controlled video coding
`so that secondary effects of quantization (i.e., undesirable
`changes in the quantization of chroma blocks) may be
`avoided.
`0026. In the first case where luma quantization parameter
`changes do not affect chroma block quantization, the first
`luma quantization parameter may map to a chroma quantiza
`tion parameter, and the second luma quantization parameter
`may map to that same chroma quantization parameter. In this
`case, no secondary effects on chroma block quality will occur
`due to changes in the luma quantization parameter since the
`chroma quantization parameter does not change, and there
`fore, evaluation of the second luma quantization parameter
`may be allowed to occur during rate-controlled video coding.
`0027. In the second case, chroma blocks may not have any
`significant (non-zero) coefficients. In this second case, any
`upward changes in chroma quantization parameters will be
`irrelevant to chroma block quantization insofar as all coeffi
`cients are already Zero and upward changes in chroma quan
`tization parameters would only cause more quantization.
`Accordingly, in this second case, no secondary effects on
`chroma block quality will occur due to upward changes in the
`luma quantization parameter, and therefore, evaluation of the
`second luma quantization parameter may be allowed during
`rate-controlled video coding.
`0028. It should also be noted that secondary effects in the
`chroma quantization parameter that actually improve chroma
`encoding are not undesirable. Therefore, if the chroma quan
`tization parameter becomes Smaller (e.g., improves) based on
`aluma quantization parameter change, the secondary effects
`may result in coding improvements that are desirable. In this
`case, testing of changed quantization parameters should
`occur even though luma quantization parameter changes
`affect chroma quantization parameters. In other words, the
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 7
`
`

`

`US 2010/0O86025 A1
`
`Apr. 8, 2010
`
`techniques of this disclosure may impose restraints on deg
`radation of chroma quantization parameters, but may allow
`secondary effects that cause improvements of chroma quan
`tization parameters.
`0029 FIG. 1 is a block diagram illustrating an exemplary
`Video encoding and decoding system 10 that may implement
`techniques of this disclosure. As shown in FIG. 1, system 10
`includes a source device 12 that transmits encoded video to a
`destination device 16 via a communication channel 15.
`Source device 12 and destination device 16 may comprise any
`of a wide range of devices. In some cases, source device 12
`and destination device 16 may comprise wireless communi
`cation device handsets, such as so-called cellular or satellite
`radiotelephones. The techniques of this disclosure, however,
`which apply more generally to video encoding, are not nec
`essarily limited to wireless applications or settings, and may
`be applied to non-wireless devices including video encoding
`capabilities.
`0030. In the example of FIG. 1, source device 12 may
`include a video source 20, a video encoder 22, a modulator/
`demodulator (modem) 23 and a transmitter 24. Destination
`device 16 may include a receiver 26, a modem 27, a video
`decoder 28, and a display device 30. In accordance with this
`disclosure, video encoder 22 of source device 12 may be
`configured to perform rate-controlled video encoding accord
`ing to a block-based encoding methodology. Video encoder
`22 may implement the techniques of this disclosure to skip the
`testing or evaluation of coding at one or more luma quantiza
`tion parameters based on a determination that luma quantiza
`tion parameter changes will cause undesirable changes in
`quantization of chroma blocks.
`0031 Video decoder 28 of destination device 16 may be
`configured to perform the reciprocal decoding techniques
`relative to video encoder 22. In any case, the illustrated sys
`tem 10 of FIG. 1 is merely exemplary. Again, the rate-con
`trolled video encoding techniques of this disclosure may be
`performed by any encoding device, and source device 12 is
`merely one example of an encoding device that can imple
`ment such techniques.
`0032 Video encoder 22 of source device 12 may encode
`video data received from video source 20 using the tech
`niques of this disclosure. Video source 20 may comprise a
`Video capture device. Such as a video camera, a video archive
`containing previously captured video, or a video feed from a
`video content provider. As a further alternative, video source
`20 may generate computer graphics-based data as the Source
`video, or a combination of live video, archived video, and
`computer-generated video. In some cases, if video source 20
`is a video camera, Source device 12 and destination device 16
`may form So-called camera phones or video phones. In each
`case, the captured, pre-captured or computer-generated video
`may be encoded by video encoder 22.
`0033. Once the video data is encoded by video encoder 22,
`the encoded video information may then be modulated by
`modem 23 according to a communication standard, e.g., Such
`as code division multiple access (CDMA) or another commu
`nication standard or technique, and transmitted to destination
`device 16 via transmitter 24. Modem 23 may include various
`mixers, filters, amplifiers or other components designed for
`signal modulation. Transmitter 24 may include circuits
`designed for transmitting data, including amplifiers, filters,
`and one or more antenna.
`0034 Receiver 26 of destination device 16 receives infor
`mation over channel 15, and modem 27 demodulates the
`
`information. Video decoder 28 decodes the demodulated
`information and display device 30 displays decoded video
`data to a user. Display device 30 may comprise any of a
`variety of display devices such as a cathode ray tube (CRT), a
`liquid crystal display (LCD), a plasma display, an organic
`light emitting diode (OLED) display, or another type of dis
`play device.
`0035 Communication channel 15 may comprise any
`wireless or wired communication medium, Such as a radio
`frequency (RF) spectrum or one or more physical transmis
`sion lines, or any combination of wireless and wired media.
`Communication channel 15 may form part of a packet-based
`network, Such as a local area network, a wide-area network, or
`a global network Such as the Internet. Communication chan
`nel 15 generally represents any Suitable communication
`medium, or collection of different communication media, for
`transmitting video data from source device 12 to destination
`device 16.
`0036 Video encoder 22 and video decoder 28 may operate
`according to a video compression standard Such as the ITU-T
`H.264 standard, alternatively referred to as MPEG-4, Part 10,
`Advanced Video Coding (AVC). However, the techniques of
`this disclosure may be readily applied to any of a variety of
`other video coding standards. Specifically, any standard that
`allows for rate-controlled, block-based video encoding via
`quantization parameter adjustments may benefit from the
`teachings of this disclosure.
`0037 Although not shown in FIG. 1, in some aspects,
`video encoder 22 and video decoder 28 may each be inte
`grated with an audio encoder and decoder, and may include
`appropriate MUX-DEMUXunits, or other hardware and soft
`ware, to handle encoding of both audio and video in a com
`mon data stream or separate data streams. If applicable,
`MUX-DEMUX units may conform to the ITU H.223 multi
`plexer protocol, or other protocols such as the user datagram
`protocol (UDP).
`0038 Video encoder 22 and video decoder 28 each may be
`implemented as one or more microprocessors, digital signal
`processors (DSPs), application specific integrated circuits
`(ASICs), field programmable gate arrays (FPGAs), discrete
`logic, Software, hardware, firmware or any combinations
`thereof. Each of video encoder 22 and video decoder 28 may
`be included in one or more encoders or decoders, either of
`which may be integrated as part of a combined encoder/
`decoder (CODEC) in a respective mobile device, subscriber
`device, broadcast device, server, or the like.
`0039. In some cases, devices 12, 16 may operate in a
`Substantially symmetrical manner. For example, each of
`devices 12, 16 may include video encoding and decoding
`components. Hence, system 10 may support one-way or two
`way video transmission between video devices 12, 16, e.g.,
`for video streaming, video playback, video broadcasting, or
`video telephony.
`0040. During the encoding process, video encoder 22 may
`execute a number of coding techniques or steps. In general,
`video encoder 22 operates on video blocks within individual
`Video frames (or other independently coded units such as
`slices) in order to encode the video blocks. The video blocks
`may have fixed or varying sizes, and may differin size accord
`ing to a specified coding standard. In some cases, each video
`frame may include a series of independently decodable slices,
`and each slice may include a series of macroblocks, which
`may be arranged into even Smallerblocks. Macroblocks typi
`cally refer to 16-by-16 blocks of data, which may be broken
`
`Unified Patents, LLC v. Elects. & Telecomm. Res. Inst., et al.
`
`Ex. 1036, p. 8
`
`

`

`US 2010/0O86025 A1
`
`Apr. 8, 2010
`
`into a set of luma and chroma blocks. The ITU-T H.264
`standard Supports intra prediction in various block sizes. Such
`as 16-by-16, 8-by-8, or 4-by-4 for luma blocks, and 8-by-8 for
`chroma components, as well as inter prediction in various
`block sizes, such as 16-by-16, 16-by-8, 8-by-16, 8-by-