[BEE ’I'RANSACTlONS ON COMMUNICATIONS: VOL. COMAEQ, NO. 12. DECEMBER 1981
`
`sional transform coding technique has a better performance at
`lower bit rates.
`'
`‘
`REFERENCES
`
`ii; A. ilabihi. "Survey of adaptive image coding techniques." IEEE
`Trans. Common” Vol. COMXZS. pp. [2574284. Nov. P977.
`A. G. Tcschcr and R. V. Cox. “Image coding: Val-able ratc DPlM
`through fixed rate channel.“ in SIPE Prov” vol, 119. San Diego.
`CA, Aug. 1977. pp. 14745.4.
`R. F Rice and l. R. Flaunt. “Adaptive variable-length coding for
`efficmnt compression or spacecraft television data," [EEE Trans.
`Commun. Techno/,. tol. COMA19. pp. 8897897, Dec.
`|97l.
`i
`All Habibi {5'69 M'7CASM'751li’uru photograph and biography. seethis
`issue. 53.
`ll‘2t'.
`
`
`
`BJTSISAMFLE
`
`HYBRID—COSiNE
`\ ZD—COSl RE
`
`38
`36
`34
`PEAK-PEAK 5 IN
`
`Fig. '7. Comparison of adaptive hybrid and 2-D cosine transforms.
`
`t
`
`923%.
`
`Interlield Hybrid Coding 0f Component
`Color Television Signals
`
`I
`
`PMCAPL02442647
`
`lGlTAL transmission of images has gained considerable~
`importance [ii—{20}, [rm-[45] in View of its applica-
`tion to satellite and carrier communication of television
`images
`(picture-phone.
`conference,
`classroom,
`industrial
`and network TViboth monochrome and color). Major re-
`search in this field is focused on bandwi
`(BW) compres-
`sion,
`i.e., removing the redundancy inl
`rent ‘
`an image or
`sequence of images both in space. and time such
`can be transmitted, at reduced bit rates. For n
`color TV digital
`transmission,
`the fidelity requirements are ;
`very rigid [2], [3], [7]—[11], [21}. Yet substantial BW re!
`duction can be achieved by considering the image redundancy
`and psychovisual characteristics of the human vision. Many
`data compression tecluiiques four picturephonc. conference
`TV.
`industrial TV. satellite images from outer space, and
`images from remotely piloted vehicles have been proposed
`and implemented. Also. prototype. commercial systems based
`on predictive or; transform coding have been designedw built,
`and are being marketed [12].
`[22l—i'l4]. Both intmfrzune
`and/or inmrfmme coding applied In color ilnugesin composite
`or component form have heen investigated. liiitinlly the ref
`ilundancy reduction has been attempted by prodirtive coding,
`
`;.
`
`FARHAD A. KAMANGAR. MEMBER.
`
`lEEE. AND K. R. RAG. SEMOR MEMBER.
`
`IEEE
`
`images. bit rate. entropy, essential maximum, and other parameters
`are utilized as the performance criteria. Also. a subjective evaluation
`of the processed images is carried out. It is shown that the interfield
`adaptive hybrid coding of color TV signals in comyonent l‘orm results
`in significant savings in‘ bit rates for transmission over a digital link.
`3
`lNTRODUCTIONJ
`
`l
`
`Abstract—Hybrid coding of color television images at broadcast.
`standards is
`investigated. This involves two-dimensional discrete
`cosine transform [2-D DCT) of each field. followed by the differential
`pulse code modulation (DPCM) between successive fields. Two 22-
`frame sequences of "Water Skier" and “Vthel of Fortune” in
`component l'orm, i.e., luminance Y and chrominance l' and Q. were
`utilized as the database. A statistical study of the prediction error”in
`the 2-D DCT domain is carried out. Three different algorithms for the
`2»D DCT system are simulated and analyzed. In the first algorithm.
`optimum (nonuniformj quantizers are used in DPCM loops followed
`by fixed-wordlength coders. Uniform quantizers along with variable-
`wordlength coders‘ are implemented in the second system. The
`performances of different deterministic coders are investigated and
`compared with Hufl‘man coders. The third scheme is an adaptive
`system. In this system each 8 x it block is divided into four subblocks.
`The activity of each subblock is monitored and when it exceeds some
`threshold, the subblock is considered to be spatially active. More bits
`are assigned to active subblocks and the ranges of corresponding
`quantizers are expanded. The prediction error of the dc coefficient is '
`monitored to determine the temporal activity of a block. For a
`temporally active block, the number of bits assigned to the two lower
`filequency suhhloclrs is increafid. Performance of these three systems
`for unmatched statistics and in the case of a scene change are studied.
`Mean square. error (MSE) between the original and reconstructed
`
`lanuary 13. 1981; revised September 28, 1981.
`Manuscript re::eiverl
`this paper was presented in psut at the National Telecommunications
`Conferences. Washington. DC, 197‘), and Houston, TX. 1980. and is
`based on the research by l‘. A. Kulnnngzlr M '4 partial requirement for
`1h? Phi). degree from the University of 'lexasul Arlington. Arlington.
`TX This work was supportml hy the:(‘ommercialTelcumnmunicatium
`Group. Rockwell lntwnutionul. Dallas. 'I X,
`The authors are With the “(minimum or lilt‘clriuril Engineering,
`Univmnlly ml
`'I'C'flzmitl Arlington. Arlington. TX ’lfifill‘?
`
`()(l‘Nl-b'l'lH/X l I' l AU!) I '74tlflthl.7§
`
`(if) l‘lHl
`
`ll’lil‘,
`
`
`
`PMC3683154
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 1
`
`

`

`KAMANGAR AND RAO: INTERFIELD HYBRID CODING OF COLOR TV SIGNALS
`
`[-741
`
`tion. The block siZe was chosen to be 8 X 8 pels based on
`these considerations.
`
`[rzrerframe/In terfield Processing
`
`2-D DCT/DPCM
`
`‘ c
`
`s
`
`The array of elements resulting from 2-D DCT [26] , [27]
`can be expressed as
`,
`Net N—l
`
`y(u.u,k_)= E 2 x(i,f, k)D(u, u, 13]);
`i=0
`j=0
`
`u, UT 0, l, "3N h i
`
`(1)
`
`where x(r', j, k) and y(u,‘ v, k) represent the (NX N) arrays of
`the kth field in data and DCT domains, respectively. and
`D(Ll, v,
`i. I) is the 2-D DCT kernel which is separable. The
`prediction error for each transform coefficient is de fined as
`
`e(u, u, k) = y(u. v, k) “flu. U. k):
`
`11,1): 0. l. -~~.N — l-
`
`(2),
`
`where for. L’, k), the predicted value ofy(a. v, k). is given by
`
`IN 6001. n, k -
`I'm. U, k) zit". U. k
`u. u w 0. l.
`N ~~ l.
`
`I);
`
`~ I). is trans"
`I), the qurmtized value of r-‘(u, u, A:
`6170!. Lh k
`mitted through the channel after coding. The image is rccorh
`.
`.
`.
`strucled at the l‘t‘t‘ClVCf Side by the lollowinginverse 2D DCT/,1
`
`PMC368315
`
`This was followed later with transform techniques as fast
`algorithms and ‘special processors were developed. Combina-
`tion of the two schemes (prediction and transform) also has
`[been suggested and applied. This combination, called [hybrid
`coding, acts as a compromise of terms of performance, capabil—
`ities. complexity, and limitations [25] .
`
`OBJECTIVE
`
`(2-D
`, The object of this paper is to develop hybrid [25]
`for
`DCT/DPCM) coding of color TV (component
`form)
`digital
`transmission at reduced bit rates. Data compression
`,is based on 2-D DCT [26],
`[27} of each field followed by
`DPCM [28] "between successive fields. Based on the histogram
`of the predictor error in the 2-D DCT domain, quantizers for
`minimizing the mean square quantization error
`(MSQE)
`[30] are developed. The performance of these optimal (non-
`uniform) quantizers with fixed length coders is compared
`with the uniform quantizers coupled with variable length
`coders. Performance criteria such as mean of the absolute
`reconstruction error
`(MARE). absolute maximum r
`‘
`struction error
`(AMRE), bit
`rate, entropy, and e
`/
`maximum [29] are utilized for evaluating these quantiz
`
`.
`
`DATABASE
`
`The database for the hybrid simulation consists of 22
`frame sequences each of “Water Skier” and “Wheel ofFortune"
`in component form ()4, I, Q). i.e., luminance Y plus chromi-
`nance I and Q. The amplitude of the picture elements (pels
`or pixels) has been originally quantized uniformly to six
`bits (64 levels). This is increased to eight bits by adding
`two zeros as the two least significant bits to the six—bit quan-
`tizer. Each of the Y, 1, and Q components is sampled at
`8.064 MHZ with 416 pels/horizontal line and 464 lines!frame
`(visible portion). The database was supplied in this format by
`NASA Ames Research Center (ARC). The bandwidths of Y,
`I, and Q are 4.2 MHz, 1.5 MHz, and 0,5 MHJ, respectively.
`
`HYBRID CODING
`
`In rroducrion
`
`The concept of hybrid coding for video signal is to remove
`the inherent redundancy in the picture by a unitary trans-
`form in one or two dimensions followed by DPCM along the
`other dimension. In this system [25] each picture is divided
`into smaller blocks. Each block is passed through a 2-D trans.
`form and then a‘bank of DPCM loops removes the redundancy
`between the corresponding transform coefficients in the con-
`secutive frames (fields).
`
`[firm/c Size
`/
`
`The choice of the block size is dependent on two factors
`1)/The larger block size dccorrelatcs more samples which
`will
`result
`in higher compression ratio. 2) The larger [thick
`2/176 requires more arithmetic operations for forward and inn
`crsc transforms. It also requires more DPCM loops which will
`/
`.
`.
`.
`(“M result
`in increased complexity in the final hardware renhzru
`
`Because two fields are interlaced to form one frame, the
`question arises if the blocks should be‘forrned by adjacent
`lines in a field or in a frame. The factors that affect this
`choice are as follows. 1) Based on the 4 to 3 aspect ratio and ‘
`the sampling rate of 8.064 MHz, the corresponding pels in
`the adjacent lines of a frame have higher correlation compared
`to the adjacent pels in each line. Also, corresponding pels in
`the adjacent lines of a field have lower correlation than the
`adjacent in a tine. Thus, if the blocks are chosen in each
`frame, rather than in each field, it will result in higher decor—
`relation in the transform domain. 2) The interfield process
`, will
`result
`in more correlation in the temporal direction,
`which results in higher effibiencyrgof'DPCM loops. This com-
`pensates for the lower decorrelation by the forward transfomi.
`3) The interframe ‘process increases the memory requirements
`in two parts of the hybrid system, a) For the transform
`process, since the lines are chosen in each,fraine, the first
`field should be held in a memory before the complete block
`can be formed and processed. This requires one field to be,
`stored in both the transmitter and receiVer, before applying
`the forward and inverse transforms. b) The shift registers used
`in the DPCM loops as the’predictors should be doubled in size
`because, in this case, the differences between the correspond-
`ing DCT coefficients in successive frames are formed and
`quantized in these loops. Based on these considerations and
`simulation of the hybrid system for interfield and interframe
`processes. the interfield process was chosen for further study.
`
`PMCAPL02442648
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 2
`
`

`

`1742
`
`DPCM operations;
`
`IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-22$. NO. 12, DECEMBER 1981
`
`TABLEI
`COVARIANCE MATRIX OF THE ROWS OF Y IN THE DCT
`DQMAIN
`
`1
`.
`
`r
`
`'lABLF n
`4
`VARIANCE MATRICES OF THE PHLLDlCTlON ERRORS IN THE
`DCT DQMAIN Y.
`l} THEORETICAL (’b) EX PERIM ENTAL FOR
`W5 SEQUENCE,
`_
`J EXPERIMENTAL FOR WF SEQUENCE.
`
`g 7::
`3??
`l :5:
`1 so
`‘
`:e:
`3m).
`
`‘
`
`*
`
`r
`a
`1
`
`c
`
`-
`
`a
`2
`
`i
`,
`
`224
`25
`u
`, n
`u
`F5
`rc
`so
`
`21]
`'2:
`I}
`n
`m
`ro
`ID
`m
`
`'2‘:
`r
`It
`n
`m
`to
`to.
`lo.
`
`2n
`25 ,
`is
`n
`m
`1!:
`In
`I:
`
`
`
`PMCAPL02442649
`
`37(u, 0, k) =Eq(u. v, k) +J7(u, vrk '1); -
`u, v=O, 1, 2, "‘,1V*l
`N~1N—r
`'
`\
`5c'(i,i,k)= 2 2 yogukmr’omnj};
`'
`u=0 u=0
`,
`.‘
`~
`r,j=o.,1,~--,Ne1
`
`‘
`
`5‘
`
`where D‘ 1(u, v, i, 1') is the 2-D IDCT kernel. The difference
`x(z‘,]', k) — §(i,f, k) is defined as the reconstruction error.
`
`\
`Statistical Model
`In order
`to fidd the quanitzer characteristics and the ;
`number of quantization levels assigned to the prediction
`‘error of each DCT coefficient, the a prion' knowledge of its
`. statistics is required. To find the statistical parameters, such
`as mean and variance of each prediction error. a statistical
`model
`fd'r
`the origifia‘d picture should be established. It is
`assumed that each 8 X 8 block of pels represents a two-
`' dimensional
`separable wide-sense stationary process. The
`statistics of the pels along horizontal and vertical directions
`are governed by an I—order Markov process. Based on this, the
`variancc’distribution of the prediction error can be easily
`determined
`‘
`‘
`
`" Covariance Matrices
`
`The correlations between the adjacent pels in each line
`and between the corresponding pels in the adjacent lines of
`a field were found to be’ 0.94' and 0.92 for Y= 0.97 and 0.96
`for I, and 0.98 and 0.97 for Q. respectively. The theoretical
`variance matrix (based on an I—order Markov process) for the
`rows of Y in the DCT domain is shown in Table I. The vari-
`ances of DCT coefficients based on analytical and experi-
`mental models are shown in Table II for Y only.
`‘
`In order to find the statistics of the prediction errors, a
`hybrid system with no quantizers in DPCM loops is simulated.
`It is assumed that the presence of the quantizers will not have
`considerable effect on these statistics. The variance matrices
`of prediction errors for 'Y only are shown in Table II. For the
`theoretical model
`(I—order Markov process)
`temporal cor-
`relations were assumed to be 0.95, 0.96, and 0.97 for Y,
`and Q respectively. The variance distributions for 1 and Q are
`similar to those of Y (see Table II).
`
`Quan tiz err
`
`in
`Design of the quantizcrs for DPCM loops involves,
`general, 1) finding the optimum allocation of bits assigned to
`each prediction error for minimum mean square quantization
`error (MSQE), assuming that
`the total number of bits per
`block and the variances are known; and 2) determining the
`decision levels and output
`levels resulting in a minimum
`MSQE for
`a specific probability density function of the
`prediction error
`[30L As the histograms of the prediction
`errors are approximately Laplaciani
`the optimum (nonuni==
`form) quantizers were designed based on the lnplacian density
`function.
`
`Uniform Quan nzers
`In order to achieve lower bit rates= often variable length
`coding is applied to the quantizer outputs. It has been shown
`[311 that for a large number of output levels the entropy of
`the equally spaced (uniform) quantizer is less than or equal to
`the entropy of the optimum (nonuniform) quantizer, assuming
`that both Quantizers have the same MSQE. This means that if
`the entropy coding is used, for a fixed bit rate output, the
`performance of the uniform quantizer in terms of MSQE is
`better than or equal to the nonuniform quantizer. Notice
`that
`the two quantizers being compared do not have the
`same number of output levels.
`In terms of local visual distortions caused by the quan-
`tizcrs.
`the nonuniform quantizcr has less granular noise be-
`cause oi’ its finer structure in the lower rimge. On the other
`hand:
`it, also introduces more noise in the Iludrange input
`levels. Because, in the 2D DCT/DPCM system, Dl’CM loops
`are removing the tcmporal'corrclution, the effect of using :1
`uniform quantizer would he more grrurular noise on those
`parts of the picture which are not changing. To compare the
`performance of
`the nonuniform quantizer with uniform
`
`PMC3683156
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 3
`
`

`

`in.
`
`KAMAl‘GAR AND RAG: INTERFIELD HYBRID RODING OF COLOR TV SIGNALS
`
`v
`
`1743.
`
`qtpintizers, ‘a set of uniform quantizers with optimum spacing
`between the output levels was designed assuming a Laplacian
`distributed input variable.
`~
`
`,
`
`TABLE ur '
`arr ALLOCATION FOR rue Pasorcrrou ERRORS FOR
`OVERALL AVERAGE BIT RATE or 2.9 BlTS/PEL.
`(a) Y, (b)1,(cl Q»
`'
`ll
`
`r
`y
`
`Eff Assignment
`
`Using block coding technique, it is shown that for Lapla-
`cian'distribution, the bit allocation for the prediction errors
`t
`‘in the‘8 X 8 block is governed by [36] , [37?
`i
`M
`r
`1' S
`—
`v
`1
`.
`(mi) —E + i? log; 031.2 «2—523; 0%:
`c]
`
`(5)
`
`2
`
`wruan-hfiw
`
`r-cnowuwwv
`
`00000000
`
`wwwr-wwNN
`
`DDDHh-HHl-d
`
`(a)
`
`o'bacww‘wu
`
`0905:0090,
`
`00000000
`
`ooocuo'oo
`
`OOOCOOOH
`
`0
`g(b)
`
`ooucouao
`
`ODOCQOOO
`
`ooocbOOOI
`
`00000000
`
`ODOOODl—F
`
`oéocoooo
`
`l‘oner—uny
`
`ooooooco
`
`09000000
`
`00000600
`
`lc)
`
`PMCAPL02442650
`
`' Laemmel Coders [32/ of Length N. (LN): In these coders
`the messages are divided into groups of length 2” — Vines—
`sages each. Each message can be identified by its grouphum-
`her, g 2 O, l, m, and the message number‘in its group r 2
`l, 2, ---, 2N — l. For‘each codeword the firsthbits are “0"
`and the lastN bits are binary representations of r.
`Golomb Coders [33/ of Length N, (ON): In these coders,
`the messages are divided into groups of 2(N_'l) messages
`each. Each message can be identified by its group number
`g = O, l, 2,
`and the message number inflits group r : l,
`2,
`2W“), Each code is constructed by.g‘bits of “O”.
`as the most significant bits followed by bit “l.” The next
`(N A 1) bits are binary representations of r'. It should be
`noted that G1 and L] are the same.
`2
`
`where (mkstands for the nearest integer tom, M is the total
`number of bits assigned to the block, S is the number of
`samples in the block, and 091.7" is the variance of the 2th pre-;
`diction Errors Modify (5) such that 23415 «an». 2 M. An
`example of the bit‘ allocation for the prediction error for
`overall average rate of 2 bits/pol (REP) is shown in Table in.
`c
`I %
`
`Coders
`;
`‘
`r
`Variable lpngth coding was used along with the uniform
`quantizersto remove further redundancy from the quantizers’
`outputs. Obviously, the most efficient coder” would be the
`Huffman coder; The, disadvantage of the Huffman coder is
`that it
`is very difficult to implement, especially for a large
`number of input messages-fiend it also needs (2 prior? knowledge
`of the probability distriblfion of the messages. The other
`factor is that the efficiency of the Huffman coders decreases
`for unmatched statistics. For ,thcsc reasons. other classes of
`deterministic coders were implemented and compared with
`the Huffman coders.
`~
`,
`V
`.
`
`PERFORMANCE CRITERIA
`
`The performance of the hybrid coder can be judged by a
`Several criteria, both qualitative and quantitative. Some of
`the latter used to evaluate the system are as follows.
`UMean of the absolute reconstruction error (MARE).
`MARE is defined as
`
`MARiim/illxliJfi) “ilk/2k)”
`
`« (o)
`
`where .Wi‘j, k) is the reconstructed value oft-(ii k)
`
`2)“!an'ance of the reconstruction error (62). For a zero
`mean process the variance of the reconstruction error is
`definedas
`'
`‘
`'
`
`e? =£uxrzp 11 k)'~>‘<ri,r, or Q}.
`
`‘
`
`(7)
`
`This is also the MSE
`
`between the original and reconstructed
`V
`images
`\
`3)Essentiallmam‘mum (EMLIhe EM for A percent is
`defined such that the differences between the intensities of
`the original and reconstructed pels occur within EM, A percent
`of the time, or simply A percént ‘of the reconstruction errors
`i fall in the range- of EMr
`‘
`'
`4) Absolute, maximum reconstruction error
`a
`AMRE for a process is defined as
`
`(AMRE).
`
`"AMRE : ms (rm; f, k) —.)E(z', j, k) r).
`
`(85
`
`Thisflshows the highest valueof the reconstruction error caused
`by overloading dre‘quantizers. AMRE also can be defined as
`a 100 percent EM.
`
`5) Signal-to-noise ratio -in dB (SNR). The -. peak-to-peak
`signrdtonoise ratio (SNR) is defined as
`
`SNR 2 —20 logm <
`
`V.»
`
`g9)
`
`where Vpp is the peak-to-peak signal which for 8 bit PCM
`is equal to 255.
`6) Entropy (12'). The entropy of the quantized prediction
`error with iVmpossible levels is defined as
`r
`
`’
`
`~
`N71
`H = — 23 Mo) log; (pom bits
`i=0
`
`e
`
`(,0)
`
`where [70],) is the probability of the ill: quantization level.
`The ratio of (MARE)2 to E2 is a measure of granular noise
`of the system in comparison to the overload noises For exani~
`ple, between two systems having equal variances of the recon-
`struction error,
`the one with larger MARE introduces more
`granular noise and less overload noise compared to the one
`with smaller MARE. A subjective evaluation of the original
`and reconstructed frame sequences yields
`the qualitative
`performance of the hybrid coder.
`The statistics ot‘tgvie hybrid system for Y: I, and Q of the
`WS sequence are shown in Table IV. The lower AMRE of
`
`PMC3683157
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 4
`
`

`

`-
`
`a
`
`{EBB TRANSACTIQNS ON COMhziiJNICATiONgiVOL. COM-29. NO. 12. DECEMBER 1981
`r»
`%
`TABLE IV
`(a) STATISTICS OF THE. HY BRID SYSTEM WITH NONUNEFDRM QUANTIZERS AND FIXED ITENGTH CODERS
`(Y OF W5 SEQUENCE)
`I
`n
`(b) STATISTICS OF THE HYBRID SYSTEM WITH UNIFORM QUANTZZERS AND FlXED LENGTH CODERS
`~
`
`(Y OF. WS SEQUENCE) '
`
`Average1r
`t
`STIR
`w
`‘l
`Average
`entropy
`‘
`.
`.
`LT
`ft of BB?
`(5??)
`
`.
`
`.033
`$533
`|‘
`.030
`}..r
`l
`.D33
`
`
`|\)pma
`
`U!{Srmtoorwr—ewoInt.
`
`
`Average
`$7 uf SPF
`
`
`
`2.5
`an}
`LS
`La
`9.5
`I 0.0
`May. number at bits assigned in each Prediction m
`Fig. 1.
`Performance of different coders for uniform quantizers.
`
`5-0
`
`0-5
`
`{.0
`
`1.5
`EFF
`
`for nonuniform
`MSQE versus average number of bits/pel
`Fig.
`quantizers and fixed length coders (system "42”) and uniform
`quantizers with G2 coders (system "5”).
`
`the system with nonuniform quantizers [Tame IV(a)] com,
`pared to that for the uniform quantiZers [Table fV(b)], for
`' equal average bit rates,
`is the result of the Wider range of
`the nonuniform quantizers. This wide range causes a faster
`recoverv in temporal direction in the case qf large frarneto
`frame changes caused by a violent motion Or a scene change.
`Fig.
`1 describes the performance of different coders used
`along with uniform quantizers. Since these coders do not
`have a constant output bit rate, a buffer is needed to smooth
`out the output bit rate The average bit rates at the outputs of
`the [437 G3, L4, and G4 coders are most‘of the lime larger than
`the average bit rates at the inputs to these coders [37]. This
`means that the use Vof these coders will decrease the efficiency
`of the system; it can he seen that for the average bit rates
`less: than L375 bits/pelfire Ll coders should be used. Above
`
`this bit rate. G2 coders result in lovver outputbit rate. Because
`the coders
`longer wordlengths have better performance
`for a larger number of input messages, a combination of dif-
`ferent coders can be used for different coefficients to improve
`the average bit rate of the system. Since the G1 and L1 coders
`are the same, we conclude that Go coders have a better
`overall performance than LN coders. Fig. 2 shows a compari-
`son in terms of the MSQE between the hybrid system with
`nonuniform quantizers and fixed length coders
`(system
`“u”) and the system‘wilh uniform quantizcrs and G1 coders
`(system “b")_V it can be seen that system “a” has lower MSQE
`at all differe’nt average bit rates, Sines the nmnher' of bits rus-
`sigucrl
`to the l and Q blocks is small,
`the variable length
`coders were not used for 1 amd Q, The original and recon—
`structed images are shown in Figs. LR.
`
`PMC3683158
`
`
`
`l
`
`‘
`
`orto,..aswMH .rxrpo‘gr«4mm‘tr»
`
`PMCAPL02442651
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 5
`
`

`

`TRANSACTIONS ON COMMUNICATIONS, VOL. CUM-29, N ). 12, DELEMER 1981
`
`Fig 7.
`
`“What: of Fortune." reconszwuted image (usiifontz quamizer,
`1‘5 biISJpeiB.
`
`Fig. 3.
`
`“Water Skier,” 131%de image,
`
`PMCAPL02442652
`
`-
`i591: ‘y shews “he result: 0f tha mmuaatmn of me hybrid
`sysiem for the W3 sequence folicwed by fine WP sequence,
`This was '* p.6memsd to investigate file effect Of a scene
`chang:.'AL£hough the MSE of this pmcess ir. ‘zess Than {hat for
`the WP sequence adamafihs AMRE for ibis case is higher.
`This is the result of the transition be?
`n the two aequences.
`These 13:36 values of reconstruction arm: at
`the transition
`time do not have canshxrableyfkct on the overafl system
`staiifiics, 3.3L:
`introduce a visual dismmon in the first
`field after
`the scene chmge In the subjective evaiuation,
`which was perfmmcd at NASA-ARC faciiities,
`these large
`Values Cf reconstmczim: err-:31 caussd the biock structure to
`be visible in the first iame afier the scene change (Fig‘ 9).
`A150. overloaded qumtizers cause the areas of the préw'ous
`field, mmesponding to large changes in luminance,
`to ‘09
`slightly Visible in the first field afte! Lhe mmsitinrx,
`
`“Water Skier,” reconstmcmd image (uniform
`buts/Pei).
`~
`
`5.
`
`‘Wfleei of Formn
`
`‘
`‘
`~
`"water Skie‘v" recgns‘rPC‘led image (nonuniform quai‘mfiv
`"0 bm’pen'
`
`A dag: five ybn‘a‘ Sys fem
`
`Tire visible distormm caused by 'a scene change or :1 violent
`motion can 73:: reduced by expanding $316 range of the quark
`mzers. But
`this W111 aiso increase me gimmla; noise in the
`unian regions. Another approach to 'this yrobiem is to
`design an adaptive system {23], [34} . E38]? [43] , The adap-
`tive sysiem proposed here is bascd on the deflation of (wo
`kinds of activities in the image ssguence. 1) A Heck with
`ma) much demii is cumidcred to be spatiajly 24mm: An examv
`3310 of such a case is a block which Limlud‘cs an alphanumeric
`dimmer»: or mm of ii. 2)
`:’\ Mack is cgmsidc’md fompmaliy
`am? if
`if flame is a huge (iii'feri‘uce between that Muck :md
`Um cm'rmponding Ewimrk in Lise pwvéuus (Eek? This kéfid 01'
`
`.
`“Wheel of Fonunef" oxfiginal image.
`
`PMC3683159
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 6
`
`

`

`IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. (30.1529. NOA 12, DECEMBER X981
`
`TABLE V
`
`ENGTH CODERS
`
`PMCAPL02442653
`
`, some parts of the picfime.
`. mid black miniming cff’mi'
`ted, the range of the quan-
`" 'hcn this
`actiwiiy is
`tizczs 52101435 be expai'idcd [c :educe the overload noise and to
`imprcve the recovery tirke.
`.
`.
`‘1 ‘
`_' of me blocks teprcscnfing the pre-
`di {ion errors in tilt? 2-D “)CT domain, each: block can be
`divided irate: n subblocks, and each subbinck cam be monitored
`independently. The activity of each subblcck is defined as its
`as energy {sum of its squared elements). The activity sf each
`subblock can be divided into m chases. Fer each c1353, 3
`specific set of bit allocations and noxmeiizazion factors 5‘33 be
`used. This a prior? lnfUmliLi(.\Il i5 availablc in both the Hans-
`miitcr and the receivcr. Only the ciassificaricm of each sub-
`block activity should be sent to the receiver as the overhead
`mfomation, This requires a 10131 of :1 log m bits for each
`block. The performance of tits $1,;sz will improve as r31 and n
`21:6 increased up to some specific values, But this will also
`increase ihe system compiexity and the eveflmad infcimation.
`The maximum value Of f? for an 8 X 8 block is 64. Wham
`each coefficient is monitcred individually, But
`ibis needs m“
`least 64 bits. of overhead ini‘cmnazicn which at the average
`iuii rate of 2.0 bits/pel is 160 peicent of the animal (12:23.
`ink;
`For pnuzlicai cunsidcmiilms, L'Eiilll
`iilwlx is dividiiii
`{our suhblncks with minimum correlation ‘hcmecw the bin: 3
`(i g.
`1(3). This grouping, was arvivmi 9i 351M an mimsiw
`. xizliia‘iual
`study ni‘ slifi‘cimsl gioupx. Nu claim is made here
`
`(3] S'l'A'I'lS'l‘K'S OF THE HYBRED SYSTEM WITH NCNUNIFCRM QUANTIZERS A.\ E] HXED L
`(Y OF W3 FOLEVQWED BY Y OF WP SEQUENCE}
`THE HYBR ID SYSTLM Willi UNIFORM GUAKTIEERQ AND FiXH') LENGTH CODERS
`(b) STA l'iS‘l'iCS
`(Y OF W5 FOLLOWED 3': E’ OF WF SEQUENCE}
`FORM Q1; NTEZERS AND fixau LEix‘sm @DERSA
`(C) STATISTICS OF THE HY BRIE SYSTEM Win-i NOR:
`* AND Q OF WP Saguamg
`(I AN9 (2 Of W5 ’EOLLOWED B
`WW 1
`,
`I Average
`l
`jl
` 4"“,fi.
`H of BB}?
`_
`g
`l
`s
`~12a)r\m
`
`.
`
`l l
`
`PMC3683160
`
`
`
`
`
`'
`
`‘i‘his K the first
`“Wheal of Fortune,” reconstructed image.
`Fig. 9.
`frame of the WI“ sequence, after the W5 sequence has been processed
`by the nonadaptivc system (nonuniform quantizer. 2.0 bits/p ),
`Noticc the colors and block stsucmrc of the WS sequence visible
`this frame_
`‘
`
`activity is caused by a fast motion, camera panning. 01 a
`scene change. Spatial activity in a block causes large abSOlili'C
`valucs for higher tx‘ansform coefficients. in the case of non:
`adaptive systems, EMS will cause a usual distortion because
`the: picdiction mum of these ccefficiem's am usually as»
`signed a small number of hits 0; «Even discarded. This visual
`clismmm; can be avoided by assigning more bits to they?
`prcdiciian cums whenever a spatially aulive lvlnck is Lie=
`{acted The mmpmal activity will rcguii in large values of the
`:Jrcdiciizm mum which,
`in tum, will Gauge overload ilfilféfl
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 7
`
`

`

`,5.
`
`KAMANGAR AND RAD? INTERFIELD HYBRID CODING 0F COLOR TV SIGNALS
`
`Fig. 10, Organization of subblocks iii an 8 X 8 block of prediction
`'
`errors in the 2-D DOT domain.
`
`Il.'03
`
`.0 m
`
`Cwlitlwe
`
`m:anLity
`
`0 0
`
`7
`
`Pmb-hllit
`
`mulltive
`
`2' o
`
`canalof!n {rob-81IMy
`
`annuluin
`
`'rob-b'lHy93 m
`
`G O
`
`l
`
`u my" Hf m “bbzoflz‘goo
`
`.c energy of m. :ubblccl u
`(c)
`
`_
`
`'
`
`'
`0“ mm” :22. “goals”,
`(,1,
`A
`'
`ll, Cumulative probability distributions of the subblocks (see
`Fig. 10).
`
`Fig.
`
`PMCAPL02442654
`
`do emffim'ln?
`
`Fig. ‘12. Cumuiutive probability distribution of the prediction error of
`the dc coefficient.
`
`to indicate that this subblock structure is optimum. The pre-
`diction error of the dc coefficient
`is not included because
`this term determines the average gray level of the block. [t
`is, however, monitored to determine the activity in the tem-
`poral direction.
`The 30 energy of each subblock is monitored independently
`and more bits are asigrred to it if itis considered to be active.
`It will, however, result in a variable output bit rate which re-
`quires a buffer to smooth out the transmission bit rate. The
`threshold of the classification for each subblock is determined
`by the 0.5 point on the probability distribution function for
`the ac energy. Fig. 11 shows these probability distributions
`for the subblocks 1—4. Fig. 12 shows the cumulative probability
`distribution of the prediction error of the dc coefficient. The
`threshold for temporal activity is chosenat the 05 point on
`the probability distribution function. Whenever the prediction
`error of the dc coeflieient exceeds this threshold, the block
`is considered to be temporally active. In this case the normal-
`ization'factors of the quantizers corresponding to the, pre-
`diction eirors in subblocks
`and 2 are increased and more
`bits are assigned to these blocks. The temporal activity over-
`rides the spatial activity for all the subblocks. For a temporally
`active block, no extra bits will be assigned to subblocks 3
`and 4 Given if they are spatially active. This is based on the
`psychovisual effect that,
`immediately after a scene change,
`the eye will, not distinguish a high resolution image from a low
`resolution image [35]. Thus, the resolution can be gradually
`added to the picture in the following fields after the scene
`change with no perceptible distortion.
`._
`Since there is a positive correlation between the activities
`in the luminance and chrominance signals, the prediction error
`- of do coefficient for Y is monitored to determine the activities
`in I and {2 blocks. More bits are assigned to 1 and Q blocks
`whenever they are considered to be active. An example ofbit
`allocation midtrices is 5110wa in Fig. 13- The number 01‘ bits
`assigned to the prediction error of the dc coefficient is 8 hits.
`
`PMC3683161
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 8
`
`

`

`IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-29, NO. 12, DECEMBER 1931
`‘4!
`
`Subhtact H,
`
`temps rel ly‘
`J
`
`r ‘ 1 ’l‘cmporally
`illillltl
`00300
`UU‘JUU
`UDJUU
`JUDHUO
`U (I U I} U U
`jot-osou
`{DOUJUU
`Li##_,,.v:
`Nonactivc
`
`Nancnive
`
`.
`{C}
`
`uh)
`(a) Bit allocation for adaptive algorithm Y. (b) Bit allocate-n
`Figs 13.
`for adaptive algorithm 1.
`(C) Bit allocation for adaptive algorithm
`Q.
`
`PMCAPL02442655
`
`This value was calculated without considering the overhead
`information bits. Four bits of information are needed to be
`sent to the receiver, which at the rate of 2.0 bits/pel is about
`3 percent of the actual data rate. Considering the overhead
`bits, the average bit rate for the bit allocation shown in Fig.
`13 is 2.0 bits/pet Table VI shows the statistical «results for
`the adaptive algorithm simulated for three different average
`bit rates. Examination of Table VI shows that:
`’
`1)»f0r equal bit rates, the MSE of the adaptive system is
`less than that for nonadaptive systems:
`2) the AMRE of the adaptive system is much less than that
`for the nonadaptive systems;
`3) although the MSE of the hdaptive system increases for
`unmatched statistics, it
`is less sensitive to picture statistics
`comparati to the rumadaptivc systems.
`
`as
`
`‘ABPP : [76 +0.75(1o + 10) + 0.25m + 63)
`
`4; 0.5(lé — 8)] (64
`= 19376 bits.
`
`'
`
`Assuming that the temporal and’spatial activities are inde-
`pendent of each other, the theoretical average number of bits/
`pel (ABPP) can be calculated as
`
`4
`
`«9
`
`ABPP = (1; + pIkI + kaQ * E piki>/64
`
`i=1
`
`M
`where b is the total number of bits assigned to Y, I, and Q
`blocks. ki
`is the number of extra bits assigned to subblock
`i. kj and kg are the number of extra bits assigned to I and Q
`blocks if they are active. 11,-, (1,, and pQ are the probabilities
`that extra bits be assigned to the ith subblock and 1 and Q
`blocks.
`respectively, Since the activities of the subblocks
`are considered to be independent, the probabilities of assign-
`ing more bits to the subblocks can be calculated as
`1
`
`p7 = 0.5 + n.5~>< 0.5,
`
`p4
`
`0.5 x 0.5, mans,
`
`‘pQwIl;
`
`plan; p3=p4fi
`
`,‘
`
`Fm the example shown in Fig. 13 the ABPP can be calculated
`
`PMC3683162
`
`PMC Exhibit 2036
`Apple v. PMC
`IPR2016-00755
`Page 9
`
`

`

`
`
`
`