Quality Assessment of
`Digital Television Signals
`
`By LEONARD S. GOLDING
`
`Procedures have existed for many years permitting the subjective evaluation of conventional
`[analog] television pictures. Subjective data has been amassed and objective test signals have
`been developed that can be used to predict with some certainty the quality of picture that
`will be presented to the viewer. It is noted that the digital case is substantially more complex
`than the analog case, but it is important and worthwhile to derive similar procedures for
`subjectively evaluating digitally encoded television pictures. A way is suggested for beginning
`such an effort. and it is foreseen that objective test signals could be developed for assuring
`a given subjectively evaluated quality level in the presence of various digital impairments.
`in this way. digitized television pictures could he validly compared with each other and with
`analog pictures. Topics discussed include: the various subjective grading scales; source coding
`and channel effects; PCM. DPCM and transform encoding; interframe processing: and
`encoding of composite and component video signals.
`
`pairment to the television signal that has
`been created. This is the type of test used
`by Batstow and Christopher. Second, there
`is a quality test where the observer is asked
`to rate the overall quality of the picture:
`and third, there is a comparison-type test
`where the observer is asked to compare the
`quality of a given picture against
`the
`quality of another picture. All three types
`ofsubjcctive test have been used in evalu-
`ating analog television signal quality and
`each has its own grading scale and test
`procedure. Table 1
`lists typical grading
`Scales that have been used for each type of
`subjective test. Table [l
`lists common
`subjective test procedures which have been
`followed by various countries such as the
`USA. and the United Kingdom and by
`several international organizations. The
`subjective test procedures must consider
`the number of observers, the type of grad~
`ing scale used, the viewing conditions and
`the type of picture material used in the test.
`These are all referred to in Table III. After
`a number of years, there has been agree-
`ment within the CC] R as to a recom-
`mended subjective testing procedure for
`testing television signals. Table III lists the
`recommended subjective testing procedure
`now internationally accepted.
`The importance of subjective testing is
`that the subjective grading scale (such as
`the impairment scale — which has grades
`of
`imperceptible, perceptible but not
`annoying, slightly annoying. annoying and
`very annoying fl as given in Table IV) is
`a universal scale which allows one to com-
`pare different kinds of impairment in the
`television picture and therefore allows one
`to compare one television system with an-
`other and one type of signal processing
`method with another. So the subjective test
`provides a universal scale that can be used
`to measure all different kinds of systems
`and compare them with each other. The
`subjective testing scale also is directly re-
`lated to picture quality as seen by the ob-
`servcr, and so permits one to easily define
`a broadcast-quality signal. In the case of
`commercial broadcast service. where the
`ultimate objective is to present a pleasing
`and high-quality picture to the observer,
`the subjective scale allows that picture
`quality to be evaluated directly.
`Let us consider the impairment test in
`greater detail. In the impairment test, one
`adds diffcrcnt amounts of an impairment
`such as noise to the original signal and dc-
`termines how the observer evaluates the
`visibility of the impairment as a function
`of the amount added to the televiSion sig-
`Three Kinds of Subjective Test
`nal. Typically, one considers a single pa-
`rameter such as the amount of noise or the
`In the area of subjective testing, there
`power of the noise and relates that to a
`are three main types of subjective test. The
`judgment on the subjective grading scale
`first is an impairment test where the ob-
`made by the observer as given in Table III.
`server is asked to judge the degree of im-
`Volume 3? March i978 SMPTE Journal
`l53
`
`come to a point where we can specify
`quantitatively the parameters that define
`quality. We would like to reach a similar
`objective in the digital case. Let us examine
`how these parameters were derived and
`how we reached the present objectives for
`the analog case.
`In the early days of television, there were
`many laboratories such as Bell Telephone
`Laboratories, R.C.A. Laboratories and
`others around the world that carried out
`subjective tests to evaluate such parameters
`as signal-to-noise ratio and differential
`phase. They determined just how these
`parameters related to a given subjective
`quality such as just-perceptible distortion
`or annoying distortion.
`In other words,
`quantitative para meters were rotated to a
`subjective measure of performance. Once
`the subjective measures were determined,
`a second phase took place where objective
`test patterns were developed which could
`measure these quantitative parameters by
`means ofa vectorscope, waveform monitor.
`oscilloscope or some othér piece of test
`equipment and relate them directly to
`picture quality based on the subjective test
`results. The use of objective test patterns
`eliminated the need for carrying out sub-
`jective testing every time one wished to
`evaluate the quality ofa particular televi-
`sion system. In the development olthe ob-
`jective test patterns.
`the quantitative
`measures needed for each impairment.
`such as noise, were originally based on the
`subjective test results that had been ob-
`taincd. As an example ofthis, Fig. 1 shows
`a block diagram ofthe experimental con-
`figuration used by Barstow and Christo-
`phcr‘ to subjectively evaluate the cffects of
`random noise on the analog television sig-
`nal. Figure 2 shows the results of these
`measurements where a subjective rating or
`quality was related to a quantitative value
`of signal-to-noisc ratio. This type of sub-
`jectivc test result forms the basis for all of
`the current analog specifications.
`
`Determining Quality Objectives
`Digital encoding and processing of the
`televiSion signal
`is rapidly becoming of
`considerable importance to the broadcast
`industry, with the proliferation of new
`digital television equipment for the studio
`and for transmission which has become
`available in recent years. A problem which
`arises with the introduction of the new
`digital television equipment is the evalua—
`tion and the measurement of the quality of
`digital television signals. Quality objectives
`must be determined in order to decide what
`the proper bit rates and proper encoding
`techniques are that should be used in the
`new equipment being designed. Before
`quality objectives can be determined, One
`must address the following question, "How
`do we get to the quality objectives that arc
`meaningful for digital television systems?"
`It is this question I will be discussing in this
`paper.
`One way of getting to the quality ob-
`jectives is to consider past history relating
`to the ease of analog television. In the early
`days of analog television, setting up quality
`objectives and establishing test procedures
`for evaluating these quality objectives were
`problems similar to those presently being
`encountered with digital television. What
`we are seeking is a set of performance pa-
`rameters, a standard you might say, similar
`to what we have for the analog case, which
`if met, would be considered to provide
`broadcast-quality television. In the analog
`case, for example, we have specifications
`on random noise, impulsive noise, linear
`distortion and nonlinear distortion. You
`may not agree with all the specifications for
`these impairments, but they are a repre-
`sentative set of numbers for these param-
`eters. which most of the industry agrees
`represents broadcast-quality television.
`The point is that for the analog television
`case we have been able, over the years, to
`
`Presented on 24 January 191'6 at the Society's Winter
`Television Conference in Detroit, by lconard S.
`Golding, Digital Communications Corp" l9 Firstfield
`Rd., Gaithersburg. MD 20760. The paper was subse-
`quently revised for publication in Digital Video {Q l???
`by the Society of Motion Pictureand Television EngiA
`Reefs. Inc. and is being reprinted here.
`
`PMC Exhibit 2028
`
`Apple v. PMC
`|PR2016-01520
`
`Page 1
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 1
`
`

`

`
`
`0 - L5 It!
`fll'l WISE
`Gilutott
`
`(PM WLJ’. FILTER
`
`Fig. i. Experimental configuration used by Barstow and Christopher for
`subjective evaluation of noise.
`
`F. EI'll'linU' Wflnwml
`
`i-
`
`ll'lilllfll WEHIWBLE
`
`5. SWIM“ WICIIWKE
`
`1.
`
`J.
`
`I.
`
`lH-Flllflll MT Wt WECHMI
`
`IIIPIIHII
`
`JUST FEKIPTIHE
`
`lIFlNI'lElY FEICEHIBLE OIL! “19"
`
`l. M" IEKE'I’IILE
`
`Elfin“ INN“ NIHIILIDE l! HI.“ I I‘fl'l *5
`
`in the case of digital television. one could
`carry out similar types of subjective testing.
`For example, in the analog-to-digital con-
`Version of the signal in a pulse-code-mod-
`ulation system. one could vary the number
`of bits per sample {related to the number
`of sampling levels) used to quantize the
`signal and evaluate a subjective quality
`associated with varying that particular
`parameter. One could also vary the sam-
`pling frequency and determine a subjective
`quality related to the amount of impair-
`ment occurring in the picture due to dif-
`ferent sampling Frequencies. As in the an-
`alog case. where varying the amount of
`noise and comparing it to subjective quality
`allowed one to determine a suitable sig~
`nal-to—noise ratio, for a broadcast—quality
`signal. one could determine the number of
`bits per sample and the sampling frequen-
`cy. based on the subjective rating. that is
`required in order to provide a broadcast-
`quality signal.
`In both these cases a broadcast-quality
`signal was determined by picking some
`value ofsubjective grade, such as “just im-
`perceptible" (1.5 on the 6-point impair-
`ment scale — meaning that half the ob-
`servers can perceive the impairment and
`halfcannot) and using that as a measure of
`broadcast quality. Thus. just as analog
`broadcast quality was equated to a sub-
`jective grade of “just imperceptible“ for a
`number of different types of impairment
`one can also similarly assess signal quality
`in the digital case. Table IV lists analog
`impairments that have been evaluated
`subjectively. The parameters are quite
`different. but the subjective test procedures
`could be quite similar. Subjective testing
`thus provides a basis for deriving specifi-
`cations on picture quality for both analog
`and digital television systems. Further»
`more, subjective Scales could provide a
`means of relating analog television systems
`to digital television systems. There are.
`however. a number of significant differ-
`ences in the digital case which makes the
`process more complicated than it was for
`the analog case.
`'
`
`Evaluating Digital Parameters
`One of the desirable features of digital
`signal processing of the television signal is
`that there is a great deal of flexibility in
`
`Fig. 2. Analog signal-to-Itoise measurements.
`
`being able to carry out a variety of different
`types of signal processing without seriously
`impairing the signal. While this feature is
`very attractive from a design standpoint, it
`makes assessment of quality more difficult.
`for it leads to many more parameters and
`cases to evaluate than was encountered in
`the analog case.
`Table V lists typical sets of parameters
`that would be of interest to evaluate for
`different coding methods. lfwe consider an
`analog-to-digital-COnversion process such
`as pulse code modulation, then such pa~
`rameters as sampling rate. number of bits
`per sample. companding law, clock jitter,
`and the description of the filtering used. are
`the kind of parameters which are important
`in determining the quality of the recon-
`structed analog signal. When other types
`of digital encoding methods are used the set
`of parameters that must be evaluated will
`vary and be dependent on the type of digi-
`tal processing carried out. So. for example.
`if we were to use differential-pulse~code
`modulation as the means of converting the
`analog television signal
`into a digitally
`encoded form. the parameters of interest
`would be different, e.g.,
`the particular
`prediction algorithm used in the differen-
`tial PCM coding, the number of bits per
`sample used in the feedback loop, and the
`number of previous samples used in pre
`dieting the next value of the signal. Other
`coding methods such as transform coding
`require yet another set of parameters to be
`evaluated. as indicated in Table V.
`
`In the case of digital television, there are
`two classes of parameters to be evaluated
`which impact the quality of the picture.
`The first class of parameters relates to the
`conversion of the analog signal into digital
`form and the conversion of the digital sig-
`nal back to analog form. a type of pro—
`cessing termed "source coding.” Parame-
`ters associated with different methods of
`source coding that are to be evaluated are
`listed in Table V.
`There are also impairments introduced
`into the picture after the signal is in digital
`form. They are typically called channel
`effects and are also listed in Table V. Such
`parameters are: random errors which occur
`on the bit stream, slips of the bit timing
`clock or jitter of the clock. burst errors. etc.
`These errors which are introduced into the
`bit stream after the television signal is in
`digital form will result in additional im-
`pairments appearing in the reconstructed
`analog signal and must also be evalu-
`ated.
`
`Corre!ated impairments
`There is another complicating difference
`associated with the digital case. however,
`and that is that the nature of the impair-
`ment in the reconstructed analog signal.
`due to channel effects occurring on the bit
`stream.
`is related to the type of source
`coding that was used to convert the analog
`signal
`into digital
`form. Because the
`number and type of bit errors introduced
`into the bit stream cause different analog
`
`Table 1. Subject gradig scales.
`
`Impairment
`Quality
`Comparison
`
`A-Excellcnt
`S-lmperceptible
`4-Perceptible but not annoying B—Gcod
`3-Somewhat annoying
`C-Fair
`2-Severely annoying
`0- Poor
`l-Unusable
`E-Bad
`
`+2 much better
`+l better
`0 the same
`—l worse
`—2 much worse
`
`I-lmperceptible
`Z-lust perceptible
`3-Definitely perceptible
`but not disturbing
`4~Somewhat objectionable
`S—Definitely objectionable
`Isl-Extremer objectionable
`
`l-Excellent
`1?ma
`3uFair|y good
`
`+3 much better
`+2 better
`+l slightly better
`
`4-Rather poor
`5~Poor
`GAVcry poor
`
`0 the same
`—l slightly worse
`-2 worse
`—3 much worse
`
`I54
`
`5 MPTE Journal March l 978 Volume 87
`
`PMC Exhibit 2028
`
`Apple v. PMC
`|PR2016-01520
`
`Page 2
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 2
`
`

`

`Table ll. Subjective test procedures.
`
`Reference
`
`Observers Category
`Number
`Grading Scale Type
`Number of Grades
`Test Pictures Number
`Viewing Conditions:
`Ratio of viewing
`distance to picture
`height
`Peak Luminance on the
`screen (cdlm’)
`
`on.
`{C.C.l.R.. I963-I966)
`
`our. 0.1.R.T..
`(C,C.I.R., 19634966)
`
`figfiq
`1953—1965)
`
`U.S.A. (C.C.I.R..
`U.S.A. (C.C.LR.,
`
` {966— | 969) l966— | 969)
`
`
`Non-Expert
`20—25
`Quality
`5
`4—8
`6
`
`Impairment Quality Comparison
`6
`6
`"l
`5
`4-6
`
`Non-Expert
`Approx. 2G0
`Quality
`6
`2—8
`6-8
`
`Expert
`>10
`Impagment
`3-4
`4
`
`Non-Expert
`Approx. 20
`Compsarison
`6
`6
`
`5
`1111131111“?
`
`50
`
`41-54
`
`10
`
`170 (monochrome)
`34 (color)
`
`50
`
`Contrast range of
`the picture
`Luminance of inactive
`tube screen (ed/m”)
`Luminance of backcloth l llluminant C
`(Cdlmzl
`
`
`Not specified
`50.5
`
`0.5
`
`Not specified
`2
`
`Approx. 0.5
`
`Table II continued.
`
`Japan
`(C.C.l.R.. 1963-1966C
`and l966—l969A)
`
`Fed. Rep. of Germany
` Reference (C.C.I.R., 1963—19663)
`
`
`Comparison
`5
`
`Observers Category
`Number
`Grading Scale Type
`Number of Grades
`Test Pictures Number
`Viewing Conditions:
`Ratio of viewing
`distance to picture
`height
`Peak Luminance on the
`screen {cdlm‘}
`
`Contrast range of
`_
`the picture I
`Luminancie of inactive
`tube screen (cdlni‘)
`
`Non-E xpert
`>10
`Quality
`5
`>5
`6
`
`50
`
`Not specified
`50.5
`
`Quality
`5
`
`Non-Expert
`2045
`Impairment
`5
`>3
`6-8
`
`Approx. 400
`(monochrome)
`74-34 (color)
`30;] to sort
`
`Approx. 5
`(mono chrome)
`0. 'l-Z (color)
`
`Table III. CUR-recommended subjective Iestingpi-ocedures.
`Viewing
`Viewing
`condition
`condition
`designation
`description
`a
`ratio ofviewing distance to picture height
`
`peak screen luminance (dd/m3)
`ratio of inactive~tube (cutoff) luminance to
`peak luminance
`ratio of screen luminance displaying black level
`in completely dark room to that
`corresponding to peak white
`ratio of luminance of background behind
`picture monitor to picture peak luminance
`
`S ecifications
`Sfl-liclasgs
`60-ileldsls
`systems
`systems
`6
`4 to 6
`
`70 :l: ll]
`0.02
`
`70 i ll]
`0.02
`
`approx. 00]
`
`f
`
`approx. 0.!
`
`approx. 0.15
`
`low
`lowI
`other room illumination
`055
`white
`chromaticity of background
`—
`29
`ratio of solid angle subtended by that part of
`the background which satisfies this
`sgcification to that subtended b; the picture
`
`in
`c
`
`:1
`
`e
`
`f
`g
`h
`
`impairments for different types of source.
`coding. there is an interrelation that must
`be considered when evaluating the quality'
`of a digital system.
`Furthermore,
`the impairments intro«
`duced in the analog-to-digital conversion
`process are correlated with the television
`signal and are strongly dependent on the
`
`characteristics of the picture material
`being digitized. The correlated nature of
`the impairments can result in some pecu-
`liar subjective effects. For example, in the
`case of pulse~code modulation, when too
`few bits per sample are used. false edges or
`contours appear in the picture. demon-
`strating a type of noise that is not normally
`
`Quantization noise
`lntersymbol
`interference
`Bit error rate
`Bit error time
`Distribution
`Bit Timing clock
`jitter
`Phase and
`amplitude hits
`Bit liming slips
`Impulsive noise
`
`
`Table IV. Picture impairments.
`
`Analog case Digital case
`Additive independent
`Sampling noise
`noise
`Random
`Impulsive
`Periodic
`Crosstalk
`Linear distortion
`Field time
`Line time
`Short time
`Chrominance/
`luminance
`Gain 3t delay
`inequalin
`Gain/frequency
`Nonlinear distortion
`Differential phase
`Differential gain
`Chrominance/
`luminance
`lntermodulation
`Luminance nonlinear
`distortion
`Chrominance
`nonlinear
`Gain and phase
`distortion
`Synchronizing pulse
`nonlinearitg
`
`seen in analog television. (In the analog
`television case, most ofthe impairments are
`uncorrelated with the television signal and
`produce a more random type of noise im-
`pairment
`in the picture. but this is not
`neceSSarily true for the digital case.)
`There are other differenow in the digital
`television case which must also be consid-
`ered. Because television signals can be
`stored in digital
`form,
`interframe or
`frame-to-frame coding of the signal
`is
`possible, and devices for accomplishing this
`have already been developed by certain
`manufacturers. This type of source coding
`required consideration of frame-to-framc
`subjective effects which must be tested with
`picture material that involves motion be—
`tween one frame and the next. This in turn
`leads into more complex subjective testing
`procedures than have been used in the an-
`alog television case.
`
`Golding: Quality Assessment ofDigital Television Signals
`
`155
`
`PMC Exhibit 2028
`
`Apple v. PMC
`|PR2016-01520
`
`Page 3
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 3
`
`

`

`Table V. Parameters in digital impairment
`
`Table Vi. Quality-assessment procedure. motion into account must also be consid-
`ered.
`
`testing.
`
`PCM
`
`I
`'
`soume COdlng Channel
`Filter
`Random error
`Parfl_m€l°r5
`“‘5
`Sampling rate
`Impr error {ale
`Not 0f bils/
`Bum dural'on
`Sample.
`__
`Colmpa "ding Clmkilllfl
`aw
`Clockjitm
`Phase/amp, hits
`B“ “look Slips
`-
`-
`Pmdmi'?"
`algomgm
`Compandlng law _
`No.0fbitsfsamplein
`reedPNk IOOD
`Loop filler Paramelcrs
`Transform Noofcoefficicnts uscd
`Companding law per
`coefficient
`No. of bits/coefficient
`Filter aramaters
`
`UPC M
`
`Compostte vs Component Encoding
`In the source coding area there is yet
`another basic choice to be made between
`coding methods. It involves choosing be-
`tween direct analog-to-digital conversion
`of the composite color television signal and
`separate encoding of the components ofthe
`television signal (the luminance signal and
`the two chrominance signals). The im-
`pairments perceived when using these dif—
`ferent analog-to—digital coding pmcesses
`are quite different — especially if there is
`interaction between the chrominance and
`luminance signals.
`
`be
`l
`h .
`I
`d.
`_
`D
`' avatierTlgcafimfienfizl diffgg‘lgcbe 0mm
`“$612;
`6 a
`_ Dcmmine key Parameters which are to be
`tested. and minimum range over which each
`parameter should bevaried, Minimize number
`ofcombinations ofdilTercnl parameters. which
`must be tested.
`- Carry out subjective impairment
`test.
`fol-
`owin internationall acee ted raclices for
`'subjeiing mung y
`P
`P
`- Compare performance with other digital sys-
`.
`.
`.
`.
`terns using subjective gradlng scale as common
`measure of performanw
`, Develop Objective test Signals and pmedurcs
`which permit evaluator, or performance or
`given digital system. using quantitative mea-
`sures on the television signal.
`
`the impairments produced.
`parameters.
`and the difficulties of subjective testing.
`We have found that in general the digital
`case is a more complex case to develop
`standards for because:
`I. The impairments are more varied as
`they are correlated with the television sig-
`nal and are a function of both the source
`coding and channel effects.
`2. Generally more parameters need to
`be subjectively tested to fully evaluate
`specific coding methods, and there are
`potentially a greater number of coding
`methods which may be useful and practical
`to consider.
`3. Frame-to—frame signal processing is
`quite feasible with digital
`techniques,
`which means that subjective tests taking
`
`4. After the signal is digitized and en-
`coded, the primary effect of further sources
`of degradation IS to increase the bit error
`rate and possibly change the error pat-
`tern.
`if error coding is employed to reduce
`5.
`the bit error rate on the digital bit stream
`than the cum- coding process used also will
`-
`-
`-
`affect hOdeli. erlrors will appear in there-
`cor-mm“: am 0g Signals' thus the Im-
`function not onl of the
`palrmems 3.11? a
`.
`y
`analog-to-dtgttal codmg process. but any
`Signal processtng carried out on the signal
`after it has been converted into digital
`form.
`6. Chrominance/luminancc impair—
`ments depend on whether the digitizing
`and encoding is done on composite video or
`on components.
`
`Overcoming the Complications
`While the digital case is more Compli-
`cated than the analog case, I believe it can
`be handled quite successfully with some
`intelligent planning. For each analog-to-
`digital coding process one can specify a
`particular set of parameters such as num-
`ber of bits per sample. sampling rate. etc,
`that have to be evaluated. impairment or
`quality testing following recommended test
`procedures could be carried out to relate
`each ofthese parameters to an equivalent
`subjective quality grade. if a subjective
`quality of “just imperceptible“ is selected,
`then through a series of subjective tests the
`
`.3
`
`g32I
`
`—0III
`
`Summarizing Digital Impairments
`
`marize what we have found about digital
`
`IIIa
`«itQ
`
`o I
`
`—
`;
`'6
`u
`3
`g
`
` At this point it may be useful to sum—
`
`
`(C)
`
`(ill
`
`(5)
`
`3a
`
`D
`III
`E
`o
`u.
`z5III
`
`33
`
`NUMBER OF [ITS
`
`NUMBER OF BITS
`
`Fig. 3. Example of measured results for PCM coding: variation in subjective
`impairment at different numbers at hits per sample. using dither. (Vertical
`bars show variation in grade for different picture sources, and open circles
`denote mean grades for all picture sources; horizontal line at Subjective
`Grade 1.25 indicates the mean score for an unquaotined picture.)
`
`E56
`
`SMPTE Journal
`
`March NH Volume 8.7
`
`Fig. 4. Effect of different sampling frequencies {PCM ending) on critical
`picture color hers and on noncritical pictures taken off-air with a receiver.
`(A) Color bars; no dither. (B) Color bars: with dither. (C) Off-air pictures;
`nodither. (D) Off-air pictures; with dither. (Solid lines denote a sampling
`l'reouencyfs of three times color subcarrier; long dashes shovvjr‘S = SSI x -
`line frequency: and short dashes show fs unlocked.)
`
`PMC Exhibit 2028
`
`Apple v. PMC
`|PR2016-01520
`
`Page 4
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 4
`
`

`

`BITS PER SAMPLE
`X3
`
`1):
`
`
`
`SUBJECTI'J‘EGRADE
`
`
`
`0.3
`0.4
`0.6
`0.3 1.0
`R.M.S. AMPLlTLiDE OF in'I'ER. NS
`
`1.5
`
`2.0
`
`Fig. 5. Measured results for timing jitter. The impairment is caused by
`white Gaussianjitter on a display of “30% colorbars. Circies, triangles
`and crosses respectively denote maximum jitter frequencies of 20 kHz.
`éflGkHzand 6 MHz.
`
`
`
`IMPNIIHEHTERNIE
`
`20
`
`30
`
`
`
`40
`
`1'0
`30
`50
`BIT-HATE. M BITfS
`
`W
`
`90
`
`correct value of the parameters to achieve
`this subjective quality could be determined
`by a series of impairment or quality tests.
`While the number of parameters may be
`large. by some careful planning and some
`preliminary screening the range of each of
`the parameters that have to be tested can
`be made relatively small. for a given sub-
`jective quality. Knowing the nature of the
`impairments introduced by the particular
`analog-to-digital coding method one could
`select a reasonably small set of picture
`materials that are effective at showing up
`these impairments and that could be used
`in carrying out the subjective tests. For
`each source-coding method the effects of
`different types of error patterns on the
`digitally encoded signal could be evaluated.
`In a very systematic way. the set of pa—
`rameters. which give a specified subjective
`quality. could be determined for each an-
`alog-to-digital coding method in the pres-
`ence of different bit error rates and bit
`error patterns that might be encountered
`in practice. The subjective quality scale
`would then provide the means of compar-
`ing different coding methods with regard
`to the bit rate and bit error rate needed to
`provide a specified subjective quality.
`
`Objective Test Signals
`Once this subjective test data had been
`compiled one could dispenSe with the fre-
`quent subjective tests (as has been done
`with analog television) and look into ob-
`jective test signals which could be used to
`evaluate a given subjective performance.
`For example. the pulse and bar pattern
`commonly used in the analog case for
`measuring the short time distortion could
`be used in the digital case to measure edge
`busyness and background noise in constant
`gray level areas of the picture. For the
`PCM analog-to-digital coding technique
`a ramp signal would be quite useful in de-
`tecting contours and quantization noise.
`While much more work must be done to
`
`Fig. 6. A digital subjective test: impairment vs bit rate for DPCM coding
`of PAL signals with sampling Frequencies of Zffland 3f“. Crosses show
`results for DPCM and circles for PCM.
`
`determine the correct objective test signals.
`the procedure for arriving at these test
`signals can follow along similar lines to
`those used to arrive at analog test signals.
`Table VI outlines what i believe to be a
`quality assessment procedure for the digital
`case that can be followed to arrive at
`quality objectives; the procedure is similar
`to that used originally to arrive at the an-
`alog quality objectives.
`Some examples of subjective tests that
`have already been carried out successfully
`on different digital coding methods are il-
`luslrated in Figs. 3, 4 and 5. This data.
`provided by the British Broadcasting
`Corporationfiii’ involved testing the ana-
`log-to-digita] encoding of the PAL televi-
`sion signal. In Fig. 3. the number of bits per
`sample was varied and related to a sub-
`jective quality. In Fig. 4. different sampling
`frequencies were evaluated and related to
`a subjective quality as a function of dif—
`ferent number of bits per sample. In Fig. 5.
`the effects of timingjitter were related to
`subjective quality for a PCM signal. The
`test procedure followed was similar to that
`recommended by the CCIR (Table III).
`Five or six different pictures were used as
`the subject material. The results. as illus-
`trated in these figures. show how quanti-
`tative values can be determined for the set
`of parameters associated with PCM en-
`coding of the signal in order to obtain a
`given specified subjective quality.
`Figure 6 illustrates how the subjective
`rating scale can be used to compare dif-
`ferent analog-to—digital coding methods. In
`this figure both pulse-code modulation and
`differential pulse-code modulation are re-
`lated to a given subjective quality at a given
`bit rate. The subjective quality is shown for
`different numbers of hits per sample and
`different sampling rates for the DPCM
`method. The PCM performance. with 8
`
`bits per sample and a sampling rate of
`twice the color subcarrier frequency is also
`plotted on the same graph. These test re-
`sults. provided by the BBC, illustrate how
`a comparison can be made. As previously
`mentioned, different digital coding meth-
`ods could also be compared to analog sys-
`tem performance by using one of the sub-
`jective grading scales such as the impair-
`ment grading scale. as a common basis of
`comparison even though the nature ofthe
`impairments may be different. One must
`be careful that the test results used. how-
`ever. apply to a sufficiently large amount
`of picture material to make the comparison
`valid.
`While the procedure to get to the quality
`objectives for digital
`television systems
`appears to involve a considerable effort
`(possibly a lot more effort than was origi-
`nally needed to arrive at the analog televi-
`sion quality objectives}. it is expected that
`this procedure would be carried out over a.
`considerable period of time. Furthermore.
`with the type of impairments being much
`more varied in the digital case than in the
`analog case. it would appear that the sub-
`jective testing procedure would be the only
`way of getting a common measure of
`quality which could be used to determine
`the correct parameters for different digital
`encoding methods. different bit error rates
`and different digital impairments which
`might occur.
`References
`l. J, M. Barstcw and H. N. Christopher. “The Mea-
`snremenl of Random Video Interference to Mono-
`chrome and Color Television Pictures." Ai‘EE
`Trans. No. 63. Nov 1962.
`2. V. G. Dcvereux. “Pulse Code Moduialion ofVideo
`Signals: Subjective Study of Coding Parameters."
`BBC Research Report No. “971/40.
`3. "Pulse Code Modulation of Video Signals: Q-Bit
`Coder and Decoder." BBC Research Report No.
`reruns.
`
`Golding: Quality Assessment ofDigr‘rai' Television Signal:
`
`IS‘Jr
`
`PMC Exhibit 2028
`
`Apple v. PMC
`|PR2016-01520
`
`Page 5
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 5
`
`