Quality Assessment of
`Digital Television Signals
`
`By LEONARD S. GOLDING
`
`Procedures haveexisted for many years permitting the subjective evaluation of conventional
`(analog)television pictures. Subjective data has been amassed and objectivetest signals have
`been developed that can be used to predict with some certainty the quality of picture that
`will be presented to the viewer. It is noted that the digital case is substantially more complex
`than the analog case, but it is important and worthwhile to derive similar procedures for
`subjectively evaluating digitally encoded television pictures. A way is suggested for beginning
`suchan effort, andit is foreseen that objective test signals could be developed for assuring
`a given subjectively evaluated quality level in the presence of various digital impairments.
`In this way, digitized television pictures could be validly compared with each other and with
`analog pictures. Topics discussed include: the various subjective grading scales; source coding
`and channel effects; PCM, DPCM and transform encoding; interframe processing; and
`encoding of composite and componentvideo signals.
`
`pairment to the television signal that has
`been created. This is the type of test used
`by Barstow and Christopher. Second, there
`is a quality test where the observeris asked
`to rate the overall quality of the picture;
`and third, there is a comparison-typetest
`where the observer is asked to compare the
`quality of a given picture against
`the
`quality of another picture. All three types
`of subjective test have been used in evalu-
`ating analog television signal quality and
`each has its own grading scale and test
`procedure. Table I
`lists typical grading
`scales that have been used for each type of
`subjective test. Table II
`lists common
`subjective test procedures which have been
`followed by various countries such as the
`U.S.A. and the United Kingdom and by
`several international organizations. The
`subjective test procedures must consider
`the numberof observers, the type of grad-
`ing scale used, the viewing conditions and
`the type of picture materialused in thetest.
`Theseareall referred to in Table III. After
`a numberof years, there has been agree-
`ment within the CCIR as to a recom-
`mended subjective testing procedure for
`testing television signals. Table II] lists the
`recommended subjective testing procedure
`now internationally accepted.
`The importance of subjective testing is
`that the subjective grading scale (such as
`the impairment scale — which has grades
`of
`imperceptible, perceptible but not
`annoying, slightly annoying, annoying and
`very annoying — as given in Table IV) is
`a universal scale which allows one to com-
`pare different kinds of impairmentin the
`television picture and therefore allows one
`to compareonetelevision system with an-
`other and one type of signal processing
`method with another. So the subjective test
`provides a universal scale that can be used
`to measure all different kinds of systems
`and compare them with each other. The
`subjective testing scale also is directly re-
`lated to picture quality as seen by the ob-
`server, and so permits oneto easily define
`a broadcast-quality signal. In the case of
`commercial broadcast service, where the
`ultimate objectiveis to present a pleasing
`and high-quality picture to the observer,
`the subjective scale allows that picture
`quality to be evaluated directly.
`Let us consider the impairmenttest in
`greaterdetail. In the impairmenttest, one
`adds different amounts of an impairment
`suchas noise to the original signal and de-
`termines how the observer evaluates the
`visibility of the impairment as a function
`of the amount addedto the television sig-
`Three Kinds of Subjective Test
`nal. Typically, one considers a single pa-
`rameter such as the amount ofnoise or the
`In the area of subjective testing, there
`power of the noise and relates that to a
`are three main typesof subjective test. The
`judgmenton the subjective grading scale
`first is an impairment test where the ob-
`madeby the observeras given in TableIII.
`server is asked to judge the degree of im-
`Volume 87 March 1978 SMPTE Journal
`153
`
`come to a point where we can specify
`quantitatively the parameters that define
`quality. We would like to reach a similar
`objective in the digital case. Let us examine
`how these parameters were derived and
`how we reached the present objectives for
`the analog case.
`In the early daysoftelevision, there were
`many laboratories such as Bell Telephone
`Laboratories, R.C.A. Laboratories and
`others around the world that carried out
`subjective tests to evaluate such parameters
`as signal-to-noise ratio and differential
`phase. They determined just how these
`parameters related to a given subjective
`quality such as just-perceptible distortion
`or annoying distortion.
`In other words,
`quantitative parameters were related to a
`subjective measure of performance. Once
`the subjective measures were determined,
`a second phase took place where objective
`test patterns were developed which could
`measure these quantitative parameters by
`meansof a vectorscope, waveform monitor,
`oscilloscope or some other piece of test
`equipment and relate them directly to
`picture quality based on the subjectivetest
`results. The use of objective test patterns
`eliminated the need for carrying out sub-
`jective testing every time one wished to
`evaluate the quality of a particular televi-
`sion system. In the developmentofthe ob-
`jective test patterns,
`the quantitative
`measures needed for each impairment,
`such as noise, were originally based on the
`subjective test results that had been ob-
`tained. As an example of this, Fig. 1 shows
`a block diagram of the experimental con-
`figuration used by Barstow and Christo-
`pher! to subjectively evaluate the effects of
`random noise on the analogtelevision sig-
`nal. Figure 2 shows the results of these
`measurements where a subjective rating or
`quality was related to a quantitative value
`of signal-to-noise ratio. This type of sub-
`jective test result forms the basis forall of
`the current analog specifications.
`
`Determining Quality Objectives
`Digital encoding and processing of the
`television signal
`is rapidly becoming of
`considerable importance to the broadcast
`industry, with the proliferation of new
`digital television equipment for the studio
`and for transmission which has become
`available in recent years. A problem which
`arises with the introduction of the new
`digital television equipment is the evalua-
`tion and the measurementofthe quality of
`digital television signals. Quality objectives
`must be determinedin order to decide what
`the proper bit rates and proper encoding
`techniques are that should be used in the
`new equipment being designed. Before
`quality objectives can be determined, one
`must address the following question, “How
`do weget to the quality objectives that are
`meaningful for digital television systems?”
`It is this question I will be discussing in this
`paper.
`One way of getting to the quality ob-
`jectives is to considerpasthistory relating
`to the case of analogtelevision. In the early
`days of analogtelevision, setting up quality
`objectives and establishing test procedures
`for evaluating these quality objectives were
`problems similar to those presently being
`encountered with digital television. What
`we are seekingis a set of performance pa-
`rameters, a standard you might say, similar
`to what wehavefor the analog case, which
`if met, would be considered to provide
`broadcast-quality television, In the analog
`case, for example, we have specifications
`on random noise, impulsive noise, linear
`distortion and nonlinear distortion. You
`maynotagree with all the specifications for
`these impairments, but they are a repre-
`sentative set of numbersfor these param-
`eters, which most of the industry agrees
`represents broadcast-quality television.
`The point is that for the analog television
`case we have been able, over the years, to
`
`Presented on 24 January 1976 at the Society's Winter
`Television Conference in Detroit, by Leonard S,
`Golding, Digital Communications Corp., 19 Firstfield
`Rd., Gaithersburg, MD 20760. The paper was subse-
`quently revised for publication in Digital Video © 1977
`by the Society of Motion Picture and Television Engi-
`neers, Inc. andis being reprinted here.
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 1
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 1
`
`

`

`‘COLOR
`
`
`
`
`P|TTLA
`
`
`See
`
`LeOBSERVER
`Att
`
`|fi|[|
`
`60
`
`50
`55
`WEIGHTED NOISE MAGNITUDE D8 BELOW | VOLT Fe
`
`7.
`
`EXTREMELY OBJECT/OMABLE
`
`. DEFINITELY GBJECTIONABLE
`
`,
`
`SOMEWHAT OBJECTIONABLE
`
`* 5
`
`4.
`
`THPADRMENT BUT WOT OBJECTIONABLE
`
`3. DEFINITELY PERCEPTIBLE OWLY SLIGHT
`IMPAIRMENT
`
`2.
`
`JUST PERCEPTIBLE
`
`.
`
`WOT PERCEPTIBLE
`
`
`
`Fig. 1, Experimental configuration used by Barstow and Christopher for
`subjective evaluation of noise.
`
`In the case of digital television, one could
`carry out similar types of subjective testing.
`For example, in the analog-to-digital con-
`version of the signal in a pulse-code-mad-
`ulation system,one could vary the number
`of bits per sample (related to the number
`of sampling levels) used to quantize the
`signal and evaluate a subjective quality
`associated with varying that particular
`parameter. One could also vary the sam-
`pling frequency and determinea subjective
`quality related to the amount of impair-
`ment occurring in the picture due to dif-
`ferent sampling frequencies. As in the an-
`alog case, where varying the amount of
`noise and comparingit to subjective quality
`allowed one to determine a suitable sig-
`nal-to-noise ratio, for a broadcast-quality
`signal, one could determine the number of
`bits per sample and the sampling frequen-
`cy, based on the subjective rating, that is
`required in order to provide a broadcast-
`quality signal.
`In both these cases a broadcast-quality
`signal was determined by picking some
`value of subjective grade, such as “just im-
`perceptible” (1.5 on the 6-point impair-
`ment scale — meaning that half the ob-
`servers can perceive the impairment and
`half cannot) and using that as a measure of
`broadcast quality. Thus, just as analog
`broadcast quality was equated to a sub-
`jective grade of “just imperceptible” for a
`number ofdifferent types of impairment
`one can also similarly assess signal quality
`in the digital case. Table IV lists analog
`impairments that have been evaluated
`subjectively. The parameters are quite
`different, but the subjective test procedures
`could be quite similar. Subjective testing
`thus provides a basis for deriving specifi-
`cations on picture quality for both analog
`and digital television systems. Further-
`more, subjective scales could provide a
`meansofrelating analog television systems
`to digital television systems. There are,
`however, a numberofsignificant differ-
`encesin the digital case which makes the
`process more complicated than it was for
`the analog case.
`.
`
`Evaluating Digital Parameters
`Oneofthe desirable features of digital
`signal processing of the television signal is
`that there is a great deal of flexibility in
`
`Fig. 2. Analog signal-to-noise measurements.
`
`being able to carry out a variety of different
`types of signal processing without seriously
`impairing the signal. While this feature is
`very attractive from a design standpoint,it
`makes assessment of quality more difficult,
`for it leads to many more parameters and
`cases to evaluate than was encountered in
`the analog case.
`Table V lists typical sets of parameters
`that would be of interest to evaluate for
`different coding methods. If we consider an
`analog-to-digital-conversion process such
`as pulse code modulation, then such pa-
`rameters as sampling rate, numberof bits
`per sample, companding law, clock jitter,
`and the description ofthefiltering used, are
`the kind of parameters which are important
`in determining the quality of the recon-
`structed analog signal. When other types
`of digital encoding methods are used theset
`of parameters that must be evaluated will
`vary and be dependent on thetypeof digi-
`tal processing carried out. So, for example,
`if we were to use differential-pulse-code
`modulation as the meansof converting the
`analog television signal
`into a digitally
`encoded form, the parameters of interest
`would be different, ¢.g.,
`the particular
`prediction algorithm usedin the differen-
`tial PCM coding, the numberofbits per
`sample used in the feedback loop, and the
`number of previous samples used in pre-
`dicting the next valueof the signal. Other
`coding methods such as transform coding
`require yet another set of parameters to be
`evaluated, as indicated in Table V.
`
`In the case ofdigital television, there are
`two classes of parameters to be evaluated
`which impact the quality of the picture.
`Thefirst class of parameters relates to the
`conversionof the analogsignal into digital
`form and the conversion ofthedigital sig-
`nal back to analog form, a type of pro-
`cessing termed “source coding.” Parame-
`ters associated with different methods of
`source coding that are to be evaluated are
`listed in Table V.
`There are also impairments introduced
`into the picture after the signalis in digital
`form. They are typically called channel
`effects and are also listed in Table V. Such
`parameters are: random errors which occur
`on the bit stream, slips of the bit timing
`clock orjitter of the clock, burst errors, etc.
`These errors which are introduced into the
`bit stream after the television signalis in
`digital form will result in additional im-
`pairments appearing in the reconstructed
`analog signal and must also be evalu-
`ated.
`
`Correlated Impairments
`Thereis another complicating difference
`associated with the digital case, however,
`and that is that the nature of the impair-
`ment in the reconstructed analog signal,
`due to channeleffects occurring on the bit
`stream,
`is related to the type of source
`coding that was used to convert the analog
`signal
`into digital
`form. Because the
`number and type ofbit errors introduced
`into the bit stream cause different analog
`
`Table I. Subject grading scales.
`
`Impairment
`Quality
`Comparison
`
`A-Excellent
`5-Imperceptible
`4-Perceptible but not annoying B-Good
`3-Somewhat annoying
`C-Fair
`2-Severely annoying
`D-Poor
`|-Unusable
`E-Bad
`
`+2 muchbetter
`+1 better
`Othe same
`—| worse
`—2 much worse
`
`|-Imperceptible
`2-Just perceptible
`3-Definitely perceptible
`but not disturbing
`4-Somewhat objectionable
`5-Definitely objectionable
`6-Extremely objectionable
`
`1-Excellent
`2-Good
`3-Fairly good
`
`+3 much better
`+2 better
`+1 slightly better
`
`4-Rather poor
`5-Poor
`6-Very poor
`
`Othesame
`—I slightly worse
`—2 worse
`—3 much worse
`
`154
`
`SMPTE Journal March 1978 Volume 87
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 2
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 2
`
`

`

`Table II. Subjective test procedures.
`UK.
`E.B.U., O.LR.T.
`Welk,
`(C.C.LR., 1963-1966)
`(C.C.1.R., 1963-1966)
`1963-1966)
`
`U.S.A, (C.C.LR.,
`U.S.A. (C.CLR.,
`
` 1966-1969) 1966-1969)
`
`
`Reference
`
`Observers Category
`Number
`Grading Scale Type
`Numberof Grades
`Test Pictures Number
`Viewing Conditions:
`Ratio of viewing
`distance to picture
`height
`Peak Luminance on the
`screen (cd/m?)
`
`Non-Expert
`20-25
`Quality
`5
`4-8
`6
`
`Impairment Quality Comparison
`6
`6
`7
`5
`4-6
`
`Non-Expert
`Approx. 200
`Quality
`6
`2-8
`6-8
`
`Expert
`>10
`nbaimeat
`34
`4
`
`Non-Expert
`Approx. 20
`companot
`6
`6
`
`Impmnt:
`5
`
`50
`
`41-54
`
`70
`
`170 (monochrome)
`34 (color)
`
`50
`
`Contrast range of
`the picture
`Luminance of inactive
`tube screen (cd/m?)
`Luminance of backcloth 1 Illuminant C
`(cd/m?)
`
`
`Notspecified
`$0.5
`
`0.5
`
`Not specified
`2
`
`Approx. 0.5
`
`Table II continued,
`Japan
`(C.C.LR., 1963-1966C
`and 1966-1969A)
`
`Fed. Rep. of Germany
`
` Reference (C.C.LR., 1963-1966B)
`
`Comparison
`5
`
`Observers Category
`Number
`Grading Scale Type
`Number of Grades
`Test Pictures Number
`Viewing Conditions:
`Ratio of viewing
`distance to picture
`height
`Peak Luminance onthe
`screen (cd/m?)
`
`Contrast range of
`;
`the picture —
`Luminance of inactive
`tube screen (cd/m?)
`
`Non-Expert
`>10
`Quality
`5
`>§
`6
`
`50
`
`Not specified
`30.5
`
`Quality
`5
`
`Non-Expert
`20-25
`Impairment
`5
`>3
`6-8
`
`Approx. 400
`(monochrome)
`74-84 (color)
`30/1 to 50/1
`
`Approx. 5
`(monochrome)
`0.7-2 (color)
`
`Table III. CCIR-recommended subjective testingprocedures.
`Viewing
`Viewing
`Specifications
`condition
`condition
`s0-Tields/s
`60-fields/s
`designation
`description
`systems
`systems
`a
`ratio of viewing distance to picture height
`6
`4to6
`
`b
`S
`
`d
`
`e
`
`f
`g
`h
`
`peak screen luminance (cd/m?)
`ratio of inactive-tube (cutoff) luminance to
`peak luminance
`ratio of screen luminance displaying black level
`in completely dark room to that
`corresponding to peak white
`ratio of luminance of background behind
`picture monitor to picture peak luminance
`
`70+ 10
`0.02
`
`70+ 10
`0.02
`
`approx. 001
`
`_
`
`approx, 0.1
`
`approx. 0.15
`
`low
`low
`other room illumination
`Des
`white
`chromaticity of background
`_
`29
`ratio of solid angle subtended by that part of
`the background which satisfies this
`specification to that subtended by the picture
`
`impairments for different types of source,
`coding, there is an interrelation that must
`be considered when evaluating the quality
`of a digital system.
`Furthermore,
`the impairments intro-
`duced in the analog-to-digital conversion
`process are correlated with the television
`signal and are strongly dependent on the
`
`characteristics of the picture material
`being digitized. The correlated nature of
`the impairments can result in some pecu-
`liar subjective effects. For example,in the
`case of pulse-code modulation, when too
`few bits per sample are used, false edges or
`contours appear in the picture, demon-
`strating a type of noise that is not normally
`
`Quantization noise
`Intersymbol
`Interference
`Bit error rate
`Bit error time
`Distribution
`Bit Timing clock
`jitter
`Phase and
`amplitude hits
`Bit timing slips
`Impulsive noise
`
`
`Table IV. Picture impairments.
`Analog case Digital case
`
`Additive independent
`Sampling noise
`noise
`Random
`Impulsive
`Periodic
`Crosstalk
`Linear distortion
`Field time
`Line time
`Short time
`Chrominance/
`luminance
`Gain & delay
`inequality
`Gain/ frequency
`Nonlinear distortion
`Differential phase
`Differential gain
`Chrominance/
`luminance
`Intermodulation
`Luminance nonlinear
`distortion
`Chrominance
`nonlinear
`Gain and phase
`distortion
`Synchronizing pulse
`nonlinearity
`
`seen in analog television. (In the analog
`television case, mostof the impairments are
`uncorrelated with the television signal and
`produce a more random typeofnoise im-
`pairment
`in the picture, but this is not
`necessarily true for the digital case.)
`There are otherdifferences in the digital
`television case which mustalso be consid-
`ered. Because television signals can be
`stored in digital
`form,
`interframe or
`frame-to-frame coding of the signal
`is
`possible, and devices for accomplishing this
`have already been developed by certain
`manufacturers. This type of source coding
`required consideration of frame-to-frame
`subjective effects which must be tested with
`picture material that involves motion be-
`tween one frameandthe next. This in turn
`leads into more complex subjective testing
`procedures than have been usedin the an-
`alog television case.
`
`Golding: Quality AssessmentofDigital Television Signals
`
`155
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 3
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 3
`
`

`

`Table VI, Quality-assessment procedure,
`
`+ Determine source coding technique to be
`evaluated and channel conditions to be con-
`sidered.
`+ Determine key parameters which are to be
`tested, and minimum range over which each
`parameter should be varied, Minimize number
`of combinationsofdifferent parameters, which
`must be tested,
`fol-
`test,
`« Carry out subjective impairment
`lowing internationally accepted practices for
`subjecting testing.
`« Compare performance with other digital sys-
`tems using subjective grading scale as common
`measure of performance.
`+ Develop objective test signals and procedures
`which permit evaluation of performance of
`given digital system, using quantitative mea-
`sures on the television signal.
`
`the impairments produced,
`parameters,
`and the difficulties of subjective testing.
`We have found that in general the digital
`case is a more complex case to develop
`standards for because:
`|. The impairments are morevaried as
`they are correlated with the television sig-
`nal and are a function of both the source
`coding and channeleffects.
`2. Generally more parameters need to
`be subjectively tested to fully evaluate
`specific coding methods, and there are
`potentially a greater number of coding
`methods which maybe useful and practical
`to consider.
`3. Frame-to-framesignal processing is
`quite feasible with digital
`techniques,
`which means that subjective tests taking
`
`motion into account must also be consid-
`ered.
`
`4. After the signal is digitized and en-
`coded, the primary effect of further sources
`of degradation is to increase the bit error
`rate and possibly change the error pat-
`tern.
`
`5. If error coding is employed to reduce
`the bit error rate on thedigital bit stream
`then the error coding process used also will
`affect how bit errors will appear in the re-
`constructed analog signals; thus the im-
`pairments are a function not only of the
`analog-to-digital coding process, but any
`signal processing carried out on the signal
`after it has been converted into digital
`form.
`6. Chrominance/luminance impair-
`ments depend on whether the digitizing
`and encoding is done on composite video or
`on components.
`
`Overcoming the Complications
`While the digital case is more compli-
`cated than the analog case, I believe it can
`be handled quite successfully with some
`intelligent planning. For each analog-to-
`digital coding process one can specify a
`particular set of parameters such as num-
`ber of bits per sample, samplingrate,etc.,
`that have to be evaluated. Impairmentor
`quality testing following recommendedtest
`procedures could be carried outto relate
`each of these parameters to an equivalent
`subjective quality grade. If a subjective
`quality of “just imperceptible”is selected,
`then througha series of subjective tests the
`
`
`
`SUBJECTIVEGRADE
` NUMBER OFBITS
`GRADE
`SUBJECTIVE
`
`Table V, Parameters in digital impairment
`testing.
`
`PCM
`
`Source Coding Channel
`Filter
`Random error
`parameters
`rate
`Sampling rate
`Imp. error rate
`No. ofbits/
`Burst duration
`sample
`Companding
`law
`Clockjitter
`
`Clockjitter
`
`Phase/amp.hits
`Bit clock slips
`
`DPCM
`
`Prediction
`algorithm
`Companding law
`No. of bits/sample in
`feedback loop
`Loopfilter parameters
`Transform No.ofcoefficients used
`Companding law per
`coefficient
`No. of bits/coefficient
`Filter parameters
`
`Composite vs Component Encoding
`In the source coding area there is yet
`another basic choice to be made between
`coding methods. It involves choosing be-
`tween direct analog-to-digital conversion
`of the composite colortelevision signal and
`separate encoding of the componentsofthe
`television signal (the luminance signal and
`the two chrominance signals). The im-
`pairments perceived when usingthesedif-
`ferent analog-to-digital coding processes
`are quite different — especially if thereis
`interaction between the chrominance and
`luminance signals.
`
`Summarizing Digital Impairments
`At this point it may be useful to sum-
`marize what we have found aboutdigital
`
`
`
`NUMBEROFBITS
`
`
`
`SUBJECTIVEGRADE
`
`Fig. 3. Example of measured results for PCM coding: variation in subjective
`impairment at different numbers of bits per sample, using dither. (Vertical
`bars showvariation in grade for different picture sources, and opencircles
`denote mean grades for all picture sources; horizontal line at Subjective
`Grade 1.25 indicates the mean score for an unquantizedpicture.)
`
`156
`
`SMPTE Journal
`
`March 1978 Volume 87
`
`Fig. 4. Effect of different sampling frequencies (PCM coding) on critical
`picture color bars and on noncritical pictures taken off-air with a receiver.
`(A) Color bars; no dither. (B) Color bars; with dither. (C) Off-air pictures;
`no dither. (D) Off-air pictures; with dither. (Solid lines denote a sampling
`frequency /, of three times color subcarrier; long dashes show/, = 851 X -
`line frequency; and short dashes show /, unlocked.)
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 4
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 4
`
`

`

` w
`
`O41
`
`0.8 1.0
`0.6
`0.4
`0.3
`~ 0.2
`R.M.S. AMPLITUDE OF JITTER, NS
`
`1.5
`
`2.0
`
`=
`
`iat
`=
`5
`g
`a
`2
`”
`
`Fig. 5. Measured results for timing jitter. The impairment is caused by
`white Gaussian jitter on a display of 100% colorbars. Circles, triangles
`and crosses respectively denote maximum jitter frequencies of 20 kHz,
`600 kHz and 6 MHz.
`
`5
`o
`e
`zWw
`z
`z=
`<a
`=
`
`4
`
`BITS PER SAMPLE
`x3
`
`ax
`
`Pa
`
`a5
`
` 40
`
`70
`60
`50
`BIT-RATE, M BIT/S
`
`80
`
`90
`
`100
`
`Fig. 6. A digital subjective test: impairment ys bit rate for DPCM coding
`of PAL signals with sampling frequencies of 2/,-and 3/,,. Crosses show
`results for DPCM and circles for PCM.
`
`correct value of the parameters to achieve
`this subjective quality could be determined
`by a series of impairmentor quality tests.
`While the numberof parameters may be
`large, by some careful planning and some
`preliminary screening the range of each of
`the parameters that have to be tested can
`be maderelatively small, for a given sub-
`jective quality. Knowing the nature ofthe
`impairments introduced by the particular
`analog-to-digital coding method one could
`select a reasonably small set of picture
`materials that are effective at showing up
`these impairments and that could be used
`in carrying out the subjective tests. For
`each source-coding method the effects of
`different types of error patterns on the
`digitally encoded signal could be evaluated.
`In a very systematic way, the set of pa-
`rameters, which give a specified subjective
`quality, could be determined for each an-
`alog-to-digital coding method in the pres-
`ence of different bit error rates and bit
`error patterns that might be encountered
`in practice. The subjective quality scale
`would then provide the means of compar-
`ing different coding methods with regard
`to the bit rate andbit error rate needed to
`provide a specified subjective quality.
`
`bits per sample and a sampling rate of
`determine the correct objectivetest signals.
`the procedure for arriving at these test
`twice the color subcarrier frequency is also
`signals can follow along similar lines to
`plotted on the same graph. These test re-
`sults, provided by the BBC,illustrate how
`those used to arrive at analogtest signals.
`Table VI outlines what I believe to be a
`a comparison can be made. Aspreviously
`mentioned, different digital coding meth-
`quality assessment procedurefor the digital
`case that can be followed to arrive at
`ods could also be comparedto analog sys-
`tem performancebyusing one of the sub-
`quality objectives; the procedureis similar
`jective grading scales such as the impair-
`to that used originally to arrive at the an-
`ment grading scale, as a common basis of
`alog quality objectives.
`comparison even thoughthe natureof the
`Some examples of subjective tests that
`impairments may be different. One must
`have already beencarried out successfully
`be careful that the test results used, how-
`on different digital coding methodsareil-
`ever, apply to a sufficiently large amount
`lustrated in Figs. 3, 4 and 5. This data,
`of picture material to make the comparison
`provided by the British Broadcasting
`valid,
`Corporation,?3 involved testing the ana-
`While the procedure to get to the quality
`log-to-digital encoding of the PAL televi-
`objectives for digital
`television systems
`sion signal. In Fig. 3, the numberofbits per
`appears to involve a considerable effort
`sample was varied and related to a sub-
`(possibly a lot more effort than was origi-
`jective quality. In Fig. 4, different sampling
`nally needed to arrive at the analogtelevi-
`frequencies were evaluated and related to
`sion quality objectives), it is expected that
`a subjective quality as a function of dif-
`this procedure would be carried out over a
`ferent numberofbits per sample. In Fig. 5,
`considerable period of time. Furthermore,
`the effects of timing jitter were related to
`with the type of impairments being much
`subjective quality fora PCM signal. The
`morevaried in the digital case than in the
`test procedure followed wassimilar to that
`analog case, it would appear that the sub-
`recommended by the CCIR (Table III).
`Objective Test Signals
`jective testing procedure would bethe only
`Five or six different pictures were used as
`way of getting a common measure of
`Oncethis subjective test data had been
`the subject material. The results, as illus-
`quality which could be used to determine
`trated in these figures, show how quanti-
`compiled one could dispense with the fre-
`tative values can be determined for the set
`the correct parameters for different digital
`quent subjective tests (as has been done
`encoding methods,different bit error rates
`with analog television) and look into ob-
`of parameters associated with PCM en-
`and different digital impairments which
`coding of the signal in order to obtain a
`jective test signals which could be used to
`might occur.
`given specified subjective quality.
`evaluate a given subjective performance.
`Figure6illustrates how the subjective
`For example, the pulse and bar pattern
`References
`rating scale can be used to compare dif-
`commonly used in the analog case for
`|. J. M. Barstow and H, N, Christopher, “The Mea-
`ferent analog-to-digital coding methods. In
`measuring the short time distortion could
`surement of Random Video Interference to Mono-
`this figure both pulse-code modulation and
`be used in the digital case to measure edge
`chrome and Color Television Pictures," 4/EE
`Trans., No. 63, Nov. 1962.
`differential pulse-code modulationare re-
`busyness and backgroundnoise in constant
`2. V.G. Devereux, “Pulse Code Modulation of Video
`gray level areas of the picture. For the
`lated to a given subjective quality at a given
`Signals: Subjective Study of Coding Parameters,”
`bit rate. The subjective quality is shown for
`PCM analog-to-digital coding technique
`BBC Research Report No. 1971/40.
`different numbersof bits per sample and
`a rampsignal would be quite useful in de-
`3. “Pulse Code Modulation of Video Signals: 8-Bit
`tecting contours and quantization noise.
`different sampling rates for the DPCM
`Coder and Decoder,” BBC Reseurch Report No.
`While much more work must be done to
`1970/25,
`method. The PCM performance, with 8
`
`Golding: Quality Assessmentof Digital Television Signals
`
`157
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 5
`
`PMC Exhibit 2028
`Apple v. PMC
`IPR2016-01520
`Page 5
`
`