Quality Assessment of
`Digital Television Signals
`
`By LEONARD S. GOLDING
`
`Procedures have existed for many years permitting the subjective evaluation of conventional
`(analog) television pictures. Subjective data has been amassed and objective test grants have
`been developed that can be used to predict with some certainty the quality of picture that
`will be presented to the viewer. It isnotcd that the digital case is substantially more complex
`than the analog case, but it is important and worthwhile to derive similar procedures for
`subjectively evaluating digitally encoded television pictures. A way is suggested for begimting
`such an effort, and it is foreseen that objective test signals could be developed for assuring
`a given subjectiver evaluated quality level in the presence of various digital impairments.
`In this way, digitized television pictures could be validly compared with each other and with
`analog pictures. Topics discussed include: the various subjective grading scales; source coding
`and channel effects; PCM, DPCM and transform encoding; intcrlrame processing; and
`encoding of composite and component video signals.
`
`153
`
`puirrncnt to the television signal that has
`been created. This is the type of test used
`by Barstow and Christopher. Second. there
`is a quality test where the observer is asked
`to rate the overall quality of the picture;
`and third, there is a comparison-type test
`where the observer is asked to compare the
`quality of a given picture against
`the
`quality of another picture. All three types
`of subjective lest have been used in cva1u~
`sting analog television signal quality and
`each has its awn grading scale and test
`procedure. Table 1 lists typical grading
`scales that have been used for each type of
`subjective test. Table ll lists common
`subjective test procedures which have been
`followed by various countries such as tho
`U.S.A. and the United Kingdom and by
`Several international organizations. The
`subjective test procedures must consider
`the number of observers. the typcof grud-
`ing scale used. the viewing conditions and
`the type of picture material used in the lost.
`These are all referred to in Table II]. After
`a number of years. there has been agree»
`ment within the CClR as to a recom-
`mended subjective testing procedure for
`testing television signals. Table III lists the
`recommended subjective testing procedure
`new internationally accepted.
`The importance of subjective testing is
`that the subjective grading scale (such as
`the impairment scale — which has grades
`of imperceptible. perceptible but not
`annoying. slightly annoying. annoying and
`very annoying —- as given in Table W) is
`a universal scale which allows one to corn—
`porc different kinds of impairment in the
`television picture and therefore allows one
`to compare one television system with an-
`other and one type of signal processing
`method with another. So the subjective test
`provides a universal scale that can be used
`to measure all different kinds of systems
`and compare them with each other. The
`subjective testing scale also is directly we
`latcd to picture quality as seen by the ob
`server, and so permits one to easily define
`a broadcast-quality signal. In the case of
`commercial broadcast service. where the
`ultimate objective is to present a pleasing
`and high-quality picture to the observer,
`the subjective scale allows that picture
`quality to be evaluated directly.
`Let us consider the impairment test in
`greater detail. In the impairment test, one
`adds different amounts of an impairment
`such as noise to the original signal and de-
`termines how the observer evaluates the
`visibility of the impairment as a function
`of the amount added to the television sig-
`Three Kinds of Subjective Test
`nal. Typically. one considers a single pa-
`rameter such as the amount of noise or the
`In the area of subjective testing. there
`power of the noise and relates that to a
`are three main types of subjective test. The
`judgment on the subjective grading scale
`first is an impairment test where the ob-
`made by the observer as given in Table Ill.
`server is asltcd to judge the degree of im-
`Volume 87 March £973 SMPTE Journal
`
`Determining Quality Objectives
`Digital encoding and processing of the
`television signal is rapidly becoming of
`considerable importance to the broadcast
`industry, with the proliferation of new
`digital television equipment for the studio
`and for transmission which has become
`available in recent years. A problem which
`arises with the introduction of the new
`digital television equipment is the evalua-
`tion and the measurement of the quality of
`digital television signals. Quality objectives
`must be determined in order to decide what
`the proper bit rates and proper encoding
`techniques are that should be used in the
`new equipment being designed. Before
`quality objectives can be determined. one
`must address the following question. “How
`do we get to the quality objectives that are
`meaningful for digital television systems?“
`It is this question I will be discussing in this
`paper.
`One way of getting to the quality ob-
`jectives is to consider post history rclating
`to the case of analog television. In the early
`days of analog television. setting up quality
`objectives and establishing test procedures
`for evaluating these quality objectives were
`problems similar to those presently being
`encountered with digital television. What
`we are seeking is a set of performance pa-
`rameters, a standard you might say, similar
`to what we have for the analog case. which
`if met. would be considered to provide
`broadcast-quality television. in the analog
`case, for example. we have specifications
`on random noise, impulsive noise. linear
`distortion and nonlinear distortion. You
`may not agree with all the specifications for
`these impairments, but they are a repre-
`sentative set of numbers for these param-
`eters, which most of the industry agrees
`represents broadcast-quality television.
`The point is that for the analog television
`case we have been able. over the years. to
`
`Presented on 241a1'luary me at the Society's Winter
`Television Conference in Detroit. by Leonard S.
`Golding. Digital Communications Com. l9 Firstl'teid
`Rd. Gaithersburg. MD 10160. The paper was subse-
`quently revised for publication in Digital Video © 197?
`by the Society of Motion Picture and Television Engi-
`neers. inc. and is being reprinted here.
`
`come to a point where Wt: can specify
`quantitatively the parameters that define
`quality. We would like to reach a similar
`objective in the digital case. Let us examine
`how those parameters were derived and
`how we reached the present objectives for
`the analog case.
`In the curly days of television, there were
`many laboratories such as Bell Telephone
`Laboratories. R.C.A. Laboratories and
`others around the world that carried out
`subjective tests to evaluate such parameters
`as signal-tortoise ratio and differential
`phase. They determined just how these
`parameters related to a given subjectivc
`quality such as just-perceptible distortion
`or annoying distortion. In other words.
`quantitative parameters were related to a
`subjective measure of performance. Once
`the subjective measures were. determined,
`a second phase tool: place. where objective
`test patterns were developed which could
`measure these quantitative parameters by
`means ofa vectorsoopc. waveform monitor,
`oscilloscope or some other piece of test
`equipment and relate them directly to
`picture quality based on the subjective test
`results. The use of objective test patterns
`eliminated the need for carrying out sub-
`jective testing every time one wished to
`evaluate the quality of u particular televi-
`sion system. In the development of the. ob-
`jective test patterns.
`the quantitative
`measures needed for each impairment.
`such as noise, were originally based on the
`subjective test results that had been ob-
`tained. As an example of this, Fig. 1 shows
`a block diagram of the experimental con-
`figuration usod by Barstow and Christo-
`pher1 to subjectively evaluate the effects of
`random noise on the analog television sig-
`nal. Figure 2 shows the results of these
`measurements where a subjective rating or
`quality was related toa quantitative value
`of signal-to-noise ratio. This type of sub-
`jective test result forms the basis for all of
`the current analog specifications.
`
`PMC Exhibit 2033
`Apple v. PMC
`IPR2016-00755
`Page 1
`
`

`

`.
`
`.
`
`.
`
`.
`
`trillith ONECTIMME
`
`llEflll'l’ElJ commoner!
`
`small!!! Illtfllml
`
`limflll W] M Wifllm
`
`. MFIIIIIIJ Pttfl‘flli Ml'l' SLIM
`It'll-II?
`
`.
`
`.llfil Pmfl'l'lli
`
`. NT Pentium:
`
`Fig. 2. Analog signal-to-noise measurements.
`
`IEIEH'IID Illll'ii WI'IIDE Ill IELW'I “I M:
`
`Fig. I. Eiperimental configuration used by Barstovt and Christopher for
`subjective evaluation of noise.
`
`-3 much worse
`
`In the case of digital television. one could
`carry out similar types ofsubjective testing.
`For example. in the analog-to-digital con-
`version of the signal in a pulse-code-rnod-
`ulation system, one could vary the number
`of bits per sample (related to the number
`of sampling levels) used to quantize the
`signal and evaluate a subjective quality
`associated with varying that particular
`parameter. One could also vary the sam-
`pling frequency and determine a subjective
`quality related to the amount of impair-
`ment occurring in the picture due to dif-
`ferent sampling frequencies. As in the an-
`alog case. where varying the amount of
`noise and comparing it to subjective quality
`allowed one to determine a suitable sig-
`nal-to-noise ratio. for a broadcast-quality
`signal, one could determine the number of
`bits per sample and the sampling frequen-
`cy. based on the subjective rating, that is
`required in order to provide a broadcast-
`quality signal.
`in both these cases a broadcast-quality
`signal was determined by picking some
`value of subjective grade, such as "just imo
`perceptible" (1.5 on the 6-point impair-
`ment scale — meaning that half the ob-
`servers can perceive the impairment and
`half cannot) and using that can measure of
`broadcast quality. Thus. just as analog
`broadcast quality was equated to a sub-
`jective grade of “just imperceptible” for a
`number of different types of impairment
`one can also similarly assess signal quality
`in the digital case. Table IV lists analog
`impairments that have been evaluated
`subjectively. The parameters are quite
`different, but the subjective test procedures
`could be quite similar. Subjective testing
`thus provides a basis for deriving specifi-
`cations ort picture quality for both analog
`and digital
`television systems. Further-
`more, subjective scales could provide a
`means of relating analog television systems
`to digital television systems. There are,
`however, a number of significant differ-
`ences in the digital case which makes the
`process more complicated than it was for
`the analog case.
`'
`
`being able to carry out a variety ofdil‘ferent
`types of signal processing without seriously
`impairing the signal. While this feature is
`very attractive from a design standpoint, it
`makes assessment of quality more difficult.
`for it leads to many more parameters and
`ones to evaluate than was encountered in
`the analog case.
`Table V lists typical sets of parameters
`that would be of interest to evaluate for
`different coding methods. If we consider an
`analog-to—digital-conversion process such
`as pulse code modulation, then such pa-
`rameters as sampling rate, number of bits
`per sample, companding law, clock jitter,
`and the description of the filtering used. are
`the kind ofparameters which are important
`in determining the quality of the recon-
`structed analog signal. When other types
`ofdigital encoding methods are used the set
`of parameters that must be evaluated will
`vary and be dependent on the type ofdigi-
`tal processing carried out. So, for example,
`if we were to use differential-pulse-eode
`modulation as the means of converting the
`analog television signal
`into a digitally
`encoded form. the parameters of interest
`would be different, (3.3., the particular
`prediction algorithm used in the differen-
`tial PCM coding, the number of bits per
`sample used in the feedback loop, and the
`number of previous samples used in pre-
`dicting the next value of the signal. Other
`coding methods such as transform coding
`require yet another set of parameters to be
`evaluated, as indicated in Table V.
`
`In the case of digital television. there are
`two classes of parameters to be evaluated
`which impact the quality of the picture.
`The first class of parameters relates to the
`conversion of the analog signal into digital
`form and the conversion of the digital sig-
`nal back to analog form, a type' of pro-
`cessing termed “source coding." Parame-
`ters associated with different methods of
`source coding that are to be evaluated are
`listed in Table V.
`i
`There are also impairments introduced
`into the picture after the signal is in digital
`form. They are typically called channel
`effects and are also listed in Table V. Such
`parameters are: random errors which occur
`on the bit stream, slips of the bit timing
`clock or jitter of the clock. burst errors. etc.
`These errors which are introduced into the
`bit stream after the television signal is in
`digital form will result in additional im-
`pairments appearing in the reconstructed
`analog signal and must also be evalu-
`ated.
`
`Correlated Impairments
`There is another complicating difference
`associated with the digital case, however,
`and that is that the nature of the impair-
`ment in the reconstructed analog signal,
`due to channel effects occurring on the bit
`stream, is related to the type of source
`coding that was used to convert the analog
`signal
`into digital
`form. Because the
`number and type of bit errors introduced
`into the bit stream cause different analog
`
`Table l. Subject gliding scales.
`Quality
`Impairment
`A-Exccllcnt
`Selmpcrceptible
`4-Perceptible but not annoying B-Good
`C-Fair
`3-Somewhat annoying
`D-Poor
`2-Severely annoying
`E~ Bad
`l-Unusa ble
`
`Comparison
`+2 much better
`+l better
`0 the same
`—l worse
`-2 much worse
`
`l-lmperceptible
`2—Just perceptible
`3~Dcfinitely perceptible
`but not disturbing
`4-Somewhat objectionable
`5- Definitely objectionable
`6sExtremcly objectionable
`
`1- Excellent
`I-Good
`JsFairly good
`
`4- Rather poor
`5- Poor
`é-Vcry poor
`
`+3 much better
`+2 better
`~l-l slightly better
`0 the some
`-I slightly worse
`«2 worse
`
`Evaluating Digital Parameters
`One of the desirable features of digital
`signal processing of the television signal is
`that there is a great deal of flexibility in
`154
`SMPTE Journal March .1978 Volume 87
`
`PMC Exhibit 2033
`Apple v. PMC
`IPR2016-00755
`Page 2
`
`

`

`Table II. Subjective test procedures.
`
`Reference
`Observers Category
`Number
`Grading Scale Type
`Number of Grades
`Test Pictures Number
`Viewing Con ditions:
`Ratio of viewing
`distance to picture
`height
`Peal: Luminance on the
`screen (ed/n1“)
`
`or.
`(c.c.l.n.. I963-l966)
`Non-Expert
`20-25
`Quality
`5
`4-8
`6
`
`50
`
`Not specified
`
`Contrast range of
`the picture
`$0.5
`Luminance of inactive
`tube screen (cdlm’)
`Luminance of backcloth l llluminant C
`(ed/m1)
`
`Table It continued.
`
`41—54
`
`.
`
`Analog case
`—".
`‘
`noise
`Additive independent
`Random
`Impulsive
`Pcriadic
`_
`Crosstth
`Linear distortion
`Field tit-rte
`Line time
`Short time
`Chrominancc/
`_
`.
`luminance
`Gal" & dfilal
`inequality
`Gain/frequency
`Nonlinear distortion
`Differential phase
`Differential gain
`Chro'min critterf
`luminance
`Intermodulation
`Luminance manna“
`Table III. COR-recommended sub!ective testing grocedum.
`distortion
`.
`ificationr
`S
`Viewing
`Viewing
`Chrom‘mm‘"
`GO-i ieldsis
`5Eiielas FE
`condition
`condition
`nonlinear
`l
`s
`toms
`s
`t
`‘ nat'on
`' t'
`d
`W Gain and phase
`a
`ratio of viewing distance to picture height
`6
`4 to 6
`distortion
`Synchronizing pulse
`nonlinearit:
`
`I55
`
`E.B.U.. 0.I.R.T..
`LP...
`(cc.
`uses—1966)
`
`Impairment Quality Comparison
`6
`6
`_'i
`5
`4-6
`
`fit“ -
`19534955]
`Non-Expert
`Approx. 200
`Quality
`6
`2-8
`6 -8
`
`U.S.A. (C.C.l.R..
`wee-1969)
`Expert
`>10
`Impairment
`'1r
`3-4
`4
`
`use. (c.c.i.R.,
`l966—l969)
`Non-Expert
`Approx. 20
`Comparison
`5
`6
`6
`
`Iumflt:
`5
`
`70
`
`170 (monochrome)
`34 (color)
`
`50
`
`Not specified
`
`2
`
`Approx. 0.5
`
`Table IV. Picture impairments.
`
`Reference
`Observers Category
`Namher
`Gfifiifigesrcgfgrflgs
`Test Pictures Number
`Viewing Conditions:
`Ratio of viewing
`distance to picture
`height
`Peal: Luminance on the
`screen (cdfrn’)
`
`Contrast range of
`the picture
`Luminance of inactive
`tube screen (cdlm')
`
`Fed. Rep. of Germany
`(C.C.I.R.. 1963-1966B}
`Non-Expert
`>10
`Qusality
`>5
`6
`
`Compsarison
`
`50
`
`Not specified
`50.5
`
`Digital case
`I
`_
`Sampling noise I
`Quamizafiufl noise
`Intcrsymbol
`Imemmncc
`Bit error rate
`Bit error time
`Distribution
`Bil Timing clock
`'iiier
`Phi,“ and
`_
`'
`amplitude hits
`Bil liming Slips
`Impulsive noise
`
`Japan
`(C.C.I.R.. 1963-19660
`and 1966-1969A)
`Non—Expert
`20-25
`Impaisrrrtent
`)3
`6-8
`
`Quality
`5
`
`Approx. 400
`(monochrome)
`74—84 (color)
`30,! 1 to 50/1
`Approx. 5
`(monochrome)
`0.7-2 (CDIDI')
`
`'
`
`b
`c
`
`1-
`g
`h
`
`peak screen luminance {ed/m2)
`ratio of inactive-tube (cutoff) luminance to
`peak luminance
`ratio of screen luminance displaying black level
`"1 ‘0'“?de dark f°°m ‘° “’3‘
`corresponding to peak white
`.
`.
`.
`alt luminance
`icture monitor to icture
`ratio of luminance of background behind
`p
`F
`De
`other mom illumination
`chromaticin of background
`ratio ofsolid angle subtended by that part of
`the background which satisfies this
`sgcification to that subtended b! the picture
`
`1'0 1: lo
`0.02
`
`70 :e If)
`0.02
`
`approx. 001
`
`approx. 0.1
`
`approx. 0.!5
`
`low
`while
`29
`
`10“,
`as,
`u
`
`impairments for different types of source.
`coding. there is an interrelation that must
`be considered when evaluating the quality.r
`of a digital system.
`Furthermore. the impairments intro-
`duocd in the analog-to-dlgital conversion
`process are correlated with the television
`signal and are strongly dependent on the
`
`characteristics of the picture material
`being digitized. The correlated nature of
`the impairments can result in some pecu-
`liar subjective effects. For example. in the
`case of pulse-code modulation, when too
`few bits per sample are used, false edges or
`contours appear in the picture, demon-
`strating a type ofnoise that is not normally
`
`seen in analog television. {in the analog
`television cam, most of the impairments are
`uncorrelated with the television signal and
`-
`-
`produce a more random type of nurse 1I'I'I.-
`.
`.
`.
`pairmcm in the picture, but this is not
`necessarily true for the digital case.)
`There are other differences in the digital
`television case which must also be consid-
`cred. Because television signals can be
`stored in digital
`form.
`interframe or
`framc.to.frame coding or the signal
`is
`possible, and devices for accomplishing this
`have already been developed by certain
`manufacturers. This type of source coding
`required consideration of frame-lo-frame
`subjective effects which must be tested with
`picture material that involves motion be-
`tween one frame and the next. This in turn
`leads into more complex subjective testing
`proceduresthan have been used in the an—
`slog television case.
`
`Gai'almg: Quality Assessment oth‘gflal Television Signals
`
`PMC Exhibit 2033
`Apple v. PMC
`IPR2016-00755
`Page 3
`
`

`

`Table V. Parameters in digital impairment
`testing.
`
`PCM
`
`Transform
`
`parameters. the impairments produced,
`and the difficulties of subjective testing.
`We have found that in general the digital
`case is a more complex case to develop
`standards for because:
`I. The impairments are more varied as
`they are correlated with the television sig-
`nal and are a function of both the source
`coding and channel effects”
`2. Generally more parameters need to
`be subjectively tested to fully evaluate
`specific coding methods, and there are
`potentially a greater number of coding
`methods which may be useful and practical
`to consider.
`3. Frame-to-framc signal processing is
`quite feasible with digital
`techniques.
`which means that subjective tests taking
`
`line frequency; and short dashes showI. unlocked.)
`
`Phase/amp. hits
`Bit clock slips
`
`Clock jitter
`
`Source Coding Channel
`Filter
`Random error
`parameters
`rate
`Sampling rate
`Imp. error rate
`No. of bits/
`Burst duration
`sample
`Compunding
`law
`Clock jitter
`Prediction
`algorithm
`Companding law
`No. of bits/sample in
`feedback loop
`Loop filter parameters
`No. of coefficients used
`Camper-ding low per
`coefficient
`No. of bits/coefficient
`Filter Eramelers
`
`Table VI. Quality-assessment procedure.
`c Determine source coding technique to be
`evaluated and channel conditions to be con-
`sidcrcd.
`I Determine key parameters which are to be
`tested. and minimum range over which each
`parameter should be varied. Minimize number
`of combinations cfdiffcrent parameters. which
`must be tested.
`fol-
`a Carry out subjective impairment test.
`lowing internationally accepted practices for
`subjecting testing.
`0 Compare performance with other digital sys~
`terns using subjective grading scale as common
`measure of performance.
`- Develop objective test signals and procedures
`which permit evaluation of performance of
`given digital system. using quantitative mea-
`sures on the television signal.
`
`motion into account must also be consid-
`erect.
`4. After the signal is digitized and en-
`coded. the primary offeet offurther sources
`of degradation is to increase the bit error
`rate and possibly change the error pat-
`tern.
`
`5. If error coding is employed to reduce
`the bit error rate on the digital bit stream
`then the error coding process used also will
`affect how bit errors will appear in the re-
`constructcd analog signals; thus the im-
`pairments are a function not only of the
`analog-to-digital coding process, but any
`signal processing carried out on the signal
`after it has been converted into digital
`form.
`6. Chrominanee/lurninance impair-
`ments depend on whether the digitizing
`and encoding is done on composite video or
`on components.
`
`Overcoming the Complications
`While the digital case is more compli-
`cated than the analog case. i believe it can
`be handled quite successfully with some
`intelligent planning. For each analog-to-
`digital coding process one can specify a
`particular set of parameters such as num-
`ber of bits per sample. sampling rate. etc.
`that have to be evaluated. impairment or
`quality testing following recommended test
`procedures could be carried out to relate
`each of these parameters to an equivalent
`subjective quality grade. [f a subjective
`quality of “just imperceptible" is selected.
`then through a series ofsubjective tests the
`
`Composite us Component Encoding
`In the source coding area there is yet
`another basic choice to be made between
`coding methods. It involves choosing be-
`tween direct analog-to-digital conversion
`of the composite color television signal and
`separate encoding of the components of the
`television signal (the luminance signal and
`the two chrominance signals). The im-
`pairments perceived when using these dif-
`ferent analog-to-digital coding processes
`are quite different — especially if there is
`interaction between the chrominance and
`luminance signals.
`
`Summarizing Digital Impairments
`At this point it may be useful to sum-
`marize what we. have found about digital
`
`
`
`$UBJEO'IWEGMDE
`
`
`
`WIJEG'INEom:
`
`NIJIIBEB 0F BITS
`Fig. 3. Example of measured results for PCM coding: variation in objective
`impairment at different numbers of bits per sample, using dither. (Vertical
`bars show variation in grade for different picture sources, and open circles
`denote mean grades for all picture sources: horizontal line at Subjective
`Grade 125 indicates the mean score for an unquantized picture.)
`
`[56
`
`SMPTE Journal March 1978 Volume 87
`
`NUMBER OF INS
`
`(DI
`
`Fig. 4. Effect of different sampling frequencies (PCM coding) on critical
`picture color bars and on noncritical pictures taken oil-air with a receiver.
`(A) Color bars; no dither. (B) Color bars; with dither. (C) Off-air pictures:
`no dither. (D) Off-air picture: with dither. (Solid lines denote a sampling
`frequencyf. of three times color subcarrier; long dashes show]. = 851 X
`
`PMC Exhibit 2033
`Apple v. PMC
`IPR2016-00755
`Page 4
`
`

`

`151'
`
`determine the correct objective mt signals.
`the procedure for arriving at these test
`signals can follow along similar lines to
`those used to arrive at analog test signals.
`Table VI outlines what I believe to be a
`quality assessment procedure for the digital
`case that can be followed to arrive at
`quality objectives; the procedure is similar
`to that used originally to arrive at the en-
`alog quality objectives.
`Some examples of subjective tests that
`have already been carried out successfully
`on different digital coding methods are il-
`lustrated in Figs. 3, 4 and 5. This data.
`provided by the British Broadcasting
`Corporation?-3 involved testing the ana-
`log-todigital encoding of the PAL televi—
`sion signal. In Fig. 3, the number of bits per
`sample was varied and related to a sub-
`jective quality. In Fig. 4. different sampling
`frequencies were evaluated and related to
`a subjective quality as a function of dif-
`ferent number of bits per sample. [n Fig. 5.
`the effects of timing jitter were related to
`subjective quality for a PCM signal. The
`test procedure followed was similar to that
`recommended by the CCIR (Table III).
`Five or six different pictures were used as
`the subject material. The results, as illus-
`trated in these figures. show how quanti-
`tative values can be determined for the set
`of parameters associated with PCM en-
`coding of the signal in order to obtain a
`given specified subjective quality.
`Figure 6 illustrates how the subjective
`rating scale can be used to compare dif-
`ferent analog-to-digital coding methods. in
`this figure both pulse—code modulation and
`differential pulsevcodc modulation are re-
`latod to a given subjective quality at a given
`bit rate. The subjective quality is shown for
`different numbers of bits per sample and
`different sampling rates for the DPCM
`method. The PCM performance. with 8
`Golding: Quality Assessment oth‘gr‘rat Television Signals
`
`
`
`WMEO'I'WEflMDE
`
`DJ
`DJ
`0.! 1.0
`n.u.s. “FLUIDS BF JI‘I'IEII, N3
`
`1.5
`
`3.0
`
`Fig. 5. Measured results for timing jitter. The impairment is caused by
`white Gaussian jitter on a display of 100% colorbars. Circles. triangles
`and crosses respectively denote maximum jitter frequencies of 20 kHz.
`600 kHz and 6 MHz.
`
`correct value ofthc parameters to achieve
`this subjective quality could be determined
`by a series of impairment or quality tests.
`While the number of parameters may be
`large. by some careful planning and some
`preliminary screening the range of each or
`the parameters that have to be tested can
`be made relatively small. fora given sub-
`jectivc quality. Knowing the nature orthe
`impairments introduced by the particular
`analog-to-digital coding method one could
`select a reasonably small set of picture
`materials that are effective at showing up
`these impairments and that could be used
`in carrying out the subjective tests. For
`each source-coding method the effects of
`different types of error patterns on the
`digitally encoded signal could be evaluated.
`in a very systematic way. the set of pa-
`rameters, which give a specified subjective
`quality, could be determined for each an-
`alog-to-digital coding method in the pros-
`enee of different bit error rates and bit
`error patterns that might be encountered
`in practice. The subjective quality scale
`would then provide the means of compar-
`ing different coding methods with regard
`to the bit rate and bit error rate needed to
`provide a specified subjective quality.
`
`Objective Test Signals
`Once this subjective test data had been
`compiled one could dispense with the fre
`quent subjective tests (as has been done
`with analog television) and look into ob-
`jective test signals which could be used to
`evaluate a given subjective performance.
`For example. the pulse and bar pattern
`commonly used in the analog case for
`measuring the short time distortion could
`be used in the digital case to measure cdgc
`busyness and background noise in constant
`gray level areas of the picture. For the
`PCM analog-to-digital coding technique
`a ramp signal would be quite useful in de-
`tecting contours and quantization noise.
`While much more work must be done to
`
`HITS PER SAMPLE
`X3
`
`IX
`
`
`
`l‘IIPAIIIIENTGRADE
`
`7|!
`'0
`BIT-RITE. ll BI'US
`Fig. 6. A digital subjective test: impairment vs bit rate for DPCM coding
`of PAL signals with sampling frequencies of 2fjrand 3f". Crosses show
`results for DPCM and circles for PCM.
`
`bits per sample and a sampling rate of
`twice the color subcarrier frequency is also
`plotted on the same graph. These test re-
`sults. provided by the BBC. illustrate how
`a comparison can be made. As previously
`mentioned, different digital coding meth-
`ods could also be compared to analog sys-
`tem performance by using one of the sub-
`jective grading scales such as the impair-
`ment grading scale. as a common basis of
`comparison even though the nature of the
`impairments may be different. One must
`be careful that the test results used. how-
`ever. apply to a sufficiently large amount
`of picture material to make the comparison
`valid.
`While the procedure to get to the quality
`objectives for digital
`television systems
`appears to involve a considerable effort
`{possibly a lot more effort than was origi-
`nally needed to arrive at the analog televi-
`sion quality objectives). it is expected that
`this procedure would be carried out over a
`considerable period of time. Furthermore.
`with the type of impairments being much
`more varied in the digital case than in the
`analog case. it would appear that the sub-
`jective testing procedure would be the only
`way of getting a common measure of
`quality which could be used to determine
`the correct parameters for different digital
`encoding methods. different bit error rates
`and different digital impairments which
`might occur.
`References
`I. J. M. Barstow and II. N. Christopher. “The Mea-
`surement of Random Video Interference to Mono-
`chrome and Color Television Pictures." AIEE
`Trans. No.63. NOW. [962.
`. V. (J. Devereux."PulscCode Modulation of Video
`Signals: Subjective Study of Coding Parameters."
`BBC Research Report No. 1971/40.
`3. "Pulse Code Modulation of Video Signals: B-Bit
`Coder and Decoder.“ BBC Research Report No.
`IMO/25.
`
`PMC Exhibit 2033
`Apple v. PMC
`IPR2016-00755
`Page 5
`
`