US006272176B1
`(10) Patent No.:
`(12) United States Patent
`US 6,272,176 B1
`Srinivasan
`(45) Date of Patent:
`Aug.7, 2001
`
`
`(54) BROADCAST ENCODING SYSTEM AND
`METHOD
`
`WO 94/11989
`
`ercsecssecssusseeeseeee HO4N/5/76
`5/1994 (WO)
`OTHER PUBLICATIONS
`
`(75)
`
`Inventor: Venugopal Srinivasan, Palm Harbor,
`FL (US)
`
`International Search Report, dated Aug. 27, 1999, Applica-
`tion No. PCT/US98/23558.
`
`(73) Assignee: Nielsen Media Research, Inc.,
`Schaumburg, IL (US)
`
`“Digital Audio Watermarking,” Audio Media, Jan./Feb.
`1998, pp. 56, 57, 59 and 61.
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`US.C, 154(b) by 0 days.
`
`(21) Appl. No.: 09/116,397
`
`Primary Examiner—Chi Pham
`Assistant Examiner—Emmanuel Bayard
`(74) Attorney, Agent,
`or Firm—Marshall, O’Toole,
`Gerstein, Murray & Borun
`
`(57)
`
`ABSTRACT
`
`(22) Tiled:
`
`Jul. 16, 1998
`
`7
`
`° a “a seesesecueneeuesnesesecaceneeseeneeneueeeees.seseee,deonss“
`(52) I 75/240; 375/253
`(58) Field of Search .....0..cccccceeeeeeeee 375/240, 245,
`375/246, 253, 254, 132; 714/752, 755,
`758; 704/203, 212, 226
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`3,845,391
`4,025,851
`4,313,197
`4,703,476
`4,931,871
`4945 412
`4.972471
`
`10/1974 Crosby deca cee cteeceeceececeeeteeeteeeees 325/64
`S/1977 Haselwood et al.
`sess 325/31
`
`1/1982 Maxemchuk ......
`370/110
`
`10/1987 Howard wo.ccicsssceeseeceees 370/76
`
`..
`.. 358/142
`6/1990 Kramer
`
`7/1990 Kramer
`358/142
`
`1/1990 Gross et al. ecssssssessesesnen 380/3
`;
`;
`(List continued on next page.)
`FOREIGN PATENT DOCUMENTS
`
`An encoderis arranged to add a binary code bit to block of
`a signal by selecting, within the block, (i) a reference
`frequencywithin the predeterminedsignal bandwidth, (ii) a
`first code frequency havinga first predetermined offset from
`the reference frequency, and (iii) a second code frequency
`having a second predetermined offset from the reference
`frequency. The spectral amplitude of the signal at the first
`code frequency is increased so as to render the spectral
`amplitude at the first code frequency a maximum in its
`neighborhood of frequencies and is decreased at the second
`code frequency so as to render the spectral amplitude at the
`second code frequency a minimum in its neighborhood of
`frequencies. Alternatively, the portion of the signal at one of
`the first and second code frequencies whose spectral ampli-
`:
`:
`op
`:
`tude is smaller may be designated as a modifiable signal
`“
`rn
`:
`:
`componentsuchthat, in order to indicate the binary bit, the
`phase of the modifiable signal componentis changedso that
`this phase differs within a predetermined amount from the
`phase of the reference signal component. Asa still further
`alternative,the spectral amplitude of the first code frequency
`may be swapped with a spectral amplitude of a frequency
`having a maximum amplitude in the first neighborhood of
`A/1994 (DE) weeeseeseeseccseceeeeeteeeeeeee G101./5/00
`43 16 297
`
`
`0 243 561=11/1987 (EP)... w HO4Q/1/457 frequencies and the spectral amplitude of the second code
`
`
`0 535 893
`4/1993 (EP)...
`. GO6T/15/332
`frequency may be swapped with a spectral amplitude of a
`
`2 170 080
`TINL986
`(GB)
`vesessssessseeseeeeeeeeeesees HO4N/5/04
`frequency having a minimum amplitude in the second
`2BOME BS (GP cosmos BMH.
`
`7 059030
`9009213
`WO 89/09985
`
`3/1995 (JP).
`1/1997 CIP).
`10/1989 (WO) wesesccceseeeeesenes G10L/5/00
`
`shorotoemAdsmy he ag
`
`39 Claims, 9 Drawing Sheets
`
`60
`
`eye
`
`38
`
`Sony Exhibit 1005
`Sony Exhibit 1005
`Sony v. MZ Audio
`Sony v. MZ Audio
`
`

`

`US 6,272,176 B1
`
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5/1997 Dougherty oo...eee 348/486
`5,629,739
`11/1997 Lee et ale coecceeecceeeseeseeeeeeeeee 375/216
`5,687,191
`.. 380/6
`6/1998 Jensenetal. ..
`5,764,763
`5,113,437 *
`5/1992 Best et al. severe 380/3
`
`
`7/1998 Fardeau et al. ov. 455/2
`5,787,334
`5,319,735
`.
`6/1994 Preuss et al.
`395/2.14
`
`
`5,450,490 5,822,360—10/1998 Lee et al. eecssssecseessnseeeseeesen 375/2009/1995 Jensenet al. ossssssesseseeeerenn 380/6
`11/1996 Fardeau et al. oe 455/2
`5,574,962
`5,963,909 * 10/1999 Warren et al. v.ccceccscecssseeeeee 705/1
`
`5,579,124
`11/1996 Aijala etal. .....
`. 386/96
`5,581,800
`12/1996 Fardeau et al. occ 455/2
`
`* cited by examiner
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 1 of 9
`
`US 6,272,176 B1
`
`9%
`
`LINNAWOHOLVLVG
`
`vl
`
`YAAIFOTY
`
`Waao5ga:Laaaoona YELLINSNVELL
`
`91
`
`ol
`
`LJUNOIS
`
`
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 2 of 9
`
`US 6,272,176 B1
`
`Input audio block v(t) with 512 samples
`
`Multiply by window function w(t)
`
`Compute Fast Fourier Transform 3{w(t)v(t)}
`
`:
`
`=
`
`Select frequencies f; and f, for modulation
`
`Analyze neighborhoods
`
`;
`
`Data Bit Value
`
`Boost and Attenuate to get 3, {v(t}w(1)}
`
`Compute Inverse Transform 37" {v(1) w(t)}
`
`Vo(t) = v(t) + (SQ {v(t)w(t)} — v(t) (4)
`
`Coded output:
`
`FIGURE 2
`
`40
`
`42
`
`44
`
`46
`
`48
`
`56
`
`62
`
`64
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 3 of 9
`
`US 6,272,176 B1
`
`50
`
`52
`
`54
`
`5.6
`
`Lu“
`- 42
`
`28
`
`5O
`
`w =
`
`<
`
`<£
`
`14
`|oa
`
`uy
`Ww 0.0
`Vi
`45 47 4951 53 5557 5961 6365 67 69 71:73 75
`FREQUENCY INDEX
`
`FIGURE 3
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 4 of 9
`
`US 6,272,176 B1
`
`AMPLITUDE 128
`
`192
`
`256 320
`
`384 448
`
`-O0.3
`
`SAMPLE NUMBER
`
`FIGURE 4
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 5 of 9
`
`US 6,272,176 B1
`
`60
`
`yy
`
`38
`
`FIGURE 5
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 6 of 9
`
`US 6,272,176 B1
`
`
`
`SPECTRALAMPLITUDE
`
`9.00
`
`6.75
`
`450
`
`2.25
`
`70
`
`0.00
`45 47 49 51 53 5557 5961 63 65 67 69 71 7375
`
`FREQUENCY INDEX
`
`FIGURE 6
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 7 of 9
`
`US 6,272,176 B1
`
`Unencoded 512 sample block
`
`synchronization
`
`FIGURE 7A
`
`66
`
`70
`\
`
`66.
`
`*
`
`”
`
`triple” tone
`
`
`Last PN15 Data of Previous Block
`
`FIGURE 7B
`
`

`

`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 8 of 9
`
`US 6,272,176 B1
`
`400
`
`102
`
`104
`
`106
`
`Update SIS[0], status array index p = 1
`
`108
`
`Enter Incremental FFT mode
`
`110
`
`Update SIS[p] , p = p+!
`
`
`
`
`
`
`FIGURE 8
`
`i12
`
`14
`
`116
`
`18
`
`120
`
`122
`
`Read Data in SIS[p}.OP
`
`124
`
`126
`
`

`

`
`orpnedojeuy/|oipne
`
`
`[elidiq~“K------------4b------r---74----nduiweansigindjnoweansyg!'AIOaplA
`
`
`
`
`
`
`
`
`juauny.indurIndjnoweonsyigolpny[ediqweansig
`regia.leusiq
`80¢
`
`
`juan;“44°91<-,Japoouyoapia,Iieusig
`
`
`|_GAejapiimJapooug
`IZCO?
`omm6owesomeoeleelllsiaswe5soe8mole|OoplARIdIg
`
`
`
`d/volpny
`
`
`
`———jndu
`
`dojeuy
`
` Iyndul;oapiadojeuy
`
`U.S. Patent
`
`Aug.7, 2001
`
`Sheet 9 of 9
`
`US 6,272,176 BI
`
`
`
`OopiAdojvuy
`
`ndjno
`
`indjno
`
`vod
`
`
`
`OsplA/OIpny~«.--L
`
`__-boLeoapiA/olpny
`
`pOC
`
`
`
`indjnoorpnyKO--p---------"
`
`90¢
`
`63YNODIs
`
`
`
`
`
`roeckn------1777~Indurorpny
`
`
`
`
`

`

`US 6,272,176 B1
`
`1
`BROADCAST ENCODING SYSTEM AND
`METHOD
`
`TECHNICAL FIELD OF THE INVENTION
`
`The present invention relates to a system and method for
`adding an inaudible code to an audio signal and subse-
`quently retrieving that code. Such a code may be used, for
`example, in an audience measurement application in order to
`identify a broadcast program.
`
`10
`
`2
`No. 5,450,490, teach an arrangement for adding a code at a
`fixed set of frequencies and using one of two masking
`signals, where the choice of masking signal is made on the
`basis of a frequency analysis of the audio signal to whichthe
`code is to be added. Jensen et al. do not teach a coding
`arrangement in which the code frequencies vary from block
`to block. The intensity of the code inserted by Jensen ct al.
`is a predetermined fraction of a measured value (e.g., 30 dB
`down from peak intensity) rather than comprising relative
`maxima or minima.
`
`BACKGROUND OF THE INVENTION
`
`30
`
`Moreover, Preusset al., in U.S. Pat. No. 5,319,735, teach
`a multi-band audio encoding arrangement in which a spread
`There are manyarrangements for adding an ancillary code
`spectrum codeis inserted in recorded music atafixed ratio
`to a signal in such a waythat the added codeis not noticed.
`to the input signal intensity (code-to-music ratio) that is
`15
`It is well knownin television broadcasting, for example, ta
`preferably 19 dB. Lee et al., in U.S. Pat. No. 5,687,191,
`hide such ancillary codes in non-viewable portions of video
`teach an audio coding arrangement suitable for use with
`by inserting them into either the video’s vertical blanking
`digitized audio signals in which the code intensity is made
`interval or horizontal retrace interval. An exemplary system
`to match the input signal by calculating a signal-to-mask
`which hides codes in non-viewable portions of video is
`ratio in each of several frequency bands and bytheninsert-
`referred to as “AMOL”and is taught
`in U.S. Pat. No.
`ing the code at an intensity that is a predetermined ratio of
`4,025,851. This system is used by the assignee of this
`the audio input in that band. As reported in this patent, Lee
`application for monitoring broadcasts oftelevision program-
`et al. have also described a method of embedding digital
`ming as well as the times of such broadcasts.
`information in a digital waveform in pending U.S. applica-
`tion Ser. No. 08/524,132.
`Other known video encoding systems have sought to bury
`the ancillary code in a portion of a television signal’s
`It will be recognized that, because ancillary codes are
`transmission bandwidth that otherwise carries little signal
`preferably inserted at low intensities in order to prevent the
`energy. An example of such a system is disclosed by
`code from distracting a listener of program audio, such
`Dougherty in U.S. Pat. No. 5, 629,739, whichis assigned to
`codes may be vulnerable to various signal processing opera-
`the assignee of the present application.
`tions. For example, although Lee et al. discuss digitized
`audio signals, it maybe noted that manyofthe earlier known
`Other methods and systems add ancillary codes to audio
`approaches to encoding a broadcast audio signal are not
`signals for the purpose of identifying the signals and,
`compatible with current and proposed digital audio
`perhaps, for tracing their courses through signal distribution
`standards, particularly those employing signal compression
`systems. Such arrangements have the obvious advantage of
`methods that may reduce the signal’s dynamic range (and
`being applicable not only to television, but also to radio
`thereby delete a low level code) or that otherwise may
`broadcasts and to pre-recorded music. Moreover, ancillary
`damage an ancillary code. In this regard, it is particularly
`codes which are added to audio signals may be reproduced
`important for an ancillary code to survive compression and
`in the audio signal output by a speaker. Accordingly, these
`subsequent de-compression by the AC-3 algorithm or by one
`arrangements offer the possibility of non-intrusively inter-
`of the algorithms recommended in the ISOMEC 11172
`cepting and decoding the codes with equipment that has
`MPEGstandard, which is expected to be widely used in
`microphones as inputs. In particular,
`these arrangements
`future digital television broadcasting systems.
`provide an approach to measuring broadcast audiences by
`the use of portable metering equipmentcarried by panelists.
`The present invention is arranged to solve one or more of
`the above noted problems.
`In the field of encoding audio signals for broadcast
`audience measurement purposes, Crosby, in U.S. Pat. No.
`SUMMARYOF THE INVENTION
`3,845,391, teaches an audio encoding approach in whichthe
`code is inserted in a narrow frequency“notch” from which
`the original audio signal is deleted. The notch is made at a
`fixed predetermined frequency(e.g., 40 Hz). This approach
`led to codes that were audible whenthe original audio signal
`containing the code was of low intensity.
`A series of improvements followed the Crosby patent.
`‘Thus, Howard, in U.S. Pat. No. 4,703,476, teaches the use
`of two separate notch frequencies for the mark and the space
`portions of a code signal. Kramer, in U.S. Pat. No. 4,931,871
`and in US. Pat. No. 4,945,412 teaches, inter alia, using a
`code signal having an amplitudethat tracks the amplitude of
`the audio signal to which the code is added.
`Broadcast audience measurement systems in which pan-
`elists are expected to carry microphone-equipped audio
`monitoring devices that can pick up and store inaudible
`codes broadcast in an audio signal are also known. For
`example, Aijalla et al., in WO 94/11989 and in US. Pat. No.
`5,579,124, describe an arrangement in which spread spec-
`trum techniques are used to add a code to an audio signal so
`that the codeis either not perceptible, or can be heard only
`as low level “static” noise. Also, Jensen et al., in U.S. Pat.
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`invention, a
`According to one aspect of the present
`method for adding a binary code bit to a block of a signal
`varying within a predetermined signal bandwidth compris-
`ing the following steps: a) selecting a reference frequency
`within the predetermined signal bandwidth, and associating
`therewith both a first code frequency having a first prede-
`termined offset from the reference trequency and a second
`code frequency having a second predetermined offset from
`the reference frequency; b) measuring the spectral power of
`the signal in a first neighborhood of frequencies extending
`about the first code frequency and in a second neighborhood
`of frequencics extending about the sccond code frequency;
`c) increasing the spectral powerat thefirst code frequency
`so as to renderthe spectral powerat the first code frequency
`a maximum in the first neighborhood of frequencies; and d)
`decreasing the spectral powerat the second code frequency
`so as to render the spectral power at
`the second code
`frequency a minimum in the second neighborhood offre-
`quencies.
`According to another aspect of the present invention, a
`method involves adding a binary code bit to a block of a
`
`

`

`US 6,272,176 B1
`
`10
`
`15
`
`5
`
`30
`
`35
`
`40
`
`3
`signal having a spectral amplitude and a phase, both the
`spectral amplitude and the phase vary within a predeter-
`mined signal bandwidth. The method comprises the follow-
`ing steps: a) selecting, within the block, (i) a reference
`frequency within the predetermined signal bandwidth,(ii) a
`first code frequency havinga first predetermined offset from
`the reference frequency, and (iii) a second code frequency
`having a second predetermined offset from the reference
`frequency; b) comparing the spectral amplitude ofthe signal
`nearthe first code frequencyto the spectral amplitude of the
`signal near the second code frequency; c) selecting a portion
`of the signal at one of the first and second code frequencies
`at which the corresponding spectral amplitude is smaller to
`be a modifiable signal component, and selecting a portion of
`the signal at the other of the first and second code frequen-
`cies to be a reference signal component; and d) selectively
`changing the phase of the modifiable signal component so
`that it differs by no more than a predetermined amount from
`the phase of the reference signal component.
`Accordingto still another aspect of the present invention,
`a method involves the reading of a digitally encoded mes-
`sage transmitted with a signal having a time-varying inten-
`sity. The signal is characterized by a signal bandwidth, and
`the digitally encoded message comprises a plurality of
`binary bits. The method comprises the following steps: a)
`selecting a reference frequency within the signal bandwidth;
`b) selecting a first code frequency at a first predetermined
`frequency offset from the reference frequency andselecting
`a second code frequency at a second predetermined fre-
`quency offset from the reference frequency; and, c) finding
`which one of the first and second code frequencies has a
`spectral amplitude associated therewith that is a maximum
`within a corresponding frequency neighborhoodandfinding
`which one of the first and second code frequencies has a
`spectral amplitude associated therewith that is a minimum
`within a corresponding frequency neighborhoodin order to
`thereby determine a value of a received one of the binary
`bits.
`
`4
`the first code frequency and in a second neighborhood of
`frequencies extending about the second code frequency. The
`bit inserter is arranged to insert the binary bit by increasing
`the spectral amplitude at the first code frequency so as to
`render the spectral amplitude at the first code frequency a
`maximum in the first neighborhood of frequencies and by
`decreasing the spectral amplitude at the second code fre-
`quency so as to render the spectral amplitude at the second
`code frequency a minimumin the second neighborhood of
`frequencies.
`Accordingto a still further aspect of the present invention,
`an encoderis arranged to add a binarybit of a codec to a block
`of a signal having a spectral amplitude and a phase. Both the
`spectral amplitude and the phase vary within a predeter-
`mined signal bandwidth. The encoder comprises a selector,
`a detector, a comparitor, and a bit inscrter. The sclector is
`arrangedto select, within the block,(i) a reference frequency
`within the predetermined signal bandwidth,(ii) a first code
`frequency having a first predetermined offset from the
`reference frequency, and (iii) a second code frequency
`having a second predetermined offset from the reference
`frequency. The detector is arranged to detect the spectral
`amplitude of the signal near the first code frequency and near
`the second code frequency. The selector is arranged to select
`the portion of the signal at one of the first and second code
`frequencies at which the corresponding spectral amplitudeis
`smaller to be a modifiable signal component, and to select
`the portion of the signal at the other of the first and second
`code frequencies to be a reference signal component. ‘lhe bit
`inserter is arranged to insert the binary bit by selectively
`changing the phase of the modifiable signal component so
`thatit differs by no more than a predetermined amount from
`the phase of the reference signal component.
`According to yet a further aspect of the present invention,
`a decoder, whichis arranged to decode a binarybit of a code
`from a block of a signal transmitted with a time-varying
`intensity, comprises a selector, a detector, and a bit finder.
`‘The selector is arranged to select, within the block, (i) a
`reference frequency within the signal bandwidth, (ii) a first
`code frequency at a first predetermined frequency offset
`According to yet another aspect of the present invention,
`from the reference frequency, and (iti) a second code fre-
`a method involves the reading of a digitally encoded mes-
`quency at a second predetermined frequencyoffset from the
`sage transmitted with a signal having a spectral amplitude
`reference frequency. The detector is arranged to detect a
`and a phasc. The signal
`is characterized by a signal
`spectral amplitude within respective predetermined fre-
`bandwidth, and the message comprises a plurality of binary
`bits. The method comprises the steps of: a) selecting a
`quency neighborhoods of the first and the second code
`reference frequency within the signal bandwidth; b) select-
`frequencies. The bit finder is arranged to find the binary bit
`
`ingafirst code frequency ata first predetermined frequency when one of the first and second code frequencies has a
`offset from the reference frequency and selecting a second
`spectral amplitude associated therewith that is a maximum
`code frequency at a second predetermined frequency offset
`within its respective neighborhood and the other of the first
`from the reference frequency; c) determining the phase of
`and second code frequencies has a spectral amplitude asso-
`the signal within respective predetermined frequency neigh-
`ciated therewith that is a minimum within its respective
`borhoodsofthe first and the second code frequencies; and d)
`neighborhood.
`determining if the phase at the first code frequencyis within
`According to another aspect of the present invention, a
`a predetermined value of the phase at
`the second code
`decoderis arranged to decode a binary bit of a code from a
`frequency and thereby determining a valueof a received one
`block of a signal transmitted with a time-varying intensity.
`of the binary bits.
`The decoder comprisesa selector, a detector, and a bit finder.
`The selector is arranged to select, within the block, (i) a
`According to a further aspect of the present invention, an
`reference frequency within the signal bandwidth, (ii) a first
`encoder, which is arranged to add a binary bit of a code to
`a block of a signal having an intensity varying within a
`code frequency at a first predetermined frequency offset
`from the reference frequency, and (iii) a second code fre-
`predetermined signal bandwidth, comprises a selector, a
`detector, and a bit inserter. The selectoris arrangedto select,
`quency at a second predetermined frequencyoffset from the
`within the block, (i) a reference frequency within the pre-
`reference frequency. The detector is arranged lo detect the
`phase of the signal within respective predetermined fre-
`determined signal bandwidth, (ii) a first code frequency
`having a first predetermined offset from the reference
`quency neighborhoods of the first and the second code
`frequency, and (iii) a second code frequency having a second
`frequencies. The bit finder is arranged to find the binary bit
`predetermined offset from the reference frequency. The
`when the phase at
`the first code frequency is within a
`detector is arranged to detect a spectral amplitude of the
`predetermined value of the phase at the second code fre-
`quency.
`signal in a first neighborhoodof frequencies extending about
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`US 6,272,176 B1
`
`5
`According to still another aspect of the present invention,
`an encoding arrangement encodes a signal with a code. The
`signal has a video portion and an audio portion. The encod-
`ing arrangement comprises an encoder and a compensator.
`The encoder ts arranged to encode oneof the portions of the
`signal. The compensator is arranged to compensate for any
`relative delay between the video portion and the audio
`portion caused by the encoder.
`According to yet another aspect of the present invention,
`a method of reading a data element from a received signal
`comprising the following steps: a) computing a Fourier
`Transform of a first block of n samples of the reccived
`signal; b) testing the first block for the data element; c)
`setting an array element SIS[a] of an SIS array to a prede-
`termined value if the data elementis foundin the first block;
`d) updating the Fouricr Transform of the first block of n
`samples for a second block of n samples of the received
`signal, wherein the second block differs from the first block
`by k samples, and wherein k<n; e) testing the second block
`for the data element; and f) setting an array element SIS[a+
`1] of the SIS array to the predetermined value if the data
`element is found in the first block.
`
`6
`FIG. 6 is a spectral plot of a “triple tone” audio block
`which formsthe first block of a preterred synchronization
`sequence, where the thin line of the plot is the spectrum of
`the original audio signal and the thick line of the plotis the
`spectrum of the modulated signal;
`FIG. 7a schematically depicts an arrangement of synchro-
`nization and information blocks usable to form a complete
`code message;
`FIG. 7b schematically depicts further details of the syn-
`chronization block shown in FIG. 7a;
`FIG. 8 is a flow chart depicting steps performed by a
`decoder of the system shown in FIG. 1; and,
`FIG. 9 illustrates an encoding arrangementin which audio
`encoding delays are compensated in the video data stream.
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`10
`
`15
`
`Audio signals are usually digitized at sampling rates that
`range between thirty-two kHz and forty-eight kHz. For
`example, a sampling rate of 44.1 kHz is commonly used
`during the digital recording of music. However, digital
`According to a further aspect of the present invention, a
`television (“DTV”)is likely to use a forty eight kHz sam-
`method for adding a binary code bit to a block of a signal
`pling rate. Besides the sampling rate, another parameter of
`varying within a predetermined signal bandwidth comprises
`interest in digitizing an audio signal is the numberof binary
`the following steps: a) selecting a
`reference frequency
`bits used to represent the audio signal at each of the instants
`within the predetermined signal bandwidth, and associating
`whenit is sampled. This numberof binary bits can vary, for
`therewith both a first code frequency havingafirst prede-
`termined offset from the reference trequency and a second
`example, between sixteen and twenty four bits per sample.
`code frequency having a second predetermined offset from
`The amplitude dynamic range resulting from using sixteen
`the reference frequency; b) measuring the spectral power of
`bits per sample of the audio signal is nincty-six dB. This
`the signal within the block in a first neighborhood of
`decibel measure is the ratio between the square of the
`highest audio amplitude (21°=65536) and the lowest audio
`frequencies extending aboutthe first code frequency and in
`amplitude (17=1). The dynamic rangeresulting from using
`a second neighborhood of frequencies extending about the
`second code frequency, wherein the first frequency has a
`twenty-four bits per sample is 144 dB. Raw audio, which is
`spectral amplitude, and wherein the second frequency has a
`sampled at the 44.1 kHz rate and which is converted to a
`spectral amplitude; c) swapping the spectral amplitude ofthe
`sixteen-bit per sample representation, results in a data rate of
`705.6 kbits/s.
`first code frequency with a spectral amplitude of a frequency
`having a maximum amplitude in the first neighborhood of
`frequencies while retaining a phase angle at both the first
`frequency and the frequency having the maximum ampli-
`tude in the first neighborhood of frequencies; and d) swap-
`ping the spectral amplitude of the second code frequency
`with a spectral amplitude of a frequency having a minimum
`amplitude in the second neighborhood of frequencies while
`retaining a phase angle at both the second frequency and the
`frequency having the maximum amplitude in the second
`neighborhood of frequencies.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`30
`
`35
`
`40
`
`45
`
`These and other features and advantages will become
`more apparent from a detailed consideration of the invention
`when taken in conjunction with the drawings in which:
`FIG. 1 is a schematic block diagram of an audience
`measurement system employing the signal coding and
`decoding arrangements of the present invention;
`FIG. 2 is flow chart depicting steps performed by an
`encoder of the system shown in FIG. 1;
`FIG. 3 is a spectral plot of an audio block, wherein the thin
`line of the plot is the spectrum ofthe original audio signal
`and the thick line of the plot is the spectrum of the signal
`modulated in accordance with the present invention;
`FIG. 4 depicts a window function which may be used to
`prevent transient effects that might otherwise occur at the
`boundarics between adjacent cneoded blocks;
`FIG. 5 is a schematic block diagram of an arrangementfor
`generating a seven-bit pseudo-noise synchronization
`sequence;
`
`50
`
`55
`
`60
`
`65
`
`Compression of audio signals is performed in order to
`reduce this data rate to a level which makes it possible to
`transmit a stereo pair of such data on a channel with a
`throughput as low as 192 kbits/s. This compression typically
`is accomplished by transform coding. A block consisting of
`N,=1024 samples, for example, may be decomposed, by
`application of a Fast Fourier Transform or other similar
`frequencyanalysis process, into a spectral representation. In
`order to prevent errors that may occur at
`the boundary
`between one block and the previous or subsequent block,
`overlapped blocks are commonlyused. In one such arrange-
`ment where 1024 samples per overlapped block are used, a
`block includes 512 samples of “old” samples (i.e., samples
`from a previous block ) and 512 samples of “new”or current
`samples. The spectral representation of such a block is
`divided into critical bands where each band comprises a
`group of several neighboring frequencies. The powerin each
`of these bands can be calculated by summing the squares of
`the amplitudes of the frequency components within the
`band.
`
`Audio compression is based on the principle of masking
`that, in the presence of high spectral energy at one frequency
`(i.e., the masking frequency), the human ear is unable to
`perceive a lower energy signal if the lower energy signal has
`a frequency (i., the masked frequency) near that of the
`higher energy signal. The lower energy signal at the masked
`frequencyis called a masked signal. A masking, threshold,
`whichrepresentseither(i) the acoustic energy required at the
`masked frequency in order to make it audible or (ii) an
`energy change in the existing spectral value that would be
`
`

`

`US 6,272,176 B1
`
`7
`perceptible, can be dynamically computed for each band.
`‘The frequency components in a masked band canbe repre-
`sented in a coarse fashion by using fewer bits based on this
`masking threshold. That is, the masking thresholds and the
`amplitudes of the frequency components in each band are
`coded with a smaller number of bits which constitute the
`
`8
`the code may use a different pair of code frequencies f, and
`f, denoted by corresponding code frequency indexes I, and
`I,. There are two preferred ways of selecting the code
`frequencies f, and f, at
`the step 46 so as to create an
`inaudible wide-band noise like code.
`
`compressed audio. Decompression reconstructs the original
`signal based on this data.
`FIG. 1 illustrates an audience measurement system 10 in
`which an encoder 12 adds an ancillary code to an audio
`signal portion 14 of a broadcast signal. Alternatively, the
`encoder 12 maybe provided, as is knownin theart, at some
`other location in the broadcast signal distribution chain. A
`transmitter 16 transmits the encoded audio signal portion
`with a video signal portion 18 of the broadcast signal. When
`the encodedsignal is received by a receiver 20 located al a
`statistically selected metering site 22, the ancillary codeis
`recovered by processing the audio signal portion of the
`received broadcast signal even though the presence of that
`ancillary code is imperceptible to a listener when the
`encoded audio signal portion is supplied to speakers 24 of
`the receiver 20. To this end, a decoder 26 is connected either
`directly to an audio output 28 available at the receiver 20 or
`to a microphone30 placed in the vicinity of the speakers 24
`through which the audio is reproduced. The reecived audio
`signal can be either in a monaural or stereo format.
`ENCODING BY SPECTRAL MODULATION
`
`(a) Direct Sequence
`
`10
`
`15
`
`One wayof selecting the code frequencies f, and f, at the
`step 46 is to compute the code frequencies by use of a
`frequency hopping algorithm employing a hop sequence H,
`and a shift index I,,,,. For example, if N, bits are grouped
`together to form a pseudo-noise sequence, H, is an ordered
`sequence of N, numbers representing the frequency devia-
`tion relative to a predetermined reference index I,,. For the
`case where N,=7, a hop sequence H,={2,5, 1, 4, 3, 2, 5} and
`a shift index I,,,,,=5 could be used. In general, the indices for
`the N, bits resulting from a hop sequence maybe given by
`the following equations:
`
`L,=lsytH1snipe
`
`and
`
`Loltltlsniper
`
`(2)
`
`(3)
`
`One possible choice for the reference frequencyf., is five
`kHz, corresponding to a predetermined reference index
`1,,=53. This value of f., is chosen because it is above the
`average maximum sensitivity frequency of the human ear.
`In order for the encoder 12 to embed digital code data in
`When encodinga first block of the audio signal, I, and I, for
`an audio data stream in a manner compatible with compres-
`the first block are determined from equations (2) and (3)
`sion technology, the encoder 12 should preferably use fre-
`usingafirst of the hop sequence numbers; when encoding a
`quencies and critical bands that match those used in com-
`second block of the audio signal, I, and I, for the second
`pression. The block length N, of the audio signal that is used
`block are determined from equations (2) and (3) using a
`for coding may be chosen suchthat, for example, jN,=N,=
`second of the hop sequence numbers; and so on. For the fifth
`1024, where j is an integer. A suitable value for N. may be,
`bit in the sequence {2,5,1,4,3,2,5}, for example, the hop
`for example, 512. As depicted by a step 40 ofthe flow chart
`sequence value is three and, using equations (2) and (3),
`shownin FIG. 2, which is executed by the encoder12, a first
`produces an index I,=51 and an index [,=61 in the case
`block v(t) of jN, samples is derived from the audio signal
`whereI_,,,,=5. In this example, the mid-frequency index is
`portion 14 by the encoder 12 such as by use of an analog to
`given by the following equation:
`digital converter, where v(t) is the time-domain representa-
`tion of the audio signal within the block. An optional
`window may be applied to v(t) at a block 42 as discussed
`below in additional detail. Assuming for the momentthat no
`such wind