US005450490A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,450,490
`Sep. 12, 1995
`
` .
`
`United States Patent
`Jensen etal.
`
`15;
`
`[75]
`
`[73] Assignee:
`
`:
`
`.
`
`Sony Exhibit 1044
`Sony Exhibit 1044
`Sony v. MZ Audio
`Sony v. MZ Audio
`
`GENERATOR
`
`7/1990 Werner ......ccsscsecrscsserssscessesseee 375/1
`4,943,973
`[54] APPARATUS AND METHODS FOR
`4,945,412 7/1990 Kramer.....
`358/142
`
`INCLUDING CODES IN AUDIO SIGNALS
`4,955,070 9/1990 Welsh et al. we 455/2
`AND DECODING
`
`demas 10/1990 Greenberg....
`358/142
`Inventors:
`James M. Jensen, Columbia;
`aiuto GAG Gar
`wo 08
`Robert B. Graybill, Ellicott City;
`
`5,113,437
`5/1992 Best etal..
`me
`Sayed Hassan, Clarksville; Wendell
`
`5,213,337
`5/1993 Sherman ...
`4595/2 X
`D. Lynch, Silver Spring; Wayne
`
`
`5,379,345
`1/1995 Greenberg
`... 380/6 X
`Sabin, Baldwin, all of Md. °
`5,394,274 2/1995 Kalin oo.cceccssssssssesssescssseeeeses 380/3 X
`The Arbitron Company, Laurel, Md.
`FOREIGN PATENT DOCUMENTS
`I aE Nei aet 1904
`31,
`.
`
`.. HO4N 7/16
`7/1986 Canada..
`1208761
`Int. CLS...
`[51]
`.. HO4L 9/00; HO4H 9/00
`
`2036205 12/1991 Canada.....
`« HO4H 9/00
`[52] U.S. C1, ooneececeetencenceetseeeeseseaeeres 380/6; 455/2;
`
`2559002
`8/1985 France esssssssssssssesesee HO4B 1/00
`,
`348/1
`W091/11062 7/1991 WIPO...
`.«. HO4B 17/00
`.
`wi
`[58] Field of Search..................... 380/6; 455/2; 348/1
`093/07689 4/1993 WIPO uu...ecccsssussscsoeee HO4B 17/00
`[56]
`References Cited
`OTHER PUBLICATIONS
`U.S. PATENT DOCUMENTS
`Namba,Seiichi, et al., “A Program Identification Code
`2,470,240 5/1949 Crosby secsssssssessseseussenseenes 250/27
`Ftansmission System Using Low-Frequency Audio
`2,573,279 10/1951 Scherbatskoy...
`.. 346/37
`
`2,630,525
`3/1953 Tomberiin etal......
`ww 250/6 pee NHK Laboratories Note; Ser. No. 314, Mar.
`2,660,511 11/1953 Scherbatskoy etal. ....
`+ 346/37
`5.
`.
`.
`.
`
`2,660,662 11/1953 Scherbatskoy sessessseecesssnasensenes 250/2
`McGraw-Hill Encyclopedia ofScience & Technology, 6th
`2,662,168 12/1953 Scherbatskoy et al.
`... 250/2
`Edition, McGraw-Hill Book Company, 1987, vol. 8,
`2,766,374 10/1956 Hoffmann ........0
`w» 250/2
`pp. 328-341.
`
`3,004,104 10/1961 Hembrooke .
`wee 179/2
`Rossing, The Science of Sound, Addison-Wesley Pub-
`
`3,397,402 8/1968 Schneider........
`.. 346/37
`lishing Company, 1990, Chapters 5 and 6 (pp. 65-108,
`3,492,577
`1/1970 Reiteret al.
`.....
`.. 325/31
`and section 16.4 (pp. 336-338).
`
`3,760,275 9/1973 Ohsawaet al.
`..
`ww. 325/31
`Pri:
`toy Ci
`
`I:
`3,803,349 4/1974 Watanabe....
`-1TB/S8R
`“grey Bzaminer. Pat lta
`
`UJ;
`3,845,391 10/1974 Crosby cescccscssssscssssssssecsssseseee 325/64
`Garg xi se eS aor ugene
`L.
`Flanagan,
`
`4,025,851 Morris&Safford5/1977 Haselwood etal. w. 325/31 urtis,
`
`
`
`
`4,225,967 9/1980 Miwaetal. .........
`... 455/68
`[57]
`ABSTRACT
`
`
`pesrerhs nyono ae Jr., et al. .
`aes Soin
`Apparatus and methods for including a code having at
`Niseeas Winks ewa * TOsI6
`least one code frequency componentin an audio signal
`
`4,450.531
`5/1984 Kenyon et al.
`"364/604
`are provided. The abilities of various frequency compo-
`4.547804 10/1985 Greenberg _.
`3587142
`“nents in the audio signal to mask the code frequency
`4613904 9/1986 Lurie .......
`358/142
`component to human hearing are evaluated and based
`4,626,904 12/1986 Lurie .......
`-. 358/84
`on these evaluations an amplitude is assigned to the
`4,639,779
`1/1987 Greenberg
`. 358/142
`code frequency component. Methods and apparatusfor
`4,681,995 10/1987 Kemp e.essssscssssssssssssscsssssnseecsens 455/2
`detecting a code in an encoded audio signal are also
`4,697,209 9/1987 Kiewit etal.
`... 358/84
`provided. A code frequency componentin the encoded
`
`4,703,476 10/1987 Howard.......
`... 370/76
`audio signal is detected based on an expected code am-
`4,718,106
`1/1988 Weinblatt 00...
`cece 455/2
`plitude or on a noise amplitude within a range of audio
`
`4,771,455 9/1988 Hareyamaet al.
`w+. 380/6
`frequenciesincluding the frequency of the code compo-
`4,805,020 2/1989 Greenberg.......
`. 358/147
`nent.
`4,843,562 6/1989 Kenyonetal.
`- 364/487
`47 Claims, 20 Drawing Sheets
`4,876,617 10/1989 Best et ab...eeeeeeeeeeeee 360/60
`40
`#
`
`
`
`
`MASKING
`UD!
`
`
`EVALUATION
`
`
`
`
` CODE
`
`FREQUENCY
`
`

`

`40
`34
`
`
`EVALUATION
`
`CODE
`GENERATOR
`
` MASKING
`FREQUENCY
`
`
`
`ENCODED
`AJ
`
`FIG. /
`
`
`
`OZJO[yo0YSS661‘ZT“dagU9}8J“SN
`
`06r‘0SP‘s
`
`

`

`
`
`
`
`
`84
`
`
`
`
`
`
`
`
`
`
`
`AUDIO OZJOZ1204S66T‘ZT“dagyusjeg*S°N
`
`ENCODED
`
`06r‘OSP‘s
`
`

`

`100
`
`MONITOR
`
`104
`
`140
`
`[06
`
`LMS
`
`,
`
`90
`
`124
`
`96
`
`120
`
`BAND
`
`ENCODED
`AUDIO
`
`146
`
`
`
`0ZJO¢30045S66L‘ZI“dagqusjeg“S°N
`
`06r'osP‘s
`
`

`

`on2 1
`
`fo
`
`f3
`
`fo
`
`fs
`
`f6
`
`fr
`
`ft
`
`fg
`
`fio aH,
`
`fy
`
`fe
`
`fg
`
`fia
`
`fis
`
`fie
`
`fiz
`
`fig
`
`fig
`
`feo
`
`for
`
`foo
`
`faz
`
`faq
`
`fo5 fog
`
`for
`
`fog
`
`fag
`
`f39
`
`|
`ony 131
`
`£32
`
`933.
`
`834
`
`£35 36 F357
`
`f3— fag
`
`fo
`
`3kHz
`
`FREQUENCY
`
`
`
`
`
`07JOp39045S661‘ZI“dagyud}Vg°S°N
`
`06r'osr‘s
`
`

`

`FIG.5
`
`M2
`
`0
`
`110
`
`MEMORY
`
`AMPLITUDE
`ADJUST
`
`I
`
`4
`
`l06
`
`DAC
`
`FREQUENCY
`RANGE
`SEPARATION
`
`13
`
`
` 146
`
`
`
`0%JO¢3094S661“ZT‘dagjuoeg“SN
`
`06r*0Sr'S
`
`

`

`ty
`
`+2
`
`106 0ZJO9JoaS661
`
`‘ZI“dagyuojeg“SN
`
`
`
`iheUTECbdeCUT
`TedLa|etPada|LL||
`falAUlf||fa
`
`FIG.6
`
`067‘0Sr's
`
`

`

`U.S. Patent
`
`Sep. 12, 1995
`
`Sheet 7 of 20
`
`5,450,490
`
`
`START
`ENCODE
`
`
`190
`
`FIG. 7A
`
`194
`
`
`
`[D6
`-_MESSAGE
`
`
` GET NEXT
`FROM HOST
`
`
`160 OUTPUT
`CODE
`
`
`SYMBOL
`
`
`
`TIMER
`TIMER
`
`INTERRUPT1
`INFERRUPT 2
`
`
`
`
`
` ACCUMULATE
`WEIGHT
`FACTORS
`
`
`
`
`RAM
`
`AUDIO im] -O©Ooo -O
`
` MESSAGE
`
`
`
`COMPLETE
`
`

`

`U.S. Patent
`
`Sep. 12, 1995
`
`Sheet 8 of 20
`
`5,450,490
`
`F-1G.7B
`
`168
`
`170
`
`I72
`
`
`
`
`
`
`COMPUTE
`
`RESOLUTION
`FFT
`
`FOR EACH
`CODE COMPONENT
`
`
`
`NA
`NEIGHBORHOOD
`
`

`

`U.S. Patent
`
`Sep. 12, 1995
`
`Sheet 9 of 20
`
`5,450,490
`
`W)
`
`FIG.7C
`
`
`
`DOMINANT
`TONAL
`
`ON
`
`76
`
`180
`
`EVALUATE NARROW
`BAND AND BROADBAND
`MASKING CAPABILITIES
`
`G
`
`YES
`
`EXIT
`SUBROUTINE
`
`194
`
`

`

`U.S. Patent
`
`Sep. 12, 1995
`
`Sheet 10 of 20
`
`5,450,490
`
`230
`
`
`SWITC
`
`
`
`
`
`2
`
`34
`
`H
`
`WEIGHTING
`
`DETERMINATION
`
`

`

`> TO MULTIPLIER 236
`
`230
`
`FIG.9
`
`
`
`
`
`U.S.PatentSep.12,1995Sheet11of205,450,490
`
`
`
`
`
`
`

`

`
`
`
`
`204
`CODE
`
`
`CODES
`
`EXTRACTION EXPECTED
`
` 0ZJOZE009S66L‘ZT“dag—Juoyeg‘SN
`
`290
`
`
`AMPLITUDE
`DETERMINATION
`
`06r‘0sSP's
`
`

`

`
`
`262
`
` ENCODED
`
`ANALOG
`CONDITIONING
`
`210
`
`MEMORY
`
`
`
`212 07JOETW945S66L“ZI“dasyuojeg‘S°Nn
`
`FIG. /T
`
`06r‘0St‘s
`
`

`

`U.S. Patent
`
`Sep. 12, 1995
`
`5,450,490
`
`Sheet 14 of 20
`
`FIG.12A
`
`

`

`U.S. Patent
`
`Sep. 12, 1995
`
`Sheet 15 of 20
`
`5,450,490
`
`FIG.12B8
`
`GATHER|44
`DATA
`
`456
`
`PERFORM
`OVERLAPPED
`FFT
`
`462
`
`
`
`
`
`
`EVALUATE
`FOR
`"@" ORI"
`
`466
`
`EVALUATE
`FOR
`“E" OR “S"
`
`
`
`

`

`ENCODED
`
`AUDIO
`
`
`
`—*
`
`COMPARE
`FUNCTION
`
`CODES
`
`
`
`
`
`OZJOOT0045S661“ZT“dagju0}eg°S°N
`
`FIG./3
`
`06r‘OSt's
`
`

`

`DETECTORS
`
`zBe F1G./4
`
`
`COMPONENT
`
` DETECTORS
` COMPONENT
`(N+1)—> 2N
`
`AUDIO
`
`
`COMPONENT
`DETECTORS
`
`(2N+ {)—> 3N
`
` CONTROL
`
`CIRCUIT
` COMPONENT
`
`
`DETECTORS
`
`
`
`
`
`
`OZJOLE3994S661“ZT“dagjus}ze8g*S°N
`
`06r‘0SP's
`
`

`

`
`NOISE
`ABSOLUTE
`ESTINATE
`VALUE
`FILTER
`CIRCUIT
`
`
`
`
` CODE
`WINDOW|PRESENCE
`COMPARATOR
`
`
`
`
`
`ABSOLUTE
`SIGNAL
`
`VA
`ESTIMATE
`
`
`CIRCUIT
`
`
`
`
`
`FILTER
`
`
`
`
`
`0ZJO8ST3204SS66L‘ZT“degzu0je8g*S°N
`
`06P‘0St's
`
`

`

`
`
`CONTROL
`
`ENCODER
`TRANSMITTER
`
`
`
` PROGRAM
`
`AUD 10
`
`SOURCE
`
` 340
`
`
`
`
`
`OZ506T3094SS661“CI“dag3u938g°S'N
`
`FIG.16
`
`06r0sr‘s
`
`

`

`
`
`
`
` 07JO071894S66I‘ZL“dagjus}e8g*S'N
`
` RADIO
`
`
`
`RECEIVER
`
`
`06r‘0SPr‘s
`
`

`

`1
`
`5,450,490
`
`APPARATUS AND METHODS FOR INCLUDING
`CODESIN AUDIO SIGNALS AND DECODING
`
`BACKGROUND OF THE INVENTION
`
`5
`
`2
`which overcome the disadvantages of the foregoing
`proposed techniques.
`It is a further object of the present invention to pro-
`vide coding apparatus and methodsfor including codes
`with audio signals so that, as sound, the codes are inau-
`dible to the human ear but can be detected reliably by
`The present invention relates to apparatus and meth-
`decoding apparatus.
`ods for including codes in audio signals and decoding
`A further object of the present invention is to provide
`such codes.
`decoding apparatus and methodsfor reliably recovering
`For manyyears, techniques have been proposed for
`codes present in audio signals.
`mixing codes with audio signals so that (1) the codes can
`In accordance with a first aspect of the present inven-
`be reliably reproduced from the audio signals, while (2)
`tion, apparatus and methodsfor including a code having
`the codes are inaudible when the audio signals are re-
`at least one code frequency component with an audio
`produced as sound. The accomplishment of both objec-
`signal having a plurality of audio signal frequency com-
`tives is essential for practical application. For example,
`ponents, comprise the meansfor and the steps of: evalu-
`broadcasters and producers of broadcast programs, as
`ating an ability of a first set of the plurality of audio
`well as those who record music for public distribution
`signal frequency components to mask the at least one
`will not tolerate the inclusion of audible codes in their
`code frequency component to human hearing to pro-
`programs and recordings.
`duceafirst masking evaluation; evaluating anability of
`Techniques for encoding audio signals have been
`20
`a second set of the plurality of audio signal frequency
`proposed at various times going back at least to U.S.
`components differing from the first set thereof to mask
`Pat. No. 3,004,104 to Hembrooke issued Oct. 10, 1961.
`the at least one code frequency component to human
`Hembrooke showed an encoding method in which
`hearing to produce a second masking evaluation; assign-
`audio signal energy within a narrow frequency band
`ing an amplitude to the at least one code frequency
`wasselectively removed to encode the signal. A prob-
`component based on a selected one of the first and
`lem with this technique arises when noise or signal
`second masking evaluations; and including the at least
`distortion reintroduces energy into the narrow fre-
`one code frequency component with the audio signal.
`quency band so that the code is obscured.
`In accordance with another aspect of the present
`In another method, U.S. Pat. No. 3,845,391 to Crosby
`invention, an apparatus for including a code having at
`proposed to eliminate a narrow frequency band from
`least one code frequency component with an audio
`the audio signal and insert a code therein. This tech-
`signal having a plurality of audio signal frequency com-
`nique evidently encountered the same problems as
`ponents, comprises: a digital computer having an input
`Hembrooke, as recounted in U.S. Pat. No. 4,703,476 to
`for receiving the audio signal,
`the digital computer
`Howard which, as indicated thereon, was commonly
`being programmed to evaluate respective abilities of
`assigned with the Crosby patent. However, the Howard
`first and second sets of the plurality of audio signal
`patent sought only to improve Crosby’s method with-
`frequency components to mask the at least one code
`out departing from its fundamental approach.
`frequency component to human hearing to produce
`It has also been proposed to encode binary signals by
`respective first and second masking evaluations, the
`spreading the binary codes into frequencies extending
`second set of the plurality of audio signal frequency
`throughout the audio band. A problem with this pro-
`posed method is that, in the absence of audio signal
`components differing from the first set thereof, the
`digital computer being further programmedto assign an
`components to mask the code frequencies, they can
`amplitude to the at least one code frequency component
`become audible. This method, therefore, relies on the
`based on a selected one of the first and second masking
`asserted noiselike character of the codes to suggest that
`evaluations; and means for including the at least one
`their presence will be ignored by listeners. However,in
`code frequency component with the audio signal.
`many cases this assumption may not be valid, for exam-
`In accordance with a further aspect of the present
`ple, in the case of classical music including portions
`invention, apparatus and methods for including a code
`with relatively little audio signal content or during
`having a plurality of code frequency components with
`pauses in speech.
`an audio signal having a plurality of audio signal fre-
`A further technique has been suggested in which dual
`quency components, the plurality of code frequency
`tone multifrequency (DTMF) codesare inserted in an
`components including a first code frequency compo-
`audio signal. The DTMF codes are purportedly de-
`nent having a first frequency and a second codefre-
`tected based on their frequencies and durations. How-
`quency component having a second frequency different
`ever, audio signal components can be mistaken for one
`from the first frequency, comprise the meansfor and the
`or both tones of each DTMFcode, so that either the
`steps of, respectively: evaluating an ability of at least
`presence of a code can be missed by the detector or
`one of the plurality of audio signal frequency compo-
`signal components can be mistaken for a DTMFcode.
`It is noted in addition that each DTMFcodeincludes a
`nents to mask a code frequency component having the
`first frequency to human hearing to produce a first
`tone common to another DTMFcode. Accordingly, a
`respective masking evaluation; evaluating an ability of
`signal component correspondingto a tone of a different
`DTMEFcode can combine with the tone of a DTMF
`at least one of the plurality of audio signal frequency
`_ components to mask a code frequency component hav-
`code which is simultaneously present in the signal to
`result in a false detection.
`ing the second frequency to human hearing to produce
`a second respective masking evaluation; assigning a
`respective amplitudeto thefirst code frequency compo-
`nent based on the first respective masking evaluation
`and assigning a respective amplitude to the second code
`frequency component based on the second respective
`
`Accordingly, it is an object of the present invention
`to provide coding and decoding apparatus and methods
`
`'
`
`15
`
`25
`
`35
`
`40
`
`45
`
`30
`
`35
`
`60
`
`65
`
`OBJECTS AND SUMMARY OF THE
`INVENTION
`
`

`

`5,450,490
`
`10
`
`30
`
`35
`
`3
`4
`masking evaluation; and including the plurality of code
`one code frequency componentin a portion of the audio
`frequency components with the audio signal.
`signal within the second audio signal interval.
`In accordance with yet another aspect of the present
`In accordancewitha still further aspect of the present
`invention, an apparatus for including a code having a
`invention, apparatus and methodsfor including a code
`5
`plurality of code frequency components with an audio
`having at least one code frequency component with an
`signal having a plurality of audio signal frequency com-
`audiosignal havingaplurality of audio signal frequency
`ponents, the plurality of code frequency components
`components, comprise the means for and the steps of,
`including a first code frequency component having a
`respectively: producinga first tonal signal representing
`first frequency and a second code frequency component
`substantially a first single one of the plurality of audio
`having a second code frequency different from the first
`signal frequency components; evaluating an ability of
`frequency, comprises; a digital computer having an
`the first single one of the plurality of audio signal fre-
`input for receiving the audio signal, the digital com-
`quency components to maskthe at least one code fre-
`puter being programmed to evaluate an ability of at
`quency component to human hearing based on thefirst
`least one of the plurality of audio signal frequency com-
`tonal signal to produceafirst masking evaluation; as-
`15
`ponents to mask a code frequency component having
`signing an amplitude to the at least one code frequency
`the first frequency to human hearing to producea first
`component based on the first masking evaluation; and
`respective masking evaluation and to evaluate an ability
`including the at least one code frequency component
`ofat least one of the plurality of audio signal frequency
`with the audio signal.
`components to mask a code frequency componenthav-
`In accordance with another aspect of the present
`ing the second frequency to human hearing to produce
`invention, an apparatus for including a code having at
`a second respective masking evaluation;
`the digital
`least one code frequency component with an audio
`computer being further programmedto assign a corre-
`signal havinga plurality of audio signal frequency com-
`sponding amplitude to the first code frequency compo-
`ponents, comprises: a digital computer having an input
`nent based on the first respective masking evaluation
`for receiving the audio signal, the digital computer
`and to assign a corresponding amplitude to the second
`being programmedto producea first tonal signal Tepre-
`code frequency componentbased on the second respec-
`senting substantially a first single one of the plurality of
`tive masking evaluation; and means for including the
`audio signal frequency components and to evaluate an
`plurality of code frequency components with the audio
`ability of the first single one of the plurality of audio
`signal.
`signal frequency components to mask the at least one
`In accordancewitha still further aspect of the present
`code frequency component to humanhearing based on
`invention, apparatus and methods for including a code
`the first tonal signal to producea first masking evalua-
`having at least one code frequency component with an
`tion; the digital computer being further programmedto
`audio signal including a plurality of audio signal fre-
`assign an amplitude to the at least one code frequency
`quency components, comprise the means for and the
`component based on the first masking evaluation; and
`steps of, respectively: evaluating an ability of at least
`means for including the at least one code frequency
`one of the plurality of audio signal frequency compo-
`component with the audio signal.
`nents within a first audio signal interval on a time scale
`In accordance with yet still another aspect of the
`of the audio signal when reproduced as sound during a
`present invention, apparatus and methods for detecting
`correspondingfirst time interval to mask the at least one
`a code in an encoded audio signal, the encoded audio
`code frequency component to human hearing when
`signal including a plurality of audio frequency signal
`reproduced as sound during a second time interval cor-
`components and at least one code frequency component
`responding to a second audio signal interval offset from
`having an amplitude and an audio frequency selected
`the first audio signal interval to produceafirst masking
`for masking the code frequency component to human
`evaluation; assigning an amplitude to the at least one
`45
`hearing by at least one of the plurality of audio fre-
`code frequency component based on the first masking
`quency signal components, comprise the meansfor and
`evaluation; and including the at least one code fre-
`the steps of, respectively: establishing an expected code
`quency component in a portion of the audio signal
`amplitudeofthe at least one code frequency component
`within the second audio signal interval.
`based on the encoded audio signal; and detecting the
`In accordance with yet still another aspect of the
`code frequency componentin the encoded audiosignal
`present invention, an apparatus for including a code
`based on the expected code amplitude thereof.
`having at least one code frequency componentwith an
`In accordance with a yet still further aspect of the
`audio signal including a plurality of audio signal fre-
`present invention, a programmed digital computer is
`quency components, comprises: a digital computer hav-
`provided for detecting a code in an encoded audio sig-
`ing an input for receiving the audio signal, the digital
`nal, the encoded audio signal including a plurality of
`computer being programmedto evaluate an ability of at
`audio frequency signal components and at least one
`least one ofthe plurality of audio signal frequency com-
`code frequency component having an amplitude and an
`ponents within a first audio signal interval on a time
`audio frequency selected for masking the code fre-
`scale of the audio signal when reproduced as sound
`quency component to human hearing byat least one of
`during a correspondingfirst time interval to mask the at
`the plurality of audio frequency signal components, the
`least one code frequency component to human hearing
`digital computer comprising: an inputfor receiving the
`when reproduced as sound during a second timeinter-
`encodedaudio signal; a processor programmedto estab-
`val corresponding to a second audio signal interval
`lish an expected code amplitudeof the at least one code
`offset from the first audio signal interval, to produce a
`frequency component based on the encoded audio sig-
`first masking evaluation;
`the digital computer being
`nal, to detect the code frequency component in the
`further programmed to assign an amplitude to the at
`encoded audio signal based on the expected code ampli-
`least one code frequency componentbased onthe first
`tude and to producea detected code outputsignal based
`masking evaluation; and meansfor includingtheat least
`on the detected code frequency component; and an
`
`65
`
`50
`
`55
`
`

`

`5,450,490
`
`6
`FIG.4 provides spectral diagramsfor use in ilustrat-
`ing frequency compositions of various data symbols as
`encoded by the embodiment of FIG. 3;
`FIGS. 5 and 6 are functional block diagrams for use
`illustrating the operation of the embodimentof FIG.
`
`3;
`
`5
`output coupled with the processor for providing the
`detected code output signal.
`In accordance with another aspect of the present
`invention, apparatus and methodsare provided for de-
`tecting a code in an encoded audio signal, the encoded
`audio signal having a plurality of frequency compo-
`nents including a plurality of audio frequency signal
`components andat least one code frequency component
`having a predetermined audio frequency and a prede-
`termined amplitude for distinguishing the at least one
`code frequency componentfrom theplurality of audio
`frequency signal components, comprise the means for
`and thesteps of, respectively: determining an amplitude
`of a frequency component of the encoded audio signal
`within a first range of audio frequencies including the
`predetermined audio frequencyofthe at least one code
`frequency component; establishing a noise amplitude
`for the first range of audio frequencies; and detecting
`the presence of the at least one code frequency compo-
`nentin the first range of audio frequencies based on the
`established noise amplitude thereof and the determined
`amplitude of the frequency component therein.
`In accordance with a further aspect of the present
`invention, a digital computer is provided for detecting a
`code in an encoded audio signal, the encoded audio
`signal having a plurality of frequency components in-
`cluding a plurality of audio frequency signal compo-
`nents and at least one code frequency component hav-
`ing a predetermined audio frequency and a predeter-
`mined amplitude for distinguishing the at least one code
`frequency component from the plurality of audio fre-
`quency signal components, comprising: an input for
`receiving the encoded audiosignal; a processor coupled
`with the input to receive the encoded audio signal and
`programmed to determine an amplitude of a frequency
`component of the encoded audio signal within a first
`range of audio frequencies including the predetermined
`audio frequency of the at least one code frequency
`component; the processor being further programmedto
`establish a noise amplitude for the first range of audio
`frequencies and to detect the presenceofthe at least one
`code frequency componentin the first range of audio
`frequencies based on the established noise amplitude
`thereof and the determined amplitude of the frequency
`component therein; the processor being operative to
`produce a code output signal based on the detected
`presence of the at least one code frequency component;
`and an output terminal coupled with the processor to
`provide the code signal thereat.
`The above, and other objects, features and advan-
`tages of the invention, will be apparentin the following
`detailed description of certain advantageous embodi-
`ments thereof whichis to be read in connection with the
`accompanying drawings forming a part hereof, and
`wherein corresponding elements are identified by the
`same reference numerals in the several views of the
`drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG.1 is a functional block diagram of an encoderin
`accordance with an aspect of the present invention;
`FIG. 2 is a functional block diagram of a digital en-
`coderin accordance with an embodimentof the present
`invention;
`FIG.3 is a block diagram of an encoding system for
`use in encoding audio signals supplied in analog form;
`
`10
`
`20
`
`25
`
`30
`
`35
`
`50
`
`55
`
`65
`
`FIGS. 7A through 7C are flow charts for illustrating
`a software routine employed in the embodiment of FIG.
`3;
`
`FIG. 8 is a block diagram of an encoder employing
`analog circuitry;
`FIG.9 is a block diagram of a weighting factor deter-
`mination circuit of the embodiment of FIG.8;
`FIG.10 is a functional block diagram of a decoder in
`accordance with certain features of the present inven-
`tion;
`FIG. 11 is a block diagram of a decoder in accor-
`dance with an embodiment of the present invention
`employing digital signal processing;
`FIGS. 12A and 12B are flow chartsfor use in describ-
`ing the operation of the decoder of FIG. 11;
`FIG.13is a functional block diagram of a decoderin
`accordance with certain embodiments of the present
`invention;
`FIG.14 is a block diagram of an embodiment of an
`analog decoder in accordance with the present inven-
`tion;
`.
`FIG. 15is a block diagram of a componentdetector
`of the embodiment of FIG. 14; and
`FIGS. 16 and 17 are block diagrams of apparatusin
`accordance with an embodiment of the present inven-
`tion incorporated in a system for producing estimates of
`audiences for widely disseminated information.
`
`DETAILED DESCRIPTION OF CERTAIN
`ADVANTAGEOUS EMBODIMENTS
`
`Encoding
`invention implements techniques for
`The present
`including codes in audio signals in order to optimize the
`probability of accurately recovering the information in
`the codes from the signals, while ensuring that the
`codes are inaudible to the human ear when the encoded
`audio is reproduced as sound evenif the frequencies of
`the codes fall within the audible frequency range.
`With reference first to FIG. 1, a functional block
`diagram of an encoder in accordance with an aspect of
`the present invention is illustrated therein. An audio
`signal to be encodedis received at an input terminal 30.
`The audio signal may represent, for example, 2 program
`to be broadcast by radio, the audio portion of a televi-
`sion broadcast, or a musical composition or other kind
`of audio signal to be recorded in some fashion. More-
`over, the audio signal may be a private communication,
`such as a telephone transmission, or a personal record-
`ing of some sort. However, these are examples of the
`applicability of the present invention and there is no
`intention to limit its scope by providing such examples.
`ASindicated by the functional block 34 in FIG. 1, the
`ability of one or more componentsofthe received audio
`signal to mask sounds having frequencies corresponding
`with those of the code frequency component or compo-
`nents to be addedto the audio signal is evaluated. Multi-
`ple evaluations may be carried out for a single code
`frequency, a separate evaluation for each of a plurality
`of code frequencies may be carried out, multiple evalua-
`tions for each of a plurality of code frequencies may be
`effected, one or more common evaluations for multiple
`
`

`

`7
`code frequencies may be carried out or a combination of
`one or more of the foregoing may be implemented.
`Each evaluation is carried out based on the frequency of
`the one or more code components to be masked and the
`frequency or frequencies of the audio signal component
`or components whose masking abilities are being evalu-
`ated. In addition, if the code component and the mask-
`ing audio component or components do notfall within
`substantially simultaneous signal
`intervals, such that
`they would be reproduced as sound at significantly
`different time intervals, the effects of differences in
`signal intervals between the code component or compo-
`nents being masked and the masking program compo-
`nent or components are also to be taken into consider-
`ation.
`in certain embodiments multiple
`Advantageously,
`evaluations are carried out for each code component by
`separately considering the abilities of different portions
`of the audio signal to mask each code component. In
`one embodiment, the ability of each of a plurality of
`substantially single tone audio signal components to
`mask a code componentis evaluated based on thefre-
`quency of the audio signal component, its “amplitude”
`(as defined herein) and timing relevant to the code com-
`ponent, such masking being referred to herein as “tonal
`masking”.
`The term “amplitude” is used herein to refer to any
`signal value or values which may be employedto evalu-
`ate masking ability, to select the size of a code compo-
`nent, to detect its presence in a reproducedsignal, or as
`otherwise used, including values such as signal energy,
`power, voltage,
`current,
`intensity and pressure,
`whether measured on an absolute orrelative basis, and
`whether measured on an instantaneous or accumulated
`basis. As appropriate, amplitude may be measured as a
`windowed average, an arithmetic average, by integra-
`tion, as a root-mean-square value, as an accumulation of
`absolute or relative discrete values, or otherwise.
`In other embodiments, in addition to tonal masking
`evaluations or in the alternative, the ability of audio
`signal components within a relatively narrow band of
`frequenciessufficiently near a given code componentto
`mask the componentis evaluated (referred to herein as
`“narrow band” masking). In still other embodiments,
`the ability of multiple code components within a rela-
`tively broad band of frequencies to mask the component
`is evaluated. As necessary or appropriate, the abilities of
`program audio components in signal intervals preced-
`ing or following a given component or components to
`mask the same on a non-simultaneous basis are evalu-
`ated. This manner of evaluation is particularly useful
`where audio signal components in a given signal inter-
`val have insufficiently large amplitudes to permit the
`inclusion of code components ofsufficiently large am-
`plitudes in the same signal interval so that they are
`distinguishable from noise.
`Preferably, a combination of two or more tonal mask-
`ing abilities, narrow band masking abilities and broad-
`band maskingabilities (and, as necessary or appropriate,
`non-simultaneous masking abilities), are evaluated for
`multiple code components. Where code components
`are sufficiently close in frequency, separate evaluations
`need not be carried out for each.
`Preferably, each evaluation provides a maximum
`allowable amplitude for one or more code components,
`so that by comparing all of the evaluations that have
`been carried out and which relate to a given compo-
`nent, a maximum amplitude may be selected therefor
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`60
`
`65
`
`3,450,490
`
`8
`which will ensure that each component will neverthe-
`less be masked by the audio signal whenit is reproduced
`as sound so thatall of the components becomeinaudible
`to human hearing. By maximizing the amplitude of each
`component, the probability of detecting its presence
`based on its amplitude,
`is likewise maximized. Of
`course, it is not essential that the maximum possible
`amplitude be employed, as it is only necessary when
`decoding to be able to distinguish a sufficiently large
`number of code components from audio signal compo-
`nents and other noise.
`Theresults of the evaluations are output as indicated
`at 36 in FIG. 1 and madeavailable to a code generator
`40. Code generation may be carried out in any of a
`variety of different ways. One particularly advanta-
`geous technique assigns a unique set of code frequency
`components to each of a plurality of data states or sym-
`bols, so that, during a given signal interval, a corre-
`sponding data state is represented by the presenceofits
`respective set of code frequency components. In this
`manner, interference with code detection by audiosig-
`nal components is reduced since, in an advantageously
`high percentage of signal intervals, a sufficiently large
`number of code components will be detectable despite
`program audio signal interference with the detection of
`other components. Moreover,
`the process of imple-
`menting the masking evaluationsis simplified where the
`frequencies of the code components are knownbefore
`they are generated.
`Other forms of encoding may also be implemented.
`For example, frequency shift keying (FSK), frequency
`modulation (FM), frequency hopping, spread spectrum
`encoding, as well as combinations of the foregoing can
`be employed. Still other encoding techniques which
`may be used in practicing the present invention will be
`appare