United States Patent [t9J
`Chu
`
`111111111111111111111111111111111111111111111111111111111111111111111111111
`US005263019A
`[It] Patent Number:
`[45] Date of Patent:
`
`5,263,019
`Nov. 16, 1993
`
`[54]
`
`[75]
`[73]
`
`[21]
`[22]
`
`[63]
`
`[51]
`[52]
`
`[58]
`
`[56]
`
`METHOD AND APPARATUS FOR
`ESTIMATING THE LEVEL OF ACOUSTIC
`FEEDBACK BETWEEN A LOUDSPEAKER
`AND MICROPHONE
`Inventor: Peter L. Cbu, Needham, Mass.
`Assignee: PictureTel Corporation, Peabody,
`Mass.
`Appl. No.: 837,729
`Filed:
`Feb. 19, 1992
`
`Related U.S. Application Data
`Continuation-in-part ofSer. No. 659,579, Feb. 21, 1991,
`which is a continuation-in-part of Ser. No. 640,477,
`Jan. 11, 1991, abandoned, Continuation of Ser. No.
`637,016, Jan. 4, 1991, abandoned.
`Int. a.s ................................................ H04J 1/00
`u.s. a .................................... 370/32.1; 379/406;
`381/83
`Field of Search ................... 370/32.1, 32; 381/46,
`381/47,66,71, 83; 379/345,392,410,406
`References Cited
`U.S. PATENT DOCUMENTS
`4,064,378 12/1977 Kitayama eta!. ................ 179/170.2
`4,126,770 11/1978 Tamura eta!. ................... 179/170.2
`4,232,400 11/1980 Yamamoto eta!. ................ 455/305
`4,479,036 10/1984 Yamamoto eta!. ............. 179/170.2
`4,525,856 6/1985 Admiraal eta!. ..................... 381/93
`4,539,675 9/1985 Fisher ................................. 370/32.1
`4,589,137 5/1986 Miller .................................... 381/94
`4,633,046 12/1986 Kitayama et al .................. 370/32.1
`4,658,426 4/1987 Chabries et al. ...................... 381/94
`4,677,676 6/1987 Eriksson ................................ 381/71
`4,677,677 6/1987 Eriksson ................................ 381/71
`4,683,590 7/1987 Miyoshi eta!. ....................... 381/71
`4,769,847 9/1988 Taguchi ................................ 381/94
`4,837,834 6/1989 Allie ...................................... 381/71
`4,965,823 10/1990 Nakagawa eta!. ................. 379/406
`5,117,418 5/1992 Chaffee eta! ...................... 370/32.1
`
`FOREIGN PATENT DOCUMENTS
`2191363 10/1986 United Kingdom .
`
`OTHER PUBLICATIONS
`P. L. Chu, "Quadrature Mirror Filter Design for an
`Arbitrary Number of Equal Bandwidth Channels",
`IEEE Trans. on ASSP, ASSP-33, No. 1, Feb., 1985, pp.
`203-218.
`P. L. Chu, "Fast Gaussian Random Noise Generator,"
`IEEE Trans. ASSP, ASSP-37, No. 10, Oct., 1989, pp.
`1593-1597.
`D. L. Duttweiler, "A Twelve-Channel Digital Voice
`·Echo Canceller," IEEE Transactions on Communica(cid:173)
`tions, COM-26, No. 5, May, 1978, pp. 647-653.
`
`(List continued on next page.)
`
`Primary Examiner-Douglas W. Olms
`Assistant Examiner-Shick Hom
`Attorney, Agent, or Firm-Fish & Richardson
`
`[57]
`ABSTRACT
`An improved echo cancelling device for reducing the
`effects of acoustic feedback between a loudspeaker and
`microphone in a communication system. The device
`includes an adjustable filter for receiving a loudspeaker
`signal and generating in response thereto an echo esti(cid:173)
`mation signal. The device subtracts the echo estimation
`signal from the microphone signal to produce an echo
`corrected microphone signal. During periods of time
`when the microphone signal is substantially derived
`from acoustic feedback between the loudspeaker and
`the microphone, the device adjusts transfer characteris(cid:173)
`tics of the filter to reduce the echo corrected micro(cid:173)
`phone signal. The improvement includes estimating
`from the adjusted transfer characteristics an energy
`transfer ratio representative of the ratio of the energy of
`the microphone signal to the energy of the loudspeaker
`signal. The device compares the microphone signal to
`the energy transfer ratio multiplied by the loudspeaker
`signal to identify periods of time when the microphone
`signal is substantially derived from acoustic feedback
`between the loudspeaker and the microphone.
`
`18 Qaims, 7 Drawing Sheets
`
`Petitioner Apple Inc.
`Ex. 1005, p. 1
`
`

`

`5,263,019
`Page 2
`
`OTHER PUBLICATIONS
`
`S. Gay, "Fast Converging Subband Acoustic Echo
`Cancellation Using RAP on the WE@ DSP/16A,"
`Proceedings of ICASSP, 1990, pp. 1141-1144.
`A. Gilloire, "Experiments with Sub-band Acoustic
`Echo Cancellers for Teleconferencing," Proceedings of
`ICASSP, 1987, pp. 2141-2144.
`M. J. Gingell, B. G. Hay, and L. D. Humphrey, "A
`Block Mode Update Echo Canceller Using Custon
`LSI," GLOBECOM Conference Record, vol. 3, Nov.,
`1983, pp. 1394-1397.
`D. G. Messerschmitt, "Echo Cancellation in Speech
`and Data Transmission," IEEE Journal on Selected
`Topics in Communications, IEEE Journal on Selected
`Topics in Communications, SAC-2 No. 2, Mar., 1984,
`pp. 283-296.
`
`Ying G. Tao, Kevin D. Kolwicz, C. W. K. Gritton, and
`Donald D. Duttweiler, "A Cascadable VLSI Echo
`Canceller", IEEE Journal on Selected Topics in Com(cid:173)
`munications, SAC-2, No.2, Mar., 1984, pp. 297-303.
`S. Yamamoto, S. Kitayama, J. Tamura, and H. Ishigami,
`"An Adaptive Echo Canceller with Linear Predictor,"
`The Transactions of the IECE of Japan, vol. E62, No.
`12, bee., 1979, pp. 851-857.
`R. Frenzel and M. E. Hennecke, "A Robust Echo Com(cid:173)
`pensator: Implementation & Realtime Measurements",
`IEEE ASSP Workshop on Applications on Signal Pro(cid:173)
`cessing to Audio & Acoutics, Oct. 20-23, 1991, New
`Paltz, N.Y.
`Hua Ye and Bo-Xiu Wu, "A New Double-Talk Detec(cid:173)
`tion Algorithm Based on the Orthogonality Theorm",
`IEEE Transaction on Communications, vol. 39, No. 11,
`Nov. 1991, pp. 1542-1545.
`
`Petitioner Apple Inc.
`Ex. 1005, p. 2
`
`

`

`U.S. Patent
`
`Nov. 16, 1993
`
`Sheet 1 of 7
`
`5,263,019
`
`"" ~--.._ _____ __,
`
`c..) --f-------.,
`I
`I
`I
`I
`I
`....... ....--~~ I
`~-"'-\1)~ I
`-~ ~ --~.&•-..... I
`
`~~~
`~ ..... (_
`~~
`"")
`
`• • •
`
`•
`
`Petitioner Apple Inc.
`Ex. 1005, p. 3
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov. 16, 1993
`Nov. 16, 1993
`
`Sheet 2 of 7
`Sheet 2 of 7
`
`5,263,019
`5,263,019
`
`|oy(WyS|x||Ta](1)%
`I}UNUNay,w\M07||ob9)|a02Ive(CUE|vieaEgSpaiT
`1@UILIWS\q|MOvAgdHHLUs
`
`(2)4s,re~[GaGo.aaa)|62UTS|MUU)|azims|g|eeeae]cus|AMTOINS
`SayaOf7|NOLLYNIIIOSs
`AMTNOINSSIMMSarn71)SyWOLLYMIITO|<a]C'S
`
`AINTNOTSS
`pce“71]az7/
`VUIAS
`TANSieee]|[oa||evaror0
`
`
`CETELLAY
`
`AINT0FES
`
`
`
`I
`
`Petitioner Apple Inc.
`Petitioner Apple Inc.
`Ex. 1005, p. 4
`Ex. 1005, p.4
`
`
`
`

`

`U.S. Patent
`
`Nov. 16, 1993
`
`Sheet 3 of 7
`
`5,263,019
`
`... ,
`...... ~ ~4~
`\
`
`H
`... ~~~ ~
`.. ~~~
`
`~
`...... ~~
`
`"'~JS~
`c::::i
`
`•~:::: 4l
`
`•~::::411'
`IE
`\..
`
`... ,
`~ 9:: w
`.. ~~~ ~
`.. ~~~
`
`~
`
`...... ~~
`
`"'~JS~
`c::::i
`
`....
`......
`"
`. ~ ·~
`l!
`\.....
`
`. ;:·~
`'-
`~ ~
`... ~~~ ~
`...... ~~
`. ~~a..;:
`
`"'~JS~
`
`~
`~
`~
`~
`
`r- r--- --- ...,
`r- ·- --- --- l
`r- r- ---r--- l
`I
`I
`I
`~·
`~·
`I
`I
`I
`I
`I ......
`I .
`I ~ ~
`I ~ ~
`I~
`Lot
`1::::
`~~ l'f I
`~~ I~ I
`I
`"'~
`I ~ "~~ ~ j
`r-+ ~~~ ~
`I ~ ~~" 1..- _lj
`:"'!:S~ ~ I
`~~~ ~
`'~~ ~
`I
`'~~
`I
`--~
`~:s
`I
`~
`I
`~
`~
`I
`I
`""-!::
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`:"'
`I
`I ·~
`I
`I
`I
`I
`I
`I
`I
`_______ _J
`
`~,
`
`1~
`
`It;
`~
`'
`~ ~
`I.e.:
`I Ail'
`:---
`I
`~
`T~ ......
`I ~
`_______ _J
`L
`
`~·
`~~~ ~
`"'
`+-
`: ·~ ~ '
`~ ~
`!'"':.
`.....,
`I
`L _____ ').J
`
`~
`
`~
`
`~·
`~
`'- ~
`,~
`~ '
`::::::!
`I.e.:
`~
`~"'
`......
`
`I
`L
`
`+-
`
`I
`
`~
`IrS
`
`IrS
`
`~
`
`Petitioner Apple Inc.
`Ex. 1005, p. 5
`
`

`

`U.S. Patent
`
`Nov. 16, 1993
`
`Sheet 4 of 7
`
`5,263,019
`
`J/1
`Nn • /SnfiJ/
`AGE•O
`TENP•O
`EnfiJ • Nn·Nn
`
`FIG.4A
`
`FIG.4B
`
`Petitioner Apple Inc.
`Ex. 1005, p. 6
`
`

`

`U.S. Patent
`
`Nov. 16, 1993
`
`Sheet 5 of 7
`
`5,263,019
`
`ENABLE TAP I'EIGHT
`ADJUSTNENT
`
`DISABLE TAP
`lfflfJHT ADJIJSTNENT
`
`116
`
`120
`
`IIECONPIJTE r; AS
`fOLLOJfS:
`r (h(/) -hr /-! )) 2
`
`I" L -/
`
`1•2
`
`NO
`
`FIG.5
`
`Petitioner Apple Inc.
`Ex. 1005, p. 7
`
`

`

`U.S. Patent
`
`Nov. 16, 1993
`
`Sheet 6 of 7
`
`5,263,019
`
`Ill
`
`CPNI'I/TE KEJIEKBEI?ATIPN EST/NATE
`I • L -I
`I: (/J(I)-11(1-!))2
`I. p
`
`NP
`
`FIG.6
`
`Petitioner Apple Inc.
`Ex. 1005, p. 8
`
`

`

`U.S. Patent
`
`Nov. 16, 1993
`
`Sheet 7 of 7
`
`5,263,019
`
`SELECT THE SAIIPLES Of THE BAND
`
`41(} ·-- L/11/TED ECHtJ COHHECTED IIICHtJPHtJNE
`
`SIGNAL ll'n (i) fHOII THE LAST T/fO
`SECONDS
`
`41! --........
`
`SEGIIENT THE SELECTED SAIIPLES
`INTtJ ltJtJ BLOCKS, 'EACH BLOCK
`COYEHING A !tJ msec TillE PEHIOD
`
`tJf ll'n fi) tJYEH EACH BLtJCK
`
`414 ·-- CALCULATE THE AYEHAGE ENEHGY
`.. ,
`41G -- ENEHGY (Jf THIS BLtJCK AS THE ENEHGY
`
`SELECT THE BLOCK lfiTH THE
`IIINIIIUII ENEHGY AND USE THE
`
`tJf THE ESTIIIATE Of ENEHGY tJf
`THE BACKGHOUNIJ NOISE IN BAND n
`tJf THE IIICHOPHtJNE SIGNAL
`
`FIG. 7
`
`Petitioner Apple Inc.
`Ex. 1005, p. 9
`
`

`

`1
`
`5,263,019
`
`METHOD AND APPARATUS FOR ESTIMATING
`THE LEVEL OF ACOUSTIC FEEDBACK
`BETWEEN A LOUDSPEAKER AND
`MICROPHONE
`
`This is a continuation-in-part of co-pending U.S. ap(cid:173)
`plication Ser. No. 07/659,579, filed Feb. 21, 1991 which
`is a continuation-in-part of U.S. application Ser. No.
`640,4 77, filed Jan. 11, 1991 (now abandoned) which is a
`continuation of U.S. application Ser. No. 637,016 filed
`Jan. 4, 1991 (now abandoned).
`
`2
`Electrically simulating the acoustic feedback is diffi(cid:173)
`cult since the acoustic feedback is determined by the
`acoustic characteristics of the room containing the mi(cid:173)
`crophone and speaker. This is complicated by variations
`S in the acoustic characteristics of different rooms and by
`the dramatic changes in a given room's characteristics
`which occur if the microphone or loudspeaker is
`moved, or if objects are moved in the room.
`To compensate for the changing characteristics of the
`10 room, many echo cancellation devices model the
`room's characteristics with an adaptive filter which
`adjusts with changes in the room. More specifically, the
`electric signal used to drive the telephone's loudspeaker
`BACKGROUND OF THE INVENTION
`is applied to a stochastic gradient least-means-squares
`The invention relates generally to reducing unwanted 15 adaptive filter whose tap weights are set to estimate the
`audio or acoustic feedback in a communication system,
`room's acoustic response. The output of the filter, be-
`and particularly to an adaptive acoustic echo cancella-
`lieved to estimate the acoustic echo, is then subtracted
`tion device for suppressing acoustic feedback between
`from the microphone signal to eliminate the component
`the loudspeaker and microphone of a telephone unit in
`of the microphone signal derived from acoustic feed-
`a teleconferencing system. The telephone unit of a typi- 20 back. The resultant "echo corrected" signal is then sent
`cal audio conferencing system includes a loudspeaker
`to listeners at the far end of the communication system.
`for broadcasting an incoming telephone signal into an
`To assure that the adaptive filter accurately estimates
`entire room. Similarly, the telephone's microphone is
`the room's response, the device monitors the echo cor-
`f
`rected signal. During moments when no one is speaking
`typically designed to pick up the voice 0 any person 25 into the microphone, the adaptive filter adjusts its tap
`within the room and transmit the voice to a remote
`weights such that the energy of the echo corrected
`telephone at the far end of the communication system.
`signal is at a minimum. In theory, the energy of the echo
`Unlike conventional hand held telephone sets, confer-
`corrected signal is minimized when the adaptive filter
`ence telephone units are prone to acoustic feedback
`removes from the microphone signal an accurate rep-
`between the loudspeaker unit and microphone. For 30 lica of the acoustic feedback. However, the adaptive
`example, a voice signal .which is broadcast into the
`process must be disabled whenever a person speaks into
`room by the loudspeaker unit may be picked up by the
`the microphone. Otherwise, the unit will attempt to
`microphone and transmitted back over the telephone
`adjust the tap weights in an effort to eliminate the
`lines. As a result, persons at the far end of the communi-
`speech.
`cation system hear an echo of their voice. The echo lags 35 Accordingly, echo cancellation devices which em-
`the person's voice by the round trip delay time for the
`ploy adaptive filters for estimating a room's response
`voice signal. Typically, the echo is more noticeable as
`typically include a "double-talk" detection device
`the lag between the person's voice and the echo in-
`which monitors the microphone signal to determine
`creases. Accordingly, it is particularly annoying in
`when a person is speaking into the microphone. One
`video conferencing systems which transmit both video 40 such detector, described in D. L. Duttweiler, "A
`and audio information over the same telephone lines.
`Twelve Channel Digital Echo Canceller", IEEE Trans.
`The additional time required to transmit video data
`On Comm., Volcom-26, No. 5, May 1978, declares
`increases the round trip delay of the audio signal,
`double talk when a sample of the microphone signal is
`thereby extending the lag between a person's voice and
`greater than or equal to one-half the largest sample of
`the echo.
`45. the loudspeaker signal within the last N samples, where
`N is a constant equal to the maximum delay between the
`Many conference telephones avoid echo by allowing
`loudspeaker and the microphone. If someone is speak-
`only half duplex communication (that is, by allowing
`communication over the phone line to occur in only one
`ing into the microphone, the energy of the microphone
`direction at a time) thereby preventing feedback. For
`signal is typically at least half that of the loudspeaker
`example, when the loudspeaker unit is broadcasting a so signal. Accordingly, the above described double talk
`voice, the telephone disables the microphone to prevent
`detector properly concludes that someone is speaking
`the loudspeaker signal from being fed back by the mi-
`into the microphone and disables the adaptive filter
`crophone.
`from adjusting its taps.
`If the loudspeaker and microphone are far apart from
`While a half duplex system avoids echo, it often cuts
`off a person's voice in mid-sentence. For example, when ss each other, the microphone includes little or no acous-
`both parties· speak simultaneously, the telephone unit
`tic feedback from the loudspeaker. Further when some-
`allows communication in only one direction, thereby
`one is speaking softly into the microphone, the energy
`clipping the voice of one party.
`of the soft voice component of the microphone signal is
`Some loudspeaker telephones employ echo cancella-
`not alone greater than half the energy of loudspeaker
`tion in an attempt to allow full-duplex communication 60 signal. Accordingly, the above described doubletalk
`without echo. Conventional echo cancellation devices
`detector falsely concludes that no one is speaking into
`attempt to remove from the microphone signal the com-
`the microphone and therefore enables the adaptive filter
`ponent believed to represent the acoustic feedback.
`to adjust its taps. The filter accordingly begins adjusting
`More specifically, these devices prepare an electric
`the taps in an effort to reduce the echo-corrected micro-
`signal which simulates the acoustic feedback between 65 phone signal to zero. Thus, by falsely concluding that
`the loudspeaker and the microphone. This electric sig-
`no one is speaking into the microphone, the device
`nal is subtracted from the microphone signal in an at-
`begins to cut off the voice of the person speaking into
`tempt to remove the echo.
`the microphone.
`
`Petitioner Apple Inc.
`Ex. 1005, p. 10
`
`

`

`5,263,019
`
`4
`signal to the room reverberation estimate. During peri(cid:173)
`ods of time when the echo corrected microphone signal
`is less than the room reverberation estimate, the device
`enables the signal clipper to attenuate the echo cor(cid:173)
`rected microphone signal. More specifically, for em(cid:173)
`bodiments in which the adjustable filter is a digital filter,
`the device calculates the room reverberation estimate
`according to the formula:
`
`REn(l) = Et X
`
`I=L-1
`l:
`I=P
`
`((hn(l) - hn(l - 2))2
`
`3
`If the loudspeaker is placed close to the microphone,
`the energy of the microphone signal may exceed half
`the energy of the loudspeaker signal regardless of
`whether someone is speaking into the microphone. For
`example, if the room includes ambient background
`noise such as generated by a fan, the microphone picks
`up this sound and adds it to the substantial acoustic
`feedback caused by the close proximity of the micro(cid:173)
`phone and loudspeaker. Accordingly, the energy of the
`microphone signal may exceed the half of the energy of 10
`the loudspeaker signal even when the loudspeaker is the
`only source of speech in the room. In this case, the
`above described doubletalk detector falsely concludes
`that someone is always speaking into the microphone
`and therefore permanently disables the adaptive filter 15 where REn(i) is the room reverberation estimate in
`from adjusting its taps.
`band n at tap i, E1 is the loudspeaker energy value, L is
`Therefore, one object of the present invention is to
`the number of taps for the fllter, P is a constant which
`provide an acoustic echo cancellation device which
`is slightly greater than the propagation time (in samples)
`includes an improved double talk detector for determin-
`for the acoustic signal to propagate from the loud-
`ing when someone is speaking into the microphone.
`20 speaker to the microphone, and hn(i) is the tap value of
`SUMMARY OF THE INVENTION
`filter tap j in band n (see also Equation lO supra).
`Other objects, features and advantages of the inven(cid:173)
`tion are apparent from the following description of
`particular preferred embodiments taken together with
`the drawings.
`
`The invention relates to an improved echo cancelling
`device for reducing the effects of acoustic feedback
`between a loudspeaker and microphone in a communi- 25
`cation system. The device includes an adjustable filter
`for receiving a loudspeaker signal and generating in
`response thereto an echo estimation signal. The device
`subtracts the echo estimation signal from the micro(cid:173)
`phone signal to produce an echo corrected microphone 30
`signal. During periods of time when the microphone
`signal is substantially derived from acoustic feedback
`between the loudspeaker and the microphone, the de(cid:173)
`vice adjusts transfer characteristics of the filter to re(cid:173)
`duce the echo in the echo corrected microphone signal. 35
`The improvement includes estimating from the adjusted
`transfer characteristics an energy transfer ratio repre(cid:173)
`sentative of the ratio of the energy of the microphone
`signal to the energy of the loudspeaker signal. The de(cid:173)
`vice compares the microphone signal to the energy 40
`transfer ratio multiplied by the loudspeaker signal to
`identify periods of time when the microphone signal is
`substantially derived from acoustic feedback between
`the loudspeaker and the microphone.
`In one embodiment, the adjustable filter is a digital 45
`filter having a plurality of taps. The value of the taps
`define the transfer characteristics of the adjustable fil(cid:173)
`ter. The device calculates the energy transfer ratio by
`first filtering a plurality of the tap values with a band(cid:173)
`pass filter to produce a plurality of filtered tap values. It 50
`then computes the square of each filtered tap value and
`sums the squared filtered tap values. More specifically,
`the device computes the energy transfer ratio from the
`plurality of the taps according to the equation:
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a block diagram of an echo cancellation
`device.
`FIG. 2 Is a block diagram of an echo cancellation
`device, showing the signal splitters in further detail.
`Fl G. 3 is a block diagram of a bank of adaptive filters
`for performing echo cancellation on a set ofbandlimited
`signals.
`FIGS. 4(a) and 4(b) are a flow chart illustrating a
`procedure U!!ed in updating the tap weights of an adapt(cid:173)
`ive filter.
`FIG. 5 is a flow chart illustrating a procedure for
`determining if the microphone signal includes near end
`speech.
`FIG. 6 is a flow chart illustrating a procedure for
`implementing a variable gain signal clipper.
`FIG. 7 is a flow chart illustrating a procedure for
`estimating the energy of the background noise in an
`echo corrected bandlimited microphone signal.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`Referring to FIG. 1, a microphone 10 converts
`speech and other acoustic signals in a room into an
`analog electronic microphone signal. The electronic
`signal is applied to input signal conditioner 12 which
`filters the signal with a 7 KHz low pass fllter and digi-
`55 tizes the filtered signal at a 16KHz sampling rate. The
`resultant digitized microphone signal m(z) (where z is
`an integer representing the time at which sample m(z)
`was taken, measured in terms of a number of samples at
`a 16KHz sampling rate) is applied to echo cancellation
`system 15 which processes the microphone signal to
`remove any echo components, and transmits the echo
`corrected signal to the far end of the communication
`system. Echo cancellation system 15 is preferably im(cid:173)
`plemented by a 60 MHz DSP16A processor.
`A digitized electronic speaker signal s(z), represent(cid:173)
`ing the voice of persons at the far end of the communi(cid:173)
`cation system, is received at the near end of the system.
`The speaker signal s(z) is applied to an output signal
`
`Gn =
`
`I=L-1
`l:
`/=2
`
`2
`((hn(l) - hn(l - 2))
`
`where Gn is the energy transfer ratio in band n, L is 60
`the total number oftaps for the filter, and hn(l) is the tap
`value for tap I in band n (see also Equation 6 supra).
`In another embodiment, the device includes a clipper
`for attenuating the echo corrected microphone signal
`during selected periods of time. The device calculates 65
`from the adjusted transfer characteristics a room rever(cid:173)
`beration estimate representative of room's reverbera(cid:173)
`tion. It then compares the echo corrected microphone
`
`Petitioner Apple Inc.
`Ex. 1005, p. 11
`
`

`

`5,263,019
`
`5
`6
`While clipping eliminates noticeable residual echo, it
`conditioner 33 which processes the signal, converting it
`introduces noticeable changes in background noise as it
`to an analog electronic signal. The analog signal is ap-
`plied is loudspeaker 32 which reproduces the voice
`is activated and deactivated. For example, assume the
`microphone picks up the sound of a fan operating in the
`signal, broadcasting the reproduced voice into the
`room. The digitized speaker signal s(z) is also applied to
`room at the near end of the communication system.
`Since this sound is not an echo, it tends to pass through
`echo cancellation system 15 for use in estimating the
`the echo cancellers 18. However, when center clipping
`echo contained in the microphone signal.
`Within echo cancellation system 15, m(z) is first
`engages to fully eliminate echo, it also suppresses the
`passed through a whitening filter 14 which spreads the
`sound of the fan. Thus, the listeners at the far end hear
`spectrum of m(z) more evenly across the bandwidth of 10 the fan drift in and out as clipping is engaged and disen-
`m(z) while preserving the voice information contained
`gaged. To eliminate this annoying side effect of center
`in m(z). The resultant whitened signal mw(z) generated
`clipping, the clipped signals are applied to a bank of
`by filter 14 is then applied to a splitter 16 which sepa-
`noise fillers which add to the clipped signals a noise
`rates mw(z) into twenty-nine distinct frequency bands
`signal which mimics the clipped background noise.
`and shifts each band limited signal into the baseband 15 After the bandlimited signals are processed by bank
`forming baseband signals mn(i).
`22 of noise fillers, they are applied to composer 24
`The bandlimited baseband signals mn(i) (where i rep-
`which assembles them into a composite signal cw(z).
`resents the time at which sample mn(i) is taken, mea-
`Finally, the composite signal cw(z) is applied to an in-
`sured in terms of a number of samples taken at a lower
`verse whitening fllter 26 which performs the inverse
`sample rate to be discussed below) are then applied to a 20 operation of the whitening filter 14, thereby returning
`bank 18 of echo cancellers which subtract from each
`the signal to a form ready for transmission to listeners at
`signal mn(i) an estimation of the echo in the band n. To
`the far end.
`estimate the echo in each band, the loudspeaker signal
`Referring to FIG. 2, the separation of the micro-
`s(z) is whitened and band filtered in the same manner as
`phone and speech signals into a set of bandlimited sig-
`the microphone signal m(z). More specifically, s(z) is 25 nals is now described in more detail. Within splitter 16,
`passed through a whitening filter 28 which is similar to
`the whitened microphone signal mw(z) is first applied to
`or identical to whitening filter 14. The whitened loud-
`a bank of digital bandpass filters 34 which separate
`speaker signal sw(z) is then separated by signal splitter
`mw(z) into its spectral components. The bandwidths of
`30 into its spectral components, represented by a set of
`the filters cover the entire 7 KHz frequency spectrum of
`twenty-nine bandpass loudspeaker signals Sb(i), and 30 mw(z) without gaps. Toward this end, the filter band-
`each component is shifted into the baseband. As will be
`widths preferably overlap.
`explained more fully below, each baseband loudspeaker
`Low complexity methods are known in the art for
`signal sn(i) is then passed through a corresponding least-
`implementing a bank of bandpass filters in which each
`means-squared filter (within the bank of echo cancellers
`filter has the same bandwidth. See e.g., R. F. Crochiere
`18) which models the response of the channel between 35 et al., "Multirate Digital Signal Processing, Prentice
`Hall, Englewood Cliffs, N.J., 1983; P. L. Chu, "Quadra-
`loudspeaker 32 and microphone 10 in the frequency
`band n. The output of each filter is used as the estimated
`ture Mirror Filter Design for an Arbitrary Number of
`echo signal to be subtracted from mn(i).
`Equal Bandwidth Channels," IEEE Trans on ASSP,
`Subtracting the estimated echo signal from the corre-
`ASSP-33, No. 1, February 1985 p.203-218. A bank of
`sponding band limited microphone signal mn(i) elimi- 40 filters made according to these techniques span frequen-
`nates most of the acoustic feedback between loud-
`cies from zero to one half the sampling rate of the signal
`speaker 32 and microphone 10 in band n. The remaining
`applied to the bank of filters. The microphone signal
`residual echo is typically not noticeable because the
`m(z) applied to the bank of bandpass filters 34 is sam-
`voice of persons speaking into microphone 10 tends to
`pled at 16KHz. Accordingly, a bank of filters imp1e-
`mask the presence of the residual echo. However, dur- 45 mented according to the sampled techniques covers
`ing moments when there is no such near end voice
`frequencies up to 8KHz, i.e., one half the sampling rate.
`signal, the residual echo is more apparent.
`However, since m(z) is previously low pass filtered by
`signal conditioner 12 to eliminate frequencies above 7
`To eliminate any noticeable residual echo, the echo
`corrected signals m'(i) are applied to a bank of twenty-
`KHz, the highest frequency filters in the bank which lie
`nine center clippers 20. Bank 20 includes a center clip- SO in the low pass filter's transition band may be ignored.
`per for each band limited microphone signal m' n(i).
`Several factors must be weighed in choosing the
`Each center clipper monitors a corrected signal m' n(i)
`number of fl.lters in the bank. For example, using a large
`to determine when it falls below a certain threshold.
`number of filters reduces the bandwidth of each filter,
`When m' n(i) drops below the threshold, the center clip-
`which, as be explained more fully below, reduces the
`per assumes that m' n(i) contains no near end speech. 55 number of computations required to process a given
`Accordingly the clipper begins gradually attenuating
`bandlimited signal. However, such reduction in band-
`the corrected signal m' n(i) to zero to eliminate the resid-
`width increases the delay introduced by each filter.
`ual echo in band n.
`Further, a large number of filters yield many bandli-
`Center clipping thus operates independently in each
`mited signals mn(i), thereby increasing the computa-
`band. If a narrow band voice signal (e.g., a high pitched 60 tional cost of implementing the bandpass filters, echo
`cancellers, center clippers and noise fillers. Accord-
`voice or a whistle) is applied to the microphone, center
`clipping will highly attenuate the microphone signal in
`ingly, in the preferred embodiment, the bank of band-
`all silent bands, allowing the bands containing the nar-
`pass filters 34 contains 32 filters covering frequencies up
`to 8 KHz. Only the lower 29 fllters are used, however,
`row band voice signal to pass without clipping. Thus,
`echo is completely eliminated in all attenuated bands 65 since the input microphone signal m(z) has only a 7
`KHz bandwidth.
`containing no near end speech. In the other bands, the
`echo cancellers 18 remove most of the echo, any resid-
`Each filter 34 is a 192 tap, symmetric FIR (finite
`ual echo being masked by the narrow band voice signal.
`impulse response) filter having a magnitude response
`
`Petitioner Apple Inc.
`Ex. 1005, p. 12
`
`

`

`8
`7
`the estimated echo from the corresponding bandlimited
`equal to the square root of a raised cosine. This response
`microphone signal. Adaptive filter SO, for example,
`is preferable since it gives a smooth transition from
`removes the acoustic echo in band n from the bandli-
`passband to stopband. Each filter thus has a 250 Hz, 3
`dB bandwidth and a 500Hz, 40 dB bandwidth. Attenua-
`mited microphone signal, mn(i). Toward this end, adapt-
`tion at the 500 Hz bandwidth must be high to prevent 5 ive filter SO includes a least-means-square ("LMS")
`filter 52 whose tap weights are chosen to model the
`aliasing.
`response of the channel between loudspeake{ 32 and
`Each bandlimited signal (with the exception of the
`output oflowpass filter 34(a) which is baseband), is then
`microphone 10 in the frequency band n.
`applied to a frequency shifter 36 which modulates the
`The bandlimited loudspeaker signal Sn(i) in the same
`bandlimited signal to shift its frequency spectrum 10 band, n, is applied to the input of LMS filter 52. In
`downward to the baseband.
`response, filter 52 generates an estimate en(i) of the
`Since the full band microphone signal m(z) is sampled
`acoustic feedback of sn(i). The estimated echo en(i) is
`at 16KHz, each band limited signal is also sampled at
`then applied to a subtracter 54 which removes the esti-
`the same 16KHz rate. However, since each bandlimited
`mated echo signal from mn(i) to produce an echo cor-
`signal has a much narrower bandwidth than the micro- 15 rected signal m' n(i).
`phone signal, many of these samples are redundant.
`Adaptive filter SO continuously monitors the cor-
`Accordingly, each bandlimited signal is decimated by a
`rected signal m'n(i) to determine whether the LMS filter
`decimation un:t 38 to reduce the sampling rate to ap-
`52 accurately models the response of the channel be-
`proximately the Nyquist rate, that is, twice the band-
`tween the loudspeaker and microphone. More specifi-
`width of the filter 34. In the preferred embodiment, 20 cally, echo canceller 18 includes for each band n, a local
`decimation units 38 subsample at 1 KHz, or one six-
`speech detector 56 which determines whether the band-
`teenth of the original sampling rate. This dram~tically
`limited microphone signal mn(i) includes any near end
`reduces the number _of sampl~s, t~er~by reduc~ng the
`speech. When no one is speaking into the microphone,
`number of computations requ1red m 1mplementmg the
`the microphone signal mn(i) contains only the acoustic
`subsequent echo cancellation, center clipping and noise 25 feedback from the loudspeaker and any background
`filli~g. ~andp~ss filters. 34, frequenci_es shifters 36_ and
`noise from the room. Thus, if LMS filter 52 properly
`decJmatJon umts 38 are Implemented m a ~ eaver smgle
`models the room response, the corrected signal m' n(i)
`sid_eband modu.~ator ~tructur~ ~s pro~osed m R. E. _cr?,-
`should be approximately zero during this time (assum-
`chJer~ et al, Mult1rate Dig_Ital Signal Pro~essmg '
`ing the background noise is relatively small). Accord-
`Prentice ~all, Englewood Chffs, N.J. (1983) mcorpo- 30 ingly, if m'n(i) is too large during a moment when local
`rated here~n by reference.
`.
`speech detector 56 indicates that no one is speaking at
`~h~ whi_tened loudspeaker Signal s,.,(z) must also be