United States Patent
`United States Patent [191
`[19]
`Lindemann et at
`Lindemann et a].
`
`111I1
`
`1111111111111111111111111111111111111111111111111111111111111
`US005651071A
`US005651071A
`[11] Patent Number:
`[11] Patent Number:
`[45] Date of Patent:
`[45] Date of Patent:
`
`5,651,071
`5,651,071
`Jut 22, 1997
`Jul. 22, 1997
`
`NOISE REDUCTION SYSTEM FOR
`[54]
`[54] NOISE REDUCTION SYSTEM FOR
`BINAURAL HEARING AID
`BINAURAL HEARING AID
`
`[75]
`[75]
`
`Eric Lindemann; John Laurence
`Inventors: Eric Lindemann; John Laurence
`Inventors :
`Melanson, both of Boulder, Colo.
`Melanson. both of Boulder. Colo.
`
`[73] Assignee: AudioLogic, Inc.. Boulder. C010.
`[73] Assignee: AudioLogic, Inc., Boulder, Colo.
`
`Appl. No.: 123,503
`[21]
`[21] Appl. No.: 123,503
`Filed:
`Sep. 17, 1993
`[22]
`[22] Filed:
`Sep. 17, 1993
`Int. CI.6
`[51]
`[51]
`..................................................... H04R 25/00
`Int. (:1.6 ................................................... .. H04R 25/00
`[52]
`[52] U.S. CI •........................ 381/68.2; 381/68.4; 395/2.35
`US. Cl. ........... ..
`381/682; 381/684; 395/235
`[58]
`[58] Field of Search ........................... 381/68.2, 68, 68.4.
`Field of Search ......................... .. 381/682. 68. 68.4.
`381/60, 26, 74, 94,46,47; 395/2.35, 2.12,
`381/60. 26. 74. 94. 46. 47; 395/235. 2.12.
`2.37,2.42
`2.37. 2.42
`
`[56]
`[56]
`
`References Cited
`References Cited
`
`U.S. PATENT DOCUlVlENTS
`U.S. PATENT DOCUMENTS
`
`4,628,529
`4,628,529 12/1986 Borth et aI ..
`12/1986 Borth et al. .
`4,630,305
`4,630,305 12/1986 Borth et aI ..
`12/1986 Borth et a1. .
`4,868,880
`9/1989 Bennett, Jr ..
`4,868,880
`9/1989 Bennett, Jr. .
`4,887,299 12/1989 Cummins et aI ..
`4,887,299
`12/1989 Cummins et a1. .
`7/1991 Chabries et aI ..
`5,029,217
`5,029,217
`7/1991 Chabn'es et a1. .
`8/1994 Hall, II et aI .......................... 39512.35
`5,341,452
`5,341,452
`8/1994 Hall, H et al. ....................... .. 395/235
`
`OTHER PUBLICATIONS
`OTHER PUBLIClITIONS
`“Multimicrophone Signal-Processing Technique to Remove
`"Multimicrophone Signal-Processing Technique to Remove
`Reverberation from Speech Signals” by J. Allen et al. vol.
`Reverberation from Speech Signals" by 1. Allen et al, vol.
`62, No.4, Oct. 1977. pp. 912-915.
`62. No. 4. Oct. 1977. pp. 912-915.
`“An Alternative Approach to Linearly Constrained Adaprive
`"An Alternative Approach to Linearly Constrained Adaprive
`Beamforruing" By L. 1. Griffiths et al, IEEE Transactions,
`Beamforming” By L. J. Gri?lths et al. IEEE Transactions.
`vol. AP-30. No. L Jan. 1982, pp. 27-34.
`vol. AP-30. No. 1. Jan. 1982. pp. 27-34.
`“Speech Enhancement Using A Minimum Mean-Square
`"Speech Enhancement Using A Minimum Mean-Square
`Error Short-Time Spectral Amplitude Estimator” By Y.
`Error Short-Time Spectral Amplitude Estimator" By Y.
`Ephraim et al, IEE Transactions. Dec. 1984. No.6.
`Ephraim et al. IEE Transactions. Dec. 1984. No. 6.
`Article Entitled "Extension of a Binaural Cross-Correlation
`Article Entitled “Extension of a Binaural Cross-Correlation
`Model by Contralateral Inhibition” By W. Lindemann. J.
`Model by Contralateral Inhibition" By W. Lindemann. 1.
`Acoust. Soc. Am. 80(6), Dec. 1986, pp. 1608-1622.
`Acoust. Soc. Am. 80(6). Dec. 1986. pp. 1608-1622.
`‘Multimicrophone Adaptive Beamforming for Interference
`''Multimicrophone Adaptive Bearnforming for Interference
`Reduction In Hearing Aids" by P. Peterson et al. Journal of
`Reduction In Hearing Aids” by P. Peterson et al. Journal of
`Rehabilitation Research and Development. vol. 24. No. 4.
`Rehabilitation Research and Development. vol. 24. No.4.
`pp. 103-110.
`pp. 103-110.
`
`“Evaluation of Two Voice-Separation Algorithms Using
`"Evaluation of 1\\'0 Voice-Separation Algorithms Using
`Normal-Hearing and Hearing-Impaired Listeners” By R.
`Normal-Hearing and Hearing-Impaired Listeners" By R.
`Stubbs et al. 1. Acoust. Soc .• Oct. 1988.
`Stubbs et al. J. Acoust. 800.. Oct. 1988.
`“Improvement of Speech Intelligibility In Noise Develop
`"Improvement of Speech Intelligibility In Noise Develop(cid:173)
`ment and Evaluation of a New Directional Hearing Instru(cid:173)
`ment and Evaluation of a New Directional Hearing Instru
`ment Based On Array Technology” By W. Soede, Delft
`ment Based On Array Technology" By W. Soede. Delft
`Univ. of Technology.
`Univ. of Technology.
`Article Entitled “Evaluation of An Adaptive Beamforming
`Article Entitled "Evaluation of An Adaptive Beamforruing
`Method for Hearing Aids” By J. Greenberg et al. J. Acoust.
`Method for Hearing Aids" By J. Greenberg et al, J. Acoust.
`Soc. Am. 91 (3). Mar. 1992. pp. 1662-1676.
`Soc. Am. 91 (3), Mar. 1992. pp. 1662-1676.
`“Digital Signal Processing for Binaural Hearing Aids” By
`"Digital Signal Processing for Binaural Hearing Aids" By
`Kollmeier et al. Proceedings International Congress on
`Kollmeier et al. Proceedings International Congress on
`Acoustics. 1992. Beijing. China.
`Acoustics. 1992. Beijing. China.
`Article Entitled “Cocktail-Party-Processing: Concept and
`Article Entitled "Cocktail-Party-Processing: Concept and
`Results” By M. Bodden. Bodden Proceedings. 1992.
`Results" By M. Bodden, Bodden Proceedings, 1992,
`Beijing. China.
`Beijing. China.
`
`(List continued on next page.)
`(List continued on next page.)
`
`Primary Examiner—Curtis Kuntz
`Primary Examiner-Curtis Kuntz
`Assistant Examiner—Hl1yen D. Le
`Assistant Examiner-Huyen D. Le
`Attorney, Agent, or Firm-Homer L. Knearl; Holland & Hart
`Attorney, Agent, or F irm—H0me1' L. Knearl; Holland & Hart
`[57]
`ABSTRACT
`[57]
`ABSTRACT
`
`In this invention noise in a binaural hearing aid is reduced
`In this invention noise in a binaural hearing aid is reduced
`by analyzing the left and right digital audio signals to
`by analyzing the left and right digital audio signals to
`produce left and right signal frequency domain vectors and
`produce left and right signal frequency domain vectors and
`thereafter using digital signal encoding techniques to pro
`thereafter using digital signal encoding techniques to pro(cid:173)
`duce a noise reduction gain vector. The gain vector can then
`duce a noise reduction gain vector. The gain vector can then
`be multiplied against the left and right signal vectors to
`be multiplied against the left and right signal vectors to
`produce a noise reduced left and right signal vector. The cues
`produce a noise reduced left and right signal vector. The cues
`used in the digital encoding techniques include
`used in the digital encoding techniques include
`directionality. short term amplitude deviation from long
`directionality, short term amplitude deviation from long
`term average, and pitch. In addition, a multidimensional
`term average. and pitch. In addition. a multidimensional
`gain function based on directionality estimate and amplitude
`gain function based on directionality estimate and amplitude
`deviation estimate is used that is more effective in noise
`deviation estimate is used that is more effective in noise
`reduction than simply summing the noise reduction results
`reduction than simply summing the noise reduction results
`of directionality alone and amplitude deviations alone. As
`of directionality alone and amplitude deviations alone. As
`further features of the invention. the noise reduction is
`further features of the invention. the noise reduction is
`scaled based on pitch-estimates and based on voice detec(cid:173)
`scaled based on pitch-estimates and based on voice detec
`tion.
`tion.
`‘
`
`14 Claims, 5 Drawing Sheets
`14 Claims, 5 Drawing Sheets
`
`242
`
`DE-
`EMPHASIS
`
`OUTPUT
`LEFT
`
`230
`
`..
`
`FFT
`
`WINDOW
`
`it
`
`202
`
`234
`
`238
`
`152
`INNER
`PRODUCT
`
`$68
`
`‘I56
`
`+
`
`*
`
`PITCH
`GAIN
`
`_ BEAM
`
`BAND
`SMOOTH
`
`25589 ‘5%
`GAIN
`
`154
`
`158
`
`GAIN
`ADJUST
`
`200
`
`I
`VOICE
`
`DETECT
`5cm:
`
`149
`
`151
`
`FFT
`
`+ 206
`
`*
`
`|
`
`204
`232
`
`236
`
`240
`
`:8
`
`rn
`
`wmnow
`
`DE-
`EMPHASIS
`244
`
`OUTPUT
`RIGHY
`
`RTL345-1_1027-0001
`
`

`
`5,651,071
`5,651,071
`Page 2
`Page 2
`
`OTHER PUBLICATIONS
`OTHER PUBLICATIONS
`
`“Microphone Array Speech Enhancement In Overdeter
`"Microphone Array Speech Enhancement In Overdeter(cid:173)
`mined Signal Scenarios” By R. Slyh et aL. Proceedings
`mined Signal Scenarios" By R. Slyh et al .. Proceedings
`IEEE International Conference on on Acoustics. Speech and
`IEEE International Conference on on Acoustics. Speech and
`Signal Processing. II-347—I1—350.
`Signal Processing. ll-347-II-350.
`
`“Separation of Speech from Interfering Speech By Means of
`"Separation of Speech from Interfering Speech By Means of
`Harmonic Selection" by T. Parsons, J. Acoust. Soc. Am .• vol.
`Harmonic Selection” by T. Parsons, J. Acoust. Soc. Am.. vol.
`60. No.4. Oct. 1976, pp. 911-918.
`60. No. 4. Oct. 1976, pp. 911-918.
`“Suppression of Acoustic Noise In Speech Using Spectral
`"Suppression of Acoustic Noise In Speech Using Spectral
`Subtraction" By S. Boll. IEEE Transactions on Acoustics,
`Subtraction” By S. Boll. IEEE Transactions on Acoustics,
`Speech and Signal Processing. vol. ASSP-27. No. 2. Apr.
`Speech and Signal Processing. vol. ASSP-27. No.2. Apr.
`1979. pp. 113-120.
`1979. pp. 113-120.
`
`RTL345-1_1027-0002
`
`

`
`139~
`
`I STEERINGh
`GAIN
`~nltfHEAsiS H STEERING I--( h
`*
`
`ALLPASS
`
`LEFT IN
`
`OUTPUT
`LEFT
`
`230
`
`144
`
`150
`
`156
`
`.. -
`
`~ L' ~~L
`1 ....... ,.. .. 1
`202
`176=vr;
`.1
`
`T
`
`BAND
`SMOOTH
`
`BEAM
`SPECTRAL
`SUBTRACT
`GAIN
`
`SUM
`MAGSQ
`
`158 I
`
`GAIN
`ADJUST
`
`200
`
`RIGHT IN
`
`PRE(cid:173)
`EMPHASIS
`
`STEERING
`ALLPASS
`
`VOICE 1 I
`DETECT
`GAIN
`SCALE
`
`206 I
`
`232--->--"
`
`1 41
`
`145
`
`STEERING 1-1 - - - '
`GAIN
`
`FI G. 1
`
`OUTPUT
`RIGHT
`
`Lj
`•
`00
`•
`~
`~
`
`~ g
`
`~
`
`~ = :-
`
`N
`~
`~
`\C
`\C
`~
`
`r:FJ
`t:I'"
`~
`
`~ -~
`
`0
`'""" Ul
`
`01
`'-8 =" 01
`'-8 =
`
`......:t
`~
`
`~
`
`RTL345-1_1027-0003
`
`

`
`u.s. Patent
`US. Patent
`
`Jul. 22, 1997
`JuI. 22, 1997
`
`Sheet 2 0f 5
`Sheet 2 of 5
`
`5,651,071
`5,651,071
`
`INNER
`INNER
`PRODUCT
`PRODUCT
`
`SUM MAG
`SQUARE
`
`INPUT
`REAL L
`
`INPUT
`IMAG L
`
`INPUT
`REAL R
`
`INPUT
`IMAG R
`
`FIG.2
`
`NOTE: THIS CIRCUIT IS
`NOTE: THIS CIRCUIT IS
`REPEATED FOR EVERY
`REPEATED F
`EVERY
`FREQUENCY F OF THE FFT
`FREQUENCY F
`THE EFT
`
`190
`19D
`
`INTEGRATE
`INTEGRATE
`TOTAL J--~
`TOTAL
`POWER
`POWER
`
`PITCH
`PITCH
`CONFIDENCE
`CONFIDENCE
`
`MAXIMUM
`MAXIMUM
`DOT
`DOT
`PRODUCT
`PRODUCT
`
`184
`
`DOT
`' PRODUCT
`
`SELECT
`SELECT
`MAXIMUM
`MAXIMUM
`
`PITCH
`GAIN
`
`182
`FFT sum >~I '2
`
`FFT
`SUM
`
`FO
`
`HARMONIC
`HARMONIC
`GRID
`GR
`TABLE
`TA
`
`186 J
`186
`
`t - - - " - - - - - -.. SELECT
`‘ SELECT
`GRID
`GRID
`
`FIG.6
`FIG.6
`
`192
`
`RTL345-1_1027-0004
`
`

`
`u.s. Patent
`US. Patent
`
`Jul. 22, 1997
`Jul. 22, 1997
`
`Sheet 3 of 5
`Sheet 3 of 5
`
`5,651,071
`5,651,071
`
`0-7
`
`NO SMOOTHING
`NO SMOOTHING
`
`0-7
`
`(
`6-17 F 4 POINT COSINE KERNALT 8-15
`4 POINT COSINE KERNAL ) 8-15
`6-17
`SMOOTHING FILTER
`SMOOTHING FILTER
`J
`13-20 6 POINT COSINE KERNAL\16—23
`13-20 6 POINT COSINE KERNAL 16-23
`SMOOTHING FILTER
`SMOOTHING FILTER
`
`l,
`
`INNER
`INNER
`PRODUCT
`PRODUCT
`gégTgoRmT
`128 POINT
`VECTOR
`
`J
`20-35 a POINT cOsINE KERNAL ‘24-31
`20-35 8 POINT COSINE KERNAL )24-31
`SMOOTHING FILTER
`SMOOTHING FILTER
`J
`—
`12 POINT COSINE KERNAL 32-47
`26-53 12 POINT COSINE KERNAL 32-47
`SMOOTHING FILTER
`SMOOTHING FILTER
`~
`J
`38-82 ( 20 POINT COSINE KERNAL l48-72
`(I) N
`2O POINT COSINE KERNAL‘ 48—72
`-
`[N 0)
`l
`SMOOTHING FILTER
`SMOOTHING FILTER
`57-127 32 POINT cOsINE KERNAC73-127
`57-127 32 POINT COSINE KERNAL 73-127
`SMOOTHING FILTER
`SMOOTHING FILTER
`)
`
`)
`
`L J
`FIG.3A
`FIG.3A
`
`157
`157
`
`0-7
`‘0:’.
`
`NO SMOOTHING
`NO SMOOTHING
`
`0-7
`0-7
`
`5-17
`6-17
`
`‘
`
`MAG sa
`MAG SQ
`SUM
`SUM
`128 POINT
`128 POINT
`VECTOR
`
`VECTOR Li L A)
`
`j
`l
`J
`J
`( 4 POINT COSINE KERNAL 8-15
`4 POINT cOsINE KERNAL‘ 8-15
`SMOOTHING FILTER
`l
`SMOOTHING FILTER
`J
`13-20 j 6 POINT COSINE KERNAL ,16-23
`13-20 6 POINT COSINE KERNAL 15-23
`SMOOTHING FILTER
`SMOOTHING FILTER
`I
`J
`<
`20-35 J 8 POINT COSINE KERNAL 124-31
`20-35 8 POINT COSINE KERNAL 24~31
`SMOOTHING FILTER
`I
`l
`)
`SMOOTHING FILTER
`26—53 12 POINT COSINE KERNAL\32—47
`26-53 I 12 POINT COSINE KERNAL 132-47
`SMOOTHING FILTER
`SMOOTHING FILTER
`J
`L
`38-82 I 20 POINT COSINE KERNAL 148-72
`38-82 20 POINT COSINE KERNAL\ 48-72
`SMOOTHING FILTER
`SMOOTHING FILTER
`J
`I
`1
`57-127 32 POINT COSINE KERNAL 173-127
`57—127 32 POINT COSINE KERNAL 73-127
`SMOOTHING FILTER
`J
`SMOOTHING FILTER
`'
`K
`157
`FIG.3B
`'57
`FIOBB
`
`INNER
`INNER
`PRODUCT
`PRODUCT
`AVERAGE
`AVERAGE
`128 POINT
`128 POINT
`VECTOR
`VECTOR
`
`MAG sa
`“A6551:
`SUM
`AVERAGE
`AVERAGE
`128 POINT
`128 POINT
`VECTOR
`VECTOR
`
`RTL345-1_1027-0005
`
`

`
`u.s. Patent
`US. Patent
`
`J ul. 22, 1997
`Jul. 22, 1997
`
`Sheet 4 of 5
`Sheet 4 of 5
`
`5,651,071
`5,651,071
`
`160
`
`INNER
`1-POLE
`1-POLE
`INNER
`PRODUCT >---~ LOWPASS
`LOWPASS
`PRODUCT
`
`166
`
`IF d<O
`THEN
`d=O
`
`IF F<500HZ
`THEN d8
`THEN dB
`IF F<1000HZ
`IF F<1000HZ
`THEN d4
`THEN d 4
`IF F<25OOHZ
`IF F<2500HZ
`THEN d2
`THEN d2
`ELSE d
`ELSE d
`
`174
`
`168
`
`2D
`2D
`GAIN
`GAIN
`FUNCTION
`FUNCTION
`TABLE
`TABLE
`
`‘><GAIN
`
`GAIN
`
`170
`
`NOTE: THIS CIRCUIT IS
`NOTE: THIS CIRCUIT IS
`REPEATED FOR EVERY
`REPEATED FOR EVERY
`FREQUENCY F OF THE FFT
`FREQUENCY F OF THE FFT
`FIG.4
`
`162
`
`1-POLE
`LOWPASS
`
`ES EG 08
`L5 TA PA
`PP E Ew _ w mm W0
`LONG TERM
`AVERAGE
`M PPTII
`RE LS
`(ONE-POLf
`l.OWPASS
`
`212
`
`TOTAL
`TOTAL
`MAG SO
`1 12
`
`210
`
`TOTAL
`TOTAL
`MAG SO
`MAG SQ
`1 12
`| I2
`214
`
`D ME
`MR A MU
`UQ
`SS
`
`L+R NOISE
`REDUCED
`FREO VECTOR
`
`L+R FREQ
`L+R FREQ
`VECTOR
`VECTOR
`
`NOISE
`REDUCTION
`GAIN
`
`MAG SQ 210%,
`
`218
`
`ATTACK /
`ATTACK/
`RELEASE
`RELEASE
`TWO
`POLE
`2164
`216
`
`10
`
`222
`222
`
`Es
`
`LIMIT
`LIMIT
`TO
`TO
`1
`1
`
`220/J
`220
`
`ADJUSTED
`ADJUSTED
`NOISE
`NOISE
`REDUCTION
`REDUCTION
`GAIN
`GAIN
`
`224
`
`226
`FIG.7
`FIG]
`
`RTL345-1_1027-0006
`
`

`
`u.s. Patent
`US. Patent
`
`D
`
`Jul. 22, 1997
`Jul. 22, 1997
`
`Sheet 5 of 5
`Sheet 5 of 5
`
`5,651,071
`5,651,071
`
`SPAD
`SPAD
`
`SPAD
`SPAD
`
`C:DGAIN FOR
`2D GAIN FOR
`SERIAL
`SERIAL
`CONNECTION
`CONNECTION
`
`FIG.SA
`FIG.5A
`
`D
`
`20
`2D
`GENERALIZED
`GENERALIZED
`GAIN
`GAIN
`
`FIG.58
`FIG.5B
`
`RTL345-1_1027-0007
`
`

`
`5,651,071
`5,651,071
`
`1
`1
`NOISE REDUCTION SYSTEM FOR
`NOISE REDUCTION SYSTEM FOR
`BINAURAL HEARING AID
`BINAURAL HEARING AID
`
`CROSS REFERENCE TO RELATED
`CROSS REFERENCE TO RELATED
`APPLIC1XITONS
`APPLICATIONS
`
`The present invention relates to patent application entitled
`The present invention relates to patent application entitled
`“Binaural Hearing Aid” Ser. No. 08/ 123.499. ?led Sep. 17.
`"Binaural Hearing Aid" Ser. No. 08/123,499. filed Sep. 17.
`1993. which describes the system architecture of a hearing
`1993. which describes the system architecture of a hearing
`aid that uses the noise reduction system of the present 10
`aid that uses the noise reduction system of the present
`invention.
`invention.
`
`2
`2
`The frequency domain approaches which have been pro
`The frequency domain approaches which have been pro(cid:173)
`posed {7. 8. 9} have performed better than delay-and-sum or
`posed {7. 8. 9} have performed better than delay-and-sum or
`adaptive ?lter approaches in reverberant listening environ
`adaptive filter approaches in reverberant listening environ(cid:173)
`ments and function with only two microphones. The prob
`ments and function with only two microphones. The prob-
`lems related to the previously-published frequency domain
`s lems related to the previously-published frequency domain
`approaches have included unacceptably long input-to-output
`approaches have included unacceptably long input-to-output
`time delay. distortion of the desired signal. spatial aliasing at
`time delay. distortion of the desired signal. spatial aliasing at
`high frequencies. and some di?iculty in reverberant envi
`high frequencies. and some difficulty in reverberant envi-
`ronments (although less than for the adaptive filter case).
`ronments (although less than for the adaptive ?lter case).
`While beamforming uses directionality to separate
`While beamforming uses directionality to separate
`desired signal from undesired signal. spectral subtraction
`desired signal from undesired signal. spectral subtraction
`makes assumptions about the differences in statistics of the
`makes assumptions about the differences in statistics of the
`undesired signal and the desired signal. and uses these
`undesired signal and the desired signal. and uses these
`differences to separate and attenuate the undesired signal.
`differences to separate and attenuate the undesired signal.
`The undesired signal is assumed to be lower in amplitude
`The undesired signal is assumed to be lower in amplitude
`then the desired signal and/or has a less time varying
`then the desired signal and/or has a less time varying
`spectrum. If the spectrum is static compared to the desired
`spectrum. If the spectrum is static compared to the desired
`signal (speech). then a long-term estimation of the spectrum
`signal (speech). then a long-term estimation of the spectrum
`will approximate the spectrum of the undesired signal. This
`will approximate the spectrum of the undesired signal. This
`20 spectrum can be attenuated. If the desired speech spectrum
`spectrum can be attenuated. If the desired speech spectrum
`is most often greater in amplitude and/or uncorrelated with
`is most often greater in amplitude and/or uncorrelated with
`the undesired spectrum. then it will pass through the system
`the undesired spectrum. then it will pass through the system
`relatively undistorted despite attenuation of the undesired
`relatively undistorted despite attenuation of the undesired
`spectrum. Examples of work in spectral subtraction include
`spectrum. Examples of work in spectral subtraction include
`references {11. 12, 13}.
`25 references {ll. 12, 13}.
`25
`Pitch-based speech enhancement algorithms use the
`Pitch-based speech enhancement algorithms use the
`pitched nature of voiced speech to attempt to extract a voice
`pitched nature of voiced speech to attempt to extract a voice
`which is embedded in noise. A pitch analysis is made on the
`which is embedded in noise. Apitch analysis is made on the
`noisy signal. If a strong pitch is detected. indicating strong
`noisy signal. If a strong pitch is detected. indicating strong
`voiced speech superimposed on the noise. then the pitch can
`voiced speech superimposed on the noise. then the pitch can
`be used to extract harmonics of the voiced speech. removing
`be used to extract harmonics of the voiced speech. removing
`most of the uncorrelated noise components. Examples of
`most of the uncorrelated noise components. Examples of
`work in pitch-based enhancement are references {17. 18}.
`work in pitch-based enhancement are references {17. 18}.
`
`BACKGROUND OF THE INVENTION
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention:
`1. Field of the Invention:
`This invention relates to binaural hearing aids. and more 15
`This invention relates to binaural hearing aids. and more
`particularly. to a noise reduction system for use in a binaural
`particularly. to a noise reduction system for use in a binaural
`hearing aid.
`hearing aid.
`2. Description of Prior Art:
`2. Description of Prior Art:
`Noise reduction. as applied to hearing aids. means the
`Noise reduction. as applied to hearing aids. means the
`attenuation of undesired signals and the ampli?cation of
`attenuation of undesired signals and the amplification of
`desired signals. Desired signals are usually speech that the
`desired signals. Desired signals are usually speech that the
`hearing aid user is trying to understand. Undesired signals
`hearing aid user is trying to understand. Undesired signals
`can be any sounds in the environment which interfere with
`can be any sounds in the environment which interfere with
`the principal speaker. These undesired sounds can be other
`the principal speaker. These undesired sounds can be other
`speakers. restaurant clatter. music. traffic noise. etc. There
`speakers. restaurant clatter. music. tra?ic noise. etc. There
`have been three main areas of research in noise reduction as
`have been three main areas of research in noise reduction as
`applied to hearing aids: directional beamforrning. spectral
`applied to hearing aids: directional beamforming. spectral
`subtraction. pitch-based speech enhancement.
`subtraction. pitch-based speech enhancement.
`The pmpose of beamforming in a hearing aid is to create 30
`The purpose of beamforming in a hearing aid is to create
`an illusion of "tunnel hearing" in which the listener hears
`an illusion of “tunnel hearing” in which the listener hears
`what he is looking at but does not hear sounds which are
`what he is looking at but does not hear sounds which are
`coming from other directions. If he looks in the direction of
`coming from other directions. If he looks in the direction of
`a desired sound-e.g.. someone he is speaking to-then
`a desired sound—e.g.. someone he is speaking to-—-then
`other distracting sounds—e.g.. other speakers-will be
`other distracting sounds-e.g.. other speakers-will be 35
`35
`attenuated. A beamformer then separates the desired "on(cid:173)
`attenuated A bearnformer then separates the desired “on
`axis” (line of sight) target signal from the undesired “off
`axis" (line of sight) target signal from the undesired "off(cid:173)
`axis" jammer signals so that the target can be amplified
`axis” jammer signals so that the target can be ampli?ed
`while the jan1iller is attenuated.
`while the jammer is attenuated.
`Researchers have attempted to use beamforming to 40
`Researchers have attempted to use bearnforming to
`40
`improve signal-to-noise ratio for hearing aids for a number
`improve signal-to-noise ratio for hearing aids for a number
`of years {References 1.2.3.7.8. 9}. Three main approaches
`of years {References 1. 2. 3. 7. 8. 9}. Three main approaches
`have been proposed. The simplest approach is to use purely
`have been proposed. The simplest approach is to use purely
`analog delay and sum techniques {2}. Amore sophisticated
`analog delay and sum techniques {2}. A more sophisticated
`approach uses adaptive FIR ?lter techniques using
`approach uses adaptive FIR filter techniques using 45
`45
`algorithms. such as the Griffiths-Jim beamformer {I. 3}.
`algorithms. such as the Grifliths-Jim beamformer {1. 3}.
`These adaptive ?lter techniques require digital signal pro
`These adaptive filter techniques require digital signal pro(cid:173)
`cessing and were originally developed in the context of
`cessing and were originally developed in the context of
`antenna array bearnforming for radar applications {5}. Still
`antenna array beamforrning for radar applications {5}. Still
`another approach is motivated from a model of the human 50
`another approach is motivated from a model of the human
`50
`binaural hearing system {14. IS}. While the first two
`binaural hearing system {14. 15}. While the ?rst two
`approaches are time domain approaches. this last approach
`approaches are time domain approaches. this last approach
`is a frequency domain approach.
`is a frequency domain approach.
`There have been a number of problems associated with all
`There have been a number of problems associated with all
`of these approaches to beamforming. The delay-and-sum
`of these approaches to bearnforming. The delay-and-sum 55
`55
`and adaptive filter approaches have tended to break down in
`and adaptive ?lter approaches have tended to break down in
`non-anechoic. reverberant listening situations: any real room
`non-anechoic. reverberant listening situations: any real room
`will have so many acoustic reflections coming off walls and
`will have so many acoustic re?ections coming off walls and
`ceilings that the adaptive filters will be largely unable to
`ceilings that the adaptive ?lters will be largely unable to
`distinguish between desired sounds coming from the front
`distinguish between desired sounds coming from the front 60
`60
`and undesired sounds coming from other directions. The
`and undesired sounds coming from other directions. The
`delay-and-sum and adaptive ?lter techniques have also
`delay-and-sum and adaptive filter techniques have also
`required a large (>=8) number of microphone sensors to be
`required a large (>=8) number of microphone sensors to be
`effective. This has made it difficult to incorporate these
`effective. This has made it di?icult to incorporate these
`systems into practical hearing aid packages. One package
`systems into practical hearing aid packages. One package
`that has been proposed consists of a microphone array across
`that has been proposed consists of a microphone array across
`the top of eyeglasses {2}.
`the top of eyeglasses {2}.
`
`SUMMARY OF THE INVENTION
`SU'MNIARY OF THE INVENTION
`
`In accordance with this invention. the above problems are
`In accordance with this invention. the above problems are
`solved by analyzing the left and right digital audio signals to
`solved by analyzing the left and right digital audio signals to
`produce left and right signal frequency domain vectors and,
`produce left and right signal frequency domain vectors and,
`thereafter. using digital signal encoding techniques to pro
`thereafter. using digital signal encoding techniques to pro(cid:173)
`duce a noise reduction gain vector. The gain vector can then
`duce a noise reduction gain vector. The gain vector can then
`be multiplied against the left and right signal vectors to
`be multiplied against the left and right signal vectors to
`produce a noise reduced left and right signal vector. The cues
`produce a noise reduced left and right signal vector. The cues
`used in the digital encoding techniques include
`used in the digital encoding techniques include
`directionality. short-term amplitude deviation from long
`directionality. short-term amplitude deviation from long(cid:173)
`term average. and pitch. In addition. a multidimensional
`term average. and pitch. In addition. a multidimensional
`gain function. based on directionality estimate and ampli(cid:173)
`gain function. based on directionality estimate and ampli
`tude deviation estimate. is used that is more effective in
`tude deviation estimate. is used that is more effective in
`noise reduction than simply summing the noise reduction
`noise reduction than simply summing the noise reduction
`results of directionality alone and amplitude deviations
`results of directionality alone and amplitude deviations
`alone. As further features of the invention, the noise reduc(cid:173)
`alone. As further features of the invention, the noise reduc
`tion is scaled based on pitch-estimates and based on voice
`tion is scaled based on pitch-estimates and based on voice
`detection.
`detection.
`Other advantages and features of the invention will be
`Other advantages and features of the invention will be
`understood by those of ordinary skill in the art after referring
`understood by those of ordinary skill in the art after referring
`to the complete written description of the preferred embodi(cid:173)
`to the complete written description of the preferred embodi
`ments in conjunction with the following drawings.
`ments in conjunction with the following drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 illustrates the preferred embodiment of the noise
`FIG. 1 illustrates the preferred embodiment of the noise
`reduction system for a binaural hearing aid.
`reduction system for a binaural hearing aid.
`FIG. 2 shows the details of the inner product operation
`FIG. 2 shows the details of the inner product operation
`and the sum of magnitudes squared operation referred to in
`and the sum of magnitudes squared operation referred to in
`65 FIG. 1.
`FIG. 1.
`65
`FIGS. 3A and 3B show the band smoothing filters 157 of
`FIGS. 3A and 3B show the band smoothing ?lters 157 of
`band smoothing operation 156 in FIG. 1.
`band smoothing operation 156 in FIG. 1.
`
`RTL345-1_1027-0008
`
`

`
`5.651.071
`5,651,071
`
`3
`3
`FIG. 4 shows the details of the beam spectral subtract gain
`FIG. 4 shows the details of the beam spectral subtract gain
`operation 158 in FIG. 1.
`operation 158 in FIG. 1.
`FIG. SA is a graph of noise reduction gains as a serial
`FIG. 5A is a graph of noise reduction gains as a serial
`function of directionality and spectral subtraction.
`function of directionality and spectral subtraction.
`FIG. SB is a graph of the noise reduction gain as a
`FIG. 5B is a graph of the noise reduction gain as a
`function of directionality estimate and spectral subtraction
`function of directionality estimate and spectral subtraction
`excursion estimate in accordance with the process in FIG. 4.
`excursion estimate in accordance with the process in FIG. 4.
`FIG. 6 shows the details of the pitch-estimate gain opera—
`FIG. 6 shows the details of the pitch-estimate gain opera(cid:173)
`tion 180 in FIG. 1.
`tion 180 in FIG. 1.
`FIG. 7 shows the details of the voice detect gain scaling
`FIG. 7 shows the details of the voice detect gain scaling
`operation 208 in FIG. 1.
`operation 208 in FIG. 1.
`
`DESCRIPfION OF THE PREFERRED
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`EMBODIMENTS
`
`Theory of Operation:
`Theory of Operation:
`In the noise-reduction system described in this invention,
`In the noise-reduction system described in this invention.
`all three noise reduction techniques. beamforming. spectral
`all three noise reduction techniques. beamforming. spectral
`subtraction and pitch enhancement. are used. Innovations
`subtraction and pitch enhancement. are used. Innovations
`will be described relevant to the individual techniques.
`will be described relevant to the individual techniques.
`especially beamforming. In addition. it will be demonstrated
`especially bearnforming. In addition, it will be demonstrated
`that a synergy exists between these techniques such that the
`that a synergy exists between these techniques such that the
`whole is greater than the sum of the parts.
`whole is greater than the sum of the parts.
`
`Multidimensional Noise Reduction:
`Multidimensional Noise Reduction:
`
`5
`
`4
`4
`By using the combination of (D,STAD) we are able to make
`By using the combination of (D.STAD) we are able to make
`a better decision about a spectral component by insisting that
`a better decision about a spectral component by insisting that
`not only must it rise above the STAD threshold. but it must
`not only must it rise above the STAD threshold. but it must
`also be reasonably on-line. There is a continuous give and
`also be reasonably on-line. There is a continuous give and
`take between these two parameters.
`take between these two parameters.
`Including fO. pitch. as a third cue gives rise to a three
`Including f0. pitch. as a third cue gives rise to a three
`dimensional noise reduction system. We found it advanta(cid:173)
`dimensional noise reduction system. We found it advanta
`geous to estimate D and STAD in parallel and then use the
`geous to estimate D and STAD in parallel and then use the
`two parameters in a single two-dimensional function for
`two parameters in a single two-dimensional function for
`10 gain. We do not want to estimate fO in parallel with D and
`10
`gain. We do not want to estimate f0 in parallel with D and
`STAD. though. because we can do a better estimate of fO if
`STAD. though. because we can do a better estimate of f0 if
`we first noise reduce the signal somewhat using D and
`we ?rst noise reduce the signal somewhat using D and
`STAD. Therefore. based on the partially noise-reduced
`STAD. Therefore. based on the partially noise-reduced
`signal. we estimate f0 and then calculate the ?nal gain using
`signal, we estimate fO and then calculate the final gain using
`15
`15 D. STAD and fO in a general three-dimensional function. or
`D. STAD and f0 in a general three-dimensional function. or
`we can use fO to adjust the gain produced from D,sTAD
`we can use f0 to adjust the gain produced from D.STAD
`estimates. When fO is included. we see that not only is the
`estimates. When f0 is included. we see that not only is the
`system more efficient because we can use arbitrary gain
`system more e?icient because we can use arbitrary gain
`functions of three parameters. but also the presence of a first
`functions of three parameters. but also the presence of a ?rst
`20 stage of noise reduction makes the subsequent fO estimation
`stage of noise reduction makes the subsequent f0 estimation
`more robust than it would be in an fO only based system.
`more robust than it would be in an f0 only based system.
`The D estimate is based on values of phase angle and
`The D estimate is based on values of phase angle and
`magnitude for the current input segment. The STAD esti
`magnitude for the current input segment. The STAD esti(cid:173)
`mate is based on the sum of magnitudes over many past
`mate is based