( 12 ) United States Patent
`Burnett
`
`US 11,122,357 B2
`( 10 ) Patent No .:
`( 45 ) Date of Patent :
`* Sep . 14 , 2021
`
`US011122357B2
`
`( * ) Notice :
`
`( 54 ) FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`MICROPHONE ARRAY ( DOMA )
`( 71 ) Applicant : Jawbone Innovations , LLC , Marshall ,
`TX ( US )
`( 72 ) Inventor : Gregory C. Burnett , Dodge Center ,
`MN ( US )
`( 73 ) Assignee : Jawbone Innovations , LLC , Marshall ,
`TX ( US )
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U.S.C. 154 ( b ) by 0 days .
`This patent is subject to a terminal dis
`claimer .
`( 21 ) Appl . No .: 13 / 959,708
`Aug. 5 , 2013
`( 22 ) Filed :
`( 65 )
`Prior Publication Data
`Jul . 3 , 2014
`US 2014/0185825 A1
`Related U.S. Application Data
`( 63 ) Continuation of application No. 12 / 139,333 , filed on
`Jun . 13 , 2008 , now Pat . No. 8,503,691 .
`( Continued )
`
`( 51 ) Int . Cl .
`H04R 3/00
`H04R 1/10
`
`( 52 ) U.S. CI .
`CPC
`
`( 2006.01 )
`( 2006.01 )
`( Continued )
`
`H04R 1/1091 ( 2013.01 ) ; GIOL 21/0208
`( 2013.01 ) ; H04R 1/406 ( 2013.01 ) ;
`( Continued )
`
`( 58 ) Field of Classification Search
`CPC
`HO4R 3/005 ; H04R 2410/05 ; HO4R 3/002 ;
`HO4R 3/04 ; HO4R 1/1091 ; HO4R 1/406 ;
`( Continued )
`References Cited
`U.S. PATENT DOCUMENTS
`
`( 56 )
`
`4,653,102 A
`4,777,649 A
`
`3/1987 Hansen
`10/1988 Carlson et al .
`( Continued )
`FOREIGN PATENT DOCUMENTS
`
`WO
`
`2009003180
`
`12/2008
`
`OTHER PUBLICATIONS
`Weiss , Howard ; Office Action mailed by U.S. Patent and Trademark
`Office dated Apr. 10 , 2012 for U.S. Appl . No. 12 / 139,333 .
`( Continued )
`Primary Examiner Marcos D. Pizarro
`( 74 ) Attorney , Agent , or Firm - Nutter McClennen &
`Fish LLP
`
`( 57 )
`ABSTRACT
`A dual omnidirectional microphone array noise suppression
`is described . Compared to conventional arrays and algo
`rithms , which seek to reduce noise by nulling out noise
`sources , the array of an embodiment is used to form two
`distinct virtual directional microphones which are config
`ured to have very similar noise responses and very dissimilar
`speech responses . The only null formed is one used to
`remove the speech of the user from V2 . The two virtual
`microphones may be paired with an adaptive filter algorithm
`and VAD algorithm to significantly reduce the noise without
`distorting the speech , significantly improving the SNR of the
`desired speech over conventional noise suppression systems .
`20 Claims , 17 Drawing Sheets
`
`202
`
`09
`
`-500
`
`201
`
`01
`
`V1
`
`A21
`
`221
`
`22N
`
`VN
`
`211 hein
`
`X
`
`Aji
`AIN
`
`ZIN
`
`502
`
`Page 1 of 37
`
`GOOGLE EXHIBIT 1001
`
`

`

`US 11,122,357 B2
`
`Page 2
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/934,551, filed on Jun.
`13, 2007, provisional application No. 60/953,444,
`filed on Aug. 1, 2007, provisional application No.
`60/954,712,
`filed on Aug. 8, 2007, provisional
`application No. 61/045,377, filed on Apr. 16, 2008.
`
`2003/0044025 Al
`2003/0228023 Al
`2007/0003082 Al
`2007/0121974 Al
`2009/0010450 Al
`
`3/2003 Ouyang et al.
`12/2003 Burnett etal.
`1/2007 Pedersen
`5/2007 Nemirovski
`1/2009 Burnett
`
`OTHER PUBLICATIONS
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 14, 2011 for U.S. Appl. No. 12/139,333.
`Copenheaver, Blaine R; International Searching Authority; Notifi-
`cation of Transmittal of the International Search Report and Written
`Opinion of the International Searching Authority of the Declaration
`for International Patent Application No. PCT/US2008/068634, dated
`Sep. 2, 2008.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 10, 2011 for U.S. Appl. No. 12/163,592.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Nov. 16, 2011 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jan. 16, 2009 for U.S. Appl. No. 11/805,987.
`Azad, Abul K..; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 6, 2008 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Oct. 10, 2006 for U.S. Appl. No. 10/159,770.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 15, 2005 for U.S. Appl. No. 10/159,770.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 31, 2013 for U.S. Appl. No. 13/436,765.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 30, 2010 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 24, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 11, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 9, 2008 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 9, 2007 for U.S. Appl. No. 10/667,207.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 3, 2016 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 7, 2015 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 21, 2014 for U.S. Appl. No. 14/224,868.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 3, 2016 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Jun. 23, 2011 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 17, 2010 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Dec. 9, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 16, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 18, 2008 for U.S. Appl. No. 10/400,282.
`
`* cited by examiner
`
`(51)
`
`(2013.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`Int. Cl.
`GOL 21/0208
`HOAR 1/40
`HOAR 3/04
`GIOL 21/0216
`(52) U.S. Cl
`CPC veeceseseeee HOAR 3/002 (2013.01); HO4R 3/005
`(2013.01); HO4R 3/04 (2013.01); GIOL
`2021/02165 (2013.01); HO4R 2460/01
`(2013.01)
`
`(58) Field of Classification Search
`CPC ...... HO4R 2460/01; G1OL 2021/02165; G10L
`21/0208
`USPC wiceceeeeeeees 381/92, 94.7; 704/233, E21.004
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5/1993 Kaneda
`5,208,864 A
`1/1994 Freeman et al.
`5,276,765 A
`10/1994 Oh et al.
`5,353,376 A
`4/1995 Silverberg et al.
`5,406,622 A
`10/1995 Bradelyet al.
`5,463,694 A
`12/1995 Cezanneet al.
`5,473,701 A
`12/1995 Yoshidaet al.
`5,473,702 A
`5/1996 Sugiyama
`5,517,435 A
`12/1996 Park etal.
`5,590,241 A
`4/1997 Matouketal.
`5,625,684 A
`5/1997 Kanamori etal.
`5,633,935 A
`9/1997 Yamaguchiet al.
`5,664,014 A
`9/1997 Nishiguchiet al.
`5,664,052 A
`3/1998 Holzrichter et al.
`5,729,694 A
`5/1998 Hosoi
`5,754,665 A
`9/1998 Claybaughet al.
`5,815,582 A
`10/1998 Andreaet al.
`5,825,897 A
`5/1999 Takada
`5,907,624 A
`12/1999 Holzrichter
`6,006,175 A
`5/2001 Choet al.
`6,233,551 Bl
`3/2004 Valveetal.
`6,707,910 Bl
`9/2004 Housni
`6,795,713 B2
`11/2005 Vaudreyet al.
`6,963,649 B2
`12/2005 Turnbull et al.
`6,980,092 B2
`4/2007 Yang et al.
`7,206,418 B2
`6/2008 Fan
`7,386,135 B2
`8,068,619 B2* 11/2011 Zhang etal. wo. 381/92
`
`.. 381/92
`8,494,177 B2*
`7/2013 Burnett
`....
`8/2013 Burnett oe 381/92
`8,503,691 B2*
`8/2015 Burnett
`9,099,094 B2
`8/2002 Watsonet al.
`2002/0110256 Al
`8/2002 Erte
`2002/0116187 Al
`
`Page 2 of 37
`
`Page 2 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 1 of 17
`
`US 11,122,357 B2
`
`
`
`Cleaned Speech
`
`
`
`Noise Removal
`
`MICI
`
`
`
`Voicing Information
`
`
`
`104. VAD
`
`m ; ( n )
`
`my ( n )
`
`DOMA
`
`MIC 2
`
`( n )
`
`& ( n )
`
`FIG.1
`
`s ( n )
`
`n ( n )
`
`H ( 2 )
`
`( z ) ' H
`
`SIGNAL s ( n )
`
`101
`
`( ???
`
`NOISE
`
`Page 3 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 2 of 17
`
`US 11,122,357 B2
`
`S
`
`d
`
`dy
`
`dy
`
`202
`
`201
`
`0
`
`0? C
`
`do
`
`O
`
`do
`FIG.2
`
`Output
`V
`FIG.3
`
`0 Speed
`
`Page 4 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 3 of 17
`
`US 11,122,357 B2
`
`202
`
`0 ,
`
`221
`
`X
`
`Agi
`
`An2
`
`0
`
`Aii WY
`Aj
`
`Z 1
`
`?????
`
`?
`
`FIG.4
`
`Page 5 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 4 of 17
`
`US 11,122,357 B2
`
`0
`
`211
`
`AIN
`
`X
`
`11N
`
`{
`
`{
`}
`
`???
`}
`
`{
`{
`
`}
`{
`
`{
`
`{
`{
`}
`{
`
`{
`
`{
`
`221
`
`Z2N
`
`.... - •
`
`-.
`
`A21
`
`AN
`
`S
`
`??
`
`VN
`
`FIG.5
`
`Page 6 of 37
`
`

`

`US 11,122,357 B2
`
`oaa“—Nai-i.moao=/Mee
`ceeee=s||ftTi;:—éi“|af”3~|.
`am™wneeer)eeeee;'“ene
`
`1
`1
`
`?
`
`{
`
`2 .
`
`}
`
`3
`
`?
`
`3
`
`}
`
`1
`
`?
`
`?
`1
`
`1
`
`i
`
`}
`3
`
`U.S. Patent
`U.S. Patent
`
`Sep. 14 , 2021
`
`=NoS
`
`Sheet 5 of 17
`
`3
`
`3
`
`$
`
`2
`3
`
`1 .
`
`3
`
`3
`1
`2
`
`$
`
`WWW
`
`MI he
`
`ht
`
`A
`
`1 +++
`
`+
`
`3
`;
`?
`
`*
`
`Soo
`o%;:~‘A.rsrt
`rr;.4a‘=:‘°‘“4i)‘,oe‘
`ow
`f?~~'al;oS¢tnmi~fDfoOf=;a;,i7%27iLewee'’a|i,2téiae
`iae‘,;a~3:yy~fa:~PenseeoTteers
`eget-iSot
`
`
` Socoenee0Omea,~Se'é~i‘Mey;‘Afoeaa°wn£MeteH.wsawwfeee—oF
`
`3
`
`*
`3
`
`3
`
`3
`
`FIG.6
`FIG.6
`
`Page 7 of 37
`
`Page 7 of 37
`
`
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 6 of 17
`
`US 11,122,357 B2
`
`Receive acoustic signals at a first physical
`microphone and a second physical microphone .
`
`-702
`
`Output first microphone signal from first physical
`microphone and second microphone signal from
`second physical microphone .
`
`Form first virtual microphone using the first combination
`of first microphone signal and second microphone signal .
`
`-706
`
`Form second virtual microphone using second combination
`of first microphone signal and second microphone signal .
`
`Generate denoised output signals having less
`acoustic noise than received acoustic signals .
`700
`FIG.7
`
`Form physical microphone array including first
`physical microphone and second physical microphone .
`
`Www
`
`wwwwwwwwwwww
`
`Form virtual microphone array including first virtual
`microphone and second virtual microphone using
`signals from physical microphone array .
`FIG.8
`
`800
`
`-802
`
`-804
`
`Page 8 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 7 of 17
`
`US 11,122,357 B2
`
`Linear response of V2 to a speech source at 0.10 meters
`
`.
`
`2
`
`3
`
`0
`
`FIG.9
`
`Lincar response of V2 to a noise source at 1 meters
`120
`
`1
`
`3
`
`210
`
`FIG.10
`
`0
`
`330
`
`300
`
`Page 9 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 8 of 17
`
`US 11,122,357 B2
`
`Linear response of VI to a speech source at 0.10 meters
`
`80
`
`FIG.11
`
`1
`
`0
`
`330
`
`Linear response of VI to a noise source at 1 meters
`
`120
`
`-
`
`150
`
`210
`
`FIG.12
`
`0
`
`Page 10 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 9 of 17
`
`US 11,122,357 B2
`
`Linear response of VI to a speech source at 0.1 meters
`2
`
`1.5
`
`MAIS
`
`30
`
`180
`
`SPEECH
`
`0
`
`25.4
`
`300
`
`Hz
`500 Hz
`1000 Hz
`
`2000 Hz
`
`3000 Hz
`4000 Hz
`
`270
`FIG.13
`
`Page 11 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 10 of 17
`
`US 11,122,357 B2
`
`5
`
`Response ( dB )
`
`00
`
`Frequency response at 0 degrees
`
`Cardioid speech
`response
`
`V1 speech
`response
`
`0
`
`2000
`
`6000
`
`Frequency ( Hz )
`
`FIG.14
`
`Page 12 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 11 of 17
`
`US 11,122,357 B2
`
`5
`
`Response ( dB )
`
`-35
`
`35
`
`30
`
`25
`
`V1 / V2 for speech ( dB )
`15
`
`Aurii
`
`5
`
`V1 ( top , dashed ) and V2 speech response vs. B assuming d = 0.1m
`
`*** ?
`
`? ?
`
`?
`
`? ? ?? -
`
`0.5
`
`1
`
`FIG.15
`V1 / V2 for speech versus B assuming ds = 0.1m
`
`0.9
`
`I
`
`B
`FIG.16
`
`Page 13 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 12 of 17
`
`US 11,122,357 B2
`
`B factor vs. actual d , assuming do = 0.1m and theta = 0 )
`
`1.2
`
`1 1
`
`1.25
`
`1.2
`
`B
`
`1.1
`
`0.05
`
`0.3
`0.25
`Actual ds ( meters )
`FIG.17
`B versus theta assuming ds = 0.1m
`
`20
`0
`theta ( degrees )
`FIG.18
`
`Page 14 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 13 of 17
`
`US 11,122,357 B2
`
`N ( s ) for B = 1 and D = -72c - 006 seconds
`
`Amplitude ( dB )
`
`0
`
`0
`
`Phase ( degrees )
`
`M
`
`0
`
`000
`
`3000
`
`Frequency ( Hz )
`
`FIG.19
`
`Page 15 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 14 of 17
`
`US 11,122,357 B2
`
`N ( s ) for B = 1.2 and D = -7.2e - 006 seconds
`
`0
`
`Amplitude ( dB )
`
`? ? ? ?
`
`?
`
`? ? ? ? ? ? ??
`
`und
`
`2000
`
`4000
`
`7000
`
`0
`
`260
`
`240
`Phase ( degrees )
`
`180
`
`Frequency ( Hz )
`
`FIG.20
`
`Page 16 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 15 of 17
`
`US 11,122,357 B2
`
`Cancellation with di = 1 , thetal = 0 , d2 = 1 , and theta2 = 30
`
`0
`
`Amplitude ( dB )
`
`wa
`
`X
`
`-30
`
`AUD
`
`Phase ( degrees )
`
`75
`
`0
`
`2000
`
`5000
`
`3000
`Frequency ( Hz )
`
`6000
`
`FIG.21
`
`Page 17 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 16 of 17
`
`US 11,122,357 B2
`
`Cancellation with dl = 1 , thotal = 0 , d2 = 1 , and theta2 = 45 HT
`
`7000
`
`*
`
`30
`
`0
`
`Phase ( degrees )
`
`60
`0
`
`2000
`
`5000
`Frequency ( Hz )
`
`6000
`
`FIG.22
`
`Page 18 of 37
`
`

`

`U.S. Patent
`
`Sep. 14 , 2021
`
`Sheet 17 of 17
`
`US 11,122,357 B2
`
`Original VI ( top ) and cleaned V1 ( bottom ) with simplified VAD ( dashed ) in noise
`0.4
`
`Win
`
`wy
`8
`
`w
`
`*** * *
`
`8 MN
`
`2
`
`2.5
`x 10 $
`
`8
`
`PHILUX
`
`1
`
`???
`
`Noisy
`
`0.2
`
`-0.2
`
`0.3
`0.2
`
`Cleaned
`
`0
`
`wwwwwwww
`
`wwwwww
`
`3
`
`- ? ? ? ? ? ? ?
`
`3
`
`8
`
`3 UMNM
`
`
`
`WWWW du *
`
`
`
`1.5
`1
`Time ( samples at 8 kHz / sec )
`
`2
`
`2.5
`x 10 $
`
`FIG.23
`
`Page 19 of 37
`
`

`

`US 11,122,357 B2
`
`20
`
`1
`2
`acoustic noise . The technology supporting the Jawbone is
`FORMING VIRTUAL MICROPHONE
`currently described under one or more of U.S. Pat . No.
`ARRAYS USING DUAL OMNIDIRECTIONAL
`7,246,058 by Burnett and / or U.S. patent application Ser .
`MICROPHONE ARRAY ( DOMA )
`Nos . 10 / 400,282 , 10 / 667,207 , and / or 10 / 769,302 . Generally ,
`5 multi - microphone techniques make use of an acoustic - based
`CROSS - REFERENCE TO RELATED
`Voice Activity Detector ( VAD ) to determine the background
`APPLICATIONS
`noise characteristics , where “ voice ” is generally understood
`to include human voiced speech , unvoiced speech , or a
`This application is a continuation of U.S. Nonprovisional
`patent application Ser . No. 12 / 139,333 , filed Jun . 13 , 2008 ,
`combination of voiced and unvoiced speech . The Jawbone
`now U.S. Pat . No. 8,503,691 , entitled “ Forming Virtual 10 improved on this by using a microphone - based sensor to
`Microphone Arrays Using Dual Omnidirectional Micro
`construct a VAD signal using directly detected speech vibra
`phone Array ( DOMA ) , ” which claims the benefit of U.S.
`tions in the user's cheek . This allowed the Jawbone to
`Provisional Patent Application No. 60 / 934,551 , filed Jun .
`aggressively remove noise when the user was not producing
`13 , 2007 , U.S. Provisional Patent Application No. 60/953 ,
`444 , filed Aug. 1 , 2007 , U.S. Provisional Patent Application 15 speech . However , the Jawbone uses a directional micro
`phone array .
`No. 60 / 954,712 , filed Aug. 8 , 2007 , and U.S. Provisional
`Patent Application No. 61 / 045,377 , filed Apr. 16 , 2008 , all
`INCORPORATION BY REFERENCE
`of which are incorporated by reference herein in their
`entirety for all purposes .
`Each patent , patent application , and / or publication men
`tioned in this specification is herein incorporated by refer
`TECHNICAL FIELD
`ence in its entirety to the same extent as if each individual
`patent , patent application , and / or publication was specifi
`The disclosure herein relates generally to noise suppres
`cally and individually indicated to be incorporated by ref
`sion . In particular , this disclosure relates to noise suppres-
`sion systems , devices , and methods for use in acoustic 25 erence .
`applications .
`BRIEF DESCRIPTION OF THE DRAWINGS
`BACKGROUND
`FIG . 1 is a two - microphone adaptive noise suppression
`Conventional adaptive noise suppression algorithms have 30 system , under an embodiment .
`FIG . 2 is an array and speech source ( S ) configuration ,
`been around for some time . These conventional algorithms
`have used two or more microphones to sample both an
`under an embodiment . The microphones are separated by a
`( unwanted ) acoustic noise field and the ( desired ) speech of
`distance approximately equal to 2d , and the speech source
`a user . 20 The noise relationship between the microphones
`is located a distance d , away from the midpoint of the array
`is then determined using an adaptive filter ( such as Least- 35 at an angle 0. The system is axially symmetric so only d , and
`Mean - Squares as described in Haykin & Widrow ,
`e need be specified .
`ISBN # 0471215708 , Wiley , 2002 , but any adaptive or sta-
`FIG . 3 is a block diagram for a first order gradient
`tionary system identification algorithm may be used ) and
`microphone using two omnidirectional elements O , and O2 ,
`that relationship used to filter the noise from the desired
`under an embodiment .
`signal .
`FIG . 4 is a block diagram for a DOMA including two
`Most conventional noise suppression systems currently in
`physical microphones configured to form two virtual micro
`phones V , and V2 , under an embodiment .
`use for speech communication systems are based on a
`single - microphone spectral subtraction technique first
`FIG . 5 is a block diagram for a DOMA including two
`develop in the 1970's and described , for example , by S. F.
`physical microphones configured to form N virtual micro
`Boll in “ Suppression of Acoustic Noise in Speech using 45 phones V1 through Vn , where N is any number greater than
`Spectral Subtraction , ” IEEE Trans . on ASSP , pp . 113-120 ,
`one , under an embodiment .
`1979. These techniques have been refined over the years , 30
`FIG . 6 is an example of a headset or head - worn device
`but the basic principles of operation have remained the
`that includes the DOMA , as described herein , under an
`same . See , for example , U.S. Pat . No. 5,687,243 of
`embodiment .
`McLaughlin , et al . , and U.S. Pat . No. 4,811,404 of Vilmur , 50
`FIG . 7 is a flow diagram for denoising acoustic signals
`et al . There have also been several attempts at multi-
`using the DOMA , under an embodiment .
`microphone noise suppression systems , such as those out-
`FIG . 8 is a flow diagram for forming the DOMA , under
`lined in U.S. Pat . No. 5,406,622 of Silverberg et al . and U.S.
`an embodiment .
`Pat . No. 5,463,694 of Bradley et al . Multi - microphone
`FIG . 9 is a plot of linear response of virtual microphone
`systems have not been very successful for a variety of 55 V2 to a 1 kHz speech source at a distance of 0.1 m , under an
`reasons , the most compelling being poor noise cancellation
`embodiment . The null is at 0 degrees , where the speech is
`performance and / or significant speech distortion . Primarily ,
`normally located .
`conventional multi - microphone systems attempt to increase
`FIG . 10 is a plot of linear response of virtual microphone
`the SNR of the user's speech by “ steering " the nulls of the
`V2 to a 1 kHz noise source at a distance of 1.0 m , under an
`system to the strongest noise sources . This approach is 60 embodiment . There is no null and all noise sources are
`limited in the number of noise sources removed by the
`detected .
`FIG . 11 is a plot of linear response of virtual microphone
`number of available nulls .
`The Jawbone earpiece ( referred to as the “ Jawbone ) ,
`V , to a 1 kHz speech source at a distance of 0.1 m , under an
`embodiment . There is no null and the response for speech is
`introduced in December 2006 by AliphCom of San Fran-
`cisco , Calif . , was the first known commercial 10 product to 65 greater than that shown in FIG . 9 .
`use a pair of physical directional microphones ( instead of
`FIG . 12 is a plot of linear response of virtual microphone
`omnidirectional microphones ) to reduce environmental
`V to a 1 kHz noise source at a distance of 1.0 m , under an
`
`40
`
`Page 20 of 37
`
`

`

`US 11,122,357 B2
`
`9
`
`3
`4
`In accordance with another embodiment , a microphone
`embodiment . There is no null and the response is very
`array is formed with a first virtual microphone formed from
`similar to V2 shown in FIG . 10 .
`a first combination of a first microphone signal and a second
`FIG . 13 is a plot of linear response of virtual microphone
`microphone signal , wherein the first microphone signal is
`Vi to a speech source at a distance of 0.1 m for frequencies
`of 100 , 500 , 1000 , 2000 , 3000 , and 4000 Hz , under an 5 generated by a first omnidirectional microphone and the
`second microphone signal is generated by a second omni
`embodiment .
`directional microphone ; and a second virtual microphone
`FIG . 14 is a plot showing comparison of frequency
`formed from a second combination of the first microphone
`responses for speech for the array of an embodiment and for
`signal and the second microphone signal , wherein the sec
`a conventional cardioid microphone .
`FIG . 15 is a plot showing speech response for V1 ( top , 10 ond combination is different from the first combination . The
`first virtual microphone has a first linear response to speech
`dashed ) and V2 ( bottom , solid ) versus B with d , assumed to
`that has a single null oriented in a direction toward a source
`be 0.1 m , under an embodiment . The spatial null in V2 is
`of the speech , wherein the speech is human speech .
`relatively broad .
`In accordance with another embodiment , a device
`FIG . 16 is a plot showing a ratio of V. / , speech 15 includes a first microphone outputting a first microphone
`responses shown in FIG . 10 versus B , under an embodiment .
`signal and a second microphone outputting a second micro
`The ratio is above 10 dB for all 0.8 < B < 1.1 . This means that
`phone signal ; and a processing component coupled to the
`the physical ß of the system need not be exactly modeled for
`first microphone signal and the second microphone signal ,
`good performance .
`the processing component generating a virtual microphone
`FIG . 17 is a plot of B versus actual d , assuming that d = 10 20 array comprising a first virtual microphone and a second
`virtual microphone , wherein the first virtual microphone
`cm and theta = 0 , under an embodiment .
`FIG . 18 is a plot of B versus theta with d = 10 cm and
`comprises a first combination of the first microphone signal
`and the second microphone signal , and wherein the second
`assu ning do = 10 cm , under an embodiment .
`FIG . 19 is a plot of amplitude ( top ) and phase ( bottom )
`virtual microphone comprises a second combination of the
`response of N ( s ) with B = 1 and D = -7.2 usec , under an 25 first microphone signal and the second microphone signal .
`embodiment . The resulting phase difference clearly affects
`The second virtual microphone have substantially similar
`responses to noise and substantially dissimilar responses to
`high frequencies more than low .
`FIG . 20 is a plot of amplitude ( top ) and phase ( bottom )
`speech .
`response of N ( s ) with B = 1.2 and D = -7.2 usec , under an
`In accordance with another embodiment , a devise
`embodiment . Non - unity B affects the entire frequency range . 30 includes a first microphone outputting a first microphone
`FIG . 21 is a plot of amplitude ( top ) and phase ( bottom )
`signal and a second microphone outputting a second micro
`response of the effect on the speech cancellation in V2 due
`phone signal , wherein the first microphone and the second
`to a mistake in the location of the speech source with q1 = 0
`microphone are omnidirectional microphones ; and a virtual
`degrees and q2 = 30 degrees , under an embodiment . The
`microphone array comprising a first virtual microphone and
`cancellation remains below -10 dB for frequencies below 6 35 a second virtual microphone , wherein the first virtual micro
`phone comprises a first combination of the first microphone
`kHz .
`FIG . 22 is a plot of amplitude ( top ) and phase ( bottom )
`signal and the second microphone signal , and the second
`virtual microphone comprises a second combination of the
`response of the effect on the speech cancellation in V2 due
`to a mistake in the location of the speech source with q1 = 0
`first microphone signal and the second microphone signal .
`degrees and q2 = 45 degrees , under an embodiment . The 40 The second combination is different from the first combi
`cancellation is below -10 dB only for frequencies below
`nation , and the first virtual microphone and the second
`virtual microphone are distinct virtual directional micro
`about 2.8 kHz and a reduction in performance is expected .
`FIG . 23 shows experimental results for a 2do = 19 mm phones .
`array using a linear B of 0.83 on a Bruel and Kjaer Head and
`In accordance with another embodiment , a device
`Torso Simulator ( HATS ) in very loud ( ~ 85 dBA ) music / 45 includes a first physical microphone generating a first micro
`speech noise environment , under an embodiment . The noise
`phone signal ; a second physical microphone generating a
`has been reduced by about 25 dB and the speech hardly
`second microphone signal ; and a processing component
`coupled to the first microphone signal and the second
`affected , with no noticeable distortion .
`microphone signal , the processing component generating a
`50 virtual microphone array comprising a first virtual micro
`SUMMARY OF THE INVENTION
`phone and a second virtual microphone . The first virtual
`microphone comprises the second microphone signal sub
`The present invention provides for dual omnidirectional
`microphone array devices systems and methods .
`tracted from a delayed version of the first microphone signal ,
`In accordance with on embodiment , a microphone array is
`and the second virtual microphone comprises a delayed
`formed with a first virtual microphone that includes a first 55 version of the first microphone signal subtracted from the
`combination of a first microphone signal and a second
`second microphone signal .
`microphone signal , wherein the first microphone signal is
`In accordance with another embodiment , a
`sensor
`generated by a first physical microphone and the second
`includes a physical microphone array including a first physi
`microphone signal is generated by a second physical micro-
`cal microphone and a second physical microphone , the first
`phone ; and a second virtual microphone that includes a 60 physical microphone outputting a first microphone signal
`second combination of the first microphone signal and the
`and the second physical microphone outputting a second
`second microphone signal , wherein the second combination
`microphone signal ; and a virtual microphone array compris
`is different from the first combination . The first virtual
`ing a first virtual microphone and a second virtual micro
`microphone and the second virtual microphone are distinct
`phone , the first curtail microphone comprising a first com
`virtual directional microphones with substantially similar 65 bination of the first microphone signal and the second
`responses to noise and substantially dissimilar responses to
`microphone signal , the second virtual microphone compris
`ing a second combination of the first microphone signal and
`speech .
`
`Page 21 of 37
`
`

`

`US 11,122,357 B2
`
`5
`
`M2 ( z ) = N ( z ) + S2 ( 2 ) ,
`
`with
`
`N2 ( z ) = N ( z ) H ( 2 )
`
`35
`
`S2 ( z ) = S ( z ) H ( 2 ) ,
`
`so that
`
`M ( z ) = S ( z ) + N ( z ) H ( 2 )
`
`5
`6
`The term “ V ” means the virtual directional “ speech ”
`the second microphone signal . The second combination is
`microphone , which has no nulls .
`different from the first combination , and the virtual micro-
`The term “ V ” means the virtual directional ” noise "
`phone array includes a single null oriented in a direction
`microphone , which has a null for the user's speech .
`toward a source of speech of a human speaker .
`The term “ Voice Activity Detection ( VAD ) signal ” means
`a signal indicating when user speech is detected .
`DETAILED DESCRIPTION
`The term “ virtual microphones ( VM ) ” or “ virtual direc
`tional microphones ” means a microphone constructed using
`A dual omnidirectional microphone array ( DOMA ) that
`two or more omnidirectional microphones and associated
`provides improved noise suppression is described herein .
`Compared to conventional arrays and algorithms , which 10 signal processing .
`seek to reduce noise by nulling out noise sources , the array
`FIG . 1 is a two - microphone adaptive noise suppression
`of an embodiment is used to form two distinct virtual
`system 100 , under an embodiment . The two - microphone
`directional microphones which are configured to have very
`system 100 including the combination of physical micro
`similar noise responses and very dissimilar speech
`phones MIC 1 and MIC 2 along with the processing or
`responses . The only null formed by the DOMA is one used 15 circuitry components to which the microphones couple
`to remove the speech of the user from V2 . The two virtual
`( described in detail below , but not shown in this figure ) is
`microphones of an embodiment can be paired with an
`referred to herein as the dual omnidirectional microphone
`adaptive filter algorithm and / or VAD algorithm to signifi-
`array ( DOMA ) 110 , but the embodiment is not so limited .
`cantly reduce the noise without distorting the speech , sig-
`Referring to FIG . 1 , in analyzing the single noise source 101
`nificantly improving the SNR of the desired speech over 20 and the direct path to the microphones , the total acoustic
`conventional noise suppression systems . The embodiments
`information coming into MIC 1 ( 102 , which can be an
`described herein are stable in operation , flexible with respect
`physical or 5 virtual microphone ) is denoted by m , ( n ) . The
`to virtual microphone pattern choice , and have proven to be
`total acoustic information coming into MIC 2 ( 103 , which
`robust with respect to speech source - to - array distance and
`can also be an physical or virtual microphone ) is similarly
`orientation as well as temperature and calibration tech- 25 labeled m2 ( n ) . In the z ( digital frequency ) domain , these are
`niques . In the following description , numerous specific
`represented as M , ( z ) and M2 ( Z ) . Then ,
`details are introduced to provide a thorough understanding
`M1 ( z ) = S ( z ) + N2 ( z )
`of , and enabling description for , embodiments of the
`DOMA . One skilled in the relevant art , however , will
`recognize that these embodiments can be practiced without 30
`one or more of the specific details , or with other compo
`nents , systems , etc. In other instances , well - known struc
`tures or operations are not shown , or are ot described in
`detail , to avoid obscuring aspects of the disclosed embodi
`ments .
`Unless otherwise specified , the following terms have the
`corresponding meanings in addition to any meaning or
`understanding they may convey to one skilled in the art .
`The term “ bleedthrough ” means the undesired presence of
`noise during speech .
`The term “ denoising ” means removing unwanted noise
`Eq . 1
`M ( z ) = N ( z ) + S ( z ) H_ ( 2 ) .
`This is the general case for all two microphone systems .
`from Micl , and also refers to the amount of reduction of
`noise energy in a signal in decibels ( dB ) .
`Equation 1 has four unknowns and only two known rela
`tionships and therefore cannot be solved explicitly .
`The term " devoicing ” means removing / distorting the
`desired speech from Mic1 .
`However , there is another way to solve for some of the
`unknowns in Equation 1. The analysis starts with an exami
`The term " directional microphone ( DM ) ” means a physi-
`nation of the case where the speech is not being generated ,
`cal directional microphone that is vented on both sides of the
`sensing diaphragm .
`that is , where a signal from the VAD subsystem 104 ( op
`The term “ Mic1 ( M1 ) ” means a general designation for an
`tional ) equals zero . In this case , s ( n ) = S ( z ) = 0 , and Equation
`adaptive noise suppression system microphone that usually 50 1 reduces to
`contains more speech than noise .
`The term “ Mic2 ( M2 ) ” means a general designation for an
`Min ( z ) = N ( z ) H ( 2 )
`adaptive noise suppression system microphone that usually
`M2n ( z ) = N ( z ) ,
`contains more noise than speech .
`The term " noise " means unwanted environmental acous- 55 where the N subscript on the M variables indicate that only
`noise is being received .
`tic noise .
`The term “ null ” means a zero or minima in the spatial
`This leads to
`response of a physical or virtual directional microphone .
`The term “ O , ” means a first physical omnidirectional
`microphone used to form a microphone array .
`The term “ 0
`” means a second physical omnidirectional
`microphone used to form a microphone array .
`The term “ speech ” means desired