I 1111111111111111 1111111111 11111 111111111111111 1111111111111111 IIII IIII IIII
`USO 111223 57B2
`
`c12) United States Patent
`Burnett
`
`(IO) Patent No.: US 11,122,357 B2
`*Sep.14,2021
`(45) Date of Patent:
`
`(54) FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`MICROPHONE ARRAY (DOMA)
`
`(71) Applicant: Jawbone Innovations, LLC, Marshall,
`TX (US)
`
`(72)
`
`Inventor: Gregory C. Burnett, Dodge Center,
`MN (US)
`
`(73) Assignee: Jawbone Innovations, LLC, Marshall,
`TX (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`This patent is subject to a terminal dis-
`claimer.
`
`(21) Appl. No.: 13/959,708
`
`(22) Filed:
`
`Aug. 5, 2013
`
`(65)
`
`Prior Publication Data
`US 2014/0185825 Al
`Jul. 3, 2014
`
`Related U.S. Application Data
`(63) Continuation of application No. 12/139,333, filed on
`Jun. 13, 2008, now Pat. No. 8,503,691.
`(Continued)
`
`(51)
`
`Int. Cl.
`H04R 3/00
`H04R 1110
`
`(2006.01)
`(2006.01)
`(Continued)
`
`(52) U.S. Cl.
`CPC ........ H04R 111091 (2013.01); Gl0L 2110208
`(2013.01); H04R 11406 (2013.01);
`(Continued)
`
`(58) Field of Classification Search
`CPC .... H04R 3/005; H04R 2410/05; H04R 3/002;
`H04R 3/04; H04R 1/1091; H04R 1/406;
`(Continued)
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4,653,102 A
`4,777,649 A
`
`3/1987 Hansen
`10/1988 Carlson et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`
`2009003180
`
`12/2008
`
`OTHER PUBLICATIONS
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Apr. 10, 2012 for U.S. Appl. No. 12/139,333.
`(Continued)
`
`Primary Examiner - Marcos D. Pizarro
`(74) Attorney, Agent, or Firm - Nutter McClennen &
`Fish LLP
`
`ABSTRACT
`(57)
`A dual onmidirectional microphone array noise suppression
`is described. Compared to conventional arrays and algo-
`rithms, which seek to reduce noise by nulling out noise
`sources, the array of an embodiment is used to form two
`distinct virtual directional microphones which are config-
`ured to have very similar noise responses and very dissimilar
`speech responses. The only null formed is one used to
`remove the speech of the user from V2 . The two virtual
`microphones may be paired with an adaptive filter algorithm
`and VAD algorithm to significantly reduce the noise without
`distorting the speech, significantly improving the SNR of the
`desired speech over conventional noise suppression systems.
`
`20 Claims, 17 Drawing Sheets
`
`202
`
`-500
`
`(
`
`201-
`
`Vl
`
`VN
`
`- 1 -
`
`Sony v. Jawbone
`
`U.S. Patent No. 11,122,357
`
`Sony Ex. 1001
`
`

`

`US 11,122,357 B2
`Page 2
`
`Related U.S. Application Data
`(60) Provisional application No. 60/934,551, filed on Jun.
`13, 2007, provisional application No. 60/953,444,
`filed on Aug. 1, 2007, provisional application No.
`60/954,712, filed on Aug. 8, 2007, provisional
`application No. 61/045,377, filed on Apr. 16, 2008.
`
`2003/0044025 Al
`2003/0228023 Al
`2007 /0003082 Al
`2007/0121974 Al
`2009/0010450 Al
`
`3/2003 Ouyang et al.
`12/2003 Burnett et al.
`1/2007 Pedersen
`5/2007 Nemirovski
`1/2009 Burnett
`
`OTHER PUBLICATIONS
`
`(51)
`
`(2013.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`Int. Cl.
`Gl0L 2110208
`H04R 1140
`H04R 3/04
`Gl0L 21/0216
`(52) U.S. Cl.
`CPC ............. H04R 3/002 (2013.01); H04R 3/005
`(2013.01); H04R 3/04 (2013.01); Gl0L
`2021/02165 (2013.01); H04R 2460/01
`(2013.01)
`
`( 58) Field of Classification Search
`CPC ...... H04R 2460/01; Gl0L 2021/02165; Gl0L
`21/0208
`USPC ................... 381/92, 94.7; 704/233, E21.004
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,208,864 A
`5,276,765 A
`5,353,376 A
`5,406,622 A
`5,463,694 A
`5,473,701 A
`5,473,702 A
`5,517,435 A
`5,590,241 A
`5,625,684 A
`5,633,935 A
`5,664,014 A
`5,664,052 A
`5,729,694 A
`5,754,665 A
`5,815,582 A
`5,825,897 A
`5,907,624 A
`6,006,175 A
`6,233,551 Bl
`6,707,910 Bl
`6,795,713 B2
`6,963,649 B2
`6,980,092 B2
`7,206,418 B2
`7,386,135 B2
`8,068,619 B2 *
`8,494,177 B2 *
`8,503,691 B2 *
`9,099,094 B2
`2002/0110256 Al
`2002/0116187 Al
`
`5/ 1993 Kaneda
`1/1994 Freeman et al.
`10/ 1994 Oh et al.
`4/1995 Silverberg et al.
`10/1995 Bradely et al.
`12/1995 Cezanne et al.
`12/1995 Yoshida et al.
`5/1996 Sugiyama
`12/1996 Park et al.
`4/1997 Matouk et al.
`5/1997 Kanamori et al.
`9/1997 Yamaguchi et al.
`9/ 1997 Nishiguchi et al.
`3/1998 Holzrichter et al.
`5/ 1998 Hosoi
`9/ 1998 Claybaugh et al.
`10/1998 Andrea et al.
`5/ 1999 Takada
`12/1999 Holzrichter
`5/2001 Cho et al.
`3/2004 Valve et al.
`9/2004 Housni
`11/2005 Vaudrey et al.
`12/2005 Turnbull et al.
`4/2007 Yang et al.
`6/2008 Fan
`................... 381/92
`11/2011 Zhang et al.
`7/2013 Burnett ........................... 381/92
`8/2013 Burnett ........................... 381/92
`8/2015 Burnett
`8/2002 Watson et al.
`8/2002 Erte
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 14, 2011 for U.S. Appl. No. 12/139,333.
`Copenheaver, Blaine R; International Searching Authority; Notifi-
`cation of Transmittal of the International Search Report and Written
`Opinion of the International Searching Authority of the Declaration
`for International Patent Application No. PCT/US2008/068634, dated
`Sep. 2, 2008.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 10, 2011 for U.S. Appl. No. 12/163,592.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Nov. 16, 2011 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jan. 16, 2009 for U.S. Appl. No. 11/805,987.
`Azad, Abu! K .. ; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 6, 2008 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Oct. 10, 2006 for U.S. Appl. No. 10/159,770.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 15, 2005 for U.S. Appl. No. 10/159,770.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 31, 2013 for U.S. Appl. No. 13/436,765.
`Lao, Lun S .. ; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 30, 2010 for U.S. Appl. No. 10/667,207.
`Lao, Lun S .. ; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 24, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S .. ; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 11, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S .. ; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 9, 2008 for U.S. Appl. No. 10/667,207.
`Lao, Lun S .. ; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 9, 2007 for U.S. Appl. No. 10/667,207.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 3, 2016 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 7, 2015 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 21, 2014 for U.S. Appl. No. 14/224,868.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 3, 2016 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Jun. 23, 2011 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 17, 2010 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Dec. 9, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 16, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 18, 2008 for U.S. Appl. No. 10/400,282.
`
`* cited by examiner
`
`- 2 -
`
`

`

`Cleaned Speech
`
`e •
`
`•
`00
`
`/
`
`{ ) '
`
`103
`MIC2
`................... -. ..,.,,
`f /tl½(n)
`
`'·
`
`,:,
`
`SI} n)
`'-·
`...,
`
`(
`
`'
`..,,
`
`!½(n)
`/
`
`r--........::::E.~~~-
`
`.1,02
`:rv n cl
`Voicing Information
`
`104 ---\____ V AD
`
`n(n)
`NOISE
`t.0:J'.9 ---~~;------~--=-..,._..,~..,"~
`f
`\
`,
`i
`101
`
`n(n)
`
`,
`
`'t
`
`z)
`H ( ..
`
`• ·1. '·
`
`I
`
`- 3 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 2 of 17
`
`US 11,122,357 B2
`
`s
`
`di
`
`•
`
`•~-
`'
`
`do
`
`•
`
`do
`FIG.2
`
`/····202
`,·
`
`/--300
`{
`
`201 -----
`
`A, ...
`
`A1
`
`Output
`V
`FIG-*3
`
`- 4 -
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sep.14,2021
`Sep. 14, 2021
`
`Sheet 3 of 17
`Sheet 3 of 17
`
`US 11,122,357 B2
`US 11,122,357 B2
`
`
`
`
`
`--✓----400
`/'
`a
`"'
`
`201---"
`tokie fanny }
`
`A 11
`
`zJl
`
`z 12
`
`Vl
`
`V2
`
`- 5 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 4 of 17
`
`US 11,122,357 B2
`
`/,(
`
`/--500
`t
`
`.
`
`201 ···",
`
`A21
`
`A2N
`
`Vl
`
`i!&
`
`<SI
`
`<SI
`
`VN
`
`A 11
`
`A1N
`
`- 6 -
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sotf‘~NAi5aneae
`wna.me~~/,ue=‘,~zeae
`+t‘i:wen
`teeereanayoneweaf2:aé=/ca2é;i?;¢fae;ae
`=‘4te=
`Yo‘‘+7‘5N~ge;,gt:
`éSm3oOétmMi—éY:e5=iN:*i4;xy:deeeceeoet:2on,=;4‘a4vo;
`«::swe"
`fi‘:i:a‘.xt
`WwimeeeeceeteenaonanearaeaaANatFEONeneewee
`
`:-}?7
`neneefo“ésyy‘
`
`ofNmtora
`ayAat:ciaaeCoed
`wen;Sot
`Menene4Me'£
`NY4ZSe2ftehteaeZN,~AL“eSfteeOe
`‘s,fwwfMe
`eeene
`
`,I
`.ll
`
`i
`
`i
`
`{
`
`-.-.. ""r • .., ,... -~
`.t
`! ,
`' ;,
`
`\
`\
`
`I
`
`,,
`
`I
`
`i
`I
`
`/l
`
`,
`
`/
`
`Sep.14,2021
`
`Sheet 5 of 17
`S.
`
`US 11,122,357 B2
`pO
`
`/ ,'
`
`I
`\
`
`i
`
`l
`
`,,
`... _.,,.-!'J.'
`
`,-,·
`
`, ........ ,.,.,. ..... ~_ ........ -
`
`' ' \
`
`\
`
`I
`I
`
`\
`
`\
`
`' ' '
`
`I
`\
`\
`
`' ' ...
`
`,,,...
`
`600
`
`' \
`\
`
`FIG.6
`
`,,,
`
`'
`
`·,
`
`\
`
`\
`
`\
`!
`[
`
`i
`!
`;
`i
`I
`
`\ '
`
`I
`1
`f
`I
`!
`!
`
`!
`I
`/
`
`....,. ___ t' ..
`!
`
`;
`l
`\
`\ ',
`
`- 7 -
`
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 6 of 17
`
`US 11,122,357 B2
`
`Receive acoustic signals at a first physical
`microphone and a second physical microphone.
`
`--'''\.----704
`
`-··,
`_-7nt:.
`t ..... ~-- uu
`
`>--'
`
`4·
`Output first microphone signal from first physical
`microphone and second microphone signal from
`second physical microphone.
`!
`Form first virtual microphone using the first combination
`of first microphone signal and second microphone signal.
`4 .
`. Form second virtual microphone using se-cond combination __
`··'<-.---70 8
`of first microphone signal and second microphone signal,
`!
`Generate denoised output signals having less
`acoustic noise than received acoustic signals,
`FIG.7
`
`100-----·)f
`
`'1 l 0
`-·-,
`·---1
`
`-
`
`Form physical microphone array including first
`physical microphone and second physical microphone.
`!
`Form virtual microphone array including first virtual
`microphone and second virtual microphone using
`signals from physical microphone array.
`800 -----~ft
`
`-
`
`- 8 -
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sep. 14, 2021
`Sep.14,2021
`
`Sheet 7 of 17
`Sheet 7 of 17
`
`US 11,122,357 B2
`US 11,122,357 B2
`
`Linear response of V2 fo a speech source at 0.10 meters 240 80
`
`240
`
`60
`
`30
`
`270
`
`- 9 -
`
`

`

`U.S. Patent
`U.S. Patent
`
`Sep. 14, 2021
`Sep.14,2021
`
`Sheet 8 of 17
`Sheet 8 of 17
`
`US 11,122,357 B2
`US 11,122,357 B2
`
`Linear response of V1 to a speech source at 0.10 meters
`
`FIG&12
`
`270
`
`iQ
`
`-10 -
`
`- 10 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 9 of 17
`
`US 11,122,357 B2
`
`Linear re:sponse of Vl to a speech source at 0.1 meters
`
`30
`
`\
`
`180 ........... ----~----···
`'
`'
`
`270
`
`- 11 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 10 of 17
`
`US 11,122,357 B2
`
`5 ........... : .......... :-....... .
`'
`
`.
`
`'
`
`,_.,.. .................... "'< ...................... ~.,, .,_..,.~ ... .., .... ..,- ......... 4
`;
`
`'
`;
`
`... ...
`
`i Cardioid speech!
`!
`·
`response
`t
`'
`
`Frequency response at O degrees
`10--------~-----~---.
`, ,
`'
`,
`:
`:
`' '
`
`~
`
`....
`
`..-1. - ..... ----·-~
`
`,
`'
`
`Vl speech ····:
`response
`
`,
`·--····!--·
`
`~ - - - - - •• " " " • »» • •
`
`~10 .............. :.
`
`M15 •••
`
`;
`
`;
`
`' : : :
`: :
`M20o.......__10-•0--20~0-0_3_00_0_4_00-•-5-0~00--60--00--70-00-. -8000
`Frequency (Hz)
`.FIG.14
`
`- 12 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 11 of 17
`
`US 11,122,357 B2
`
`1
`
`)
`
`C
`
`~~ - - -
`
`"'"' ......................... ! ............................ ._ ... ,
`
`'
`
`.,..,.,,.,,..,.,.~,.,~--..,,..,...,...,,..,...,..,,,J
`C
`l
`i
`'
`
`• """''""'" •••• ..c, .. .,,. ••• .,_. •• , •• --....... ~ •••••• _. ........ .
`
`1
`
`'
`
`"?'"'O ..... "lo,;
`.... ..,,. ...
`
`<
`
`:
`1
`
`-~
`"s:j ·-· ~
`
`~
`~
`
`0 n.
`
`t;n
`~
`~
`
`Vl (top, dashed) and V2 speech response vs. B assuming d8 = O. lm
`a
`----~--..,.___
`-ov..----~--
`:
`.
`'
`;
`;
`.
`-5
`····· 1···· . ., ...... ., .. ~;---~~·...;::,::.:;~·:.:.;~·:.:.~_:-.... •:-.···· ........ ., .... 1 ............ •·•••• -~ ---··· •»••·. •••••
`........... ,., ....... ..,,.. .... ! ........... ,.,..,..,~-:.;.1--~--~ .... i ,-¥.,.. ............... ...
`-10
`~15
`-20
`-25
`»30
`-35
`'
`-r··----
`··-{--
`40.___,,,__ _ _,_ _ _,_ _ _,_ _____ _,_,_.____,_ _ __.
`' '
`'
`0,6
`0,9
`0,8
`0.7
`0,4
`t1
`0,5
`1
`B
`FIG~15
`Vl/V2 for speech versus B assuming d5 = 0.lm
`4Q,-..---.--~--.,-----,-----.,.,---.---,
`:
`' <
`... .., ....... ,, ......... -~- ... -................. ...
`35
`.,.. ...... _ ........................... > ....................... ~ ...... ~ ................................... ~,.., .......... ,.. ................ ,:
`'
`' ' '
`'
`:
`i
`' <
`30
`--... " ... ; .... -·---- -··-- -.. {----·· .. ·····•· .. ,-., -:----·· --
`""· ..... ----:.- ... '
`'
`'
`'
`'
`'
`25
`____ J ___ _
`' ' < '
`20
`"••················:·-················:······
`< :
`'
`............. ,,, ......... "''"'t~ ............... ..................... v ... ,,..,., ...... ....,.. ....... ,.., ... r..,,.. ........... .,..'"'.,,.." i'"'"" ..,..,,.,,., ..,,...,...,,.
`
`' '
`' '
`'
`:
`'
`'
`..... ~ ..... - ....... ·····• •-·· ..... ··-· ..... --
`'
`'
`'
`'
`....... 'I' .......... ,,.. ............. - ---tr--............................. ,...,,.. ....... ,..., ......... ,.. ..... "' .... ....
`' <
`' ' '
`................... , ....... ··--. ______ ,.,. , ....
`'
`<
`···--i-·
`'
`
`..,,.,,. ............. ., ...... ,,;; .......................... y
`
`t
`'
`
`'
`'
`
`•
`(
`
`)
`
`<
`<
`
`'
`
`'
`
`t
`
`<
`<
`
`<
`
`<
`;
`
`l
`•.••••.• •»••· ~ ......... ..
`
`'
`t
`
`:
`
`---- -
`
`......................... ...
`
`'
`
`'
`'
`
`'
`
`10
`
`-..... -y·---- . .......
`'
`
`0
`
`'.
`OA
`
`0,5
`
`0.6
`
`0.9
`
`1
`
`t1
`
`OJ
`0,8
`B
`FIG&16
`
`- 13 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 12 of 17
`
`US 11,122,357 B2
`
`<
`
`B factor vs. actual ds assuming ds = O,lm and theta;;:; 0
`.. L . <
`' '
`········-·l-...
`'
`.
`
`1.2
`t1
`1
`
`I
`•• .. ~•••••• ••--••
`
`•
`••, ••••'
`
`--t~------- -......
`'
`
`,., .......... ,.,_..,,. ·....
`
`' ' "f"'"' ,,._,,.
`' ' ' '
`............. -;---·-· ........ .. y ... ...... .
`'
`
`'
`-·······f. ...
`'
`!
`'
`
`.
`
`t
`••••••••o•v••-•1: •••
`
`I
`
`I
`. <
`'
`
`:
`!
`'
`'
`. .. ···! ................ {·-
`<
`!
`;
`'
`<
`,
`• ••••••~, ••••• c•v••~•-••)
`!
`<
`. ··-- ~ .. •-.(·• .... -- . -.... -~.
`'
`'
`i
`:
`'
`'
`<
`'
`t
`.......... ,., ......... ,_...,,_.,. .., ........... ,.,...,. .... J: ,.,_ ....... ,., ......... ,.,'
`!
`f
`C
`!
`);,,.,,.,.,,,,.,,.,., •. ,.,., ... • ,,,. ..... ,., •• .,,..,,,< .. ,,....,
`s
`;
`~
`'
`<
`'
`<
`l
`{
`
`•
`
`................... ,., ....
`
`'
`
`C
`
`'
`
`~
`••• ••-••{ •• •• -
`>
`'
`<
`-- .. ")
`<
`:
`<
`'
`:
`~
`• ....... ,. .. ..,.J,,.,,., .... _,. .., ........ ½ .,..,,.,,._,_.,. ..,..,..1, ... ,.,., ....... ,.,_ ....... '
`~
`
`.
`!
`0.7
`'
`.
`.
`'
`'
`.. .... · .... · --; .. ·· · .... · ._ .... t .,. ... · .,. ..... ·r ... · ., ........... t • ._ ....... ,. ••• • ; ....... " .... • .. .
`0.6
`!
`!
`;
`... .-.:......
`'
`'
`0,5
`'
`······ .;....
`.) ......... - .... ~
`;
`!
`'
`<
`(L4 - - - - - - - - ' - - - - ' ____ __,
`<
`'
`(l.05
`0,25
`0.3
`0.35
`0.4
`0,45
`0.5
`0.2
`0,1
`0,15
`Actual ds (meters)
`FIG~l7
`B versus theta assuming d8 = O,lm
`1.25,-.---...------,--.._..---------
`: I
`I : I
`.,,. ...... ,.,..,. ... ,.,.. .... ~ ............ ,..., .... .,,_,...,! .... ,. .......... ,. ,.,.,,., t ........ ,_ ......... ...,..,~ .... ,. _, ...... ,., ........... : ......... ..., .......... ~ ...... t ...,.,,. ...... ,.. .......... ,.,
`'
`'
`'
`' ' <
`' <
`
`I
`
`;
`I
`<
`
`1.2 -··
`
`>
`
`t
`'
`
`f
`
`I
`
`I
`C
`
`"""f""
`
`'
`·····c
`'
`
`'
`-· -- ' ~ -·- .• ~------ -
`
`I
`
`.. !· ~
`
`' ' ' '
`' ' ' ' ' I ' '
`
`1.15 ...... .
`
`1.1
`
`1.05 v
`
`-
`
`u
`
`•
`
`· ---1··-.. -- -· - .•
`
`• .-. . ,,; . . . . . . . . . . . . . . . . . ~ " " " " ' " " " ' " • •
`
`. <
`,
`' < :
`' < '
`
`' . < . <
`..... ". ~--·: -- ------ ~ -
`' < : :
`'
`................. ; ................... ; ....... .
`··········· ... ; ............... ,} .. ······ ........... ; ................ .
`'
`' '
`' :
`1,..__,__ _ __., ________ ~-~--'--.....,,,_ ____ _,_--J
`~ ~ ~ ~ 0 W ~ 00
`00
`theta (degrees)
`FIG*18
`
`- 14 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 13 of 17
`
`US 11,122,357 B2
`
`0
`
`:
`
`;
`
`1
`
`'
`
`:
`
`Q.....---i - - - -~ - -~ , - - - - - - - - - - - .
`'
`'
`'
`,,;;-, ~20 ······ · ······ ·· ... !. ··············· · ············· · ········ ······ · .. t. · ······ · ······ · .L. · ······· ······ ·
`'
`'
`'
`· ····· ·········!············
`······ ·
`g
`:
`:
`@) -40 ......... , ...... · .... : . ., ...... .,......
`· ........ """ · .... ; .. · ...... · ...... · ... ~ .. · ...... · ...... · .,; .... · ...... · ...... · · ····-· ......... : .............. ········
`· '"·t" ·" +-·'"··-->"
`I
`-60 ······· !"·- "
`S
`, "''"'" ·
`:
`:
`:
`~
`~ Mao ~~~ ~~~~-~-~~~~-~~~~ ~~~{~-~~ ~~-~ ~~~ ~~n~ •-n~ ~~l~.~n~ ~~n~~~L~ ~~~~~~~~ ~;
`:
`:
`'
`'
`~1000--10-00--20_0_0 -3-00_0_4-00_0_5_00_0_6_0_00--70-00 _
`Frequency (Hz)
`
`~
`
`__,8000
`
`- 15 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 14 of 17
`
`US 11,122,357 B2
`
`N(s) for B = 1.2 and D-= -7.2e-006 seconds
`10r----, - - - - - - , - - - - - - - - - - .
`,
`'
`'
`'
`'
`'
`~ O ------·------ i---
`t·----
`·--
`:
`,
`·10 ''"A"'''"''"'''"'':''"'"''"•''"'""•' ;.'°-"'"'°"'"''"•'~'"'°"A'"'°"•''"'°-:
`~--
`--0
`:
`~~ ·20 m,p"'m•~•--.•~•••m
`•-tn•--•• '""~
`:
`~
`~
`~ ~30 --···--··----·+-··----· .. ·····
`.......... "":
`... ·-......... ~ ·-............ .
`'
`,
`~40---· - - - - - - - - - - - - - -
`0
`1000
`2000
`3000
`4000
`5000
`6000
`7000
`8000
`
`180w;_-----------------------------....!
`0
`1000
`2000
`3000
`4000
`5000
`6000
`7000
`8000
`Frequency (Hz)
`FIG~20
`
`- 16 -
`
`

`

`U.S. Patent
`
`Sep. 14,2021
`
`Sheet 15 of 17
`
`US 11,122,357 B2
`
`~
`
`~
`
`)
`
`I
`
`1
`
`:
`
`,
`90
`85 ............................... ,~1-,.,,.............
`~...-......
`'
`'
`... y .. .,,.,,, . . . . . . . . ~_._..,..,__ . . . . .,,. ........ ~ ... ,., . . .,.,,,,,...,.,. ... ,.,,.,.,,
`...
`............... ~,.,......
`.
`i
`~) 60 ··············--·+················: ............. ' ..................................... ; ·················:···
`............... .
`.. •······
`~ 75 '" .................. 1 ............... ., ... ~ .. " ...........
`. ............... ~ ...... ,. ... ._ ....... !. ........... .-.-...... ., ..... .-.... .
`: ........ .-....
`l
`i
`................ · ..
`70 ............ · ...... j-- ..
`· · · .. · · ...... ·;- · ...... · !
`;
`: .............. ..
`-
`65 ········ .. ···--··~·~··--------.•
`
`....... •--··:····················; ............
`!
`~
`l
`moo
`600
`5000
`2000
`6000
`3000
`4000
`Frequency (Hz)
`FIG.21
`
`:
`
`i
`
`~ . . - . y
`
`: ...... -
`
`........
`
`'
`
`i
`
`7000
`
`8000
`
`- 17 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 16 of 17
`
`US 11,122,357 B2
`
`10
`
`Cancellation ,vith dl = l, theta.I = 0, d2 = l, and theta2 = 45
`'
`,
`'
`'
`'
`'
`'
`
`'. ........ = ........ _.--J~~t--= ....... ~ ... : ......... = ...... ri=-....... -.: ...... ~ .... ~ .. J .. ;~: :::::~~::~:: :1 .. , .. ~-:~: : ~: ::1::~:
`
`------ .....
`
`- '" -- ·- • - -- .. - - -y- • - -- - - ., ....
`
`' : <
`' ' < '
`*30 ........................ T.... ..............
`.. .................... ..
`---------l-----·-·-
`'
`*40---· _____________ __,
`'
`0
`1000
`2000
`3000
`4000
`5000
`6000
`7000
`8000
`so~--,----,.---,-----,---.....---,----.----,
`' '
`......... 7,., ...................... .., ... ; ....... --~ .. .,, ............... r .. ,,, .......... ,,, ............ :~ ........................... i .... ,., ................ ~,., .~ ............... ..,. ......... ...
`'
`... w"
`~) 80 ................................. ]'...............................
`.. ....................... ; ........................ ; ............................ --!- .................................. ;............................
`.. .............................. ..
`~ 75 ................................. ; .......................................................... ;.............................
`'
`....................................................... ..
`~ 70 ..................................... + .............................. ..
`_,,
`:
`~ p..
`
`~
`
`j
`
`)
`
`l
`
`C
`
`C
`
`:
`
`I
`
`:
`
`•
`
`,
`
`1000
`
`2000
`
`6000
`
`7000
`
`8000
`
`- ------ - ........ >---.- - -·---
`
`' ' '
`: < '
`5000
`4000
`3000
`Frequency (Hz)
`
`- 18 -
`
`

`

`U.S. Patent
`
`Sep.14,2021
`
`Sheet 17 of 17
`
`US 11,122,357 B2
`
`Original Vl (top) and cleaned Vl (bottom) with simplified VAD (dashed) in noise
`0.4.....----..----..----...-----,-----.----,
`0.3
`0.2 -
`0.1
`0
`~0.1
`»0,2 .
`0
`
`0.5
`
`1
`
`1.5
`
`2
`
`2.5
`~
`X 10"
`
`I""" ....... "" ....... "".......
`
`I
`
`I
`
`'
`'
`'
`'
`
`0,3
`
`0.2 1 0.1--
`
`0
`\\)
`0 »0.1 -
`»0,2 .
`0
`
`--~
`
`'
`'
`'
`
`'
`
`'
`
`' ' ' ' '
`
`0.5
`
`1
`1.5
`Time (samples at 8 kHz/sec)
`
`2
`
`2.5
`.::
`X 10"
`
`- 19 -
`
`

`

`US 11,122,357 B2
`
`1
`FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`MICROPHONE ARRAY (DOMA)
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`2
`acoustic noise. The technology supporting the Jawbone is
`currently described under one or more of U.S. Pat. No.
`7,246,058 by Burnett and/or U.S. patent application Ser.
`Nos. 10/400,282, 10/667,207, and/or 10/769,302. Generally,
`5 multi-microphone techniques make use of an acoustic-based
`Voice Activity Detector (VAD) to determine the background
`noise characteristics, where "voice" is generally understood
`to include human voiced speech, unvoiced speech, or a
`combination of voiced and unvoiced speech. The Jawbone
`10 improved on this by using a microphone-based sensor to
`construct a VAD signal using directly detected speech vibra-
`tions in the user's cheek. This allowed the Jawbone to
`aggressively remove noise when the user was not producing
`speech. However, the Jawbone uses a directional micro-
`phone array.
`
`This application is a continuation of U.S. Nonprovisional
`patent application Ser. No. 12/139,333, filed Jun. 13, 2008,
`now U.S. Pat. No. 8,503,691, entitled "Forming Virtual
`Microphone Arrays Using Dual Onmidirectional Micro-
`phone Array (DOMA)," which claims the benefit of U.S.
`Provisional Patent Application No. 60/934,551, filed Jun.
`13, 2007, U.S. Provisional Patent Application No. 60/953,
`444, filed Aug. 1, 2007, U.S. Provisional Patent Application 15
`No. 60/954,712, filed Aug. 8, 2007, and U.S. Provisional
`Patent Application No. 61/045,377, filed Apr. 16, 2008, all
`of which are incorporated by reference herein in their
`entirety for all purposes.
`
`INCORPORATION BY REFERENCE
`
`20
`
`Each patent, patent application, and/or publication men-
`tioned in this specification is herein incorporated by refer-
`ence in its entirety to the same extent as if each individual
`patent, patent application, and/or publication was specifi-
`cally and individually indicated to be incorporated by ref-
`25 erence.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`TECHNICAL FIELD
`
`The disclosure herein relates generally to noise suppres-
`sion. In particular, this disclosure relates to noise suppres-
`sion systems, devices, and methods for use in acoustic
`applications.
`
`BACKGROUND
`
`40
`
`Conventional adaptive noise suppression algorithms have
`been around for some time. These conventional algorithms
`have used two or more microphones to sample both an
`(unwanted) acoustic noise field and the (desired) speech of
`a user. 20 The noise relationship between the microphones
`is then determined using an adaptive filter (such as Least-
`Mean-Squares as described
`in Haykin & Widrow,
`ISBN#0471215708, Wiley, 2002, but any adaptive or sta-
`tionary system identification algorithm may be used) and
`that relationship used to filter the noise from the desired
`signal.
`Most conventional noise suppression systems currently in
`use for speech communication systems are based on a
`single-microphone spectral subtraction
`technique
`first
`develop in the 1970's and described, for example, by S. F.
`Boll in "Suppression of Acoustic Noise in Speech using
`Spectral Subtraction," IEEE Trans. on ASSP, pp. 113-120,
`1979. These techniques have been refined over the years, 30
`but the basic principles of operation have remained the
`same. See, for example, U.S. Pat. No. 5,687,243 of
`McLaughlin, et al., and U.S. Pat. No. 4,811,404 ofVilmur, 50
`et al. There have also been several attempts at multi-
`microphone noise suppression systems, such as those out-
`lined in U.S. Pat. No. 5,406,622 of Silverberg et al. and U.S.
`Pat. No. 5,463,694 of Bradley et al. Multi-microphone
`systems have not been very successful for a variety of 55
`reasons, the most compelling being poor noise cancellation
`performance and/or significant speech distortion. Primarily,
`conventional multi-microphone systems attempt to increase
`the SNR of the user's speech by "steering" the nulls of the
`system to the strongest noise sources. This approach is 60
`limited in the number of noise sources removed by the
`number of available nulls.
`The Jawbone earpiece (referred to as the "Jawbone),
`introduced in December 2006 by AliphCom of San Fran-
`cisco, Calif., was the first known commercial 10 product to
`use a pair of physical directional microphones (instead of
`onmidirectional microphones) to reduce environmental
`
`FIG. 1 is a two-microphone adaptive noise suppression
`30 system, under an embodiment.
`FIG. 2 is an array and speech source (S) configuration,
`under an embodiment. The microphones are separated by a
`distance approximately equal to 2d0 , and the speech source
`is located a distance ds away from the midpoint of the array
`35 at an angle 8. The system is axially symmetric so only ds and
`8 need be specified.
`FIG. 3 is a block diagram for a first order gradient
`microphone using two onmidirectional elements 0 1 and 0 2 ,
`under an embodiment.
`FIG. 4 is a block diagram for a DOMA including two
`physical microphones configured to form two virtual micro-
`phones V 1 and V 2 , under an embodiment.
`FIG. 5 is a block diagram for a DOMA including two
`physical microphones configured to form N virtual micro-
`45 phones V 1 through V N, where N is any number greater than
`one, under an embodiment.
`FIG. 6 is an example of a headset or head-worn device
`that includes the DOMA, as described herein, under an
`embodiment.
`FIG. 7 is a flow diagram for denoising acoustic signals
`using the DOMA, under an embodiment.
`FIG. 8 is a flow diagram for forming the DOMA, under
`an embodiment.
`FIG. 9 is a plot of linear response of virtual microphone
`V 2 to a 1 kHz speech source at a distance of0.1 m, under an
`embodiment. The null is at O degrees, where the speech is
`normally located.
`FIG. 10 is a plot of linear response of virtual microphone
`V 2 to a 1 kHz noise source at a distance of 1.0 m, under an
`embodiment. There is no null and all noise sources are
`detected.
`FIG. 11 is a plot of linear response of virtual microphone
`V 1 to a 1 kHz speech source at a distance of0.1 m, under an
`embodiment. There is no null and the response for speech is
`65 greater than that shown in FIG. 9.
`FIG. 12 is a plot of linear response of virtual microphone
`V 1 to a 1 kHz noise source at a distance of 1.0 m, under an
`
`- 20 -
`
`

`

`3
`embodiment. There is no null and the response is very
`similar to V2 shown in FIG. 10.
`FIG. 13 is a plot oflinear response of virtual microphone
`V 1 to a speech source at a distance of0.1 m for frequencies
`of 100, 500, 1000, 2000, 3000, and 4000 Hz, under an
`embodiment.
`FIG. 14 is a plot showing comparison of frequency
`responses for speech for the array of an embodiment and for
`a conventional cardioid microphone.
`FIG. 15 is a plot showing speech response for V 1 (top,
`dashed) and V 2 (bottom, solid) versus B with ds assumed to
`be 0.1 m, under an embodiment. The spatial null in V2 is
`relatively broad.
`FIG. 16 is a plot showing a ratio of V 1N 2 speech 15
`responses shown in FIG. 10 versus B, under an embodiment.
`The ratio is above 10 dB for all 0.8<8<1.1. This means that
`the physical ~ of the system need not be exactly modeled for
`good performance.
`FIG. 17 is a plot ofB versus actual ds assuming that ds =10
`cm and theta=0, under an embodiment.
`FIG. 18 is a plot of B versus theta with ds=l0 cm and
`assuming ds=l0 cm, under an embodiment.
`FIG. 19 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=l and D=-7.2 µsec, under an
`embodiment. The resulting phase difference clearly affects
`high frequencies more than low.
`FIG. 20 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=l.2 and D=-7.2 µsec, under an
`embodiment. Non-unity B affects the entire frequency range.
`FIG. 21 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V 2 due
`to a mistake in the location of the speech source with q 1 =0
`degrees and q2=30 degrees, under an embodiment. The
`cancellation remains below -10 dB for frequencies below 6
`kHz.
`FIG. 22 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V 2 due
`to a mistake in the location of the speech source with q 1 =0
`degrees and q2=45 degrees, under an embodiment. The
`cancellation is below -10 dB only for frequencies below
`about 2.8 kHz and a reduction in performance is expected.
`FIG. 23 shows experimental results for a 2d0=19 mm
`array using a linear~ of0.83 on a Brue! and Kjaer Head and
`Torso Simulator (HATS) in very loud (-85 dBA) music/ 45
`speech noise environment, under an embodiment. The noise
`has been reduced by about 25 dB and the speech hardly
`affected, with no noticeable distortion.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides for dual onmidirectional
`microphone array devices systems and methods.
`In accordance with on embodiment, a microphone array is
`formed with a first virtual microphone that includes a first
`combination of a first microphone signal and a second
`microphone signal, wherein the first microphone signal is
`generated by a first physical microphone and the second
`microphone signal is generated by a second physical micro-
`phone; and a second virtual microphone that includes a
`second combination of the first microphone signal and the
`second microphone signal, wherein the second combination
`is different from the first combination. The first virtual
`microphone and the second virtual microphone are distinct
`virtual directional microphones with substantially similar
`responses to noise and substantially dissimilar responses to
`speech.
`
`US 11,122,357 B2
`
`4
`In accordance with another embodiment, a microphone
`array is formed with a first virtual microphone formed from
`a first combination of a first microphone signal and a second
`microphone signal, wherein the first microphone signal is
`5 generated by a first onmidirectional microphone and the
`second microphone signal is generated by a second omni-
`directional microphone; and a second virtual microphone
`formed from a second combination of the first microphone
`signal and the second microphone signal, wherein the sec-
`10 ond combination is different from the first combination. The
`first virtual microphone has a first linear response to speech
`that has a single null oriented in a direction toward a source
`of the speech, wherein the speech is human speech.
`In accordance with another embodiment, a device
`includes a first microphone outputting a first microphone
`signal and a second microphone outputting a second micro-
`phone signal; and a processing component coupled to the
`first microphone signal and the second microphone signal,
`the processing component generating a virtual microphone
`20 array comprising a first virtual microphone and a second
`virtual microphone, wherein the first virtual microphone
`comprises a first combination of the first microphone signal
`and the second microphone signal, and wherein the second
`virtual microphone comprises a second combination of the
`25 first microphone signal and the second microphone signal.
`The second virtual microphone have substantially similar
`responses to noise and substantially dissimilar responses to
`speech.
`In accordance with another embodiment, a devise
`30 includes a first microphone outputting a first microphone
`signal and a second microphone ou