Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 1 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 1 of 38
`
`
`
`
`EXHIBIT G
`EXHIBIT G
`
`
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 2 of 38
`ee”TETTTT
`
`US011122357B2
`
`a2) United States Patent
`US 11,122,357 B2
`(0) Patent No.:
`*Sep. 14, 2021
`(45) Date of Patent:
`Burnett
`
`(54)
`
`FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`
`MICROPHONE ARRAY (DOMA)
`
`(71)
`
`(72)
`
`Applicant: Jawbone Innovations, LLC, Marshall,
`TX (US)
`
`Inventor: Gregory C. Burnett, Dodge Center,
`MN (US)
`
`(73)
`
`Assignee:
`
`Jawbone Innovations, LLC, Marshall,
`TX (US)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21)
`
`Appl. No.: 13/959,708
`
`(22)
`
`Filed:
`
`Aug. 5, 2013
`
`(65)
`
`(63)
`
`Prior Publication Data
`
`US 2014/0185825 Al
`
`Jul. 3, 2014
`
`Related U.S. Application Data
`
`Continuation of application No. 12/139,333, filed on
`Jun. 13, 2008, now Pat. No. 8,503,691.
`(Continued)
`
`(51)
`
`Int. Cl.
`
`HOAR 3/00
`HOAR 1/10
`
`(52)
`
`USS. Cl.
`
`(2006.01)
`(2006.01)
`(Continued)
`
`CPC waeees HO4R 1/1091 (2013.01); GIOL 21/0208
`(2013.01); HO4R 1/406 (2013.01);
`(Continued)
`
`(58) Field of Classification Search
`CPC .... HO4R 3/005; HO4R 2410/05; HO4R 3/002;
`HO4R 3/04; HO4R 1/1091; HO4R 1/406;
`(Continued)
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4,653,102 A
`4,777,649 A
`
`3/1987 Hansen
`10/1988 Carlson et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`
`2009003180
`
`12/2008
`
`OTHER PUBLICATIONS
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Apr. 10, 2012 for U.S. Appl. No. 12/139,333.
`(Continued)
`
`Primary Examiner — Marcos D. Pizarro
`(74) Attorney, Agent, or Firm — Nutter McClennen &
`Fish LLP
`
`ABSTRACT
`(57)
`A dual omnidirectional microphone array noise suppression
`is described. Compared to conventional arrays and algo-
`rithms, which seek to reduce noise by nulling out noise
`sources, the array of an embodimentis used to form two
`distinct virtual directional microphones which are config-
`ured to have very similar noise responses and very dissimilar
`speech responses. The only null formed is one used to
`remove the speech of the user from V,. The two virtual
`microphonesmaybepaired with an adaptive filter algorithm
`and VADalgorithm to significantly reduce the noise without
`distorting the speech, significantly improving the SNR of the
`desired speech over conventional noise suppression systems.
`
`20 Claims, 17 Drawing Sheets
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 3 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 3 of 38
`
`US 11,122,357 B2
`
`Page 2
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/934,551, filed on Jun.
`13, 2007, provisional application No. 60/953,444,
`filed on Aug. 1, 2007, provisional application No.
`60/954,712,
`filed on Aug. 8, 2007, provisional
`application No. 61/045,377, filed on Apr. 16, 2008.
`
`2003/0044025 Al
`2003/0228023 Al
`2007/0003082 Al
`2007/0121974 Al
`2009/0010450 Al
`
`3/2003 Ouyanget al.
`12/2003 Burnett etal.
`1/2007 Pedersen
`5/2007 Nemirovski
`1/2009 Burnett
`
`OTHER PUBLICATIONS
`
`(51)
`
`(2013.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`Int. Cl.
`GOL 21/0208
`HOAR 1/40
`HOAR 3/04
`GIOL 21/0216
`(52) U.S. Cl
`CPC veeceseseeee HOAR 3/002 (2013.01); HO4R 3/005
`(2013.01); HO4R 3/04 (2013.01); GIOL
`2021/02165 (2013.01); HO4R 2460/01
`(2013.01)
`
`(58) Field of Classification Search
`CPC ...... HO4R 2460/01; GIOL 2021/02165; G10L
`21/0208
`USPC wiceceeeeeeees 381/92, 94.7; 704/233, E21.004
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5/1993 Kaneda
`5,208,864 A
`1/1994 Freeman et al.
`5,276,765 A
`10/1994 Oh et al.
`5,353,376 A
`4/1995 Silverberg et al.
`5,406,622 A
`10/1995 Bradelyet al.
`5,463,694 A
`12/1995 Cezanneet al.
`5,473,701 A
`12/1995 Yoshidaet al.
`5,473,702 A
`5/1996 Sugiyama
`5,517,435 A
`12/1996 Park etal.
`5,590,241 A
`4/1997 Matouketal.
`5,625,684 A
`5/1997 Kanamori et al.
`5,633,935 A
`9/1997 Yamaguchietal.
`5,664,014 A
`9/1997 Nishiguchiet al.
`5,664,052 A
`3/1998 Holzrichter etal.
`5,729,694 A
`5/1998 Hosoi
`5,754,665 A
`9/1998 Claybaughet al.
`5,815,582 A
`10/1998 Andreaet al.
`5,825,897 A
`5/1999 Takada
`5,907,624 A
`12/1999 Holzrichter
`6,006,175 A
`5/2001 Choet al.
`6,233,551 Bl
`3/2004 Valveetal.
`6,707,910 Bl
`9/2004 Housni
`6,795,713 B2
`11/2005 Vaudreyetal.
`6,963,649 B2
`12/2005 Turnbull et al.
`6,980,092 B2
`4/2007 Yang et al.
`7,206,418 B2
`6/2008 Fan
`7,386,135 B2
`8,068,619 B2* 11/2011 Zhang etal. we. 381/92
`.. 381/92
`8,494,177 B2*
`7/2013 Burnett
`....
`
`8/2013 Burnett ...0 ee 381/92
`8,503,691 B2*
`8/2015 Burnett
`9,099,094 B2
`2002/0110256 Al
`8/2002 Watsonet al.
`2002/0116187 Al
`8/2002 Erte
`
`Weiss, Howard; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 14, 2011 for U.S. Appl. No. 12/139,333.
`Copenheaver, Blaine R; International Searching Authority; Notifi-
`cation of Transmittal of the International Search Report and Written
`Opinion of the International Searching Authority of the Declaration
`for International Patent Application No. PCT/US2008/068634, dated
`Sep. 2, 2008.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 10, 2011 for U.S. Appl. No. 12/163,592.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Nov. 16, 2011 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jan. 16, 2009 for U.S. Appl. No. 11/805,987.
`Azad, Abul K..; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 6, 2008 for U.S. Appl. No. 11/805,987.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Oct. 10, 2006 for U.S. Appl. No. 10/159,770.
`Shah, Paras D.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 15, 2005 for U.S. Appl. No. 10/159,770.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 31, 2013 for U.S. Appl. No. 13/436,765.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 30, 2010 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 24, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 11, 2009 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Jul. 9, 2008 for U.S. Appl. No. 10/667,207.
`Lao, Lun S..; Office Action mailed by U.S. Patent and Trademark
`Office dated Feb. 9, 2007 for U.S. Appl. No. 10/667,207.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Mar. 3, 2016 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Aug. 7, 2015 for U.S. Appl. No. 14/224,868.
`Tran, Long K.; Office Action mailed by U.S. Patent and Trademark
`Office dated Dec. 21, 2014 for U.S. Appl. No. 14/224,868.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 3, 2016 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Jun. 23, 2011 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 17, 2010 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Dec. 9, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Mar. 16, 2009 for U.S. Appl. No. 10/400,282.
`Faulk, Devona E.; Office Action mailed by U.S. Patent and Trade-
`mark Office dated Aug. 18, 2008 for U.S. Appl. No. 10/400,282.
`
`* cited by examiner
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 4 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 4 of 38
`
`US 11,122,357 B2
`
`U.S. Patent
`
`
`
`Sep. 14, 2021
`
`Sheet 1 of 17
`
`yooads
`
`powest5 001-7
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 5 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 5 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 2 of 17
`
`US 11,122,357 B2
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 6 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 6 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 3 of 17
`
`US 11,122,357 B2
`
`
`
`
`
`Ay
`
`Ay
`
`Ay
`
`Ay
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 7 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 7 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 4 of 17
`
`US 11,122,357 B2
`
`
`
`241
`
`x
`
`oy
`
`x}
`
`24
`
`:
`
`:
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 8 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 8 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 5 of 17
`
`US 11,122,357 B2
`
`.
`
`\on
`
`ne
`
`FIG.6
`
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 9 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 9 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 6 of 17
`
`US 11,122,357 B2
`
` Receive acoustic signals at a first physical
`mi
`ical
`ie
`microphone and a second physical microphone.
`
`
`
`
`po
`poTO
`&
`
`
`
`Output first microphone signal fromfirst physica
`microphone and second microphone signal from
`second physical microphone.
`
`
`|
`
`
`
`
`
`
`Formfirst virtual microphone using the first combination | 706
`of first microphone signal and second microphone signal.
`| ~~
`
`
`
`Form second virtual microphone using second combination |
`of first microphone signal and second microphone signal.
`
`
`Generate denoised output signals having less
`
`acoustic noise than received acoustic signals.
`
`
`
`Form physical microphone array including first
`physical microphone and second physical microphone.
`
`signals from physical microphone array.
`
`Porm virtual microphone array including first virtual
`microphone and second virtual microphone using
`
`300"
`
`FIGS
`
`
`
`|
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 10 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 10 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 7 of 17
`
`US 11,122,357 B2
`
`Linear response of V2 to a speech source af 0.10 meters
`
`120.60
`8-
`
`‘
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 11 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 11 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 8 of 17
`
`US 11,122,357 B2
`
`Linear response of V1 fo a speech source af 0.10 meters
`80.
`
`0.8
`
`iQ
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 12 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 12 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 9 of 17
`
`US 11,122,357 B2
`
`Linear response of V1 to a speech source at 0.1 meters
`
`90
`2
`ee
`
`Nerena, Pannenne
`
`eg
`
`é
`
`.
`
`%
`
`on
`
`earn eT
`
`”
`
`¥
`
`‘
`
`Ns
`\
`*
`\
`
`%
`
`“
`
`»
`
`aaa
`
`*
`
`*
`
`:
`
`;
`
`o
`
`<
`
`:
`
`s
`
`s
`
`-
`
`%
`
`%,
`
`ot
`
`.
`
`‘,
`SO
`ak
`%,
`
`.
`
`te
`
`“
`
`.
`
`x
`
`.
`
`a
`\
`.*
`an
`Ba,
`a.
`ig
`
`Re
`
`Ry
`
`Dg
`
`.
`
`Ty
`
`SA
`Tree
`em
`
`Fk
`
`-
`&
`
`00
`
`“Prag,
`t
`Pome,
`ay
`abo
`~
`
`:
`
`2
`
`7
`
`3
`i
`FY
`f
`»*
`go ; FON
`i
`3
`j
`ae
`}
`as
`‘¢
`al
`x
`Hein og
`%
`x
`;
`or:
`f
`on
`i
`conor +
`if
`i
`oor:
`:
`oe?
`sea
`Bre
`“Poemanragoosoas00 SOR ‘
`’
`,
`.é
`
`270
`
`FIG.13
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 13 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 13 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 10 of 17
`
`US 11,122,357 B2
`
`10,
`
`Frequency response at 0 degrees
`|
`
`faeencdn eoneeeedaenn eranniawnnaeenethenesian tedawmnaanetd enaaemanved
`
`6) ooneccnnnntee
`pan,
`S|
`= & |.
`is
`ae
`|.oseeennosfosoneono9c00cse0009eoronsosoeoeonanejeonsndsseosenenn9dssannspenasaxoessashaosodnepnosousscanansenenascaddecannnagodsesboa0eca;aoosbesscanabesesescnaqgseosasoaouaSsonoooSUeOHOGDNONS
`= 40
`:
`|
`AB)
`
`.
`
`response
`
`OF
`
`TR00 B00
`
`
`4000
`BOGS
`BOG
`FHT
`Sooo
`3060
`Frequency (Hz)
`
`FIG.14
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 14 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 14 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 11 of 17
`
`US 11,122,357 B2
`
`TS A 0 0 nee 928paagg AE I my 28 ay a NB ‘
`
`BR new
`:
`ae Se
`»
`FE aon
`np
`:
`
`The
`
`
`
`Response(dB)
`
`
`
`V1/V2forspeech(dB)
`
`V1 (top, dashed) and V2 speechrespon evs, 8 assuming d, =im wo Po a oo AS oe,
`V1/V2 for speech versus B assuming d, = 0.1m eeESE
`
`a)eH
`
`4
`
`05
`
`06
`
`9
`
`):
`
`i
`
`08
`07
`FIG.15
`
`04.
`
`05
`
`06
`
`O7
`
`08
`
`08
`
`1
`
`44
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 15 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 15 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 12 of 17
`
`US 11,122,357 B2
`
`ech cee cee cee bee ee eee eee ee ereDe ceeee cee cenceeee ee creeBee creecrab a creer een
`i
`‘
`‘
`,
`‘
`
`$
`
`:
`
`;
`
`3
`
`:
`
`‘
`3
`1
`:
`‘
`i
`i
`:
`i
`
`:
`‘
`:
`:
`:
`:
`:
`
`t
`£
`b
`t
`i
`
`005
`
`01
`
`O18
`
`:
`:
`:
`i
`£
`:
`:
`:
`
`)
`t
`
`:
`
`3
`i
`.
`!
`i
`:
`i
`3
`
`3
`$
`Q
`i
`>
`
`:
`:
`‘
`:
`:
`
`'
`t
`t
`c
`z
`»
`r
`
`
`03
`035
`O04
`O48
`05
`025
`02
`Actnal d, (meters)
`FIG.17
`
`B versus theta assuming d, = @.1m
`
`»
`r
`t
`©
`*
`r
`»
`t
`t
`‘
`i
`tr
`:
`‘
`Dy
`i
`&
`:
`c%
`:
`re
`ree °
`t
`.
`.
`~
`4 4 ok eh ee ee wt deen ee nee ae ee eh ce ee
`.
`tf'
`‘
`:
`t
`%
`:
`a
`§
`:
`t
`:
`i
`i
`t
`i
`:
`t
`:
`?
`?
`?
`t
`»
`
`.
`
`cnet, ho
`
`3
`
`’
`
`3
`
`«
`
`i
`i
`i
`1
`i
`
`.
`:
`i
`i
`:
`$
`
`f
`
`¢
`
`é
`
`Z
`
`¢
`
`+
`t
`t
`r
`+
`z
`»
`
`‘
`t
`t
`:
`t
`»
`
`6
`
`80
`
`B factor vs. actual d, assuming d, = 0.le and theta = 0
`
`wineneenaeeeteeenebeeenha:t
`
`‘,
`
`»
`3
`i
`:
`t
`
`‘
`
`%
`
`%
`
`+ ‘ t t t + : : + + % ‘ 4 + : £ ‘ + : ‘~~beneeeeRoe
`
`i
`i
`i
`:
`:
`
`‘
`>
`t
`
`‘
`
`ts
`
`et
`&
`ok
`?
`g
`
`?
`
`;
`
`‘
`:
`:
`i
`+
`
`i
`i
`‘
`5
`:
`
`4
`:
`3
`q
`
`‘
`2
`
`#
`
`#
`
`4
`3°
`Ft
`?
`a
`
`°0 «2 4
`8 4 20
`theta (degrees)
`FIG.18
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 16 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 16 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 13 of 17
`
`US 11,122,357 B2
`
`
`2000
`3000
`4000-000
`8000
`7000
`8000
`
`. {000
`
`
`
`Amplitude(dB)
`
`
`
`Phase(degrees)
`
`N(s) for B= | and D =-7.2e-006 seconds
`
`
`6
`
`1900
`
`«2000
`
`«300040005000
`
`«80607000
`
`«BOON
`
`Frequency (Hz)
`
`FIG.19
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 17 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 17 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 14 of 17
`
`US 11,122,357 B2
`
`
`
`Amplitude(dB)
`
`
`
`Phase(degrees)
`
`N(s) for B = 1.2 and D = -7.2e-006 seconds
`
`
`
`
`
`
`0
`T0a0 NG BGT
`AGT
`BRT
`HORT
`FHT
`B00
`
`
`
`6
`1000
`3060
`3000
`R00
`HOST
`BGT
`Frequency (Hz)
`
`
`7600
`Ba00
`
`FIG.20
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 18 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 18 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 15 of 17
`
`US 11,122,357 B2
`
`0 Cancellation withdl =|, thetal = 6, d2 = 1, and theta2 = 30
`
`
`ttn
`c —
`,
`:
`:
`,
`,
`SB
`10)
`= at) |
`3)
`
`Go
`TRGS
`Badd
`000
`4000
`O00
`BOT
`7OGd-Ba00
`
`She
`
`
`
`Phase(degrees)
`
`
`
`
`
`9
`FRG0 G0 SORT
`FBGT G0
`GOUT
`FUGT
`B00
`Frequency (Hz)
`
`FIG.2]
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 19 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 19 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 16 of 17
`
`US 11,122,357 B2
`
`
`
`Amplitude(dB)
`
`
`
`Phase(degrees)
`
`Cancellation with dl = 1, thetal
`=0,d2=
`1, andtheta2 = 45
`
`
`
`
`{000
`2000
`3000
`40005000
`000
`7006
`8000
`
`"
`
`
`
`5000
`4000 O00 O00
`Oo
`{000-2606
`Frequency (Hz)
`
`
`7000
`B00
`
`FIG,22
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 20 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 20 of 38
`
`U.S. Patent
`
`Sep. 14, 2021
`
`Sheet 17 of 17
`
`US 11,122,357 B2
`
`Original V1 (top) and cleaned Vi (bottom) with strplilied VAD (dashed) in noise
`
`Br ne ne ane ane ane a
`
`x10
`
`yrrwawmenen
`
`FIG.23
`
`YELLE
`
`eertte
`
`¥ii44334433443344aa‘
`
`3&3kr
`
`$$&h
`
`k
`
`pewesyg
`
`
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 21 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 21 of 38
`
`US 11,122,357 B2
`
`1
`FORMING VIRTUAL MICROPHONE
`ARRAYS USING DUAL OMNIDIRECTIONAL
`
`MICROPHONE ARRAY (DOMA)
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. Nonprovisional
`patent application Ser. No. 12/139,333, filed Jun. 13, 2008,
`now U.S. Pat. No. 8,503,691, entitled “Forming Virtual
`Microphone Arrays Using Dual Omnidirectional Micro-
`phone Array (DOMA),” which claims the benefit of US.
`Provisional Patent Application No. 60/934,551, filed Jun.
`13, 2007, U.S. Provisional Patent Application No. 60/953,
`444,filed Aug. 1, 2007, U.S. Provisional Patent Application
`No. 60/954,712, filed Aug. 8, 2007, and U.S. Provisional
`Patent Application No. 61/045,377, filed Apr. 16, 2008,all
`of which are incorporated by reference herein in their
`entirety for all purposes.
`
`TECHNICAL FIELD
`
`The disclosure herein relates generally to noise suppres-
`sion. In particular, this disclosure relates to noise suppres-
`sion systems, devices, and methods for use in acoustic
`applications.
`
`BACKGROUND
`
`Conventional adaptive noise suppression algorithms have
`been around for some time. These conventional algorithms
`have used two or more microphones to sample both an
`(unwanted) acoustic noise field and the (desired) speech of
`a user. 20 The noise relationship between the microphones
`is then determined using an adaptive filter (such as Least-
`Mean-Squares
`as
`described in Haykin & Widrow,
`ISBN#0471215708, Wiley, 2002, but any adaptive or sta-
`tionary system identification algorithm may be used) and
`that relationship used to filter the noise from the desired
`signal.
`Most conventional noise suppression systems currently in
`use for speech communication systems are based on a
`single-microphone
`spectral
`subtraction technique
`first
`develop in the 1970’s and described, for example, by S. F.
`Boll in “Suppression of Acoustic Noise in Speech using
`Spectral Subtraction,” IEEE Trans. on ASSP, pp. 113-120,
`1979. These techniques have been refined over the years, 30
`but the basic principles of operation have remained the
`same. See,
`for example, U.S. Pat. No. 5,687,243 of
`McLaughlin,et al., and U.S. Pat. No. 4,811,404 of Vilmur,
`et al. There have also been several attempts at multi-
`microphone noise suppression systems, such as those out-
`lined in U.S. Pat. No. 5,406,622 of Silverberg et al. and U.S.
`Pat. No. 5,463,694 of Bradley et al. Multi-microphone
`systems have not been very successful for a variety of
`reasons, the most compelling being poor noise cancellation
`performanceand/or significant speech distortion. Primarily,
`conventional multi-microphone systems attempt to increase
`the SNRof the user’s speech by “steering” the nulls of the
`system to the strongest noise sources. This approach is
`limited in the number of noise sources removed by the
`numberofavailable nulls.
`The Jawbone earpiece (referred to as the “Jawbone),
`introduced in December 2006 by AliphCom of San Fran-
`cisco, Calif., was the first known commercial 10 product to
`use a pair of physical directional microphones (instead of
`omnidirectional microphones)
`to reduce environmental
`
`me 5
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`acoustic noise. The technology supporting the Jawbone is
`currently described under one or more of U.S. Pat. No.
`7,246,058 by Burnett and/or U.S. patent application Ser.
`Nos. 10/400,282, 10/667,207, and/or 10/769,302. Generally,
`multi-microphone techniques make use of an acoustic-based
`Voice Activity Detector (VAD)to determine the background
`noise characteristics, where “voice” is generally understood
`to include human voiced speech, unvoiced speech, or a
`combination of voiced and unvoiced speech. The Jawbone
`improved on this by using a microphone-based sensor to
`construct a VAD signal using directly detected speech vibra-
`tions in the user’s cheek. This allowed the Jawbone to
`
`aggressively remove noise whenthe user wasnot producing
`speech. However, the Jawbone uses a directional micro-
`phonearray.
`
`INCORPORATION BY REFERENCE
`
`Each patent, patent application, and/or publication men-
`tioned in this specification is herein incorporated by refer-
`ence in its entirety to the same extent as if each individual
`patent, patent application, and/or publication was specifi-
`cally and individually indicated to be incorporated by ref-
`erence.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a two-microphone adaptive noise suppression
`system, under an embodiment.
`FIG. 2 is an array and speech source (S) configuration,
`under an embodiment. The microphonesare separated by a
`distance approximately equal to 2d, and the speech source
`is located a distance d, away from the midpoint of the array
`at an angle 0. The system is axially symmetric so only d, and
`8 need be specified.
`FIG. 3 is a block diagram for a first order gradient
`microphone using two omnidirectional elements O, and O,,
`under an embodiment.
`FIG. 4 is a block diagram for a DOMAincluding two
`physical microphones configured to form two virtual micro-
`phones V, and V., under an embodiment.
`FIG. 5 is a block diagram for a DOMAincluding two
`physical microphones configured to form N virtual micro-
`phones V, through V,,, where N is any numbergreater than
`one, under an embodiment.
`FIG. 6 is an example of a headset or head-worn device
`that includes the DOMA, as described herein, under an
`embodiment.
`
`FIG. 7 is a flow diagram for denoising acoustic signals
`using the DOMA, under an embodiment.
`FIG. 8 is a flow diagram for forming the DOMA, under
`an embodiment.
`FIG.9 is a plot of linear response of virtual microphone
`V, to a1 kHz speech source at a distance of 0.1 m, under an
`embodiment. The null is at 0 degrees, where the speech is
`normally located.
`FIG. 10 is a plot of linear response of virtual microphone
`V, toa 1 kHz noise source at a distance of 1.0 m, under an
`embodiment. There is no null and all noise sources are
`detected.
`FIG. 11is a plot of linear response of virtual microphone
`V, toa 1 kHz speech source at a distance of 0.1 m, under an
`embodiment. There is no null and the response for speech is
`greater than that shown in FIG.9.
`FIG. 12 is a plot of linear response of virtual microphone
`V, toa 1 kHz noise source at a distance of 1.0 m, under an
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 22 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 22 of 38
`
`US 11,122,357 B2
`
`3
`4
`In accordance with another embodiment, a microphone
`embodiment. There is no null and the response is very
`array is formed withafirst virtual microphone formed from
`similar to V, shown in FIG. 10.
`a first combination of a first microphone signal and a second
`FIG.13 is a plot of linear response of virtual microphone
`microphone signal, wherein the first microphone signal is
`V, to a speech source at a distance of 0.1 m for frequencies
`generated by a first omnidirectional microphone and the
`of 100, 500, 1000, 2000, 3000, and 4000 Hz, under an
`embodiment.
`second microphonesignal is generated by a second omni-
`directional microphone; and a second virtual microphone
`formed from a second combination ofthe first microphone
`signal and the second microphone signal, wherein the sec-
`ond combination is different from the first combination. The
`
`FIG. 14 is a plot showing comparison of frequency
`responses for speech for the array of an embodimentand for
`a conventional cardioid microphone.
`FIG. 15 is a plot showing speech response for V, (top,
`dashed) and V,, (bottom, solid) versus B with d, assumed to
`be 0.1 m, under an embodiment. The spatial null in V, is
`relatively broad.
`FIG. 16 is a plot showing a ratio of V,/V, speech
`responses shownin FIG. 10 versus B, under an embodiment.
`Theratio is above 10 dB for all 0.8<B<1.1. This means that
`
`first virtual microphonehasa first linear response to speech
`that has a single null oriented in a direction toward a source
`of the speech, wherein the speech is human speech.
`In accordance with another embodiment,
`a device
`includes a first microphone outputting a first microphone
`signal and a second microphoneoutputting a second micro-
`phone signal; and a processing component coupled to the
`first microphone signal and the second microphonesignal,
`the processing component generating a virtual microphone
`array comprising a first virtual microphone and a second
`virtual microphone, wherein the first virtual microphone
`comprises a first combination of the first microphonesignal
`and the second microphone signal, and wherein the second
`virtual microphone comprises a second combination of the
`first microphone signal and the second microphonesignal.
`The second virtual microphone have substantially similar
`responses to noise and substantially dissimilar responses to
`speech.
`a devise
`In accordance with another embodiment,
`includes a first microphone outputting a first microphone
`signal and a second microphoneoutputting a second micro-
`phonesignal, wherein the first microphone and the second
`microphone are omnidirectional microphones; and a virtual
`microphonearray comprising a first virtual microphone and
`a secondvirtual microphone, whereinthefirst virtual micro-
`phone comprises a first combination ofthe first microphone
`signal and the second microphonesignal, and the second
`virtual microphone comprises a second combination of the
`first microphone signal and the second microphonesignal.
`The second combination is different from the first combi-
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`the physical 6 of the system need not be exactly modeled for
`good performance.
`FIG. 17 is a plot of B versus actual d, assuming that d,=10
`cm and theta=0, under an embodiment.
`FIG. 18 is a plot of B versus theta with d=10 cm and
`assuming d=10 cm, under an embodiment.
`FIG. 19 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=1 and D=-7.2 usec, under an
`embodiment. The resulting phase difference clearly affects
`high frequencies more than low.
`FIG. 20 is a plot of amplitude (top) and phase (bottom)
`response of N(s) with B=1.2 and D=-7.2 usec, under an
`embodiment. Non-unity B affects the entire frequency range.
`FIG. 21 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V, due
`to a mistake in the location of the speech source with q1=0
`degrees and q2=30 degrees, under an embodiment. The
`cancellation remains below -10 dB for frequencies below 6
`kHz.
`
`FIG. 22 is a plot of amplitude (top) and phase (bottom)
`response of the effect on the speech cancellation in V, due
`to a mistake in the location of the speech source with q1=0
`degrees and q2=45 degrees, under an embodiment. The
`cancellation is below -10 dB only for frequencies below
`about 2.8 kHz and a reduction in performance is expected.
`FIG. 23 shows experimental results for a 2d,=19 mm
`array using a linear B of 0.83 on a Bruel and Kjaer Head and
`Torso Simulator (HATS) in very loud (~85 dBA) music/
`speech noise environment, under an embodiment. The noise
`has been reduced by about 25 dB and the speech hardly
`affected, with no noticeable distortion.
`
`SUMMARY OF THE INVENTION
`
`microphone and the second virtual microphoneare distinct
`virtual directional microphones with substantially similar
`responses to noise and substantially dissimilar responses to
`speech.
`
`nation, and the first virtual microphone and the second
`virtual microphone are distinct virtual directional micro-
`phones.
`a device
`In accordance with another embodiment,
`includesa first physical microphonegenerating a first micro-
`phone signal; a second physical microphone generating a
`second microphone signal; and a processing component
`coupled to the first microphone signal and the second
`microphonesignal, the processing component generating a
`virtual microphone array comprising a first virtual micro-
`phone and a second virtual microphone. The first virtual
`microphone comprises the second microphonesignal sub-
`The present invention provides for dual omnidirectional
`tracted from a delayed version ofthe first microphonesignal,
`microphone array devices systems and methods.
`and the second virtual microphone comprises a delayed
`In accordance with on embodiment, a microphonearrayis
`version of the first microphone signal subtracted from the
`formed with a first virtual microphone that includesa first
`second microphonesignal.
`combination of a first microphone signal and a second
`sensor
`a_
`In accordance with another embodiment,
`microphone signal, wherein the first microphone signal is
`
`generated by a first physical microphone and the second includes a physical microphonearray includingafirst physi-
`microphonesignal is generated by a second physical micro-
`cal microphone and a second physical microphone, the first
`phone; and a second virtual microphone that includes a
`physical microphone outputting a first microphone signal
`second combination of the first microphone signal and the
`and the second physical microphone outputting a second
`second microphonesignal, wherein the second combination
`microphonesignal; and a virtual microphone array compris-
`is different from the first combination. The first virtual
`ing a first virtual microphone and a secondvirtual micro-
`phone,the first curtail microphone comprising a first com-
`bination of the first microphone signal and the second
`microphonesignal, the second virtual microphone compris-
`ing a second combinationofthefirst microphone signal and
`
`

`

`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 23 of 38
`Case 6:21-cv-00984-ADA Document 19-7 Filed 12/23/21 Page 23 of 38
`
`US 11,122,357 B2
`
`5
`the second microphonesignal. The second combination is
`different from the first combination, and the virtual micro-
`phone array includes a single null oriented in a direction
`toward a source of speech of a human speaker.
`
`DETAILED DESCRIPTION
`
`A dual omnidirectional microphone array (DOMA)that
`provides improved noise suppression is described herein.
`Compared to conventional arrays and algorithms, which
`seek to reduce noise by nulling out noise sources, the array
`of an embodiment
`is used to form two distinct virtual
`
`directional microphones which are configured to have very
`similar noise
`responses
`and very dissimilar
`speech
`responses. The only null formed by the DOMAis one used
`to remove the speech of the user from V,. The two virtual
`microphones of an embodiment can be paired with an
`adaptive filter algorithm and/or VAD algorithm to signifi-
`cantly reduce the noise without distorting the speech, sig-
`nificantly improving the SNR of the desired speech over
`conventional noise suppression systems. The embodiments
`described herein are stable in operation,flexible with respect
`to virtual microphonepattern choice, and have proven to be
`robust with respect to speech source-to-array distance and
`orientation as well as temperature and calibration tech-
`niques.
`In the following description, numerous specific
`details are introduced to provide a thorough understanding
`of, and enabling description for, embodiments of the
`DOMA. One skilled in the relevant art, however, will
`recognize that these embodiments can be practiced without
`one or more of the specific details, or with other compo-
`nents, systems, etc. In other instances, well-known struc-
`tures or operations are not shown, or are not described in
`detail, to avoid obscuring aspects of the disclosed embodi-
`ments.
`
`Unless otherwise specified, the following terms have the
`corresponding meanings in addition to any meaning or
`understanding they may convey to one skilled in theart.
`The term “bleedthrough” meansthe undesired presence of
`noise during speech.
`The term “denoising” means removing unwanted noise
`from Micl, and also refers to the amount of reduction of
`noise energy in a signal in decibels (dB).
`The term “devoicing” means removing/distorting the
`desired speech from Mic.
`The term “directional microphone (DM)” meansa physi-
`cal directional microphonethat is vented on both sides of the
`sensing diaphragm.
`The term “Mic1 (M1)” meansa general designation for an
`adaptive noise suppression system microphonethat usually
`contains more speech than noise.
`The term “Mic2 (M2)” meansa general designation for an
`adaptive noise suppression system microphonethat usually
`contains more noise than speech.
`The term “noise” means unwanted environmental acous-
`tic noise.
`
`The term “null” means a zero or minima in the spatial
`response of a physical or virtual directional microphone.
`The term “O,” means a first physical omnidirectional
`microphone used to form a microphonearray.
`The term “O,” means a second physical omnidirectional
`microphone used to form a microphonearray.
`The term “speech” means desired speech of the user.
`The term “Skin Surface Microphone (SSM)”is a micro-
`phone used in an earpiece (e.g.,
`the Jawbone earpiece
`available from Aliph of San Francisco, Calif.) to detect
`speech vibrations on the user’s skin.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`The term “V,” means the virtual directional “speech”
`microphone, which has no nulls.
`The term “V,” means the virtual directional” noise”
`microphone, which has a null for the user’s speech.
`The term “Voice Activity Detection (VAD