UNITED STATES PATENT AND TRADEMARK OFFICE
`__________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________________________
`
`AMAZON.COM, INC.,
`Petitioner,
`
`v.
`
`JAWBONE INNOVATIONS, LLC,
`Patent Owner.
`
`IPR2023-00251
`U.S. Patent No. 11,122,357
`
`DECLARATION OF RICHARD M. STERN, Ph.D.
`IN SUPPORT OF PETITION FOR INTER PARTES REVIEW
`OF CLAIMS 1-20 OF U.S. PATENT NO. 11,122,357
`
`Meta Platforms, Inc. Exhibit 1002
`Page - Cover
`
`
`__________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________________________
`
`AMAZON.COM, INC.,
`Petitioner,
`
`v.
`
`JAWBONE INNOVATIONS, LLC,
`Patent Owner.
`
`IPR2023-00251
`U.S. Patent No. 11,122,357
`
`DECLARATION OF RICHARD M. STERN, Ph.D.
`IN SUPPORT OF PETITION FOR INTER PARTES REVIEW
`OF CLAIMS 1-20 OF U.S. PATENT NO. 11,122,357
`
`Meta Platforms, Inc. Exhibit 1002
`Page - Cover
`
`
`
`TABLE OF CONTENTS
`
`BACKGROUND ---------------------------------------------------------------------- 1
`
`Experience and Qualifications ----------------------------------------------- 1
`
`Materials Considered ---------------------------------------------------------- 2
`
`APPLICABLE LEGAL STANDARDS ------------------------------------------- 4
`
`Claim Construction ------------------------------------------------------------ 4
`
`Obviousness -------------------------------------------------------------------- 6
`
`PERSON OF ORDINARY SKILL IN THE ART ------------------------------- 9
`
`TECHNOLOGY BACKGROUND AND OVERVIEW OF
`PRIOR ART --------------------------------------------------------------------------- 11
`
`[1982] Griffiths-Jim: The GSC / Griffiths-Jim
`Beamformer ------------------------------------------------------------------- 11
`
`[1982-c.2001] Use of GSC with Microphone Arrays for
`Speech Applications ---------------------------------------------------------- 15
`
`[2001] Brandstein ------------------------------------------------------------- 16
`
`[2001] Gannot ----------------------------------------------------------------- 21
`
`[2001-2002] Brandstein and McCowan Discuss Near-
`Field Design ------------------------------------------------------------------- 24
`
`[2003] Burnett ’848 ---------------------------------------------------------- 30
`
`THE ’357 PATENT ------------------------------------------------------------------ 33
`
`CLAIMS 1-20 OF THE ’357 PATENT WOULD HAVE BEEN
`OBVIOUS ----------------------------------------------------------------------------- 37
`
`Table of Contents, Page 1
`
`Meta Platforms, Inc. Exhibit 1002
`Page i of 90
`
`
`
`TABLE OF AUTHORITIES
`(Cont’d)
`
`Claims 1-20 Would Have Been Obvious in View of
`Brandstein and Gannot ------------------------------------------------------- 37
`
`Claim 1 ----------------------------------------------------------------- 37
`
`Preamble -------------------------------------------------------- 37
`
`1[a]: First Virtual Microphone Comprising a
`Combination of Signals from First and
`Second Physical Microphones ------------------------------ 38
`
`1[b][1]: Second Virtual Microphone ----------------------- 41
`
`1[b][2]: Substantially Similar Responses to
`Noise and Substantially Dissimilar
`Responses to Speech ----------------------------------------- 42
`
`1[c][1]: Signal Processor Operative to
`Combine Microphone Signals by Filtering
`and Summing in the Time Domain ------------------------ 45
`
`1[c][2]: Applying a Varying Linear Transfer
`Function -------------------------------------------------------- 48
`
`1[c][3]: Generating an Output Signal with
`Reduced Noise ------------------------------------------------ 50
`
`Further Comments on Claim 1 in View of
`Brandstein and Gannot --------------------------------------- 51
`
`Claim 2 ----------------------------------------------------------------- 52
`
`Claim 3 ----------------------------------------------------------------- 53
`
`Claim 4 ----------------------------------------------------------------- 54
`
`Claim 5 ----------------------------------------------------------------- 56
`
`Table of Contents, Page 2
`
`Meta Platforms, Inc. Exhibit 1002
`Page ii of 90
`
`
`
`TABLE OF AUTHORITIES
`(Cont’d)
`
`Claim 6 ----------------------------------------------------------------- 64
`
`Claim 7 ----------------------------------------------------------------- 67
`
`Claim 8 ----------------------------------------------------------------- 67
`
`Claim 9 ----------------------------------------------------------------- 68
`
`Claim 10 ---------------------------------------------------------------- 69
`
`Claim 11 ---------------------------------------------------------------- 70
`
`Claim 12 ---------------------------------------------------------------- 72
`
`Claim 13 ---------------------------------------------------------------- 73
`
`Claim 14 ---------------------------------------------------------------- 74
`
`Claim 15 ---------------------------------------------------------------- 76
`
`Preamble -------------------------------------------------------- 76
`
`15[a]: First Virtual Microphone ---------------------------- 76
`
`15[b][2]: Second Virtual Microphone --------------------- 76
`
`15[b][2]: Substantially Similar Responses to
`Noise and Substantially Dissimilar
`Responses to Speech ----------------------------------------- 77
`
`15[c]: Virtual Microphone Array with a
`Single Null ----------------------------------------------------- 77
`
`15[d][1]: Signal Processor ----------------------------------- 78
`
`15[d][2]: Applying a Varying Linear
`Transfer Function --------------------------------------------- 79
`
`Table of Contents, Page 3
`
`Meta Platforms, Inc. Exhibit 1002
`Page iii of 90
`
`
`
`TABLE OF AUTHORITIES
`(Cont’d)
`
`15[d][3]: Generating an Output Signal with
`Reduced Noise ------------------------------------------------ 79
`
`Further Comments on Claim 15 in View of
`Brandstein and Gannot --------------------------------------- 79
`
`Claim 16 ---------------------------------------------------------------- 80
`
`Claim 17 ---------------------------------------------------------------- 80
`
`Claim 18 ---------------------------------------------------------------- 82
`
`Claim 19 ---------------------------------------------------------------- 82
`
`Claim 20 ---------------------------------------------------------------- 82
`
`Claims 4, 13, 14, 18, and 19 Would Have Been Obvious
`in View of Brandstein, Gannot, and Griffiths-Jim ---------------------- 83
`
`Claims 4-6, 13, 14, 18, and 19 Would Have Been
`Obvious in View of Brandstein, Gannot, and McCowan -------------- 86
`
`CONCLUSION ----------------------------------------------------------------------- 88
`
`Table of Contents, Page 4
`
`Meta Platforms, Inc. Exhibit 1002
`Page iv of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`1.
`I am making this declaration at the request of Petitioner Amazon.com,
`
`Inc. (“Amazon”). I have been retained by Amazon as a technical expert in this
`
`matter.
`
`2.
`
`I am being compensated for my work on this case. My compensation
`
`does not depend on the content of this Declaration or the outcome of these
`
`proceedings. I do not own any stock in Amazon and, to my knowledge, I have no
`
`financial interest in Amazon.
`
`BACKGROUND
`
`Experience and Qualifications
`
`3.
`
`I am a Professor at Carnegie Mellon University (CMU) in the
`
`Department of Electrical and Computer Engineering, the Department of Computer
`
`Science, and the Language Technologies Institute. I have been on the faculty of
`
`Carnegie Mellon since 1977.
`
`4.
`
`I received the S.B. degree from the Massachusetts Institute of
`
`Technology (MIT) in 1970, the M.S. from the University of California, Berkeley, in
`
`1972, and the Ph.D. from MIT in 1977, all in electrical engineering.
`
`5.
`
`I am a fellow of the Institute of Electrical and Electronics Engineers
`
`(IEEE), the Acoustical Society of America, and the International Speech
`
`Communication Association (ISCA). I was the ISCA 2008-2009 Distinguished
`
`Lecturer, a recipient of the Allen Newell Award for Research Excellence in 1992,
`
`-1-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 1 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`and I served as the General Chair of Interspeech 2006. Interspeech is the world’s
`
`largest technical conference focused on speech processing and application.
`
`6. Much of my current research is in spoken language systems, where I
`
`am particularly concerned with the development of techniques with which automatic
`
`speech recognition can be made more robust with respect to changes in environment
`
`and acoustical ambience.
`
`7.
`
`I have actively worked on the theory and application of systems using
`
`microphone arrays over a period of decades (e.g., Stern et al., 2008; Stern and
`
`Menon, 2020), and my research group has developed several array-based algorithms
`
`to improve speech recognition accuracy in difficult acoustical environments (e.g.,
`
`Seltzer et al., 2004; Stern et al., 2007; Kim et al., 2009; Moghimi and Stern, 2019).
`
`My relevant publications, including those cited above, are available on Carnegie
`
`Mellon’s web site at http://www.cs.cmu.edu/afs/cs/user/robust/www/papers.html.
`
`8.
`
`I understand a copy of my current curriculum vitae, which lists my
`
`publications for the last ten years, is being submitted as Exhibit 1012.
`
` Materials Considered
`
`9.
`
`In preparing this declaration, I have considered the following materials:
`
`Exhibit No.
`
`Description
`
`1001
`
`U.S. Patent No. 11,122,357 (“the ’357 patent”)
`
`-2-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 2 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`Exhibit No.
`
`Description
`
`1003
`
`1004
`
`1005
`
`1006
`
`1007
`
`1008
`
`1009
`
`1010
`
`1011
`
`1012
`
`Excerpts of MICROPHONE ARRAYS: SIGNAL PROCESSING
`TECHNIQUES AND APPLICATIONS (Michael Brandstein & Darren
`Ward eds., Springer-Verlag 2001) (“Brandstein”)
`
`Sharon Gannot et al., Signal Enhancement Using Beamforming
`and Nonstationarity with Applications to Speech, vol. 49, no. 8
`IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1614 (Aug. 2001)
`(“Gannot”)
`
`Lloyd Griffiths & Charles Jim, An Alternative Approach to
`Linearly Constrained Adaptive Beamforming, vol. AP-30, no. 1
`IEEE TRANSACTIONS ON ANTENNAS AND PROPAGATION, 27 (Jan.
`1982) (“Griffiths-Jim”)
`
`Iain A. McCowan et al., Near-Field Adaptive Beamformer for
`Robust Speech Recognition, vol. 12, no. 1 DIGITAL SIGNAL
`PROCESSING, 87 (Jan. 2002) (“McCowan”)
`
`U.S. Patent No. 5,651,071 (“Lindemann”)
`
`U.S. Patent No. 5,627,799 (“Hoshuyama”)
`
`U.S. Patent Publication No. 2003/0128848 (“Burnett ’848”)
`
`Excerpts from the ’357 patent’s file history
`
`U.S. Provisional Patent Application No. 61/045,377
`
`Curriculum Vitae of Richard M. Stern, Ph.D.
`
`
`
`10.
`
`I have also relied on my education, training, and experience, and my
`
`knowledge of pertinent literature in the field of the ’357 patent.
`
`-3-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 3 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
` APPLICABLE LEGAL STANDARDS
`
`11.
`
`I have been asked to provide my opinion as to whether the claims of the
`
`’357 patent would have been obvious to a person of ordinary skill in the art at the
`
`time of the alleged invention, in view of the prior art.
`
`12.
`
`I am an electrical engineer by training and profession. The opinions I
`
`am expressing in this report involve the application of my training and technical
`
`knowledge and experience to the evaluation of certain prior art with respect to the
`
`’357 patent.
`
`13. Although I have been involved as a technical expert in patent matters
`
`before, I am not an expert in patent law. Therefore, attorneys from Knobbe, Martens,
`
`Olson & Bear have provided me with guidance as to the applicable patent law in this
`
`matter. The paragraphs below express my understanding of how I must apply
`
`current principles related to patent validity to my analysis.
`
` Claim Construction
`
`14.
`
`It is my understanding that in determining whether a patent claim is
`
`anticipated by or obvious in view of the prior art, the Patent and Trademark Office
`
`construes the claim by giving the claim terms their plain and ordinary meaning, as
`
`they would have been understood by a person of ordinary skill in the art at the time
`
`of the invention in view of the intrinsic record (patent claims, patent specification,
`
`and file history). For the purposes of this review, and to the extent necessary, I have
`
`-4-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 4 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`interpreted each claim term in accordance with its plain and ordinary meaning as it
`
`would have been understood by a person of ordinary skill in the art at the time of the
`
`invention, in view of the intrinsic record. I have been instructed to assume that the
`
`time of the invention is June 13, 2007, which I understand to be the filing date of the
`
`earliest provisional application to which the ’357 patent claims priority.
`
`15.
`
`I understand that a patent and its prosecution history are considered
`
`“intrinsic evidence” and are the most important sources for interpreting claim
`
`language in a patent. I also understand that in reading the claim, I must not import
`
`limitations from the specification into the claim terms; in other words, I must not
`
`narrow the scope of the claim terms by implicitly adding disclosed limitations that
`
`have no express basis in the claims. The prosecution history of related patents and
`
`applications can also be relevant.
`
`16.
`
`I understand that sources extrinsic to a patent and its prosecution history
`
`(such as dictionary definitions and technical publications) may also be used to help
`
`interpret the claim language, but that such extrinsic sources cannot be used to
`
`contradict the unambiguous meaning of the claim language that is evident from the
`
`intrinsic evidence.
`
`17. Unless expressly stated herein, I have applied the plain and ordinary
`
`meaning of the claim terms, which I understand is the meaning that a person of
`
`-5-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 5 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`ordinary skill in the art would have given to terms in June 2007 based on a review
`
`of the intrinsic evidence.
`
`Obviousness
`
`18.
`
`It is my understanding that a claim is “obvious” if the claimed subject
`
`matter as a whole would have been obvious to a person of ordinary skill in the art at
`
`the time of the alleged invention. I understand that an obviousness analysis involves
`
`a number of considerations. I understand that the following factors must be
`
`evaluated to determine whether a claim would have been obvious: (i) the scope and
`
`content of the prior art; (ii) the differences, if any, between each claim of the ’357
`
`patent and the prior art; (iii) the level of ordinary skill in the art in June 2007; and
`
`(iv) additional considerations, if any, that indicate that the invention was obvious or
`
`not obvious. I understand that these “additional considerations” are often referred
`
`to as “secondary considerations” or “objective indicia” of non-obviousness or
`
`obviousness.
`
`19.
`
`I also understand that the frame of reference when evaluating
`
`obviousness is what a hypothetical person of ordinary skill in the pertinent art would
`
`have known in June 2007. I understand that the hypothetical person of ordinary skill
`
`is presumed to have knowledge of all pertinent prior art references.
`
`20.
`
`I understand that a prior art reference may be a pertinent prior art
`
`reference (or “analogous art”) if it is in the same field of endeavor as the patent or if
`
`-6-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 6 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`it is pertinent to the problem that the inventors were trying to solve. I understand
`
`that a reference is reasonably pertinent if it logically would have commended itself
`
`to an inventor’s attention in considering the problem at hand. If a reference relates
`
`to the same problem as the claimed invention, I understand that supports use of the
`
`reference as prior art in an obviousness analysis. Here, all of the references relied
`
`on in my obviousness analysis below are in the same field of endeavor as the ’357
`
`patent, e.g., microphone array signal processing and noise reduction. The references
`
`are also pertinent to a particular problem the inventor was focused on, e.g., noise
`
`suppression.
`
`21.
`
`It is my understanding that the law recognizes several rationales for
`
`combining references or modifying a reference to show obviousness of claimed
`
`subject matter. Some of these rationales include:
`
`• combining prior art elements according to known methods to yield
`
`predictable results;
`
`• simple substitution of one known element for another to obtain
`
`predictable results;
`
`• a predictable use of prior art elements according to their established
`
`functions;
`
`• using known techniques to improve similar devices (methods, or
`
`products) in the same way;
`
`-7-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 7 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`• applying a known technique to a known device (method, or product)
`
`ready for improvement to yield predictable results;
`
`• choosing from a finite number of identified, predictable solutions, with
`
`a reasonable expectation of success (in which case a claim would have
`
`been obvious to try);
`
`• known work in one field of endeavor may prompt variations of it for
`
`use in either the same field or a different one based on design incentives
`
`or other market forces if the variations would have been predictable to
`
`one of ordinary skill in the art; and
`
`• some teaching, suggestion, or motivation in the prior art that would
`
`have led one of ordinary skill to modify the prior art reference or to
`
`combine prior art reference teachings to arrive at the claimed invention.
`
`22.
`
`I understand that “secondary considerations” must be considered as part
`
`of the obviousness analysis when present. I further understand that the secondary
`
`considerations may include: (1) a long-felt but unmet need in the prior art that was
`
`satisfied by the claimed invention; (2) the failure of others; (3) skepticism by experts;
`
`(4) commercial success of a product covered by the patent; (5) unexpected results
`
`achieved by the claimed invention; (6) industry praise of the claimed invention; (7)
`
`deliberate copying of the invention; and (8) teaching away by others. I also
`
`understand that evidence of the independent and nearly simultaneous “invention” of
`
`-8-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 8 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`the claimed subject matter by others is a secondary consideration supporting an
`
`obviousness determination and may support a conclusion that a claimed invention
`
`was within the knowledge of a person of ordinary skill as of June 2007. I am not
`
`aware of any evidence of secondary considerations that would suggest that the
`
`claims of the ’357 patent would have been nonobvious in June 2007.
`
`23.
`
`I understand that when assessing obviousness, using hindsight is
`
`impermissible; that is, what is known today or what was learned from the teachings
`
`of the patent should not be considered. The patent should not be used as a road map
`
`for selecting and combining items of prior art. Rather, obviousness must be
`
`considered from the perspective of a person of ordinary skill at the time the alleged
`
`invention was made—which in this case, I have been asked to assume is June 2007.
`
`24.
`
`I also understand that an obviousness analysis must consider the
`
`invention as a whole, as opposed to just a part or element of the invention. I
`
`understand this “as a whole” assessment to require showing that one of ordinary skill
`
`in the art at the time of invention, confronted by the same problems as the inventor
`
`and with no knowledge of the claimed invention, would have selected the elements
`
`from the prior art and combined them in the claimed manner.
`
` PERSON OF ORDINARY SKILL IN THE ART
`
`25.
`
`It is my understanding that when interpreting the claims of the ’357
`
`patent and evaluating whether a claim would have been obvious, I must do so based
`
`-9-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 9 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`on the perspective of a person of ordinary skill in the art at the time of invention. I
`
`understand that the earliest claimed priority date of the ’357 patent is June 13, 2007,
`
`and I have been instructed to assume for the purposes of my opinions that this is the
`
`invention date for the ’357 patent.
`
`26.
`
`I understand that in determining the level of ordinary skill in the art,
`
`several factors are considered. Those factors may include: (i) the type of problems
`
`encountered in the art; (ii) prior art solutions to those problems; (iii) the rapidity with
`
`which innovations are made; (iv) the sophistication of the technology; and (v) the
`
`educational level of active workers in the field. A person of ordinary skill in the art
`
`must have the capability of understanding the scientific and engineering principles
`
`applicable to the pertinent art.
`
`27. Based on my review of the specification and claims of the ’357 patent,
`
`it is my opinion that a person of ordinary skill in the art would have had a minimum
`
`of a bachelor’s degree in computer engineering, computer science, electrical
`
`engineering, mechanical engineering, or a similar field, and approximately three
`
`years of industry or academic experience in a field related to acoustics, speech
`
`recognition, speech detection, or signal processing. Work experience could
`
`substitute for formal education, and additional formal education could substitute for
`
`work experience.
`
`-10-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 10 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`28. My conclusions below that the claims of the ’357 patent would have
`
`been obvious would remain the same even if the invention date, field of endeavor,
`
`or level of ordinary skill were slightly different.
`
`29.
`
`I meet the above definition of a person of ordinary skill in the art, and
`
`did so as of June 2007. Also, I have worked with persons of ordinary skill in the art
`
`through my professional and academic experiences, and I have an understanding of
`
`their skill level around June 2007.
`
`TECHNOLOGY BACKGROUND AND OVERVIEW OF
`PRIOR ART
`
`[1982] Griffiths-Jim: The GSC / Griffiths-Jim
`Beamformer
`
`30. One noise-reduction technique known for decades before 2007 was the
`
`Generalized Sidelobe Canceler, or GSC. The GSC is sometimes also called the
`
`Griffiths-Jim beamformer.
`
`31.
`
`The GSC was described in a paper by Lloyd Griffiths and Charles Jim
`
`titled An Alternative Approach to Linearly Constrained Adaptive Beamforming,
`
`included in the January 1982 volume of IEEE Transactions on Antennas and
`
`Propagation. I will refer to this paper as “Griffiths-Jim” and I understand it is
`
`Exhibit 1005. I have been teaching the Griffiths-Jim algorithm for decades
`
`(including well before 2006) in my CMU course 18-792 Advanced Digital Signal
`
`-11-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 11 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`Processing, working from the description in the standard text Adaptive Signal
`
`Processing by Widrow and Stearns, which was published in 1985.
`
`32. Griffiths-Jim described “a simple time-varying beamformer which can
`
`be used to combine the outputs of an array of sensors.” (Ex. 1005, 27.) “The purpose
`
`of the beamformer is to minimize the effects of noise and interference at the array
`
`output” while capturing the target signal. (Id., 27.) Griffiths-Jim called the
`
`beamformer a “generalized sidelobe canceling” structure or model. (Id., 29.)
`
`33. Figure 4 of Griffiths-Jim shows the GSC (highlighting and annotations
`
`added):
`
`(Id., 30 (annotated).)
`
`
`
`-12-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 12 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`34. As shown in Figure 4, the signal processor had two main substructures:
`
`the top branch was a “conventional beamformer” designed to capture the target
`
`signal plus noise, and the bottom branch was the “sidelobe canceling path” that
`
`would be used to capture only noise so that the noise could be subtracted or canceled
`
`out.
`
`35.
`
`I will refer to the top branch as the first virtual sensor. The first virtual
`
`sensor was formed by combining the outputs of the array sensors. The sensor
`
`outputs were combined by multiplying the sensor output signals by “weights”
`
`𝑤𝑤𝑐𝑐1,…,𝑤𝑤𝑐𝑐𝑐𝑐 (sometimes also called “gains”) and summing the weighted sensor
`signals to produce a “nonadaptive-beamformed signal 𝑦𝑦𝑐𝑐(𝑘𝑘).” (Id., 29.) (The
`variable 𝑘𝑘 in Griffiths-Jim’s equations is time expressed in samples. (E.g., id., 27
`(“We denote the sampled output of the 𝑚𝑚th time-delayed sensor by 𝑥𝑥𝑚𝑚(𝑘𝑘).”), 28
`(“We let 𝐀𝐀𝑙𝑙 and 𝐗𝐗(𝑘𝑘 − 𝑙𝑙) represent the filter coefficient and signal vectors at the
`𝑙𝑙th delay point . . . .”).) Griffiths-Jim’s discussion is thus in the time domain, which
`sensor (top branch) is 𝑦𝑦𝑐𝑐′(𝑘𝑘), which “is obtained by filtering 𝑦𝑦𝑐𝑐(𝑘𝑘) and the FIR
`operator containing . . . constraint values 𝑓𝑓(𝑙𝑙)” (id., 30):
`
`refers to analyzing signals as a function of time.) The output of the first virtual
`
`
`
`-13-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 13 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`(Id., 30.) This output 𝑦𝑦𝑐𝑐′(𝑘𝑘) contains the target signal plus noise.
`
`36.
`
`“The lower path in Fig. 4 is the sidelobe canceling path” (id., 30), which
`
`I will refer to as the second virtual sensor. The second virtual sensor is formed
`
`from a combination of the outputs of the array sensors that differs from the
`
`combination used for the first virtual sensor.
`
`37. The lower path includes a blocking matrix designed to “block the
`
`desired signal 𝑠𝑠(𝑘𝑘) from the lower path.” (Id., 30.) In Griffiths-Jim, the blocking
`matrix is denoted by 𝑊𝑊𝑠𝑠����. (Id., 30.)
`
`38.
`
`In the lower path, the array sensor outputs were combined by applying
`
`the blocking matrix and adaptive filter coefficients and summing the result:
`
`where
`
`
`
`
`
`(Id., 30.)
`
`39. The second virtual sensor’s output “𝑦𝑦𝐴𝐴(𝑘𝑘) contains no desired signal
`
`terms” and instead “contains only noise and interference terms.” (Id., 30.) In other
`
`words, the second virtual sensor is designed to capture noise only.
`
`-14-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 14 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`40. The overall output of the GSC, 𝑦𝑦(𝑘𝑘), was produced by subtracting the
`
`noise-only output of the second virtual sensor from the target-plus-noise output of
`
`the first virtual sensor:
`
`
`
`(Id., 30.) The result was a signal with reduced noise. The response of the processor
`
`to the desired signal 𝑠𝑠(𝑘𝑘)𝟏𝟏 is that produced only by 𝑦𝑦𝑐𝑐′(𝑘𝑘),” i.e., only by the first
`
`virtual sensor (top branch). (Id., 30.) The “output due to the presence of only the
`
`desired signal satisfies the constraint defined by (9)” (id., 30), which Griffiths-Jim
`
`explains is commonly made to be a constraint of “zero distortion” (id., 28).
`
`
`
`[1982-c.2001] Use of GSC with Microphone Arrays
`for Speech Applications
`
`41. After Griffiths and Jim’s GSC article, over the next two decades, the
`
`GSC was used in many signal-processing applications, including with microphone
`
`arrays to reduce noise in speech applications.
`
`42.
`
`I understand Exhibit 1007
`
`is U.S. Patent No. 5,651,071
`
`(“Lindemann”). Filed in 1993, Lindemann cites the Griffiths-Jim article and
`
`explains that using a Griffiths-Jim beamformer “to improve signal-to-noise ratio for
`
`hearing aids” was known. (Ex. 1007, 1:40-46, 12:12-14.1)
`
`
`1 Patent citations are in the form of column:line.
`
`-15-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 15 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`43.
`
`I understand Exhibit 1008
`
`is U.S. Patent No. 5,627,799
`
`(“Hoshuyama”). Filed in 1995, Hoshuyama “relates generally to interference
`
`cancelers, and more particularly to a generalized sidelobe canceler, or adaptive
`
`beamformer for an array of sensors such as microphones and the like.” (Ex. 1008,
`
`1:8-11.) Describing the prior art at the time, Hoshuyama states:
`
`According to a prior art microphone array, signals detected by an array
`of microphones are lowpass filtered and summed together to detect a
`target signal that arrives in a particular direction. The adaptive
`microphone array beamformer is one form of the generalized sidelobe
`canceler as described in an article “An alternative Approach to Linearly
`Constrained Adaptive Beamforming”, Lloyd J. Griffiths and Charles
`W. Jim, the IEEE Transactions on Antenna and Propagation, Vol. AP-
`30, No. 1, January 1982, pages 27-34.
`
`(Id., 1:17-26.)
`
`
`
`[2001] Brandstein
`
`44.
`
`In 2001, Springer-Verlag published the reference book MICROPHONE
`
`ARRAYS: SIGNAL PROCESSING TECHNIQUES AND APPLICATIONS (“Brandstein”),
`
`edited by Michael Brandstein of Harvard University and Darren Ward of Imperial
`
`College. In 2001, Springer-Verlag was already a well-established publishing house
`
`and a leading publisher of technical monographs and serial publications consulted
`
`by academics, scientists, engineers, and other practitioners across a wide spectrum
`
`of technical disciplines, including audio-signal processing. I purchased a copy of
`
`-16-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 16 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`Brandstein before the end of 2002, and Exhibit 1003 is a scanned copy of excerpts
`
`from my copy of Brandstein.
`
`45. Following its publication, Brandstein quickly became a standard
`
`reference for those working in the field of audio signal processing.
`
`46. Chapter 5 of Brandstein concerns robust adaptive beamforming, and at
`
`the beginning of that chapter Brandstein states: “Applications of beamforming
`
`include microphone arrays for speech enhancement. The goal of speech
`
`enhancement is to remove undesirable signals such as noise and reverberation.” (Ex.
`
`1003 (Brandstein), 87.) Brandstein further notes: “Among various adaptive
`
`beamformers, the Griffiths-Jim beamformer (GJBF), or the generalized sidelobe
`
`canceler, is most widely known.” (Id., 88 (internal citation omitted).)
`
`
`
`
`
`[continued on next page]
`
`-17-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 17 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`47. As Brandstein states, “Figure 5.1 depicts the structure of the GJBF.”
`
`(Id., 88.) Figure 5.1 appears below with highlighting and annotations:
`
`(Id., 89, Fig. 5.1 (annotated).)
`
`
`
`48. As illustrated in the top branch of Figure 5.1, the signals of at least two
`
`the signals to form a fixed beamformer that is a first virtual microphone. The first
`
`virtual microphone captures the target speech signal plus noise. As in the case of
`
`physical microphones, 𝑥𝑥0(𝑘𝑘) and 𝑥𝑥1(𝑘𝑘), are combined by filtering and summing
`Griffiths-Jim, the variable 𝑘𝑘 is time expressed in samples. Brandstein’s discussion
`
`is thus in the time domain.
`
`-18-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 18 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`49.
`
`In the bottom branch of Figure 5.1, the signals of the two physical
`
`microphones are combined by filtering and summing the signals in a different way
`
`to form a second virtual microphone. The second virtual microphone includes a
`
`blocking matrix (BM). “[T]he BM forms a null in the look direction so that the
`
`target signal is suppressed and all other signals are passed through.” (Id., 88.) “The
`
`BM was named after its function, which is to block the target signal.” (Id.) As a
`
`result, the second virtual microphone captures noise only.
`
`50. The output of the GSC (or GJBF) is the output of the top branch (first
`
`virtual microphone) minus the output of the bottom branch (second virtual
`
`microph
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
