`__________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________________________
`
`AMAZON.COM, INC.,
`Petitioner,
`
`v.
`
`JAWBONE INNOVATIONS, LLC,
`Patent Owner.
`
`IPR2023-00251
`U.S. Patent No. 11,122,357
`
`DECLARATION OF RICHARD M. STERN, Ph.D.
`IN SUPPORT OF PETITION FOR INTER PARTES REVIEW
`OF CLAIMS 1-20 OF U.S. PATENT NO. 11,122,357
`
`Meta Platforms, Inc. Exhibit 1002
`Page - Cover
`
`
`
`TABLE OF CONTENTS
`
`BACKGROUND ---------------------------------------------------------------------- 1
`
`Experience and Qualifications ----------------------------------------------- 1
`
`Materials Considered ---------------------------------------------------------- 2
`
`APPLICABLE LEGAL STANDARDS ------------------------------------------- 4
`
`Claim Construction ------------------------------------------------------------ 4
`
`Obviousness -------------------------------------------------------------------- 6
`
`PERSON OF ORDINARY SKILL IN THE ART ------------------------------- 9
`
`TECHNOLOGY BACKGROUND AND OVERVIEW OF
`PRIOR ART --------------------------------------------------------------------------- 11
`
`[1982] Griffiths-Jim: The GSC / Griffiths-Jim
`Beamformer ------------------------------------------------------------------- 11
`
`[1982-c.2001] Use of GSC with Microphone Arrays for
`Speech Applications ---------------------------------------------------------- 15
`
`[2001] Brandstein ------------------------------------------------------------- 16
`
`[2001] Gannot ----------------------------------------------------------------- 21
`
`[2001-2002] Brandstein and McCowan Discuss Near-
`Field Design ------------------------------------------------------------------- 24
`
`[2003] Burnett ’848 ---------------------------------------------------------- 30
`
`THE ’357 PATENT ------------------------------------------------------------------ 33
`
`CLAIMS 1-20 OF THE ’357 PATENT WOULD HAVE BEEN
`OBVIOUS ----------------------------------------------------------------------------- 37
`
`Table of Contents, Page 1
`
`Meta Platforms, Inc. Exhibit 1002
`Page i of 90
`
`
`
`TABLE OF AUTHORITIES
`(Cont’d)
`
`Claims 1-20 Would Have Been Obvious in View of
`Brandstein and Gannot ------------------------------------------------------- 37
`
`Claim 1 ----------------------------------------------------------------- 37
`
`Preamble -------------------------------------------------------- 37
`
`1[a]: First Virtual Microphone Comprising a
`Combination of Signals from First and
`Second Physical Microphones ------------------------------ 38
`
`1[b][1]: Second Virtual Microphone ----------------------- 41
`
`1[b][2]: Substantially Similar Responses to
`Noise and Substantially Dissimilar
`Responses to Speech ----------------------------------------- 42
`
`1[c][1]: Signal Processor Operative to
`Combine Microphone Signals by Filtering
`and Summing in the Time Domain ------------------------ 45
`
`1[c][2]: Applying a Varying Linear Transfer
`Function -------------------------------------------------------- 48
`
`1[c][3]: Generating an Output Signal with
`Reduced Noise ------------------------------------------------ 50
`
`Further Comments on Claim 1 in View of
`Brandstein and Gannot --------------------------------------- 51
`
`Claim 2 ----------------------------------------------------------------- 52
`
`Claim 3 ----------------------------------------------------------------- 53
`
`Claim 4 ----------------------------------------------------------------- 54
`
`Claim 5 ----------------------------------------------------------------- 56
`
`Table of Contents, Page 2
`
`Meta Platforms, Inc. Exhibit 1002
`Page ii of 90
`
`
`
`TABLE OF AUTHORITIES
`(Cont’d)
`
`Claim 6 ----------------------------------------------------------------- 64
`
`Claim 7 ----------------------------------------------------------------- 67
`
`Claim 8 ----------------------------------------------------------------- 67
`
`Claim 9 ----------------------------------------------------------------- 68
`
`Claim 10 ---------------------------------------------------------------- 69
`
`Claim 11 ---------------------------------------------------------------- 70
`
`Claim 12 ---------------------------------------------------------------- 72
`
`Claim 13 ---------------------------------------------------------------- 73
`
`Claim 14 ---------------------------------------------------------------- 74
`
`Claim 15 ---------------------------------------------------------------- 76
`
`Preamble -------------------------------------------------------- 76
`
`15[a]: First Virtual Microphone ---------------------------- 76
`
`15[b][2]: Second Virtual Microphone --------------------- 76
`
`15[b][2]: Substantially Similar Responses to
`Noise and Substantially Dissimilar
`Responses to Speech ----------------------------------------- 77
`
`15[c]: Virtual Microphone Array with a
`Single Null ----------------------------------------------------- 77
`
`15[d][1]: Signal Processor ----------------------------------- 78
`
`15[d][2]: Applying a Varying Linear
`Transfer Function --------------------------------------------- 79
`
`Table of Contents, Page 3
`
`Meta Platforms, Inc. Exhibit 1002
`Page iii of 90
`
`
`
`TABLE OF AUTHORITIES
`(Cont’d)
`
`15[d][3]: Generating an Output Signal with
`Reduced Noise ------------------------------------------------ 79
`
`Further Comments on Claim 15 in View of
`Brandstein and Gannot --------------------------------------- 79
`
`Claim 16 ---------------------------------------------------------------- 80
`
`Claim 17 ---------------------------------------------------------------- 80
`
`Claim 18 ---------------------------------------------------------------- 82
`
`Claim 19 ---------------------------------------------------------------- 82
`
`Claim 20 ---------------------------------------------------------------- 82
`
`Claims 4, 13, 14, 18, and 19 Would Have Been Obvious
`in View of Brandstein, Gannot, and Griffiths-Jim ---------------------- 83
`
`Claims 4-6, 13, 14, 18, and 19 Would Have Been
`Obvious in View of Brandstein, Gannot, and McCowan -------------- 86
`
`CONCLUSION ----------------------------------------------------------------------- 88
`
`Table of Contents, Page 4
`
`Meta Platforms, Inc. Exhibit 1002
`Page iv of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`1.
`I am making this declaration at the request of Petitioner Amazon.com,
`
`Inc. (“Amazon”). I have been retained by Amazon as a technical expert in this
`
`matter.
`
`2.
`
`I am being compensated for my work on this case. My compensation
`
`does not depend on the content of this Declaration or the outcome of these
`
`proceedings. I do not own any stock in Amazon and, to my knowledge, I have no
`
`financial interest in Amazon.
`
`BACKGROUND
`
`Experience and Qualifications
`
`3.
`
`I am a Professor at Carnegie Mellon University (CMU) in the
`
`Department of Electrical and Computer Engineering, the Department of Computer
`
`Science, and the Language Technologies Institute. I have been on the faculty of
`
`Carnegie Mellon since 1977.
`
`4.
`
`I received the S.B. degree from the Massachusetts Institute of
`
`Technology (MIT) in 1970, the M.S. from the University of California, Berkeley, in
`
`1972, and the Ph.D. from MIT in 1977, all in electrical engineering.
`
`5.
`
`I am a fellow of the Institute of Electrical and Electronics Engineers
`
`(IEEE), the Acoustical Society of America, and the International Speech
`
`Communication Association (ISCA). I was the ISCA 2008-2009 Distinguished
`
`Lecturer, a recipient of the Allen Newell Award for Research Excellence in 1992,
`
`-1-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 1 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`and I served as the General Chair of Interspeech 2006. Interspeech is the world’s
`
`largest technical conference focused on speech processing and application.
`
`6. Much of my current research is in spoken language systems, where I
`
`am particularly concerned with the development of techniques with which automatic
`
`speech recognition can be made more robust with respect to changes in environment
`
`and acoustical ambience.
`
`7.
`
`I have actively worked on the theory and application of systems using
`
`microphone arrays over a period of decades (e.g., Stern et al., 2008; Stern and
`
`Menon, 2020), and my research group has developed several array-based algorithms
`
`to improve speech recognition accuracy in difficult acoustical environments (e.g.,
`
`Seltzer et al., 2004; Stern et al., 2007; Kim et al., 2009; Moghimi and Stern, 2019).
`
`My relevant publications, including those cited above, are available on Carnegie
`
`Mellon’s web site at http://www.cs.cmu.edu/afs/cs/user/robust/www/papers.html.
`
`8.
`
`I understand a copy of my current curriculum vitae, which lists my
`
`publications for the last ten years, is being submitted as Exhibit 1012.
`
` Materials Considered
`
`9.
`
`In preparing this declaration, I have considered the following materials:
`
`Exhibit No.
`
`Description
`
`1001
`
`U.S. Patent No. 11,122,357 (“the ’357 patent”)
`
`-2-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 2 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`Exhibit No.
`
`Description
`
`1003
`
`1004
`
`1005
`
`1006
`
`1007
`
`1008
`
`1009
`
`1010
`
`1011
`
`1012
`
`Excerpts of MICROPHONE ARRAYS: SIGNAL PROCESSING
`TECHNIQUES AND APPLICATIONS (Michael Brandstein & Darren
`Ward eds., Springer-Verlag 2001) (“Brandstein”)
`
`Sharon Gannot et al., Signal Enhancement Using Beamforming
`and Nonstationarity with Applications to Speech, vol. 49, no. 8
`IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1614 (Aug. 2001)
`(“Gannot”)
`
`Lloyd Griffiths & Charles Jim, An Alternative Approach to
`Linearly Constrained Adaptive Beamforming, vol. AP-30, no. 1
`IEEE TRANSACTIONS ON ANTENNAS AND PROPAGATION, 27 (Jan.
`1982) (“Griffiths-Jim”)
`
`Iain A. McCowan et al., Near-Field Adaptive Beamformer for
`Robust Speech Recognition, vol. 12, no. 1 DIGITAL SIGNAL
`PROCESSING, 87 (Jan. 2002) (“McCowan”)
`
`U.S. Patent No. 5,651,071 (“Lindemann”)
`
`U.S. Patent No. 5,627,799 (“Hoshuyama”)
`
`U.S. Patent Publication No. 2003/0128848 (“Burnett ’848”)
`
`Excerpts from the ’357 patent’s file history
`
`U.S. Provisional Patent Application No. 61/045,377
`
`Curriculum Vitae of Richard M. Stern, Ph.D.
`
`
`
`10.
`
`I have also relied on my education, training, and experience, and my
`
`knowledge of pertinent literature in the field of the ’357 patent.
`
`-3-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 3 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
` APPLICABLE LEGAL STANDARDS
`
`11.
`
`I have been asked to provide my opinion as to whether the claims of the
`
`’357 patent would have been obvious to a person of ordinary skill in the art at the
`
`time of the alleged invention, in view of the prior art.
`
`12.
`
`I am an electrical engineer by training and profession. The opinions I
`
`am expressing in this report involve the application of my training and technical
`
`knowledge and experience to the evaluation of certain prior art with respect to the
`
`’357 patent.
`
`13. Although I have been involved as a technical expert in patent matters
`
`before, I am not an expert in patent law. Therefore, attorneys from Knobbe, Martens,
`
`Olson & Bear have provided me with guidance as to the applicable patent law in this
`
`matter. The paragraphs below express my understanding of how I must apply
`
`current principles related to patent validity to my analysis.
`
` Claim Construction
`
`14.
`
`It is my understanding that in determining whether a patent claim is
`
`anticipated by or obvious in view of the prior art, the Patent and Trademark Office
`
`construes the claim by giving the claim terms their plain and ordinary meaning, as
`
`they would have been understood by a person of ordinary skill in the art at the time
`
`of the invention in view of the intrinsic record (patent claims, patent specification,
`
`and file history). For the purposes of this review, and to the extent necessary, I have
`
`-4-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 4 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`interpreted each claim term in accordance with its plain and ordinary meaning as it
`
`would have been understood by a person of ordinary skill in the art at the time of the
`
`invention, in view of the intrinsic record. I have been instructed to assume that the
`
`time of the invention is June 13, 2007, which I understand to be the filing date of the
`
`earliest provisional application to which the ’357 patent claims priority.
`
`15.
`
`I understand that a patent and its prosecution history are considered
`
`“intrinsic evidence” and are the most important sources for interpreting claim
`
`language in a patent. I also understand that in reading the claim, I must not import
`
`limitations from the specification into the claim terms; in other words, I must not
`
`narrow the scope of the claim terms by implicitly adding disclosed limitations that
`
`have no express basis in the claims. The prosecution history of related patents and
`
`applications can also be relevant.
`
`16.
`
`I understand that sources extrinsic to a patent and its prosecution history
`
`(such as dictionary definitions and technical publications) may also be used to help
`
`interpret the claim language, but that such extrinsic sources cannot be used to
`
`contradict the unambiguous meaning of the claim language that is evident from the
`
`intrinsic evidence.
`
`17. Unless expressly stated herein, I have applied the plain and ordinary
`
`meaning of the claim terms, which I understand is the meaning that a person of
`
`-5-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 5 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`ordinary skill in the art would have given to terms in June 2007 based on a review
`
`of the intrinsic evidence.
`
`Obviousness
`
`18.
`
`It is my understanding that a claim is “obvious” if the claimed subject
`
`matter as a whole would have been obvious to a person of ordinary skill in the art at
`
`the time of the alleged invention. I understand that an obviousness analysis involves
`
`a number of considerations. I understand that the following factors must be
`
`evaluated to determine whether a claim would have been obvious: (i) the scope and
`
`content of the prior art; (ii) the differences, if any, between each claim of the ’357
`
`patent and the prior art; (iii) the level of ordinary skill in the art in June 2007; and
`
`(iv) additional considerations, if any, that indicate that the invention was obvious or
`
`not obvious. I understand that these “additional considerations” are often referred
`
`to as “secondary considerations” or “objective indicia” of non-obviousness or
`
`obviousness.
`
`19.
`
`I also understand that the frame of reference when evaluating
`
`obviousness is what a hypothetical person of ordinary skill in the pertinent art would
`
`have known in June 2007. I understand that the hypothetical person of ordinary skill
`
`is presumed to have knowledge of all pertinent prior art references.
`
`20.
`
`I understand that a prior art reference may be a pertinent prior art
`
`reference (or “analogous art”) if it is in the same field of endeavor as the patent or if
`
`-6-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 6 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`it is pertinent to the problem that the inventors were trying to solve. I understand
`
`that a reference is reasonably pertinent if it logically would have commended itself
`
`to an inventor’s attention in considering the problem at hand. If a reference relates
`
`to the same problem as the claimed invention, I understand that supports use of the
`
`reference as prior art in an obviousness analysis. Here, all of the references relied
`
`on in my obviousness analysis below are in the same field of endeavor as the ’357
`
`patent, e.g., microphone array signal processing and noise reduction. The references
`
`are also pertinent to a particular problem the inventor was focused on, e.g., noise
`
`suppression.
`
`21.
`
`It is my understanding that the law recognizes several rationales for
`
`combining references or modifying a reference to show obviousness of claimed
`
`subject matter. Some of these rationales include:
`
`• combining prior art elements according to known methods to yield
`
`predictable results;
`
`• simple substitution of one known element for another to obtain
`
`predictable results;
`
`• a predictable use of prior art elements according to their established
`
`functions;
`
`• using known techniques to improve similar devices (methods, or
`
`products) in the same way;
`
`-7-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 7 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`• applying a known technique to a known device (method, or product)
`
`ready for improvement to yield predictable results;
`
`• choosing from a finite number of identified, predictable solutions, with
`
`a reasonable expectation of success (in which case a claim would have
`
`been obvious to try);
`
`• known work in one field of endeavor may prompt variations of it for
`
`use in either the same field or a different one based on design incentives
`
`or other market forces if the variations would have been predictable to
`
`one of ordinary skill in the art; and
`
`• some teaching, suggestion, or motivation in the prior art that would
`
`have led one of ordinary skill to modify the prior art reference or to
`
`combine prior art reference teachings to arrive at the claimed invention.
`
`22.
`
`I understand that “secondary considerations” must be considered as part
`
`of the obviousness analysis when present. I further understand that the secondary
`
`considerations may include: (1) a long-felt but unmet need in the prior art that was
`
`satisfied by the claimed invention; (2) the failure of others; (3) skepticism by experts;
`
`(4) commercial success of a product covered by the patent; (5) unexpected results
`
`achieved by the claimed invention; (6) industry praise of the claimed invention; (7)
`
`deliberate copying of the invention; and (8) teaching away by others. I also
`
`understand that evidence of the independent and nearly simultaneous “invention” of
`
`-8-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 8 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`the claimed subject matter by others is a secondary consideration supporting an
`
`obviousness determination and may support a conclusion that a claimed invention
`
`was within the knowledge of a person of ordinary skill as of June 2007. I am not
`
`aware of any evidence of secondary considerations that would suggest that the
`
`claims of the ’357 patent would have been nonobvious in June 2007.
`
`23.
`
`I understand that when assessing obviousness, using hindsight is
`
`impermissible; that is, what is known today or what was learned from the teachings
`
`of the patent should not be considered. The patent should not be used as a road map
`
`for selecting and combining items of prior art. Rather, obviousness must be
`
`considered from the perspective of a person of ordinary skill at the time the alleged
`
`invention was made—which in this case, I have been asked to assume is June 2007.
`
`24.
`
`I also understand that an obviousness analysis must consider the
`
`invention as a whole, as opposed to just a part or element of the invention. I
`
`understand this “as a whole” assessment to require showing that one of ordinary skill
`
`in the art at the time of invention, confronted by the same problems as the inventor
`
`and with no knowledge of the claimed invention, would have selected the elements
`
`from the prior art and combined them in the claimed manner.
`
` PERSON OF ORDINARY SKILL IN THE ART
`
`25.
`
`It is my understanding that when interpreting the claims of the ’357
`
`patent and evaluating whether a claim would have been obvious, I must do so based
`
`-9-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 9 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`on the perspective of a person of ordinary skill in the art at the time of invention. I
`
`understand that the earliest claimed priority date of the ’357 patent is June 13, 2007,
`
`and I have been instructed to assume for the purposes of my opinions that this is the
`
`invention date for the ’357 patent.
`
`26.
`
`I understand that in determining the level of ordinary skill in the art,
`
`several factors are considered. Those factors may include: (i) the type of problems
`
`encountered in the art; (ii) prior art solutions to those problems; (iii) the rapidity with
`
`which innovations are made; (iv) the sophistication of the technology; and (v) the
`
`educational level of active workers in the field. A person of ordinary skill in the art
`
`must have the capability of understanding the scientific and engineering principles
`
`applicable to the pertinent art.
`
`27. Based on my review of the specification and claims of the ’357 patent,
`
`it is my opinion that a person of ordinary skill in the art would have had a minimum
`
`of a bachelor’s degree in computer engineering, computer science, electrical
`
`engineering, mechanical engineering, or a similar field, and approximately three
`
`years of industry or academic experience in a field related to acoustics, speech
`
`recognition, speech detection, or signal processing. Work experience could
`
`substitute for formal education, and additional formal education could substitute for
`
`work experience.
`
`-10-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 10 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`28. My conclusions below that the claims of the ’357 patent would have
`
`been obvious would remain the same even if the invention date, field of endeavor,
`
`or level of ordinary skill were slightly different.
`
`29.
`
`I meet the above definition of a person of ordinary skill in the art, and
`
`did so as of June 2007. Also, I have worked with persons of ordinary skill in the art
`
`through my professional and academic experiences, and I have an understanding of
`
`their skill level around June 2007.
`
`TECHNOLOGY BACKGROUND AND OVERVIEW OF
`PRIOR ART
`
`[1982] Griffiths-Jim: The GSC / Griffiths-Jim
`Beamformer
`
`30. One noise-reduction technique known for decades before 2007 was the
`
`Generalized Sidelobe Canceler, or GSC. The GSC is sometimes also called the
`
`Griffiths-Jim beamformer.
`
`31.
`
`The GSC was described in a paper by Lloyd Griffiths and Charles Jim
`
`titled An Alternative Approach to Linearly Constrained Adaptive Beamforming,
`
`included in the January 1982 volume of IEEE Transactions on Antennas and
`
`Propagation. I will refer to this paper as “Griffiths-Jim” and I understand it is
`
`Exhibit 1005. I have been teaching the Griffiths-Jim algorithm for decades
`
`(including well before 2006) in my CMU course 18-792 Advanced Digital Signal
`
`-11-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 11 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`Processing, working from the description in the standard text Adaptive Signal
`
`Processing by Widrow and Stearns, which was published in 1985.
`
`32. Griffiths-Jim described “a simple time-varying beamformer which can
`
`be used to combine the outputs of an array of sensors.” (Ex. 1005, 27.) “The purpose
`
`of the beamformer is to minimize the effects of noise and interference at the array
`
`output” while capturing the target signal. (Id., 27.) Griffiths-Jim called the
`
`beamformer a “generalized sidelobe canceling” structure or model. (Id., 29.)
`
`33. Figure 4 of Griffiths-Jim shows the GSC (highlighting and annotations
`
`added):
`
`(Id., 30 (annotated).)
`
`
`
`-12-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 12 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`34. As shown in Figure 4, the signal processor had two main substructures:
`
`the top branch was a “conventional beamformer” designed to capture the target
`
`signal plus noise, and the bottom branch was the “sidelobe canceling path” that
`
`would be used to capture only noise so that the noise could be subtracted or canceled
`
`out.
`
`35.
`
`I will refer to the top branch as the first virtual sensor. The first virtual
`
`sensor was formed by combining the outputs of the array sensors. The sensor
`
`outputs were combined by multiplying the sensor output signals by “weights”
`
`𝑤𝑤𝑐𝑐1,…,𝑤𝑤𝑐𝑐𝑐𝑐 (sometimes also called “gains”) and summing the weighted sensor
`signals to produce a “nonadaptive-beamformed signal 𝑦𝑦𝑐𝑐(𝑘𝑘).” (Id., 29.) (The
`variable 𝑘𝑘 in Griffiths-Jim’s equations is time expressed in samples. (E.g., id., 27
`(“We denote the sampled output of the 𝑚𝑚th time-delayed sensor by 𝑥𝑥𝑚𝑚(𝑘𝑘).”), 28
`(“We let 𝐀𝐀𝑙𝑙 and 𝐗𝐗(𝑘𝑘 − 𝑙𝑙) represent the filter coefficient and signal vectors at the
`𝑙𝑙th delay point . . . .”).) Griffiths-Jim’s discussion is thus in the time domain, which
`sensor (top branch) is 𝑦𝑦𝑐𝑐′(𝑘𝑘), which “is obtained by filtering 𝑦𝑦𝑐𝑐(𝑘𝑘) and the FIR
`operator containing . . . constraint values 𝑓𝑓(𝑙𝑙)” (id., 30):
`
`refers to analyzing signals as a function of time.) The output of the first virtual
`
`
`
`-13-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 13 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`(Id., 30.) This output 𝑦𝑦𝑐𝑐′(𝑘𝑘) contains the target signal plus noise.
`
`36.
`
`“The lower path in Fig. 4 is the sidelobe canceling path” (id., 30), which
`
`I will refer to as the second virtual sensor. The second virtual sensor is formed
`
`from a combination of the outputs of the array sensors that differs from the
`
`combination used for the first virtual sensor.
`
`37. The lower path includes a blocking matrix designed to “block the
`
`desired signal 𝑠𝑠(𝑘𝑘) from the lower path.” (Id., 30.) In Griffiths-Jim, the blocking
`matrix is denoted by 𝑊𝑊𝑠𝑠����. (Id., 30.)
`
`38.
`
`In the lower path, the array sensor outputs were combined by applying
`
`the blocking matrix and adaptive filter coefficients and summing the result:
`
`where
`
`
`
`
`
`(Id., 30.)
`
`39. The second virtual sensor’s output “𝑦𝑦𝐴𝐴(𝑘𝑘) contains no desired signal
`
`terms” and instead “contains only noise and interference terms.” (Id., 30.) In other
`
`words, the second virtual sensor is designed to capture noise only.
`
`-14-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 14 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`40. The overall output of the GSC, 𝑦𝑦(𝑘𝑘), was produced by subtracting the
`
`noise-only output of the second virtual sensor from the target-plus-noise output of
`
`the first virtual sensor:
`
`
`
`(Id., 30.) The result was a signal with reduced noise. The response of the processor
`
`to the desired signal 𝑠𝑠(𝑘𝑘)𝟏𝟏 is that produced only by 𝑦𝑦𝑐𝑐′(𝑘𝑘),” i.e., only by the first
`
`virtual sensor (top branch). (Id., 30.) The “output due to the presence of only the
`
`desired signal satisfies the constraint defined by (9)” (id., 30), which Griffiths-Jim
`
`explains is commonly made to be a constraint of “zero distortion” (id., 28).
`
`
`
`[1982-c.2001] Use of GSC with Microphone Arrays
`for Speech Applications
`
`41. After Griffiths and Jim’s GSC article, over the next two decades, the
`
`GSC was used in many signal-processing applications, including with microphone
`
`arrays to reduce noise in speech applications.
`
`42.
`
`I understand Exhibit 1007
`
`is U.S. Patent No. 5,651,071
`
`(“Lindemann”). Filed in 1993, Lindemann cites the Griffiths-Jim article and
`
`explains that using a Griffiths-Jim beamformer “to improve signal-to-noise ratio for
`
`hearing aids” was known. (Ex. 1007, 1:40-46, 12:12-14.1)
`
`
`1 Patent citations are in the form of column:line.
`
`-15-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 15 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`43.
`
`I understand Exhibit 1008
`
`is U.S. Patent No. 5,627,799
`
`(“Hoshuyama”). Filed in 1995, Hoshuyama “relates generally to interference
`
`cancelers, and more particularly to a generalized sidelobe canceler, or adaptive
`
`beamformer for an array of sensors such as microphones and the like.” (Ex. 1008,
`
`1:8-11.) Describing the prior art at the time, Hoshuyama states:
`
`According to a prior art microphone array, signals detected by an array
`of microphones are lowpass filtered and summed together to detect a
`target signal that arrives in a particular direction. The adaptive
`microphone array beamformer is one form of the generalized sidelobe
`canceler as described in an article “An alternative Approach to Linearly
`Constrained Adaptive Beamforming”, Lloyd J. Griffiths and Charles
`W. Jim, the IEEE Transactions on Antenna and Propagation, Vol. AP-
`30, No. 1, January 1982, pages 27-34.
`
`(Id., 1:17-26.)
`
`
`
`[2001] Brandstein
`
`44.
`
`In 2001, Springer-Verlag published the reference book MICROPHONE
`
`ARRAYS: SIGNAL PROCESSING TECHNIQUES AND APPLICATIONS (“Brandstein”),
`
`edited by Michael Brandstein of Harvard University and Darren Ward of Imperial
`
`College. In 2001, Springer-Verlag was already a well-established publishing house
`
`and a leading publisher of technical monographs and serial publications consulted
`
`by academics, scientists, engineers, and other practitioners across a wide spectrum
`
`of technical disciplines, including audio-signal processing. I purchased a copy of
`
`-16-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 16 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`Brandstein before the end of 2002, and Exhibit 1003 is a scanned copy of excerpts
`
`from my copy of Brandstein.
`
`45. Following its publication, Brandstein quickly became a standard
`
`reference for those working in the field of audio signal processing.
`
`46. Chapter 5 of Brandstein concerns robust adaptive beamforming, and at
`
`the beginning of that chapter Brandstein states: “Applications of beamforming
`
`include microphone arrays for speech enhancement. The goal of speech
`
`enhancement is to remove undesirable signals such as noise and reverberation.” (Ex.
`
`1003 (Brandstein), 87.) Brandstein further notes: “Among various adaptive
`
`beamformers, the Griffiths-Jim beamformer (GJBF), or the generalized sidelobe
`
`canceler, is most widely known.” (Id., 88 (internal citation omitted).)
`
`
`
`
`
`[continued on next page]
`
`-17-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 17 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`47. As Brandstein states, “Figure 5.1 depicts the structure of the GJBF.”
`
`(Id., 88.) Figure 5.1 appears below with highlighting and annotations:
`
`(Id., 89, Fig. 5.1 (annotated).)
`
`
`
`48. As illustrated in the top branch of Figure 5.1, the signals of at least two
`
`the signals to form a fixed beamformer that is a first virtual microphone. The first
`
`virtual microphone captures the target speech signal plus noise. As in the case of
`
`physical microphones, 𝑥𝑥0(𝑘𝑘) and 𝑥𝑥1(𝑘𝑘), are combined by filtering and summing
`Griffiths-Jim, the variable 𝑘𝑘 is time expressed in samples. Brandstein’s discussion
`
`is thus in the time domain.
`
`-18-
`
`Meta Platforms, Inc. Exhibit 1002
`Page 18 of 90
`
`
`
`Amazon.com, Inc. v. Jawbone Innovations, LLC
`Declaration of Dr. Richard M. Stern – U.S. Patent 11,122,357
`
`
`
`49.
`
`In the bottom branch of Figure 5.1, the signals of the two physical
`
`microphones are combined by filtering and summing the signals in a different way
`
`to form a second virtual microphone. The second virtual microphone includes a
`
`blocking matrix (BM). “[T]he BM forms a null in the look direction so that the
`
`target signal is suppressed and all other signals are passed through.” (Id., 88.) “The
`
`BM was named after its function, which is to block the target signal.” (Id.) As a
`
`result, the second virtual microphone captures noise only.
`
`50. The output of the GSC (or GJBF) is the output of the top branch (first
`
`virtual microphone) minus the output of the bottom branch (second virtual
`
`microph