`571-272-7822
`
`Paper 59
`Entered: January 12, 2015
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`TOYOTA MOTOR CORPORATION,
`Petitioner,
`
`v.
`
`AMERICAN VEHICULAR SCIENCES LLC,
`Patent Owner.
`____________
`
`Case IPR2013-00419
`Patent 6,772,057 B2
`____________
`
`
`
`Before JAMESON LEE, TREVOR M. JEFFERSON, and
`LYNNE E. PETTIGREW, Administrative Patent Judges.
`
`
`PETTIGREW, Administrative Patent Judge.
`
`
`
`
`
`FINAL WRITTEN DECISION
`35 U.S.C. § 318(a) and 37 C.F.R. § 42.73
`
`I. BACKGROUND
`
`Petitioner, Toyota Motor Corporation (“Toyota”), filed a Petition for
`
`inter partes review of claims 1–4, 7–10, 30–34, 37–41, 43, 46, 48, 49, 56,
`
`59–62, and 64 of U.S. Patent No. 6,772,057 B2 (Ex. 1001, “the ’057
`
`patent”). Paper 3 (“Pet.”). Patent Owner, American Vehicular Sciences
`
`LLC (“AVS”), filed a Preliminary Response. Paper 17 (“Prelim. Resp.”).
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`On January 13, 2014, pursuant to 35 U.S.C. § 314, we instituted an inter
`
`partes review for all challenged claims on certain grounds of unpatentability
`
`asserted in the Petition. Paper 19 (“Dec. on Inst.”).
`
`Subsequent to institution, AVS filed a Patent Owner Response (Paper
`
`33), and later filed a Revised Patent Owner Response (Paper 45, “PO
`
`Resp.”).1 Toyota filed a Reply to Patent Owner’s Response (Paper 40, “Pet.
`
`Reply”).
`
`A consolidated oral hearing for this proceeding and Toyota Motor
`
`Corp. v. American Vehicular Sciences, LLC, IPR2013-00424, involving the
`
`same parties and similar issues, was held on August 18, 2014. A transcript
`
`of the consolidated hearing is included in the record. Paper 58 (“Tr.”).
`
`We have jurisdiction under 35 U.S.C. § 6(c). This Final Written
`
`Decision is issued pursuant to 35 U.S.C. § 318(a) and 37 C.F.R. § 42.73.
`
`As explained below, Toyota has shown by a preponderance of the
`
`evidence that claims 30, 32–34, 37–40, 43, 46, 48, and 49 of the ’057 patent
`
`are unpatentable, but Toyota has not shown by a preponderance of the
`
`evidence that claims 1–4, 7–10, 31, 41, 56, 59–62, and 64 of the ’057 patent
`
`are unpatentable.
`
`A. Related Proceedings
`
`Toyota and AVS indicate that the ’057 patent has been asserted by
`
`AVS in the following district court cases: American Vehicular Sciences
`
`LLC v. Toyota Motor Corp., No. 6:12-cv-00410 (E.D. Tex.) (filed June 25,
`
`2012); American Vehicular Sciences LLC v. BMW Group, No. 6:12-cv-
`
`
`
`1 We authorized AVS to file a Revised Patent Owner Response to make
`certain non-substantive corrections to the Patent Owner Response. See
`Papers 39, 44 (Orders on Conduct of the Proceedings).
`
`2
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`00415 (E.D. Tex.) (filed June 25, 2012); American Vehicular Sciences LLC
`
`v. Subaru of Am. Inc., No. 6:12-cv-00230 (E.D. Tex.) (filed Mar. 8, 2013);
`
`and American Vehicular Sciences LLC v. Mercedes-Benz U.S. Int’l, Inc.,
`
`No. 6:13-cv-00309 (E.D. Tex.) (filed Apr. 3, 2013). Pet. 1; Paper 27, 2–3.
`
`B. The’057 Patent
`
`The ’057 patent, titled “Vehicular Monitoring Systems Using Image
`
`Processing,” generally relates to a vehicle monitoring arrangement for
`
`monitoring an environment exterior of a vehicle. Ex. 1001, Abstract. One
`
`embodiment of such an arrangement described in the ’057 patent includes a
`
`transmitter that transmits electromagnetic waves into the environment
`
`exterior of a vehicle and one or more receivers that receive reflections of the
`
`transmitted waves from exterior objects, such as approaching vehicles.
`
`Id. at 14:8–12, 14:32–37, 38:7–13, Fig. 7. In a preferred implementation,
`
`the transmitter is an infrared transmitter, and the receivers are CCD (charge
`
`coupled device) transducers that receive the reflected infrared waves.
`
`Id. at 38:10–12, 39:25–28. One or more receivers may be arranged on a rear
`
`view mirror of the vehicle. Id. at 14:58–60, 38:22–25. The system also may
`
`include radar or pulsed laser radar (lidar) for measuring distance between the
`
`vehicle and exterior objects. Id. at 14:38–40, 39:1–6.
`
`The waves received by the receivers contain information about
`
`exterior objects in the environment, and the receivers generate signals
`
`characteristic of the received waves. Id. at 14:12–14, 39:44–49. A trained
`
`pattern recognition means, such as a neural computer or neural network,
`
`processes the signals to provide a classification, identification, or location of
`
`an exterior object. Id. at 14:17–25, 39:49–54. Training of a neural network
`
`to provide classification, identification, or location of objects is
`
`3
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`accomplished by conducting a large number of experiments in which the
`
`system is taught to differentiate among received signals corresponding to
`
`different objects. Id. at 36:22–39 (describing a neural network training
`
`session in connection with an embodiment that monitors an interior of a
`
`vehicle, particularly the passenger seat). The classification, identification, or
`
`location of an exterior object may be used to affect operation of other
`
`systems in the vehicle, e.g., to show an image or icon on a display viewable
`
`by a driver or to deploy an airbag. Id. at 14:21–31, 39:54–62.
`
`C. Illustrative Claims
`
`Of the challenged claims, claims 1, 30, 40, and 56 are independent.
`
`Claims 1, 30, and 40 are illustrative:
`
`A monitoring arrangement for monitoring an
`1.
`environment exterior of a vehicle, comprising:
`
`at least one receiver arranged to receive waves from the
`environment exterior of the vehicle which contain information
`on any objects in the environment and generate a signal
`characteristic of the received waves; and
`
`a processor coupled to said at least one receiver and
`comprising trained pattern recognition means for processing the
`signal to provide a classification, identification or location of
`the exterior object, said trained pattern recognition means being
`structured and arranged to apply a trained pattern recognition
`algorithm generated from data of possible exterior objects and
`patterns of received waves from the possible exterior objects to
`provide the classification, identification or location of the
`exterior object;
`
`whereby a system in the vehicle is coupled to said
`processor such that the operation of the system is affected in
`response to the classification, identification or location of the
`exterior object.
`
`4
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`30. A vehicle including a monitoring arrangement for
`monitoring an environment exterior of
`the vehicle,
`the
`monitoring arrangement comprising:
`
`at least one receiver arranged on a rear view mirror of
`the vehicle to receive waves from the environment exterior of
`the vehicle which contain information on any objects in the
`environment and generate a signal characteristic of the received
`waves; and
`
`a processor coupled to said at least one receiver and
`arranged to classify or identify the exterior object based on the
`signal and thereby provide the classification or identification of
`the exterior object;
`
`whereby a system in the vehicle is coupled to said
`processor such that the operation of the system is affected in
`response to the classification or identification of the exterior
`object.
`
`40. A monitoring arrangement for monitoring an
`environment exterior of a vehicle, comprising:
`
`a plurality of receivers arranged apart from one another
`and to receive waves from different parts of the environment
`exterior of the vehicle which contain information on any objects
`in the environment and generate a signal characteristic of the
`received waves; and
`
`a processor coupled to said receivers and arranged to
`classify, identify or locate the exterior object based on the
`signals generated by said receivers and thereby provide the
`classification[,] identification or location of the exterior object,
`
`whereby a system in the vehicle is coupled to said
`processor such that the operation of the system is affected in
`response to the classification, identification or location of the
`exterior object.
`
`Ex. 1001, 54:13–32, 55:58–56:6, 56:37–52 (emphases added).
`
`5
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`D. Pending Grounds of Unpatentability
`
`We instituted inter partes review based on the following grounds of
`
`unpatentability asserted in the Petition:
`
`Reference(s)
`
`Basis
`
`Claim(s)
`
`Lemelson2
`
`35 U.S.C. § 102(e)
`
`1–4, 7–10, 40, 41, 46, 48,
`49, 56, 59–61, and 64
`
`Lemelson and
`Borcherts3
`Lemelson and
`Asayama4
`Lemelson, Borcherts,
`and Asayama
`Yamamura5 and
`Borcherts
`
`Dec. on Inst. 38.
`
`35 U.S.C. § 103(a) 30–34, 37–39, and 62
`
`35 U.S.C. § 103(a) 4, 43, and 59
`
`35 U.S.C. § 103(a) 34
`
`35 U.S.C. § 103(a) 30, 32, and 37–39
`
`II. ANALYSIS
`
`A. Claim Construction
`
`In the Decision on Institution, we construed several claim terms of the
`
`’057 patent, as set forth in the following table:
`
`
`
`2 U.S. Patent No. 6,553,130, issued Apr. 22, 2003 (Ex. 1002).
`3 U.S. Patent No. 5,245,422, issued Sept. 14, 1993 (Ex. 1004).
`4 U.S. Patent No. 5,214,408, issued May 25, 1993 (Ex. 1005).
`5 Japanese Unexamined Patent Application Publication No. H06-124340,
`published May 6, 1994 (Ex. 1012). Citations to Yamamura refer to its
`English translation (Ex. 1013).
`
`6
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`Claim Term
`
`Board’s Construction
`
`“trained pattern
`recognition algorithm”
`
`“trained pattern
`recognition means”
`
`“identify”
`
`“exterior object”
`
`“rear view mirror”
`
`“transmitter”
`
`“an algorithm that processes a signal that is
`generated by an object, or is modified by
`interacting with an object, in order to
`determine to which one of a set of classes
`the object belongs, the algorithm having
`been taught, through a variety of examples,
`various patterns of received signals
`generated or modified by objects”
`“a neural computer or neural network
`trained for pattern recognition, and
`equivalents thereof”
`“determine that the object belongs to a
`particular set or class”
`“a material or physical thing outside the
`vehicle, not a part of the roadway on which
`the vehicle travels”
`“a mirror that faces to the rear”
`“device that transmits any type of
`electromagnetic waves, including visible
`light”
`
`Dec. on Inst. 8–15. AVS does not contest these constructions for purposes
`
`of this proceeding, PO Resp. 9–10, and Toyota does not dispute these
`
`constructions in its Reply. We maintain these constructions for this Final
`
`Written Decision.
`
`B. Principles of Law
`
`To prevail in challenging AVS’s claims, Toyota must demonstrate by
`
`a preponderance of the evidence that the claims are unpatentable. 35 U.S.C.
`
`§ 316(e); 37 C.F.R. § 42.1(d). A claim is anticipated if a single prior art
`
`reference either expressly or inherently discloses every limitation of the
`
`claim. Orion IP, LLC v. Hyundai Motor Am., 605 F.3d 967, 975 (Fed. Cir.
`
`2010). A claim is unpatentable under 35 U.S.C. § 103(a) if the differences
`
`7
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`between the claimed subject matter and the prior art are such that the subject
`
`matter, as a whole, would have been obvious at the time of the invention to a
`
`person having ordinary skill in the art. KSR Int’l Co. v. Teleflex, Inc., 550
`
`U.S. 398, 406 (2007).
`
`C. Anticipation by Lemelson
`Claims 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61, and 64
`
`Toyota asserts that claims 1–4, 7–10, 40, 41, 46, 48, 49, 56, 59–61,
`
`and 64 are unpatentable under 35 U.S.C. § 102(e) as anticipated by
`
`Lemelson. Pet. 10–24. In support of this assertion, Toyota provides detailed
`
`analysis and claim charts explaining how Lemelson discloses each claim
`
`limitation. Id. Toyota also relies on the testimony of Dr. Nikolaos
`
`Papanikolopoulos, as set forth in his Declaration (Ex. 1016) and Reply
`
`Declaration (Ex. 1023).
`
`AVS responds that Lemelson does not disclose each limitation of the
`
`challenged claims. PO Resp. 11–25. For support, AVS relies on the
`
`testimony of Dr. Cris Koutsougeras, as set forth in his Declaration
`
`(Ex. 2001).
`
`Having considered the parties’ contentions and supporting evidence,
`
`we determine that Toyota has demonstrated by a preponderance of the
`
`evidence that claims 40, 46, 48, and 49 are anticipated by Lemelson, but
`
`Toyota has not demonstrated by a preponderance of the evidence that claims
`
`1–4, 7–10, 41, 56, 59–61, and 64 are anticipated by Lemelson.
`
`1. Lemelson
`
`Lemelson discloses a computerized system in a motor vehicle that
`
`identifies possible obstacles on a roadway and either warns the driver or
`
`controls the operation of vehicle systems, such as the brakes or steering
`
`8
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`mechanism, to avoid or lessen the effect of a collision. Ex. 1002, Abstract,
`
`5:15–29, 8:38–39. The system includes at least one video camera, mounted
`
`at the front end of the vehicle, such as the front end of the roof, bumper, or
`
`hood. Id. at 5:31–34. The system may include multiple cameras for front,
`
`side, and rear viewing, and for stereo imaging capabilities. Id. at 6:27–42.
`
`The video camera also may be implemented with other technologies,
`
`including infrared imaging methods. Id. at 6:34–37. In addition, the system
`
`may use radar or lidar for range detection. Id. at 5:67–6:4. “[V]ideo
`
`scanning and radar or lidar scanning may be jointly employed to identify and
`
`indicate distances between the controlled vehicle and objects ahead of, to the
`
`side(s) of, and to the rear of the controlled vehicle.” Id. at 6:5–8.
`
`The analog signal output from the video camera(s) is digitized in an
`
`analog-to-digital convertor and passed to an image analyzing computer
`
`(IAC), which is
`
`provided, implemented and programmed using neural networks
`and artificial intelligence as well as fuzzy logic algorithms to
`(a) identify objects on the road ahead such as other vehicles,
`pedestrians, barriers and dividers, turns in the road, signs and
`symbols, etc., and generate identification codes, and (b) detect
`distances from such objects by their size (and shape) and
`provide codes indicating same for use by a decision computer[]
`23, which generates coded control signals which are applied
`through the computer 11 or are directly passed to various
`warning and vehicle operating devices such as a braking
`computer or drive[] 35, which operates a brake servo 33, a
`steering computer or drive(s) 39 and 40 which operate steering
`servos 36; . . . a headlight controller 41 for flashing the head
`lights, a warning light control 42 for flashing external and/or
`internal warning lights; a horn control 43, etc.
`
`Id. at 5:39–59. The IAC also may display symbols representing hazard
`
`objects. Id. at 6:52–55, 9:60–62.
`
`9
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`Lemelson discloses further details regarding a neural network
`
`embodiment of the IAC for identifying objects:
`
`Neural networks used in the vehicle . . . warning system are
`trained to recognize roadway hazards which the vehicle is
`approaching including automobiles, trucks, and pedestrians.
`Training involves providing known inputs to the network
`resulting
`in desired output responses. The weights are
`automatically adjusted based on error signal measurements until
`the desired outputs are generated. Various learning algorithms
`may be applied. Adaptive operation is also possible with on-
`line adjustment of network weights
`to meet
`imaging
`requirements. The neural network embodiment of the image
`analysis computer 19 provides a highly parallel
`image
`processing structure with rapid, real-time image recognition
`necessary for the Motor Vehicle Warning and Control System.
`
`Id. at 8:1–14.
`
`2. “Generated from” Limitation
`
`A central dispute between the parties is whether Lemelson discloses a
`
`“trained pattern recognition algorithm generated from data of possible
`
`exterior objects and patterns of received waves from the possible exterior
`
`objects.” This limitation appears in independent claims 1 and 56 and in
`
`claim 41, which depends from claim 40. Although we construed “trained
`
`pattern recognition algorithm” in our Decision on Institution, we did not
`
`provide an express construction for the “generated from” language following
`
`that term in the claims. For this Final Written Decision, we construe the
`
`“generated from” limitation according to its broadest reasonable
`
`interpretation in light of the specification of the ’057 patent. See 37 C.F.R.
`
`§ 42.100(b).
`
`AVS contends that the “generated from” limitation in claims 1, 41,
`
`and 56 requires a particular type of training to generate the “trained pattern
`
`10
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`recognition algorithm” recited in the claims. PO Resp. 6. Relying on the
`
`Declaration of Dr. Koutsougeras, AVS argues that the ’057 patent discloses
`
`and claims a specific method for training the pattern recognition means
`
`using “data of possible exterior objects and patterns of received waves from
`
`the possible exterior objects.” Id. (citing Ex. 2001 ¶¶ 19, 20, 53). For
`
`example, if a vehicle uses a radar receiver, AVS contends that the claims
`
`require training using “real radar waves” received from actual examples of
`
`possible exterior objects placed in front of the radar system, along with data
`
`indicating the identity or classification of the objects. Id. at 6–7 (citing
`
`Ex. 2001 ¶¶ 19–20). AVS contrasts the use of real waves to train a pattern
`
`recognition system, as recited in the claims, with other methods of training,
`
`such as using simulated data (e.g., a computer simulation of radar waves).
`
`Id. at 7 (citing Ex. 2001 ¶¶ 46, 55–63).
`
`Toyota argues that the “generated from” language of claims 1, 41, and
`
`56 does not require training with real data. Pet. Reply 2, 4. Because the
`
`claims refer to “data of possible exterior objects” and “patterns of received
`
`waves,” rather than “data from” and “patterns from,” Toyota contends that
`
`the claim language encompasses training using simulated data and patterns
`
`that represent possible objects and received waves, respectively.
`
`Id. at 2, 4–5.
`
`As neither party asserts that the “generated from” language is defined
`
`in the ’057 patent, we give the claim language its ordinary and customary
`
`meaning as would be understood by a person having ordinary skill in the art
`
`in the context of the entire patent disclosure. See In re Translogic Tech.,
`
`Inc., 504 F.3d 1249, 1257 (Fed. Cir. 2007). In describing the training of
`
`pattern recognition systems, such as neural networks, for use with the
`
`11
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`invention, the ’057 patent explains that a large number of experiments are
`
`conducted in which different objects are placed in numerous positions and
`
`orientations, and signals from a CCD array are returned from the objects and
`
`measured by sensors or transducers. Ex. 1001, 36:22–39. This is the sole
`
`example of a pattern recognition training session provided in the ’057 patent.
`
`Although the described training session relates to objects inside a vehicle,
`
`the ’057 patent indicates that pattern recognition systems for identifying
`
`exterior objects are trained in a similar manner. See, e.g., id. at 40:1–9.
`
`According to the plain language of the disputed limitation, the
`
`algorithm must be generated from both “data of possible exterior objects”
`
`and “patterns of received waves from the possible exterior objects.” We are
`
`not persuaded by Toyota’s argument that the claimed “patterns of received
`
`waves,” in contrast to “patterns from received waves,” need only be patterns
`
`representing what received waves would look like. Neither the written
`
`description of the ’057 patent nor the claim language, in context, supports
`
`such parsing, particularly when the claim language further requires the
`
`waves to be received “from” possible exterior objects.
`
`In view of the claim language and the description in the ’057 patent of
`
`a training session using signal patterns actually received from real objects,
`
`we see no reasonable basis for interpreting “generated from . . . patterns of
`
`received waves from the possible exterior objects” to encompass training of
`
`a pattern recognition algorithm using simulated wave patterns. Therefore,
`
`the broadest reasonable construction of the claim language at issue requires a
`
`pattern recognition algorithm that has been generated using patterns of
`
`waves actually received from possible exterior objects.
`
`12
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`3. Claims 1–4, 7–10, 41, 56, 59–61, and 64
`
`The monitoring arrangement in independent claim 1 classifies,
`
`identifies, or locates an exterior object using a “trained pattern recognition
`
`algorithm generated from data of possible exterior objects and patterns of
`
`received waves from the possible exterior objects.” Independent claim 56
`
`and dependent claim 41, which depends from claim 40, recite the same
`
`limitation. In its Petition, Toyota contends that Lemelson’s IAC classifies,
`
`identifies, and locates objects through the use of a neural network for pattern
`
`recognition that has been trained on a data set. Pet. 12 (citing Ex. 1016
`
`¶¶ 52–55). According to Toyota, “Lemelson explains that the neural
`
`network in the IAC may be ‘trained’ using ‘known inputs.’” Id. at 13 (citing
`
`Ex. 1002, 7:47–8:10, 8:21–23).
`
`AVS contends that Lemelson does not disclose, either expressly or
`
`inherently, the specific type of training of the pattern recognition algorithm
`
`recited in claims 1, 41, and 56. PO Resp. 11. AVS admits that Lemelson
`
`discloses a system for identifying objects exterior to a vehicle using a type of
`
`pattern recognition algorithm (a neural network). Id. at 12. AVS contends,
`
`however, that Lemelson’s reference to “known inputs” fails to disclose a
`
`trained pattern recognition algorithm that is generated in the manner
`
`required by the claims. Id. at 14–21.
`
`As discussed above, we have construed “trained pattern recognition
`
`algorithm generated from data of possible exterior objects and patterns of
`
`received waves from the possible exterior objects” to require training using
`
`patterns of waves actually received from possible exterior objects. For the
`
`following reasons, we find that Toyota has not demonstrated by a
`
`13
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`preponderance of the evidence that Lemelson discloses a trained pattern
`
`recognition algorithm generated in this manner.
`
`As AVS correctly asserts, id. at 12, Toyota’s Petition and supporting
`
`Declaration of Dr. Papanikolopoulos cite only one sentence from Lemelson
`
`as relating to training a neural network for pattern recognition: “Training
`
`involves providing known inputs to the network resulting in desired output
`
`responses.” Ex. 1002, 8:4–6 (emphasis added); see Pet. 11, 13, 16–17;
`
`Ex. 1016 ¶ 55. AVS’s arguments focus on whether a person having ordinary
`
`skill in the art would have understood Lemelson’s “known inputs” to refer to
`
`patterns of waves actually received from possible exterior objects. AVS
`
`submits that there are other ways Lemelson’s system could have generated
`
`its pattern recognition algorithm and, therefore, Lemelson’s known inputs
`
`are not necessarily “patterns of received waves from the possible exterior
`
`objects,” as recited in the claims. PO Resp. 14–21.
`
`First, AVS argues that Lemelson could have involved generating the
`
`pattern recognition algorithm using “simulated data.” PO Resp. 14–15
`
`(citing Ex. 2001 ¶¶ 55–63). Relying on the testimony of Dr. Koutsougeras,
`
`AVS explains that “[s]imulated data is data that does not include any
`
`‘patterns of received waves from the possible exterior objects.’ Rather, it is
`
`generated by computer programs that simulate what sensors would be
`
`reading if they were detecting an object.” Id. at 15 (citing Ex. 2001 ¶¶ 55–
`
`63). Dr. Koutsougeras testifies that Lemelson’s “known inputs” very well
`
`could have been simulated data, as using simulated data for training neural
`
`networks was widely known, and using simulated data for training a neural
`
`network on a vehicle had been described in a published thesis. Ex. 2001
`
`¶¶ 58–59 (citing Ex. 2004, 38). AVS also argues that Lemelson’s “known
`
`14
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`inputs” could have involved training with waves actually received from
`
`objects other than exterior objects to be classified or identified, e.g., training
`
`with waves from license plates rather than vehicles. PO Resp. 19–21 (citing
`
`Ex. 2001 ¶¶ 64–71).
`
`Toyota responds that a person of ordinary skill in the art would have
`
`understood that training a neural network to identify exterior objects in
`
`Lemelson would have been done with “real data,” and not with simulated or
`
`“partial data” (i.e., actual waves received from objects other than those being
`
`detected). Pet. Reply 8 (citing Ex. 1023 ¶¶ 10–26). Initially, we note that
`
`Toyota introduces this argument for the first time in its Reply, along with a
`
`supporting Reply Declaration from Dr. Papanikolopoulos (Ex. 1023). A
`
`reply may only respond to arguments raised in the patent owner response.
`
`37 C.F.R. § 42.23(b). Furthermore, “a reply that raises a new issue or
`
`belatedly presents evidence will not be considered.” Office Patent Trial
`
`Practice Guide, 77 Fed. Reg. 48,756, 48,767 (Aug. 12, 2014).
`
`Even if we consider Toyota’s newly proffered argument and evidence
`
`as responsive to AVS’s Patent Owner Response, we are not persuaded that
`
`Toyota has shown by a preponderance of the evidence that Lemelson’s
`
`“known inputs” are patterns of waves actually received from possible
`
`exterior objects, as required by the claims. Instead, we find credible the
`
`testimony of AVS’s expert, Dr. Koutsougeras, that one of ordinary skill in
`
`the art would have understood that Lemelson’s “known inputs” could
`
`include real or simulated data for training a neural network. See Ex. 2001
`
`¶¶ 55–63; see also Ex. 1022, 132:24–138:5, 157:12–159:14, 163:18–164:7
`
`15
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`(deposition testimony of Dr. Koutsougeras).6 This understanding is
`
`supported by Toyota’s counsel, who was asked at the hearing whether
`
`“known inputs” in Lemelson refers to “just real sensor data or is . . .
`
`understood as both” real data and simulated data. Tr. 27:25–28:1. He
`
`answered that “one of ordinary skill in the art would have understood
`
`[known inputs] as real sensor data, but it is not to the exclusion of simulated
`
`data.” Id. at 28:4–6 (emphasis added). In addition, Toyota’s expert,
`
`Dr. Papanikolopoulos, acknowledges in his deposition testimony that use of
`
`simulated data was a possibility for training pattern recognition algorithms.
`
`See Ex. 2002, 102:5–14 (stating that “[i]n this particular domain, you go to
`
`simulated data, or if you don’t have access to real data, to real images” for
`
`training pattern recognition systems to detect automobiles); see also id.
`
`at 48:2–9 (stating that using simulated data rather than images was a
`
`possibility).
`
`In his Reply Declaration, Dr. Papanikolopoulos opines that one of
`
`ordinary skill in the art at the time of the invention would have understood
`
`“known inputs” in Lemelson to refer to real data because an ordinarily
`
`skilled artisan “would have known that training with ‘real data’ would have
`
`yielded the best results for” the purpose of identifying exterior objects.
`
`Ex. 1023 ¶ 9. In essence, Dr. Papanikolopoulos’s supporting analysis
`
`indicates that one of ordinary skill in the art would have preferred real data
`
`over simulated or partial data for various applications. See id. ¶¶ 10–26.
`
`This preference, however, is not sufficient to show that Lemelson’s
`
`description of training with “known inputs” expressly discloses generating a
`
`
`
`6 We refer to the page numbers of the deposition transcript rather than the
`exhibit page numbers.
`
`16
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`pattern recognition algorithm using waves actually received from possible
`
`exterior objects, as required by the claims.7 Nor does Toyota’s evidence
`
`show by a preponderance of the evidence that Lemelson’s “known inputs”
`
`inherently, or necessarily, refer to waves actually received from possible
`
`objects, because the “known inputs” could refer to simulated data.
`
`We disagree with Toyota’s argument that Dr. Koutsougeras’s
`
`testimony should be given little weight because he has limited experience
`
`with pattern recognition in vehicles. See Pet. Reply 11–12.
`
`Dr. Koutsougeras testified that his “dissertation [was] on neural networks,
`
`particularly methods for training neural networks,” and he has taught classes
`
`and directed student theses on neural networks or for which neural networks
`
`were a substantial component. Ex. 1022, 19:19–20:24. We are not
`
`persuaded by Toyota’s argument that experience in training neural networks
`
`specifically for vehicle exterior monitoring systems is necessary to support
`
`testimony regarding an ordinarily skilled artisan’s understanding of
`
`Lemelson’s disclosure of training using “known inputs.”
`
`In addition, we are not persuaded by Toyota’s untimely citation in its
`
`Reply to a portion of Lemelson discussing “adaptive operation” and “online
`
`adjustment” of a neural network. See Pet. Reply 11 (citing Ex. 1002, 8:9–
`
`10, Figs. 1, 2). This passage refers to alterations to the neural network after
`
`training has occurred. Thus, Toyota’s evidence of “real-world” data being
`
`
`
`7 We are not persuaded by Toyota’s unsupported argument, improperly
`raised for the first time in its Reply, that “known inputs” in Lemelson are
`analogous to a genus, and “real data” is a claimed species, so that the
`“generated from” claim language is met by Lemelson’s disclosure of
`“known inputs.” See Pet. Reply 7.
`
`17
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`used to adjust operations of a neural network does not show that Lemelson
`
`discloses training with real data.
`
`Toyota also argues for the first time in its Reply that the “generated
`
`from” limitation is not a limitation for purposes of patentability because it is
`
`a process step within an apparatus claim and should not be given patentable
`
`weight. See Pet. Reply 3–4. We find this argument to be untimely.
`
`Toyota’s Petition does not treat the “generated from” language as if it is not
`
`a limitation, and Toyota makes no argument in its Petition that any claim
`
`contains product-by-process language. See Pet. 12–13.
`
`In any event, we are not persuaded by Toyota’s argument (Pet.
`
`Reply 4) that because the resulting pattern recognition algorithm is the same
`
`whether the algorithm is trained using real data or simulated data, the
`
`“generated from” language is not a limitation for purposes of patentability.
`
`See In re Thorpe, 777 F.2d 695, 697 (Fed. Cir. 1985) (“If the product in a
`
`product-by-process claim is the same as or obvious from a product of the
`
`prior art, the claim is unpatentable even though the prior product was made
`
`by a different process.”). Toyota’s proffered expert testimony and attorney
`
`argument suggest that a pattern recognition algorithm trained with real data
`
`differs from one that is trained using simulated data. See, e.g., Ex. 1023
`
`¶¶ 18, 21 (testimony from Dr. Papanikolopoulos that simulated data would
`
`have been ineffective at the time of the invention for training a neural
`
`network to distinguish between types of objects, resulting in “garbage in-
`
`garbage out”); see also id. ¶¶ 13–15 (testimony from Dr. Papanikolopoulos
`
`that training with partial data would have been ineffective). Accordingly,
`
`we do not find that the “generated from” language of claims 1, 41, and 56 is
`
`not a limitation for purposes of patentability.
`
`18
`
`
`
`IPR2013-00419
`Patent 6,772,057 B2
`
`Based on the foregoing, Toyota’s Petition and supporting evidence
`
`fail to establish that the reference in Lemelson to training with “known
`
`inputs” discloses training with patterns of waves actually received from
`
`possible exterior objects. In addition, Toyota’s Reply fails to provide timely
`
`and persuasive evidence or argument that a person having ordinary skill in
`
`the art would have understood that training a neural computer using “known
`
`inputs” in Lemelson necessarily describes training with patterns of waves
`
`actually received from possible exterior objects.
`