throbber
·1· ·UNITED STATES PATENT AND TRADEMARK OFFICE
`· · ·BEFORE THE PATENT TRIAL AND APPEAL BOARD
`·2· ·-----------------------------------------X
`· · ·TOYOTA MOTOR CORPORATION,
`·3
`
`·4· · · · · · · · · · ·Petitioner,
`
`·5· · · · · · · · · · · · · · ·IPR2013-00412
`· · ·VS.· · · · · · · · · · · ·IPR2013-00413
`·6· · · · · · · · · · · · · · ·IPR2013-00416
`
`·7
`· · ·AMERICAN VEHICULAR SCIENCES, LLC,
`·8
`· · · · · · · · · · · ·Patent Owner.
`·9· ·-----------------------------------------X
`
`10
`
`11
`
`12· · · · · · · · · DEPOSITION OF
`
`13· · · · ·NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`
`14· · · · · · ·Monday, February 24, 2014
`
`15· · · · · · · · New York, New York
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23· ·Reported By:
`
`24· ·LINDA J. GREENSTEIN
`
`25· ·JOB NO. 89371
`
`AVS EXHIBIT 2003
`Toyota Inc. v. American Vehicular Sciences LLC
`IPR2013-00424
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· ·exploited -- we had a camera mounted inside
`
`·3· ·the car, and we use a location of the
`
`·4· ·license plate with respect to the rest of
`
`·5· ·the car, and we tried to compute also other
`
`·6· ·characteristics; say, characteristics of
`
`·7· ·the license plate.
`
`·8· · · · · · · And then based on the size, we
`
`·9· ·try to keep the size constant, it allows us
`
`10· ·to keep constant distance as part of a
`
`11· ·vehicle following application.
`
`12· · · · Q.· · Did that system use a pattern
`
`13· ·recognition algorithm?
`
`14· · · · · · · MR. BERKOWITZ:· Object to form.
`
`15· · · · A.· · Again, let me clarify since we
`
`16· ·have these terms in my -- the claim
`
`17· ·construction I was given.· Do you want me
`
`18· ·to answer --
`
`19· · · · Q.· · As you normally use that term,
`
`20· ·did that system, the one that you refer to
`
`21· ·in paragraph 4 of your 419 application, use
`
`22· ·a pattern recognition algorithm?
`
`23· · · · · · · MR. BERKOWITZ:· Object to form.
`
`24· · · · · · · Lack of foundation.
`
`25· · · · A.· · Normally use -- again, under the
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· ·assumption of the claim construction that I
`
`·3· ·was given, the answer is yes.
`
`·4· · · · Q.· · What about under your ordinary
`
`·5· ·usage of that term?
`
`·6· · · · · · · MR. BERKOWITZ:· Object to form
`
`·7· ·and lack of foundation.
`
`·8· · · · A.· · By using the claim construction
`
`·9· ·of pattern recognition, yes.
`
`10· · · · Q.· · I wasn't asking about the claim
`
`11· ·construction in this case.
`
`12· · · · · · · I was just asking about as you
`
`13· ·would normally use that term in your
`
`14· ·research, did the system of paragraph 4 use
`
`15· ·a pattern recognition algorithm?
`
`16· · · · · · · MR. BERKOWITZ:· Object to form.
`
`17· · · · · · · Lack of foundation.
`
`18· · · · A.· · Given the precision that I need
`
`19· ·to provide with respect -- since we have
`
`20· ·many terms which are used, I think I need
`
`21· ·to stick to use of the term in my
`
`22· ·declaration.· So in this case, yes.
`
`23· · · · Q.· · So you've used the term "pattern
`
`24· ·recognition" in some of your papers,
`
`25· ·haven't you?
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· ·part of the claim construction, so I'm
`
`·3· ·going to ask you the same clarification.
`
`·4· · · · Q.· · First, as you've construed the
`
`·5· ·claims in this case, did your license plate
`
`·6· ·system involve trained pattern recognition?
`
`·7· · · · · · · MR. BERKOWITZ:· Object to form
`
`·8· ·and foundation.
`
`·9· · · · A.· · So given the claim construction
`
`10· ·I have here, we didn't use trained pattern
`
`11· ·recognition means.
`
`12· · · · Q.· · And then as you used that term
`
`13· ·before these IPR matters, would you have
`
`14· ·said that that used trained pattern
`
`15· ·recognition?
`
`16· · · · · · · MR. BERKOWITZ:· Object to form
`
`17· ·and lack of foundation.
`
`18· · · · A.· · The term "trained pattern
`
`19· ·recognition" is not really common.· I mean,
`
`20· ·we use other terms, so I would say no.
`
`21· · · · Q.· · Okay.
`
`22· · · · · · · How did the system detect a
`
`23· ·license plate?
`
`24· · · · A.· · So I can give you my
`
`25· ·recollection.
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· ·perimeter of the license plate, so it was a
`
`·3· ·scale or a number, to I don't want to be --
`
`·4· ·but mainly images.
`
`·5· · · · Q.· · Okay.
`
`·6· · · · · · · And did you have on the system
`
`·7· ·any data or images relating to license
`
`·8· ·plates?
`
`·9· · · · A.· · I don't recall this, to be
`
`10· ·precise.
`
`11· · · · · · · One thing I recall is actually
`
`12· ·we used to have rectangles, or models of
`
`13· ·rectangles.
`
`14· · · · · · · I don't recall exactly if we had
`
`15· ·the license plates or -- and also, I'm not
`
`16· ·sure in Yue Du's thesis -- I asked him do
`
`17· ·this, but I'm not sure if this was ever
`
`18· ·implemented.
`
`19· · · · Q.· · So it's possible in a system of
`
`20· ·this type to have the system look for
`
`21· ·rectangles rather than specifically compare
`
`22· ·captured images to stored images of license
`
`23· ·plates?
`
`24· · · · · · · MR. BERKOWITZ:· Object to form.
`
`25· · · · · · · Foundation.
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOEAMOTORvaAMS
`
`_
`
`,
`
`February 24, 2014
`48
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`In a system like that, could you
`
`have used simulated data rather than
`
`images?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`Under the assumption that
`
`different situations, different
`
`applications might require different ~—
`
`‘ it's a possibility.
`
`But for us, it was a
`
`requirement, a contractual requirement,
`
`to
`
`demonstrate something like this.
`
`o.
`
`Okay.
`
`MR. BERKOWITZ: Whenever it's a
`
`convenient time, we've been going around an
`
`hour and ten.
`
`BY MR. SCHARFF:.
`
`Q.
`
`Earlier I asked you about the
`
`claim constructions that you applied in
`
`your declaration, and you said that those
`
`were proyided to you.
`
`Were those provided to you by
`
`the attorneys for Toyota?
`
`A.
`
`Yes.
`
`From Kenyon & Kenyon,
`
`just
`
`
`
`1.
`
`2.
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`
`
`1
`
`to be ~~
`5i: ESQUIRE
`
`s o .. u 1
`
`. o r. s
`
`~ 800.211.0EPO (3376)
`
`EsquireSo/utions.oom
`
`

`

`NlKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTAMOTORvsANS
`
`February 24, 2014
`89
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`Q.
`
`So the claim doesn't just say "a
`
`trained pattern recognition algorithm
`generated from examples," does it?
`
`A.
`
`Q.
`
`It doesn't say.
`
`You understand then that Claim 1
`
`requires that the trained pattern
`
`recognition algorithm be generated from’
`
`data of possible exterior objects and
`
`patterns of received waves from the
`
`possible exterior objeCts; correct?
`
`A.
`
`Q.
`
`Yes.
`
`Not every trained pattern
`
`recognition algorithm would be generated
`
`using those specific requirements, would
`
`they?
`'
`
`I
`
`MR. BERKOWITZ: Object to form.
`
`Foundation.
`
`A.
`
`Can you clarify, because this is
`
`a little bit broad?
`broad.
`
`I would say it's too
`
`Q.
`
`What do you regard as the
`
`difference between just a trained pattern
`
`recognition algorithm in general, versus a
`
`trained pattern recognition algorithm
`
`\qumU'II-PUJNH
`
`H O
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`
`
`ESQUIRE
`
`5 o . u T .
`
`u N s
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`
`February 24, 2014
`90
`
`koooqmmithH
`
`l“o
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`1'7
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24;
`
`25
`
`
`
`
`
`Nikolaos PapanikolopoulOs, Ph.D.
`
`generated from data of possible exterior
`
`objects and patterns of received waves from
`
`the possible exterior objects?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`You see, again,
`I have to stick
`to the construction that was given, so I
`
`checked the trained pattern recognition
`
`algorithm, as you stated.
`
`Q.
`
`I'm sorry,
`
`that wasn't my
`
`question.
`
`Can you think of examples of a
`
`trained pattern recognition algorithm that
`
`are not generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects.
`
`MR. BERKOWITZ: Object to form.
`
`MR. SCHARFF:
`
`I'll restate.
`
`BY MR.. SCHARFF:
`
`Q.
`
`So can you think of an example
`
`of a trained pattern recognition algorithm
`
`that is not generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects?
`
`A.
`
`With respect to the specific
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`r” ESQUIRE
`
`s o . u T . a n s
`
`800.21 1.05130 (3376)
`EsquireSo/utions.com
`
`

`

`NMOLAOSPAPANMOLOPOULOSJWiD.
`TOYOTA MOTOR vs. AVS
`
`February 24., 2014
`91
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`claim construction that I had to examine,
`
`no.
`
`Q.
`
`I'm sorry, it was an open—ended
`
`question.
`
`So you can't think of any
`
`examples of a trained pattern recognition
`
`algorithm that are not generated from data
`
`of possible exterior objects and patterns
`of received waves from the possible
`
`exterior objects?
`
`A.
`
`Again, my answer is with respect
`
`to this particular claim construction that
`
`I had to examine,
`
`I had to look at the
`
`whole claim.
`
`The claim construction I was
`
`given,
`
`I had to offer an opinion.
`
`Q.
`
`You have a claim construction
`
`for trained pattern recognition algorithm,
`
`but I'm asking you about the next phrase.
`
`A.
`
`Q.
`
`And this is the one -*
`
`Did you ignore, for purposes of
`
`your opinions,
`
`the phrase "generated from
`
`data of possible exterior objects and
`
`patterns of received waves from the
`
`possible exterior objects"?
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`
`
`
`
`5%”? ‘ ESQUIRE
`
`
`s o L u T
`
`. a r. s
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`LOUJQG‘XU'II‘DLMNH
`HHHHHHmswmHo
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`IWKOLAOSPAPANMOLOPOULOS,HiD.
`TOYOTAMOTORvaANS
`
`February 24, 2014
`92
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`A.
`
`No.
`
`It is included in my
`
`declaration. That is a statement from
`
`Lemelson which addresses this.
`
`So if you look at my declaration
`
`~~ this is the '000 —— you want me to do
`
`the '000?
`
`Q.
`
`The '057. We're on the '057
`
`patent right now.
`
`The IPR.
`
`A.
`
`Q.
`
`Which one do you want?
`
`For the '057 patent.
`
`In your opinion, Lemelson
`
`
`
`discloses:
`
`"A trained pattern recognition
`
`algorithm generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects."
`
`Is that right?
`
`A.
`
`What I am saying is actually
`
`Lemelson explicitly teaches ~~ explicitly
`
`discloses the teaching based on data of
`
`possible exterior objects, and I have a
`
`statement to that effect.
`
`So I have to look at the claim
`
`construction that was given and the
`
`particular claims in order to ~—
`
` fig ESQUIRE
`“’
`s r.
`,. u T
`. o N s
`
`800.211.DEPO (3376)
`Esqu1reSo/ut10ns.oom
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOIAMOTORvaANS
`
`February 24, 2014
`93
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`Q.
`
`So I'm asking you, what type of
`
`trained pattern recognition algorithm is
`
`not generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects?
`
`MR. BERKQWITZ: Objection.
`
`Foundation.
`
`
`
`A.
`
`My reply to this is that I was
`
`not asked this.
`
`I was asked to examine *—
`
`Q.
`
`I understand.
`
`I think that's the difficultly
`
`with you responding to my questions.
`
`I'm allowed to ask you these
`
`questions, and I understand you weren't
`
`previously asked this specific type of
`
`question, but I'm asking this now, and I'm
`
`asking what your answer is.
`
`And you can't just say:
`
`I don't
`
`have an answer because I wasn't asked this
`
`previously or because it wasn't this
`
`specific question that you addressed with
`
`respect to Lemelson.
`
`
`
`
`ommqmmswml—I
`
`23
`
`24
`
`25
`
`slaw
`
`My question is:
`
`
`ESQUIRE
`800.21 1.05130 (3376)
`
`s o L u T . o u s
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`
` Nikolaos Papanikolopoulos, Ph.D.'
`
`Are there trained pattern
`
`recognition algorithms that are not
`
`generated from data of possible exterior
`
`objects and patterns of received waves from
`the possible exterior objects?
`
`MR. BERKOWITZ: Object to form;
`
`Foundation.
`
`A.
`
`You see, it would be highly
`
`speculative on my part to enumerate cases
`
`like this when I have —— my declaration is
`
`very specific.
`
`Q.
`
`I'm sorry, but you do have to
`
`answer the question.
`
`MR. BERKOWITZ: Object as
`
`argumentative.
`
`I think he is answering the
`
`question.
`
`MR. SCHARFF: No, he isn't.
`
`He's saying he's refusing to
`
`
`
`answer it because it wasn't part of the
`
`specific analysis he did from Lemelson.
`MR. BERKOWITZ:
`I disagree with
`
`all that.
`
`MR. SCHARFF: And, by the way,
`
`that isn't an allowed objection under the
`
`
`
`omm4mmswwH<
`
`15
`
`l6
`
`l7
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24:
`
`25
`
`
`, ESQUIRE
`
`s o .. u T . a u s
`
`'
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTAMOTORvsAVS
`
`February 24,- 2014
`95
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`rules.
`
`MR. BERKOWITZ:
`
`If you start to
`
`argue with my witness,
`
`I can make that
`
`objection.
`
`MR. SCHARFF:
`
`It‘s not in the ~~
`
`MR. BERKOWITZ:
`
`In the Appendix?
`
`I disagree with that.
`
`Can you read the question back.
`
`(Requested portion of record
`
`read.)
`
`BY MR. SCHARFF:
`
`Q.
`
`That should be a simple
`
`yes—or—no answer.
`
`Are there or aren't there?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`So let me state, page ~~
`
`paragraph 55 of my '057,
`
`this is from
`
`column 8,
`
`row I through 8.
`
`"Lemelson explains that neural
`
`networks used in the vehicle warning system
`
`are trained to recognize roadway hazards
`
`which the vehicle is approaching,
`
`including
`
`automobiles,
`
`trucks and pedestrians.
`
`"Training involves providing
`
`DUN
`
`\OCOQOWUTFP
`
`10
`
`ll
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`:9
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`
`9‘37 ESQUIRE
`
`
`so.u1.u"s
`
`‘
`
`800.211.DEPO (3376)
`
`EmeeSoMMwmtom
`
`

`

`BHKOLAOSFVUWVMKOLOPOULOS,PHll
`February 24, 2014
`96
`TOYOTA MOTOR VS. AVS
`
`j
`
`
`
`Nikolaos Papanikolopdulos, Ph.D.
`
`known inputs to the network resulting in
`
`desired output responses.
`
`"The weights are automatically
`
`adjusted based on error signal measurements
`
`until the desired outputs are generated.
`
`"Various learning algorithms may
`
`be applied."
`
`And if you look at figure 5 of
`
`Lemelson, basically makes very clear that
`
`the training is done ~- explicitly
`
`disclosed training based on basically data.
`
`Q.
`
`That was wasn't my question.
`
`I
`
`wasn‘t asking you what Lemelson discloses.
`
`I wasn't asking you what does satisfy
`
`generating a trained pattern recognition.
`
`I was asking you specifically:
`
`Are there ways to train a
`
`pattern recognition algorithm that don't
`
`involve generating from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects?
`
`Just based on your knowledge,
`
`your experience, your research.
`
`MR. BERKOWITZ: Object to form.
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`
`
` ESQUIRE
`
`s o .. u T
`
`. o r. s
`
`800.21 1.DEPO (3376)
`EsquireSo/utions.com
`
`

`

`NHfilAOSPAPAMKOLOPOULOSJWiD.
`TOYOTAMOTORVSAVS
`
`February 24, 2014
`97
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`
`
`WM
`
`koooqmmhb
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23'
`
`24
`
`25
`
`A.
`
`Again, it will be highly
`
`speculative because it depends on the
`
`application.
`
`And I will mention, if you go to
`
`my bio, you are going to see I have a paper
`
`about using trainable ~— trained pattern
`
`recognition algorithms to recognize vehicle
`
`occupants.
`
`I think it's journal paper 50
`
`or 51.
`
`The training there involve
`
`images and patterns, so given the analysis,
`
`I cannot think in the context of other
`
`examples.
`
`Q.
`
`Okay.
`
`So you're unable to answer the
`
`question? You don't know whether or not
`
`there are ways that you can train a pattern
`
`recognition algorithm that don't involve
`
`generating it from data of possible
`
`exterior objects and patterns of received
`
`waves from possible exterior objects?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`' You see, given the assumption
`
`that we're dealing with a particular
`
`
` 5” J ESQUIRE
`
`'
`
`s
`
`r.-
`
`.v u T . o N s
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`MKOLAOSPAPAMKOLOPOULOSJWiD
`TOYOEAMOTORVSAVS
`
`February 24, 2014
`102
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`But if you have access to real
`
`data, you‘re going to have useless
`
`information in—useless information out.
`
`Q.
`
`Is it technologically possible
`
`to use simulated data for a pattern
`
`recognition system for training that system
`
`for detecting automobiles?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`If you look at my declaration,
`
`I
`
`mention several systems in this domain.
`
`In this particular domain, you
`
`go to simulated data, or if you don't have
`
`access to real data,
`
`to real images.
`
`So the fact that someone might
`
`go that way will be highly speculative
`
`because ——
`
`Q.
`
`I understand it may not make
`
`sense to you, but is it technologically
`
`possible to use simulated data ——
`
`
`
`A.
`
`Q.
`
`Now? Then?
`
`Now.
`
`Is it possible today to use
`
`simulated data to train a pattern
`
`recognition algorithm to detect an
`
`ES QUIRE
`
`s o L u T
`
`. o u s
`
`800.21 1.DEPO (3376)
`
`EsquireSo/utions.com
`
`DJN
`
`qmmpb
`
`10
`
`11‘
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`103
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`automobile?
`
`A.
`
`It's one of the questions that
`
`is not a yes or no, but sometimes you can
`
`use very advanced simulated data that are
`
`really as close. But also you have very
`
`inexpensive means to access the real data,
`
`the real images, so I don't see any —~
`
`Q.
`
`I'm just asking if it's
`
`technologically possible. Not whether or
`
`not that would be the way that you would do
`
`it.
`
`A.
`
`I suppose.
`
`But you don't want useless stuff
`
`in~useless stuff out.
`
`Q.
`
`Could training a pattern
`
`recognition algorithm with simulated data
`
`involve less data that needs to be stored
`
`on the system?
`
`MR. BERKOWITZ: Object to form.
`
`Q.
`
`As compared to training with
`
`images, for example, real life images.
`
`MR. BERKOWITZ:
`
`Same objection.
`
`
`
`LoooqmmiwaI—l
`mHHHHHHHHHHOKDODQG‘AU'ipD-UJNHO
`
`21
`
`22
`
`23
`
`24:
`
`25
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A.
`‘You use different —— with the
`
` fig; ESQUIRE
`800.211.DEPO (3376)
`
`assumption that you can use different
`
`s o . u T
`
`. a r. s
`
`EsquireSo/uz‘ions.com
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· ·obstructions of images -- in other words,
`
`·3· ·you can use edge images, boundary images,
`
`·4· ·the answer is probably yes.
`
`·5· · · · · · · But, again, it goes back to my
`
`·6· ·earlier comment.· You don't want to have
`
`·7· ·useless stuff in or noise in, because it
`
`·8· ·will be noise out.
`
`·9· · · · Q.· · Back in 1995, would it have been
`
`10· ·technologically possible to train a pattern
`
`11· ·recognition algorithm for detecting
`
`12· ·vehicles using simulated data?
`
`13· · · · A.· · Simulated data at that time,
`
`14· ·technologically possible is a -- let me
`
`15· ·explain to you.
`
`16· · · · · · · If you use a silicon graphics
`
`17· ·machine processor, the data that could be
`
`18· ·produced has nothing to do with what the
`
`19· ·real image would have been.
`
`20· · · · · · · So I can speculate that anyone
`
`21· ·can produce collections of 0s and 1s, but
`
`22· ·if they had any relevance with reality and
`
`23· ·the task is a different story.
`
`24· · · · Q.· · For example, back in 1995, was
`
`25· ·it technologically possible to just train a
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· · · · Q.· · Okay.
`
`·3· · · · · · · Would it have been possible in
`
`·4· ·1995 to use a trained pattern recognition
`
`·5· ·system to detect something other than a
`
`·6· ·vehicle and assume that a vehicle was
`
`·7· ·detected; for example, taillights?
`
`·8· · · · · · · MR. BERKOWITZ:· Object to form.
`
`·9· · · · A.· · Simulated data or real data?
`
`10· · · · Q.· · Either simulated or real.
`
`11· · · · A.· · We have some of the references
`
`12· ·here discussing this, and I think the --
`
`13· ·it's an easier problem than -- the overall
`
`14· ·problem that --
`
`15· · · · Q.· · So in 1995, it would have been
`
`16· ·possible to train a pattern recognition
`
`17· ·algorithm to detect taillights, and from
`
`18· ·that, assume that a vehicle was present?
`
`19· · · · A.· · Can you repeat the question?
`
`20· · · · Q.· · Sure.· In 1995, it was possible
`
`21· ·to train a pattern recognition algorithm to
`
`22· ·detect taillights; and then, from that,
`
`23· ·assume a vehicle was present?
`
`24· · · · A.· · You're asking by saying -- it's
`
`25· ·very easy to confuse -- in cases like this,
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`123
`
`
`
`KDOOQOWU‘IHBUJNH
`HHHHHHHHqmmamNHo
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`less storage space requirements than using
`
`digital image data for vehicle detection?
`
`A.
`
`To me, it makes no sense to use
`
`simulated data in a scenario like this.
`
`Images —~ and the assumption is
`
`
`
`actually,
`
`I'm talking about
`
`'95 —~
`
`the
`
`simulation couldn't have reached the level
`
`of complexity that a picture could provide.
`And if you are a little bit more,
`
`careful with the management of the data and
`
`the processing and the storage of the data,
`
`this was, as it has been demonstrated by
`
`some of the papers that I'm including in my
`
`writeup, you can do it, so I don't see the
`
`reason why you would go to simulated data.
`
`Now, if you take a raw image and
`
`you just save it...
`
`Q.
`
`Could in 1995 simulated data
`
`involve smaller file sizes than digital
`
`image data?
`
`A.
`
`Potentially, yes.
`But, again,
`I don't see -— I
`
`don't see the usefulness.
`
`Again, why do we need to have
`
`
`
`
`ESQUIRE
`
`s o L u T
`
`. a u s
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`·1· · · · Nikolaos Papanikolopoulos, Ph.D.
`
`·2· · · · A.· · So I'm on page 45, above
`
`·3· ·paragraph 110, the Figure 1, that
`
`·4· ·basically, the camera 12 -- to me, this is
`
`·5· ·a disclosure.
`
`·6· · · · Q.· · Again, I wasn't asking about
`
`·7· ·Borcherts.
`
`·8· · · · · · · In your opinion, in a
`
`·9· ·hypothetical prior art reference that
`
`10· ·discloses a receiver that's arranged near
`
`11· ·the rearview mirror, but is definitely not
`
`12· ·on the rearview mirror, is that good
`
`13· ·enough, in your opinion, to satisfy the
`
`14· ·claim requirement in Claim 30 of "at least
`
`15· ·one receiver arranged on a rearview mirror
`
`16· ·of the vehicle"?
`
`17· · · · A.· · Even the claim construction --
`
`18· ·yes.
`
`19· · · · Q.· · So in your opinion, near the
`
`20· ·rearview mirror, that's good enough to
`
`21· ·count as on a rearview mirror?
`
`22· · · · · · · MR. BERKOWITZ:· Object to form.
`
`23· · · · A.· · For someone with ordinary skill
`
`24· ·in the art, I think this will be
`
`25· ·sufficient.
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`162
`
`
`
`KOCOQGNU‘II‘P-UJNI-J
`mNNHHHHHHHHHHmHommqmmswMHo
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`Q.
`
`A.
`
`Q.
`
`Yes,
`
`'057.
`
`Yes.
`
`So,
`
`in particular, you state
`
`that:
`
`"Lemelson discloses a processor that
`
`processes the signals derived from the
`
`camera image(s) and provides a
`
`classification,
`
`identification or location
`
`of the exterior object.
`"In particular, Lemelson teaches
`
`that this can be accomplished using a
`
`neural network. Specifically,
`
`the signal
`
`output from the camera(s)
`
`is digitized and
`
`passed to an image analyzing computer
`
`(which, as set forth in greater detail
`
`below, meets the 'trained pattern
`
`recognition means'
`
`limitation of Claims 1
`
`and 56)."
`
`So,
`
`in addition, as we talked
`
`about earlier, Claim 1 of the '057 patent,
`
`for example, requires, among other things,
`
`a trained patent recognition algorithm;
`
`23~
`
`right?
`
`24
`
`25
`
`A.
`
`Q.
`
`Yes.
`
`So let's go to page 55 of your
`
`
`
`'J‘V‘
`
` ESQUIRE
`
`s a .. u 7
`
`, o r. s
`
`V
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`163
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`declaration.
`
`A.
`
`Q.
`
`Okay.
`
`There you state that:
`
`
`
`"Lemelson explains that neural
`
`networks used in the vehicle warning system
`
`are trained to recognize roadway hazards
`
`which the vehicle is approaching,
`
`including
`
`automobiles,
`
`trucks and pedestrians.
`
`"Training involves providing
`
`known inputs to the network resulting in
`
`desired output responses."
`
`Now,
`
`in your declaration —4 so
`
`then,
`
`in your opinion —~ where Lemelson
`
`describes "providing known inputs," that's
`
`the disclosure that you point to in
`
`Lemelson regarding training with the neural
`
`network;
`
`is that right?
`
`MR. BERKOWITZ: Object to form.
`
`It's one of the places.
`
`Q.
`
`Where else in Lemelson in your
`
`opinion does it discuss training of the
`
`patent recognition algorithm?
`
`Let me give you Lemelson.
`
`I
`
`don't believe it's already in evidence.
`
`g?”4», M
` ESQUIRE
`
`s o L u 1
`
`.
`
`.3 u s
`
`800.211.0590 (3376)
`
`EsquireSo/utions.com
`
`

`

`N IKOLAOS PAPANIKOLOPOULOS, PHD.
`February 24, 2014
`164
`TOYOTAMOTORvaAVS
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`(Toyota Exhibit 1002 for
`
`identification in IPR 419, Lemelson
`
`document.)
`
`BY MR. SCHARFF:
`
`Q.
`
`-
`
`I show you what Toyota has
`
`marked as 1002 in the 419 IPR.
`
`Do you recognize this document?
`
`.Yes.
`
`It is the Lemelson.
`
`Other than the sentence that you
`
`A.
`
`Q.
`
`quote in your declaration that states
`
`“training involves providing known inputs
`
`to the network resulting in desired output
`
`responses," what else in Lemelson in your
`
`opinion relates to training of the pattern
`
`recognition algorithm?
`
`A.
`
`I think, first of all, we have
`
`to mention Figure 1.
`
`I think I copied my declaration,
`
`because it shows a structure and how —— for
`
`example,
`
`the processor means in 19. And
`
`then I want to mention Figure 2.
`
`Q.
`
`Sure.
`
`Before we move on from Figure 1,
`
`there's nothing in Figure 1 that
`
`
`
`LIJN
`
`4mms
`
`(I)
`
`10
`
`ll
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`#7“
`
`
`i‘x ESQUIRE
`
`5 o . u T
`
`. o r. s
`
`
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`165
`
`ODQONU'IeP-UJNH
`
`k0
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`specifically refers to training,
`
`is there?
`
`A.
`
`If you look at image analyzing
`
`computer, and the way that is discussed in
`
`the —— let me find the exact paragraph
`
`somewhere.
`
`If you look at the description
`
`of the drawing, Figure 1 says it's a
`
`"diagram of the overall model, basically
`
`warning and control system,
`
`illustrating
`
`system sensors, computer displays,
`
`inputeoutput device and other key
`
`elements."
`
`So we go to ~-
`
`Q.
`
`Before we move on,
`
`that sentence
`
`that you just read to me ~—
`
`MR. BERKOWITZ: He‘s got to be
`
`able to finish here.
`
`MR. SCHARFF: Well, if he's
`
`going to read ten different sentences, and
`
`then I have to go back to them individually
`
`and ask him questions about that —~
`
`MR. BERKOWITZ: You could do
`
`that at the end.
`
`And referring to the Appendix
`
`I am required to
`that you mentioned before,
`
`
`
`
`
`* ESQUIRE I
`
`s a L u T
`
`. a N s
`
`'
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`February 24, 2014
`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`166
`TOYOTA MOTOR vs. AVS
`
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11‘
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`object
`
`to opposing parties' conduct, so I
`
`do object to the fact that you're cutting
`
`him off from his answers.
`
`MR. SCHARFF:
`
`I let him complete
`
`his sentence. Before he moves on to
`
`another sentence,
`
`I can ask if we can ask
`
`questions about that sentence.
`
`Again,
`
`this isn't an allowable
`
`objection. You're coaching the witness.
`
`MR. BERKOWITZ:
`
`I am actually
`
`required to make that objection.
`
`So let the record reflect that
`
`counsel is cutting the witness off.
`
`MR. SCHARFF: And, again,
`
`let
`
`the record reflect I didn't cut him off in
`
`the middle of a sentence.
`
`MR. BERKOWITZ:
`
`In the middle of
`
`an answer.
`
`BY MR. SCHARFF:
`
`Q.
`
`Let‘s take your support for the
`
`pattern recognition training one item at a
`
`time.
`
`We talked previously about the
`
`. 25
`
`training with known inputs sentence in
`
`
`
`ESQUIRE
`
`s a . u T
`
`.
`
`o N s
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`mmqmmewmr—I
`
`1..) O
`
`ll
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`167
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`Lemelson that you point to in your
`
`declaration.
`
`What's the next instance?
`
`A.
`
`With all due respect, may I
`
`explain a little bit the thought process
`
`here?
`
`Q.
`A.
`
`Okay.
`There are issues of text and
`
`figures that are interconnected, so if you
`
`let me mention the figures and the texts,
`
`then I can answer the questions here and
`
`the logic that I followed here.
`
`Q.
`
`That's fine.
`
`If that's what you
`
`need to do, go ahead.
`
`I was just trying to
`
`make it a little more concise.
`
`A.
`
`So the ones I consider is
`
`actually Figure l and its expected
`
`description; Figure 2 and Figure 5.
`
`And also, as part of this,
`
`the
`
`Figure 3 and 4.
`
`I go to the ~- so column 4,
`
`where it has descriptions of the different
`
`figures. And then I'm using 515 down to
`
`55. And then I'm using 81, and I think I
`
`
`5” ESQUIRE
`
`s o . u T . u r: s
`
`800.211.DEPO (3376)
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR vs. AVS
`
`..
`
`February 24, 2014
`168
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`stopped at row 8.
`
`The idea here is actually trying
`
`to understand how the components are put
`
`together, and also how the different claim
`
`construction in terms of what was given to
`
`me map this combination of text and
`
`figures.
`
`So if you see the column 8 text,
`
`it says some other interesting things that
`
`teach the training based on data of
`
`possible exterior objects.
`
`It says, for example:
`
`"The neural network of the image
`,analysis computer 19" ~— so that's why 1
`
`want to use the Figure 1
`
`~— "provides a
`
`highly parallel image processing structure
`
`with rapid, real~time image recognition
`
`necessary for the Motor Vehicle Warning and
`
`Control System."
`
`"Very Large Scale Integrated
`
`(VLSI) Circuit implementation of the neural
`
`processing elements permits low—cost,
`
`low—weight limitation."
`
`And I want to emphasize this:
`
`.7; ESQUIRE
` w:
`
`s
`
`r:
`
`L u w ‘ a r: s
`
`800.211.0sPo (3376)
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`Febnmwy24,2014
`TOYOTA MOTOR VS. AVS
`169
`
`
`
`
`[\3
`
`DJ
`
`\1CH.U'IEPA
`
`(I)
`
`10
`
`11
`
`12
`
`13'
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`"Also, a neural network has
`certain reliability advantages important in
`
`a safety warning system.
`
`Loss of one
`
`processing element does not necessarily
`
`result in a processing system failure."
`
`In other words, if one of the
`
`elements goes off for whatever reason,
`
`the
`
`system is able to learn online and adopt,
`
`so all these are elements that I took into
`
`consideration.
`
`Now, one important question that
`
`I had to face as part of this is actually
`
`the data and patterns are actually used.
`
`This is for the training. This
`
`is the Figure 5, because it shows an image
`
`processer that receives image data, and an
`
`image data bus that connects the virtual
`
`processing elements,
`
`so all of these were a
`
`part of this.
`
`So I don‘t know if this has been
`
`helpful.
`
`Q.
`
`A.
`
`Are you done with your answer?
`
`One last thing.
`
`Another thing that I had to
`
`
`fig: ESQUIRE
`
`
`s o . u T . o u s
`
`800.211.DEPO (3376)
`
`EsquireSo/utions.com
`
`

`

`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`170
`
`Nikolaos Papanikolopoulos, Ph.D.
`consider as part of the claim was actually
`
`the idea of patterns. And for me, patterns
`
`are actually relationships of pixels.
`
`Q.
`
`Okay.
`
`So let's go to your
`
`declaration.
`
`In your 419 declaration,
`
`paragraphs 47 to 64 relates to anticipation
`
`by Lemelson;
`
`is that right?
`
`A.
`
`Q.
`
`Yes.
`
`And out of those paragraphs in
`
`your declaration,
`
`the only spot where you
`
`refer to "training" is in paragraph 55,
`
`when you quote to where Lemelson says:
`
`”Training involves providing
`
`known inputs to the network resulting in
`
`desired output responses."
`
`MR. BERKOWITZ: Object to form.
`
`Foundation.
`
`A.
`
`There is also various learning
`
`elements may be applied and also the
`
`weights are automatically adjusted.
`
`Q.
`
`I'm talking specifically where
`
`you use the word "training."
`
`That only appears in paragraph
`
`UJN
`
`4mma
`
`10
`
`11
`
`12
`
`13
`
`14c
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24:
`
`25
`
`
`
`
`
`5W ESQUIRE
`
`
`s c:
`
`L u T
`
`. o n s
`
`800.211.0EPO (3376)
`
`EsquireSo/utions.com
`
`

`

`NlKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`17.1
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`55 in the sentence:
`
`"Training involves
`
`providing known inputs to the network
`
`resulting in desired output responses."
`
`Is that correct?
`
`A.
`
`To me, we need to look at
`
`training as part of the learning in the
`
`adjustments of weights, given the structure
`
`that I have here.
`
`Q.
`
`So, again,
`
`the only place
`
`between paragraphs 47 and 64 where you use
`
`the word "training" is in paragraph 55;
`
`is
`
`that correct?
`
`MR. BERKOWITZ: Object to form
`
`and foundation.
`
`
`
`A.
`
`These are components of the
`
`training.
`
`I cannot
`
`ignore this.
`
`I gave
`
`you the line of thought I followed in order
`
`to form an opinion.
`
`Q.
`
`Sure.
`
`I get that. But my
`
`question is:
`
`Is the only instance in which
`
`you use the word "training" in paragraphs
`
`47 to 64 in parag

This document is available on Docket Alarm but you must sign up to view it.


Or .

Accessing this document will incur an additional charge of $.

After purchase, you can access this document again without charge.

Accept $ Charge
throbber

Still Working On It

This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.

Give it another minute or two to complete, and then try the refresh button.

throbber

A few More Minutes ... Still Working

It can take up to 5 minutes for us to download a document if the court servers are running slowly.

Thank you for your continued patience.

This document could not be displayed.

We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.

You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.

Set your membership status to view this document.

With a Docket Alarm membership, you'll get a whole lot more, including:

  • Up-to-date information for this case.
  • Email alerts whenever there is an update.
  • Full text search for other cases.
  • Get email alerts whenever a new case matches your search.

Become a Member

One Moment Please

The filing “” is large (MB) and is being downloaded.

Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!

If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document

We are unable to display this document, it may be under a court ordered seal.

If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.


Access Government Site

We are redirecting you
to a mobile optimized page.





Document Unreadable or Corrupt

Refresh this Document
Go to the Docket

We are unable to display this document.

Refresh this Document
Go to the Docket