`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`1
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_ _ _ _ _ _ _ _ _ _ . . _ _ _ . _ . . . . . _ . . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _1X
`TOYOTA MOTOR CORPORATION,
`
`VS.
`
`Petitioner,
`
`IPR2013-00412
`IPR2013-00413
`IPR2013-00416
`
`AMERICAN VEHICULAR SCIENCES, LLC,
`
`Patent Owner.
`_ _ _ _ _ _ . _ _ _ _ . _ _ _ _ 1 _ _ _ 1 . _ _ . . . . _ _ _ _ . _ _ . _ _ _ -_X
`
`DEPOSITION OF
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`
`Monday, February 24, 2014
`
`New York, New York
`
`Reported By:
`
`LINDA J. GREENSTEIN
`
`JOB NO.
`
`89371
`
`
`
`ESQUIRE
`
`SDLUIIONS
`
`800.211.DEPO (3376)
`Esquiresolutions. com
`
`CORRECTED
`AVS EXHIBIT 2002 .
`
`Toyota Inc. v. American Vehicular Sciences LLC
`IPR2013—OO419
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`MKOLAOSPAPANMOLOPOULOS,M+D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`34
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`exploited ~~ we had a camera mounted inside
`
`the car, and we use a location of the
`
`license plate with respect to the rest of
`
`the car, and we tried to compute also other
`
`characteristics; say, characteristics of
`
`the license plate.
`
`And then based on the size, we
`try to keep the size constant, it allows us
`
`to keep constant distance as part of a
`
`Vehicle following application.
`
`Q.
`
`Did that system use a pattern
`
`recognition algorithm?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`Again,
`
`let me clarify since we
`
`have these terms in my —— the claim
`
`construction I was given.
`
`Do you want me
`
`to answer --
`
`Q.
`
`As you normally use that term,
`
`did that system,
`
`the one that you refer to
`
`in paragraph 4 of your 419 application, use
`
`a pattern recognition algorithm?
`
`MR. BERKOWITZ: Object to form.
`
`Lack of foundation.
`
`2
`
`A.
`
`Normally use —- again, under the
`
`10
`
`ll
`
`l2
`
`l3
`
`:5
`
`2.6
`
`2.7
`
`19
`
`2O
`
`21
`
`22
`
`23
`
`24
`
`25
`
` ESQUIRE
`
`., 0 ..
`
`U T
`
`. 0 N
`
`800.211.DEPO (3376)
`EsquireSoluz‘ions.com
`
`
`
`NIKOLAOS PAPAMKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`35
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`assumption of the claim construction that I
`
`was given,
`
`the answer is yes.
`
`Q.
`
`What about under your ordinary
`
`usage of that term?
`
`MR. BERKOWITZ: Object to form
`
`and lack of foundation.
`
`A.
`
`By using the claim construction
`
`of pattern recognition, yes.
`
`Q.
`
`I wasn't asking about the claim
`
`construction in this case.
`
`I was just asking about as you
`
`would normally use that term in your
`
`research, did the system of paragraph 4 use
`
`a pattern recognition algorithm?
`
`MR. BERKOWITZ: Object to form.
`
`Lack of foundation.
`
`A.
`
`Given the precision that I need
`
`to provide with respect —~ since we have
`
`many terms which are used,
`
`I think I need
`
`to stick to use of the term in my
`
`declaration.
`
`So in this case, yes.
`
`Q.
`
`So you've used the term "pattern
`
`recognition" in some of your papers,
`
`haven't you?
`
`
`V‘‘$4
`
`
`
` ESQUIRE
`
`.:O1.UTlC|NS
`
`800.211.DEPO (3376)
`Esquiresolut/'ons.com
`
`M k
`
`DCDQO\U'!I«l>Ld
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`2O
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`part of the claim construction,
`
`so I'm
`
`going to ask you the same clarification.
`
`Q.
`
`First, as you've construed the
`
`claims in this case, did your license plate
`
`system involve trained pattern recognition?
`
`MR. BERKOWITZ: Object to form
`
`and foundation.
`
`A.
`
`So given the claim construction
`
`I have here, we didn't use trained pattern
`
`recognition means.
`
`Q.
`
`And then as you used that term
`
`before these IPR matters, would you have
`
`said that that used trained pattern
`
`recognition?
`
`MR. BERKOWITZ: Object to form
`
`KOm\10‘\U1
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17.
`
`and lack of foundation.
`
`18
`
`19
`
`2O
`
`21
`
`22
`
`23
`
`24
`
`25
`
`A.
`
`The term "trained pattern
`
`recognition" is not really common.
`
`I mean,
`
`we use other terms,
`
`so I would say no.
`
`Q.
`
`Okay.
`
`How did the system detect a
`
`license plate?
`
`A.
`
`So I can give you my
`
`recollection.
`
`
`gfii
`
`ESQUIRE
`
`.>0LUTIONS
`
`800.211.DEPO (3376)
`EsquireSolutions.com
`
`
`
`MKOLAOSPAPAMKOLOPOULOSJWiD
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`46
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`perimeter of the license plate,
`
`so it was a
`
`scale or a number,
`
`to I don't want to be --
`
`but mainly images.
`
`Q.
`
`Okay.
`
`And did you have on the system
`
`any data or images relating to license
`
`plates?
`
`A.
`
`I don't recall this,
`
`to be
`
`precise.
`
`One thing I recall is actually
`
`we used to have rectangles, or models of
`
`rectangles.
`
`I don't recall exactly if we had
`
`the license plates or —- and also,
`
`I'm not
`
`sure in Yue Du's thesis —— I asked him do
`
`this, but I'm not sure if this was ever
`
`implemented.
`
`Q.
`
`So it's possible in a system of
`
`this type to have the system look for
`
`rectangles rather than specifically compare
`
`captured images to stored images of license
`
`plates?
`
`MR. BERKOWITZ: Object to form.
`
`Foundation.
`
`[\J
`
`O‘\U1r-I>
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`2O
`
`21
`
`22
`
`23
`
`24
`
`25
`
`ESQUIRE
`
`.>O(UTl0N5
`
`_.._____l
`
`800.211.DEPO (3376)
`EsquireSoIutions.com
`
`
`
`omm4mm9mNH
`
`E—‘
`
`F‘ I-‘
`
`I-’ N
`
`!-‘ U3
`
`!-‘ 1-P-
`
`F4 U‘!
`
`i-‘ O'\
`
`I-‘ \l
`
`1"’ CD
`
`I-‘ LO
`
`[\J O
`
`N 1-‘
`
`N N
`
`[0 LA}
`
`24
`
`25
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTAMOTORVaAYS
`
`February 24, 2014
`48
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`In a system like that, could you
`
`have used simulated data rather than
`
`images?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`Under the assumption that
`
`different situations, different
`
`applications might require different «-
`
`it's a possibility.
`
`But for us, it was a
`
`requirement, a contractual requirement,
`
`to
`
`demonstrate something like this.
`
`Q.
`
`Okay.
`
`MR. BERKOWITZ: Whenever it's a
`
`convenient time, we've been going around an
`
`hour and ten.
`
`BY MR. SCHARFF: .
`
`Q.
`
`Earlier I asked you about the
`
`claim constructions that you applied in
`
`your declaration, and you said that those
`
`were provided to you.
`
`Were those provided to you by
`
`the attorneys for Toyota?
`
`A.
`
`Yes.
`
`From Kenyon & Kenyon,
`
`just
`
`to be --
`
`ESQUIRE
`
`SOLUTIITNS
`
`' 800.211.DEPO (3376)
`EsquireSoIutions.c:om
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`Februany24,2014
`89
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`i
`
`Q.
`
`So the claim doesn‘t just say "a
`
`trained pattern recognition algorithm
`generated from examples," does it?
`
`A.
`
`Q.
`
`It doesn't say.
`
`You understand then that Claim 1
`
`requires that the trained pattern
`
`recognition algorithm be generated from.
`
`data of possible exterior objects and
`
`‘patterns of received waves from the
`possible exterior objects; correct?
`
`A.
`
`Q.
`
`Yes.
`
`Not every trained pattern
`
`recognition algorithm would be generated
`
`using those specific requirements, would
`
`they?
`9
`
`A
`
`MR. BERKOWITZ: Object to form.
`
`Foundation.
`
`A.
`
`Can you clarify, because this is
`
`a little bit broad?
`
`I would say it's too
`
`broad.
`
`Q.
`
`What do you regard as the
`
`difference between just a trained pattern
`recognition algorithm in general, versus a
`trained pattern recognition algorithm
`
`\1G\U’lI~l>LaUI.\)!-‘
`
`10
`
`ll
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
` ESQUIRE
`
`SDLUTIDN-5
`
`800.211.DEPO (3376)
`EsquireSoIutions. com
`
`
`
`CJ\OUD\IO\U‘|x-¥>(.rJl\)I-‘
`
`I--‘
`
`1-‘ 1-’
`
`I-‘ N
`
`E-‘ OJ
`
`H 51>
`
`}-‘ U1
`
`1-‘ Ch
`
`I—‘ <1
`
`!-‘ 03
`
`I-‘ K0
`
`N O
`
`[U I-‘
`
`[U N
`
`[0 DJ
`
`24
`
`25
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR Vs. AVS
`
`February 24, 2014
`90
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`generated from data of possible exterior
`
`objects and patterns of received waves from
`
`the possible exterior objects?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`You see, again,
`
`I have to stick
`
`to_the construction that was given, so I
`
`checked the trained pattern recognition
`
`algorithm, as you stated.
`
`Q.
`
`I'm sorry,
`
`that wasn't my
`
`question.
`
`Can you think of examples of a
`
`trained pattern recognition algorithm that
`
`are not generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects.
`
`MR. BERKOWITZ: Object to form.
`
`MR. SCHARFF:
`
`I'll restate.
`
`BY MR..SCHARFF:
`
`Q.
`
`So can you think of an example
`
`of a trained pattern recognition algorithm
`
`that is not generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects?
`
`A.
`
`With respect to the specific
`
`
`
`ESQUIRE
`SOLUTIIINS
`
`..=,’.'
`-V‘
`X
`
`800.211.DEPO (3376)
`Esquireso/utionacom
`
`
`
`IWKOLAOSPARMWKOLOPOULOS,MiD.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`91
`
`O\0C0\‘!0\U'It~l>-UJEOE-‘
`
`l-‘
`
`I-‘ (-4
`
`E-‘ N
`
`I-‘ UJ
`
`i-’ IP-
`
`|-'‘ U’!
`
`1-‘ G1
`
`I-‘ \1
`
`I—‘ CC
`
`I-‘ L0
`
`[0 O
`
`N !-‘
`
`[\J N
`
`[0 DJ
`
`M 14>
`
`[U U’!
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`claim construction that I had to examine,
`
`no.
`
`Q.
`
`I'm sorry, it was an open~ended
`
`question.
`
`So you can't think of any
`
`examples of a trained pattern recognition
`
`algorithm that are not generated from data
`
`of possible exterior objects and patterns
`of received waves from.the possible
`
`exterior objects?
`
`A.
`
`Again, my answer is with respect
`
`to this particular claim construction that
`
`I had to examine,
`
`I had to look at the
`
`whole claim.
`
`The claim construction I was
`
`given,
`
`I had to offer an opinion.
`
`Q.
`
`You have a claim construction
`
`for trained pattern recognition algorithm,
`
`but I'm asking you about the next phrase.
`
`A.
`
`Q.
`
`And this is the one --
`
`Did you ignore, for purposes of
`
`your opinions,
`
`the phrase "generated from
`
`data of possible exterior objects and
`
`patterns of received waves from the
`
`possible exterior objects"?
`
`ESQUIRE
`
`SOLUTIONS
`
`I>‘~
`
`800. 21 1.DEPO (3376)
`Esquiresolutionaoom
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`Februany24,2014
`92_
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`A.
`
`No.
`
`It is included in my
`
`declaration. That is a statement from
`
`Lemelson which addresses this.
`
`So if you look at my declaration
`
`—— this is the ‘O00 ~~ you want me to do
`
`the 'OOO?
`
`Q.
`
`The ‘O57. We're on the ‘D57
`
`patent right now.
`
`The IPR.
`
`A.
`
`Q.
`
`Which one do you want?
`
`For the 'o57 patent.
`
`In your opinion, Lemelson
`
`discloses:
`
`"A trained pattern recognition
`
`algorithm generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects."
`
`Is that right?
`
`A.
`
`What I am saying is actually
`
`Lemelson explicitly teaches ~— explicitly
`
`discloses the teaching based on data of
`
`possible exterior objects, and I have a
`
`statement to that effect.
`
`So I have to look at the claim
`
`construction that was given and the
`
`particular claims in order to ~-
`
`GLD03\10\Ln_s£>bJ[\3!—‘
`
`I-‘
`
`I-‘ l_,.
`
`F‘ M
`
`F‘ L1.)
`
`E-‘ H’-5
`
`I-4 U‘!
`
`{-4 CW
`
`E-‘ \'I
`
`P‘ CO
`
`i—‘ K!)
`
`N O
`
`N l—'
`
`N N
`
`23
`
`24
`
`25
`
`a ESQUIRE
`SOLUTIONS
`
`800.211.DEPO (3376)
`Esqu/reSoIutions. com
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`.
`
`'
`
`February 24, 2014
`93
`
`A 1
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`Q.
`
`So I'm asking you, what type of
`
`trained pattern recognition algorithm is
`
`not generated from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects?
`
`MR. BERKQWITZ: Objection.
`
`Foundation;
`
`A.
`
`My reply to this is that I was
`
`not asked this.
`
`VI was asked to examine ——
`
`Q
`
`I understand.
`
`I think that‘s the difficultly
`
`14 with you responding to my questions.
`
`15
`
`I-‘ Ch
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`I'm allowed to ask you these
`
`questions, and I understand you weren't
`
`previously asked this specific type of
`
`question, but I'm asking this now, and I'm
`
`asking what your answer is.
`
`And you can't just say:
`
`I don't
`
`have an answer because I wasn't asked this
`
`previously or because it wasn't this
`
`specific question that you addressed with
`
`respect to Lemelson.
`
`My question is:
`
`
`
`rgyx‘
`
`s u 1. u T 1 u u 5
`
`.
`
`300.211.0500 (3376)
`
`ESC]UI'I‘eS'O/ul‘iOI75'.COITl
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR vs. AVS
`
`'
`
`February 24, 2014
`94
`
`Nikolaos Papanikolopoulos, Ph.D.'
`
`Are there trained pattern
`
`recognition algorithms that are not
`
`generated from data of possible exterior
`
`objects and patterns of received waves from
`
`the possible exterior objects?
`
`MR. BERKOWITZ: Object to iforml‘
`
`Foundation.
`
`A.
`
`You see, it would be highly
`
`speculative on my part to enumerate cases
`
`like this when I have —— my declaration is
`
`very specific.
`
`Q.
`
`I'm sorry, but you do have to
`
`answer the question.
`
`MR. BERKOWITZ: Object as
`
`argumentative.
`
`I think he is answering the
`
`question.
`
`MR. SCI-IARFF: No, he isn't.
`
`He's saying he's refusing to
`
`answer it because it wasn't part of the
`
`specific analysis he did from Lemelson.
`MR’. BERKOWITZ:
`I disagree with
`
`all that.
`
`MR. SCHARFF: And, by the way,
`
`that isn't an allowed objection under the
`
`...fjfi
`
`5 0 .
`
`.. T .
`
`.7 .. 5
`
`t
`
`*
`
`800.211.DEPO (3376)
`Esqulresolutionacom
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PHD’.
`TOYOTAMOTORvaAVS
`
`February24,2014
`.
`V95
`
`Nikolaos éapanikolopoulos, Ph.D.
`
`rules.
`
`
`
`
`MR. BERKOWITZ:
`
`If you start to
`
`argue with my witness,
`
`I can make that
`
`objection.
`
`
`
` MR. SCHARFF:
`MR. BERKOWITZ:
`
`It's not in the --
`
`
`In the Appendix?
`
`
`
`I disagree with that.
`
`Can you read the question back.
`
`(Requested portion of record
`
`read.)
`
`BY MR. SCHARFF:
`
` Q.
`yes—or~no answer.
`
`That should be a simple
`
`Are there or aren't there?
`
`
`
`
`MR. BERKOWITZ: Object to form.
` A.
`
`So let me state, page ~~
`this is from
`paragraph 55 of my ‘O57,
`
`
`
`column 8,
`row 1 through 8.
`"Lemelson explains that neural
`
`
`
`
`networks used in the vehicle warning system
`
`are trained to recognize roadway hazards
`
`which the vehicle is approaching,
`
`
`automobiles,
`trucks and pedestrians.
`"Training involves providing
`
`including
`
`
`
`ESQUIRE
`
`'SOLU'\'lDHS
`
`800.211.DEPO (3376)
`Esquiresolutionacom
`
`
`
`CDKOCOQ01U1i«I>LO[0P-’
`
`f_,.
`
`I-‘ l—‘
`
`I-‘ N
`
`1-‘ DJ
`
`1-4 11>-
`
`1*" U1
`
`1-‘ G\
`
`1"‘ \'l
`
`I-‘E-1KO-03
`
`M 0
`
`DO I-‘
`
`[0 N
`
`N DJ
`
`[\J 14>-
`
`[0 U1
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`Februany24,2014
`96
`
`
`Nikolaos Papanikolopoulos, Ph.D.
`known inputs to the network resulting in
`
`
`
`desired output responses.
`
`"The weights are automatically
`
`adjusted based on error signal measurements
`until the desired outputs are generated.
`
`"Various learning algorithms may
`
`be applied."
`
`And if you look at figure 5 of
`
`Lemelson, basically makes very clear that
`
`the training is done ~— explicitly
`
`disclosed training based on basically data.
`
`Q.
`
`That was wasn't my question.
`
`I
`
`wasn't asking you what Lemelson discloses.
`
`I wasn't asking you what does satisfy
`
`generating a trained pattern recognition.
`
`
`
`I was asking you specifically:
`
`Are there ways to train a
`
`pattern recognition algorithm that don't
`
`involve generating from data of possible
`
`exterior objects and patterns of received
`
`waves from the possible exterior objects?
`
`Just based on your knowledge,
`
`your experience, your research.
`
`MR. BERKOWITZ: Object to form.
`
`ESQUIRE
`
`SOLUTIONS
`
`800.211.DEPO (3376)
`EsqulreSoIL/tionacom
`
`OK0OO~\‘lChU114>UJI-*‘
`
`I-‘
`
`i--1 l-‘
`
`H N
`
`|—‘ La)
`
`I--‘ I45
`
`I-4 U1
`
`!-‘ Cfi
`
`F-‘ \]
`
`F‘ 0.3
`
`s-3 Lo
`
`M O
`
`[\J I--‘
`
`22
`
`23
`
`24.‘
`
`25
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`97
`
`kD03\3O‘\U1x-I5-(.U[\Jl—‘
`
`1D
`
`11
`
`12
`
`13
`
`14:
`
`15
`
`16
`
`1'7
`
`18
`
`19
`
`20
`
`_21
`
`22
`
`23'
`
`24
`
`2.5
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`A.
`
`Again, it will be highly
`
`speculative because it depends on the
`
`application.
`
`And I will mention, if you go to
`
`my bio, you are going to see I have a paper
`
`about using trainable ~— trained pattern
`
`recognition algorithms to recognize vehicle
`
`occupants.
`
`I think it's journal paper 50
`
`or 51.
`
`‘The training there involve
`
`images and patterns,
`
`so given the analysis,
`
`I cannot think in the context of other
`
`examples.
`
`Q.
`
`Okay.
`
`So you're unable to answer the
`
`question? You don‘t know whether or not
`
`there are ways that you can train a pattern
`
`recognition algorithm that don't involve
`
`generating it from data of possible
`
`exterior objects and patterns of received
`
`waves from possible exterior objects?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`' You see, given the assumption
`
`that we're dealing with a particular
`
`ESQUIRE
`5U|.UTIl7NS
`
`800.21 1.DEPO (3376)
`EsquireSolutions.com
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`Februany24,2014
`102
`
`Nikolaos Papanikolopoulos, Ph.D.
`But if you have access to real
`data, you're going to have useless
`
`information in—useless information out.
`
`Q.
`
`Is it technologically possible
`
`to use simulated data for a pattern
`
`recognition system for training that system
`
`for detecting automobiles?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`If you look at my declaration,
`
`I
`
`mention several systems in this domain.
`
`In this particular domain, you
`
`go to simulated data, or if you don't have
`
`access to real data,
`
`to real images.
`
`So the fact that someone might
`
`go that way will be highly speculative
`
`because --
`
`Q.
`
`I understand it may not make
`
`sense to you, but is it technologically
`
`possible to use simulated data ——
`
`A.
`
`Q.
`
`Now? Then?
`
`Now.
`
`
`
`Is it possible today to use
`
`simulated data to train a pattern
`
`recognition algorithm to detect an
`
`
`ESQUIRE
`SOLUTIONS
`
`800.21 1.DEPO (3376)
`EsquireSoIu1‘ions.C0m
`
`03\1G\U‘lrDL-J[\J!-‘
`
`10
`
`ll-
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`103
`
`
`
`}..1.0K0CO\1G'\U‘!e~¥>OJ[0I-‘
`
`I--‘ l-‘
`
`E-‘ N
`
`I-" OJ
`
`1-‘ 1-D-
`
`I-4 U‘!
`
`2-‘ ON
`
`1-‘ <1
`
`l—‘ 03
`
`l-' KO
`
`[U C)
`
`M §-‘
`
`N N
`
`N U.)
`
`[U 14>-
`
`[0 U1
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`automobile?
`
`A.
`
`It's one of the questions that
`
`is not a yes or no, but sometimes you can
`
`use very advanced simulated data that are
`
`really as close. But also you have very
`
`inexpensive means to access the real data,
`
`the real images, so I don't see any ~-
`
`Q.
`
`I'm just asking if it's
`
`technologically possible. Not whether or
`
`not that would be the way that you would do
`
`it.
`
`I suppose.
`
`But you don't want useless stuff
`
`in~useless stuff out.
`
`Q.
`
`Could training a pattern
`
`recognition algorithm with simulated data
`
`involve less data that needs to be stored
`
`on the system?
`
`MR. BERKOWITZ: Object to form.
`
`Q.
`
`As compared to training with
`
`images, for example, real life images.
`
`MR. BERKOWITZ:
`
`Same objection.
`
`A.
`
`‘You use different —— with the
`
`assumption that you can use different
`
`ESQUIRE
`SULUTIDNS
`
`800.211.DEPO (3376)
`EsqulreSo/utions.com
`
`
`
`NMOLAOSPAPANMOLOPOULOS,MiD.
`TOYOTAMOTORvsAVS
`
`February 24, 2014
`104
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`obstructions of images —— in other words,
`
`you can use edge images, boundary images,
`
`the answer is probably yes.
`
`But, again, it goes back to my
`
`earlier comment. You don't want to have
`
`useless stuff in or noise in, because it
`
`will be noise out.
`
`Q.
`
`Back in 1995, would it have been
`
`technologically possible to train a pattern
`
`recognition algorithm for detecting
`
`vehicles using simulated data?
`
`A.
`
`Simulated data at that time,
`
`technologically possible is a —— let me
`
`explain to you.
`
`If you use a silicon graphics
`
`machine processor,
`
`the data that could be
`
`produced has nothing to do with what the
`
`real image would have been.
`
`So I can speculate that anyone
`
`can produce collections of Os and ls, but
`
`if they had any relevance with reality and
`
`the task is a different story.
`
`Q.
`
`For example, back in 1995, was
`
`it technologically possible to just train a
`
`[U
`
`OO\10‘\U1!-D
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`,5:
`
`
`r 4
` ESQUIRE
`
`.>U1.UTlC|NS
`
`800.211.DEPO (3376)
`EsquireSolutions.com
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`108
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`Q.
`
`Okay.
`
`Would it have been possible in
`
`1995 to use a trained pattern recognition
`
`system to detect something other than a
`
`vehicle and assume that a vehicle was‘
`
`detected; for example,
`
`taillights?
`
`MR. BERKOWITZ: Object to form.
`
`Simulated data or real data?
`
`Either simulated or real.
`
`We have some of the references
`
`A.
`
`Q.
`
`A.
`
`here discussing this, and I think the --
`
`it's an easier problem than —~
`
`the overall
`
`problem that -~
`
`Q.
`
`So in 1995, it would have been
`
`possible to train a pattern recognition
`
`algorithm to detect taillights, and from
`
`that, assume that a vehicle was present?
`
`A.
`
`Q.
`
`Can you repeat the question?
`
`Sure.
`
`In 1995, it was possible
`
`to train a pattern recognition algorithm to
`
`detect taillights; and then,
`
`from that,
`
`assume a Vehicle was present?
`
`A.
`
`You're asking by saying -— it's
`
`very easy to confuse -— in cases like this,
`
`[\)
`
`\lO\U'1I-I>-
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`ESQUIRE
`
`.. o ». u T
`
`» o N
`
`800.211.DEPO (3376)
`Esc/uireSoIutions.com
`
`
`
`MKOLAOSPAPAMKOLOPOULOS,PHD.
`February 24, 2014
`123
`TOYOTAMOTORVSAVS
`
`
`CDLDCD\'IO\U’ls-i>l.zJL\J!'-‘
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`less storage space requirements than using
`
`digital image data for vehicle detection?
`
`A.
`
`To me, it makes no sense to use
`
`simulated data in a scenario like this.
`
`Images —~ and the assumption is
`
`actually,
`
`I'm talking about
`
`'95 ~— the
`
`simulation couldn't have reached the level
`
`of complexity that a picture could provide.
`
`And if you are a little bit more.
`
`careful with the management of the data and
`the processing and the storage of the data,
`
`this was, as it has been demonstrated by
`
`some of the papers that I'm_including in my
`
`writeup, you can do it,
`
`so I don't see the
`
`reason why you would go to simulated data.
`
`Now, if you take a raw image and
`
`you just save it...
`
`Q.
`
`Could in 1995 simulated data
`
`involve smaller file sizes than digital
`
`image data?
`
`A.
`
`Potentially, yes.
`But, again,
`I don't see ~— I
`
`don't see the usefulness.
`
`Again, why do we need to have
`
`
`
`ESQUIRE
`
`SOLUTIONS
`
`800.21 1.DEPO (3376)
`Esqu1reSo/utions.com
`
`
`
`[\)
`
`\'IO\U’lsJ>UJ
`
`1O
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`Nl KOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`143
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`A.
`
`So I'm on page 45, above
`
`paragraph 110,
`
`the Figure 1,
`
`that
`
`basically,
`
`the camera 12 —-
`
`to me,
`
`this is
`
`a disclosure.
`
`Q.
`
`Again,
`
`I wasn't asking about
`
`Borcherts.
`
`In your opinion,
`
`in a
`
`hypothetical prior art reference that
`
`discloses a receiver that's arranged near
`
`the rearview mirror, but is definitely not
`
`on the rearview mirror,
`
`is that good
`
`enough,
`
`in your opinion,
`
`to satisfy the
`
`claim requirement
`
`in Claim 30 of "at least
`
`one receiver arranged on a rearview mirror
`
`of the vehicle"?
`
`A.
`
`Even the claim construction ——
`
`yes.
`
`Q.
`
`So in your opinion, near the
`
`rearview mirror,
`
`that's good enough to
`
`count as on a rearview mirror?
`
`MR. BERKOWITZ: Object to form.
`
`A.
`
`For someone with ordinary skill
`
`in the art,
`
`I think this will be
`
`sufficient.
`
`L.
`J2‘
`$71
`
`
`
`ESQUIRE
`
`SOLUTIONS
`
`800.211.DEPO (3376)
`Esquiresolutionacom
`
`
`
`IWKOLAOSPAPANWIKOPOULOS,PHD.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`155
`
`O\DC0\'iChU11¥>LIJl\3l-‘
`
`F‘
`
`I-‘ I-‘
`
`H M
`
`I-3 DJ
`
`I-4 H3
`
`1-‘ U1
`
`I--‘ 01
`
`F‘ x1
`
`I-‘ 03
`
`I-4 \D
`
`M O
`
`N I-4
`
`N N
`
`[0 DJ
`
`[\J t-¥>
`
`M 01
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`A.
`
`Q.
`
`I stick to my previous answer.
`
`If I put the receiver on the
`
`back bumper of the car, can you tell me if
`
`that's arranged on the rearview mirror?
`
`A.
`
`Q.
`
`What is the size in this case?
`
`Any receiver.
`
`Any receiver that's positioned
`on the back bumper of a car,
`is that
`
`positioned on the rearview mirror?
`
`A.
`
`Is this supposed to be a
`
`receiver that's too big to fit on the rear
`
`end of the car?
`
`Q.
`
`-I'm not asking anything like
`
`If I point to a car —— I put a
`
`car in front of you.
`
`I point to the camera
`
`that's on the rear bumper and I ask you:
`
`"Is that camera on the rear
`
`bumper on the rearview mirror?", you can't
`
`answer yes or no?
`
`A.
`
`Because there is no yes~or-no
`
`answer on this.
`
`MR. SCI-IARFF: All right.
`
`Thank you. Let's take a break
`
`ESQUIRE
`
`SULUTIVNE
`
`800.21 1,DEPO (3376)
`Esquiresolutionacom
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, F'H.D.
`TOYOTA MOTOR vs. AVS
`
`Februany24,2D14
`162
`
`
`
` Nikolaos Papanikolopoulos,
`
`Q.
`Yes;
`A.
`Yes.
`
`Ph.D.
`
`‘D57.
`
`O\O0D\‘IO’\U1i¥>-LULUI-J
`
`!—’
`
`I-4 H
`
`H}-‘LU-Ex)
`
`I-4 rb
`
`Q.
`
`So,
`
`in particular, you state
`
`that:
`
`"Lemelson discloses a processor that
`
`processes the signals derived from the
`
`camera image(s) and provides a
`
`classification,
`
`identification or location
`
`of the exterior object.
`"In particular, Lemelson teaches
`
`that this can be accomplished using a
`
`neural network. Specifically,
`
`the signal
`
`output from the camera(s)
`
`is digitized and
`
`passed to an image analyzing computer
`
`l5,
`
`(which, as set forth in greater detail
`
`l6
`
`l7
`
`l8
`
`l9
`
`20
`
`2l
`
`22
`
`below, meets the.'trained pattern
`
`recognition means’
`
`limitation of Claims 1
`
`and 56)."
`
`So,
`
`in addition, as we talked
`
`about earlier, Claim 1 of the ‘O57 patent,
`
`for example, requires, among other things,
`
`a trained patent recognition algorithm;
`
`23-
`
`right?
`
`24.
`
`25
`
`A.
`
`Q.
`
`Yes.
`
`So let's go to page 55 of your
`
`
`
`
`‘ ES QUIRE
`SULUT!0NS
`
`800.211.DEPO (3376)
`EsqulreSolutions.com
`
`*‘
`
`
`
`N!KOLAOS PAPVANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`Februany24,2D14
`168
`
`omm4om9wmH
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`declaration.
`
`A.
`
`Q.
`
`Okay.
`
`There you state that:
`
`"Lemelson explains that neural
`
`networks used in the vehicle warning system
`
`are trained to recognize roadway hazards
`
`which the vehicle is approaching,
`
`including
`
`automobiles,
`
`trucks and pedestrians.
`
`"Training involves providing
`
`known inputs to the network resulting in
`
`desired output responses."
`
`Now,
`
`in your declaration -4 so
`
`then,
`
`in your opinion —— where Lemelson
`
`describes "providing known inputs," that's
`
`the disclosure that you point to in
`
`Lemelson regarding training with the neural
`
`network;
`
`is that right?
`
`MR. BERKOWITZ: Object to form.
`
`It's one of the places.
`
`Where else in Lemelson in your
`
`A.
`
`Q.
`
`22
`
`23
`
`24;
`
`25
`
`opinion does it discuss training of the
`
`patent recognition algorithm?
`
`Let me give you Lemelson.
`
`I
`
`don't believe it's already in evidence.
`
` ESQUIRE
`
`SDLIJTIOHIE
`
`‘ 800.211.DEPO (3376)
`Esquireso/utions. com
`
`
`
`N IKOLAOS PAPANIKOLOPOU LOS, PH.D.
`TOYOTAMOTORV&AVS
`
`February 24, 2014
`164
`
`OK0103\1ChU114>Us)[0I-’~
`
`I-‘
`
`E-‘ I-‘
`
`l-‘ N
`
`1-‘ DJ
`
`I--‘ H5
`
`1-‘ U1
`
`1-‘ Ch
`
`I-’ \1
`
`I--‘ CO
`
`l---‘ K!)
`
`N O
`
`[U l--‘
`
`[U M
`
`[0 U3
`
`I\J IJ>
`
`M U1
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`(Toyota Exhibit 1002 for
`
`identification in IPR 419, Lemelson
`
`document.)
`
`BY MR . SCHARFF:
`
`Q.
`
`~
`
`I show you what Toyota has
`
`marked as 1002 in the 419 IPR.
`
`Do you recognize this document?
`
`.Yes.
`
`It is the Lemelson.
`
`Other than the sentence that you
`
`A.
`
`Q.
`
`quote in your declaration that states
`
`"training involves providing known inputs
`
`to the network resulting in desired output
`
`responses," what else in Lemelson in your
`
`opinion relates to training of the pattern
`
`recognition algorithm?
`
`A.
`
`I think, first of all, we have
`
`to mention Figure 1.
`
`I think I copied my declaration,
`
`because it shows a structure and how —— for
`
`example,
`
`the processor means in 19. And
`
`then I want to mention Figure 2.
`
`Q.
`
`Sure.
`
`Before we move on from Figure 1,
`
`there's nothing in Figure I that
`
`ESQUIRE
`
`50l.lITI\IHS
`
`800.21 1.DEPO (3376)
`Esquiresolutionaoom
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`165
`
`.
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`specifically refers to training,
`
`is there?
`
`A.
`
`If you look at image analyzing
`
`computer, and the way that is discussed in
`
`the —— let me find the exact paragraph
`
`somewhere.
`
`If you look at the description
`
`of the drawing, Figure 1 says it's a
`
`"diagram of the overall model, basically
`
`warning and control system,
`
`illustrating
`
`system sensors, computer displays,
`
`input—output device and other key
`
`elements."
`
`So we go to ~-
`
`Q.
`
`Before we move on,
`
`that sentence
`
`that you just read to me --
`
`MR. BERKOWITZ: He's got to be
`
`able to finish here,
`
`MR. SCHARFF: Well, if he's
`
`going to read ten different sentences, and
`
`then I have to go back to them individually
`
`and ask him questions about that --
`
`MR. BERKOWITZ: You could do
`
`that at the end.
`
`And referring to the Appendix
`
`that you mentioned before,
`
`I am required to
`
`5 I1
`
`I. u T .
`
`r:
`
`.4 5
`
`'
`
`'
`
`.
`
`800.211.DEPO (3376)
`
`Esquiresolutionecom
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR vs. AVS
`
`February 24, 2014
`166
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`
`
`to opposing parties‘ conduct, so I
`
`object
`
`do object to the fact that you're cutting
`
`him off from his answers.
`
`MR. SCHARFF:
`
`I let him complete
`
`
`
`
`his sentence. Before he moves on to
`
`
`
`
`another sentence,
`
`I can ask if we can ask
`
`
` Again,
`
`
`objection. You're coaching the witness.
`
`
` MR. BERKOWITZ:
`
`
`questions about that sentence.
`
`this isn't an allowable
`
`MR. BERKOWITZ:
`
`I am actually
`
`required. to make that objection.
`
`So let the record reflect that
`
`counsel is cutting the witness off.
`MR. SCHARFF: And, again,
`let
`
`the record reflect I didn't cut him off in
`
`the middle of a sentence.
`
`an BIISWEI‘ .
`
`In the middle of
`
`
`
`Q.
`
`Let's take your support for the
`
`BY MR . SCHARFF:
`
`time .
`
`We talked previously about the
`
`training with known inputs sentence in
`
` pattern recognition training one item at a
`
`
`ESQUIRE
`
`SCILUTIDHE
`
`
`800.21 1.DE-PO (3376)
`Esquiresolutionaoom
`
`oLooo<:mLns4>-’w£\'7I—-‘
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PHD.
`TOYOTA MOTOR VS. AVS
`
`_
`
`February 24, 2014
`167
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`Lemelson that you point to in your’
`
`declarations
`
`What's the next instance?
`
`A.
`
`With all due respect, may I
`
`explain a little bit the thought process
`
`here?
`
`Q.
`A.
`
`Okay.
`There are issues of tent and
`
`figures that are interconnected, so if you
`
`let me mention the figures and the texts,
`
`then I can answer the questions here and
`
`the logic that I followed here.
`
`Q.
`
`That's fine.
`
`If that's what you
`
`need to do, go ahead.
`
`I was just trying to
`
`make it a little more concise.
`
`_A.
`
`So the ones I consider is
`
`actually Figure 1 and its expected
`
`description; Figure 2 and Figure 5.
`
`And also, as part of this,
`
`the
`
`Figure 3 and 4.
`
`I go to the ~~ so column 4,
`
`where it has descriptions of the different
`
`figures. And then I'm using 515 down to
`
`55. And then I'm using 81, and I think I
`
`ES
`
`s u L U T I 0 :4 s
`
`800.211.DEPO (3376)
`
`ESC]UI'f'9S0/UtI'OnS.COfT7
`
`
`
`MKOLAOS PAPANIKOLOPOULQS, PH.D.
`TOYOTAMOTORvaAVS
`
`February 24, 2014
`168
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`stopped at row 8.
`
`The idea here is actually trying
`
`to understand how the components are put
`
`together, and also how the different claim
`construction in terms of what was given to
`
`me map this combination of text and
`
`figures.
`
`
`
`
`
`
`
`
`So if you see the column 8 text,
`
`
`
`
`it says some other interesting things that
`teach the training based on data of
`possible exterior objects.
`It says, for example:
`"The neural network of the image
`_analysis computer 19" ~- so that's why i
`
`
`
`
`Want to use the Figure 1 ~~ "provides a
`
`highly parallel image processing structure
`
`
`
`
`
`with rapid, real~time image recognition
`
`necessary for the Motor Vehicle Warning and
` "Very Large Scale Integrated
`processing elements permits low—cost,
`
`Control System."
`
`
`
` (VLSI) Circuit implementation of the neural
`
`low—weight limitation."
`
`
`
`ESQUIRE
`
`SDLUTIDNS
`
`800.211.DEPO (3376)
`EsqulreSo/utionaoom
`
`And I want to emphasize this:
`
`
`
`
`CD\'10\U1s¥>LrJ!.\3|-‘
`
`10
`
`ll
`
`12
`
`13
`
`14
`
`15
`
`16
`
`l7
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24;
`
`25
`
`
`
`NIKOLAOS PAPANiKDLOPt)ULDS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`169
`
`Nikolaos Papanikolopoulos, Ph.D.
`
`"Also, a neural network has
`certain reliability advantages important in
`
`a safety warning system. Loss of one
`
`processing element does not necessarily
`
`result in a processing system failure."
`
`In other words, if one of the
`
`elements goes off for whatever reason,
`
`the
`
`system is able to learn online and adopt,
`
`so all these are elements that I took into
`
`consideration.
`
`Now, one important question that
`
`I had to face as part of this is actually
`
`the data and patterns are actually used.
`
`This is for the training. This
`
`is the Figure 5, because it shows an image
`
`processer that receives image data, and an
`
`image data bus that connects the virtual
`
`processing elements,
`
`so all of these were a
`
`part of this.
`
`So I don't know if this has been
`
`helpful.
`
`Q.
`
`A.
`
`Are you done with your answer?
`
`One last thing.
`
`Another thing that I had to
`
`7~~‘;§_ ES
`
`s 0 L u 'r
`
`1
`
`an n s
`
`800.211.DEPO (3376)
`
`Esqu/I/‘BS0/U1‘/.0nS.COn7
`
`
`
`NIKOLAOS PAPANIKOLOPOULOS, PH.D.
`TOYOTA MOTOR VS. AVS
`
`February 24, 2014
`170
`
`QO\U'1r£>~bJL\Jl'-3
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`Nikolaos Papanikolopoulos, Ph.D.
`consider as part of the claim was actually
`
`the idea of patterns. And for me, patterns
`
`are actually relationships of pixels.
`
`Q.
`
`Okay.
`
`So let's go to your
`
`declaration.
`
`In your 419 declaration,
`
`paragraphs 47 to 64 relates to anticipation
`
`by Lemelson;
`
`is that right?
`
`A.
`
`