`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`_________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`_________________
`
`APPLE INC.,
`
` Petitioner,
`
` v.
`
`ZENTIAN LIMITED,
`
` Patent Owner
`
`_________________
`
`Inter Partes Review Case No. IPR2023-00037
`
`U.S. Patent No. 10,971,140
`
`Inter Partes Review Case No. IPR2023-00033
`
`U.S. Patent No. 7,587,319
`
` REMOTE DEPOSITION
`
` of
`
` CHRISTOPHER SCHMANDT
`
`Reported by:
`
`Robin LaFemina, RPR, CLR
`
`Job No.: 7862
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 2
`
` September 6, 2023
`
` 11:05 a.m.
`
` REMOTE DEPOSITION of
`
`CHRISTOPHER SCHMANDT, held via Zoom, with
`
`all parties appearing from their respective
`
`locations, before Robin LaFemina, a
`
`Registered Professional Reporter, Certified
`
`LiveNote Reporter and Notary Public within
`
`and for the State of New York.
`
`1 2 3 4 5
`
`6
`
`7 8 9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 3
`
`APPEARANCES:
`
`(ALL PARTIES APPEARING REMOTELY)
`
`ERISE IP
`
`Attorneys for Petitioner
`
` 5299 DTC Boulevard
`
` Suite 1340
`
` Greenwood Village, Colorado 80111
`
`BY: CHRISTINA CANINO, ESQ.
`
` christina.canino@eriseip.com
`
`ERISE IP
`
`Attorneys for Petitioner
`
` 7015 College Boulevard
`
` Suite 700
`
` Overland Park, Kansas 66211
`
`BY: JENNIFER C. BAILEY, ESQ.
`
` jennifer.bailey@eriseip.com
`
`(Continued on following page)
`
`1 2
`
`3
`
`4 5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 4
`
`APPEARANCES: (C'td.)
`
`(ALL PARTIES APPEARING REMOTELY)
`
`NOZOORI PC
`
`Attorneys for Patent Owner
`
` 11601 Wilshire Boulevard
`
` Suite 2170
`
` Los Angeles, California 90025
`
` (310) 975-7074
`
`BY: KAYVAN B. NOROOZI, ESQ.
`
` knoroozi@noroozilaw.com
`
` PETER KNOPS, ESQ.
`
` pknops@noroozilaw.com
`
`1 2
`
`3
`
`4 5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 5
`
` THE COURT REPORTER: Good
`
` morning. Today's date is September 6,
`
` 2023.
`
` My name is Robin LaFemina and I
`
` am the court reporter representing
`
` TransPerfect Legal Solutions.
`
` This deposition is being taken
`
` remotely and the court reporter will
`
` not be physically present with the
`
` witness whose deposition is being
`
` taken.
`
` The parties agree not to
`
` challenge the validity of any oath
`
` administered by the court reporter,
`
` even if the court reporter is a notary
`
` of a state other than where the
`
` deponent is currently physically
`
` located.
`
` Would each attendee, starting
`
` with the examining attorney, please
`
` indicate your agreement by stating
`
` your name, your firm affiliation and
`
` your agreement on the record, after
`
` which I will swear in the witness.
`
`1 2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 6
`
` MR. NOROOZI: Kayvan Noroozi,
`
` Noroozi PC, representing Zentian
`
` Limited. We agree.
`
` MS. CANINO: Christina Canino
`
` with Erise IP representing Apple Inc.
`
` We agree.
`
` THE COURT REPORTER: Raise your
`
` right hand, please, Mr. Schmandt.
`
` Do you swear the testimony you
`
` are about to give will be the truth,
`
` the whole truth, and nothing but the
`
` truth, and do you acknowledge that
`
` although I am not physically present in
`
` the room with you, that this oath is
`
` administered with the same force and
`
` effect as though I were?
`
` THE WITNESS: Yes, I do.
`
` THE COURT REPORTER: You may
`
` begin, counsel.
`
`CHRISTOPHER SCHMANDT,
`
` called as a Witness, having been first
`
` duly sworn by Robin LaFemina, a Notary
`
` Public within and for the State of New
`
` York, was examined and testified as
`
`1 2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 7
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` follows:
`
`EXAMINATION BY
`
`MR. NOROOZI:
`
` Q. Good morning, Mr. Schmandt.
`
`How did you prepare for this deposition?
`
` A. Before I dive into that, I'd
`
`like to get a few corrections from my
`
`declaration on the record. Is this an
`
`appropriate time to do that?
`
` Q. Okay. What's the nature of the
`
`corrections? Is it typographical or --
`
` A. Yes.
`
` Q. -- something else?
`
` A. Typos.
`
` Q. Okay. Go ahead. Walk me
`
`through the areas you'd like to make
`
`corrections to.
`
` A. Okay. These are both in the
`
`'140 declaration.
`
` Q. Okay.
`
` A. Section X, i.e. 10, it says --
`
` Q. Sorry. Could you give me a
`
`paragraph number?
`
` A. No. This is the header. This
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 8
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`is in the Table of Contents.
`
` Q. I see.
`
` A. And it will also be imported
`
`into that page number which we can do
`
`afterwards, if you'd like. Okay? So the
`
`header that says Section X, change Ground 4
`
`to Ground 3.
`
` Q. In the version that I have, it
`
`says Opinions Regarding Ground 3. Let's
`
`see. I'm looking at the Table of Contents,
`
`so -- and I'm also looking at the
`
`declaration itself at exhibit page 101. It
`
`says Section X, Opinions Regarding Ground 3
`
`and then subsection A, claims 53 and 58-59.
`
` A. Sorry. This is the '140?
`
` Q. I believe. Oh, is it --
`
` A. There is no claim 50s in the
`
`'140.
`
` Q. Got it. Okay.
`
` So okay, Section X with respect
`
`to the '140, it says opinions regarding
`
`Ground 4 and it should say Ground 3?
`
` A. Correct.
`
` Q. Page 93?
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 9
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` A. Well, I'm looking at the Table
`
`of Contents, but yes, it should also be on
`
`page 93.
`
` Q. Okay. What else?
`
` A. Section XI, that is 11, it
`
`should say Opinions Regarding Claim 5, I'm
`
`reading off of my notes here, so instead of
`
`Ground 6, it's Ground 5. And subsection A
`
`should read claim 6, not claim 4. And that
`
`should also be carried through under page
`
`94.
`
` Q. Anything else?
`
` A. That's all. Thank you.
`
` Q. In reviewing your two decs for
`
`the '319 patent and '140 patent, prior to
`
`today, did you identify any areas of
`
`substantive testimony that you want to
`
`correct or modify?
`
` A. No.
`
` Q. In preparing for today's
`
`deposition, did you identify any documents
`
`that are not exhibits in the '140 patent
`
`proceeding or the '319 patent proceeding
`
`that you intend to introduce into either
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 10
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`proceeding going forward?
`
` A. No. I think you may have
`
`phrased that in terms of exhibits. I looked
`
`at the Patent Owner Preliminary Response and
`
`the Decision. I don't know if those have
`
`exhibit numbers.
`
` Q. Right. I'm referring
`
`specifically to exhibits from you, meaning
`
`evidence on your behalf in support of your
`
`opinions through testimony, whether already
`
`expressed or things that you intend to
`
`express.
`
` A. And I think the question was
`
`whether I intended to introduce anything
`
`additional. If that is my understanding of
`
`the question, my answer is no.
`
` Q. In preparing for today's
`
`depositions, are there any further
`
`explanations or articulations that you
`
`decided that you should provide or would
`
`like to provide that are not already set
`
`forth in your declarations?
`
` A. No.
`
` Q. Let's start with the '140 patent
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 11
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`declaration. This is in Case No.
`
`IPR2023-00037, and it's Exhibit 1003 in that
`
`proceeding, and I'd like to direct you to
`
`paragraph 4 of your declaration.
`
` A. Sure. Okay.
`
` Q. You said in paragraph 4, I built
`
`the first conventional computer system
`
`utilizing speech recognition and synthesis
`
`("Put That There") starting in 1980.
`
` Do you see that?
`
` A. Yes.
`
` Q. What is your basis for saying
`
`that you built the first one?
`
` A. Because I don't know of any
`
`other one. To be perfectly clear, there was
`
`a system out of Bell Labs by Levinson that
`
`was airline reservation information with a
`
`speech interface and he and I have talked
`
`about it and decided that we were
`
`simultaneously the first to do this, so he
`
`should -- it should -- maybe shouldn't say
`
`first, it should say tied for the first or
`
`one of the first.
`
` Q. Okay.
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 12
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` You say you built this
`
`conversational computer system utilizing
`
`speech recognition and synthesis.
`
` What do you mean by built?
`
` A. I did not specify the hardware.
`
`The hardware was specified as part of the
`
`research proposal. I accepted the hardware
`
`into the lab, I wrote device drivers for
`
`said hardware so that my software could
`
`communicate with it, and then I wrote half
`
`the software for the Put That There system,
`
`the other half being written by a colleague.
`
` Q. And what was the Put That There
`
`system capable of doing?
`
` A. The Put That There system
`
`accepted voice and gesture as input. The
`
`target application was manipulating objects
`
`in a geographic database, so there were
`
`spatial extents. The Put That There system
`
`took voice and gesture as inputs. When it
`
`took the inputs, it built what's known in
`
`the art as a dialogue frame using a slot and
`
`filler approach where a semantic model
`
`parses the speech and tries to take each
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 13
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`word and use it in a context of a model of
`
`user intent, so it would build a dialogue
`
`frame and then the application interface to
`
`the geospatial model could look at that
`
`dialogue frame and determine whether it was
`
`complete. If it was complete, the system
`
`could perform the action which was usually
`
`graphical, but could be auditory. If it was
`
`not complete, then the dialogue system would
`
`indicate which parts of the frame needed to
`
`be filled. The system would then speak back
`
`to the user and ask a question directed to
`
`filling the parts of the dialogue frame that
`
`were missing or misrecognized, the user
`
`would say something or point at something,
`
`and this process would repeat until there
`
`was a dialogue frame that had adequate
`
`information to perform some action within
`
`the confines of the application for which it
`
`was written.
`
` Q. In the next sentence of your
`
`paragraph 4, you say, I built one of the
`
`first graphical user interfaces for audio
`
`editing, employing keyword recognition on
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 14
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`voice memos in 1982 (Intelligent Ear).
`
` What do you mean by I built in
`
`this context?
`
` A. Again, with one colleague, I
`
`came up with the idea, I came up with the
`
`techniques of doing the keyword recognition
`
`and I implemented half of the software.
`
` Q. You used hardware that someone
`
`else had built?
`
` A. Yes. I actually used the same
`
`hardware that I -- some of the same hardware
`
`that I had used in the Put That There system.
`
` Q. In the next sentence, you say, I
`
`built the first research-grade unified
`
`messaging system, etc.
`
` What do you mean by I built in
`
`that context?
`
` A. I came up with the idea. I
`
`can't claim sole ownership of the idea. The
`
`idea was certainly refined by conversation
`
`among my peers at the laboratory. I took on
`
`a grad student to work on it with me. I
`
`acquired and incorporated a new speech
`
`synthesizer and I wrote half the software,
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 15
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`roughly half.
`
` Q. Did you build the hardware?
`
` A. Some of the hardware.
`
` Q. What aspects?
`
` A. There was -- so there were a
`
`number of audio interconnects and telephone
`
`line interconnects. This was a system that
`
`could answer the phone and take voice
`
`messages, and if I called in, it could play
`
`me those voice messages and it could play me
`
`synthetic speech messages over the phone, so
`
`it needed a telephone line interface which I
`
`built and it needed some switchable audio
`
`interconnects so these various audio devices
`
`could get inputs and outputs from the phone
`
`line as needed.
`
` Q. Did you build the processor and
`
`memory architecture?
`
` A. No.
`
` Q. Where did you get that?
`
` A. That was applied by the Magic 6
`
`operating system running on an Interdata
`
`Model 732 computer, mini computer.
`
` Q. What kind of grad student helped
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 16
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`you with the project that you call Phone
`
`Slave?
`
` A. A good one. I'm not sure what
`
`you're asking. He was male.
`
` Q. Let me --
`
` A. He's now dead.
`
` Q. I'm sorry to hear that. I'm
`
`very sorry to hear that. Let me specify.
`
` Was it a Ph.D. graduate student
`
`or a master's graduate student who helped
`
`you with Phone Slave?
`
` A. A master's.
`
` Q. And what did the graduate
`
`student provide?
`
` A. He wrote half the code.
`
` Q. For doing what?
`
` A. Oh, well the system as a whole
`
`had a graphical user interface and a voice
`
`user interface. So the graphical user
`
`interface had to display the messages, it
`
`was a touch sensitive interface, so it would
`
`accept user interaction. So he built a
`
`parser, specifically he built a parser that
`
`would lay out the screen and detect where a
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 17
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`user had touched, and on the basis of where
`
`the user had touched would formulate a
`
`command. This is interesting because I
`
`wrote the voice user interface, which was
`
`the user talking to the system, and we made
`
`sure through this compiled -- compiled code
`
`that the -- when the speech recognizer --
`
`for example, when the speech recognizer --
`
`when I said play my messages to the speech
`
`recognizer, that was the equivalent of the
`
`user touching a column of all messages, so
`
`those were reduced to the same command, the
`
`command was a symbolic, it was a number.
`
`Okay? So the front end would generate the
`
`same number whether I touched a column of
`
`messages or said play my messages, that
`
`command could then be fed into the
`
`application support system, which would
`
`actually play the audio or synthesize the
`
`text without having to write two versions of
`
`that, one for voice input and one for speech
`
`input, so he wrote that parser part and he
`
`wrote most of the graphical user interface,
`
`and I wrote the voice user interface and the
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 18
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`code that would actually answer the phone.
`
`So this was an interesting system in that it
`
`didn't just say, hi, you've reached Chris
`
`Schmandt, please leave a message, beep.
`
`This was in the early days of answering
`
`machines and voicemail. So it would
`
`actually say, hello, Chris's telephone,
`
`who's calling, please? And it would record
`
`that snippet.
`
` And then it would say, what's
`
`this in reference to?
`
` And it asked a series of five
`
`questions, and because it asked a series of
`
`five questions, later on I could say who
`
`left messages, and it could simply play
`
`those audio snippets, and there was a
`
`sociological aspect of this as well because
`
`in -- at least in the U.S., the protocol is
`
`that the called party says hello, and then
`
`it is the obligation of the calling party to
`
`identify themselves, so if you answer the
`
`phone saying who's calling, please, people
`
`answer that. And that was the hook. You
`
`know, once they started leaving a message,
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 19
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`they would continue to leave the message as
`
`opposed to hang up and think about it and
`
`maybe call back, maybe not.
`
` Q. Okay.
`
` So what was the nature of the
`
`speech recognition capability of the Phone
`
`Slave system that you say you built in 1983?
`
` A. I'm sorry. What was the
`
`question? You froze for a second. Could
`
`you repeat the question, please?
`
` THE COURT REPORTER: I think he
`
` is frozen now as well.
`
` THE WITNESS: I think so.
`
` MR. KNOPS: Hold on. I'll text
`
` him to let him know.
`
` (Pause in the proceedings.
`
`CONTINUED BY MR. NOROOZI:
`
` Q. Can you guys hear me?
`
` A. Now we can hear you. I did not
`
`hear the question.
`
` Q. Thank you.
`
` What was the nature of the
`
`speech recognition capability of the Phone
`
`Slave system that you say you built in 1983?
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 20
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` A. Speaker independent, recognizer
`
`connected speech, 120 word vocabulary.
`
` Q. And did your graduate student
`
`build the speech recognition software, did
`
`you build the speech recognition software,
`
`did you both do it, or did you get it from
`
`somewhere?
`
` A. No, we got it. It was a
`
`product, NEC-DP100.
`
` Q. With respect to Back Seat Driver
`
`in paragraph 4 of your declaration, you say
`
`that along with your students, you built the
`
`first system for real time spoken driving
`
`directions.
`
` What do you mean by you built?
`
` A. In that case, I architected
`
`the -- I came up with the concept and
`
`architected the system, although it was
`
`built on some previous work that he had
`
`done. In that system, he wrote pretty much
`
`all the code.
`
` Q. Can I ask you to specify who he
`
`is?
`
` A. Jim Davis. James Raymond Davis.
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 21
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` Q. And why do you refer to him as
`
`he? Is he someone that you have already
`
`spoken about today?
`
` A. I -- I don't get your question
`
`at all. He was male --
`
` Q. No, no.
`
` A. -- so I call him he.
`
` Q. Sorry. I think my connection
`
`cut out.
`
` Is this person, Mr. Davis,
`
`somebody that you already alluded to earlier
`
`today?
`
` A. No.
`
` Q. Okay.
`
` The reason I asked is because
`
`you said he and that suggested to me that
`
`you had an antecedent basis for this he in
`
`your earlier testimony. So to clarify,
`
`Mr. Davis is not someone that you've already
`
`alluded to today before we asked about Back
`
`Seat Driver?
`
` A. Not before --
`
` MS. CANINO: Objection. Form.
`
` A. -- we asked about Back Seat
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 22
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`Driver. If you wanted to know whether there
`
`was an antecedent basis, we could read back
`
`my response, but I don't think it matters.
`
`This was the only project listed here which
`
`Jim Davis worked on.
`
` Q. And just so you understand, I'm
`
`not trying to play word games. I'm asking
`
`because I want to understand if there's a
`
`common collaborator or graduate student that
`
`worked on the prior projects that we talked
`
`about who also worked on Back Seat Driver
`
`that you were referring to.
`
` A. No. Would you like to know the
`
`names of the people who worked on the prior
`
`projects?
`
` Q. No. Don't need to.
`
` A. Okay.
`
` Q. Thank you.
`
` Did you design the processor to
`
`memory architecture for Back Seat Driver?
`
` A. Only indirectly in terms of
`
`working at the architecture, so the system
`
`could provide realtime driving directions.
`
`The first version of the system, the
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 23
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`software ran on a symbolics Lisp machine,
`
`Lisp, L-I-S-P, machine that was in our lab,
`
`and the vehicle would send a location of
`
`velocity every second over a telephone
`
`connection. And then there was another
`
`telephone connection to send the voice back
`
`to the vehicle, but the problem was cellular
`
`modems in 1988 weren't so great, and
`
`sometimes the data would drop out and we
`
`wouldn't get any data from the car for eight
`
`to ten seconds. Ordinarily that doesn't
`
`matter, but if you're traveling at 40 miles
`
`an hour, you might miss a turn, so that had
`
`issues, so Jim and I worked out that we
`
`could take the latest SPARCstation
`
`processor, which was at that point a
`
`SPARCstation 1+ that was just able to run
`
`common Lisp and put that in a vehicle
`
`although we needed to build a vibration
`
`damping platform because it had an internal
`
`hard disk. So I didn't design SPARCstation 1
`
`or SPARCstation 1+, but Jim and I had to
`
`design the physical setup for how the
`
`machine was placed in the car.
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 24
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` Q. Did you write the speech
`
`recognition software for Back Seat Driver?
`
` A. Oh, no, I didn't.
`
` Q. Where did you get that?
`
` A. There actually wasn't any
`
`recognition software in Back Seat Driver,
`
`the input was touchtones, so you would spell
`
`a street name with touchtones. The output
`
`was synthetic speech because, of course, it
`
`was real time spoken driving directions.
`
` Q. In the next sentence of
`
`paragraph 4 of your '140 patent declaration,
`
`you refer to Conversational Desktop 1985,
`
`Chatter 1993 and MailCall 1996.
`
` Did you build the processor to
`
`memory architecture in any of those systems
`
`that you reference?
`
` A. No.
`
` Q. Did you write the speech
`
`recognition software for any of those
`
`systems?
`
` A. For the Chatter system, we had
`
`software -- my recollection is we had
`
`software recognition from some small company
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 25
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`which we had to modify to put into an
`
`appropriate software architecture with their
`
`help for our application, but we did not
`
`rewrite the recognizer. The Conversational
`
`Desktop used the aforementioned DP200
`
`recognizer and MailCall used a recognizer
`
`from -- recognizer software from Texas
`
`Instruments called Dagger.
`
` Q. And what about Chatter 1993?
`
` A. That was the one I was just
`
`talking about. We took some small company's
`
`recognizer and we made some modifications to
`
`it, but I would not say we actually changed
`
`the recognition algorithm. I think we
`
`worked mostly with the API.
`
` Q. And with the assistance of the
`
`company who wrote the speech recognizer;
`
`right?
`
` A. I can't remember exactly how
`
`that worked. Sorry. That was 30 years ago.
`
` Q. What processor and memory
`
`hardware did you use for Conversational
`
`Desktop 1985, Chatter 1993 and MailCall
`
`1996, if you recall?
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 26
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` MS. CANINO: Objection --
`
` A. Conversational --
`
` MS. CANINO: -- form.
`
` A. Conversational Desktop was
`
`Magic -- 1985. 1985. It was a hybrid of
`
`Magic 6 operating system running on
`
`PerkinElmer 3220 computers, some version of
`
`DOS running on a PC and Berkeley UNIX
`
`running on one of the early multi-bus Sun
`
`microsystems computer, so Conversational
`
`Desktop had multiple stations. It was
`
`designed for two parties to interact with
`
`each other. It was easy to whip the code
`
`off for the PerkinElmer, we had great
`
`software to do that, but we only had one of
`
`those, so we had to build a version of the
`
`software to run under UNIX because we were
`
`then switching to the Sun computers. It was
`
`all a very distributed and network system.
`
` Q. Did you write the speech
`
`recognition software for Put That There?
`
` A. No.
`
` Q. What did you use for that?
`
` A. NEC-DP100.
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 27
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` Q. Referring to the systems you say
`
`you built for handheld mobile devices in
`
`paragraph 4 of your declaration, those being
`
`ComMotion 1999, Nomadic Radio 2000,
`
`Impromptu 2001, and Symphony 2004, did you
`
`build the processor to memory architecture
`
`for any of those systems?
`
` A. No.
`
` Q. Did you use existing processor
`
`to memory architecture that was in the
`
`relevant mobile handheld devices?
`
` A. Well, we had to build these.
`
`ComMotion, we had to build the hardware. We
`
`used an existing processor board. The same
`
`with Nomadic Radio. Impromptu used a
`
`combination of software that was native on
`
`the Hewlett Packard IPAQ, that's I-P-A-Q,
`
`and some software that we wrote on our own
`
`UNIX machines for server. And Symphony, I
`
`can't remember the details of. I can't
`
`remember whether the recognizer was done,
`
`that was a server-based system as well. I
`
`can't remember the details of that.
`
` Q. For the speech recognition and
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 28
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`handheld mobile devices systems identified
`
`in paragraph 4 of your declaration, did you
`
`build the speech recognizer software for any
`
`of those systems?
`
` A. No.
`
` Q. Where did you get the speech
`
`recognition software?
`
` A. Those came from all over the
`
`place. Nomadic Radio was open -- Nomadic
`
`Radio was an AT&T recognizer, I believe the
`
`ComMotion was Open Source Sphinx. I can't
`
`remember the other two.
`
` Q. In the next sentence of your
`
`declaration, you refer to Memory Prosthesis
`
`2004. Did you build the memory and
`
`processor architecture for Memory Prosthesis
`
`2004?
`
` A. No.
`
` Q. Did you write the speech
`
`recognition software for Memory Prosthesis
`
`2004?
`
` A. No.
`
` Q. Where did you get the speech
`
`recognition software for Memory Prosthesis
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 29
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`2004?
`
` A. I believe we used a network
`
`version of IBM ViaVoice.
`
` Q. And where did you get the
`
`processor to memory architecture for Memory
`
`Prosthesis 2004?
`
` A. So the -- there were a number of
`
`parts -- there were a number of parts to
`
`that system. Some parts of that system ran
`
`real time on Hewlett Packard IPAQ, which I
`
`mentioned before. The recognition was all
`
`done offline over a network. On UNIX work
`
`station, I think it was a NeXT computer,
`
`N-E-X-T, but I'm not sure.
`
` Q. Was the Memory Prosthesis 2004 a
`
`real time speech recognition system?
`
` A. Oh, no. Well, some parts of it
`
`worked in real time, but the whole test, it
`
`was a memory prosthesis, the whole test was
`
`would it help you remember something when
`
`you looked back two years later.
`
` Q. So it did not need to be real
`
`time and it was not real time; fair?
`
` A. The --
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 30
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
` MS. CANINO: Objection. Form.
`
` A. How close to real time it was
`
`would be a question of network latency, but
`
`it was not a design constraint.
`
` Q. With respect to Synthetic News
`
`19 -- withdrawn.
`
` With respect to Synthetic News
`
`Radio 1999, did you build the processor to
`
`memory architecture for that system?
`
` A. No.
`
` Q. With respect to Synthetic News
`
`Radio 1999, did you build the speech
`
`recognition software for that system?
`
` A. Well, there were several aspects
`
`of it, meaning how broadly you used the term
`
`recognition. So basically what we were
`
`doing was capturing newscasts and doing
`
`basically a rough recognition on them using
`
`large vocabulary speaker independent
`
`recognizer, and then we were processing, and
`
`I would consider this to be, based on the
`
`literature that I read, I would consider
`
`this to be part of the recognition stage.
`
`We took those results which were very noisy
`
`TransPerfect Legal Solutions
`212-400-8845 - Depo@TransPerfect.com
`
`229b546b-a925-468b-a88f-daea5df568e8
`
`
`
`Page 31
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`11
`
`12
`
`13
`
`14
`
`15
`
`16
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25
`
` Schmandt
`
`and did latent semantic analysis of those
`
`results. Latent semantic analysis attempts
`
`to identify words that are different from
`
`other words, it attempts to cluster things
`
`by topic, so our goal here was not to
`
`understand every word, but to understand the
`
`topic, so we wanted to find in hourly
`
`newscasts which of the audio was
`
`continuation of a