Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 1 of 219
`
`
`
`
`
`
`
` Exhibit 29
`
`
`
`
`
`
`
`
`
`
`
` Exhibit 29
`
`
`
`
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/2CbgwflfifébltopY
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 2 of 219
`
`UNITED STATES DISTRICT COURT
`
`FOR THE SOUTHERN DISTRICT OF NEW YORK
`
`NETWORK-1 TECHNOLOGIES,
`
`INC.
`
`Plantiff
`
`
`
`vs.
`
`Case No. 14—CV-2396
`
`GOOGLE,
`
`INC., and YOUTUBE, LLC
`
`Defendant
`
`CONFIDENTIAL
`
`DEPOSITION OF SHUMEET BALUJA, Ph.D.
`
`June 19, 2015
`
`Michele E. Edd Notar Public
`393685
`Y'
`Y
`
`1972 ®
`
`BARKLEY
`
`barkley.com
`
`(858) 455-5444 San Diego
`(949) 955-0400 Irvine
`(415) 433-5777 ‘San Francisco
`(310) 207-8000 Los Angeles
`(951) 686-0606 Riverside
`(760) 322-2240 Palm Springs
`(408) 885-0550 San Jose
`(916) 922-5777 Sacramento
`(518) 490-1910 Albany
`(347) 821 -4611 Brooklyn
`(212) 808—8500 New York City
`(818) 702—0202 Woodland Hills
`(702) 366-0500 Las Vegas
`(312) 379-5566 Chicago
`(914) 510-9110 White Plains
`(516) 277-9494 Garden City
`00+1+800 222 1231 Paris
`00+1+800 222 1231 Dubai
`001+1+800 222 1231 Hong Kong
`
`
`
` 1 UNITED STATES DISTRICT COURT
`
` 2 FOR THE SOUTHERN DISTRICT OF NEW YORK
`
` 3
`
` 4 NETWORK-1 TECHNOLOGIES, INC.
`
` 5 Plantiff
`
` 6 vs. Case No. 14-CV-2396
`
` 7 GOOGLE, INC., and YOUTUBE, LLC
`
` 8 Defendant
`
` 9 _______________________________/
`
`10
`
`11 CONFIDENTIAL
`
`12 The deposition of SHUMEET BALUJA, Ph.D., was
`
`13 held on the 19th day of June, 2015, commencing at 9:09
`
`14 a.m., at the Law Offices of Finnegan, Henderson, Farabow,
`
`15 Garrett & Dunner, LLP, Two Freedom Drive, Reston,
`
`16 Virginia, before Michele E. Eddy, Notary Public.
`
`17
`
`18
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`25 REPORTED BY: MICHELE E. EDDY, RPR, CRR, CLR
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 3 of 219
`
`2
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
` 1 APPEARANCES:
`
` 2
`
` 3 ON BEHALF OF THE PLAINTIFF:
`
` 4 BRIAN D. LEDAHL, ESQUIRE
`
` 5 Russ August & Kabat
`
` 6 12424 Wilshire Boulevard, 12th Floor
`
` 7 Los Angeles, California 90025
`
` 8 Telephone: (310) 826-7474
`
` 9 Facsimile: (310) 826-6991
`
`10 Email: Bledahl@raklaw.com
`
`11
`
`12
`
`13 ON BEHALF OF THE DEFENDANTS:
`
`14 DOUGLAS R. NEMEC, ESQUIRE
`
`15 Skadden, Arps, Slate, Meagher & Flom, LLP
`
`16 4 Times Square
`
`17 New York, New York 10036
`
`18 Telephone: (212) 735-2419
`
`19 Facsimile: (917) 777-2419
`
`20 Email: Douglas.nemec@skadden.com
`
`21
`
`22 ALSO PRESENT: James Sherwood, In-house Counsel, Google
`
`23 Daniel Holmstock, Videographer
`
`24
`
`25
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 4 of 219
`
`3
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
` 1 INDEX
`
` 2 Deposition of SHUMEET BALUJA, Ph.D.
`
` 3 June 19, 2015
`
` 4 Examination By: Page
`
` 5 Mr. Ledahl 7
`
` 6 Exhibit No. Marked
`
` 7 Exhibit 68 E-mail chain; top e-mail 96
`
` 8 dated 12-8-06 from Shumeet
`
` 9 Baluja to Jay Yagnik;
`
`10 GOOG-NETWORK-00656181-186
`
`11 Exhibit 69 Content ID Moma Posting; 114
`
`12 GOOG-NETWORK-00598291-92
`
`13 Exhibit 70 Paper titled "Waveprint: 118
`
`14 Efficient Wavelet-based
`
`15 Audio Fingerprinting" by
`
`16 Shumeet Baluja and Michele
`
`17 Covell;
`
`18 GOOG-NETWORK-00610574-87
`
`19 Exhibit 71 Abstract titled "Permutation 153
`
`20 Grouping: Intelligent Hash
`
`21 Function Design for Audio &
`
`22 Image Retrieval" by Shumeet
`
`23 Baluja, Michele Covell, and
`
`24 Sergey Ioffe;
`
`25 GOOG-NETWORK-00610856-59
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 5 of 219
`
`4
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
` 1 INDEX (Continued)
`
` 2 Deposition of SHUMEET BALUJA, Ph.D.
`
` 3 June 19, 2015
`
` 4
`
` 5 Exhibit No. Marked
`
` 6 Exhibit 72 PowerPoint titled "Audio and 158
`
` 7 Video Snippet Identification"
`
` 8 by Michele Covell, Jeff Faust,
`
` 9 Sergey Ioffe, Dave Marwood,
`
`10 Shumeet Baluja, and Jay Yagnik;
`
`11 GOOG-NETWORK-00633973-4049
`
`12 Exhibit 73 U.S. Patent No. US 2009/0052785; 165
`
`13 GOOG-NETWORK-00696867-890
`
`14 Exhibit 74 Abstract titled "Learning to 176
`
`15 Hash: Forgiving Hash Functions
`
`16 and Applications" by Shumeet
`
`17 Baluja and Michele Covell;
`
`18 GOOG-NETWORK-00613418-464
`
`19 Exhibit 75 U.S. Patent No. 8,184,953; 188
`
`20 GOOG-NETWORK-00697356-76
`
`21
`
`22
`
`23
`
`24
`
`25
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 6 of 219
`
`5
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
` 1 PROCEEDINGS,
`
` 2 THE VIDEOGRAPHER: This is video number
`
`09:09 3 1 in the video recorded deposition of Shumeet
`
`09:09 4 Baluja taken by the plaintiff in the matter of
`
`09:09 5 Network-1 Technologies, Inc., Plaintiff, versus
`
`09:09 6 Google, Inc., and YouTube, LLC, Defendants,
`
`09:09 7 pending before the United States District Court
`
`09:09 8 for the Southern District of New York, Case Number
`
`09:09 9 14-CV-2396.
`
`09:09 10 This deposition is being held at the law
`
`09:09 11 offices of Finnegan Henderson at 11955 Freedom
`
`09:09 12 Drive in Reston, Virginia, on June 19th, 2015.
`
`09:09 13 The time on the video screen is 9:09 a.m.
`
`09:10 14 My name is Daniel Holmstock. With me is
`
`09:10 15 Michele Eddy. We are in association with Barkley
`
`09:10 16 Court Reporters, Inc., located at 1875 Century
`
`09:10 17 Park East, Suite 1300, in Los Angeles, California.
`
`09:10 18 For the record now will counsel please
`
`09:10 19 introduce themselves and whom they represent.
`
`09:10 20 MR. LEDAHL: Brian Ledahl, from Russ
`
`09:10 21 August & Kabat, on behalf of the plaintiff
`
`09:10 22 Network-1.
`
`09:10 23 MR. NEMEC: Doug Nemec, from Skadden
`
`09:10 24 Arps, on behalf of the defendants Google and
`
`09:10 25 YouTube, and with me is Jim Sherwood, in-house
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 7 of 219
`
`6
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:10 1 counsel for Google.
`
`09:10 2 THE VIDEOGRAPHER: Will the court
`
`09:10 3 reporter please administer the oath.
`
`09:10 4 Whereupon,
`
`09:10 5 SHUMEET BALUJA, Ph.D.,
`
`09:10 6 having been duly sworn, testified as follows:
`
`09:10 7 EXAMINATION BY MR. LEDAHL:
`
`09:10 8 Q Good morning, Mr. Baluja.
`
`09:10 9 A Hi.
`
`09:10 10 Q Thank you for coming in today. At the
`
`09:10 11 outset I'm going to want to cover a couple, sort
`
`09:10 12 of, preliminary matters, but before I do that,
`
`09:10 13 have you ever had your deposition taken before
`
`09:10 14 today?
`
`09:10 15 A No, this is the first.
`
`09:10 16 Q Okay. So I'm going to cover a couple
`
`09:10 17 things that hopefully will make the process a
`
`09:10 18 little smoother for both us and for the court
`
`09:10 19 reporter. First of all, do you understand that
`
`09:11 20 the oath you took a moment ago is the same oath
`
`09:11 21 you would take if you were testifying live in a
`
`09:11 22 court in front of a judge and a jury?
`
`09:11 23 A I do.
`
`09:11 24 Q You understand that that oath carries
`
`09:11 25 the same penalties of perjury as if you were
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 8 of 219
`
`7
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:11 1 testifying live in court?
`
`09:11 2 A I do.
`
`09:11 3 Q Now, as you may have noticed, we have a
`
`09:11 4 court reporter and a video recording of the
`
`09:11 5 deposition proceedings today. As a consequence of
`
`09:11 6 that, there are a couple of things I'm going to
`
`09:11 7 ask that we try to observe to make the process a
`
`09:11 8 little smoother and to make sure we have a clear
`
`09:11 9 record of the proceedings.
`
`09:11 10 Because the court reporter takes down
`
`09:11 11 what we say, it's important that we do our best to
`
`09:11 12 try not to speak at the same time since she can
`
`09:11 13 only record one of us at a time. So, to that end,
`
`09:11 14 I'll do my best to wait until you finish answering
`
`09:11 15 one of my questions to ask another. Likewise, if
`
`09:11 16 you can wait until I finish my question to start
`
`09:11 17 your answer, it will make things a lot clearer and
`
`09:11 18 simpler for the court reporter. Is that
`
`09:11 19 acceptable?
`
`09:11 20 A Yes.
`
`09:11 21 Q Similarly, because it's very hard for
`
`09:12 22 her to capture things like nods or shakes of the
`
`09:12 23 head or words like uh-hmm and huh-uh clearly, I'm
`
`09:12 24 going to ask that you do your best to remember to
`
`09:12 25 give clear verbal responses like yes or no so that
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 9 of 219
`
`8
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:12 1 we have a clear record. Is that acceptable?
`
`09:12 2 A Yes.
`
`09:12 3 Q Now, after the deposition, the court
`
`09:12 4 reporter will prepare a transcript and you will
`
`09:12 5 have an opportunity to review that and make
`
`09:12 6 corrections or changes if you believe they are
`
`09:12 7 necessary. However, I want to caution you that to
`
`09:12 8 the extent you change the substance of your
`
`09:12 9 testimony, I will have the opportunity at some
`
`09:12 10 point, at a trial, perhaps, to comment on whether
`
`09:12 11 that may affect or implicate your credibility. So
`
`09:12 12 to the extent possible, I'm going to ask you to do
`
`09:12 13 your best to give your best and clearest
`
`09:12 14 recollection today. Do you understand?
`
`09:12 15 A Yes.
`
`09:12 16 Q And is there any reason, such as a
`
`09:12 17 medical reason or medication, that would prevent
`
`09:12 18 you from providing your best and most accurate
`
`09:12 19 recollections today?
`
`09:12 20 A No.
`
`09:12 21 Q During the course of the day, if at any
`
`09:12 22 point you don't understand my question, please let
`
`09:12 23 me know. I'll do my best to try to clarify it for
`
`09:13 24 you. Otherwise if you can provide an answer, I'll
`
`09:13 25 assume you understood the question. Okay?
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 10 of 219
`
`9
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:13 1 A Okay.
`
`09:13 2 Q Similarly, as we go along, if at any
`
`09:13 3 point you need a break, please let me know. I'll
`
`09:13 4 ask that we don't take breaks while we have a
`
`09:13 5 question to you pending but otherwise we'll do our
`
`09:13 6 best to accommodate that as soon as we can.
`
`09:13 7 A Okay.
`
`09:13 8 Q Now, Mr. Baluja, where are you currently
`
`09:13 9 employed?
`
`09:13 10 A Google.
`
`09:13 11 Q How long have you been at Google?
`
`09:13 12 A Almost 12 years.
`
`09:13 13 Q So since about 2003?
`
`09:13 14 A Correct.
`
`09:13 15 Q I would like to start a little bit with
`
`09:13 16 your, sort of, educational background. So
`
`09:13 17 starting with any college or university education,
`
`09:13 18 where, if at all, did you attend university?
`
`09:13 19 A I went to University of Virginia, and
`
`09:13 20 then I went to Carnegie Mellon for my Ph.D.
`
`09:13 21 Q So at UVA you -- what degree did you
`
`09:13 22 obtain there?
`
`09:13 23 A Computer science and philosophy.
`
`09:13 24 Q Was that a bachelor's degree?
`
`09:13 25 A Correct.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 11 of 219
`
`10
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:14 1 Q What year did you graduate and receive
`
`09:14 2 your degree from UVA?
`
`09:14 3 A I think '92, I think. '91 or '92. I
`
`09:14 4 don't remember.
`
`09:14 5 Q Then you said you went to Carnegie
`
`09:14 6 Mellon from there?
`
`09:14 7 A Correct.
`
`09:14 8 Q And that was for, I think you said, a
`
`09:14 9 Ph.D.?
`
`09:14 10 A Correct.
`
`09:14 11 Q What was the area of study for your
`
`09:14 12 Ph.D.?
`
`09:14 13 A Computer science and robotics.
`
`09:14 14 Q And did you complete your Ph.D. studies
`
`09:14 15 there?
`
`09:14 16 A Yes.
`
`09:14 17 Q What year did you receive your degree?
`
`09:14 18 A '96.
`
`09:14 19 Q Did you receive any interim degrees such
`
`09:14 20 as a master's degree along the way?
`
`09:14 21 A No.
`
`09:14 22 Q Do you have any other postgraduate
`
`09:14 23 education other than the Ph.D. from Carnegie
`
`09:14 24 Mellon?
`
`09:14 25 A No.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 12 of 219
`
`11
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:14 1 Q And did I understand correctly that you
`
`09:14 2 went essentially straight through from UVA to
`
`09:14 3 Carnegie Mellon?
`
`09:14 4 A Yes.
`
`09:14 5 Q Who was your -- well, strike that.
`
`09:14 6 Did you have a faculty advisor or Ph.D.
`
`09:15 7 advisor at Carnegie Mellon?
`
`09:15 8 A Yes.
`
`09:15 9 Q Who was that?
`
`09:15 10 A Dean Pomerleau and Tom Mitchell.
`
`09:15 11 Q Can you spell Pomerleau for me?
`
`09:15 12 A Yes, P-O-M-E-R-L-E-A-U.
`
`09:15 13 Q And the other person you said was?
`
`09:15 14 A Tom Mitchell.
`
`09:15 15 Q Tom Mitchell. That, I can probably
`
`09:15 16 spell.
`
`09:15 17 After receiving your Ph.D. at Carnegie
`
`09:15 18 Mellon, did you have employment shortly
`
`09:15 19 thereafter?
`
`09:15 20 A Yes.
`
`09:15 21 Q Where did you work first after your
`
`09:15 22 Ph.D.?
`
`09:15 23 A Justsystem Pittsburgh Research Center.
`
`09:15 24 Justsystem, J-U-S-T.
`
`09:15 25 Q Got it.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 13 of 219
`
`12
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:15 1 A Yes.
`
`09:15 2 Q And what was the business or activity of
`
`09:15 3 Justsystem?
`
`09:15 4 A They were basically the Microsoft of
`
`09:15 5 Japan so they did all the same stuff that
`
`09:15 6 Microsoft does but in Japan.
`
`09:15 7 Q What did you do at Justsystem?
`
`09:16 8 A Research.
`
`09:16 9 Q In any particular area or focus?
`
`09:16 10 A General computer science. I was all
`
`09:16 11 over the place.
`
`09:16 12 Q How long did you work at Justsystem?
`
`09:16 13 A Two years.
`
`09:16 14 Q So roughly until 1998?
`
`09:16 15 A Yes, right around then, yes.
`
`09:16 16 Q Where did you go after working at
`
`09:16 17 Justsystem?
`
`09:16 18 A Lycos, Incorporated.
`
`09:16 19 Q Is that L-Y-C-O-S?
`
`09:16 20 A Correct.
`
`09:16 21 Q What did you do at Lycos?
`
`09:16 22 A I was a chief scientist.
`
`09:16 23 Q What was Lycos' business, generally?
`
`09:16 24 A Web search.
`
`09:16 25 Q How long were you at Lycos?
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 14 of 219
`
`13
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:16 1 A About a year and a half or so.
`
`09:17 2 Q After Lycos, where did you go next?
`
`09:17 3 A It was called eCompanies.
`
`09:17 4 Q That would have been in about 2000 or
`
`09:17 5 thereabouts?
`
`09:17 6 A Yes, right around there. I don't
`
`09:17 7 remember the exact dates.
`
`09:17 8 Q What did you do at eCompanies?
`
`09:17 9 A I was the senior vice president of
`
`09:17 10 research.
`
`09:17 11 Q What was eCompanies' business generally?
`
`09:17 12 A They're an incubator.
`
`09:17 13 Q Did you focus on particular companies or
`
`09:17 14 developing companies within eCompanies?
`
`09:17 15 A Data mining and generally the technology
`
`09:17 16 side of companies.
`
`09:18 17 Q So was your role as providing some kind
`
`09:18 18 of research or scientific and technical support to
`
`09:18 19 various companies that were incubated at
`
`09:18 20 eCompanies?
`
`09:18 21 A Yeah, it was much less scientific. When
`
`09:18 22 you're incubating companies, it's more technical,
`
`09:18 23 yes.
`
`09:18 24 Q Fair enough.
`
`09:18 25 Do you know approximately how many
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 15 of 219
`
`14
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:18 1 companies, if any, spun out from that incubator
`
`09:18 2 during the time you were there?
`
`09:18 3 A Yes, I think we had three or four IPOs
`
`09:18 4 or buyouts.
`
`09:18 5 Q Do you remember the names of any of the
`
`09:18 6 companies?
`
`09:18 7 A Sure. Business.com, Jamdat Mobile, USB
`
`09:18 8 X. I think that was one. I don't remember the
`
`09:18 9 other one.
`
`09:18 10 Q How long were you at e- --
`
`09:19 11 A Oh, Boingo.
`
`09:19 12 Q Boingo.
`
`09:19 13 How long were you at eCompanies?
`
`09:19 14 A A year and a half or so.
`
`09:19 15 Q Where did you go next after working at
`
`09:19 16 eCompanies?
`
`09:19 17 A Jamdat Mobile.
`
`09:19 18 Q What did you do at Jamdat?
`
`09:19 19 A I was the CTO, chief technology officer.
`
`09:19 20 Q And, generally speaking, what was
`
`09:19 21 Jamdat's business at that time?
`
`09:19 22 A Mobile games.
`
`09:19 23 Q How long did you stay there?
`
`09:19 24 A Two years, I think.
`
`09:19 25 Q If I've got the timing correct, from
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 16 of 219
`
`15
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:19 1 there did you go to Google?
`
`09:20 2 A I took a year off.
`
`09:20 3 Q Why did you leave Jamdat?
`
`09:20 4 A My dad had health issues in Virginia so
`
`09:20 5 I left there.
`
`09:20 6 Q After you -- so was it after this year
`
`09:20 7 off that you began working at Google?
`
`09:20 8 A Correct.
`
`09:20 9 Q What led you to join Google?
`
`09:20 10 A A lot of my friends were there.
`
`09:20 11 Q Did you work at Google? Like where
`
`09:20 12 physically, geographically?
`
`09:20 13 A I went to Mountain View.
`
`09:20 14 Q When you first started at Google, what
`
`09:20 15 was your position or role there?
`
`09:20 16 A Researcher.
`
`09:20 17 Q Was there any particular focus or area
`
`09:21 18 that you were involved in?
`
`09:21 19 A No. Research is pretty open.
`
`09:21 20 Q Was your -- I take it there is a
`
`09:21 21 research department or research group at Google;
`
`09:21 22 is that right?
`
`09:21 23 A Correct.
`
`09:21 24 Q Was there any kind of group organization
`
`09:21 25 within that such that there were particular focus
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 17 of 219
`
`16
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:21 1 areas of any kind?
`
`09:21 2 A There were, but I was not a part of
`
`09:21 3 that.
`
`09:21 4 Q I see.
`
`09:21 5 Did at you focus personally on anything
`
`09:21 6 in particular when you started?
`
`09:21 7 A It would be -- okay, so, caveats on this
`
`09:21 8 is this was 10, 12 years ago so I don't remember.
`
`09:21 9 The first project, I can tell you what I worked
`
`09:21 10 on. I started the wireless group at Google.
`
`09:21 11 Q How long were you working on that
`
`09:21 12 project?
`
`09:22 13 A So I'll say I worked two or three years
`
`09:22 14 on it, but then that wasn't full-time, right, so
`
`09:22 15 you pick up projects as you go along.
`
`09:22 16 Q What was, in a general sense, the
`
`09:22 17 wireless group's focus?
`
`09:22 18 A Your cell phone that you have, we
`
`09:22 19 started that.
`
`09:22 20 Q And was that in the context of wireless
`
`09:22 21 communication or operating systems for phones or
`
`09:22 22 some other focus?
`
`09:22 23 A Yes.
`
`09:22 24 Q Both?
`
`09:22 25 A Everything.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 18 of 219
`
`17
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:22 1 Q Then what was the next major, sort of,
`
`09:22 2 project or area that you focused on?
`
`09:22 3 A Kid Safe Search. So I tried to keep
`
`09:22 4 kids safe on the Internet.
`
`09:23 5 Q And did that work start sometime during
`
`09:23 6 that two to three years or so that you mentioned?
`
`09:23 7 A Exactly, yes.
`
`09:23 8 Q What was the next, sort of, major
`
`09:23 9 project or area that you focused on?
`
`09:23 10 A Two, image processing and advertising --
`
`09:23 11 oh, and graphs.
`
`09:23 12 Q What was the image processing area that
`
`09:23 13 you mentioned?
`
`09:23 14 A Everything from face recognition to
`
`09:23 15 analyzing images.
`
`09:23 16 Q What was the advertising project that
`
`09:23 17 you mentioned?
`
`09:23 18 A Social -- all I can say is social
`
`09:23 19 advertising.
`
`09:23 20 Q Can you give me just a very high-level
`
`09:24 21 explanation of what that means?
`
`09:24 22 A Sure. You're familiar with the
`
`09:24 23 advertising you see on Facebook?
`
`09:24 24 Q Sure.
`
`09:24 25 A So this is on our social networks.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 19 of 219
`
`18
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:24 1 Q So, essentially, connecting advertising
`
`09:24 2 in some way to content on someone's social
`
`09:24 3 network --
`
`09:24 4 A Sure.
`
`09:24 5 Q -- or relating thereto?
`
`09:24 6 A I can accept that.
`
`09:24 7 Q Okay. And what was the graphs project
`
`09:24 8 or area that you mentioned?
`
`09:24 9 A So, like I mentioned, you know, Facebook
`
`09:24 10 is a big social graph. We also have social graphs
`
`09:24 11 as well as connectivity between web pages, the
`
`09:24 12 graphs of the whole web, so graph algorithms.
`
`09:24 13 Q What's the purpose of these kind of
`
`09:24 14 graph algorithms?
`
`09:24 15 A In general, it falls under
`
`09:24 16 understanding -- understanding the content of a
`
`09:24 17 web page.
`
`09:24 18 Q So are you referring generally to, sort
`
`09:24 19 of, the kind of complex interconnection and
`
`09:25 20 linkages that occur between multiple websites and
`
`09:25 21 locations and --
`
`09:25 22 A Correct.
`
`09:25 23 Q -- recognizing those relationships?
`
`09:25 24 A Yes. So you're familiar -- and the
`
`09:25 25 big -- the big one is PageRank, as I'm sure you've
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 20 of 219
`
`19
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:25 1 heard Google was based on when it started. So
`
`09:25 2 that's an example of a graph algorithm.
`
`09:25 3 Q What was the next major area that you,
`
`09:25 4 sort of, picked up? And I recognize -- am I
`
`09:25 5 correct that different of these sort of overlap in
`
`09:25 6 time?
`
`09:25 7 A They all -- at some point it's just one
`
`09:25 8 big mix. Right? Let's see. I think that's the
`
`09:25 9 big ones.
`
`09:25 10 Q At some point did you become involved in
`
`09:25 11 technologies relating to either audio or video
`
`09:25 12 recognition?
`
`09:25 13 A Yes. That falls under the image
`
`09:25 14 processing.
`
`09:25 15 Q I see.
`
`09:25 16 When did you start working in the image
`
`09:25 17 processing area?
`
`09:25 18 A So, for example, if you talk about the
`
`09:26 19 image processing where I do face detection and
`
`09:26 20 face recognition, that was two years, two or three
`
`09:26 21 years into Google.
`
`09:26 22 Q And over time am I understanding you
`
`09:26 23 correctly that various other aspects of image
`
`09:26 24 processing arose --
`
`09:26 25 A Yes.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 21 of 219
`
`20
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:26 1 Q -- in your research?
`
`09:26 2 A Yes, of course.
`
`09:26 3 Q Now, I mentioned the general category a
`
`09:26 4 moment ago of audio or video recognition. How do
`
`09:26 5 you understand that concept? What is that, in
`
`09:26 6 your estimation?
`
`09:26 7 A Are you talking about -- ask that
`
`09:26 8 question again.
`
`09:26 9 Q Sure.
`
`09:26 10 You mentioned that you understood that
`
`09:26 11 to be part of your work in image processing.
`
`09:27 12 A Right.
`
`09:27 13 Q Let's start with, when did you start
`
`09:27 14 having any focus in that context of audio or video
`
`09:27 15 recognition?
`
`09:27 16 A So when you say "video recognition," I
`
`09:27 17 don't actually work with videos. I work with
`
`09:27 18 images. So most of the work I did was on images,
`
`09:27 19 and images are a component of videos, right? So
`
`09:27 20 that's where I come in. And -- yes.
`
`09:27 21 Q When did you start having involvement in
`
`09:27 22 connecting that to video, if you will?
`
`09:27 23 A Probably 2006, 2005, somewhere around
`
`09:27 24 there. Probably 2006.
`
`09:27 25 Q And were you also involved in audio
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 22 of 219
`
`21
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:27 1 recognition?
`
`09:27 2 A So what's interesting is the way we did
`
`09:27 3 image recognition, we applied the same thing to
`
`09:28 4 audio.
`
`09:28 5 Q So let's talk about, first, image
`
`09:28 6 recognition then.
`
`09:28 7 A Okay.
`
`09:28 8 Q What -- can you describe to me generally
`
`09:28 9 the approach you are mentioning as to how you
`
`09:28 10 approached image recognition.
`
`09:28 11 A It's all statistics and machine
`
`09:28 12 learning, is the high-level approach. We used a
`
`09:28 13 lot of machine learning classifiers to understand
`
`09:28 14 what is in an image and did matching based on
`
`09:28 15 that.
`
`09:28 16 Q And so when you say "we used machine
`
`09:28 17 learning classifiers," give me an example of what
`
`09:28 18 you mean by that.
`
`09:28 19 A A neural network or AdaBoost.
`
`09:28 20 Q How did you use AdaBoost, this neural
`
`09:28 21 network, for this purpose?
`
`09:28 22 A You have a set of training images that
`
`09:28 23 have some particular object or image type. You
`
`09:29 24 train a machine learning classifier to be able to
`
`09:29 25 distinguish those images from other images.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 23 of 219
`
`22
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:29 1 Q So is this a system where essentially
`
`09:29 2 you provide some constellation of two or more
`
`09:29 3 images to this system and then it looks for
`
`09:29 4 mechanisms to discriminate between them and, over
`
`09:29 5 time, using multiple sets, it develops some
`
`09:29 6 mechanism to systematically do so?
`
`09:29 7 A Usually you -- sorry, there's a
`
`09:29 8 distinguishing characteristic between train time
`
`09:29 9 and test time.
`
`09:29 10 Q Okay.
`
`09:29 11 A When you train, you present millions of
`
`09:29 12 images, right. So --
`
`09:29 13 Q And -- and do you present them all as a
`
`09:29 14 group or in some pattern -- groupings or
`
`09:29 15 subformations?
`
`09:29 16 A The way machine learning works is you
`
`09:29 17 usually separate it out between the class you wish
`
`09:29 18 to distinguish and the others.
`
`09:29 19 Q And so give me an example, kind of on an
`
`09:30 20 orders of magnitude --
`
`09:30 21 A Okay.
`
`09:30 22 Q -- when you're talking about
`
`09:30 23 presenting -- I assume we're talking about still
`
`09:30 24 images, to this AdaBoost network.
`
`09:30 25 A Yes.
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 24 of 219
`
`23
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:30 1 Q Give me an example of how that, in a
`
`09:30 2 practical sense, proceeds.
`
`09:30 3 A So, for example, let's say I wanted to
`
`09:30 4 detect images that have faces in them. I will
`
`09:30 5 give it millions of examples of images with faces
`
`09:30 6 and even more billions of examples of images
`
`09:30 7 without faces, and it statistically learns to
`
`09:30 8 recognize the difference.
`
`09:30 9 Q What's the sort of output, if you will,
`
`09:30 10 of this kind of a process? What do you get from
`
`09:30 11 having the neural network perform this training?
`
`09:30 12 A Ideally, you will get a yes or no, it
`
`09:30 13 has a face.
`
`09:30 14 Q And do you -- does that provide you with
`
`09:30 15 some tool or mechanism that can then be applied
`
`09:30 16 elsewhere to essentially perform these kinds of
`
`09:30 17 recognitions, some sort of algorithm for the
`
`09:31 18 discrimination?
`
`09:31 19 A Sorry, could you explain.
`
`09:31 20 Q Well, the output -- I understand, the
`
`09:31 21 neural network, ideally you -- it is trained to
`
`09:31 22 discriminate between an image with a face and
`
`09:31 23 without a face in your example, correct?
`
`09:31 24 A Correct.
`
`09:31 25 Q Does the system also then provide you
`
`Case 1:14-cv-02396-PGG-SN Document 241-4 Filed 11/12/20 Page 25 of 219
`
`24
`
`SHUMEET BALUJA, Ph.D. - CONFIDENTIAL
`
`
`
`09:31 1 with some kind of mechanism or algorithm to
`
`09:31 2 implement on another system, or do you then simply
`
`09:31 3 have the neural network that processes queries in
`
`09:31 4 the future for further use?
`
`09:31 5 A Sorry, the reason I don't -- I can't
`
`09:31 6 answer that, I don't know what you mean by "on
`
`09:31 7 another system."
`
`09:31 8 Q Okay. So let me go back a step then.
`
`09:31 9 A Okay.
`
`09:31 10 Q What was the purpose of training the
`
`09:31 11 neural network to discriminate, in your example,
`
`09:31 12 between faces and -- images with faces and without
`
`09:31 13 faces?
`
`09:31 14 A Right. The purpose was to find images
`
`09:31 15 of the faces.
`
`09:31 16 Q But to what end?
`
`09:31 17
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
