Filed on Behalf of NETWORK-I TECHNOLOGIES, INC.
`
`By: Charles R. Macedo (Reg. No. 32,781}
`Brian A. Con1ack{Reg. No. 45,343}
`Amster, Rothatein & Ebenstein LLP
`90 Park Avenue
`
`New York, NY 10016
`Telephone: (212) 336-80?-=1
`cmacedo@are1aw_com
`N l-Google- [P R[@areIaw.c om
`
`Gregory Dovei (admitted pro hnc vice)
`Dove] 3.5 Loner, LLP
`201 Santa Monica Bl'«.rd., Suite 600
`Santa Monica, CA 9040]
`
`Telephone: (310) 656-7066
`gr_eg{n:do*_u'elIaw.con1
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`GOOGLE INC.
`
`Petitioner
`
`‘U.
`
`NETWORK-I TECHNOLOGIES
`
`Patent Owner
`
`Cases IPR20I5-003-=13, IPR20I5-00345, IPR20I5-0034?, and IPREIH 5-00343
`
`Patents 3,640,179, 3,205,237, 3,010,983, and 8,656,441
`
`DECLARATION OF DR. GEORGE I-(ARYPIS
`
`NETWORK-I EXHIBIT 2005
`
`Google Inc. v. Network-I Technoiogies, Inc.
`
`IPR20l5-00345
`
`Page ] of292
`
`
`
`By: Charles R. Macedo (Reg. No. 32,781}
`Brian A. Con1ack{Reg. No. 45,343}
`Amster, Rothatein & Ebenstein LLP
`90 Park Avenue
`
`New York, NY 10016
`Telephone: (212) 336-80?-=1
`cmacedo@are1aw_com
`N l-Google- [P R[@areIaw.c om
`
`Gregory Dovei (admitted pro hnc vice)
`Dove] 3.5 Loner, LLP
`201 Santa Monica Bl'«.rd., Suite 600
`Santa Monica, CA 9040]
`
`Telephone: (310) 656-7066
`gr_eg{n:do*_u'elIaw.con1
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`GOOGLE INC.
`
`Petitioner
`
`‘U.
`
`NETWORK-I TECHNOLOGIES
`
`Patent Owner
`
`Cases IPR20I5-003-=13, IPR20I5-00345, IPR20I5-0034?, and IPREIH 5-00343
`
`Patents 3,640,179, 3,205,237, 3,010,983, and 8,656,441
`
`DECLARATION OF DR. GEORGE I-(ARYPIS
`
`NETWORK-I EXHIBIT 2005
`
`Google Inc. v. Network-I Technoiogies, Inc.
`
`IPR20l5-00345
`
`Page ] of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`Table of Contents
`
`Background to thy opinions in this Declaration. ........................................... ..I
`
`A.
`
`B. Assignment.
`
`C- Approach.
`
`D.
`
`Understanding ofthe law.
`
`Ii.
`
`Summary ofthe [PR Patents and asserted
`
`A.
`
`The IPR
`
`1.
`
`2.
`
`3.
`
`4.
`
`‘23? pater1t{Ex. 100] ‘23? IPR} ............................................... ..8
`
`‘988 patent{Ex. 100] "983
`
`‘|‘?9 patent{Ex. 1001 ‘H9 IPR) ............................................. ..ll
`
`‘~44! patent (Ex. 100] ‘-441 IPR} ............................................. ..l3
`
`B.
`
`The asserted
`
`I.
`
`2.
`
`3.
`
`4.
`
`5.
`
`Overview ofGhias—Ex. ltllt} (addressed in the ‘Z37’, ‘93S,
`
`‘W9, and ‘4-4] IPRs].
`
`-16
`
`Overview oflwamut-a—Ex. 1012 [addressed in the ‘Z3? and
`
`‘Q88 lPRs). .............................................................................. .. 19
`
`Overview ofConwe1i~—Ex. I009 (addressed in the ‘ I ‘I9 and
`‘£141
`
`22
`
`Overview of Phi|1vaw—Ex. I014 {addressed in the ‘ 1?‘) and
`544] |PRs as a secondary reference}. ...................................... ..24
`
`Overview ofChen—Ex. 1008 (addressed in the ‘23?‘' [FR as a
`secondary ret'erence}.
`
`III.
`
`IV.
`
`General
`
`General concerns with the IPR Petitions and Dr. Moulin's
`
`Declarations (Eats. 1004 in each
`
`ClaimConstructions............
`
`A.
`
`suh-linear {‘23'.=' patent).
`
`1.
`
`2.
`
`3.
`
`the words used in the construction: “size ofthe dataset" ........4l
`
`‘E3?
`
`Petitioner’s
`
`B.
`
`non-exhaustive search (‘Z31 ‘Q88, ‘N9. and ‘44l patents) ............. ..54
`
`i
`
`Page 2 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`l.
`
`2.
`
`The Board’s preliminary construction of “non-exltaustive
`search” is consistent with the understanding ofone ofordinary
`skill in the art: “a search that locates a match without a
`
`comparison ofall possible matches.”
`
`The Board properly rejected Petitioner’s assertion that a “non-
`exhaustive search" should be construed as “a search that locates
`
`a match without conducting a brute force comparison ofall
`possible matches. and all data within all possible matches.” ...56
`
`C.
`
`D.
`
`neighbor search I identifying a neighbor I neighbor 2’ near neighbor
`(‘23'i', “J88, “ I79, and ‘r-141
`
`approximate nearest neighbor search {*2}? patent}.
`
`1.
`
`2.
`
`VI.
`
`“identifying a close match that is not necessarily the closest
`match"
`
`“sublinear"
`
`A.
`
`‘Z37 Ground l: The instituted claims ofthe ‘Z37 patent are ttot
`anticipated by lwamura.
`
`l.
`
`2.
`
`3.
`
`4.
`
`5.
`
`sub-linear time seat'ch (claims elements lib] and 5(b.2}} ....... .156
`
`approximate nearest neighbor search {claim elements 9{b} and
`
`nonexhaustive search {claim element 25[b)}.
`
`identify a neighbor it near neighbor {claims elements l[b}, 5[b),
`and
`
`sublinear approximate nearest neighbor search (claim element
`
`13.
`
`‘Z3’? Ground 2: The instituted claims ofthe ‘23T patent at'e not
`anticipated by Ghias.
`
`I.
`
`2.
`
`sublinear time search [claim elements lib) and 5(b.2}]. ........I34
`
`approximate nearest neighbor search (claim elements 9{b1I and
`l3I[b.2}] .................................................................................. .. I 55
`
`C.
`
`‘237 Ground 3: The instituted claims ofthe ‘237 patent are not
`obvious over lwamura and
`
`I66
`
`‘it'll.
`
`‘938patent.
`
`A.
`
`‘993 Grottnd l: The instituted claims ofthe ‘988 Patent at'e not
`
`anticipated by Ghias.
`
`[68
`
`ii
`
`Page 3 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and IPR20l5-00348
`
`Declaration of George Karypis
`
`l.
`
`2.
`
`3.
`
`non-exhaustive search [claim elenient l5{b}}.........................169
`
`search identifying a neighbor [claim element I5(b]}I....._......_. I36
`
`determining an action based on the identification [claim
`element l5[c}). ...................................................................... .. [90
`
`‘£3138 Ground 2: The instituted claims ofthe ‘998 patent are not
`obvious over Ghias.....-......-... --
`..
`
`I 93
`
`‘983 Ground 3: The instituted clairns ofthe ‘998 patent are not
`anticipated by Ii.va1nura.l94
`
`B.
`
`C.
`
`I.
`
`2.
`
`non-exhaustive search {claim I5(b)} ..................................... .. I 94
`
`identifying a neighbor {claim
`
`VIII.
`
`‘ IT‘?patent.
`
`A.
`
`‘I TO Ground I: The instituted claims ofthe ‘W9 patent are not
`anticipated by
`
`I.
`
`2.
`
`neighbor search {claims I. I3,
`
`non-exhaustive search [claims I, I3.
`
`B.
`
`‘ I79 Ground 2: The instituted claims ofthe ‘ 179 Patent are not
`
`obvious in view of Ghias and
`
`I.
`
`2.
`
`non-exhaustive search [claims I, I3,
`
`neighbor search (claims I, I3,
`
`IX. "4-4]
`
`A.
`
`‘44I Ground 1: The instituted claims oftlie ‘dill Patent are not
`
`anticipated by Conweil .................................................................... .256
`
`I.
`
`2.
`
`neighbor search {claims I. I3.
`
`non-exhaustive search [claims I, I3.
`
`B.
`
`‘44l Ground 2: The instituted claims of the ‘MI Patent are not
`
`obvious over Ghias and
`
`I.
`
`2.
`
`non-exhaustive search [claims I, I3,
`
`neighbor search (claims I, I3,
`
`iii
`
`Page 4 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345. IPRZGIS-0034?, and lPR2{}l5-00348
`
`Declaration of George Karypis
`
`L George Karypis, declare:
`
`[ am making this Declaration at the request of Patent Owner Network-l
`
`Technologies. Inc. in the Fo1|owing!mcr!’rrrre.s- Reviews ofU.S. Patent Nos.
`
`3,2fl5,237 {‘237 patent}, 3,[llfl,988 ["988 patclttl, 8,640,179 (‘I79 patent], and
`
`3,656,441 {‘4-4| patent} (collectively the “]PR Patents”):
`
`-
`
`-
`
`I
`
`Ir
`
`IPR2o1s-oo34s P23? patent},
`
`[PR2{1|I5-lJt]34? (‘983 patent},
`
`IPRZDIS-00343 If I 79 patent], and
`
`IPRZDIS-00348 (‘A41 patent].
`
`(collectively the “lPRs"], all initiated by petitioner Googie lnc. [“Petitioner").
`
`I.
`
`Background to my opinions in this Declaration.
`
`A.
`
`Expertise.
`
`l.
`
`I am a Professor in the Department of Computer Science and
`
`Engineering at the University of Minnesota.
`
`I hold a Ph.D. in Computer Science
`
`from the University of Minnesota. granted in 19%.
`
`I began my post-graduate
`
`school career as a Research Associate in my current department.
`
`I became an
`
`Assistant Professor in I999. an Associate Professor in 2004. and a Professor in
`
`2009.
`
`I teach courses in Algorithms and Data Structures. Parallel Programming.
`
`and Data Mining. among other subjects.
`
`I
`
`Page 5 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`2.
`
`I am a inember ofthe Editorial Boarcl ofa number ofacaclemic
`
`joumais, and I have chaired a number ofacademic conferences.’
`
`I am a co-author
`
`of the books iiiti-ridut-rim: to Priro.-'.-’ei if ‘riiiipiitmg, and .-"nrror.~’ric'iioii to .“rrrrih'c.-’
`
`{'riiiipntiiig.' f)e.rign and x€l'.vr,1.-’_v.si.s ofxi .-’gtJriH'mi.s'.
`
`1 am an author of more than 80
`
`published journal papers, and tnoI'e than I 15 published conference papers:
`
`Representative academic conferences include:
`
`I Program Connnittee co-C hair of the AC M Recoininender Systems
`
`Conference (RecSvs‘ 13), Hong Kong, China {Z013};
`
`a Program Committee co-Chair of the l3"‘Inten1ational Conference on Data
`
`Mining [ICD1'v“I), Da|1as,Texas(Decetnber 2013); and
`
`a Program Coinrnittee Co-Chair ofthe international Conference on Data
`
`Science and Advanced Analytics (DSAA 2014), Shanghai, China, {November
`
`2'31-=1}.
`
`l'\—l
`
`Representative papers include:
`
`Ir
`
`"i.2Kniig.'
`
`i"'ti.st .-";'xrir.'t K-Nearest Neighhtir (irrnili f'riii.s'ti'm;'Iitiii will: L2-
`
`Norm .-"i-iniiiig " David C. Anastasia and George Karypis, 24"‘ ACM
`
`International Conference on Infonnation and Kiiowledge Management
`
`tCIKM). Melbourne, Australia (2015).
`
`2
`
`Page 6 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`3.
`
`I have also developed a number of SUfiWfll‘E svstents for a variety of
`
`fiinctions, including software for analyzing high-diinensionai data sets. A copy of
`
`my curriculum vitae is attached as Exhibit A to this Declaration.
`
`it contains a
`
`more complete listing ofmy professional activities and background.
`
`B.
`
`Assignment.
`
`4.
`
`[ have been retained by Patent Owner Network-I Technologies. Inc.
`
`as a technical consultant.
`
`I am being compensated for my time at my standard
`
`consulting rate of $350 per hour.
`
`1 am not receiving any compensation that
`
`depends on the outcome ofthe IPRs.
`
`5.
`
`This declaration addresses the validity of:
`
`I
`
`“i'.2Al’.'
`
`I-it.i'! I ’o.i-me .'~1‘iii1i.-’rm'i_1r .'s'corci'i with l’refi.r 12-2 Norm B.1’Jt‘.1'll‘d.fl‘" David
`
`Anastasiu and George K}11‘}’[JiS, 30"‘ IEEE lntemational Conference on Data
`
`Engineering (ICDE), pp. 784—T95 (ZDI-4].
`
`I
`
`“If 'rJnipr.rri.i'tm of'I)e.sc*r.iprrJr S_nac‘e.v_fi*.i1" ('heiii.r'cau" ( 'rni:r_nr.innt1' Retrieval avid
`
`(‘!o.~r.o)1'ic'oiitiri“ Nil-cii Wale and George Karypis, IEEE International
`
`Conference on Data Mining [ICDML pp. 6'i'8—68‘-) (2006).
`
`I
`
`“t':'mpiricct.-’ rind i'iI'ieov'e!.ic'o1' f 'mnpctri.i'tm.-.' tgfSe1'eci'eta’ {"ri!critJit .-'-i1tnc!i'oii.i'_fi)t'
`
`.’Jo1:.'mnen1'I".-’i.1.-.'1'r:1ri1'1g” Ting Zhao and George Karvpis, Machine Learning, 55,
`
`pp. 311-331 12004}.
`
`3
`
`Page 7 of292
`
`
`
`IPRZDIS-00343, [P112015-00345, IPRZGIS-0034?, and lPR20l5-00348
`
`Declaration of George Karypis
`
`-I
`
`claims 1, 3-5, T-9, 11-13, 15, 16, 21-2?, 29, 30, 33, 34, 35, 37', and 38 of
`
`the ‘Z3? patent;
`
`claims 15—l?, 21-23, 31-33, 51, and S2 ofthe ‘998 patent;
`
`claims 1-3, I5, 8-14, 18, 1'3‘, 21-2?, 29-31, and 34-3? ofthe ‘ H9 patent; and
`
`claims 1-3, 6, 8-14, 13, 1'9, 21-2?, 29, and 30 ofthe ‘£141 patent-
`
`C.
`
`Approach.
`
`6.
`
`To develop my opinions, I have read:
`
`the four [PR Patents [the ‘23'?, ‘938, ‘I39, and ‘44] patents];
`
`the four Petitions for Inter P-:.r.*'.*r»:.s Reviews;
`
`the exhibits accoinpanyirig the Petitions, including the four Declarations of
`
`Dr. Pierre Mou1in{Eas. 1004 in each {PR}:
`
`the four Decisions instituting the IPRs; and
`
`the testirnony oi‘Dr. Pien'e Mouliu, dated August 19-20, 2015 (Ex. zooms}.-‘
`
`Ir
`
`-I
`
`I
`
`-
`
`Ir
`
`I
`
`1-
`
`-
`
`[n this Declaration, I identify the specific Petition, Declaration, and Decision
`
`that [ ant citing by including the corresponding patent abbreviation in a
`
`parenthetical. For example, I refer to the Petition addressing the ‘23? patent as Pet.
`
`(‘E37’) at X; and the Moulin Declaration addressing the ‘ 1 T9 patent as Moulin
`
`Decl. 1," [T9] 11X. Because there is only one Dr. Moulin Deposition transcript for all
`
`4
`
`Page 8 of292
`
`
`
`IPRZDIS-00343. IPRZUIS-00345. [P112015-UU34'r'. and lPR20l5-00348
`
`Declaration c-fGeorge Karypis
`
`T.
`
`In addition, I relied oI1 my personal knowledge and experience with
`
`both research and development in the technology underlying the IPR Patents and
`
`the art asserted against the IPR Patents.
`
`D. Understanding ofthe law.
`
`8.
`
`My understanding regarding the law as applicable to this Declaration
`
`is based on my discussions with counsel.
`
`I have included in the text of my
`
`Declaration quotations from or references to certain legal cases or statutes that
`
`were provided to me by counsel to provide me with an understanding of the
`
`relevant law.
`
`E.
`
`9.
`
`Person of ordinary skill in the art.
`
`Through my education, experience and training, in academia and
`
`industry, and my analysis ofthe [FR Patents, I am familiar with the knowledge ofa
`
`person ofordinary skill in the field of the IPR Patents at the time ofinvention in
`
`2000.
`
`It}.
`
`For the purposes ofthis Declaration, I am ofthe opinion that a person
`
`ofordinary skill in the art with respect to the [PR Patents is a person with a
`
`Bachelor’s degree in computer science, mathematics, or a similar discipline and
`
`two to three years of relevant experience. or a graduate degree in the same area.
`
`four lPRs (Ex. 2006], I simply refer to Dr. Mouliifs deposition testimony as
`
`Moulin Depo. Z.
`
`5
`
`Page 9 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?, and lPR20l5-00348
`
`Declaration of George Karypis
`
`In determining what would be the level ofordinary skill in the field as ofthe 2000
`
`time frame, 1 considered the following:
`
`{a}
`
`the educational level ofthe inventor, [ngemar J. Cox (it is my
`
`understanding that Dr. Cox has a bachelor's degree in electronics and
`
`computer science from University College London {I930} and a PhD.
`
`from Oxford {[9831};
`
`[b]
`
`the type of problems encountered in the art—i.c., how to identify a digital
`
`work without modifying the work (see e.;:., ‘23';', 1:30-36);
`
`{C}
`
`the prior art solutions to those problems {see e.g., ‘23':', 1:3?-4:4, and the
`
`prior art asserted by the Petitioner in the IPRs addressing related problems
`
`involving searching, matching, and identifying melodies, audio files, and
`
`other digital files within clatabases—Conwell, Ghias, Iwamura, Chen, and
`
`Philyaw};
`
`{d} the rapidity with which innovations are made [based on my observations
`
`over the past 2U plus years, major innovations in content identification
`
`occur about every 5 to 10 years];
`
`(e) the sophistication of the technology {developing content identification
`
`solutions is a moderately sophisticated technology); and
`
`6
`
`Page 10 of 292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`{ii} the educational level of workers in the field (workers in the field generally
`
`had have at least a bachelor’s degree in computer science, mathematics or
`
`a similar discipline and at least two to three years ofrelevant experience].
`
`I 1.
`
`Based on these factors, it is my conclusion that a person ofordinaly
`
`skill in the art at the time would have been a person with a Bache1or’s degree in
`
`computer science, mathematics, or a similar discipline and two to three years of
`
`relevant experience, or a graduate degree in the same or related area.
`
`I2.
`
`I note that Dr. lvloulin suggests that the person of ordinary skill in the
`
`art “would have been highly skilled, and typically would have possessed at least an
`
`MS. in computer science, electricai engineering, or mathematics; knowledge of
`
`video and audio processing techniques; and 1-2 years of experience i11 audio,
`
`video, or image processing." See .-:-.'._r:., Pvioulin Decl. (‘E37’) 117'; Pet. {‘23?'] at 4. Dr.
`
`1‘v’Ioulin‘s opinion as to the person of ordinary skill in the art is similar to mine with
`
`respect to the degrees and years of experience, but I note that: Ifl] Dr. Moulin does
`
`not provide any rational underpinnings for his opinion; and (2) the phrase “highly
`
`skilled” used by Dr. Moulin in his description is a relative term and Dr. lvioulin
`
`does not provide the context for this phrase.
`
`7'
`
`Page ll of 292
`
`
`
`[PFt20l5-00343; lPR20l5-00345; IPR201S-0034?; and lPR20l5-00348
`
`Declaration of George Karypis
`
`II.
`
`Summary of the [PR Patents and asserted art.
`
`l3.
`
`In this Declaration:
`
`an
`
`I use the term “v_vp$" to mean the item (e_;;;., a digital audio or image tile} to
`
`be identified using the search {see e.g., ‘Z31’, 6:5 1-56; ‘988, 'l':l T-20; ‘ I T9,
`
`6:18-21; 544}; 6:-49-52];
`
`-
`
`I use the term " ’ to mean one ofthe units in the reference database
`
`that the extracted features of the work may be compared to {see
`
`‘23?',
`
`6:16-20; ‘E188; 6:46-50; ‘ [T9, 6:21-24; ‘-441; 6: I5-I 3}; and
`
`-
`
`[ use the term ‘ “J,”or to mean the collection of
`
`all records to be sea1'elied(.-we e.g., “23'r', 6:23-30; ‘988, 6:50-60; ‘I79, 6:30-
`
`36; ‘-441, 6:24-30}.
`
`A.
`
`The IPR Patents.
`
`I4.
`
`Each IPR Patent (the ‘23'?, ‘ 1 T0, *938, and ‘£141 patents} involves a
`
`search that compares features from a given work to records in a reference database
`
`of potential matches to identify an action to be taken.
`
`I.
`
`‘Z3? patent (Ex. I001 ‘23? IPR}.
`
`15.
`
`The independent claims ofthe ‘Z37 patent include the following
`
`elements:
`
`[I] receiving or obtaining Features extracted f'rom a work;
`
`8
`
`Page 12 of 292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`[2] identifyittg the work using the extracted features. to perform a search of
`
`the database, where the search is:
`
`I
`
`II
`
`II
`
`Ir
`
`a sub—]inear time search to identify a neighbor [claims I and 5);
`
`an approximate nearest neighbor search {claims 9 and 13};
`
`a non-exhaustive search
`
`to idetttify a near neighbor [claim 25}; or
`
`a sublinear approximate nearest neighbor search [claim 33}; and
`
`[3] either ti) transmitting infonnation about the identified work to the client
`
`device, or (ii) determining an action based on the identity ofthe work.
`
`16.
`
`The invention claimed in the ‘Z3? patent includes two key features:
`
`1?,
`
`Feature 1: Although the language varies among the claims. each
`
`claim requires that the *‘identifying“ he performed based on a search that has two
`
`properties:
`
`[ I ) a suh-linear or non-exhaustive property [reflected in the underlined
`
`language):
`
`I
`
`I
`
`sub-linear tinte search
`
`to identify a neighbor {ciaims I and 5};
`
`aggroximate nearest neighbor search {ciaims 9 and 13];
`
`I non-exhaustive search
`
`to identify a near neighbor {claim 25}; and
`
`Ir
`
`sublinear approximate nearest neighbor sea1'ch (claim 33).
`
`(2) a neighbor property {reflected in the underlined language}:
`
`9
`
`Page 13 of 292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`I-
`
`identify a neighbor [claims 1 and 5):
`
`I approximate nearest neighbor search {ciaims 9 and 13}:
`
`It non-exhaustive search
`
`to iIIiEI]tif}" a near neighbor {claim 25]; and
`
`-0
`
`sublineat approximate nearest neighbor search (claim 33).
`
`18.
`
`Feature 2: The system tnust either determine an “action" based on the
`
`identification {claims 25 and 33); or transmit information about the identified
`
`media work to a “client device" {claims 1, 5, 9, and 13).
`
`It is not sufficient to
`
`simply identify a match. Rather, an action must also be identified or information
`
`about the identified work must be transmitted to the client device.
`
`2.
`
`‘988 ]1atent{Ex. H10] ‘988 IPR).
`
`I9.
`
`The independent claims ofthe “J38 patent include the following
`
`elements:
`
`[I] extracting features from a work;
`
`[2] identifying the work based on the extracted features by perfonning “a
`
`non-exhaustive searclt identifying a neighbor;”'
`
`[3] detemtining an action based on the identity oftlte work; and
`
`[4] performing the action.
`
`20.
`
`The invention claimed in the ‘988 patent includes two relevant
`
`distinguishing features:
`
`ID
`
`Page 14 of 292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`[1] the “ide11tifying" must be peifornied using a “non-exhaustive search
`
`identifying a neighbor;” and
`
`(2) the system must “deterrnin[e] an action” and “perFonn[] the action“
`
`based on the identity ofthe work.
`
`It is not sufficient to identify a match.
`
`Rather, “an action” associated with the match Inust be “detern1in[ed]”
`
`and “perform[ed]." “.988, claim 15.
`
`2].
`
`I note that the Board did not institute trial for independent ciaini I of
`
`the ‘.983 patent and any claims dependent on claim I. Accordingly, I do not
`
`address these claims in this Deciaration.
`
`3.
`
`‘I79 patent (Ex. 100] ‘I79 IPR).
`
`22.
`
`The independent claims ot'the ‘H9 patent [claims I, 13, and 25}
`
`inciude the following five elements for identifying a work and perfonning a
`
`corresponding action:
`
`[I] a database comprising: {a] electronic representations of works; and {I3}
`
`eiectronic data related to an action corresponding to works;
`
`[2] obtaining extracted features of an unknown work;
`
`[3] iCiBfltif}"il1g the unknown work by comparing the extracted features and
`
`electronic representations using a “non-exhaustive neighbor search ;"
`
`1 I
`
`Page 15 of 292
`
`
`
`IPRZUIS-00343, IPRZUIS-00345, IPR20l5-0034?, and lPR2{}15-00348
`
`Declaration of George Karypis
`
`[4] determining an appropriate action based on the electronic data related to
`
`an action; and
`
`[5] associating the determined action with the identified work.
`
`‘ [T9, claims i, I3. and 25.
`
`23.
`
`The ciaimeti steps are illustrated in Figure I:
`
`”5"""~’“E
`E5(TFt.F.t2T|DN
`oPEi=..I.TIoH:5
`
`FEATURE TO
`woatt In
`-,_.,,_-_,G,H,5
`PER,-WW3.
`Di.‘-|'loB-I‘-\5E
`I5-EP£R.fl.I'|DH
`GPERATEPIH
`
`I-hAT|.J'£h
`[\I'ECTDR]t.|I.TRAET1Dfi>l
`UPER-5-TIIUNEEJ
`
`FEATLRE
`If"|"E|iiTC|'H.] LDDi4.U1'
`DPERATIONISJ
`
`I
`
`I
`"‘
`'
`''''I
`rrwrunsret micron:
`
`.
`
`_
`
`J.
`
`'-t.
`
`"'~-
`
`'«"°‘l"-*350'3lfiTED
`|N'FUHH.N.T1DH LCQKUP
`|3"EH‘-"-\T|CiN[5-]
`
`1411
`
`135
`
`|:u.t.I.a-L55
`otneiutton
`Qpffigrggpqgi
`
`""-...__l
`
`-.._‘I.-~-..
`
`mama 11"“ 111
`
`ACTION
`|NiTlM'|DN
`DFERATIDIHSJ
`
`m
`FIGURE 1
`
`Figure 1 iilustrates {“for work @t2"}:
`
`Iv “feature {vector} extraction operation[s}”{14fl] that extract features from the
`
`work {‘ l'i'9, 6145-437}:
`
`12
`
`Page 16 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`I
`
`“feature [vector] lockup operatiouts)" [150] that identify the work by
`
`searching for a matching feature vector {‘ I79, 6:50-52};
`
`Ir
`
`“work-associated infonnatioo lookup operation(s}” (160) that retrievets)
`
`associated infonnation, such as an action {‘ 1 T9, 6:55-58}; and
`
`II
`
`“action initiation operationtsl" IE 170} that performtsl some action based on
`
`the associated information F179, s:s3-so).
`
`24.
`
`The invention claimed in the ‘ITS’! patent includes two relevant
`
`distinguisliing features:
`
`[I] the “identifying,” must be performed by comparing the extracted features
`
`to the electronic representations using a “non-exhaustive neighhor
`
`search:” and
`
`[2] the system must determine or associate an ‘‘action‘‘ based on the
`
`identified work.
`
`It is not sufficient to simply identify a tnatch. Rather,
`
`“an action” associated with the match must be “determined” or
`
`“associated."
`
`‘I79, claims 1, I3, and 25.
`
`4.
`
`‘MI patent (Ex. lllfll ‘44l IPR}.
`
`25.
`
`The independent claims of the ‘-441 patent (claims 1, 13, and 25}
`
`include the following five elements for identifying a work and performing a
`
`13
`
`Page 1? of 292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, [P112015-0034?, and lPR2015-00348
`
`Declaration of George Karypis
`
`corresponding action:
`
`[1] a database with (a) first data related to records, and {is} second data
`
`related to action information corresponding to the records:
`
`[2] extracting features from a work;
`
`[3] identifying the work by comparing the extracted features and the data
`
`related to the records using “a norkexltaustive neighbor search;"
`
`[4] determining an action based on the identity ofthe electronic work; and
`
`[5] perfonning the action.
`
`‘44] , claims 1, I3, and 25.
`
`26.
`
`The claimed steps are illustrated in Figure 1:
`
`l4
`
`Page 18 of 292
`
`
`
`IPRZUIS-00343, IPRZUIS-00345, IPR20l5-0034?, and lPR2{}l5-D0348
`
`Declaration of George Karypis
`
`FEATURE
`;'..'E|!.'.'TfiI"¥;|E.'|.‘|'|'-'f.-'ul3-"|'l|L'Ir-i
`UFLHAIIDNL-3|
`
`FE-I‘-‘.'|'LIFlE
`I
`I
`°'s:;a¢a":-t
`L
`
`FEATURE
`Eutnamiou
`‘3"E"-“"'¢'-"'55
`
`FEATURE to
`“MK,”
`Tm“
`-oPEFu.tIoh-:5
`
`l.‘nh'I'.i.P.F$E
`EE HERA‘-l1|I;|N
`oasnatuorcsp
`
`'I'I"||}
`IHFDRHATIDH
`
`1»
`Fehruncrsmcctoaz 4 ~I|.|''''‘'1_12
`“- 1.35
`I'.I.il.TAfl‘.fi.'5.E
`
`woar:..-ussncunsn D .;~,g_..EH,.L.,.:.,.
`ll'l:UFl|\l.AT‘.|I'.‘IH LOIDILUF
`upEfi_.|._-hams]
`1.-noaac
`DFEFt.-1TI-:1MI$I
`:.‘l_
`cisoauatnon _
`
`lfivq
`....
`m nssuocuteonuronsutnouug ncnom qu"-- in
`
`-a
`
`action
`INITIATICIN
`l3F'EF|.fi.T||.".|H|51
`
`mg
`
`Figure 1 illustrates (“for work @t2’’}:
`
`1- “feature ( vector} extraction operation[s}" (140) that extract{s} features from
`
`the work (441 , 6:39-41};
`
`Ir “feature {vector} Iookup operationfs)" (150) that identify the work by
`
`searching for a matching feature vector {‘-441, 6:44-48};
`
`- “work-associated infonnation lockup operation[s}" (160) that retrie-acts)
`
`associated information, such as an action (‘-441 , 6:49-51}; and
`
`II
`
`“action initiation operation{s}" ( ITO} that perforrn[s) some action based on
`
`the associated information [‘44l, 6:52-54].
`
`15
`
`Page 19 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345. IPRZGIS-0034?. and lPR2{}l5-00348
`
`Declaration of George Karypis
`
`2?.
`
`The invention claimed in the ‘£141 patent includes two relevant
`
`tiistinguishing features:
`
`(I ) the "identifying" must be performed by comparing the extracted features
`
`to the electronic representations using a “non-exhaustive neighbor
`
`search?’ and
`
`(2) the system must determine or associate an “action" based on the
`
`identified work.
`
`It is not sufficient to simply identify a match. Rather.
`
`“an action" associated with the match must be “determined" or
`
`“associated."
`
`‘£141. claims I, I3. and 25.
`
`B.
`
`The asserted art.
`
`28.
`
`The four lPRs address three primary references and two secondary
`
`reterences.
`
`I address each reference in turn. starting with the pmuary references
`
`and then turning to the secondary references.
`
`I.
`
`Overview of Ghias—Es. ]l]I[l {addressed in the ‘Z37. ‘983,
`‘I79. and "44! {PR5}.
`
`29. Ghias [Patent No. 5,S':'4,o86) discloses “an apparatus [for] searching
`
`melodies.” Gliias, Abstract. As illustrated in Figure l ofGhias, a “tune 12 is
`
`hummed by a person [3 into a microphone 20.” Ghias, 2:4 I -42.
`
`16
`
`Page 20 of 292
`
`
`
`[PR2Di5-00343, [PR2{JI5-00345, [P112015-0034?, and lPR2{}15-00348
`
`Declaration of George Karypis
`
`cQ|,qP|_|TEfi
`
`|’LEI'W_P7A§§' F ‘ ‘ ‘ ‘ ‘-
`|
`¢‘I..IEI!C
`I
`————— ——
`}w.wELE'rI '..F."-_TEE_II SPLINE I
`I
`I
`' TFi:iii:{iiIin ' ————— ——. I""$‘I’i?:'LtuETi
`""" " LSEEJEI twsesi
`
`
`QUERY ENGINE
`
`AUTDDGRRELATION
`BHSED PITCH
`DETECTOR
`
`
`
`
`
`
`RANKED LIST
`GF MATCHING
`H:EI.OC.1|E.5
`
`%:'_'IG.I
`
`The data from the microphone is Fed into “a pitch tracking module 22 in computer
`
`16” which extracts “a contour representation” ofthe melody (23). Ghias, 2:41-50.
`
`The computer uses a “query engine 24” which “searches the melody database I4."
`
`Ghias, 2:50-52. The disclosed search can produce a ranked list ofmatching
`
`meIodies——-“ranked by how well they matched the query" {Ghias, 6:60-63} as
`
`illustrated at 26.
`
`30.
`
`As I explain below in detail, all searches disclosed in Ghias are linear
`
`[not sub-iinear} with respect to the size of the data set being searched.
`
`In
`
`addressing “the problem of approximate string matching," Ghias identifies the
`
`running times ofseveral algorithms:
`
`1'?
`
`Page 2! of292
`
`
`
`[PR2Dl5-00343, [PR2{Jl5-00345, [P112015-0034?, and lPR20l5-00348
`
`Declaration of George Karypis
`
`1-Several Algorithms have been developed that address the
`pniliiern of approximate string matching. Running times
`:5 have ranged [rum Uimfi for the brute force algorithm to
`
`|
`
`ttfltfi orililogirni. where "ill" means "on the orderof." m I
`
`is the number of pitch difiierences in the query, and_ .
`E
`
`Ghias, 6:23-28.
`
`In each identified instance, the running time ofthe search is not
`
`sub-linear with respect to the data set. As clarified in this passage from Ghias (and
`
`as I address in detail below]:
`
`I
`
`“in is the number of pitch differences in the query” corresponding to the
`
`length of the query (highlighted in green in the passage above]; and
`
`-
`
`“n is the size ofthe string {song]" corresponding to the size ofa record being
`
`searched [highlighted in orange in the passage above}.
`
`3 l.
`
`The disclosed searches may be sub-linear with respect to the length of
`
`the query being searched “m
`
`the nttmber ofpiteh differences in the query.“
`
`Speeificaliy, the referenced search with a running time of‘O(niog(n1}) is subiinear
`
`with respect to “of” because it is a function of 1og(rn}), The disclosed searches,
`
`however, are never sub-linear with respect to “n. . the size of the string (song]" or
`
`the size ofthe data set {N} [i'.e., the number ofsongs to be compared}. Rather, the
`
`search time will grow linearly with each additional song to be searched and the
`
`length ofthe song.
`
`32. Also as I describe in detail below, the searches discioseti in Ghias are
`
`exhaustive rather than “ttonexhaustive." The “query engine 24” compares the
`
`18
`
`Page 22 of 292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`work {user input 23] to “all the songs" in the melody database I4 {the library].
`
`Ghias. 5:66-6:2. After searching all possible matches. the system “output[s] a
`
`ranked list ofapproximately matching melodies.” Ghias, 2:50-53.
`
`33,
`
`Finally, as I describe in detail below, the searches disclosed in Ghias
`
`aI'e not “neighbor” searches because the searches always necessarily identify the
`
`exact or closest match—they are guaranteed to identify an exact match or the
`
`closest match. Ghias does not identify any search in which an exact or the closest
`
`match is not guaranteed to be identified.
`
`2.
`
`Overview ol‘Iwan1ura—Ex. ]l]I2 [addressed in the ‘Z3? and
`"9S8 IPRs}.
`
`34.
`
`lwamura [Patent No. fi,l83,Dli}} discloses a “method to enable one to
`
`search for a song title when only its melody is known." Iwamura. Abstract. “A
`
`remote music database with melody information is searched for the melody entered
`
`by the user. using for example. a peak or differential matching algorithm.”
`
`lwamura, Abstract. Figure 1 illustrates “an example of a search interface”
`
`[lwamura. 2:45-46]:
`
`19
`
`Page 23 of 292
`
`
`
`lPR2Dl5-00343. lPR2{Jl5-00345, [P112015-0034?, and lPR2{}l5-00348
`
`Declaration of George Karypis
`
`FIG. 1
`
`:£.|llflI'I|1Il||'.|'l
`
`Ilalnap Search
`
`Trafimsrei Ft. scliumann
`
`35.
`
`Iwatnura discloses a searching algorithm that is designed to be more
`
`efficient than alternatives by matching up peak notes from the work to be identified
`
`with the peak notes of the records in the database when comparing the notes from
`
`the work to be identified with the notes in the records. “Peak notes are also
`
`detected and marked when the data base is built.” lwamura, 6:59-60. “A fast
`
`search is performed by using a peak or differential matching algorithm.“ Iwamura.
`
`I2: 1-2.
`
`36.
`
`As I explain in detail below, the search disclosed in Iwamura is
`
`exhaustive rather than the claimed “non-exhaustit.ie.“ “sublinear,“ or “approximate
`
`nearest neighbor“ search. While the individual comparisons ofa work and a
`
`record in the library can be more efficient using the “peak note“ approach
`
`20
`
`Page 24 of292
`
`
`
`IPRZDIS-00343, IPRZUIS-00345, IPRZGIS-0034?. and lPR20l5-00348
`
`Declaration of George Karypis
`
`disclosed in lwantura [ search speed can be increased]. in doing so each record in
`
`the library is searched as part ofthe disclosed algorithm and “[t]he reference
`
`melody that gives the least difference is returned as a search result.” lwamura.
`
`7:53-55.
`
`37. Moreover, the Boyer-Moore algorithm referenced in Iwamura
`
`searches “word by word from the beginning of the database to the end.” lwamura,
`
`9:5 I-55. As a result. as I explain in detail below. while the Boyer-Moore
`
`algorithm may be sublinear with respect to the length ofthe query {the work to be
`
`ide11tified],4 it is not sub-linear with respect to the relevant size ofthe dataset being
`
`searched.
`
`If the query pre-processing step of the Boyer-Ivloore algorithm is included as
`
`part ofthe execution time. then the algorithm may be linear in terms ofthe length
`
`of the query. If the query is used repeatedly, however, the pre-processing
`
`execution time will only be incurred once. One can think ofconcatenating all the
`
`database strings to give as an aggregate length of n. If“m” is the query length.
`
`then the worse-case complexity is Thetatm) (){n], which is linear with respect to
`
`both the database It and the query length m.
`
`2]
`
`Page 25 of 2.92
`
`
`
`[PR2Dl5-00343, lPR2{J]5-00345, [P112015-0034?, and lPR2{}l5-00348
`
`Declaration of George Karypis
`
`3.
`
`Overview of Conwell—Ex. 1l]l]9 (add ressed in the ‘I79 and
`‘=14! IP'Rs}.
`
`38.
`
`Conwell {Patent No. 6,970,366} discioses associating media content,
`
`such as MP3 files, with identifiers and URLs. Conwell, Abstract. As illustrated in
`
`Figure 3, the identifiers {e.g., "(J3-'-1”} are associated with corresponding URLs (e.g.,
`
`'!'.I-
`“www.songrnusic.co1nfcatalogf05o34.htm] }:
`
`E w
`
`anu.:son1n11u5ic.I:um!:a1ald-gJ'UEllJ14.htrrl
`www.:iupaflracks.oon1Jindexu‘ariis.tsu'ta.1fior.h'lt1I
`
`nnuI.u'.|DUd&'y'B.oom!rap!199'9M$?55fS4E.htm|
`
`vwvw.amu3io.oo1'rIl'Dé§Eié53r:.pdf
`
`www.cdw.oorrutmus|c.tEaaturad_CDsa"-ndax.html
`
`E vMw.sony1nu5ic.contF::ata|ogI{lI}231.hufl
`E wutnnI.pol5rgrarn.oomffranklirtFadf_‘2‘34.hh'n
`
`3‘-E}.
`
`Conwell discloses two approaches to identifying a work: {I}
`
`assigning identifiers, or (2) implicitly generating identifiers derived from the data
`
`using a hashing algorithm. Conwell, Abstract.
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
