Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 1 of 30
`Case 1:14-cv-02396—PGG-MHD Document 153-7 Filed 06/28/19 Page 1 of 30
`
`EXHIBIT F
`
`EXHIBIT F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 2 of 30
`
`(12) Ulllted States Patent
`Cox
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 8,640,179 B1
`*Jan. 28, 2014
`
`US008640179B1
`
`(54) METHOD FOR USING EXTRACTED
`FEATURES FROM AN ELECTRONIC WORK
`
`( * ) Notice:
`
`(75) Inventor: Ingemar J. Cox, London (GB)
`_
`(73) Ass1gnee: Network-1 Security Solutions, Inc.,
`New York, NY (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 8 days.
`patent is subject TO a terminal dlS-
`claimer.
`_
`(21) App1.No.. 13/338,079
`(22) Filed:
`Dec‘ 27’ 2011
`_
`_
`Related U‘s‘ Apphcatlon Data
`(63) Continuation of application NO 1 1/977’202, ?led on
`Oct 23 2007 HOW Pat NO 8 205 237 which is a
`continuation of application No. 11/445,928, ?led on
`Jun. 2, 2006, noW Pat. No. 8,010,988, Which is a
`continuation-in-part of application No. 09/950,972,
`?led on Sep. 13, 2001, noW Pat. No. 7,058,223.
`(60) Provisional application No. 60/232,618, ?led on Sep.
`14, 2000.
`(51) Int. Cl.
`H04N 7/1 73
`(52) us CL
`USPC __________ __ 725/115; 72 5 /1 10; 725/114; 72 5 /1 16
`(58) Field of Classi?cation Search
`None
`See application ?le for Complete Search history
`References Cited
`U-S- PATENT DOCUMENTS
`3,919,479 A 11/1975 Moon et al.
`4,230,990 A 10/1980 Lert, Jr. et al.
`
`(2011.01)
`
`(56)
`
`5/1984 Kenyon et a1.
`4,450,531 A
`6/1984 Heffron er a1~ _
`4,454,594 A
`1/1985 Baranoff-Rosslne
`4,495,526 A
`2/1985 Matthews
`4,499,601 A
`4/l985 Kohler et a1‘
`4,511,917 A
`4,547,804 A 10/1985 Greenberg
`4,634,966 A
`1/1987 Nakataniet al.
`4,639,779 A
`1/1987 Greenberg
`4,677,455 A
`6/1987 Okajima
`(Continued)
`FOREIGN PATENT DOCUMENTS
`1354 276 B1
`12/2007
`1354276 B1
`12/2007
`(Continued)
`OTHER PUBLICATIONS
`Peter N. Yianlos, Excluded Middle Vantage Point Forest for Nearest
`Neighbor Search, Aug. 1, 1999, pp. 1-12.*
`Martin Ester et al., “A Density-Based Algorithm for Discovering
`Clusters in Large Spatial Databases With Noise,” Proceedings of 2nd
`Internatlonal Conference on Knowledge Dlscovery and Data M1n1ng
`(KDD'96)’ 1996'
`
`EP
`EP
`
`(Continued)
`_
`_
`.
`Prlmary Examm” * Cal Chen
`(74) Attorney, Agent, or Firm * Amster, Rothstein &
`Ebenstein LLp
`ABSTRACT
`(57)
`A computer-implemented method including the steps of
`maintaining, by a computer system including at least one
`computer, a database in Which is stored ?rst data related to
`identi?cation of one or more Works and second data related to
`information corresponding to each of the one or more Works
`as identi?ed by the ?rst data. Extracted features of a Work to
`be identi?ed are obtained. The Work is identi?ed by compar
`mg the extracted features of the Work W1th the ?rst data 1n the
`database using a non-exhaustive neighbor search. The infor
`mation corresponding to the identi?ed Work is determined
`based on the second data in the database. The determined
`information is associated With the identi?ed Work.
`37 Claims, 10 Drawing Sheets
`
`WORK @n
`
`WORK l2 @ 122
`
`FEATURE
`EXTRACT‘ON
`OP£RATION(S
`
`FEATURE TO
`WORK m
`TAGGXNG
`oPER/moms
`
`124
`
`EATURE
`(VECTOR) EXTRACUON
`ovsrwxoms)
`
`FEATURE
`(VECTOR) LOOKUP
`OF'ERAT1ON(S)
`
`DATABASE
`GENERATION
`opsmnoms)
`
`W10
`rNFORMATsoN
`
`WORK'ASSQCIATED
`1NFORMATION LOOKUP
`OPERATIONS)
`
`DATABASE
`GENERATION
`OPERA‘HOMS)
`
`WIDYACTION
`INFORMATION _>
`
`,
`
`_.
`
`WORK )0
`
`ACT‘ON
`moi/mom
`OPERA‘HOMS)
`
`170
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 3 of 30
`
`US 8,640,179 B1
`Page 2
`
`(56)
`
`References Cited
`U_S_ PATENT DOCUMENTS
`4,677,466 A
`6/1987 Len, Jr. et al.
`4,682,370 A
`7/1987 Matthews
`4,697,209 A
`9/1987 Kiewit et 31‘
`4,739,398 A
`4/1988 Thomas et 31‘
`4,776,017 A 10/1988 pujimoto
`4,805,020 A
`2/1989 Greenberg
`4,843,526 A
`6/1989 Price’HI
`4,843,562 A
`6/1989 Kenyon et 31‘
`4,918,730 A
`4/1990 Schulze
`5,210,820 A
`5/1993 Kenyon
`5,283,819 A
`2/1994 Glick etal.
`5,437,050 A
`7/1995 Lamb et a1‘
`5,465,353 A 11/1995 Hull et al.
`5,481,294 A
`1/1996 Thomas et a1‘
`5,504,518 A
`4/1996 Ellis et al.
`5,550,735 A
`8/1996 Slade et 31‘
`5,581,658 A 12/1996 O>Hagan etal‘
`5594934 A
`1/1997 Lu et 31‘
`5,629,739 A
`5/1997 DOugheI-ty
`5,634,012 A
`5/1997 Ste?k etal.
`5,638,443 A
`6/1997 Ste?k et 31‘
`5,692,213 A 11/1997 Goldbergetal.
`5,701,452 A 12/1997 Siefert
`5,701,542 A 12/1997 sasayama
`5,706,364 A
`1/1998 Kopec etal.
`5,724,605 A
`3/1998 Wissner
`5,745,900 A
`4/1998 Burrows
`5,748,783 A
`5/1998 Rhoads
`5,768,426 A
`6/1998 Rhoads
`5,798,785 A
`8/1998 Hendricks et al.
`5,822,436 A 10/1998 Rhoads
`5,832,119 A 11/1998 Rhoads
`5,841,978 A 11/1998 Rhoads
`5,850,490 A 12/1998 Johnson
`5,862,260 A
`1/1999 Rhoads
`5,892,536 A
`4/1999 Logan et 31‘
`5918223 A
`6/1999 Blum et 31‘
`5,953,415 A
`9/1999 Nielsen
`5,963,966 A 10/1999 Mitchell et al.
`5,973,723 A 10/1999 DeLuca
`5,978,791 A 11/1999 Farber et 31‘
`5,983,171 A 11/1999 Yokoyamaetal.
`5,983,176 A 11/1999 HOfren etal.
`6,006,256 A 12/1999 Zdepskietal.
`6,009,410 A 12/1999 LeMole etal.
`6,011,758 A
`1/2000 Dockesetal.
`6,023,693 A
`2/2000 Masuoka et 31‘
`6,026,439 A
`2/2000 Chowdhury et 31‘
`6,044,402 A
`3/2000 Jacobson et 31‘
`6,052,693 A
`4/2000 Smith et 31‘
`6,057,872 A
`5/2000 Candelore
`6,061,056 A
`5/2000 Menard etal. .............. .. 715/704
`6,067,369 A
`5/2000 Kamei
`6,088,455 A
`7/2000 Logan et 31‘
`6,088,707 A
`7/2000 Bates etal.
`6,096,961 A
`8/2000 Bruti et 31‘
`6,118,450 A
`9/2000 P106111 etal.
`6,119,124 A
`9/2000 Broderetal.
`6,121,530 A
`9/2000 Sonoda
`6,154,737 A 11/2000 Inaba et 31‘
`6,169,986 B1
`1/2001 Bowman et 31‘
`6,173,406 B1
`1/2001 Wang et 31‘
`6,195,693 B1
`2/2001 Berryetal.
`6,229,922 B1
`5/2001 Sasakawa etal.
`6,233,682 B1
`5/2001 Fritsch
`6,236,758 B1
`5/2001 sodagar et 31‘
`6,240,409 B1
`5/2001 Aiken
`6,243,725 B1
`6/2001 Hempleman et al.
`6,247,133 B1
`6/2001 Palage et al.
`6,253,193 B1
`6/2001 Ginter et al.
`6,263,348 B1
`7/2001 KathroW et al.
`6,269,275 B1
`7/2001 Slade
`6,279,010 B1
`8/2001 Anderson
`6,317,885 B1
`11/2001 Fries
`6,330,593 B1
`12/2001 Roberts et al.
`
`2/2002 Milsted et al.
`6,345,256 B1
`2/2002 Broder et al.
`6,349,296 B1
`3/2002 Judd et al.
`6,360,215 B1
`3/2002 Kravets et al.
`6,363,377 B1
`4/2002 Hejnmlr
`6374225 B1
`4/2002 Hoffert et al.
`6,374,260 B1
`4/2002 Fujiwara et al.
`6,381,601 B1
`5/2002 W1_ser et 31.
`6,385,596 B1
`6/2002 La1et al._
`6,407,680 B1
`6/2002 Abecassls
`6,408,128 B1
`7/2002 Farber et al.
`6,415,280 B1
`7/2002 HuItado et al.
`6,418,421 B1
`8/2002 Malik et al.
`6,438,556 B1
`9/2002 Kortge I
`6,446,068 B1
`9/2002 Kumagal
`6,449,226 B1
`9/2002 Otsuka et al.
`6,452,874 B1
`9/2002 Laroche
`6,453,252 B1
`10/2002 Pace et al.
`6,460,050 B1
`11/2002 Cremia
`6,477,704 B1
`12/2002 Chen et al.
`6,490,279 B1
`12/2002 VanZoestet al.
`6,496,802 B1
`12/2002 Dustin et al.
`6,496,857 B1
`1/2003 Levy et al.
`6,505,160 B1
`4/2003 Foot?
`6,542,869 B1
`4/2003 CorWln et al.
`6,550,001 B1
`4/2003 511118, 111
`6,550,011 B1
`4/2003 Hasegawa et al.
`6,552,254 B2
`6/2003 Evans et al.
`6,577,746 B1
`7/2003 K198
`6591245 B1
`7/2003 HeJna, Jr.
`6,598,228 B2
`8/2003 Story et al.
`6,609,105 B2
`11/2003 Stern
`6,654,757 B1
`12/2003 Crow et al.
`6,665,661 B1
`1/2004 B0118 etal
`6,675,174 B1
`2/2004 Gould et al.
`6,693,236 B1
`7/2004 Harley
`6,766,523 B2
`8/2004 Ellis et al.
`6,774,926 B1
`10/2004 Saw
`6,810,388 B1
`12/2004 Fuller et al.
`6,833,865 B1
`12/2004 IkeZoye et al. .............. .. 709/231
`6,834,308 B1
`3/2005 MacQueen et 31.
`6,871,200 B2
`3/2005 Morris
`6,871,231 B2
`3/2005 Bates et a1~
`6,873,982 B1
`8/2005 Yamanaka
`6,928,423 B1
`8/2005 Farber et al.
`6,928,442 B2
`8/2005 Logan eta1~
`6,931,451 B1
`8/2005 wllfet a1~
`6,937,766 B1
`9/2005 swlerczek
`6,941,275 B1
`9/2005 Stern
`6,944,632 B2
`6,968,337 B2 11/2005 Wold _
`6,978,419 B1
`12/2005 KantroWltZ
`6,978,461 B2 12/2005 Shapiro et al.
`6,983,371 B1
`1/2006 HuItado et al.
`6,990,453 B2
`1/2006 Wang et al.
`6,999,111 B2
`2/2006 McIntyre et al.
`7,013,301 B2
`3/2006 H019} eta1~
`7,020,635 B2
`3/2006 Ham1lton et al.
`7,043,473 B1
`5/2006 Rassool et al.
`7,058,223 B2
`6/2006 C08
`7,065,709 B2
`6/2006 Ell1s et al.
`7,092,953 B1
`8/2006 Haynes
`7,103,906 B1
`9/2006 KaFZ eta1~
`7,106,904 B2
`9/2006 Shlma
`7,155,449 B2 12/2006 Pingel et al.
`7,158,929 B2
`1/2007 Wouters et 31.
`7,168,083 B2
`1/2007 Kalker et al.
`7,171,016 B1
`1/2007 Rhoads
`7,174,293 B2
`2/2007 Kényon etal
`7,184,100 B1
`2/2007 Wilfet al.
`7,191,190 B2
`3/2007 Debique et al.
`7,225,455 B2
`5/2007 Bennington etal.
`7,243,153 B2
`7/2007 McIntyre et al.
`7,272,788 B2
`9/2007 Anderson et al.
`7,302,574 B2 11/2007 ConWellet al.
`7,308,413 B1
`12/2007 Tota et al.
`7,346,472 B1
`3/2008 MoskoWitZ et al.
`7,363,278 B2
`4/2008 Schmelzer et al.
`7,366,718 B1
`4/2008 Pugh et al.
`7,366,787 B2
`4/2008 Salas et al.
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 4 of 30
`
`US 8,640,179 B1
`Page 3
`
`(56)
`
`References Cited
`U_S_ PATENT DQCUMENTS
`7,369,677 B2
`5/2008 petrovic et a1‘
`7,370,017 B1
`5/2008 Lindeman et a1‘
`7,421,723 B2
`9/2008 Harkness et a1‘
`7,423,771 B2
`9/2008 Ohata et al.
`7,477,739 B2
`V2009 Haitsma et a1‘
`7,483,958 B1
`1/2009 Elabbady et al.
`7,493,643 B2
`2/2009 Ellis
`7,500,007 B2
`3/2009 lkezoye et a1‘
`7,523,312 B2
`4/2009 Kalker et a1‘
`7,529,659 B2
`5/2009 Wold
`7,562,012 B1
`7/2009 Wold et a1‘
`7,562,392 B1
`7/2009 Rhoads et a1‘
`7,565,327 B2
`7/2009 Schmelzer
`7,587,728 B2
`9/2009 Wheeler et a1‘
`7,595,914 B2
`9/2009 Haining
`7,624,337 B2 11/2009 Sun et a1‘
`7,647,604 B2
`V2010 Ramaswamy
`7,650,616 B2
`V2010 Lee
`7,660,700 B2
`2/2010 MOSkOWitZ et a1‘
`7,707,088 B2
`4/2010 Schmelzer
`7,711,652 B2
`5/2010 Schmelzer
`7,738,704 B2
`6/2010 Lienhart et a1‘
`7,757,248 B2
`7/2010 Harkness et a1.
`7,783,489 B2
`8/2010 Kenyon et a1‘
`7,797,249 B2
`9/2010 SchmelZer et al.
`7,853,664 B1
`12/2010 Wang et a1‘
`7,877,438 B2
`1/2011 Schrempp et a1‘
`7,917,645 B2
`3/2011 lkezoye et a1‘
`7,949,494 B2
`5/2011 MoskoWitZ et al.
`8,006,314 B2
`8/2011 Wold
`8,082,150 B2 l2/2011 Wold
`8,086,445 B2 l2/2011 Wold et a1‘
`8,090,605 B2
`1/2012 Tota et al.
`8,094,949 B1
`1/2012 Rhoads
`8,112,776 B2
`2/2012 Schein et al.
`8,171,509 B1
`5/2012 Girouard et al.
`8,214,175 B2
`7/2012 MoskoWitZ et al.
`8,340,994 B2 12/2012 Tota et al.
`2001/0001160 A1 *
`5/2001 Shoff et al. ................... .. 725/51
`2001/0003818 A1
`6/2001 Pingel et al.
`2001/0047298 A1 11/2001 Moore et al.
`2001/0049625 A1 12/2001 MoWry
`2002/0023020 A1
`2/2002 Kenyon et a1.
`2002/0026369 A1
`2/2002 Miller et al.
`2002/0032698 A1
`3/2002 Cox
`2002/ 0038296 A1
`3/ 2002 Margolus et a1.
`2002/0056123 A1
`5/2002 LiWerant et al.
`2002/0059610 A1
`5/2002
`2002/0082731 A1
`6/2002 Pltman et _a1~
`2002/0083005 A1
`6/2002 Lowensteln et al'
`2002/0087885 A1
`7/2002 Peled et 31'
`2002/0120925 A1
`8/2002 Logan
`20020133499 Al
`90002 Ward et a1‘
`2002/0150164 A1 10/2002 Felts et al.
`2002/0l56760 A1 100002 Lawrence et a1‘
`2002/0178276 A1 1 1/2002 Mccmney et 31‘
`2002/0186887 A1 12/2002 Rhoads
`2003/0028489 A1
`2/2003 Williamson
`2003/0037010 A1
`2/2003 SchmelZer
`2003/0061490 A1
`3/2003 Abaji?n
`2003/0093790 A1
`5/2003 Logan et a1~
`2003/0095660 A1
`5/2003 Lee et 31'
`2003/0101144 A1
`5/2003 Moren?
`2003/0106017 A1
`6/2003 Kanch1rayappa et al.
`2003/0l46940 A1
`800% Ellis et a1‘
`2003/0202660 A1 100003 Zhou et a1
`2003/0233930 A1 12/2003 OZick
`200200003398 A1
`1/2004 Donian et 31‘
`2004/0010602 A1
`1/2004 Van V1ec1< e131,
`2004/0015608 A1
`1/2004 E1115 et 31,
`2004/0025174 A1
`2/2004 (jerrato
`2004/0163106 A1
`8/2004 Schrempp et a1.
`2004/ 0170335 A1
`9/ 2004 Pearlman et al.
`2004/0199387 A1* 10/2004 Wang et a1. ................. .. 704/243
`2004/0221118 A1 11/2004 Slater et al.
`
`2004/0243540 A1 12/2004 MoskoWitZ et al.
`2005/ 0044189 A1
`2/2005 IkeZoye et al.
`2005/0080846 A1
`4/2005 McCleskey et al.
`2005/0102515 A1
`5/2005 Jaworski et al.
`2005/0154892 A1
`7/2005 Mihcak et al.
`2005/0160363 A1
`7/2005 Bhogal et a1.
`2005/0193016 A1
`9/2005 Seet et al.
`2005/0213826 A1
`9/2005 Neogi
`2005/0246752 A1 11/2005 LiWerant et a1.
`2005/0289065 A1 12/2005 Weare
`2006/0031870 A1
`2/2006 Jarman et a1.
`2006/0080356 A1
`4/2006 Burges et al.
`2006/0085816 A1
`4/2006 Funk et al.
`2006/0101069 A1
`5/2006 Bell et al.
`2006/0110137 A1
`5/2006 Tsuda et al.
`2006/0187358 A1
`8/2006 Lienhart et al.
`2006/0195859 A1
`8/2006 Konig et a1.
`2006/0195860 A1
`8/2006 Eldering et al.
`2006/0206462 A1
`9/2006 Barber
`2006/0212927 A1
`9/2006 Riku et al.
`2006/0271947 A1 11/2006 Lienhart et al.
`2007/0041667 A1
`2/2007 COX
`2007/0071330 A1
`3/2007 Oostveen et 31.
`2007/0083510 A1
`4/2007 McArdle
`2007/0101360 A1
`5/2007 Gutta et al.
`2007/0118375 A1
`5/2007 Kenyon et a1.
`2007/0124698 A1
`5/2007 Majumder
`2007/0130580 A1
`6/2007 Covell et al.
`2007/0180537 A1
`8/2007 Héet a1~
`2007/0203911 A1
`8/2007 Ch1u
`2007/0282472 A1 12/2007 Seldman
`2007/0288518 A1 12/2007 Crigler et al.
`Zoos/0052783 A1
`2/2008 Levy
`2008/0091684 A1
`4/2008 Ell1s et a1.
`2008/0250241 A1 10/2008 Ginter et a1.
`2009/0052784 A1
`2/2009 Covell et al.
`FOREIGN PATENT DOCUMENTS
`1485 315 B1
`7/2009
`EP
`2369203 A
`5/2002
`GB
`2003-2422g1
`55/2003
`JP
`94/060g4 A1
`3/1994
`W0
`99/5077g A1 10/1999
`W0
`WO0122730 A1
`3/2001
`W0
`W0 WO 2002/011033 A1
`2/2002
`W0 W0 2002003963 A1 12/2002
`OTHER PUBLICATIONS
`Yossi Rubner et al., “Adaptive Color Image Embeddings for Database
`Navigation*,” Proceedings of the 1998 IEEE Asian Conference on
`Computer Vision,
`Roger Weber et al., “A Quantitative Analysis and Performance Study
`for Similarity-Search Methods in High Dimensional Spaces,” Pro
`ceedings of 24th VLDB Conference, 1998.
`P Y. .1
`“D S
`d A1
`. hm f N
`N .
`b
`.
`1an1-os,
`ata truc-tures an
`gor1t
`Is or earest e1gh or
`Search 1n General Metr1c Spaces ” Proceedlngs of the ACM-SIAM
`.
`.
`.
`’
`Sympos1um on D1screte algor1thms, 1993, pp. 311.321.
`Peter N.Yianlos, Excluded Middle Vantage Point Forests for Nearest
`Neighbor Search, Jul. 20, 1998,1911 l-l2~
`Peter N. Yianlos “Locally Lifting the Curse of Dimensionality for
`Nearest Neighbor Search” SODA 2000, pp. 361-370.
`L. Baum et al., “A Maximation Technique Occuring in the Statistical
`Analysis of Probabilistic Functions of Markov Chains,” The Annals
`of Mathematical Statistics, vol. 41, No. 1, pp. 164-171 (1970).
`A. P. Dempster et al., “Maximum Likelihood from Incomplete Data
`-
`-
`,,
`-
`-
`-
`v1a-the $EM$ Algor1thm, Journal of the Royal Stat1st1cal Soc1ety,
`Ser1es B(Methodolog1cal),vol. 39, Issue 1, pp. 1-38 (1977).
`D. Reynolds et al., “Robust Text-Independent Speaker Identi?cation
`Using Gaussian Mixture Speaker Models,” IEEE Transactions on
`Speech and Audio Processing, vol. 3, No. 1, pp. 72-83 (Jan. 1995).
`D. Bouktache, “A fast Algorithm for the nearest neighbor classi?er,”
`IEEE Transactions on Pattern Analysis and Machine Intelligence,
`Mar. 1997, pp. 277-282.
`Nene et al., “A simple algorithm for nearest neighbor search in high
`dimensions,” IEEE Transactions on Pattern Analysis and Machine
`Intelligence, Sep. 1997, pp. 989-1003.
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 5 of 30
`
`US 8,640,179 B1
`Page 4
`
`(56)
`
`References Cited
`OTHER PUBLICATIONS
`Sunil Arya et al., “Approximate Nearest Neighbor Queries in Fixed
`Dimensions,” Proceedings of the 4th annual ACM-SIAM Sympo
`sium on Discrete algorithms, 1993, pp. 271-280.
`K. Fukunaga et al., A branch and bound algorithm for computing
`k-nearest neighbors, IEEE Trans. Comput., C24:750-753, Jul. 1975.
`Charles D. Feustel et al., “The nearest neighbor problem in an
`abstract metric space,” Pattern Recognition Letters, pp. 125-128,
`Dec. 1982.
`Dennis Shasha et al., “New Techniques for Best-Match Retrieval,”
`ACM Transactions on Information Systems, 8(2); 140{158, Apr.
`1990.
`J. Uhlmann, “Satisfying general proximity/ similarity queries with
`metric trees”, Information Processing Letters, 40 (4): 175{9, Nov.
`1991.
`Sergey Brin, “Near Neighbor Search in Large Metric Spaces,” Pro
`ceedings of the 21st VLDB Conference, Zurich, Switzerland, Sep.
`1995.
`Daniel P. Huttenlocher, et al., “Comparing Images Using the
`Hausdorff Distance,” IEEE Transactions on Pattern Analysis and
`Machine Intelligence, vol. 15, No. 9, pp. 850-863, Sep. 1993.
`Thomas Seidl et al., “Optimal Multi-Step K-Nearest Neighbor
`Search,” Proceedings of ACM SIGMOD International Conference of
`Management ofData, Jun. 1998, pp. 154-165.
`W. A. Burkhard et al., “Some Approaches to Best-Match File Search
`ing,” Communications ofthe ACM, vol. 16, No. 4, Apr. 1973.
`Eyal KushilevitZ et al., “Ef?cient Search for Approximate Nearest
`Neighbor in High Dimensional Spaces,” Proceedings of the 30th
`annual ACM Symposium on the Theory of computing, 1998, pp.
`457-474, vol. 30, No. 2.
`J. Nievergelt et al., “The Grid File: An Adaptable, Symmetric
`Multikey File Structure,” ACM Transactions on Database Systems,
`vol. 9, No. 1, pp. 38-71 (Mar. 1984).
`Nevin HeintZe, “Scalable Document Fingerprinting,” Proc. USENIX
`Workshop on Electronic Commerce (1996).
`Erling Wold et al., “Content-Based Classi?cation, Search, and
`Retrieval of Audio,” IEEE Multimedia, vol. 3, Issue 3, pp. 27-63
`(1996).
`Bir Bhanu et al., “Learning Feature Relevance and Similarity Metrics
`in Image Databases,” Proceedings of the IEEE Workshop on Content
`Based Access of Image and Video Libraries, pp. 14-19 (1998).
`A. Del Bimbo et al., “Using Weighted Spatial Relationships in
`Retrieval by Visual Contents,” Image Description and Retrieval, pp.
`161-192 (1998).
`P. Indyk et al., “Approximate Nearest Neighbors: Towards Removing
`the Curse of Dimensionality,” Proceeding of the Thirtieth Annual
`ACM Symposium on Theory of Computing, pp. 604-613 Jul. 21,
`1999.
`Marco La Cascia, “Combining Textual and Visual Cues for Content
`based Image Retrieval on the World Wide Web,” Proceedings of the
`IEEE Workshop on Content-Based Access of Image and Video
`Libraries, pp. 24-29 (1998).
`Atsuo Yoshitaka et al., “A Survey on Content-Based Retrieval for
`Multimedia Databases,” IEEE Transactions on Knowledge and Data
`Engineering, vol. 11, No. 1, pp. 81-93 (Jan/Feb. 1999).
`Steve Lawrence et al., “Digital Libraries and Automonous Citation
`Indexing,” IEEE Computer, pp. 67-71 (Jun. 1999).
`Akisato Kimura et al., “Very Quick Audio Searching: Introducing
`Global Pruning to the Time-Series Active Search,” IEEE Conf on
`Acoustics, Speech and Signal Processing, (ICASSP ’01), vol. 3, pp.
`1429-1432, 2001.
`Edgar Chavez et al., “Searching in Metric Spaces,” ACM Computing
`Surveys, vol. 33, No. 3, pp. 273-321 (Sep. 2001).
`Jaap Haitsma et al., “Robust Audio Hashing for Content Identi?ca
`tion,” Workshop on Content Based Multimedia Indexing, Brescia,
`Italy (Sep. 19-21, 2001).
`Jaap Haitsma et al., “A Highly Robust Audio Fingerprinting System,”
`Journal of New Music Research, 1744-5027, vol. 32, Issue 2, pp.
`211-221 (2003).
`
`Saul Schleimer et al., “Winnowing: Local Algorithms for Document
`Fingerprinting,” ACM SIGMOD, Jun. 9-12, 2003.
`Edward Chang et al., “Searching Near-Replicas of Images via Clus
`tering,” SPIE Symposium of Voice, Video and Data Communica
`tions, 1999.
`EdwardY. Chang et al., “RIME: A Replicated Image Detector for the
`World-Wide Web,” SPIE, 1998.
`Hector Garcia-Molina et al., “Safeguarding and Charging for Infor
`mation on the Internet,” Proceedings of ICDE, 1998.
`Sergey Brin et al., “Copy Detection Mechanisms for Digital Docu
`ments,” Proceedings of ACM SIG-MOD, May 1995.
`Stefan Berchtold “The x-tree: An Index Structure for High-Dimen
`sional Data,” Proceedings of the 22nd VLDB, Aug. 1996.
`Norio Katayama et al., “The SR-tree: An Index Structure for High
`Dimensional Nearest Neighbor Queries,” Proceedings of ACM
`SIGMOD, May 1997.
`John T. Robinson, “The K-D-B-Tree: A Search Structure for Large
`Multidimensional Dynamic Indexes,” Proceedings of ACM
`SIGMOD, Apr. 1981.
`Myron Flickner et al., “Query by Image and Video Content: The
`QBIC System,” IEEE Computer 28(9), pp. 23-32, 1995.
`Amarnath Gupta et al., “Visual Information Retrieval,” Communica
`tions ofthe ACM, vol. 40, No. 5, pp. 69-79, May 1997.
`John R. Smith et al., “VisualSEEk: A fully automated content-based
`image query system,” ACM Multimedia Conference, 1996.
`David A. White et al., “Similarity Indexing: Algorithms and Perfor
`mance,” Proc. SPIE, vol. 2670, San Diego, 1996.
`Norbert Beckmann et al., “The R*-tree: An Ef?cient and Robust
`Access Method for Points and Rectangles,” Proceedings of ACM
`Sigmod, May 1990.
`A. Guttman, “R-Trees: A Dynamic Index Structure for Spatial
`Searching,” Proceedings ofACM Sigmod, Jun. 1984.
`David A. White et al., “Similarity Indexing with the SS-tree’,” Pro
`ceedings of the 12th ICDE, Feb. 1996.
`King Lin et al., “The TV-Tree: An Index Structure for High-Dimen
`sional Data,”VLDB, Journal 3, No. 4, 1994, pp. 517-542.
`Paolo Ciaccia et al., “M-tree: An Ef?cient Access Method for Simi
`larity Search in Metric Spaces,” Proceedings of the 23rdVLDB, Aug.
`1997.
`Nick Roussopoulos et al., Nearest Neighbor Queries, Proceedings of
`ACM Sigmod, May 1995.
`C. Li et al., “An extensible hashing index for high-dimensional simi
`larity search,” Stanford Techncial Report, Aug. 1998.
`Jon M. Kleinberg, “Two Algorithms for Nearest-Neighbor Search in
`High Dimensions,” Proc 29th Stoc, Feb. 7, 1997.
`US. Appl. No. 60/222,023, ?led Jul. 31, 2000; Avery Li-Chun Wang
`and Julius O. Smith III, Inventors; Palo Alto, CA.
`Liu et a1 ., “An Investigation of Practical Approximate Nearest Neigh
`bor Algorithms”, Advances in Neural Information Processing Sys
`tems (NIPS) 2004.
`KL. Clarkson, “Nearest-Neighbor Searching and Metric Space
`Dimensions”, Nearest-Neighbor Methods for Learning and Vision:
`Theory and Practice:, Apr. 2005.
`Swaminathan et al., “Robust an Secure Image Hashing”, IEEE Trans
`actions on Information Forensics and Security, Jun. 2006, pp. 215
`230, vol. 1, No. 2.
`Burges et al., “Duplicate Detection and Audio Thumbnails with
`Audio Fingerprinting” [online]. 2004, [retrieved on Nov. 21, 2006].
`Retrieved on the Internet: <URL: www.research.microsoft.com/
`cburges/techireports/tr-2004-19.pdf>, 5 pages.
`Cana et al., “A Review of Algorithms for Audio Fingerprinting”
`[online]. 2002, [retrieved on Nov. 21, 2006]. Retrieved from the
`Internet: <URL: www.iua.upf.es/mtg/publications/MMSP-2002
`pcano.pdf>, 5 pages.
`Haitsma and Kaiker, “A Highly Robust Audio Fingerprinting Sys
`tem” [online]. 2002, [retrieved on Nov. 16, 2006]. Retrieved from the
`Internet: <url: www.ismir2002.ismir.net/proceedings/02-FP04-2.
`pdf>, 9 pages.
`Jacobs el al., “Fast Multiresolution Image Querying” [online]. 1995,
`[retrieved on Nov. 21, 2006]. Retrieved from the Internet: <URL:
`www.grail.cs.washington.edu/projects/query.pdf>, 10 pages.
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 6 of 30
`
`US 8,640,179 B1
`Page 5
`
`(56)
`
`References Cited
`OTHER PUBLICATIONS
`Ke et al., “Computer Vision for Music Identi?cation” [online]. 2005,
`[retrieved on Nov. 21, 2006]. Retrieved from the Internet: <URL:
`Www.cs.cmu.edu/~yke/musicretrieval/cvpr2005-mr.pdf>, 8 pages.
`ShaZam, ShaZam Enterprise Brings Music Recognition to Windows
`Mobile 5.0 Powered Smartphones [online]. 2006, [retrieved on Nov.
`16, 2006]. Retrieved from the Internet: <URL: www.shaZam.com/
`music/portal/sp/s/media-type/html/user/anon/page/default/tem
`plate/pages/p/companyirelease30.html>, 1 page.
`Stanford, “CS276 Information Retrieval and Web Mining” [online].
`2005, [retrieved on Nov. 16, 2006] Retrieved from the Internet:
`<URL: WWw.stanford.edu/class/cs276/handouts/lecture19.pdf>, 8
`pages.
`Stanford, “Data Mining: Associations” [online], 2002, [retrieved on
`Nov. 16, 2006]. Retrieved from the Internet: <URL: www.stanford.
`edu/class/cs206/cs206-2.pdf>, 11 pages.
`StollnitZ et al., Wavelets for Computer Graphics: A Primer, Part 1:
`[online]. 1995, [retrieved on Nov. 21, 2008]. Retrieved from the
`Internet: <URL: WWW. grail .c s .washington .edu/pub/stoll/waveletl .
`pdf>, 8 pages.
`StollnitZ et al., Wavelets for Computer Graphics: A Primer, Part 2:
`[online]. 1995, [retrieved on Nov. 21, 2006]. Retrieved from the
`Internet: <URL: WWW. grail .c s .washington .edu/pub/stoll/wavelet2.
`pdf>, 9 pages.
`Yang, “MACS: Music Audio Characteristic Sequence Indexing for
`Similarity Retrieval”, Oct. 21-24, 2001, New PeitZ, New York.
`Viola and Janes, Robust Real-Time Object Detection, Int. J. Com
`puter Vision, 2002.
`Burges et al., “Using Audio Fingerprinting for Duplicate Detection
`and Thumbnail Generation,” Mar. 2005, 4 pages.
`Lin, et al., Input Data Representation for Self-Organizing Map in
`Software Classi?cation, Knowledge Acquisition and Modeling,
`2009. KAM ’09. Second International Symposium on vol. 2, Digital
`Object Identi?ed: 10.11 09/KAM.2009.151 Publication Year: 2009,
`pp. 360-353.
`Baiuja et al., “Content Fingerprinting Using Wavelets”, 3rd European
`Conference on Visual Media Production, 2006, pp. 198-207.
`Cohen et al., “Finding Interesting Associations without Support
`Pruning”, IEEE Transactions on Knowledge and Data Engineering,
`2001, pp. 64-78, vol. 13, issue 1.
`Yang, Ef?cient Video Identi?cation based on locality sensitive hash
`ing and triangle inequality, National University of Singapore, 2005,
`pp. 1-64.
`U.S. Appl. No. 60/304,647, ?led Jul. 10, 2001.
`U.S.Appl.No.60/281,881,?ledApr. 5,2001.
`Metadata Mediation: Representation and Protocol, Tsuyoshi Sakata,
`Hiroyuki Tada, Tomohisa Ohtake, Digital Vision Laboratories, 7-3
`37 Akasaka, Minato, Tokyo, Japan.
`US. Appl. No. 13/800,573, ?led Mar. 13,2013.
`U.S. Appl. No. 13/800,890, ?led Mar. 13,2013.
`U.S. Appl. No. 13/829,717, ?led Mar. 14, 2013.
`US. Appl. No. 13/830,447, ?led Mar. 14,2013.
`U.S. Appl. No. 13/830,626, ?led Mar. 14,2013.
`U.S. Appl. No. 13/830,986, ?led Mar. 14,2013.
`U.S. Appl. No. 13/842,068, ?led Mar. 15, 2013.
`US. Appl. No. 13/338,079, ?led Dec. 27, 2011.
`US. Appl. No. 60/134,782, ?led May 19, 1999.
`US. Appl. No. 60/193,948, ?led Mar. 31, 2000.
`US. Appl. No. 60/195,535, ?led Apr. 7, 2000.
`US. Appl. No. 60/206,384, ?led May 23, 2000.
`US. Appl. No. 60/221,843, ?led Jul. 28, 2000.
`US. Appl. No. 60/133,247, ?led May 5, 1999.
`US. Appl. No. 60/155,064, ?led Sep. 21, 1999.
`US. Appl. No. 60/218,824, ?led Jul. 18, 2000.
`Indyk, Piotr et al., “Locality-Preserving Hashing in Multidimen
`sional Spaces,” Feb. 25, 1997.
`Gibson, David, “Name That Clip: Music retrieval using audio clips,”
`Aug. 19, 1999.
`
`Declaration of David A. Gibson, Inter Partes Review of US. Patent
`7,174,293, Aug. 30, 2013.
`Declaration of David A. Gibson, Inter Partes Review of US. Patent
`7,783,489, Aug. 30, 2013.
`Intersil, “Glossary of Communication Terms,” Dec. 1996.
`Declaration of Dr. Ton Kalker, Inter Partes Review of US. Patent
`7,174,293, Aug. 30, 2013.
`Declaration of Dr. Ton Kalker, Inter Partes Review of US. Patent
`7,783,489, Aug. 30, 2013.
`ArdiZZone, Edoardo et al., “Motion and Color-Based Video Indexing
`and Retrieval,” Universita di palermo, Departimento di Ingegneria
`Elettrica, pp. 135-139, Viale delle ScienZe, Palermo, Italy, IEEE
`1996.
`Deng, Yining et al., “Content-based Search of Video Using Color,
`Texture, and Motion,” Dept. of Electrical and Computer Engineering,
`University of California, Santa Barbara, CApp. 534-537, IEEE 1997.
`Fang, Min et al., “Computing Iceberg Queries Ef?ciently,” Dept. of
`Computer Science, Stanford, CA, Paper No. 234, pp1-25.
`Flickner, Myron et al., “Query by Image and Video Content: The
`QBIC System,” IBM Almaden Research Center, Sep. 1995, pp.
`23-32, IEEE 1995.
`Gargi, U et al., “Performance Characterization and Comparison of
`Video Indexing Algorithms,” Dept. of Computer Science & Engi
`neering, The Pennsylvania State University, University Park, PA.
`Gionis, Aristides et al., “Similarity Search in High Dimensions via
`Hashing,” Dept. of Computer Science, Stanford University, Stanford,
`CA, pp. 518-529, Proceeding of the 25th VLDB Conference,
`Edinburgh, Scotland, 1999.
`Indyk, Piotr et al., “Approximate Nearest Neighbors: Towards
`Removing the Curse of Dimensionality” (preliminary version) Dept.
`of Computer Science, Stanford University, Stanford, CA, pp. 1-13 &
`i-vii, Jul. 21, 1999.
`Iyengar, Giridharan et al., “Models for automatic classi?cation of
`video sequences,” MIT Media Laboratory, Cambridge, MA.
`Jain, Anil K., et al., “Image Retrieval using Color and Shape,” Dept.
`of Computer Science, Michigan State University, Eas Lansing, MI,
`pp. 1-24, May 15, 1995.
`Ogle, Virginia E., et al., “Chabot: Retrieval from a Relational Data
`base of Images,” University of California at Berkeley, Computer pp.
`40-48, IEEE 1995.
`Pentland, A. et al., “Photobook: Content-Based Manipulation of
`Image Databases,” Perceptual Computing Section, the Media Labo
`ratory, Massachusetts Institute of Tech., International Journal of
`ComputerVision 18(3), pp. 233-254 (1996), 1996 KluwerAcademic
`Publishers. Manuf. In the Netherlands.
`Shivakumar, Narayanan et al., “Scam: a Copy Detection Mechanism
`for Digital Documents,” Dept. of Computer Science, Stanford Uni
`versity, Stanford, CA, pp. 1-13.
`Shivakumar, Narayanan et al., “Building a Scalable and Accurate
`Copy Detection Mechanism,” Dept. of Computer Science, Stanford
`University, Stanford, CA.
`Srihari, Rohini K., “Automatic Indexing and Content-Based
`Retrieval of Captioned Images,” State University of New York, Buf
`falo, Theme Feature, pp. 49-56, Sept. 1995, IEEE 1995.
`Swain, Michael and Ballard, Dana H., “Color Indexing,” Interna
`tional Journal ofComputerVision 7:1, p. 11-32 (1991), 1991 Kluwer
`Academic Publishers. Manuf. in the Netherlands.
`Wactlar, Howard D. et al., “Intelligent Access to Digital Video:
`Informedia Project,” Carnegie Mellon University, Digital Library
`Initiative: Carnegie Mellon University, Computer, pp. 46-52, IEEE
`1996.
`Yeo, Boon-Lock et al., “Rapid Scene Analysis on Compressed
`Video,” IEEE Transactions on Circuits and Systems for Video Tech
`nology, vol. 5, No. 6, pp. 533-544, Dec. 1995, Dept. of Electrical
`Engineering, Princeton University, Princeton, NJ, IEEE Log No.
`9415901.
`Indyk, Piotr et al., “Finding pirated video sequences on the Internet,”
`Dept. of Computer Science, Stanford University, Palo Alto, CA,
`Paper No. 199.
`* cited by examiner
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 7 of 30
`
`US. Patent
`
`Jan. 28, 2014
`
`Sheet 1 0f 10
`
`US 8,640,179 B1
`
`WORK @t2
`
`WORK @n
`
`FEATURE
`EXTRACTICJN
`OPERATIONS
`
`FEATURE
`(VECTOR) EXTRACTION
`OPERATIONS)
`
`140
`
`120
`
`DATABASE
`GENERATION
`QPERATIOMS)
`
`124
`
`
`
`1 50
`
`FEATURE
`(VECTOR) LOOKUP
`OPERATIONS)
`
`160
`WORK-ASSOCIATED
`INFORMATION LOOKUP
`OPERATIOMS)
`
`a
`
`1.13.
`p/
`FEATURES) (VECTOR) WORK ID
`
`DATABASE
`GENERATION
`OPERATIONS)
`
`WID-ACTION
`INFORMATION
`
`WORK ID ASSGCIATED INFORMATION (8.9., ACTION)
`
`ACTION
`INETIATION
`OPERATIONS)
`
`170
`
`10
`FIGURE 1
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 8 of 30
`
`US. Patent
`
`Jan. 28, 2014
`
`Sheet 2 0f 10
`
`US 8,640,179 B1
`
`SATELLITE CABLE
`
`T- _ _ _ _ h _ —
`
`OR TERRESTRIAL
`TV BROADCAST
`
`USER COMPUTER,
`SET-TOP-BOX OR
`EQUIVALENT
`
`FEATURE
`(VECTOR)
`EXTRACTION
`OPERATIONS
`
`FEATURE
`(VECTOR) LOCJKUP
`OPERATIOMS)
`
`1 1 0a
`
`WI D
`DATABASE
`
`120a/138a
`
`DATABASE
`GENERATION
`OPERATIONS
`
`WORK
`ASSOCIATED
`INFORMATIQN
`LOOKUP
`OPERATIOMS
`
`WID
`DATABASE
`
`WI DAT
`DATABASE
`
`WI DAT
`DATABASE
`
`ACTION
`INITIATION
`OPERATIOMS)
`
`210
`FIGURE 2
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 9 of 30
`
`US. Patent
`
`Jan. 28, 2014
`
`Sheet 3 0f 10
`
`US 8,640,179 B1
`
`32(}
`
`SATELLITE, CABLE I“ - -— — -_ ....... __ _ ___'
`OR TERRESTRIAL
`W BROADCAST
`
`]
`
`USER COMPUTER,
`SET-TOP-BOX OR
`EQUIVALENT
`
`FEATURE
`(VECTOR)
`EXTRACTION
`OPERATIONIS
`
`FEATURE
`(VECTOR) LOOKUP
`OPERATIOMS)
`
`DATABASE
`
`ACTION
`INITIATION
`OPERATIONIS)
`
`MONITORINFCEITTER
`DATABASE
`GENERATION
`OPERATIONS)
`
`11Gb’
`
`WORK-ASSOCIATED
`INFORMATION LOOKUP
`OPERATIOMS)
`
`i___._____ I I I I I I I I I I
`
`WID
`DATABASE
`
`DATABASE
`
`340
`
`FIGURE 3
`
`

`

`Case 1:14-cv-02396-PGG-MHD Document 153-7 Filed 06/28/19 Page 10 of 30
`
`US. Patent
`
`Jan. 28, 2014
`
`Sheet 4 0f 10
`
`US 8,640,179 B1
`
`420
`
`SATELLITE, CABLE
`OR TERRESTRIAL
`Tv BROADCAST
`
`[- -—— -—— — — _ _ _ _ _I
`
`i
`
`USER COMPUTER,
`SET-TOP-BOX OR
`EQUIVALENT
`
`FEATURE
`(VECTOR)
`EXTRACTION
`OPERATIONS
`
`INTTIATTON
`QPERAT|ON(S)
`
`FEATURE
`(VECTOR) LOOKUP
`OPERATTOMS)
`
`WORK-ASSOCSATED
`INFORMATION LOOKUP
`OPERATlON(S)
`
`DATABASE
`- GENERATEON
`OPERATIONS)
`
`1 10¢
`
`W! D
`DATABASE
`
`W1 DAT
`DATA BASE
`
`1300
`
`MONITORING CENTER