(12) United States Patent
`Barney
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9,075,849 B2
`*Jul. 7, 2015
`
`US009075849B2
`
`IVIETHOD AND SYSTEIVI FOR
`PROBABILISTICALLY QUAN TIFYIB G AN D
`VISUALIZING RELEVANCE BETWEEN TVVO
`OR MORE CITATIONALLY OR
`CONTEXTUALLY RELATED DATA OBJECTS
`
`Applicant: PatentRatings, LLC, Irvine, CA (US)
`Inventor:
`Jonathan A. Barney, Newport Beach,
`CA (US)
`Assignee: PA'1‘EN'1‘lLA'1‘lN GS, LLC, Irvine, CA
`(US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`Notice:
`
`This patent is subject to a terminal dis-
`claimer.
`
`Appl. No.2 14/338,208
`Filed:
`Jul. 22, 2014
`Prior Publication Data
`
`US 2015/0046420A1
`
`Feb. 12, 2015
`
`Related U.S. Application Data
`
`Continuation of application No. 13/958,386, filed on
`Aug. 2, 2013, now Pat. No. 8,818,996, which is a
`continuation of application No. 13/411,441, filed on
`Mar. 2, 2012, now Pat. No. 8,504,560, which is a
`
`(Continued)
`
`(2006.01)
`
`Int. Cl.
`G06F 17/30
`U.S. Cl.
`CPC .... .. G06F 17/3053 (2013.01); G06F 17/30675
`(2013.01); G06F 17/30716 (2013.01);
`(Continued)
`Field of Classification Search
`CPC .................. .. G06F 17/30728; G06F 17/30861;
`G06F 17/30011; G06F 17/30014; G06F
`17/30722, G06F 17/3053; G06F17/30675,
`(10614 17/30716;
`(30614 17/30864
`
`USPC ....... .. 707/705, 722, 726, 728, 731, 923, 930,
`707/933, 937, 999.1
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,175,681 A
`5,576,954 A
`
`12/1992 Iwai et :11.
`11/1996 Driscoll
`
`(Continued)
`FOREIGN PAT,:N1 DOCUM,
`
`EP
`W0
`W0
`
`1 215599
`WO 00/75851
`W0 01/35277
`
`6/2002
`12/2000
`5/2001
`
`OTHER PUBLICATIONS
`“Bond Rating," Printout from unknown wcbsitc. 1997.
`(Continued)
`
`Primary Examiner — Harcs Jami
`(74) Atmrnqx, Agent, or Firm — Knobbe, Martens, Olson &
`Bear, LLP
`ABSTRACT
`(57)
`In one embodiment a method for probabilistically quantify-
`ing a degree of relevance between two or more citationally or
`contextually related data objects, such as patent documents,
`non-patent documents, web pages, personal and corporate
`contacts information, product
`information, consumer to
`behavior, technical or scientific information, address infor-
`mation, and the like is provided. In another embodiment a
`method for visualizing and displaying relevance between two
`or more citationally or contextually related data objects is
`provided. I11 another embodiment a search input/output inter-
`face that utilizes an iterative self—organizing mapping tech-
`nique to automatically generate a visual map of relevant pat-
`cnts and’or other rclatcd documents desired to be explored,
`searched or analyzed is provided. In another embodiment, a
`search input/output interface that displays and/or communi-
`cates search input criteria and corresponding search results in
`a way that facilitates intuitive understanding and visualiza-
`tion of the logical relationships between two or more related
`concepts being searched is provided.
`
`20 Claims, 12 Drawing Sheets
`
`. Panama"
`=X
`
`tPnr'7(‘
`M.»
`V :§-m a2. 5gm (:4 »...»....»...-w»
`
`mum: an
`v
`mzmrv
`smtmuuv
`swnmu
`mm: NEE:
`-PVI,
`wt
`umrfiunr
`nzsaess
`-
`uuz
`u
`,->
`(G71/tn? Nrl
`
`Tm
`
`Ocean Tomo Ex. 1001-001
`
`

`
`US 9,075,849 B2
`Page 2
`
`Related U.S. Application Data
`
`division of application No. 12/749,381, filed on Mar.
`29, 2010, now Pat. No. 8,131,701, which is a division
`ofapplication No. 11/236,965, filed on Sep. 27, 2005,
`now Pat. No. 7,716,226.
`U.S. Cl.
`CPC .... .. G06F17/30864 (2013.01); G06F 2216/1]
`(2013.01), YIOS 707/933 (2013.01), YIOS
`707/912 (2013.01), YJOS 707/923 (2013.01),
`1/10s 707/93 (2013.01), YIOS 707/937
`(2013.01), G06F 17/30572 (2013.01), G06F
`1 7/30728 (2013.01); G06F 17/30861 (2013.01);
`G06F 1 7/301 (2013 .01 ); G06F 1 7/30321
`(2013.01), G06F 17/30554 (2013.01)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`.............. .. 1/1
`
`...... .. 715/206
`
`.............. .. 1/1
`
`.............. .. 1/1
`
`707/3
`
`>>>D>J>D>>>D>>>>J>D>>>
`
`5,594,897
`5,608,620
`5,680,305
`5,694,592
`5,721,910
`5,754,840
`5,774,833
`5,799,325
`5,832,494
`5,848,409
`5,991,751
`5,999,907
`6,018,714
`6,018,749
`6,038,561
`6,154,725
`6,175,824
`6,202,058
`6,263,314
`6,286,018
`6,330,547
`6,389,418
`6,499,026
`6,526,440
`6,556,992
`6,604,114
`6,754,873
`6,799,176
`6,832,211
`7,188,069
`7,213,198
`7,292,994
`7,451,388
`7,912,842
`7,962,511
`2002/0002524
`2002/0004775
`2002/0022974
`2002/0035499
`2002/0046038
`200210077835
`200710082778
`20020087442
`20020099638
`20020156760
`20030036945
`20030065658
`2003/0074350
`20030212572
`20040010393
`2004/0068453
`2004/0103112
`2004/0122841
`2004/0133433
`2005/0021434
`2005/0071174
`
`7
`
`*
`
`1/1997
`3/1997
`10/1997
`1 2/ 1997
`2/1998
`5/1998
`6/1998
`8/1998
`1 1/ 1998
`12/1998
`1 1/ 1999
`1 2/ 1999
`1/2000
`1/2000
`3/2000
`1 1/2000
`1/2001
`3/2001
`7/2001
`9/2001
`1 2/2001
`5/2002
`12/2002
`2/2003
`4/2003
`8/2003
`6/2004
`9/2004
`1 2/2004
`3/2007
`5/2007
`1 1/2007
`1 1/2008
`3/201 1
`6/201 1
`1/2002
`1/2002
`2/2002
`3/2002
`4/2002
`6/2002
`6/2002
`7/2002
`7/2002
`10/2002
`2/2003
`4/2003
`4/2003
`1 1/2003
`1/2004
`4/2004
`5/2004
`6/2004
`7/2004
`1/2005
`3/2005
`
`....... ..
`
`Goffman ............. . .
`Lundgren
`Apgar, IV
`Driscoll
`Unger
`Rivette et al.
`Newman
`Rivette et al.
`Egger et a1.
`Ahn
`Rivcttc et al.
`Donner
`Risen, Jr. et al.
`Rivette et al.
`Snyder et al.
`Donner
`Breitzrnan et al.
`Rose et al.
`Donner
`Pitkow et al.
`Martin
`Boyack et 31.
`***Smith etal.
`Bharat
`Barney et al.
`Toong et al.
`Law et al.
`Page ................... ..
`Thomas et al.
`Hagelin
`Harik
`Prokoski
`Henzinger et al.
`Bayliss
`Barney
`Kossovsky ct a1.
`Kossovsky et a1.
`Iindh
`Germeraad et al.
`Prokoski
`Hagelin
`Barnett et al.
`Reader
`Coffman et al.
`Lawrence et al.
`Del Vecchio et al.
`Matsubayashi et al.
`Tsuda
`Poltorak
`Barney
`Duan et al.
`Colson et al.
`Goodman ct al.
`Lee et al.
`I)’I,oren
`Leibowitz et a1.
`
`200 5/0071743
`2005/0149420
`2006/00364 52
`2006/00364 53
`2006/0036529
`2006/0036632
`2006/0036635
`2006/0074867
`2006/0224999
`2007/0073625
`2007/0073748
`2007/0088738
`2007/0094297
`2007/0150298
`2007/0208669
`2008/0091620
`
`A1*
`A1
`A1
`A1
`A1
`A1
`A1
`A1*
`A1
`A1
`A1
`A1
`A1
`A1
`A1
`A1
`
`3/2005
`7/2005
`2/2006
`2/2006
`2/2006
`2/2006
`2/2006
`4/2006
`10/2006
`3/2007
`3/2007
`4/2007
`4/2007
`6/2007
`9/2007
`4/2008
`
`715/500
`
`Harrington et al.
`Hagelin et al
`Williams
`Williams
`Williams
`Williams
`Williams
`Breitzman ...................... .. 707/3
`Albrecht et al.
`Shelton
`Barney
`Barney et al.
`Barney
`Barney
`Rivette et al.
`Vollenweider ct al.
`
`OTHER PUBLICATIONS
`
`“Calculation of Indicated Market Value” TRRU IP Valuation Report
`Dec. 14, 2000.
`“Empirical Evidence of Patent Validity,” AIPLA Quarterly Journal,
`vol. 28: 185. pp. 187-275. 1998.
`“Intellectual Property,” VVebsite: www.edtn.con1. Feb. 15, 1999.
`“Patent Cafe Website of Satisfy the Inventor’s Heartiest Appetite,”
`Desert Maller News. Sep. 21, 1999.
`“Patent Claim Analysis.” Site Hawk Date Unknown.
`“The Open Platform for Intellectual Property Asset Management"
`AURGIN products pamphlet for Aureka 7.0 software. 1999.
`“Value Relevance of Nonfinancial Information: The Case of Patent
`Data.” Nov. 2001.
`AIPLA., “Report ofEconomic Survey” (1991).
`Albert, M.B., D. Avery, F. Narin and P. McAllister “Direct Validation
`of Citation Counts as Indicators oflndustrially Important Patents”,
`Research Policy,vo1. 20. 1991. pp. 251-259.
`Allison, J.R., M.A. Leinley, K.A. Moore and R.D. Tr11nkey“Valuable
`Patents”, Boalt Working Papers in Public law, Paper 28, 2003.
`American Intellectual Property Law Association, Report of Eco-
`nomic Survey, pp. 63-63 (1999).
`Aurigin Systems. Inc. Aureka Cite Module 1998.
`Barney.U.A, “A Study Patent Mortality Rates: USing Statistical S111‘-
`vival Analysis to Rate and Value Patent Assets”, AIPLA Quarterly
`Journal, vol. 30, No. 3, pp. 317-352.
`Barron, Russell 1. “Better Accounting for Patent Portfolios” Legal
`Times. Oct. 16, 2000. pp. 91-92.
`Branrson, Robert S “Valuing Patents, Technologies and Portfolios:
`Rules of Thumb.” Website: www.ventius.com May 1, 2000.
`Buchanan,
`.I.M. “Patent Rankings: The Numbers Garne v. Patent
`Quality”, blog post, downloaded from wwW.rethinkip.com, Apr. 20,
`2005.
`Cockburn, Iair1, et al., “Industry Effects and Appropriability Mea-
`sures in the Stock Market’ sVa1uation of R&D and Patents.” National
`Bureau ofEconomic Research. Cambridge, MA. Dec. 1987.
`Freewing Aerial Robotics Corporation Determination of Fair Market
`Value Jun. 1, 1997.
`Grilliches, 7,vi. “Patent Statistics as Economic Indicators: A Survey
`Part I.” National Bureau of Economic Research. Cambridge. MA.
`Mar. 1990.
`Hall, Bronwyn H “Market Value and Patent Citations: A First Look”
`May 2001 .
`Hall, Bronwyn 11. “Innovation and Market Value.” National Bureau
`of Economic Research, Cambridge. MA. Feb. 1999.
`Harhoff, D., F. Narin, F M. Scherer and K. Vopel “Citation Frequency
`and the Value ofPatented Innovation”, Discussion paper 97-27, Aug.
`1997.
`Heiden, Bowman J., "The Microeconomic Asset Value of a Patent:
`An Ernpiracle Study of Highly valuable Swedish-owned Patents,”
`Center for Intellectual Property Studies Dept. of Industrial Manage-
`ment & Economics, Chalmers University ofTechnologv (Apr. 2001).
`Jaffe, Adam B., et al., “International Knowledge Flows: Evidence
`from Patent Citations.” National Bureau of Economic Research.
`Cainbridge, MA. Apr. 1998
`
`Ocean Tomo Ex. 1001-002
`
`

`
`US 9,075,849 B2
`Page 3
`
`(56)
`
`References Cited
`OTHER PUBLICATIONS
`
`Khan, B. Zorina. “Legal Monopoly: Patents andAntitrust Litiation in
`U.S. Manufacturing. 1970-1998.” National Bureau of Economic
`Research Cambridge, MA Apr. 1999.
`Kim, Ad., .\I.W. Partee, T.J. Reynolds and M.A. Santamaria “Patent
`Litigation Risk-Scoring Model”, Proceedings ofthe 2002 IEEE Sys-
`tems and Information Design Symposium, 2002, pp. 13-17.
`Kohonen, T.. S. Kaski, K. I.agus, J. Salojarvi, J. Ilonkela, V. Paatero
`and A Saarela “Self Organization of a Massive Document Collec-
`tion”, IEEE Transactions of Ncural Networks. vol. 11, No. 3, May
`2000, pp. 574-585.
`Lanjouw, J.0. and M. Schankerrnan “Characteristics of Patent Liti-
`gation: A \Vindow on Competition”, RAND Journal of Economics,
`vol. 32, No. 1, Spring 2001, pp. 129-151.
`Lanj ouw, Jcan 0.. ct al. "How to Count Patents and Value Intellectual
`Property: Uses of Patent Renewal and Application Dam.” National
`Bureau of Economic Research. Cambridge, MA. Jul. 1996.
`Lanjouw, Jean 0., et al. “Stylized Facts ofPatent I igation.” National
`Bureau of Economic Research. Cambridge, MA. Oct. 1999.
`Lanjouw, Jean 0., et al. “Stylized Facts of Patent Litigation: Value,
`Scope and Ownership.” National Bureau of Economic Research.
`Cambridge. MA. Mar. 1990.
`Lanjouw, Jean 0., et al. “The Enforcement of Intellectual Property
`Rights: A survey of the Empirical Literature.” National Bureau of
`Economic Research. Cambridge, MA. Dec. 1997.
`Los Angeles Times, “Marketplace of Ideas: Selling Patents Online ”
`Oct. 25. 1999. Section C.
`Malki, Tlli “Intellectual Property andtheValuation ofBiotechnology
`Companies: GEN—dex versus Dow Jones.” 1997.
`Malki, Elli. "Intellectual Property Intensity (IPI) and the Value-
`Growth Effect.” Nov. 12, 1997.
`Mark A. Lernley. et al.. Valuable Patents. 92 Georgetown Law Journal
`435 (2004).
`McGavock, Daniel M. of IPC Group, Inc. “Assessing the Value of
`your Client’s Intellectual Property Rights for Licensing, Sale or
`Litigation.” Presentation for Knobbe, Martens, Olson & Bear, LLP.
`Sep. 27. 1999.
`Multiple Regression http://www.2.chase.ncsu.edu/garson/pa765/re-
`gress.htm Dec. 13, 2001.
`Neifeld, R.A “A Macro-Economic Model Providing Patcnt Valuation
`and Patent Based Conrpany Financial Indicators”, JPTOS vol. 83,
`No. 3, Mar. 2001. pp. 211-222.
`Neifeld, R.A. “Patent Valuation from a Practical View Point, and
`Some Interesting Patent Value Statistics from the PatentValuePredic-
`tor Model”, PatentCafe Magazine, downloaded from www.
`PatentCafe.com, Apr. 14, 2004.
`
`Patent Evaluation Index. The Japan Technomark Foundation (Mar.
`2000).
`Putnam, J.D. “The Value of International Patent Rights”. Ph.D. dis-
`seration, Yale University, May 1996
`Reitzig, Markus “Improving Patent Valuation Methods for Manage-
`ment Validating Indicators by Understanding Patenting Strategies”
`Oct. 2001.
`Rivette. Kevin G.. et al. “Discovering New Value in Intellectual
`Property.” Harvard Business Review. Jan-Feb. 2000.
`S. Benninga, Financial Modeling, 2nd Ed., MIT Press (2000).
`SA Solla, '1'.K. Leen, and KR. Mul1er’s "Leaming the Similarity of
`Documents: An Information-Geometric Approach to Document
`Retrieval and Categorization,”ANIPS, V. 12, pp. 914-920. MIT Press,
`2000.
`Savikas, “Survey Lets Judges Render Some Opinions About the
`Patent Bar,” Nat’l L.J., Jan. 18, 1993. at 57.
`Schankerman, M. and A Pakes “Estimates of the Value of Patent
`Rights in European Countries During the Post—1950 Period”. NBER
`Workina Paper No. 1650, Ju11. 1985.
`Smith & Par, Valuation ofIntellectual Property and IntangibleAssets,
`2nd Ed. (1989).
`Survey on Solutions to Prior Art Searching Internet Patent News
`Service Jun. 30, 1999.
`“The Anatomy of Large-Scale Hypertextual Search Engine.” by
`Sergey Brin and Lawrence Page.
`Impact upon Patent
`'1homas,
`1’. “The Effect of Technological
`Renewal Decisions”. Technology Analysis & Strategic Management,
`vol. II,No. 2, 1999, pp. 181-197.
`Trajtenberg, M. “A Pa11ny for Your Quotes: Patent Citations and the
`Value of innovations", RAND Journal of Economics, vol. 21, No. 1,
`Serino 1990, op. 172-181.
`Trippe, Anthony. “Software Tools for Analyzing Patents,” Website:
`www.tripp.EPSILON.go—concepts.corn. Apr. 1999.
`U.S. Pate11t and Trademark Office, Board of Patent Appeals and
`Interferences. Ex parts Donnor. No. 96-2552, Decided Mar. 26, 1999.
`53 USl’Q2d, pp. 1699-1702.
`United States Patent & Trademark Office, FY 2000 USPTO Annual
`Report.
`van der Drift, J. “Statistics of European Patents on Legal Status and
`Granting Data", World Patent Information, vol. 10, No. 4, May 3-5,
`1988, pp. 243-249.
`Willigan, Walter L “LeveragingYour Intellectual Property: A Proved
`Path to Value Extraction” Dec. 1998.
`Winkless, B., B. O’Connor a11d J. Cooney “Invention Quality Mea-
`surement (IQM) 1. Patent Valuation: The Methods that TRIZ Forgot".
`'1he TRIZ Journal, downloaded from vWWv.triz-journal.com, Sep.
`2003.
`
`* cited by examiner
`
`Ocean Tomo Ex. 1001-003
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21f01LI.6ehS
`
`US 9,075,849 B2
`
`
`
`.§e3Ife!__As_n£!.a€1.
`w13:i.uI1u.1LiI..v15un...2.
`
`43%!..1.Zuu5l«I3131._
`
`(1 .
`
`laid._
`
`E
`
`
`_uIIF...w«w1mNmfiza§..lll1.lI
`
`
`
`llifigwlg_43.u1Im..
`
`1.€.ll|.5.:
`
`_u.Ile
`
`WE
`
`assȴ
`
`«WI;
`
`....I1inI:|_..lln!:si.ls$:
`
`-.n_._.ac.l_1.r:+Iu.._I.:.IIlIn:n.
`
`Ilwrwu_..I
`
`1.3:if!...m.
`1.!E§r.Ewa:nW_..»
`lilj
`._..In....,:..................
`
`.tlu..l..._l,_
`
`
`
`miiawsw.in....1
`
`fi§suK:;D,..DoDM.».:%wrm
`
`
`
`....3£.3..,_..L.:..___=
`
`.1l£.i_.I.w
`
`
`1....gigr.m.
`
`l..fl.....r-,2.3...5.
`IE!~..l,:rE....!-:u$rx«M.;»;.a
`
`t4iai..._;.....:4...H.§..§
`.,m3\1xl.1.s.!.c...3h..
`
`
`.il_.!I
`
`Sun
`.550
`
`_u_m:_._:_m=
`
`Eng350
`
`3_m:E:__
`
`Ocean Tomo Ex. 1001-004
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`Sheet 2 of 12
`
`US 9,075,849 B2
`
`4th Year Maintenance Rate
`vs
`IPQ Snore
`
`4}
`
`AtoU16a939 ‘ll.ofTotal
`
`5%
`
`.
`
`l
`
`l
`
`SD
`
`90
`
`100
`
`110
`
`120
`
`130
`
`149
`
`.« D%
`x
`,
`15D 16D ==~1TD
`
`Intellectual Property Quotient ("IPQ")
`
`Estimated Life Expectancy
`vs.
`IPQ Score
`
`%Maintained
`
`
`
`LifeExpectancy
`
`«SD
`
`TC!
`
`80
`
`90
`
`100
`
`11C!
`
`120
`
`13D 14D
`
`150
`
`160 =°"17l3
`
`FIG. 2B
`
`Intellectual Property Quotient ("IPQ")
`
`Ocean Tomo Ex. 1001-005
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`Sheet 3 of 12
`
`US 9,075,849 B2
`
`110 ---W-4,
`
`120
`
`‘N.
`5 HNPQT P,e,T_P[]P_';-5-'1'
`fI:a=£A1,,A2.,.An}z’
`*’°'v--4»«-
`_,.,,,_.,w-._J
`
`-
`
`~
`
`.
`
`....
`
`125
`
`,
`
`E
`\E_"_”‘_"__‘X
`
`no
`’
`
`_,.,
`,
`,
`g‘TfiF7Cfi Pm.PtJP."3" 31.,
`"W-~*? I:+4.:k.I=a.es.1:TEr2IsT|z:s;1’-:c.*W,,,,__..;,,,,,,,,,,,,,.,.,,,,.__,,,,,.. Patent Populatmn 3
`5" Cb={B1,,B$_:.m;m"|?IW:13,/r
`'
`cgmar
`
`'
`ZCUMFEARE Caztb
`
`r-* 140
`
`’
`
`’ STATISTICALLY
`SIGNIFIC‘.-f5sNT
`DIFFERENCES
`= PV1, F"x-'2.
`.
`. PM-"n
`
`CUNSTHUCT
`HEGHESSIUN
`MDDEL -~} CVm=
`f{F’V’1,F"V2...F".-‘:13
`
`‘
`
`'
`
`150 r
`
`,
`
`:
`‘VINPUI PArENT§Pn1J,v:,«a__*,,,,.
`F, CHARACTERISTICS; w""""""""""""""""WW“-*“"*“*
`,....w———~-=*
`‘LE3: = {E1,C2. ,
`. __n} f
`
`‘‘ F‘alentPnpulatian"C"
`E Wm == ??
`
`E
`
`»eoEm~,,,g«;;gU,,L:gg
`EE‘v'm[Pn] =
`[f{F"+‘1.F\/2_..P‘w'n}
`
`
`
`E
`
`E
`
`V
`
`~~~~
`
`/‘"‘”nu:Pu"fi5”“"EREnncfEo:,7
`A
`'
`
`PATENT QUAUTY 5,59,,»
`JA‘!
`= EVm[Pn]
`
`“M- 170
`
`~*” 180
`
`.
`
`,5”
`
`Ocean Tomo Ex. 1001-006
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`Sheet 4 of 12
`
`US 9,075,849 B2
`
`€_._..,._......mn».
`
`2”3"*“”i._,,§lF¥!3,T,,:3'
`
`%’~i‘fi'I:fwuIw§."|:“EE”V.fl|RIABLE5:-
`I
`.a=b=I:=d=l].25
`;
`ha mgbmficshgj = u 35
`=
`,co =15] -_— n = rn=0
`
`/204 (‘Z00
`
`..l
`
`mpur mxrermna
`}--~e- CHARACTERISTICS
`C, =T 1-, X? K
`
`.
`
`»~
`
`7
`—
`T
`mam PATENT(n)
`_
`7 oaseaveo QUALITY wk-
`
`210 -—.J"""*“ c#["E‘uLA”1:"é"”FT3.°”TEMT[n)
`, PREDICTED QUALITY
`
`i
`
`PWALID) = a + b>C1;+
`
`VFVREDICTIOIH
`VINCORRECT:
`'1N=1N+_1
`
`_
`
`'
`
`P('~;'.'5.LID)bD.S?Hf,\
`M1,.’
`,
`““~.,_
`("I
`
`'
`
`,~
`..
`
`T
`
`,,___,
`,,A
`,
`,:
`Y = INW“-ID
`
`;’_'
`,,
`r 3»-cw “EXT P-‘VENT ,
`[.m.;+1,,
`,..,_ .,
`,
`,.
`
`PATENT
`
`COUNT:
`n =
`
`no
`
`M, was
`W,‘ LAST P.a.TENT;,,:=
`
`\.
`
`'
`
`M
`
`~
`.n.ccuR.n.cv(3.tx)
`(230
`I
`§*Es§‘?‘3'=CC?!L*3.9:t1,'i‘l"‘”“-,_,228
`»
`W-~:;j§;1,(m)>s.n(m-1);_”_‘::
`
`"
`
`aa-'-(Aa/'2)
`
`C0 = IN= 0
`
`238’!
`
`235
`
`.
`
`,
`
`f#3fi(TE5';l:§,_TES.Fa___a4612
`
`SP.(rn—1)? f,
`”\,W_
`/“J./’
`
`., VV
`
`_~.,..W....,N
`
`).,.v*
`
`-; ~:....Wl ab= «ab/2>
`~g/
`CO ___ IN: 0
`.,‘~_M
`218
`
`7'53.
`
`smasn9'"
`~w.fi\§“'q(m_1);fr,x
`,w‘
`
`, _
`‘TEST b— b Ab./Z5
`.,,.._..,Wm....._,.._. W.
`
`,-.
`
`3
`
`>-.
`
`~§1'-5i«:;1;b:>:Aa.A=.Adu%:~:>
`~\MMW
`“Av,-
`
`l
`
`"
`
`. 4 ’
`
`‘
`
`,,«~' "S.u(rEsT)>
`“N.
`‘H RS4 (m -1)?
`
`.....,
`
`,.
`
`. u_..........,
`
`W m‘n_“’~S,qEm,-|;’A2:3 ’“
`
`sn res
`'*-.
`
`TEST1d=CI-11d/*2!‘
`
`‘rfise,
`
`{ Ap
`
`Ocean Tomo Ex. 1001-007
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21f05LI.6ehS
`
`US 9,075,849 B2
`
`Ocean Tomo Ex. 1001-008
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21f06LI.6ehS
`
`US 9,075,849 B2
`
`adi
`
`Ocean Tomo Ex. 1001-009
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`:107AI.6ehS
`
`211
`
`US 9,075,849 B2
`
`
`
` ...\moms.E__%Emmsm
`
`.aE9=m
`
`co_w_>H830
`
`_mE§=E2Emesm
`
`_mWcmfimz
`
`_8s8_m
`
`_mn_.:EE
`
`Ocean Tomo Ex. 1001-010
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`AI.6ehS
`
`21f000
`
`US 9,075,849 B2
`
`
`
`xmoms€;__%...¢mm:_m
`
`.o£men_
`
`tmmsm
`
`.>=m_Eon_
`
`:o_m_>EEO
`
`$9.5
`
`
`
`
`
`LL--_m.Em£.22..:%5w
`
`Ocean Tomo Ex. 1001-011
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21f09LI.6ehS
`
`US 9,075,849 B2
`
`
`
`
`
`wa
`
`
`
` 3985.00388we5328:;WmumsE__o_mEmMVWang.mam:o_§_5E_o_§o>
`
`co_m_>M230
`
`bmmam
`
`..ilIL|I
`
`
`
`
`
`I§Ei|l._mc.:w£E9.émmsm
`
`m_wo£mo_n_
`
`>._m9:m
`
`E.28¢soWismN8.o228.oE:§o.oo:§o.o
`
`Ocean Tomo Ex. 1001-012
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21r1001LI.eehS
`
`US 9,075,849 B2
`
`
`
` En:_._m__._u.L E33rut
`
`Ocean Tomo Ex. 1001-013
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21f011AI.6ehS
`
`US 9,075,849 B2
`
`
`
`:mEE_._..m....._o.=.=._ozP
`
`
`
`mu__:o.:um_m_mm_._m:I@__uz._...$:oIw,
`
`
`
`E_._Eom.n_..m_4
`
`Ocean Tomo Ex. 1001-014
`
`

`
`U.S. Patent
`
`Jul. 7, 2015
`
`21r1021LI.eehS
`
`US 9,075,849 B2
`
`S.93
`
`‘
`
`
`
`Mafia.hN_..___.D.VMED.hfl—.___—D
`
`
`
`
`
`
`
`._u_..m_.__._uHm_.,_Hu_m:_._E=
`
`Ocean Tomo Ex. 1001-015
`
`

`
`US 9,075,849 B2
`
`1
`METHOD AND SYSTEM FOR
`PROBABILISTICALLY QUANTIFYING AND
`VISUALIZING RELEVANCE BETVVEEN TVVO
`OR MORE CITATIONALLY OR
`CONTEXTUALLY RELATED DATA OBJECTS
`
`CROSS—R 4 F 4 RENC 4 TO RELATED
`APPLICATIONS
`
`'lhe present application is a continuation of U.S. applica-
`tion Ser. No. 13/958,386, filed Aug. 2, 2013, which is a
`continuation of U.S. application Ser. No. 13/411,441, filed
`Mar. 2, 2012. now U.S. Pat. No. 8,504,560, which is a divi-
`sional ofU.S. application Ser. No. 12/749,381, filed Mar. 29,
`2010, now U.S. Pat. No. 8,131,701, which is a divisional of
`U.S. application Ser. No. 11/236,965, filed Sep. 27, 2005,
`now U.S. Pat. No. 7,716,226, Each of the above—referenced
`applications are hereby incorporated by reference in their
`entireties.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The present invention relates generally to the field of docu-
`ment searching, data mining and data visualization.
`2. Description ofthe Related Art
`The field of data searching and data/text mining is replete
`with various search methods and algorithms for helping
`determine the identity and/or location of documents that may
`have relevance to a particular subject matter of interest. The
`most basic search techniques involve locating specific words
`or word combinations within one or more of a quantity of
`documents contained in a database. This search methodology,
`while very simple to implement, suffers from a number of
`significant drawbacks,
`including slow search processing
`time, limited ability to construct and execute complex search
`queries, and other well-docurnented limitations inherent in
`the use of keywords as search criteria. Improvements to the
`basic keyword search include the use of structured queries
`(e.g., based on Boolean logic), word stemming, wildcards,
`fuzzy logic, contextual analysis and latent semantic analysis.
`Despite its well—doeumented drawbacks, simple key—word
`based searching is still a good entry point to quickly locate
`documents of general interest to a relevant subject matter. It is
`sufficient in many searching applications to locate a particular
`desired piece of information contained within one or more
`documents being searched. However, there are many special-
`ized searching applications, particularly in the science, tech-
`nology, academic and legal fields, where keyword searching
`(even with the various improvements to date) provides an ,
`unsatisfactory approach for locating some or all of the rel-
`evant documents that may be of interest to a researcher. The
`primary underlying difficulty is that words and word phrases
`are imprecise by their nature. Different words and word
`phrases can have completely different meanings in different
`associative contexts. As a result, key—word based searching in
`these and other specialized searching applications tends to be
`a slow and tedious process, typically producing significant
`numbers of irrelevant documents or “false hits” and often
`failing to turn up one or more desired relevant documents.
`More advanced searching techniques rely on contextual or
`bibliographical linkages between two or more documents.
`For example, U.S. Pat. No. 6,754,873 issued Jun. 22, 2004 to
`Law, et. al. describes a search technique for finding related
`hyperlinked documents located on the world—wide—web using
`link-based analysis. hi this case backlink and forwardlink sets
`are utilized to find web pages that are related to a particular
`
`2
`selected web page ofinterest. The resulting list ofrelated web
`pages is typically sorted in accordance with a calculated
`relevancy score, the intent being that presumably tl1e most
`relevant and/or highest quality hits would be listed toward the
`top of the search results page and the least relevant and/or
`lowest quality hits would be listed toward the bottom of the
`search results page.
`Relevancy scores are typically calculated as an arbitrary
`score or metric based on one or more selected factors deter-
`mined (or assumed) to be informative as to the quality or
`relevance ofthe search output relative to the search input. For
`example, tl1e search engine may assign an arbitrary rank or
`score to each hit calculated according to the munber or fre-
`quency of keyword occurrences in each document, the intent
`being that the otal score would roughly correspond to the
`relevance or importance of the particular located document
`relative to tl1e input search query. Another example, described
`in the article entitled "The Anatomy of a Large-Scale Hyper-
`textual Search Engine,” by Sergey Brin and Lawrence Page,
`‘ assigns a degree ofimportance to a web page based on the link
`structure of the web page. In this manner, the Brin and Page
`algorithm attempts to quantify the importance of a web page
`based not on its content, but on the number and quality of
`linkages to and from other web pages.
`U.S. Pat. No. 6,526,440 issued Feb. 25, 2003 to Bharat and
`assigned to Google, Inc. describes a similar search engine for
`searching a corpus of data and refining a standard relevancy
`score based on the interconnectivity of the initially returned
`set of documents. The search engine obtains an initial set of
`relevant documents by matching search terms to an index of
`a corpus. A re-ranking component in the search engine then
`refines the initially returned document rankings so that docu-
`ments frequently cited in the initial set of relevant documents
`are preferred over documents that are less frequently cited
`within the initial set. The resulting hits in each case are typi-
`cally displayed in a text-scrolled list, with the relative place-
`ment of each hit on the list being determined in accordance
`with the calculated relevancy score. This, in essence, is the
`primary search and relevance ranking algorithm behind the
`popular Google® search engine.
`As with the Google® search engine, many of the more
`sophisticated search engines today are primarily optimized
`toward the task of searching the world wide web for relevant
`documents ofa general-content nature and focusing typically
`on a single item of information or a single concept. Most
`searches conducted using these types of search algorithms
`seek to find particular items ofinfonnation that are essentially
`known to exist and that can be described with a few simple
`key words. The probability that a user would be able to
`successfully use a search engine in this context to locate at
`least one source of information satisfying the user’s need is
`fairly high. However, in certain specialized searching appli-
`cations, particularly in tl1e science, technology, academic and
`legal fields, conventional search engines provide an unsatis-
`factory approach for locating some or all of the relevant
`documents that may be of interest to a researcher.
`For example, those skilled in the intellectual property arts
`and the patent legal field in general will readily appreciate the
`difficulty and challenge of searching through vast databases
`of case law, patents and related scientific documents looking
`for “prior art" documents relevant to a particular issued patent
`or pending application and/or cases relevant to a particular
`point of law. For patents the difliculty and challenge stems
`from the confluence of several unique factors affecting pat-
`ents and patent—related documents. These factors include the
`shear volume of potentially relevant pate11t documents and
`related scientific literature (estimated at over 80 million docu-
`
`Ocean Tomo Ex. 1001-016
`
`

`
`US 9,075,849 B2
`
`3
`ments worldwide), latent inaccuracies and inconsistencies in
`the technology classifications used by the various national
`and international patent ofiices, the complex scientific nature
`ofpatent disclosures, the ever evolving lexicon for describing
`novel patented concepts and structures, language translation 5
`issues in the case of relevant foreign patent documents and
`scientific literature, and the proclivity of patent attorneys and
`agents to use complex legalese and coined lexicon to describe
`novel concepts. The purpose of the patent search is also quite
`different than the normal search context. The point is not so
`much to find useful information relevant to a concept of
`interest, but to establish and document legal evidence of the
`existence or non-existence of a particular concept or idea in
`combination witl1 one or more other related concepts or ideas
`at a particular point in time.
`Traditional search engines are not particularly adept at
`efliciently handling these and other types of specialized
`searching applications. The standard inputjoutput text inter-
`face ofmost conventional search engines also does a poorjob
`of displaying and communicating input/output search criteria
`and search results in a way that facilitates intuitive under-
`standing and visualization of the logical relationships sought
`to be explored between two or more related concepts being
`searched. It would be of particular benefit to provide an '
`improved search algorithm, database and user interface that
`would overcome or at least mitigate some or all of the above-
`noted problems and limitations.
`
`~
`
`SUMMARY OF THE INVENTION
`
`In one embodiment the present invention provides a novel
`method for probabilistically quantifying a degree of rel-
`evance between two or more citationally or contextually
`related data objects. Data objects may include, for example
`and without limitation, patent documents, non-patent docu-
`ments, reported case law, web pages, personal and corporate
`contacts information, product information, consumer behav-
`ior, technical or scientific information, address information,
`and the like.
`
`In another embodiment the present invention provides a
`novel method for visualizing and displaying relevance
`bctwccn two or more citationally or contextually rclatcd data
`objects.
`In anothcr embodiment the prcscnt invention provides a
`novel search input/output interface that displays and/or com-
`municates search input criteria and corresponding search
`results in a way that facilitates intuitive imderstanding and
`visualization ofthe logical relationships between two or more ,
`related concepts being searched.
`In another embodiment the present invention provides a
`r1ovel search input/output interface that utilizes an iterative
`self-organizing mapping (“SOM”) technique to automati-
`cally gcncratc a visual map of rclcvant patcnts and/or othcr
`related documents desired to be explored, searched or ana-
`lyzed.
`In another embodiment the present invention provides a
`statistically optimized relevance scoring system for statisti-
`cally quantifying the degree of relevance between two or
`more citationally and/or contextually related documents
`according to a calculated event probability that a particular
`selected relationship exists between the two or more selected
`documents.
`In another embodiment the present invention provides an
`improved search algorithm having capability to statistically
`quantify a degree of relevance between two or more citation-
`
`4
`ally and/or contextually related documents and to provide an
`interactive visual interface for displaying and interacting with
`the resulting data set.
`In another embodiment the present invention provides an
`improved search method and algorithm for locating patent
`documents and’or other related documents of interest. A first
`group of patents is identified representing the closest known
`references to a particular technology or search topic of inter-
`est. Relevance analysis is performed on the first group to
`generate a second group of relevant patents, each having an
`associated relevance score to the first group. A user reviews
`the second group of relevant patents and selectively adds any
`desired additional relevant patents to the first group. The
`search method is iteratively repeated as many times as desired
`to generate a desired list of most relevant patents and/’or otl1er
`documents of interest.
`In another embodiment the present invention provides a11
`improved method and system for probabilistically quantify-
`i11g the degree of relevance between two or more citationally
`and/or contextually related documents and an interactive
`visual interface for representing a resulting determined rel-
`cvant documcnt sct in thc form of a sclf-organizing map
`(“SOM”) comprising one or more depicted subject matter
`domains or “landscapes.”
`In another embodiment tl1e present invention provides a11
`improved method and system for rating and analyzing patents
`using relational citation analysis in conjunction with a self-
`organizing mapping technique that maps or categorizes pat-
`ents by iteratively adjusting or optimizing an arbitrary or
`scaled distance between citationally related and/or imrelated
`patents within a multi-dimensional space.
`In another embodiment the present invention provides an
`improved model approach for quantitatively measuring a
`degree ofrelevance between two or more patents a11d/or otl1er
`documents of interest and to thereby group, map and/or clus-
`tcr rclcvant patents and related documents objcctivcly and
`repeatable.
`In another embodiment the present invention provides an
`improved model approach for quantitatively measuring a
`degree of relevance between two or more patents and/or other
`documents of interest by analyzing citational relationships
`between multiple related documents (“relational citation
`analysis”). Relational citation analysis is a novel technique
`that exploits citational and/or contextual relationships ("rel-
`cvancc links”) bctwccn two or more patent documents and/or
`other related documents of interest for the purpose of quan-
`titatively measuring a degree of relevance.
`In another embodiment the present invention a determined
`relevance regression transform function is executed by a
`high— speed computer across an entire database of potentially
`relevant documents. Relevance scores are calculated between
`each document and each other document (or potentially rel-
`evant document) in the database and the results are stored in
`an accessible indcx so that rclcvancc scores can bc instantly
`accessed on the fly as needed.
`In another embodiment the present invention provides an
`improved technique for measuring contextual relatedness or
`contextual
`simila