UNITED STATES PATENT AND TRADEMARK OFFICE
`________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`________________
`
`MINDGEEK USA INC., MINDGEEK S.À.R.L.,
`MG FREESITES LTD., MG FREESITES II LTD.,
`MG CONTENT RK LTD., MG CONTENT DP LTD.,
`MG CONTENT RT LTD., MG PREMIUM LTD.,
`MG CONTENT SC LTD., MG CYPRUS LTD.,
`LICENSING IP INTERNATIONAL S.À.R.L.,
`9219-1568 QUÉBEC INC. d/b/a ENTREPRISE MINDGEEK CANADA, and
`COLBETTE II LTD.,
`
`Petitioners
`
`v.
`
`UNIVERSITY OF SOUTHERN CALIFORNIA
`
`PRESERVATION TECHNOLOGIES LLC
`
`Patent No. 6,574,638
`
`Issue Date: June 3, 2003
`
`Title: Method and Apparatus for Cataloguing Multimedia Data Using Surveying
`Data
`____________________________________________________________
`
`PETITION FOR INTER PARTES REVIEW
`UNDER 35 U.S.C. §§ 311-319 AND 37 C.F.R. § 42.100 ET SEQ.
`
`

`

`EXHIBIT LIST
`
`Exhibit Number Description
`
`1001
`
`1002
`
`1003
`
`1004
`
`1005
`
`1006
`
`1007
`
`1008
`
`1009
`
`1010
`
`1011
`
`U.S. Patent No. 6,574,638 to Gustman
`
`U.S. Patent No. 5,832,495 to Gustman
`
`Declaration of Dr. Jagadish
`
`U.S. Patent No. 6,574,638 File History
`
`DexMedia, Inc. v. Click-to-Call Technologies, Inc., et al.,
`
`Case No. 15-1242, Dkt. Nos. 127-128.
`
`Dr. Jagadish CV
`
`Query by Image and Video Content: The QBIC System to
`
`Flickner et al.
`
`Efficient Organization and Access of Multi-Dimensional
`
`Datasets on Tertiary Storage Systems to Chen et al.
`
`A Distributed Hierarchical Storage Manager for a Video-on-
`
`Demand System to Federighi et al.
`
`Content-Based Video Indexing and Retrieval to Smoliar et
`
`al.
`
`Media Streams: An Iconic Visual Language for Video
`
`Representation to Davis
`
`i
`
`

`

`1012
`
`1013
`
`An objective video quality assessment system based on
`
`human perception to Webster et al.
`
`IDIC: Assembling Video Sequences from Story Plans and
`
`Content Annotations to Sack et al.
`
`ii
`
`

`

`TABLE OF CONTENTS
`INTRODUCTION..............................................................................................1
`
`I.
`
`II. MANDATORY NOTICES................................................................................2
`
`A. Lead and Backup Counsel and Service Information..................................2
`
`B. Real Parties-in-Interest ...............................................................................2
`
`C. Related Matters...........................................................................................3
`
`III. CERTIFICATION OF GROUNDS FOR STANDING.....................................5
`
`IV. THRESHOLD REQUIREMENT FOR INTER PARTES REVIEW..................6
`
`V. STATEMENT OF PRECISE RELIEF REQUESTED......................................7
`
`VI. OVERVIEW OF THE ’638 PATENT AND PRIOR ART ...............................7
`
`A. State of the Technology Prior to the ’638 Patent .......................................7
`
`B. The ’638 Patent...........................................................................................8
`
`C. The Prior Art.............................................................................................10
`
`1. Gustman...............................................................................................10
`
`VII. CLAIM CONSTRUCTION.............................................................................11
`
`VIII.
`
`LEVEL OF ORDINARY SKILL IN THE ART .....................................12
`
`IX. CLAIM-BY-CLAIM EXPLANATION OF GROUNDS FOR
`
`UNPATENTABILITY.............................................................................................12
`
`iii
`
`

`

`A. Ground 1: Claims 1-26 are Anticipated by Gustman..............................12
`
`X. CONCLUSION ................................................................................................42
`
`iv
`
`

`

`I.
`
`INTRODUCTION
`
`MINDGEEK USA INC., MINDGEEK S.À.R.L., MG FREESITES LTD.,
`
`MG FREESITES II LTD., MG CONTENT RK LTD., MG CONTENT DP LTD.,
`
`MG CONTENT RT LTD., MG PREMIUM LTD., MG CONTENT SC LTD., MG
`
`CYPRUS LTD., LICENSING IP INTERNATIONAL S.À.R.L., 9219-1568
`
`QUÉBEC INC. d/b/a ENTREPRISE MINDGEEK CANADA, and COLBETTE II
`
`LTD. (“Petitioners” or “MindGeek”) petition for Inter Partes Review (“IPR”) under
`
`35 U.S.C. §§ 311-319 and 37 C.F.R. § 42 claims 1-26 (“the Challenged Claims”) of
`
`US 6,574,638 (“’638 Patent”).
`
`Preservation Technologies LLC has alleged that, “The Asserted Patents are
`
`currently owned by the University of Southern California and Preservation has
`
`obtained a license with all necessary rights from the Shoah Foundation of the
`
`University of Southern California (the ‘Shoah Foundation’) to enforce these patents
`
`against Defendants in its own name.” Preservation Technologies LLC v. MindGeek
`
`USA Inc. et al, Case No. 2:17-cv-08906-DOC-JPR (C.D. Cal.), Dkt. No. 63, ¶41.
`
`The University of Southern California and Preservation Technologies LLC are
`
`collectively referred to as “Patent Owner.”
`
`1
`
`

`

`II. MANDATORY NOTICES
`
`A.
`
`Lead and Backup Counsel and Service Information
`
`LEAD
`
`BACK-UP COUNSEL
`
`Frank M. Gasparo (Reg. No. 44,700)
`
`William Hector (Reg. No. 66,823)
`
`(Lead)
`
`Address:
`
`Venable LLP
`
`(Backup)
`
`Address:
`
`Venable LLP
`
`1270 Avenue of the Americas, 24th
`
`San Francisco, CA
`
`Floor
`
`101 California Street, Suite 3800
`
`New York, NY 10020
`
`San Francisco, CA 94111
`
`Phone and Fax:
`
`P: (212) 370-6273.
`
`F: (212) 307-5598
`
`Phone and Fax:
`
`P: (415) 653-3750
`
`F: (415) 653-3755
`
`Please send all correspondence to the lead counsel at the address shown
`
`above. Petitioners consent to service by email at: FMGasparo@Venable.com.
`
`Powers of Attorney are filed concurrently herewith under 37 C.F.R. § 42.10(b).
`
`The Office is authorized to charge the fee set forth in 37 C.F.R. § 42.15(a) to
`
`Deposit Account No. 22-0261, and any other fees that might be due in connection
`
`with this Petition.
`
`B.
`
`Real Parties-in-Interest
`
`2
`
`

`

`The real parties-in-interest for this Petition are MINDGEEK USA INC.,
`
`MINDGEEK S.À.R.L., MG FREESITES LTD., MG FREESITES II LTD., MG
`
`CONTENT RK LTD., MG CONTENT DP LTD., MG CONTENT RT LTD., MG
`
`PREMIUM LTD., MG CONTENT SC LTD., MG CYPRUS LTD., LICENSING IP
`
`INTERNATIONAL S.À.R.L., 9219-1568 QUÉBEC INC. d/b/a ENTREPRISE
`
`MINDGEEK CANADA, and COLBETTE II LTD.
`
`Based on Preservation Technologies LLC’s allegations above, we understand
`
`the real parties-in-interest regarding ownership of the ’638 Patent are University of
`
`Southern California and Preservation Technologies LLC.
`
`C.
`
`Related Matters
`
`Petitioners identify the following matters related to the ’638 Patent:
`
`Ongoing
`
`1.
`
`Preservation Technologies LLC v. MindGeek USA Inc. et al., Case
`
`No. 2:17-cv-08906-DOC-JPR (C.D. Cal.);
`
`2.
`
`Preservation Technologies LLC v. MG Content RK Ltd. et al., Case
`
`No. 2:18-cv-03058-DOC-JPR (C.D. Cal.) (consolidated with the above case. Dkt.
`
`No. 14);
`
`Dismissed
`
`3.
`
`Preservation Technologies LLC v. MindGeek USA Inc., Case No.
`
`1:14-cv-01292 (D. Del.);
`
`3
`
`

`

`4.
`
`Preservation Technologies LLC v. Target Brands Inc., Case No. 1:13-
`
`cv-01692 (D. Del.);
`
`5.
`
`Preservation Technologies LLC v. ESPN, Inc., Case No. 8:12-cv-
`
`01542 (C.D. Cal.);
`
`6.
`
`Preservation Technologies LLC v. Hulu LLC, Case No. 8:11-cv-
`
`01985 (C.D. Cal.);
`
`7.
`
`Preservation Technologies LLC v. Dish Network Corp., Case No.
`
`2:11-cv-10692 (C.D. Cal.);
`
`8.
`
`Preservation Technologies LLC v. Sony Corporation of America, Case
`
`No. 2:11-cv-10694 (C.D. Cal.);
`
`9.
`
`Preservation Technologies LLC v. Facebook, Inc., Case No. 2:11-cv-
`
`10701 (C.D. Cal.);
`
`10.
`
`Preservation Technologies LLC v. Netflix, Inc. et al., Case No. 8:11-
`
`cv-01860 (C.D. Cal.); and
`
`11.
`
`Preservation Technologies LLC v. Netflix, Inc. et al., Case No. 8:11-
`
`cv-01862 (C.D. Cal.).
`
`Contemporaneously with this filing, Petitioners are also filing a petition for
`
`IPR of three other patents: U.S. Patent Nos. 6,092,080; 6,199,060; and 6,212,527.
`
`4
`
`

`

`III. CERTIFICATION OF GROUNDS FOR STANDING
`
`On October 10, 2014, Preservation Technologies filed a complaint against
`
`Mindgeek USA Incorporated in the District of Delaware listing, among eleven
`
`patents, U.S. Patent Nos. 6,092,080, 6,199,060, 6,212,527, and 6,574,638.
`
`Preservation Technologies LLC v. Mindgeek USA Incorporated, 1:14-cv-01292-
`
`SLR (D. Del.). The complaint was served on October 14, 2014. Preservation
`
`Technologies LLC v. Mindgeek USA Incorporated, 1:14-cv-01292-SLR (D. Del.),
`
`Dkt. No. 4. Plaintiff filed a notice of voluntary dismissal without reason on February
`
`2, 2015 (id., Dkt. No. 10) and remained silent until it filed its subsequent case on
`
`December 11, 2017. Preservation Technologies LLC v. MindGeek USA Inc. et al,
`
`Case No. 2:17-cv-08906-DOC-JPR (C.D. Cal.), Dkt. No. 1.
`
`Petitioners acknowledge recent Federal Circuit precedent that even if a
`
`complaint is voluntary dismissed, the one year IPR time bar applies to such a
`
`complaint. Click-To-Call Techs., LP v. Ingenio, Inc., 899 F.3d 1321, 1328 (Fed. Cir.
`
`2018) (“The principal question on appeal is whether the Board erred in interpreting
`
`the phrase served with a complaint alleging infringement of a patent recited in §
`
`315(b) such that the voluntary dismissal without prejudice of the civil action in
`
`which the complaint was served does not trigger the bar. We hold that it did.”)
`
`(quotations and citation omitted).
`
`5
`
`

`

`In dissent, Judge Dyk (joined by Judge Lourie) outlined several reasons why
`
`the decision is incorrect and that “section 315(b) time-bar should not apply when the
`
`underlying complaint alleging infringement has been voluntarily dismissed without
`
`prejudice.” Click-To-Call Techs., 899 F.3d at 1355. We understand that DexMedia,
`
`Inc. (formerly Ingenio, Inc.) has been granted an extension to January 11, 2019, to
`
`file writ of certiorari with the United States Supreme Court. DexMedia, Inc. v. Click-
`
`to-Call Technologies, Inc., et al., Case No. 15-1242, Dkt. No. 128. See Exhibit 1005.
`
`In Preservation Technologies LLC v. MindGeek USA Inc. et al, Case No. 2:17-
`
`cv-08906-DOC-JPR (C.D. CA), the complaint was filed on December 11, 2017 with
`
`service of the complaint of certain entities on December 12, 2017.
`
`To the extent Click-To-Call is relevant to the present IPR, we believe Click-
`
`To-Call was incorrectly decided and to preserve our rights of appeal and otherwise,
`
`we are filing the ’638 Patent petition. Thus, based on the foregoing reasons,
`
`Petitioners certify the ’638 Patent is eligible for IPR and that Petitioners are not
`
`barred or estopped from requesting IPR challenging the patent claims on the
`
`Grounds identified herein.
`
`IV. THRESHOLD REQUIREMENT FOR INTER PARTES REVIEW
`
`As further detailed below, claims 1-26 are invalid under 35 U.S.C. §102.
`
`Thus, “there is a reasonable likelihood that the petitioner would prevail with
`
`respect to at least 1 of the claims challenged in the petition.” 35 U.S.C. § 314(a).
`
`6
`
`

`

`V.
`
`STATEMENT OF PRECISE RELIEF REQUESTED
`
`Petitioners respectfully request cancellation of claims 1-26 of the ’638 Patent
`
`(Exhibit 1001) based on the following Ground of Unpatentability, set forth below:
`
`Ground 1: Claims 1-26 are unpatentable under 35 U.S.C. §102(b) as
`
`anticipated over U.S. Patent No. 5,832,495 to Gustman (“Gustman”) (Exhibit 1002).
`
`VI. OVERVIEW OF THE ’638 PATENT AND PRIOR ART
`
`A.
`
`State of the Technology Prior to the ’638 Patent
`
`It was typical practice to associate a set of keywords or index terms with a
`
`document, and to index only these in a catalog. Exhibit 1003, ¶27. Attributes or
`
`descriptions of the stored multimedia object could be stored and indexed in a catalog
`
`separate from the object itself. Id., ¶28.
`
`By the early 1990s, computers had become powerful enough and digital
`
`storage capacities had increased enough that it became feasible to digitize and store
`
`videos: not just short clips but even entire movies. Id., ¶30. There was tremendous
`
`interest in so-called “video-on-demand.” Id., ¶30. There was a great deal of progress
`
`made on the storage and retrieval of videos. Id., ¶30. There was also work reported
`
`on building these servers with tertiary storage. Id., ¶31. And also in a distributed
`
`environment. Id., ¶31.
`
`When a user specifies some search terms to retrieve a text document, it is often
`
`useful for the system to point the user to specific portions of the document that are
`
`7
`
`

`

`relevant, particularly if the document is long.
`
`Id., ¶32. Technology for such
`
`identification of “segments” or “snippets” of a text document has long been known.
`
`Id., ¶32.
`
`Similarly, given a long video, it is helpful for a user to be pointed to specific
`
`portions of it that are relevant to a particular search request. Id., ¶33. Fortunately,
`
`standard methods for video storage already segment videos into scenes. Id., ¶33.
`
`B.
`
`The ’638 Patent
`
`The ’638 Patent was filed on September 12, 2000 with no priority claim. The
`
`’638 Patent issued on June 3, 2003.
`
`The ‘638 Patent relates to a method and apparatus for cataloguing multimedia
`
`data using survey data. Exhibit 1001, 1:9-12. According to the patent, it is useful
`
`to catalogue multimedia data in a multimedia system such that the data may be
`
`identified and retrieved by searching the catalog, e.g., descriptive information of a
`
`catalogue may be associated to segments of multimedia data, and the descriptive
`
`information may be queried and used to identify relevant segments of multimedia
`
`data. Id., 1:42-48.
`
`The ‘638 Patent asserts that existing systems do not provide the ability to
`
`relate a survey database to a multimedia system, so it would be beneficial to
`
`incorporate the results of a survey into a catalogue that may be used to search and
`
`retrieve multimedia data, the patent states. Id., 1:33-35, 50-55.
`
`8
`
`

`

`To this end, the ‘638 Patent contends that a survey scheme comprises elements
`
`that may be used to build a survey and store answers to survey questions, and
`
`elements of the survey may be linked to elements of a multimedia catalogue that
`
`are associated with the multimedia data, and survey data may be queried to identify
`
`a catalogue element and the multimedia data associated with the catalogue element.
`
`Id., 227-38.
`
`For example, the ‘638 Patent asserts that survey data 1032 comprises elements
`
`such as question 1402 and answer 1404, and that catalogue 1304 comprises
`
`elements such as keyword 1406 and segment 1408 that are related to multimedia
`
`data 1306, such that either an instance of keyword 1406 or segment 1408 or both
`
`may be used to identify multimedia data 1306. Id., 4:33-63, Figures 13-15.
`
`According to ‘638 Patent, a survey question 1402 may be queried to identify
`
`keywords 1406 that are associated with question 1402, or a set of segments may be
`
`identified that comprises instances of segment 1408 that are related to an instance
`
`of answer 1404, the patent says. Id., 5:35-47, 6:37-7:11.
`
`The ‘638 Patent asserts that the catalogue has a catalogue element referred to
`
`as a “phrase” 1502 or 1906, which has a plurality of attributes (e.g., keyword 1406
`
`or 1910, person, image, video, proposed person, proposed keyword, etc.), where the
`
`keyword catalogue element has attributes such as thesaural keyword, keyword, and
`
`“type.” Id., 16-46-63.
`
`9
`
`

`

`C.
`
`The Prior Art
`
`1.
`
`Gustman
`
`Gustman, U.S. Patent No. 5,832,495, is entitled “Method and Apparatus for
`
`Cataloguing Multimedia Data.” It was filed on July 8, 1996 and issued on November
`
`3, 1998. Exhibit 1002. Therefore, it is 35 U.S.C. §102(b) prior art to the ’638 Patent
`
`which was filed on September 12, 2000. Gustman was cited on the face of the ’638
`
`Patent. Id., 2.
`
`However, 35 U.S.C. § 325(d) is inapplicable because the reference was not
`
`applied against the claims. Exhibit 1004. Importantly, there is no evidence that the
`
`Examiner considered the particular Gustman disclosures outlined by Petitioners
`
`herein. Id.
`
`Gustman associates user input, such as, for example, answers to questions,
`
`with the catalogue. Gustman teaches the same catalogue described in the ‘638 Patent
`
`(indeed, the ‘638 Patent appears to have lifted much of Gustman’s description and
`
`drawings intact, including the description of the catalogue). In more detail, as shown
`
`in Gustman Figures 2A, 2B, and 2C, the catalogue includes catalogue elements
`
`including “phrase” 206, segment 204, keyword 210, proposed keyword 214,
`
`thesaural keyword 212, type 216, person 218, proposed person 224, and thesaural
`
`person 222, all of which are associated with one another. Exhibit 1002, 4:8-22, 8:10-
`
`24, 31-35 (phrase); id., 4:25-34, 8:25-29 (segment); id., 8:36-43 (keyword); id.,
`
`10
`
`

`

`8:64-9:14; id., 10:4-21 (type); id., 4:35-46, 10:50-11:13; see also Id., 3:48-55, 4:8-
`
`22, 8:10-24, 31-35.
`
`Gustman teaches that data may be entered by a user and associated with
`
`catalogue elements. Id., 14:9-15. For example, Gustman discloses that a user may
`
`enter a description (referred to as a “descriptive phrase” (not to be confused with
`
`phrase 206)) for a phrase, segment, or image instance.
`
`Id., 15:32-41, 16:21-26,
`
`17:48-51, 19:5-9, 33-45; Figure 2D.
`
`As another example, Gustman discloses that when button 318 (Fig. 4) is
`
`pressed, a descriptive note is parsed, and the user can identify a keyword that is not
`
`identified during the parsing. Exhibit 1002, 16:27-48. Alternatively, the user can
`
`enter a proposed keyword 214 in block 410 along with an associated comment, or
`
`proposed keyword 214 can also be created by typing a value into block 406. Id.;
`
`Id., 20:17-29. As yet another example, Gustman also discloses that the user may
`
`enter a thesaural attribute element (e.g., thesaural keyword 212) in block 412. Id.,
`
`16:40-47; 19:67-20:6.
`
`Further, Gustman discloses that the system poses a series of questions to the
`
`user to assess the quality of data collection. Id. 10:50-11:13, 16:49-17:7, 17:8-39.
`
`VII. Claim Construction
`
`Petitioners propose, for purposes of this IPR only, that all claim terms of the
`
`’638 Patent take on their ordinary and customary meaning that the terms would have
`
`11
`
`

`

`to one of ordinary skill in the art—no constructions are necessary. Petitioners’
`
`proposal in this proceeding should not be viewed as a concession as to the proper
`
`scope of any claim term in any litigation nor a waiver of any indefiniteness
`
`arguments.
`
`VIII. LEVEL OF ORDINARY SKILL IN THE ART
`
`Petitioners submit that a person of ordinary skill in the art (“POSITA”) would
`
`have had at least a B.S. degree in computer science or electrical engineering (or
`
`comparable degree) and two years of experience in databases or networking. Exhibit
`
`1003, ¶22.
`
`IX. CLAIM-BY-CLAIM EXPLANATION OF GROUNDS FOR
`UNPATENTABILITY
`
`Claims 1-26 are unpatentable as shown in the detailed explanation below.
`
`Exhibit 1003, ¶45.
`
`A.
`
`Ground 1: Claims 1-26 are Anticipated by Gustman.
`
`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`[1a] 1. In a
`computer
`system,
`associating
`multimedia
`data with
`surveying
`
`Gustman
`
`[1a] “An embodiment of the invention can be implemented as
`computer software in the form of computer readable program
`code executed on a general purpose computer such as illustrated
`in FIG. 1. A keyboard 110 and mouse 111 are coupled to a
`bidirectional system bus 118. The keyboard and mouse are for
`introducing user input to the computer system and communicating
`that user input to central processing unit (CPU) 113. Other suitable
`input devices may be used in addition to, or in place of, the mouse
`
`12
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`data
`comprising:
`
`[1b]
`obtaining an
`association
`between
`survey data
`and at least
`one
`catalogue
`element,
`
`Gustman
`
`111 and keyboard 110. I/O (input/output) unit 119 coupled to
`bidirectional system bus 118 represents such I/O elements as a
`printer, network communications card, modem, A/V (audio/video)
`I/O, etc. The computer system of FIG. 1 also includes a video
`memory 114, main memory 115 and mass storage 112, all coupled
`to bidirectional system bus 118 along with keyboard 110, mouse
`111 and CPU 113.” Exhibit 1002, 7:12-61.
`
`“The invention catalogues data such as multimedia data.” Id.,
`7:64.
`
`Regarding “[survey] data,” see 1b below.
`[1b] “Catalogue, attribute and event elements are instantiated and
`maintained using cataloguing and relationship management
`facilities. The cataloguing and relationship management facilities
`are provided to accept user input and generate elements and
`their attributes. The collection of element instances created using
`the cataloguing and relationship management facilities can be used
`as an interface by any system to access input data. One attribute of
`a phrase element is referred to as a descriptive phrase. A user enters
`a descriptive phrase associated with an instance of the phrase
`element using the cataloguing facility interface.” Exhibit 1002,
`4:46-58.
`
`“FIGS. 3A-3F illustrate a cataloguing facility user interface
`screen for reviewing input data and associating the input data
`with catalogue and attribute elements according to an
`embodiment of the invention.” Id., 6:15-18.
`
`“Descriptions or notes can be entered and associated with an
`element using the cataloguing interface. Referring to FIG. 3A, an
`input area 312 is provided to enter a description for a phrase,
`segment, or image instance, for example. The description is
`referred to as a descriptive phrase. A descriptive phrase
`associated with an instance of phrase 206 is parsed to identify a
`
`13
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`Gustman
`
`plurality of instances of keyword 210 and person 218 to be
`associated with the instance of phrase 206.” Id., 15:32-41.
`
`“Control button 300 is used to save the descriptive phrase for a
`phrase. When control button 300 is pressed, the descriptive phrase
`is parsed to identify existing attribute elements (e.g., keyword 210)
`that are related to the new instance of phrase 206.” Id., 16:22-26.
`
`“Referring to FIG. 3A, a user can view the input data in area 310,
`enter a description in area 312. By selecting button 320, the user
`can create an instance of segment 204. Button 300 allows the
`user to create an instance [of] phrase 206. Using button 318, a
`user can delete an instance of segment 204 or phrase 206.” Id.,
`17:48-51.
`
`“At step 902, the creation process creates an ID for the new instance
`of segment 204. The descriptive phrase entered by the user (in
`area 312 of FIG. 3A, for example) is used to initialize the
`
`14
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`Gustman
`
`descriptive phrase attribute of the new instance at step 904. At step
`906, the segment instance is created.” Id., 19:5-9.
`
`“In addition to the catalogue and attribute elements, the invention
`associates a plurality of event elements with the input data. An
`event element is associated with an input data portion. An event
`element is a quality assurance event, for example, that marks an
`input data portion that has relevance to quality assurance data
`collection. Quality assurance events can be identified as a
`positive, negative, or informational event. A quality assurance
`event can also identify a fact or piece of information. An event
`element can also be associated with other elements such as a
`catalogue element (e.g., phrase) or a container element (e.g.,
`segment).” Id., 4:35-46.
`
`“In the preferred embodiment, a series of questions are posed to
`the user to assess the quality of the data collection process.
`Referring to FIG. 3A, button 301 is used to jump to the quality
`assurance questions screen. The user can scroll
`through the
`questions to specify answers. The questions screen allows the user
`to view the input data as well. FIG. 5 provides an example of a
`quality assurance entry screen including quality assurance
`questions according to an embodiment of the invention. Block 502
`provides an area for viewing the input data. Controls 504 allow the
`user to navigate through the input data. Blocks 506, 508, 510 are as
`previously described in FIG. 3C (e.g., blocks 344, 346, and 348).
`The screen also contains areas 512A-512C for viewing and
`responding to a question. Block 516 identifies the number of the
`question. Block 518 contains the quality assurance question. The
`user can respond with a short answer (e.g., a yes or no answer)
`using block 520. In addition, the user can enter comments in
`block 522. A scrolling capability is provided to allow the user to
`scroll through a comment. The user can scroll
`through the
`questions using blocks 534 and 536. For example, by pressing block
`536, question one (block 512A) scrolls off the screen, questions two
`
`15
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`Gustman
`
`and three (blocks 512B and 512C) scroll up, and a fourth question
`appears at the bottom of the screen. The user can return to the screen
`Id., 16:49
`illustrated in FIGS. 3A-3F by pressing block 532.”
`through 17:7.
`
`Id., Figure 5.
`
`“A thesaural attribute element (e.g., thesaural keyword 212)
`can be entered in block 412. In block 412, the attribute element is
`entered in the ‘Word Used’ column and the thesaural term is
`entered in the ‘Preferred Term’ column.” Id., 16:40-47.
`
`“The user can also enter proposed and thesaural attribute
`elements using this screen (e.g., blocks 410, 412, and 418). FIGS.
`10 and 11 provide examples of a process flow for creating a
`thesaural attribute element (e.g.,
`thesaural keyword 212 and
`thesaural person 222) and a proposed attribute element (e.g.,
`proposed
`keyword
`214
`and
`proposed
`person
`224),
`respectively. Referring to FIG. 10, the element identified in the
`input screen (e.g., block 412 of FIG. 4) is obtained at step 1004. The
`16
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`Gustman
`
`thesaural value is obtained from the input screen (e.g., block 412 of
`FIG. 4) at step 1006. An instance of the thesaural attribute element
`(e.g., thesaural keyword 212 and thesaural person 22) is created at
`step 1008. The element relationships are created at step 1010.
`Referring to FIG. 2B, a relationship can exist between keyword 210
`and thesaural keyword 212 (e.g., relationship 238), and between
`person 218 and thesaural person 222 (e.g., relationship 234). At step
`1010, the element located at step 1004 is related to the new thesaural
`instance created at step 1008.” Id., 19:67-20:16.
`
`“The user can identify or propose keywords that are not
`identified during the parsing of the descriptive note. An instance of
`proposed keyword 214 can be entered in block 410 along with
`an associated comment. … A new instance of keyword 210 can
`be proposed (i.e., a proposed keyword 214 can be created) by
`highlighting a value from block 402 and pressing block 404. A
`
`17
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`[1c] said at
`least one
`catalogue
`element
`associated
`with said
`multimedia
`data;
`
`Gustman
`
`proposed keyword 214 can also be created by typing a value into
`block 406.” Id., 16:27-48.
`[1c] “The multimedia data catalogue used in the invention… one
`catalogue element that is referred to as a phrase. A phrase is
`associated with a portion of multimedia data. … In the invention,
`input data is decomposed into one or more pieces, or fragments. An
`instance of phrase 206 is associated with each input data fragment.
`Phrase 206 is a catalogue element.” Exhibit 1002, 4:8-22; 8:10-35.
`
`“Each instance of phrase 206 has one or more attributes and/or
`attribute elements. Attributes such as an in timecode and an out
`timecode identify a portion of the video from an interview that
`is associated with an instance of phrase 206.” Id., 25:14-19.
`
`“A phrase has a plurality of attributes some of which are attribute
`elements. The attribute elements that are attributes of a phrase
`include keyword, person,
`image, video (e.g., documentary
`footage), proposed person, and proposed keyword. The keyword,
`person, image, proposed person and proposed keyword attribute
`elements can have attributes that are also attribute elements. For
`example, attribute elements that are attributes of the keyword
`attribute element include thesaural keyword, thesaural person,
`keyword, and type. … Segment 204 is a container element. It can
`contain other elements. For example, segment 204 can contain one
`or more instances of phrase 206.” Id., 4:8-22; 8:10-35.
`
`18
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`Gustman
`
`19
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`[1d]
`searching
`said survey
`data to
`identify said
`at least one
`catalogue
`element;
`
`[1e]
`identifying
`said
`multimedia
`data using
`said at least
`one
`catalogue
`element.
`2. The
`surveying
`method of
`claim 1
`wherein
`
`Gustman
`
`[1d] “The cataloguing and relationship management facilities are
`provided to accept user input and generate elements and their
`attributes. The collection of element instances created using the
`cataloguing and relationship management facilities can be used as
`an interface by any system to access input data.” Exhibit 1002,
`4:46-58.
`
`“An index is built on the attributes and attribute elements. The
`index can be used to navigate through the catalogue (e.g., search
`for phrases). A set of catalogue elements can be identified by
`querying the attribute elements.” Id., 4:7-23.
`
`“An element of a catalogue has one or more attributes. An attribute
`provides information that can be used to search for, answer
`questions about, and navigate through a catalogue.” Id., Abstract.
`
`“A query operation can be performed on the attribute elements to
`examine other attribute elements associated with a catalogue
`element. A query operation identifies a set of cataloguing
`elements (e.g., phrases) that satisfy the criteria specified in the
`query.” Id., 4:25-27.
`[1e] “A phrase is associated with a portion of multimedia data.”
`Exhibit 1002, 4:7-10.
`
`“Each instance of phrase 206 has one or more attributes and/or
`attribute elements. Attributes such as an in timecode and an out
`timecode identify a portion of the video from an interview that
`is associated with an instance of phrase 206.” Id., 25:14-19.
`
`See 1b above.
`
`“An element of a catalogue has one or more attributes. An
`attribute provides information that can be used to search for,
`
`20
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`said survey
`data
`comprises at
`least one
`survey
`element,
`said survey
`element
`comprises at
`least one
`attribute
`that
`associates
`said at least
`one survey
`element
`with said at
`least one
`catalogue
`element.
`3. The
`method of
`claim 2
`wherein
`said survey
`data
`comprises at
`least one
`question
`element.
`
`Gustman
`
`answer questions about, and navigate through a catalogue.” Exhibit
`1002, Abstract.
`
`“An attribute of phrase 206 is keyword 210. An instance of phrase
`206 can be associated with one or more instances of keyword 210.
`Keyword 210 further defines aspects of an input data fragment.
`Preferably, an attribute of keyword 210 identifies content, or
`substance, for an input data fragment. The content or substance
`identified by keyword 210 is preferably expressed as a single word.
`However, content or substance can be expressed using multiple
`words.” Id., 8:36-43.
`
`See 1b above.
`
`“In the preferred embodiment, a series of questions are posed to the
`user to assess the quality of the data collection process. Referring
`to FIG. 3A, button 301 is used to jump to the quality assurance
`questions screen. The user can scroll through the questions to
`specify answers. The questions screen allows the user to view the
`input data as well. FIG. 5 provides an example of a quality
`assurance entry screen including quality assurance questions
`according to an embodiment of the invention.” Exhibit 1002, 16:49-
`57.
`
`“Block 502 provides an area for viewing the input data. Controls
`504 allow the user to navigate through the input data. Blocks 506,
`508, 510 are as previously described in FIG. 3C (e.g., blocks 344,
`
`21
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`
`Gustman
`
`346, and 348). The screen also contains areas 512A-512C for
`viewing and responding to a question. Block 516 identifies the
`number of the question. Block 518 contains the quality assurance
`question. The user can respond with a short answer (e.g., a yes or
`no answer) using block 520. In addition,
`the user can enter
`comments in block 522. A scrolling capability is provided to allow
`the user to scroll through a comment. The user can scroll through
`the questions using blocks 534 and 536. For example, by pressing
`block 536, question one (block 512A) scrolls off the screen,
`questions two and three (blocks 512B and 512C) scroll up, and a
`fourth question appears at the bottom of the screen. The user can
`return to the screen illustrated in FIGS. 3A-3F by pressing block
`532.” Id., 16:58-17:7.
`
`Id., Figure 5.
`See 1b and Claim 3 above.
`
`4. The
`method of
`claim 2
`
`22
`
`

`

`U.S. Patent
`No.
`6,574,638
`Claim
`Language
`wherein
`said survey
`data
`comprises at
`least one
`answer
`element.
`
`5. The
`method of
`claim 2
`wherein
`said at least
`one attribute
`identifies at
`least one
`segment
`element of
`said
`catalogue.
`
`Gustman
`
`“Block 502 provides an area for viewing the input data. Controls
`504 allow the user to navigate through the input data. Blocks 506,
`508, 510 are as previously described in FIG. 3C (e.g., blocks 344,
`346, and 348). The screen also contains areas 512A-512C for
`vi