Trials@uspto.gov
`571-272-7822
`
`
`
`
`
`
`
`Paper 47
`Entered: July 1, 2020
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`CANON INC., CANON U.S.A., INC., and
`AXIS COMMUNICATIONS AB,
`Petitioner,
`
`v.
`
`AVIGILON FORTRESS CORPORATION,
`Patent Owner.
`____________
`
`IPR2019-00314
`Patent 7,923,923 B2 & C1
`____________
`
`
`
`Before GEORGIANNA W. BRADEN, KIMBERLY McGRAW, and
`JESSICA C. KAISER, Administrative Patent Judges.
`
`McGRAW, Administrative Patent Judge.
`
`
`
`
`JUDGMENT
`Final Written Decision
`Determining No Challenged Claims Unpatentable
`35 U.S.C. § 318(a)
`
`
`
`
`
`
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`
`I. INTRODUCTION
`In this inter partes review, instituted pursuant to 35 U.S.C. § 314,
`Canon Inc., Canon U.S.A., Inc., and Axis Communications AB (collectively
`“Petitioner”) challenge claims 1–41 of U.S. Patent No. 7,932,923 B2 & C1
`(Ex. 1001, “the ’923 patent”), owned by Avigilon Fortress Corporation
`(“Patent Owner”). This Final Written Decision is entered pursuant to
`35 U.S.C. § 318(a) and 37 C.F.R. § 42.73. For the reasons discussed below,
`Petitioner has not shown by a preponderance of the evidence that claims 1–
`41 of the ’923 patent are unpatentable.
`A. Procedural History
`Petitioner filed a Petition for inter partes review of claims 1–41 of
`the ’923 patent. Paper 1 (“Pet.”). Patent Owner filed a Preliminary
`Response. Paper 9. Petitioner then filed an authorized Reply to address
`Patent Owner’s arguments that the asserted references are not printed
`publications (Paper 11), to which Patent Owner filed an authorized Sur-reply
`(Paper 12). Applying the standard set forth in 35 U.S.C. § 314(a), which
`requires demonstration of a reasonable likelihood that Petitioner would
`prevail with respect to at least one challenged claim, we instituted an inter
`partes review of the challenged claims. Paper 13 (“Inst. Dec.”).
`Following institution, Patent Owner filed a Patent Owner Response
`(Paper 27, “PO Resp.”), Petitioner filed a Reply (Paper 31, “Pet. Reply”),
`and Patent Owner filed a Sur-reply (Paper 38, “PO Sur-reply”). An oral
`hearing was held on April 8, 2020, and a copy of the hearing transcript has
`been entered into the record. Paper 46 (“Tr.”).
`B. Related Matters
`Concurrent with the instant Petition, Petitioner filed another petition
`for inter partes review of the ’923 patent in IPR2019-00311. Canon Inc. et
`
`2
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`al. v. Avigilon Fortress Corp., IPR2019-00311 (PTAB Nov. 12, 2018)
`(Paper 1). We issue a final written decision in IPR2019-00311 concurrently
`with this Decision.
`Petitioner also has filed several petitions challenging patents related to
`the ’923 patent.1 For example, Petitioner filed petitions in IPR2018-00138
`and IPR2018-00140 challenging claims of related U.S. Patent
`No. 8,564,661 B2 (“the ’661 patent”). In both of these proceedings, we
`determined that Petitioner had shown by a preponderance of the evidence
`that each of the challenged claims of the ’661 patent are unpatentable. Axis
`Commc’ns AB et al. v. Avigilon Fortress Corp., IPR2018-00138, Paper 25
`(PTAB May 30, 2019); Axis Commc’ns AB et al. v. Avigilon Fortress Corp.,
`IPR2018-00140, Paper 25 (PTAB May 30, 2019).
`Petitioner also filed petitions in IPR2019-00235 and IPR2019-00236
`challenging claims of related U.S. Patent No. 7,868,912 B2. We denied
`institution of both of these proceedings. See Canon Inc. et al. v. Avigilon
`Fortress Corp., IPR2019-00235, Paper 19 (PTAB June 4, 2019) (stating that
`Petitioner did not show the asserted reference qualified as a prior art printed
`publication); Canon Inc. et al. v. Avigilon Fortress Corp., IPR2019-00236,
`Paper 12 (PTAB June 4, 2019) (exercising discretion under 35 U.S.C.
`§ 325(d) to decline institution).
`C. The ’923 Patent (Ex. 1001)
`The ’923 patent, titled “Video Surveillance System Employing Video
`Primitives,” is generally directed to methods, devices, and computer
`readable storage media for video surveillance. See Ex. 1001, codes (54),
`
`
`1 Petitioner states the ’923 patent and U.S. Patent Nos. 8,564,661 B2
`and 7,868,912 B2 are related as each claim priority to U.S. Application No.
`09/694,712. Pet. 8.
`
`3
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`(57), Reexamination Certificate 1:29–4:28. In one embodiment, the
`disclosed video surveillance system operates by (1) obtaining source video,
`(2) extracting “video primitives” from the video, (3) archiving the video
`primitives, (4) extracting “event occurrences” from the video primitives
`using “event discriminators,” and (5) undertaking a response, as appropriate.
`Id. at Fig. 4, 4:30–31, 11:63–65. “Video primitive” refers to an “observable
`attribute” of an object viewed in a video feed, such as the size, shape,
`position, speed, color, and texture of the object. Id. at 7:6–12. The ’923
`patent explains that event discriminators are used to filter the video
`primitives to determine if any event occurrences occurred. Id. at 10:66–
`11:1. For example, an event discriminator can look for a “wrong way” event
`as defined by a person traveling the “wrong way” into an area between
`9:00 a.m. and 5:00 p.m. Id. at 11:1–4. The event discriminator checks the
`video primitives and determines if any video primitives with the following
`properties exist: a timestamp between 9:00 a.m. and 5:00 p.m., a
`classification of “person” or “group of people,” a “position inside the area,”
`and a “wrong direction of motion.” Id. at 11:4–9.
`D. Illustrative Claims
`Petitioner challenges claims 1–41 of the ’923 patent. Claims 1, 8, 9,
`20, 22, 29, and 30 are independent. Claim 1 is representative and is
`reproduced below.
`1. A method comprising:
`[a] detecting an object in a video from a single camera;
`[b] detecting a plurality of attributes of the object by analyzing the
`video from said single camera, the plurality of attributes
`including at least one of a physical attribute and a temporal
`attribute, each attribute representing a characteristic of the
`detected object;
`
`4
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`[c] selecting a new user rule after detecting the plurality of
`attributes;
`[d1] after detecting the plurality of attributes and after selecting the
`new user rule, identifying an event of the object that is not one
`of the detected attributes of the object by applying the new
`user rule to the plurality of detected attributes,
`[d2] wherein the applying the new user rule to the plurality of
`detected attributes comprises applying the new user rule to
`only the plurality of detected attributes;
`[e] wherein the plurality of attributes that are detected are
`independent of which event is identified,
`[f] wherein the step of identifying the event of the object identifies
`the event without reprocessing the video, and
`[g] wherein the event of the object refers to the object engaged in
`an activity.
`
`
`Ex. 1001, Reexamination Certificate, 1:34–55 (matter in brackets added for
`clarity; matter in italics indicates additions made to the claim during the
`reexamination proceeding).
`E. Asserted Challenge to Patentability
`Petitioner asserts that the claims 1–41 are unpatentable based on the
`following challenge (Pet. 3):
`Claims Challenged
`35 U.S.C. §
`1–41
`1032
`
`References
`Dimitrova,3 Brill4
`
`
`2 The Leahy-Smith America Invents Act, Pub. L. No. 112-29, 125 Stat. 284
`(2011) (“AIA”), amended 35 U.S.C. § 103. Because the challenged claims
`of the ’923 patent has an effective filing date before the effective date of the
`applicable AIA amendments, we refer to the pre-AIA versions of § 103.
`3 Nevenka Dimitrova, Motion Recovery for Video Content Classification,
`ACM Transactions on Information Systems, Vol. 13, No. 4, Oct. 1995, 408–
`439 (Ex. 1006, “Dimitrova”).
`4 Frank Brill et al., Event Recognition and Reliability Improvements for the
`Autonomous Video Surveillance System, Proceedings of a Workshop held in
`Monterey California, Nov. 20–23, 1998, pp. 267–283 (Ex. 1004, “Brill”).
`
`5
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`
`
`F. Testimonial Evidence
`In support of its unpatentability contentions, Petitioner relies on a
`declaration by John R. Grindon, D.Sc. (Ex. 1005). Patent Owner cross-
`examined Dr. Grindon via deposition. See Ex. 2018. In support of its Patent
`Owner Response, Patent Owner relies on a declaration by Alan Bovik, Ph.D.
`(Ex. 2019). Petitioner cross-examined Dr. Bovik via deposition. See Ex.
`1056.
`
`II. DISCUSSION
`A. Legal Principles
`To prevail on its challenge to Patent Owner’s claims, Petitioner must
`demonstrate by a preponderance of the evidence that the claims are
`unpatentable. 35 U.S.C. § 316(e) (2012); 37 C.F.R. § 42.1(d) (2018). The
`petitioner “has the burden from the onset to show with particularity why the
`patent it challenges is unpatentable.” Harmonic Inc. v. Avid Tech., Inc., 815
`F.3d 1356, 1363 (Fed. Cir. 2016) (citing 35 U.S.C. § 312(a)(3) (requiring
`inter partes review petitions to identify “with particularity . . . the evidence
`that supports the grounds for the challenge to each claim”)). This burden
`never shifts to Patent Owner. See Dynamic Drinkware, LLC v. Nat’l
`Graphics, Inc., 800 F.3d 1375, 1378 (Fed. Cir. 2015) (citing Tech. Licensing
`Corp. v. Videotek, Inc., 545 F.3d 1316, 1326–27 (Fed. Cir. 2008))
`(discussing the burden of proof in inter partes review).
`A claim is unpatentable for obviousness under 35 U.S.C. § 103(a) if
`the differences between the claimed subject matter and the prior art are
`“such that the subject matter, as a whole, would have been obvious at the
`time the invention was made to a person having ordinary skill in the art to
`which said subject matter pertains.” KSR Int’l Co. v. Teleflex Inc., 550 U.S.
`
`6
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`398, 406 (2007). The question of obviousness is resolved on the basis of
`underlying factual determinations, including (1) the scope and content of the
`prior art; (2) any differences between the claimed subject matter and the
`prior art; (3) the level of skill in the art; and (4) when in evidence, objective
`indicia of non-obviousness (i.e., secondary considerations). Graham v. John
`Deere Co., 383 U.S. 1, 17–18 (1966).
`B. Level of Ordinary Skill in the Art
`In determining whether an invention would have been obvious at the
`time it was made, we consider the level of ordinary skill in the pertinent art
`at the time of the invention. Graham, 383 U.S. at 17 (1966). Before
`institution, the parties provided slightly different definitions of the level of
`skill in the art. See Pet. 8–9 (citing Ex. 1005 ¶¶ 76–80); Prelim. Resp. 7. In
`our Decision on Institution, we adopted Petitioner’s proposed definition of a
`person of ordinary skill in the art as someone having (i) a Bachelor of
`Science degree in electrical engineering, computer engineering, or computer
`science, with approximately two years of experience or research related to
`video processing and/or surveillance systems, or (ii) equivalent training and
`work experience in computer engineering and video processing and/or
`surveillance systems. Inst. Dec. 7–8 (citing Pet. 8–9; Ex. 1005 ¶¶ 76–80).
`Neither party disputes our definition of the level of ordinary skill in
`the art. We see no reason to change our definition based on the complete
`record and, thus, maintain our definition for the purposes of this Decision.
`We further note that our analysis would be the same under either parties’
`definition.
`
`C. Claim Construction
`In an inter partes review filed before November 13, 2018, such as
`here, we construe claim terms in an unexpired patent according to their
`
`7
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`broadest reasonable construction in light of the specification of the patent in
`which they appear. 37 C.F.R. § 42.100(b) (2018);5 Cuozzo Speed Techs.,
`LLC v. Lee, 136 S. Ct. 2131, 2144–46 (2016) (upholding the use of the
`broadest reasonable interpretation standard as the claim interpretation
`standard to be applied in inter partes reviews). Consistent with the broadest
`reasonable construction, claim terms are presumed to have their ordinary and
`customary meaning as understood by a person of ordinary skill in the art in
`the context of the entire patent disclosure. In re Translogic Tech., Inc., 504
`F.3d 1249, 1257 (Fed. Cir. 2007).
`The parties propose constructions for a number of claim terms,
`including the phrase “wherein the applying the new user rule to the plurality
`of detected attributes comprises applying the new user rule to only the
`plurality of detected attributes,” as recited in claims 1, 8, 9, 22, and 29 and
`as similarly recited in claims 20 and 306 (hereinafter the “only” limitations).
`See Pet. 9–23; PO Resp. 5–15; Pet. Reply 4–10; PO Sur-reply 5–9.
`This language was added during the reexamination of the ’923 patent
`to distinguish over certain prior art references. See Ex. 1016 (Oct. 30, 2013
`Amendment), 78–79; Ex. 1017 (June 4, 2014 Final Action), 37–38; Ex. 1018
`
`
`5 The 2018 amendment to this rule does not apply here because the Petition
`was filed before November 13, 2018. See Changes to the Claim
`Construction Standard for Interpreting Claims in Trial Proceedings Before
`the Patent Trial and Appeal Board, 83 Fed. Reg. 51,340 (Oct. 11, 2018)
`(now codified at 37 C.F.R. pt. 42 (2019)) (amending 37 C.F.R. § 42.100(b)
`effective November 13, 2018).
`6 Claim 20 recites “wherein the analysis of the combination of the attributes
`to detect the event comprises analyzing only the combination of attributes.”
`Ex. 1001, Reexam. Cert. 2:46–48. Claim 30 recites “wherein the applying
`the selected new user rule to the plurality of attributes stored in memory
`comprises applying the selected new user rule to only the plurality of
`attributes stored in memory.” Id. at 4:23–24.
`
`8
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`(April 16, 2014 Amendment and Reply), 3–6, 9. In its October 2013
`amendment, Patent Owner stated that the added claim language “require[s]
`application of the user rule ‘only’ to the detected attributes.” Ex. 1016, 78.
`Patent Owner further stated that in contrast, the queries of the prior art
`references “are not applied to attributes . . . alone but are applied to object-
`oriented abstractions.” Id. at 78–79. The Examiner agreed with Patent
`Owner and found the “conceptual queries utilized in the video query system
`of Day-I are not applied to ‘only’ the detected attributes, but require the use
`of object-oriented abstractions to interface with the VSDG model.”
`Ex. 1017, 38; see also id. (stating as shown in the ’923 patent, “an event
`discriminator (‘user rule’) can be applied only against video primitives as
`opposed to abstractions thereof”).
`Petitioner argues that the added claim language cannot be construed to
`require searching only attributes and exclude the searching of abstractions
`because there is no support for such a limitation in the specification of the
`’923 patent. See Pet. 20 (citing Ex. 1005 ¶ 133); id. at 7 (stating “the word
`‘only’ is used for excluding reprocessing source video as opposed to
`excluding something other than video primitives, such as abstractions”
`(citing Ex. 1033 ¶ 30, 148; Ex. 1018, 7; Ex. 1005 ¶ 69)). Thus, Petitioner
`contends any “argument that the claims include a negative limitation
`precluding the searching of abstractions . . . must fail because [there is] no
`written description support for that limitation in the ’923 patent.” Pet.
`Reply 9; see also id. at 9–10 (stating a “construction of the claims forbidding
`searching abstractions lacks written description support in the ’923 patent
`and would be an error”).
`We disagree with Petitioner and determine that the plain and ordinary
`meaning of the phrase “applying the selected new user rule to only the
`
`9
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`plurality of attributes” requires that the new user rule be applied to only the
`plurality of attributes. Even if the added claim language was not adequately
`supported,7 the language of the claim clearly states that the new user rule is
`applied to only the plurality of attributes, and as such, we construe the claim
`in accordance with its plain meaning. Petitioner has not persuasively shown
`that claim construction should deviate from the clear meaning of the claim.
`The prosecution history also supports a construction in accordance with the
`plain and ordinary meaning of the claim language. During the
`reexamination proceeding, Patent Owner expressly stated that the added
`language “require[s] application of the user rule ‘only’ to the detected
`attributes” in “contrast [to] the requirements of Day-I [which] are not
`applied to the attributes stored in the VSDG alone but are applied to object-
`oriented abstractions.” Ex. 1016, 78; see also id. at 79 (stating the “queries
`of Day-II are also not applied to the attributes stored in the VSDG alone but
`are applied to object-oriented abstractions”). Thus, the prosecution history
`supports a construction as requiring that the new user rule be applied to only
`the plurality of attributes (i.e., the new user rule is not applied to object-
`oriented abstractions).
`The parties remaining disputes regarding the scope of the “only”
`limitations need not be resolved as they are not necessary to resolution of the
`issues before us. We need only construe claim terms that are in controversy,
`and only to the extent necessary to resolve the controversy. Nidec Motor
`Corp. v. Zhongshan Broad Ocean Motor Co., 868 F.3d 1013, 1017 (Fed.
`Cir. 2017) (applying Vivid Techs., Inc. v. Am. Sci. & Eng’g, Inc., 200 F.3d
`
`
`7 We make no determination as to whether the added claim language is
`adequately supported.
`
`10
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`795, 803 (Fed. Cir. 1999) in the context of an inter partes review). To the
`extent we find it necessary to further address the meaning of any other claim
`terms, we do so below in the context of our unpatentability analysis.
`D. Asserted Obviousness of Claims 1–41 over Dimitrova and Brill
`Petitioner contends that claims 1–41 are unpatentable under 35 U.S.C.
`§ 103(a) as obvious over the combined teachings of Dimitrova and Brill.
`Pet. 34–73; Pet. Reply 11–25. Patent Owner opposes. PO Resp. 16–51;
`PO Sur-reply 10–24. For the reasons explained below, Petitioner has not
`established by a preponderance of the evidence that claims 1–41 of the
`’923 patent would have been obvious over Dimitrova and Brill.
`1. Overview of Dimitrova
`Dimitrova is an article titled “Motion Recovery for Video Content
`Classification” that is directed to content retrieval based on video data for
`video on demand, automated surveillance systems, and other similar
`systems. Ex. 1006, 3–4. Dimitrova discloses that the distinction between
`still images and moving pictures is typically based in movements and
`variations, and, accordingly, the analysis of the motion of objects allows for
`the extraction of information that is unique to a video sequence. Id.
`Dimitrova stores the video information using object attributes (“O”),
`object motion attributes (“M”), and video attributes (“V”), or an “OMV”
`triplet. See id. at 19–20. The OMV triplet is the basis for the query function
`that allows for retrieval of video data based on its content. Id. at 3, 20. For
`example the query expression
`“V_Seq(O_category = pet, (Activity = walking, Trajectory = t1))” translates
`into “retrieve all the video sequences in which a pet walks and makes a
`trajectory t1.” Id. at 20. Dimitrova states that the answer to that query will
`include all the “objects (animals) that are classified as pets: cats, dogs, fish
`
`11
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`called Wanda.” Id. Another example is the function “Object_motion : O x
`V → P (M)” which takes any object description and a particular video
`sequence and returns a set of motion descriptions related to that object. Id.
`Dimitrova states that these functions “allow for inexactness, and by default,
`they return results that are approximately similar to the precise answer.” Id.
`at 21. To make the retrieval exact, the symbol “!” is placed in front of the
`query function. Id. (stating that the query “!Object_motion” will return only
`the motion descriptions of objects that match the give characteristics).
`Dimitrova also explains that these retrieval functions are imbedded
`into the framework of a multimedia functional query language called
`“EVA,” which is “object oriented and supports objects, object classes,
`attributes and methods of objects, and relationships between objects.” Id. at
`21–22. EVA deals with the temporal and spatial aspects of multimedia
`information retrieval and delivery, in addition to the usual capabilities
`provided by the ordinary database languages. Id. at 21.
`2. Overview of Brill (Ex. 1004)
`Brill, an article titled “Event Recognition and Reliability
`Improvements for the Autonomous Video Surveillance System,” is directed
`to an Autonomous Video Surveillance (“AVS”) system that “processes live
`video streams from surveillance cameras to automatically produce a real-
`time map-based display of the locations of people, objects and events in a
`monitored region.” Ex. 1004, 4. Specifically, the AVS system of Brill has
`been enhanced to utilize collateral information sources, camera hand-off,
`vehicle recognition, and complex event recognition. Id.
`3. Petitioner’s Contentions
`Petitioner contends that the combined teachings of Dimitrova and
`Brill render claims 1–41 unpatentable as obvious. See Pet. 33–73. With
`
`12
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`respect to claim 1, Petitioner contends Dimitrova discloses the limitations of
`claim element 1[a], which require detecting an object in a video from a
`single camera, as Dimitrova detects objects and their motion in a video
`obtained from a single camera. See id. at 33 (citing Ex. 1006, 8–19, Figs. 4–
`6; Ex. 1005 ¶¶ 345–347; Ex. 1006, 4–5). Petitioner also asserts Dimitrova
`discloses the “detecting a plurality of attributes” limitations of claim element
`1[b] as Dimitrova’s system uses “motion analysis” techniques to detect basic
`activity attributes of an object, such as strolling, walking, or hurrying, and
`observable physical attributes, such as size, shape, color, or velocity. Id.
`at 33–36 (citing Ex. 1006, 17, 25; Ex. 1005 ¶¶ 348–349, 351, 359).
`Petitioner contends Brill also discloses detecting physical and temporal
`attributes because Brill discloses events that use multiple physical attributes
`(e.g., “person” and “near the door”), an activity attribute (“loiter”), and a
`temporal attribute (“more than 5 seconds”) Id. at 36–37 (citing Ex. 1005
`¶ 360).
`Petitioner further contends Dimitrova discloses the “selecting a new
`user rule” limitations of claim element 1[c] because Dimitrova discloses “a
`tasking mechanism that allows users to create ad hoc queries based on a
`plurality attributes that are already detected.” Id. at 37–38 (citing Ex. 1005
`¶¶ 362–363; Ex. 1006, 20, 28–29). In particular, Petitioner points to
`Dimitrova’s “Parade Query” as specifying a combination of a set of already
`detected activity, physical, and temporal attributes for identifying a Parade
`Event. Id. at 38 (citing Ex. 1005 ¶¶ 364–365). Petitioner also contends that
`the proper construction of a “new user rule” does not require setting a
`response, but that if a response is required, both Dimitrova and Brill teach
`this feature. Id. at 38 (citing Ex. 1004, Fig. 11; Ex. 1006, 11, 27; Ex. 1005
`¶ 366).
`
`13
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`Claim element 1[d1] recites, inter alia, “identifying an event of the
`object that is not one of the detected attributes of the object by applying the
`new user rule to the plurality of detected attributes.” Petitioner asserts that
`“applying” encompasses “any mechanism for analyzing the detected
`attributes to determine if they satisfy the new user rule criteria, e.g.,
`querying a database.” Pet. 12 (citing Ex. 1005 ¶¶ 100–103); see also Pet.
`Reply 7 (stating that a “‘query’ must examine the attributes to find a match
`for the collection of attributes specified by the user rule” (emphasis omitted)
`(citing Ex. 2019, 42)). Petitioner asserts Dimitrova’s “Parade Query”
`satisfies the limitations of claim element 1[d1] because the query applies at
`least two different attributes (e.g., “waving,” “walking,” “tall,” president,”
`“person” and four frame numbers), from the plurality of detected attributes.
`Pet. 39 (citing Ex. 1006, 25; Ex. 1005 ¶ 368). Petitioner further contends the
`Parade Query includes a temporal attribute requiring that the two objects
`meet the specified criteria at the same time. Id. at 40 (citing Ex. 1005
`¶ 370). Petitioner also contends Dimitrova satisfies the limitations of claim
`element 1[d1] that require applying the new user rule after detecting
`attributes and creating the query because the Parade Event is created after-
`the-fact and applied to existing recorded attributes. Id. at 39–40 (citing
`Ex. 1006, 33, 34, 24–25, 29; Ex. 1005 ¶¶ 369, 371).
`Petitioner asserts the limitations of claim element 1[d2] that recite
`“applying the new user rule to the plurality of detected attributes comprises
`applying the new user rule to only the plurality of detected attributes”
`require that “the prior art have the ability to search only the attributes
`themselves.” Id. at 40 (citing Pet. 19–21). Petitioner argues Dimitrova
`discloses this limitation by providing a query operator called “exact” (using
`symbol “!”). Id. (citing Ex. 1004, 21; Ex. 1005 ¶¶ 372–373). Petitioner
`
`14
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`contends this operator allows the system to retrieve only objects that have
`the same detected attributes as the user specified in the query. Id. at 40–41
`(citing Ex. 1005 ¶ 373). Petitioner asserts this “operator prevents the system
`from returning higher-level abstractions based on the queried attributes,
`which would merely represent approximations of the searched attributes.”
`Id. at 41 (citing Ex. 1005 ¶ 373). Petitioner also asserts that although
`“Dimitrova discloses schemas that define logical relationships between
`objects and attributes, those schemas do not define the storage structure of
`the data in the Dimitrova database.” Id. at 41 (citing Ex. 1006, 22–23; Ex.
`1005 ¶ 374). Petitioner contends that Dimitrova merely teaches that the
`attributes are stored as “OMV triplets,” without requiring any further
`hierarchy and that Dimitrova “can apply its user rules to only the attributes,
`e.g., the OMV triplet data, and therefore “does not require searching that
`involves higher-level abstractions,” such as the abstractions of the Day
`references that Patent Owner was trying to distinguish with the “only”
`limitation. Ex. 1005 ¶ 374 (citing Ex. 1006, 19).
`Petitioner also contends the attributes queried in Dimitrova’s Parade
`Query are “independent of which event is identified” as required by claim
`element 1[e] because attributes are pre-collected and stored in a database
`before the Parade Query is defined by the user. See Pet. 41–43 (citing
`Ex. 1006, 20, 25, 29; Ex. 1005 ¶¶ 375–380). In addition, Petitioner contends
`that because Dimitrova searches stored attributes in its database, Dimitrova
`performs the “identifying” step “without reprocessing the video,” as required
`by claim element 1[f]. See id. at 43 (citing Ex. 1003, 24–27; Ex. 1005
`¶ 381). Finally, Petitioner asserts Dimitrova’s Parade Query satisfies the
`limitations of claim element 1[g] (“wherein the event of the object refers to
`the object engaged in an activity”) because the Parade Query identifies a
`
`15
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`“parade” event by searching for activity attributes (i.e., “waving” and
`“talking”) that refer to an object engaged in an activity. See id. at 44 (citing
`Pet. 41–43; Ex. 1005 ¶ 382).
`
`4. Analysis
`In our Institution Decision, we determined Petitioner made an
`adequate showing for institution purposes that Dimitrova and Brill teach or
`suggest each limitation of claim 1. See Inst. Dec. 24–31. Nevertheless, on
`the complete record now before us, as discussed in detail below, we find that
`a preponderance of the evidence does not establish that the combined
`teachings of Dimitrova and Brill teach or suggest “wherein the applying the
`new user rule to the plurality of detected attributes comprises applying the
`new user rule to only the detected attributes” as recited in independent
`claim 1, element [d2], as well as in independent claims 8, 9, 22, and 29 and
`similarly recited in independent claims 20 and 30 (hereinafter the “only”
`limitations). Accordingly, we determine that Petitioner has not shown by a
`preponderance of the evidence that independent claims 1, 8, 9, 20, 22, 29
`and 30, or the claims that depend therefrom, would have been obvious over
`Dimitrova and Brill.
`As described above, Petitioner contends the “Parade Query” is a “new
`user rule.” See Pet. 38–39 (stating that the creation of ad hoc queries using a
`set of attributes, such as the “Parade Query” discloses selecting a “new user
`rule”). According to Petitioner, the Parade Query, when used with a “query
`operator called ‘exact’ (using symbol ‘!’),” applies the new user rule to only
`the plurality of detected attributes because the “exact” operator “allows the
`system to retrieve only objects that have the same detected attributes as the
`user specified in the query.” See id. at 40–41 (citing Ex. 1004, 21; Ex. 1005
`
`16
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`¶¶ 372–373).8 Petitioner asserts that the “exact” operator prevents the
`system from returning higher-level abstractions based on queried attributes,
`which would merely represent approximations of the searched attributes. Id.
`at 41 (citing Ex. 1005 ¶ 373).
`Patent Owner responds that Dimitrova does not teach “applying the
`new user rule to only the plurality of detected attributes” because Dimitrova,
`even when using the “exact” operator, “queries object-oriented abstractions”
`(abstraction data) and thus does not query only attributes. See PO Resp. 18–
`22. To support its argument that Dimitrova queries abstraction data in
`addition to attributes (and thus does not apply the new user rule to “only”
`attributes), Patent Owner points to Dimitrova’s disclosure that “its retrieval
`functions ‘need to relate the motion at a higher level of abstraction of the
`object to the detailed motion of parts of objects.’” Id. at 19 (quoting
`Ex. 1006, 33) (citing Ex. 1006, 21, 27; Ex. 2018, 130:13–24; Ex. 1006, 27).
`Patent Owner also contends that Dimitrova explicitly states that its query
`operates on abstractions as the “V_seq” operator “operates on “any
`description that can be provided at any level of the spatial hierarchy.” Id.
`(quoting Ex. 1006, 20); see also Ex. 1006, 21–22 (stating that the language
`used in the system is “object oriented and supports objects, classes, attributes
`and methods of objects, and relationships between objects” (emphasis
`added)). Patent Owner further argues that Dimitrova states that it uses “a
`semantic multiresolution hierarchy for spatiotemporal representation
`
`
`8 Petitioner also argues that the “only” limitations cannot be construed to
`require searching only attributes and “exclude searching abstractions”
`because the ’923 patent provides no written description support for that
`limitation. See, e.g., Pet. 19–20; Pet. Reply 13–14. We disagree with this
`argument for the reasons stated above in Section II.C.
`
`17
`
`

`

`IPR2019-00314
`Patent 7,932,923 B2 & C1
`(Figure 7) because it helps video analysis at various resolution levels” and
`that even Petitioner’s expert agrees that “there is a spatial hierarchy in
`Dimitrova that is involved in the query process.” PO Resp. 20 (quoting Ex.
`1006, 17; Ex. 2018, 145:17–146:9) (citing Ex. 1006, Fig. 7; Ex. 2018,
`146:10–15; Ex. 1022, 1, 2, 5, 7). In addition, Patent Owner contends
`Dimitrova’s OMV triplet “is an abstraction of the video data” because it
`represents a way to store the analyzed video in an indexed way with
`metadata that associates different O, M, and V values as a group and is more
`than jus