`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`_______________________
`
`ROBERT BOSCH LLC
`Petitioner
`
`v.
`
`MONUMENT PEAK VENTURES, LLC
`Patent Owner
`_______________________
`
`Case IPR2019-01473
`Patent 6,654,507
`_______________________
`
`PATENT OWNER’S RESPONSE
`
`
`
`
`i
`
`
`
`
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TABLE OF CONTENTS
`
`INTRODUCTION ........................................................................................... 1
`
`Regions of the Image are Assigned Values Indicating Likelihoods that
`Each Respective Region in the Image Corresponds to a Main Subject of the
`Image ................................................................................................................. 6
`
`in a Scene ......................................................................................................... 10
`
`Displayed in More (or Less) Detail from a Normal Appearance .................... 11
`
`I.
`II. THE ’507 Patent ........................................................................................... 2
`A. Claim Constructions .................................................................................... 6
`1. A Belief Map is a Representation of an Image in Which Respective
`2. A Main Subject of an Image is What a Photographer Tries to Capture
`3. A Zoom Factor is a Factor or Ratio by Which an Image is Presented or
`4. Summary ................................................................................................... 11
`III. ANALYSIS .................................................................................................. 11
`A. Petitioner’s Declarant Merely Repeats the Arguments in the Petition
`B. Patentability over Toyama in View of Itti and Neubauer ...................... 30
`1. Toyama (Exhibit 1005) ............................................................................. 30
`2.
`3. Neubauer (Exhibit 1007) .......................................................................... 33
`4. Petitioner has Failed to Prove that Claims 1, 8, and 14 of the ’507 Patent
`IV. CONCLUSION. .......................................................................................... 45
`
`Without Further Analysis or Insight ............................................................... 12
`
`Itti (Exhibit 1006) ..................................................................................... 32
`
`Are Unpatentable in View of Toyama, Itti, and Neubauer .............................. 33
`
`
`
`ii
`
`

`

`CASES
`
`
`
`TABLE OF AUTHORITIES
`
`Alza Corp. v. Mylan Labs., Inc.,
`464 F.3d 1286 (Fed. Cir. 2006) ........................................................................... 43
`
`
`CFMT, Inc. v. Yieldup Intern. Corp.,
`349 F.3d 1333 (Fed. Cir. 2003) ..................................................................... 35, 38
`
`
`Hartness Int’l. Inc. v. Simplimatic Engineering Co.,
`819 F.2d 1100 (Fed. Cir. 1987) ..................................................................... 38, 44
`
`
`In re Magnum Oil Tools International, Ltd.,
`829 F.3d 1364 (Fed Cir. 2016) ...................................................................... 30, 39
`
`
`KSR Int’l v. Teleflex Inc.,
`550 U.S. 398 (2007) .............................................................................................. 8
`
`
`Markman v. Westview Instruments, Inc.,
`52 F.3d 967 (Fed. Cir. 1995) ................................................................................. 9
`
`
`Phillips v. AWH Corp.,
`415 F.3d 1303 (Fed. Cir. 2005) ............................................................................. 7
`
`
`TRW Automotive US LLC v. Magna Elecs., Inc.,
`IPR2014-00258 (PTAB Aug. 27, 2014) ............................................................. 30
`
`
`Vitronics Corp. v. Conceptronics, Inc.,
`90 F.3d 1576 (Fed. Cir. 1996) ............................................................................... 9
`
`REGULATIONS
`
`
`
`37 C.F.R. § 42.65(a) ......................................................................................... 30, 38
`
`37 C.F.R. § 42.100(b) ........................................................................................... 6, 7
`
`37 C.F.R. § 42.104(b)(4) ........................................................................................ 38
`
`37 C.F.R. § 42.108(c) ............................................................................................. 38
`
`
`
`
`
`iii
`
`

`

`EXHIBIT LIST
`
`Description
`
`U.S. Patent 6,282,317 to Luo et al.
`
`Transcript of deposition of John R. Grindon, D. Sc.
`
`
`
`Exhibit No.
`
`2001
`
`2002
`
`
`
`
`
`
`
`
`
`iv
`
`

`

`I. INTRODUCTION
`The challenged claims of U.S. Patent 6,654,507 (the “’507 patent”) should
`
`not be found unpatentable because the Petitioner has failed to prove by a
`
`preponderance of evidence that the challenged claims are unpatentable.
`
`Petitioner relies solely on Toyama for teaching claim 1’s requirement of
`
`determining a crop window having a shape factor and a zoom factor where the
`
`shape and zoom factors determine the size of the crop window. However,
`
`Petitioner’s argument that this limitation is met because, “Toyama discloses a
`
`zoom factor to determine a size of the crop window because it discloses that sub-
`
`regions are ‘defined for a limited range of scales’ and that each cropped image is
`
`resized into a canonical image size,” misses the mark because the “scales” referred
`
`to by Toyama are not zoom factors, but rather represent how large a section of the
`
`image is to be evaluated. Consequently, Petitioner has failed to prove its
`
`obviousness case with respect to claim 1 and its dependent claims.
`
`Further, Petitioner’s contention that Toyama alone or the combination of
`
`Toyama and Itti teach or suggest computing a belief map, as required by claim 1, is
`
`wrong. Toyama only describes a hypothesis that may indicate areas of an image
`
`that should be examined for a particular object (a face), but there is no suggestion
`
`that the face is the “main subject” of the image, or that any likelihood values are
`
`assigned to any of the sub-regions thought to contain faces. Moreover, and
`
`
`
`1
`
`

`

`Neubauer’s teachings notwithstanding, a person of ordinary skill in the art would
`
`not find it obvious to combine the teachings of Toyama and Itti inasmuch as Itti
`
`itself recognizes that its model “cannot detect conjunctions of features” as would
`
`be found in faces.
`
`These and further reasons why the challenged claims are not unpatentable in
`
`view of the cited references are discussed in detail below.
`
`
`
`II. THE ’507 PATENT
`The ’507 patent relates to methods and systems that “automatically zoom[]
`
`and crop[] digital images according to an analysis of the main subject in the
`
`scene.” Ex. 1001 at 4:40-42. Cropping is, generally, a process employed to remove
`
`content, often unwanted content, from a photographic image, for example to
`
`improve the framing of a subject in the image, to change the aspect ratio of the
`
`image, or to otherwise accentuate a subject in the image. Ex. 2002 at 13:6 – 15:8;
`
`18:18 – 19:3. Zooming is a process by which an image is altered, e.g., in terms of
`
`its presentation on a display, to allow a view of more or less detail in the image. Id.
`
`at 17:24 – 19:3. It is similar to magnification in that zooming enlarges or reduces
`
`the apparent size of an item being viewed. Ex. 1001 at 1:35-37. The apparent
`
`enlargement or reduction in size from the original item is referred to as a zoom
`
`factor. See id. at 5:11-15.
`
`
`
`2
`
`

`

`“One advantage of the invention lies in the ability to automatically crop and
`
`zoom photographic images based upon the scene contents.” Id. at 3:52-54. In
`
`particular, the main subject of the image is identified and the cropping and
`
`zooming is performed based on that identified main subject. This provides high-
`
`quality zoomed/cropped images automatically. Id. at 3:54-61. As explained in U.S.
`
`Application No. 09/223,860, now U.S. Patent 6,282,317 (Ex. 2001) (the “’317
`
`patent”) and incorporated by reference in the ’507 patent, id. at 4:45-47, a “main
`
`subject” is what a photographer tries to capture in a scene. Ex. 2001 at 1:12-13.
`
`Main subject detection (“MSD”) identifies those regions in an image that are
`
`most likely to contain the main subject of the image and “provides a measure of
`
`saliency or relative importance for different regions of the image.” Ex. 1001 at
`
`4:48-50. More specifically, “[t]he output of MSD used by the invention is a list of
`
`segmented regions ranked in descending order of their likelihood (or belief) as
`
`potential main subjects for a generic or specific application.” Id. at 5:17-20. The
`
`output list of segmented regions is “converted into a map in which the brightness
`
`of a region is proportional to the main subject belief of the region. Therefore, this
`
`map can be called a main subject ‘belief’ map.” Id. at 5:20-23.
`
`The belief map of the ’507 patent indicates the location and the associated
`
`likelihood of the main photographic subject in an image. In such a belief map, a
`
`belief value at a given pixel location is proportional to the likelihood that the
`
`
`
`3
`
`

`

`location is part of the main subject. Id. at 5:23-27. The belief map also informs a
`
`cropping decision, id. at 5:39-40, in which placement of a cropping window is
`
`based on a likelihood that the main subject of the image is included within the
`
`window. Id. at 5:42-60. The amount of crop and the allowable zoom factors of a
`
`resulting image are restricted, based on the findings of consumer studies and other
`
`factors. Id. at 5:61 – 6:15.
`
`The cropping process is further described with reference to Figure 3,
`
`reproduced below, as applied to a digital
`
`image shown in Figure 5. Id. at 6:63-67.
`
`At the outset, step 200, an image is input
`
`and a belief map is created using MSD.
`
`Id. at 7:1. Fig. 5 shows the original
`
`image. Id. at 6:67.
`
`
`
`
`
`In step 201-203, a zoom factor and a crop window are automatically
`
`determined. Id. at 7:21-4. The zoom factor is determined based on an estimate of
`
`the size of the main subject from the belief map. Id. at 7:4-7. As shown in Figure
`
`
`
`4
`
`

`

`6, the crop window 80 is positioned such that it is the smallest rectangle of the
`
`original aspect ratio of the image that encompasses the
`
`effective bounding rectangle 70 defining the region of
`
`high subject content of the image specified by the
`
`belief values of the belief map. Id. at 7:16 – 8:4.
`
`Initially, as shown in Figure 7, the crop window is
`
`positioned around the centroid or center-of-mass,
`
`of nonzero beliefs (step 204). It is then moved, as
`
`shown in Figure 8, so that it is entirely within the
`
`original image (step 205). And finally, it is moved
`
`once more, as shown in Figure 9, so that all the
`
`regions of the highest belief value (the “main
`
`subject”) are included within the crop window (step
`
`206). Id. at 8:5-15.
`
`At step 207, the process “determines
`
`whether an acceptable solution has been found,
`
`i.e., whether it is possible to include at least the
`
`regions of the highest belief values in the cropping window.” Id. at 8:16-19. If an
`
`acceptable solution does exist, then at step 208 the window is again moved “to
`
`optimize a subject content index for the crop window.” Id. at 8:20-22. In the
`
`
`
`5
`
`

`

`current example, this moves the crop window, as shown in Figure 10, to include a
`
`portion of the flowers (so-called secondary objects) to increase the sum of the
`
`belief values within the crop window. Id. at 8:22-
`
`39. Once done, then at step 209 the image is
`
`cropped according to the calculated crop window
`
`position. Id. at Fig. 3.
`
`Referring back to Figure 3, if no acceptable solution was found at step 207,
`
`the final position of the crop window is restored to that shown in Figure 8, and the
`
`original image is cropped accordingly. Id. at 9:1-3; Fig. 3.
`
`
`
`A. Claim Constructions
`1. A Belief Map is a Representation of an Image in Which Respective Regions
`of the Image are Assigned Values Indicating Likelihoods that Each
`Respective Region in the Image Corresponds to a Main Subject of the
`Image
`Of the challenged claims, claim 1 is the sole independent claim, and it
`
`recites a “belief map.” In inter partes reviews, claims are construed “using the
`
`same claim construction standard that would be used to construe the claim in a
`
`civil action under 35 U.S.C. § 282(b).” 37 C.F.R. § 42.100(b). Accordingly, the
`
`term “belief map” must be construed “in accordance with [its] ordinary and
`
`customary meaning[s] . . . as understood by one of ordinary skill in the art and the
`
`
`
`6
`
`

`

`prosecution history pertaining to the patent.” Id.; Phillips v. AWH Corp., 415 F.3d
`
`1303, 1312–13 (Fed. Cir. 2005).
`
`In determining the level of ordinary skill in the art, various factors may be
`
`considered, including the “type of problems encountered in the art; prior art
`
`solutions to those problems; rapidity with which innovations are made;
`
`sophistication of the technology; and educational level of active workers in the
`
`field.” In re GPAC, Inc., 57 F.3d 1573, 1579 (Fed. Cir. 1995) (quotation marks
`
`omitted). Here, Petitioner asserts that a person of ordinary skill in the art would
`
`have “a bachelor’s degree in electrical engineering, computer science, physics, or a
`
`related field, and two to four years of experience (or the academic equivalent) in
`
`the field of computer/machine vision or image processing. Less work experience
`
`may be compensated by a higher level of education, and vice versa.” Pet. 10 (citing
`
`Ex. 1004 ¶ 17).
`
`The ’507 patent relates to methods and systems “for automatically
`
`crop[ping] and zoom[ing] digital images according to an analysis of the main
`
`subject in the scene.” Ex. 1001 at 4:40-42. During prosecution of the application
`
`that led to the ’507 patent, the Examiner cited Jia, U.S. Patent 6,430,320, as
`
`teaching, inter alia, computation of a belief map of a digital image, determining a
`
`crop window having a shape factor and a zoom factor, and cropping the digital
`
`image in response to the belief map and the crop window, Ex. 1002, part 1, pp. 77-
`
`
`
`7
`
`

`

`80, but ultimately agreed that Jia did not teach a belief map in which a probability
`
`of the location of the main subject in the image is assigned. Ex. 1002, part 2, pp.
`
`28-29. From this prosecution history, it is apparent that a person of ordinary skill in
`
`the art would need to be someone practiced in solutions for automatically cropping
`
`an image. In re GPAC, Inc., 57 F.3d at 1579; and see Ex. 2002 at 46:12 – 51:9.
`
`That person of ordinary skill would also possess the level of creativity typical of
`
`persons in that field. KSR Int’l v. Teleflex Inc., 550 U.S. 398, 418 (2007).
`
`According to the Specification:
`
`The output of MSD used by the invention is a list of segmented
`regions ranked in descending order of their likelihood (or
`belief) as potential main subjects for a generic or specific
`application. This list can be readily converted into a map in
`which the brightness of a region is proportional to the main
`subject belief of the region. Therefore, this map can be called a
`main subject “belief” map. This “belief” map is more than a
`binary map that only indicates location of the determined main
`subject. The associated likelihood is also attached to each
`region so that regions with large values correspond to regions
`with high confidence or belief of being part of the main subject.
`
`Ex. 1001 at 5:17-28 (emphases added). And further,
`
`The crop window is 80 then moved so that the entire crop
`window is within the original image (e.g. item 205) as shown in
`FIG. 8. In item 206, the crop window 80 is moved again so that
`
`
`
`8
`
`

`

`all the regions of the highest belief values (“main subject”) are
`included within the crop window and to create a margin 81, as
`shown in FIG. 9.
`
`Id. at 8:7-12
`
`Because the claims do not stand alone, Markman v. Westview Instruments,
`
`Inc., 52 F.3d 967, 978 (Fed. Cir. 1995), and must be read both in terms of the
`
`specification, id. at 979, understanding the claims requires consideration of this
`
`description. Indeed, Petitioner’s declarant, Dr. Grindon agreed that “belief map” is
`
`not a term of art with a single, known meaning, and that the one of ordinary skill in
`
`the art would look to the specification to determine its meaning. Ex. 2002 at 63:4 –
`
`64:4.
`
`In the above passages from the specification, the inventors emphasized that
`
`the belief map is more than just an indication of a determined location of a main
`
`subject of an image. The belief map associates likelihoods of regions being the
`
`main subject to each region in the image. Accordingly, a belief map, as recited in
`
`the claims, should be regarded as a representation of an image in which respective
`
`regions of the image are assigned values indicating likelihoods that each
`
`respective region in the image corresponds to a main subject of the image. See
`
`Vitronics Corp. v. Conceptronics, Inc., 90 F.3d 1576, 1582 (Fed. Cir. 1996) (the
`
`specification "is always highly relevant to the claim construction analysis. Usually,
`
`it is dispositive; it is the single best guide to the meaning of a disputed term.").
`
`
`
`9
`
`

`

`
`
`
`
`2. A Main Subject of an Image is What a Photographer Tries to Capture in a
`Scene
`The ’507 patent indicates that regions of the belief map having the highest
`
`belief values are the “main subject” of an image. Ex. 1001 at 8:7-12. However,
`
`construing the main subject in this fashion leads to a somewhat circular definition
`
`of the belief map, which becomes unhelpful: a belief map would be regarded as a
`
`representation of an image in which respective regions of the image are assigned
`
`values indicating likelihoods that each respective region in the image corresponds
`
`to a region of the belief map having the highest belief value. To better elucidate the
`
`meaning of these terms a person of ordinary skill in the art would look to the ’317
`
`patent, which is incorporated by reference in the ’507 patent and from which the
`
`’507 patent’s description of main subject detection is taken. Ex.1001 at 4:45-47,
`
`The ’317 patent defines the main subject of the image as what a photographer tries
`
`to capture in a scene. Ex. 2001 at 1:12-13. It is apparent from the description of
`
`main subject detection in the ’507 patent, that it uses the terms main subject in the
`
`same way as the term is used in the ’317 patent. Ex. 1001 at 4:48 – 5:60.
`
`
`
`
`
`10
`
`

`

`3. A Zoom Factor is a Factor or Ratio by Which an Image is Presented or
`Displayed in More (or Less) Detail from a Normal Appearance
`A zoom factor is a factor or ratio by which an image is presented or displayed
`
`in more (or less) detail from a normal appearance. See, e.g., Ex. 2002 at 18:8-16
`
`(Petitioner’s declarant testifying that zooming involves changing the resolution of
`
`an image, i.e., its pixel density, by interpolation or decimation).
`
`
`
`4. Summary
`For the reasons discussed above, the Board should construe the term belief
`
`map as a representation of an image in which respective regions of the image are
`
`assigned values indicating likelihoods that each respective region in the image
`
`corresponds to a main subject of the image, the term main subject as what a
`
`photographer tries to capture in a scene, and the term zoom factor as a factor or
`
`ratio by which an image is presented or displayed in more (or less) detail from a
`
`normal appearance.
`
`
`
`III. ANALYSIS
`Petitioner alleges that claims 1, 8, and 14 ae are unpatentable as being
`
`obvious in view of Toyama, U.S. Patent 6,792,135 (Ex. 1005) when considered in
`
`combination with Itti et al., “A Model of Saliency-Based Visual Attention for
`
`Rapid Scene Analysis,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND
`
`
`
`11
`
`

`

`MACHINE INTELLIGENCE, vol. 20, no. 11 (Ex. 1006), and Neubauer, U.S. Patent
`
`6,533,131 (Ex. 1007). Pet. at 3-4; 22 et seq.
`
`
`
`A. Petitioner’s Declarant Merely Repeats the Arguments in the Petition
`Without Further Analysis or Insight
`Before examining the failings in the petition with respect to the challenges in
`
`view of the cited references, it is important to discuss the petition’s reliance on Dr.
`
`Grindon’s declaration, Ex. 1004. Beginning at p. 22, the petition sets forth
`
`arguments for unpatentability of the challenged claims. Pet. at 22 et seq.
`
`Throughout these arguments, the petition cites paragraphs of Dr. Grindon’s
`
`declaration for support of the propositions being made.1 However, when one
`
`examines Dr. Grindon’s testimony, it is apparent that it is nothing but a repetition,
`
`for the most part verbatim, of the attorney argument made in the petition. For
`
`example, consider the arguments for the alleged unpatentability of claim 1:
`
`Petition p. 22
`Toyama discloses a method of
`producing an image of at least a
`portion of a digital image, (see
`Toyama, 3:52-57, Fig. 3; Ex. 1004),
`
`Ex. 1004 ¶ 49
`Toyama discloses a method of
`producing an image of at least a
`portion of a digital image, (see
`Toyama, 3:52-57, Fig. 3; Ex. 1004),
`
`
`1 The petition cites Dr. Grindon’s declaration, but makes no attempt to indicate it is
`
`quoting that declaration. Thus, the petition is attorney argument and nothing more.
`
`
`
`12
`
`

`

`as shown, for example, in Fig. 4:
`
`
`as shown, for example, in Fig. 4:
`
`
`Petition p. 23
`In Toyama’s method, a hypothesis
`module generates a hypothesis and
`defines the dimensions of a sub-
`region, or cropped image, in a
`received raw image where a face
`may be found. Id., 6:32-36.
`
`
`Petition pp. 23-24
`Toyama discloses providing a digital
`image having pixels because it
`discloses that a digital raw image
`that is cropped is input into the face
`detection system. Toyama, 7:2-10,
`10:37-39 (explaining that the
`working example described was
`“performed using an image from a
`live color video camera that was
`captured using an image digitizer”)
`(emphasis added). Toyama further
`discloses an image with pixels when
`describing a preferred embodiment
`where “the image property used is
`edge density, although other suitable
`
`Ex. 1004 ¶ 50
`In Toyama’s method, a hypothesis
`module generates a hypothesis and
`defines the dimensions of a sub-
`region, or cropped image, in a
`received raw image where a face
`may be found. Id., 6:32-36.
`
`
`Ex. 1004 ¶ 51
`Toyama discloses providing a digital
`image having pixels because it
`discloses that a digital raw image
`that is cropped is input into the face
`detection system. Toyama, 7:2-10,
`10:37-39 (explaining that the
`working example described was
`“performed using an image from a
`live color video camera that was
`captured using an image digitizer”)
`(emphasis added). Toyama further
`discloses an image with pixels when
`describing a preferred embodiment
`where “the image property used is
`edge density, although other suitable
`
`
`
`13
`
`

`

`properties (such as pixel color) may
`also be used. Id., 2:35-37 (emphasis
`added). Moreover, Toyama
`frequently describes image pixels
`when discussing the resizing module,
`equalization module, and feature
`extraction module. Id., 7:62-64
`(explaining that “the resizing module
`resizes the cropped image … using
`such methods as, for example, …
`pixel interpolation”), 8:9-15, 8:21-
`37, 8: 44-48.
`
`
`Petition p. 24
`Element 1[b] includes two sub-
`elements: (i) “computing a belief
`map of the digital image by using the
`pixels of the digital image to
`determine a series of features;” and
`(ii) “using such features to assign a
`probability of a location of a main
`subject of the digital image in the
`belief map.” Toyama suggests but
`does not expressly disclose
`computing a belief map as claimed,
`and Itti and Neubauer resolves any
`
`properties (such as pixel color) may
`also be used. Id., 2:35-37 (emphasis
`added). Moreover, Toyama
`frequently describes image pixels
`when discussing the resizing module,
`equalization module, and feature
`extraction module. Id., 7:62-64
`(explaining that “the resizing module
`resizes the cropped image … using
`such methods as, for example, …
`pixel interpolation”), 8:9-15, 8:21-
`37, 8: 44-48.
`
`
`Ex. 1004 ¶ 52
`Element 1[b] includes two sub-
`elements: (i) “computing a belief
`map of the digital image by using the
`pixels of the digital image to
`determine a series of features;” and
`(ii) “using such features to assign a
`probability of a location of a main
`subject of the digital image in the
`belief map.” Toyama suggests but
`does not expressly disclose
`computing a belief map as claimed,
`and Itti and Neubauer resolves any
`
`
`
`14
`
`

`

`dispute that computing a belief map
`and assigning a probability of a main
`subject’s location was known. Ex.
`1004, ¶ 52 .
`
`
`dispute that computing a belief map
`and assigning a probability of a main
`subject’s location was known.
`
`
`
`Petition p. 25
`This step would have been obvious
`to a person of ordinary skill in the art
`because they would have looked to,
`for example, Itti to fill a gap where
`Toyama suggests a certain
`functionality but does not provide a
`detailed implementation of it. Ex.
`1004, ¶ 53. As further explained
`below, Toyama discloses generating
`a hypothesis of where a face may be
`located through “vision processing
`that target regions of the image most
`likely to contain a face.” Id., 7:18-20
`(emphasis added). As Dr. Grindon
`explains, it would have been obvious
`for the hypothesis module’s output of
`“regions of the image most likely to
`contain a face,” which is input into
`the cropping module 410, to be in the
`form of a belief map. Ex. 1004, ¶ 53.
`
`Ex. 1004 ¶ 53
`This step would have been obvious
`to a person of ordinary skill in the art
`because they would have looked to,
`for example, Itti to fill a gap where
`Toyama suggests a certain
`functionality but does not provide a
`detailed implementation of it. Ex.
`1004, ¶ 53. As further explained
`below, Toyama discloses generating
`a hypothesis of where a face may be
`located through “vision processing
`that target regions of the image most
`likely to contain a face.” Id., 7:18-20
`(emphasis added). It would have
`been obvious for the hypothesis
`module’s output of “regions of the
`image most likely to contain a face,”
`which is input into the cropping
`module 410, to be in the form of a
`belief map.
`
`
`
`15
`
`

`

`
`
`
`
`Petition p. 25
`As discussed above, a “belief map”
`as the term is used in the ’507 patent
`means a probability-related map
`indicating importance of a region by
`providing a measure of relative
`importance for different regions in an
`image. Supra, Section V.A. Toyama
`suggests but does not explicitly
`disclose computing a belief map
`because it discloses a cropping
`module that receives a hypothesis
`from a generation module that
`identifies the regions “most likely” to
`contain a face in the raw image. Id.,
`7:13-20. That is, the regions where
`the hypothesis engine believes the
`face is located. Ex. 1004, ¶ 54.
`
`
`Ex. 1004 ¶ 54
`As discussed above, a “belief map”
`as the term is used in the ’507 patent
`means a probability-related map
`indicating importance of a region by
`providing a measure of relative
`importance for different regions in an
`image. Supra, Section V.A. Toyama
`suggests but does not explicitly
`disclose computing a belief map
`because it discloses a cropping
`module that receives a hypothesis
`from a generation module that
`identifies the regions “most likely” to
`contain a face in the raw image. Id.,
`7:13-20. That is, the regions where
`the hypothesis engine believes the
`face is located.
`
`
`Petition p. 26
`Toyama similarly discloses that the
`generation module can use “vision
`processing” to target these regions,
`id., 7:18-20, and that the generation
`module generates multiple sub-
`
`Ex. 1004 ¶ 55
`Toyama similarly discloses that the
`generation module can use “vision
`processing” to target these regions,
`id., 7:18-20, and that the generation
`module generates multiple sub-
`
`
`
`16
`
`

`

`regions when multiple regions are
`most likely to contain a face. Id.,
`7:18-20, 7:35-38 (explaining that
`“once the dimensions and shape of
`the sub-region are defined, the entire
`image is searched by cycling each
`sub-region through the face detection
`system” (emphasis added)). Toyama
`further discloses that the hypothesis
`may be based “at least partially on
`the non-intensity image property.”
`Id., claim 14, 7:18-22 (explaining
`that the feature used may include
`skin color or ellipse-shaped blobs).
`
`
`
`Petition p. 27
`A person of skill would have
`understood Toyama to suggest
`providing the hypothesis via a belief
`map because so-called belief maps
`identify likely regions of interest
`using vision processing techniques—
`matching Toyama’s disclosure of
`outputting the hypothesis using
`“vision processing that target
`
`regions when multiple regions are
`most likely to contain a face. Id.,
`7:18-20, 7:35-38 (explaining that
`“once the dimensions and shape of
`the sub-region are defined, the
`entire image is searched by cycling
`each sub-region through the face
`detection system” (emphasis
`added)). Toyama further discloses
`that the hypothesis may be based “at
`least partially on the non-intensity
`image property.” Id., claim 14,
`7:18-22 (explaining that the feature
`used may include skin color or
`ellipse-shaped blobs).
`
`
`Ex. 1004 ¶ 56
`A person of skill would have
`understood Toyama to suggest
`providing the hypothesis via a belief
`map because so-called belief maps
`identify likely regions of interest
`using vision processing
`techniques—matching Toyama’s
`disclosure of outputting the
`hypothesis using “vision processing
`
`
`
`17
`
`

`

`regions of the image most likely to
`contain a face.” Id., 7:18-20
`(emphasis added); Ex. 1004, ¶ 56.
`And even the ’507 patent itself
`recognizes that such object detection
`and content determination based on
`the semantic meaning was
`“conventional wisdom in the field of
`computer vision.” ’507 patent, 4:54-
`59.
`
`
`Petition p. 27
`To the extent Toyama does not
`disclose computing a belief map by
`using the pixels to determine a series
`of features, Itti expressly does so. Itti
`discloses a belief map in the form of
`a “saliency map” that “defines the
`most salient image location to which
`the focus of attention (FOA) should
`be directed,” created by combining
`and normalizing feature maps. Itti, p.
`1255. The most salient locations are
`determined by normalizing and
`summing three conspicuity maps
`which combine a series of detected
`
`that target regions of the image most
`likely to contain a face.” Id., 7:18-20
`(emphasis added); And even the
`’507 patent itself recognizes that
`such object detection and content
`determination based on the semantic
`meaning was “conventional wisdom
`in the field of computer vision.”
`’507 patent, 4:54-59.
`
`
`
`Ex. 1004 ¶ 57
`To the extent Toyama does not
`disclose computing a belief map by
`using the pixels to determine a series
`of features, Itti expressly does so. Itti
`discloses a belief map in the form of
`a “saliency map” that “defines the
`most salient image location to which
`the focus of attention (FOA) should
`be directed,” created by combining
`and normalizing feature maps. Itti, p.
`1255. The most salient locations are
`determined by normalizing and
`summing three conspicuity maps
`which combine a series of detected
`
`
`
`18
`
`

`

`features “into the final input S to the
`saliency map:
`
`features “into the final input S to the
`saliency map:
`
`
`
`
`
`Id.
`
`
`Id.
`
`
`Petition p. 27
`Moreover, as Itti’s system is run, its
`focus of attention shifts from the
`most salient region to the next and to
`the next, as shown in the saliency
`maps represented on the right side of
`the figure below.
`
`
`Petition p. 28
`And Toyama discloses which
`features (skin color and ellipse-
`shaped blobs) should be used (or
`more heavily weighted) to
`specifically target faces. Toyama,
`7:18-23. Thus, combining Toyama
`and Itti would have been obvious
`because it was known to use saliency
`maps for image detection and
`Toyama teaches which features
`should be used. Ex. 1004, ¶ 59.
`
`Ex. 1004 ¶ 58
`Moreover, as Itti’s system is run, its
`focus of attention shifts from the
`most salient region to the next and to
`the next, as shown in the saliency
`maps represented on the right side of
`the figure below.
`
`
`Ex. 1004 ¶ 59
`And Toyama discloses which
`features (skin color and ellipse-
`shaped blobs) should be used (or
`more heavily weighted) to
`specifically target faces. Toyama,
`7:18-23. Thus, combining Toyama
`and Itti would have been obvious
`because it was known to use saliency
`maps for image detection and
`Toya