`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`Robert Bosch LLC
`Petitioner
`
`v.
`
`Monument Peak Ventures, LLC,
`Patent Owner
`
`
`
`Case IPR2019-01473
`Patent No. 6,654,507
`_________________________
`
`
`PETITION FOR INTER PARTES REVIEW
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`I.
`II.
`
`III.
`
`
`
`TABLE OF CONTENTS
`
`Introduction .................................................................................................... 1
`Statement of Precise Relief Requested for Each Claim Challenged ......... 3
`A.
`Claims for Which Review Is Requested ............................................... 3
`B.
`Statutory Grounds.................................................................................. 3
`’507 Patent Overview .................................................................................... 4
`A.
`’507 Patent Disclosure and Claims ....................................................... 5
`B.
`’507 Patent Prosecution History ............................................................ 9
`IV. Level of Ordinary Skill in the Art .............................................................. 10
`V. Claim Construction ..................................................................................... 11
`A.
`Belief Map ........................................................................................... 11
`B. Main Subject ........................................................................................ 12
`VI. Claims 1, 8, and 14 Are Unpatentable Over the Prior Art ...................... 13
`A. Ground 1: Toyama, Itti, and Neubauer Render claims 1, 8, and
`14 Obvious. ......................................................................................... 13
`1.
`Summary of Prior Art ............................................................... 14
`a.
`Toyama ........................................................................... 14
`b.
`Itti .................................................................................... 17
`Neubauer ......................................................................... 20
`c.
`Claim 1 ...................................................................................... 22
`Claim 1 [preamble]: “A method of producing an image of
`at least a portion of a digital image, comprising:” ......... 22
`
`2.
`
`
`
`ii
`
`

`

`3.
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`Claim 1[a]: “a) providing a digital image having pixels;” ....... 23
`Claim 1[b]: “b) computing a belief map of the digital
`image by using the pixels of the digital image to
`determine a series of features and using such
`features to assign a probability of a location of a
`main subject of the digital image in the belief
`map;” ............................................................................... 24
`Claim 1[c]: “c) determining a crop window having a
`shape factor and a zoom factor, the shape and the
`zoom factors determining a size of the crop
`window; and” .................................................................. 32
`Claim 1[d]: “d) cropping the digital image to include a
`portion of the image of high subject content in
`response to the belief map and the crop window.” ......... 34
`Claim 8 ...................................................................................... 36
`“The method of claim 1 wherein the crop window is
`completely within the digital image.” ............................ 36
`Claim 14 .................................................................................... 36
`“A computer storage product having at least one
`computer storage medium having instructions
`stored therein causing one or more computers to
`perform the method of claim 1.” .................................... 36
`VII. Grounds for Standing .................................................................................. 37
`VIII. Mandatory Notices Under 37 C.F.R. § 42.8 ............................................... 37
`A.
`Real Party-in-Interest .......................................................................... 37
`B.
`Related Matters .................................................................................... 37
`C.
`Lead and Back-Up Counsel ................................................................. 38
`D.
`Service Information ............................................................................. 38
`IX. Certification Under 37 C.F.R. § 42.24(d) .................................................. 38
`iii
`
`4.
`
`
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`Payment of Fees ........................................................................................... 38
`X.
`XI. Conclusion .................................................................................................... 38
`
`
`
`
`
`
`
`
`iv
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`LIST OF EXHIBITS
`
`Description
`Exhibit
`Ex. 1001 U.S. Patent No. 6,654,507 to Jiebo Luo et al. (the ’507 patent”).
`Ex. 1002 Patent Prosecution History of U.S. Patent No. 6,654,507.
`Ex. 1003 Curriculum Vitae of John Grindon, Ph.D.
`Ex. 1004 Declaration of John Grindon, Ph.D.
`Ex. 1005 U.S. Patent No. 6,792,135 to Toyama. (“Toyama”).
`Ex. 1006
`Itti, Laurent et al., “A Model of Saliency-Based Visual Attention for
`Rapid Scene Analysis,” IEEE Transactions on Pattern Analysis and
`Machine Intelligence, Vol. 20, No. 11, November 1998 (“Itti”).
`Ex. 1007 U.S. Patent No. 6,553,131 to Neubauer et al. (“Neubauer”).
`Ex. 1008
`IEEE Xplore page for IEEE Transactions on Pattern Analysis and
`Machine Intelligence, available at
`https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34 (last
`visited, Aug. 23, 2019).
`IEEE Editorial Style Manual, available at
`https://www.ieee.org/content/dam/ieee-
`org/ieee/web/org/conferences/style_references_manual.pdf (last
`visited, Aug. 23, 2019).
`IEEE Xplore page for A Model of Saliency-Based Visual Attention
`for Rapid Scene Analysis, available at
`https://ieeexplore.ieee.org/document/730558 (last visited, Aug. 23,
`2019).
`
`Ex. 1009
`
`Ex. 1010
`
`
`
`
`
`v
`
`
`
`
`
`
`
`

`

`Introduction
`Petitioner, Robert Bosch LLC, requests inter partes review of claims 1, 8,
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`I.
`
`and 14 of U.S. Patent No. 6,654,507 (“the ’507 patent”), currently assigned to
`
`Monument Peak Ventures LLC (“MPV”).
`
`The challenged claims are directed to a system and method for automatically
`
`creating cropped and zoomed versions of digital images. ’507 patent, Abstract. As
`
`the ’507 patent admits, conventional systems already cropped images based on, for
`
`example, “relatively homogenous margins around the borders of an image” or
`
`“different intensity levels within the image.” Id., 2:22-23, 32-33. The ’507 patent
`
`claims that the major drawback of these techniques is that “both techniques cannot
`
`deal with images with nonuniform backgrounds.” Id., 2:50-51. The patent asserts
`
`that this resulted in manual cropping still being preferred over those automated
`
`methods.
`
`The ’507 patent asserts that the advantage of the disclosed invention is the
`
`ability to automatically crop and zoom images based on a so-named “belief” map,
`
`which helps identify areas of interest in an image based upon their saliency in the
`
`map. ’507, 3:53-60. But belief, or saliency, maps were not novel as of the filing
`
`date of the ’507 patent. In fact, such maps (and algorithms for developing such
`
`maps) were already known and taught by others, including algorithms based on
`
`research attempting to model the way humans (and other primates) detect objects
`
`
`
`1
`
`

`

`of interest in an image. Itti, p. 1254. Simply automating human behavior by
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`applying a known technique for modeling such behavior is not inventive. See
`
`Princeton Biochemicals, Inc. v. Beckman Coulter, Inc., No. CIV.A. 96-5541
`
`(MLC), 2004 WL 1398227, at *18 (D.N.J. June 17, 2004), aff'd, 411 F.3d 1332
`
`(Fed. Cir. 2005) (“It has been well-established that the obvious automation of a
`
`known manual process is not patentable.”) (citing In re Vener & Bowser,
`
`262 F.2d 91(C.C.P.A. 1958), and In re Rundell, 48 F.2d 958 (C.C.P.A. 1931)).
`
`And the technique claimed by the ’507 patent of locating an object of interest in an
`
`image and cropping around it was similarly conventional. See Itti, p. 1256
`
`(discussing the focus of attention as “a simple disk”). Indeed, the challenged
`
`claims recite nothing more than automating manual processes by combining well-
`
`known machine vision principles to determine main subjects in a photograph (that
`
`model the first step of the manual process) and well-known techniques to crop and
`
`zoom images (that model the second step of the manual process). Such automation
`
`of a manual process is not patentable. See Princeton Biochemical , 2004 WL
`
`1398227, at *18. Therefore, with further reference to the following information
`
`discussed below, the Board should institute inter partes review of the ’507 patent
`
`and cancel claims 1, 8, and 14 based on the grounds presented.
`
`
`
`2
`
`

`

`II.
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`Statement of Precise Relief Requested for Each Claim Challenged
`A. Claims for Which Review Is Requested
`Petitioner respectfully requests review under 35 U.S.C. § 311 of claims 1, 8,
`
`and 14 of the ’507 patent and cancellation of those claims as unpatentable.
`
`Statutory Grounds
`B.
`Each asserted reference identified in the table below, issued, published,
`
`and/or was filed before December 14, 2000, the earliest purported priority date of
`
`challenged claims. Thus, each asserted reference is prior art under 35 U.S.C.
`
`§§ 102(a), (b), and/or (e).
`
`Prior Art References
`
`Ref. 1: Toyoma, U.S. Patent No. 6,792,135, filed on October 29, 1999 (Ex.
`
`1005).
`
`Ref. 2:
`
`Itti, “A Model of Saliency-Based Visual Attention for Rapid Scene
`
`Analysis,” IEEE Transactions on Pattern Analysis and Machine
`
`Intelligence, Vol. 20, No. 11, Published November 1998 (Ex. 1006).
`
`Ref. 3 Neubauer, U.S. Patent No. 6,553,131, filed on September 15, 1999 (Ex.
`
`1007).
`
`
`Claims 1, 8, and 14 are unpatentable under the following ground based on 35
`
`U.S.C. § 103:
`
`
`
`3
`
`

`

`Ground
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`Grounds of Unpatentability
`
`Ground 1 Toyama, Itti, and Neubauer render claims 1, 8, and 14 obvious.
`
`
`
`III.
`
`’507 Patent Overview
`The ’507 patent issued from U.S. Patent App. No. 09/736,825, filed
`
`December 14, 2000. The ’507 patent is not a continuation, continuation-in-part, or
`
`divisional application and does not otherwise claim priority to another patent
`
`application, although it does incorporate by reference the disclosure of other patent
`
`applications by the same inventor, U.S. Patent App. Nos. 09/490,915 and
`
`09/223,860. Accordingly, the earliest purported effective filing date of the
`
`challenged claims is December 14, 2000.
`
`The ’507 patent provides “[a] method of producing an image of at least a
`
`portion of a digital image.” ’507 patent, Abstract. Before the ’507 patent, there
`
`were two main types of prior art: one examines images in a line-by-line fashion
`
`and the other, examines images block-by-block to identify uniform areas. The first
`
`type automatically cropped images by (1) examining image borders; (2)
`
`determining, by using grayscale images, whether a row of pixels is to be cropped
`
`based on pixel variation; (3) determining if a row of pixels passes the criterion to
`
`be included in the final cropped image; (4) examining the next row inward until all
`
`rows are complete; and (5) determining the final cropped image. Id., 2:8-21. In
`
`
`
`4
`
`

`

`other words, these prior art programs removed homogenous margins around the
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`image border and did not examine image contents. Id., 2:22-24. The second type
`
`cropped images based on different intensity levels within an image. Id., 2:30-40.
`
`The ’507 patent purports to improve prior art systems by using a belief map
`
`of the image to identify main subjects. Id., 3:28-32. But as Bosch’s expert, Dr.
`
`John Grindon, explains, belief mapping has been used in various computer vision
`
`applications, and using it to crop an image would have been obvious to one of
`
`ordinary skill. Ex. 1004, ¶ 22.
`
`’507 Patent Disclosure and Claims
`
`A.
`
`The ’507 patent claims methods of and systems for cropping a digital image
`
`comprising three main steps: (1) developing a belief map indicative of main
`
`subjects in the image; (2) determining a crop window; and (3) cropping the image
`
`to include a portion of the image with high subject content. See generally ’507
`
`Patent, Claims 1-28. Claim 1 is illustrative of the challenged claims:
`
`1. A method of producing an image of at least a portion of a digital
`image, comprising:
`
`a) providing a digital image having pixels;
`
`b) computing a belief map of the digital image by using the pixels
`of the digital image to determine a series of features and using such
`features to assign a probability of a location of a main subject of the
`digital image in the belief map;
`
`
`
`5
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`c) determining a crop window having a shape factor and a zoom
`factor, the shape and the zoom factors determining a size of the crop
`window; and
`
`d) cropping the digital image to include a portion of the image of
`high subject content in response to the belief map and the crop window.
`
`Id., 11:57-12:3.
`
`The specification of the ’507 patent describes in significantly more detail a
`
`process for automatically cropping an image. For example, in explaining how a
`
`belief map is created, the ’507 patent describes a main subject detection (“MSD”)
`
`system that provides a “discriminative” “measure of saliency or relative
`
`importance for different regions” in an image for a number of applications,
`
`“including automatic crop and zoom.” Id., 4:48-51, 4:53. The MSD system
`
`performs various sub-tasks, including: “region segmentation, perceptual grouping,
`
`feature extraction, and probabilistic and semantic reasoning.” Id., 4:65-66. Features
`
`are extracted for each segmented image region to represent a variety of visual
`
`saliency properties which are input into a probability network to create a non-
`
`binary belief map. Id., 4:67-5:4.
`
`Through MSD, regions belonging to a main subject are differentiated from
`
`the image background. Id., 5:5-7. The MSD system outputs a “list of segmented
`
`regions ranked in descending order of their likelihood (or belief) as potential main
`
`subjects for a generic or specific application” and can also be converted into a
`
`
`
`6
`
`

`

`belief map, which can be represented for visual purposes as an image wherein the
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`region’s brightness is proportional to the belief or probability that the region is a
`
`main subject. Id., 5:17-20, 5:25-27 (“The associated likelihood is also attached to
`
`each region so that regions with large values correspond to regions with high
`
`confidence or belief…”). These “brighter” regions, i.e. those having larger belief
`
`values, represent regions with high confidence of being part of the main subject.
`
`Id., 5:26-28.
`
`FIG. 3 further explains how it uses the belief map results for additional
`
`image processing. In step 200, the image is input, and a belief map is created using
`
`MSD. Id., 7:1. A zoom factor, based on either an operator’s selection or
`
`determined automatically on a main subject size estimate, and a cropped window
`
`are selected in step 201. Id., 7:2-7.
`
`
`
`7
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`
`
`In step 201, belief map regions are clustered using a k-means clustering
`
`process and the lowest belief cluster, or the background, is zeroed using a
`
`predefined threshold. Id., 7:16-18. Next, the centroid of non-zero beliefs are
`
`computed in step 202, and a crop window is centered at the centroid in step 204.
`
`
`
`8
`
`

`

`Id., 7:23-25, 8:5-6. The centroid is calculated by summing the values of the beliefs
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`at each pixel location. Id., 7:29-38.
`
`
`
`In step 205, the crop window is moved so that the entire crop window is
`
`within the original image. Id., 8:7-8. The cropped window is then moved again in
`
`item 206 so that all regions of the highest belief subject are included in the cropped
`
`window, thereby capturing the entire subject of interest. Id., 8:9-12. Step 207
`
`determines whether an “acceptable solution” has been found, meaning whether it is
`
`possible to include “at least the regions of the highest belief values” in the
`
`cropping window. Id., 8:16-19. If an acceptable solution exists, the window is
`
`moved again in item 208 to maximize the belief sums within the window. Id., 8:20-
`
`22. If an acceptable solution is not produced, the final position of the crop window
`
`is restored to that of step 205. Id., 9:1-3.
`
`’507 Patent Prosecution History
`B.
`During prosecution of the application that issued as the ’507 patent, the
`
`Examiner rejected challenged claims 1, 8, and 14 under 35 U.S.C. § 102(e) as
`
`anticipated by Jia (U.S. Patent No. 6,430,320). Ex. 1002, p. 77-80. The examiner
`
`also rejected claims 1, 8, and 14 for obviousness-type double patenting. Id., p. 81.
`
`The applicant also argued that Jia did not disclose “belief maps” and agreed
`
`to file a terminal disclaimer upon an indication of allowability. Id., 275-79.
`
`
`
`9
`
`

`

`The Examiner allowed claims 1-28 because “none of the prior art, teach or
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`fairly suggest computing a belief map of the digital image, by using the pixels of
`
`the digital image to determine a series of features, and using such features to assign
`
`a probability of a location of a main subject of the digital image in the belief map.”
`
`Id., 291. The examiner noted that Jia (U.S. Patent 6,430,320) discloses “a method
`
`of automatically cropping an image according to a belief of where in the image a
`
`main subject is located . . . [but] does not each or fairly suggest the use of a belief
`
`map in which a probability of the location of the main subject is assign[ed].” Id.,
`
`291-92.
`
`The Examiner, however, never considered the references applied in the
`
`petition— Toyama, Itti or Neubauer—nor were these references cited by the
`
`applicant. See Ex. 1002; ’507 patent, cover page. The proposed ground teaches a
`
`belief map with corresponding belief values that the Examiner found lacking in the
`
`prior art considered and which led to the allowance of the claims.
`
`IV. Level of Ordinary Skill in the Art
`A person of ordinary skill in the art for the ’507 patent would have a
`
`bachelor’s degree in electrical engineering, computer science, physics, or a related
`
`field, and two to four years of experience (or the academic equivalent) in the field
`
`of computer/machine vision or image processing. Less work experience may be
`
`compensated by a higher level of education, and vice versa. Id. Ex. 1004, ¶ 17.
`
`
`
`10
`
`

`

`V. Claim Construction
`
`A claim in a patent subject to inter partes review is subject to the claim
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`construction standard of Phillips v. AWH Corp., 415 F.3d 1303 (Fed. Cir. 2005)
`
`(en banc). See 83 Fed. Reg. 51,340 (October 11, 2018). Under the Phillips
`
`standard, claims are given their ordinary and customary meaning to a person of
`
`ordinary skill in the art at the time of the invention. Phillips, 415 F.3d at 1313. The
`
`following terms requires construction for this proceeding. Claim terms not
`
`addressed below should be given their plain and ordinary meaning.
`
`A. Belief Map
`In the ’507 patent, a “belief map” as used in claim 1 means a probability-
`
`related map indicating importance of a region by providing a measure of relative
`
`importance for different regions in an image. Ex. 1004, ¶ 34. As Dr. Grindon
`
`explains, synonyms for “belief” maps are “likelihood,” “probability,” or “saliency”
`
`maps. Id.
`
`The ’507 patent states that the “‘belief’ map is more than a binary map that
`
`only indicates location of the determined main subject;” instead, the “associated
`
`likelihood is also attached to each region so that regions with large values
`
`correspond to regions with high confidence or belief of being part of the main
`
`subject.” ’507 patent, 5:23-28 (emphasis added); see also id., 5:17-22 (explaining
`
`that the main subject detection (MSD) output used “is a list of segmented regions
`
`
`
`11
`
`

`

`ranked in descending order of their likelihood (or belief) as potential main subjects
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`for a generic or specific application. This list can be readily converted into a map
`
`in which the brightness of a region is proportional to the main subject belief of the
`
`region.”).
`
`Each pixel in the belief map corresponds to a pixel in the original image and
`
`is assigned a likelihood value of the region where it is in the original image. See
`
`id., 7:36-38. The region(s) with the highest likelihood value are designated as the
`
`main subject. Thresholds may be applied to select other belief values—those that
`
`are lower than the highest but higher than the lowest belief values—and designate
`
`them as secondary main subjects. Id., 5:51-54 (explaining that these secondary
`
`main subjects can be “included according to a descending order of belief values
`
`once the main subject of highest belief values are included.”); see also id., 8:43-46,
`
`9:59-67. Accordingly, Petitioner requests that “belief map” be construed as “a map
`
`indicating importance of a region by providing a measure of relative importance
`
`for different regions in an image.” Ex. 1004, ¶ 36.
`
`B. Main Subject
`A “main subject,” as used in claim 1 is a “photographic subject having a
`
`highest belief value in a belief map.” Ex. 1004, ¶ 37; see ’507 patent, 4:48-50
`
`(“Main subject detection provides a measure of saliency or relative importance for
`
`different regions that are associated with different subjects in an image.”) The ’507
`
`
`
`12
`
`

`

`patent states that the “output of [main subject detection] … is a list of segmented
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`regions ranked in descending order of their likelihood (or belief) as potential main
`
`subjects.” Id., 5:17-19. The ’507 patent explains that the belief map “reflects the
`
`inherent uncertainty” of determining a main subject because “different observers
`
`may disagree on certain subject matter while agreeing on other subject matter in
`
`terms of main subjects . . .” Id., 5:29-33.
`
`This list of regions can be converted to a map where the region brightness is
`
`proportional to the main subject belief of the region. Id., 5:20-22; see also id.,
`
`5:25-28 (explaining that the belief map attaches a likelihood “to each region so that
`
`regions with large values correspond to regions with high confidence or belief of
`
`being part of the main subject”) (emphases added). However, a “binary decision”
`
`can also be obtained by “using an appropriate threshold on the belief map.” Id.,
`
`5:34-36.
`
`VI. Claims 1, 8, and 14 Are Unpatentable Over the Prior Art
`A. Ground 1: Toyama, Itti, and Neubauer Render claims 1, 8, and 14
`Obvious.
`Toyama discloses a system and method for producing at least a portion of an
`
`image by determining a crop window and cropping the image to a portion of the
`
`image containing a main subject as claimed. Ex. 1004, ¶ 39. Toyama suggests
`
`computing a belief map to determine the location of the main subject, and Itti and
`
`Neubauer resolve any dispute that computing a belief map and assigning a
`13
`
`
`
`

`

`probability of a main subject’s location was known. Id. Itti discloses a belief map
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`through its saliency map that represents “the conspicuity—or ‘saliency’—at every
`
`location” in the image created from normalized conspicuity maps representing
`
`various features. Itti, p. 1255. Neubauer similarly discloses a probability-like
`
`saliency map where the “highest peak values in the intensity regions have the
`
`highest probability of being the license plate and are selected for further
`
`processing.” Neubauer, 4:29-31, 5:4-7.
`
`1.
`
`Summary of Prior Art
`Toyama
`a.
`As shown below in FIGS. 3 and 4, Toyama discloses a face detection system
`
`210 and method for detecting a face within an image. Toyama, 2:18-23. The
`
`system includes: (1) a hypothesis module that defines a sub-region in which to
`
`search for a face; (2) a feature extraction module for extracting image feature
`
`values based on a non-intensity image property; and (3) a relational template
`
`module that uses a relational template and facial regions to determine whether a
`
`face has been detected. Id., 2:30-42.
`
`
`
`14
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`
`
`First, the face detection system receives the raw image 200 and sends it to a
`
`hypothesis module 300. Id., 6:32-36. The hypothesis module generates a
`
`hypothesis and defines the dimensions of a sub-region in the raw image 200 (or
`
`cropped image) where a face may be found. Id. The cropped image is preprocessed
`
`by preprocessing module 320, id., 6:36-38, before being sent to a feature extraction
`
`module, id., 6:38-40.
`
`
`
`15
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`
`
`Based on a relational analysis of various features extracted from the cropped
`
`image, the system 210 determines whether a face has been detected in a cropped
`
`image (box 360). Id., 6:53-55. If a face is not found in the sub-region, and a
`
`different sub-region is produced (box 370) by returning to the hypothesis module
`
`300 and another hypothesis is generated about where a face may be located within
`
`the image 200. Id., 6:55-60. If a face is detected in the cropped image, the face
`
`information, including the location and dimensions of the cropped image, is sent as
`
`an output (380). Id., 6:62-67.
`
`
`
`16
`
`

`

`Itti
`b.
`“A Model of Saliency-Based Visual Attention for Rapid Scene Analysis”
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`(“Itti”) is an article published by the IEEE in November 1998. Ex. 1006.
`
`According to the IEEE Xplore database, the article appears in volume 20, issue 11
`
`of the “IEEE Transactions on Pattern Analysis and Machine Intelligence magazine,
`
`published in November 1998, at pages 1254-59. Ex. 1006; Ex. 1011. To that end,
`
`Itti bears an IEEE copyright line on the first page: 0162-8828/98/$10.00 © 1998
`
`IEEE” that represents publication date, a price, and an ISSN code. Ex. 1006; Ex.
`
`1010. According to the IEEE, the Transactions on Pattern Analysis and Machine
`
`Intelligence publication aims to “present most important research results in areas”
`
`within pattern analysis and machine intelligence and is published monthly. Ex.
`
`1009. This evidence establishes public accessibility of Itti before the earliest
`
`priority date of the ’507 patent in December 2000. As the Board has noted in
`
`previous proceedings, “IEEE is a well-known, reputable compiler and publisher of
`
`scientific and technical publications,” and the Board can “take Official Notice that
`
`members in the scientific and technical communities who both publish and engage
`
`in research rely on the information published on the copyright line of IEEE
`
`publications.” Ericsson, Inc. v. Intellectual Ventures I LLC, IPR2014-00527, Paper
`
`41 at 11 (May 18, 2015).
`
`
`
`17
`
`

`

`Itti relates to a visual attention system where “[m]ultiscale image features
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`are combined into a single topographical saliency map.” Itti, p. 1254. The saliency
`
`map represents “the conspicuity—or ‘saliency’—at every location” in the image.
`
`Id., p. 1255 (explaining that “the maximum of the saliency map (SM) defines the
`
`most salient image location, to which the focus of attention (FOA) should be
`
`directed.”). Itti discloses various feature maps which are combined into three
`
`“conspicuity maps” for intensity, color, and orientation. Id. The three conspicuity
`
`maps are normalized and summed into the final saliency map input, where the
`
`maximum value defines the most salient image location. Id.
`
`Itti then uses a winner-take-all (WTA) neural network which marks the most
`
`salient region. Itti, p. 1256. All WTA neurons evolve independently until one
`
`
`
`
`
`18
`
`

`

`“winner” reaches a threshold, which triggers three simultaneous mechanisms:
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`(1) the focus of attention is shifted to the winner neuron location; (2) all WTA
`
`neurons are completely inhibited, or reset; and (3) local inhibition is activated in
`
`the saliency map in the area and size of the focus of attention to allow the next
`
`most salient location to subsequently become the winner. Id. The results over 260
`
`ms of simulated time are shown below.
`
`
`
`
`
`19
`
`

`

`Neubauer
`c.
`Neubauer relates to a license plate recognition system using an intelligent
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`camera. To first begin to find a region where a license plate may be located,
`
`Neubauer discloses that “[c]oarse localization hones in on the license plate image
`
`within the image captured” by performing “sub-sampling … to reduce the number
`
`of pixels therein.” In block 16 of the system, “vertical edges are computed (since
`
`vertical edges dominate in license plates”). Neubauer, 4:19-25, 4:65-5:2
`
`(explanation that “coarse localization relies on the fact that a license plate (text)
`
`includes a high amount of vertical edges compared to the remaining parts of a car
`
`and its surrounding in an image”). But Neubauer also explains that “[h]orizontal
`
`edges or other local features may be employed for coarse localization as well.” Id.,
`
`5:14-15.
`
`
`
`20
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`
`
`Afterwards, in block 18, the system generates a saliency map that “blends
`
`vertical edges to achieve intensity regions in the image.” Id., 4:27-29. Neubauer
`
`explains that “highest peak values in the intensity regions have the highest
`
`probability of being the license plate and are selected for further processing.” Id.,
`
`4:29-31, 5:4-7 (explaining that “[p]ixels with high intensity in the saliency map
`
`correspond to positions that are likely to include the license plate. The highest
`
`pea[k] value in the saliency map corresponds to the license plate”). If other peaks
`
`occur, “a candidate list of possible locations is ordered by the intensity of the
`
`
`
`21
`
`

`

`vertical edge feature which is computed and evaluated until a reasonable result is
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`found.” Id., 5:7-10. In block 20, the system “hones in on the image of the license
`
`plate to provide the course localization.” Id., 4:31-33.
`
`In block 24, after coarse localization, the system performs fine localization
`
`by determining and correcting the license plate orientation and determining the
`
`height and position of the license plate within the image “to more accurately
`
`retrieve the region of interest. Id., 5:18-23, 25-28 (explaining that “[a]fter the
`
`position, orientation and size of the license plate are determined, the region of
`
`interest is resampled to a predetermined vertical resolution (e.g., 20 pixels)).
`
`2.
`
`Claim 1
`Claim 1 [preamble]: “A method of producing an
`image of at least a portion of a digital image,
`comprising:”
`Toyama discloses a method of producing an image of at least a portion of a
`
`digital image, (see Toyama, 3:52-57, Fig. 3; Ex. 1004), as shown, for example, in
`
`Fig. 4:
`
`
`
`22
`
`

`

`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`
`
`In Toyama’s method, a hypothesis module generates a hypothesis and
`
`defines the dimensions of a sub-region, or cropped image, in a received raw image
`
`where a face may be found. Id., 6:32-36.
`
`Claim 1[a]: “a) providing a digital image having
`pixels;”
`Toyama discloses providing a digital image having pixels because it
`
`discloses that a digital raw image that is cropped is input into the face detection
`
`system. Toyama, 7:2-10, 10:37-39 (explaining that the working example described
`
`was “performed using an image from a live color video camera that was captured
`
`using an image digitizer”) (emphasis added). Toyama further discloses an image
`23
`
`
`
`

`

`with pixels when describing a preferred embodiment where “the image property
`
`Case IPR2019-01473
`U.S. Patent No. 6,654,507
`
`
`used is edge density, although other suitable properties (such as pixel color) may
`
`also be used. Id., 2:35-37 (emph