`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`SAMSUNG ELECTRONICS CO., LTD.; AND
`SAMSUNG ELECTRONICS AMERICA, INC.
`Petitioner
`
`v.
`
`IMAGE PROCESSING TECHNOLOGIES, LLC
`Patent Owner
`
`____________________
`
`Patent No. 8,805,001
`____________________
`
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,805,001
`
`
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`
`
`I.
`
`TABLE OF CONTENTS
`
`Contents
`INTRODUCTION ........................................................................................... 1
`
`II. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8 ................................... 1
`
`III.
`
`PAYMENT OF FEES UNDER 37 C.F.R. § 42.15(a) .................................... 3
`
`IV. GROUNDS FOR STANDING ........................................................................ 3
`
`V.
`
`PRECISE RELIEF REQUESTED .................................................................. 3
`
`VI. LEGAL STANDARDS ................................................................................... 3
`
`A.
`
`B.
`
`C.
`
`Claim Construction ............................................................................... 3
`
`Level Of Ordinary Skill In The Art ....................................................... 4
`
`This Petition Is Not Redundant ............................................................. 4
`
`VII. OVERVIEW OF THE RELEVANT TECHNOLOGY AND ’001
`PATENT .......................................................................................................... 5
`
`VIII. DETAILED EXPLANATION OF GROUNDS ............................................ 14
`
`A. Overview Of The Prior Art References .............................................. 14
`
`1.
`
`2.
`
`3.
`
`U.S. Patent No. 5,481,622 to Gerhardt (Ex. 1013) ................... 14
`
`U.S. Patent No. 6,044,166 to Bassman (Ex. 1014) ................... 21
`
`Alton L. Gilbert et al., A Real-Time Video Tracking
`System, PAMI-2 No. 1 IEEE Transactions on Pattern
`Analysis and Machine Intelligence 47 (Jan. 1980)
`(“Gilbert”) (Ex. 1005) ............................................................... 24
`
`4.
`
`U.S. Patent No. 5,150,432 (“Ueno”) (Ex. 1007) ..................... 33
`
`IX. Specific Explanation Of Grounds For Invalidity........................................... 38
`
`A. Ground 1: Gerhardt In View Of Bassman Renders Obvious
`Claims 5-13 ......................................................................................... 38
`
`i
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`Reasons To Combine Gerhardt And Bassman ......................... 38
`
`Elements Incorporated Into Claims 5-7 As Claims
`Dependent From An Independent Claim .................................. 41
`
`Claim 5: “The process according to claim 1, wherein said
`image processing system comprises at least one
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 47
`
`Claim 6: “The process according to claim 1, wherein said
`image processing system comprises at least two
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 52
`
`Claim 7: “The process according to claim 1, wherein
`forming the at least one histogram further comprises
`successively increasing the size of a selected area until
`the boundary of the target is found” ......................................... 53
`
`Claim 8 ...................................................................................... 53
`
`Claim 9: “The process according to claim 8, wherein
`identifying the target in said at least one histogram
`further comprises determining a center of the target to be
`between X and Y minima and maxima of the target” .............. 54
`
`Claim 10: “The process according to claim 8, wherein
`identifying the target in said at least one histogram
`further comprises determining the center of the target at
`regular intervals” ....................................................................... 54
`
`Claim 11: “The process according to claim 8 further
`comprising drawing a tracking box around the target” ............ 55
`
`1.
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`7.
`
`8.
`
`9.
`
`10. Claim 12: “The process according to claim 8, wherein
`said image processing system comprises at least one
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 56
`
`11. Claim 13: “The process according to claim 8, wherein
`said image processing system comprises at least two
`
`
`
`
`
`-ii-
`
`
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 56
`
`12. Gerhardt and Bassman Are Not Cumulative ............................ 56
`
`B.
`
`Ground 2: Gerhardt In View Of Gilbert And Ueno Renders
`Obvious Claims 5-13 .......................................................................... 57
`
`1.
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`7.
`
`8.
`
`9.
`
`Reasons To Combine Gilbert, Gerhardt, And Ueno ................. 57
`
`Elements Incorporated Into Claims 5-7 As Claims
`Dependent From An Independent Claim .................................. 60
`
`Claim 5: “The process according to claim 1, wherein said
`image processing system comprises at least one
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 66
`
`Claim 6: “The process according to claim 1, wherein said
`image processing system comprises at least two
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 69
`
`Claim 7: “The process according to claim 1, wherein
`forming the at least one histogram further comprises
`successively increasing the size of a selected area until
`the boundary of the target is found” ......................................... 70
`
`Claim 8 ...................................................................................... 70
`
`Claim 9: “The process according to claim 8, wherein
`identifying the target in said at least one histogram
`further comprises determining a center of the target to be
`between X and Y minima and maxima of the target” .............. 71
`
`Claim 10: “The process according to claim 8, wherein
`identifying the target in said at least one histogram
`further comprises determining the center of the target at
`regular intervals” ....................................................................... 72
`
`Claim 11: “The process according to claim 8 further
`comprising drawing a tracking box around the target” ............ 73
`
`
`
`
`
`-iii-
`
`
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`10. Claim 12: “The process according to claim 8, wherein
`said image processing system comprises at least one
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 77
`
`11. Claim 13: “The process according to claim 8, wherein
`said image processing system comprises at least two
`components selected from a memory, a temporal
`processing unit, and a spatial processing unit” ......................... 77
`
`12. Gilbert, Gerhardt, And Ueno Are Not Cumulative ................... 77
`
`X.
`
`CONCLUSION .............................................................................................. 80
`
`
`
`
`
`
`
`-iv-
`
`
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`LIST OF EXHIBITS1
`
`U.S. Patent No. 8,805,001 (“the ’001 Patent”)
`Declaration of Dr. John C. Hart
`Curriculum Vitae for Dr. John C. Hart
`Prosecution File History of U.S. Patent No. 8,805,001
`Alton L. Gilbert et al., A Real-Time Video Tracking System,
`PAMI-2 No. 1 IEEE Transactions on Pattern Analysis and
`Machine Intelligence 47 (Jan. 1980) (“Gilbert”)
`Reserved
`U.S. Patent 5,150,432 (“Ueno”)
`D. Trier, A. K. Jain and T. Taxt, “Feature Extraction Methods
`for Character Recognition-A Survey”, Pattern Recognition, vol.
`29, no. 4, 1996
`M. H. Glauberman, “Character recognition for business
`machines,” Electronics, vol. 29, pp. 132(136), Feb. 1956
`Declaration of Gerard P. Grenier (authenticating Ex. 1005)
`Reserved
`Reserved
`U.S. Patent No. 5,481,622 to Gerhardt
`U.S. Patent No. 6.044,166 to Bassman
`
`1001
`1002
`1003
`1004
`1005
`
`1006
`1007
`1008
`
`1009
`
`1010
`1011
`1012
`1013
`1014
`
`
`
` 1
`
` Citations to non-patent publications are to the original page numbers of the
`
`publication, and citations to U.S. patents are to column:line number of the patents.
`
`v
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`I.
`
`INTRODUCTION
`
`Samsung Electronics Co., Ltd. and Samsung Electronics America, Inc.
`
`(collectively, “Petitioner”) request inter partes review (“IPR”) of Claims 5-13 of
`
`U.S. Patent No. 8,805,001 (“the ’001 Patent”) (Ex. 1001), which, on its face, is
`
`assigned to Image Processing Technologies, LLC (“Patent Owner”). This Petition
`
`presents several non-cumulative grounds of invalidity that the PTO did not
`
`consider during prosecution. These grounds are each likely to prevail and this
`
`Petition, accordingly, should be instituted on all grounds and the challenged claims
`
`should be cancelled.
`
`
`
`II. MANDATORY NOTICES UNDER 37 C.F.R. § 42.8
`Real Parties-in-Interest: Petitioner identifies the following real parties-in-
`
`interest: Samsung Electronics Co., Ltd.; Samsung Electronics America, Inc.
`
`Related Matters: Patent Owner has asserted the ’001 Patent against
`
`Petitioner in Image Processing Technologies LLC v. Samsung Elecs. Co., No.
`
`2:16-cv-00505-JRG (E.D. Tex.). Patent Owner has also asserted U.S. Patent Nos.
`
`6,959,293; 7,650,015; 8,983,134; 8,989,445; and 6,717,518 in the related action.
`
`Petitioner is concurrently filing IPR petitions for all of these asserted patents.
`
`Petitioner has previously filed the following IPR petitions against the ’001 Patent
`
`and the first four patents listed above:
`
`1
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`IPR2017-00347 against the ’001 Patent, filed 11/29/2016.
`
`IPR2017-00336 against U.S. Patent No. 6,959,293, filed 11/29, 2016.
`
`IPR2017-00357 against U.S. Patent No. 8,989,445, filed 11/30/2016.
`
`IPR2017-00355 against U.S. Patent No. 7,650,015, filed 11/30, 2016.
`
`IPR2017-00353 against U.S. Patent No. 8,983,134, filed 11/30/2016.
`
`IPR2017-01190 against U.S. Patent No. 6,717,518, filed 3/29/2017.
`
`IPR2017-01212 against U.S. Patent No. 8,989,445, filed 3/30/2017.
`
`IPR2017-01189 against U.S. Patent No. 6,959,293, filed 3/30/2017.
`
`Lead and Back-Up Counsel:
`
`• Lead Counsel: John Kappos (Reg. No. 37,861), O'Melveny & Myers
`
`LLP, 610 Newport Center Drive, 17th Floor, Newport Beach,
`
`California 92660. (Telephone: 949-823-6900; Fax: 949-823-6994;
`
`Email: jkappos@omm.com.)
`
`• Backup Counsel: Nick Whilt (Reg. No. 72,081), Brian M. Cook (Reg.
`
`No. 59,356), O'Melveny & Myers LLP, 400 S. Hope Street, Los
`
`Angeles, CA 90071. (Telephone: 213-430-6000; Fax: 213-430-6407;
`
`Email: nwhilt@omm.com, bcook@omm.com.)
`
`Service Information: Samsung consents to electronic service by email to
`
`IPTSAMSUNGOMM@OMM.COM. Please address all postal and hand-delivery
`
`correspondence to lead counsel at O’Melveny & Myers LLP, 610 Newport Center
`
`2
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`Drive, 17th Floor, Newport Beach, California 92660, with courtesy copies to the
`
`email address identified above.
`
`III. PAYMENT OF FEES UNDER 37 C.F.R. § 42.15(a)
`The Office is authorized to charge an amount in the sum of $23,000 to
`
`Deposit Account No. 50-2862 for the fee set forth in 37 CFR § 42.15(a), and any
`
`additional fees that might be due in connection with this Petition.
`
`IV. GROUNDS FOR STANDING
`Petitioner certifies that the ’001 Patent is available for IPR and Petitioner is
`
`not barred or estopped from requesting IPR on the grounds identified herein.
`
`V.
`
`PRECISE RELIEF REQUESTED
`
`Petitioner respectfully requests review of Claims 5-13 of the ’001 Patent,
`
`and cancellation of these claims, based on the grounds listed below:
`
`• Ground 1: Claims 5-13 are obvious under 35 U.S.C. § 103(a) over
`
`Gerhardt in view of Bassman; and
`
`• Ground 2: Claims 5-13 are obvious under 35 U.S.C. § 103(a) over
`
`Gilbert in view of Ueno and further in view of Gerhardt.
`
`VI. LEGAL STANDARDS
`A. Claim Construction
`The ’001 Patent will expire on December 2, 2017—within 18 months of the
`
`Notice of Filing Date. Thus, for purposes of this proceeding, Petitioner has
`
`interpreted each claim term according to its plain and ordinary meaning under
`
`3
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`Phillips v. AWH Corp., 415 F.3d. 1303 (Fed. Cir. 2005).
`
`Level Of Ordinary Skill In The Art
`
`B.
`One of ordinary skill in the art the time of the alleged invention of the ’001
`
`Patent would have had either (1) a Master’s Degree in Electrical Engineering or
`
`Computer Science or the equivalent plus at least a year of experience in the field of
`
`image processing, image recognition, machine vision, or a related field or (2) a
`
`Bachelor’s Degree in Electrical Engineering or Computer Science or the equivalent
`
`plus at least three years of experience in the field of image processing, image
`
`recognition, machine vision, or a related field. Additional education could
`
`substitute for work experience and vice versa. Ex. 1002, ¶51.
`
`C. This Petition Is Not Redundant
`This Petition is not redundant to earlier filed IPR2017-00347 (the “’347
`
`Petition”) pertaining to the ’001 Patent. First, this Petition is necessitated because
`
`after Samsung filed the ’347 Petition, Patent Owner moved to add new claims to its
`
`infringement contentions that were originally served August 16, 2016, over three
`
`months earlier in the EDTX litigation. The motion for leave to amend was granted
`
`February 28, 2017. Thus, Samsung promptly prepared and filed this second
`
`Petition to address the newly-added claims. See, e.g., Microsoft Corp. v.
`
`Proxyconn, Inc., Case No. IPR2013-00109, slip op., 3 (P.T.A.B. Feb. 25, 2014)
`
`(Paper 15) (instituting IPR because additional claims asserted in concurrent district
`
`4
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`court litigation). Samsung has also included any remaining, unchallenged, claims
`
`in this Petition as a protective measure against Patent Owner continuing to assert
`
`new claims in the district court litigation. See Silicon Labs. Inc. v. Cresta Tech.
`
`Corp., Case No. IPR2015-00615, slip op. 24 (P.T.A.B. Aug. 14, 2015) (Paper 9)
`
`(instituting where petitioner filed to “challenge the remaining claims that the Patent
`
`Owner may likely assert in the district court case”).
`
`Second, this Petition raises new arguments not raised in the ’347 Petition.
`
`See id. For example, this Petition seeks institution only on claims that were not the
`
`subject of the ’347 Petition. See, e.g., Cepheid v. Roche Molecular Sys., Inc., Case
`
`No. IPR2015-00881 (P.T.A.B. Sept. 17, 2015) (Paper 9). This Petition does not
`
`seek institution on any claim that was the subject of the earlier ’347 Petition.
`
`Because these new claims have different scope, this Petition raises new arguments
`
`to address new limitations. Moreover, all grounds are new—this Petition relies on
`
`different prior art not included in the ’347 Petition to address the limitations of the
`
`newly-added claims. Facebook, Inc. v. TLI Commc’ns, LLC, Case No. IPR2015-
`
`00778, Paper 17, 26-27 (P.T.A.B. Aug. 28, 2015) (instituting where prior art and
`
`arguments were not substantially similar to previous petitions).
`
`VII. OVERVIEW OF THE RELEVANT TECHNOLOGY AND ’001
`PATENT
`
`The purported invention of the ’001 Patent relates to identifying and tracking
`
`a target in an input signal using one or more histograms derived from an image
`
`5
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`frame in the video signal. See e.g., Ex. 1001, Claims 5-13; Ex. 1002, ¶¶32-34.
`
`Video image processing and the use of histograms to identify and track targets, and
`
`to derive other information from a video signal were well known at the time the
`
`asserted patents were filed. Ex. 1002, ¶¶24-31, 49-50. An input signal used in the
`
`purported invention has “a succession of frames, each frame having a succession of
`
`pixels.” Ex. 1001, 3:28-31. The input signal may be a video signal or any other
`
`signal that “generates an output in the form of an array of information
`
`corresponding to information observed by the imaging device,” such as
`
`“ultrasound, IR, Radar, tactile array, etc.” Ex. 1001, 9:26-29; Ex. 1002, ¶34. The
`
`’001 Patent then constructs a histogram showing the frequency of pixels meeting a
`
`certain characteristic. The characteristics used to form histograms are referred to
`
`as “domains” in the ’001 Patent. Ex. 1002, ¶35. The ’001 Patent teaches that “the
`
`domains are preferably selected from the group consisting of i) luminance, ii)
`
`speed (V), iii) oriented direction (DI), iv) time constant (CO), v) hue, vi)
`
`saturation, and vii) first axis (x(m)), and viii) second axis (y(m)).” Ex. 1001, 4:2-6;
`
`Ex. 1002, ¶¶35-37. Figure 11 shows histogram processors that can create
`
`histograms in various domains:
`
`6
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`
`
`The histograms include a plurality of “classes” within a given domain. Ex.
`
`1002, ¶36. Figure 14a (and its accompanying description) illustrates an example of
`
`“classes” within a domain:
`
`FIG. 14a shows an example of the successive classes C1
`
`
`
`7
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`C2…Cn−1 Cn, each representing a particular velocity, for a
`
`hypothetical velocity histogram, with
`
`their being
`
`categorization for up to 16 velocities (15 are shown) in
`
`this example. Also shown is envelope 38, which is a
`
`smoothed representation of the histogram.
`
`Ex. 1001, 20:54-59.
`
`The ’001 Patent then uses the histograms to identify a target in the input
`
`signal. For example, one embodiment of the ’001 Patent performs “automatic
`
`framing of a person…during a video conference.” Id. at 22:10-12, Figure 15:
`
`
`
`The system constructs histograms in the X- and Y-domains counting the
`
`number of pixels where the differences in luminance between successive frames
`
`are above certain threshold values:
`
`The pixels with greatest movement within the image will
`
`normally occur at the peripheral edges of the head of the
`
`8
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`subject, where even due to slight movements, the pixels
`
`will vary between the luminance of the head of the
`
`subject and the luminance of the background. Thus, if
`
`the system of the invention is set to identify only pixels
`
`with DP=1, and to form a histogram of these pixels, the
`
`histogram will detect movement peaks along the edges of
`
`the face where variations in brightness, and therefore in
`
`pixel value, are the greatest, both in the horizontal
`
`projection along Ox and in the vertical projection along
`
`Oy.
`
`Id. at 22:49-59, 10:30-58 (explaining that DP is set to “1” when pixel value of the
`
`pixel under consideration has “undergone significant variation as compared
`
`to…the same pixel in the prior frame”); Ex. 1002, ¶¶37-38. Figures 16 and 17
`
`show camera setup and the histogram constructed using this method:
`
`9
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`Ex. 1001, Fig. 16
`
`
`
`Ex. 1001, Fig. 17
`
`In addition, the system may also be used to automatically track a target by “a
`
`
`
`spotlight or a camera. Using a spotlight the invention might be used on a
`
`helicopter to track a moving target on the ground, or to track a performer on a stage
`
`during an exhibition. The invention would similarly be applicable to weapons
`
`targeting systems.” Ex. 1001, 23:40-46; Ex. 1002, ¶¶39-40. In such applications,
`
`the system uses X- and Y-minima and maxima of the histograms in X- and Y-
`
`domains to determine the center of the target:
`
`In a preferred embodiment, the new center of the area is
`
`determined to be (XMIN+XMAX)/2, (YMIN+YMAX)/2, where
`
`XMIN and XMAX are the positions of the minima and
`
`maxima of the x projection histogram, and YMIN and
`
`YMAX are the positions of the minima and maxima of the
`
`y projection histogram.
`
`10
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`Ex. 1001, 24:50-55. The patent defines “the positions of the minima” of a
`
`projection histogram to be the smallest X- (and Y-) coordinate of any pixel in the
`
`image region whose validation signal is “1.” Ex. 1002, ¶41. Similarly the
`
`maximum is the largest X- (and Y) coordinate of any pixel in the image region
`
`whose validation signal is “1.” Id. The system may recalculate the histograms at
`
`regular intervals and use those points to again find the center coordinates as the
`
`new frames are received. Ex. 1002, ¶¶41-42. The ’001 Patent teaches that the
`
`system determines the center position “at regular intervals, and preferably in each
`
`frame” and uses this information “to modify the movement of camera 13 to center
`
`face V.” Ex. 1001, ’001 Patent at 23:28-32; Ex. 1002, ¶43. In other words, for
`
`each frame, the system would recalculate the histograms and use those points to
`
`again find the center coordinates.
`
`In addition, the ’001 Patent teaches that as part of the identification and
`
`tracking process, the imaging system may place a tracking box around the target
`
`that “may be displayed on monitor 212, or on another monitor as desired to
`
`visually track the target.” Ex. 1001, 25:20-25; Ex. 1002 ¶44. For example, Figure
`
`23 shows an example of the tracking box in a frame:
`
`11
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`Ex. 1001, Fig. 23
`
`
`
`In addition, the ’001 Patent teaches that the image processing system may
`
`include a memory, a temporal processing unit, and a spatial processing unit. Ex.
`
`1002, ¶45. The ’001 Patent states:
`
`Referring to FIG. 2, image processing system 11 includes
`
`a first assembly 11a, which consists of a temporal
`
`processing unit 15 having an associated memory 16, a
`
`spatial processing unit 17 having a delay unit 18 and
`
`sequencing unit 19, and a pixel clock 20, which generates
`
`a clock signal HP, and which serves as a clock for
`
`temporal processing unit 15 and sequencing unit 19.
`
`Clock pulses HP are generated by clock 20 at the pixel
`
`rate of the image, which is preferably 13.5 MHZ.
`
`Ex. 1001, 10:5-12.
`
`12
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`
`
`
`
`The temporal processing unit 15 of the ’001 Patent “smooth[s] the video
`
`signal and generate[s] a number of outputs that are utilized by spatial processing
`
`unit 17.” Ex. 1001, 10:13-16. Specifically, the temporal processing unit of the
`
`’001 Patent “generates a binary output signal DP for each pixel, which identifies
`
`whether the pixel has undergone significant variation, and a digital signal CO,
`
`which represents the updated calculated value of time constant C.” Ex. 1001,
`
`10:25-29; Ex. 1002, ¶46. Thus, the “temporal processing unit” of the ’001 Patent
`
`refers to a component that generates signal based on the information obtained by
`
`13
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`two (or more) frames in the video signal representing images at different times.
`
`Ex. 1002, ¶47. The spatial processing unit of the ’001 Patent receives input from
`
`the temporal processing unit, and determines the parameters relating to the
`
`movement of the target. Ex. 1001, 15:36-60; Ex. 1002, ¶48.
`
`The ’001 Patent also teaches a method by which the system will “process
`
`pixels only within a user-defined area.” Ex. 1001, 21:17-29. For example, the
`
`system can receive user input instructing it to “process pixels only in a defined
`
`rectangle by setting the XMIN and XMAX, and YMIN and YMAX values as
`
`desired.” Id. The size of the area may be incrementally increased until the box
`
`bounding the processed area overlaps the boundary of the target. Ex. 1001, 24:39-
`
`42 (“This process is continued until the histogram formed by either of histogram
`
`formation units 28 and 29 contains meaningful information, i.e., until the box
`
`overlaps the boundary of the target.”); Ex. 1002, ¶40.
`
`VIII. DETAILED EXPLANATION OF GROUNDS
`A. Overview Of The Prior Art References
`1.
`U.S. Patent No. 5,481,622 to Gerhardt (Ex. 1013)
`The ’001 Patent’s purported invention relates to a process of identifying a
`
`target in digitized visual input by using histograms of pixel characteristics and
`
`tracking the target. This technology was, however, already developed by Lester A.
`
`Gerhardt and Ross M. Sabolcik, researchers at Rensselaer Polytechnic Institute,
`
`14
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`and published as U.S. Patent No. 5,481,622 (“Gerhardt”). Gerhardt issued on
`
`January 2, 1996, and thus qualifies as prior art at least under pre-AIA 35 U.S.C. §
`
`102(b). Although Gerhardt was of record during prosecution, it was not applied in
`
`any office action. Ex. 1002. ¶56.
`
`Gerhardt discloses an image processing system that allows a user to interface
`
`with a computer without hands. Instead, Gerhardt’s system tracks the position of a
`
`user’s pupil to generate input to the computer. Ex. 1002. ¶57. In one example,
`
`Gerhardt’s system uses a video camera 12 mounted on a helmet, as shown in
`
`Figures 1 and 2.
`
`
`
`Gerhardt’s system receives an input signal from a “camera means for
`
`acquiring a video image” and a “frame grabber means [that is] coupled to the
`
`camera means.” Ex. 1013, 2:25-44. The “frame grabber” converts video data
`
`(which inherently contains a plurality of frames) to digital pixel data (a plurality of
`
`pixels). Ex. 1002, ¶58. For each frame input, Gerhardt generates a histogram
`
`based on the pixels’ intensity values to identify and track the user’s pupil. Ex.
`
`15
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`1013, 9:39-61. Gerhardt forms a histogram of the eye image with bins along the
`
`horizontal axis, where the “vertical axis indicates the pixel count of each bin, and
`
`the horizontal axis indicates the magnitude of the pixel intensity of each bin.” Ex.
`
`1013, 9:39-61. In one embodiment, Gerhardt teaches classification according to
`
`the continuous variable of intensity and that intensity may be “represented by a 7-
`
`bit greyscale, or in other words, divided up into 128 bins.” Id. An example
`
`histogram formed based on the eye image is shown in Figure 5:
`
`
`
`From the intensity histogram, Gerhardt identifies the pupil (i.e., the target).
`
`Ex. 1002. ¶59. Gerhardt uses an intensity threshold level that will divide pixel data
`
`into two sets—a darker set (pixels with intensity below the threshold) that has total
`
`pixel area substantially equal to the expected size of the use’s pupil in the eye
`
`image, and a lighter set (the remaining pixels). In the example shown in Figure 5,
`
`the threshold intensity (about 61) is chosen such that the pixels below the threshold
`
`16
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`(shown in black in Figure 5 above) take up about 5% of the image area.
`
`After finding the intensity threshold corresponding to the pupil (i.e., the
`
`target), Gerhardt creates a binary image that shows only the pixels belonging to the
`
`pupil. Ex. 1013, 10:6-34; Ex. 1002. ¶60. A binary image created from the eye
`
`image is shown in Figure 6.
`
`
`
`Once pixels belonging to the target (pupil) are identified in the histogram,
`
`Gerhardt then “locat[es] the pupil, map[s] the pupil coordinates to display screen
`
`coordinate, and inform[s] peripheral devices of the pupil location.” Ex. 1013,
`
`8:34-37. This is done by first identifying the “blobs” or “set[s] of contiguous
`
`pixels” in the image using a region-growing method. Ex. 1013, 12:32-61. The
`
`system then “selects one of these blobs as corresponding to the user’s pupil” based
`
`on the blob’s properties (such as its size, centroid, X- and Y-minima and maxima
`
`of the pixels in the blob, the length-to-width ratio of the blob’s bounding rectangle,
`
`the perimeter of…the blob, or the moment of inertia. Ex. 1013, 9:7-17; 12:32-61.
`
`Examples of the “bounding rectangle[s]…that correspond[] to the x and y-
`
`coordinate maxima…and minima” of the identified blobs are shown in Figure 10,
`
`reproduced below. See Ex. 1002. ¶61.
`
`17
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`
`
`Once the system selects a blob as the target (the pupil), Gerhardt’s system
`
`maps the pupil’s centroid in (x,y) image coordinates “into a corresponding location
`
`in screen coordinates (corresponding, for example, to the user’s point of regard on
`
`a display screen).” Ex. 1013, 15:22-27. The screen coordinates are used by the
`
`interface to provide feedback to the operator. See Ex. 1013, 15:32-39; Ex. 1002.
`
`¶62.
`
`The above-described process of generating a histogram and locating the
`
`pupil blob in the image is repeated for each frame of the video signal. See Ex.
`
`1013, 8:45-52, 9:62-10:1. Figure 15, reproduced below, shows a flow chart of the
`
`image processing steps described above in a continuous loop. See also id., 8:45-52
`
`(the process of identifying and locating the pupil is performed in a “continuous
`
`loop, which involves continually acquiring an eye image with camera 12 and
`
`attempting to locate the pupil position.”); Ex. 1002. ¶63.
`
`18
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`
`
`
`For each image frame, the threshold intensity level found in the intensity
`
`histogram may change, because Gerhardt uses the area criterion (e.g., the 5% area
`
`threshold), which “permits the threshold level to be changed for each image frame
`
`to adjust for changes in lighting conditions.” Ex. 1013, 9:65-10:1; Ex. 1002. ¶64.
`
`Gerhardt’s system also displays a bounding rectangle associated with the
`
`pupil (target) at a display location, which is based on the target location. Ex. 1002.
`
`¶65. Figure 12, reproduced below, illustrates the bounding rectangle around the
`
`19
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`blob identified as the pupil and as seen on the display of the image processing
`
`system. Ex. 1013, 14:33-52 (“[t]he pupil selection method according to the present
`
`invention is able to successfully select pupil blob 150 from the image of Fig. 12.”);
`
`Ex. 1002. ¶65. In addition to a bounding rectangle, the “perimeter of the blob”
`
`may also be used to select the target. Ex. 1013, 12:58-61.
`
`
`
`To improve processing efficiency, Gerhardt’s system may identify a
`
`rectangular area within the image frame and generate a histogram based only on
`
`the plurality of pixels within the identified rectangular area. Ex. 1013, 21:1-18;
`
`Ex. 1002. ¶66. One method of identifying such rectangular area is by “keeping a
`
`running average of the centroid location for previously-selected pupil blobs.” Ex.
`
`1013, 21:8-11. Histograms are generated in the “active area” that is “centered
`
`about the running average centroid location.” Id. If the pupil is not found in the
`
`rectangular area considered, “the size of the active window can be incrementally
`
`increased until the pupil blob is again successfully selected.” Ex. 1013, 21:1-18.
`
`20
`
`

`

`Petition for Inter Partes Review
`Patent No. 8,805,001
`In some cases, Gerhardt’s system receives a user input designating the
`
`position of the pupil (target). Ex. 1002. ¶67. For example, during calibration of
`
`the system “a cursor is placed at a known location on the user interface…and the
`
`user then looks at the cursor for a set period of time.” Ex. 1013, 18:40-58. This
`
`provides the input of the pupil position to the system, and enables the system to
`
`calibrate by determining the user’s pupil location. Ex. 1013, 18:40-58.
`
`U.S. Patent No. 6,044,166 to Bassman (Ex. 1014)
`
`2.
`A similar process and apparatus is also described in a patent issued to
`
`researchers at Sarnoff Corporation of Princeton, New Jersey. Ex. 1002. ¶68. U.S.
`
`Patent No. 6,044,166 to Bassman was filed on February 23, 1996, and thus
`
`qualifies as prior art at least under pre-AIA 35 U.S.C. § 102(e). Bassman was not
`
`of record and was not considered during prosecution of the ’001 Patent. Bassman
`
`discloses an image processing system for tracking vehicles (targets) on a roadway.
`
`Ex. 1014, 2:39-3:13.
`
`Bassman’s image processor receives input from a video camera, and