`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`SAMSUNG ELECTRONICS CO., LTD.; and
`SAMSUNG ELECTRONICS AMERICA, INC.
`Petitioners
`
`v.
`
`IMAGE PROCESSING TECHNOLOGIES, LLC
`Patent Owner
`
`____________________
`
`Patent No. 6,959,293
`____________________
`
`DECLARATION OF DR. JOHN C. HART
`IN SUPPORT OF PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 6,959,293
`
`
`
`
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`Page 1 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`TABLE OF CONTENTS
`
`INTRODUCTION .............................................................................................................. 1
`
`BACKGROUND AND EXPERIENCE ............................................................................. 1
`
`A.
`
`B.
`
`Qualifications .......................................................................................................... 1
`
`Previous Testimony ................................................................................................ 4
`
`TECHNICAL BACKGROUND ......................................................................................... 5
`
`THE ’293 PATENT .......................................................................................................... 11
`
`SUMMARY OF OPINIONS ............................................................................................ 14
`
`LEVEL OF ORDINARY SKILL IN THE ART .............................................................. 15
`
`
`I.
`
`II.
`
`III.
`
`IV.
`
`V.
`
`VI.
`
`VII. CLAIM CONSTRUCTION .............................................................................................. 16
`
`VIII. THE PRIOR ART TEACHES OR SUGGESTS EVERY FEATURE OF THE
`CHALLENGED CLAIMS OF THE ’293 PATENT ........................................................ 17
`
`A.
`
`Overview of the Prior Art References .................................................................. 17
`
`1.
`
`2.
`
`3.
`
`4.
`
`International Patent Publication WO 99/36893 (“Pirim”) ........................ 17
`
`U.S. Patent No. 5,239,594 (“Yoda”) ......................................................... 22
`
`C. International Patent Publication WO 99/35606 (“Qian”) .................... 23
`
`Eriksson et al., “Eye-Tracking for Detection of Drive Fatigue,”
`(IEEE 1998) (“Eriksson”) ......................................................................... 26
`
`B.
`
`Ground 1: the combination of Pirim and Yoda teaches, suggests, or
`discloses every element of Claims 3-17 ................................................................ 27
`
`1.
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`Reasons to combine Pirim and Yoda ........................................................ 27
`
`Claim 3 ...................................................................................................... 29
`
`Claim 4 ...................................................................................................... 46
`
`Claim 5 ...................................................................................................... 50
`
`Claim 6 ...................................................................................................... 51
`
`Claim 7 ...................................................................................................... 53
`
`i
`
`SAMSUNG EXHIBIT 1002
`
`Page 2 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`Claim 8 ...................................................................................................... 55
`
`Claim 9 ...................................................................................................... 56
`
`Claim 10 .................................................................................................... 56
`
`Claim 11 .................................................................................................... 57
`
`Claim 12 .................................................................................................... 60
`
`Claim 13 .................................................................................................... 61
`
`Claim 14 .................................................................................................... 62
`
`Claim 15 .................................................................................................... 63
`
`Claim 16 .................................................................................................... 64
`
`Claim 17 .................................................................................................... 65
`
`7.
`
`8.
`
`9.
`
`10.
`
`11.
`
`12.
`
`13.
`
`14.
`
`15.
`
`16.
`
`C.
`
`Ground 2: The Combination of Pirim and Eriksson teaches, suggests, or
`discloses every element of Claims 20-21 .............................................................. 67
`
`1.
`
`2.
`
`3.
`
`Reasons to Combine Pirim and Eriksson .................................................. 67
`
`Claim 20 .................................................................................................... 69
`
`Claim 21 .................................................................................................... 74
`
`D.
`
`Ground 3: Pirim teaches, suggests, or discloses every element of Claims 2,
`23, and 28 .............................................................................................................. 75
`
`1.
`
`2.
`
`3.
`
`Claim 2 ...................................................................................................... 75
`
`Claim 23 .................................................................................................... 80
`
`Claim 28 .................................................................................................... 84
`
`E.
`
`Ground 3: The combination of Pirim and Qian teaches, suggests, or
`discloses every element of Claims 24-27 .............................................................. 86
`
`1.
`
`2.
`
`3.
`
`4.
`
`Reasons to Combine Pirim and Qian ........................................................ 86
`
`Claim 24 .................................................................................................... 86
`
`Claim 25 .................................................................................................... 89
`
`Claim 26 .................................................................................................... 90
`
`
`
`ii
`
`
`
`SAMSUNG EXHIBIT 1002
`
`Page 3 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`Claim 27 .................................................................................................... 90
`
`5.
`
`IX.
`
`CONCLUSION ................................................................................................................. 91
`
`
`
`
`
`iii
`
`
`
`SAMSUNG EXHIBIT 1002
`
`Page 4 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`I, John C. Hart, declare as follows:
`
`I.
`INTRODUCTION
`1. I have been retained by Samsung Electronics Co., Ltd. and Samsung
`
`Electronics America, Inc. (collectively, “Petitioner”) as an independent expert
`
`consultant in this proceeding before the United States Patent and Trademark
`
`Office (“PTO”).
`
`2. I have been asked to consider whether certain references teach or suggest the
`
`features recited in Claims 2-17, 20-21, and 23-28 of U.S. Patent No. 6,959,293
`
`(“the ’293 Patent”) (Ex. 1001), which I understand is allegedly owned by
`
`Image Processing Technologies, LLC (“Patent Owner”). My opinions and the
`
`bases for my opinions are set forth below.
`
`3. I am being compensated at my ordinary and customary consulting rate for my
`
`work.
`
`4. My compensation is in no way contingent on the nature of my findings, the
`
`presentation of my findings in testimony, or the outcome of this or any other
`
`proceeding. I have no other interest in this proceeding.
`
`II. BACKGROUND AND EXPERIENCE
`A. Qualifications
`5. I have more than 25 years of experience in computer graphics and image
`
`processing technologies. In particular, I have devoted much of my career to
`
`1
`
`SAMSUNG EXHIBIT 1002
`
`Page 5 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`researching and designing graphics hardware and systems for a wide range of
`
`applications.
`
`6. My research has resulted in the publication of more than 80 peer-reviewed
`
`scientific articles, and more than 50 invited papers and talks in the area of
`
`computer graphics and image processing.
`
`7. I have authored or co-authored several publications that are directly related to
`
`target identification and tracking in image processing systems. Some recent
`
`publications include:
`
`• P.R. Khorrami, V.V. Le, J.C. Hart, T.S. Huang, “A System for
`
`Monitoring the Engagement of Remote Online Students using Eye
`
`Gaze Estimation,” Proc. IEEE ICME Workshop on Emerging
`
`Multimedia Systems and Applications, July 2014.
`
`• V. Lu, I. Endres, M. Stroila, and J.C. Hart, “Accelerating Arrays of
`
`Linear Classifiers Using Approximate Range Queries,” Proc. Winter
`
`Conference on Applications of Computer Vision, Mar. 2014.
`
`• M. Kamali, E. Ofek, F. Iandola, I. Omer, J.C. Hart, “Linear Clutter
`
`Removal from Urban Panoramas,” Proc. International Symposium on
`
`Visual Computing. Sep. 2011.
`
`8. From 2008-2012, as a Co-PI of the $18M Intel/Microsoft Universal Parallelism
`
`Computing Research Center at the University of Illinois, I led the AvaScholar
`
`2
`
`SAMSUNG EXHIBIT 1002
`
`Page 6 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`project for visual processing of images that included face identification,
`
`tracking and image histograms.
`
`9. I am a co-inventor of at least one U.S. patent relating to image processing—
`
`U.S. Patent Number 7,365,744.
`
`10. I have served as the Director for Graduate Studies for the Department of
`
`Computer Science, an Associate Dean for the Graduate College, and am
`
`currently serving as the Executive Associate Dean at the University of Illinois.
`
`I am also a professor in the Department of Computer Science at the University
`
`of Illinois, a position I have held since 2000. As a professor I have taught
`
`classes on image processing and graphics technology and have conducted
`
`research into specific applications of these technologies.
`
`11. From 1992 to 2000, I worked first as an Assistant Professor and then as an
`
`Associate Professor in the School of Electrical Engineering and Computer
`
`Science at Washington State University.
`
`12. From 1991-1992, I was a Postdoctoral Research Associate at the Electronic
`
`Visualization Laboratory at the University of Illinois at Chicago, and at the
`
`national Center for Supercomputing Applications at the University of Illinois at
`
`Urbana-Champaign.
`
`13. I earned a Doctor of Philosophy in Electrical Engineering and Computer
`
`Science from the University of Illinois at Chicago in 1991.
`
`3
`
`SAMSUNG EXHIBIT 1002
`
`Page 7 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`14. I earned a Master’s Degree in Electrical Engineering and Computer Science
`
`from the University of Illinois at Chicago in 1989.
`
`15. I earned a Bachelor of Science in Computer Science from Aurora University in
`
`1987.
`
`16. I have been an expert in the field of graphics and image processing since prior
`
`to February 2000, the alleged priority date of the ’293 Patent. I am qualified to
`
`provide an opinion as to what a person of ordinary skill in the art would have
`
`understood, known, or concluded as of 2000.
`
`17. Additional qualifications are detailed in my curriculum vitae, which I
`
`understand has been submitted as Exhibit 1003 in this proceeding.
`
`B.
`Previous Testimony
`18. In the previous five years, I have testified as an expert at trial or by deposition
`
`or have submitted declarations in the following cases:
`
`• Certain Computing or Graphics Systems, Components Thereof,
`
`and Vehicles Containing Same, Inv. No. 337-TA-984, USITC Pub.
`
`• ZiiLabs Inc., Ltd v. Samsung Electronics Co. Ltd. et al., No. 2:14-
`
`cv-00203 (E.D. Tex. Feb. 4, 2016).
`
`• Graphics Property Holding v. Toshiba, Certain Consumer
`
`Electronics with Display and Processing Capabilities, U.S.
`
`International Trade Commission Case #337-TA-884.
`
`4
`
`SAMSUNG EXHIBIT 1002
`
`Page 8 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`• The following petitions for Inter Partes Review in Image
`
`Processing Techs., LLC v. Samsung Electronics Co., Ltd., et al.:
`
`o IPR2017-00336 against the ’293 Patent,
`
`o IPR2017-00357 against U.S. Patent 8,989,445,
`
`o IPR2017-00355 against U.S. Patent 7,650,015,
`
`o IPR2017-00347 against U.S. Patent 8,805,001, and
`
`o IPR2017-00353 against U.S. Patent 8,983,134.
`
`III. TECHNICAL BACKGROUND
`19. Image processing systems have long used histograms as a mathematical tool to
`
`identify and track image features and to adjust image properties. The use of
`
`histograms to identify and track image features dates back to well before the
`
`’293 Patent was filed. See, e.g., Martin Eriksson and Nikoalaos P.
`
`Papanikolopoulos, “Eye-Tracking for Detection of Drive Fatigue,” 0-7803-
`
`4269-0/97 (IEEE 1998) (Ex. 1008); WO 99/35606, Richard Jungiang Qian,
`
`“System for Human Face Tracking,” published July 15, 1999 (Ex. 1007).
`
`20. A digital image is represented by a number of picture elements, or pixels,
`
`where each pixel has certain properties, such as brightness, color, position,
`
`velocity, etc., which may be referred to as domains. For each pixel property or
`
`domain, a histogram may be formed. A histogram is a type of statistical tool.
`
`In image processing, histograms are often used to count the number of pixels in
`
`5
`
`SAMSUNG EXHIBIT 1002
`
`Page 9 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`an image in a certain domain of the pixel. Histograms have multiple bins,
`
`where each bin in the histogram counts the pixels that fall within a range for
`
`that domain. For example, for the continuous variable of luminance (also
`
`called brightness), the luminance value for each pixel can be sampled by a
`
`camera and then digitized and represented by an 8-bit value. Then, those
`
`luminance values could be loaded into a luminance histogram. The histogram
`
`would have one bin for each range of luminance values, and each bin would
`
`count the number of pixels in the image that fall within that luminance value
`
`range. As shown below, a luminance histogram may reveal certain properties
`
`of an image, such as whether it is properly exposed, based on whether an
`
`excessive number of pixels fall on the dark end or light end of the luminance
`
`range.
`
`21. Histograms of other pixel properties can also be formed. For example, the
`
`figure below illustrates two histograms formed by counting the number of
`
`black pixels having each x-coordinate value (i.e., the x-coordinate domain) and
`
`
`
`6
`
`SAMSUNG EXHIBIT 1002
`
`Page 10 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`the number having each y-coordinate value (i.e., the y-coordinate domain).
`
`
`
`22. Such histograms are sometimes called “projection histograms” because they
`
`represent the image projected onto each axis. In the example above, the image
`
`was pure black and white, but projection histograms of a greyscale image can
`
`also be formed in a similar manner by defining a luminance threshold and
`
`projecting, for example, only those pixels that have a luminance value lower
`
`than 100.
`
`23. A more complex greyscale image is shown below, along with its luminance
`
`histogram (black = 0; white = 255):
`
`7
`
`SAMSUNG EXHIBIT 1002
`
`Page 11 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`24. Here, the peak in the dark luminance region (luminance = 0-50) corresponds to
`
`the dark suit and tie and relatively dark background. The peak in the light
`
`luminance region (luminance > 230) corresponds to the white shirt, while the
`
`central peak (between luminance 130 and 170) corresponds largely to the
`
`medium brightness of the face. If one were to select only the subset of pixels
`
`with brightness between 130 and 170 and plot them according to their x and y
`
`8
`
`SAMSUNG EXHIBIT 1002
`
`Page 12 of 95
`
`

`

`position, one would get the following image:
`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`25. Taking projection histograms of this subset of pixels with luminance between
`
`130 and 170, then, provides an indication of location of the face in the image.
`
`On the left, below, is a projection of this subset of pixels onto the x axis, and
`
`on the right is a similar projection onto the y axis.
`
`9
`
`SAMSUNG EXHIBIT 1002
`
`Page 13 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`26. Histograms may also be formed of pixel color properties in much the same
`
`way. Color is typically represented by three values: hue, saturation and
`
`luminance. Hue (aka “tone”) is an angle ranging from 0° to 360° around a
`
`
`
`color wheel that indicates which “color” is bring represented, e.g. 0° = red, 60°
`
`= yellow, 120° = green, 180° = cyan, 240° = blue, and 300° = magenta.
`
`Saturation, which may also range from 0 to 255, represents how “brilliant” the
`
`color is. For example, if a color with a saturation of 255 represents red, then a
`
`saturation of 128 would represent pink and a saturation of 0 would represent
`
`gray. Luminance ranges from 0 to 255 and represents the “brightness” of the
`
`color. If luminance = 0, then the color is black, regardless of the other values.
`
`Given a color image, the luminance values of the pixels would yield the
`
`“black-and-white” or grayscale version of the image.
`
`10
`
`SAMSUNG EXHIBIT 1002
`
`Page 14 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`IV. THE ’293 PATENT
`27. The ’293 Patent, titled “Method and Device for Automatic Visual Perception,”
`
`was filed on February 23, 2001 and issued on October 25, 2005. The ’293
`
`Patent names Patrick Pirim as the sole inventor. I understand that the ’293
`
`Patent claims a priority date of February 24, 2000.
`
`28. The ’293 Patent describes a system for acquiring histograms of various
`
`parameters associated with the pixels that make up a scene. Ex. 1001, ’293
`
`Patent at 7:55-64. Figure 3 of the ’293 Patent, annotated below, illustrates an
`
`embodiment:
`
`11
`
`SAMSUNG EXHIBIT 1002
`
`Page 15 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`29. In Figure 3, DATA(A), corresponding to parameter A, flows through input
`
`multiplexer 105 (shaded green) to the address input of histogram memory 100
`
`(shaded red). For example, if each DATA(A) were an 8-bit value representing
`
`pixel brightness (ranging from 0 to 255) for a pixel in the frame, the histogram
`
`memory would increment the value stored at the address representing the
`
`brightness value for that pixel. In other words, once the frame is processed, the
`
`histogram memory would contain a value at each of 256 memory addresses
`
`representing the number of pixels having the brightness value corresponding to
`
`12
`
`SAMSUNG EXHIBIT 1002
`
`Page 16 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`that address. Ex. 1001, ’293 Patent at 8:45-64.
`
`30. Classifier unit 101 (shaded blue) compares DATA(A) to a classification
`
`condition stored in register 101r. Ex. 1001, ’293 Patent at 9:31-34. For
`
`example, register 203 of the 256 registers in 101r might be set to a “1” for
`
`comparison with the value DATA(A). If DATA(A) is equal to 203, the output
`
`of the classifier, signal 101s, would indicate that the condition is met. Signal
`
`101s is sent to coincidence bus 111 (shaded yellow). Ex. 1001, ’293 Patent at
`
`9:36-42. Coincidence bus 111 also carries output signals from other classifiers
`
`in the system to coincidence unit 102 (shaded purple).
`
`31. Other embodiments of the classifier evaluate whether data falls within a certain
`
`range, or above or below a threshold. For example, Figure 12 and 13a disclose
`
`a classifier (119) that evaluates whether data P is greater than a threshold Q.
`
`Ex. 1001, ’293 Patent at Figs. 12, 13a.
`
`32. The coincidence unit 102 (purple) generates a validation signal that enables
`
`histogram memory 100 to be incremented when certain classification
`
`conditions are met. Ex. 1001, ’293 Patent at 9:36-50. For example, the system
`
`could be configured to enable the generation of a brightness histogram for only
`
`those pixels that have a particular range of color values.
`
`33. Figure 13a, annotated below, is an example of an embodiment in which the
`
`classifier 119 evaluates whether input data P is greater than classification
`
`13
`
`SAMSUNG EXHIBIT 1002
`
`Page 17 of 95
`
`

`

`threshold Q.
`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`34. In this example, threshold Q need not be set to a static value but rather can be
`
`automatically updated based on histogram data. For example, RMAX is the
`
`number of counts in the most populated bin of a histogram and NBPTS is the
`
`total number of points in (or, more accurately, pixels used to generate) a
`
`histogram. Ex. 1001, ’293 Patent at 10:7-31.
`
`35. As the circuit of 13a shows, the threshold Q might be set to ½ of RMAX, or
`
`can be set to some other value loaded through block 123. Ex. 1001, ’293
`
`Patent at 10:7-31. .
`
`V.
`SUMMARY OF OPINIONS
`36. In preparing this declaration, I have reviewed at least the documents labeled
`
`Exhibits 1001-1009 and other materials referred to herein in connection with
`
`14
`
`SAMSUNG EXHIBIT 1002
`
`Page 18 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`providing this declaration. In addition to these materials, I have relied on my
`
`education, experience, and my knowledge of practices and principles in the
`
`relevant field, e.g., image processing. My opinions have also been guided by
`
`my appreciation of how one of ordinary skill in the art would have understood
`
`the claims and specification of the ’293 Patent around the time of the alleged
`
`invention, which I have been asked to assume is February 24, 2000.
`
`37. Based on my experience and expertise, it is my opinion that certain references
`
`teach or suggest all the features recited in Claims 2-17, 20-21, and 23-28 of the
`
`’293 Patent, as explained in detail below. Specifically, it is my opinion that
`
`Claims 3-17 are taught or disclosed by International Patent Publication WO
`
`99/36893 (“Pirim”) in combination with U.S. Patent No. 5,239,594 (“Yoda”).
`
`It is also my opinion that Claims 20-21 are taught or disclosed by Pirim in
`
`combination with Eriksson et. al, “Eye-Tracking for Detection of Drive
`
`Fatigue,” (IEEE 1998) (“Eriksson”). It is also my opinion that Claims 2, 23,
`
`and 28 are taught or disclosed by Pirim. It is also my opinion that Claims 24-
`
`27 are taught or disclosed by Pirim in combination with International Patent
`
`Publication WO 99/35606 (“Qian”).
`
`VI. LEVEL OF ORDINARY SKILL IN THE ART
`38. Based on my review of the ’293 Patent specification, claims, and file history, I
`
`believe one of ordinary skill in the art around the time of the alleged invention
`
`15
`
`SAMSUNG EXHIBIT 1002
`
`Page 19 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`of the ’293 Patent would have had either (1) a Master’s Degree in Electrical
`
`Engineering or Computer Science or the equivalent plus at least a year of
`
`experience in the field of image processing, image recognition, machine vision,
`
`or a related field or (2) a Bachelor’s Degree in Electrical Engineering or
`
`Computer Science or the equivalent plus at least three years of experience in
`
`the field of image processing, image recognition, machine vision, or a related
`
`field. Additional education could substitute for work experience and vice
`
`versa.
`
`39. In determining the level of ordinary skill in the art, I was asked to consider, for
`
`example, the type of problems encountered in the art, prior art solutions to
`
`those problems, the rapidity with which innovations are made, the
`
`sophistication of the technology, and the educational level of active workers in
`
`the field.
`
`40. I was one of at least ordinary skill in the art as of February 2000, and my
`
`opinions concerning the ’293 Patent claims are from the perspective of a
`
`person of ordinary skill in the art, as set forth above.
`
`VII. CLAIM CONSTRUCTION
`41. I have been instructed to interpret all claim terms in accordance with their
`
`broadest reasonable plain meanings in light of the patent specification, and I
`
`have applied this interpretation to my analysis.
`
`16
`
`SAMSUNG EXHIBIT 1002
`
`Page 20 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`VIII. THE PRIOR ART TEACHES OR SUGGESTS EVERY FEATURE OF
`THE CHALLENGED CLAIMS OF THE ’293 PATENT
`A. Overview of the Prior Art References
`1.
`International Patent Publication WO 99/36893 (“Pirim”)
`42. Pirim discloses a system for detecting whether a driver is falling asleep by
`
`acquiring pictures of the driver and forming histograms to analyze opening and
`
`closing of the driver’s eyes. Ex. 1005, Pirim, at 5. Pirim’s image processing
`
`system “receives a digital video signal S originating from a video camera or
`
`other imaging device 13 which monitors a scene 13a.” Id. at 12. “Signal S(PI)
`
`represents signal S composed of pixels PI.” Id. at 13. Each video frame
`
`comprises horizontal scanned lines, each including “a succession of pixels or
`
`image points PI, e.g., a1.1, a1.2, and a1.3 for line l1.1.” Id.
`
`43. With reference to Figure 14, annotated below, Pirim discloses a histogram unit
`
`having a memory 100 (shaded red). Data(V), representing pixel parameter V,
`
`proceeds through input multiplexer 104 (shaded green) to the address input of
`
`memory 100. Id. at 29. Just as in the ’293 Patent, a value stored at the address
`
`corresponding to the value of the input data parameter is incremented to
`
`accumulate a histogram of the parameter. Id.
`
`44. Pirim further discloses a “classifier 25b” (shaded blue) that receives the
`
`data(V) value and compares it to a “register 106 that enables the classification
`
`criteria to be set by the user, or by a separate computer program.” Id. at 29-30.
`
`17
`
`SAMSUNG EXHIBIT 1002
`
`Page 21 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`45. The output of classifier 25b proceeds to a bus 23 (shaded yellow), which also
`
`carries the output of other classifiers in the system. Id. at 31. These signals
`
`proceed to validation unit 31 (shaded purple). “Each validation unit generates
`
`a validation signal which is communicated to its associated histogram
`
`formation block 24-29. The validation signal determines, for each incoming
`
`pixel, whether the histogram formation block will utilize that pixel in forming
`
`18
`
`SAMSUNG EXHIBIT 1002
`
`Page 22 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`it histogram.” Id. Thus, the operation of the system is summarized as follows:
`
`Thus, using the classifiers in combination with validation units
`30-35, the system may select for processing only data points in
`any selected classes within any selected domains. For example,
`the system may be used to detect only data points having speed
`2, direction 4, and luminance 125 by setting each of the
`following registers to “1”: the registers in the validation units
`for speed, direction, and luminance, register 2 in the speed
`classifier, register 4 in the direction classifier, and register 125
`in the luminance classifier. In order to form those pixels into a
`block, the registers in the validation units for the x and y
`directions would be set to “1” as well.
`
`Id. at 31.
`
`46. Pirim also discloses that statistical characteristics of the histogram are
`
`calculated, including “the minimum (MIN) of the histogram, the maximum
`
`(MAX) of the histogram, the number of points (NBPTS) in the histogram, the
`
`position (POSRMAX) of the maximum of the histogram.” Id. at 32. Such
`
`statistics may be used to automatically set limits of the classifiers:
`
`the envelopes of
`represents
`Fig. 13 diagrammatically
`histograms 38 and 39, respectively in x and y coordinates, for
`velocity data. In this example, XM and YM represent the x and y
`coordinates of the maxima of the two histograms 38 and 39,
`whereas la and lb for the x axis and lc and ld for the y axis
`represent the limits of the range of significant or interesting
`
`19
`
`SAMSUNG EXHIBIT 1002
`
`Page 23 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`speeds, la and lc being the longer [sic] limits and lb and ld being
`the upper limited [sic] of the significant portions of the
`histograms. Limits la, lb, lc, and ld may be set by the user or by
`an application program using the system, may be set as a ratio
`of the maximum of the histogram, e.g., XM/2, or may be set as
`otherwise desired for the particular application.
`Id. at 36-37 (emphasis added). In other words, among the ways the classification
`
`criterion can be set, it can be set to a statistic derived from the histogram, such as
`
`half of the maximum value (of the velocity data in this example).
`
`47. Pirim also discloses that the controller in the system can read any of these
`
`statistics and can execute a program to update the classification criteria in the
`
`classifiers (among other parameters in the system):
`
`Controller 42 is in communication with data bus 23, which
`allows controller 42 to run a program to control various
`parameters that may be set in the system and to analyze the
`results. In order to select the criteria of pixels to be tracked,
`controller 42 may also directly control the following: i) content
`of each register in classifiers 25b, ii) the content of each register
`in validation units 31 . . . Controller 42 may also retrieve i) the
`content of each memory 100 and ii) the content of registers 112,
`in order to analyze the results of the histogram formation
`process.”
`Id. at 38-39. Pirim further discloses that the controller may adjust these
`
`classification thresholds dynamically to automatically adapt the system to the
`
`20
`
`SAMSUNG EXHIBIT 1002
`
`Page 24 of 95
`
`

`

`observed scene:
`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`Controller 42 constantly adapts operation of the system,
`especially in varying lighting levels. Controller 42 may detect
`varying lighting conditions by periodically monitoring the
`luminance histogram and adapting the gain bias of the sensor to
`maintain as broad a luminance spectrum as possible. Controller
`42 may also adjust the thresholds that are used to determine
`shadowing, etc. to better distinguish eye and nostril shadowing
`from noise, e. g. shadowing on the side of the nose, and may
`also adjust the sensor gain to minimize this effect. If desired
`controller 42 may cause the histogram formation units to form a
`histogram of the iris. This histogram may also be monitored for
`consistency, and the various thresholds used in the system
`adjusted as necessary.
`
`Id. at 57.
`
`48. Pirim also discloses an anticipation function for predicting the future value of a
`
`parameter based on statistics about the parameter in prior frames:
`
`As discussed above, the system of the invention may be used to
`search for objects within a bounded area defined by XMIN,
`XMAX, YMIN and YMAX. Because moving object may leave
`includes an
`the bounded area
`the system preferably
`anticipation function which enables XMIN, XMAX, YMIN
`and YMAX to be automatically modified by the system to
`compensate for the speed and direction of the target. This is
`accomplished
`by
`determining
`values
`for O-MVT,
`
`21
`
`SAMSUNG EXHIBIT 1002
`
`Page 25 of 95
`
`

`

`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`corresponding to orientation (direction) of movement of the
`target within the bounded area using the direction histogram,
`and I-MVT, corresponding to the intensity (velocity) of
`movement. Using these parameters, controller 42 may modify
`the values of XMIN, XMAX, YMIN and YMAX on a frame-
`by-frame basis to ensure that the target remains in the bounded
`box being searched.
`Id. at 37-38 (emphasis added).
`
`2.
`U.S. Patent No. 5,239,594 (“Yoda”)
`49. Yoda discloses a system of “self-organizing classifiers” for analyzing images
`
`on the basis of multiple characteristics. For example, Yoda states that
`
`classifying on a single feature, such as brightness, is not always sufficient:
`
`FIG. 4 provides an example wherein a single feature is used.
`In particular, it shows the distributions 13 and 14 of brightness
`features F1 for ash wood and birch wood, respectively… In Fig.
`4 the