UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`SAMSUNG ELECTRONICS CO., LTD.; and
`SAMSUNG ELECTRONICS AMERICA, INC.
`Petitioners
`
`v.
`
`IMAGE PROCESSING TECHNOLOGIES, LLC
`Patent Owner
`
`____________________
`
`Patent No. 6,959,293
`____________________
`
`DECLARATION OF DR. JOHN C. HART
`IN SUPPORT OF PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 6,959,293
`
`
`
`
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`TABLE OF CONTENTS
`
`INTRODUCTION .............................................................................................................. 1
`
`BACKGROUND AND EXPERIENCE ............................................................................. 1
`
`A.
`
`B.
`
`Qualifications .......................................................................................................... 1
`
`Previous Testimony ................................................................................................ 4
`
`TECHNICAL BACKGROUND ......................................................................................... 5
`
`THE ’293 PATENT .......................................................................................................... 11
`
`SUMMARY OF OPINIONS ............................................................................................ 14
`
`LEVEL OF ORDINARY SKILL IN THE ART .............................................................. 15
`
`
`I.
`
`II.
`
`III.
`
`IV.
`
`V.
`
`VI.
`
`VII. CLAIM CONSTRUCTION .............................................................................................. 16
`
`VIII. THE PRIOR ART TEACHES OR SUGGESTS EVERY FEATURE OF THE
`CHALLENGED CLAIMS OF THE ’293 PATENT ........................................................ 17
`
`A.
`
`Overview of the Prior Art References .................................................................. 17
`
`1.
`
`2.
`
`3.
`
`4.
`
`International Patent Publication WO 99/36893 (“Pirim”) ........................ 17
`
`U.S. Patent No. 5,546,125 to Tomitaka et al. (“Tomitaka”) ..................... 21
`
`Robert B. Rogers, “Real-Time Video Filtering With Bit-Slice
`Microprogrammable Processors,” Ph.D. Dissertation, New Mexico
`State University (1978) (“Rogers”) .......................................................... 24
`
`Alton L. Gilbert et al., “A Real-Time Video Tracking System,”
`IEEE Transactions on Pattern Analysis and Machine Intelligence,
`Vol. PAMI-2, No. 2, January 1980 (“Gilbert”) ........................................ 31
`
`B.
`
`Ground 1: the combination of Pirim and Tomitaka teaches, suggests, or
`discloses every element of Claims 1, 18, 19, 22, and 29 ...................................... 34
`
`1.
`
`2.
`
`3.
`
`4.
`
`Reasons to combine Pirim and Tomitaka ................................................. 34
`
`Claim 1 ...................................................................................................... 36
`
`Claim 18 .................................................................................................... 44
`
`Claim 19 .................................................................................................... 52
`
`i
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`Claim 22 .................................................................................................... 53
`
`Claim 29 .................................................................................................... 55
`
`5.
`
`6.
`
`C.
`
`Ground 2: the combination of Rogers and Gilbert teaches, suggests, or
`discloses every limitation of Claims 1, 18, 19, 22, and 29. .................................. 57
`
`1.
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`Reasons to Combine Rogers and Gilbert .................................................. 57
`
`Claim 1 ...................................................................................................... 58
`
`Claim 18 .................................................................................................... 62
`
`Claim 19 .................................................................................................... 66
`
`Claim 22 .................................................................................................... 67
`
`Claim 29 .................................................................................................... 71
`
`D.
`
`Ground 3: the combination of Tomitaka and Rogers teaches, suggests, or
`discloses every element of Claims 1, 18, 19, 22, and 29 ...................................... 72
`
`1.
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`Reasons to Combine Tomitaka and Rogers .............................................. 72
`
`Claim 1 ...................................................................................................... 76
`
`Claim 18 .................................................................................................... 81
`
`Claim 19 .................................................................................................... 84
`
`Claim 22 .................................................................................................... 84
`
`Claim 29 .................................................................................................... 87
`
`IX.
`
`CONCLUSION ................................................................................................................. 89
`
`ii
`
`
`
`SAMSUNG EXHIBIT 1002
`
`
`
`
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`I, John C. Hart, declare as follows:
`
`I.
`
`INTRODUCTION
`1. I have been retained by Samsung Electronics Co., Ltd. and Samsung
`
`Electronics America, Inc. (collectively, “Petitioner”) as an independent expert
`
`consultant in this proceeding before the United States Patent and Trademark Office
`
`(“PTO”).
`
`2. I have been asked to consider whether certain references teach or suggest
`
`the features recited in Claims 1, 18, 19, 22, and 29 of U.S. Patent No. 6,959,293
`
`(“the ’293 Patent”) (Ex. 1001), which I understand is allegedly owned by Image
`
`Processing Technologies, LLC (“Patent Owner”). My opinions and the bases for
`
`my opinions are set forth below.
`
`3. I am being compensated at my ordinary and customary consulting rate
`
`for my work.
`
`4. My compensation is in no way contingent on the nature of my findings,
`
`the presentation of my findings in testimony, or the outcome of this or any other
`
`proceeding. I have no other interest in this proceeding.
`
`II. BACKGROUND AND EXPERIENCE
`A. Qualifications
`5. I have more than 25 years of experience in computer graphics and image
`
`processing technologies. In particular, I have devoted much of my career to
`
`1
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`researching and designing graphics hardware and systems for a wide range of
`
`applications.
`
`6. My research has resulted in the publication of more than 80 peer-
`
`reviewed scientific articles, and more than 50 invited papers and talks in the area of
`
`computer graphics and image processing.
`
`7. I have authored or co-authored several publications that are directly
`
`related to target identification and tracking in image processing systems. Some
`
`recent publications include:
`
`• P.R. Khorrami, V.V. Le, J.C. Hart, T.S. Huang, “A System for
`
`Monitoring the Engagement of Remote Online Students using Eye
`
`Gaze Estimation,” Proc. IEEE ICME Workshop on Emerging
`
`Multimedia Systems and Applications, July 2014.
`
`• V. Lu, I. Endres, M. Stroila, and J.C. Hart, “Accelerating Arrays of
`
`Linear Classifiers Using Approximate Range Queries,” Proc. Winter
`
`Conference on Applications of Computer Vision, Mar. 2014.
`
`• M. Kamali, E. Ofek, F. Iandola, I. Omer, J.C. Hart, “Linear Clutter
`
`Removal from Urban Panoramas,” Proc. International Symposium on
`
`Visual Computing. Sep. 2011.
`
`8. From 2008-2012, as a Co-PI of the $18M Intel/Microsoft Universal
`
`Parallelism Computing Research Center at the University of Illinois, I led the
`
`2
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`AvaScholar project for visual processing of images that included face
`
`identification, tracking and image histograms.
`
`9. I am a co-inventor of at least one U.S. patent relating to image
`
`processing—U.S. Patent Number 7,365,744.
`
`10. I have served as the Director for Graduate Studies for the Department of
`
`Computer Science, an Associate Dean for the Graduate College, and am currently
`
`serving as the Executive Associate Dean at the University of Illinois. I am also a
`
`professor in the Department of Computer Science at the University of Illinois, a
`
`position I have held since 2000. As a professor I have taught classes on image
`
`processing and graphics technology and have conducted research into specific
`
`applications of these technologies.
`
`11. From 1992 to 2000, I worked first as an Assistant Professor and then as
`
`an Associate Professor in the School of Electrical Engineering and Computer
`
`Science at Washington State University.
`
`12. From 1991-1992, I was a Postdoctoral Research Associate at the
`
`Electronic Visualization Laboratory at the University of Illinois at Chicago, and at
`
`the national Center for Supercomputing Applications at the University of Illinois at
`
`Urbana-Champaign.
`
`13. I earned a Doctor of Philosophy in Electrical Engineering and Computer
`
`Science from the University of Illinois at Chicago in 1991.
`
`3
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`14. I earned a Master’s Degree in Electrical Engineering and Computer
`
`Science from the University of Illinois at Chicago in 1989.
`
`15. I earned a Bachelor of Science in Computer Science from Aurora
`
`University in 1987.
`
`16. I have been an expert in the field of graphics and image processing since
`
`prior to February 2000, the alleged priority date of the ’293 Patent. I am qualified
`
`to provide an opinion as to what a person of ordinary skill in the art would have
`
`understood, known, or concluded as of 2000.
`
`17. Additional qualifications are detailed in my curriculum vitae, which I
`
`understand has been submitted as Exhibit 1003 in this proceeding.
`
`B.
`Previous Testimony
`18. In the previous five years, I have testified as an expert at trial or by
`
`deposition or have submitted declarations in the following cases:
`
`• Certain Computing or Graphics Systems, Components Thereof,
`
`and Vehicles Containing Same, Inv. No. 337-TA-984, USITC Pub.
`
`• ZiiLabs Inc., Ltd v. Samsung Electronics Co. Ltd. et al., No. 2:14-
`
`cv-00203 (E.D. Tex. Feb. 4, 2016).
`
`• Graphics Property Holding v. Toshiba, Certain Consumer
`
`Electronics with Display and Processing Capabilities, U.S.
`
`International Trade Commission Case #337-TA-884.
`
`4
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`III. TECHNICAL BACKGROUND
`19. Image processing systems have long used histograms as a mathematical
`
`tool to identify and track image features and to adjust image properties. The use of
`
`histograms to identify and track image features dates back to well before 1997.
`
`See, e.g., Alton L. Gilbert et al., “A Real-Time Video Tracking System,” IEEE
`
`Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-2, No. 2,
`
`January 1980 (Ex. 1008); D. Trier, A. K. Jain and T. Taxt, “Feature Extraction
`
`Methods for Character Recognition-A Survey”, Pattern Recognition, vol. 29, no. 4,
`
`1996, pp. 641–662 (Ex. 1010 at 10) (citing M. H. Glauberman, “Character
`
`recognition for business machines,” Electronics, vol. 29, pp. 132-136, Feb. 1956
`
`(Ex. 1011)).
`
`20. A digital image is represented by a number of picture elements, or
`
`pixels, where each pixel has certain properties, such as brightness, color, position,
`
`velocity, etc., which may be referred to as domains. For each pixel property or
`
`domain, a histogram may be formed. A histogram is a bar chart counting the
`
`number of pixels in the image, where each bar counts the pixels that share a certain
`
`characteristic. For example, if the luminance (brightness) of each pixel were
`
`represented by an 8-bit value, a luminance histogram would be a bar chart of the
`
`number of pixels in the image indicating, for each luminance value from 0 to 255,
`
`the count of the number of pixels in the image with that luminance value. As
`
`5
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`shown below, a luminance histogram may reveal certain properties of an image,
`
`such as whether it is properly exposed, based on whether an excessive number of
`
`pixels fall on the dark end or light end of the luminance range.
`
`
`
`21. Histograms of other pixel properties can also be formed. For example,
`
`the figure below illustrates two histograms formed by counting the number of
`
`black pixels having each x-coordinate value (i.e., the x-coordinate domain) and the
`
`number having each y-coordinate value (i.e., the y-coordinate domain).
`
`6
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`22. Such histograms are sometimes called “projection histograms” because
`
`they represent the image projected onto each axis. In the example above, the
`
`image was pure black and white, but projection histograms of a greyscale image
`
`can also be formed in a similar manner by defining a luminance threshold and
`
`projecting, for example, only those pixels that have a luminance value lower than
`
`100.
`
`23. A more complex greyscale image is shown below, along with its
`
`luminance histogram (black = 0; white = 255):
`
`7
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`24. Here, the peak in the dark luminance region (luminance = 0-50)
`
`corresponds to the dark suit and tie and relatively dark background. The peak in
`
`the light luminance region (luminance > 230) corresponds to the white shirt, while
`
`the central peak (between luminance 130 and 170) corresponds largely to the
`
`medium brightness of the face. If one were to select only the subset of pixels with
`
`brightness between 130 and 170 and plot them according to their x and y position,
`
`8
`
`SAMSUNG EXHIBIT 1002
`
`

`
`one would get the following image:
`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`25. Taking projection histograms of this subset of pixels with luminance
`
`between 130 and 170, then, provides an indication of location of the face in the
`
`image. On the left, below, is a projection of this subset of pixels onto the x axis,
`
`and on the right is a similar projection onto the y axis.
`
`9
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`26. Histograms may also be formed of pixel color properties in much the
`
`same way. Color is typically represented by three values: hue, saturation and
`
`luminance. Hue (aka “tone”) is an angle ranging from 0° to 360° around a color
`
`wheel that indicates which “color” is bring represented, e.g. 0° = red, 60° =
`
`yellow, 120° = green, 180° = cyan, 240° = blue, and 300° = magenta. Saturation,
`
`which may also range from 0 to 255, represents how “brilliant” the color is. For
`
`example, if a color with a saturation of 255 represents red, then a saturation of 128
`
`would represent pink and a saturation of 0 would represent gray. Luminance
`
`ranges from 0 to 255 and represents the “brightness” of the color. If luminance =
`
`0, then the color is black, regardless of the other values. Given a color image, the
`
`luminance values of the pixels would yield the “black-and-white” or grayscale
`
`version of the image.
`
`10
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`IV. THE ’293 PATENT
`27. The ’293 Patent, titled “Method and Device for Automatic Visual
`
`Perception,” was filed on February 23, 2001 and issued on October 25, 2005. The
`
`’293 Patent names Patrick Pirim as the sole inventor. I understand that the ’293
`
`Patent claims a priority date of February 24, 2000.
`
`28. The ’293 Patent describes a system for acquiring histograms of various
`
`parameters associated with the pixels that make up a scene. Ex. 1001, ’293 Patent
`
`at 7:55-64. Figure 3 of the ’293 Patent, annotated below, illustrates an
`
`embodiment:
`
`11
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`29. In Figure 3, DATA(A), corresponding to parameter A, flows through
`
`input multiplexer 105 (shaded green) to the address input of histogram memory
`
`100 (shaded red). For example, if each DATA(A) were an 8-bit value representing
`
`pixel brightness (ranging from 0 to 255) for a pixel in the frame, the histogram
`
`memory would increment the value stored at the address representing the
`
`brightness value for that pixel. In other words, once the frame is processed, the
`
`histogram memory would contain a value at each of 256 memory addresses
`
`representing the number of pixels having the brightness value corresponding to that
`
`12
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`address. Ex. 1001, ’293 Patent at 8:45-64.
`
`30. Classifier unit 101 (shaded blue) compares DATA(A) to a classification
`
`condition stored in register 101r. Ex. 1001, ’293 Patent at 9:31-34. For example,
`
`register 101r might store a particular brightness value, such as 203, for comparison
`
`with DATA(A). Signal 101s is the output of the classifier, indicating whether the
`
`condition is met, and is sent to coincidence bus 111 (shaded yellow). Ex. 1001,
`
`’293 Patent at 9:36-42. Coincidence bus 111 may also carry output signals from
`
`other classifiers in the system to coincidence unit 102 (shaded purple).
`
`31. Other embodiments of the classifier evaluate whether data falls within a
`
`certain range, or above or below a threshold. For example, Figure 12 and 13a
`
`disclose a classifier (119) that evaluates whether data P is greater than a condition
`
`Q. Ex. 1001, ’293 Patent at Figs. 12, 13a.
`
`32. The coincidence unit 102 (purple) generates a validation signal that
`
`enables histogram memory 100 to be incremented when certain classification
`
`conditions are met. Ex. 1001, ’293 Patent at 9:36-50. For example, the system
`
`could be configured to enable a brightness histogram for only those pixels that
`
`have a particular range of color values.
`
`33. Figure 13a, annotated below, is an example of an embodiment in which
`
`the classifier 119 evaluates whether input data P is greater than classification
`
`threshold Q.
`
`13
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`
`34. In this example, threshold Q need not be set to a static value but rather
`
`can be automatically updated based on histogram data. For example, RMAX is the
`
`number of counts in the highest bin of a histogram and NBPTS is the total number
`
`of points in a histogram. Ex. 1001, ’293 Patent at 10:7-31.
`
`35. As the circuit of 13a shows, the threshold Q might be set to ½ of
`
`RMAX, or can be set to some other value loaded through block 123. Ex.
`
`1001, ’293 Patent at 10:7-31. .
`
`V.
`
`SUMMARY OF OPINIONS
`36. In preparing this declaration, I have reviewed at least the documents
`
`labeled Exhibits 1001-1012 and other materials referred to herein in connection
`
`with providing this declaration. In addition to these materials, I have relied on my
`
`education, experience, and my knowledge of practices and principles in the
`
`14
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`relevant field, e.g., image processing. My opinions have also been guided by my
`
`appreciation of how one of ordinary skill in the art would have understood the
`
`claims and specification of the ’293 Patent around the time of the alleged
`
`invention, which I have been asked to assume is February 24, 2000.
`
`37. Based on my experience and expertise, it is my opinion that certain
`
`references teach or suggest all the features recited in Claims 1, 18, 19, 22, and 29
`
`of the ’293 Patent, as explained in detail below. Specifically, it is my opinion that
`
`Claims 1, 18, 19, 22, and 29 are taught or disclosed by International Patent
`
`Publication WO 99/36893 (“Pirim”) in combination with U.S. Patent No.
`
`5,546,125 (“Tomitaka”). It is also my opinion that Claims 1, 18, 19, 22, and 29 are
`
`taught or disclosed by Robert B. Rogers, “Real-Time Video Filtering With Bit-
`
`Slice Microprogrammable Processors,” Ph.D. Dissertation, New Mexico State
`
`University (1978) (“Rogers”) in combination with Alton L. Gilbert et al., “A Real-
`
`Time Video Tracking System,” IEEE Transactions on Pattern Analysis and
`
`Machine Intelligence, Vol. PAMI-2, No. 2, January 1980 (“Gilbert”). It is also my
`
`opinion that Claims 1, 18, 19, 22, and 29 are taught or disclosed by Tomitaka in
`
`combination with Rogers.
`
`VI. LEVEL OF ORDINARY SKILL IN THE ART
`38. Based on my review of the ’293 Patent specification, claims, and file
`
`history, I believe one of ordinary skill in the art around the time of the alleged
`
`15
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`invention of the ’293 Patent would have had either (1) a Master’s Degree in
`
`Electrical Engineering or Computer Science or the equivalent plus at least a year of
`
`experience in the field of image processing, image recognition, machine vision, or
`
`a related field or (2) a Bachelor’s Degree in Electrical Engineering or Computer
`
`Science or the equivalent plus at least three years of experience in the field of
`
`image processing, image recognition, machine vision, or a related field. Additional
`
`education could substitute for work experience and vice versa.
`
`39. In determining the level of ordinary skill in the art, I was asked to
`
`consider, for example, the type of problems encountered in the art, prior art
`
`solutions to those problems, the rapidity with which innovations are made, the
`
`sophistication of the technology, and the educational level of active workers in the
`
`field.
`
`40. I was one of at least ordinary skill in the art as of February 2000, and my
`
`opinions concerning the ’293 Patent claims are from the perspective of a person of
`
`ordinary skill in the art, as set forth above.
`
`VII. CLAIM CONSTRUCTION
`41. I have been instructed to interpret all claim terms in accordance with
`
`their broadest reasonable plain meanings in light of the patent specification, and I
`
`have applied this interpretation to my analysis.
`
`16
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`VIII. THE PRIOR ART TEACHES OR SUGGESTS EVERY FEATURE OF
`THE CHALLENGED CLAIMS OF THE ’293 PATENT
`A. Overview of the Prior Art References
`1.
`International Patent Publication WO 99/36893 (“Pirim”)
`42. Pirim discloses a system for detecting whether a driver is falling asleep
`
`by acquiring pictures of the driver and forming histograms to analyze opening and
`
`closing of the driver’s eyes. Ex. 1005, Pirim, at 5. Pirim’s image processing
`
`system “receives a digital video signal S originating from a video camera or other
`
`imaging device 13 which monitors a scene 13a.” Id. at 12. “Signal S(PI)
`
`represents signal S composed of pixels PI.” Id. at 13. Each video frame comprises
`
`horizontal scanned lines, each including “a succession of pixels or image points PI,
`
`e.g., a1.1, a1.2, and a1.3 for line l1.1.” Id.
`
`43. With reference to Figure 14, annotated below, Pirim discloses a
`
`histogram unit having a memory 100 (shaded red). Data(V), representing pixel
`
`parameter V, proceeds through input multiplexer 104 (shaded green) to the address
`
`input of memory 100. Id. at 29. Just as in the ’293 Patent, a value stored at the
`
`address corresponding to the value of the input data parameter is incremented to
`
`accumulate a histogram of the parameter. Id.
`
`44. Pirim further discloses a “classifier 25b” (shaded blue) that receives the
`
`data(V) value and compares it to a “register 106 that enables the classification
`
`criteria to be set by the user, or by a separate computer program.” Id. at 29-30.
`
`17
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`
`
`45. The output of classifier 25b proceeds to a bus 23 (shaded yellow), which
`
`also carries the output of other classifiers in the system. Id. at 31. These signals
`
`proceed to validation unit 31 (shaded purple). “Each validation unit generates a
`
`validation signal which is communicated to its associated histogram formation
`
`block 24-29. The validation signal determines, for each incoming pixel, whether
`
`the histogram formation block will utilize that pixel in forming it histogram.” Id.
`
`18
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`Thus, the operation of the system is summarized as follows:
`
`Thus, using the classifiers in combination with validation units
`30-35, the system may select for processing only data points in
`any selected classes within any selected domains. For example,
`the system may be used to detect only data points having speed
`2, direction 4, and luminance 125 by setting each of the
`following registers to “1”: the registers in the validation units
`for speed, direction, and luminance, register 2 in the speed
`classifier, register 4 in the direction classifier, and register 125
`in the luminance classifier. In order to form those pixels into a
`block, the registers in the validation units for the x and y
`directions would be set to “1” as well.
`
`Id. at 31.
`
`46. Pirim also discloses that statistical characteristics of the histogram are
`
`calculated, including “the minimum (MIN) of the histogram, the maximum (MAX)
`
`of the histogram, the number of points (NBPTS) in the histogram, the position
`
`(POSRMAX) of the maximum of the histogram.” Id. at 32. Such statistics may be
`
`used to automatically set limits of the classifiers:
`
`the envelopes of
`represents
`Fig. 13 diagrammatically
`histograms 38 and 39, respectively in x and y coordinates, for
`velocity data. In this example, XM and YM represent the x and y
`coordinates of the maxima of the two histograms 38 and 39,
`whereas la and lb for the x axis and lc and ld for the y axis
`represent the limits of the range of significant or interesting
`
`19
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`speeds, la and lc being the longer [sic] limits and lb and ld being
`the upper limited [sic] of the significant portions of the
`histograms. Limits la, lb, lc, and ld may be set by the user or by
`an application program using the system, may be set as a ratio
`of the maximum of the histogram, e.g., XM/2, or may be set as
`otherwise desired for the particular application.
`Id. at 36-37 (emphasis added). In other words, among the ways the classification
`
`criterion can be set, it can be set to a statistic derived from the histogram, such as
`
`half of the maximum value (of the velocity data in this example).
`
`47. Pirim also discloses classifiers based on X and Y position of pixels that
`
`restrict histogram processing to only a particular rectangular region:
`
`In order to process pixels only within a user-defined area, the x-
`direction histogram formation unit 28 may be programmed to
`process pixels only in a class of pixels defined by boundaries,
`i.e., XMIN and XMAX. This is accomplished by setting the
`XMIN and XMAX values in a user-programmable memory in
`x-direction histogram
`formation unit 28 or
`in
`linear
`combination units 30-35. Any pixels outside of this class will
`not be processed. Similarly, y-direction histogram formation
`unit 29 may be used to process pixels only in a class of pixels
`defined by boundaries YMIN and YMAX.
`Id. at 35. These X and Y, MIN and MAX classification criteria may also be
`
`changed automatically by the system using statistics derived from the histograms:
`
`20
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`Because the moving object may leave the bounded area the
`system preferably includes an anticipation function which
`enables XMIN, XMAX, YMIN, and YMAX
`to be
`automatically modified by the system to compensate for the
`direction of the target. This is accomplished by determining
`values for O-MVT, corresponding to the orientation (direction)
`of movement of the target within the bounded area using the
`direction histogram, and I-MVT, corresponding to the intensity
`(velocity) of movement. Using these parameters, controller 42
`may modify the values of XMIN, XMAX, YMIN, and YMAX
`on a frame-by-frame basis to ensure that the target remains in
`the bounded box being searched.
`Id. at 39-40. Thus, Pirim discloses automatic updating of classification criteria
`
`based on statistical data derived from the histograms.
`
`2.
`U.S. Patent No. 5,546,125 to Tomitaka et al. (“Tomitaka”)
`48. Tomitaka discloses a “video signal follow-up processing system for
`
`adaptively tracking to the moving of a subject.” Ex. 1007, Tomitaka at Abstract.
`
`As illustrated in Figure 1, annotated below, a color video signal from optical
`
`system 1 is digitized in A/D 6, and the color of each pixel is separated into
`
`brightness (Y or L) and chroma (C) signals. Id. at 4:7-16. The chroma color signal
`
`is further demodulated into individual color difference signals R-Y and B-Y to
`
`form three color values Y, R-Y, and B-Y. Id. at 4:17-32. The color data Y, R-Y,
`
`and B-Y is converted into the HLS color coordinate system and written to image
`
`21
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`memory 15 (shaded orange) for processing. Id. at 4:39-50. Brightness and hue are
`
`then processed by two histogram units 19 and 20 (shaded blue and red) which form
`
`hue and brightness histograms, respectively. Id. at 6:1-6.
`
`
`49. Hue histogram signal S13 and brightness histogram signal S14 are sent
`
`to follow-up signal processor 16 (shaded purple). Id.at 5:42-65. The follow-up
`
`signal processor 16 forms “feature patterns” from the hue and brightness
`
`histograms that are compared with reference measurements to track an object:
`
`“they are compared with an image portion in a reference measurement frame so
`
`that the panning and tilting of the lens block 1 is adaptively controlled to always
`
`move the position of a detected measurement frame having an image with the
`
`22
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`highest similarity to the signal of the reference measurement frame.” Id. at 6:64-
`
`7:2.
`
`50. However, “when the values of components of the hue signal HUE and
`
`the components of the brightness signal Y are close to the threshold values
`
`corresponding to each sort value, the sort values to be included become uncertain
`
`depending on presence or absence of noise.” Id. at 6:14-19. To address this issue,
`
`certain pixels are prevented from being included in the Hue histogram by logic
`
`involving gate 18 (shaded green), comparator 25 (shaded yellow), and noise
`
`threshold signal S15 from the follow-up signal processor 16 (shaded purple). As
`
`seen in Figure 1, a “noise determination signal S15” is determined by the follow-
`
`up signal processor 16 and sent to comparator circuit 25 (shaded yellow), where it
`
`is compared to the saturation value. Id. at 6:40-49.
`
`When the hue signal HUE detected at the saturation/hue
`detection circuit 14 is close to the L axis (shown in FIG. 2),
`there is a possibility that the hue signal HUE may not have
`meaning as information since it is buried in noise because of
`having low saturation. Such a meaningless hue signal HUE is
`removed in the gate circuit 18.
`Id. at 6:50-55. Thus, pixels that fail the classification condition set up in
`
`comparator 25 are prevented by logic in gate 18 from being included in the HUE
`
`histogram formed by histogram unit 19. Furthermore, the classification criterion
`
`23
`
`SAMSUNG EXHIBIT 1002
`
`

`
`Declaration of Dr. John C. Hart
`U.S. Patent No. 6,959,293
`for rejecting such a pixel, represented by signal S15 from the follow-up signal
`
`processor 16, is based on histogram data inputs S13 and S14.
`
`3.
`
`Robert B. Rogers, “Real-Time Video Filtering With Bit-
`Slice Microprogrammable Processors,” Ph.D. Dissertation,
`New Mexico State University (1978) (“Rogers”)
`51. Rogers describes a system for tracking a missile or similar object by
`
`digitizing video images and analyzing the pixel intensity (brightness) using
`
`