UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`SAMSUNG ELECTRONICS CO., LTD.; and
`SAMSUNG ELECTRONICS AMERICA, INC.
`Petitioners
`
`v.
`
`IMAGE PROCESSING TECHNOLOGIES, LLC
`Patent Owner
`
`____________________
`
`Patent No. 7,650,015
`____________________
`
`DECLARATION OF DR. JOHN C. HART
`IN SUPPORT OF PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 7,650,015
`
`
`
`
`
`
`
`Page 1of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`TABLE OF CONTENTS
`
`INTRODUCTION .............................................................................................................. 1
`
`BACKGROUND AND EXPERIENCE ............................................................................. 1
`
`A.
`
`B.
`
`Qualifications .......................................................................................................... 1
`
`Previous Testimony ................................................................................................ 4
`
`TECHNOLOGICAL BACKGROUND.............................................................................. 5
`
`THE ’015 PATENT .......................................................................................................... 11
`
`SUMMARY OF OPINIONS ............................................................................................ 21
`
`LEVEL OF ORDINARY SKILL IN THE ART .............................................................. 22
`
`
`I.
`
`II.
`
`III.
`
`IV.
`
`V.
`
`VI.
`
`VII. CLAIM CONSTRUCTION .............................................................................................. 23
`
`VIII. THE PRIOR ART TEACHES OR SUGGESTS EVERY STEP AND FEATURE
`OF THE CHALLENGED CLAIMS OF THE ’015 PATENT ......................................... 23
`
`A.
`
`Overview Of The Prior Art References ................................................................ 23
`
`1.
`
`2.
`
`3.
`
`4.
`
`Alton L. Gilbert et al., A Real-Time Video Tracking System,
`PAMI-2 No. 1 IEEE Transactions on Pattern Analysis and
`Machine Intelligence 47 (Jan. 1980) (“Gilbert”) (Ex. 1005) .................... 23
`
`U.S. Patent No. 5,521,843 (“Hashima”) (Ex. 1006) ................................. 32
`
`U.S. Patent No. 5,150,432 (“Ueno”) (Ex. 1007) ....................................... 40
`
`W. B. Schaming, Adaptive Gate Multifeature Bayesian Statistical
`Tracker, 359 Applications of Digital Image Processing IV 68
`(1982) (“Schaming”) (Ex. 1008)............................................................... 45
`
`B.
`
`Ground 1: Gilbert In View Of Schaming Teaches Or Suggests Every
`Step And Feature Of Claim 6 ............................................................................ 51
`
`1.
`
`2.
`
`3.
`
`Reasons To Combine Gilbert And Schaming ....................................... 52
`
`Claim 6 ..................................................................................................... 56
`
`Detailed Application Of Gilbert And Schaming To Claim 6 ................... 69
`
`C.
`
`Ground 2: Gilbert In View Of Ueno Teaches Or Suggests Every Step
`And Feature Of Claim 6 ..................................................................................... 82
`
`
`
`i
`
`Page 2of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`Reasons To Combine Gilbert And Ueno ............................................... 82
`
`Claim 6 ..................................................................................................... 86
`
`Detailed Application Of Gilbert And Ueno To Claim 6 ......................... 100
`
`1.
`
`2.
`
`3.
`
`D.
`
`Ground 3: Hashima In View Of Schaming Teaches Or Suggests Every
`Step And Feature Of Claim 6 .......................................................................... 115
`
`1.
`
`2.
`
`3.
`
`Reasons To Combine Hashima And Schaming .................................. 115
`
`Claim 6 ................................................................................................... 117
`
`Detailed Application Of Hashima And Schaming To Claim 6 ............... 127
`
`IX.
`
`CONCLUSION ............................................................................................................... 140
`
`ii
`
`
`
`
`
`
`
`
`
`Page 3of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`I, John C. Hart, declare as follows:
`
`1.
`
`I.
`
`INTRODUCTION
`2.
`
`I have been retained by Samsung Electronics Co., Ltd. and Samsung
`
`Electronics America, Inc. (collectively, “Petitioner”) as an independent expert
`
`consultant in this proceeding before the United States Patent and Trademark Office
`
`(“PTO”).
`
`3.
`
`I have been asked to consider whether certain references teach or
`
`suggest the features recited in Claim 6 of U.S. Patent No. 7,650,015 (“the ’015
`
`Patent”) (Ex. 1001), which I understand is allegedly owned by Image Processing
`
`Technologies, LLC (“Patent Owner”). My opinions and the bases for my opinions
`
`are set forth below.
`
`4.
`
`I am being compensated at my ordinary and customary consulting rate
`
`for my work.
`
`5. My compensation is in no way contingent on the nature of my
`
`findings, the presentation of my findings in testimony, or the outcome of this or
`
`any other proceeding. I have no other interest in this proceeding.
`
`II. BACKGROUND AND EXPERIENCE
`A. Qualifications
`6.
`I have more than 25 years of experience in computer graphics and
`
`image processing technologies. In particular, I have devoted much of my career to
`
`
`
`1
`
`Page 4of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`researching and designing graphics hardware and systems for a wide range of
`
`applications.
`
`7. My research has resulted in the publication of more than 80 peer-
`
`reviewed scientific articles, and more than 50 invited papers, and talks in the area
`
`of computer graphics and image processing.
`
`8.
`
`I have authored or co-authored several publications that are directly
`
`related to target identification and tracking in image processing systems. Some
`
`recent publications include:
`
`•
`
`•
`
`•
`
`P.R. Khorrami, V.V. Le, J.C. Hart, T.S. Huang. A System for
`
`Monitoring the Engagement of Remote Online Students using Eye
`
`Gaze Estimation. Proc. IEEE ICME Workshop on Emerging
`
`Multimedia Systems and Applications, July 2014.
`
`V. Lu, I. Endres, M. Stroila and J.C. Hart. Accelerating Arrays of
`
`Linear Classifiers Using Approximate Range Queries. Proc. Winter
`
`Conference on Applications of Computer Vision, Mar. 2014.
`
`M. Kamali, E. Ofek, F. Iandola, I. Omer, J.C. Hart Linear Clutter
`
`Removal from Urban Panoramas. Proc. International Symposium on
`
`Visual Computing. Sep. 2011.
`
`9.
`
`From 2008-2012, as a Co-PI of the $18M Intel/Microsoft Universal
`
`Parallelism Computing Research Center at the University of Illinois, I led the
`
`
`
`2
`
`Page 5of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`AvaScholar project for visual processing of images that included face
`
`identification, tracking and image histograms.
`
`10.
`
`11.
`
`I am a co-inventor of U.S. Patent No. 7,365,744.
`
`I have served as the Director for Graduate Studies for the Department
`
`of Computer Science, an Associate Dean for the Graduate College, and I am
`
`currently serving as the Executive Associate Dean of the Graduate College at the
`
`University of Illinois. I am also a professor in the Department of Computer
`
`Science at the University of Illinois, where I have served on the faculty since
`
`August 2000. As a professor I have taught classes on image processing and
`
`graphics technology and have conducted research into specific applications of
`
`these technologies.
`
`12. From 1992 to 2000, I worked first as an Assistant Professor and then
`
`as an Associate Professor in the School of Electrical Engineering and Computer
`
`Science at Washington State University.
`
`13. From 1991-1992, I was a Postdoctoral Research Associate at the
`
`Electronic Visualization Laboratory at the University of Illinois at Chicago, and at
`
`the National Center for Supercomputing Applications at the University of Illinois
`
`at Urbana-Champaign.
`
`14.
`
`I earned a Doctor of Philosophy in Electrical Engineering and
`
`Computer Science from the University of Illinois at Chicago in 1991.
`
`
`
`3
`
`Page 6of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`I earned a Master’s Degree in Electrical Engineering and Computer
`
`15.
`
`Science from the University of Illinois at Chicago in 1989.
`
`16.
`
`I earned a Bachelor of Science in Computer Science from Aurora
`
`University in 1987.
`
`17.
`
`I have been an expert in the field of graphics and image processing
`
`since before 1996. I am qualified to provide an opinion as to what a person of
`
`ordinary skill in the art (“POSA”) would have understood, known, or concluded as
`
`of 1996.
`
`18. Additional qualifications are detailed in my curriculum vitae, which I
`
`understand has been submitted as Exhibit 1003 in this proceeding.
`
`B.
`19.
`
`Previous Testimony
`
`In the previous five years, I have testified as an expert at trial or by
`
`deposition or have submitted declarations in the following cases:
`
`20. Certain Computing or Graphics Systems, Components Thereof, and
`
`Vehicles Containing Same, Inv. No. 337-TA-984.
`
`21. ZiiLabs Inc., Ltd v. Samsung Electronics Co. Ltd. et al., No. 2:14-cv-
`
`00203 (E.D. Tex. Feb. 4, 2016).
`
`22. Certain Consumer Electronics with Display and Processing
`
`Capabilities, Inv. No. 337-TA-884.
`
`
`
`4
`
`Page 7of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`III. TECHNOLOGICAL BACKGROUND
`23.
`Image processing systems have long used histograms as a
`
`mathematical tool to identify and track image features and to adjust image
`
`properties. The use of histograms to identify and track image features dates back
`
`to well before 1997. See, e.g., Alton L. Gilbert et al., “A Real-Time Video
`
`Tracking System,” IEEE Transactions on Pattern Analysis and Machine
`
`Intelligence, Vol. PAMI-2, No. 2, January 1980 (Ex. 1005); D. Trier, A. K. Jain
`
`and T. Taxt, “Feature Extraction Methods for Character Recognition-A Survey”,
`
`Pattern Recognition, vol. 29, no. 4, 1996, pp. 641–662 (Ex. 1009 at 10) (citing M.
`
`H. Glauberman, “Character recognition for business machines,” Electronics, vol.
`
`29, pp. 132-136, Feb. 1956 (Ex. 1010)).
`
`24. A digital image is represented by a number of picture elements, or
`
`pixels, where each pixel has certain properties, such as brightness, color, position,
`
`velocity, etc., which may be referred to as domains. For each pixel property or
`
`domain, a histogram may be formed. A histogram is a type of bar chart used for
`
`counting the number of items meeting certain criteria. In image processing,
`
`histograms count the number of pixels in the image having certain characteristics,
`
`where each bar counts the pixels that share a specific value or value range for that
`
`characteristic. For example, if the luminance (brightness) of each pixel were
`
`represented by an 8-bit value, a luminance histogram would be a bar chart
`
`
`
`5
`
`Page 8of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`indicating, for each luminance value from 0 to 255, the count of the number of
`
`pixels in the image with that luminance value. As shown below, a luminance
`
`histogram may reveal certain properties of an image, such as whether it is properly
`
`exposed, based on whether an excessive number of pixels fall on the dark end or
`
`light end of the luminance range.
`
`
`
`25. Histograms of other pixel properties can also be formed. For
`
`example, the figure below illustrates two histograms formed by counting the
`
`number of black pixels having each X-coordinate value (i.e., the X-coordinate
`
`domain) and the number having each Y-coordinate value (i.e., the Y-coordinate
`
`domain).
`
`
`
`6
`
`Page 9of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`
`
`26. Such histograms are sometimes called “projection histograms”
`
`because they represent the image projected onto each axis. In the example above,
`
`the image was pure black and white, but projection histograms of a greyscale
`
`image can also be formed in a similar manner by defining a luminance threshold
`
`and projecting, for example, only those pixels that have a luminance value lower
`
`than 100.
`
`27. A more complex greyscale image is shown below, along with its
`
`luminance histogram (black = 0; white = 255):
`
`
`
`7
`
`Page 10of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`
`
`
`
`28. Here, the peak in the dark luminance region (luminance = 0-50)
`
`corresponds to the dark suit and tie and relatively dark background. The peak in
`
`the light luminance region (luminance > 230) corresponds to the white shirt, while
`
`the central peak (between luminance 130 and 170) corresponds to the medium
`
`brightness of the face. If one were to select only the subset of pixels with
`
`
`
`8
`
`Page 11of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`brightness between 130 and 170 and plot them according to their x and y position,
`
`one would get the following image:
`
`
`
`29. Taking projection histograms of this subset of pixels with luminance
`
`between 130 and 170, then, provides an indication of location of the face in the
`
`image. On the left, below, is a projection of this subset of pixels onto the x axis,
`
`and on the right is a similar projection onto the y axis.
`
`
`
`9
`
`Page 12of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`
`
`30. Histograms may also be formed of pixel color properties in much the
`
`same way. Color is typically represented by three values: hue, saturation and
`
`luminance. Hue (aka “tone”) is an angle ranging from 0° to 360° around a color
`
`wheel that indicates which “color” is bring represented, e.g., 0° = red, 60° =
`
`yellow, 120° = green, 180° = cyan, 240° = blue, and 300° = magenta. Saturation,
`
`which may also range from 0 to 255, represents how “brilliant” the color is. For
`
`example, if a color with a saturation of 255 represents red, then a saturation of 128
`
`would represent pink and a saturation of 0 would represent gray. Luminance
`
`ranges from 0 to 255 represent the “brightness” of the color. If luminance = 0,
`
`then the color is black, regardless of the other values. Given a color image, the
`
`luminance values of the pixels would yield the “black-and-white” or grayscale
`
`version of the image.
`
`
`
`10
`
`Page 13of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`
`
`
`
`IV. THE ’015 PATENT
`31. The ’015 Patent, entitled “Image Processing Method,” was filed on
`
`February 20, 2007, and issued on January 19, 2010. The ’015 Patent names
`
`Patrick Pirim as the sole inventor. I understand that the ’015 Patent claims a
`
`priority date of July 22, 1996.
`
`32. The ’015 Patent is generally directed to tracking a target using an
`
`image processing system. For example, in the Abstract, the ’015 Patent notes that
`
`the invention relates to “[a] method and apparatus for localizing an area in relative
`
`movement and for determining the speed and direction thereof.” Ex. 1001, ’015
`
`Patent at Abstract.
`
`33. The ’015 Patent uses the pixels in a frame of an image of a video
`
`signal to form one or more histograms in order to identify and track a target. See,
`
`
`
`11
`
`Page 14of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`e.g., Ex. 1001, ’015 Patent at Claim 6. The input signal employed in the ’015
`
`Patent is comprised of a “succession of frames, each frame having a succession of
`
`pixels.” Ex. 1001 at 3:13-16. Although the disclosed embodiments relate
`
`primarily to a video signal, the ’015 Patent also teaches that the input signal could
`
`correspond to other types of signals, for example “ultrasound, IR, Radar, tactile
`
`array, etc.” Ex. 1001, ’015 Patent at 9:10-15.
`
`34. The ’015 Patent teaches constructing a histogram showing the
`
`frequency of the pixels meeting a certain characteristic. In the ’015 Patent, these
`
`characteristics—such as luminance or speed—are referred to as “domains.”
`
`Histograms may be constructed in a variety of domains, for example, the ’015
`
`Patent teaches that examples of possible domains include pixel data such as “i)
`
`luminance, ii) speed (V), iii) oriented direction (D1), (iv) time constant (CO), v)
`
`hue, vi) saturation, and vii) first axis (x(m)), and viii) second axis (y(m)).” Id. at
`
`3:54-58.
`
`35. A domain can be further subdivided into classes, each class consisting
`
`of the subset of pixels with similar domain values. Figure 14a (and the
`
`accompanying text) illustrates an example of “classes” within a domain:
`
`
`
`12
`
`Page 15of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`
`
`FIG. 14 a shows an example of the successive classes C1
`
`C2 . . . Cn−1 Cn, each representing a particular velocity, for
`
`a hypothetical velocity histogram, with their being
`
`categorization for up to 16 velocities (15 are shown) in
`
`this example. Also shown is envelope 38, which is a
`
`smoothed representation of the histogram.
`
`Ex. 1001, ’015 Patent at 20:47-52.
`
`36. The hypothetical histogram in Figure 14a would be constructed using
`
`histogram formation block 25 in Figure 11. In this figure, various histogram
`
`processors (numbered 24-29) are shown that allow creation of histograms in
`
`various domains. Block 25 is disclosed as creating velocity histograms. Id. at
`
`17:3-9.
`
`
`
`13
`
`Page 16of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`A detailed depiction of histogram block 25 is shown in Figure 13:
`
`
`
`
`
`14
`
`Page 17of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`
`
`37.
`
`In Figure 13, velocity data for a pixel is input into a memory address
`
`and into classifier 25b. The classifier contains registers 106 that correspond to
`
`classes within the particular domain. Thus, for a classifier in a velocity histogram
`
`formation block, the classifier would have a register for each velocity class.
`
`Because the histogram will only be incremented for pixels satisfying the
`
`classification criteria that, when met, outputs a classification signal of “1,” the
`
`histogram in Figure 14a would be constructed using a classifier 25b that has each
`
`of the velocity-class registers set to “1.” In this example, each pixel that is input
`
`into classifier 25b would generate a classification signal of “1.” The histogram
`15
`
`
`
`Page 18of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`would then be updated to include the input pixel, which it would do by
`
`incrementing the histogram bin corresponding to the appropriate velocity class. In
`
`other examples, a classifier may output a classification signal of “1” for only
`
`specific classes of a domain, rather than for all of the classes in a domain as in
`
`Figure 14a. Thus, for example, the classifier could choose pixels with only
`
`specific velocities for consideration in subsequent histograms. This feature may be
`
`used in conjunction with the output from other classification units to create
`
`histograms identifying only pixels meeting multiple classification criteria in a
`
`variety of domains.
`
`38. The ’015 Patent discloses that its teachings are applicable to a broad
`
`range of applications. For example, in one embodiment, the ’015 Patent performs
`
`“automatic framing of a person . . . during a video conference.” Ex. 1001, ’015
`
`Patent at 22:4-6. In this application, histograms are constructed in the X- and Y-
`
`domains and count the number of pixels between successive frames where the
`
`differences in luminance are above certain threshold values. Ex. 1001, ’015 Patent
`
`at 22:44-54. By this method, the ’015 Patent teaches the system is able to
`
`determine the boundaries of the target based on peaks in the histograms generated.
`
`Ex. 1001, ’015 Patent at 22:55-23:22. This application and result are shown in
`
`Figure 16 and 17, reproduced below:
`
`
`
`16
`
`Page 19of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`Ex. 1001, Fig. 16
`
`
`
`Ex. 1001, Fig. 17
`
`
`
`
`39. The ’015 Patent discloses using a mounted spotlight or camera to
`
`automatically track a target, for example using a spotlight mounted on a helicopter
`
`to track a target on the ground or using automated stage lights to track a performer
`
`during a performance. Ex. 1001, ’015 Patent at 23:40-45. The ’015 Patent also
`
`teaches that “[t]he invention would similarly be applicable to weapons targeting
`
`systems.” Ex. 1001, ’015 Patent at 23:39-40.
`
`40.
`
`In each of these embodiments, the ’015 Patent finds the X- and Y-
`
`minima and maxima of the histograms and uses these points to determine a center
`
`point of the target. The patent states that the “XMIN and XMAX” for the X-projection
`
`histogram and “YMIN and YMAX” for the y projection histogram, are “key
`
`characteristics” of the histogram “which include the minimum (MIN) of the
`
`histogram” and “the maximum (MAX) of the histogram.” Id. at 19:40-44. It
`
`teaches that these key characteristics are computed by the condition:
`17
`
`
`
`Page 20of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`For each pixel with a validation signal V2 of “l”:
`
`(a) if the data value of the pixel<MIN (which is initially
`
`set to the maximum possible value of the histogram),
`
`then write data value in MIN,
`
`(b) if the data value of the pixel>MAX (which is initially
`
`set to the minimum possible value of the histogram), then
`
`write data value in MAX . . .
`
`Id. at 19:50-55. Hence the ‘015 Patent defines the minimum of the X-projection
`
`histogram is the smallest X-coordinate of any pixel in the image region whose
`
`validation signal is “1.” Similarly the maximum is the largest X-coordinate of any
`
`pixel in the image region whose validation signal is “1.” The same holds true for
`
`the Y-projection histogram where the maximum and minimum are computed in the
`
`same way, but using the y coordinate axis.
`
`41. After determining the X- and Y-minima and maxima of the
`
`histograms, the ’015 Patent teaches that a center point of the target image may be
`
`found at the coordinates of (XMIN+XMAX)/2 and (YMIN+YMAX)/2. Ex. 1001, ’015
`
`Patent at 24:46-51. This is only one possible way of computing a center point of
`
`an object. Indeed, for an irregularly shaped object like those shown in Figure 20,
`
`below, there is not one clear center point. Rather, various center points such as
`
`center of mass, center-of-area, and center of gravity might all give different
`
`
`
`18
`
`Page 21of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`coordinates, but would nevertheless accomplish the purpose of the ’015 Patent.
`
`
`
`42.
`
`In addition, the ’015 Patent teaches that as part of the identification
`
`and tracking process, the imaging system may place a tracking box around the
`
`target that “may be displayed on monitor 212, or on another monitor as desired to
`
`visually track the target.” Ex. 1001, ’015 Patent at 25:16-21. For example, Figure
`
`23 shows an example of the tracking box in a frame:
`
`
`
`19
`
`
`
`Page 22of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`43. After determining a center, the ’015 Patent teaches that the center may
`
`be used “to modify the movement of camera 13 to center face V.” Ex. 1001, ’015
`
`Patent at 23:23-27. In other words, for each frame, the system moves the direction
`
`of the camera toward the center point of the target, so that the target (face V) will
`
`be displayed at the center of the screen.
`
`FIG. 15 shows an example of use of the system of the
`
`invention to perform automatic framing of a person
`
`moving, for example, during a video conference. A
`
`video camera 13 observes the subject P, who may or may
`
`not be moving. A video signal S from the video camera
`
`is transmitted by wire, optical fiber, radio relay, or other
`
`communication means to a monitor 10b and to the image
`
`processing system of the invention 11. The image
`
`processing system determines the position and movement
`
`of the subject P, and controls servo motors 43 of camera
`
`13 to direct the optical axis of the camera towards the
`
`subject and particularly towards the face of the subject,
`
`as a function of the location, speed and direction of the
`
`subject, and may vary the zoom, focal distance and/or the
`
`focus of the camera to provide the best framing and
`
`
`
`20
`
`Page 23of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`image of the subject.
`
`Ex. 1001, ’015 Patent at 22:4-17.
`
`V.
`
`SUMMARY OF OPINIONS
`44.
`
`In preparing this declaration, I have reviewed at least the documents
`
`labeled Exhibits 1001-1012 and other materials referred to herein in connection
`
`with providing this declaration. In addition to these materials, I have relied on my
`
`education, experience, and my knowledge of practices and principles in the
`
`relevant field, e.g., image processing. My opinions have also been guided by my
`
`appreciation of how one of ordinary skill in the art would have understood the
`
`claims and specification of the ’015 Patent around the time of the alleged
`
`invention, which I have been asked to assume is the earliest claimed priority date
`
`of July 22, 1996.
`
`45. Based on my experience and expertise, it is my opinion that certain
`
`references teach or suggest all the features recited in Claim 6 of the ’015 Patent, as
`
`explained in detail below. Specifically, it is my opinion that Claim 6 is disclosed
`
`by Alton L. Gilbert et. al., A Real-Time Video Tracking System (“Gilbert”) in
`
`combination with W. B. Schaming, Adaptive Gate Multifeature Bayesian
`
`Statistical Tracker. It is also my opinion that Claim 6 are disclosed by Gilbert in
`
`combination with U.S. Patent No. 5,150,432 (“Ueno”). Finally, it is my opinion
`
`that Claim 6 are disclosed by U.S. Patent No. 5,521,843 (“Hashima”) in
`
`
`
`21
`
`Page 24of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`
`combination with Gilbert.
`
`VI. LEVEL OF ORDINARY SKILL IN THE ART
`46. Based on my review of the ’015 Patent specification, claims, file
`
`history, and prior art, I believe one of ordinary skill in the art around the time of
`
`the alleged invention of the ’015 Patent would have had either (1) a Master’s
`
`Degree in Electrical Engineering or Computer Science or the equivalent plus at
`
`least a year of experience in the field of image processing, image recognition,
`
`machine vision, or a related field or (2) a Bachelor’s Degree in Electrical
`
`Engineering or Computer Science or the equivalent plus at least three years of
`
`experience in the field of image processing, image recognition, machine vision, or
`
`a related field. Additional education could substitute for work experience and vice
`
`versa.
`
`47.
`
`In determining the level of ordinary skill in the art, I was asked to
`
`consider, for example, the type of problems encountered in the art, prior art
`
`solutions to those problems, the rapidity with which innovations are made, the
`
`sophistication of the technology, and the educational level of active workers in the
`
`field.
`
`48. My opinions concerning the ’015 Patent claims are from the
`
`perspective of a person of ordinary skill in the art (“POSA”), as set forth above.
`
`
`
`22
`
`Page 25of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`VII. CLAIM CONSTRUCTION
`49. For my analysis of the ’015 Patent, I have interpreted all claim terms
`
`according to their plain and ordinary meaning.
`
`VIII. THE PRIOR ART TEACHES OR SUGGESTS EVERY STEP AND
`FEATURE OF THE CHALLENGED CLAIMS OF THE ’015 PATENT
`A. Overview Of The Prior Art References
`1.
`Alton L. Gilbert et al., A Real-Time Video Tracking System,
`PAMI-2 No. 1 IEEE Transactions on Pattern Analysis and
`Machine Intelligence 47 (Jan. 1980) (“Gilbert”) (Ex. 1005)
`50. Gilbert arose out of efforts by researchers at U.S. Army White Sands
`
`Missile Range, New Mexico, in collaboration with New Mexico State University,
`
`Las Cruces, who developed a system that utilizes histograms to identify and track
`
`targets in flight such as rockets and aircraft. Gilbert was published in the January
`
`1980 issue of IEEE Transactions on Pattern Analysis and Machine Intelligence, a
`
`peer-reviewed journal published by the Institute of Electrical and Electronics
`
`Engineers (“IEEE”). Ex. 1011, Grenier Decl.
`
`51. Gilbert generally relates to “object identification and tracking
`
`applications of pattern recognition at video rates.” Ex. 1005, Gilbert at 47,
`
`Abstract. Gilbert discloses “a system for missile and aircraft identification and
`
`tracking . . . applied in real time to identify and track objects.” Ex.1005, Gilbert at
`
`47. It accomplishes this tracking by employing a series of processors, including an
`
`IO processor, a video processor, a projection processor, a tracker processor, and a
`
`
`
`23
`
`Page 26of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`control processor. Ex. 1005, Gilbert at 48. These elements are shown in Gilbert
`
`Figure 1:
`
`
`
`Ex. 1005, Gilbert at 48. Each of these processors performs a different role in
`
`identifying, locating, and ultimately tracking a target. Ex. 1005, Gilbert at 48.
`
`52. For example, the video processor receives a digitized video signal as
`
`input that comprises 60 frames/s. Ex. 1005, Gilbert at 48. Although Gilbert uses
`
`the word “field” instead of “frame,” a POSA would have understood that a “frame”
`
`consists of two “fields” in the context in which they are used in Gilbert. (At the
`
`time, video signals were interlaced such that a frame of a non-interlaced video
`
`consisted of two fields. The first field would be the odd numbered scanlines and
`
`the second field would be the even numbered scanlines recorded 1/60th of a second
`
`
`
`24
`
`Page 27of 143
`
`SAMSUNG EXHIBIT 1002
`Samsung v. Image Processing Techs.
`
`

`
`Declaration of Dr. John C. Hart
`Inter Partes Review of U.S. Patent No. 7,650,015
`later than the first field.) These frames each consist of a succession of pixels,
`
`arranged in a matrix of n X m pixels. Ex. 1005, Gilbert at 48. For each frame, the
`
`video processor creates a pixel intensity histogram across 256 gray levels. Id.
`
`These 256 gray levels represent the plurality of classes across which the histogram
`
`is formed. In other words, the Video Processor of Gilbert creates histograms using
`
`the luminance domain over classes of all luminance values. Id. The video
`
`processor then uses these histograms and a probability estimate to determine
`
`whether a p