U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`______________________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`______________________________________________
`
`
`
`
`LG ELECTRONICS INC. and LG ELECTRONICS USA, INC.,
`Petitioners
`
`v.
`
`IMAGE PROCESSING TECHNOLOGIES, LLC,
`Patent Owner.
`
`
`
`
`DECLARATION OF DR. JEFFREY J. RODRIGUEZ IN SUPPORT OF
`PETITION FOR INTER PARTES REVIEW OF U.S. PATENT NO. 6,959,293
`
`
`
`Petitioner LG Ex-1002, 0001
`
`

`

`
`
`TABLE OF CONTENTS
`
`
`I.
`INTRODUCTION .......................................................................................... 1
`BACKGROUND AND QUALIFICATIONS ................................................ 1
`II.
`III. MATERIALS CONSIDERED ....................................................................... 5
`IV. RELEVANT LEGAL STANDARDS ............................................................ 7
`A.
`Claim Interpretation ............................................................................. 7
`B.
`Person of Ordinary Skill in the Art ...................................................... 7
`C.
`Obviousness .......................................................................................... 8
`LEVEL OF ORDINARY SKILL IN THE ART .......................................... 11
`V.
`VI. TECHNOLOGY BACKGROUND .............................................................. 11
`VII. SUMMARY OF MY OPINIONS ................................................................ 15
`VIII. OVERVIEW OF THE ’293 PATENT ......................................................... 15
`IX. OVERVIEW OF THE PRIOR ART ............................................................ 19
`A. Overview of Pirim .............................................................................. 19
`1.
`Pirim’s Generic Image Processing System .............................. 19
`2.
`Documents Incorporated By Reference In Pirim ..................... 24
`Overview of Bolle .............................................................................. 27
`B.
`X. DATE OF INVENTION .............................................................................. 31
`XI. CLAIM CONSTRUCTION ......................................................................... 31
`A.
`“histogram” ........................................................................................ 31
`B.
`“the [at least two] histogram calculation units being configured
`to form a histogram representative of the parameter” ........................ 32
`“wherein the validation signal is produced from time
`coincidences signals from the time coincidence bus so that the
`calculation of the histogram depends on the classification
`signals carried by the time coincidence bus” ..................................... 32
`
`
`C.
`
`
`
`
`
`
`
`i
`
`
`
`Petitioner LG Ex-1002, 0002
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`
`XII. DETAILED EXPLANATION OF UNPATENTABILITY
`GROUNDS ................................................................................................... 33
`A. Grounds 1 and 2: Claim 1 is anticipated and/or rendered
`obvious by Pirim ................................................................................ 33
`1.
`Pirim discloses two anticipatory embodiments (Ground
`1), or, at a minimum, two embodiments that each render
`Claim 1 obvious (Ground 2) .................................................... 33
`Claim 1 ..................................................................................... 34
`2.
`Ground 3: Claim 1 is Rendered Obvious by Pirim in View of
`Bolle ................................................................................................... 67
`1. Motivations to Combine Pirim and Bolle ................................ 67
`2.
`Claim 1 ..................................................................................... 75
`XIII. CONCLUSION ............................................................................................. 88
`
`
`B.
`
`
`
`
`
`ii
`
`
`
`Petitioner LG Ex-1002, 0003
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`I, Jeffrey J. Rodriguez, declare as follows.
`
`1.
`
`I.
`
`INTRODUCTION
`I have been retained by LG Electronics Inc. and LG Electronics USA,
`2.
`
`Inc. (“Petitioner”) as an independent expert consultant in this inter partes review
`
`(“IPR”) proceeding before the United States Patent and Trademark Office
`
`(“PTO”).
`
`3.
`
`I have been asked by LG’s counsel (“Counsel”) to consider whether
`
`certain references disclose, teach, and/or suggest the features recited in Claim 1 of
`
`U.S. Patent No. 6,959,293 (the “’293 Patent”) (Ex-1001), assigned to Image
`
`Processing Technologies, Inc. (“Patent Owner”). My opinions and the bases for
`
`my opinions are set forth below.
`
`4.
`
`I am being compensated at my ordinary and customary consulting rate
`
`for my work, which is $575 per hour. My compensation is in no way contingent
`
`on the nature of my findings, the presentation of my findings in testimony, or the
`
`outcome of this or any other proceeding. I have no other financial interest in this
`
`proceeding.
`
`II. BACKGROUND AND QUALIFICATIONS
`I am a professor at The University of Arizona in the Department of
`5.
`
`Electrical and Computer Engineering, where I hold or have held the following
`
`positions: (a) Associate Professor of Electrical and Computer Engineering, with
`
`1
`
`Petitioner LG Ex-1002, 0004
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`tenure (1997–present), (b) Associate Professor of Biomedical Engineering, with
`
`tenure (2017–present), (c) Director of the Signal and Image Laboratory (1990–
`
`present), (d) Director of Image Analysis, Cancer Imaging Shared Services, Arizona
`
`Cancer Center (2009–2014), (e) Co-Director of Connection One, a National
`
`Science Foundation industry/university cooperative research center for wired and
`
`wireless communication circuits and systems, (f) Director of Graduate Studies for
`
`the Department of Electrical and Computer Engineering (2000–2003, 2005–2016).
`
`6. My formal education includes a Bachelor of Science degree in
`
`Electrical Engineering from the University of Texas at Austin in May 1984, a
`
`Master of Science degree in Electrical Engineering from the Massachusetts
`
`Institute of Technology in June 1986, and a Ph.D. Degree in Electrical Engineering
`
`from the University of Texas at Austin in May 1990.
`
`7.
`
`I teach courses at both the graduate and undergraduate level through
`
`the Dept. of Electrical and Computer Engineering and the College of Optical
`
`Sciences. The courses I have taught include Digital Image Processing, Digital
`
`Image Analysis, Digital Signal Processing, Advanced Digital Signal Processing,
`
`Signals and Systems, and Circuit Analysis. I have been awarded the Outstanding
`
`Teaching Award by the IEEE and Eta Kappa Nu (IEEE-HKN), given by students
`
`at The University of Arizona to one outstanding professor each year.
`
`2
`
`Petitioner LG Ex-1002, 0005
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`8. My research activity is generally directed to systems for digital
`
`signal/image/video processing and analysis. My current research activities include
`
`automated image analysis of ultrasound thyroid images, analysis of uncertainty in
`
`machine-learning systems, automated image analysis of sweat pore activation,
`
`deep-learning classification of chest X-ray images, convolutional neural networks
`
`for driver distraction detection, and automated algorithms for object detection and
`
`tracking in video. My research has also included content-adaptive improved error
`
`concealment methods for H.264/AVC video communication.
`
`9.
`
`One of my recent research projects involved the design and
`
`development of a real-time image and video processing system for automated
`
`detection and behavioral analysis of zebrafish for use in ototoxicity assessment of
`
`drugs. The system we designed and built includes an array of Raspberry Pi
`
`microcomputer systems configured for parallel video capture of sixteen zebrafish
`
`populations. Each Raspberry Pi features a system on a chip, which includes a
`
`CPU, a video graphics processing unit (GPU), a disk storage system, and a
`
`memory system. Our zebrafish analysis system automatically captures and
`
`transmits compressed MPEG video data to a local personal computer or to a high-
`
`performance computing cluster to decode the video and implement customized
`
`algorithms for detection of zebrafish pose and motion, resulting in automated
`
`assessment of zebrafish swimming behavior.
`
`3
`
`Petitioner LG Ex-1002, 0006
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`I am a Senior Member of the Institute of Electrical and Electronics
`
`10.
`
`Engineers (IEEE) and the IEEE Signal Processing Society. I served as Treasurer
`
`of the 2020 IEEE Southwest Symposium on Image Analysis and Interpretation
`
`(SSIAI), General Chair of SSIAI 2016, and General Chair of the 2007 IEEE
`
`International Conference on Image Processing (ICIP). In addition, I served on the
`
`IEEE Signal Processing Society Technical Committee on Image, Video, and
`
`Multidimensional Signal Processing (2005–2011) and served as Associate Editor
`
`of the IEEE Transactions on Image Processing (1996–2000). Over the years, I
`
`have served on numerous other professional committees, and I have served as a
`
`technical reviewer for numerous journals and professional conferences.
`
`11.
`
`I am named as an inventor on a patent involving polar dynamic
`
`programing for automated image segmentation. I have authored or co-authored
`
`over 140 technical publications. Over the past 30 years, I have trained hundreds of
`
`students in signal/image/video processing, including directing the research of
`
`numerous graduate students, resulting in more than 50 graduate theses and
`
`dissertations. I have served as an expert in patent cases since 2005, involving
`
`signal/image/video processing, data compression systems, video receivers,
`
`document cameras, cellular camera phones, signal/image/video transmission over
`
`wireless networks, etc.
`
`4
`
`Petitioner LG Ex-1002, 0007
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`12. My current curriculum vitae is attached as Exhibit A, which includes a
`
`detailed listing of my education, work experience, honors, awards, professional
`
`associations, publications, and a list of expert consulting activities in which I have
`
`testified as an expert at deposition or trial.
`
`III. MATERIALS CONSIDERED
`In forming my opinions, I have relied on my own knowledge and
`13.
`
`experience, including my education, training, and work experience in the field of
`
`electrical and computer engineering, my research and teaching experience, my
`
`experience in working with others in this field, and my experience in the design,
`
`development, and operation of relevant systems. I have considered the following
`
`materials and any other materials described or cited in this declaration:
`
`Ex-1001
`Ex-1003
`Ex-1004
`
`Ex-1005
`
`Ex-1006
`
`Ex-1007
`
`Ex-1008
`
`U.S. Patent No. 6,959,293
`Curriculum Vitae of Dr. Jeffrey Rodriguez
`Prosecution History of the ‘293 Patent
`Samsung Elecs. Co., Ltd. v. Image Processing Technologies LLC,
`Case No. IPR2017-00336, Paper 38 (May 9, 2018) (Final Written
`Decision)
`Samsung Electronics Co., Ltd. v. Image Processing Techs. LLC,
`Case IPR2017-01189, Paper 9 (P.T.A.B. August 18, 2017)
`(Institution Decision).
`Ex parte Image Processing Techs, LLC, Reexamination Control
`No. 90/014,056 (P.T.A.B. Aug. 29, 2019) (Decision on Appeal).
`Ex parte Image Processing Techs, LLC, Reexamination Control
`No. 90/014,056 (U.S.P.T.O. June 26, 2018) (Declaration of Alan
`Bovik).
`
`5
`
`Petitioner LG Ex-1002, 0008
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`Ex-1010
`
`Ex-1011
`
`Ex-1018
`
`Ex-1019
`
`Ex-1021
`
`Ex-1022
`
`Ex-1023
`
`Ex-1024
`
`Ex-1025
`
`Ex-1028
`
`Image Processing Techs., LLC v. Samsung Electronics Co., Ltd.,
`No. 2:20-cv-00050, Dkt. 192 (E.D. Tex. June 26, 2020) (Order
`Regarding Prosecution History Estoppel)
`Image Processing Techs., LLC v. Samsung Electronics Co., Ltd.,
`No. 2:16-cv-505, Dkt. 174 (E.D. Tex. June 21, 2017) (Claim
`Construction Order)
`WIPO Patent Publication WO 99/36893 (“Pirim”) [cites to Ex-
`1018 refer to the original page number at the bottom center of the
`page]
`U.S. Patent No. 5,546,475 (“Bolle”)
`Publication of International Application No. PCT/EP98/05383 (an
`international application incorporated by reference in Pirim)
`Annotated Claim 1 of the ’293 Patent
`European Patent Office File history for Pirim (including
`PCT/EP98/05383)
`Translation of PCT/FR97/01354 (document incorporated by
`reference in Pirim) [cites to Ex-1024 refer to the pdf page number]
`Original, French version of PCT/FR97/01354, translation, and
`supporting affidavit
`Certified WIPO file history for Pirim
`
`14.
`
`I reserve the right to supplement, amend, or modify the opinions set
`
`forth in this declaration or in any other declaration, report, witness statement,
`
`deposition, or in testimony based on any new contentions or other information that
`
`may emerge in this proceeding, including any other documents, information,
`
`materials, or testimony.
`
`6
`
`Petitioner LG Ex-1002, 0009
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`IV. RELEVANT LEGAL STANDARDS
`I am not an attorney and offer no legal opinions. For the purposes of
`15.
`
`this Declaration, I have been informed about certain aspects of the law that are
`
`relevant to my analysis, as summarized below.
`
`A. Claim Interpretation
`I have been informed and understand that in an IPR proceeding,
`16.
`
`claims are to be interpreted according to the Phillips claim construction standard. I
`
`have been informed and understand that claim construction is a matter of law and
`
`that the final claim constructions for this proceeding will be determined by the
`
`Patent Trial and Appeal Board (“PTAB”).
`
`17.
`
`In this Declaration, I have been asked to apply the constructions that
`
`were previously set by the Patent Office and a district court. See infra, Part XI.
`
`B.
`18.
`
`Person of Ordinary Skill in the Art
`I have been informed and understand that a patent is to be understood
`
`from the perspective of a hypothetical “person of ordinary skill in the art”
`
`(“POSITA”) as of the time of the claimed invention. Such an individual is
`
`considered to possess normal skills and knowledge in a particular technical field
`
`(as opposed to being a genius). I understand that in considering what the claims of
`
`a patent require, what was known prior to that patent, what a prior art reference
`
`7
`
`Petitioner LG Ex-1002, 0010
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`discloses, and whether an invention is obvious or not, one must use the perspective
`
`of such a POSITA.
`
`C. Obviousness
`I have been informed and understand that a patent claim is obvious
`19.
`
`under 35 U.S.C. §103, and therefore invalid, if the claimed subject matter, as a
`
`whole, would have been obvious to a POSITA as of the date of the claimed
`
`invention of the patent based on one or more prior art references and/or the
`
`knowledge of a POSITA.
`
`20.
`
`I understand that an obviousness analysis must consider (1) the scope
`
`and content of the prior art, (2) the differences between the claims and the prior art,
`
`(3) the level of ordinary skill in the pertinent art, and (4) secondary considerations,
`
`if any, of non-obviousness (such as unexpected results, commercial success, long-
`
`felt but unmet need, failure of others, copying by others, and skepticism of
`
`experts).
`
`21.
`
`I understand that a prior art reference may be combined with other
`
`references to disclose each element of the invention under 35 U.S.C. §103. I
`
`understand that a reference may also be combined with the knowledge of a
`
`POSITA, and that this knowledge may be used to combine multiple references. I
`
`further understand that a POSITA is presumed to know the relevant prior art. I
`
`8
`
`Petitioner LG Ex-1002, 0011
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`understand that the obviousness analysis may take into account the inferences and
`
`creative steps that a POSITA would employ.
`
`22.
`
`In determining whether a prior art reference would have been
`
`combined with other prior art or other information known to a POSITA, I
`
`understand that the following principles may be considered:
`
`a. whether the references to be combined involve non-analogous art;
`
`b. whether the references to be combined are in different fields of endeavor
`
`than the alleged invention in the Patent;
`
`c. whether the references to be combined are reasonably pertinent to the
`
`problems to which the inventions of the Patent are directed;
`
`d. whether the combination is of familiar elements according to known
`
`methods that yields predictable results;
`
`e. whether a combination involves the substitution of one known element for
`
`another that yields predictable results;
`
`f. whether the combination involves the use of a known technique to
`
`improve similar items or methods in the same way that yields predictable results;
`
`g. whether the combination involves the application of a known technique to
`
`a prior art reference that is ready for improvement, to yield predictable results;
`
`h. whether the combination is “obvious to try”;
`
`i. whether the combination involves the known work in one field of
`
`9
`
`Petitioner LG Ex-1002, 0012
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`endeavor prompting variations of it for use in either the same field or a different
`
`one based on design incentives or other market forces, where the variations are
`
`predictable to a POSITA;
`
`j. whether there is some teaching, suggestion, or motivation in the prior art
`
`that would have led one of ordinary skill in the art to modify the prior art reference
`
`or to combine prior art reference teachings to arrive at the claimed invention;
`
`k. whether the combination requires modifications that render the prior art
`
`unsatisfactory for its intended use;
`
`l. whether the combination requires modifications that change the principle
`
`of operation of the reference;
`
`m. whether the combination is reasonably expected to be a success; and
`
`n. whether the combination possesses the requisite degree of predictability at
`
`the time the invention was made.
`
`23.
`
`I understand that in determining whether a combination of prior art
`
`references renders a claim obvious, it is helpful to consider whether there is some
`
`teaching, suggestion, or motivation to combine the references and a reasonable
`
`expectation of success in doing so. I understand, however, that a teaching,
`
`suggestion, or motivation to combine is not required.
`
`10
`
`Petitioner LG Ex-1002, 0013
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`V. LEVEL OF ORDINARY SKILL IN THE ART
`24. A person of ordinary skill in the art (“POSITA”) at the time of the
`
`alleged invention of the ’293 Patent would have had either (1) a master’s degree in
`
`Electrical Engineering or Computer Engineering or the equivalent plus at least a
`
`year of experience in the field of image processing, image recognition, machine
`
`vision, or a related field or (2) a Bachelor’s Degree in Electrical Engineering or
`
`Computer Engineering or the equivalent plus at least three years of experience in
`
`the field of image processing, image recognition, machine vision, or a related field.
`
`Additional education could offset a lesser amount of experience and vice versa.
`
`VI. TECHNOLOGY BACKGROUND
`Images, such as frames in a video, can by represented as a set of
`25.
`
`“pixels,” which is a shorthand term for “picture element.” When a camera captures
`
`an image digitally, it stores certain information for each pixel, such as luminance,
`
`color, position, etc. For example, the below image of the number “5” has seven
`
`rows and five columns of pixels, each pixel having a square shape:
`
`11
`
`Petitioner LG Ex-1002, 0014
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`
`
`26.
`
`It has long been known, since the 1980s and before, that one way to
`
`analyze the pixels of an image is to use a statistical tool called a histogram. See,
`
`e.g., U.S. Patent No. 4,353,092 at 1:5–56. Histograms are useful for numerous
`
`purposes. One example is contrast enhancement. For example, if there are many
`
`dark pixels in an image, the histogram of the image will show an unbalanced
`
`distribution of pixel values (left image in the figure below). The contrast can be
`
`enhanced via histogram modification to make the image more useful to a viewer
`
`(right image in the figure below). The figure below shows histograms of the
`
`brightness or greyscale of a grayscale image. The left histogram shows unbalanced
`
`intensities (related to brightness) concentrated in the middle of the grayscale range,
`
`while the right histogram shows more evenly distributed intensities after
`
`enhancement.
`
`12
`
`Petitioner LG Ex-1002, 0015
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`
`
`Source: https://www.mathworks.com/help/images/contrast-adjustment.html
`
`27. Histograms can be formed of any parameter. One common parameter
`
`is hue. For example, the image of a cup, below left, can be represented with the
`
`hue histogram shown in the middle below. The horizontal axis shows a range of
`
`hue values (which can be thought of as different colors), and the vertical axis
`
`counts the number of pixels having each hue value. The histogram has two peaks
`
`that generally indicate a large number of pixels in shades of red and brown.
`
`
`
`Source: https://scikit-
`
`image.org/docs/stable/auto_examples/color_exposure/plot_rgb_to_hsv.html
`
`28. The red dashed line in the hue histogram is a threshold. Excluding
`
`pixels with colors below the threshold results in the simplified image on the right.
`
`13
`
`Petitioner LG Ex-1002, 0016
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`This process, called segmentation, helps a computer to distinguish the object (a cup
`
`and saucer) from the background.
`
`29. Another common example of a histogram is a histogram of x position
`
`or y position, which is called a projection histogram. Here is a picture of a sample
`
`image, its x-position histogram, and its y-position histogram:
`
`
`
`Image
`
`
`
` x position histogram
`
`
`
`y position histogram
`
`30. As can be seen in this example, the x position histogram has five
`
`values on the horizontal axis, which correspond to each x position in the image.
`
`For example, we can label the columns of the image as x = 1 through x = 5, and we
`
`can label the rows as y = 1 through y = 7. For example, we can label the left
`
`column as x = 1. Since there are five black pixels in that column of the image, the
`
`x position histogram shows a count of 5 for x = 1.
`
`31. For a univariate histogram, the variable on the horizontal axis of the
`
`histogram is sometimes called a domain or parameter. For a univariate histogram,
`
`14
`
`Petitioner LG Ex-1002, 0017
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`the vertical axis of a histogram shows a count of how many pixels in the image
`
`have each value of the parameter.
`
`VII. SUMMARY OF MY OPINIONS
`32. As explained below in detail in this Declaration, it is my opinion that:
`
`Ground 1: Claim 1 of the ’293 Patent is anticipated by Pirim.
`
`Ground 2: Claim 1 of the ’293 Patent is rendered obvious by Pirim.
`
`Ground 3: Claim 1 of the ’293 Patent is rendered obvious by the
`
`combination of Pirim and Bolle.
`
`VIII. OVERVIEW OF THE ’293 PATENT
`33. The ’293 Patent was filed on February 23, 2001, naming Patrick Pirim
`
`as the inventor. ’293 Patent (Ex-1001), Cover. It is directed to “methods and
`
`devices for processing image signals using ...histogram calculation units.” Id.,
`
`1:6–14. It shows an example histogram in Figure 13c:
`
`
`
`15
`
`Petitioner LG Ex-1002, 0018
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`34. A histogram is a common “statistical distribution” used to show how
`
`data varies across the range of possible values for a parameter. Id., 32:1–3. The
`
`’293 Patent explains that a “histogram” can be formed of various “parameters” of
`
`the pixels in a frame, such as “tone (hue), saturation and luminance signals, as well
`
`as the velocity, direction, and intensity signals.” Id., 19:35–41, 24:18–20. The
`
`horizontal axis shows the range of possible values of the parameter, and the
`
`vertical axis shows a count of how many pixels in the image have each value of the
`
`parameter.
`
`35. The ’293 Patent claims a device for “detecting an event” that could be
`
`“aural and/or visual phenomena” using “two histogram calculation units.” Id.,
`
`Claims 1, 18, 22. A histogram is formed using a histogram block (referred to as a
`
`“histogram calculation unit” in the ’293 patent) shown in Figure 3:
`
`16
`
`Petitioner LG Ex-1002, 0019
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`
`
`Ex-1001, Fig. 3, 8:37–43.1 The histogram block includes a “memory 100” (red)
`
`for storing a histogram, which is a “conventional ...digital memory” with
`
`“addresses ...equal to the number of possible levels for the values of the parameter
`
`A that must be discriminated,” where each address “stores at least number of pixels
`
`contained in a frame (i.e., in an image).” Id., 8:37–52.
`
`36.
`
`“Classifier unit 101 includes a register 101r capable of storing certain
`
`possible level values ...for the levels of parameter A” so that it can “sort[] the
`
`pixels, and provide, on its output 101s, a value 1 when the parameter A associated
`
`
`1 All coloring, highlighting, and annotations in figures, as well as all bold and italic
`emphasis applied to text, in this Declaration is added unless otherwise indicated.
`
`17
`
`Petitioner LG Ex-1002, 0020
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`with the said pixel has a level corresponding to that contained in the register 101r
`
`and the zero value conversely.” Id., 9:27–35. The classifier outputs a
`
`classification signal to the time coincidences bus 111 (yellow in Fig. 3, above).
`
`Id., 9:35–41.
`
`37. The ’293 Patent contemplates that multiple histogram blocks could be
`
`used, each evaluating a different parameter for classification. See id., Fig. 3
`
`(showing time coincidences unit 102 receiving classification inputs inE, ...inC,
`
`inB, inA from histogram units E, ...C, B, and A); 9:36–50. All of the classification
`
`signals can be output to a common time coincidence bus 111 (yellow in Figure 3,
`
`above), so that all time coincidence units (purple) can consider the results of the
`
`classifications. Id.
`
`38. The time coincidences unit 102 (purple in Fig. 3, above) “compares
`
`the values” received from various classifiers to values programmed into its
`
`“register 102r” and “transmits on its output 102s, for each pixel, an enabling signal
`
`equal to 1 when there is a coincidence between the register values equal to 1 and
`
`the corresponding data received from the bus 111, and a zero value in the reverse
`
`case.” Id., 9:36–50. If the time coincidences unit outputs an enabling signal equal
`
`to 1, the memory (red in Fig. 3, above) is incremented by 1 at a memory address
`
`corresponding to the value aij of the parameter. Id., 8:53–64.
`
`18
`
`Petitioner LG Ex-1002, 0021
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`IX. OVERVIEW OF THE PRIOR ART
`A. Overview of Pirim
`Pirim’s Generic Image Processing System
`1.
`International patent application having international publication no.
`
`39.
`
`WO 99/36893 (“Pirim”), titled “Method and Apparatus for Detection of
`
`Drowsiness,” has named inventors Patrick Pirim and Thomas Binford and
`
`identifies an international publication date of July 22, 1999.
`
`40. Pirim discloses a “generic image processing system” that includes “a
`
`video camera” that “monitors a scene.” Ex-1018, 10. The video camera signal,
`
`S(PI) is shown in Figure 11 below, entering Pirim’s system on the left. Figure 11
`
`includes “histogram processor 22a” (also less frequently referred to as a
`
`“Histogram Formation Unit”) which is shown in a magnified detail view in Figure
`
`12. Id., 24–25.
`
`19
`
`Petitioner LG Ex-1002, 0022
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`
`
`
`SPATIAL
`UT SToGRA
`Arvo
`
`
`Fotmato
`Ter Peetar
` Ot
`Pec cessioe
`VmT
`
`
`
`Ex-1018, Figs. 11 and 12. Figure 12 is a generalized, configurable histogram
`
`processor 22a, and it showssix histogram blocks (elements 24—29) (also referred to
`
`as “histogram formation blocks”in Pirim) for parameters including luminance SR,
`
`velocity or speed V, direction of displacement DI, time constant CO,x position,
`
`and y position. Jd., 7, 12, 14, 16, 25. “The function of each histogram blockis to
`
`enable a histogram to be formed for the domain associated with that block.” Jd.,
`
`25.
`
`41.
`
`Pirim’s embodiments disclose specific configurations for histogram
`
`processor 22a (Figure 12) that modify Figure 12 to include histogram units for the
`
`shown parameters and other parameters. Jd., 47, 53 (disclosing using parameters
`
`20
`
`Petitioner LG Ex-1002, 0023
`
`Petitioner LG Ex-1002, 0023
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`including “CO, DP, velocity, direction, luminance, hue, and saturation”). Thus,
`
`Figure 12 is not fixed—it is highly configurable; Pirim, including its incorporated
`
`document, discloses numerous possible configurations. Id., 42–57 (disclosing
`
`various configurations for detecting a driver entering a vehicle and detecting
`
`drowsiness); Ex-1021, 33–34 (disclosing various configurations for detecting
`
`vehicles, a person’s head, a ground target, and performers on a stage); Ex-1001,
`
`10:36–40 (disclosing “extrapolation to any number of [histogram] units is
`
`evident”). Further, in incorporated document PCT/EP98/05383, Pirim discloses
`
`using two copies of Figure 12 (which appears in PCT/EP98/05383 labeled as
`
`Figure 11) to track (A) multiple targets or (B) a single target in the event one of the
`
`processors loses tracking. Ex-1018, 10; Ex-1021, 37:7–21.
`
`42. Each histogram block forms a histogram using the circuitry shown in
`
`Figure 14 (which is essentially the same as the histogram block shown in Figure 3
`
`of the ’293 Patent):
`
`21
`
`Petitioner LG Ex-1002, 0024
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`
`
`
`Ex-1018, Fig. 14. Pirim explains that Figure 14 shows an example
`
`“histogram…block” that can be connected to other histogram blocks, each of
`
`which is “identical to the others and functions in the same manner.” Id., 26.
`
`43. For clarity, this Declaration refers to element 22a as a “histogram
`
`processor” and refers to an individual one of elements 24–29 within the histogram
`
`processor as a “histogram block.”
`
`44. The operation of the various components of the histogram block are
`
`materially the same as in the ’293 Patent and not repeated here. Compare Ex-
`
`1001, 7:54–64, 8:37–52, 9:27–50, Fig. 3 with Ex-1018, 24–29, Figs. 2, 4, 11, 12,
`
`14. The ’293 Patent adopts new terminology to refer to some of the preexisting
`
`components. In the ’293 Patent, the applicant switched to referring to the
`
`validation unit as a “time coincidences unit.” Ex-1001, 9:36–41, Fig. 3; see also
`
`22
`
`Petitioner LG Ex-1002, 0025
`
`

`

`U.S. Patent No. 6,959,293
`Declaration of Dr. Jeffrey J. Rodriguez
`Ex-1018, 29. And the ’293 Patent interchangeably refers to the output of the
`
`classifier (which is the input to the validation unit) as either a classification signal
`
`or as a time coincidences signal. Id., 2:22–28, 21:9, 9:27–41, Fig. 3, Claim 1; see
`
`also Ex-1018, 28–29.
`
`45. Pirim explains that generally the purpose of the classifier is to select
`
`pixels that meet specified criteria. Ex-1018, 27–29. For example, if a given
`
`histogram formation block is processing the speed parameter, the classifier could
`
`be programmed to output a signal of “1” if the pixel has a speed value in the
`
`desired range, and “0” otherwise. Id. Other histogram formation blocks similarly
`
`classify other parameters such as hue, luminance, direction, etc. Id. The resulting
`
`classification signals are transferred to the validation unit of each other histogram
`
`formation block via bus 23. Id. The validation unit of each histogram formation
`
`block can then c