UNITED STATES PATENT AND TRADEMARK OFFICE
`
`________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`________________
`
`UNIFIED PATENTS INC.,
`Petitioner,
`
`v.
`
`ADVANCED SILICON TECHNOLOGIES LLC
`Patent Owner
`________________
`
`Case IPR2016-01060
`U.S. Patent No. 8,933,945 B2
`________________
`
`DECLARATION OF JOHN C. HART, PH.D. IN SUPPORT OF PATENT
`OWNER ADVANCED SILICON TECHNOLOGIES LLC’S
`PRELIMINARY RESPONSE
`
`Case IPR2016-01060
`Unified Patents Inc. v.
`
` Advanced Silicon Technologies LLC
`
`AST Exhibit 2001
`
`

`
`I.
`
`INTRODUCTION
`
`1.
`
`I have been retained by counsel for Patent Owner Advanced Silicon
`
`Technologies LLC (“Patent Owner” or “AST”) as an expert to provide this
`
`declaration and certain opinions regarding claims 1-3, 9, 10 and 21 of United
`
`States Patent No. 8,933,945 (“the ‘945 Patent”). More specifically, I have been
`
`asked to form opinions on whether Claims 1-3, 9, 10 and 21 of the ‘945 patent are
`
`rendered obvious by certain combinations of prior art references, including
`
`European Patent Application No. EP 1 195 717 to Seiler et al. (“Seiler”), U.S.
`
`Patent No. 6,864,896 to Perego (“Perego”), and U.S. Patent No. 5,757,385 to
`
`Narayanaswami et al. (“Narayanaswami”). All of the opinions and conclusions
`
`found in this declaration are my own and I have personal knowledge of all the facts
`
`set forth herein. If called to testify, I would competently testify and verify that
`
`contained herein.
`
`2.
`
`I am being compensated at my hourly rate of $550. I am also being
`
`separately reimbursed for out of pocket expenses. My compensation does not
`
`depend in any way on the outcome of these cases or the particular testimony or
`
`opinions I express.
`
`3.
`
`I understand that Unified Patents Inc. (“Unified”) has filed a petition
`
`in this proceeding contending claims 1-3, 9, 10 and 21 of the ’945 Patent are
`
`2
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`obvious over certain multi-reference combinations of Seiler, Perego and
`
`Narayanaswami.
`
`II.
`
`EXPERT QUALIFICATIONS
`4.
`
`I have been a professor of computer graphics since 1992. I am
`
`currently a tenured Professor in the Department of Computer Science at the
`
`University of Illinois at Urbana-Champaign, a department consistently ranked in
`
`the top-5 by US News and World Report. I am also the Executive Associate Dean
`
`of the Graduate College of the University of Illinois at Urbana-Champaign.
`
`5.
`
`I received a Bachelor’s of Science degree in Computer Science from
`
`Aurora University in 1987, a Master’s of Science in Electrical Engineering and
`
`Computer Science from the University of Illinois at Chicago in 1989, and a Ph.D.
`
`in Electrical Engineering and Computer Science from the University of Illinois at
`
`Chicago in 1991.
`
`6.
`
`I worked as an intern in 1998 at the IBM TJ Watson Research Center
`
`in Hawthorn, New York, and in 1999 at AT&T Pixel Machines in Holmdel, New
`
`Jersey. I worked as a postdoc in 1991-1992 at the Electronic Visualization
`
`Laboratory at the University of Illinois at Chicago with funding from the National
`
`Center for Supercomputing Applications at the University of Illinois at Urbana-
`
`Champaign. I worked as a visiting researcher in the summer of 2007 for the
`
`Graphics Research Group of Adobe Systems in Seattle, Washington.
`
`3
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`7.
`
`From 1997-2000, I designed, simulated and patented graphics
`
`hardware for Silicon Reality and the Evans & Sutherland Computer Corp. I have
`
`also consulted for visual effects companies (e.g., Kleiser-Walczak, Blue Sky
`
`VIFX), defense contractors (e.g., SAIC, Pratt & Whitney) and medical imaging
`
`companies (e.g., Intrinsic Medical Imaging).
`
`8. My research in computer graphics has been supported by Adobe,
`
`DARPA, Intel, Nokia, NVIDIA, NSF and Microsoft. I have published over 75
`
`reviewed papers on computer graphics, including papers in the most rigorous and
`
`prestigious venues, including the SIGGRAPH Annual Conference Proceedings, the
`
`ACM Transactions on Graphics, and the IEEE Transactions on Visualization and
`
`Computer Graphics. From 2002-2008, I was the longest-serving Editor-in-Chief of
`
`ACM Transactions on Graphics, the top journal in the field of computer graphics.
`
`9.
`
`I was an Executive Producer for the ACM SIGGRAPH documentary
`
`“The Story of Computer Graphics” in 1999. I served from 1996-2000 on the ACM
`
`SIGGRAPH Executive Committee. I am currently the computer graphics area
`
`editor of the ACM Books series, and oversaw the recent publication of “The VR
`
`Book: Human Centered Design for Virtual Reality.”
`
`10.
`
`I co-authored the book “Real-Time Shading” in 2002 on the
`
`implementation of procedural shaders on graphics processors became they became
`
`fully user programmable GPU’s. I contributed two chapters to the book “Modeling
`
`4
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`and Texturing: A Procedural Approach” in 2002, including a chapter on procedural
`
`texturing using rasterization and texturing operations.
`
`11. My CV is attached as Exhibit A.
`
`III. MATERIALS CONSIDERED
`12.
`
`In forming the opinions set forth in this declaration, I have considered
`
`and relied upon my education, knowledge of the relevant field, and experience.
`
`The materials I have reviewed and considered include Unified’s Petition, the ‘945
`
`Patent, its prosecution history, and the prior art cited by Unified.
`
`IV. PERSON OF ORDINARY SKILL IN THE ART
`13. The field of the ‘945 Patent is computer graphics hardware.
`
`Professionals in the field of computer graphics hardware will generally be
`
`electrical engineers, computer engineers, or computer scientists.
`
`14.
`
`It is my opinion that a POSITA of computer graphics hardware would
`
`have a degree in electrical engineering, computer engineering, computer science,
`
`or a related field. Such a person would also have at least 3-5 years' experience
`
`working in computer graphics hardware/computer architecture or related fields, or
`
`an equivalent combination of graduate education and/or work experience.
`
`15.
`
`I meet this criteria and I consider myself a person with at least
`
`ordinary skill in the art pertaining to the ‘945 Patent. In addition, I would have
`
`been a POSITA at the time of the invention of the ‘945 Patent.
`
`5
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`V.
`
`U.S. PATENT NO. 8,933,945
`16. The ‘945 patent describes a “graphics processing circuit” consisting of
`
`multiple graphics pipelines on the same chip. When multiple graphics pipelines
`
`operate in parallel, an important goal for efficient graphics processing is “load
`
`balancing,” which means all of the graphics pipelines remain busy and none lay
`
`idle, such that the graphics processing unit can produce an image as quickly as
`
`possible.
`
`17. One of these graphics pipelines is illustrated in Fig. 2 (reproduced
`
`below) of the ‘945 Patent, which I annotated to identify one of the graphics
`
`pipelines using a red rectangle. It consists of “front end circuity,” a “scan
`
`converter” and “back end circuitry.”
`
`Raster Engine
`
`6
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`18.
`
`“The front end circuitry 35 generates the pixel data 36 by performing,
`
`for example, clipping, lighting, spatial transformations, matrix operations and
`
`rasterizing operations on the primitive data.” (‘945 Patent at 4:39-42.) For
`
`example, the front end circuitry would convert the vertices of a triangle from their
`
`3-D XYZ positions on the sphere to their corresponding 2-D XY positions on the
`
`display screen, as shown below. The “rasterizing operations” include computing
`
`vertex values useful for the next scan converter stage.
`
`Z
`
`XYZ
`
`XYZ
`
`XYZ
`
`Y
`
`X
`
`XY
`
`XY
`
`XY
`
`Y
`
`X
`
`19.
`
`“The scan converter 37 of the first graphics pipeline 101 receives the
`
`pixel data 36 and sequentially provides the position (e.g. x, y) coordinates 60 in
`
`screen space of the pixels to be processed by the back end circuitry 39…” (‘945
`
`Patent at 4:45-48.) As shown below, a scan converter accepts a triangle described
`
`by three vertices and produces a collection of pixels designed to fill the triangle.
`
`7
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`XY
`
`XY
`
`Y
`
`XY
`
`X
`
`20.
`
`“Back end circuitry 39 may include, for example, pixel shaders,
`
`blending circuits, z-buffers or any other circuitry for performing pixel appearance
`
`attribute operations (e.g. color, texture blending, z-buffering) on those pixels
`
`located, for example, in tiles…” (‘945 Patent at 4:66-5:3.) As illustrated below,
`
`the back end circuitry determines the colors of the pixels produced by the scan
`
`converter.
`
`21.
`
`In order to provide better load balancing of the available resources,
`
`the ‘945 Patent divides the screen up into tiles (below left), such that each tile can
`
`be assigned to one graphics pipeline for graphics processing. For example, as
`
`8
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`illustrated in below right, tiles shaded in red are assigned to one graphics pipeline
`
`while tiles shaded in blue are assigned to the other.
`
`As shown in Fig 2 of the ‘945 Patent, excerpted below left, a single front end
`
`circuit processes triangle vertices, but the remainder of the pipeline is performed
`
`by two different circuits “A” and “B,” which handle the portions of the screen in
`
`Fig. 3 of the ‘945 Patent, shown below right.
`
`22. A memory controller 46 accepts pixels 43 and 44 generated by back
`
`end circuitry A 39 and B 42 and writes pixels from both sources into the same
`
`graphics memory. Thus, the claimed memory controller is operative to transfer
`
`9
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`data between the pipelines and from the pipelines to the shared memory. This
`
`memory controller 46 also manages the transfer of pixel data 49 and 50 from
`
`graphics memory to the display, as illustrated by Fig. 2 excepted below, and
`
`described by the ‘945 Patent at 5:37-44.
`
`VI. UNIFIED’S REFERENCES
`
`A.
`23.
`
`SEILER
`
`Seiler discloses a specialized system used for volume rendering, e.g.
`
`for medical data imaging of CT and MRI scan data stored as a 3-D voxel array.
`
`While competitive graphics processors like Narayanaswami would have used
`
`specialized circuity including a scan converter and a texture unit (for example,
`
`10
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`Narayanaswami at 1:22 and 38-39), Seiler used an alternative to scan conversion
`
`known as ray casting. (Seiler ¶ 6.) Ray casting determines for each pixel, which
`
`triangle(s) affect it, and is viewed as the inverse approach of scan conversion.
`
`24.
`
`Seiler describes a “memory interface 210 [that] implements all
`
`accesses to the rendering memory 160, arbitrates the requests of the bus logic 220
`
`and the controller 400, and distributes array data across the modules and the
`
`rendering memory 160 for high bandwidth access and operation.” (Seiler ¶ 16.)
`
`25.
`
`Seiler does not describe memory interface 210 as transferring
`
`processed pixel data (or any data) from one pipeline to another. For example,
`
`Seiler’s Figure 2 (reproduced below) shows that memory interface 210 acts as an
`
`interface for data transfers between rendering memory 160, on the other hand, and
`
`either the bus logic 220 or controller 400, on the other. Figure 2 and the Seiler
`
`reference does not describe or depict memory interface 210 as transferring data
`
`from one pipeline to another.
`
`11
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`B.
`26.
`
`PEREGO
`
`Perego does not refer to or advocate the use of a memory shared
`
`among different pipelines operating in parallel. In Perego’s described system (as
`
`illustrated in Figure 3 reproduced below), each purported pipeline (rendering
`
`engine 312) has a dedicated memory that it shares with the CPU/Memory
`
`Controller 302 but not with any other rendering engines 312.
`
`12
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`27.
`
`Perego’s use of dedicated memory diametrically differs from Seiler’s
`
`single shared memory system and is the essence of Perego’s purported innovation:
`
`by having a dedicated memory for each rendering engine/module, Perego is able to
`
`achieve its goal of providing a scalable modular system in which individual
`
`engines/modules can be added or removed as needed. (See Perego at 2:61-65.)
`
`28.
`
`Perego’s design thus teaches away from the design disclosed by
`
`Seiler, which uses a memory controller connected to a single shared memory
`
`13
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`system. Perego’s choice of a local dedicated memory is incompatible with the
`
`single shared memory systems taught by Seiler (and the ‘945 Patent). A POSITA
`
`at the time would have understood that incorporating the tiling proposed by Perego
`
`into the multiple parallel pipeline system proposed by Seiler but without the
`
`dedicated memory also disclosed by Perego would have sacrificed the advantages
`
`of the Perego design, leaving no benefit and no motivation for the combination.
`
`29.
`
`Perego does not disclose a pipeline that is “operative to process data
`
`in a dedicated tile” as required by each of the challenged ’945 Patent claims.
`
`Although Perego states generally that certain tiles may be assigned to a particular
`
`rendering engine, Perego does not describe how this assignment and any
`
`corresponding processing is done. (See Perego at 5:19-27.) Perego does not
`
`describe a system in which a rendering engine only processes the pixels in the
`
`particular tile or tiles assigned to that engine.
`
`30. Nor is such a system inherent in light of the assignment described in
`
`Perego because such assignment can be achieved without the rendering engines
`
`only processing the pixels in a particular tile. For example, as described below, the
`
`Narayanaswami reference describes a system in which the rendering of certain tiles
`
`is assigned to particular processors, but each processor processes pixel data that is
`
`not within the tiles assigned to that processor.
`
`C.
`
`NARAYANASWAMI
`
`14
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`31. Narayanaswami was filed by IBM to find “computer graphics
`
`systems” that could “render multiple objects into a frame buffer for display as
`
`quickly as possible.” (Narayanaswami at 1:15-17.) Narayanaswami recognized
`
`that “the rendering process has become more complex and computationally
`
`intensive as users demand more detailed results … using more computationally
`
`intensive processing techniques”. (Id. at 1:18-24.)
`
`32. Narayanaswami teaches that multiprocessing has been utilized to
`
`handle an ever increasing graphical workload. (Id. at 1:17- 25.) Narayanaswami
`
`attributes that complexity to user demand for “more detailed results using more
`
`objects rendered more quickly, including providing realtime motion, while using
`
`more computationally intensive processing techniques such as color, texture,
`
`lighting, transparency and other rendering techniques.” (Id.)
`
`33. Thus, Narayanaswami was not motivated primarily by speed but by
`
`making the graphics pipeline more flexible through software programming on
`
`parallel processors to enable higher quality images.
`
`34. Narayanaswami describes graphics adapter processors 220, depicted
`
`in Fig. 1 below.
`
`15
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`35. Narayanaswami describes (and illustrates in Fig. 6 below) the
`
`preferred method steps to be performed by each processor: “FIG. 6 is a flowchart
`
`illustrating a preferred method for each processor to handle the graphics workload
`
`while determining ownership of pixels or regions. This process may be executed
`
`concurrently and in parallel by all processors.” (Narayanaswami at 6:66 – 7:3.)
`
`16
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`36. As depicted in Figure 6 and the accompanying description, each
`
`Narayanaswami processor processes all pixels, whether they are in a particular
`
`region or not. (Narayanaswami at 7:21-28.) To illustrate, step 610 requires each
`
`processor to scan convert the subobject. Narayanaswami describes the processor
`
`as completing this step before determining which pixels are “owned by the
`
`17
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`processor,” resulting in the processor scan converting and processing all of the
`
`pixels, not just those in a particular region:
`
`In step 615, the subobject is then scan converted into
`pixels, In step 620, each pixel is checked to see if it is owned by
`the processor by comparing the identifier of the processor with
`the identifier stored in the pixel ownership buffer or the region
`ownership list for that pixel.
`If yes in step 620, indicating that the processor owns the
`pixel, then the processer processes the pixel.
`
`(Id. at 7:21-28.)
`
`VII. RAY CASTING VS. SCAN CONVERTING
`37. At the time of the ‘945 patent, a POSITA would have understood that
`
`scan conversion (as utilized in Narayanaswami) was used for real-time graphics,
`
`and ray casting (as utilized in Seiler) was used for offline rendering of non-realtime
`
`graphics such as for feature film production where each image can take e.g. 20
`
`minutes to render.
`
`38. Ray casting is designed and used to achieve more realistic images.
`
`Because of this purpose, ray casting algorithms were known to require relatively
`
`large amounts of processing time and power and were thus typically run on the so-
`
`called “supercomputers” of the time or general purpose server-class CPUs rather
`
`than dedicated special purpose graphic processors.
`
`39. Thus, Seiler’s ray casting approach to rendering was considered for
`
`18
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`specialized purposes and not intended for purposes requiring speed. At the time, a
`
`POSITA would not have considered a ray casting approach for general graphics
`
`processors because, as discussed in the ‘945 Patent, the goal of those general
`
`graphics processor was speed. (See ‘945 Patent at 1:61-64) (“The amount of time
`
`taken for an entire frame of information to be calculated and provided to the frame
`
`buffer becomes a bottleneck in graphics systems as the calculations associates with
`
`the graphics become more complicated.”)
`
`40. Thus, at the time, a POSITA would understand that Seiler’s
`
`specialized approach allowed it to support real-time ray casting into this special
`
`form of volume data of medical scans, but was unsuited for the variety of graphics
`
`applications that were supported at that time by graphics accelerators.
`
`41. By contrast, Narayanaswami characterizes a “typical graphics system”
`
`which states that the renderer “breaks each remaining drawing primitive into
`
`visible pixels,” which implies scan conversion, as opposed to the fundamentally
`
`opposite ray casting approach taught by Seiler which would determine for each
`
`pixel which drawing primitive (e.g. voxel) it displays. (Narayanaswami at 4:7-37.)
`
`Narayanaswami also expressly discloses that its approach is based on scan
`
`conversion. (Narayanaswami at 7:21.)
`
`42.
`
`Seiler’s real-time ray casting was not designed to achieve the speed
`
`necessary to “render … for display as quickly as possible” as stated as a
`
`19
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`preliminary goal in Narayanaswami and it would have been poorly suited to that
`
`type of application.
`
`43. A POSITA at the time of the ‘945 patent would have understood the
`
`fundamental differences between ray casting and scan conversion, as well as a
`
`between volumetric primitives (voxels) and surface primitives (triangles), and
`
`would not have been motivated to combine such systems.
`
`44. Accordingly, for the foregoing reasons, a POSITA would not have
`
`been motivated to combine Narayanaswami with Seiler.
`
`20
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`I declare under penalty of perjury under the laws of the United States of America
`
`that the foregoing is true and correct to the best of my knowledge.
`
`Executed on: August 23, 2016
`
`21
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`Exhibit A
`
`Exhibit A
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`
`
`
`
`Curriculum Vitæ
`
`
`John C. Hart
`
`(217) 333-8740
`jch@illinois.edu
`http://graphics.cs.uiuc.edu/~jch
`
`
`
`
`
`Professor
`Dept. of Computer Science
`3227 Siebel Center
`
`Executive Associate Dean
`Graduate College
`223 Coble Hall
`
`University of Illinois at Urbana Champaign
`
`
` Experience 
`
`
`
`
`
`
`
`
`
`Education
`
` Ph.D. Electrical Engineering and Computer Science Dept., University of Illinois at Chicago, 1991. Thesis
`title: ``Computer Display of Linear Fractal Surfaces.'' Advisor: Thomas A. DeFanti.
`
` M.S. EECS Dept., University of Illinois at Chicago, 1989. Thesis title: ``Image Space Algorithms for
`Visualizing Quaternion Julia Sets.'' Advisor: Thomas A. DeFanti.
`
` B.S. College of Liberal Arts and Science, Aurora University, 1987. Major: Computer Science.
`
`Academia
`
` Executive Associate Dean. Graduate College, University of Illinois, 2015-.
`
` Associate Dean. Graduate College, University of Illinois, 2014-2015.
`
` Director for Graduate Studies. Dept. of Computer Science. 2013.
`
` Full Professor. Dept. of Computer Science, University of Illinois, 2006-.
`
` Associate Professor. Dept. of Computer Science, University of Illinois, 2000-2006.
`
` Associate Professor. School of Electrical Engineering and Computer Science, Washington State
`University, 1998-2000.
`
` Assistant Professor. School of Electrical Engineering and Computer Science, Washington State
`University, 1992-1998.
`
` Postdoctoral Research Associate. Electronic Visualization Laboratory, University of Illinois at Chicago,
`and National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, 1991-
`1992.
`
` Research Assistant. Electronic Visualization Laboratory, Electrical Engineering and Computer Science
`Dept., University of Illinois at Chicago, 1988-1991.
`
` Teaching Assistant. Analog & digital circuit design courses, EECS Dept. University of Illinois at Chicago,
`1987-1988.
`
`John C. Hart
`
`Page 1
`
`2/28/2016
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
`Industry
`
`
`
`Intrinsic Medical Imaging LLC, Bloomfield Hills, MI. Consultant, 2012-13. Mesh construction and
`simulation of coronary arterial blood flow.
`
` Pratt & Whitney, Hartford, CT. Consultant, 2011. Flux estimation in a lensed system.
`
` Science Applications International Corp. (SAIC), Champaign, IL. Consultant, 2008. Ray-NURBS
`intersection.
`
` Adobe Systems, Inc., Seattle, WA. Visiting Researcher, 2007. Project: Rendering meshed objects as
`stylized vector art.
`
` The Teaching Company, Chantilly, VA. Consultant, 2003. Produced custom educational video elements
`on the platonic solids.
`
` Evans & Sutherland Computer Corp., Salt Lake City, Utah. Consultant, 1999-2000. Project: Design of
`the Evans & Sutherland Multi-Texturing Language.
`
` Blue Sky | VIFX, Inc., Culver City, California. Consultant, 1998. Project: development of a custom
`software plug-in for the Houdini procedural animation package to polygonize a feature film villain modeled
`as a complex implicit surface.
`
` Evans & Sutherland Computer Corp., (group formerly known as Silicon Reality, Inc.) Federal Way,
`Washington. Consultant, 1997-8. Project: design of graphics hardware to support antialiased procedural
`solid texturing.
`
` Kleiser-Walczak Construction Company (a visual effects production company). Lennox,
`Massachussetts. Consultant, 1992-1993. Project: development of a new fractal-based video transition effect
`for an attraction at the Luxor Hotel, Las Vegas.
`
` AT&T Pixel Machines. AT&T Bell Laboratories, Holmdel, New Jersey. Summer Intern, 1990.
`
`
`
`IBM T.J. Watson Research Center. Hawthorne, New York. Summer Intern, 1989.
`
`Expert Services
`
` ZiiLabs (subsidiary of Creative Technology). Graphics Hardware Expert on graphics hardware, 2015.
`Authored reports on validity and damages. Deposed on both July 2015.
`
` Graphics Property Holdings (formerly SGI). Graphics Hardware Expert on floating point rasterization,
`2013-2014. Authored report on patent validity. Deposed Feb. 2014. Testified to the International Trade
`Commission Apr. 2014.
`
` Microsoft XBox. Graphics Hardware Expert on deinterlacing, through Sidley Austin LLP, 2010. Authored
`reports on patent invalidity and non-infringement.
`
` NVIDIA & AMD (ATI), Graphics Hardware Expert on graphics card features, through Cooley Godward
`Kronish LLP, San Francisco. 2008.
`
` Broadcom. Graphics Hardware Expert on graphics accelerators, through McAndrews, Held & Malloy, Ltd,
`2006.
`
`Awards and Honors
`
` Outstanding Advisor. College of Engineering, 2010 and 2015.
`
` Winner. 2004 Fantasy Graphics League.
`
`John C. Hart
`
`Page 2
`
`2/28/2016
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
` Listed. International Who's Who of Professionals, 1997.
`
` Champion. SIGGRAPH Bowl II, SIGGRAPH 94. Captain of the EVL Alumni Team.
`
` NSF Research Initiation Award. “Modeling, Rendering and Animation of Implicit Surfaces,” to support
`the generalization of methods developed for the visualization of fractal models to the more common
`implicit surfaces used in CAGD and entertainment, 1993.
`
` First Runner-Up. Truevision Videographics Competition, SIGGRAPH 90.
`
` Graduate Fellowship. Graduate College, University of Illinois at Chicago, 1990-1991.
`
`
`
` Research Funding 
`
`From Industry
`
`
`
`
`
`
`
`Intel. Intel Illinois Parallelism Center. $2,500,000, Co-PI, 2011-2012.
`
`Intel. Leveraging Larrabee’s Programmable Rasterization. $130,000, 2009-2011.
`
`Intel/Microsoft. Universal Parallel Computing Research Center. $18,000,000, Co-PI, 2008-2012.
`
` NAVTEQ. “Surface Classification and Reconstruction from LIDAR and images.” $60,000, 2008.
`
` Thomas M. Siebel. “MethMorph: Visual Simulation of Methamphetamine Addiction.” $35,000, Spring
`2006.
`
` NVidia Corp. $15,000 annually though the UIUC CS Affiliates Program, 2004–, and $125,000 total to
`date in Ph.D. student fellowships ($25K directly to the student to support one year of Ph.D. research): Nate
`Carr: 2002 & 2003, Jesse Hall: 2003 & 2004, Jared Hoberock: 2005 & 2008.
`
` Microsoft Research. “Precomputed Radiance Transfer Compression.” $15,000, Sep. 2002.
`
` Evans & Sutherland Computer Corp. “Real Time Procedural Solid Texturing.” $83,246, June 1999 –
`May 2000.
`
` Evans & Sutherland Computer Corp. and Washington Technology Center. “APST: Computer
`Graphics Hardware for Antialiased Procedural Solid Texturing.” $77,089. Aug. 16, 1998 - Aug. 15, 1999.
`
`
`
`
`
`Intel Natural Data Types Group. “Procedural Modeling.” $43,000. Aug. 16, 1997 - Aug. 15, 1998.
`
`Intel Natural Data Types Group. “Recurrent Modeling -- Beyond Fractal Block Coding.” (Co-PI with
`P. Flynn.) $93,000. Apr. 1, 1995 - Mar. 31, 1997.
`
`From Government Agencies
`
` NSF. #OCI-1216788 “Collaborative Research: Conceptualizing an Institute for Using Inter-Domain
`Abstractions to Support Inter-Disciplinary Applications” (Co-PI with David Padua and Philippe Geubelle,
`and other collaborators at Purdue and UT-Austin), $135,000, Oct. 2012 – indefinite.
`
` NSF. #EF-1115112 “Collaborative Research: Digitization TCN: InvertNet--An Integrative Platform for
`Research on Environmental Change, Species Discovery and Identification” (co-PI with Christopher
`Dietrich, Christopher Taylor, Nahil Sobh and Umberto Ravaioli), $2,809,463, July 2011-June 2015.
`
` NSF. #OCI-1047764 “SI2-SSE: Collaborative Research: Lagrangian Coherent Structures for Accurate
`Flow Structure Analysis” (co-PI with Shawn Shadden) $251,643, Sep. 2010 – Aug. 2013.
`
`John C. Hart
`
`Page 3
`
`2/28/2016
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
` NSF. #IIS-0534485 “Analysis and Visualization of Complex Graphs” (co-PI with Michael Garland)
`$300,000, Sep. 2006 – Aug. 2009.
`
` UIUC Critical Research Initiative. “A New Approach to Bone Replacement.” (Co-PI with Russ Jamison,
`Michael Goldwasser, Ben Grosser, Matei Stroila and Amy Wagoner Johnson.) $100,000, Sep. 2005 – Aug.
`2006.
`
` NSF Small Grant for Exploratory Research. “Application Directed Surface Parameterization.” $97,868,
`Jan. 2005 – Dec. 2005.
`
` NSF/DARPA CARGO (Computational and Algorithmic Representations of Geometric Objects)
`Award. “Robust Lagrangian Surface Propagation with Topological Control.” (Lead PI, with Michael
`Heath, Jim Jiao and John Sullivan), $400,000, May 2003 – May 2006.
`
` NSF Information Technology Research. #NSG-0219594 “Making 3D Visibility Practical.” (Co-PI with
`Steve LaValle, Jeff Erickson, Fredo Durand) $499,923. Aug. 2002 – Aug. 2005.
`
` CNRS (Centre National de la Recherche Scientifique). Supplement to “Making 3D Visibility Practical”
`to support UIUC – INRIA collaboration, $7,000, 2003-5.
`
` NSF Information Technology Research. #ACI-0113968 “Multipass Programming for Personal High-
`Performance Computing.” $489,671, Aug. 2001 – July 2006.
`
` NSF Information Technology Research. #ACI-0121288 “Procedural Representation and Visualization
`Enabling Personalized Computational Fluid Dynamics.” (Co-PI with David Ebert, David Marcum, Kelly
`Gaither and Penny Rheingans) $3,989,773. Aug. 2001 – July 2006.
`
` NSF. #NSG-9732379 “Applications of Morse Theory and Catastrophe Theory to Computer Graphics.”
`(Co-PI with R. Lewis.) $220,541. Aug. 16, 1997 – Aug. 15 2000.
`
` NSF. #CCR-9529809 “Recurrent Modeling.” (Co-PI with P. Flynn.) $206,435. June 15, 1996 - May 31,
`1999. Research Experience for Undergraduates Supplement: $5,000. Jan. 1, 1998 - May. 31, 1999.
`
` NSF Research Initiation Award. #CCR-9309210 “Modeling, Rendering and Animation of Implicit
`Surfaces.” $97,925. July 1, 1993 - June 30, 1996. Research Experience for Undergraduates Supplement:
`$4,885. Jan. 1, 1995 - Dec. 31, 1995.
`
`Education Projects
`
` Nokia University Cooperation Funding. “Teaching Mobile Augmented Reality on the Windows Phone
`Platform.” $11,377.64. 2013.
`
` UIUC College of Engineering Strategic Instructional Initiatives Program. Improvement of key ME
`courses (Statics, Dynamics and Solid Mechanics), $450,000, with MechSE Profs. Tortorelli, Dullerud
`Keane and West, 2012-2014.
`
` NSF Special Project. #EIA-9911033 “The Story of Computer Graphics Documentary Project.” $48,000.
`Sept. 15, 1999 – Aug. 31, 2000.
`
`Equipment Grants and Donations
`
` Nokia. Two Velodyne LIDAR scanners, 2013.
`
` NVidia. K20 graphics card, 2012.
`
`
`
`
`
`Intel. Dell XPS 12, $1,000, 2012.
`
`Intel. Two Knights Ferry development workstations, $4,940, 2009.
`
` NVidia. Various graphics cards, 2002-4.
`
`John C. Hart
`
`Page 4
`
`2/28/2016
`
`AST Exhibit 2001
`
`AST Exhibit 2001
`
`

`
` ATI. One graphics card, 2002.
`
` Tektronix Phaser 340 Color Printer. Tektronix. $5,000. Oct. 1995.
`
` High-Performance Networking and Computing Infrastructure for Imaging Research. (Co-PI with
`T. Fischer, P. Flynn and R. Bamberger.) NSF Research Instrumentation and Infrastructure. CISE
`Instrumentation Program, Office of Cross-Disciplinary Activites. $100,000. March 1, 1994 - April 30,
`1996. Awarded: Jan. 1995.
`
` Silicon Graphics Inc. Workstation Upgrade. $9,000. Apr. 1993.
`
` XAOS Tools. Software Donation. Pandemonium and n-title. $6,500. Feb. 1993.
`
` Karen Guzak. Art Donation. 12 Prints. $4,800. May 1995.
`
`
`
` Publications 
`
`Most of the following publications are available online via
`http://graphics.cs.illinois.edu/~jch/papers.
`
`Reviewed Publications
`
`1. M. Chen, S. Shadden and J.C. Hart. “Fast Coherent Particle Advection through Time-Varying Unstructured
`Flow Datasets.” To appear: IEEE Transactions on Visualization and Computer Graphics, 2016.
`
`2. M. Chen, J.C. Hart and S. Shadden. Hierarchical Watershed Ridges for Visualizing Lagrangian Coherent
`Structures. Presented at TopoInVis: Topology-Based Methods in Visualization, 2015. To appear: Springer
`Mathematics + Visualization Series.
`
`3. J. Kratt, M. Spicker, A. Guayaquil, M. Fiser, S. Pirk, O. Deussen, J.C. Hart, B. Benes. Woodification: User-
`Controlled Cambial Growth Modeling. Proc. Eurographics, Computer Graphics Forum 23(2), May 2015, pp.
`361-372.
`
`4. V. Lu, J.C. Hart. Multicore Construction of k-d Trees for High Dimensional Point Data. Proc. Advances in Big
`Data Analytics, July 2014.
`
`5. P.R. Khorrami, V.V. Le, J.C. Hart, T.S. Huang. A System for Monitoring the Engagement of Remote Online
`Students using Eye Gaze Estimation. Proc. IEEE ICME Workshop on Emerging Multimedia Systems and
`Applications, July 2014.
`
`6. V. Lu, I. Endres, M. Stroila and J.C. Hart. Accelerating Arrays of Linear Classifiers Using Approximate Range
`Queries. Proc. Winter Conference on Applications of Computer Vision, Mar. 2014.
`
`7. D. Mayerich, J.C. Hart, Volume Visualization of Serial Electron Microscopy Images Using Local Variance.
`Proc. BioVis (IEEE Symposium on Biological Data Visualization), pp. 9-16, Oct. 2012
`
`8. C. Dietrich, J. Hart, D. Raila, U. Ravaioli, N. Sobh, O. Sobh, C. Taylor. InvertNet: a new paradigm for digital
`access to invertebrate collections. ZooKeys 209, July 2012, pp. 165-181.
`
`9. Y. Jia, M. Garland, and J.C. Hart. Social Network Clustering and Visualization using Hierarchical Edge
`Bundles. Computer Graphics Forum 30(8), Dec. 2011, pp. 2314–2327.
`
`10. Y. Jia, V. Lu, J. Hoberock, M.J. Garland, J.C. Hart. Edge v. Node Parallelism for Graph Centrality Metrics.