`
`In re Patent of: McFarland et al.
`U.S. Patent No.: 5,959,633 Attorney Docket No.: 19473-0309IP1
`Issue Date:
`Sep. 28, 1999
`Appl. Serial No.: 08/726,091
`Filing Date:
`Oct. 4, 1996
`Title:
`METHOD AND SYSTEM FOR PRODUCING GRAPHICAL IMAGES
`
`
`DECLARATION OF DR. ANSELMO LASTRA
`
`I, Dr. Anselmo Lastra, of Chapel Hill, NC, declare that:
`
`QUALIFICATIONS AND BACKGROUND INFORMATION
`1.
`I have been a professor in the Department of Computer Science at the
`
`University of North Carolina at Chapel Hill since 1991. I am currently a tenured Professor
`
`and Chairman of the Computer Science Department. My qualifications for formulating my
`
`analysis on this matter are summarized here and are addressed more fully in my curriculum
`
`vitae, which is attached hereto as Appendix A.
`
`2.
`
`I received my Bachelor of Science degree in Electrical Engineering from the
`
`Georgia Institute of Technology in 1972. In 1981, I received a Master of Arts degree in
`
`Computer Science from Duke University. In 1988, I received a Ph.D. in Computer Science
`
`from Duke University.
`
`3.
`
`Since 1972 I have been involved in the research, development, and design of
`
`hardware and software for computer systems, including computer graphics systems.
`
`4.
`
`From 1972 to 1975, I was an engineer at Scidata Inc. in New York, New York
`
`and Miami, Florida. From 1975 to 1979, I was an engineer and then project manager at
`
`Page 1 of 83
`
`1
`
`GOOGLE-1003
`
`
`
`Coulter Electronics in Miami, Florida. While holding these positions, I designed electronic
`
`hardware, software, and medical instruments and for a time I supervised a team of
`
`engineers and technicians.
`
`5.
`
`From 1988 to 1991, I was a Research Assistant Professor at Duke University
`
`in Durham, North Carolina. While holding this position, I taught computer science classes
`
`and performed research in the areas of parallel processing and performance, and
`
`computational science, with a particular interest in graphics and pattern recognition.
`
`6.
`
`In 1991, I joined the Computer Science Department at the University of North
`
`Carolina at Chapel Hill as a Research Assistant Professor. In 2001, I was promoted to
`
`Associate Professor. In 2006, I was promoted to full Professor, and since 2009 I have
`
`served as Chairman of the Computer Science Department.
`
`7.
`
`At the University of North Carolina at Chapel Hill, I worked on the Pixel-
`
`Planes and PixelFlow projects. With support from the National Science Foundation and the
`
`Defense Advanced Research Projects Agency, we developed a heterogeneous graphics
`
`supercomputer called Pixel-Planes 5 that was, to the best of my knowledge, the fastest of
`
`its time. We later developed another heterogeneous graphics supercomputer called
`
`PixelFlow.
`
`8.
`
`At the University of North Carolina at Chapel Hill, I have taught courses in
`
`computer graphics, graphics hardware, computer architecture, 3D computer animation,
`
`image generation, robotics, digital logic, programming, and other subjects.
`
`Page 2 of 83
`
`2
`
`
`
`9.
`
`I have authored or co-authored over 80 research papers and articles, many of
`
`which relate to computer graphics and were presented in prestigious venues, such as the
`
`annual conference of the Association for Computing Machinery’s Special Interest Group on
`
`Computer Graphics and Interactive Techniques (“SIGGRAPH”). A complete list of my
`
`publications is contained in my curriculum vitae in Appendix A.
`
`10.
`
`I co-authored perhaps the first publication in the area now known as General
`
`Purpose Computing on Graphics Processors (“GPGPU”): Harris, Coombe, Scheuermann,
`
`and Lastra, “Physically-Based Visual Simulation on Graphics Hardware,” Graphics
`
`Hardware 2002, 109-118. With two colleagues from the University of North Carolina at
`
`Chapel Hill, I organized the 2004 Workshop on GPGPU held in Los Angeles, which brought
`
`this technology to a wider audience.
`
`11.
`
`I served a term on the editorial board of the IEEE Transactions on
`
`Visualization and Computer Graphics. I also served a term on the editorial board of IEEE
`
`Computer Graphics and Applications. I was on the SIGGRAPH conference organization
`
`committee. I served as papers chair, program chair, treasurer and general chair of the
`
`Graphics Hardware conference, and I also served on the advisory board. In 2009, we
`
`merged Graphics Hardware with the Symposium on Interactive Ray Tracing and created the
`
`High Performance Graphics Conference. I currently serve on the steering committee.
`
`Page 3 of 83
`
`3
`
`
`
`12.
`
`Based on my above-described 40 years of experience in computer system
`
`design and graphics hardware and software, I believe that I am considered to be an expert
`
`in computer graphics.
`
`13.
`
`In writing this Declaration, I have considered the following: my own
`
`knowledge and experience, including my work experience in the fields of computer science,
`
`computer engineering, and computer graphics systems; my experience in teaching those
`
`subjects; and my experience in working with others involved in those fields. In addition, I
`
`have analyzed the following publications and materials:
`
` The disclosure and claims of U.S. Patent No. 5,959,633 (“the ‘633 patent”);
`
` The prosecution history of the '633 patent;
`
` U.S. Patent No. 5,883,639 to Walton et al. (“Walton”);
`
` U.S. Patent No. 5,564,048 to Eick et al. (“Eick”); and
`
` Inside Visual C++, Second Edition: Version 1.5 by David J. Kruglinski, September
`
`1, 1994 (“Kruglinski”)
`
` The additional references and background materials specifically identified below
`
`in this Expert Declaration, which are attached as Appendices A-C.
`
`14.
`
`Although for the sake of brevity this Declaration refers to selected portions of
`
`the cited references, it should be understood that one of ordinary skill in the art would view
`
`the references cited herein in their entirety, and in combination with other references cited
`
`Page 4 of 83
`
`4
`
`
`
`herein or cited within the references themselves. The references used in this Declaration,
`
`therefore, should be viewed as being incorporated herein in their entirety.
`
`15.
`
`I am not, and never was, an employee of Google Inc., Samsung
`
`Telecommunications America, LLC, Samsung Electronics America, Inc., or Samsung
`
`Electronics Col., Ltd. I have been engaged in the present matter to provide my independent
`
`analysis of the issues raised in the petition for inter partes review of the ‘633 patent. I
`
`received no compensation for this declaration beyond my normal hourly compensation
`
`based on my time actually spent studying the matter, and I will not receive any added
`
`compensation based on the outcome of this inter partes review of the ’633 patent.
`
`OVERVIEW OF CONCLUSIONS FORMED
`16.
`This expert Declaration explains the conclusions that I have formed based on
`
`my analysis. To summarize those conclusions:
`
` Based upon my knowledge and experience and my review of the prior art publications
`
`listed above, I believe that claims 1-4, 6, 8-11, 13, and 15 of the ‘633 patent are
`
`anticipated by U.S. Patent No. 5,883,639 (“Walton”).
`
` Based upon my knowledge and experience in this art and my review of the prior art
`
`publications listed above, I believe that claims 1-4, 6, 8-11, 13, and 15 of the ‘633 patent
`
`are rendered obvious in light of U.S. Patent No. 5,564, 048 (“Eick”) in view of Inside
`
`Visual C++, Second Edition: Version 1.5 by David J. Kruglinski, September 1, 1994
`
`(“Kruglinski”).
`
`Page 5 of 83
`
`5
`
`
`
`
`
`BACKGROUND KNOWLEDGE ONE OF SKILL IN THE ART WOULD HAVE HAD PRIOR
`TO THE FILING OF THE ‘633 PATENT
`17.
`The technology in the '633 patent at issue generally relates to a computer
`
`graphics programming system that allows a graphics program to access a shape library
`
`stored external to the program and delegate drawing functions to “external capabilities” of
`
`the accessed shapes. (‘633 patent at 1:41-50.) Prior to the filing date of the ‘633 patent,
`
`there existed numerous products, publications, and patents that implemented or described
`
`the functionality claimed in the ‘633 patent. Thus, the methodology of the ‘633 patent was
`
`well-known in the prior art. Further, to the extent there was any problem to be solved in the
`
`'633 patent, it had already been solved in the prior art systems before the filing date of the
`
`'633 patent.
`
`18.
`
`Based upon my experience in this area, a person of ordinary skill in the art in
`
`this field before October 1996 (“POSITA”) would have had a combination of experience and
`
`education in object-oriented programming languages and graphical user interface design.
`
`This typically would consist of a minimum of a bachelor of science in computer science or a
`
`related engineering field plus 2-5 years of experience or the equivalent.
`
`19.
`
`Based on my experiences, I have a good understanding of the capabilities of
`
`a POSITA. Indeed, I have taught, participated in organizations, and worked closely with
`
`Page 6 of 83
`
`6
`
`
`
`many such persons over the course of my career, including the relevant time frame before
`
`October 1996.
`
`
`
`INTERPRETATIONS OF THE ‘633 PATENT CLAIMS AT ISSUE
`
`20.
`
`I understand that, for purposes of my analysis in this inter partes review
`
`proceeding, the terms appearing in the patent claims should be interpreted according to
`
`their “broadest reasonable construction in light of the specification of the patent in which it
`
`appears.” 37 C.F.R. § 42.100(b). In that regard, I understand that the best indicator of
`
`claim meaning is its usage in the context of the patent specification as understood by a
`
`POSITA. I further understand that the words of the claims should be given their plain
`
`meaning unless that meaning is inconsistent with the patent specification or the patent’s
`
`history of examination before the Patent Office. I also understand that the words of the
`
`claims should be interpreted as they would have been interpreted by a POSITA at the time
`
`of the invention was made (not today); because I do not know at what date the invention as
`
`claimed was made, I have used the filing date of the ‘633 patent as the point in time for
`
`claim interpretation purposes. That date was October 4, 1996. Without exception,
`
`however, the interpretations that I provide below would have also been correct if the date of
`
`invention was anywhere within the mid-1990s. I have been asked to provide my
`
`interpretation of the following terms and phrases of the ‘633 patent set forth below.
`
`Page 7 of 83
`
`7
`
`
`
`21. Claims 1 and 8 recite “external shape stored outside the computer program.”
`
`First, I have interpreted “external shape” to refer to an object, library component, or other
`
`data structure that contains source code, that is stored external to a computer program, and
`
`that includes data for use in producing a graphical image. For example, the ‘633 patent
`
`describes accessing an “eternal shape library” stored in a computer memory. (‘633 patent at
`
`2:66-3:7.) The external shapes are stored as code objects that include “information used by
`
`computer graphics application 122 to produce graphical images on output device 116.” (Id.
`
`at 3:3-7; see also 4:39-53.) Second, I have interpreted “stored outside the computer
`
`program” to mean “external to a computer program,” but not necessarily in a different
`
`physical location than the computer program, in accordance with the plain meaning of this
`
`language. This is also consistent with the specification of the ‘633 patent, which discloses
`
`that the “computer graphics application 122” and the “external shape library 124” are both
`
`“stored in memory 118.” (Id. at 2:58-3:3.) Therefore, in accordance with the claim
`
`interpretation principles set forth above, I believe that a person of ordinary skill in the art
`
`would recognize the broadest reasonable interpretation of “external shape stored outside
`
`the computer program” to be “a library or object containing code that is external to a
`
`computer program, and which includes data defining a graphical image.”
`
`22. Claims 1, 8, and 15 recite “external capabilities.” The term “capabilities” is
`
`defined in the ‘633 patent to mean “action methods, symbol methods, or any other functions
`
`that allow the generation of information required to produce a graphical image.” (Id. at 3:29-
`
`Page 8 of 83
`
`8
`
`
`
`31.) I therefore interpret the term “external capabilities” to refer to such capabilities that are
`
`external to a computer program. Accordingly, I will apply this definition for the term
`
`“external capabilities” to my analysis of the claims of the ‘633 patent.
`
`23. Claims 1, 2, 8, and 9 recite “delegate.” The plain meaning of “delegate” is “to
`
`commit or entrust to another.” (Appendix C, The American Heritage Dictionary of the
`
`English Language 493 (3rd ed. 1992).) The ‘633 patent does not provide a definition for the
`
`term “delegate” and in fact (other than in the claims) the term “delegate” only appears three
`
`times within the specification of the ‘633 patent. I have therefore applied the above recited
`
`dictionary definition of the word “delegate” to my analysis of the ’633 patent.
`
`24. Claims 2-4 and 9-11 recite “external action.” The ‘633 patent describes
`
`external actions as including functions and data that are operable “to receive user
`
`interaction received by the computer graphics application 122 from input device 114 for
`
`manipulation of shapes that are not contained within computer graphics application 122.”
`
`(Id. at 6:19-29.) The ‘633 patent further discloses that “[a]n external action, such as external
`
`action 352, allows computer graphics application 122 to pass user interaction from input
`
`device 114, such as mouse and keyboard inputs, to external symbol methods 612, which as
`
`discussed below are functions that operate to create, edit, render, modify, read, or write a
`
`graphical object on output device 116.” Therefore, in accordance with the claim
`
`interpretation principles set forth above, I believe that a person of ordinary skill in the art
`
`would recognize the broadest reasonable interpretation of the term “external actions” to be
`
`Page 9 of 83
`
`9
`
`
`
`“functions and data that are external to the computer program and that are operable to
`
`receive user interaction.”
`
`25. Claims 2, 3, 6, 9, 10, and 13 recite “external symbol.” The ‘633 patent
`
`describes external symbols as “functions that operate on external symbol data 610 to
`
`create, edit, render, modify, read, or write a graphical object.” (Id. at 7:14-16.) The ‘633
`
`patent further describes that the functions of the external symbols are performed in
`
`response to user input. (Id. at 5:6-12.) Therefore, in accordance with the claim
`
`interpretation principles set forth above, I believe that a person of ordinary skill in the art
`
`would recognize the broadest reasonable interpretation of the term “external symbols” to be
`
`“functions and data that are external to the computer program and that are operable for
`
`creating, editing, rendering, modifying, reading, and/or writing a graphical image.”
`
`ANALYSIS OF WALTON (CLAIMS 1-4, 6, 8-11, 13, and 15)
`26. Walton is one of many prior art examples of computer graphics systems and
`
`programs that utilize external graphical object libraries for storing graphical object data and
`
`input, drawing, and manipulation functions for various graphical objects to allow user
`
`applications to access the graphical objects and associated functionality stored within the
`
`external libraries. (See Walton at 9:24-25; 9:46-47; 6:8-10; 6:17-20; 8:54-65; 21:10-16.) As
`
`shown below, annotated Fig. 3 of Walton shows a modular arrangement for storing
`
`graphical object data and functionality in a “library of graphical objects 320” that is stored
`
`externally to an application, labeled as “user’s source code 360:”
`
`Page 10 of 83
`
`10
`
`
`
`shape objects are
`stored external to the
`computer program
`
`
` Walton discloses that graphical objects are “stored as objects in an object-
`
`27.
`
`oriented database system and connected to other objects or user code” and that such
`
`techniques for storing the graphical objects external to the user code were “commonly used
`
`in object-oriented systems” at the time that Walton was written. (Id. at 8:54-59.) Walton
`
`further discloses that these external graphical objects can be “accessed by the user code
`
`120” by connecting to “a client server via an interprocess communications mechanism of a
`
`Page 11 of 83
`
`11
`
`
`
`type known to those skilled in the art.” (Id. at 8:58-62.) Walton describes that storing the
`
`graphical objects externally to the user application is “particularly advantageous in that the
`
`user application code can read input and send outputs to the display screen using the
`
`created graphics objects without requiring the designer to write interface code.” (Id. at 8:24-
`
`27.)
`
`28. Walton further describes delegating drawing functions and other functions for
`
`the external shapes to the externally stored graphical objects. For example, Walton
`
`indicates that “a graphical object in accordance with the invention must be able to draw
`
`itself if asked to do so.” (emphasis added) (Id. at 13:19-21.) Walton additionally discloses
`
`that each “graphical object is responsible for controlling itself” and that a graphical object is
`
`responsible for “chang[ing] its graphical representation . . . on the display” in response to
`
`user input or other events. (Id. at 11:8-9; 13:26-30.)
`
`29.
`
`As set forth in more detail below, all of which is based upon my knowledge
`
`and experience in this art and my review of Walton, I believe that Walton anticipates every
`
`element of claims 1-4, 6, 8-11, 13, and 15.
`
`[1.P] A computerized system comprising:
`30.
`I believe that a POSITA would have recognized that the Walton patent
`
`discloses a computerized system. For example, Walton describes "[a] visual software
`
`engineering system." (WALTON at 7:44-47.) Walton further describes that the visual
`
`software engineering system can be implemented using UNIX work stations running source
`
`Page 12 of 83
`
`12
`
`
`
`code written in C or C++ computer languages. (Id. at 7:44-59.) Walton discloses using “[a]
`
`system for providing a simple, easy to learn and flexible means of creating user interfaces to
`
`products under development without the need of a programming language or the need to
`
`learn a large set of complicated commands.” (Id. at Abstract.) Walton also discloses that
`
`"a system is provided for creating and animating graphical objects by directly
`
`manipulating the graphical objects on a display screen." (emphasis added) (Id. at 4:38-41.)
`
`31. Walton further indicates that the system is “preferably implemented in
`
`software on a computer and is accessed via a window system 402,” a window system
`
`being a common type of computer interface system. (emphasis added) (Id. at 9:52-54.) A
`
`person of ordinary skill in the art would have recognized that the above mentioned systems
`
`used to implement computer code written in a computer language are “computerized
`
`systems” as recited in claim 1.
`
`[1.1] a storage medium;
`32.
`I believe that a POSITA would have recognized that the Walton patent
`
`discloses a storage medium. For example, Walton discloses that a “user can also store the
`
`entire design of his or her user interface on disk” and that “objects [can] be saved and
`
`restored from disk files.” (Walton at 4:29-33; 21:44-46.) The term “disk” when used in this
`
`context, at the time of the filing of the ‘633 patent, would include internal hard drives,
`
`external hard drives, removable disks (e.g., floppy disks), CD-ROMs and other types of
`
`storage devices, any of which constitutes a “storage medium” as recited in claim 1.
`
`Page 13 of 83
`
`13
`
`
`
`Additionally, Walton describes that its system is “implemented in software on a computer
`
`and is accessed via a window system 402." (Id. at 9:52-54.) As mentioned above, Walton
`
`also discloses implementation in C or C++ on UNIX workstations. (Id. at 7:51-57.) A person
`
`of ordinary skill in the art would have recognized that software requires a storage medium to
`
`execute on a computer.
`
`[1.2] a processor coupled to the storage medium;
`33.
`I believe that a POSITA would have recognized that the Walton patent
`
`discloses a processor coupled to the storage medium. For example, Walton discloses that
`
`“[a] software interface is also easily connected to the actual code being constructed by the
`
`developer, where such code may be running on the development host, in an instruction
`
`set simulator on the host, in the physical product's processor, or on all three at once.”
`
`(emphasis added) (Walton 6:29-34.) A person of ordinary skill in the art would have
`
`recognized that the term “coupled” when used the context of this claim element means that
`
`the processor is in communication with the storage medium so as to allow the processor to
`
`access code stored on the storage medium. A POSITA would further have recognized that
`
`for the processor to be “running” the code (as taught by Walton), it must be coupled to a
`
`storage medium containing the code.
`
`[1.3] a computer program stored in the storage medium, the computer program
`operable to run on the processor, the computer program further operable to:
`34.
`I believe that a POSITA would have recognized that Walton discloses a
`
`computer program stored in the storage medium, the computer program operable to run on
`
`Page 14 of 83
`
`14
`
`
`
`the processor and further operable to perform the functions recited in claim 1. For example,
`
`Walton describes a computer program in the form of “user software [that] may be used to
`
`access the graphical object” and that “the graphical object may be manipulated on the
`
`display screen directly from the user application code.” (Walton at 8:16-21); Walton also
`
`describes a program in the form of developer code in that “[a] software interface is also
`
`easily connected to the actual code being constructed by the developer, where such
`
`code may be running on the development host, in an instruction set simulator on the host, in
`
`the physical product's processor, or on all three at once.”) (emphasis added) (Id. at 6:29-
`
`34.) Furthermore, Walton describes “a source code listing of the following preferred
`
`embodiment of the VSE system 400” and that “graphical objects” can be “linked to user
`
`source code 360.” (Id. at 10:17-21; 9:35-40.)
`
`35. Walton describes the user code (program) as being executed on a computer.
`
`For example, Walton discloses that “the VSE system 400 is preferably implemented in
`
`software on a computer and is accessed via a window system 402.") (emphasis added)
`
`(Id. at 9:50-54.) Based on Walton’s teaching and the broadest reasonable interpretation of
`
`this claim element, a POSITA would have understood that the computer program running on
`
`the processor of the computer would be stored in the storage medium.
`
`[1.4] access an external shape stored outside the computer program, the
`external shape comprising external capabilities; and
`36.
`I believe that a POSITA would have recognized that Walton discloses
`
`accessing an external shape stored outside the computer program, the external shape
`
`Page 15 of 83
`
`15
`
`
`
`comprising external capabilities. As explained above, the proper construction for “external
`
`shape stored outside the computer program” is “an object, library, or other data structure
`
`containing code that is stored external to a computer program and that includes data for use
`
`in producing a graphical image.” Walton discloses exactly that. For example, Walton
`
`describes accessing external shapes in the form of graphical objects stored in an external
`
`library, each object including an external shape (“graphic element”) and external capabilities
`
`(“behavior element”). (Walton at 13:13-17.) Walton discloses that “[a] graphical object is
`
`the key object for the VSE system” and that a “VSE object consists of two major parts, the
`
`graphic element and the behavior element.” (Id.) The behavior elements taught by Walton
`
`define functions that manipulate the display of graphical objects in response to user input.
`
`(See Walton at 5:62-6:15.)
`
`37.
`
`A POSITA would have understood that Walton’s “behavior element”
`
`corresponds to the recited “external capabilities,” which as noted above are properly defined
`
`as “action methods, symbols methods, or any other functions that allow the generation of
`
`information required to produce a graphical image.” Walton describes that the behavior
`
`element of each VSE object includes “a behavior function (graphics manipulation)” and a
`
`“behavior event” which allows “the VSE object [to] change its graphical representation and
`
`update itself on the display.” (Id. at 13:26-31.) Thus, the behavior elements of Walton
`
`correspond to the recited “external capabilities” because, among other reasons, the
`
`behavior elements define behavior functions (graphic output) for creating and manipulating
`
`Page 16 of 83
`
`16
`
`
`
`graphics on a display in response to behavior events (user input). (See Id. at 13:19-41;
`
`11:8-9; 11:23-30.)
`
`38. Walton describes computer programs in the form of user source code. (Id. at
`
`9:36-47.) Walton uses the term “client application” to refer to computer programs capable of
`
`accessing and interacting with external graphical objects. (Id. at 8:35-54.) Annotated FIG. 3
`
`of Walton shows the library of graphical objects 320 (i.e., “external shapes”) being stored
`
`external to the computer program (user source code 360).
`
`Page 17 of 83
`
`17
`
`
`
`shape objects are
`stored external to the
`computer program
`
`
`39. Walton discloses that shapes are stored as objects. (Id. at 6:38-40) For
`
`example, Walton discloses “using an object based technology where a drawn shape is a
`
`separately movable object.” (Id.; see also 12:57-67.) Walton discloses that “the user
`
`software may be used to access the graphical object, and by providing the behavior function
`
`name and the desired behavior state, the graphical object may be manipulated on the
`
`display screen directly from the user application code.” (Id. at 8:16-21.) Walton further
`
`Page 18 of 83
`
`18
`
`
`
`discloses that “the user program preferably includes an interface 230 for accessing this
`
`animation information and for calling the object and its associated behavior and
`
`behavior value for the chosen animation” where the called “object” includes the external
`
`shape and the called “associated behavior” is the external capabilities associated with the
`
`called shape object. (emphasis added) (Id. at 9:4-8.)
`
`40.
`
`For example, Walton discloses that each shape object has external
`
`capabilities in the form of “input and output behavior states” and that these “behavior states
`
`can also be stored in external database 370.” (Id. at 9:24-25; 9:46-47.) As noted above,
`
`“external capabilities” are defined in the specification of the ‘633 patent as “action methods,
`
`symbols methods, or any other functions that allow the generation of information required to
`
`produce a graphical image.” (‘633 Patent at 3:29-31.) The behavior states of Walton are
`
`“functions that allow the generation of information required to produce a graphical image” in
`
`that the behavior states define functions that manipulate the display of graphical objects in
`
`response to user input. (See Walton at 5:62-6:15.)
`
`41. Walton gives other various examples of external capabilities being stored
`
`along with shape objects, including “storing the defined output behavior with the
`
`corresponding graphical object for respective input states.” (Id. at 5:34-35.) Other examples
`
`of storing and accessing external capabilities with external shape objects disclosed by
`
`Walton include “storing the defined input behavior states with corresponding graphical
`
`objects for respective output behavior states” and “retrieving graphical objects with pre-
`
`Page 19 of 83
`
`19
`
`
`
`defined input behavior states from a library.” (Id. at 6:8-10; 6:17-20.) Walton discloses that
`
`graphical objects are stored external to the computer program in that the graphical objects
`
`are “stored as objects in an object-oriented database system and connected to other objects
`
`or user code” and that such techniques for storing the graphical objects external to the user
`
`code were “commonly used in object-oriented systems” at the time that the Walton patent
`
`was written. (Id. at 8:54-59.) Walton further discloses that these external graphical objects
`
`can be “accessed by the user code 120” by connecting to “a client server via an
`
`interprocess communications mechanism of a type known to those skilled in the art.” (Id. at
`
`8:58-62.) Walton further discloses that the user code accesses graphical objects stored
`
`externally by the VSE system, in that “[t]he client server 414 is the part of the VSE system
`
`400 that connects user code to the VSE system 400 . . . client server 414 establishes the
`
`connection between a user program and the VSE system 400 and passes information
`
`between the two as required." (Id. at 21:10-16.) This information passed between the VSE
`
`system and the user source code includes “graphical objects with pre-defined input behavior
`
`states” retrieved from the library of graphical objects 320. (Id. at 6:17-20.) Walton also
`
`discloses that user code can “receive graphical objects from other graphical editors outside
`
`of the VSE system 400.” (Id. at 14:19-21.)
`
`42. Walton further describes that the graphics objects can be saved to disk,
`
`external to the user code. For example, "Universe 420 is also provided as a place for storing
`
`all VSE graphical objects." (Id. at 10:1-2.) Additionally, “[t]he universe 420 also allows
`
`Page 20 of 83
`
`20
`
`
`
`objects to be saved and restored from disk files and views of the universe 420 to be added
`
`and deleted.” (Id. at 21:44-46.) Walton additionally discloses that objects can be saved to
`
`disk in the form of graphical files:
`
`“The graphical file is created to store the graphical representations of
`objects for displaying or printing the appearance of those objects, while
`the design file contains the information the graphical file does along with
`descriptions of object relationships and object behavior functions. In other
`words, the graphical file stores pictures of objects, while the design file
`stores the appearance and behavior of objects.”
`
`(Id. at 11:23-30.)
`
`43. Walton further describes that the external capabilities (“behavior states”) of
`
`the graphics objects are stored external to user source code. (Id. at 9:35-42.) For example,
`
`“the behavior states of the graphical objects created by the graphics editor 310 may also be
`
`linked to user source code 360” and that the behavior states include capabilities such as the
`
`ability to “manipulate the states of the graphical objects.” (Id.) Walton further discloses that
`
`the capabilities are stored external to the user source code in that “the user may access the
`
`stored behavior function and values for both input and output." (Id.)
`
`44. Walton further discloses that the graphical objects (i.e., “external shapes” in
`
`the parlance of the ‘633 patent) include external capabilities in the form of functions that
`
`allow the graphical objects to produce a graphical image. (Id. at 13:19-22) For example,
`
`Walton discloses that “a graphical object in accordance with the invention must be able to
`
`Page 21 of 83
`
`21
`
`
`
`draw itself if asked to do so and to indicate that it has been selected." (emphasis added)
`
`(Id.) Walton additionally discloses that each “graphical object is responsible for controlling
`
`itself” and that a graphical object is responsible for “chang[ing] its graphical representation .
`
`. . on the display” in response to user input or other events. (Id. at 11:8-9; 13:26-30.)
`
`Walton includes numerous other examples of externally stored graphical objects having
`
`associated capabilities for manipulating the graphical objects:
`
`“the graphics for creating an interface to an object are created and then
`input and output behavior are added (as by clicks on mouse buttons).
`The object is then tested and saved. The user then creates the desired
`application code and runs the application which calls upon particular
`objects. The user application code is then connected to the VSE system
`400 and the results displayed.”)
`
`
`(emphasis added) (Id. at 25:62-26:4)
`
`
`45. Walton describes the Visual Software Engineering