`
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
` ____________
`
`SONY COMPUTER ENTERTAINMENT AMERICA LLC
`Petitioner
`
`v.
`
`APLIX IP HOLDINGS CORPORATION
`Patent Owner
`
`____________
`
`Case No. IPR2015-00729
`Patent 7,280,097
` ____________
`
`
`
`DECLARATION OF DR. GREGORY F. WELCH
`
`
`
`
`
`SCEA Ex. 1009 Page 1
`
`
`
`
`
`I, Gregory F. Welch, hereby declare the following:
`I.
`BACKGROUND AND QUALIFICATIONS
`1.
`I have summarized in this section my educational background, career
`
`history, and other relevant qualifications. I have also attached a current version of
`
`my Curriculum Vitae as Exhibit 1010.
`
`2.
`
`I am the Florida Hospital Endowed Chair in Healthcare Simulation at
`
`the University of Central Florida (UCF) with appointments in the College of
`
`Nursing, the Computer Science Division of the Department of Electrical
`
`Engineering and Computer Science, and the Institute for Simulation & Training. I
`
`am also an Adjunct Professor of Computer Science at the University of North
`
`Carolina at Chapel Hill (UNC), and a Visiting Professor in the Graduate School of
`
`Media Design at Keio University (Japan).
`
`3.
`
`In 1986 I received a B.S. degree in Electrical Technology from Purdue
`
`University (with Highest Distinction), in 1995 I received a M.S. in Computer
`
`Science from UNC, and in 1997 I received a Ph.D. in Computer Science from
`
`UNC.
`
`4.
`
`Previously I have been a Research Professor at the University of North
`
`Carolina at Chapel Hill, a Senior Engineer at Northrop-Grumman’s Defense
`
`Systems Division where I worked on the AN/ALQ-135 electronic countermeasures
`
`system for the U.S. Air Force F-15 Eagle, and a member of the technical staff of
`
`
`
`1
`
`SCEA Ex. 1009 Page 2
`
`
`
`
`
`NASA’s Jet Propulsion Laboratory where I worked on the Voyager Spacecraft
`
`project.
`
`5. My current research interests include human-computer interaction,
`
`virtual and augmented reality, human motion tracking systems, three-dimensional
`
`(3D) telepresence, projector-based graphics, computer vision and view synthesis,
`
`and medical applications of computers for training, assessment, and practice. I
`
`have co-authored over 100 peer-reviewed publications in these areas, and I am a
`
`co-inventor on multiple patents. I currently supervise over $2M in research
`
`funding (active grants at UCF and UNC), and am jointly responsible for over
`
`$23M in grants overall since 1996, from (for example) the Office of Naval
`
`Research (ONR), the National Science Foundation (NSF), The National Institutes
`
`of Health National Library of Medicine (NIH-NLM), the Defense Advanced
`
`Research Projects Agency (DARPA), the Department of Energy (DOE), and
`
`private companies—all
`
`involving multi-disciplinary and multi-institutional
`
`projects. I am the Co -Director of the Synthetic Reality Laboratory and the
`
`Interactive Systems & User Experience Research Cluster of Excellence at UCF.
`
`6.
`
`I have co-chaired major academic conferences (including IEEE
`
`ISMAR 2012 and Virtual Reality 2013), served on numerous program committees,
`
`co-chaired workshops, and serve as a peer reviewer for many conferences and
`
`journals. I serve on the editorial board of the International Journal of Virtual
`
`
`
`2
`
`SCEA Ex. 1009 Page 3
`
`
`
`
`
`Reality, and I am an Associate Editor for the journal Presence: Teleoperators and
`
`Virtual Environments, and an Associate Editor for the journal Frontiers in Virtual
`
`Environments.
`
`7.
`
`I am a member of the IEEE Computer Society, the Association for
`
`Computing Machinery (ACM), the Southern Nursing Research Society (SNRS),
`
`the International Nursing Association for Clinical Simulation & Learning
`
`(INACSL), and the Society for Simulation in Healthcare (SSH).
`
`8. My work in human interface systems and the associated the computer-
`
`based sensing (e.g., hardware, software, sources/sensors, signal processing, and
`
`algorithms) goes back at least to the early 1980s when I was an undergraduate at
`
`Purdue University, e.g., with the co-development of an environmentally aware
`
`“smart wheelchair” for children with Cerebral Palsy. Fellow student and co-
`
`developer James Williams and I received an “Outstanding Senior Design Project
`
`award for “The Easy Chair” in 1986. One of my core contributions to the
`
`wheelchair project was the development of a novel customizable touch pad to be
`
`used by the children to control the wheelchair. The touch pad was customizable to
`
`allow caregivers to design an interface that was tailored to each child and their
`
`unique (limited) affordances.
`
`9. My work in computer-based sensing continued into the late 1980s and
`
`early 1990s when I worked at NASA’s Jet Propulsion Laboratory (the Voyager
`
`
`
`3
`
`SCEA Ex. 1009 Page 4
`
`
`
`
`
`Project) and Northrop-Grumman’s Defense Systems Division (a radar jammer). In
`
`particular in 1992 I attended graduate school at the University of North Carolina at
`
`Chapel Hill (UNC) where I studied/worked under the direction of Prof. Gary
`
`Bishop and others as a graduate student. My Ph.D. work, which I completed in
`
`1996, introduced a new Kalman filter-based Single Constraint at a Time (SCAAT)
`
`approach to sensing for applications such as human motion tracking in Virtual
`
`Environments. It was one of the critical aspects of the HiBall system for tracking
`
`heads, hands, and user interface devices. This system was commercialized by
`
`3rdTech and sold until approximately 2012.
`
`10. While a research faculty member at UNC from 1996-2011, I co-led/led
`
`the Tracker Research Group, the 3D Computer Vision Group, and the Office of the
`
`Future Group. This includes conception and acquisition of contracts and grants;
`
`leading the subsequent research efforts; advising students; serving on Ph.D.
`
`committees; etc. In the mid-to-late 1990s I co-developed methods for tracking
`
`human motion by combining measurements from cameras that recognize and track
`
`natural features in the environment, with inertial and other sensing devices
`
`(accelerometers and gyros). Along the way I have developed human interface
`
`devices for research (e.g., physician interfaces for medical visualization and
`
`telepresence), and supervised the development of human interface devices by
`
`students in a Virtual Worlds course at UNC.
`
`
`
`4
`
`SCEA Ex. 1009 Page 5
`
`
`
`
`
`11. Presently I am leading the development of several human interface
`
`projects related to healthcare simulation and training at UCF. For example, we
`
`recently developed a system that comprises a plastic model of a human head with
`
`rear-projected dynamic (live) animated facial imagery, and whole-head touch
`
`sensing for medical training, e.g., to train healthcare professionals in the diagnosis
`
`of strokes. We are also working on a full-body version of the same (an entire
`
`“Physical-Virtual Patient Bed”).
`
`12. While
`
`leading
`
`research at UNC and UCF,
`
`I have co-
`
`developed/prototyped many sensing systems associated with human user
`
`interfaces, including systems that used cameras and image processing/computer
`
`vision techniques to recognize and track features and objects (both natural and
`
`instrumented) for applications such as human motion tracking, vehicle tracking
`
`(for 3D environment mapping), Augmented Reality, and automatic camera-
`
`projector calibration.
`
`13. Handheld devices have been used in the field of Augmented Reality
`
`for almost as long as the field has existed, and I have been engaged in the research
`
`community for almost that long, reviewing papers, co-offering tutorials on related
`
`topics, and organizing conferences. My personal research has also included
`
`handheld devices. For example, in 1995, I authored a paper surveying power
`
`management techniques for mobile (i.e., handheld) devices. I have also developed
`
`
`
`5
`
`SCEA Ex. 1009 Page 6
`
`
`
`
`
`systems for remote medical consultation using handheld mobile devices, such as
`
`PDAs, and associated methods for novel tracking and interaction.
`
`14.
`
`I have been retained as an expert in this matter by Petitioner. As part of
`
`my work in connection with this proceeding, I have analyzed the following:
`
`• U.S. Patent 7,280,097 (Ex. 1001);
`• International Publication No. WO 2004/007041 to Mollinari et al. (Ex.
`1003);
`• U.S. Patent No. 5,903,257 to Nishiumi et al. (Ex. 1004);
`• U.S. Patent Publication No. 2004/0139254 to Tu et al. (Ex. 1005);
`• U.S. Patent Publication No. 2004/0137983 to Kerr et al. (Ex. 1007);
`• U.S. Patent Publication No. 2005/0221894 to Lum et al. (Ex. 1008);
`• Jenny Preece, Yvonne Rogers, Helen Sharp, David Benyon, Simon
`Holland, and Tom Carey, Human-Computer Interaction, Addison-
`Wesley (1994) (Ex. 1012);
`• Microsoft Computer Dictionary 5th Edition, Microsoft Corporation
`(2002) (Ex. 1013);
`• Paul E. Ceruzzi, A History of Modern Computing, The MIT Press (1998)
`(Ex. 1014);
`• Ivan E. Sutherland, The Ultimate Display, Proceedings of IFIP Congress
`pp. 506-508 (1965) (Ex. 1015);
`• Paul Atkinson, A Bitter Pill to Swallow: The Rise and Fall of the Tablet
`Computer, MIT Design Issues: Vol. 24, No. 4 (Autumn 2008) (Ex.
`1016);
`• William K. English, Douglas C. Engelbart, Melvyn L. Berman, Display-
`Selection Techniques for Text Manipulation, IEEE Transactions of
`Human Factors in Electronics, Vol. HFE-8, No. 1 (March 1967) (Ex.
`1017);
`• Mary Brandel, 1963: The debut of ASCII,
`http://edition.cnn.com/TECH/computing/9907/06/1963.idg/ (posted on
`July 6, 1999) reprinted from ComputerWorld April 12, 1999 at p. 87 (Ex.
`1018);
`• Mark J.P. Wolf, Before the Crash Early Video Game History, Wayne
`State University Press (2012) (Ex. 1019);
`• Andrew Cunningham, The NES turns 30, How it Began, worked, and
`saved an industry, http://arstechnica.com/gaming/2013/07/time-to-feel-
`
`
`
`6
`
`SCEA Ex. 1009 Page 7
`
`
`
`
`
`old-inside-the-nes-on-its-30th-birthday (Posted July 15, 2013) (Ex.
`1020);
`• N64: The Launch, Nintendo Power Vol. 88, pp. 10-13(September 1996)
`(Ex. 1021);
`• M. Wiley, Nintendo WaveBird Review,
`http://www.ign.com/articles/2002/06/11/nintendo-wavebird-review
`(Posted June 11, 2002)(Ex. 1022);
`• Michael McCandless, The PalmPilot and the handheld revolution, IEEE
`Expert pp. 6-8 (November/December 1997) (Ex. 1023);
`• Neil J. Salkind, PalmPilotTM and PalmTM Organizers! I Didn’t Know You
`Could Do That…TM, Sybex Inc. (2000) (Ex. 1024);
`• Newton Solutions Guide Software Peripherals and Accessories for
`Newton PDAs, JointSolutions Marketing (1995) (Ex. 1025);
`• Newton Apple MessagePad Handbook, Apple Computer, Inc. (1995)
`(Ex. 1026);
`• Jason O.B. Soh, Bernard C.Y. Tan, Mobile Gaming, Communications of
`the ACM Vol. 51, No. 3, pp. 35-39 (March 2008) (Ex. 1027);
`• N-GAGE Extended User’s Guide, Nokia Corporation (2003) (Ex. 1028);
`• User’s Guide PCS Vision Picture Phone VGA1000, Samsung
`Telecommunications America (2003) (Ex. 1029);
`• Graeme Wearden, Sony Ericsson unveils latest phones,
`http://www.zdnet.com/article/sony-ericsson-unveils-latest-phones/
`(posted September 3, 2003) (Ex. 1030).
`
`II. LEGAL FRAMEWORK
`A. Obviousness
`15.
`I am a technical expert and do not offer any legal opinions. However,
`
`counsel has informed me that a person cannot obtain a patent on an invention if his
`
`or her invention would have been obvious to a person of ordinary skill in the art at
`
`the time the invention was made. A conclusion of obviousness may be founded
`
`upon more than a single item of prior art. In determining whether prior art
`
`references render a claim obvious, counsel has informed me that courts consider
`
`
`
`7
`
`SCEA Ex. 1009 Page 8
`
`
`
`
`
`the following factors: (1) the scope and content of the prior art, (2) the differences
`
`between the prior art and the claims at issue, (3) the level of skill in the pertinent
`
`art, and (4) secondary considerations of non-obviousness. In addition, the
`
`obviousness inquiry should not be done in hindsight. Instead, the obviousness
`
`inquiry should be done through the eyes of one of skill in the relevant art at the
`
`time the patent was filed.
`
`16.
`
`In considering whether certain prior art renders a particular patent
`
`claim obvious, counsel has informed me that courts allow a technical expert to
`
`consider the scope and content of the prior art, including the fact that one of skill in
`
`the art would regularly look to the disclosures in patents, trade publications,
`
`journal articles, industry standards, product literature and documentation, texts
`
`describing competitive technologies, requests for comment published by standard
`
`setting organizations, and materials from industry conferences. I believe that all of
`
`the references that my opinions in this IPR are based upon are well within the
`
`range of references a person of ordinary skill in the art would consult to address the
`
`type of problems described in the Challenged Claims.
`
`17.
`
`I understand that the United States Supreme Court’s most recent
`
`statement on the standard for determining whether a patent is obvious was stated in
`
`2007 in the KSR decision. Specifically, I understand that the existence of an
`
`explicit teaching, suggestion, or motivation to combine known elements of the
`
`
`
`8
`
`SCEA Ex. 1009 Page 9
`
`
`
`
`
`prior art is a sufficient, but not a necessary, condition to a finding of obviousness.
`
`Thus, the teaching suggestion-motivation test is not to be applied rigidly in an
`
`obviousness analysis. In determining whether the subject matter of a patent claim
`
`is obvious, neither the particular motivation nor the avowed purpose of the
`
`patentee controls. Instead, the important consideration is the objective reach of the
`
`claim. In other words, if the claim extends to what is obvious, then the claim is
`
`invalid.
`
` I further understand the obviousness analysis often necessitates
`
`consideration of the interrelated teachings of multiple patents, the effects of
`
`demands known to the technological community or present in the marketplace, and
`
`the background knowledge possessed by a person having ordinary skill in the art.
`
`All of these issues may be considered to determine whether there was an apparent
`
`reason to combine the known elements in the fashion claimed by the patent.
`
`18.
`
`I also understand that in conducting an obviousness analysis, a precise
`
`teaching directed to the specific subject matter of the challenged claim need not be
`
`sought out because it is appropriate to take account of the inferences and creative
`
`steps that a person of ordinary skill in the art would employ. I understand that the
`
`prior art considered can be directed to any need or problem known in the field of
`
`endeavor at the time of invention and can provide a reason for combining the
`
`elements of the prior art in the manner claimed. In other words, the prior art need
`
`not be directed towards solving the same specific problem as the problem
`
`
`
`9
`
`SCEA Ex. 1009 Page 10
`
`
`
`
`
`addressed by the patent. Further, the individual prior art references themselves
`
`need not all be directed towards solving the same problem. Under the KSR
`
`obviousness standard, common sense is important and should be considered.
`
`Common sense teaches that familiar items may have obvious uses beyond their
`
`primary purposes.
`
`19.
`
`I also understand that the fact that a particular combination of prior art
`
`elements was “obvious to try” may indicate that the combination was obvious even
`
`if no one attempted the combination. If the combination was obvious to try
`
`(regardless of whether it was actually tried) or leads to anticipated success, then it
`
`is likely the result of ordinary skill and common sense rather than innovation. I
`
`further understand that in many fields it may be that there is little discussion of
`
`obvious techniques or combinations, and it often may be the case that market
`
`demand, rather than scientific literature or knowledge, will drive the design of an
`
`invention. I understand that an invention that is a combination of prior art must do
`
`more than yield predictable results to be non-obvious.
`
`20.
`
`I understand that for a patent claim to be obvious, the claim must be
`
`obvious to a person of ordinary skill in the art at the time of the invention. I
`
`understand that the factors to consider in determining the level of ordinary skill in
`
`the art include (1) the educational level and experience of people working in the
`
`field at the time the invention was made, (2) the types of problems faced in the art
`
`
`
`10
`
`SCEA Ex. 1009 Page 11
`
`
`
`
`
`and the solutions found to those problems, and (3) the sophistication of the
`
`technology in the field.
`
`21.
`
`I understand that at least the following rationales may support a finding
`
`of obviousness:
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`
`22.
`
`Combining prior art elements according to known methods to yield
`predictable results;
`Simple substitution of one known element for another to obtain
`predictable results;
`Use of a known technique to improve similar devices (methods, or
`products) in the same way;
`Applying a known technique to a known device (method, or product)
`ready for improvement to yield predictable results;
`“Obvious to try”—choosing from a finite number of identified,
`predictable solutions, with a reasonable expectation of success;
`A predictable variation of work in the same or a different field of
`endeavor, which a person of ordinary skill would be able to
`implement;
`If, at the time of the alleged invention, there existed a known problem
`for which there was an obvious solution encompassed by the patent’s
`claim;
`Known work in one field of endeavor may prompt variations of it for
`use in either the same field or a different one based on technological
`incentives or other market forces if the variations would have been
`predictable to one of ordinary skill in the art; and/or
`Some teaching, suggestion, or motivation in the prior art that would
`have led one of ordinary skill to modify the prior-art reference or to
`combine prior-art reference teachings to arrive at the claimed
`invention.
`
`I understand that even if a prima facie case of obviousness is
`
`established, the final determination of obviousness must also consider “secondary
`
`considerations” if presented. In most instances, the patentee raises these secondary
`
`
`
`11
`
`SCEA Ex. 1009 Page 12
`
`
`
`
`
`considerations of non-obviousness. In that context, the patentee argues an
`
`invention would not have been obvious in view of these considerations, which
`
`include: (a) commercial success of a product due to the merits of the claimed
`
`invention; (b) a long-felt, but unsatisfied need for the invention; (c) failure of
`
`others to find the solution provided by the claimed invention; (d) deliberate
`
`copying of the invention by others; (e) unexpected results achieved by the
`
`invention; (f) praise of the invention by others skilled in the art; (g) lack of
`
`independent simultaneous invention within a comparatively short space of time;
`
`(h) teaching away from the invention in the prior art.
`
`23.
`
` I further understand that secondary considerations evidence is only
`
`relevant if the offering party establishes a connection, or nexus, between the
`
`evidence and the claimed invention. The nexus cannot be based on prior art
`
`features. The establishment of a nexus is a question of fact. While I understand
`
`that Patent Owner has not offered any secondary considerations at this time, I will
`
`supplement my opinions in the event that Patent Owner raises secondary
`
`considerations during the course of this proceeding.
`
`III. OPINION
`A. Background of the Technology
`24. Computers have always required interfaces in order to interact with
`
`humans. In the beginning of the modern computing era (i.e., from the 1950s until
`
`the 1970s), users conventionally interacted with computers using punch cards.
`
`
`
`12
`
`SCEA Ex. 1009 Page 13
`
`
`
`
`
`Exhibit 1012, Preece at pp. 4-5. At that time, computers were large, costly and ill
`
`suited for use by private individuals. Id. This limited the potential user pool to
`
`engineers, programmers, and scientists. Id. However, despite these limitations,
`
`early researchers were endeavoring to make computers more interactive and easier
`
`to use. The first input devices were developed as a result of these efforts.
`
`25. An input device is widely understood to be a peripheral piece of
`
`computer hardware used to provide information to a computer system. Ex. 1013,
`
`Microsoft Computer Dictionary at p. 274. Once the video display terminal was
`
`developed, typewriter keyboards enabled users to enter text into the computer and
`
`view it on the display. Ex. 1014, Ceruzzi at p. 256. By the mid-1960s, keyboards
`
`were the most common types of computer input devices because they were
`
`inexpensive, reliable, and produced easily
`
`transmitted signals. Ex. 1015,
`
`Sutherland at p. 1. In addition to inputting text, the computer keyboard also
`
`enabled the user to navigate through on-screen menus. Ex. 1014, Ceruzzi at p.
`
`256.
`
`26. The light pen, which was initially developed at MIT in the 1950s, was
`
`the one of the earliest input devices that allowed the user to interact with a video
`
`display. Ex. 1016, Atkinson at p. 7. Using a light pen, the user could point to items
`
`on the display or draw on the display in order to provide input to the computer.
`
`
`
`13
`
`SCEA Ex. 1009 Page 14
`
`
`
`
`
`Ex. 1015, Sutherland at p. 1. Joysticks, knobs and pushbuttons were also common
`
`input elements. Id.
`
`27. Douglas Engelbart and his group of researchers developed the
`
`computer mouse in an effort to improve communication between humans and
`
`computers. Ex. 1014, Ceruzzi at p. 260. Using the mouse, users could control the
`
`position of a cursor in order to select items on a CRT display. Ex. 1017, English at
`
`pp. 5-6. The computer mouse was shown to be more efficient and effective at
`
`selection tasks than other peripheral input devices of the time including the light
`
`pen and the joystick. Ex. 1014, Ceruzzi at p. 260.
`
`Ex. 1017, English at p. 5.
`
`
`
`
`
`14
`
`SCEA Ex. 1009 Page 15
`
`
`
`
`
`28. For as long as there have been input devices, computers have been able
`
`to map user inputs to particular functions. In the 1960s, the ASCII code standard
`
`mapped specific keys on a keyboard to codes that could be interpreted by
`
`computers as characters. Ex. 1014, Ceruzzi at p. 226; see also, Ex. 1018, Brandel.
`
`29. The first personal computers were developed in the 1970s. Ex. 1012,
`
`Preece at p. 5. These computers were much smaller, less costly, and more
`
`interactive, which enabled their use by the general population. Id. By the early
`
`1980s, most personal computers, including the Apple Macintosh, for example,
`
`utilized keyboards and computer mice as primary input devices. Ex. 1014, Ceruzzi
`
`at p. 274. At this time, these general-purpose personal computers were used for
`
`many applications including, word processing and games. Id. at p. 275. The first
`
`“transportable” computers were also developed in the early 1980s. One such early
`
`portable computer was the Osborne, which included a peripheral keyboard:
`
`
`
`15
`
`SCEA Ex. 1009 Page 16
`
`
`
`
`
`
`
`Ex. 1014, Ceruzzi at p. 278.
`
`30. The first home video game consoles, which were introduced in the
`
`1970s, also included peripheral input devices. The Magnavox Odyssey 2, for
`
`example, included at 48-character keyboard and handheld joystick controllers:
`
`
`
`16
`
`SCEA Ex. 1009 Page 17
`
`
`
`
`
`GIVE THE ULTIMATE
`COMPUTER VIDEO GAME.
`
`store nearest you.
`
`The program-
`mable compuIer
`video game sys-
`Icm. Odyssuy’ hm
`u sophisiicoled 49
`(horodvr oIpho»
`numvrio: keyboard.
`Over 30 games
`’0 play. Cdynvy"
`offers a wide variety
`oforcodo sports and
`(‘dUCGIIOntlI games.
`
`J-gamc can-
`Iridgc and hand
`conlrols included.
`Odyswy" gnu-x you
`mon- mun olhu v:dv:o
`games Odyswy‘
`“w ulhmnu- qunw
`Hm ulhmate gm. Fvom
`the ongvnnvor
`Mognovox
`Dial 800-447-
`4700 Go" Imc Ior
`
`Fug. 2.2. Magnavox Odyssey2 advertisement. From www.relrolst.com/zoo9/o4/15/magnavox-odys-
`sey-z-ad-from-198o.
`
`
`
`I
`
`Ex. 1019, Wolf at p. 33.
`EX. 1019, Wolfat p. 33.
`
`
`
`17
`17
`
`SCEA EX. 1009 Page 18
`
`SCEA Ex. 1009 Page 18
`
`
`
`
`
`31. Over time, video game console manufacturers began designing
`
`handheld controllers with combinations of digital direction pads (d-pads), analog
`
`control sticks, and various digital buttons.
`
` For example, the Nintendo
`
`Entertainment System, which was released in the United States in 1985, included
`
`controllers with a d-pad and four other buttons on a front surface:
`
`Ex. 1020, Cunningham. The Nintendo 64, which was released in 1996, included a
`
`controller with input elements on the top, bottom, and side surfaces:
`
`
`
`
`
`18
`
`SCEA Ex. 1009 Page 19
`
`
`
`
`
`
`
` Ex. 1021, Nintendo Power at p. 12. The input elements included buttons, a D-pad,
`
`and an analog joystick. Id.; see also, Ex. 1004, Nishiumi.
`
`32.
`
`Initially, game controllers were coupled to video game consoles
`
`through the use of various wired technologies; however, controllers including
`
`combinations of digital and analog control elements
`
`that used wireless
`
`communication technologies were being sold as early as 2002 with the introduction
`
`of the Nintendo WaveBird that came with a wireless adapter. Ex. 1022, Wiley.
`
`
`
`19
`
`SCEA Ex. 1009 Page 20
`
`
`
`
`
`33. General-purpose handheld computing devices
`
`started gaining
`
`popularity in the early 1990s and achieved widespread use when the first PalmPilot
`
`was introduced in 1996 by 3Com. Ex. 1023, McCandless at p. 6. The PalmPilot
`
`came with several built-in applications, including a calendar, to do list, and address
`
`book, and also allowed users to install additional applications, including games.
`
`Id.; see also, Ex. 1024, Salkind. Other Personal Digital Assistants (PDAs) were on
`
`the market at this time that had similar functionalities and also offered optional
`
`peripheral input devices. The Apple Newton MessagePad, for example, was able
`
`to connect to a peripheral keyboard that could be used in lieu of a handwriting
`
`recognition system or onscreen keyboard to input text. Ex. 1025, Newton
`
`Solutions Guide at p. 46; Ex. 1026, Newton MessagePad Handbook at p. 23.
`
`34. By the early 2000s, the line between PDAs and mobile phones was
`
`beginning
`
`to blur, and “mobile gaming” was viewed by mobile phone
`
`manufactures as a way to attract new customers or as an incentive to get existing
`
`customers to upgrade their mobile devices. Ex. 1027, Soh at p. 38. In 2002, Nokia
`
`announced the N-Gage, which combined the features of a mobile phone, MP3
`
`player and mobile gaming device. Id. Like previous PDAs, the N-Gage also
`
`allowed the user to run messaging, calendar, and game applications. Ex. 1028, N-
`
`GAGE Extended User’s Guide at p. 8.
`
`
`
`20
`
`SCEA Ex. 1009 Page 21
`
`
`
`
`
`35. At this time, some mobile phone manufacturers also began offering
`
`peripheral game controllers as accessories for cellular phones in order to provide
`
`home game console-like control to mobile games. Ex. 1027, Soh at p. 38. For
`
`example, Sprint PCS released a game pad controller for use with Samsung cellular
`
`phones in 2003. Id. Users could download games to their Samsung phone and
`
`play them using the game pad controller. Ex. 1029, User’s Guide PCS Vision
`
`Picture Phone VGA1000 at pp. 138-139. The users could also “re-map the keys to
`
`allow easier game play.” Id. at p. 140. The key re-mapping function allowed the
`
`user “to customize how the game pad operates when a specific button is pressed.”
`
`Id.
`
`36. Also at this time (i.e., September 2003), Sony Ericsson released the
`
`Gameboard EGB-10:
`
`
`
`21
`
`
`
`SCEA Ex. 1009 Page 22
`
`
`
`
`
`The EGB-10 was essentially a simplified version of the Sony Playstation controller
`
`that was used to control game applications running on the Sony Ericsson Z600
`
`mobile phone. Ex. 1030, Wearden.
`
`37. Thus, from the large, room-sized mainframe computers of the
`
`midcentury to the small, handheld computers and cellular phones of the early
`
`2000s, computers, no matter their size, have always had peripheral input devices.
`
`B.
`
`Level of a Person Having Ordinary Skill in the Art
`
`38.
`
`In determining the characteristics of a hypothetical person of ordinary
`
`skill in the art of the ‘097 Patent at the time of the claimed invention, which
`
`counsel has informed me is October 11, 2005, I considered several factors,
`
`including the type of problems encountered in the art, the solutions to those
`
`problems, the rapidity with which innovations are made in the field, the
`
`sophistication of the technology, and the education level of active workers in the
`
`field. I also placed myself back in the time frame of the claimed invention, and
`
`considered the students who I had taught and with whom I had worked at that time.
`
`39.
`
`In my opinion, a person of ordinary skill in the art would be a person
`
`with (1) an undergraduate degree in computer science, computer engineering,
`
`electrical engineering, or similar technical fields; (2) a working knowledge of
`
`computers - including handheld computing devices, and their processing, storage,
`
`hardware—including input devices, and software; (3) two to four years of
`
`
`
`22
`
`SCEA Ex. 1009 Page 23
`
`
`
`
`
`experience (or, with a graduate degree in the above-stated fields, one to two years
`
`of experience) with designing and developing human-computer interfaces and the
`
`associated technologies.
`
`40. Based on my education, training, and professional experience in the
`
`field of the claimed invention, I am familiar with the level and abilities of a person
`
`of ordinary skill in the art at the time of the claimed invention. Additionally, I was
`
`at least a person having ordinary skill in the art as of the priority date of the ‘097
`
`Patent.
`
`C. Obvious to Combine Mollinari and Nishiumi
`
`41. Mollinari recognizes the difficulties associated with using a mobile
`
`phone keypad to control a game application. Ex. 1003 at 1:30-2:2. To solve this
`
`problem, Mollinari proposes a game controller used to control a game application
`
`running on a mobile telecommunications terminal. Id. at Abstract. The game
`
`controller includes input elements that are used to activate specific game functions.
`
`Id. at 6:31-35; Fig. 1(a).
`
`
`
`23
`
`SCEA Ex. 1009 Page 24
`
`
`
`
`
`
`
`This enables the user to utilize the input elements of the game controller instead of
`
`the keypad of the mobile terminal while playing games. Id. at 3:4-9.
`
`42. Mollinari recognizes that many applications used on the device may
`
`benefit from a variety of controls, including combinations of digital and analog
`
`controls. Id. at 7:4-18. Additionally, Mollinari recognizes that providing an
`
`ergonomic shape to the game controller (e.g., comfortable controller shape and
`
`input control placement) allows bimanually control and easy access to multiple
`
`input controls (e.g., thumb-only control of joysticks). Id. at 6:28-7:2, 5:14–18,
`
`12:30–32. Like Mollinari, the ‘097 Patent similarly recognizes the difficulties
`
`associated with using a mobile phone keypad to control a game application. Ex.
`
`1001 at Abstract, 1:11-18, 14:36-42. Thus, the ‘097 Patent also provides a game
`
`controller that functions as a remote controller to control software applications
`
`
`
`24
`
`SCEA Ex. 1009 Page 25
`
`
`
`
`
`running on the handheld host device. Id. at Abstract, 8:25-34. Additionally, the
`
`game controller includes input elements, “such as keys, buttons, pressure sensor
`
`pads, touch pads, rotary dials, thumb joysticks, linear strip sensors or other
`
`actuators associated with one or more sensors that can be manipulated by one or
`
`both of a human user's thumbs or fingers.” Id. at 9:6-11. Thus, both the ‘097
`
`Patent and Mollinari are directed to providing small, handheld devices that enable
`
`the user to quickly and efficiently input data through a peripheral input device.
`
`Mollinari is therefore in the same field of endeavor and is analogous to the ‘097
`
`Patent.
`
`43. Like Mollinari, Nishiumi also describes a game controller having
`
`multiple input elements. Ex. 1004 at 6:45-7:5, Figs. 7-8. The game controller of
`
`Nishiumi provides input signals to an image processing system such as a personal
`
`computer or video game system. Id. at 4:21-61, Figs. 1-2.
`
`The game controller includes input elements located on the front, side, and rear
`
`surfaces. Id. at 6:45-7:5, Figs. 7-8. Like the ‘097 Patent, Nishiumi aims to provide
`
`
`
`25
`
`
`
`SCEA Ex. 1009 Page 26
`
`
`
`
`
`small, handheld devices that enable the user to quickly and efficiently input data
`
`through a peripheral input device. Thus, Nishiumi is also in the same field of
`
`end