Case: IPR2015-00230
`Patent: 7,463,245
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`________________________
`
`SONY COMPUTER ENTERTAINMENT AMERICA LLC
`Petitioner
`
`v.
`
`APLIX IP HOLDINGS CORPORATION
`Patent Owner
`
`________________________
`
`Case No. IPR2015-00230
`
`Patent No. 7,463,245
`
`DECLARATION OF DR. KARON MACLEAN
`
`Mail stop PATENT BOARD
`Patent Trial and Appeal Board
`U.S. Patent & Trademark Office
`P.O. Box 1450
`Alexandra, VA 22313-145
`
`
`Patent: 7,463,245
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`________________________
`
`SONY COMPUTER ENTERTAINMENT AMERICA LLC
`Petitioner
`
`v.
`
`APLIX IP HOLDINGS CORPORATION
`Patent Owner
`
`________________________
`
`Case No. IPR2015-00230
`
`Patent No. 7,463,245
`
`DECLARATION OF DR. KARON MACLEAN
`
`Mail stop PATENT BOARD
`Patent Trial and Appeal Board
`U.S. Patent & Trademark Office
`P.O. Box 1450
`Alexandra, VA 22313-145
`
`
`
`TABLE OF CONTENTS
`
`Page
`
`I.
`
`Background & Qualifications.................................................................................... 1
`
`A.
`
`B.
`
`C.
`
`D.
`
`E.
`
`Education background and career history.................................................... 1
`
`Research expertise ........................................................................................... 2
`
`Collaborations with Industry ......................................................................... 6
`
`Professional service and recognitions........................................................... 7
`
`Exhibits analyzed............................................................................................. 8
`
`II.
`
`Legal Framework ...................................................................................................... 10
`
`A.
`
`B.
`
`C.
`
`D.
`
`Scope and Content of the Prior Art............................................................ 10
`
`Differences Between the Art and the Invention ....................................... 11
`
`The Level of Skill in the Art......................................................................... 13
`
`Objective Indicia............................................................................................ 13
`
`III. Opinion ...................................................................................................................... 14
`
`A.
`
`Background of the Technology ................................................................... 14
`
`1.
`
`2.
`
`3.
`
`Short history of handheld computing devices up to 2003............ 14
`
`Short History of touchpads and use on computers generally and
`also on handheld devices up to 2003............................................... 15
`
`Designing Handheld Devices........................................................... 18
`
`B.
`
`Summary of the ’245 Patent......................................................................... 20
`
`ii
`
`
`
`C.
`C.
`
`Review of Petition Arguments..................................................................... 23
`Review of Petition Arguments ................................................................... .. 23
`
`1.
`1.
`
`2.
`
`2.
`
`3.
`3.
`
`4.
`
`4.
`
`5.
`
`5.
`
`Person of ordinary skill in the art .................................................... 23
`Person of ordinary skill in the art .................................................. .. 23
`
`Liebenow in view of Andrews.......................................................... 24
`
`Liebenow in View of Andrews ........................................................ .. 24
`
`Liebenow in view of Hedberg.......................................................... 43
`Liebenow in View of Hedberg ........................................................ .. 43
`
`Liebenow in view of Martin ............................................................. 55
`
`Liebenow in View of Martin ........................................................... .. 55
`
`Griffin in view of Liebenow............................................................. 62
`
`Griffin in View of Liebenow ........................................................... .. 62
`
`IV.
`
`IV.
`
`Conclusion ........................................................................................................... 83
`
`Conclusion ......................................................................................................... .. 83
`
`iii
`
`iii
`
`
`
`I. Background & Qualifications
`
`1.
`
`I have summarized in this section my educational background, career
`
`history, and other relevant qualifications. I have also attached a current version of my
`
`Curriculum Vitae as Ex. 2004.
`
`A.
`
`2.
`
`Educational background and career history
`
`I am presently a Full Professor at the University of British Columbia,
`
`with a regular appointment in Computer Science in the Faculty of Science, and a
`
`courtesy appointment in Mechanical Engineering in the Faculty of Applied Science. I
`
`have recently been a Visiting Professor at the University of Colorado (Boulder,
`
`Colorado, USA) and at the University of Canterbury (Christchurch, NZ).
`
`3.
`
`In 1986 I received a B.S. degree in Mechanical Engineering and
`
`Biological Sciences from Stanford University. In 1988 I received a M.S. in Mechanical
`
`Engineering from Massachusetts of Technology, and 1996 a Ph.D. in Mechanical
`
`Engineering from Massachusetts of Technology.
`
`4.
`
`From 1989 to 1991 I worked as a project engineer at the University of
`
`Utah’s Center for Engineering Design in Salt Lake City, UT. From 1996 to 2000 I was
`
`a Member of Research Staff and Project Lead at Interval Research Corporation in
`
`Palo Alto, CA.
`
`1
`
`
`
`B.
`
`5.
`
`Research expertise
`
`The majority of my research relates to different aspects of the design of
`
`physical interfaces for human use. These draw on disciplines of Human Computer
`
`Interaction (HCI), robotics and mechatronics, human biomechanics, psychophysics
`
`and cognition, and design practices more broadly. As a sample, my research includes
`
`design of interaction techniques for handheld devices, invention of flexible sensors
`
`for touch sensing and machine-learning recognition of human gestural touch,
`
`prototyping tools for haptic designers, psychophysically-based tools for creating sets
`
`of “haptic icons” (meaningful signals rendered by tactile and force-feedback
`
`actuators), and design and deployment of emotional touch in therapeutic applications
`
`of touch-based affective robots. Other research interests include human-robot
`
`interaction (HRI), accelerometer based motion recognition and interactive motion
`
`guidance for users, and mobile tools for systematizing health issue investigation.
`
`6.
`
`With my students and postdocs, I have co-authored over 100 peer-
`
`reviewed publications in these areas. In the fields of HCI and HRI, top-tier (peer-
`
`reviewed, 20-25% acceptance) conferences are a primary mode of publication. My
`
`team has received seven “Best Paper” awards and two additional runners-up in the
`
`last 10 years at such conferences, including the ACM Conference on Human Factors
`
`in Computing Systems (CHI), ACM/IEEE Human Robot Interaction (HRI), IEEE
`
`HAPTICS and ACM ICMI (Int’l Conference on Multimodal Interaction).
`
`2
`
`
`
`7.
`
`I am a co-inventor on four patents, filed between 1999 and 2001 and
`
`listed in my CV, which address novel physical interaction design concepts arising
`
`from research of the team I led at Interval Research in Palo Alto While they rely on
`
`creative and sometimes sophisticated disposition of technology elements that were
`
`newly available at the time, their primary contributions were in how to make
`
`technology work in a physical way for real problems that people had or foreseeably
`
`would have – for example, in controlling new forms of digital media, and the
`
`emerging challenges of mobile interactions.
`
`8.
`
`I have graduated 33 graduate students and supervised 6 postdocs and 52
`
`directed undergraduate research projects since 2000. Many of these individuals are
`
`presently employed in influential human-computer interaction roles in prominent
`
`companies including Apple, RIM, Microsoft, Google, Phillips, Canada’s National
`
`Research Council, Immersion Corp., Dolby, and the National Basketball Association
`
`as well as numerous smaller startups and other ventures. Two initiated a startup based
`
`on the research they did with me (Haptok Inc.).
`
`9.
`
`I currently am solely or jointly responsible for CAN$0.7M in research
`
`funding (FY 2014 funding at UBC and elsewhere) and for CAN$7.8M in grants
`
`overall since 2000 from (for example) Canada’s NSERC, CIHR and CFI funding
`
`agencies, the USA’s NSF and numerous private companies including Nokia, Nissan,
`
`3
`
`
`
`Immersion, and General Motors. The majority of this funding involves multi-
`
`disciplinary and multi-institutional projects.
`
`10. My work in human interface systems has its roots in an undergraduate
`
`research job at Stanford/VA Rehabilitation Research & Development Center, where
`
`in the early 1980’s I worked with Felix Zajac and his group on modeling the human
`
`musculoskeletal system. I continued in bioengineering with my Master’s research in
`
`MIT’s Mechanical Engineering Design Division, and a rehabilitation group creating
`
`computer-controlled prosthetics and robotics. My own research in functional
`
`neuromuscular stimulation was ultimately directed at restoring function to spinal-cord
`
`injured humans; with my supervisor William Durfee I characterized motor responses
`
`to neural stimulation, and devised a system by which to control motor activation. I
`
`then worked for two years at the University of Utah’s Center for Engineering Design,
`
`a world leader in complex human-controlled robotics, on the design and digital
`
`controls for high-degree of freedom, teleoperated, force-feedback robots for undersea
`
`and animatronic applications.
`
`11.
`
`For my MIT PhD (1990-1996) I developed and psychophysically
`
`characterized one of the very first of what later came to be known as haptic interfaces.
`
`At a time we knew absolutely nothing of how humans would perceive virtual force
`
`displays or how to create high fidelity haptic representations of real environments. I
`
`studied this by simulating and haptically rendering linear pushbutton switch profiles
`
`4
`
`
`
`with high performance motors and examining human responses to these displays
`
`relative to traditional switches. MIT at this time was a haptics breeding-ground,
`
`spawning the majority of early haptics researchers and entrepreneurs internationally.
`
`For example, my fellow students and friends included Thomas Massie, inventor of the
`
`Phantom haptic device; Margaret Minsky, credited with publication of a first haptic
`
`application; Ed Colgate, who later became the first Editor-in-Chief of the new IEEE
`
`Transactions on Haptics, and also chaired the first haptics conference in 1992; and
`
`Elaine Chen, a haptic inventor.
`
`12.
`
`I took up a postdoc and then researcher / team leader roles at Interval
`
`Research Corporation (1996-2000), a Palo Alto technology think-tank funded by Paul
`
`Allen, where I led a diverse team of designers and engineers in developing novel
`
`haptic interaction techniques and technology, predominantly for handheld interaction
`
`with physical devices. This was a period of frenetic development in user interface
`
`technology, much of it centered in Silicon Valley, and Interval had a front-row seat to
`
`observe and participate in it.
`
`13.
`
`At UBC, I am now primarily engaged on three threads of work: physical
`
`and generally handheld interface design including using these devices for motion
`
`sensing and guidance feedback, with a special focus on attentional processes; affective
`
`human-robot interaction; and mobile health applications. My background, spanning
`
`robot controls, bioengineering /psychophysics and human computer interaction
`
`5
`
`
`
`(HCI) design, is somewhat unusual and I have exploited this to bridge my various
`
`communities. At present, I am leading a group of about ten students and postdocs at
`
`UBC from a mix of computational, engineering and arts backgrounds, working on
`
`interaction techniques and sensing technology for mobile and HRI contexts.
`
`14.
`
`In summary, my research has been largely focused on physical human
`
`interfaces and human neuro, musculoskeletal and cognitive systems since the early
`
`1980’s, and I have been actively engaged in communities at the forefront of
`
`developments in these areas throughout my career.
`
`C.
`
`15.
`
`Collaborations with Industry
`
`As an HCI designer, it has always been useful to align with end-users
`
`and industries who inform me of human problems and industry needs. I will mention
`
`a few. Immersion Corp has been a primary holder of haptic intellectual property since
`
`the late 1990’s, whom I consulted for in 2000, and have received funding from and
`
`sent students to for employment ever since. I have worked closely with two
`
`automotive companies, Nissan on cockpit controls and General Motors on human-
`
`robot interaction for collaborative automation. I have given invited talks at Apple
`
`regarding my interaction design techniques, and three of my graduates work there
`
`now on the Apple Watch project. At RIM, one of my graduates spearheaded the push
`
`to put haptics onto the Blackberry in the late 2000’s. I am on the advisory board of
`
`Tangible Haptics, a Northwestern startup by Edward Colgate commercializing
`
`6
`
`
`
`variable-friction surface haptic feedback, a concept which he invented; my students
`
`and I developed interaction techniques for it which then won a Best of CHI award. I
`
`am on the advisory board of the EU research consortium WEARHAP. I have also
`
`worked with numerous small companies over the years in collaborative and advisory
`
`capacities.
`
`D.
`16.
`
`Professional service and recognition
`I have co-chaired major academic conferences, most recently the 2010
`
`and 2012 IEEE Haptics Symposium, and served in 2014 as Area Chair (Interaction
`
`Techniques and Devices) for the largest HCI conference, ACM CHI. I currently serve
`
`on the editorial board of the Int’l Journal of Human-Computer Studies (IJHCS), and
`
`was a founding associate editor of the IEEE Transactions on Haptics, whose 2008
`
`creation I orchestrated. I have served on numerous program committees in HCI,
`
`robotics, haptics and mobile interaction. In 2014-2015, I was on the Awards
`
`committees for all three major haptics conferences (Eurohaptics 2014, Haptics
`
`Symposium 2014 and Worldhaptics 2015).
`
`17.
`
`I am a Senior Member of the Association for Computing Machinery
`
`(ACM), and a Senior Member of the Institute for Electrical and Electronics
`
`Engineering (IEEE), active in the Computer Society and the Robotics and
`
`Automation Society.
`
`7
`
`
`
`18.
`
`I have been the prime architect of UBC’s HCI curriculum, creating it in
`
`2000 and twice re-vamping a program that has reappeared in many other universities;
`
`it is now among the largest in North America. I have also taught numerous summer
`
`schools and workshops with the aim of bringing HCI practices to engineering haptics
`
`audiences, and vice versa.
`
`19.
`
`I am currently leading a major pan-university initiative to create an HCI
`
`Institute at UBC, including proposals to secure major funding for it.
`
`20.
`
`I was the recipient of the 2001 Peter Wall Early Career Scholar, the 2006
`
`Izzak Walton Killam Memorial Faculty Research Fellowship, the 2008 Charles A
`
`McDowell Award (the top research prize at UBC), and received a 2013 NSERC
`
`Accelerator Grant, awarded to top science researchers nationally in Canada.
`
`E.
`
`Exhibits analyzed
`
`Exhibit 1001
`Exhibit 1003
`Exhibit 1004
`Exhibit 1005
`Exhibit 1006
`Exhibit 1007
`Exhibit 1010
`
`Exhibit 2001
`
`Petition for Inter Partes Review of U.S. Patent No. 7,463,245
`Decision Instituting Inter Partes Review of U.S. Patent No.
`7,463,245 (Paper 16)
`U.S. Patent No. 7,463,245 to Beth Marcus et al.
`U.S. Patent Application Publication No. 2002/0118175 to
`Liebenow et al.
`International Publication No. WO2000/59594 to Andrews et al.
`International Publication No. WO1999/18495 to Hedberg
`U.S. Patent No. 7,336,260 to Martin et al.
`U.S. Patent Application Publication No. 2003/0020692 to
`Griffin et al.
`Expert Declaration of Dr. Gregory F. Welch
`Amended Complaint in Aplix IP Holdings Corporation v. Sony
`Computer Entertainment Inc. and Sony Computer Entertainment
`America LLC, Case No. 1:14-cv-12745
`
`8
`
`
`
`Exhibit 2002
`
`Exhibit 2007
`
`Exhibit 2008
`
`Exhibit 2009
`
`Exhibit 2010
`
`Exhibit 2011
`
`Exhibit 2012
`
`Exhibit 2013
`
`Exhibit 2014
`
`Exhibit 2015
`
`Exhibit 2016
`
`Exhibit 2017
`
`Exhibit 2018
`
`Exhibit 2019
`
`Office Action dated October 5, 2006, from the file history of
`U.S. Ser. No. 10/699,555 (parent of the application for U.S.
`Patent 7,463,245).
`Allen, J. P., Handheld Computing Predictions: What Went Wrong?,
`Proceedings of the 1st International Symposium on Handheld
`and Ubiquitous Computing, Karlsruhe, Germany: Springer-
`Verlag, 1999, pp. 117-123
`Wikipedia entry on “List of Blackberry products” at
`https://en.wikipedia.org/wiki/List_of_BlackBerry_products,
`accessed 8/3/2015
`Keyboard image at http://www.computerhistory.org/
`collections/catalog/102642008, accessed 8/2/2015
`Patent US 5,305,017
`Wikipedia entry on “Touchpad” at
`https://en.wikipedia.org/wiki/Touchpad, accessed 8/1/2015
`Buxton, W., Multi-Touch Systems that I Have Known and Loved, at
`www.billbuxton.com/multitouchOverview.html, accessed
`8/2/2015
`Walker, G., A Review of Technologies for Sensing Contact Location on
`the Surface of a Display, Journal of the Society for Information
`Display, vol. 20:8, pp. 413-440, 2012
`Wikipedia entry on “IBM Simon” at
`https://en.wikipedia.org/wiki/ IBM_Simon, accessed 8/2/2015
`Wikipedia entry on “Casio PB 1000” at
`https://en.wikipedia.org/wiki/Casio_PB-1000, accessed
`8/5/2015
`Blickenstorfer, C., NeoNode N1, Can a unique interface put this
`compelling smart phone on the map? At
`http://pencomputing.com/WinCE/neonode-n1-review.html,
`accessed 8/2/2015
`Wikipedia entry on “List of iPod models” at
`https://en.wikipedia.org/wiki/List_of_iPod_models, accessed
`8/2/2015
`Barker, M., Microsoft Teams with Interlink Electronics for Xbox
`Controllers, at www.Gamasutra.com, accessed 8/2/2015
`Hinckley, K., Pierce, J., Sinclair, M., and Horvitz, E., Sensing
`Techniques for Mobile Interaction, Proceedings of the 13th Annual
`ACM Symposium on User Interface Software and Technology,
`San Diego, California, USA: ACM, 2000, pp. 91-100
`
`9
`
`
`
`Exhibit 2020
`
`Exhibit 2021
`
`Exhibit 2022
`
`Wikipedia entry on “Camera phone” at
`https://en.wikipedia.org/wiki/Camera_phone, accessed
`8/3/2015
`Partridge, K., Chatterjee, S., Sazawal, V., Borriello, G., and
`Want, R., Tilttype: Accelerometer-Supported Text Entry for Very Small
`Devices,” in Proceedings of the 15th annual ACM symposium on
`User interface software and technology. Paris, France: ACM,
`2002, pp. 201-204
`Wigdor, D. and Balakrishnan, R., Tilttext: Using Tilt for Text Input
`to Mobile Phones” in Proceedings of the 16th annual ACM
`symposium on User interface software and technology.
`Vancouver, Canada: ACM, 2003, pp. 81-90
`
`II. Legal Framework
`21.
`Counsel has informed me that a patent claim is obvious if the
`
`differences between it and the prior art are such that it would have been obvious at
`
`the time the invention was made to a person having ordinary skill in the relevant art.
`
`To determine obviousness I must consider the scope and content of the prior art, the
`
`differences between the prior art and the claimed invention, the level of ordinary skill
`
`in the field of the invention, and any relevant objective considerations of
`
`nonobviousness. I have taken each of these factors into account in my analysis
`
`below.
`
`A.
`22.
`
`Scope and Content of the Prior Art
`I am informed that prior art for obviousness purposes is limited to art
`
`that is analogous, meaning that it is from the same “field of endeavor” as the patented
`
`invention, or is “reasonably pertinent” to the particular problem with which the
`
`inventor was involved.
`
`10
`
`
`
`23.
`
`I understand that two pieces of art that relate to the same general subject
`
`matter are not necessarily in the same field of endeavor. Similarity in the structure
`
`and function of the invention and the prior art is indicative that the prior art is within
`
`the same field of endeavor, and differences in structure and function suggest the
`
`contrary.
`
`24.
`
`I am further informed that another way that art can be considered
`
`analogous is if it is “reasonably pertinent” to the problem the inventor was trying to
`
`solve. A reasonably pertinent reference is one that logically would have commended
`
`itself to an inventor's attention in considering the problem the inventor was trying to
`
`solve.
`
`25.
`
`Even when art is analogous, however, I understand that it is also
`
`important to consider the differences between the art and the invention as claimed
`
`and whether a person of ordinary skill would have had a motivation to combine the
`
`references in the manner of the claim.
`
`Differences Between the Art and the Invention
`B.
`26. Most inventions are an assembly of pre-existing technology. It is
`
`therefore not proper to say that an invention was obvious merely because it is a
`
`recombination of the prior art. Such an invention might be perfectly obvious in
`
`hindsight, but that is not the test for validity purposes. Rather, the test is it would
`
`have been obvious at the time of the invention to a person of ordinary skill who did
`
`11
`
`
`
`not have the benefit of the inventor’s teachings. Moreover, sometimes it is the
`
`recognition of the problem to be solved, rather than the claimed solution, that was
`
`non-obvious.
`
`27.
`
`A reason to combine references might come from one of various
`
`sources. For instance, there might be a specific marketplace pressure or existing
`
`design demand that would motivate a person of ordinary skill in the art to make the
`
`combination. It does not matter what the source of the reason to combine is. What
`
`is important is clearly to identify a reason that a person of ordinary skill at the relevant
`
`time would have appreciated. Otherwise, the combination is obvious only in
`
`hindsight.
`
`28.
`
`I understand rather that there has to be some set of facts apart from the
`
`teachings of the patent itself to provide the glue to combine the art in the manner of
`
`the claims. One cannot simply put together pieces of art like a jigsaw puzzle using the
`
`patent as the picture on the puzzle box lid. That is impermissible hindsight.
`
`29.
`
`I understand that it can also be pertinent whether there was a reason in
`
`the art not to make the claimed combination. It could be that a person of ordinary
`
`skill would have considered the combination difficult to make or that the prior art
`
`“teaches away” from the claimed invention.
`
`12
`
`
`
`C.
`30.
`
`The Level of Skill in the Art
`I am informed that obviousness is to be judged from the perspective of
`
`a person of ordinary skill, not the perspective of an expert. Even if the invention
`
`would have been obvious to the brightest minds in the field, that does not render it
`
`unpatentable. I understand that highly educated and trained individuals know more,
`
`and thus more inventions are obvious to them. On the other hand, individuals with
`
`little education or training would have more difficulty in making connections and
`
`therefore fewer inventions would be obvious to them.
`
`D. Objective Indicia
`31.
`Finally, I am informed that one should take into account any objective
`
`evidence that suggest that the invention may have been obvious or nonobvious. For
`
`instance, if many different inventors came up with the same combination around the
`
`same time, that may suggest that it was an obvious combination to make. On the
`
`other hand, if the elements of the invention were around for a long time without
`
`anyone making the combination, that is an objective fact suggesting that the
`
`combination may not have been so obvious. I understand that there is no exhaustive
`
`list of objective factors.
`
`13
`
`
`
`III. Opinion
`
`A.
`
`Background of the Technology
`
`1.
`
`Short history of handheld computing devices up to 2003
`
`32.
`
`A number of technological and commercial developments were
`
`influential or significant with respect to handheld computing device interfaces and
`
`usage. While hand-sized, portable computers have existed in some form since at least
`
`the late 1980s, Palm Computing’s 1996 release of the Palm Pilot inaugurated the
`
`“PDA” (personal digital assistant) era and the idea of a handheld device that could
`
`work as a complement to a personal computer (Ex. 2007, Allen, at p. 5). Pen-based
`
`user input dominated handheld devices around this time.
`
`33.
`
`The decade from 1995 through the mid-2000’s saw enormous
`
`exploration and innovation in handheld input modalities, in part because the purpose
`
`and potential of handheld computing was far from resolved: what would they be good
`
`for, who would use them, and how would these devices fit into the growing
`
`ecosystem of other computational instruments at users’ disposal?
`
`34. Devices during this period tended to be specialized. Cell phones, PDAs,
`
`music players, cameras, game consoles and remote controls had and still have the
`
`liberty to focus on working well for one thing. Devices were marketed as
`
`“smartphones” as early as 2000 (e.g. the Ericsson R380, Palm Kyocera, and Treo 90),
`
`but combined a small number of relatively compatible functions, e.g., a PDA plus a
`
`14
`
`
`
`mobile phone. Blackberry released the first “convergent” 6000 series smartphone in
`
`2003 (Ex. 2008 at p. 1), with push email, mobile telephony, text, internet fax and web
`
`browsing.
`
`35. However, it was not until the 2007 iPhone and the 2008 Android HTC
`
`Dream that the “Swiss Army knife” smartphone truly emerged. From that point,
`
`smartphone uses spanned business, music and gaming as well as telephony and
`
`calendaring, and third party “apps” that it seemed could do just about everything else.
`
`After this point, smartphones lost most differentiation in size, shape and input
`
`modalities, with general convergence onto a thin slab with touchscreen controls.
`
`36.
`
`Innovation driven by use cases and enabling of helpful interaction techniques:
`
`Academic literature around this time illustrates considerable attention to how users
`
`might provide input to handheld devices, and also how users might use these devices
`
`to interact with other aspects of their environment. Their reports address both
`
`purpose (potential tasks, often newly envisioned ones) and interaction techniques
`
`(often enabled by new sensing modalities).
`
`2.
`
`Short history of touchpads, and their use on computers
`generally and on handheld devices up to 2003
`
`37.
`
`Touch sensors have proved attractive computer input elements in
`
`contexts where space is limited. The first appearance of touchpads as a commercially
`
`available computer input modality is commonly cited as an inclusion on the Apollo
`
`computer’s keyboard (Exs. 2009, 2011 at p. 2) in 1982, although there were earlier
`
`15
`
`
`
`commercial uses. It spread more widely as GlidePoint technology, invented in the
`
`1980’s (U.S. Patent No. 5,305,017, Ex. 2010), developed by Cirque Corporation
`
`(1991) and incorporated into early Apple products (1994 PowerBook series). Several
`
`US companies, including Logitech, Synaptics and MicroTouch Systems, released their
`
`own versions in the mid-1990’s; descendants of these early technologies have
`
`continued to dominate consumer electronics ever since (many sources, including a
`
`reasonable summary on Wikipedia, Ex. 2011). Multitouch variants and interaction
`
`techniques that exploited them attained substantial visibility by the mid 1980’s (e.g.,
`
`Ex. 2012, Buxton, at p. 1), and appeared in some commercial settings, e.g. air traffic
`
`control terminals even earlier, in the 1960’s (Ex. 2013, Walker, p. 413).
`
`38.
`
`Leading technologies: The earliest technology families utilized as computer
`
`input elements are capacitive (allows sensing of presence and absolute position of a bare
`
`finger based on its electrical properties) and resistive (measures the resistance drop
`
`when two electrically striped sheets are pressed together, thus requiring some touch
`
`pressure). These have different strengths and weaknesses; for example, resistive
`
`touchpads (can have higher resolution, require nonzero pressure) work well with
`
`stylus input, whereas capacitive touchpads and touchscreens require a bare finger but
`
`accommodate light touch. The capacitive touch screen we see in smartphones today is
`
`comprised of multiple layers of glass with fused conductive coating, but other
`
`16
`
`
`
`construction methods and materials support sensing on curved and even flexible
`
`surfaces. Many variants have been derived from both.
`
`39.
`
`Touch sensing on handhelds: IBM’s Simon Personal Communicator (1994),
`
`was touted as the first device to include telephone and PDA features, and also had a
`
`stylus touchscreen with handwriting input (Ex. 2014 at p. 1); but it was predated by
`
`non-phone devices such as the 1987 Casio pocket computer PB-1000 (Ex. 2015),
`
`which had 16 touch-sensitive keys (“LCD sensitive areas”) built into the screen in a
`
`4x4 grid. The 2004 Neonode N1 Mobile phone (announced in 2002) had an optical
`
`touch-screen, and was considered the first touchscreen to go beyond finger-tapping
`
`on buttons to make use of finger swipe-type gestures (Ex. 2016 at pp. 1-3). The
`
`iPhone, responsible for massively popularizing multitouch input in handhelds, was
`
`released in 2007.
`
`40. Of non-screen touch sensors, the best known in this period may be
`
`Apple’s iPod, first released in 2001 with a mechanical scroll wheel, and with a one-
`
`dimensional touch-sensitive wheel in 2002 (seee.g.Ex. 2017). The “click wheel”
`
`innovation where mechanical tactile switches were incorporated beneath the touch
`
`wheel did not appear until 2004.
`
`41.
`
`Another relevant player is Interlink Electronics (1984), a company which
`
`has produced wide varieties of flexible force-sensing resistors packaged in ways
`
`convenient for prototyping and manufacture, and initially most popular with
`
`17
`
`
`
`electronic music instrument controllers. In the early 2000’s, these enabled inexpensive
`
`analog sensing of point and 2-D position and pressure. Interlink participated in the
`
`Xbox console design (2001), providing six buttons with pressure sensitivity, e.g. for
`
`sensitive control of character movements (Ex. 2018).
`
`42.
`
`The incorporation of touch sensing on hand-sized surfaces to enable
`
`interactions in ways other than buttons or stylus movements on a flat touch-sensitive
`
`screen developed more slowly than gestural control. Ex. 2019, a paper related to
`
`sensing techniques for mobile interaction, describes a simple prototype capacitive
`
`sensor applied to a curved surface to indicate touch contact or non-contact over a
`
`larger surface (e.g., a mouse) without specific location information. This led others to
`
`envision more involved uses that demanded more sophisticated but low-cost sensing
`
`on arbitrary surfaces, which developed in turn; as we see in the ’245 patent herein.
`
`3.
`
`Designing handheld devices
`
`43.
`
`Several core requirements dominate the design of handheld device user
`
`interfaces, and differentiate them from design of other types of systems. First, they are
`
`hand-sized and hand-used, as opposed to used while sitting on a desk or mounted in a
`
`wall. The device must afford accessing and activating its input modalities, typically
`
`with fingers and thumbs, while at least one hand is also supporting and stabilizing it.
`
`Second, the majority of handheld devices in 2003 were for mobile use. Interaction may
`
`18
`
`
`
`occur while the user is physically in motion, in environments with demands on
`
`attention and perception. The device must be conveniently carried.
`
`44.
`
`The need to be both small and usable while held imposes harsh
`
`restrictions on everything else it does, particularly viewing, data entry and talking into
`
`it. Solving these constraints successfully requires judicious choices and tradeoffs.
`
`Added physical features and capabilities that do not adequately fit with or
`
`complement the device as a whole take up space and interfere with other uses. All
`
`input elements and displays need to meet ergonomic aspects of usability.
`
`45.
`
`An example of these tradeoffs is illustrated in the choice and layout of
`
`input elements. A laptop computer can be assumed to need a keyboard and a pointing
`
`device of some kind; but even miniaturized versions of these elements will take up
`
`most of the surface area of a handheld device. A designer must consider what the
`
`device will be used for, in what conditions. If the user needs to provide text, this
`
`might be done with a small keyboard (physical or touchpad based) or a vast variety of
`
`alternative text entry methods that require both an input element and an interaction
`
`technique that effectively exploits it. Any of these require careful thought to how the
`
`device is grasped and supported, what digits are free to slide, press or tap while
`
`supported, and visibility of a graphical display. This is before considering other
`
`utilities the device must provide. If text entry is not a frequent or extensive use
`
`19
`
`
`
`modality, an experienced designer will likely diminish the surface area devoted to its
`
`support.
`
`46. General purpose handheld devices are extremely hard to design. It is no
`
`accident that specialized devices preceded the general-purpose “Swiss Army knife”
`
`smartphone described above by many years; the latter’s
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
