IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`In re Patent of: Magnus Goertz
`U.S. Patent No.:
`8,812,993 Attorney Docket No.: 50095-0016IP1
`Issue Date:
`August 19, 2014
`
`Appl. Serial No.: 13/310,755
`
`Filing Date:
`December 4, 2011
`
`Title:
`USER INTERFACE
`
`
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 8,812,993 PURSUANT TO 35 U.S.C. §§311–319, 37 C.F.R. §4
`
`

`

`III.
`
`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`TABLE OF CONTENTS
`EXHIBITS ............................................................................................................... iii
`I.
`REQUIREMENTS FOR IPR UNDER 37 C.F.R. §42.104 ............................. 1
`II.
`THE ’993 PATENT ......................................................................................... 3
`A.
`The ’993 Patent Alleged Invention ....................................................... 3
`B.
`The ’993 Patent Prosecution History .................................................... 7
`C.
`Claim Construction ............................................................................... 8
`PRIOR ART ..................................................................................................... 8
`A. Well-Known Concepts by the Critical Date.......................................... 8
`B.
`Prior Art Combinations ....................................................................... 10
`1.
`Hisatomi, Ren, Allard-656, Tanaka, and Kodama .................... 10
`2.
`Hansen and Gillespie ................................................................ 18
`IV. EXPLANATION OF UNPATENTABILITY ............................................... 27
`A. Ground 1A: Claims 1-3, 7, and 8 are rendered obvious by
`Hisatomi in view of POSA knowledge and/or Ren ............................ 27
`Ground 1B: Claim 4 is rendered obvious by Hisatomi in view
`of POSA Knowledge and Allard-656 and/or Ren and Allard-
`656 ....................................................................................................... 54
`Ground 1C: Claim 5 is Rendered Obvious by Hisatomi in view
`of POSA knowledge and Tanaka and/or Ren and Tanaka .................. 58
`D. Ground 1D: Claim 6 is Rendered Obvious by Hisatomi in view
`of POSA knowledge and Kodama and/or Ren and Kodama .............. 60
`Ground 2A: Claims 1-3, 7, and 8 are rendered obvious by
`Hansen in view of Gillespie ................................................................ 63
`
`B.
`
`C.
`
`E.
`
`i
`
`

`

`F.
`
`V.
`
`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`Grounds 2B, 2C, and 2D: Claims 4-6 are rendered obvious by
`Hansen and Gillespie in view of Allard-656, Tanaka, or
`Kodama................................................................................................ 79
`INSTITUTION SHOULD NOT BE DENIED UNDER 35 U.S.C. §
`325(d) ............................................................................................................. 82
`A.
`Petitioner’s Grounds and Obviousness Rationales Were Not
`Previously Considered by The Office ................................................. 83
`The Office Erred in Granting the ’993 Patent ..................................... 86
`B.
`VI. PTAB DISCRETION UNDER § 314(a) SHOULD NOT PRECLUDE
`INSTITUTION .............................................................................................. 88
`A.
`Factor 1: Institution Will Enable a Stay .............................................. 88
`B.
`Factor 2: Uncertain District Court Schedule ...................................... 89
`C.
`Factor 3: Early Stage of Parallel Proceedings .................................... 90
`D.
`Factor 4: The Petition Raises Unique Issues ...................................... 91
`E.
`Factor 5: The Petition Will Enable Cancellation of Claims That
`Might Be Reasserted ........................................................................... 92
`Factor 6: Other Considerations Support Institution ........................... 93
`F.
`VII. CONCLUSION .............................................................................................. 93
`VIII. MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 93
`A.
`Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) .......................... 93
`B.
`Related Matters Under 37 C.F.R. § 42.8(b)(2) ................................... 93
`C.
`Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) ............... 94
`D.
`Service Information ............................................................................. 94
`
`
`
`
`
`
`ii
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EXHIBITS
`
`Exhibit No.
`1001
`1002
`1003
`1004
`1005
`
`1006
`
`1007
`1008
`1009
`1010
`1011
`1012
`1013
`1014
`
`1015
`
`1016
`1017
`
`1018
`
`1019
`
`Description
`U.S. Patent No. 8,812,993 (“the ’993 patent”)
`Declaration of Benjamin B. Bederson
`File History for U.S. Patent No. 8,812,993
`Benjamin B. Bederson CV
`Certified translation of JP Published Patent Application No.
`2002-55750 (“Hisatomi”), published February 20, 2002
`Xiangshi Ren & Shinji Moriyama, “Improving Selection on Pen-
`Based Systems: A Study of Pen-Based Interaction for Selection
`Tasks,” ACM Transactions on Computer-Human Interaction,
`Vol. 7, No. 3, September 2000, pp. 384-416 (“Ren”)
`U.S. Patent No. 5,422,656 to Allard et al. (“Allard-656”)
`U.S. Patent No. 5,249,296 to Tanaka (“Tanaka”)
`U.S. Patent No. 5,615,384 to Allard et al. (“Allard-384”)
`U.S. Patent No. 5,537,608 to Beatty et al. (“Beatty”)
`U.S. Patent No. 5,903,268 to Hirayama (“Hirayama”)
`U.S. Patent No. 5,305,435 to Bronson (“Bronson”)
`U.S. Patent No. 6,133,898 to Ludolph et al. (“Ludolph”)
`Tammara T. A. Combs and Benjamin B. Bederson “Does
`zooming improve image browsing?” Proceedings of the Fourth
`ACM Conference on Digital Libraries (DL ’99), ACM, New
`York, NY, USA, (August 1999) 130-137
`Dean Harris Rubine, “The Automatic Recognition of Gestures,”
`CMU-CS-91-202, December, 1991.
`U.S. Patent No. 6,710,791 to Kodama et al. (“Kodama”)
`IBM Corp., User’s Manual, “Simon Says ‘Here’s How!’” Part.
`No. 82G2557 (1994) (“IBM”)
`Benjamin B. Bederson & James D. Hollan, Pad++: A Zooming
`Graphical Interface for Exploring Alternate Interface Physics,
`UIST ’94 Proceedings of the 7th Annual ACM Symposium on
`User Interface Software and Technology 17 (1994), DOI:
`http://dx.doi.org/10.1145/192426.192435
`David Rogers et al., Tossing Objects in a Desktop Environment,
`submitted to Conference on Human Factors in Computing
`Systems (1996)
`
`iii
`
`

`

`1020
`
`1021
`
`1022
`
`1023
`
`1024
`
`1025
`
`1026
`1027
`1028
`1029
`1030
`
`1031
`1032
`
`1033
`1034
`1035
`
`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`David Rogers et al., Exemplar Figure of Tossing from Tossing
`Objects in a Desktop Environment, submitted to Conference on
`Human Factors in Computing Systems (1996)
`Benjamin B. Bederson, Fisheye Menus, UIST ’00 Proceedings of
`ACM Conference on User Interface Software and Technology
`217 (2000), DOI: 10.1145/354401.317382
`Leslie E Chipman et al., SlideBar: Analysis of a Linear Input
`Device, 23 Behaviour & Info. Tech. 1 (2004), DOI:
`10.1080/01449290310001638487
`Hilary Browne et al., Designing a Collaborative Finger Painting
`Application for Children, HCIL-2000-17, CS-TR-4184,
`UMIACS-TR-2000-66 (Sept. 2000), available at
`https://hcil.umd.edu/pub-perm-link/?id=2000-17
`Pekka Parhi, Amy K. Karlson, and Benjamin B. Bederson. 2006.
`Target size study for one-handed thumb use on small touchscreen
`devices. In Proceedings of the 8th Conference on Human-
`Computer Interaction with Mobile Devices and Services
`(MobileHCI ’06). Association for Computing Machinery, New
`York, NY, USA, 203–210.
`DOI:https://doi.org/10.1145/1152215.1152260
`Karlson, Amy & Bederson, Benjamin & Contreras-Vidal, José.
`(2008). Understanding One-Handed Use of Mobile Devices.
`Handbook of Research on User Interface Design and Evaluation
`for Mobile Technology. 86-101. DOI:10.4018/978-1-59904-871-
`0.ch006
`Apple Newton Message Pad Handbook (1993)
`Handbook for Palm m500 Series Handhelds (1998)
`HP Jornada 520 Series Pocket PC User Guide (2001)
`U.S. Patent No. 5,821,930 to Hansen (“Hansen”)
`U.S. Patent Application Publication No. 2005/0024341
`(“Gillespie”)
`Declaration of Mr. Jacob Munford
`Norman, D. A. (1988). The psychology of everyday things.
`BasicBooks. IBSN: 0-465-06709-3.
`U.S. Publication No. 2001/0043189 to Brisebois (“Brisebois”)
`Trial Delay Statistics
`Order Governing Proceedings - Patent Case, Neonode
`Smartphone LLC v. Apple Inc., 6:20-cv-00505 (W.D.Tex. Oct. 5,
`2020)
`
`iv
`
`

`

`1036
`
`1037
`
`1038
`
`1039
`
`1040
`
`
`1041
`
`1042
`
`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`Order Governing Proceedings - Patent Case, Neonode
`Smartphone LLC v. Samsung Electronics Co. Ltd. and Samsung
`Electronics America, Inc., 6:20-cv-00507 (W.D.Tex. Oct. 5,
`2020)
`Order Granting Motion Continue Case Management Conference
`(CMC), Neonode Smartphone LLC v. Apple Inc., 6:20-cv-00505
`(W.D.Tex.) (W.D.Tex. [[DATE]])
`Order Granting Motion Continue Case Management Conference
`(CMC), Neonode Smartphone LLC v. Samsung Electronics Co.
`Ltd. and Samsung Electronics America, Inc., 6:20-cv-00507
`(W.D.Tex. Oct. 7, 2020)
`Order Setting Markman Hearing, Neonode Smartphone LLC v.
`Apple Inc., 6:20-cv-00505 (W.D. Tex. Oct. 26, 2020)
`Order Setting Markman Hearing, Neonode Smartphone LLC v.
`Samsung Electronics Co. Ltd. and Samsung Electronics America,
`Inc., 6:20-cv-00507 (W.D. Tex. Oct. 26, 2020)
`November 6, 2020 Letter from Apple Counsel to Neonode
`Counsel
`November 6, 2020 Letter from Samsung Counsel to Neonode
`Counsel
`
`v
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`Samsung Electronics Co. Ltd. and Samsung Electronics America, Inc.
`
`(“Samsung”) and Apple Inc. (collectively “Petitioners”) petition for Inter Partes
`
`Review (“IPR”) of claims 1-8 (“the Challenged Claims”) of U.S. Patent No.
`
`8,812,993 (“the ’993 patent”).
`
`I.
`
`REQUIREMENTS FOR IPR UNDER 37 C.F.R. §42.104
`A. Grounds for Standing Under 37 C.F.R. §42.104(a)
`
`Petitioners certify the ’993 patent is available for IPR. This petition is being
`
`filed within one year of service of complaints against Petitioners. Petitioners are
`
`not barred or estopped from requesting this review of the Challenged Claims.
`
`B. Challenge Under §42.104(b) and Relief Requested
`
`Petitioners request cancellation of the Challenged Claims on the grounds
`
`below.
`
`Ground Claims
`
`Basis (§103)
`
`1A
`
`1B
`
`1C
`
`1D
`
`2A
`
`1-3, 7, 8 Hisatomi, Ren
`
`4
`
`5
`
`6
`
`Hisatomi, Ren, Allard-656
`
`Hisatomi, Ren, Tanaka
`
`Hisatomi, Ren, Kodama
`
`1-3, 7, 8 Hansen, Gillespie
`
`1
`
`

`

`Ground Claims
`
`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`Basis (§103)
`
`2B
`
`2C
`
`2D
`
`4
`
`5
`
`6
`
`Hansen, Gillespie, Allard-656
`
`Hansen, Gillespie, Tanaka
`
`Hansen, Gillespie, Kodama
`
`For this Petition, Petitioners treat 12/10/2002 as the earliest effective filing
`
`date of the Challenged Claims (“the Critical Date”). The following references are
`
`prior art:
`
`Reference
`Hisatomi
`
`Date(s)
`02/20/2002
`
`Basis
`§102(a)
`
`Ren
`
`09/20001
`
`§102(b)
`
`Allard-656
`
`6/6/1995
`
`§102(b)
`
`Tanaka
`
`09/28/1993
`
`§102(b)
`
`Kodama
`
`1/31/1997
`
`§102(e)
`
`Hansen
`
`10/13/1998
`
`§102(b)
`
`
`
` 1
`
` EX1031, ¶¶6-8 (authenticating public availability date).
`
`2
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`Basis
`§102(e)
`
`Date(s)
`4/17/20022
`
`Reference
`Gillespie
`
`II. THE ’993 PATENT
`A. The ’993 Patent Alleged Invention
`The ’993 Patent “relates to a user interface for a mobile handheld computer
`
`unit, which … comprises a touch sensitive area, and [the] touch sensitive area is
`
`divided into a menu area and a display area.” EX1001, 1:14-17, 3:57-65; EX1002,
`
`¶¶34-35.
`
`The ’993 patent’s FIG. 1 illustrates “a user interface according to the present
`
`invention [] specifically adapted to computer units comprising a touch sensitive
`
`area 1, which is divided into a menu area 2 and a display area 3.” EX1001, 3:58-
`
`61. The ’993 patent describes that “[a]ccording to the present invention the menu
`
`area 2 is adapted to present a representation of a first 21, a second 22 and a third 23
`
`predefined function.” EX1001, 4:8-10. FIG. 1 illustrates a display area 3 (red) and
`
`a menu area (blue). EX1002, ¶36.
`
`
`
` 2
`
` Gillespie is actually entitled to a prior art date of 5/16/01 pursuant to Dynamic
`
`Drinkware, LLC, v. National Graphics, Inc., 800 F.3d 1375 (Fed. Cir. 2015).
`
`3
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EX1001, FIG. 1 (annotated).
`
`
`
`The ’993 patent explains “FIG. 2 shows that any one of these three functions
`
`21, 22, 23 can be activated when the touch sensitive area 1 detects a movement of
`
`an object 4 with its starting point A within the representation of a function on the
`
`menu area 2 and with a direction B from the menu area 2 to the display area 3.”
`
`EX1001, 4:14-19. Figure 2 describes a touch beginning within menu area 2 (blue)
`
`and continuing into display area 3 (red). EX1002, ¶37.
`
`4
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EX1001, FIG. 2.
`
`
`
`The ’993 patent states “[t]he first function 21 is a general application
`
`dependent function”, and “if the first function 21 is activated, then the display area
`
`3 is adapted to display icons … representing services or functions depending on the
`
`current active application.” Id., 4:11-23. “If no application is currently active on
`
`the computer unit, then the icons 211, 212, 213, 214, 215, 216 are adapted to
`
`represent services or settings of the operations system of the computer unit, such as
`
`background picture, clock, alarm 215, users 213, help 211, etc.” EX1001, 4:36-40’
`
`EX1002, ¶¶38-39.
`
`5
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EX1001, FIG. 3 (annotated, icons in green).
`
`
`
`“FIG. 4 shows that selection of a preferred service or setting is done by
`
`tapping C, D on corresponding icon 213.” EX1001, 4:41-42, FIG. 4; EX1002,
`
`¶¶40-41.
`
`6
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EX1001, FIG. 4 (annotated, location of an icon in green).
`
`
`
`B.
`The ’993 Patent Prosecution History
`After several office actions and responses, Applicant amended the claims to
`
`require the otherwise activatable graphic be present “in a strip along at least one
`
`edge of the display screen.” EX-1003, 148-149. The Examiner rejected this
`
`feature based on Brisebois (EX1033), which was alleged to teach a physical,
`
`“flexible strip on [an] active edge” through which a “user can touch or press an
`
`7
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`area of active edge input device.” EX1003, 92; EX1002, ¶¶42-46.
`
`Applicant argued that Brisebois’s active edge 120 through which a user
`
`provides input is separate and distinct from display 110, so the user’s input cannot
`
`leave the active edge 120. EX1003, 77-79; see also EX1033, FIG. 1, ¶¶[0028]-
`
`[0030]. To distinguish Brisebois, Applicant amended the claims “to recite that the
`
`object glides away from and out of the strip,” as part of the multi-step gesture.
`
`EX1003, 73, 79 (emphasis in original). Applicant argued Brisebois fails to
`
`disclose this feature. Id. The Examiner subsequently allowed the claims.
`
`EX1003, 25-27; EX1002, ¶¶47-48.
`
`C. Claim Construction
`When determining validity, “claim terms need only be construed to the
`
`extent necessary to resolve the controversy.” Wellman, Inc. v. Eastman Chem. Co.,
`
`642 F.3d 1355, 1361 (Fed. Cir. 2011). Petitioners reserve the right to respond to
`
`constructions offered by Patent Owner or adopted by the Board. Petitioners are not
`
`waiving arguments under 35 U.S.C. §112 or arguments regarding claim scope
`
`possibly raised in litigation. Petitioners acknowledge that the present analysis is
`
`performed under the Phillips standard. EX1002, ¶53.
`
`III. PRIOR ART
`A. Well-Known Concepts by the Critical Date
`By 2000, touch sensing technologies were deployed in a wide variety of
`
`products, including tablets, PDA’s, and mobile phones. See, e.g., EX1026 (Apple
`
`8
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`Newton Message Pad); EX1027 (Palm m500); EX1028 (HP Jornada Pocket PC).
`
`One example was the IBM Simon, available by 1995. Id.; EX1017; EX1002,
`
`¶¶53-54.
`
`EX1017, 10-11.
`
`
`
`At the time of the ’993 patent, user interfaces that transitioned between
`
`states in response to a mouse or touch input were known in the art. See, e.g.,
`
`EX1012, 5:54-6:3, 7:23-35, 8:1-22, FIGS. 1, 4-5, 7-9; EX1013, 18:14-33, 19:62-
`
`65, FIG. 4A, 4B, 5A, 5B, 9A, 9B, 9C, 10A, 10B, 10C; EX1002, ¶¶54-72.
`
`EX1012, Figs 4, 5 (annotated).
`
`
`
`9
`
`
`
`
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EX1013, FIGS. 4A, 4B.
`
`
`
`
`
`B.
`
`Prior Art Combinations
`1. Hisatomi, Ren, Allard-656, Tanaka, and Kodama
`a. Hisatomi
`Hisatomi discloses a “portable information processing device 01 [that] is a
`
`notebook-sized portable information terminal (PDA) that is mounted with an
`
`image display screen 09 with the capability to display full-color image information
`
`in high definition, and receives the coordinate instructions mainly from a pen-type
`
`input device 05.” EX1004, ¶[0012]; EX1002, ¶¶73-80.
`
`Hisatomi also teaches “an image display area 11E and [pull-out] menu
`
`display trigger areas 11A to 11D.” EX1004, ¶[0018]. Initially, only the whole
`
`main image and the menu trigger areas are visible in the display area. EX1004,
`
`FIGS. 5, 6, 10, 11; EX1002, ¶75.
`
`10
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`EX1004, FIG. 5.
`
`
`
`When the input device 05 is moved from the trigger area toward the center
`
`of the touch panel sensor 11, the pull-out menu and the icons it contains will be
`
`displayed. EX1004, ¶¶[0018], [0025], FIGS. 6, 10, 11, 12; EX1002, ¶¶76.
`
`11
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1004, FIG. 12.
`
`b.
`Ren
`Ren discloses pen-based selection strategies for input on pen-based systems
`
`incorporating small touch-sensitive screens. EX1006, 384-385; EX1002, ¶¶81-84.
`
`One of the strategies disclosed and studied by Ren is the Direct Off selection
`
`strategy in which “the target is highlighted only while the pen is touching it. The
`
`selection is made at the moment the pen is taken off the target.” EX1006, 390.
`
`12
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1006, 390. The Direct Off strategy “was already in common use.” EX1006,
`
`391. EX1002, ¶83.
`
`c.
`Allard-656
`Allard-656 discloses a personal communicator such as a “cellular phone and
`
`a complete data processing system.” EX1009, 3:28-31; EX1002, ¶¶85-88. The
`
`device includes a “touchscreen display 26 . . . and [a] user interface for entering
`
`information into and receiving information from.” EX1009, 3:45-50, 3:54-57;
`
`EX1002, ¶¶86.
`
`13
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1009, FIG. 1. The device of Allard-656 loads a phone application that interacts
`
`with the “Navigator program to create a Phone screen 108.” EX1009, 5:40-55;
`
`EX1002, ¶87.
`
`14
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1009, FIGS 4A-4B. Allard-656’s interface is very similar to that of the IBM
`
`Simon device, available by 1995. EX1017, pp. 10, 11; EX1002, ¶88.
`
`d.
`Tanaka
`Tanaka discloses an “electronic ‘organizer’ notebook device” that includes a
`
`tablet 8 with a display unit 9 and interface 2. EX1010, 3:46-65; EX1002, ¶¶89-92.
`
`When a TOOLBOX icon is selected, a TOOLBOX window is displayed. EX1010,
`
`4:17-19. The toolbox “window contains various icons.” EX1010, 4:19; EX1002,
`
`¶92.
`
`15
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1010, FIG. 5.
`
`e.
`Kodama
`Kodama discloses “a portable information gather apparatus which can
`
`function to manage captured information on a page basis.” EX1016, 1:9-14;
`
`EX1002, ¶93. Kodama’s pen computer displays icons—referred to as “stamps”—
`
`that are associated with various functions a user can select, including, e.g., an
`
`alarm stamp 101. See EX1016, 5:35-43, 8:21-9:4.
`
`16
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1016, FIG. 4.
`
`f. Motivation to Combine Hisatomi, Ren, Allard-656,
`Tanaka, and Kodama
`Hisatomi in combination with Ren, Allard-656, Tanaka, and Kodama render
`
`obvious the challenged claims. A person of ordinary skill in the art as of the
`
`Critical Date (“POSA”)3 would have been motivated to combine the teachings of
`
`these references. EX1002, ¶94. All five references relate to GUIs on a touch screen
`
`display of a handheld device, and the ability of a user to navigate and select targets
`
`
`
` 3
`
` See EX1002, ¶¶25, 49-52 (defining a POSA).
`
`17
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`within the user interface in ways that make optimal use of the space on a small
`
`screen. Thus, the fact that all five references are from the same technology area
`
`and address the same set of challenges provides motivation for a POSA to combine
`
`the teachings of Hisatomi with Ren, and further with Allard-656, Tanaka, and
`
`Kodama. Additional motivations to combine are set forth for specific claim
`
`limitations below.
`
`2. Hansen and Gillespie
`a. Hansen
`Hansen discloses a touch-sensitive user interface focused on providing
`
`improved user interactivity relative to traditional interfaces that rely on a mouse
`
`and keyboard. EX1029, 2:18-3:28, 4:26-33; EX1002, ¶¶95-97. Hansen’s interface
`
`enables detection and processing of an “input stroke”—a type of touch input that
`
`involves “continuous movement” of a user’s finger from a predetermined area
`
`(e.g., the edge) of a touch-sensitive display towards the center of the display.
`
`EX1029, 4:34-47. Using an input stroke, a user can access a computer program in
`
`an “intuitive” manner. Id., 4:26-33. Hansen’s Figure 2 (reproduced below)
`
`illustrates an example input stroke. EX1002, ¶95.
`
`18
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1029, FIG. 2. As shown, a user performs an input stroke on touch-sensitive
`
`screen 35. EX1029, 4:8-20. The input stroke includes two steps: (1) a user’s
`
`finger touches screen 35 within a predetermined area (e.g., the left-hand edge); and
`
`(2) while still touching screen 35, the user slides the finger towards the middle of
`
`screen 35 along arrow 45. EX1029, 4:34-47; EX1002, ¶96.
`
`Figure 3A (reproduced below) shows a display of Hansen’s interface before
`
`the user completes an input stroke. EX1029, 5:9-11.
`
`19
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1029, FIG. 3A. The Figure 3A interface displays “any of a number of windows
`
`w1, w2 as are commonly displayed by a computer system.” EX1029, 5:11-13;
`
`EX1002, ¶97.
`
`Figure 3B (reproduced below) shows a subsequent display of Hansen’s
`
`interface after the user has completed the two-step input stroke so that the display
`
`represents an outcome of the completed input stroke. EX1029, 5:6-13.
`
`20
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1029, FIG. 3B. As shown above, the Figure 3B display includes icons 50 in a
`
`line along the left-hand edge of screen 35. EX1029, 5:13-17. Each icon
`
`corresponds to a computer program, which is accessed through selection of the
`
`icon. Id., 5:14-25. The user performs selection by “momentarily touching one of
`
`the plurality of icons 50,” i.e., by tapping an icon. Id., 5:26-35. Once a user
`
`selects an icon, a corresponding computer program is launched in a “working
`
`window,” as shown in Figure 3C (reproduced below). EX1029, 5:34-44; EX1002,
`
`¶97.
`
`21
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`
`
`EX1029, FIG. 3C.
`
`b. Gillespie
`Gillespie describes user interfaces presented on a touch screen. EX1030,
`
`Abstract, [0033]; EX1002, ¶¶98-101. Specifically, Gillespie proposes
`
`incorporating a touch screen into a computer in a “similar way as a touch pad” and
`
`“could be introduced in addition to the other interface devices of the computer.”
`
`Id., ¶[0036]; EX1002, ¶98.
`
`Gillespie describes numerous “touch screen modes” for its touch screen—
`
`e.g., “iconic,” “auxiliary,” and “pop-up.” EX1030, ¶[0045]. In the “iconic” mode,
`
`the touch screen is configured to operate as a “touch pad pointing device” in which
`
`finger motions and taps are “generally interpreted the same as when the touch
`
`screen operates as a conventional touch pad.” Id., ¶[0046]; EX1002, ¶99.
`
`Figure 4 (reproduced below) shows an example of an interface image 400
`
`22
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`displayed by the touch screen when operating in the “iconic” mode. EX1002,
`
`¶100.
`
`
`
`EX1030, FIG. 4. As shown above, image 400 shows various visual conventions
`
`that “indicate special tapping or scrolling regions on the touch sensor surface.” Id.,
`
`¶¶ [0044], ¶ [0046]-[0056]. For example, dashed lines 426 and 428 in Figure 4
`
`identify regions that can accept touch input. Id., ¶[0049]. Similarly, dashed lines
`
`516, 518, 520, and 522 in Figure 5 are used to indicate an “activated state of the
`
`touch screen,” communicating to a user that touch input in the corresponding
`
`regions “will be interpreted as special commands…” Id., ¶[0057]. Gillespie
`
`describes examples of other visual conventions, such as “solid or colored lines,”
`
`“colored or inverted backgrounds,” or changes to the display or behavior of icons.
`
`Id.
`
`23
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`More broadly, Gillespie focuses on improving usability of touch-sensitive
`
`interfaces by displaying visual conventions and adjusting their appearance to
`
`reduce user confusion. EX1030, ¶[0059]. In this regard, Gillespie’s visual
`
`conventions afford users an understanding of how to use the touch interfaces
`
`described within its disclosure. EX1002, ¶101.
`
`c. Motivation to Combine Hansen and Gillespie
`As explained above, Hansen discloses a touch-sensitive user input focused
`
`on improving interactivity. EX1029, 2:18-3:28, 4:26-33. Hansen describes that it
`
`is “desirable to remove barriers that exist between the user and the computer
`
`system” by allowing users to provide touch input—specifically input strokes—to
`
`allow the user to “become more interactive with the computer system” so that
`
`performing tasks are “intuitive.” EX1029, 2:3-14; EX1002, ¶102.
`
`Yet, Hansen’s disclosure does not address techniques of assisting users to
`
`become comfortable using this type of touch input. EX1002, ¶103. For example,
`
`Hansen does not address the known need to assist a user transitioning from a
`
`device employing a mouse and keyboard interface to a devices employing a touch-
`
`sensitive interface. See, e.g., EX1029, 1:21-2:15; id. Hansen instead assumes that
`
`a user is proficient in using touch-sensitive interfaces to achieve its intended
`
`improvements in interactivity. EX1002, ¶103. Hansen therefore does not address
`
`scenarios in which a user is less experienced with touch-sensitive interfaces, or
`
`24
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`accesses the disclosed interface for the first time. Id.
`
`Given this gap in Hansen’s disclosure, a POSA looking to implement
`
`Hansen’s touch-sensitive user interface would have found it obvious to rely upon
`
`well-known prior art techniques to supplement Hansen’s teachings to further
`
`improve Hansen’s interface. For instance, a POSA would have understood that
`
`less-experienced users may not know how to initiate the input stroke disclosed in
`
`Hansen without any visible indicators. EX1002, ¶104. Indeed, in its Background
`
`section, Gillespie recognizes that many touch-sensitive user interfaces “have
`
`proven useful to expert users but confusing to novice users...” EX1030, ¶[0004].
`
`Given these usability challenges to Hansen’s interface, a POSA would have
`
`been motivated to modify Hansen’s interface to include one or more indicators that
`
`visually represent the “predetermined area” within which a user’s touch initiates
`
`the two-step input stroke. EX1002, ¶¶105-06. To perform this modification, a
`
`POSA would have turned to Gillespie’s technique of using visual conventions to
`
`“indicate control regions that respond specifically to finger motions and/or
`
`finger taps, either at all times or only when the touch screen has been activated in a
`
`special way.” EX1030, ¶[0056]4. Given the various types of visual conventions
`
`disclosed in Gillespie, a POSA would have understood that such a modification
`
`
`
` 4
`
` Emphasis is added by Petitioners unless otherwise noted.
`
`25
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`could be performed in various ways. EX1002, ¶106; EX1030, ¶[0059].
`
`One way in which a POSA would have modified Hansen to incorporate
`
`Gillespie’s visual conventions would be to modify Hansen’s Figure 3A display to
`
`incorporate a variation of Gillespie’s “solid or colored lines” and “colored or
`
`inverted backgrounds.” See EX1030, ¶[0059]; EX1002, ¶107. The added visual
`
`conventions would have identified the boundaries of the region of Hansen’s
`
`interface representing the predetermined area as a control region for initiating an
`
`input stroke. Id. An example of such a modified Figure 3A display is shown
`
`below:
`
`
`EX1029, FIG. 3A (modified to incorporate a Gillespie-type visual convention, e.g.,
`
`26
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`
`dashed lines and colored background).5
`
`A POSA designing user interfaces would have understood that visually
`
`identifying Hansen’s predefined area as shown in the modified Figure 3A display
`
`would have provided an “affordance”—a well-known design concept defined as
`
`“the perceived and actual properties an object, such as the fundamental properties
`
`that determine just how an object could possibly be used.” EX1002, ¶108 (citing
`
`to EX1032, 9). A POSA would have found it obvious that incorporation of
`
`Gillespie’s visual indicators into the Figure 3A display have “afforded” a user the
`
`understanding that the “predetermined area” could be used to transition to
`
`Hansen’s Figure 3B, display a menu of icons 50. Id. EX1002, ¶¶109-13.
`
`IV. EXPLANATION OF UNPATENTABILITY
`A. Ground 1A: Claims 1-3, 7, and 8 are rendered obvious by
`Hisatomi in view of POSA knowledge and/or Ren
`Claim 1 - [1.pre] “A non-transitory computer readable
`medium storing instructions, which, when executed by a
`processor of an electronic device having a touch-sensitive
`
`
`
` 5
`
` As explained by Dr. Bederson, the modified figures provided throughout the
`
`petition are not intended to be to scale or to represent the actual graphics a POSA
`
`would have used to provide “affordance.” EX1002, ¶107. Instead, the modified
`
`figures are intended solely to be illustrative. Id.
`
`27
`
`

`

`Attorney Docket No. 50095-0016IP1
`IPR of U.S. Patent No. 8,812,993
`display screen, cause the processor to enable a user interface
`of the device,”
`Hisatomi discloses this element. EX1002, ¶¶114-20. Hisatomi discloses
`
`“notebook-sized portable information terminal[s]” that included “touch-sensitive
`
`keys on [a] liquid crystal display” such as a PDA. EX1004, 0002. 0012.
`
`Hisatomi’s device includes a display screen 9 and touch panel sensor 11. EX1004,
`
`0012, 0015-18; see also id., 1031 and 21-22 (describing part 01, 05, 09, and 11).
`
`EX1002, ¶115.
`
`
`
`EX1004, FIGS. 3 and 4 (annotated, device (blue), display (red), and touch panel
`
`sensor (green)); see also id., FIGS. 5 and 8. A user provides input to the device 01
`
`by touching the touch panel sensor 11, thereby providing “coordinate instructions
`
`mainly from a pen-type input device 05.” EX1004, 0015.
`
`Hisatomi discloses one or more non-transitory computer readable medium
`
`(i.e., RAM, ROM, F