`571-272-7822
`
`Paper 19
`Date: January 13, 2022
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`GOOGLE LLC,
`Petitioner,
`v.
`NEONODE SMARTPHONE LLC,
`Patent Owner.
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`
`
`
`
`
`
`
`
`Before MICHELLE N. ANKENBRAND, KARA L. SZPONDOWSKI, and
`CHRISTOPHER L. OGDEN, Administrative Patent Judges.
`ANKENBRAND, Administrative Patent Judge.
`
`DECISION
`Granting Institution of Inter Partes Review
`35 U.S.C. § 314
`
`
`
`
`
`
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`INTRODUCTION
`I.
`Google LLC (“Petitioner”) filed a Petition requesting an inter partes
`review of claims 1–7, 9, 12, 13, and 15–17 of U.S. Patent No. 8,095,879 B2
`(Ex. 1001, “the ’879 patent”). Paper 1 (“Pet.”). Neonode Smartphone LLC
`(“Patent Owner”) filed a Preliminary Response. Paper 15 (“Prelim. Resp.”).
`With our authorization, Petitioner filed a Pre-Institution Reply (Paper 17,
`“Reply”), and Patent Owner filed a Pre-Institution Sur-reply (Paper 18,
`“Sur-reply”).
`We have authority to determine whether to institute an inter partes
`review. 35 U.S.C. § 314 (2018); 37 C.F.R. § 42.4(a) (2020). The standard
`for instituting an inter partes review is set forth in 35 U.S.C. § 314(a), which
`provides that an inter partes review may not be instituted “unless the
`Director determines . . . there is a reasonable likelihood that the petitioner
`would prevail with respect to at least [one] of the claims challenged in the
`petition.”
`For the reasons set forth below, we determine the information
`presented in the Petition establishes a reasonable likelihood that Petitioner
`would prevail with respect to at least one of the challenged claims.
`Accordingly, we institute an inter partes review.
`II. BACKGROUND
`A. Related Matters
`The parties identify the following district court proceedings involving
`the ’879 patent: Neonode Smartphone LLC v. Apple Inc., Case No. 6:20-cv-
`00505 (W.D. Tex.) and Neonode Smartphone LLC v. Samsung Electronics
`Co. Ltd., Case No. 6:20-cv-00507 (W.D. Tex.). Pet. 106; Paper 3, 2. The
`parties also note that the ’879 patent is challenged in IPR2021-00144, which
`Samsung Electronics Co. Ltd., Samsung Electronics America, Inc.
`
`2
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`(collectively, “Samsung”), and Apple Inc. (“Apple”) filed (the “prior
`petition” or “Samsung and Apple petition”). Pet. 106, Paper 3, 2.
`Patent Owner also identifies as related IPR2021-00145 (challenging
`U.S. Patent No. 8,812,993 B2). Paper 3, 2.
`B. The ’879 Patent
`The ’879 patent, titled “User Interface for Mobile Handheld Computer
`Unit,” issued on January 10, 2012. Ex. 1001 at [45], [54]. The ’879 patent
`describes a user interface for a mobile handheld computer that has a touch
`sensitive area divided into a menu area and a display area. Id. at 1:6–9. The
`menu area shows a representation of a first, a second, and a third predefined
`function that “can be activated when the touch sensitive area detects a
`movement of an object with its starting point within the representation of the
`function on the menu area and with a direction from the menu area to the
`display area.” Id. at 1:65–2:5, 2:11–14. “[T]he first function is a general
`application dependent function, the second function is a keyboard function,
`and the third function is a task and file manager.” Id. at 2:7–10.
`The user interface is “specifically adapted to be used with a small
`computer unit” having a touch sensitive area that is approximately 2–3
`inches and also is adapted so that a user can operate it with one hand. Id. at
`3:1–6.
`
`3
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`We reproduce Figure 1 of the ’879 patent below.
`
`
`Figure 1 is a schematic view of touch sensitive area 1 on a mobile handheld
`computer unit, depicting menu area 2 adapted to present a representation of
`first 21, second, 22, and third 23 predefined functions. Id. at 3:21–22, 4:1–3.
`We reproduce Figure 2 of the ’879 patent below.
`
`
`Figure 2 is a schematic side view illustrating how a user activates a function.
`Id. at 3:24–25. A user can activate any one of functions 21, 22, or 23 when
`
`4
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`touch sensitive area 1 detects a movement of object 4 with its starting point
`A within the representation of a function on menu area 2 and with direction
`B from menu area 2 to display area 3. Id. at 4:7–11. Object 4 can be a
`finger, a pen, or another pointing device. Id. at 6:11–15.
`The ’879 patent explains that when a user activates, for example, the
`first function, the display area is adapted to display icons representing
`services or settings, depending on the current active application. Id. at 2:18–
`20. Figure 3, which we reproduce below, illustrates first function 21
`activated.
`
`
`Figure 3 is a schematic illustration of the first function. Id. at 3:26. Figure 3
`shows that after a user activates first function 21 with the movement
`depicted in Figure 2, display area 3 displays icons 211–216, each
`representing services or functions depending on the current active
`application. Id. at 4:12–15. If, for example, the active application handles a
`picture, then the icons showing on display area 3 after a user activates the
`first function “can be services such as ‘save to disk’, ‘send as SMS’, or
`‘delete’, and they can be setting such as ‘resolution’, ‘colour’, or
`‘brightness’.” Id. at 4:24–28. If no application is active on the computer,
`then icons 211–216 represent services or settings of the operating system,
`
`5
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`such as background picture clock, alarm 215, users 213, and help 211. Id. at
`4:29–32.
`The ’879 patent describes, in similar fashion, how a user activates the
`second keyboard function and the third task and file manager function. See
`id. at 4:34–5:63.
`
`C. Illustrative Claim
`Petitioner challenges claims 1–7, 9, 12, 13, and 15–17 of the ’879
`patent (collectively, “the challenged claims”). Claim 1, the only
`independent claim, is illustrative of the claimed subject matter, and recites:
`1. A non-transitory computer readable medium storing a
`computer program with computer program code, which, when
`read by a mobile handheld computer unit, allows the computer to
`present a user interface for the mobile handheld computer unit,
`the user interface comprising:
`a touch sensitive area in which a representation of a function is
`provided, wherein the representation consists of only one
`option for activating the function and wherein the function is
`activated by a multi-step operation comprising (i) an object
`touching the touch sensitive area at a location where the
`representation is provided and then (ii) the object gliding along
`the touch sensitive area away from the touched location,
`wherein the representation of the function is not relocated or
`duplicated during the gliding.
`Ex. 1001, 6:45–59.
`
`
`
`
`
`
`6
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`D. Asserted Grounds of Unpatentability
`Petitioner asserts that the challenged claims are unpatentable on the
`following grounds:
`References/Basis
`35 U.S.C. §
`Claim(s) Challenged
`Robertson, 2 Maddalozzo3
`103(a)1
`1–5, 13, 15–17
`Robertson, Maddalozzo, Vayda4
`103(a)
`6, 7, 9
`Robertson, Maddalozzo,
`103(a)
`12
`Bedford-Roberts5
`Tarpenning6
`103(a)
`1, 4–6, 13, 15–17
`Tarpenning, Vayda
`103(a)
`2, 3, 7, 9
`Tarpenning, Bedford-Roberts
`103(a)
`12
`Pet. 1–2. Petitioner relies on the Declaration of Dr. Jacob O. Wobbrock
`(Ex. 1003) to support its asserted grounds of unpatentability. Patent Owner
`disputes that Petitioner’s asserted grounds render any of the challenged
`claims unpatentable. See generally Prelim. Resp. Patent Owner relies on
`the Declaration of Dr. Craig Rosenberg (Ex. 2001).
`III. DISCRETION UNDER 35 U.S.C. § 314(A) DUE TO PRIOR
`PETITION
`Patent Owner argues that we should exercise discretion to deny
`institution under 35 U.S.C. § 314(a) in view of the prior petition Samsung
`
`
`1 The Leahy-Smith America Invents Act, Pub. L. No. 112-29, 125 Stat. 284
`(2011) (“AIA”), amended 35 U.S.C. § 103. Because the ’879 patent has an
`effective filing date before the effective date of the applicable AIA
`amendments, we refer to the pre-AIA version of 35 U.S.C. § 103.
`2 George G. Robertson et al., Buttons as First Class Objects on an
`X Desktop, UIST: Proceedings of the ACM Symposium on User Interface
`Software and Technology: Hilton Head, South Carolina, USA, 35–44 (Nov.
`11–13, 1991) (Ex. 1005).
`3 U.S. Patent No. 7,768,501, issued Aug. 3, 2010 (Ex. 1006).
`4 U.S. Patent No. 5,745,717, issued Apr. 28, 1998 (Ex. 1007).
`5 U.S. Patent No. 5,870,092, issued Feb. 9, 1999 (Ex. 1008).
`6 U.S. Patent No. 6,181,344, issued Jan. 30, 2001 (Ex. 1009).
`
`7
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`and Apple filed in IPR2021-00144 challenging the ’879 patent. Prelim.
`Resp. 1–2; Sur-reply 1–3. Petitioner argues that we should not exercise
`discretion to deny institution under 35 U.S.C. § 314(a). Pet. 105; Reply 1.
`For the reasons stated below, we do not exercise discretion to deny
`institution in view of the prior petition.
`Institution of an inter partes review is discretionary. See 35 U.S.C.
`§ 314(a) (authorizing institution of an inter partes review under particular
`circumstances, but not requiring institution under any circumstances);
`Cuozzo Speed Techs., LLC v. Lee, 136 S. Ct. 2131, 2140 (2016) (“[T]he
`agency’s decision to deny a petition is a matter committed to the Patent
`Office’s discretion.”); SAS Inst. Inc. v. Iancu, 138 S. Ct. 1348, 1356 (2018)
`(“[Section] 314(a) invests the Director with discretion on the question
`whether to institute review . . . .” (emphasis omitted)); Harmonic Inc. v. Avid
`Tech., Inc., 815 F.3d at1356, 1367 (Fed. Cir. 2016) (“[T]he PTO is
`permitted, but never compelled, to institute an IPR proceeding.”).
`When determining whether to exercise discretion to deny institution in
`view of a prior petition challenging the same patent, we consider the
`following non-exclusive factors:
`1. whether the same petitioner previously filed a petition
`directed to the same claims of the same patent;
`2. whether at the time of filing of the first petition the
`petitioner knew of the prior art asserted in the second petition or
`should have known of it;
`3. whether at the time of filing of the second petition the
`petitioner already received the patent owner’s preliminary
`response to the first petition or received the Board’s decision on
`whether to institute review in the first petition;
`
`8
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`4. the length of time that elapsed between the time the
`petitioner learned of the prior art asserted in the second petition
`and the filing of the second petition;
`5. whether the petitioner provides adequate explanation
`for the time elapsed between the filings of multiple petitions
`directed to the same claims of the same patent;
`6. the finite resources of the Board; and
`7. the requirement under 35 U.S.C. § 316(a)(11) to issue a
`final determination not later than 1 year after the date on which
`the Director notices institution of review.
`General Plastic Co., Ltd. v. Canon Kabushiki Kaisha, IPR2016-01357,
`Paper 19, 16−17 (PTAB Sept. 6, 2017) (precedential) (“General Plastic”).
`We address each of these factors in turn.
`1. Factor 1: petitioner and challenged claims
`As we note above, Samsung and Apple previously filed a petition
`challenging the ’879 patent. Patent Owner contends that the Samsung and
`Apple petition and the present petition challenge the same patent, and of the
`thirteen claims challenged in the follow-on petition, eleven overlap the
`previous petition. Prelim. Resp. 6–7; Sur-reply 1. Patent Owner further
`contends Petitioner has a sufficiently close relationship with Samsung and
`Apple warranting exercise of discretion to deny institution. Prelim. Resp. 7–
`9 (citing Valve Corp. v. Elec. Scripting Prods., Inc., IPR2019-00062, Paper
`11 at 2 (PTAB Apr. 2, 2019) (precedential) (“Valve”); PayPal, Inc. v.
`IOENGINE LLC, IPR2019-00884, Paper 22 (PTAB Oct. 3, 2019)
`(“PayPal”). Patent Owner argues that three facts establish a sufficiently
`close relationship: (1) Petitioner sued Samsung for infringing the ’879
`patent and Samsung sells the accused Galaxy devices preinstalled with
`Petitioner’s operating system and other of Petitioner’s mobile apps;
`(2) Petitioner offers through Google Play and Apple’s App Store its virtual
`
`9
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`keyboard operation—GBoard—that Patent Owner identifies as infringing in
`the complaint against Apple and that is accessible on the accused Samsung
`and Apple phones; and (3) Apple relied on the presence of Petitioner’s
`employees in the Northern District of California to support a motion to
`transfer Patent Owner’s case and to support a mandamus petition Apple filed
`after the district court denied the motion to transfer. Id. at 8; Sur-reply 1.
`Petitioner asserts that it is not a party to any litigation involving the
`’879 patent and, therefore, the Valve and PayPal decisions are
`distinguishable. Reply 2–3. Petitioner further asserts that it is not a
`petitioner in, and was not involved with the prior petition, Patent Owner fails
`to show any “significant relationship” with Samsung or Apple, and the
`present Petition challenges additional claims that were not included in the
`first petition, all of which favor declining to exercise discretion to deny
`institution. Pet. 105; Reply 1–3.
`As both parties acknowledge, “when different petitioners challenge
`the same patent [the Board] consider[s] any relationship between those
`petitioners when weighing the General Plastic factors.” Valve, Paper 11 at
`9. In Valve, the Board explained that the first General Plastic factor
`weighed in favor of exercising discretion to deny institution because “Valve
`[the second petitioner] and HTC [the first petitioner] were co-defendants in
`the District Court litigation and were accused of infringing the [challenged]
`patent based on the same product.” Id. at 9–10. Thus, the Board in Valve
`found three facts sufficient to establish a significant relationship between the
`two petitioners—the first and second petitioners were co-defendants, the
`challenged patent had been asserted against both petitioners because the first
`petitioner had licensed the allegedly infringing technology from the second
`petitioner, and the second petitioner provided the first petitioner with
`
`10
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`technical assistance during development of the accused product. Id. In
`PayPal, the Board determined a significant relationship existed between the
`first and second petitioners even though they were not co-defendants
`because the first petitioner and second petitioner were accused of infringing
`the same patent based on the same products, the first petitioner had an
`indemnification agreement with the second petitioner, and the first petitioner
`filed a declaratory judgment action based on the indemnification agreement.
`PayPal, Paper 22 at 9, 10–11.
`Here, in contrast, Petitioner is not a party to any litigation. Thus,
`Petitioner is not a co-defendant with Samsung or Apple. Nor is there any
`evidence to indicate that Petitioner has an indemnification agreement with
`Samsung or Apple. At most, as Patent Owner argues, the accused Samsung
`devices are preinstalled with Petitioner’s operating system and other of
`Petitioner’s mobile applications, and Petitioner’s GBoard application, which
`is accused of infringing, is available for download on Petitioner’s store app
`and Apple’s app store. 7 In our view, these facts do not create a significant
`relationship under Valve. Accordingly, on the record before us, we
`determine that Petitioner does not have a significant relationship with
`Samsung or Apple.
`As for the challenged claims, we agree with Patent Owner that the
`current Petition challenges most of the claims that were challenged in the
`
`
`7 We do not find as relevant to our analysis the fact that Apple relied on the
`location of Petitioner’s witnesses in its transfer motion and mandamus
`petition. As Petitioner notes, Reply 2, the motion and petition rely on the
`court’s power to compel third-party witnesses to testify, see Ex. 2015, 10–
`11; Ex. 2016, 21–22, and Patent Owner does not explain why that is relevant
`to our determination of whether a significant relationship exists between
`Petitioner and Apple and/or Samsung.
`
`11
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`prior petition. An overlap in claims alone, however, is not enough to tip this
`factor in favor of exercising discretion to institute review. See Valve, Paper
`11 at 10 (“The complete overlap in claims and the significant relationship
`between Valve and HTC favor denying institution.”). As a result, we
`determine that the first factor weighs against exercising our discretion to
`deny institution.
`2. Factor 2: knew, or should have known, of the prior art
`Patent Owner argues that Petitioner should have known of the prior
`art references asserted in its Petition at the time the prior petition was filed
`because the prior art references are all publicly available on either
`Petitioner’s own search engine, its own patent search platform, or the Patent
`and Trademark Office’s “Public PAIR” platform. Prelim. Resp. 10–11;
`Sur-reply 2. Petitioner asserts that it was not aware of the asserted prior art
`references until six months after the prior petition was filed, and Patent
`Owner does not adequately explain why Petitioner should have known of
`them at the time the previous petition was filed. Pet. 105–106; Reply 3–4.
`Petitioner further asserts that this factor is “of limited relevance” because the
`current Petition asserts different prior art than the Samsung and Apple
`petition. Reply 3.
`We agree with Petitioner that this factor is of limited relevance in our
`analysis because the current Petition relies on different prior art than the
`Samsung and Apple petition. See Foursquare Labs, Inc. v. Mimzi, LLC,
`IPR2019-01287, Paper 11 at 8–9 (PTAB Jan. 14, 2020). Further, as we
`discuss in connection with the first General Plastic factor, Petitioner is not a
`party to any litigation. This fact undermines Patent Owner’s argument that
`Petitioner should have known of the prior art references asserted in the
`Samsung and Apple petition, as we agree with Petitioner that Patent Owner
`
`12
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`fails to address why Petitioner should have searched for or known of prior
`art in November 2020 when Petitioner was not a party to any litigation.
`Accordingly, the second factor weighs against exercising our discretion to
`deny institution.
`3. Factor 3: had the benefit of preliminary response and prior decision on
`institution
`Patent Owner argues that it filed a preliminary response to the
`Samsung and Apple petition on March 17, 2021, and Petitioner filed its
`Petition on June 14, 2021, so Patent Owner’s preliminary response was
`available to Petitioner when it began drafting the Petition. Prelim. Resp. 11–
`12; Sur-reply 2. Petitioner asserts that it filed the Petition before a decision
`on institution of the Samsung and Apple petition and Patent Owner offers no
`evidence that Petitioner obtained an unfair benefit or tactical advantage from
`Patent Owner’s preliminary response. Pet. 105; Reply 4.
`We agree with Petitioner. Petitioner relies on different prior art here
`than the art asserted in the Samsung and Apple petition, and Patent Owner
`does not identify any unfair benefit or tactical advantage that Petitioner
`would have gained from reviewing Patent Owner’s preliminary response to
`the Samsung and Apple petition. Accordingly, the third factor weighs
`against exercising our discretion to deny institution.
`4. Factors 4 and 5: delay in filing the petition
`For factor four, Patent Owner argues that the Board should consider
`that: (1) the prior petition was available to Petitioner since November 6,
`2020, seven months before Petitioner filed the current Petition; (2) Patent
`Owner’s preliminary response was available Petitioner since March 17,
`2021, three months before its petition was filed; and (3) the prior art
`references that Petitioner cites in its petition were available to Petitioner on
`
`13
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`its own search platforms and the Patent and Trademark Office’s Public
`PAIR platform at all material times. Prelim. Resp. 12–14; Sur-reply 2–3.
`Similarly, for factor five, Patent Owner contends Petitioner’s explanation for
`the elapsed time between the original petition and the current petition is
`unsupported and fails to address why Petitioner delayed seven months after
`the original petition and three months after Patent Owner’s preliminary
`response to file the current Petition when the prior art references Petitioner
`asserts were available to Petitioner on its own search platforms and the
`Patent and Trademark Office’s Public PAIR platform at all material times.
`Prelim. Resp. 14–15; Sur-reply 2–3. Petitioner argues that it: (1) was not a
`party to any previous litigation; (2) became aware of the prior art references
`asserted in the previous petition around May 3, 2021; and (3) filed its
`petition six weeks after becoming aware of the prior art references. Pet. 106;
`Reply 4–5.
`Because Petitioner and prior petitioners Samsung and Apple are
`unrelated, we find that this factor is only marginally relevant to our analysis.
`Here, Petitioner represents on the record that it only became aware of the
`prior art references asserted in the Samsung and Apple petition in May 2021,
`and filed its petition six weeks later. Pet. 106; Reply 4–5. Nothing in the
`record calls Petitioner’s representations into question, which we credit.
`Moreover, Patent Owner’s arguments that Petitioner should have known
`about the prior petition and preliminary response imply a relationship
`between Petitioner and Samsung or Apple. But as we discuss in connection
`with the first factor, Petitioner does not have a significant relationship with
`either Samsung or Apple. Accordingly, the fourth and fifth factors weigh
`against exercising our discretion to deny institution.
`
`14
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`5. Factors 6 and 7: Board resources and statutory deadline
`Patent Owner asserts that, “in general, having multiple petitions
`challenging the same patent . . . is inefficient and tends to waste resources.”
`Prelim. Resp. 15; Sur-reply 3. Other than that general assertion, Patent
`Owner reiterates the same arguments it presents for factors four and five.
`See Prelim. Resp. 15–16. Petitioner contends it only recently learned of the
`’879 patent and is not a party to any litigation involving it. Pet. 106.
`Neither party addresses the Board’s statutory deadline.
`We are mindful that the “intent in formulating the [General Plastic]
`factors was to take undue inequities and prejudices to Patent Owner into
`account.” General Plastic, Paper 19, at 17. We are also mindful, however,
`of potential prejudice to a second petitioner if institution of review based on
`the second petition is denied—as was originally the case with the Samsung
`and Apple petition—or the proceeding involving the first petitioner is
`terminated. In view of those two competing concerns, we find the sixth
`factor neutral. We find that the seventh factor weighs against exercising our
`discretion to deny institution because there is no argument or evidence that
`instituting review in this case will cause any delay in meeting our statutory
`deadline for issuing a final written decision in either this case or the case
`involving the Samsung and Apple petition.
`6. Conclusion
`We have considered the circumstances and facts before us in view of
`the General Plastic factors. Because our analysis is fact-driven, no single
`factor is determinative of whether we exercise our discretion to deny
`institution under § 314(a). Evaluating all of the factors on this record, we
`determine that the circumstances presented here do not support exercising
`our discretion under § 314(a) to deny institution of an inter partes review.
`
`15
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`IV. ANALYSIS
`A. Level of Ordinary Skill in the Art
`For Petitioner, Dr. Wobbrock testifies that a person of ordinary skill in
`the art at the time of the invention, would have had “at least a bachelor’s
`degree in Computer Science, Human-Computer Interaction, Symbolic
`Systems, or related engineering disciplines, and at least two years of
`experience designing and programming graphical user interfaces,” but
`“[r]elevant work experience can substitute for formal education and
`advanced degree studies could substitute for work experience.” Ex. 1003
`¶ 49. Patent Owner’s expert, Dr. Rosenberg, applies Dr. Wobbrock’s
`description of the ordinarily skilled artisan. Ex. 2001 ¶ 35. We adopt
`Dr. Wobbrock’s description of the level of ordinary skill in the art because it
`is consistent with the problems and solutions the ’879 patent identifies and
`with the prior art.
`
`B. Claim Construction
`Neither party contends that there are claim terms in dispute, or that we
`need to construe any terms for purposes of this decision. See Pet. 4; see
`generally Prelim. Resp. After reviewing the parties’ arguments and
`evidence, we determine that we do not need to expressly construe any claim
`terms for purposes of this Decision. See Nidec Motor Corp. v. Zhongshan
`Broad Ocean Motor Co., 868 F.3d 1013, 1017 (Fed. Cir. 2017).
`C. Asserted References
`Before turning to Petitioner’s asserted grounds, we provide a brief
`summary of the asserted references.
`1. Robertson (Ex. 1005)
`Robertson describes a high-level user interface toolkit called
`“XButtons,” that supports on-screen buttons as first-class objects on an
`
`16
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`“X window system desktop.” Ex. 1005, 35. Robertson further describes
`that XButtons: typically appear as small rectangular screen objects; usually
`have some text that indicates what their action is, and may also have a field
`for some editable text. Id. at 38. A group of XButtons is shown in Figure 1,
`which we reproduce below.
`
`
`Figure 1 illustrates a sample set of XButtons. Id. at 38–39. XButtons have
`multiple actions which are selected by simple mouse or pen gestures, where
`there are two classes of gestures, touching gestures and grasping gestures.
`Id. at 39. Whenever a user gestures at an XButton, a gesture parser
`interprets mouse or pen movement and classifies it as one of a set of easily
`differentiated gestures (e.g., “flick left,” “flick right,” “flick up,” “flick
`down,” “click,” “rubout,” “check,” or “insert”). Id. A menu for an XButton
`is shown in Figure 2, which we reproduce below.
`
`17
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`
`Figure 2 illustrates a sample menu for an XButton. Id. at 40. The menu
`shows all potential user manipulations of the XButton. Id. A button editor
`for an XButton is shown in Figure 3, which we reproduce below.
`
`18
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`
`Figure 3 shows a button editor for the XButton whose menu is shown in
`Figure 2. Id. The button editor is a structured property-sheet editor
`designed specifically for editing XButtons. Id. The editor fields at the top
`specify the appearance of the button. Id. The middle of the editor specifies
`the action language, the various actions, and the text to appear in the menu
`for each action. Id. Toward the bottom of the editor are fields (“Attribute”
`and “Value”) that allow the user to view and replace user-defined properties.
`Id.
`
`19
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`2. Maddalozzo (Ex. 1006)
`Maddalozzo describes a portable computer having a display screen
`where data is displayed initially within the display screen. Ex. 1006, 2:15–
`18. The display screen is partitioned into a touch-sensitive input area and a
`display area, where data input at the touch-sensitive input area is
`simultaneously displayed in the display area, in response to a particular
`input. Id. at 2:18–22. A graphical user interface displayed within a display
`screen is shown in Figure 6, which we reproduce below.
`
`Figure 6 illustrates a graphical user interface window 83 displayed in a
`keyboard mode within a display screen. Id. at 6:36–38. When a user
`touches “keys” displayed on display screen 80, keyboard input is reflected
`onto the screen beneath transparent keyboard 110. Id. at 6:48–50. Further, a
`
`
`
`20
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`smaller “active” window 100 is displayed to indicate the actual typing taking
`place. Id. at 7:1–3.
`
`3. Vayda (Ex. 1007)
`Vayda describes a menu generating computer system for displaying a
`graphical display including a menu on a display of the computer system.
`Ex. 1007, [57]. The menu can be activated by a button combination on a
`user input device or another input device, such as a touch screen. Id. at
`6:14–19. An example menu is shown in Figure 14, which we reproduce
`below.
`
`
`Figure 14 illustrates a menu screen 1400 comprising a select/execute method
`1402. Id. at 14:67–15:1. Select/execute menu 1402 comprises a vertical
`command bar 1450, a horizontal command bar 1452, and diagonal command
`bars 1460 and 1462. Id. at 15:1–4. Vayda describes that commands are
`
`21
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`selected by scrolling horizontally, vertically and/or diagonally through a
`focus position. Id. at 14:65–67.
`4. Bedford-Roberts (Ex. 1008)
`Bedford-Roberts describes a device for displaying pages of
`electronically stored information comprising a pen/finger sensitive symbol
`enabling a user to browse the information by page turning. Ex. 1008, 1:43–
`45. The user can stroke the symbol to riffle through pages. Id. at 1:51–53.
`The device is shown in Figure 1, which we reproduce below.
`
`
`
`22
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`Figure 1 illustrates a pen-based handheld device 10 for storing and
`displaying pages of electronic information comprising a display screen 12
`and a pen for user input. Id. at 2:24–26. The device also comprises a book
`symbol 18 for enabling users to browse pages of stored information. Id. at
`2:26–28. Symbol 18 is shown in greater detail in Figure 2, which we
`reproduce below.
`
`Figure 2 shows symbol 18 from Figure 1, where symbol 18 depicts an open
`book. Id. at 2:19, 30. The symbol 18 is notionally divided into three
`segments 20, 21, and 22, the boundaries of which are indicated by dotted
`lines in Figure 2. Id. at 2:37–39. The symbol 18 includes a representation
`
`
`
`23
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`24 of an open book and a double-headed arrow 26. Id. at 2:39–41. The left
`hand head 28 of the arrow 26 is in the region 20 and the right hand head 30
`of the arrow is in the region 22. Id. at 2:41–43. The book representation 24
`and the body 34 of the arrow 26 are in the region 21. Id. at 2:43–44. The
`book representation 24 comprises page delimiters 36 which are ridges
`formed in the symbol 18 which the user can feel as the pen 14 is moved over
`the symbol 18. Id. at 2:47–49.
`In use, as the user moves the pen 14 along the body 34 of the arrow,
`or anywhere within the region 21, successive pages of stored information are
`displayed, depending on the direction of movement along the arrow body 34.
`Id. at 2:50–53. As a page delimiter 36 is traversed by the pen 14, the next
`page in the direction of movement of the pen 14 is displayed. Id. at 2:54–55.
`If the user taps one of the arrow heads 28 or 30, this causes the device 10 to
`display the next page in the direction indicated by the tapped arrow head.
`Id. at 2:56–58. In order to riffle through a large number of pages of stored
`information, the user repeatedly moves the pen 14 in a single direction in the
`region 21 over the body 34 of the arrow 26. Id. at 2:61–63.
`5. Tarpenning (Ex. 1009)
`Tarpenning describes a hand-held reader device that includes a touch-
`sensitive display and graphical user interface, where the user access various
`software-implemented features associated with the display and management
`of the content. Ex. 1009, [57]. A perspective view of the hand-held reader
`device is shown in Figure 2, which we reproduce below.
`
`24
`
`
`
`IPR2021-01041
`Patent 8,095,879 B2
`
`Figure 2 illustrates a hand-held computing device adapted to be used as a
`portable reading device allow