I 1111111111111111 11111 1111111111 11111 111111111111111 IIIII IIIIII IIII IIII IIII
`US009703468B2
`
`c12) United States Patent
`Reeves et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 9,703,468 B2
`Jul. 11, 2017
`
`(54) UNIFIED DESKTOP INDEPENDENT FOCUS
`IN AN APPLICATION MANAGER
`
`(71) Applicant: Z124, George Town (KY)
`
`(72)
`
`Inventors: Paul E. Reeves, Oakville (CA); Sanjiv
`Sirpal, Oakville (CA); Martin Gimpl,
`Helsinki (FI)
`
`(73) Assignee: Z124, George Town (KY)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 983 days.
`
`(21) Appl. No.: 13/627,679
`
`(22) Filed:
`
`Sep. 26, 2012
`
`(65)
`
`Prior Publication Data
`
`US 2013/0111371 Al May 2, 2013
`
`(63)
`
`Related U.S. Application Data
`
`Continuation-in-part of application No. 13/605,740,
`filed on Sep. 6, 2012, now Pat. No. 9,529,494, and a
`continuation-in-part of application No. 13/605,482,
`filed on Sep. 6, 2012, and a continuation-in-part of
`application No. 13/605,145, filed on Sep. 6, 2012,
`now Pat. No. 8,990,713, and a continuation-in-part of
`application No. 13/604,960, filed on Sep. 6, 2012,
`now abandoned, and a continuation-in-part of
`(Continued)
`
`(51)
`
`(52)
`
`(2013.01)
`(2013.01)
`(2013.01)
`(2006.01)
`
`Int. Cl.
`G06F 3/048
`G06F 3/0486
`G06F 3/0488
`G06F 3/14
`U.S. Cl.
`CPC ........ G06F 3/0486 (2013.01); G06F 3/04883
`(2013.01); G06F 3/1423 (2013.01); G06F
`2203/04803 (2013.01)
`
`(58) Field of Classification Search
`None
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,757,371 A * 5/1998 Oran et al ..................... 715/779
`10/2001 Shields et al.
`6,304,261 Bl
`(Continued)
`
`OTHER PUBLICATIONS
`
`Soper "Sams Teach Yourself Microsoft Windows® 7 in 10 Min(cid:173)
`utes." Sams, 2010, Safari books, Web, Jul. 1, 2014 [techbus.
`safaribooksonline.corn/book/operating-systems/
`9780132121897?bookview~overview] 4 pages.
`(Continued)
`
`Primary Examiner - Ryan Barrett
`Assistant Examiner - Haimei Jiang
`(74) Attorney, Agent, or Firm - Sheridan Ross P.C.
`
`(57)
`
`ABSTRACT
`
`Methods and devices for selectively presenting a user inter(cid:173)
`face or "desktop" across two devices are provided. More
`particularly, a unified desktop is presented across a device
`and a computer system that comprise a unified system. The
`unified desktop acts as a single user interface that presents
`data and receives user interaction in a seamless environment
`that emulates a personal computing environment. To func(cid:173)
`tion within the personal computing environment, the unified
`desktop includes a process for docking and undocking the
`device with the computer system. The unified desktop
`presents a new user interface to allow access to functions of
`the unified desktop.
`
`20 Claims, 104 Drawing Sheets
`
`;1100
`
`7216
`
`• 7304
`
`Window 5
`
`Window a
`
`Apple Ex. 1019
`Apple Inc. v. Firstface Co., Ltd.
`IPR2019-01011
`Page 00001
`
`

`

`12/2008 Sabatelli et al.
`12/2008 Kumar et al.
`4/2009 Treder et al .................. 715/738
`6/2009 Gasn et al.
`10/2009 Bamford et al.
`11/2009 Duarte et al.
`12/2009 Kohavi ................. G06F 9/5027
`718/1
`
`2/2010 Sides
`4/2010 Lee et al.
`6/2010 Marano ................. G06F 3/1415
`715/804
`
`4/2011 Galicia et al.
`6/2011 Mir et al. ...................... 715/779
`8/2011 Mazzaferri
`9/2011 Abramson et al.
`12/2011 Robert et al.
`4/2012 Yusupov et al.
`4/2012 Reeves et al.
`4/2012 Reeves et al.
`4/2012 Yusupov et al.
`4/2012 Reeves et al.
`4/2012 Reeves et al.
`4/2012 Reeves et al.
`4/2012 Sirpal ................... G06F 1/1616
`715/783
`
`4/2012
`4/2012
`4/2012
`4/2012
`4/2012
`4/2012
`4/2012
`4/2012
`4/2012
`4/2012
`5/2012
`6/2012
`6/2012
`7/2012
`11/2012
`
`Gimp! et al.
`Benedek et al.
`Benedek et al.
`Reeves et al.
`Reeves et al.
`Reeves et al.
`Liu
`Reeves et al.
`Reeves et al.
`Ording et al.
`Reeves et al.
`Reeves et al.
`Hilerio et al.
`Dharawat
`Momchilov
`
`US 9,703,468 B2
`Page 2
`
`Related U.S. Application Data
`
`application No. 13/604,384, filed on Sep. 5, 2012,
`now Pat. No. 8,990,712, and a continuation-in-part of
`application No. 13/603,136, filed on Sep. 4, 2012,
`now abandoned, and a continuation-in-part of appli(cid:173)
`cation No. 13/410,931, filed on Mar. 2, 2012, now
`Pat. No. 8,872,727, and a continuation-in-part of
`application No. 13/566,336, filed on Aug. 3, 2012,
`now Pat. No. 9,405,459, and a continuation-in-part of
`application No. 13/566,244, filed on Aug. 3, 2012,
`now abandoned, and a continuation-in-part of appli(cid:173)
`cation No. 13/566,168, filed on Aug. 3, 2012, and a
`continuation-in-part of application No. 13/566,103,
`filed on Aug. 3, 2012, now abandoned, and a con(cid:173)
`tinuation-in-part of application No. 13/543,678, filed
`on Jul. 6, 2012, now Pat. No. 9,268,518, and a
`continuation-in-part of application No. 13/543,635,
`filed on Jul. 6, 2012, now abandoned, and a continu(cid:173)
`ation-in-part of application No. 13/408,530, filed on
`Feb. 29, 2012, now abandoned, and a continuation(cid:173)
`in-part of application No. 13/410,958, filed on Mar. 2,
`2012, now Pat. No. 9,122,441, and a continuation(cid:173)
`in-part of application No. 13/410,983, filed on Mar. 2,
`2012, now Pat. No. 9,003,311, and a continuation-in(cid:173)
`part of application No. 13/411,034, filed on Mar. 2,
`2012, now Pat. No. 8,910,061, and a continuation(cid:173)
`in-part of application No. 13/411,075, filed on Mar. 2,
`2012, now abandoned, and a continuation-in-part of
`application No. 13/436,593, filed on Mar. 30, 2012,
`now Pat. No. 9,069,518, and a continuation-in-part of
`application No. 13/436,823, filed on Mar. 30, 2012,
`now Pat. No. 9,213,516, and a continuation-in-part of
`application No. 13/436,826, filed on Mar. 30, 2012,
`now abandoned, and a continuation-in-part of appli(cid:173)
`cation No. 13/485,734, filed on May 31, 2012, now
`Pat. No. 8,874,894, and a continuation-in-part of
`application No. 13/485,743, filed on May 31, 2012,
`now Pat. No. 8,904,165.
`
`(60) Provisional application No. 61/539,884, filed on Sep.
`27, 2011.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6,883,143 B2
`7,369,843 B2
`7,614,018 Bl
`7,987,432 Bl*
`
`4/2005
`5/2008
`11/2009
`7/2011
`
`8,291,344 B2 * 10/2012
`
`8,296,728 Bl* 10/2012
`8,458,286 B2
`6/2013
`8,612,883 B2 * 12/2013
`
`8,677,274 B2
`9,317,195 Bl*
`2004/0172588 Al
`2004/0201628 Al
`2004/0267815 Al
`2006/0230156 Al *
`
`3/2014
`4/2016
`9/2004
`10/2004
`12/2004
`10/2006
`
`2006/0288306 Al* 12/2006
`
`Driskell
`Horiguchi
`Ohazama et al.
`Grechishkin ......... G06F 9/4443
`715/769
`Chaudhri ............ G06F 3/04883
`715/736
`Webster ........................ 717/109
`Freitas et al.
`Louch ................... G06F 3/0481
`715/792
`
`Runov et al.
`Grechishkin ......... G06F 3/0486
`Mattaway
`Johanson et al.
`De Mes
`Shappir ................. G06F 3/1454
`709/227
`Mahajan ............... G06F 9/4445
`715/804
`
`2008/0307350 Al
`2008/0313632 Al
`2009/0094523 Al*
`2009/0164915 Al
`2009/0265628 Al
`2009/0278806 Al
`2009/0328033 Al*
`
`2010/0037145 Al
`2010/0088598 Al
`2010/0138780 Al *
`
`2011/0093836 Al
`2011/0138314 Al*
`2011/0197141 Al
`2011/0225553 Al
`2011/0307783 Al
`2012/0081353 Al
`2012/0081380 Al
`2012/0081383 Al
`2012/0081396 Al
`2012/0084480 Al
`2012/0084481 Al
`2012/0084542 Al
`2012/0084710 Al*
`
`2012/0084737 Al
`2012/0084791 Al
`2012/0084792 Al
`2012/0084793 Al
`2012/0084798 Al
`2012/0086716 Al
`2012/0086717 Al
`2012/0089906 Al
`2012/0089992 Al
`2012/0096396 Al*
`2012/0124245 Al
`2012/0143944 Al
`2012/0166959 Al
`2012/0174021 Al
`2012/0296959 Al*
`
`2013/0019183 Al
`2013/0021262 Al
`2013/0024778 Al
`2013/0024812 Al
`2013/0076592 Al
`2013/0076593 Al
`2013/0076594 Al
`2013/0076664 Al
`2013/0076665 Al
`2013/0076780 Al
`2013/0080143 Al
`2013/00807 59 Al
`2013/0080899 Al
`2013/0080909 Al
`2013/0080933 Al
`2013/0080934 Al
`2013/0080935 Al
`2013/0080936 Al
`2013/0080938 Al
`2013/0080939 Al
`2013/0080940 Al
`2013/0080941 Al
`2013/0080942 Al
`2013/0080943 Al
`2013/0080944 Al
`2013/0080969 Al
`2013/0104051 Al
`2013/0104062 Al
`
`................ 715/799
`
`.............. G06F 9/54
`709/203
`.............. G06F 3/14
`715/738
`
`2012/0324365 Al* 12/2012 Momchilov
`
`1/2013 Reeves
`1/2013 Chen
`1/2013 Reeves
`1/2013 Reeves
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Sirpal et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`3/2013 Reeves et al.
`4/2013 Reeves et al.
`4/2013 Reeves et al.
`
`2007/0011622 Al
`2007/0022387 Al
`2007 /0083829 Al
`2007 /0089064 Al
`
`1/2007 Chae et al.
`1/2007 Mayer et al.
`4/2007 Lauridsen et al.
`4/2007 Facemire et al.
`
`OTHER PUBLICATIONS
`
`Official Action for U.S. Appl. No. 13/604,384, mailed Jun. 4, 2014
`11 pages.
`
`IPR2019-01011 Page 00002
`
`

`

`US 9,703,468 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Official Action for U.S. Appl. No. 13/604,960, mailed May 6, 2014
`26 pages.
`Official Action for U.S. Appl. No. 13/605,145, mailed Jun. 5, 2014
`13 pages.
`Official Action for U.S. Appl. No. 13/605,740, mailed Jul. 16, 2014
`27 pages.
`Zain "Pin Wesbites' bookmarks in Tasklist for Quickest Access
`[Windows 7]." Archived Oct. 10, 2010, Accessed Web. Jan. 15,
`2015.
`http://www.tipsotricks.corn/2010/ 10/pin-wesbites-book(cid:173)
`marks-in-tasklist-for-quickest-access.html, 6 pages.
`Notice of Allowance for U.S. Appl. No. 13/604,384, mailed Oct. 7,
`2014 9 pages.
`Official Action for U.S. Appl. No. 13/604,960, mailed Sep. 23, 2014
`29 pages.
`Official Action for U.S. Appl. No. 13/604,960, mailed Mar. 26, 2015
`32 pages.
`Notice of Allowance for U.S. Appl. No. 13/605,145, mailed Oct. 15,
`2014 10 pages.
`Official Action for U.S. Appl. No. 13/605,740, mailed Jan. 29, 2015
`29 pages.
`U.S. Appl. No. 13/485,734, filed May 31, 2012, Reeves et al.
`U.S. Appl. No. 13/543,678, filed Jul. 6, 2012, Reeves et al.
`Google Image Result for Fujitsu Dual Screen Phone, published date
`unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from: www.
`google.corn/imgres?imgurl~http://www.computerriver.com/im(cid:173)
`ages/dual-screen-phone.jpg.
`Google Image Result for LG Dual Touch Screen Concept Phone by
`Eugene Kim, published date unknown, [retrieved Apr. 18, 2011], 1
`page. Retrieved
`from: www.google.corn/imgres?imgurl~http://
`fgadgets.corn/wp-content/uploads/2010/08/Ig-dual-touch-screen(cid:173)
`phone-Eugene-Kim-0 l .jpg.
`Google Image Result for Fujitsu Dual Screen Phone, published date
`unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from: www.
`google.corn/imgres?imgurl~http://www.gsmdome.com/wp-con(cid:173)
`tent/uploads/2010/ 10/fujitsu-dual-screen-phone_ w2cP7 _54 .jpg.
`Google Image Result for Kyocera Echo, published date unknown,
`[retrieved Apr. 18, 2011], 1 page. Retrieved from: www.google.
`
`corn/imgres?imgurl~http://www.hardwaresphere.com/wp-content/
`uploads/2011/02/kyocera-echo-dual-screen-android-phone-for(cid:173)
`sprint-network.jpg.
`Google Image Result for HTC Triple Viper, published date
`unknown, [retrieved Apr. 18, 2011], 1 page. Retrieved from:www.
`google.corn/imgres?imgurl~http://www. santafemods.corn/Forurn/
`AndroidForums/htcTripleViper.png.
`Google Image Result for Dual-Screen Phone, [retrieved Apr. 18,
`2011],
`1
`page.
`Retrieved
`from:
`www.google.com/
`imgres?imgurl~http://www.netshet.org/wp-content/uploads/2011/
`02/Dual-Scree .
`Website entitled "Lapdock™ for Motorola ATRIX," Motorola
`Mobility, Inc, 2011, [retrieved on Apr. 18, 2011], 1 page. Retrieved
`from: www.motorola.com/Consumers/US-EN/Consumer-Product(cid:173)
`and-Services/Mobile+Ph .
`Website entitled "Motorola ATRIX 4G Laptop Dock Review,"
`phoneArena.com, posted Mar. 2, 2011, [retrieved on Apr. 18, 2011],
`6 pages. Retrieved from: www.phonearena.com/reviews/Motorola(cid:173)
`ATRIX-4G-Laptop-Dock-Review _id2667.
`Burns, C., "Motorola ATRIX 4G Laptop Dock Review," Android
`Community, Feb. 20, 2011, [retrieved on Apr. 18, 2011], 5 pages.
`Retrieved from: www.androidcommunity.com/motorola-atrix-4g(cid:173)
`laptop-dock-review-20110220/.
`Catacchio, "This smartphone has two huge screens ... that rotate,"
`The Next Web, Inc., Oct. 7, 2010, [retrieved on Jul. 21, 2011], 2
`pages. Retrieved from: www.thenextweb.com/asia/2010/10/07 /this(cid:173)
`smartphone-has-two-huge-screens-that-rotate/.
`Posted by Harman03, "Kyocera Echo Dual-screen Android Phone,"
`posted4 weeks from Apr. 18, 2011, [retrieved on Apr. 18, 2011], 3
`pages. Retrieved
`from: www.unp.me/fl06/kyocera-echo-dual(cid:173)
`screen-android-phone-l43 800/.
`Stein, S., "How does the Motorola Atrix 4G Lapdock compare with
`a laptop?" Crave----CNET, Feb. 9, 2011 [retrieved on Apr. 18, 2011],
`7 pages. Retrieved from: www.news.cnet.com/8301-17938_105-
`20031251-l.html.
`Website entitled, "Kyocera Echo," Kyocera Communications, Inc.,
`2011, [retrieved on Aug. 27, 2012], 6 pages. Retrieved from:
`www.echobykyocera.com/.
`Website entitled, "Sony Tablet," Sony Corporation, 2012, [retrieved
`on Aug. 27, 2012], 3 pages. Retrieved from: www.store.sony.com/
`webapp/wcs/ stores/servlet/CategoryDisplay?catalogid~ 105 51
`&storeid~ 10151 &langid~- l&categoryid~8 l 9855 292164479 55 21.
`
`* cited by examiner
`
`IPR2019-01011 Page 00003
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 1 of 104
`
`US 9,703,468 B2
`
`110
`
`114
`
`124
`
`116
`
`116c
`116b
`
`FIG. 1A
`
`108
`
`124
`
`110
`
`114
`
`FIG. 1B
`
`112
`
`112
`
`IPR2019-01011 Page 00004
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 2 of 104
`
`US 9,703,468 B2
`
`100~
`
`FIG. 1F
`
`158
`
`144
`
`148
`
`152
`
`.
`--·----.
`'
`L ... -j_ 176
`iL 128
`
`132
`
`136
`
`140
`
`FIG. 1E
`
`.S" 172a
`....... •
`
`164?.
`t .. .l.
`
`FIG. 1D
`
`FIG. 1C
`
`172a ~ =-
`
`(
`
`12. 172=;;=
`
`FIG., 1G
`
`IPR2019-01011 Page 00005
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 3 of 104
`
`US 9,703,468 B2
`
`0
`
`FIG. 1H
`
`100~
`
`FIG. 1J
`
`FIG. 1/
`
`IPR2019-01011 Page 00006
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 4 of 104
`
`US 9,703,468 B2
`
`100
`
`Screen 1
`--------
`I Gesture Capture I
`I
`Region
`I
`'"'--------
`r-------1
`I
`I
`I
`I
`I Touch Sensitive I
`Display
`I
`I
`I
`I
`I
`I
`I
`L-------
`I Con!!_gurable Are~
`
`120
`
`104
`
`110
`
`112
`
`124
`
`108
`
`Screen 2
`--------
`I Gesture Capture I
`I
`Region
`I
`'----------
`r-------,
`I
`I
`I
`I
`I Touch Sensitive I
`Display
`I
`I
`I
`I
`116 L - - - - - - - -
`Con!!_gurable Area_2
`
`114
`
`216a
`
`208
`
`212
`
`204
`
`216b
`
`Display
`Controller
`
`Memory
`
`Data
`Storage
`
`Processor
`
`Display
`Controller
`
`Cellular
`Telephony
`Module
`
`Other Wireless
`communication
`Module
`
`Power Source and
`Power control Module
`
`220
`
`232
`
`252
`
`256
`
`Port
`Interface
`
`Integrated
`Audio 1/0
`
`236
`
`Hardware
`Buttons
`
`Position
`Sensor{s) Accelerometer(s)
`
`Image
`Capture
`
`1/0
`Audio 1/0
`Interface Module
`
`158
`
`172
`
`176
`
`240
`
`FIG. 2
`
`IPR2019-01011 Page 00007
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 5 of 104
`Sheet 5 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`(:LJ~
`
`340
`
`100
`
`Oc:=
`
`336
`336
`
`w
`
`
`
`328
`
`®
`
`348
`
`100
`
`344
`
`FIG. 3A
`
`IPR2019-01011 Page 00008
`
`IPR2019-01011 Page 00008
`
`

`

`PORTRAfl'
`OPEN CLOSED EASEL MODIFIED PHONE
`EASEL
`BT
`HAT
`X
`HT
`X
`
`HT
`X
`HT
`X
`HT
`
`HT
`HAT
`X
`HT
`HT
`
`p
`p
`p
`X
`p
`
`352
`
`LANDSCAPE
`Ilv1AGE i OPEN CLOSED EASEL MODIFIED PHONE
`VIDEO
`EASEL
`HAT
`l
`HAT
`I
`HAT
`l
`I
`HAT
`X
`HAT
`
`AT
`HA.T
`HAT
`HAT
`HAT
`
`HAT
`AT
`HAT
`HAT
`HAT
`
`HAT
`HAT
`HAT
`HAT
`HAT
`
`p
`p
`p
`X
`X
`
`356
`
`IMAGE
`/ VIDEO
`I
`I
`I
`[
`HAT
`
`/360
`
`\
`
`X
`HT
`HT
`HT
`HT
`
`p
`OPEN
`0 CLOSED
`R
`EASEL
`T
`PH01'-l'E
`R
`IMAGE/
`I
`VIDEO
`A
`T
`OPEN
`L
`AT
`A CLOSED HAT
`N
`EASEL
`HAT
`D MODIFIED HAT
`s
`EASEL
`C
`IMAGE/ HAT
`A
`VIDEO
`p
`E
`
`HAT
`AT
`HAT
`HAT
`
`HAT
`
`HAT
`HAT
`HAT
`HAT
`
`HAT
`HAT
`HAT
`HAT
`
`p
`p
`p
`p
`
`I
`I
`I
`I
`
`HAT
`
`HAT
`
`HAT
`
`AT
`
`X
`HT
`HT
`HT
`
`HT
`
`HT
`X
`HT
`HT
`
`HT
`
`HAT
`HAT
`X
`HAT
`
`.HT
`
`HAT
`HAT
`HAT
`X
`
`HAT
`
`p
`p
`p
`p
`
`p
`
`I
`I
`I
`I
`
`X
`
`\ DOCKED
`
`.,J 364 ~368
`
`Key:
`H - Hall Effect Sensor(s)
`a - accelerometer( s)
`T-Timer
`P- communications Trigger
`I - Image I Video capture Request
`
`I
`
`DOCKING SIGN/-,L
`
`FIG. 3B
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`:-'
`
`2'
`....
`....
`0 ....
`
`'"
`N
`
`-....J
`
`('D
`
`O'I
`
`rJJ =(cid:173)
`('D ....
`0 ....
`....
`
`0
`.i;...
`
`d
`rJl.
`_."-0
`
`~
`0--,
`
`-....l = w
`00 = N
`
`IPR2019-01011 Page 00009
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 7 of 104
`
`US 9,703,468 B2
`
`Tap
`
`Long Press
`
`Drag
`
`Flick
`
`Pinch
`
`Spread
`
`428a 1d432a
`rd432b
`
`428b
`
`,420
`
`•
`FIG. 4A
`
`+
`
`,424
`
`FIG. 4B
`
`•
`
`,400
`
`+-
`
`,404
`
`FIG. 4C
`
`FIG. 4D
`
`428b
`
`428b
`
`428a
`
`432b
`
`..
`
`,408
`
`~-. _ ___,, 410
`
`FIG. 4E
`
`~432a
`~
`
`FIG. 4F
`
`IPR2019-01011 Page 00010
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 8 of 104
`
`US 9,703,468 B2
`
`420~3~ a 428b
`428a
`
`..,. __ __,r- 412
`
`FIG. 4G
`
`43~ a
`420~
`
`428a
`
`~400
`
`.,._
`
`r-416
`
`FIG. 4H
`
`IPR2019-01011 Page 00011
`
`

`

`500\
`
`508
`
`504
`
`Memory
`
`520 ~ -
`
`-Frame~ork
`
`524c__
`
`n
`
`518 7
`
`i Kernel l
`,
`.
`
`·
`I
`lMulti-Display Management Module!
`
`532
`
`~ Wind~w Management Module I
`
`550
`
`:
`: Window Stack
`:______ _ ___ ___J
`
`Processor
`
`548
`
`Frame Buffers
`
`552
`
`5401..
`
`Task
`Management
`Module
`
`Task Stack
`
`542
`
`Application Model
`Manager
`
`512a
`
`512b
`
`512c
`
`2
`,.
`
`FIG. 5A
`
`Driver 1 !
`1 Driv~--;;1
`7 Driv~r 31
`
`I 1 ,
`I :
`
`• • •
`
`536? _
`556 -~-ln_p_u_t M-a-na_g_e_m_e_n_t -M-(cid:144)-d-ul_e__, 1
`~ Event Buffer 1
`' _ __:::===-··· ~
`~ Display Controller
`I ~ - - - - - - - - -
`528, I ~ Surface Cache Module i
`!
`'--' - - - - - - -~
`
`!
`
`I
`
`;
`
`544
`
`Operating System
`----- ~··-- -- --- .
`
`562
`
`Application
`Manager
`
`566
`
`Desktop
`
`5560
`
`,···-···-------·--------
`
`i Application Store :
`
`- - - - - - - - - - - - - - - - - 1
`
`Application 1
`564a
`
`Application 2
`564b
`
`• • •
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`:-'
`
`2'
`....
`.... .
`0 ....
`
`N
`
`-....J
`
`rJJ =(cid:173)
`
`('D
`...,,.,
`('D
`1,0
`
`0 ....
`....
`
`0
`.i;...
`
`d r.,;_
`
`'"'-.0
`
`~
`0--,
`
`--..l = w
`00 = N
`
`IPR2019-01011 Page 00012
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 10 of 104
`
`US 9,703,468 B2
`
`524
`
`Multi-Display Management Module
`
`Display Configuration Module
`
`Preferen~:~odule
`
`I
`
`568
`
`572
`
`574
`
`Device State Module-=]
`
`576 t___ r· .. ---.----------------
`7
`Gesture Module
`I
`~ - - - - - - - - - - - - -
`
`580
`
`584
`
`--------]
`
`Event Module
`
`588
`
`~~---B_i_n_di_n_g ~o~ule
`
`]
`
`FIG. 5B
`
`IPR2019-01011 Page 00013
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 11 of 104
`
`US 9,703,468 B2
`
`N
`CL
`
`co
`0 c.o
`
`'q'
`
`0 c.o
`
`0 .....
`.....
`
`(Ill
`@
`r
`
`CQ
`(0
`
`ci
`tc
`
`~
`
`CL
`
`(Ill
`@
`r
`
`.
`
`~
`.
`Cl tc
`
`IPR2019-01011 Page 00014
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 12 of 104
`Sheet 12 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`a:,
`608
`0
`(!)
`
`FIG.6C
`
`\
`
`<II
`0
`r
`
`(II
`@
`
`,,.,,..... "
`
`0
`<0 .
`u:
`
`(!)
`
`N
`C
`ll.
`
`~
`
`C
`ll.
`
`co
`0 .....
`
`"'" 0 .,,...
`
`. ~
`
`0
`
`O
`0 .....
`r"
`.....
`\—
`
`V
`O
`LO
`
`IPR2019-01011 Page 00015
`
`IPR2019-01011 Page 00015
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 13 of 104
`Sheet 13 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`N
`..J
`
`0
`
`co
`0 .....
`
`.....
`N
`tO
`
`FIG.6E
`FIG.6D
`
`LU
`(0
`
`cj n:
`
`Q
`(0
`
`ci n:
`
`) @
`
`IPR2019-01011 Page 00016
`
`IPR2019-01011 Page 00016
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 14 of 104
`Sheet 14 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`104
`
`""" T""
`114
`
`T""
`
`CX)
`
`0 ....
`108
`
`0
`
`(
`
`.
`
`)
`
`FIG.6F
`
`~
`
`C
`..J
`
`N
`C
`..J
`
`0 ....
`....
`
`)
`
`0 ~ -
`
`~ ---
`
`612
`
`«3v.
`(0
`(D
`
`.... (0
`
`IPR2019-01011 Page 00017
`
`IPR2019-01011 Page 00017
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 15 of 104
`Sheet 15 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`620
`
`114
`
`C\I w
`
`FIG.6H
`
`108
`FIG.BG
`
`104
`
`co
`108
`0 .....
`
`(!)
`(0
`
`(!) u:
`
`IPR2019-01011 Page 00018
`
`IPR2019-01011 Page 00018
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 16 of 104
`Sheet 16 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`110
`
`CX)
`
`0 ...-
`
`><
`ca
`
`0
`
`FIG.6!
`
`(II
`0
`( .....
`
`(II
`©
`r
`
`......
`
`<.o . (!)
`it
`
`IPR2019-01011 Page 00019
`
`VT
`D
`
`N(
`
`104
`
`IPR2019-01011 Page 00019
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 17 of 104
`Sheet 17 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`304
`
`CO
`co
`0v-
`0 .....
`
`0
`
`
`
`.......
`~ 0
`"')
`·-~ ====::t====:lt:::======~
`
`co
`628
`N
`{O
`
`IPR2019-01011 Page 00020
`
`IPR2019-01011 Page 00020
`
`

`

`700~
`
`,;-760
`r---------
`
`V
`
`704
`
`708
`
`764~
`
`-----------,
`
`Active
`Window 1
`(Touch
`Sensitive
`Display 1)
`
`Active
`Window 2
`(Touch
`Sensitive
`Display 2)
`
`FIG. 7A
`
`712
`
`1"
`716
`)
`- - - - -
`1"718
`l _ _ L - -
`I
`I ____ _
`Inactive
`I
`I
`.
`1
`Window 3 1 Inactive
`I
`I
`(Touch
`I Window 4
`I Desktop 2
`I
`Sensitive
`1 (T o~~h
`l
`(Touch
`I
`Display 2) 1 Sensitive
`I Sensitive
`I
`I Display 2)
`I Display 2)
`I
`-----f
`I
`I
`-----r
`I
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`2' :-'
`....
`....
`0 ....
`
`-....J
`
`'"
`N
`
`('D
`('D
`
`QO
`
`rJJ =(cid:173)
`.....
`....
`0 ....
`....
`
`0
`.i;...
`
`d r.,;_
`
`_."-0
`
`~
`0--,
`
`-....l = w
`00 = N
`
`____ ,
`1221 _ l
`-----
`
`724
`
`7201
`
`I
`_____ I
`Inactive
`I
`I
`I
`I Window 5
`Inactive
`I
`I
`I Desktop 11 Window 6 1
`{Touch
`I
`(Touch I
`(To~~h
`I S_ensitive
`I Sensitive I ~ens1t1ve I Display 1)
`I Display 1 )I Display 1) I
`7-----
`I
`I
`I
`- , - - - - -
`
`IPR2019-01011 Page 00021
`
`

`

`728~
`
`732
`
`736
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`7401_
`7441_ r-- _
`r--
`-1
`I
`I
`I
`I
`Window 2
`I Window 1 1 (Inactive)
`I
`(Inactive) :
`I
`I I
`I I
`
`758 <_J L--
`
`:
`
`I
`I
`
`I
`7----,
`I
`- Desktop
`-
`(Active -- Touch Sensitive :
`Display 1 & 2)
`1
`I
`I
`----------
`
`I
`I
`Winqow 3
`(Active -- T o,ch Sensitive
`Displa~ 1 & 2)
`I
`I
`
`_
`
`J748
`J_752
`,
`-
`..5_756
`I-_ _....
`--.
`1--
`I
`I
`I
`I
`I
`I
`Window4 I
`I
`(Inactive) 1 Window 5 I
`I Window 6 I
`1 (Inactive)
`I
`1 (Inactive)
`L ____ r
`1
`l
`I
`I
`._ ___ 7 1
`._ ____ I
`
`~ = :-'
`....
`....
`0 ....
`
`'"
`N
`
`-....J
`
`('D
`
`1,0
`
`rJJ =-('D
`.....
`....
`0 ....
`....
`
`0
`.i;...
`
`FIG. 7B
`
`d r.,;_
`
`'"1,0
`
`~
`0--,
`
`-....l = w
`00 = N
`
`IPR2019-01011 Page 00022
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11, 2017
`Jul. 11, 2017
`
`Sheet 20 of 104
`Sheet 20 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`wt.
`
`vow
`
`C\'.I ct! o.-U) a. 06
`
`
`
`aaFEmmamémcmw53b
`
`I--
`
`co
`wmh
`(0
`,-...
`
`0t.
`
`JIown
`
`,..__
`N
`o6
`..-
`>, >,
`
`
`
`>m§m5mgmogEoo
`
`Q)
`(I)
`:!= >
`~ :e
`a. U)
`E a3
`0 (f)
`.c.
`0
`(.)
`:::i
`0
`
`co
`3:
`0
`"O
`
`Q)
`
`C s:
`> +:;
`(.)
`<t:
`
`(")
`~
`0
`"O
`C
`~
`Cl)
`>
`:;:::
`(.)
`<t:
`
`
`
`m2,852.,95%
`
`w302.2,9:3v32255952
`
`'<;t
`
`::::
`0
`
`-0 c:: s:
`
`Cl)
`>
`:;::.
`(.)
`<t:
`
`0 ......
`Us.OE
`•
`(!) u:
`
`IPR2019-01011 Page 00023
`
`IPR2019-01011 Page 00023
`
`
`
`
`

`

`>
`
`~
`
`768
`
`Window3
`
`760\..
`
`770-,
`
`778-,
`
`768
`782
`
`Window4
`
`Window2
`
`Window 1
`
`Window8
`Window?
`Window4
`Window6
`Window2
`Windows
`
`Window 1
`Desktop
`
`FIG. 7D
`
`Window3
`
`Desktop
`
`FIG. 7E
`
`0
`
`... __.1
`
`774
`L.57~
`...
`L-5i
`78
`"'
`
`.---5 782
`5 786
`...
`
`774
`
`j '-790
`
`Window8
`Window 7
`
`Window6
`
`Windows
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`:-'
`
`2'
`....
`.... ~
`0 ....
`
`N
`
`-....J
`
`('D
`
`rJJ =-('D
`.....
`N ....
`0 ....
`....
`
`0
`.i;...
`
`\0
`
`d r.,;_
`~ = w
`00 = N
`
`~
`0--,
`
`IPR2019-01011 Page 00024
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11, 2017
`Jul. 11, 2017
`
`Sheet 22 of 104
`Sheet 22 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`•
`~ 00
`wNw
`N
`00
`...
`..... m
`.2
`"O
`C:
`
`0
`
`(l.)
`>
`ti
`<(
`...
`
`(l.)
`i.+::
`:.;::;
`C:
`(l.)
`"O
`>,
`~
`a.
`1,/'J
`0
`... 0)
`
`;:;:::
`:.;::;
`C:
`(I.)
`"O
`
`BEBE:m>uo<
`
`5352Emma
`
`
`mafia:Samoaxofiw
`0 .\
`.ax
`
`0
`omw
`N
`co
`
`(0
`
`.....
`wwm
`00
`
`.....
`N
`Nww
`00
`
`00
`wow
`0
`00
`
`sq-
`0
`00
`
`r--.Jf
`J.cow
`0
`0
`co
`
`262E, vow
`
`C
`:e
`1/)
`0
`0...
`~
`(..)
`ro
`....,
`(j)
`
`(/)
`C:
`.Q
`1/)
`C:
`Q) .s
`
`0
`
`wcoficwEE
`5%:de
`
`,_
`Q)
`!+=
`:.;::;
`C:
`Q)
`32
`~
`0
`"O
`C:
`~
`
`sq-
`vmw
`N
`co
`
`.
`co
`
`m..OE
`(!)
`tr:
`
`IPR2019-01011 Page 00025
`
`IPR2019-01011 Page 00025
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 23 of 104
`
`US 9,703,468 B2
`
`900\
`
`908
`
`912
`
`916
`
`920
`
`904
`
`Start
`
`Receive activation of a window
`
`Determine associated display
`
`Determine position in stack
`
`Generate logic structure
`
`924
`
`FIG. 9
`
`IPR2019-01011 Page 00026
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 24 of 104
`
`US 9,703,468 B2
`
`1004
`
`Start
`
`;1000
`
`1008
`
`Execute (Phone) application
`
`1012
`
`1016
`
`Create manifest - - - - - - -
`
`Receive manifest
`
`1020
`
`1028
`
`Determine application display mode _ __,
`
`1024
`
`Set display binding
`
`Provide display binding to the application
`
`1032
`
`1036
`
`1040
`
`NO
`
`YES
`
`YES
`
`FIG. 10
`
`IPR2019-01011 Page 00027
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 25 of 104
`Sheet 25 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0 .....
`.....
`
`21100
`
`0
`0 .....
`
`-q'
`
`0 .....
`.....
`
`"1-t
`
`"1-t .
`• 0'> u::
`
`Fig.11
`
`I
`
`IPR2019-01011 Page 00028
`
`IPR2019-01011 Page 00028
`
`

`

`1205
`
`1210
`
`1215
`
`1220
`
`1225
`
`CPU(S)
`
`Input
`Device(s)
`
`Output
`Device(s)
`
`Storage
`Device(s)
`
`Computer
`Readable/Storage
`Media Reader
`
`;1104
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`2' :-'
`....
`....
`0 ....
`
`-....J
`
`'"
`N
`
`Communication
`Systems
`
`,----·----,
`----\ --
`
`Processing
`Acceleration
`
`I
`I
`I
`
`1230
`
`1235
`
`Fig. 12
`
`I
`I
`I
`
`Working
`Memory
`
`Operating
`System
`
`Other Code
`(Programs)
`
`1255
`
`I
`
`I I 2-1245
`
`I")
`
`1250
`
`1240
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`N
`O'I
`
`0 ....
`....
`
`0
`,i;...
`
`d r.,;_
`
`_."-0
`
`~
`0--,
`
`-....l = w
`00 = N
`
`IPR2019-01011 Page 00029
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 27 of 104
`Sheet 27 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`1300
`
`\
`
`13081‘
`
`Fig.13
`
`1304K
`
`IPR2019-01011 Page 00030
`
`IPR2019-01011 Page 00030
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 28 of 104
`Sheet 28 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`1434
`
`1430 1438
`
`1442
`
`1400
`
`1304K
`
`Fig.14
`
`IPR2019-01011 Page 00031
`
`IPR2019-01011 Page 00031
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 29 of 104
`Sheet 29 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`:2“
`
`2”
`
`3L0V.
`
`1500
`
`1308K
`
`Fig.15
`
`IPR2019-01011 Page 00032
`
`IPR2019-01011 Page 00032
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 30 of 104
`
`US 9,703,468 B2
`
`;1600
`
`1604
`
`Start
`
`1608
`
`1612
`
`1616
`
`1620
`
`Connect device and computer system
`
`Generate unified desktop
`
`Determine status indicators
`
`Provide status indicators together
`
`1624
`
`End
`
`FIG. 16
`
`IPR2019-01011 Page 00033
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 31 of 104
`
`US 9,703,468 B2
`
`;1700
`
`1704
`
`Start
`
`1708
`
`1712
`
`1716
`
`1720
`
`Receive selection of status indicator
`
`Provide information/user interface device
`
`Receive selection of user interface device
`
`Provide menu for status indicator
`
`1724
`
`End
`
`FIG. 17
`
`IPR2019-01011 Page 00034
`
`

`

`W«b
`
`:(<•:-':~s.:.:-:;
`
`-:J~!i,::.~)
`
`l.❖}':❖~
`
`:~{(,:.~}::~:;,:::
`
`M{-:t-:-
`
`::;;.:;:~M
`
`Mow BP's 'top kl!r W(cid:144)!::iil'l;
`
`,,-,,m W(cid:144)!'k
`
`, ~--:,:,
`•·:·::::;:.:-::::,:
`• (• .. -•:•~: ~=··
`
`:::_❖:•:
`
`.-:❖:•··
`·,•;:,:-::
`'•"· :; •. -•: ;,;-:•:•
`
`f h-.ite (O!Pf;idenc.<?. t1ttp:l!blt.
`ly/16Mldk R ,t. 8ARJ\CU$. iik;:,,,
`and di,l;liB$
`
`:::_~rr:t":dHlC•f 01rin<Jc; tob;f'.·::
`
`J !"s~t€ cninf.klen.:€. htqy.//bic
`j ly; l 6Ngdk fL4 S;°\.RACUS. HK€:S
`:::nd dis!lkes
`
`Fig. 18
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`2' :-'
`....
`.... .
`0 ....
`
`N
`
`--.J
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`~
`N
`
`0 ....
`....
`
`0
`.i;...
`
`d r.,;_
`
`'"'-.0
`
`~
`0--,
`
`-....l = w
`00 = N
`
`IPR2019-01011 Page 00035
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 33 of 104
`
`US 9,703,468 B2
`
`1804\
`
`1904
`
`1908
`
`1912
`
`[~ ,.,.,,__
`ti
`II
`
`para ce1ulares prornete. http://
`biUylEIUAp lQulen usa la
`versl6n actual en SU celular?
`
`II
`l\
`I~
`
`1 s: :··:(:·~:f '.: ,;,;··1
`: I hate coincidence. http://bit,
`ly/16N8dk BA BARACUS, Hkes
`and dislikes
`edgar
`1 s. h:>::r,., -::t:l
`@modulor cuando cobres
`lnvitas?
`
`1 :;. t: ~:i ~; r ::: .:::;~)
`'. I hate coinc1dence. http://blt.
`'~ ly/i 6N8dk BA BARACUS, likes
`and disliki~s
`edgar
`1s.;-,,:,",-'<n
`@rnodulor cuando cobres
`lnvitas?
`
`I IIH Every time [ think! might not
`'~ -c ~--------
`
`E
`
`1928
`
`Fig. 19
`
`IPR2019-01011 Page 00036
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 34 of 104
`
`US 9,703,468 B2
`
`;2000
`
`2004
`
`Start
`
`Connect device and computer system
`
`Generate unified desktop
`
`Execute device application
`
`2008
`
`2012
`
`2016
`
`2020
`
`Display application interface in freeform window
`
`2024
`
`End
`
`FIG. 20
`
`IPR2019-01011 Page 00037
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 35 of 104
`
`US 9,703,468 B2
`
`;2100
`
`2104
`
`Start
`
`2108
`
`Receive selection of device application
`
`2112
`
`2116
`
`2120
`
`Determine device application features and/or controls
`
`Provide a freeform window for display of device application
`
`Provide user interface device(s) in the freeform
`window to enable features and/or controls
`
`2124
`
`End
`
`FIG. 21
`
`IPR2019-01011 Page 00038
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 36 of 104
`Sheet 36 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`22200 K
`N \
`
`Fig.22
`
`2216
`
`0
`0
`N
`
`1304
`
`2208
`
`
`
`Nr.
`
`NN
`
`
`
`2204
`
`1304
`
`IPR2019-01011 Page 00039
`
`IPR2019-01011 Page 00039
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 37 of 104
`Sheet 37 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`N
`
`2200
`
`N \
`
`Fig.23 N‘—
`
`N .....
`N
`N
`
`NN
`
`IPR2019-01011 Page 00040
`
`IPR2019-01011 Page 00040
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 38 of 104
`Sheet 38 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`N
`
`2—2200
`N \
`
`2408K1304K 2416
`
`Fig.24
`
`2404K 1304K
`
`N‘—
`
`NN
`
`IPR2019-01011 Page 00041
`
`IPR2019-01011 Page 00041
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 39 of 104
`Sheet 39 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`
`
`<1»
`8
`v'
`r\/
`CD
`co
`0
`8
`LO
`N
`N
`
`q
`N
`a
`
`(0‘._.
`..-
`LON
`I.()
`N
`
`0
`8
`N
`Lo
`LC)
`N
`N
`
`to
`N
`-
`i?
`
`ooNN
`
`0
`0
`N
`
`N \
`
`-s::i-
`0
`I.()
`N
`
`;~
`.•::,
`
`.~.·
`1
`
`1304K N\--
`2504K
`
`IPR2019-01011 Page 00042
`
`..-
`N
`N
`
`NN
`
`J , I ! :t ,,!, l 1 ... !, .,l .tl 1.,!:,
`
`IPR2019-01011 Page 00042
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 40 of 104
`Sheet 40 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`N
`
`2-2200
`N \
`
`2608K1304K
`
`Fig.26
`
`2616
`
`2-2604
`
`2620
`
`1304K Nv
`
`
`IPR2019-01011 Page 00043
`
`NN
`
`IPR2019-01011 Page 00043
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 41 of 104
`Sheet 41 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`N
`
`
`A;2200
`N \
`
`2708K1304K 2716
`
`Fig.27
`
`;2704
`
`2720 NT.
`
`NN
`
`IPR2019-01011 Page 00044
`
`IPR2019-01011 Page 00044
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 42 of 104
`Sheet 42 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`N
`
`2200
`
`N \
`
`Fig.28 2816
`
`
`
`Nv-
`
`NN
`
`;2804
`
`2820
`
`
`
`IPR2019-01011 Page 00045
`
`IPR2019-01011 Page 00045
`
`

`

`2900\
`
`2908-:>
`
`2912-:>
`
`2904~
`
`, 2916
`
`'
`
`\, State when Asleep
`Screen Lock
`Require Screen
`Unlock on Device
`Wake PC and
`Device Interface
`
`Fig. 29
`
`Passcode Lock
`Receive Passcode in PC
`or Device Interface
`Either Allow Call without
`Unlock or Unlock and
`Receive Call
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`~ = :-'
`....
`....
`0 ....
`
`'"
`N
`
`-....J
`
`rJJ =-('D
`.....
`
`('D
`
`.i;...
`~
`
`0 ....
`....
`
`0
`.i;...
`
`d r.,;_
`
`_."-0
`
`~
`0--,
`
`.....J = w
`00 = N
`
`2920-:>
`
`I
`
`I
`
`\,
`Sleep
`Event
`2924-:> IWake Any Input Reveals PC
`and Device Interface
`I Wake PC and Device
`Interface
`
`2928-:>
`
`I Call
`
`IPR2019-01011 Page 00046
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 44 of 104
`
`US 9,703,468 B2
`
`;3000
`
`3004
`
`Start
`
`3008
`
`3012
`
`3016
`
`3020
`
`System sleeps
`
`Receive event
`
`3024
`
`Determine state
`
`Inform user
`of incorrect
`input
`
`NO
`
`3028
`
`3032
`
`YES
`
`Wake system
`
`End
`
`FIG. 30
`
`IPR2019-01011 Page 00047
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 45 of 104
`Sheet 45 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`/3100
`
`1308R
`
`3112
`
`3108
`
`23104
`
`1304K
`
`Fig.31
`
`IPR2019-01011 Page 00048
`
`IPR2019-01011 Page 00048
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 46 of 104
`Sheet 46 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`N
`
`3200
`
`C'? \
`
`3204
`
`3212
`
`3208
`
`Fig.32
`
`IPR2019-01011 Page 00049
`
`IPR2019-01011 Page 00049
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jul. 11,2017
`Jul. 11, 2017
`
`Sheet 47 of 104
`Sheet 47 of 104
`
`US 9,703,468 B2
`US 9,703,468 B2
`
`0
`0
`C'0
`
`2‘3300
`C"? \
`
`3304
`
`3112
`
`3204
`
`1304K
`
`Fig.33
`
`3108
`
`IPR2019-01011 Page 00050
`
`IPR2019-01011 Page 00050
`
`

`

`U.S. Patent
`
`Jul. 11, 2017
`
`Sheet 48 of 104
`
`US 9,703,468 B2
`
`;3400