throbber
1111111111111111 IIIIII IIIII 11111 1111111111 11111 lllll lllll lllll lllll 111111111111111 11111111
`US 20080036743Al
`
`c19) United States
`c12) Patent Application Publication
`Westerman et al.
`
`c10) Pub. No.: US 2008/0036743 Al
`Feb. 14, 2008
`(43) Pub. Date:
`
`(54) GESTURING WITH A MULTIPOINT
`SENSING DEVICE
`
`(75)
`
`Inventors: Wayne C. Westerman, San Francisco,
`CA (US); Steven P. Hotelling, San
`Jose, CA (US); Myra Haggerty, San
`Mateo, CA (US); Bas Ording, San
`Francisco, CA (US); Nima Parivar,
`South San Francisco, CA (US); Duncan
`Robert Kerr, San Francisco, CA (US)
`
`Correspondence Address:
`APPLE C/O MORRISON AND FOERSTER
`,LLP
`LOS ANGELES
`555 WEST FIFTH STREET SUITE 3500
`LOS ANGELES, CA 90013-1024 (US)
`
`(73) Assignee: Apple Computer, Inc.
`
`(21) Appl. No.:
`
`11/700,636
`
`(22) Filed:
`
`Jan.31, 2007
`
`Related U.S. Application Data
`
`(63) Continuation-in-part of application No. 10/903,964,
`filed on Jul. 30, 2004.
`Continuation-in-part of application No. 11/015,434,
`filed on Dec. 17, 2004, which is a continuation of
`application No. 09/236,513, filed on Jan. 25, 1999,
`now Pat. No. 6,323,846.
`
`(60) Provisional application No. 60/072,509, filed on Jan.
`26, 1998.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`G06F 3/041
`(2006.01)
`(52) U.S. Cl. .............................................................. 345/173
`
`ABSTRACT
`(57)
`Methods and systems for implementing gestures with sens(cid:173)
`ing devices are disclosed. More particularly, methods and
`systems related to gesturing with multipoint sensing devices
`are disclosed.
`
`START
`
`12
`
`PROVIDE MUL Tl POINT SENSING
`DEVICE
`
`14
`
`STANDBY
`
`DETERMINE CHORD ASSOCIATED
`WITH TOUCH
`
`YES
`
`DETERMINE GESTURE SET
`ASSOCIATING ONE OR MORE
`ACTIONS TO ONE OR MORE
`GESTURE EVENTS
`
`MONITOR TOUCH FOR GESTURE
`EVENT
`
`22
`
`PERFORM ACTION ASSOCIATED
`WITH GESTURE EVENT
`
`16
`
`18
`
`20
`
`24
`
`Petitioner Samsung Ex-1008, 0001
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 1 of 49
`
`US 2008/0036743 Al
`
`START
`
`12
`
`PROVIDE MULTIPOINT SENSING
`DEVICE
`
`14
`
`DETERMINE CHORD ASSOCIATED
`WITH TOUCH
`
`YES
`
`DETERMINE GESTURE SET
`ASSOCIATING ONE OR MORE
`ACTIONS TO ONE OR MORE
`GESTURE EVENTS
`
`MONITOR TOUCH FOR GESTURE
`EVENT
`
`NO
`
`26
`
`22
`
`NO
`
`YES
`
`PERFORM ACTION ASSOCIATED
`WITH GESTURE EVENT
`
`FIG. 1
`
`18
`
`20
`
`24
`
`Petitioner Samsung Ex-1008, 0002
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 2 of 49
`
`50,
`
`US 2008/0036743 Al
`
`START
`
`52
`
`DETERMINE GESTURE SET FOR TOUCH
`
`MONITOR TOUCH FOR GESTURE EVENT
`
`INITIATE COMMAND WHEN GESTURE EVENT
`IS PERFORMED
`
`FIG. 2
`
`so,
`
`RECOGNIZE ARRANGEMENT OF CONTACTS
`
`•
`+
`
`COMPARE ARRANGEMENT OF CONTACTS
`TO A STORED SET OF ARRANGEMENT OF
`CONTACTS
`
`LOAD GESTURE SET ASSIGNED TO INITIAL
`ARRANGEMENT OF CONTACTS
`
`54
`
`56
`
`58
`
`62
`
`j
`
`6 4
`_)
`
`6 6
`
`l)
`
`FIG. 3
`
`Petitioner Samsung Ex-1008, 0003
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 3 of 49
`
`US 2008/0036743 Al
`
`70~
`
`DETECT TOUCH
`
`1 r
`
`DETERMINE LOCATION OF TOUCH
`
`72
`
`7 4
`
`DETERMINE ARRANGEMENT OF CONTACTS J
`
`,
`
`I
`
`7 6
`
`, r
`
`DETERMINE ACTIVE APPLICATION
`
`7 8
`
`J
`
`, r
`
`
`
`8 0
`
`DETERMINE STATE OF ACTIVE APPLICATION J
`
`, '
`SET GESTURE SET BASED ON THE ABOVE
`
`82
`l)
`
`FIG. 4
`
`Petitioner Samsung Ex-1008, 0004
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 4 of 49
`
`US 2008/0036743 Al
`
`1 0 0~
`
`START
`
`102
`
`106---'-(cid:173)
`I
`I
`I
`I
`I
`I
`I
`L------- ----------------------------
`
`DETERMINE# OF
`CONTACTS
`
`----108
`
`I
`I
`
`110
`
`COMPARE RECOGNIZED ARRANGEMENT OF
`CONTACTS TO STORED ARRANGEMENT OF .__ ______ __,
`CONTACTS IN GESTURE MAP
`
`112
`
`116
`
`118
`
`LOAD GESTURE SET ASSOCIATED WITH
`INITIAL ARRANGEMENT OF CONTACTS
`
`MONITOR TOUCH FOR GESTURE EVENTS
`ASSOCIATED WITH THE GESTURE SET
`
`INITIATE COMMAND ASSOCIATED WITH
`GESTURE EVENT WHEN GESTURE EVENT IS
`PERFORMED.
`
`FIG. 5
`
`Petitioner Samsung Ex-1008, 0005
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 5 of 49
`
`US 2008/0036743 Al
`
`1 2 0~
`
`DETECT TOUCH L/
`
`122
`
`, r
`
`1 24
`DETERMINE CHORD ASSOCIATED WITH __/
`TOUCH
`
`V
`
`1 26
`DETERMINE INPUT FUNCTIONALITY _/
`ASSOCIATED WITH CHORD
`
`LOAD GESTURE SET ASSOCIATED WITH _/
`
`, r
`
`1 28
`
`INPUT FUNCTIONALITY
`
`, r
`
`1 30
`
`INITIATE ACTIONS ASSOCIATED WITH
`GESTURE EVENTS WHEN GESTURE EVENTS
`ARE PERFORMED
`
`FIG. 6
`
`Petitioner Samsung Ex-1008, 0006
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 6 of 49
`
`US 2008/0036743 Al
`
`144
`
`146
`
`148
`
`150
`
`152
`
`DETECT TOUCH
`
`RECOGNIZE ARRANGEMENT OF CONTACTS
`
`DETERMINE CHORD ASSOCIATED WITH
`ARRANGEMENT OF CONTACTS
`
`LOAD GESTURE SET ASSOCIATED WITH
`CHORD
`
`MONITOR TOUCH FOR GESTURE EVENTS
`PERFORMED BY ANY ARRANGEMENT OF
`CONTACTS DURING THE TOUCH
`
`151
`
`154
`
`INITIATE CONTROi/COMMAND ASSOCIATED
`WITH GESTURE EVENT
`
`FIG. 7
`
`Petitioner Samsung Ex-1008, 0007
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 7 of 49
`
`US 2008/0036743 Al
`
`(160
`
`START
`
`164
`
`168
`
`172
`
`PERFORM POINTING
`IN ACCORDANCE
`WITH MOTION
`
`PERFORM
`DRAGGING IN
`ACCORDANCE WITH
`MOTION
`
`PERFORM
`SECONDARY
`DRAGGING IN
`ACCORDANCE WITH
`MOTION
`
`FIG. 8
`
`Petitioner Samsung Ex-1008, 0008
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 8 of 49
`
`US 2008/0036743 Al
`
`180~
`
`182
`
`184
`
`DETERMINE BASE CHORD
`
`186
`
`DETECT MOTION
`
`DETECT NEW
`FIRST DIGIT
`
`DETECT NEW
`SECOND DIGIT
`
`190
`
`PERFORM
`POINTING
`
`PERFORM FIRST
`COMMAND EACH
`TIME DIGIT IS
`DETECTED
`
`PERFORM
`SECOND
`COMMAND EACH
`TIME DIGIT IS
`DETECTED
`
`.188
`
`194
`
`192
`
`FIG. 9
`
`Petitioner Samsung Ex-1008, 0009
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 9 of 49
`
`US 2008/0036743 Al
`
`, - - - - - - - - - - - - - - 1~ DETECTTOUCH
`
`202
`
`204
`
`~ - - - - -
`
`RECOGNIZE ARRANGEMENT
`OF CONTACTS
`
`RECOGNIZE CHORD ASSOCIATED WITH
`ARRANGEMENT OF CONTACTS
`
`LOAD GESTURE SET ASSOCIATED WITH
`CHORD
`
`206
`
`208
`
`210
`
`MONITOR TOUCH FOR A GESTURE EVENT
`PERFORMED BY ANY ARRANGEMENT OF i----------,
`CONTACTS
`
`INITIATE
`CONTROL/
`COMMAND
`ASSOCIATED
`WITH GESTURE
`EVENT
`
`FIG. 10
`
`Petitioner Samsung Ex-1008, 0010
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 10 of 49
`
`US 2008/0036743 Al
`
`(220
`
`START
`
`222
`
`226
`
`FIG. 11
`
`224
`
`228
`
`232
`
`236
`
`PERFORM POINTING
`IN ACCORDANCE
`WITH MOTION
`
`PERFORM
`DRAGGING IN
`ACCORDANCE WITH
`MOTION
`
`PERFORM
`SECONDARY
`DRAGGING IN
`ACCORDANCE WITH
`MOTION
`
`PERFORM
`SCROLLING IN
`ACCORDANCE WITH
`MOTION
`
`Petitioner Samsung Ex-1008, 0011
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 11 of 49
`
`US 2008/0036743 Al
`
`240 \_
`
`START
`
`INITIATE POINTING
`- - - - - - - - UNTIL LIFT OFF OF
`ALL FINGERS
`
`254
`
`DROP
`
`FIG. 12
`
`244
`
`246
`
`250
`
`CLICK
`
`DRAG
`
`248
`
`252
`
`253
`
`Petitioner Samsung Ex-1008, 0012
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 12 of 49
`
`US 2008/0036743 Al
`
`260\
`
`262
`
`DETECT TOUCH L)
`
`1,
`
`RECOGNIZE SIDE OF PINKY
`
`264
`
`L)
`
`11'
`
`DETECT SWIPE WITH SIDE OF PINKY
`
`'I
`
`PERFORM CONTROL/COMMAND
`ASSOCIATED WITH SWIPING PINKY
`
`2 66
`
`L)
`
`2 68
`
`FIG. 13
`
`Petitioner Samsung Ex-1008, 0013
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 13 of 49
`
`US 2008/0036743 Al
`
`START
`
`PROVIDE GESTURE SET MAPPING UPWARD SLIDE TO
`EXPOSE AND DOWNWARD SLIDE TO DASHBOARD
`
`282
`
`RECOGNIZE BASE CHORD
`
`284
`
`288
`
`OPEN
`EXPOSE
`
`286
`
`296
`
`OPEN
`DASHBOARD
`
`298
`
`300
`
`294
`
`290
`
`301
`
`292
`
`CLOSE
`EXPOSE
`
`CLOSE
`DASHBOARD
`
`PERFORM POINTING
`WITH ALTERED
`CHORD
`
`302
`
`FIG. 14
`
`Petitioner Samsung Ex-1008, 0014
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 14 of 49
`
`US 2008/0036743 Al
`
`3 1 0~
`
`DETECT TOUCH
`
`312
`L)
`
`,,
`14
`3
`DETERMINE ARRANGEMENT OF CONTACTS L)
`
`,r
`
`PERFORM ACTION BASED ON
`ARRANGEMENT OF CONTACTS
`
`31
`6
`L)
`
`FIG. 15
`
`320~
`
`MONITOR TOUCH MOTION
`
`322
`L)
`
`324
`, '
`DIFFERENTIATE MOTION BETWEEN A FLICK L)
`
`AND SWIPE
`
`, r
`
`326
`
`PERFORM FIRST l)
`
`ACTION IF MOTION
`IS A FLICK
`
`,,
`PERFORM FIRST l)
`
`3 28
`
`ACTION IF MOTION IS A
`SWIPE
`
`FIG. 16
`
`Petitioner Samsung Ex-1008, 0015
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 15 of 49
`
`US 2008/0036743 Al
`
`330\
`
`DETECT TOUCH
`
`IDENTIFY TWO CONTACTS
`
`332
`
`3 34
`
`J
`J
`
`,,
`MONITORING THE TOUCH FOR A ROTATION
`GESTURE EVENT
`
`J
`
`3 36
`
`,,
`INITIATE ROTATE COMMAND WHEN A FIRST _/
`
`CONTACT IS STATIONARY AND A SECOND
`CONTACT IS ROTATED ABOUT THE FIRST
`CONTACT
`
`3 38
`
`FIG. 17
`
`Petitioner Samsung Ex-1008, 0016
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 16 of 49
`
`US 2008/0036743 Al
`
`340~
`
`I
`
`PROVIDE MULTIPLE WINDOWS
`
`'
`DETECT POINTING MECHANISM OVER FIRST WINDOW
`
`342
`
`344
`
`346
`
`MONITOR TOUCH TO DETERMINE IF CHORD ASSOCIATED WITH~
`POINTING MECHANISM HAS BEEN PRESENTED
`
`INITIATE GESTURE SET ASSOCIATED WITH THE CHORD AND ;t
`35 0 u
`35 2 J
`
`FIRST WINDOW
`
`'
`PERFORM ACTIONS IN FIRST WINDOW WHEN GESTURE EVENTS
`ASSOCIATED WITH THE GESTURE SET ARE PERFORMED WHILE
`POINTING MECHANISM IS DETECTED OVER FIRST WINDOW
`
`'
`DETECT POINTING MECHANISM OVER SECOND WINDOW
`
`11'
`
`MONITOR TOUCH TO DETERMINE IF CHORD ASSOCIATED WITH
`POINTING MECHANISM HAS BEEN PRESENTED
`
`INITIATE SECOND GESTURE SET ASSOCIATED WITH THE CHORD
`AND SECOND WINDOW
`
`35 4
`
`l_/
`
`35 6
`
`~
`
`358
`
`PERFORM ACTIONS IN SECOND WINDOW WHEN GESTURE EVENTS
`ASSOCIATED WITH THE SECOND GESTURE SET ARE PERFORMED
`WHILE POINTING MECHANISM IS DETECTED OVER SECOND WINDOW
`
`FIG. 18
`
`Petitioner Samsung Ex-1008, 0017
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 17 of 49
`
`US 2008/0036743 Al
`
`360\
`
`DETECT TOUCH
`
`362
`
`RECOGNIZE ARRANGEMENT OF CONTACTS ASSOCIATED WITH
`FIRST HAND
`
`LOAD GESTURE SET BASED ON ARRANGEMENT OF CONTACTS
`ASSOCIATED WITH FIRST HAND
`
`RECOGNIZE SECOND ARRANGEMENT OF CONTACTS
`ASSOCIATED WITH SECOND HAND
`
`CHANGE OR MODIFY GESTURE SET FOR FIRST ARRANGEMENT OF
`CONTACTS BASED ON SECOND ARRANGEMENT OF CONTACTS
`
`FIG. 19
`
`364
`
`366
`
`368
`
`370
`
`Petitioner Samsung Ex-1008, 0018
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 18 of 49
`
`US 2008/0036743 Al
`
`380~
`
`PROVIDE MOUSE AND TOUCH SURFACE ARE PROVIDED
`
`MONITOR MOUSE FOR MOUSE EVENTS
`
`SIMULTANEOUSLY MONITOR TOUCH SURFACE FOR GESTURE
`EVENTS
`
`PERFORM MOUSE OPERATIONS IN ACCORDANCE WITH
`MOUSE EVENTS
`
`SIMULTANEOUSLY PERFORM GESTURE OPERATIONS IN
`ACCORDANCE WITH GESTURE EVENTS
`
`4 0 0~
`
`FIG. 20
`
`PROVIDE MOUSE AND TOUCH SURFACE ARE PROVIDED
`
`MONITOR MOUSE FOR MOUSE EVENTS
`
`SIMULTANEOUSLY MONITOR TOUCH SURFACE FOR
`ARRANGEMENT OF CONTACTS AND/OR GESTURE EVENTS
`
`CHANGE MOUSE MODE BASED ON ARRANGEMENT OF
`CONTACTS AND/OR GESTURE EVENTS
`
`FIG. 21
`
`382
`
`388
`
`390
`
`402
`
`404
`
`406
`
`408
`
`Petitioner Samsung Ex-1008, 0019
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 19 of 49
`
`US 2008/0036743 Al
`
`4 1 0~
`
`DETECT FIRST TOUCH ON MUL Tl POINT SENSING SURFACE
`
`RECOGNIZE FIRST ARRANGEMENT OF CONTACTS
`(ASSOCIATED WITH FIRST HAND)
`
`SIMULTANEOUSLY DETECT FIRST TOUCH ON MULTIPOINT
`SENSING SURFACE
`
`SIMULTANEOUSLY RECOGNIZE SECOND ARRANGEMENT OF
`CONTACTS (ASSOCIATED WITH SECOND HAND)
`
`LOAD GESTURE SET ASSOCIATED WITH FIRST ARRANGEMENT
`OF CONTACTS
`
`SIMULTANEOUSLY LOAD GESTURE SET ASSOCIATED WITH
`SECOND ARRANGEMENT OF CONTACTS
`
`412
`
`414
`
`416
`
`418
`
`420
`
`422
`
`424
`
`MONITOR FIRST TOUCH FOR GESTURE EVENTS
`
`426
`
`SIMULTANEOUSLY MONITOR SECOND TOUCH FOR GESTURE
`EVENTS
`
`i - - - -~
`
`INITIATE ACTIONS ASSOCIATED WITH FIRST GESTURE EVENTS
`~ - - - WHEN WHEN FIRST GESTURE EVENTS ARE PERFORMED
`
`428
`
`430
`
`SIMULTANEOUSLY INITIATE ACTIONS ASSOCIATED WITH
`SECOND GESTURE EVENTS WHEN SECOND GESTURE EVENTS---~
`ARE PERFORMED
`.
`
`FIG. 22
`
`Petitioner Samsung Ex-1008, 0020
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 20 of 49
`
`US 2008/0036743 Al
`
`440~
`
`DISPLAY GUI OBJECT
`
`DETECT MULTIPLE POINTERS AT DIFFERENT POINTS OF
`DISPLAYED OBJECT
`
`LOCK POINTERS TO DISPLAYED OBJECT
`
`MONITOR POSITION OF POINTERS RELATIVE TO THEIR
`LOCKED POSITION
`
`MODIFY DISPLAYED GUI OBJECT WHEN ONE OR BOTH OF THE
`POINTERS IS MOVED RELATIVE TO THEIR LOCKED POSITION
`
`4 5 0~
`
`FIG. 23
`
`DETECT TOUCH
`
`RECOGNIZE FIRST ARRANGEMENT OF CONTACTS
`
`DETECT SECOND TOUCH
`
`442
`
`445
`
`446
`
`448
`
`452
`
`SIMULTANEOUSLY RECOGNIZE SECOND ARRANGEMENT OF
`CONTACTS
`
`MONITOR FIRST TOUCH FOR GESTURE EVENTS
`
`SIMUL T. MONITOR SECOND TOUCH FOR GESTURE EVENTS
`
`~ - - - - - - - - - _ _ _ . . _ - - - - - - - - -~ 464
`PERFORM FIRST POINTING OPERATIONS WITH FIRST
`ARRANGEMENT OF CONTACTS
`
`466
`
`SIMULTANEOUSLY PERFORM SECOND POINTING OPERATIONS
`WITH SECOND ARRANGEMENT OF CONTACTS
`
`FIG. 24
`
`Petitioner Samsung Ex-1008, 0021
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 21 of 49
`
`US 2008/0036743 Al
`
`4 7 0~
`
`DISPLAY GROUP OF SELECTABLE ITEMS
`
`DETECT FIRST POINTING MECHANISM OVER GROUP OF
`SELECT ABLE ITEMS
`
`ACTIVATE ITEM UNDERNEATH FIRST POINTING MECHANISM
`
`SIMULTANEOUSLY DETECT SECOND POINTING MECHANISM
`OVER WINDOW
`
`APPLY SELECTED ITEM TO SECOND POINTING MECHANISM
`
`5 0 0~
`
`FIG. 25
`
`DISPLAY GROUP OF MEDIA ITEMS
`
`DETECT POINTING MECHANISM OVER AN ITEM THEREBY
`HIGHLIGHTING THE THE ITEM
`
`DETECT TOUCH ON TOUCH SURFACE
`
`RECOGNIZE ARRANGEMENT OF CONTACTS ASSOCIATED WITH
`TOUCH
`
`INSERT METADATA TIED TO THE RECOGNIZED ARRANGEMENT
`OF CONTACTS INTO THE HIGHLIGHTED ITEM
`
`FIG. 26
`
`472
`
`474
`
`476
`
`478
`
`480
`
`502
`
`506
`
`508
`
`510
`
`Petitioner Samsung Ex-1008, 0022
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 22 of 49
`
`US 2008/0036743 Al
`
`522
`
`524
`
`526
`
`DISPLAY CONTROL OBJECT
`
`DETECT TOUCH OVER CONTROL OBJECT
`
`RECOGNIZE ARRANGEMENT OF CONTACTS ASSOCIATED WITH
`TOUCH
`
`527
`
`528
`
`530
`
`NO
`
`YES
`
`IGNORE TOUCH
`
`MONITOR TOUCH FOR
`GESTURE EVENT
`ASSOCIATED WITH
`CONTROLLING THE
`CONTROL OBJECT
`
`FIG. 27
`
`532
`
`APPLY ACTION
`ASSOCIATED WITH ·
`GESTURE EVENT TO
`CONTROL OBJECT WHEN
`GESTURE EVENT IS
`PERFORMED
`
`Petitioner Samsung Ex-1008, 0023
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 23 of 49
`
`US 2008/0036743 Al
`
`540\
`
`PROVIDE LIST OF INPUT FUNCTIONS
`
`ASSIGN INPUT FUNCTIONS TO CHORDS
`
`THEREAFTER LINK INPUT FUNCTION TO CHORD WHEN CHORD
`IS RECOGNIZED
`
`FIG. 28
`
`580\
`
`PROVIDE LIST OF INPUT FUNCTIONS
`
`PROVIDE COMMANDS ASSOCIATED WITH INPUT FUNCTIONS
`
`PROVIDE LIST OF CHORDS
`
`PROVIDE GESTURE EVENTS ASSOCIATED WITH CHORDS
`
`542
`
`544
`
`546
`
`582
`
`584
`
`586
`
`590
`
`ASSIGN INPUT FUNCTION TO CHORD
`
`ASSIGN COMMANDS TO GESTURE EVENTS
`
`FIG. 30
`
`Petitioner Samsung Ex-1008, 0024
`
`

`

`Standard Trackpad
`
`5
`
`<Custom Set>
`
`List of Chords
`
`l!I
`
`552
`
`558
`
`1 FINGER
`INPUT FUNCTION
`
`2 FINGERS
`INPUT FUNCTION
`
`I~~
`- -f~
`- -rr 2 N~;ctttt~~;tGERS
`~ 3 FINGERS
`- -rr
`
`INPUT FUNCTION
`
`4 FINGERS
`INPUT FUNCTION
`
`Gesture Control Panel
`
`Basic Multitouch
`
`Advanced Multitouch
`
`550
`
`(')
`
`~ .....
`
`""O
`~ .....
`('D = .....
`> "e
`"e -....
`.... 0 = ""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`(')
`
`~ .....
`
`"f'j
`('D
`
`N
`0
`0
`QO
`
`('D
`
`rJJ
`
`=-('D
`.....
`N ...
`0 .... ... 1,0
`
`554
`
`.... ...
`
`List of Input Functions
`Input Function 1
`
`Gesture Set
`
`gesture
`event
`
`gesture
`event
`
`gesture
`event
`
`command
`
`command
`
`command
`
`56
`
`gesture
`event
`
`gesture
`event
`
`gesture
`event
`
`command
`
`command
`
`command
`
`~
`
`gesture
`event
`
`gesture
`event
`
`gesture
`event
`
`command
`
`command
`
`command
`
`FIG. 29A
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0025
`
`

`

`Gesture Contr
`
`Standard Trackpad
`
`Basic Multitouc
`
`< Custom Set>
`
`1~ I
`
`__Si
`552 = List of Chords
`558 ~
`~
`~
`
`1 FINGER
`PoinUclick
`
`2 FINGERS
`Scroll and Secondary click
`
`2 NON ADJACENT FINGERS
`none
`
`~ 3FINGERS
`~ /
`Primary Click and Drag
`
`~·. · .. E~~,~~~~]1~b6ard
`
`.... ...
`
`None
`Point
`PoinUClick
`Scroll
`Scroll & Secondary Click
`Primary Click & Drag
`Secondary Click and Drag
`Move Window
`Expose & Dashboard
`Web Browser Operations
`Editing Operations
`File Operations
`
`X
`
`554
`
`550 )
`
`i
`
`expose all
`
`56
`
`...
`expose
`applications
`
`...
`expose
`desktop
`
`dashboard
`
`!
`
`FIG. 298
`
`""O
`~ .....
`('D = .....
`
`(')
`
`~ .....
`
`(')
`
`~ .....
`
`t "e -....
`.... 0 =
`""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`"f'j
`('D
`
`N
`0
`0
`QO
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`N
`Ul
`
`0 .... ... 1,0
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0026
`
`

`

`None
`Point
`Point/Click
`Scroll
`Scroll & Secondary Click
`gx,,,-;.'.:::'-~erimaryiClicf<t~&IDrag~~~;;\,:i~;~
`Secondary Click and Drag
`Move Window
`Expose & Dashboard
`Web Browser Operations
`Editing Operations
`File Operations
`
`554
`
`550 )
`
`DRAG
`
`DRAG
`
`~ i
`0
`
`56
`
`/
`
`DRAG
`
`--
`DRAG
`
`DRAG
`
`Gesture Contr
`
`Standard Trackpad
`
`Basic Multitouc
`
`_ )
`
`< Custom Set>
`
`1~ I
`
`552
`
`558
`
`List of Chords
`
`1 FINGER
`Point/click
`
`I~
`- -
`~ 2FINGERS
`Scroll and Secondary click
`
`- -J!'1 2 NON ADJACENT FINGERS
`
`none
`
`(cid:141)
`DRAG
`
`Expose and Dashboard
`
`....
`
`'T
`
`DRAG
`
`/
`
`FIG. 29C
`
`CLICK
`
`DRAG
`
`i ~
`
`""O
`~ .....
`('D = .....
`
`(')
`
`~ .....
`
`t "e -....
`.... 0 =
`""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`(')
`
`~ .....
`
`"f'j
`('D
`
`N
`0
`0
`QO
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`N
`O'I
`
`0 .... ... 1,0
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0027
`
`

`

`Gesture Contr
`
`Standard Trackpad
`
`Basic Multitouc
`
`5523
`
`< Custom Set >
`
`List of Chords
`
`@]
`
`~ 2 NON ADJACENT FINGERS
`~ /
`none
`
`558
`
`~ 3FINGERS
`~ /
`Primary Click and Drag
`
`~ 4FINGERS
`~ /
`Expose and Dashboard
`
`None
`Point
`Point/Click
`Scroll
`Scroll & Secondary Click
`Primary Click & Drag
`Secondary Click and Drag
`Move Window
`Expose & Dashboard
`Web Browser Operations
`Editing Operations
`File Operations
`
`x.
`
`554
`
`550 )
`
`(
`
`open
`
`t
`'\,.._
`new
`
`/
`
`6
`
`· ; ; a _~ .~ · THUMB+: 1 FINGER
`
`~. tr -···. FHeQperations
`~ I
`
`-~ THUMB + 2 FINGERS
`none
`
`..... ...
`
`•
`/'-save
`
`FIG. 290
`
`}
`
`close
`
`""O
`~ ....
`('D = ....
`
`(')
`
`~ ....
`
`(')
`
`~ ....
`
`"f'j
`('D
`
`t "e -....
`.... 0 =
`""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`N
`0
`0
`QO
`
`('D
`
`rJJ =(cid:173)
`('D ....
`0 .... ... 1,0
`
`N
`-....J
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0028
`
`

`

`Standard Trackpad
`
`_j
`552 = List of Chords
`
`< Custom Set>
`
`Gesture Contr
`
`Basic Multitouc
`
`I: I
`
`558
`
`~ 4FINGERS
`~ )
`Expose and Dashboard
`
`~ THUMB + 1 FINGER
`
`File Operations
`
`····~·••·
`
`-··-·-··· ..
`
`-····-··.
`
`·• .. ·.Editing,Gp~•=~:::~.;:•.:••·••·=:::c:.
`. -·······-····-······-·-·-·········-···- ........ , ... , ..... _ ..... ,, ......... , .. ,_,_ ........... , ___ _
`
`~ THUMB+ 3 FINGERS
`
`none
`
`-~ THUMB+ 4 FINGERS
`\
`)
`none
`
`....
`....
`
`None
`Point
`PoinUClick
`Scroll
`Scroll & Secondary Click
`Primary Click & Drag
`Secondary Click and Drag
`Move Window
`Expose & Dashboard
`Web Browser Operations
`Editing Operations·
`File Operations
`
`X
`
`r t
`
`cancel
`
`..
`
`undo
`
`0
`
`bktab
`
`t
`/ ' cut
`
`copy
`redo
`
`i
`
`FIG. 29E
`
`554
`
`550 )
`
`56
`
`/
`
`t
`'\,..
`paste
`..
`
`tab
`
`}
`
`select all
`
`""O
`~ .....
`('D = .....
`
`(')
`
`~ .....
`
`(')
`
`~ .....
`
`t "e -....
`.... 0 =
`""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`"f'j
`('D
`
`N
`0
`0
`QO
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`N
`QO
`
`0 .... ... 1,0
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0029
`
`

`

`Gesture Contr
`
`Standard Trackpad
`
`Basic Multitouc
`
`552_5
`/f==-~
`I ~ ) Expose and Dashboard
`~ THUMB+ 1 FINGER
`
`< Custom Set >
`
`List of Chords
`
`4FINGERS
`
`File Operations
`
`5{8
`
`@]
`
`1-1
`
`_ ~ THUMB + 2 FINGERS
`~ )
`Editing Commands
`
`.~;L . . • .. THUM~
`. - ·····--··------
`
`: : FINGERS .
`0
`
`- ~ THUMB + 4 FINGERS
`~ )
`none
`
`~ ....
`~ ....
`
`None
`Point
`PoinUClick
`Scroll
`Scroll & Secondary Click
`Primary Click & Drag
`Secondary Click and Drag
`Move Window
`Expose & Dashbo~~d ... .....
`· Web Browser Operations
`Editing Operations
`File Operations
`
`_
`x
`
`554
`
`ll..,
`
`550 )
`
`home
`
`stop
`
`spot light
`
`6
`
`~ i /
`0
`..
`
`-
`
`back
`
`Cmd
`
`forward
`
`find
`
`/
`
`refresh
`
`Book mark
`
`i ~
`
`FIG. 29F
`
`""O
`~ ....
`('D = ....
`
`(')
`
`~ ....
`
`(')
`
`~ ....
`
`"f'j
`('D
`
`t "e -....
`.... 0 =
`""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`N
`0
`0
`QO
`
`('D
`
`rJJ =(cid:173)
`('D ....
`0 .... ... 1,0
`
`N
`1,0
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0030
`
`

`

`Gesture Contr~
`
`X
`
`Standard Trackpad
`
`Basic Multitouc
`
`552_5
`
`< Custom Set >
`
`List of Chords
`
`~
`
`I
`
`554
`
`.. No~e
`Point.·
`Point/Click
`Scroll
`Scroll & Secondary Click
`Primary Click & Drag
`Secondary Click and Drag
`Move Window
`Expose & Dashboard
`Web Browser Operations
`Editing Operations
`File Operations
`
`550 )
`
`File Operations
`
`Editing Commands
`
`none
`
`~ t /
`
`point
`
`point
`
`point
`
`56
`
`...
`point
`
`point
`
`/
`
`...
`
`point
`
`point
`
`point
`
`i ~
`
`....
`....
`
`FIG. 29G
`
`Expose and Dashboard
`
`- -
`
`- -
`
`558 ~ 4 FINGERS
`~ THUMB + 1 FINGER
`~ THUMB + 2 FINGERS
`~ THUMB+ 3 FINGERS
`'-.: c~.·.·•••.••·••··.• ; THUMB-+A FINGERS
`'!;.: :} .
`none
`
`""O
`~ .....
`('D = .....
`
`(')
`
`~ .....
`
`t "e -....
`.... 0 =
`""O = O" -....
`.... 0 =
`?' ....
`~ ...
`
`(')
`
`~ .....
`
`"f'j
`('D
`
`N
`0
`0
`QO
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`~
`0
`
`0 .... ... 1,0
`
`c
`rJJ
`N
`0
`0
`
`0
`~
`O'I
`
`QO ---0
`-....J ... ~
`> ....
`
`Petitioner Samsung Ex-1008, 0031
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 31 of 49
`
`US 2008/0036743 Al
`
`6 0 0 ,
`
`602
`
`PLACE SYSTEM IN GESTURE MAPPING MODE
`
`DETECT TOUCH
`
`STORE ARRANGEMENT OF CONTACTS ASSOCIATED WITH
`TOUCH
`
`MONITOR TOUCH FOR GESTURE EVENTS
`
`STORE GESTURE EVENT AND LINK TO ARRANGEMENT OF
`CONTACTS
`
`ASSIGN COMMAND TO THE ARRANGEMENT OF CONTACTS/
`GESTURE EVENT
`
`6 2 0 ,
`
`FIG. 31
`
`OPEN COMMAND MENU
`
`DISPLAY LIST OF COMMANDS
`
`HIGHLIGHT A COMMAND WITHIN COMMAND MENU
`
`DETECT TOUCH
`
`STORE ARRANGEMENT OF CONTACTS ASSOCIATED WITH
`TOUCH
`
`MONITOR TOUCH FQR GESTURE EVENT
`
`ASSIGN INITIAL ARRANGEMENT OF CONTACTS/GESTURE
`EVENTS TO HIGHLIGHTED COMMAND
`
`FIG. 32
`
`612
`
`622
`
`628
`
`Petitioner Samsung Ex-1008, 0032
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 32 of 49
`
`US 2008/0036743 Al
`
`640~
`
`DETECT TOUCH
`
`RECOGNIZE ARRANGEMENT OF CONTACTS
`
`SET GESTURE SET BASED AT LEAST IN PART ON
`ARRANGEMENT OF CONTACTS
`
`DETECT PAUSE
`
`DISPLAY GESTURE SET SO THAT USER CAN VIEW COMMANDS
`AND GESTURE EVENTS ASSOCIATED THEREWITH
`
`REMOVE DISPLAYED GESTURE SET WHEN A GESTURE EVENT
`IS STARTED OR AFTER A PRESET TIME OUT
`
`642
`
`646
`
`648
`
`652
`
`FIG. 33
`
`664
`
`660~
`
`662
`
`667
`
`COMMAND 1
`
`(cid:141)
`
`COMMAND4
`
`•
`
`j(~
`COMMAND7
`
`• JI ':i4..
`
`COMMAND 3
`
`..
`
`COMMANDS
`
`__)
`
`COMMAND9
`
`COMMAND2
`
`COMMAND 5
`
`I
`0
`i
`
`COMMANDS
`
`FIG. 34
`
`Petitioner Samsung Ex-1008, 0033
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 33 of 49
`
`US 2008/0036743 Al
`
`ACTIVATE TOUCH SCREEN KEYBOARD
`
`DURING TYPING OPERATION, MONITOR KEY ENTRIES OVER
`KEYBOARD
`
`DURING TYPING OPERATIONS, MONITOR SURFACE FOR
`FINGER SWIPES
`
`682
`
`686
`
`688
`
`692
`
`696
`
`700
`
`690
`
`SPACE
`
`694
`
`BACKSPACE
`
`698
`
`DOWN SWIPE?
`
`ENTER/RETURN
`
`702
`
`UP SWIPE?
`
`COMPLETE WORD
`
`FIG. 35
`
`Petitioner Samsung Ex-1008, 0034
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 34 of 49
`
`US 2008/0036743 Al
`
`700\
`
`RATE EASIEST CHORD
`
`702
`
`__/
`
`__)
`GROUP OR MATCH EASIEST CHORDS WITH u
`
`7 04
`
`7 06
`
`'
`RATE MOST FREQUENT OPERATIONS
`
`MOST FREQUENT OPERATIONS AND GROUP
`OR MATCH HARDEST CHORDS WITH LESS
`FREQUENT OPERATIONS
`
`FINE TUNE EACH GROUP ~
`
`7 08
`
`FIG. 36
`
`Petitioner Samsung Ex-1008, 0035
`
`

`

`FINGER COMBO
`1 FINGER
`1 FINGER
`1 FINGER
`2 FINGER
`' 2 FINGER
`2 FINGER
`2 FINGER
`3 FINGER
`4 FINGER
`5 FINGER
`THUMB + 1 FINGER
`THUMB+ 2 FINGER
`THUMB+ 3 FINGER
`
`GESTURE
`SLIDE
`TAP
`TAP+ 1/2
`2 FING. DOWN
`SLIDE
`TAP
`SPREAD
`-
`-
`-
`-
`-
`-
`
`SPACE
`*
`
`*
`
`*
`
`**
`
`**
`
`*
`
`***
`
`***
`
`****
`*****
`****
`****
`*****
`
`SPEED
`5
`5
`3
`4
`5
`5
`3
`5
`4
`5
`4
`3
`3
`
`RELIABILITY COMFORT
`5
`5
`2
`5
`2
`3
`4
`3
`5
`5
`5
`5
`2
`3
`4
`4
`4
`3
`5
`5
`3
`3
`3
`3
`3
`3
`
`EASE
`5
`5
`2
`4
`4
`4
`3
`4
`4
`4
`3
`2
`2
`
`OVERALL
`5
`4.25
`2.5
`3.75
`4.75
`4.75
`2.75
`4.25
`3.75
`4.75
`3.25
`2.5
`2.5
`
`FIG. 37
`
`""O
`~
`
`~ = .....
`
`t "e -....
`
`(')
`~
`Q;
`
`0 =
`~
`O" = (')
`.... 0 =
`~ .....
`"f'j
`('D
`?'
`....
`J·
`N
`0
`0
`QO
`
`('D
`('D
`
`rJJ =(cid:173)
`.....
`(,H
`Ul
`
`0 ....
`
`.i;...
`1,0
`
`c
`rJJ
`N
`0
`0
`
`QO ---0
`
`0
`(,H
`O'I
`-....J
`.i;...
`(,H
`
`> ....
`
`Petitioner Samsung Ex-1008, 0036
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 36 of 49
`
`US 2008/0036743 Al
`
`COMMAND/
`OPERATION
`
`POINT
`CLICK
`DRAG
`SCROLL
`BACK
`FORWARD
`CONTEXT MENU
`ZOOM
`CUT
`COPY
`PASTE
`SAVE
`EXPOSE
`DASHBOARD
`SPOTLIGHT
`OPEN
`CLOSE
`NEW
`SAVE
`PRINT
`EXIT
`FIND/REPLACE
`UNDO/REDO
`TAB
`ENTER
`
`FREQUENCY
`
`5
`4.5
`4
`3.5
`3
`2.5
`2.5
`2.5
`2
`2
`2
`2
`2
`1.5
`1
`1
`1
`1
`0.5
`0.5
`0.5
`0.5
`0.5
`0.5
`0.5
`
`FIG. 38
`
`Petitioner Samsung Ex-1008, 0037
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 37 of 49
`
`US 2008/0036743 Al
`
`7 4 0~
`
`742
`
`)
`
`744
`
`~
`J
`
`74 6
`
`74 8
`
`i_)
`
`7 50
`
`__)
`
`PROVIDE BASE GESTURE MAP
`
`,,
`
`OPEN FIRST PROGRAM
`
`"
`
`ADJUST BASE GESTURE MAP IN ACCORDANCE WITH
`FIRST PROGRAM
`
`,,
`
`OPEN SECOND PROGRAM
`
`"
`ADJUST BASE GESTURE MAP IN ACCORDANCE WITH
`SECOND PROGRAM
`
`FIG. 39
`
`Petitioner Samsung Ex-1008, 0038
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 38 of 49
`
`US 2008/0036743 Al
`
`7 6 0~
`
`START
`
`764
`
`762
`
`INITIATE EXPOSE
`FUNCTION 1
`
`768
`
`766
`
`INITIATE EXPOSE
`FUNCTION 2
`
`772
`
`770
`
`_)
`
`INITIATE EXPOSE
`FUNCTION 3
`
`774
`
`776
`
`DOWNWARD
`SWIPE
`
`- - - INITIATE DASHBOARD
`
`FIG. 40
`
`Petitioner Samsung Ex-1008, 0039
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 39 of 49
`
`US 2008/0036743 Al
`
`7 8 0~
`
`782
`
`784
`
`786
`
`DETECT BASE CHORD
`
`DETECT PAUSE
`
`OPEN CONTEXTUAL
`MENU
`
`788
`
`_)
`
`790
`
`MOVE HIGHLIGHT BAR
`THROUGH
`CONTEXTUAL MENU
`
`792
`
`794
`
`FIG. 41
`
`IMPLEMENT
`FUNCTIONALITY OF
`HIGHLIGHTED ITEM
`
`Petitioner Samsung Ex-1008, 0040
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 40 of 49
`
`US 2008/0036743 Al
`
`Hand
`primary
`
`# of fingers
`1
`
`primary
`
`2 adj.
`
`primary
`
`2 non adj.
`
`primary
`primary
`
`3
`3
`
`primary
`
`Thumb +1
`
`primary
`
`Thumb +2
`
`primary
`primary
`
`Thumb +3
`Thumb +4
`
`gesture event
`tap
`Position/motion
`tap
`Position/motion
`tap
`Position/motion
`Motion
`Slide right
`Slide left
`Slide up
`Slide down
`Slide right
`Slide left
`Slide up
`Slide down
`Rotate CCW
`Rotate CW
`spread
`contract
`tap
`Rotate CCW
`Rotate CW
`spread
`contact
`motion
`motion
`
`FIG. 42
`
`behavior
`click
`point
`click
`drag
`click
`Secondary drag
`Scroll/pan
`forward
`back
`stop
`reload
`tab
`bktab
`undo
`redo
`cancel
`Select all
`paste
`cut
`copy
`open
`close
`new
`save
`Move window
`point
`
`Petitioner Samsung Ex-1008, 0041
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 41 of 49
`
`US 2008/0036743 Al
`
`Hand
`primary
`
`# of fingers
`1
`
`primary
`
`2
`
`primary
`primary
`primary
`primary
`
`3
`4
`5
`Thumb +1
`
`primary
`
`Thumb +2
`
`primary
`
`Thumb +3
`
`secondary
`
`secondary
`
`secondary
`
`secqndary
`
`1
`
`2
`
`3
`
`4
`
`secondary
`
`5
`
`behavior
`click
`point
`click
`drag
`Move window
`Scroll/pan
`point
`undo
`redo
`paste
`cut
`copy
`Expose 1
`Expose 2
`Expose 3
`dashboard
`replace
`find
`back
`forward
`open
`close
`new
`save
`
`Arrow up (text cursor)
`Arrow down
`Arrow right
`Arrow left
`Apply alUoption to other hand
`
`Select up (e.g., <shift> arrow)
`Select down
`Select right
`Select left
`Move/resize window in
`
`Conjunction with opposite hand
`Page up
`Page down
`begin
`endline
`Apply shift to opposite hand
`Zoom out screen
`Zoom in screen
`Rotate CCW
`Rotate CW
`
`gesture event
`tap
`Position/motion
`tap
`Position/motion
`motion
`motion
`motion
`Slide up
`Slide down
`spread
`contract
`tap
`Slide left
`Slide right
`Slide up
`Slide down
`spread
`contract
`Slide left
`Slide right
`Rotate CCW
`Rotate CW
`spread
`contact
`
`Slide up
`Slide down
`Slide right
`Slide left
`pause
`Slide up
`Slide down
`Slide right
`Slide left
`motion
`
`Slide up
`Slide down
`Slide right
`Slide left
`pause
`spread
`contact
`Rotate CCW
`Rotate CW
`
`FIG. 43
`
`Petitioner Samsung Ex-1008, 0042
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 42 of 49
`
`US 2008/0036743 Al
`
`Hand
`primary
`
`# of fingers
`1
`
`primary
`
`primary
`primary
`
`primary
`primary
`
`primary
`
`primary
`
`2
`
`3
`4
`
`5
`Thumb+
`Middle+
`ring
`Thumb+
`index+
`pinky
`L3 + R3
`
`gesture event
`tap
`Position/motion
`tap
`spread
`contract
`slide
`rotate
`motion
`motion
`tap
`motion
`motion
`
`behavior
`click
`point
`Secondary click (right click)
`Enlarge object
`shrink object
`pan
`Rotate object
`Primary drag
`Scroll pan
`Secondary click (right click)
`point
`Secondary drag
`
`tap
`motion
`Slide up
`tap
`motion
`
`expose
`Window drag, resize
`Expose 3
`Maximize/restore
`
`Opposite comer frame drag/resize
`
`FIG. 44
`
`Petitioner Samsung Ex-1008, 0043
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 43 of 49
`
`US 2008/0036743 Al
`
`FIG. 45
`
`FIG. 46
`
`FIG. 4 7
`
`FIG. 48
`
`Petitioner Samsung Ex-1008, 0044
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 44 of 49
`
`US 2008/0036743 Al
`
`800~
`
`DETECT FIRST FINGER
`
`,,
`
`DETERMINE STATE OF FIRST FINGER
`
`,,
`
`DETECT ADDITIONAL FINGER(S)
`
`,,
`
`DETERMINE STATE OF ADDITIONAL
`FINGER(S)
`
`802
`l_/
`
`8 04
`l_/
`
`8 06
`l_/
`
`08
`8
`L)
`
`,,
`
`81 0
`
`IMPLEMENT DIFFERENT INPUT MODES l)
`
`BASED ON TIMING OF STATES OF FIRST AND
`ADDITIONAL FINGER(S) RELATIVE TO ONE
`ANOTHER
`
`FIG. 49
`
`Petitioner Samsung Ex-1008, 0045
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 45 of 49
`
`US 2008/0036743 Al
`
`820~
`
`DETECT FIRST FINGER
`
`822
`
`J
`
`1r
`
`DETERMINE STATE OF FIRST FINGER J
`
`8 24
`
`H
`
`8 26
`IMPLEMENT FIRST MODE WITH FIRST _/
`FINGER WHEN FIRST FINGER IS IN FIRST
`STATE
`
`1'
`
`MONITOR FOR MODIFIER FINGER EVENT
`WHEN FIRST FINGER IN SECOND STATE
`
`28
`8
`I__)
`
`''
`IMPLEMENT SECOND MODE WITH FIRST _)
`
`FINGER WHEN FIRST FINGER IS IN SECOND
`STATE AND MODIFIER FINGER EVENT IS
`DETECTED
`
`8 30
`
`FIG. 50
`
`Petitioner Samsung Ex-1008, 0046
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 46 of 49
`
`US 2008/0036743 Al
`
`840~
`
`INITIALLY DETECT FIRST FINGER
`
`842
`__/
`
`,r
`
`FINGER WHEN FIRST FINGER IS MOVED J
`
`IMPLEMENT FIRST MODE WITH FIRST
`
`8 44
`
`1'
`
`DETECT SECOND FINGER AFTER
`DETECTING FIRST FINGER
`
`8 46
`__/
`
`,,
`
`8 48
`
`SWITCH FROM FIRST MODE TO SECOND V
`
`MODE WHEN FIRST FINGER IS STATIONARY
`AND SECOND FINGER IS DETECTED
`
`1'
`
`8 50
`
`IMPLEMENT SECOND MODE WITH AT LEAST l)
`
`FIRST FINGER WHEN FIRST FINGER IS
`MOVED
`
`FIG. 51
`
`Petitioner Samsung Ex-1008, 0047
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 47 of 49
`
`US 2008/0036743 Al
`
`860~
`
`START
`
`862
`
`NO
`
`866
`
`PERFORM POINTING
`OPERATION WITH
`FIRST FINGER
`MOVEMENT
`
`NO
`
`868
`
`NO
`
`870
`
`YES
`
`NO
`
`872
`
`YES
`
`PERFORM DRAGGING
`OPERATION WITH
`FIRST FINGER
`MOVEMENT
`
`FIG. 52
`
`Petitioner Samsung Ex-1008, 0048
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 48 of 49
`
`US 2008/0036743 Al
`
`880~
`
`START
`
`NO
`>---~ STANDBY
`
`884
`
`IMPLEMENT
`POINTING MODE
`
`886
`
`STATIONARY
`
`892
`
`MOVING
`
`LOOK FOR MODIFIER
`EVENT
`
`IGNORE MODIFIER
`EVENT
`
`NO
`
`896
`
`888
`
`PERFORM POINTING
`OPERATIONS
`
`SWITCH FROM
`POINTING MODE TO 1 - -~ PERFORM DRAGGING
`DRAGGING MODE
`OPERATIONS
`
`898
`
`FIG. 53
`
`Petitioner Samsung Ex-1008, 0049
`
`

`

`Patent Application Publication Feb. 14, 2008 Sheet 49 of 49
`
`US 2008/0036743 Al
`
`00
`00
`0)
`
`>-0::
`0
`~
`UJ
`~
`
`0::
`0
`CJ')
`CJ')
`UJ
`<.)
`0
`0::
`CL
`
`UJ
`<.)
`
`> UJ
`0
`I-
`:::)
`CL z
`
`00
`LO
`0)
`
`;
`
`CL
`CJ')
`0
`
`co
`LO
`0)
`
`00 co
`
`0)
`
`0
`I"-
`0)
`
`UJ
`0::
`:::)
`LO ------------- 1-
`00
`Cl)
`UJ
`0)
`(.9
`
`0) co
`
`0)
`
`v
`LO .
`CJ
`LL
`
`Petitioner Samsung Ex-1008, 0050
`
`

`

`US 2008/0036743 Al
`
`Feb. 14,2008
`
`1
`
`GESTURING WITH A MULTIPOINT SENSING
`DEVICE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application claims priority benefit of U.S.
`Provisional Patent Application No. 60/763,605, filed Jan. 30,
`2006, and entitled "GESTURING WITH A MULTIPOINT
`SENSING DEVICE," which is herein incorporated by ref(cid:173)
`erence.
`
`[0002] This application is a Continuation-in-Part of U.S.
`patent application Ser. No. 10/903,964, filed on Jul. 30,
`2004, entitled "GESTURES FOR TOUCH SENSITIVE
`INPUT DEVICES," and which is herein incorporated by
`reference.
`
`[0003] This application is a Continuation-in-Part of U.S.
`patent application Ser. No. 11/015,434, filed on Dec. 17,
`2004, entitled "METHOD AND APPARATUS FOR INTE(cid:173)
`GRATING MANUAL INPUT," which claims priority to
`U.S. Pat. No. 6,323,846, issued on Nov. 27, 2001, which
`claims priority to U.S. Provisional Patent Application No.
`60/072,509 filed Jan. 26, 1998, all of which are herein
`incorporated by reference.
`
`[0004] This application

This document is available on Docket Alarm but you must sign up to view it.


Or .

Accessing this document will incur an additional charge of $.

After purchase, you can access this document again without charge.

Accept $ Charge
throbber

Still Working On It

This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.

Give it another minute or two to complete, and then try the refresh button.

throbber

A few More Minutes ... Still Working

It can take up to 5 minutes for us to download a document if the court servers are running slowly.

Thank you for your continued patience.

This document could not be displayed.

We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.

You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.

Set your membership status to view this document.

With a Docket Alarm membership, you'll get a whole lot more, including:

  • Up-to-date information for this case.
  • Email alerts whenever there is an update.
  • Full text search for other cases.
  • Get email alerts whenever a new case matches your search.

Become a Member

One Moment Please

The filing “” is large (MB) and is being downloaded.

Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!

If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document

We are unable to display this document, it may be under a court ordered seal.

If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.


Access Government Site

We are redirecting you
to a mobile optimized page.





Document Unreadable or Corrupt

Refresh this Document
Go to the Docket

We are unable to display this document.

Refresh this Document
Go to the Docket