Apple Ex. 1013
`Apple Inc. v. Firstface Co., Ltd.
`IPR2019-00614
`Page 00001
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 1 0f 12
`
`US 2010/0017872 A1
`
` 4 /
`'--
`-\
`121
`1“
`121
`102
`121
`\
`i
`3
`‘
`3
`m
`
`100
`
`1007
`100
`100 7
`
`IPR2019-00614 Page 00002
`
`IPR2019-00614 Page 00002
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 2 0f 12
`
`US 2010/0017872 A1
`
`200 ~~~~~
`
`200 -----
`
`IPR2019-00614 Page 00003
`
`IPR2019-00614 Page 00003
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 3 0f 12
`
`US 2010/0017872 A1
`
`over the internet (RSS Feed)
`
`Mikael Hagma
`Steve Jobs
`12:00 Lunch Mika
`Rod Johnston
`“‘00 Dev.
`& A
`V
`w?" '
`You have
`t
`G
`.
`‘
`'l
`“fig "9‘3 ms ~
`-
`
`C
`
`.
`
`gwrrerafIASP. Please call me
`1mm; Summer. CALL ME”
`
`ESQRSS
`iKicl shot down scool class in
`l___—_—__—l
`'
`‘
`
`A gadget is an application running on
`the phone. in their icon like state
`Gadgets are also running showing
`information like latest incoming
`message, current song played,
`currency, wheather etc. A gadget can
`be any type of application.
`
`.
`‘
`-Streamlng Radio
`—Note book
`-Web ma”
`‘
`'
`-Phone application
`-Clock .
`etc.
`
`-
`
`’1'
`7
`‘»
`
`A Gadget can be updated with
`information both from the phone
`(new SMS) or from a third party
`
`A Gadget in icon state at the
`home screen can only take a
`defined space
`
`In most cases a gadget can beexpandedeither displayed like an overlay
`directly on the home screen like the volume control or displayedas a full
`screen application like the camera.
`
`IPR2019-00614 Page 00004
`
`IPR2019-00614 Page 00004
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 4 0f 12
`
`US 2010/0017872 A1
`
`Problem: it is hard to hit the right item on a touch display because when you move the finger to and from the
`surface it is also easy move it in X-Y direction by mistake especialy when used with one hand .
`
`Solution: Use a conventional touchtechnology to find the object in X-Y position. Add tactile or graphical ilustration
`were your finger is and use a force sensor to the display to read the activation;
`
`Fig. 6
`E.g. Links in a web site
`Find object by sweeping the screen
`
`
`
`
`
`Activate with force
`
`Find object by sweeping the screen.
`
`E.g. Links in a web site
`
`
`
`
`Fig. 7
`
`Fig. 8
`
`IPR2019-00614 Page 00005
`
`IPR2019-00614 Page 00005
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 5 of 12
`
`US 2010/0017872 A1
`
`Power OFF
`
`
`
`
`
`
`
`Press Home key
`
`Key lock
`
`Press of! Gadget or
`hold Home key for 5 second
`
`Key lock
`high security
`
`
`
`Tap key lock to look
`
`Tap home button
`to start
`
`Tap home button
`to slam. Tap code
`one. iwo, ihree or four digits
`
`Home/ return from application
`Fig. 17
`Fig. 16
`
`; 05543 454 23
`' Lennart
`, Lisa b
`
`—-
`'533III
`B'21.=.a.."m .
`‘1Q 0
`
`
` I]
`
`‘
`
`E3
`
`IPR2019-00614 Page 00006
`
`IPR2019-00614 Page 00006
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 6 0f 12
`
`US 2010/0017872 A1
`
`Scroll alternatives
`
`Fig. 18
`".
`;:‘
`
`.
`
`a»! meow
`‘
`1201 morn-wmas!
`i
`- m
`"
`"WWW
`
`.‘Zv.
`
`Y
`
`t
`'
`‘
`
`u
`
`Alt 4
`Scroll directly on screen.
`Then we miss possibility to
`touch and drag on other
`gadgets
`
`1207 9997““
`
`Fig. 19
` .@
`
`Sula
`5M5
`
`Alt 1
`Scrollbar put finger
`on the side and scroll
`down to get lower down
`on the page
`
`Alt 2
`Drag button
`Put finger on button and
`drag screen up and down
`
`~
`
`Alt 3
`Two drag buttons. Same
`function as Alt 2.
`Put finger on the lower scroll
`button and drag screen up
`or on the upper icon and
`drag down.
`
`IPR2019-00614 Page 00007
`
`IPR2019-00614 Page 00007
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 7 of 12
`
`US 2010/0017872 A1
`
`Notes: I
`
`Danna , Milk.
`
`Visit Art museum .
`Fig. 22
`
`
`
`
`lléili.‘§mai| Traffic
`34'"
`jam
`
`07:00 Jubb
`
`09:0 Frukost
`
`
`
` Telephone
`idea =
`@ Igg® -:§:
`I
`tax-49$
`
`i MediaPlayer
`m hln B
`
`
`
`in Lisa
`i Ls: a-irm Al
`
`
`
`
` NEON
`
`Home
`area
`
`Home screen Based on scrollAltl
`
`Optional feature: Put your finger on the scroll bar
`handle and drag up to display upper Gadgets that are
`less frequently used.The screen is automatically
`scrolled back 3 seconds after the finger is removed
`
`'Put your finger on the screen
`and scroll bar handle and drag down to display display
`more Gadgets down the page.
`Use Home button to get back to the home area.
`
`
`
`
`G
`W
`
`finding content
`on the web
`swarm-arc» —[—-—- Read about new Gadgets
`Updated by
`Wt:
`See VldDD clips
`on the web gal
`mast seen film on the
`saeen
`
`You Tube
`
`Web new: lrom CNN
`Including an RSS teed
`
`IPR2019-00614 Page 00008
`
`IPR2019-00614 Page 00008
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 8 0f 12
`
`US 2010/0017872 A1
`
`Fav 5
`23
`
`-
`Fig. 24 ‘
`
`HIStOry
`Fig. 25
`
`We 2343
`
`; nu: mom
`}
`1107 «mm
`
`Liéa moan
`“P M.
`«one but:
`(but.
`
`‘
`
`I
`
`'
`. you:
`
`ka‘kmC-MIZEA 4
`Wri'ld'h'
`
`Isms
`
`JALL
`
`, a
`35
`Home! Bin 98’! .137
`
`Tap fav 5
`
`Tap Call to make a call
`Tap SMS to SMS that person
`More for rearranging/edit...
`
`Tap history
`
`Tap to call or SMS
`
`Optionai: lncIuded SMS history.
`
`Telephone
`
`Fig. 27
`
`Fig. 28
`
`
`
`
`
`
` D
`
`’
`
`‘
`
`‘
`‘
`,» 01mm
`
`
`1207 ”Gina! Im ‘
`W
`‘
`
`IPR2019-00614 Page 00009
`
`IPR2019-00614 Page 00009
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 9 0f 12
`
`US 2010/0017872 A1
`
`t
`!
`‘
`
`
`
`‘
`
`
`
`
`MP3 player
`ng. 33
`Fig. 34
`1}
`«1)
`050
`.
`VJQ
`
`0:30
`.
`-‘
`:lh.‘ I799“
`i201 worn-mm
`is m...
`iiiiim
`
`Volume
`
`-
`
`
`
`
`l
`
`A
`V
`
`Press vo ume
`button
`
`At (his point you can select to
`change another sound source.
`Control disappears after 4
`seconds
`
`Tap player
`
`Controil disappears after 4
`seconds
`
`Operator add
`Fig. 35
`
`Bureaus
`
`
`Market 100 Free SMS
`User group: A3¢
`
`Add:
`
`0 e ators screen
`p r
`
`4. Tap Add —> operator web page
`or java app.
`
`Alt 1 End user can move it around. A click trough creates kickback.
`Alt 2 Fixed position. Reduced monthly fee.
`
`IPR2019-00614 Page 00010
`
`IPR2019-00614 Page 00010
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 10 0f 12
`
`US 2010/0017872 A1
`
`
`
`Tap (he screen and use controls
`Controls disappears after 4
`
`Tap video
`
`Camera
`
`seconds
`Use controis
`
`L n H
`
`"— 0mm
`'
`12.070‘"~W|
`1
`1 mm
`.
`.
`
`.
`
`1
`A
`
`Lennart
`Lisa b
`Lona
`
`Tap Camera
`
`Send Picture to
`
`IPR2019-00614 Page 00011
`
`IPR2019-00614 Page 00011
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 11 of 12
`
`US 2010/0017872 A1
`
`Messaging SMS
`Fig. 44
`
`@%m
`“£0 Moth-31050 938737 moo Fathnr Him!
`
`14:00 Kano
`
`_
`
`Tap Message gadget
`
`Seiect Message
`
`Coutd also be same structure as call history
`See previous description.
`
`IPR2019-00614 Page 00012
`
`IPR2019-00614 Page 00012
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 12 of 12
`
`US 2010/0017872 A1
`
`Home screen
`
`Fig. 46
`
` G1; Phone status
`
`
`fig). @251" mini-m
`G3. Web CNN '
`
`Pan home screen
`
`
`
`‘
`
`Pan up and down.
`Pm you linger on the screen
`and drag up or down
`
`Neonode 2008-04-20
`
`Magnus Goertz
`
`Problem: Only one browser session can be opend at a time.
`How can we tell the user that in an intuitive way and still stay
`in the same screen and keep important information at the top?
`
`Solution: Always open the browser to the bottom of the screen
`and keep content above the browser Gadget.
`
`Open gadget! browser
`Fig. 48
`
`Open gadget / browser
`Fig. 49
`
`099“ gadget / browser
`Fig. 50
`
`
`,,
`r ,
`i G3. Web CNN
`
`CNN Headlines
`L , ; ___; _; /
`
`
`
`
`MGSIS
`
`precede
`larch relay “1
`
`
`
`
`[63, Web oi
`CNN Headline.
`
`
`
`E;,\
`G5.'G-fl'\ali
`
`Push/Tap and release 8
`Gadget lo open/activate
`
`When Web-Gadget is activated -all
`gadgets beneath Web—Gadget slides
`down and disappears. instead the
`browser takes this space
`
`;(
`
`PaningGadget
`
`CI in Headlines
`
`
`
`
`Paning WED
`Fig. 54
`GI, Phone status
`
`"isfl‘igfitmfin/
`——
`
`
`(33, Web CNN
`t
`.
`rw- , — ~-
`
`lii
`
`Paning Web
`Fig. 53
`61. Phone status
`
`
`[7324 «EDIE-11$”),]
`~—:‘
`
`G3. We!) CNN
`_
`,
`F
`k‘
`
`CNN Headlines
`
`
`
`Paning Gadget
`Fig_ 52
`31‘ Phone status
`
`731I1'353ll raw
`.
`;;.;
`~.
`63. We!) CNN
`‘
`Crm Heaclines
`
`
`
`
`
`: 63. Web CNN
`
`Paning a_bow bawser window will result
`in changing size of browser Window.
`
`Paning inside bowser window will result
`in paning browser content.
`
`IPR2019-00614 Page 00013
`
`IPR2019-00614 Page 00013
`
`

`

`US 2010/0017872 A1
`
`Jan. 21,2010
`
`USER INTERFACE FOR MOBILE
`COMPUTER UNIT
`
`CROSS REFERENCES TO RELATED
`APPLICATIONS
`
`[0001] This application is a continuation-in-part ofpending
`application U.S. Ser. No. 10/315,250, filed on Dec. 10,2002,
`entitled USER INTERFACE FOR MOBILE HANDHELD
`
`COMPUTER UNIT. This application claims priority from
`provisional application No. 61/132,469, filed on Jun. 19,
`2008, entitled IMPROVED KEYPAD FOR CHINESE
`CHARACTERS.
`
`FIELD OF THE INVENTION
`
`[0002] The field of the present invention is user interfaces
`for electronic devices and, more particularly, to touch screen
`user interfaces.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Touch screens provide user interfaces through
`which a user enters input to a computing device by touching
`a screen at a selected location, with a stylus or with his finger.
`[0004] Conventional touch screens are limited as to the
`types of user inputs that they can recognize. For example,
`conventional touch screens are unable to distinguish between
`a soft tap and a hard press. In some prior art embodiments
`users initially select an item on the screen, and then subse-
`quently activate the selected item. However, because prior art
`touch screens do not distinguish between degrees ofpressure,
`the user is required to remove his finger or stylus from the
`screen and activate his selection with a second tap. It would be
`advantageous to produce touch screens that distinguish
`between varying amounts ofpressure applied to the screen so
`that a user can select an item by touching its location on the
`screen, and then activate the item by applying additional
`pressure to the touch location without having to first remove
`his finger or stylus from the screen.
`[0005]
`In conventional touch screens the keys are often
`small relative to the touch area, especially in handheld
`devices. The keys are also often situated close together. This
`can make it difficult to determine which key is being pressed
`by the user. It wouldbe advantageous to clearly indicate to the
`user which key the user has selected, and furthermore, to
`allow the user to accept or reject the selected key without first
`removing his finger or stylus from the screen in order to
`perform a second tap.
`[0006] Current user interfaces are basic and often require
`navigation through a series of menus in order to perform a
`desired operation. The present invention presents a user inter-
`fere that is versatile in providing the user with many options,
`while requiring only few selections to activate a desired func-
`tion. To further enhance user experience, certain functions are
`performed automatically without requiring the user to enter a
`selection.
`
`SUMMARY OF THE DESCRIPTION
`
`[0007] Aspects of the present invention relate to user inter-
`faces designed for use with a touch screen. The present inven-
`tion relates to computer readable media storing computer
`programs with computer program code, which, when read by
`a computer unit, allows the computer to present a user inter-
`face for the computer unit.
`
`In accordance with embodiments of the present
`[0008]
`invention, the computer unit features a touch sensitive display
`area. According to preferred embodiments an initial display
`configuration presents a plurality of gadgets on the display.
`These gadgets are small areas of the screen that indicate
`which functions each gadget will perform when activated by
`the user. When a gadget is activated, typically by the user
`touching the area of the display on which the gadget is dis-
`played, the gadget increases in size and provides the user with
`icons and information about the gadget’s functions. Signifi-
`cantly, the gadget does not cover the entire display area. Thus,
`when a gadget, or even a plurality of gadgets, is active, the
`primary display of all available gadgets is still accessible.
`This primary display can be compared to a desktop in com-
`puter operating system user interfaces. However, this primary
`display in the user interface ofthe present invention is not the
`same as a desktop where active windows can cover icons on
`the desktop. In the present invention, gadgets are arranged in
`a manner that open gadgets do not cover other gadgets.
`Rather, when an open gadget expands in size, other gadgets
`are shifted to make room for the expended gadget. This allows
`the user to scroll the primary display or desktop to view any
`gadget. In the context of the present invention, this primary
`display area that includes both open and closed gadgets is
`called the home window. The user scrolls the home window to
`
`view gadgets that are shifted beyond the viewable area of the
`display. An expanded, or activated, gadget has an expanded
`window, but often it is not large enough to display everything
`contained in that gadget window. To view contents of the
`gadget not displayed in the visible portion of the gadget
`window, the user scrolls the gadget window. Thus, two dif-
`ferent scroll operations are provided: scrolling the home win-
`dow and scrolling a gadget window. According to one
`embodiment, scrolling is executed by gliding a finger or sty-
`lus along the touch screen to shift the active display area ofthe
`home window or ofthe gadget. The scrolling affects the home
`window if the finger glide began in an area of the screen that
`does not belong to an active gadget; the scroll affects an active
`gadget window if the finger glide began inside that active
`gadget window.
`[0009] Various embodiments of the invention support sev-
`eral methods of scrolling a window. According to one
`embodiment, scrolling is done when the user touches the
`display inside the window area, for example at an edge of the
`window, or on an icon, such is a arrow or scrollbar, indicating
`a scroll operation. According to another embodiment, scroll-
`ing is done by the user touching the window with a finger or
`stylus and then gliding the finger or stylus along the torch
`sensitive screen in a direction indicating the desired direction
`ofthe scroll. When the content ofthe home display is larger in
`two dimensions than the actual display screen, this operation
`is like panning an image or map. When it is larger along only
`one axis (e.g., only vertically), the scrolling only scrolls in the
`one axis even when the glide is not orthogonal along the one
`ax1s.
`
`[0010] Another aspect of the present invention relates to
`computer readable media storing a computer program with
`computer program code, which, when read by a mobile hand-
`held computer unit, allows the computer to present a user
`interface for the mobile handheld computer unit. The user
`interface features a touch sensitive area in which representa-
`tions of a plurality of keys are displayed, and each key is
`mapped to a corresponding location in the touch sensitive
`area at which the representation ofthe key is displayed. A key
`
`IPR2019-00614 Page 00014
`
`IPR2019-00614 Page 00014
`
`

`

`US 2010/0017872 A1
`
`Jan. 21,2010
`
`in this context includes, inter alia, alphabetic keys such as in
`a QWERTY keypad, numeric keys and also icons represent-
`ing programs or functions. Akey is selected, but not activated,
`when an object touches the corresponding location. This
`intermediate status of being selected but not activated facili-
`tates the user to subsequently activate a desired key and avoid
`activating a neighboring key that the user selected but does
`not wish to activate. A selected key is activated when the
`object touching it applies additional pressure to the key loca-
`tion.
`
`[0011] According to preferred embodiments of the inven-
`tion, when a key is selected, the user interface generates a
`secondary representation of the key, such as a callout balloon
`containing the key representation. The callout balloon is
`placed away from the key location (being touched) so that the
`user can easily view which key is selected without lifting his
`finger. According to another embodiment, an audio represen-
`tation of the selected key is generated so the user hears which
`key was selected.
`[0012] According to still further features in preferred
`embodiments of the invention, the user touches the screen
`(with a finger or stylus) at a first location, for example select-
`ing a first key. The user then glides his finger or stylus over the
`screen to additional locations. At each additional location a
`
`new key is selected and the previously selected key is dese-
`lected. The user can activate any selected key by applying
`additional pressure to the screen. The user does not have to
`remove the object from the screen to glide and select addi-
`tional keys even after activating a first key.
`[0013] Additional touch pressure is detected in various
`ways according to several embodiments. According to one
`embodiment, the touch sensitive area is a light-based touch
`screen operable to detect different levels of touch pressure.
`For example, light-based touch screens typically include a
`calculating unit operable to identify the size, shape and con-
`tours of an area being touched based or a pattern ofobstructed
`light. See applicant’s co-pending US. patent application Ser.
`No. 10/494,055, titled ON A SUBSTRATE FORMED OR
`RESTING DISPLAY ARRANGEMENT,
`the contents of
`which are incorporated herein by reference. When a finger or
`flexible object is used as a touch object, as additional pressure
`is applied to the touch surface, the contact area ofthe finger or
`object touching the screen increases. Thus, additional pres-
`sure is detected when an increase in the contours of the
`covered touch area is detected.
`
`[0014] Alternatively, or in combination with the above, the
`touch sensitive area features both a touch screen operable to
`identify a touch location on the screen and a pressure sensor
`operable to detect pressure applied to the screen but not
`sensitive to the location of the object applying the pressure.
`[0015] Other aspects of the present invention relate to con-
`venient arrangement and function oficons to perform popular
`functions within a user interface. Thus, a camera gadget fea-
`tures a multimedia messaging service (MMS) button facili-
`tating sending an active photo in an MMS message; a keylock
`gadget locks the computer and displays an instrumental key-
`pad for entering a musical code to unlock the computer; a
`reporting gadget displays information for a first period of
`time and is then automatically deactivated. Several reporting
`gadgets are provided, including a gadget that displays the
`time of day; a gadget displays a weather forecast; a gadget
`that displays stock market information.
`[0016] According to still further features in preferred
`embodiments the reporting gadget continues to display its
`
`information for a second period of time if the gadget is
`touched during the first period of time. I.e., automatic deac-
`tivation after the first period of time is canceled.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0017] The present invention will be more fully understood
`and appreciated from the following detailed description,
`taken in conjunction with the drawings in which:
`[0018]
`FIG. 1 is a series of screens illustrating Chinese
`stroke input,
`in accordance with an embodiment of the
`present invention;
`[0019]
`FIG. 2 is a series of screens illustrating Chinese
`Pinyin input,
`in accordance with an embodiment of the
`present invention;
`[0020]
`FIGS. 3 is a screen of a handheld terminal display-
`ing a plurality of gadgets, in accordance with an embodiment
`of the present invention;
`[0021]
`FIGS. 4-6 are diagrams illustrating selection of ele-
`ments displayed on a touch screen. A selected element is
`displayed in a callout as shown in FIGS. 5-6. The user does
`not lift his finger off the screen between selection of different
`elements, as in FIGS. 5 and 6. The use simply glides his finger
`along the screen. This is shown in FIG. 7. To activate a
`selected element the user presses on the screen without first
`lifting his finger off the screen as shown in FIG. 8 and in the
`series of FIGS. 7-8, all in accordance with an embodiment of
`the present invention;
`[0022]
`FIGS. 7-8 illustrate a side view of a user selecting
`and activating an on-screen element, in accordance with an
`embodiment of the present invention;
`[0023]
`FIGS. 9-11 illustrate turning a phone on and off.
`FIG. 9 illustrates pressing the home key on a turned-off
`phone. FIG. 10 illustrates the phone when it is turned on. FIG.
`11 illustrates pressing the home key to turn a phone off.
`[0024]
`FIGS. 12-15 illustrate locking and unlocking a
`phone. FIG. 12 illustrates activating a lock gadget onscreen to
`lock the phone. FIG. 13 illustrates pressing the home button
`on a locked phone to unlock the phone. FIG. 14 illustrates the
`unlocked phone. FIG. 15 shows pressing the home key on a
`locked phone to display a keypad for entering a security code
`to unlock the phone.
`[0025]
`FIGS. 16-17 illustrate pressing the home key to
`return to the home display screen. FIG. 16 shows an open
`active calling gadget. Pressing the home button presents the
`home screen shown in FIG. 17.
`
`FIGS. 18-21 illustrate scrolling options. In the
`[0026]
`phone shown in FIG. 18 scrolling is done by gliding a finger
`or stylus along the touch sensitive screen. FIG. 19 illustrates
`a scrollbar along one edge of the screen for scrolling up and
`down. FIGS. 20-21 illustrate arrow buttons for scrolling up
`and down.
`
`FIG. 22 illustrates scrolling the home display
`[0027]
`screen, in accordance with an embodiment of the present
`invention;
`[0028]
`FIGS. 23 -24 illustrate a Fav5 gadget, in accordance
`with an embodiment of the present invention;
`[0029]
`FIGS. 25-26 illustrate a History gadget, in accor-
`dance with an embodiment of the present invention;
`[0030]
`FIGS. 27-28 illustrate a Calling gadget, in accor-
`dance with an embodiment of the present invention;
`[0031]
`FIGS. 29-30 illustrate placing a call using a Calling
`gadget, in accordance with an embodiment of the present
`
`IPR2019—00614 Page 00015
`
`IPR2019-00614 Page 00015
`
`

`

`US 2010/0017872 A1
`
`Jan. 21,2010
`
`invention; FIG. 30 illustrates the shortcut buttons presented
`when an incoming call is received (i.e., when the phone is
`ringing).
`FIGS. 31-32 illustrate volume control, in accor-
`[0032]
`dance with an embodiment of the present invention;
`[0033]
`FIGS. 33-34 illustrate an MP3 player gadget, in
`accordance with an embodiment of the present invention;
`[0034]
`FIGS. 35-37 illustrate an operator promotion gad-
`get, in accordance with an embodiment of the present inven-
`tion;
`FIGS. 38-39 illustrate a video player gadget, in
`[0035]
`accordance with an embodiment of the present invention;
`[0036]
`FIGS. 40-42 illustrate a camera gadget and shortcut
`buttons provided when the camera gadget is activated, in
`accordance with an embodiment of the present invention;
`[0037]
`FIGS. 43-45 illustrate an SMS gadget and shortcut
`buttons provided when the SMS gadget is activated, in accor-
`dance with an embodiment of the present invention; FIG. 45
`illustrates how related SMS messages are displayed as a
`conversation in an active SMS gadget.
`[0038]
`FIGS. 46-54 illustrate scrolling the home display
`screen and gadget windows, in accordance with an embodi-
`ment of the present invention;
`
`DETAILED DESCRIPTION
`
`[0039] Touch screen keypads offer great flexibility in key-
`pad interface designiflexibility that cannot be achieved with
`electromechanical keypads. Custom design keypads can be
`generated on a touch screen, where the markings on each key
`and the functions that each key provides are optimized for a
`designated application. Moreover, touch screen keypads can
`change modes, from one pad ofkeys and associated functions
`to a different pad of keys and associated functions. Custom
`keypads are of particular advantage for multi-lingual appli-
`cations.
`
`[0040] A general description of touch screen keypad inter-
`faces, in accordance with embodiments of the present inven-
`tion, and several examples thereof, are provided in the Appen-
`dix and described in detail herein below.
`
`invention relate to
`[0041] Embodiments of the present
`improved keypads for inputting Chinese characters using
`XT9 stroke input, and using Chinese Pinyin. XT9 stroke input
`builds Chinese characters using six basic strokes, and offers
`selection of possible characters and phrases based on a set of
`strokes that have been input. Chinese Pinyin uses Latin char-
`acters that transliterate a sound or a syllable, in combination
`with a digit that represents an intonation or inflection. E.g.,
`Ma in a rising tone is m-a-1, and Ma in a descending tone is
`m-a-2.
`
`[0042] Reference is now made to FIG. 1, which is an illus-
`tration of an improved stroke-based keypad for entering Chi-
`nese characters by inputting strokes, in accordance with an
`embodiment of the present invention. Shown in FIG. 1 is a
`keypad 100, used for inputting basic strokes. Also shown in
`FIG. 1 is a sequence of strokes 101-106 that are successively
`input by a user, and a corresponding sequence of Chinese
`characters 111-114 that are entered in a message being com-
`posed.
`In accordance with an embodiment of the present
`[0043]
`invention, keypad 100 is generated and displayed on a touch
`screen. Keypad 100 has fewer than the standard 12 keys in a
`touch pad, allowing more room on screen for displaying
`characters.
`
`Further in accordance with an embodiment of the
`[0044]
`present invention, the keys of keypad 100 are customized so
`that they contain only relevant information. For example, a
`prior art keypad displays a digit, 3-4 characters, and a basic
`Chinese stroke, all inside one key, even though in XT9 stroke
`mode the basic Chinese stroke is the only useful one. The
`custom keys of the present invention display only the basic
`Chinese strokes, or the strokes and numbers, but no charac-
`ters.
`
`[0045] There are two types of key presses supported by
`keypad 1007regular and long. A regular key press adds the
`stroke shown on the key to the series of strokes 101-106
`already pressed. As strokes are successively entered, a num-
`bered array 121 of Chinese characters or phrases is dynami-
`cally displayed along the top of the keypad. These characters
`or phrases are predicted based on the key presses already
`entered. In order to select one of the numbered elements of
`
`array 121, the user performs a long key press on that number.
`Alternatively, the user may keep entering strokes until only
`one option remains.
`[0046] Often, however, more predicted characters or
`phrases exist than can be displayed along the top of the
`keypad. The N2 phone, manufactured by Neonode of Stock-
`holm, Sweden, has a joystick button at the bottom of the
`phone. Twitching the joystick up/down displays different sets
`of predicted characters or phrases. When the intended char-
`acter or phrase is displayed and associated with a given digit,
`a long press on that digit serves to select the intended char-
`acter or phrase.
`[0047] Reference is now made to FIG. 2, which is an illus-
`tration of an improved keypad for inputting Chinese charac-
`ters using Pinyin, in accordance with an embodiment of the
`present invention. Shown in FIG. 2 is a keypad 200, used for
`inputting Latin characters.Also shown in FIG. 2 is a sequence
`of Latin characters M-o-H-S-a-n that are successively input
`by a user, and a corresponding sequence of Chinese charac-
`ters that are entered in a message being composed.
`[0048]
`In accordance with an embodiment of the present
`invention, keypad 200 uses accent characters, rather than
`digits, to convey an intended intonation or inflection. Further
`in accordance with an embodiment of the present invention,
`keypad 200 displays only information relevant for Pinyin
`input on each key; no Chinese basic strokes are shown
`[0049] There are two types of Pinyin input. A user enters a
`Latin transliteration of an intended word using the Latin
`keypad input (12 keys). For each key, several letters are pos-
`sible. The list ofpredicted Latin syllables based on the current
`sequence of keypad presses is displayed. Twitching the joy-
`stick right or left selects the desired combination. Also, a
`series of predicted Chinese characters or phrases is displayed
`and selected by a long press on a respective digit. Twitching
`the joystick up/down displays other predicted Chinese char-
`acters or phrases. Entering a space after a series of letters
`indicates the end of a previous character or phrase.
`[0050]
`In accordance with an embodiment of the present
`invention, the user is able to combine stroke and Pinyin input,
`and compose a sequence of at least two Chinese characters
`using XT9 stroke input for at least one character and Pinyin
`input for at least one other character. The user switches
`between XT9 stroke input mode and Pinyin input mode by
`performing a sweeping motion in relation to the touch screen,
`such as, inter alia, sweeping a finger across the top of the
`
`IPR2019-00614 Page 00016
`
`IPR2019-00614 Page 00016
`
`

`

`US 2010/0017872 A1
`
`Jan. 21,2010
`
`touch screen. The series of at least two Chinese characters
`
`may be a text message, a name, a data entry, or any other such
`input.
`Further in accordance with an embodiment of the
`[0051]
`present invention, the user is able to compose a series of at
`least one Chinese character and at least one non-Chinese
`term, wherein the non-Chinese term includes at least one
`Latin character, digit, emoticon, punctuation mark, another
`non-Chinese symbol, or any combination thereof. The series
`is composed by switching input modes for each alphabet or
`Chinese input or digit input by sweeping across the top of the
`touch screen. For example, the user may input at least one
`Chinese character using either Pinyin or stroke input, or a
`combination thereof. The user may then perform a sweeping
`motion in relation to the touch screen to change the input
`mode to English. For example, the user may sweep a finger
`across the top of the touch screen to change the input mode to
`English. In this mode, the keypad presents Latin characters.
`The user then proceeds to input Latin characters using the
`Latin keypad displayed on the touch screen. Alternatively, the
`user may repeat a series of sweeping motions; e.g., the user
`sweeps a finger across the top ofthe touch screen, repeatedly,
`changing the input mode with each sweeping motion, until a
`digit keypad is displayed and digit input mode is active. The
`user may then proceed to enter at least one digit, adding the at
`least one digit to the series of Chinese characters already
`contained in the message. It will thus be appreciated that the
`user may switch between different input modes while com-
`posing a single message, a command, a name, a data entry or
`another such input, including at least two different types of
`characters, in an easy and simple and convenient manner.
`[0052]
`Further in accordance with an embodiment of the
`present invention, a keypad displaying emoticons is dis-
`played. In this mode, the user may select an emoticon to be
`entered into the text of a message, or such other input.
`[0053] Yet further in accordance with an embodiment ofthe
`present invention, drawings, including inter alia, emoticons,
`are constructed in a similar manner to XT9 stroke input. In
`this mode,
`the user interface displays the basic building
`blocks for the drawing, such as a curve, a semicolon, a circle,
`and other symbols. As the user taps multiple symbols, pos-
`sible drawings or emoticons that can be formed using the
`selected elements are displayed, and the user may either
`select the desired complete drawing or emoticon from the
`displayed list, or may continue entering additional building
`blocks until only one option remains. This mode of input is
`convenient as the number and size of the keys presented is
`optimized for the number of available building blocks, and
`each key only displays information relevant for the active
`input mode.
`invention provide
`[0054] Embodiments of the present
`methods and systems for enabling multiple input modes,
`whereby the screen display in each input mode is optimized
`for that mode. Optimizations include (i) configuring the num-
`ber of keys displayed, (ii) configuring the size, position and
`shape ofthe keys in relation to the screen, (iii) configuring the
`size, position and shape ofthe area ofthe display showing text
`already entered, (iv) configuring the size, position and shape
`of the area of display showing possible completions for the
`current character, phrase or symbol, and (v) displaying only at
`least one character, symbol, digit or other figure that is rel-
`evant to the active input mode on each key.
`[0055] Embodiments of the present invention also provide
`methods and systems for enabling multiple input modes and
`
`switching between the input modes by performing a sweep-
`ing motion in relation to the screen. These methods and sys-
`tems are easier and more convenient than using a menu inter-
`face to switch input modes. Additionally, these methods do
`not use up screen space to provide a switching key, to switch
`between input m