Apple Ex. 1013
`Apple Inc. v. Firstface Co., Ltd.
`IPR2019-01012
`Page 00001
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 1 of 12
`
`US 2010/0017872 Al
`
`103~Ot«2Y
`
`'
`
`400
`
`
`400
`
`4030 za
`
`405 424
`
`38t2t
`
`IPR2019-01012 Page 00002
`
`IPR2019-01012 Page 00002
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 2 of 12
`
`US 2010/0017872 Al
`
`200 -----
`
`200 -----
`
`FIG, 2
`
`IPR2019-01012 Page 00003
`
`IPR2019-01012 Page 00003
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 3 of 12
`
`US 2010/0017872 A1
`
`2” From: Karl ASp, Please call me
`FF eo: Summer, CALL ME!!
`% 55 From: Dick, We have a greate d
`
`|Kid shot down scool class in
`Lo we ot
`I cect
`itt ar ie
`Wictomeciuirge
`«
`dec?
`>
`12:00 Lunch Mika
`Steve Jobs
`14:00 Dev
`Rod Johnston
`
`;
`
`
`
`A gadgetis an application running on
`the phone. In their icon like state
`Gadgets are also running showing
`information like latest incoming
`message, current song played,
`currency, wheather etc. A gadget can
`be any type of application.
`
`-Streaming Radio
`-Note book
`-Web mail
`-Phone application
`-Clock —
`etc.
`
`A Gadget can be updated with
`information both from the phone
`(new SMS) or from a third party
`over the internet (RSS Feed)
`
`are
`ae OC ens mae
`
`A Gadgetin icon state at the
`home screen can only take a
`defined space
`
`In most cases a gadget can beexpandedeither displayed like an overlay
`directly on the home screen like the volume control or displayedasa full
`screen application like the camera.
`
`IPR2019-01012 Page 00004
`
`IPR2019-01012 Page 00004
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 4 of 12
`
`US 2010/0017872 Al
`
`Problem: It is hard to hit the right item on a touch display because when you movethe finger to and fram the
`surfaceitis also easy moveit in X-Y direction by mistake especialy when used with one hand .
`Solution: Use a conventional touchtechnology to find the object in X-Y position. Add tactile oror graphical ilustration
`were your finger is and use a force sensor to the display to read the activation.
`
`Fig. 6
`
`Find object by sweeping the screen.
`
`E.g. Links in a website
`
` Activate with force
`
`Find object by sweeping the screen.
`
`
`
`
`E.g. Links in a website
`
`Fig. 7
`
`Fig. 8
`
`IPR2019-01012 Page 00005
`
`IPR2019-01012 Page 00005
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 5 of 12
`
`US 2010/0017872 Al
`
`Power ON
`
`Fig. 9 |
`
`Press Home key
`
`Key lock
`Fig. 12°
`
`Power OFF
`
`
`
`Press off Gadget or
`
`hald Home key for 5 secondKeylock
`|
`high security
`
`Tap key lock to tock
`
`Tap home button
`to start
`
`Tap home button
`to start. Tap code
`one, two, three ar four digits.
`
`Home?return from application
`Fig. 16
`
`
`Fig.10°
`
`Fig, 13°
`
`
`Fig. 17
`
`
`WM
`
`08-543 464 23
`
`a(bareNadatebedoat=es
`
`E
`
`Tap home
`
`IPR2019-01012 Page 00006
`
`IPR2019-01012 Page 00006
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 6 of 12
`
`US 2010/0017872 Al
`
`Scroll alternatives
`
`Alt 4
`Scroll directly on screen.
`Then we miss possibility to
`touch and drag on other
`gadgets
`
`Fig. 18
` a
`
`Sign200SMS
`
`Alt 4
`Scrollbar put finger
`on the side and scroll
`downto get lower down
`on the page
`
`Alt 2
`Drag button
`Put finger on button and
`drag screen up and down
`
`Alt3
`Two drag buttons. Same
`function as Alt 2.
`Put finger on the lower scroll
`bution and drag screen up
`or on the upper icon and
`drag down.
`
`IPR2019-01012 Page 00007
`
`IPR2019-01012 Page 00007
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 7 of 12
`
`US 2010/0017872 Al
`
`Fig. 22
`
` Notes:
`
`Dinner , Milk,
`Visit Art museum.
`
`Optional feature: Put yourfinger on the scroll bar
`handle and drag up to display upper Gadgets that are
`less frequently used.The screen is automatically
`scrolled back 3 secondsafter the finger is removed
`
`NeixGmail| Traffic
`
`:00 Jobb
`
`09:0 Frukost
`
`
`12:07
`=
`
`Home SCreen Based onscroll Alt 1
`“| jam
`
`Message
`
`Home
`
`NEON
`
`
`+12%
`Volume
`
`
`
`
`
`
`were
`emus
` ‘Put your finger on the screen
`area j tsa Bina at
`Telephone
`| MediaPlayer
`and scroll bar handle and drag downto display display
`
`more Gadgets down the page.
`Use Homebutton to get back to the homearea.
`Video
` You€i
`
`Face|
`
`book? More Gadgets
`
`
`
`Buy moreGadgets
`= Geesesea Read about new Gadgets
`G ‘ontewed
`Updated by
`9
`tottotsk
`See video dips
`on the web get
`“
`mostseenfilm on the
`YouWleg|ceeen
`
`
`
`
`Web news from CNN
`Including en RSS feed
`
`tortor
`
`
`
`IPR2019-01012 Page 00008
`
`IPR2019-01012 Page 00008
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 8 of 12
`
`US 2010/0017872 Al
`
`Fav 5
`
`;
`
`> Dene
`
`sy
`peORSRASS
`
`07:00 200
`BGESEG
`12:07 o90Frace
`
` B
`
`Er BeSS
`Fig. 27
`
`
`Tap fav §
`
`Tap Call to make a call
`Tap SMS to SMS that person
`More for rearranging/edit...
`
`Tap history
`
`Telephone
`
`Tap telephone
`
`Active call
`
`Gis
`ce
`ernee
`a)
`
`Tapto call or SMS
`
`Optional: Included SMS history.
`
`IPR2019-01012 Page 00009
`
`IPR2019-01012 Page 00009
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 9 of 12
`
`US 2010/0017872 A1
`
`Volume
`
`Mp3 player
`
`
`
`
`
`
`User group: A34
`
`
`
`Press volume
`bulten
`
`At this point you can select to
`change anather sound source.
`Control disappears after 4
`seconds
`
`Tap player
`
`Controll disappears after 4
`seconds
`
`Operator add
`Fig. 35
`
`Market 100 Free SMS
`
`Operators screen
`
`4. Tap Add -> operator web page
`or java app.
`
`Alt 1 End user can moveit around. A click trough creates kickback.
`Alt 2 Fixed position. Reduced monthly fee.
`
`IPR2019-01012 Page 00010
`
`IPR2019-01012 Page 00010
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 10 of 12
`
`US 2010/0017872 Al
`
`seconds
`
`eee
`
`
`Tap the screen and use controls
`Controls disappearsafter 4
`
`Tap video
`
`Camera
`
`Use controls
`
`eer Bala
`
`Fr
`
`Tap Camera
`
`Send Picture to
`
`IPR2019-01012 Page 00011
`
`IPR2019-01012 Page 00011
`
`

`

`Patent Application Publication
`
`Jan. 21,2010 Sheet 11 of 12
`
`US 2010/0017872 Al
`
`|Emorrmesssoevs
`20:21 ‘Rod 07234012234
`Riceeesda
`bike OO78
`
`Messaging SMS
`Fig. 44
`
`|
`
`Tap Message gadget
`
`Select Message
`
`Could also be samestructure as call history
`See previous description.
`
`IPR2019-01012 Page 00012
`
`IPR2019-01012 Page 00012
`
`

`

`Patent Application Publication
`
`Jan. 21, 2010 Sheet 12 of 12
`
`US 2010/0017872 A1
`
`Home screen
`Fig. 46
`
`coMeaTiistoryg
`
` G1, Phone status
`G3, Web CNN”
`
`
`Pan homescreen
`Fig. 47
`
`Song oka
`
`Pan up and down.
`Put you finger on the screen
`and drag up or down
`
`Neonode 2008-04-20
`Magnus Goertz
`
`Problem: Only one browsersession can be opend at a time.
`How can wetell the user that in an intuitive way andstill stay
`in the same screen and keep important information at the top?
`
`Solution: Always open the browser to the bottom of the screen
`and keep content above the browser Gadget.
`
`
`
`
`Paning Web
`Fig. 54
`G1, Phonestatus
`
`Open gadget/ browser
`Fig. 50
`
`T=
` fa
`Keaneaaa) S77
`ferNereelsco
`
`Open gadget / browser
`Fig. 49
`
`CS2MealEston
`
`
`
`
`
`
`
`orme 9
`
`
`forBatelelia
`
`
`
`
`
`
`Maken A,
`i
`G5, 'G-mail
`
`I
`
`
`
`
`
`Paning Web
`Fig. 53
`
`
`
`ro
`acticin2]
`foeeee
`
` CNN Headlines
`
`|
`
`When Web-Gadgetis activated -all
`gadgets beneath Web-Gadgetslides
`down and disappears. Instead the
`browsertakes this space
`
`Paning Gadget
`
`Fig. 52
`
`
`ACCACcILisicyam
`
`
`G3,WebCNN
`|
`Petescosy
`
`
`
`
`
`Open gadget/ browser
`Fig. 48
`
`
`
`
`Push/Tap and release a
`Gadgetto open/activate
`
`Paning Gadget
`
`' G3, Web CNN
`Porters
`
`Boosenslonmam
`ae
`lexEsLy
`i
`
` cae
`
`i
`
`
`
`Paning abow bowser window will resuft
`in changing size of browser window.
`
`Paning inside bowser window will result
`in paning browser content.
`
`IPR2019-01012 Page 00013
`
`IPR2019-01012 Page 00013
`
`

`

`US 2010/0017872 Al
`
`Jan. 21, 2010
`
`USER INTERFACE FOR MOBILE
`COMPUTER UNIT
`
`CROSS REFERENCES TO RELATED
`APPLICATIONS
`
`[0001] This application is a continuation-in-part ofpending
`application U.S. Ser. No. 10/315,250,filed on Dec. 10, 2002,
`entitled USER INTERFACE FOR MOBILE HANDHELD
`COMPUTER UNIT. This application claims priority from
`provisional application No. 61/132,469, filed on Jun. 19,
`2008, entitled IMPROVED KEYPAD FOR CHINESE
`CHARACTERS.
`
`FIELD OF THE INVENTION
`
`Thefield of the present invention is user interfaces
`[0002]
`for electronic devices and, more particularly, to touch screen
`user interfaces.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Touch screens provide user interfaces through
`which a user enters input to a computing device by touching
`a screen at a selected location, with a stylus or with his finger.
`[0004] Conventional touch screens are limited as to the
`types of user inputs that they can recognize. For example,
`conventional touch screens are unable to distinguish between
`a soft tap and a hard press. In some prior art embodiments
`users initially select an item on the screen, and then subse-
`quently activate the selected item. However, becausepriorart
`touch screensdo notdistinguish between degreesofpressure,
`the user is required to remove his finger or stylus from the
`screen and activate his selection witha secondtap. It would be
`advantageous to produce touch screens that distinguish
`between varying amounts ofpressure applied to the screen so
`that a user can select an item by touching its location on the
`screen, and then activate the item by applying additional
`pressure to the touch location without havingto first remove
`his finger or stylus from the screen.
`[0005]
`In conventional touch screens the keys are often
`small relative to the touch area, especially in handheld
`devices. The keysare also often situated close together. This
`can makeit difficult to determine which key is being pressed
`by the user. It would be advantageousto clearly indicate to the
`user which key the user has selected, and furthermore, to
`allow the user to accept or reject the selected key withoutfirst
`removing his finger or stylus from the screen in order to
`perform a secondtap.
`[0006] Current user interfaces are basic and often require
`navigation through a series of menusin order to perform a
`desired operation. The present invention presents a user inter-
`fere that is versatile in providing the user with manyoptions,
`while requiring only few selections to activate a desired func-
`tion. To further enhanceuser experience,certain functions are
`performed automatically without requiring the userto enter a
`selection.
`
`SUMMARY OF THE DESCRIPTION
`
`[0007] Aspects of the present inventionrelate to user inter-
`faces designed for use with a touch screen. The present inven-
`tion relates to computer readable media storing computer
`programs with computer program code, which, when read by
`a computer unit, allows the computer to present a user inter-
`face for the computer unit.
`
`In accordance with embodiments of the present
`[0008]
`invention, the computer unit features a touch sensitive display
`area. According to preferred embodiments an initial display
`configuration presents a plurality of gadgets on the display.
`These gadgets are small areas of the screen that indicate
`which functions each gadget will perform whenactivated by
`the user. When a gadgetis activated, typically by the user
`touching the area of the display on which the gadgetis dis-
`played,the gadget increasesin size and provides the user with
`icons and information about the gadget’s functions. Signifi-
`cantly, the gadget does not coverthe entire display area. Thus,
`when a gadget, or even a plurality of gadgets, is active, the
`primary display of all available gadgets is still accessible.
`This primary display can be compared to a desktop in com-
`puter operating system user interfaces. However, this primary
`display in the user interface ofthe present invention is not the
`same as a desktop where active windowscan cover icons on
`the desktop. In the present invention, gadgets are arranged in
`a manner that open gadgets do not cover other gadgets.
`Rather, when an open gadget expands in size, other gadgets
`are shifted to make room for the expended gadget. This allows
`the userto scroll the primary display or desktop to view any
`gadget. In the context of the present invention, this primary
`display area that includes both open and closed gadgets is
`called the home window.The userscrolls the home window to
`view gadgets that are shifted beyond the viewable area of the
`display. An expanded,or activated, gadget has an expanded
`window,but often it is not large enough to display everything
`contained in that gadget window. To view contents of the
`gadget not displayed in the visible portion of the gadget
`window,the user scrolls the gadget window. Thus, two dif-
`ferent scroll operations are provided: scrolling the home win-
`dow and scrolling a gadget window. According to one
`embodiment, scrolling is executed by gliding a finger or sty-
`lus along the touch screento shift the active display area ofthe
`home windoworofthe gadget. The scrolling affects the home
`windowif the finger glide began in an area of the screen that
`does not belong to an active gadget;the scroll affects an active
`gadget window if the finger glide began inside that active
`gadget window.
`[0009] Various embodiments of the invention support sev-
`eral methods of scrolling a window. According to one
`embodiment, scrolling is done when the user touches the
`display inside the window area, for example at an edge of the
`window,or on an icon, suchis a arrow orscrollbar, indicating
`a scroll operation. According to another embodiment, scroll-
`ing is done by the user touching the window with a finger or
`stylus and then gliding the finger or stylus along the torch
`sensitive screen in a direction indicating the desired direction
`ofthe scroll. When the content ofthe homedisplay is larger in
`two dimensionsthan the actual display screen, this operation
`is like panning an image or map. Whenitis larger along only
`one axis (e.g., only vertically), the scrolling only scrolls in the
`one axis even whenthe glide is not orthogonalalong the one
`axis.
`
`[0010] Another aspect of the present invention relates to
`computer readable media storing a computer program with
`computer program code, which, when read by a mobile hand-
`held computer unit, allows the computer to present a user
`interface for the mobile handheld computer unit. The user
`interface features a touch sensitive area in which representa-
`tions of a plurality of keys are displayed, and each key is
`mapped to a corresponding location in the touch sensitive
`area at which the representation ofthe key is displayed. A key
`
`IPR2019-01012 Page 00014
`
`IPR2019-01012 Page 00014
`
`

`

`US 2010/0017872 Al
`
`Jan. 21, 2010
`
`in this context includes, interalia, alphabetic keys such as in
`a QWERTYkeypad, numeric keys and also icons represent-
`ing programsor functions. A key is selected, but not activated,
`when an object touches the corresponding location. This
`intermediate status of being selected but not activated facili-
`tates the user to subsequently activate a desired key and avoid
`activating a neighboring key that the user selected but does
`not wish to activate. A selected key is activated when the
`object touchingit applies additional pressure to the key loca-
`tion.
`
`[0011] According to preferred embodiments of the inven-
`tion, when a key is selected, the user interface generates a
`secondary representation of the key, such as a callout balloon
`containing the key representation. The callout balloon is
`placed away from the key location (being touched) so that the
`user can easily view which keyis selected withoutlifting his
`finger. According to another embodiment, an audio represen-
`tation of the selected key is generated so the user hears which
`key wasselected.
`[0012] According to still further features in preferred
`embodiments of the invention, the user touches the screen
`(with a fingeror stylus) ata first location, for example select-
`ing a first key. The user then glides his finger or stylus over the
`screen to additional locations. At each additional location a
`
`new key is selected and the previously selected key is dese-
`lected. The user can activate any selected key by applying
`additional pressure to the screen. The user does not have to
`remove the object from the screen to glide and select addi-
`tional keys even after activatinga first key.
`[0013] Additional touch pressure is detected in various
`ways according to several embodiments. According to one
`embodiment, the touch sensitive area is a light-based touch
`screen operable to detect different levels of touch pressure.
`For example, light-based touch screens typically include a
`calculating unit operable to identify the size, shape and con-
`tours of an area being touchedbasedora pattern ofobstructed
`light. See applicant’s co-pending U.S. patent application Ser.
`No. 10/494,055, titled ON A SUBSTRATE FORMED OR
`RESTING DISPLAY ARRANGEMENT,the contents of
`whichare incorporated herein by reference. Whena finger or
`flexible object is used as a touch object, as additional pressure
`is appliedto the touch surface, the contact area ofthe finger or
`object touching the screen increases. Thus, additional pres-
`sure is detected when an increase in the contours of the
`covered touch area is detected.
`
`[0014] Alternatively, or in combination with the above, the
`touch sensitive area features both a touch screen operable to
`identify a touch location on the screen and a pressure sensor
`operable to detect pressure applied to the screen but not
`sensitive to the location of the object applying the pressure.
`[0015] Other aspects of the present inventionrelate to con-
`venient arrangementand function oficonsto perform popular
`functions within a user interface. Thus, a camera gadgetfea-
`tures a multimedia messaging service (MMS) button facili-
`tating sending an active photo inan MMS message; a keylock
`gadget locks the computer and displays an instrumental key-
`pad for entering a musical code to unlock the computer; a
`reporting gadget displays information for a first period of
`time andis then automatically deactivated. Several reporting
`gadgets are provided, including a gadget that displays the
`time of day; a gadget displays a weather forecast; a gadget
`that displays stock market information.
`[0016] According to still further features in preferred
`embodiments the reporting gadget continues to display its
`
`information for a second period of time if the gadget is
`touched during the first period of time.I.e., automatic deac-
`tivation after the first period of time is canceled.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0017] The present invention will be more fully understood
`and appreciated from the following detailed description,
`taken in conjunction with the drawings in which:
`[0018]
`FIG. 1 is a series of screens illustrating Chinese
`stroke input,
`in accordance with an embodiment of the
`present invention;
`[0019]
`FIG. 2 is a series of screens illustrating Chinese
`Pinyin input,
`in accordance with an embodiment of the
`present invention;
`[0020]
`FIGS. 3 is a screen of a handheld terminal display-
`ing a plurality of gadgets, in accordance with an embodiment
`of the present invention;
`[0021]
`FIGS. 4-6 are diagramsillustrating selectionof ele-
`ments displayed on a touch screen. A selected elementis
`displayed in a callout as shown in FIGS. 5-6. The user does
`notlift his finger off the screen between selection ofdifferent
`elements, as in FIGS. 5 and 6. The use simply glideshis finger
`along the screen. This is shown in FIG. 7. To activate a
`selected element the user presses on the screen without first
`lifting his finger off the screen as shown in FIG.8 andin the
`series of FIGS. 7-8, all in accordance with an embodimentof
`the present invention;
`[0022]
`FIGS. 7-8 illustrate a side view of a user selecting
`and activating an on-screen element, in accordance with an
`embodimentof the present invention;
`[0023]
`FIGS. 9-11 illustrate turning a phone on andoff.
`FIG. 9 illustrates pressing the home key on a turned-off
`phone. FIG. 10 illustrates the phone whenitis turned on. FIG.
`11 illustrates pressing the homekey to turn a phoneoff.
`[0024]
`FIGS. 12-15 illustrate locking and unlocking a
`phone. FIG.12 illustratesactivating a lock gadget onscreen to
`lock the phone. FIG. 13 illustrates pressing the home button
`ona locked phoneto unlock the phone. FIG. 14 illustrates the
`unlocked phone. FIG. 15 showspressing the home key ona
`locked phoneto display a keypadfor entering a security code
`to unlock the phone.
`[0025]
`FIGS. 16-17 illustrate pressing the home key to
`return to the home display screen. FIG. 16 shows an open
`active calling gadget. Pressing the home button presents the
`home screen shownin FIG. 17.
`
`FIGS. 18-21 illustrate scrolling options. In the
`[0026]
`phone shownin FIG.18 scrolling is done by gliding a finger
`or stylus along the touch sensitive screen. FIG. 19 illustrates
`a scrollbar along one edge of the screen for scrolling up and
`down. FIGS. 20-21 illustrate arrow buttons for scrolling up
`and down.
`
`FIG. 22 illustrates scrolling the home display
`[0027]
`screen, in accordance with an embodiment of the present
`invention;
`[0028]
`FIGS. 23-24 illustrate a Fav5 gadget, in accordance
`with an embodimentof the present invention;
`[0029]
`FIGS. 25-26 illustrate a History gadget, in accor-
`dance with an embodimentofthe present invention;
`[0030]
`FIGS. 27-28 illustrate a Calling gadget, in accor-
`dance with an embodimentofthe present invention;
`[0031]
`FIGS. 29-30 illustrate placing a call using a Calling
`gadget, in accordance with an embodiment of the present
`
`IPR2019-01012 Page 00015
`
`IPR2019-01012 Page 00015
`
`

`

`US 2010/0017872 Al
`
`Jan. 21, 2010
`
`invention; FIG. 30 illustrates the shortcut buttons presented
`when an incoming call is received (i.e., when the phoneis
`ringing).
`FIGS. 31-32 illustrate volume control, in accor-
`[0032]
`dance with an embodimentof the present invention;
`[0033]
`FIGS. 33-34 illustrate an MP3 player gadget, in
`accordance with an embodimentof the present invention;
`[0034]
`FIGS. 35-37 illustrate an operator promotion gad-
`get, in accordance with an embodimentofthe present inven-
`tion;
`FIGS. 38-39 illustrate a video player gadget, in
`[0035]
`accordance with an embodimentof the present invention;
`[0036]
`FIGS. 40-42 illustrate a camera gadget and shortcut
`buttons provided when the camera gadget is activated, in
`accordance with an embodimentof the present invention;
`[0037]
`FIGS. 43-45 illustrate an SMS gadget and shortcut
`buttons provided when the SMSgadgetis activated, in accor-
`dance with an embodimentofthe present invention; FIG. 45
`illustrates how related SMS messages are displayed as a
`conversation in an active SMSgadget.
`[0038]
`FIGS. 46-54 illustrate scrolling the home display
`screen and gadget windows, in accordance with an embodi-
`mentof the present invention;
`
`DETAILED DESCRIPTION
`
`[0039] Touch screen keypadsoffer great flexibility in key-
`pad interface design—flexibility that cannot be achieved with
`electromechanical keypads. Custom design keypads can be
`generated on a touch screen, where the markings on each key
`and the functions that each key provides are optimized for a
`designated application. Moreover, touch screen keypads can
`change modes, from one pad ofkeys and associated functions
`to a different pad of keys and associated functions. Custom
`keypads are of particular advantage for multi-lingual appli-
`cations.
`
`[0040] A general description of touch screen keypadinter-
`faces, in accordance with embodimentsof the present inven-
`tion, and several examplesthereof, are provided in the Appen-
`dix and described in detail herein below.
`
`invention relate to
`[0041] Embodiments of the present
`improved keypads for inputting Chinese characters using
`XT9stroke input, and using Chinese Pinyin. XT9 stroke input
`builds Chinese characters using six basic strokes, and offers
`selection of possible characters and phrases based ona set of
`strokes that have been input. Chinese Pinyin uses Latin char-
`acters that transliterate a sound or a syllable, in combination
`with a digit that represents an intonation or inflection. E.g.,
`Main a rising tone is m-a-1, and Main a descending tone is
`m-a-2.
`
`[0042] Reference is now madeto FIG.1, whichis an illus-
`tration of an improved stroke-based keypad for entering Chi-
`nese characters by inputting strokes, in accordance with an
`embodimentof the present invention. Shown in FIG. 1 is a
`keypad 100, used for inputting basic strokes. Also shown in
`FIG.1 is a sequence of strokes 101-106 that are successively
`input by a user, and a corresponding sequence of Chinese
`characters 111-114 that are entered in a message being com-
`posed.
`In accordance with an embodimentofthe present
`[0043]
`invention, keypad 100 is generated and displayed on a touch
`screen. Keypad 100 has fewer than the standard 12 keys ina
`touch pad, allowing more room on screen for displaying
`characters.
`
`Further in accordance with an embodiment of the
`[0044]
`present invention, the keys of keypad 100 are customized so
`that they contain only relevant information. For example, a
`prior art keypad displays a digit, 3-4 characters, and a basic
`Chinesestroke, all inside one key, even though in XT9 stroke
`mode the basic Chinese stroke is the only useful one. The
`custom keys of the present invention display only the basic
`Chinese strokes, or the strokes and numbers, but no charac-
`ters.
`
`[0045] There are two types of key presses supported by
`keypad 100—+egular and long. A regular key press adds the
`stroke shown on the key to the series of strokes 101-106
`already pressed. As strokes are successively entered, a num-
`bered array 121 of Chinese characters or phrases is dynami-
`cally displayed along the top of the keypad. These characters
`or phrases are predicted based on the key presses already
`entered. In order to select one of the numbered elements of
`
`array 121, the user performsa long key press on that number.
`Alternatively, the user may keep entering strokes until only
`one option remains.
`[0046] Often, however, more predicted characters or
`phrases exist than can be displayed along the top of the
`keypad. The N2 phone, manufactured by Neonodeof Stock-
`holm, Sweden, has a joystick button at the bottom of the
`phone. Twitching the joystick up/downdisplaysdifferent sets
`of predicted characters or phrases. When the intended char-
`acteror phrase is displayed and associated with a given digit,
`a long press on that digit serves to select the intended char-
`acter or phrase.
`[0047] Reference is now madeto FIG. 2, whichis an illus-
`tration of an improved keypad for inputting Chinese charac-
`ters using Pinyin, in accordance with an embodimentof the
`present invention. Shownin FIG.2 is a keypad 200, used for
`inputting Latin characters. Also shownin FIG.2 is a sequence
`of Latin characters M-o-H-S-a-n that are successively input
`by a user, and a corresponding sequence of Chinese charac-
`ters that are entered in a message being composed.
`[0048]
`In accordance with an embodimentofthe present
`invention, keypad 200 uses accent characters, rather than
`digits, to convey an intended intonation or inflection. Further
`in accordance with an embodimentof the present invention,
`keypad 200 displays only information relevant for Pinyin
`input on each key; no Chinese basic strokes are shown
`[0049] There are two types of Pinyin input. A user enters a
`Latin transliteration of an intended word using the Latin
`keypad input (12 keys). For each key, severalletters are pos-
`sible. Thelist ofpredicted Latin syllables based on the current
`sequence of keypad presses is displayed. Twitching the joy-
`stick right or left selects the desired combination. Also, a
`series of predicted Chinese characters or phrasesis displayed
`and selected by a long press on a respective digit. Twitching
`the joystick up/down displays other predicted Chinese char-
`acters or phrases. Entering a space after a series of letters
`indicates the end of a previous character or phrase.
`[0050]
`In accordance with an embodimentofthe present
`invention, the user is able to combinestroke and Pinyin input,
`and compose a sequence of at least two Chinese characters
`using XT9 stroke input for at least one character and Pinyin
`input for at least one other character. The user switches
`between XT9 stroke input mode and Pinyin input mode by
`performing a sweeping motionin relation to the touch screen,
`such as, inter alia, sweeping a finger across the top of the
`
`IPR2019-01012 Page 00016
`
`IPR2019-01012 Page 00016
`
`

`

`US 2010/0017872 Al
`
`Jan. 21, 2010
`
`touch screen. The series of at least two Chinese characters
`may be a text message, aname,a data entry, or any other such
`input.
`Further in accordance with an embodiment of the
`[0051]
`present invention, the user is able to composea series of at
`least one Chinese character and at least one non-Chinese
`term, wherein the non-Chinese term includes at least one
`Latin character, digit, emoticon, punctuation mark, another
`non-Chinese symbol, or any combination thereof. The series
`is composed by switching input modes for each alphabet or
`Chineseinput ordigit input by sweeping across the top of the
`touch screen. For example, the user may input at least one
`Chinese character using either Pinyin or stroke input, or a
`combination thereof. The user may then perform a sweeping
`motion in relation to the touch screen to change the input
`mode to English. For example, the user may sweep a finger
`across the top of the touch screen to change the input mode to
`English. In this mode, the keypad presents Latin characters.
`The user then proceeds to input Latin characters using the
`Latin keypad displayed on the touch screen. Alternatively, the
`user may repeat a series of sweeping motions; e.g., the user
`sweeps a finger across the top ofthe touch screen, repeatedly,
`changing the input mode with each sweeping motion, until a
`digit keypad is displayed and digit input modeis active. The
`user may then proceedto enterat least one digit, adding the at
`least one digit to the series of Chinese characters already
`contained in the message. It will thus be appreciated that the
`user may switch between different input modes while com-
`posing a single message, a command, a name, a data entry or
`another such input, including at least two different types of
`characters, in an easy and simple and convenient manner.
`[0052]
`Further in accordance with an embodimentof the
`present invention, a keypad displaying emoticons is dis-
`played. In this mode, the user may select an emoticon to be
`entered into the text of a message, or such other input.
`[0053] Yet further in accordance with an embodimentofthe
`present invention, drawings, includinginter alia, emoticons,
`are constructed in a similar manner to XT9 stroke input. In
`this mode,
`the user interface displays the basic building
`blocksfor the drawing, such as a curve, a semicolon, a circle,
`and other symbols. As the user taps multiple symbols, pos-
`sible drawings or emoticons that can be formed using the
`selected elements are displayed, and the user may either
`select the desired complete drawing or emoticon from the
`displayed list, or may continue entering additional building
`blocks until only one option remains. This modeof inputis
`convenient as the numberand size of the keys presented is
`optimized for the numberof available building blocks, and
`each key only displays information relevant for the active
`input mode.
`invention provide
`[0054] Embodiments of the present
`methods and systems for enabling multiple input modes,
`whereby the screen display in each input modeis optimized
`for that mode. Optimizations include(i) configuring the num-
`ber of keys displayed, (i1) configuring the size, position and
`shape ofthe keys in relation to the screen, (iti) configuring the
`size, position and shapeofthe area ofthe display showing text
`already entered, (iv) configuring the size, position and shape
`of the area of display showing possible completions for the
`current character, phrase or symbol, and (v) displaying only at
`least one character, symbol, digit or other figure that is rel-
`evant to the active input mode on each key.
`[0055] Embodiments ofthe present invention also provide
`methods and systems for enabling multiple input modes and
`
`switching between the input modes by performing a sweep-
`ing motion in relation to the screen. These methods and sys-
`temsare easier and more convenient than using a menu inter-
`face to switch input modes. Additionally, these methods do
`not use up screen space to provide a switching key, to switch
`between input modes, and, as such, screen space may be used
`for information related to the current input mode and the
`current text. These methods and systems also enable switch-
`ing between many input modes, by repeating the sweeping
`motions to advance through the input modes. In the N2 phone
`manufactured by Neonode AB, this mode of switching active
`keypads is used with standard 12-key keypads, for example,
`inter alia, switching between numeric, Swedish and English
`keypads. However, switching the layout of the keys is not
`used in the Neonode N2 phone. Moreover, other, prior art
`methods of switching between input modes require more than
`one action to advance through a list of input modes.
`[0056] Reference is now