US007463245B2
`
`(12) United States Patent
`Marcus et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,463,245 B2
`*Dec. 9, 2008
`
`(54) HUMAN INTERFACE SYSTEM
`
`FOREIGN PATENT DOCUMENTS
`
`(75) Inventors: Beth Marcus, Bedford, MA (US); W.
`David Lee, Newton, MA (US)
`
`(73) Assignee: Zeemote, Inc., Chelmsford, MA (US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis-
`c1aimer_
`
`(21) Appl. No.: 11/747 863
`’
`M 11 2007
`ay ’
`prior publication Data
`
`22 F1 d:
`(
`)
`1e
`(65)
`
`Related US. Application Data
`
`(63) Continuation of application NO- 10/ 669,555, ?led 011
`OCR 31, 2003,110W Pat N0~ 7,218,313
`
`(2006 01)
`/00
`(51)
`345/169 3 4 5 /1 68
`'
`(52) U 5 Cl
`(58) Field of 615515831511 éléilif... ............ .. 3345/156
`345/168’ 169, 173; 400/472; 341/22
`See application ?le for Complete Search history
`
`(56)
`
`References Cited
`U_S_ PATENT DOCUMENTS
`
`4,793,312 A 12/1988 Doinaga et al.
`4,867,028 A
`9/1989 Jones
`4,891,777 A
`1/1990 Lapeyre
`4,896,554 A
`1/ 1990 Culver
`4,912,462 A
`3;1990 WaShiZuka et a1~
`5,189,416 A
`2 1993 Estes
`5,365,589 A 11/1994 Gutowitz
`
`EP
`
`0251477
`
`1/ 1988
`
`(Continued)
`OTHER PUBLICATIONS
`
`Donner, J. (2005). Research Approaches to Mobile Use in Develop
`ing World: A Review of the Literature. International Conference on
`Mobile Communications and Asian Modernities City University of
`Hong Kong, Jun. 7-8, 2005.
`
`(Continued)
`
`Primary ExamineriRicardoLOsorio
`(74) Attorney, Agent, or FirmiFish & Richardson P.C.
`
`A human interface con?gured to optimiZe a biomechanical
`effect of a human user’s opposing thumb and ?ngers by
`including, on one surface, one or more software con?gurable
`input elements manipulatable by a user’s thumb(s) or a stylus,
`and, on another surface, one or more software con?gurable
`“16cm elements manipulatable by a user’s ?nger<s>~ A
`selection element may be a pressure sensor pad con?gurable
`to represent delineated active areas that are mapped to one or
`more input functions. Shape changing media may be pro
`vided to permit a user to tactilely discriminate between delin
`eated active areas. Tactile feedback may be provided to a user
`through palpable detents, vibratory or force producing units.
`-
`-
`.
`.
`lnputting data may 1nclude mapping each select1on element
`to a shlft funct1on, mapping each 1nput element to text func
`tions, and using the selection elements to shift between text
`functions associated with an input element to input a desired
`-
`text funcnon'
`
`(Continued)
`
`20 Claims, 9 Drawing Sheets
`
`SCEA Ex. 1001 Page 1
`
`

`

`US 7,463,245 B2
`Page 2
`
`US. PATENT DOCUMENTS
`
`7/1995 Matthews
`5,432,510 A
`5,473,325 A 12/1995 McAlindon
`5,512,919 A
`4/1996 Araki
`5,515,305 A
`5/1996 Register et al.
`5,612,690 A
`3/1997 Levy
`5,782,642 A
`7/1998 Goren
`5,824,931 A 10/1998 Papadopoulos
`5,859,629 A
`1/1999 TognaZZini
`5,900,864 A
`5/1999 Macdonald
`5,973,621 A 10/1999 Levy
`6,005,496 A 12/1999 Hargreaves et al.
`6,084,576 A
`7/2000 Leu et al.
`6,107,988 A
`8/2000 Phillipps
`6,115,028 A
`9/2000 Balakrishnan et al.
`6,184,804 B1
`2/2001 Harrison
`6,219,731 B1
`4/2001 GutowitZ
`6,228,709 B1
`5/2001 Hsieh
`6,232,956 B1
`5/2001 Mailman
`6,297,752 B1
`10/2001 Ni
`6,377,685 B1
`4/2002 Krishnan
`RE37,723 E
`6/2002 Goren
`6,512,511 B2
`1/2003 Willner et al.
`6,520,699 B2
`2/2003 Abe
`6,541,715 B2
`4/2003 Swanson
`6,542,091 B1
`4/2003 Rasanen
`6,546,239 B1
`4/2003 PaZdersky et al.
`6,573,844 B1
`6/2003 Venolia et al.
`6,606,486 B1
`8/2003 Cubbage et al.
`6,654,733 B1
`11/2003 Goodman et al.
`6,703,963 B2
`3/2004 Higginson
`6,738,045 B2
`5/2004 Hinkley et al.
`6,741,235 B1
`5/2004 Goren
`6,760,013 B2
`7/2004 Willner et al.
`6,865,718 B2
`3/2005 Montalcini
`6,885,317 B1
`4/2005 GutowitZ
`6,885,318 B2
`4/2005 Bickerton
`6,909,424 B2
`6/2005 Liebenow et al.
`6,911,608 B2
`6/2005 Levy
`6,947,028 B2
`9/2005 Shkolnikov
`6,980,200 B2 12/2005 Goren
`7,072,975 B2
`7/2006 Kato
`7,092,734 B2
`8/2006 Herle et a1.
`2002/0019259 A1* 2/2002 Armstrong ................. .. 463/36
`2/ 2002 Mercalf
`2002/0023265 A1
`11/2002 Pallakoff
`2002/0163504 A1
`2003/0020692 A1
`1/2003 Grif?n et al.
`2003/0048205 A1
`3/2003 He
`2003/0061103 A1
`3/2003 Kanai
`2003/0169188 A1
`9/2003 Chang et a1.
`2003/0193418 A1 10/2003 Shi
`10/ 2004 Dechene
`2004/0208681 A1
`2005/0093846 A1
`5/2005 Marcus et al.
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`EP
`JP
`KR
`WO
`WO
`
`0585730
`1103883
`1293882
`2000-267787 A
`1999-0072889
`91/05303
`03/042805
`
`3/1994
`5/2001
`3/2003
`9/2000
`9/1999
`4/1991
`5/2003
`
`OTHER PUBLICATIONS
`
`Min, Lin and Sears, Andrew (2005). Graphics Matter: A Case Study
`of Mobile Phone Keypad Design for Chinese Input. CHI 2005, Late
`Breaking Results: Posters, Portland, Oregon. Apr. 2-7, 2005.
`MacKenZie, S., & Soukoreff, W. (2002). Text entry for mobile com
`puting: Models and methods, theory and practice. Human-Computer
`Interaction. 17. p. 147-198. http://www.yorku.ca/macldhci3-2002.
`pdf.
`
`Starner, T. “Keyboards Redux: Fast Mobile Text Entry”. Pervasive
`Computing, Jul-Sep. 2004, pp. 97-101. http://www.cc.gatech.edu/
`fac/Thad.Starner/p/magaZine/2004-3 -keyboard-redux.pdf.
`Lee Butts , Andy Cockburn, An evaluation of mobile phone text input
`methods, Third Australasian conference on User interfaces, p. 55-59,
`Jan. 1, 2002, Melbourne, Victoria, Australia http://www.crpit.com/
`confpapers/CRPITV7Butts.pdf.
`Chording and TiltingiDaniel Wigdor (thesis)i2004- describes
`chordtap and tilttap (also covered in depth in the paper referenced
`below) http ://www. dgp.toronto .edu/~dwigdor/research/thesi s/sub
`mitted.html.
`Wigdor, D. and Balakrishnan, R. “A Comparison of Consecutive and
`Concurrent Input Text Entry Techniques for Mobile Phones”, Con
`ference on Human Factors, Apr. 24-29, 2004, vol. 6, No. 1, pp. 81-88
`http://portal.acm.org/citation.cfm?id:985703
`http://www.dgp.
`toronto .edu/~ravin/papers/chi2004ic oncurrentextinput .pdf.
`OnisZcZak, A., & MacKenZie, I. S. (2004). A comparison of two input
`methods for keypads on mobile devices. Proceedings of NordiCHI
`2004, pp. 101-104. New York: ACM. http://www.yorku.ca/macld
`nordichi2004.pdf.
`K. Lyons. Everyday wearable computer use: A case study of an expert
`user. In Proceedings of Mobile HCI 2003, pp. 61-75, 2003. http://
`www.cc.gatech.edu/ccg/publications/everydayicase.pdf.
`K. Lyons, T. Starner, D. Plaisted, J. Fusia, A. Lyons, A. Drew, E. W.
`Looney, 2004. “Twiddler Typing: One-Handed Chording Text Entry
`for Mobile Phones,” Proc. Conf. Human Factors in Computing Sys
`tems (SIGCHI 01), ACM Press, 2004, pp. 671-678. http://www.cc.
`gatech .edu/ fac/ Thad. Starner/ p/ 03 Oil 0iMTE/twiddler-chi04 .pdf.
`Daniel Wigdor, Ravin Balakrishnan, TiltText: using tilt for text input
`to mobile phones, Proceedings of the 16th annual ACM symposium
`on User interface software and technology, p. 81-90, Nov. 2-5, 2003,
`Vancouver, Canada http://portal.acm.org/citation.cfm?id:964705
`http://www.dgp.toronto.edu/~ravin/papers/uist2003itilttext.pdf.
`M. D. Dunlop and A. Crossan, “Dictionary based text entry method
`for mobile phones”, published in Brewster, SA, and Dunlop, M.D.,
`(editors). Proceedings of Second Workshop on Human Computer
`Interaction Mobile Devices, Aug. 1999. http://www.cis.strath.ac.uld
`~mdd/research/publications/99dunlopcrossan.pdf.
`Andriy Pavlovych, Wolfgang SturZlinger: Less-Tap: A Fast and
`Easy-to-learn Text Input Technique for Phones. Graphics Interface
`2003,
`97-104
`http://www.graphicsinterface.org/cgi-bin/
`DownloadPaper?name:2003/170/paper170.pdf.
`Scott MacKenZie , Hedy Kober , Derek Smith , Terry Jones , Eugene
`Skepner, LetterWise: pre?x-based disambiguation for mobile text
`input, Proceedings of the 14th annual ACM symposium on User
`interface software and technology, Nov. 11-14, 2001, Orlando,
`Florida.
`H. Kober, E. Skepner, T. Jones, H. GutowitZ, S. MacKenZie, 2001.
`Linguistically Optimized Text Entry on a Cell Phone. In Proceedings
`of the CHI 2001. http://www.eatoni.com/research/chipdf.
`Mikael Goldstein and Didier Chincholle The Finger-Joint-Gesture
`Wearable Keypad. Ericsson Radio Systems.
`Rosenberg, R. (1998). Computing without Mice and Keyboards: Text
`and Graphis Input Devices for Mobile Computing. Ph.D. Thesis,
`Dept. of Computer Science, University College, London, 1998.
`http://www.obscure.org/ro senberg/.
`MacKay, B., Dearman, D., Inkpen, K., andWatters, C. 2005. Walk ’n
`scroll: a comparison of software-based navigation techniques for
`different levels of mobility. In Proceedings of the 7th international
`Conference on Human Computer interaction with Mobile Devices
`&Amp; Services (SalZburg, Austria, Sep. 19-22, 2005). MobileHCI
`’05, vol. 111.ACM Press, NewYork, NY, 183-190. http://portal.acm.
`org/citation.cfm?id:1085808&coll:GUIDE&dl:GUIDE
`&CFID:66591340&CFTOKEN:6294934.
`KranZ, M., Holleis. P., Schmidt, A. “DistScrollia new one-handed
`interaction device”. In Proceedings of the 5th International Workshop
`on Smart Appliances and Wearable Computing, Jun. 10, 2005. http://
`www.hcilab.org/documents/DistScrol
`lAnewOneHandedInteractionDevice-KranZHolleisSchmidt
`IWSAWC2005.pdf.
`Daniel Fallmana, Andreas Lund, Mikael Wiberg, ScrollPad: Tangible
`Scrolling with Mobile Devices, Proceedings of the Proceedings of
`the 37th Annual Hawaii International Conference on System Sci
`
`SCEA Ex. 1001 Page 2
`
`

`

`US 7,463,245 B2
`Page 3
`
`ences (HICSS’04)iTrack 9, p. 90286.3, Jan. 5-8, 2004. http://portal.
`acm.org/citation.cfm?id:963347&coll:GUIDE8dl:GUIDE
`&CFID:66483658&CFTOKENI36023921 http://danielfallman.
`org/resources/papers/fallman-hicss37.pdf.
`Chipman, L. E., Bederson, B. B., and Golbeck, J. A. 2004. SlideBar:
`analysis ofa linear input device. Behav. Inf. Tech. 23, 1 (Jan. 2004),
`1-9. http://portal.acm.org/citation.cfm?id:993182.993184# http://
`www.cs.umd.edu/Library/TRs/CS-TR-4471/CS-TR-4471.pdf.
`Orientation-based interaction for Mobile Devices. J. Darnauer, S.
`Garrity and T. Jim, Stanford University, pp. 1-4, found on the internet
`at http://hci .stanford.edu/srk/c s3 77a-mobile/proj ect/?nal/darnauer
`garrity-kim.pdf.
`Rekimoto, J. Tilting operations for small screen interfaces. Proceed
`ings of the 9th annual ACM symposium on User Interface software
`and technology, pp. 167-168, Nov. 6-8, 1996, Seattle. http://portal.
`acm.org/citation.cfm?id:2371 15&coll:GUIDE&dl:GUIDE
`&CFID:6648365 8&CFTOKENI3 6023921 .
`Hinckley, K., Pierce, J., HorvitZ, E., Sinclair, M. Foreground and
`Background Interaction with Sensor-enhanced Mobile Devices,
`ACM TOCHI (Transactions on Computer-Human Interaction) Spe
`cial Issue on Sensor-Based Interaction, 12 (1), Mar. 2005, pp. 31-52.
`http://portal.acm.org/citation.cfm?id:1057240&coll:GUIDE
`&dl:GUIDE&CFID:66591340&CFTOKEN:6294934.
`Hinckley, K., Cutrell, E., Bathiche, S., and Muss, T. 2002. Quantita
`tive analysis of scrolling techniques. In Proceedings of the SIGCHI
`Conference on Human Factors in Computing Systems: Changing
`Our World, Changing Ourselves (Minneapolis, Minnesota, USA,
`Apr. 20-25, 2002). CHI ’02. ACM Press, New York, NY, 65-72.
`http://doi.acm.org/10.1145/503376.503389.
`Harrison, B. L., Fishkin, K. P., Gujar, A., Mochon, C., and Want, R.
`1998. Squeeze me, hold me, tilt me! An exploration of manipulative
`user interfaces. In Proceedings of the SIGCHI Conference on Human
`Factors in Computing Systems (Los Angeles, California, United
`States, Apr. 18-23, 1998). C. Karat, A. Lund, J. CoutaZ, and J. Karat,
`Eds. Conference on Human Factors in Computing Systems. ACM
`Press/Addison-Wesley Publishing Co., NewYork, NY, 17-24. http://
`portal.acm.org/citation.cfm?id:274647&coll:Portal&dl:GUIDE
`&CFID:66588306&CFTOKEN:73460863&CFID:66588306
`&CFTOKEN:73460863#.
`Kiyokuni Kawachiya , Hiroshi Ishikawa, NaviPoint: an input device
`for mobile information browsing, Proceedings of the SIGCHI con
`ference on Human factors in computing systems, p. 1-8, Apr. 18-23,
`1998, Los Angeles, California, United States http://portal.acm.org/
`citation.cfm?id:274645&coll:Portal&dl:GUIDE
`&CFID:66588306&CFTOKEN:73460863.
`Hinkley, K., Pierce, J ., Sinclair, M., and HorvitZ, E. Sensing Tech
`niques for Mobile Interaction. UIST 2000: ACM Symposium on User
`Interface Software and Technology, pp. 91-100. http://portalacm.
`org/citation.cfm?id:3 54417&coll:GUIDE&dl:GUIDE
`&CFID:6648365 8&CFTOKENI3 6023921.
`Baillie, L., KuncZier, H., and Anegg, H. 2005. Rolling, rotating and
`imagining in a virtual mobile world. In Proceedings of the 7th inter
`national Conference on Human Computer interaction with Mobile
`Devices &Amp; Services (SalZburg, Austria, Sep. 19-22, 2005).
`MobileHCI ’05, vol. 111. ACM Press, New York, NY, 283 -286.
`http://doi.acm.org/10.1145/1085777.1085833.
`Amy K. Karlson, Benjamin B. Bederson, John SanGiovanni, 2003.
`AppLens and LaunchTile: Two Designs for One-Handed Thumb Use
`on Small Devices http://hcil.cs.umd.edu/trs/2004-37/2004-37.html.
`Virpi Roto, Nokia Research. Browsing on Mobile Phones. http://
`www.research.att.com/~rjana/WF12iPaper1.pdf.
`Qualcomm
`Slingshot
`http://wireless.ign.com/articles/657/
`657041p1.html.
`GamePad http://www.mobilemag.com/content/100/345/C5578/.
`Kyocera Candid KX16 http://www.mobilemag.com/content/ 100/
`340/C4392/.
`XEG Mobile Phone Pad http://us.giZmodo.com/gadgets/cellphones/
`gaming-on-the-go-with-Xeg-136414.php
`http://www.
`akihabaranews.com/en/news-10615
`XEG%2C+the+mobile+phone+pad.html.
`Samsung Game Pad http://www.cellphonemall.net/wireless/store/
`accessorydetail.asp?id:23198&phoneid:334.
`
`http://www.
`
`Thumbscript http://www.thumbscript.com/indeX.html http://www.
`thumbscript.com/howitworks.htrnl
`http://www.thumbscript.com/
`technotes.html.
`EXideas
`http://www.exideascom/ME/indexhtml
`eXideas.com/ME/HardKeyhtml.
`NE-Ware
`http://www.n-e-ware.com/Downloads/KeySticld330/
`KSUserManual330i01.pdf.
`Synaptics
`http://wwwsynaptics.com/products/pdf/
`mobiletouchipbpdf.
`Atrua: sensor company http://www.atrua.com/s-mobilephones.html.
`Varatouch: sensor company http://www.esato.com/news/articlephp/
`id:388.
`Eleksen www.eleksen.com.
`SHARP Vodafone ZTCJ01 http://www.slashphone.com/93/3123.
`htrnl.
`The GamePad http://www.kotaku.com/gaming/cell-phones/killer
`cell-phone-game-controller-130968.php.
`Sony Clie game controller PEGA-GC10 http://www.palminfocenter.
`com/viewistory.asp?ID:4295.
`Blutetooth
`GPS
`http://mobilitytoday.com/news/005986/
`mobilityibuyGPSnowii -Blueiblueto othiGPS.
`Alphagrip http://www.alphagrips.com/AlphagripAG3UsersManual.
`pdf.
`http://www.tegic.com/pdfs/salessheets/
`TegiciT9
`T9%20Adaptive%20TeXt%20Input%20Sales%20Sheet%201.pdf
`http://www.tegic.com/pdfs/salessheets/
`T9%20Adaptive%20TeXt%20Input%20Sales%20Sheet%202.pdf
`http://www.tegic.com/pdfs/salessheets/
`T9%20Adaptive%20TeXt%20Input%20Sales%20Sheet%203.pdf
`http://www.tegic.com/pdfs/salessheets/
`Sloppy%20Type%Sales%20Sheet.pdf.
`Zicorp4eZiTap http://www.Zipcorp.com/eZiTap.htm.
`MotorolaiiTAP
`http://news.Zdnet.co.uldhardware/mobile/
`0,39020360,39118435,00.htm.
`DigitWireless: FastTap http://www.digitwireless.com/?ash/down
`load/fastap.pdf.
`Microth KeyWheel http:/ /www.microth.com/ circumscript/ overview.
`asp.
`Yuvee: special keypad layout www.yuvee.com http://www.yuvee.
`com/builtin1.shtml http://www.yuvee.com/builtiinibshtml. http://
`www.yuvee .com/testdrive2 .shtml.
`Twiddler http://www.handykey.com/ http://www.handykey.com/
`Keymap.pdf.
`Sengital Ltd. Tilt sensor replacement for PDA http://sengitalmanu
`facturerglobalsources.com/si/6008823523892/ProductDetail/PDA
`keyboard/procutsiid-100105013 5/action-GetProduct.htm.
`Howard.co.kriThe mouse phone http://www.howard.co.kr/com
`puter/mouse/mousephone.htrn.
`Nokia 6620 with a control stick http://nds2.nokia.com/?les/support/
`nam/phones/guides/6620iUSien.PDF.
`Sega into cell phones http://www.phoneyworld.com/newspage.
`aspX?n:1745 .
`Prevalent Devices LLC http://www.prevalentdevices.com/manual3
`5-06.pdf.
`M. Kolsch, M. Turk, 2002. Keyboards without Keyboards: A Survey
`of Virtual Keyboards. UCSB Technical Report 2002-21, Jul. 12,
`2002. http://www.cs.ucsb.edu/research/techireports/reports/2002
`21.pdf.
`Jeong-Hoon Shin and Kwang-Seok Hong. An improved alphanu
`meric input algorithm using gloves. http://www.complexity.org.au/
`conference/upload/shin0 1/shin0 1 .pdf.
`C. MetZger, M. Anderson, T. Starner, 2004. FreeDigiter: A Contact
`Free Device for Gesture Control. Eighth IEEE International Sympo
`sium on Wearable Computers (ISWC’04) pp. 18-21. http://www.
`wirelessrerc.gatech.edu/projects/development/D1?les/iswc04 -
`freedigiterpdf.
`Rakkolainen, I. (2003). MobiVRia novel user interface concept for
`mobile computing. In: Bieber, K. & Kirste, T. (eds.), Proceedings of
`the 4th International Workshop on Mobile Computing, IMC 2003,
`Jun. 17-18, 2003, Rostock, Germany, pp. 107-112.http://www.cs.tut.
`fM~ira/IMC2003.pdf.
`Lumsden, J ., Gammell, A. (2004). Mobile Note Taking: Investigating
`the Efficacy of Mobile Text Entry. In Proc. of Mobile Human-Com
`
`SCEA Ex. 1001 Page 3
`
`

`

`US 7,463,245 B2
`Page 4
`
`puter Interaction (MobileHCI 2004), Glasgow, UK, Sep. 2004: In S.
`Brewster and M. Dunlop (Eds.). Mobile Human-Computer
`InteractioniMobileHCI 2004, Lecture Notes in Computer Science,
`vol. 3160, Berlin: Springer, 156-168.
`MacKenZie, I. S., & Soukoreff, R. W. Phrase sets for evaluating text
`entry techniques. Extended Abstracts of the ACM Conference on
`Human Factors in Computing Systems4CHI 2003, pp. 754-755
`New York: ACM.
`MacKenZie, I. S. (2002). KSPC (keystrokes per character) as a char
`acteristic of text entry techniques. Proceedings of the Fourth Inter
`national Symposium on Human-Computer Interaction with Mobile
`Devices, pp. 195-210. Heidelberg, Germany: Springer-Verlag.
`Soukoreff, R. W. and MacKenZie, I. S. (2004). Recent Developments
`in Text-Entry Error Rate Measurement. CHI 2004, Late Breaking
`Results Paper, Vienna Austria, Apr. 24-29, 2004.
`Lee, S. and Hong S.H.. Chording as a Text Entry Method in Mobile
`Phones. In Proceedings of the MobileHCI 2004: 6th International
`Symposium, Glasgow, UK, Sep. 13-16, 2004.
`Green, N., Kruger, J., Faldu, C., and St. Amant, R. 2004. A reduced
`QWERTY keyboard for mobile text entry. in CHI ’04 Extended
`Abstracts on Human Factors in Computing Systems (Vienna, Aus
`tria, Apr. 24-29, 2004). CHI ’04. ACM Press, NewYork, NY, 1429
`1432. http://portal.acm.org/citation.cfm?id:986082&coll:GUIDE
`&dl:GUIDE&CFID:66591340&/CRTOKEN:6294934.
`Partridge, K., Chatterjee, S., SaZawal,V., Borriello, G., and Want, R.
`TiltType: accelerometer-supported text entry for very small
`devices, Proceedings of the 15th annual ACM symposium on User
`interface software and technology, Oct. 27-30, 2002, Paris, France.
`Mikael Goldstein, Didier Chincholle, Morten Back (2000). Assess
`ing Two New Wearable Input Paradigms: The Finger-Joint-Gesture
`Palm-Keypad Glove and the Invisible Phone Clock. Personal and
`Ubiquitous Computing, vol. 4, Issue 2/3.
`Bartlett, J. F. 2000. Rock ‘n’ Scroll Is Here to Stay. IEEE Comput.
`Graph. Appl. 20, 3 (May 2000), 40-45. http://portal.acmorg/citation.
`cfm?id:618728&coll:Portal&d1:GUIDE&CFID:66588306
`&CFTOKEN:73460853#.
`Eslambolchilar, P, Murray-Smith, R. (2004). Tilt-Based Automatic
`Zooming and Scaling in Mobile Devices-A state-space implemen
`tation. In Proc. Of Mobile Human-Computer Interaction (MobileHCI
`2004), Glasgow, UK, Sep. 2004: In S. Brewster and M. Dunlop (Eds).
`Mobile Human-Computer-InteractioniMobile HCI 2004, Lecture
`Notes in Computer Science, vol. 3160, Berlin: Springer, 120-131.
`
`Zhai, S., Smith BA, and Selker, T. Improving Browsing Perfor
`mance: A study of four input devices for scrolling and pointing tasks,
`Proceedings of the IFIP TC13 International Conference on Human
`Comuter Interaction, p. 286-293, Jul. 14-18, 1997.
`Wobbrock, J. O., ForliZZi, J., Hudson, S. E. and Myers, B. A.
`WebThumb: interaction techniques for small-screen browsers. Proc.
`UIST, ACM Press (2002), 205-208.
`Lee, S., Hong, S. H., and Jeon, J. W. 2002. Designing a universal
`keyboard using chording gloves. SIGCAPH Comput. Phys. Handi
`cap. , 73-74 (Jun. 2002), 142-147. http://doi.acm.org/10.1145/
`960201.957230.
`Pirhonen, A., Brewster, S., and Holguin, C. 2002. Gestural and audio
`metaphors as a means of control for mobile devices. In Proceedings
`of the SIGCHI Conference on Human Factors in Computing Sys
`tems: Changing Our World, Changing Ourselves (Minneapolis, Min
`nesota, USA, Apr. 20-25, 2002). CHI ’02. ACM Press, NewYork, NY,
`291-298. http://doi.acm.org/10.1145/503376.503428.
`Kjeldskov, J. and Graham, C. (2003). A Review of Mobile HCI
`Research Methods. In Proc. of Mobile Human-Computer-Interaction
`(MobileHCI 2003), Udine Italy, Sep. 2003: In L. Chittaro (Ed.).
`Mobile Human-Computer-InteractioniMobileHCI 2003, Lecture
`Notes in Computer Science, vol. 2795, Berlin: Springer, 317-335.
`Kjeldskov, J. Stage, J. (2004). New Techniques for Usability Evalu
`ation of Mobile Systems. International Journal of Human-Computer
`Studies, May 2004, 60 (5-6): 599-620.
`Buxton, “A Directory of Sources for Input Technologies”, Oct. 1,
`2003, http://www.billbuxton.com/InputSources.html.
`“Introducing the Dvorak Keyboard”, http://www.mwbrooks.com/
`dvorald.
`Innovative Ergonomic Solutions, Cirque Pocket Keyboard, http//
`www.iesproducts.com/key-misc-pockethtml.
`Buxton, “Human Input to Computer Systems: Theories, Techniques
`and Technology”, http://billbuxton.com/inputManuscripthtlm.
`Buxton, “An Introduction to Human Input to Computers”, Apr. 6,
`1999, http://www.billbuxton.com/input01.Introduction.pdf.
`Unidenti?ed and Undated Document discussing alternative designs
`to QWERTY Keyboard, pp. 2-10.
`Examiner Takashi ShinoZuka; JPO Noti?cation of Reason(s) for
`Refusal; Dispatch Date: Jul. 8, 2008; Dispatch No.: 396667.
`
`* cited by examiner
`
`SCEA Ex. 1001 Page 4
`
`

`

`US. Patent
`
`Dec. 9, 2008
`
`Sheet 1 of9
`
`US 7,463,245 B2
`
`wj
`
`S23E00 5&5
`
`M .3
`
`35
`
`N2
`
`3
`
`“83005
`
`§
`
`E82
`
`@0602
`
`a
`
`36¢
`
`%
`
`SCEA Ex. 1001 Page 5
`
`

`

`U.S. Patent
`
`7,
`
`2B542.}3
`
`%:m
`
`w:55E95mmm9.,uummM
` 53%m9l__ME:H:1u2_OHN“w5&8on“5:05:00“:9:lmmaymcommEnfiomm< Mda”mbnfiomm<m3%:
`
`
`
`
`NEswfi
`
`SCEA Ex. 1001 Page 6
`
`SCEA Ex. 1001 Page 6
`
`

`

`U S. Patent
`
`Dec. 9, 2008
`
`Sheet 3 of9
`
`US 7,463,245 B2
`
`SCEA Ex. 1001 Page 7
`
`

`

`US. Patent
`
`Dec. 9, 2008
`
`Sheet 4 of9
`
`US 7,463,245 B2
`
`a
`
`mwlm
`
`mm
`
`em 3
`
`SCEA Ex. 1001 Page 8
`
`

`

`US. Patent
`
`Dec. 9, 2008
`
`Sheet 5 of9
`
`US 7,463,245 B2
`
`g
`
`g
`
`SCEA Ex. 1001 Page 9
`
`

`

`U S. Patent
`
`Dec. 9, 2008
`
`Sheet 6 of9
`
`US 7,463,245 B2
`
`SCEA Ex. 1001 Page 10
`
`

`

`U S. Patent
`
`Dec. 9, 2008
`
`Sheet 7 0f 9
`
`US 7,463,245 B2
`
`Fig 6a
`
`SCEA Ex. 1001 Page 11
`
`

`

`U S. Patent
`
`Dec. 9, 2008
`
`Sheet 8 of9
`
`US 7,463,245 B2
`
`WI w
`
`New
`
`SCEA Ex. 1001 Page 12
`
`

`

`US. Patent
`
`Dec. 9, 2008
`
`Sheet 9 of9
`
`US 7,463,245 B2
`
`w PBwE
`
`E02
`
`www
`
`a a5 vEEE/H
`
`
`
`#8362 Eooq
`
`Q
`
`@235
`
`%%
`
`55mm
`
`i
`
`SCEA Ex. 1001 Page 13
`
`

`

`US 7,463,245 B2
`
`1
`HUMAN INTERFACE SYSTEM
`
`PRIORITY INFORMATION
`
`This application is a continuation of US. application Ser.
`No. 10/699,555, ?led on Oct. 31, 2003 now US. Pat. No.
`7,218,313; the disclosure of the prior application is consid
`ered part of (and is incorporated by reference in) the disclo
`sure of this application.
`
`BACKGROUND
`
`The following description relates to human interface and
`input systems for electronic devices, particularly hand-held
`electronic devices, such as cell phones, personal digital assis
`tants (“PDAs”), pocket personal computers, smart phones,
`hand-held game devices, bar-code readers, remote controls,
`and other similar input devices having a keypad or one or
`more input elements.
`Electronic devices have become increasingly sophisticated
`and physically smaller due in part to a decrease in the price of
`processing poWer and a concurrent increase in demand by
`consumers for smaller devices. Such devices, hoWever, tend
`to be limited in function and utility by the user’s ability to
`interface With the device for data input (e.g., text, numeric,
`and functional input) and/or device control, Which becomes
`increasingly more di?icult to do as the available space on the
`device’s surface for positioning the input elements, Which are
`used for data input and/or device control, continues to
`decrease.
`Various human interface and input systems and techniques
`for hand-held electronic devices have been developed for data
`input and device control. These include miniature keyboards
`and keypads used in combination With chordal input tech
`niques, modal input techniques and/or smart keys; and touch
`screens used in combination With on-screen keyboard or key
`pad softWare or hand-Writing recognition softWare.
`Keyboard or Key Pad Used With Chordal, Modal and Smart
`Key Techniques
`Miniature keyboards and keypads are similar to their stan
`dard full-siZe versionsii.e., a keyboard generally has a full
`set or substantially full set of numeric, character, and func
`tional input elements, While key pads typically have a reduced
`set of numeric, character and/or functional input elements
`compared to keyboards. These miniature input devices typi
`cally are designed to ?t the available space on one surface of
`a hand-held electronic device or are designed as small, easily
`transportable, external plug-in devices. Thus, as hand-held
`electronic devices become smaller, the siZe of the input ele
`ments typically has been reduced in order for the desired
`number of input elements to ?t on one surface of the elec
`tronic device.
`For data input and device control, miniature keyboards and
`keypads typically either require one of tWo input tech
`niquesiuse of one or more thumbs or ?ngers to press the
`desired input elements or use of a stylus to “peck” the desired
`input elements (Which is usually done Where the input ele
`ment is of smaller siZe). Various techniques, such as chordal
`input techniques, modal input techniques and smart keys,
`have been developed and implemented to improve the e?i
`ciency and effectiveness of using miniature keyboards and
`keypads.
`Chordal Input Techniques
`Chordal input techniques generally are based upon the
`principle that characters, symbols, Words, phrases or concepts
`can be represented by a reduced set of input elements. Thus,
`by only having to press a reduced combination of input ele
`
`5
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`ments, functionality can be increased and quicker and more
`accurate data input can be realiZed. Chordal input techniques
`can be used on any keyboard or keypad con?guration or any
`device having more than one input element, and typically
`results in feWer input elements or more functions compared to
`conventional keyboards or keypads. An example of an elec
`tronic device using tWo-handed chordal input techniques is a
`court reporter or stenographer’ s typeWriter. One chordal input
`technique using a keypad to decrease the number of actua
`tions to achieve a large number of functions is described in
`US. Pat. No. 5,973,621 to Levy, entitled “Compact Keyed
`Input Device,” Which is incorporated herein by reference.
`Modal Input Techniques
`Modal input techniques are based on the concept that func
`tions of the electronic device, e.g., text messaging in a cell
`phone or PDA, can be accessed by pressing a particular input
`element (or combination of elements) on a keyboard or key
`pad. Once that particular input element is pressed, the func
`tionality of all or a portion of the input elements on the
`keyboard or keypad may change. Modal techniques typically
`are used in calculators, cell-phones, and PDAs. For example,
`in cell phones, a modal technique called multi-tap is common,
`in Which individual input elements on the keypad are associ
`ated With multiple symbols, such as characters, letters, num
`bers, icons or other types of symbols, Which tends to reduce
`the number of input elements required to achieve the desired
`functions, e. g., a tWelve-input-element keypad can be used to
`represent all letters of the English alphabet and the decimal
`digits. A user can input a desired symbol Within a set of
`symbols associated With a certain input element by tapping on
`that particular input element With a thumb, ?nger, or stylus,
`one or more times to input the desired character. Thus, if a
`user desires to send a text message, the user may press a
`functional input element, e.g., a mode key, to access the text
`messaging functionality of the cell phone and then tap an
`individual input element one or more times to select the
`associated symbol for input. The number of taps needed to
`input a particular symbol may differ depending on the lan
`guage character set chosen. For example, Japanese keypad or
`keyboards typically require a minimum set of 46 characters
`for text input, While English or American keyboards and
`keypads usually require a minimum set of 26 characters for
`text input. These modal input techniques have gained some
`popularity as users perform more text functions.
`Smart Keys
`Smart keys are typically used on keypads and refer to a
`single key or combination of keys that, When pressed, predict
`the users next logical action. Some implementations Work
`better than others and some applications reduce the number of
`keystrokes required to complete a function better than others.
`Word-predictor softWare, for example, attempts to predict the
`Word or character the user intends to input based upon one or
`more letters inputted by the user and the likely probabilities
`Within a given language. The probability of the softWare
`guessing correctly increases With the length of the Word or
`number of letters or characters inputted. In a device using
`smart keys on the keypad, a user may tap the keys 2, 2 and 8
`in sequence to generate the Word “cat” and the device Would
`display that Word ?rst because it is usually the most common
`combination, Whereas the Word “bat,” Which canbe generated
`by pressing the same keys, Would not be displayed ?rst
`because it is not as common. Also, the Word “cat” may be
`displayed after pressing the 2 key the second time based on a
`guess by the Word-predictor softWare.
`Smart keys also are typically used for Japanese data input
`Where a user phonetically inputs letters representing the
`sound of the Japanese character (e.g., a Kanji character).
`
`SCEA Ex. 1001 Page 14
`
`

`

`US 7,463,245 B2
`
`3
`Based on the inputted letters, the predictor software guesses
`the Japanese character. To select the character, a user would
`press the accept button or use the scrolling function to go to
`the next character with a similar set of phonetic inputs.
`Touch Screen Using On-Screen Keyboard or Handwriting
`Recognition Software
`Using on-screen keyboard or keypad software with a touch
`screen offers users the ability to enter data with ?ngers or
`thumbs on a screen-siZed keyboard or buttons, allowing faster
`data input without a stylus or physical keyboard or keypad
`accessory; while using handwriting recognition software
`with a touch screen, such as Graf?tiO on the Palm operating
`system, offers users the ability to enter text with a stylus by
`writing the text directly on the touch screen. Touch screens
`usually consume more power and are more expensive than
`non touch-sensitive screens. This higher power consumption
`can be a problem for hand-held electronic devices, which
`typically have limited power resources. Moreover, touch
`screens usually require the user to use both hands (e.g., one
`hand is used to hold and steady the device while the other
`hand is used to grasp the stylus), which is generally undesir
`able for interfacing with and controlling one handed hand
`held electronic device, such as cell phones. Handwriting rec
`ognition software has improved the slowness and
`awkwardness inherent in stylus, ?nger or thumb input but
`other drawbacks still remain, such as high power consump
`tion, the necessity to use both hands, and lack of tactile
`feedback to inform a user when an input element has been.
`Moreover, recognition software requires training to use prop
`erly, and, even then, still results in a high error rate.
`Game Control
`For game control, many of the above approaches have been
`used, but in most hand-held electronic devices, a user typi
`cally controls game play through the use of some form of
`input element, such as on a miniature keypad and/or direc
`tional pad (“D-pad”), which typically is located on the front
`surface of the device. Game control on some hand-held elec
`tronic devices, such as cell phones, is inherently one handed
`or at most two thumbed because of the siZe of the device,
`while game control on other hand-held electronic devices,
`su