US007656393B2
`
`US 7,656,393 B2
`(10) Patent No.:
`a2) United States Patent
`Kinget al.
`(45) Date of Patent:
`Feb. 2, 2010
`
`
`(75)
`
`(54) ELECTRONIC DEVICE HAVING DISPLAY
`AND SURROUNDING TOUCH SENSITIVE
`BEZEL FOR USER INTERFACE AND
`CONTROL
`Inventors: Nick King, San Jose, CA (US); Duncan
`3
`3
`3
`Kerr, San Francisco, CA (US); Paul
`Herbst, Morgan Hill, CA (US); Steven
`P Hotelling, San Jose, CA (US)
`(73) Assignee: Apple Inc., Cupertino, CA (US)
`.
`.
`.
`.
`oa.
`(*) Notice:
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 1546) by 588 days.
`(21) Appl. No.: 11/426,078
`
`(22)
`
`(65)
`
`Filed:
`
`Jun. 23, 2006
`oo.
`.
`Prior Publication Data
`US 2006/0238517 Al
`Oct. 26, 2006
`
`Related U.S. Application Data
`(63) Continuation-in-part of application No. 11/367,749,
`filed on Mar. 3, 2006.
`(60) Provisional application No. 60/663,345, filed on Mar.
`16, 2005, provisional application No. 60/658,777,
`filed on Mar. 4, 2005.
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 3/041
`(2006.01)
`G09G 5/00
`(2006.01)
`GO06F 3/02
`(2006.01)
`GO6F 3/045
`(2006.01)
`HO4B 1/40
`(2006.01)
`A6IM 37/00
`(52) U.S.C ceeeccccccccesseeee 345/173; 345/156; 345/169;
`345/174; 455/77; 604/6.13
`(58) Field of Classification Search.......... 345/156-178
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`US. PATENT DOCUMENTS
`3,333,160 A
`7/1967. Gorski
`.
`Ci
`tt
`d
`(Continued)
`FOREIGN PATENT DOCUMENTS
`1243096
`10/1988
`
`CA
`
`(Continued)
`
`OTHER PUBLICATIONS
`Quantum Research Group, “QT510/ QWheel(TM)TouchSlider IC,”
`copyright 2004-2005, 14-pgs.
`
`(Continued)
`Primary Examiner—Sumati Lefkowitz
`Assistant Examiner—Charles Hicks
`(74) Attorney, Agent, or Firm—Morrison & Foerster LLP
`
`(57)
`ABSTRACT
`An electronic device has a display and has a touch sensitive
`bezel surrounding the display. Areas on the bezel are desig-
`nated for controls used to operate the electronic device. Visual
`guides corresponding to the controls are displayed on the
`display adjacent the areas of the bezel designated for the
`controls. Touch data is generated by the bezel when a user
`touches an area of the bezel. The device determines which of
`the controls has been selected based on which designated area
`is associated with the touch data from the bezel. The device
`then initiates the determined control. The device can have a
`sensor for determiningthe orientation ofthe device. Based on
`the orientation, the device can alter the areas designated on
`the bezel for the controls and can alter the location of the
`visual guides for the display so that they match the altered
`areas on the bezel.
`
`45 Claims, 13 Drawing Sheets
`
`920
`
`IF
`
`800
`
`
`
`
`APPLE 1018
`
`APPLE 1018
`
`1
`
`

`

`US 7,656,393 B2
`
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`3,541,541 A
`3,662,105 A
`3,798,370 A
`4,246,452 A
`4,550,221 A
`4,566,001 A
`4,672,364 A
`4,672,558 A
`4,692,809 A
`4,695,827 A
`4,733,222 A
`4,734,685 A
`4,746,770 A
`4,771,276 A
`4,788,384 A
`4,806,846 A
`4,896,370 A
`4,898,555 A
`4,933,660 A
`4,968,877 A
`5,003,519 A
`5,017,030 A
`5,178,477 A
`5,189,403 A
`5,194,862 A
`5,224,861 A
`5,241,308 A
`5,252,951 A
`5,281,966 A
`5,305,017 A
`5,345,543 A
`5,376,948 A
`5,379,057 A
`5,398,310 A
`5,442,742 A
`5,463,388 A
`5,463,696 A
`5,483,261 A
`5,488,204 A
`5,495,077 A
`5,513,309 A
`5,523,775 A
`5,530,455 A
`5,543,590 A
`5,543,591 A
`5,563,632 A
`5,563,996 A
`5,565,658 A
`5,579,036 A
`5,581,681 A
`5,583,946 A
`5,590,219 A
`5,592,566 A
`5,594,471 A
`5,594,810 A
`5,596,694 A
`5,612,719 A
`5,631,805 A
`5,633,955 A
`5,634,102 A
`5,636,101 A
`5,642,108 A
`5,644,657 A
`5,666,113 A
`5,666,502 A
`5,666,552 A
`5,675,361 A
`5,675,362 A
`5,677,710 A
`5,689,253 A
`5,710,844 A
`
`............. 455/77
`
`11/1970 Englebart
`5/1972 Hurst etal.
`3/1974 Hurst
`1/1981 Chandler
`10/1985 Mabusth
`1/1986 Mooreet al.
`6/1987 Lucas
`6/1987 Beckesetal.
`9/1987 Beiningetal.
`9/1987 Beininget al.
`3/1988 Evans
`3/1988 Watanabe
`5/1988 McAvinney
`9/1988 Parks
`11/1988 Bruere-Dawsonetal.
`2/1989 Kerber
`1/1990 Kasparian et al.
`2/1990 Sampson
`6/1990 Wynne,Jr.
`11/1990 McAvinneyetal.
`3/1991 Noijean
`5/1991 Crews
`1/1993 Gambaro
`2/1993 Franzetal.
`3/1993 Edwards
`7/1993 Glass et al.
`8/1993 Young
`10/1993 Tannenbaum et al.
`1/1994 Walsh
`4/1994 Gerpheide
`9/1994 Cappsetal.
`12/1994 Roberts
`1/1995 Clough etal.
`3/1995 Tchaoet al.
`8/1995 Greysonet al.
`10/1995 Boieetal.
`10/1995 Beernink etal.
`1/1996 Yasutake
`1/1996 Meadet al.
`2/1996 Miller et al.
`4/1996 Meier et al.
`6/1996 Capps
`6/1996 Gillick et al.
`8/1996 Gillespie et al.
`8/1996 Gillespie et al.
`10/1996 Roberts
`10/1996 Tchao
`10/1996 Gerpheide etal.
`11/1996 Yates, IV
`12/1996 Tchaoet al.
`12/1996 Gourdol
`12/1996 Gourdol
`1/1997 Pagalloetal.
`1/1997 Deeran etal.
`1/1997 Gourdol
`1/1997 Capps
`3/1997 Beernink et al.
`5/1997 Bonsall
`5/1997 Bozinovicet al.
`5/1997 Capps
`6/1997 Bonsall et al.
`6/1997 Gopheretal.
`7/1997 Cappsetal.
`9/1997 Logan
`9/1997 Capps
`9/1997 Greysonet al.
`10/1997 Santilli
`10/1997 Clough et al.
`10/1997 Thompson-Rohrlich
`11/1997 Hargreavesetal.
`1/1998 Cappsetal.
`
`5,729,250 A
`5,730,165 A
`5,736,976 A
`5,741,990 A
`5,745,116 A
`5,745,716 A
`5,748,269 A
`5,764,818 A
`5,767,457 A
`5,767,842 A
`5,790,104 A
`5,790,107 A
`5,802,516 A
`5,808,567 A
`5,809,267 A
`5,821,690 A
`5,821,930 A
`5,823,782 A
`5,825,351 A
`5,825,352 A
`5,854,625 A
`5,880,411 A
`5,898,434 A
`5,920,309 A
`5,923,319 A
`5,933,134 A
`5,943,044 A
`6,002,389 A
`6,002,808 A
`6,020,881 A
`6,031,524 A
`6,037,882 A
`6,050,825 A
`6,052,339 A
`6,072,494 A
`6,084,576 A
`6,107,997 A
`6,128,003 A
`6,131,047 A
`6,131,299 A
`6,135,958 A
`6,144,380 A
`6,188,391 Bl
`6,198,515 Bl
`6,208,329 Bl
`6,222,465 Bl
`6,239,790 BL
`6,243,071 Bl
`6,246,862 BL
`6,249,606 Bl
`6,288,707 Bl
`6,289,326 Bl
`6,292,178 BL
`6,297,811 B1*
`6,323,846 Bl
`6,347,290 Bl
`6,377,009 Bl
`6,380,391 B2
`6,411,287 Bl
`6,413,233 BL*
`6,414,671 Bl
`6,421,234 Bl
`6,452,514 Bl
`6,457,355 BL
`6,466,036 Bl
`6,466,203 B2
`6,515,669 Bl
`6,525,749 BL
`6,535,200 B2
`6,543,684 BL
`6,543,947 B2
`6,570,557 Bl
`6,593,916 Bl
`
`3/1998 Bishopetal.
`3/1998 Philipp
`4/1998 Cheung
`4/1998 Davies
`4/1998 Pisutha-Arnond
`4/1998 Tchaoetal.
`5/1998 Harris et al.
`6/1998 Cappsetal.
`6/1998 Gerpheideetal.
`6/1998 Korth
`8/1998 Shieh
`8/1998 Kasser etal.
`9/1998 Shwartsetal.
`9/1998 McCloud
`9/1998 Moran etal.
`10/1998 Martensetal.
`10/1998 Hansen
`10/1998 Marcusetal.
`10/1998 Tam
`10/1998 Bissetetal.
`12/1998 Frisch et al.
`3/1999 Gillespieet al.
`4/1999 Smalletal.
`7/1999 Bisset et al.
`7/1999 Bishopetal.
`8/1999 Shieh
`8/1999 Martinelli et al.
`12/1999 Kasser
`12/1999 Freeman
`2/2000 Naughtonetal.
`2/2000 Kunert
`3/2000 Levy
`4/2000 Nicholet al.
`4/2000 Frenkelet al.
`6/2000 Nguyen
`7/2000 Leuet al.
`8/2000 Ure
`10/2000 Smith et al.
`10/2000 Hayes,Jr. etal.
`10/2000 Raabet al.
`10/2000 Mikula-Curtiset al.
`11/2000 Shwartsetal.
`2/2001 Seely et al.
`3/2001 Cole
`3/2001 Ballare
`4/2001 Kumar etal.
`5/2001 Martinelli et al.
`6/2001 Shwartset al.
`6/2001 Grivasetal.
`6/2001 Kiraly etal.
`9/2001 Philipp
`9/2001 LaFleur
`9/2001 Bernsteinetal.
`
`10/2001 Kentetal.
`11/2001 Westerman etal.
`2/2002 Bartlett
`4/2002 Philipp
`4/2002 Gillespieetal.
`6/2002. Scharff et al.
`
`wo... 604/6.13
`7/2002 Sitesetal.
`7/2002 Gillespieetal.
`7/2002: Ricksetal.
`9/2002 Philipp
`10/2002 Philipp
`10/2002 Philipp
`10/2002 Van Ee
`2/2003 Mohri
`2/2003 Moran etal.
`3/2003 Philipp
`4/2003. White etal.
`4/2003 Lee
`5/2003 Westerman etal.
`7/2003 Aroyan
`
`oo... 345/173
`
`2
`
`

`

`US 7,656,393 B2
`
`Page 3
`
`2006/0044259 Al
`2006/0053387 Al
`2006/0066582 Al
`2006/0079969 Al
`2006/0085757 Al
`2006/0097991 Al
`2006/0197753 Al
`2006/0232567 Al
`2006/0238518 Al
`2006/0238519 Al
`2006/0238520 Al
`2006/0238521 Al
`2006/0238522 Al
`2007/0229464 Al
`2007/0236466 Al
`2007/0247429 Al
`2007/0257890 Al
`2008/0088602 Al
`2008/0238879 Al
`2008/0297476 Al
`
`3/2006 Hotellinget al.
`3/2006 Ording
`3/2006 Lyonetal.
`4/2006 Seguin
`4/2006 Andreetal.
`5/2006 Hotelling etal.
`9/2006 Hotelling
`10/2006 Westerman etal.
`10/2006 Westerman etal.
`10/2006 Westerman etal.
`10/2006 Westerman etal.
`10/2006 Westerman etal.
`10/2006 Westerman etal.
`10/2007 Hotelling etal.
`10/2007 Hotelling
`10/2007 Westerman
`11/2007 Hotelling etal.
`4/2008 Hotelling
`10/2008 Jaeger etal.
`12/2008 Hotelling etal.
`
`........... 345/169
`
`7/2003
`6,597,384
`Harrison
`8/2003
`6,610,936
`Gillespieet al.
`9/2003
`6,624,833
`Kumar etal.
`10/2003
`6,639,577
`Eberhand.
`Hurstetal.
`11/2003
`6,650,319
`12/2003
`6,658,994
`McMillan
`12/2003
`6,670,894
`Mehring
`Westerman
`1/2004
`6,677,932
`Blanchard
`1/2004
`6,677,934
`Crawford
`4/2004
`6,724,366
`Orossetal.
`6/2004
`6,757,002
`7/2004
`D493,157
`Yang
`7/2004
`D493,158
`Yang
`10/2004
`D497,606
`Yang
`10/2004
`6,803,906
`Morrisonet al.
`12/2004
`D500,298
`Yang
`1/2005
`6,842,672
`Straubet al.
`5/2005
`6,888,532
`Wong etal. we 345/156
`5/2005
`6,888,536
`Westerman et al.
`5/2005
`6,900,795
`Knight, III et al.
`Liebenow et al.
`6/2005
`6,909,424
`8/2005
`6,927,761
`Badayeetal.
`9/2005
`D509,819
`‘Yang
`5/2004
`10251296 Al
`DE
`9/2005
`D509,833
`‘Yang
`11/1988
`0 288 692
`EP
`9/2005
`D510,081
`‘Yang
`1/1992
`0 464 908
`EP
`McAllister et al.
`9/2005
`6,942,571
`7/1995
`0 664 504
`EP
`11/2005
`DS511,512
`‘Yang
`1/2002
`1014 295
`EP
`11/2005
`DS511,528
`‘Yang
`2/2005
`1505 484 Al
`EP
`11/2005
`6,965,375
`Gettemyetal.
`4/2004
`2 393 688 A
`GB
`12/2005
`D512,403
`‘Yang
`8/1998
`10-228350 A
`JP
`12/2005
`DS512,435
`‘Yang
`5/1997
`W09718547 Al
`WO
`Akitt et al.
`12/2005
`6,972,401
`TN997
`WO-97/023738
`WO
`Hedrick
`12/2005
`6,977,666
`4/1998
`WO-98/ 14863
`WO
`Straubet al.
`1/2006
`6,985,801
`
`WO WO003088176 Al—10/2003
`1/2006
`6,992,659
`Gettemy
`WO WO-2004/111816 A2=12/2004
`
`4/2006
`Bornet al.
`7,031,228
`WO
`WO-2004/111816 A3
`12/2004
`5/2006
`D520,516
`‘Yang
`WO
`WO-2006/023569
`3/2006
`9/2006
`7,109,978
`Gillespie et al... 345/173
`WO
`WO-2006/094308 A2
`9/2006
`10/2006
`Steinfeld et al.
`7,129,416
`WO
`WO-2006/094308 A3
`9/2006
`12/2006
`7,145,552
`Hollingsworth «0.0.0.0... 345/168
`WO
`WO-2007/103631 A2
`9/2007
`1/2007
`Middleton
`7,170,496
`WO
`WO-2007/103631 A3
`9/2007
`7/2007
`7,240,289
`Naughton
`3/2008
`Westerman et al.
`RE40,153
`200 1/0043 189
`11/2001
`Briseboiset al.
`Baron
`2002/0093492
`7/2002
`2002/0118848
`8/2002
`Karpenstein
`2003/0006974
`1/2003
`Clough etal.
`2003/0074977
`4/2003
`Doemensetal.
`2003/0076301
`4/2003
`Tsuket al.
`2003/0076303
`4/2003
`Huppi
`2003/0076306
`4/2003
`Zadesky et al.
`2003/0095095
`5/2003
`Pihlaja
`2003/0095096
`5/2003
`Robbinetal.
`2003/0098858
`5/2003
`Perski et al.
`2003/0206202
`11/2003
`Moriya
`2003/0234768
`12/2003
`Rekimotoetal.
`2004/0012572
`1/2004
`Sowdenetal.
`2004/0263484
`12/2004
`Mantysalo etal.
`2005/0012723
`1/2005
`Pallakoff
`2005/0052425
`3/2005
`Zadesky et al.
`2005/0104867
`5/2005
`Westerman et al.
`2005/0110768
`5/2005
`Marriott et al.
`2005/0146509
`7/2005
`Geaghan et al... 345/173
`2005/0154798
`7/2005
`Nurmi
`2005/02 12760
`9/2005
`Marvitet al.
`2005/0219228
`10/2005
`Alameh
`2006/0022955
`2/2006
`Kennedy
`2006/0022956
`2/2006
`Lengelinget al.
`2006/0026521
`2/2006
`Hotellinget al.
`2006/0026535
`2/2006
`Hotellinget al.
`2006/0026536
`2/2006
`Hotellinget al.
`2006/0032680
`2/2006
`Eliasetal.
`2006/0033724
`2/2006
`Chaudhrietal.
`
`Kionx, “KXP84 Series Summary Data Sheet,” copyright 2005, dated.
`Oct. 21, 2005, 4-pgs.
`EBV Elektronik, “TSOP6238 IR Reciever Modules for Infrared
`Remote Control Systems,” dated Jan. 2004, 1-pg.
`Texas Instruments, “TSC2003 /12C Touch Screen Controller,” Data
`Sheet SBAS 162, dated Oct. 2001, 20-pgs.
`European Examination Report mailed Apr. 21, 2008, for EP Appli-
`cation No. 067375 15.4, filed Mar. 3, 2006, five pages.
`International Search Report mailed Oct. 6, 2006, for PCT Application
`No. PCT/US2006/08349, filed Mar. 3, 2006, three pages.
`International Search Report mailed Jan.3, 2008, for PCT Application
`No. PCT/US2007/062474, filed Feb. 21, 2007, three pages.
`Press Release, “iriver clix Delivers Complete Package for Portable
`Entertainment Fans,” obtained from www. iriveramerica.com/im-
`ages/pdf/iriv__clix.pdf, dated May 17, 2006, 3-pages.
`iriver clix Product Guide, copyright 1999-2006, 38-pages.
`iriver clix Quick Start Guide, undated, 2-pages.
`Gear Live Review: iRiver clix Review, obtained from http://www.
`gearlive.com/index.php/news/article/gear-live-review-iriver-clix-re-
`view-713400/, dated Jul. 13, 2006, 8-pages.
`Anonymous. “Touch Technologies Overview” 2001, 3M Touch Sys-
`tems, Massachusetts.
`Anonymous. “4-Wire Resistive Touchscreens” obtained from http://
`www.touchscreens.com/intro-touchtypes-4resistive-html generated
`Aug. 5, 2005.
`Anonymous. “5-Wire Resistive Touchscreens” obtained from http://
`www.touchscreens.com/intro-touchtypes-resistive.html
`generated
`Aug. 5, 2005.
`
`FOREIGN PATENT DOCUMENTS
`
`OTHER PUBLICATIONS
`
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`
`3
`
`

`

`US 7,656,393 B2
`Page 4
`
`Anonymous. “PenTouch Capacitive Touchscreens” obtained from
`Anonymous. “A Brief Overview of Gesture Recognition” obtained
`http://www.touchscreens.com/intro-touchtypes-pentouch.html gen-
`from http://www. Dai. Ed. Ac.uk/Cvonline/(LOCA_COPIES/
`erated Aug. 5, 2005.
`COHEN/gesture_overview. Html, generated Apr. 20, 2004.
`Anonymous. “Surface Acoustic Wave Touchscreens” obtained from
`Anonymous. “Capacitive Position Sensing” obtained from http://
`http://www.touchscreens.com/intro-touchtypes-saw.html generated
`www.synaptics.com/technology/cps.cfin generated Aug. 5, 2005.
`Aug. 5, 2005.
`Anonymous. “Capacitive Touchscreens” obtained from http://www.
`Anonymous. “Symbol Commander” obtained from http://www.
`touchscreens.com/intro-touchtypes-capacitive.html generated Aug.
`5, 2005.
`sensiva.com/symbolcommander/, generated Aug. 30, 2005.
`Anonymous.
`“Tips
`for Typing”
`FingerWorks
`http://www.
`Anonymous. “Comparing Touch Technologies”obtained from http://
`fingerworks.com/mini_typing.html generated Aug. 30, 2005.
`www.touchscreens.com/intro-touchtypes.html generated Oct. 10,
`2004.
`Anonymous. “Wacom Components—Technology” obtained from
`http://www.wacom-components.com/english/tech.asp generated on
`Anonymous. “FingerWorks—Gesture Guide—Application Switch-
`Oct. 10, 2004.
`ing,”
`obtained
`from
`http://www.fingerworks.com/
`gesture_guide_apps.html, generated on Aug. 27, 2004, 1-pg.
` http://rsb.info.nith.gov/ij/
`“Watershed Algorithm”
`Anonymous.
`plugins/watershed.html generated Aug. 5, 2005.
`Anonymous. “FingerWorks—Gesture Guide—Editing,” obtained
`from
`http://www.fingerworks.com/gesure_guide_editing.html,
`Bieret al., “Toolglass and Magic Lenses: The see-throughinterface”
`generated on Aug. 27, 2004, 1-pg.
`In James Kijiya, editor, Computer Graphics (SIGGRAPH ’93 Pro-
`ceedings), vol. 27, pp. 73-80, Aug. 1993.
`Anonymous. “FingerWorks—Gesture Guide—File Operations,”
`obtained from http://www.fingerworks.com/gesture_guide_files.
`Douglas et al., The Ergonomics of Computer Pointing Devices
`html, generated on Aug. 27, 2004, 1-pg.
`(1997).
`Anonymous. “FingerWorks—Gesture Guide—Text Manipulation,”
`European Search Report received in EP 1 621 989 (@ Beyer Weaver
`obtained.
`from
`http://www.fingerworks.com/
`& Thomas, LLP) dated Mar. 27, 2006.
`gesture_guide_text_manip.html, generated on Aug. 27, 2004, 2-pg.
`Fisheret al., “Repetitive Motion Disorders: The Design of Optimal
`Anonymous. “FingerWorksGesture Guide_Tips and Tricks,”
`Rate- Rest Profiles,’ Human Factors, 35(2):283-304 (Jun. 1993).
`obtained from http://www.fingerworks.com/gesture__guide_tips.
`Fukumoto and Yoshinobu Tonomura, “Body Coupled Fingering:
`html, generated Aug. 27, 2004, 2-pgs.
`Wireless Wearable Keyboard,” CHI97, pp. 147-154 (Mar. 1997).
`Fukumotoet al., “ActiveClick: Tactile Feedback for Touch Panels,”
`Anonymous. “FingerWorks—Gesture Guide—Web,” obtained from
`http://www.fingerworks.com/gesture_guide_web.html, generated
`In CHI 2001 Summary, pp. 121-122, 2001.
`on Aug. 27, 2004, 1-pg.
`Hardy, “Fingerworks” Mar. 7, 2002; BBC World OnLine.
`Anonymous. “FingerWorks—Guide to Hand Gestures for USB
`Hillier and Gerald J. Lieberman, Jntroduction to Operations
`
`Touchpads,” from_http://www.fingerworks.com/obtained
`
`Research (1986).
`igesture_userguide.html, generated Aug. 27, 2004, 1-pg.
`International Search Report dated Mar. 3, 2006 (PCT/US 05/03325;
`Details,”
`Anonymous.
`“FingerWorks—iGesture—Technical
`119-0052WO).
`obtained from http://www.fingerworks.com/igesture_tech.html,
`Jacob et al., “Integrality and Separability of Input Devices,’ ACM
`generated Aug. 27, 2004, 1-pg.
`Transactions on Computer-Human Interaction, 1:3-26 (Mar. 1994).
`Anonymous. “FingerWorks—The Only Touchpads with Ergonomic
`Kinkley etal., “Touch-Sensing Input Devices,” in CHI ’99 Proceed-
`Full-Hand Resting and Relaxation!” obtained from http://www.
`ings, pp. 223-230, 1999.
`fingerworks.com/resting html, Copyright 2001, 1-pg.
`Lee,“A Fast Multiple-Touch-Sensitive Input Device,” Master’s The-
`Anonymous. “FingerWorks—Tipsfor Typing on the Mini,” obtained
`sis, University of Toronto (1984).
`from http://www.fingerworks.com/mini_typing.html, generated on
`Lee, S.K.et al., “A Multi-Touch Three Dimensional Touch-Sensitive
`Aug. 27, 2004, 2-pgs.
`Tablet,” in CHI ’85 Proceedings, pp. 121-128, 2000 [(Apr. 1985).
`Anonymous. “GlidePoint®” obtained from http://www.cirque.com/
`Conference Proceedings: Human Factors in Computing Systems, pp.
`technology/technology_qp-html generated Aug. 5, 2005.
`21-251]
`Anonymous. “Gesture Recognition” http://www.fingerworks.com/
`Matsushitaet al., “HoloWall: Designing a Finger, Hand, Body and.
`gesture_recognition.htm1>, Jul. 2, 2006.
`Object Sensitive Wall,” In Proceedings of UIST ’97, Oct. 1997.
`Anonymous. “How do touchscreen monitors know where you’re
`Non-Final Office Action mailed Feb. 26, 2009, for U.S. Appl. No.
`touching?” obtained from http://www.electronics.howstuffworks.
`11/367,749, filed Mar. 3, 2006, 10 pages.
`com/question716.html generated Aug. 5, 2005.
`Quek, “Unencumbered Gestural Interaction,” JEEE Multimedia,
`Anonymous. “How does a touchscreen work?”obtained from http://
`3:36-47 (Winter 1996).
`Radwin, “Activation Force and Travel Effects on Overexertion in
`www.touchscreens.com/intro-anatomy.html generated Aug. 5, 2005.
`Repetitive Key Tapping,” Human Factors, 39(1):130-140 (Mar.
`Anonymous. “iGesture Pad—the MultiFinger USB TouchPad with
`1997).
`Whole-HandGestures,”obtained from http://www.fingerworks.com/
`igesture.html, generated Aug. 27, 2004, 2-pgs.
`Rekimotoetal., “ToolStone: Effective Use of the Physical Manipu-
`lation Vocabularies of Input Devices,” In Proc. Of UIST 2000.
`Anonymous. “iGesture Products for Everyone (learn in minutes)
`Product Overview’FingerWorks.com downloaded Aug. 30, 2005.
`Rekimoto, J. (2002). “SmartSkin: An Infrastructure for Freehand.
`Manipulation on Interactive Surfaces,” CHT 2002, Apr. 20-25, 2002.
`Anonymous. “Infrared Touchscreens” obtained from http://www.
`[(Apr. 25, 2002). 4(1):113-120.]
`touchscreens.com/intro-touchtypes-infrared.html generated Aug. 5,
`2005.
`Rubineet al., “Programmable Finger-Tracking Instrument Control-
`lers,” Computer Music Journal, vol. 14, No. 1 (Spring 1990).
`Rutledgeet al., “Force-To-Motion Functions for Pointing,” Human-
`Computer Interaction—INTERACT(1990).
`Subatai Ahmad, “A Usable Real-Time 3D Hand Tracker,” Proceed-
`ings of the 28” Asilomar Conference on Signals, Systems and
`Computers—Part 2 (of2), vol. 2 (Oct. 1994).
`U.S. Appl. No. 10/789,676, filed Feb. 27, 2004 entitled “Shape
`Detecting Input Device”.
`U.S. Appl. No. 11/332,861, filed Jan. 13, 2006, which is a Reissue of
`6,677,932.
`Wellner, “The Digital Desk Calculators: Tangible Manipulation on a
`Desk Top Display’In ACM UIST ’91 Proceedings, pp. 27-34, Nov.
`1991.
`
`Anonymous. “Mouse Emulation” FingerWorks obtained from http://
`www.fingerworks.com/gesture_guide_mouse.html generated Aug.
`30, 2005.
`Anonymous. “Mouse Gestures in Opera” obtained from http://www.
`opera.com/products/desktop/mouse/index.dml generated Aug. 30,
`2005.
`
`Anonymous. “Mouse Gestures,” Optim oz, May 21, 2004.
`Anonymous. “MultiTouch Overview” FingerWorks obtained from
`http://finger.works.com/multoverview.html
`generated Aug.
`30,
`2005.
`
`Anonymous. “Near Field Imaging Touchscreens” obtained from
`http://www.touchscreens.com/intro-touchtypes-nfi.html generated
`Aug. 5, 2005.
`
`4
`
`

`

`US 7,656,393 B2
`Page 5
`
`Williams, “Applications for a Switched-Capacitor Instrumentation
`Building Block” Linear Technology Application Note 3, Jul. 1985,
`pp. 1-16.
`Yamadaet al., “A Switched-Capacitor Interface for Capacitive Pres-
`sure Sensors” IEEE Transactions on Instrumentation and Measure-
`ment, vol. 41, No. 1, Feb. 1992, pp. 81-86.
`Yeh et al., “Switched Capacitor Interface Circuit for Capacitive
`Transducers” 1985 IEEE.
`
`Zhaiet al., “Dual Stream Input for Pointing and Scrolling,” Proceed-
`ings of CHP 97 Extended Abstracts (1997).
`Zimmerman et al., “Applying Electric Field Sensing to Human-
`Computer Interfaces,” In CHI ’85 Proceedings, pp. 280-287, 1995.
`Final Office Action mailed Aug. 25, 2009, for U.S. Appl. No.
`11/367,749, filed Mar. 3, 2006, 12 pages.
`
`* cited by examiner
`
`5
`
`

`

`U.S. Patent
`
`Feb. 2, 2010
`
`Sheet 1 of 13
`
`US 7,656,393 B2
`
`
`
`6
`
`

`

`Feb. 2, 2010
`
`Sheet 2 of 13
`
`US 7,656,393 B2
`
`U.S. Patent
`
`7
`
`

`

`U.S. Patent
`
`Feb. 2, 2010
`
`Sheet 3 of 13
`
`US 7,656,393 B2
`
`
`
`8
`
`

`

`U.S. Patent
`
`US 7,656,393 B2
`
`Feb.2, 2010
`
`Sheet 4 of 13
`
`|
`
`COLUMN MUX
`CIRCUTRY
`
`9
`
`

`

`U.S. Patent
`
`Feb.2, 2010
`
`Sheet 5 of 13
`
`US 7,656,393 B2
`
`
`DESIGNATE AREAS ON
`BEZEL FOR CONTROLS
`
`|
`|
`
`GENERATE VISUAL GUIDES
`
`DISPLAY VISUAL GUIDES
`
`ADJACENT DESIGNATED AREAS
`
`
`
`SoONTOUCHBEZEL
`
`= |
`S|
`.bi
`
`i
`|
`40
`|
`
`OBTAIN TOUCH
`DATA FRON BEZEL
`:
`COMPARE TOUCH DATA 10
`
`DESIGNATIONS ON BEZEL
`
`|
`
`|
`
`|
`|
`
`DETERMINE CONTROL
`
`IMPLICATED BY DESIGNATION
`
`INITIATE DETERMINED
`CONTROL
`
`
`
`PROCESS
`COLUMNS
`
`10
`
`

`

`U.S. Patent
`
`Feb. 2, 2010
`
`Sheet 6 of 13
`
`US 7,656,393 B2
`
`lie
`
`wo)>SS
`
`me=Ry)=tae
`
`Boole
`
`11
`
`

`

`Sheet 7 of 13
`
`US 7,656,393 B2
`
`Feb. 2, 2010
`
`U.S. Patent
`
`12
`
`12
`
`

`

`U.S. Patent
`
`Feb.2, 2010
`
`Sheet 8 of 13
`
`US 7,656,393 B2
`
`nrvthincttirtilenttnntllettlinatiandtlandtlentrdtatele.tlaratilradilernstlnns Senattl natin inttitrttaradtlratthenes
`
`
`SNSIB
`
`SNS1A
`
`SNS28
`
`SNS2A
`
`
`
`
`
`
`
`a SNS38
`Peeper
`
`13
`
`

`

`U.S. Patent
`
`Feb.2, 2010
`
`Sheet 9 of 13
`
`US 7,656,393 B2
`
`eeeci =
`
`
`009
`
`14
`
`14
`
`
`
`

`

`U.S. Patent
`
`Feb.2, 2010
`
`Sheet 10 of 13
`
`
`
`US 7,656,393 B2
`
`15
`
`15
`
`

`

`Sheet 11 of 13
`
`US 7,656,393 B2
`
`Feb. 2, 2010
`
`U.S. Patent
`
`16
`
`

`

`U.S. Patent
`
`Feb. 2, 2010
`
`Sheet 12 of 13
`
`US 7,656,393 B2
`
`832,
`
`Spee
`
` Beoeheeee
`
`ios\escactiFehe
`
`a
`
`RP
`
`17
`
`17
`
`
`

`

`U.S. Patent
`
`Feb. 2, 2010
`
`Sheet 13 of 13
`
`US 7,656,393 B2
`
`‘
`
`UbINoy
`
`|S|Jepual,
`
`
`
`sumolguosyorr
`
`SA07]
`
`ulSIQAMET
`
`a0ueg
`
`passe
`whgee
`Is0ngey
`sa)s004
`
`0.
`
`
`
`
`
`oulf
`
`lu
`
`DU
`
`lud 04)9807]9)SveIN
`
`18
`
`18
`
`

`

`US 7,656,393 B2
`
`1
`ELECTRONIC DEVICE HAVING DISPLAY
`AND SURROUNDING TOUCH SENSITIVE
`BEZEL FOR USER INTERFACE AND
`CONTROL
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`2
`FIGS. 1A-1F are diagrams of various hand-held electronic
`devices including for example a telephone 10A (FIG. 1A), a
`PDA 10B (FIG.1B), a media player 10C (FIG. 1C), a remote
`control 10D (FIG. 1D), a camera 10E (FIG. 1E), and a Global
`Positioning System (GPS) module 10F (FIG. 1F). In each of
`these devices 10, a display 12, which is secured inside the
`housing of the device 10 and which can be seen through an
`opening in the housing, is typically positioned inafirst region
`of the electronic device 10. Each of these devices 10 also
`This is a continuation-in-part of U.S. patent application
`Ser. No. 11/367,749, filed Mar. 3, 2006 and entitled “Multi-
`Functional Hand-Held Device,” which is incorporated herein
`by referencein its entirety, to which priority is claimed, and
`which claimspriority to: (1) U.S. Provisional Patent Appli-
`cation No. 60/658,777, entitled “Multi-Functional Hand-
`Held Device,” filed Mar. 4, 2005 and (2) U.S. Provisional
`Patent Application No. 60/663,345, entitle titled “Multi-
`Functional Hand-Held Device,”filed Mar. 16, 2005, each of
`whichis hereby incorporated by reference.
`This application is also related to the following applica-
`tions, whichare all herein incorporated by reference: (1) U.S.
`patent application Ser. No. 10/188,182, entitled “Touch Pad
`for Handheld Device,” filed on Jul. 1, 2002; (2) U.S. patent
`application Ser. No. 10/722,948, entitled “Touch Pad for
`Handheld Device,” filed on Nov. 25, 2003; (3) U.S. patent
`application Ser. No. 10/643,256, entitled “Movable Touch
`Pad with Added Functionality,” filed on Aug. 18, 2003; (4)
`USS. patent application Ser. No. 10/654,108, entitled “Ambi-
`dextrous Mouse,”filed on Sep. 2, 2003; (5) U.S. patent appli-
`cation Ser. No. 10/840,862, entitled “Multipoint Touch
`Screen,”filed on May6, 2004; (6) U.S. patent application Ser.
`No. 10/903,964, entitled “Gestures for Touch Sensitive Input
`Devices,” filed on Jul. 30, 2004; (7) U.S. patent application
`Ser. No. 11/038,590, entitled “Mode-Based Graphical User
`Interfaces for Touch Sensitive Input Devices,” filed on Jan.
`18, 2005; and (8) U.S. patent application Ser. No. 11/057,050,
`entitled “Display Actuator,”filed on Feb. 11, 2005, (9) U.S.
`patent application Ser. No. 11/115,539, entitled “Hand-Held
`Electronic Device with Multiple Touch Sensing Devices,”
`filed Apr. 26, 2005.
`
`include one or more input devices 14, which are typically
`positioned in a secondregionof the electronic device 10 next
`to the display 12.
`To elaborate, the telephone 10A typically includes a dis-
`play 12 such as a character or graphical display, and input
`devices 14 such as a numberpad and in somecasesa naviga-
`tion pad. The PDA 10Btypically includes a display 12 such as
`a graphical display, and input devices 14 such as a stylus
`based resistive touch screen and buttons. The media player
`10C typically includes a display 12 such as a character or
`graphic display and input devices 14 such as buttons or
`wheels. The iPod® media player manufactured by Apple
`Computer, Inc. of Cupertino, California is one example of a
`media player that includes both a display and input devices
`disposed next to the display. The remote control 10D typically
`includesan input device 14 such as a keypad and may or may
`not have a character display 12. The camera 10E typically
`includes a display 12 such as a graphic display and input
`devices 14 such as buttons. The GPS module 10F typically
`includes a display 12 such as graphic display and input
`devices 14 such as buttons, and in somecases a joy pad.
`Such prior art devices 10A-10F often employa userinter-
`face in conjunction withthe display 12 and input device 14. In
`one example, FIG. 2A showsan electronic device 20, such as
`a portable media player. The electronic device 20 has a dis-
`play 24 and an input device 26 accordingto the priorart. The
`display 22 can show various forms of information (e.g., menu
`items, songtitles stored in memory,etc.) of a user interface.
`The display 24 and input device 26 used in conjunction with
`the user interface allows the user to make selections (e.g.,
`select a song), to operate functions of the device (e.g., play,
`stop, or pause a song,etc.), and to perform other functions. In
`this device 20, the input devices 26 is a “click wheel,” such as
`used on an iPod® media player manufactured by Apple Com-
`puter, Inc. of Cupertino, Calif.
`The electronic device 20 has a housing 22 that contains the
`display 24 and the input device 26. The input device 26
`typically requires a number of components, such as pressure
`pads, printed circuit board, integrated circuits, etc. Accord-
`ingly, the housing 22 for the electronic device 20 musttypi-
`cally be extended or enlarged beyondthe size of the display
`24 so that the electronic device 20 can accommodate the
`components of the input device 26. Consequently, due to the
`required components for the input device 26, the size of the
`housing 22 may in some cases belarger than is actually
`required to housejust the display 24 and any other necessary
`components(i.e., processor, memory, powersupply, etc.) for
`the device 20. In addition, placement ofthe display 22 and the
`input device 26 typically accommodate only one orientation
`of the device 20 whenheld by a user.
`In another example, FIG. 2B shows another electronic
`device 30 having a display 34 and an input device 36 accord-
`ing to the prior art. The electronic device 30 can be a laptop
`computeror the like, and the input device 36 can be a touch
`pad used to control functionsofthe device 30, such as moving
`a cursor, making selections, etc. The touch pad 36 is posi-
`tioned on a housing 32 of the device 30 in conjunction with
`conventional components ofa keyboard 38 andother physical
`
`20
`
`30
`
`35
`
`40
`
`FIELD OF THE DISCLOSURE
`
`The subject matter of the present disclosure relates to an
`electronic device having a display and a surrounding touch
`sensitive bezel for user interface and control.
`
`45
`
`BACKGROUNDOF THE DISCLOSURE
`
`There exist today many types of hand-held electronic
`devices, each of which utilizes some sort of user interface.
`The user interface typically includes an output device in the
`form of a display, such as a Liquid Crystal Display (LCD),
`and one or more input devices, which can be mechanically
`actuated (e.g., switches, buttons, keys, dials, joysticks, joy
`pads) or electrically activated (e.g., touch pads or touch
`screens). The displayis typically configured to present visual
`information such as text and graphics, and the input devices
`are typically configured to perform operations such as issuing
`commands, making selections, or moving a cursor or selector
`of the electronic device. Each of these well-known devices
`
`has considerations such as size and shape limitations, costs,
`functionality, complexity, etc. that must be taken into account
`when designing the hand-held electronic device. In most
`cases, the user interface is positioned on the front face (or
`front surface) of the hand-held device for easy viewing of the
`display and easy manipulation of the input devices.
`
`50
`
`55
`
`60
`
`65
`
`19
`
`19
`
`

`

`US 7,656,393 B2
`
`3
`inputs. The touch pad 36 can be categorized as either “resis-
`tive” or “capacitive.” In the resistive category, the touch pad
`36 is coated with a thin metallic electrically conductive layer
`and a resistive layer. When the touch pad 36 is touched, the
`conductive layers come into contact through the resistive
`layer causing a change in resistance (typically measured as a
`change in current) that is used to identify where on the touch
`pad 36 the touch event occurred.In the capacitive category, a
`first set of conductive traces run in a first direction on the
`
`touch pad 36 andare insulated by a dielectric insulator from
`a second set of conductive traces running in a second direc-
`tion (generally orthogonalto the first direction) on the touch
`pad 36. The grid formedby the overlapping conductive traces
`creates an array of capacitors that can store electrical charge.
`Whenan object (e.g., a user’s finger) is brought into proxim-
`ity or contact with the touch pad 36, the capacitance of the
`capacitors at that location changes. This change can then be
`usedto identify the location of the touch event.
`In yet another example, FIG. 2C illustrates an electronic
`device 40 having a touch screen display 44 according to the
`prior art as an input device. The electronic device 40 is a
`Personal Digital Assistant or the like. The touch screen dis-
`play 44 is positioned on a housing 42, and the electronic
`device 40 typically has some physical controls 46 on the
`housing 42. A stylus48 is used to touch locationsof the touch
`screen display 44 to perform various functions. The stylus 48
`is typically used like a mouse andarrow,and the display 44
`can show various menu items and other user interface fea-
`
`tures. Touching a menu item on the display 44 with the stylus
`48 can generate a pop-up or window 45 in which the user can
`then make a selection with the stylus 48. The pop-ups or
`windows45 overlay the content being displayed and tend to
`obscureit.
`
`traditionally separate hand-held electronic
`Recently,
`devices have begun to be combined in limited ways. For
`example, the functionalities of a telephone have been com-
`bined with the functionalities of a PDA. One problem that has
`been encountered is in the way inputs are made into the
`device. Each of these devices has a particular set of input
`mechanismsor devices for providing inputs into the device.
`Someofthese input mechanismsare generic to all the devices
`(e.g., power button) while others are not. The onesthat are not
`generic are typically dedicated to a particular functionality of
`the device. By way of example, PDAs typically include a
`touch screen and a few dedicated buttons while cell phones
`typically include a numeric keypad andatleast two dedicated
`buttons.
`
`Thus,it is a challenge to design a device with limited input
`mechanisms without adversely affecting the numerous pos-
`sible functions that the device can perform. As will be appre-
`ciated, it is preferable not to overload the electronic devices
`with a large number of input mechanismsas this tends to
`confuse the user and to take up valuable space, i.e., “real
`estate.” In the case of hand-held devices, space is at a pre-
`mium becauseof their small size. At some point, there is not
`enough space on the device to houseall the necessary buttons
`and switches, etc. This is especially true when considering
`thatall these devices need a display that typically takes up a
`large amount of space on its own. To increase the number of
`input devices beyond somelevel, designers would have to
`decrease the size ofthe display. However, this will often lea