IJSOO7656393B2
`
`(12) Unlted States Patent
`(10) Patent No.:
`US 7,656,393 B2
`
`King et al.
`(45) Date of Patent:
`Feb. 2, 2010
`
`(54) ELECTRONIC DEVICE HAVING DISPLAY
`AND SURROUNDING TOUCH SENSITIVE
`BEZEL FOR USER INTERFACE AND
`
`(75)
`
`CONTROL
`a
`a
`a
`Inventors: Nick King San Jose CA (US)' Duncan
`Kerr, San Francisco, CA (US); Paul
`Herbst, Morgan Hill, CA (US); Steven
`PHOtelllng,SanJose,CA(US)
`(73) Assignee: Apple Inc., Cupertino, CA (US)
`.
`.
`.
`.
`.
`( * ) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U'S'C. 15403) by 588 days.
`(21) Appl. No.: 11/426,078
`
`(22)
`
`(65)
`
`Filed:
`
`Jun. 23, 2006
`_
`_
`_
`Pr10r Publlcatlon Data
`US 2006/0238517 A1
`Oct. 26, 2006
`
`(56)
`
`References Cited
`
`U~S~ PATENT DOCUMENTS
`
`3,333,160 A
`
`7/1967 Gorski
`C t
`d
`-
`( on mue )
`FOREIGN PATENT DOCUMENTS
`1243096
`10/1988
`
`CA
`
`(Continued)
`
`OTHER PUBLICATIONS
`Quantum Research Group, “QT5 10 / QWhee1(TM) Touch Slider IC,”
`copyright 2004-2005, l4-pgs.
`
`(Continued)
`Primary Examinerisumati Lefl<owitz
`Assistant Examiner%harles Hicks
`(74) Attorney, Agent, or FirmiMorrison & Foerster LLP
`
`Related US. Application Data
`
`(57)
`
`ABSTRACT
`
`(63) Continuation-in-part Of application NO’ 11/367,749,
`filed on Mar. 3’ 2006'
`provisional application No. 60/663,345, filed on Mar.
`163 20053 provisional application No. 60/658,777,
`filed on Mar. 4, 2005.
`
`(60)
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 3/041
`(2006.01)
`G09G 5/00
`(2006.01)
`G06F 3/02
`(2006.01)
`G06F 3/045
`(2006.01)
`H04B 1/40
`(2006.01)
`A61M 37/00
`(52) US. Cl.
`....................... 345/173; 345/156; 345/169;
`345/174; 455/77; 604/613
`(58) Field of Classification Search .......... 345/1567178
`See application file for complete search history.
`
`An electronic device has a display and has a touch sensitive
`bezel surrounding the display. Areas on the bezel are desig-
`nated for controls used to operate the electronic device. Visual
`guides corresponding to the controls are displayed on the
`display adjacent the areas of the bezel designated for the
`controls. Touch data is generated by the bezel when a user
`touches an area of the bezel. The device determines which of
`the controls has been selected based on which designated area
`is associated with the touch data from the bezel. The device
`then initiates the determined control. The device can have a
`sensor for determining the orientation ofthe device. Based on
`the orientation, the device can alter the areas designated on
`the bezel for the controls and can alter the location of the
`visual guides for the diSPIay so that they match the altered
`areas 011 the bezel.
`
`45 Claims, 13 Drawing Sheets
`
`920 V
`
`839
`
`
`\
`
`
`
`
`APPLE 1018
`
`APPLE 1018
`
`1
`
`

`

`US 7,656,393 B2
`
`Page2
`
`U.S. PATENT DOCUMENTS
`
`3,541,541 A
`3,662,105 A
`3798370 A
`4,246,452 A
`4,550,221 A
`4,566,001 A
`4,672,364 A
`4,672,558 A
`4,692,809 A
`4,695,827 A
`4733222 A
`4,734,685 A
`4,746,770 A
`4771276 A
`4,788,384 A
`4,806,846 A
`4,896,370 A
`4,898,555 A
`4,933,660 A
`4,968,877 A
`5003519 A
`5017030 A
`5,178,477 A
`5,189,403 A
`5,194,862 A
`5,224,861 A
`5,241,308 A
`5,252,951 A
`5,281,966 A
`5,305,017 A
`5345543 A
`5376948 A
`5,379,057 A
`5,398,310 A
`5,442,742 A
`5,463,388 A
`5,463,696 A
`5483261 A
`5488204 A
`5,495,077 A
`5513309 A
`5,523,775 A
`5,530,455 A
`5,543,590 A
`5,543,591 A
`5,563,632 A
`5,563,996 A
`5,565,658 A
`5,579,036 A
`5,581,681 A
`5,583,946 A
`5590219 A
`5592566 A
`5,594,471 A
`5,594,810 A
`5,596,694 A
`5612719 A
`5,631,805 A
`5,633,955 A
`5,634,102 A
`5,636,101 A
`5,642,108 A
`5,644,657 A
`5,666,113 A
`5666502 A
`5,666,552 A
`5,675,361 A
`5,675,362 A
`5,677,710 A
`5,689,253 A
`5,710,844 A
`
`............. 455/77
`
`11/1970 Englebart
`5/1972 Hurstet a1.
`3/1974 Hurst
`1/1981 Chandler
`10/1985 Mabusth
`1/1986 Mooreetal.
`6/1987 Lucas
`6/1987 Beckes etal.
`9/1987 Beining etal.
`9/1987 Beining etal~
`3/1988 Evans
`3/1988 Watanabe
`5/1988 MoAvinney
`9/1988 Parks
`11/1988 Bruere-Dawsonet a1.
`2/1989 Kerber
`1/1990 Kasparianetal.
`2/1990 Sarnpson
`6/1990 WynneaJr
`11/1990 MoAvinneyetal~
`3/1991 Noijean
`5/1991 Crews
`1/1993 Gambaro
`2/1993 Franz etal.
`3/1993 Edwards
`7/1993 Glass et a1.
`8/1993 Young
`10/1993 Tannenbaumet a1.
`1/1994 Walsh
`4/1994 Gerpheide
`9/1994 CappsetaL
`12/1994 Roberts
`1/1995 Clough et a1.
`3/1995 Tchao etal.
`8/1995 Greyson et a1.
`10/1995 Boie etal~
`10/1995 Beernink etal~
`1/1996 Yasutake
`1/1996 Mead etal.
`2/1996 Miller et al~
`4/1996 Meier etal.
`6/1996 Capps
`6/1996 Gillick et a1.
`8/1996 Gillespie et a1.
`8/1996 GilleSPie etal.
`10/1996 Roberts
`10/1996 Tohao
`10/1996 Gerpheide et a1.
`11/1996 Yates, IV
`12/1996 Tchao et a1.
`12/1996 Gourdol
`12/1996 Gourdol
`1/1997 Pagallo etal~
`1/1997 Deeran et a1.
`1/1997 Gourdol
`1/1997 Capps
`3/1997 Beernink etal~
`5/1997 Bonsall
`5/1997 Bozinovic et a1.
`5/1997 Capps
`6/1997 Bonsalletal.
`6/1997 Gopher et a1.
`7/1997 CappsetaL
`9/1997 Logan
`9/1997 Capps
`9/1997 Greyson etal.
`10/1997 Santilli
`10/1997 Clough etal.
`10/1997 Thompson-Rohrlich
`11/1997 Hargreaves et a1.
`1/1998 Capps et a1.
`
`5,729,250 A
`5,730,165 A
`5,736,976 A
`5,741,990 A
`5,745,116 A
`5,745,716 A
`5,748,269 A
`5,764,818 A
`5,767,457 A
`5,767,842 A
`5,790,104 A
`5,790,107 A
`5,802,516 A
`5,808,567 A
`5,809,267 A
`5,821,690 A
`5,821,930 A
`5,823,782 A
`5,825,351 A
`5,825,352 A
`5,854,625 A
`5,880,411 A
`5,898,434 A
`5,920,309 A
`5,923,319 A
`5,933,134 A
`5,943,044 A
`6,002,389 A
`6,002,808 A
`6,020,881 A
`6,031,524 A
`6,037,882 A
`6,050,825 A
`6,052,339 A
`6,072,494 A
`6,084,576 A
`6,107,997 A
`6,128,003 A
`6,131,047 A
`6,131,299 A
`6,135,958 A
`6,144,380 A
`6,188,391 B1
`6,198,515 B1
`6,208,329 B1
`6,222,465 B1
`6,239,790 B1
`6,243,071 B1
`6,246,862 B1
`6,249,606 B1
`6,288,707 B1
`6,289,326 B1
`6,292,178 B1
`6,297,811 B1
`6,323,846 B1
`6,347,290 B1
`6,377,009 B1
`6,380,391 B2
`6,411,287 B1
`6,413,233 B1
`6,414,671 B1
`6,421,234 B1
`6,452,514 B1
`6,457,355 B1
`6,466,036 B1
`6,466,203 B2
`6,515,669 B1
`6,525,749 B1
`6,535,200 132
`6,543,684 B1
`6,543,947 B2
`6,570,557 B1
`6,593,916 B1
`
`3/1998 Bishop et a1.
`3/1998 Philipp
`4/1998 Cheung
`4/1998 Davies
`4/1998 Pisutha-Arnond
`4/1998 Tchao etal.
`5/1998 Harris etal.
`6/1998 Cappsetal.
`6/1998 Gerpheide et a1.
`6/1998 Korth
`8/1998 Shieh
`8/1998 Kasser et a1.
`9/1998 Shwartsetal.
`9/1998 McCloud
`9/1998 Moran et a1.
`10/1998 Manensetal.
`10/1998 Hansen
`10/1998 Marcus et a1.
`10/1998 Tam
`10/1998 Bisset et a1.
`12/1998 Frisch et a1.
`3/1999 Gillespie etal.
`4/1999 Small et a1.
`7/1999 Bisset et a1.
`7/1999 Bishop et a1.
`8/1999 Shieh
`8/1999 Martinellietal.
`12/1999 Kasser
`12/1999 Freeman
`2/2000 Naughton et 31.
`2/2000 Kunert
`3/2000 Levy
`4/2000 Nichol et a1.
`4/2000 Frenkel et a1.
`6/2000 Nguyen
`7/2000 Leuetal.
`8/2000 Ure
`10/2000 Smith etal.
`10/2000 Hayes, Jr. et a1.
`10/2000 Raab et a1.
`10/2000 Mikula-Curtis et a1.
`11/2000 Shwarts et a1.
`2/2001 Seelyetal.
`3/2001 Cole
`3/2001 Ballare
`4/2001 Kumar et 31,
`5/2001 Martinellietal.
`6/2001 Shwartsetal.
`6/2001 Grivasetal.
`6/2001 Kiraly et 31,
`9/2001 Philipp
`9/2001 LaFleur
`9/2001 Bernstein et a1.
`
`10/2001 Kentetal.
`11/2001 Westerman et a1.
`2/2002 Bartlett
`4/2002 Philipp
`4/2002 Gillespie etal.
`6/2002 Scharffet a1.
`
`........ 604/6.13
`7/2002 Sites et a1.
`7/2002 Gillespie et 31.
`7/2002 Ricks et a1.
`9/2002 Philipp
`10/2002 Philipp
`10/2002 Philipp
`10/2002 Van Be
`2/2003 Mohri
`2/2003 Moran et a1.
`3/2003 Philipp
`4/2003 White et a1.
`4/2003 Lee
`5/2003 Westerman et a1.
`7/2003 Aroyan
`
`...... 345/173
`
`2
`
`

`

`US 7,656,393 B2
`
`Page 3
`
`6,597,384 B1
`6,610,936 B2
`6,624,833 B1
`6,639,577 B2
`6,650,319 B1
`6,658,994 B1
`6,670,894 B2
`6,677,932 B1
`6,677,934 B1
`6,724,366 B2
`6,757,002 B1
`D493,157 S
`D493,158 S
`D497,606 S
`6,803,906 B1
`D500,298 S
`6,842,672 B1
`6,888,532 B2
`6,888,536 B2
`6,900,795 B1
`6,909,424 B2
`6,927,761 B2
`D509,819 S
`13509333 S
`D510,081 S
`6,942,571 B1
`D511,512 s
`D511,528 s
`6,965,375 B1
`D512,403 s
`D512,435 S
`6,972,401 B2
`6,977,666 B1
`6,985,801 B1
`6,992,659 B2
`7,031,228 B2
`D520,516 S
`7,109,978 B2
`7,129,416 B1
`7,145,552 B2
`7,170,496 B2
`7,240,289 B2
`RE40,153 E
`2001/0043189 A1
`2882/83)ng4918 :1
`3883;895:3347! :1
`2003/0076301 A1
`2003/0076303 A1
`2003/0076306 A1
`2003/0095095 A1
`2003/0095096 A1
`2003/0098858 A1
`2003/0206202 A1
`2003/0234768 A1
`2004/0012572 A1
`2004/0263484 A1
`2005/0012723 A1
`2005/0052425 A1
`2005/0104867 A1
`2005/0110768 A 1
`2005/0146509 A1
`2005/0154798 A1
`2005/0212760 A1
`2005/0219228 A1
`2006/0022955 A1
`2006/0022956 A1
`2006/0026521 A1
`2006/0026535 A1
`2006/0026536 A1
`2006/0032680 A1
`2006/0033724 A1
`
`........... 345/169
`
`............ 345/173
`
`7/2003 Harrison
`8/2003 Gillespie et al.
`9/2003 Kumar et a1.
`10/2003 Eberhand
`11/2003 Hurst et a1.
`12/2003 McMillan
`12/2003 Mehring
`1/2004 Westerman
`1/2004 Blanchard
`4/2004 Crawford
`6/2004 Oross et al.
`7/2004 Yang
`7/2004 Yang
`10/2004 Yang
`10/2004 Morrison et a1.
`12/2004 Yang
`1/2005 Straub et a1.
`................. 345/156
`5/2005 Wong et al.
`5/2005 Westerman et al.
`5/2005 Knight, III et al.
`6/2005 Liebenow et al.
`8/2005 Badaye et al.
`9/2005 Yang
`9/2005 Yang
`9/2005 Yang
`9/2005 McAllister et al.
`11/2005 Yang
`11/2005 Yang
`11/2005 Gettemy etal.
`12/2005 Yang
`12/2005 Yang
`12/2005 Akitt etal.
`12/2005 Hedrick
`1/2006 Straub etal.
`1/2006 Gettemy
`4/2006 Born et al.
`5/2006 Yang
`9/2006 Gillespie et al.
`10/2006 Steinfeld et al.
`12/2006 Hollingsworth ............. 345/168
`1/2007 Middleton
`7/2007 Naughton
`3/2008 Westerman et al.
`11/2001 Briseboisetal.
`$388; Rigginstein
`All/388:3; 8206;321:2351.
`4/2003 Tsuk et al.
`4/2003 Huppi
`4/2003 Zadesky et al.
`5/2003 Pihlaja
`5/2003 Robbin et 31.
`5/2003 Perski et al.
`“/2003 Moriya
`12/2003 Rekimoto et al.
`“2004 Sowden et al.
`12/2004 Mantysalo et 31.
`1/2005 Pallakoff
`3/2005 Zadesky et 31.
`5/2005 Westerman et 31.
`5/200 5 Marriott et a1.
`7/2005 Geaghan et 31,
`7/2005 Nurmi
`9/2005 Marvit etal.
`10/2005 Alameh
`2/2006 Kennedy
`2/2006 Lengeling et al.
`2/2006 Hotelling et al.
`2/2006 Hotelling et al.
`2/2006 Hotelling et a1.
`2/2006 Elias et a1.
`2/2006 Chaudhri et a1.
`
`,,,,,,,,,,,,, 345/173
`
`2006/0044259 A1
`2006/0053387 A1
`2006/0066582 A1
`2006/0079969 A1
`2006/0085757 A1
`2006/0097991 A1
`2006/0197753 A1
`2006/0232567 A1
`2006/0238518 A1
`2006/0238519 A1
`2006/0238520 A1
`2006/0238521 A1
`2006/0238522 A1
`2007/0229464 A1
`2007/0236466 A1
`2007/0247429 A1
`2007/0257890 A1
`2008/0088602 A1
`2008/0238879 A1
`2008/0297476 A1
`
`3/2006 Hotelling et al.
`3/2006 Ording
`3/2006 Lyon et al.
`4/2006 Seguin
`4/2006 Andre et a1.
`5/2006 Hotelling et al.
`9/2006 Hotelling
`10/2006 Westerman et al.
`10/2006 Westerman et al.
`10/2006 Westerman et al.
`10/2006 Westerman et al.
`10/2006 Westerman et al.
`10/2006 Westerman et al.
`10/2007 Hotelling et al.
`10/2007 Hotelling
`10/2007 Westerman
`11/2007 Hotelling et al.
`4/2008 Hotelling
`10/2008 Jaeger et al.
`12/2008 Hotelling et al.
`
`FOREIGN PATENT DOCUMENTS
`
`DE
`EP
`EP
`EP
`EP
`EP
`GB
`JP
`W0
`W0
`W0
`WO
`WO
`WO
`W0
`W0
`W0
`WO
`WO
`
`10251296 A1
`0 288 692
`0 464 908
`0 664 504
`1014 295
`1505 484 A1
`2393 688 A
`10-228350 A
`W09718547 A1
`WOW/023738
`WO'98/14863
`W003088176 A1
`WO-2004/111816 A2
`WO-2004/111816 A3
`W0'2006/023569
`W0'2006/094308 AZ
`WO'2006/094308 A3
`WO-2007/103631 A2
`WO-2007/103631 A3
`
`5/2004
`11/1988
`“1992
`7/1995
`“2002
`”005
`“004
`8/1998
`5/1997
`7/1997
`“998
`10/2003
`12/2004
`12/2004
`”006
`”006
`”006
`9/2007
`9/2007
`
`OTHER PUBLICATIONS
`gionpgi‘fiziggfsfijSeries SummaryData Sheet,” copyright 2005, dated
`ct.
`,
`,
`-pgs.
`EBV Elektronik, “TSOP6238 IR Reciever Modules for Infrared
`Remote Control Systems,” dated Jan. 2004, 1-pg.
`Texas Instruments, “TSC2003 / 12C Touch Screen Controller,” Data
`Sheet SEAS 162: d‘ited 0“ 2001220'Pgs'
`.
`European Exam1nat10n Report ma11ed Apr. 21, 2008, for EP App11-
`catlon No. 067375154, filed Mar. 3, 2006, five pages.
`International SearchReportmailedOct.6,2006,forPCTApplication
`No. PCT/US2006/08349, filed Mar. 3, 2006, three pages.
`International SearchReportmailedJan.3,2008, forPCTApplication
`No. PCT/US2007/062474, filed Feb. 21, 2007, three pages.
`Press Release, “iriver clix Delivers Complete Package for Portable
`Entertainment Fans,” obtained from w. iriveramerica.com/im-
`ages/pdf/iriviclixpdf, dated May 17, 2006, 3-pages.
`iriver clix Product Guide, copyright 1999-2006, 38-pages.
`iriver clix Quick Start Guide, undated, 2-pages.
`Gear Live Review: iRiver clix Review, obtained from http://www.
`gearlive.com/index.php/news/article/gear-live-review-iriver-cliX-re-
`view-7134000 dated Jul. 13, 2006, 8-pages
`Anonymous. “Touch Technologies Overview” 2001, 3M Touch Sys-
`tems, Massachusetts.
`Anonymous. “4-Wire Resistive Touchscreens” obtained from http://
`www.touchscreens.com/intro-touchtypes-4resistive.html generated
`Aug. 5, 2005.
`Anonymous. “5-Wire Resistive Touchscreens” obtained from http://
`www.touchscreens.com/intro-touchtypes-resistive.html
`generated
`Aug. 5, 2005.
`
`3
`
`

`

`US 7,656,393 B2
`
`Page 4
`
`Anonymous. “A Brief Overview of Gesture Recognition” obtained
`from http://www. Dai. Ed. Ac.uk/Cvonline/LOCAiCOPIES/
`COHEN/gestureioverview. Html, generated Apr. 20, 2004.
`Anonymous. “Capacitive Position Sensing” obtained from http://
`www.synaptics.com/technology/cps.cfin generated Aug. 5, 2005.
`Anonymous. “Capacitive Touchscreens” obtained from http://www.
`touchscreens.com/intro-touchtypes-capacitive.html generated Aug.
`5, 2005.
`Anonymous. “Comparing Touch Technologies” obtained from http://
`www.touchscreens.com/intro-touchtypeshtml generated Oct. 10,
`2004.
`Anonymous. “FingerWorks4Gesture GuideiApplication Switch-
`ing,”
`obtained
`from
`http://www.fingerworks.com/
`gestureiguideiappshtml, generated on Aug. 27, 2004, 1-pg.
`Anonymous. “FingerWorks4Gesture GuideiEditing,” obtained
`from
`http: //www.fingerworks .com/gesureiguideiediting .htrnl,
`generated on Aug. 27, 2004, 1-pg.
`Anonymous. “FingerWorks4Gesture GuideiFile Operations,”
`obtained from http://www.fingerworks.com/gestureiguideifiles.
`html, generated on Aug. 27, 2004, 1-pg.
`Anonymous. “FingerWorksiGesture GuideiText Manipulation,”
`obtained
`from
`http://www.fingerworks.com/
`gestureiguideitextimaniphtml, generated on Aug. 27, 2004, 2-pg.
`Anonymous. “FingerWorksiGesture GuideiTips and Tricks,”
`obtained from http://www.fingerworks.com/gestureiguideitips.
`html, generated Aug. 27, 2004, 2-pgs.
`Anonymous. “FingerWorksiGesture GuideiWeb,” obtained from
`http://www.fingerworks.com/gestureiguideiwebhtml, generated
`on Aug. 27, 2004, 1-pg.
`Anonymous. “FingerWorks4Guide to Hand Gestures for USB
`Touchpads,”
`obtained
`from
`http://www.fingerworks.com/
`igestureiuserguidehtml, generated Aug. 27, 2004, 1-pg.
`Details,”
`Anonymous.
`"FingerWorksiiGestureiTechnical
`obtained from http://www.fingerworks.com/igestureitechhtml,
`generated Aug. 27, 2004, 1-pg.
`Anonymous. “FingerWorksiThe Only Touchpads with Ergonomic
`Full-Hand Resting and Relaxation!” obtained from http://www.
`fingerworks.com/restinghtml, Copyright 2001, 1-pg.
`Anonymous. “FingerWorksiTips for Typing on the Mini,” obtained
`from http://www.fingerworks.com/miniitypinghtml, generated on
`Aug. 27, 2004, 2-pgs.
`Anonymous. “G1idePoint®” obtained from http://www.cirque.com/
`technology/technologyiqphtml generated Aug. 5, 2005.
`Anonymous. “Gesture Recognition” http://www.fingerworks.com/
`gestureirecognition.htm1>, Jul. 2, 2006.
`Anonymous. “How do touchscreen monitors know where you’re
`touching?” obtained from http://www.electronics.howstuffworks.
`com/question716.html generated Aug. 5, 2005.
`Anonymous. “How does a touchscreen work?” obtained from http://
`www.touchscreens.com/intro-anatomy.html generated Aug. 5, 2005.
`Anonymous. “iGesture Padithe MultiFinger USB TouchPad with
`Whole-Hand Gestures,”obtained from http://www.fingerworks.com/
`igesture.html, generated Aug. 27, 2004, 2-pgs.
`Anonymous. “iGesture Products for Everyone (learn in minutes)
`Product Overview”FingerWorks.com downloaded Aug. 30, 2005.
`Anonymous. “Infrared Touchscreens” obtained from http://www.
`touchscreens.com/intro-touchtypes-infrared.html generated Aug. 5,
`2005.
`
`Anonymous. “Mouse Emulation” FingerWorks obtained from http://
`www.fingerworks.com/gestureiguideimousehtml generated Aug.
`30, 2005.
`Anonymous. “Mouse Gestures in Opera” obtained from http://www.
`opera.com/products/desktop/mouse/index.dml generated Aug. 30,
`2005.
`
`Anonymous. “Mouse Gestures,” Optim 02, May 21, 2004.
`Anonymous. “MultiTouch Overview” FingerWorks obtained from
`http://finger.works.com/multoverview.html
`generated Aug.
`30,
`2005.
`
`Anonymous. “Near Field Imaging Touchscreens” obtained from
`http://www.touchscreens.com/intro-touchtypes-nfi.htrnl generated
`Aug. 5, 2005.
`
`Anonymous. “PenTouch Capacitive Touchscreens” obtained from
`http://www.touchscreens.com/intro-touchtypes-pentouch.html gen-
`erated Aug. 5, 2005.
`Anonymous. “Surface Acoustic Wave Touchscreens” obtained from
`http://www.touchscreens.com/intro-touchtypes-sawhtml generated
`Aug. 5, 2005.
`Anonymous. “Symbol Commander” obtained from http://www.
`sensiva.com/symbolcommander/, generated Aug. 30, 2005.
`Anonymous.
`“Tips
`for Typing”
`FingerWorks
`http://www.
`fingerworks.com/miniitypinghtml generated Aug. 30, 2005.
`Anonymous. “Wacom ComponentsiTechnology” obtained from
`http://www.wacom-components.com/english/techasp generated on
`Oct. 10,2004.
`“Watershed Algorithm” http://rsb.info.nih.gov/ij/
`Anonymous.
`plugins/watershedhtml generated Aug. 5, 2005.
`Bier et al., “Toolglass and Magic Lenses: The see-through interface”
`In James Kijiya, editor, Computer Graphics (SIGGRAPH ’93 Pro-
`ceedings), vol. 27, pp. 73-80, Aug. 1993.
`Douglas et al., The Ergonomics of Computer Pointing Devices
`(1997).
`European Search Report received in EP 1 621 989 (@ Beyer Weaver
`& Thomas, LLP) dated Mar. 27, 2006.
`Fisher et al., “Repetitive Motion Disorders: The Design of Optimal
`Rate- Rest Profiles,” Human Factors, 35(2):283-304 (Jun. 1993).
`Fukumoto and Yoshinobu Tonomura, “Body Coupled Fingering:
`Wireless Wearable Keyboard,” CHI97, pp. 147-154 (Mar. 1997).
`Fukumoto et al., “ActiveClick: Tactile Feedback for Touch Panels,”
`In CHI 2001 Summary, pp. 121-122, 2001.
`Hardy, “Fingerworks” Mar. 7, 2002; BBC World On Line.
`Hillier and Gerald J. Lieberman,
`Introduction to Operations
`Research (1986).
`International Search Report dated Mar. 3, 2006 (PCT/US 05/03325;
`119-0052WO).
`Jacob et al., “Integrality and Separability of Input Devices,” ACM
`Transactions on Computer—Human Interaction, 1:3-26 (Mar. 1994).
`Kinkley et al., “Touch-Sensing Input Devices,” in CHI ’99 Proceed-
`ings, pp. 223-230, 1999.
`Lee, “A Fast Multiple-Touch-Sensitive Input Device,” Master’s The-
`sis, University of Toronto (1984).
`Lee, S.K. et al., “A Multi-Touch Three Dimensional Touch-Sensitive
`Tablet,” in CHI ’85 Proceedings, pp. 121-128, 2000 [(Apr. 1985).
`Conference Proceedings: Human Factors in Computing Systems, pp.
`21-25 1.]
`Matsushita et al., “HoloWall: Designing a Finger, Hand, Body and
`Object Sensitive Wall,” In Proceedings ofUIST ’97, Oct. 1997.
`Non-Final Office Action mailed Feb. 26, 2009, for US. Appl. No.
`11/367,749, filed Mar. 3, 2006, 10 pages.
`Quek, “Unencumbered Gestural Interaction,” IEEE Multimedia,
`3:36-47 (Winter 1996).
`Radwin, “Activation Force and Travel Effects on Overexertion in
`Repetitive Key Tapping,” Human Factors, 39(1): 130-140 (Mar.
`1997).
`Rekimoto et al., “ToolStone: Effective Use of the Physical Manipu-
`lation Vocabularies of Input Devices,” In Proc. Of UIST 2000.
`Rekimoto, J. (2002). “SmartSkin: An Infrastructure for Freehand
`Manipulation on Interactive Surfaces,” CH12002, Apr. 20-25, 2002.
`[(Apr. 25, 2002). 4(1):113-120.]
`Rubine et al., “Programmable Finger-Tracking Instrument Control-
`lers,” Computer Music Journal, vol. 14, No. 1 (Spring 1990).
`Rutledge et al., “Force-To-Motion Functions for Pointing,” Human-
`Computer InteractioniINTERACT (1990).
`Subatai Ahmad, “A Usable Real-Time 3D Hand Tracker,” Proceed-
`ings of the 28’h Asilomar Conference on Signals, Systems and
`ComputersiPart 2 (of2), vol. 2 (Oct. 1994).
`US. Appl. No. 10/789,676, filed Feb. 27, 2004 entitled “Shape
`Detecting Input Device”.
`U.S. Appl. No. 11/332,861, filed Jan. 13, 2006, which is a Reissue of
`6,677,932.
`Wellner, “The Digital Desk Calculators: Tangible Manipulation on a
`Desk Top Display”In ACM UIST ’91 Proceedings, pp. 27-34, Nov.
`1991.
`
`4
`
`4
`
`

`

`US 7,656,393 B2
`Page 5
`
`Williams, “Applications for a Switched-Capacitor Instrumentation
`Building Block” Linear Technology Application Note 3, Jul. 1985,
`pp. 1-16.
`Yamada et al., “A Switched-Capacitor Interface for Capacitive Pres-
`sure Sensors” IEEE Transactions on Instrumentation and Measure-
`ment, vol. 41, No. 1, Feb. 1992, pp. 81-86.
`Yeh et al., “Switched Capacitor Interface Circuit for Capacitive
`Transducers” 1985 IEEE.
`
`Zhai et al., “Dual Stream Input for Pointing and Scrolling,” Proceed—
`ings ofCHI’ 97 Extended Abstracts (1997).
`Zimmerman et al., “Applying Electric Field Sensing to Human-
`Computer Interfaces,” In CHI ’85 Proceedings, pp. 280-287, 1995.
`Final Office Action mailed Aug. 25, 2009, for US. Appl. No.
`11/367,749, filed Mar. 3, 2006, 12 pages.
`
`* cited by examiner
`
`5
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet10f13
`
`US 7,656,393 B2
`
`
`
`6
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet20f13
`
`US 7,656,393 B2
`
`
`
`7
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet30f13
`
`US 7,656,393 B2
`
`
`
`8
`
`

`

`US. Patent
`
`Feb. 2, 2010
`
`Sheet 4 of 13
`
`US 7,656,393 B2
`
`Cfiiflfifi fiflx
`
`..
`
`, n: ................
`,
`jj§
`i
`:3;
`
`MUM
`55am
`
`EERESEFRY
`
`:
`
`.
`
`.
`
`I
`
`.
`
`.
`
`USER APP
`
`S‘fSEEMAPP
`
`>
`
`:
`
`9
`
`

`

`US. Patent
`
`Feb. 2, 2010
`
`Sheet 5 of 13
`
`US 7,656,393 B2
`
`
`
`BfiflMfifiEflGfi
`BEEEL ESE Cfi§§RQLS
`
`fl
`
`GENERAYE ViSSAL GUEEES
`
`BiSPifiaY VBSUAE. GUEEES
`
`§§i
`
`,,,,,,,,,,,,,,,,,,9§HFQ¥$3HEEZ§L,,,,,,,,,,,,,,,,,,
`
`i: :
`5%
`,,,,,,,,,3§é
`
`;
`§
`45$,
`
`GEEfiN EGGCE
`5A1a ERGM BEZEi
`,
`
`§
`5
`
`§
`g
`
`AfijéiEi‘é? EESEQMFEE} AREAS
`
`.ifiigfifig...’
`é?
`
`CGWfiRE E'fiijifi mm 3%
`
`EE§EGNAY§QNS 6&5 BEZEE.
`
`BE’EERMENE CGMREEE,
`
`Ei‘iPHflHEfl EY BESEGNAEEQE
`
`ififfiAfE EEYERMENEE
`cafifaaL
`
`PEQCESS
`
`§3§€E§,
`
`10
`
`10
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet60f13
`
`US 7,656,393 B2
`
`nun“
`
`RQSgESM«U.
`
`:3.
`
`11
`
`11
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet70f13
`
`US 7,656,393 B2
`
`5172 '
`
`
`12
`
`12
`
`
`

`

`US. Patent
`
`Feb. 2, 2010
`
`Sheet 8 of 13
`
`US 7,656,393 B2
`
`
`
`
`
`
`
`
`
`13
`
`13
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet90f13
`
`US 7,656,393 132
`
`
`.....II-IIIII“““IIIIIIIIII
`
`
`
`
`mmmrmmmmk
`
`
`
`IL
`
`2%
`
`14
`
`14
`
`
`

`

`US. Patent
`
`Feb. 2, 2010
`
`Sheet 10 of 13
`
`US 7,656,393 B2
`
`
`
`15
`
`15
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet11,0f13
`
`US 7,656,393 B2
`
`
`
`16
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet120f13
`
`US 7,656,393 B2
`
`
`
`17
`
`17
`
`

`

`U.S. Patent
`
`Feb.2,2010
`
`Sheet130f13
`
`US 7,656,393 B2
`
`
`
`$chSag”.33
`
`
`
`53me
`
`5
`mam
`
`mam
`
`Eu
`
`"Em
`a
`
`8wmmi£2Mag
`
`2%mgmmEgg
`
`gagEgg,
`
`$3gwfiafi
`
`18
`
`18
`
`

`

`US 7,656,393 B2
`
`1
`ELECTRONIC DEVICE HAVING DISPLAY
`AND SURROUNDING TOUCH SENSITIVE
`BEZEL FOR USER INTERFACE AND
`CONTROL
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This is a continuation-in-part of US. patent application
`Ser. No. 11/367,749, filed Mar. 3, 2006 and entitled “Multi-
`Functional Hand-Held Device,” which is incorporated herein
`by reference in its entirety, to which priority is claimed, and
`which claims priority to: (1) US. Provisional Patent Appli-
`cation No. 60/658,777, entitled “Multi-Functional Hand-
`Held Device,” filed Mar. 4, 2005 and (2) US. Provisional
`Patent Application No. 60/663,345, entitle titled “Multi-
`Functional Hand-Held Device,” filed Mar. 16, 2005, each of
`which is hereby incorporated by reference.
`This application is also related to the following applica-
`tions, which are all herein incorporated by reference: (1) US.
`patent application Ser. No. 10/188,182, entitled “Touch Pad
`for Handheld Device,” filed on Jul. 1, 2002; (2) US. patent
`application Ser. No. 10/722,948, entitled “Touch Pad for
`Handheld Device,” filed on Nov. 25, 2003; (3) US. patent
`application Ser. No. 10/643,256, entitled “Movable Touch
`Pad with Added Functionality,” filed on Aug. 18, 2003; (4)
`US. patent application Ser. No. 10/654,108, entitled “Ambi-
`dextrous Mouse,” filed on Sep. 2, 2003; (5) US. patent appli-
`cation Ser. No. 10/840,862, entitled “Multipoint Touch
`Screen,” filed on May 6, 2004; (6) US. patent application Ser.
`No. 10/903,964, entitled “Gestures for Touch Sensitive Input
`Devices,” filed on Jul. 30, 2004; (7) US. patent application
`Ser. No. 11/038,590, entitled “Mode-Based Graphical User
`Interfaces for Touch Sensitive Input Devices,” filed on Jan.
`18, 2005; and (8) US. patent application Ser. No. 11/057,050,
`entitled “Display Actuator,” filed on Feb. 11, 2005, (9) US.
`patent application Ser. No. 1 1/1 15,539, entitled “Hand-Held
`Electronic Device with Multiple Touch Sensing Devices,”
`filed Apr. 26, 2005.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`FIELD OF THE DISCLOSURE
`
`The subject matter of the present disclosure relates to an
`electronic device having a display and a surrounding touch
`sensitive bezel for user interface and control.
`
`45
`
`BACKGROUND OF THE DISCLOSURE
`
`There exist today many types of hand-held electronic
`devices, each of which utilizes some sort of user interface.
`The user interface typically includes an output device in the
`form of a display, such as a Liquid Crystal Display (LCD),
`and one or more input devices, which can be mechanically
`actuated (e.g., switches, buttons, keys, dials, joysticks, joy
`pads) or electrically activated (e.g., touch pads or touch
`screens). The display is typically configured to present visual
`information such as text and graphics, and the input devices
`are typically configured to perform operations such as issuing
`commands, making selections, or moving a cursor or selector
`of the electronic device. Each of these well-known devices
`
`has considerations such as size and shape limitations, costs,
`functionality, complexity, etc. that must be taken into account
`when designing the hand-held electronic device. In most
`cases, the user interface is positioned on the front face (or
`front surface) of the hand-held device for easy viewing of the
`display and easy manipulation of the input devices.
`
`50
`
`55
`
`60
`
`65
`
`19
`
`2
`
`FIGS. 1A-1F are diagrams of various hand-held electronic
`devices including for example a telephone 10A (FIG. 1A), a
`PDA 10B (FIG. 1B), a media player 10C (FIG. 1C), a remote
`control 10D (FIG. 1D), a camera 10E (FIG. 1E), and a Global
`Positioning System (GPS) module 10F (FIG. 1F). In each of
`these devices 10, a display 12, which is secured inside the
`housing of the device 10 and which can be seen through an
`opening in the housing, is typically positioned in a first region
`of the electronic device 10. Each of these devices 10 also
`
`include one or more input devices 14, which are typically
`positioned in a second region of the electronic device 10 next
`to the display 12.
`To elaborate, the telephone 10A typically includes a dis-
`play 12 such as a character or graphical display, and input
`devices 14 such as a number pad and in some cases a naviga-
`tion pad. The PDA 10B typically includes a display 12 such as
`a graphical display, and input devices 14 such as a stylus
`based resistive touch screen and buttons. The media player
`10C typically includes a display 12 such as a character or
`graphic display and input devices 14 such as buttons or
`wheels. The iPod® media player manufactured by Apple
`Computer, Inc. of Cupertino, California is one example of a
`media player that includes both a display and input devices
`disposed next to the display. The remote control 1 0D typically
`includes an input device 14 such as a keypad and may or may
`not have a character display 12. The camera 10E typically
`includes a display 12 such as a graphic display and input
`devices 14 such as buttons. The GPS module 10F typically
`includes a display 12 such as graphic display and input
`devices 14 such as buttons, and in some cases a joy pad.
`Such prior art devices 10A-10F often employ a user inter-
`face in conjunction with the display 12 and input device 14. In
`one example, FIG. 2A shows an electronic device 20, such as
`a portable media player. The electronic device 20 has a dis-
`play 24 and an input device 26 according to the prior art. The
`display 22 can show various forms of information (e.g., menu
`items, song titles stored in memory, etc.) of a user interface.
`The display 24 and input device 26 used in conjunction with
`the user interface allows the user to make selections (e.g.,
`select a song), to operate functions of the device (e.g., play,
`stop, or pause a song, etc.), and to perform other functions. In
`this device 20, the input devices 26 is a “click wheel,” such as
`used on an iPod® media player manufactured by Apple Com-
`puter, Inc. of Cupertino, Calif.
`The electronic device 20 has a housing 22 that contains the
`display 24 and the input device 26. The input device 26
`typically requires a number of components, such as pressure
`pads, printed circuit board, integrated circuits, etc. Accord-
`ingly, the housing 22 for the electronic device 20 must typi-
`cally be extended or enlarged beyond the size of the display
`24 so that the electronic device 20 can accommodate the
`
`components of the input device 26. Consequently, due to the
`required components for the input device 26, the size of the
`housing 22 may in some cases be larger than is actually
`required to house just the display 24 and any other necessary
`components (i.e., processor, memory, power supply, etc.) for
`the device 20. In addition, placement ofthe display 22 and the
`input device 26 typically accommodate only one orientation
`of the device 20 when held by a user.
`In another example, FIG. 2B shows another electronic
`device 30 having a display 34 and an input device 36 accord-
`ing to the prior art. The electronic device 30 can be a laptop
`computer or the like, and the input device 36 can be a touch
`pad used to control functions ofthe device 30, such as moving
`a cursor, making selections, etc. The touch pad 36 is posi-
`tioned on a housing 32 of the device 30 in conjunction with
`conventional components ofa keyboard 38 and other physical
`
`19
`
`

`

`US 7,656,393 B2
`
`3
`inputs. The touch pad 36 can be categorized as either “resis-
`tive” or “capacitive.” In the resistive category, the touch pad
`36 is coated with a thin metallic electrically conductive layer
`and a resistive layer. When the touch pad 36 is touched, the
`conductive layers come into contact through the resistive
`layer causing a change in resistance (typically measured as a
`change in current) that is used to identify where on the touch
`pad 36 the touch event occurred. In the capacitive category, a
`first set of conductive traces run in a first direction on the
`
`touch pad 36 and are insulated by a dielectric insulator from
`a second set of conductive traces running in a second direc-
`tion (generally orthogonal to the first direction) on the touch
`pad 36. The grid formed by the overlapping conductive traces
`creates an array of capacitors that can store electrical charge.
`When an object (e. g., a user’s finger) is brought into proxim-
`ity or contact with the touch pad 36, the capacitance of the
`capacitors at that location changes. This change can then be
`used to identify the location of the touch event.
`In yet another example, FIG. 2C illustrates an electronic
`device 40 having a touch screen display 44 according to the
`prior art as an input device. The electronic device 40 is a
`Personal Digital Assistant or the like. The touch screen dis-
`play 44 is positioned on a housing 42, and the electronic
`device 40 typically has some physical controls 46 on the
`housing 42. A stylus 48 is used to touch locations of the touch
`screen display 44 to perform various functions. The stylus 48
`is typically used like a mouse and arrow, and the display 44
`can show various menu items and other user interface fea-
`
`tures. Touching a menu item on the display 44 with the stylus
`48 can generate a pop-up or window 45 in which the user can
`then make a selection with the stylus 48. The pop-ups or
`windows 45 overlay the content being displayed and tend to
`obscure it.
`
`traditionally separate hand-held electronic
`Recently,
`devices have begun to be combined in limited ways. For
`example, the functionalities of a telephone have been com-
`bined with the functionalities of a PDA. One problem that has
`been encountered is in the way inputs are made into the
`device. Each of these devices has a particular set of input
`mechanisms or devices for providing inputs into the device.
`Some ofthese input mechanisms are generic to all the devices
`(e. g., power button) while others are not. The ones that are not
`generic are typically dedicated to a particular functionality of
`the device. By way of example, PDAs typically include a
`touch screen and a few dedicated buttons while cell phones
`typically include a numeric keypad and at least two dedicated
`buttons.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`Thus, it is a challenge to design a device with limited input
`mechanisms without adversely affecting the numerous pos-
`sible functions that the device can perform. As will be appre-
`ciated, it is preferable not to overload the electronic devices
`with a large number of input mechanisms as this tends to
`confuse the user and to take up valuable space, i.e., “real
`estate.” In the case of hand-held devices, space is at a pre-
`mium because of their small size. At some point, there is not
`enough space on the device to house all the necessary buttons
`and switches, etc. This is especially true when considering
`that all these devices ne