(12) United States Patent
`Nelson et al.
`
`USOO691. 1969B1
`(10) Patent No.:
`US 6,911,969 B1
`(45) Date of Patent:
`Jun. 28, 2005
`
`(54) HANDHELD COMPUTER APPARATUS
`(75) Inventors: Scott A. Nelson, Eagan, MN (US);
`James C. Lee, Plymouth, MN (US);
`Michael A. Helgeson, Eagan, MN (US)
`(73) Assignee: Honeywell International Inc.,
`Morristown, NJ (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 843 days.
`(21) Appl. No.: 09/071,488
`(22) Filed:
`May 1, 1998
`
`(*) Notice:
`
`(51) Int. Cl. .................................................. G09G 5/08
`(52) U.S. Cl. ............................................ 345/163; 345/7
`
`(56)
`
`(58) Field of Search ................................. 345/7-9, 163,
`345/169, 156-158,347, 352, 357
`References Cited
`U.S. PATENT DOCUMENTS
`3.971,013 A 7/1976 Challoner et al.
`4.025903 A 5/1977 Kaufman et al.
`4,060,848. A 11/1977 Hyatt
`4,224,615. A 9/1980 Penz
`4,257,306 A 3/1981 Laflamme
`4,274,093 A 6/1981 Judge
`4,310,849 A 1/1982 Glass
`4,545,023 A 10/1985 Mizzi
`3. A 3.
`Shiman
`46392.25 A 1/1987 Washizuka
`4706117 A 11/1987 Schoolman
`4.737.972 A
`4f1988 Schoolman
`4,758,717 A
`7/1988 Shepard et al.
`(Continued)
`FOREIGN PATENT DOCUMENTS
`O540393 A 5/1993
`O637794. A 2/1995
`95.21408 A 8/1995
`
`EP
`EP
`WO
`
`
`
`OTHER PUBLICATIONS
`Siewiorok D P; “Wearable Computers: Merging Information
`Space with the WorkSpace” Proceedings of the Intl. Conf.
`On Computer Design: VLSI in Computers and Processors,
`Cambridge, MA. Oct. 3–6, 1993.
`Asim Smailagic et al; Modalities of Interaction with CMU
`Wearable Computers IEEE Personal Communications, vol.
`3, No. 1, Feb. 1, 1996.
`Rekimoto J et al. The World Through the Computer: Com
`puter Augmented Interaction With Real World Environ
`ments: UIST 95.8" Annual Symposium on User Interface
`Software and Technology. Proceedings of the ACM Sym
`posium on User Interface and Technology, Pittsburgh, PA.,
`Nov. 14-17, 1995.
`
`(Continued)
`Primary Examiner Jimmy H. Nguyen
`(74) Attorney, Agent, or Firm-Mueting Raasch &
`Gebhardt, PA.
`(57)
`
`ABSTRACT
`
`A computer apparatus includes a computer processing unit
`including a processor device for operating upon data and
`providing data for display. The apparatus further includes a
`handheld input/output display unit. The display unit includes
`a housing sized to be grasped by a users hand. A computer
`display is located within the housing at a position to allow
`the user to view the computer display through an eyepiece.
`A display driver is used for presenting data from the pro
`cessing unit on the computer display. Navigational input
`tools are integrated with the computer display in the housing
`and operatively connected to provide user input to the
`processing unit. The navigational input tools are positioned
`for operation by one or more fingers of the user's grasping
`hand. The computer processing unit may be integrated
`within the housing of the handheld input/output display unit.
`The navigational input tools may include a mouse having a
`force actuated pointer and at least one button, e.g., a Single
`button mouse or a two button mouse, or may include a three
`button interface.
`
`19 Claims, 23 Drawing Sheets
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 1 of 34
`
`

`

`US 6,911,969 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`4.905,001 A 2/1990 Penner
`4,916,441 A 4/1990 Gombrich
`4,931,950 A 6/1990 Isle et al.
`4,994.987 A 2/1991 Baldwin
`5,003,300 A 3/1991 Wells
`5,015,831 A 5/1991 Eastman et al.
`5,052,504 A 10/1991 Ikeda et al.
`5,144,119 A 9/1992 Chadima, Jr. et al.
`5,148,155 A 9/1992 Martin et al.
`5,175,534 A 12/1992 Thatcher
`5,208.449 A 5/1993 Eastman et al.
`5,267,181 A 11/1993 George
`5,281957 A
`1/1994 Schoolman
`5,285,398 A 2/1994 Janik
`5,305,244 A 4/1994 Newman et al.
`5,432,510 A * 7/1995 Matthews .................... 341/20
`5,450.596 A 9/1995 Felsenstein
`5,491,651 A 2/1996 Janik
`5,579,165 A 11/1996 Michel et al.
`5,964,830 A * 10/1999 Durrett ....................... 709/200
`OTHER PUBLICATIONS
`
`Product Literature, “The ProViewTM 30 Head Mounted
`Display System', Kaiser Electro-Optics Laser, Inc., 4pgs.,
`(available at least in Apr. 1997).
`C. Murray, “Head-Mounted Display Simplifies Surgery',
`Design News, pp. 102-103, Aug. 11, 1997.
`D. Baum, “VIMAD,” Scientific Honeyweller, 5(2), pp.
`45-51, Jun. 1984.
`J. Robertson, “Remote users eye tiny FPDs.” Electronic
`Buyers News, May 1997.
`Product Literature, "Advanced Flat Panel Head Mounted
`Display Program”, R&D Programs, Electronics Technology
`Office, http://esto.sysplan.com, 2 pg.S., (accessed via internet
`Apr. 1997) (last updated Apr. 1996).
`Product Literature, “HIDEF Family”, KEO, http://www.keo.
`com, 2 pgs., (accessed via internet Apr. 1997).
`Product Literature, “Kaiser Electro-Optics”, KEO, http://
`www.keo.com, 4 pgs., (accessed via internet Apr. 1997)
`(copyright 1997).
`Product Literature, “Full Immersion Head Mounted Display
`(FIHMD)', ISO Planning & C3 Program Portfolio, http://
`maco.dc.isX.com, 2 pgs., (accessed via internet Apr. 1997).
`Product Literature, “Head Mounted Display”, http://ece.
`clemson.edu, 1 pg., (accessed via internet Apr. 1997).
`Product Literature, “Head Mounted Displays (HMD)", R&D
`Programs, Electronics Technology Office, http://esto.sys
`plan.com, 3 pgs., (accessed via internet Apr. 1997).
`Product Literature, Intervision, http://www.intervisionsys
`tems.com, 6 pgs., (accessed via internet May 1997) (copy
`right 1995, 1996).
`Product Literature, “Look Past Today With TrekkerTM”,
`Personal Information Systems, http://www.cacd.rock
`well.com, 4pgs., (accessed via internet May 1997) (copy
`right 1996).
`Product Literature, “Boeing Wearable Computer Workshop
`Breakout Session Summaey', Boeing Wearable Computer
`Workshop, http://www.cs.Smu.edu, 3 pgs., (accessed via
`internet May 1997).
`Product Literature, “The MIT Wearable Computing Page',
`http://lcs.www.media.mit.edu, 5 pgs., (accessed via internet
`May 1997).
`Product Literature, “Wearable Computing Research Group',
`University of Oregon Computer & Information Science,
`http://www.cs.uoregon.edu, 2 pg.S., (accessed via internet
`May 1997) (last updated Jan. 1997).
`
`Product Literature, “Electronic Performance Support Sys
`tem', EPSS, http://mime 1.marc.gatech.edu, 6 pgS.,
`(accessed via internet May 1997) (last updated Apr. 1997).
`Product Literature, “Shape Deposition Manufacturing”,
`http://www-rpl.stanford.edu, 2 pgs., (accessed via internet
`May 1997).
`Product Literature, “The DeVry Student Chapter I.E.E.E.
`Official Wearable Computer Homepage”, http://www.devry
`cols.edu, 3 pgs., (accessed via internet May 1997) (copyright
`1997).
`Product Literature, “Jeff Hartman's Wearable Computer
`Technical page:', http://www.netwalk.com, 2 pgS.,
`(accessed via internet Apr. 1997) (Last updated Jan. 1997)
`(copyright 1996).
`Product Literature, “The Ultimate Portable Computer'?”,
`News & Views, Copyright 1994-1997, http://www.byte.com,
`2 pgs., (accessed via internet Apr. 1997) (copyright
`1994–1997).
`Product Literature, “Oregon Wearable Computer Wiring
`Diagram,” http://www.cs.uoregon.edu/research/wearableS/
`Oregon/netman3.gi, 3 pgs., (accessed via internet May
`1997).
`Product Literature, “Details about the WetPCTM, WetPC
`(tm)diagrams, http://www.aims.gov.au/pages/wetpc/wpcdi
`ag.html, 1 pg., (accessed via internet May 1997) (last
`updated Jan. 1997).
`Product Literature, “Commercialising the Technology,”
`WetPC (tm) commercialisation, http://www.aims.gov.au/
`pages/wetpc/wpccom.html, 2 pg.S., (accessed via internet
`May 1997) (last updated Jan. 1997).
`Product Literature, “Technical backgrounder.” WetPC (tm)
`Backgrounder,
`http://www.aims.gov.au/pages/wetpc/
`wpcbkgrinder.html#specs, 5 pgs., (accessed via internet May
`1997) (last updated Jan. 1997).
`Product Literature, Wearable, Tetherless, Computer-Medi
`ated Reality (with possible future applications for the dis
`abled), http://www.wearcam.org/tetherleSS/, 1 pg., (accessed
`via internet May 1997).
`Product Literature, “General Information,” General Infor
`mation-On-Site Wearable Computer Systems, http://ww
`w.cs.cmu.edu/afs/cs.cmu.edu/project/vuman/www/gener
`al.html, 2 pgs., (accessed via internet May 1997) (last
`updated Jun. 1995).
`Product Literature, “About Cybernaut Corporation & The
`Mobile Assistant
`IITM,” http://wwww.xybernaut.com/
`about1.html, 5 pgs., (accessed via internet May 1997).
`Product Literature, “WS 1000 Wearable System with RS 1
`Ring Scanner.” WS 1000 Wearable System with RS 1 Ring
`Scanner, http://www.symbol.com/STO00262.HTM, 8 pgs.,
`(accessed via internet May 1997).
`Product Literature, “RS 1 Ring Scanner, RS 1 Ring Scan
`ner, http://www.symbol.com/STO00266.HTM, 5 pgs.,
`(accessed via internet May 1997).
`Product Literature, “Virtuo Vue Palm-Sized Portable Dis
`play, 1 pg.
`Product Literature, “Office-on-an-arm hits the Streets.”
`News Trends, 1 pg.
`Product Literature, “A Cyberscreen So Tiny It Fits. On A
`Dime,” Business Week, p. 126C, Apr. 21, 1997.
`Product Literature, “Wearable Computer.” Appliance Manu
`facturer, p. 8, Mar. 1996.
`* cited by examiner
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 2 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 1 of 23
`
`US 6,911,969 B1
`
`
`
`Fig. 1
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 3 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 2 of 23
`
`US 6,911,969 B1
`
`22 N
`
`
`
`23 -
`
`24
`-
`
`1.
`
`UTER
`2
`
`COMPUTER
`3
`
`27
`
`Main
`Access
`
`35
`
`33
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 4 of 34
`
`

`

`U.S. Patent
`
`US 6,911,969 B1
`
`1 O
`
`
`
`CN
`CO
`
`
`
`---------
`
`v
`
`65
`
`14
`41
`
`28
`
`20
`
`?WIdISICI
`
`?HOLO?INNOO
`
`DISPLAY
`
`16
`
`Fig. 3
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 5 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 4 of 23
`
`US 6,911,969 B1
`
`
`
`Fig. 4
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 6 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 5 of 23
`
`
`
`Fig.5
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 7 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 6 of 23
`
`US 6,911,969 B1
`
`
`
`Fig.6
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 8 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 7 of 23
`
`
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 9 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 8 of 23
`
`US 6,911,969 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`08V8:64, H.
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 10 of 34
`
`

`

`U.S. Patent
`
`Jun.28, 2005
`
`Sheet 9 of 23
`
`US 6,911,969 B1
`
`sdig[|
`|ggbua 3IOMBBJM
`
`
`
`
`‘aaiBe},useop{|Sjeag]4]“s8e)/6;46)s043UMOpmolq‘oo26eJouOps/2Adj
`
`
`“OOZNNAFeS|9A9]Huimoyo}ayyul![12D
`
`
`JeeuUsBO]SoUel|dUOD/e]USWUOAUYaU}ejajdui0|[4]eai]-aor
`
`
`
`“SUOTESIPU![9AQ]AJEdas7asO}Japlo
`
`
`OOZTNMA32SiaAe]Buimo}io}ou}Ul[2D:
`
`
`
`yopawn]siGujyGyjequassa-uoy|UOS/ed‘fT-SlWeN
`
`uaeUNOsayv1Oza[|tobzo1]02-3|wnudda34sav10z4WOLLOSdOud3d£023xWOLLOS
`
`
`
`
`WOLLOEddOud3dS0za929201S02-3
`
`
`
`XN1d3dydOud3d$0za8SS201dbe-d
`
`
`
`
`€€4201L0z-4
`
`uopdyosedAlsyseys,ABepol
`
`WOLLOS2-Aen
`
`
`Jaio|dxyJeuUa\U]Yosausw-
`djaSeOAeyOFMaIAWPFOld
`
`esezoT£0¢-4
`
`98©
`
`APPL-1042 / IPR2018-0036 1
`Apple v. Uniloc / Page 11 of 34
`
`uoydusseq
`
`eZ“ulus
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 11 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 10 Of 23
`
`US 6,911,969 B1
`
`
`
`90
`
`Fig.9
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 12 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 11 Of 23
`
`US 6,911,969 B1
`
`
`
`104
`
`106
`
`108
`
`100
`
`102
`
`Fig. 10
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 13 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 12 of 23
`
`US 6,911,969 B1
`
`
`
`Honeywell
`
`Copyright,1996 Honeywell
`
`Press and hold the middle key
`to view the application menu...
`
`Fig. 1 1A
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 14 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 13 of 23
`
`US 6,911,969 B1
`
`144
`
`Personnel
`
`>Alkylation N
`FCCU - 146
`> Suifer -
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`143
`
`
`
`Matt Johnson is logged on
`Alkylation
`
`12/10/1996 1 13:26 PM
`
`
`
`Current Shift: 3.
`Remaining Time in Shift:
`Shift Coverage:
`
`Personnel
`Maintenance
`Process
`Procedure
`EMail
`Exit
`
`Cancel
`
`View Map
`
`147
`
`141
`
`140
`
`Fig. 1 1B
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 15 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 14 of 23
`
`US 6,911,969 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`143
`
`
`
`
`
`144
`
`142
`
`Personnel
`
`No-one is logged on
`
`12110/1996 1:16:25 PM
`
`d Alkylation
`V7 FCCU
`d Shift Team #1
`d Shift Team #2
`> Suffer
`
`
`
`FCC
`
`Current Shift: 3.
`
`Remaining Time in Shift:
`Shift Coverage:
`
`
`
`
`
`
`
`Shift Team
`
`Board Operator: George Dunkirk
`
`Process Engineer: Jim Keam
`
`James WallaCe
`
`John Bailey
`
`147
`
`Personnel
`Maintenance
`Process
`Procedures
`EMai
`Exit
`
`Cancel
`
`View Map
`
`141
`
`Fig. 1 1 C
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 16 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 15 0f 23
`
`US 6,911,969 B1
`
`
`
`PerSonnel
`
`152
`
`Jim Kleam is logged on
`FCC
`
`12/10/1996 f:18:02 PM
`
`d Shift Team #1
`V7 Shift Team 2
`George Dunkirk
`y Jim Keam
`ZJim Keam
`James WallaCe
`John Bailey
`d Sulfer
`Y
`162
`
`Current Shift: 3.
`Remaining Time in Shift:
`Shift Coverage:
`
`Shift Team
`
`Board Operator: George Dunkirk
`Process Engineer: Jim Kleam
`Field Opi; 1: James Wallace
`Personnel
`John Bailey
`Maintenance
`Process
`Procedures
`EMail
`Exit
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 17 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 16 of 23
`
`US 6,911,969 B1
`
`
`
`Personnel Locator Map
`
`1. N
`
`XYZ Chemical Company, Vanilla Bayou
`{
`NE Gate
`FCCU lo
`
`Ayia
`
`O Reservoir
`
`Tank Farm
`O O
`
`Personnel
`
`Maintenance
`Process
`Procedures
`EMail
`Exit
`
`Close Map
`
`Suifer ill
`{ Evacuation
`Assembly Area
`
`X Fire Station
`
`revised 10/02/96
`
`170
`
`Fig.11 E
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 18 of 34
`
`

`

`og}>)191At raINDadId
`TAIMalAZeal
`
`
`
`Wd6S°ZE:+966L/0L/C1
`
`uope6S07siweerwit
`
`JOMO|JoziueNqegEDswedey
`
`
`
`JOSsaldU0y)SBDJOABIIAIaSq
`
`61
`
`SsOUBUS]UIBW
`
`
`
`
`
`PSLJAMO]JAZIUBINGS€DNedeyA
`
`ualE|Os]|Avoumopinys¢q
`
`uOnEOS|
`
`
`
`
`
`(S)a/vPrl-J0}SWOYO¢-9UOpuliqeppedoO}HeIsuy[4]
`
`OOLSEL4Woyelu!E-DUOPULaIpped,¢eySU|[7]
`
`
`
`G/YZ1-a01JeNNOE-—9UOputiqelppedOLWeisu)CJ
`
`
`
`
`
`OU]PRALUAOE-DUOPUNsipped,gI/ISU][7]
`
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`U.S. Patent
`
`Jun.28, 2005
`
`Sheet 17 of 23
`
`US 6,911,969 B1
`US 6,911,969 B1
`
`
`
`281
`
`881
`
`681.
`
`
`
`
`
`
`
`B/VPL-3Woypag;jajul€--UOpuliqjYyBiasunBy.gBumesq
`
`SoueUS}UIEWdnueis
`
`
`JOUUOSIBdesl——~_=>uonejosiagqsOURUOJUIEW<q
`
`
`
`
`
`aINWaS<¢SOINPSIOldJOPIOWOMMANUsdo<SS@00)dOZsoBWNY
`
`
`
`
`
`
`
`suonneg
`
`spayeiduiog
`
`Agpeyejdwog
`
`‘uOReIIUNWWOD
`
`teW>
`
`wa
`
`jaoued
`
`Lbrug
`
`APPL-1042 / IPR2018-0036 1
`Apple v. Uniloc / Page 19 of 34
`
`
`
`GadZi-aWoyJIU!E-OUOpullsipped,g|Hiv?L-3]Woypeasjaqul€-9UOpuljgyybieeunBy,gBums}CJjeisuj[2]
`
`
`
`
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 19 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 18 of 23
`
`US 6,911,969 B1
`
`
`
`isolate C-3 Debutanizer Tower El
`
`196
`
`From F1331 OC
`Control Station
`
`194
`
`195
`
`192
`
`191
`
`C Personnel
`Maintenance
`Process
`Procedures
`
`Close D L
`
`NZe==3-3s N194
`i . To E-14A/B(ts)
`
`PO OWS
`
`Fig. 1 1 G
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 20 of 34
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun.28, 2005
`
`Sheet 19 of 23
`
`US 6,911,969 B1
`US 6,911,969 B1
`
`
`
`WdSO:Z1:2
`
`
`
`deziuejngag€9D
`
`
`
`pees]8}e}S
`
`Ja|jojueDeunsseld
`
`jauuosieg
`
`S02\
`
`DNEWALSMAA
`
`BIBLUEDMAIA
`
`pUuel|MAA
`
`902
`
`802
`
`S66L/0L/2L
`
`me[sor]a.Hew3
`€0ZOL)O9/y|/BIOUED|-adA,
`
`Car]Ce)pay)
`SPOdodsSoUBLIE]UIEY
`66L0002dLMHOZA|:uojeaynuap|
`
`uOpeSGo7siweepwit
`
`
`punaryduind£02D4hdaimjeusdwayweessozIUe;NgedONTILFMoly
`
`
`JO[|ONUODGINSSAldPEEYIEAOZ0ZOdDA
`
`MOl4WES}SJeZIUEINGSQ102D4hd
`
`
`JalONUODaNSsaid'd’D€0ZOdbel
`oz4soewny¢JaljoujUODMO}Pe94ZOZO4bX
`
`JOMO]JOZIUBINGSG£9A
`
`sossoudwogsegie¢
`oe|CHEALOUT
`
`
`
`sse00ld
`
`20g
`
`„ H / / '64, H.
`
`APPL-1042 / IPR2018-0036 1
`Apple v. Uniloc / Page 21 of 34
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 21 of 34
`
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 20 of 23
`
`US 6,911,969 B1
`
`212
`
`C-3 Debutanizer Tower
`212
`To Atm.
`A
`X
`- N
`(a)
`
`N
`Feed From ---
`E-14A/B(ss)
`
`To E-13A(ss)
`
`From F1331 OC
`- Control Station
`
`tree
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`-
`
`
`
`212
`
`2
`
`213
`
`212
`
`PC 203 C.P. Pressure Controller
`
`O
`
`g
`Personnel
`Maintenance
`Process
`Procedures
`EMail
`Exit
`
`X
`
`(S. ty
`(FC 2040 MSCFD
`scance
`oMSCFD
`Dessagnac: S 212
`Mawi: w
`View Camera
`A
`TO E-14A/BtS
`to CWS'''
`(ts)
`S.
`
`
`
`210
`
`Fig. 1 1 I
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 22 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 21 of 23
`
`US 6,911,969 B1
`
`[×]Spueu L
`
`?ZZ
`
`
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 23 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 22 of 23
`
`US 6,911,969 B1
`
`22
`-N
`
`23
`7-
`
`24
`-
`
`COMPUTER
`1.
`
`COMPUTER
`2
`
`COMPUTER
`3
`
`27
`
`Main
`Access
`
`29-1
`
`3OO
`
`N 301
`
`
`
`328
`
`326
`
`Fig. 12
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 24 of 34
`
`

`

`U.S. Patent
`
`Jun. 28, 2005
`
`Sheet 23 of 23
`
`US 6,911,969 B1
`
`
`
`
`
`9992
`
`NWWIQIH
`
`THÁOW.H™HGHINI
`NOIJWÈ) LAWN
`
`
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 25 of 34
`
`

`

`1
`HANDHELD COMPUTER APPARATUS
`
`US 6,911,969 B1
`
`FIELD OF THE INVENTION
`The present invention is generally related to computing
`devices. More particularly, the present invention pertains to
`hand-held computer devices and user interfaces for Such
`devices.
`
`2
`obstructions in a processing plant environment or any other
`user environment. Further, wearable and tablet computers
`typically have displays plagued by a limited Screen size
`and/or have various user input challenges. For example,
`users of Such personal information Systems may wear gear
`Such as gloves or be required to carry a large quantity of
`accessories or gear. Inputting information into a personal
`information System when wearing gloves becomes a difficult
`process with most conventional Systems. Even just navigat
`ing through information available in Such Systems when in
`the field with or without gloves may be challenging.
`
`SUMMARY OF THE INVENTION
`Generally, the present invention integrates a display and
`navigational input tools for use with a computing unit to
`address one or more of the problems relating to conventional
`devices described above.
`A computer apparatus according to the present invention
`includes a computer processing unit including a processor
`device for operating upon data and providing data for
`display. The apparatus further includes a handheld input/
`output display unit. The display unit includes a housing
`sized to be grasped by a user's hand. A computer display is
`located within the housing at a position to allow the user to
`View the computer display through an eyepiece. A display
`driver is used for presenting data from the processing unit on
`the computer display. Navigational input tools are integrated
`with the computer display in the housing and operatively
`connected to provide user input to the processing unit. The
`navigational input tools are positioned for operation by one
`or more fingers of the user's grasping hand.
`In one embodiment of the apparatus, the computer pro
`cessing unit is positioned within the housing of the handheld
`input/output display unit.
`In another embodiment of the apparatus, the navigational
`input tools include a mouse having a force actuated pointer
`and at least one button, e.g., a single button mouse or a two
`button mouse. In a further embodiment, the navigational
`input tools may include a three button interface.
`In yet another embodiment of the apparatus, the housing
`includes a recessed thumb indent which includes a Switch.
`The Switch may be used to control power to the apparatus
`and/or control activation of the mouse.
`A handheld apparatus for use with a computing unit
`according to the present invention includes a housing sized
`to be grasped by a user's hand having a longitudinal axis
`extending therethrough from a first end to a Second end. The
`housing includes an upper Surface and a lower Surface
`extending from the first end to the second end. Further, the
`housing includes an eyepiece at the first end of the housing.
`A computer display is located within the housing at a
`position to allow the user to view the computer display
`through the eyepiece. Navigational input tools are integrated
`within the housing with the computer display Such that the
`navigational input tools are operable at the upper Surface of
`the housing by one or more fingers of the user's grasping
`hand.
`
`BACKGROUND OF THE INVENTION
`Display technologies are emerging which have impor
`tance for a variety of applications. For example, new display
`technologies are being used in hand-held personal digital
`assistants (PDAs), head-mounted displays (HMDs), minia
`ture monoculars or binoculars, etc. Recent advances in
`hardware and Software technologies enable the application
`of powerful computing platforms in a comfortable, body
`worn format. Application of wearable computerS allows for
`individual users to remain integrated with information SyS
`tems while operating hands-free and/or without hard-wire
`connection of Such computers to other components. For
`example, wearable computers provide only a limited advan
`tage if the computing platform must continue to be con
`nected to a fixed Structure for power and/or network con
`nectivity. Consequently, wearable Systems rely upon
`wireleSS network technologies to enable these Systems to be
`integrated into collaborative environments.
`There is an increasing need for field-portable personal
`information processing systems (PIPSs) which allow col
`laborative communication between a user and other parties
`or information systems that can provide the user with needed
`information and/or direction. For example, collaborative
`communication, diagnosis, and action among control room
`and field perSonnel in the context of large-scale processing
`plants is highly desirable. For example, Such functions are
`required between control room and field perSonnel in the
`context of large refineries, petrochemical plants, and pulp
`and paper mills. Field-portable personal information pro
`cessing Systems allow highly mobile, wireleSS access to a
`plant information infrastructure Such as distributed control
`System (DCS) point data and on-line procedures. Personal
`information processing Systems are used by field operations
`and maintenance perSonnel to enhance situation awareness,
`increase collaboration among and between field and control
`room perSonnel, and reduce operator-induced control and
`communication errors in the day-to-day operations of a
`processing plant.
`Conventionally, computer devices have been used to
`provide personal information processing System functional
`ity. Such computer devices generally have included various
`display devices and user interfaces. For example, Such
`devices may include notebook pad-type devices, tablet input
`devices, head-mounted displays for hands-free operation,
`Small or wearable keyboards, or voice actuation devices.
`However, conventional devices used for personal infor
`mation processing Systems do not recognize problems asso
`ciated with the human interface requirements for Such
`Systems. Personal display Systems impose various problems
`asSociated with the input devices being used, ease of move
`ment of the user, and compatibility with changing and/or
`harsh ambient environments where users require Such
`devices. For example, the wearing of a display on a helmet
`may provide obstruction of a user's vision. Helmet-mounted
`displays add weight to the already heavy helmets causing
`StreSS on the user's body. Further, Such head-mounted dis
`plays get easily bumped around as the wearer walks under
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a general illustration of a user grasping a
`hand-held input/output display unit for use with a wearable
`computer unit according to the present invention.
`FIG. 2 is a block diagram of a personal information
`processing System including a hand-held input/output dis
`play unit according to the present invention.
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 26 of 34
`
`

`

`3
`FIG. 3 is a block diagram of one embodiment of the
`hand-held input/output display unit of FIG. 1 useable with a
`Separate wearable processing unit.
`FIG. 4 is a perspective view of one illustrative embodi
`ment of the hand-held input/output display unit of FIG. 1
`according to the present invention integrating a single
`button mouse with a display in the unit.
`FIG. 5 is a top view of the hand-held input/output display
`unit of FIG. 4.
`FIG. 6 is a bottom view of the hand-held input/output
`display unit of FIG. 4.
`FIG. 7 is an exploded view of the hand-held input/output
`display unit of FIG. 4.
`FIGS. 8A-8B are display screens used to describe the
`graphical user interface for use with the navigational input
`tools integrated in the hand-held input/output display unit of
`FIG. 4.
`FIG. 9 is a top view of an alternate embodiment of the
`hand-held input/output display unit of FIG. 1 according to
`the present invention integrating a two-button mouse with a
`display in the unit.
`FIG. 10 is a top view of an alternate configuration of the
`hand-held input/output display unit of FIG. 1 according to
`the present invention integrating a three-button user inter
`face with a display in the unit.
`FIGS. 11A-11J are display screens used for describing a
`graphical user interface in conjunction with the three-button
`user interface of FIG. 10 according to the present invention.
`FIG. 12 is a block diagram of an alternate personal
`information processing System including an alternate hand
`held computer device according to the present invention,
`wherein the computing unit is integrated within the hand
`held device with the display and navigational input tools.
`FIG. 13 is a block diagram of further detail of the
`hand-held computer device of FIG. 12 including an inte
`grated processing unit.
`DETAILED DESCRIPTION OF THE
`EMBODIMENTS
`The present invention shall be described generally with
`reference to FIGS. 1 and 2. Thereafter, a more detailed
`description of various embodiments of the present invention
`shall be described with reference to FIGS. 3-13.
`FIG. 1 shows a user 11 grasping in the user's hand 13 a
`hand-held input/output display unit 12 according to the
`present invention. The hand-held input/output display unit
`12 is used by the user 11 to view a miniature computer
`display integrated therein with navigational input tools for
`allowing the user to navigate in computer program applica
`tions being executed by a wearable computing unit 14. The
`hand-held input/output display unit 12 is connected to the
`wearable computer unit 14 by cable 15, e.g., an umbilical
`cable, to form operative computer apparatus 10. One skilled
`in the art will recognize that with use of component/
`packaging miniaturization and battery miniaturization, the
`wearable computer unit 14 may be completely integrated
`into the hand-held input/output display unit 12, as further
`described herein with reference to FIGS. 12-13. Further, the
`connection 15 between the computer unit 14 and the hand
`held input/output display unit 12 may be wireleSS.
`The computer apparatus 10 allows the user 11 to move
`about in an environment, e.g., in an industrial or processing
`plant environment, a military environment, an educational
`environment, etc., while Still having access to the environ
`ment's information infrastructure. For example, in a plant
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,911,969 B1
`
`4
`environment the computer apparatus can be used by
`personnel, Such as field operations and maintenance
`perSonnel, to enhance situation awareness, increase collabo
`ration among and between field and control perSonnel, and
`reduce operator-induced control and communication errors
`in the day-to-day operations of the environment. The present
`invention is not to be limited to use in any particular
`environment as the present invention may be useful in many
`different environments, Such as any environment where a
`wearable computer would provide application advantages.
`In particular, although the present invention is described in
`conjunction with applications for a processing plant
`environment, the present invention is clearly not limited to
`Such an environment. Further, the present invention may be
`used Solely as a wearable computer (i.e., including the
`input/output display unit having integrated navigational
`tools) without the computer unit having a transceiver for
`wireleSS connection.
`FIG. 2 shows a personal information processing System
`19 wherein computer apparatus 10 is used. The computer
`apparatus 10 according to the present invention includes
`computer unit 14 operatively connected by an electrical
`connection 15 to hand-held input/output display unit 12.
`Generally, the personal information processing System 19
`includes wireless transmission link 25 to deliver digital
`information from the information infrastructure 33 of the
`environment in which the processing System 19 is used to
`computer apparatus 10. The information infrastructure 33 is
`generally represented by the computer Systems 22-24 and
`main acceSS point 27. One skilled in the art will recognize
`that Such an infrastructure may include any configuration of
`hardware, Software, firmware, transmission links, etc., and
`that the present invention is limited to no particular infra
`Structure.
`The transmission link 25 Supports wireleSS/digital com
`munication. Generally, the wireleSS transmission link or
`network 25 of personal information processing system 19
`may be any conventional link as would be readily known to
`one skilled in the art. For example, the transmission link,
`e.g., an RF network 25, may provide a digital wireless local
`area network (LAN) to Support data applications associated
`with the environment of the user 11. Further, for example, in
`the environment where the user is a processing plant user,
`Seamless connectivity and a maximized transmission data
`rate may be required. Such is typically difficult in processing
`plant environments where the environment includes a large
`quantity of metallic Structures and reinforced buildings and
`may require a different transmission link than is required for
`other environments.
`In general, the transmission network 25 consists of a main
`access point 27 which includes electronics for functioning as
`an interface between a wired infrastructure 33 or network,
`Such as is generally shown by computer Stations 22-24, and
`the transceiver 35 of the main access point 27. The trans
`ceiver 35 is used for transmission/reception via antenna 29.
`For example, the electronics may convert data from Ethernet
`into packet data for transmission. Further, the transmission
`network 25 may include Several repeaters (not shown)
`configured to resend Signals transmitted from the main
`access point 27 to extend the range of the wireleSS commu
`nication link to computer apparatus 10. Yet further, the
`transmission network 25 includes a wireless transceiver 31
`associated with the computer unit 14 worn by the user 11
`allowing the user 11 to receive and transmit data through
`antenna 17 associated with the computer unit 14. Generally,
`one skilled in the art will recognize that conventional
`modem electronics integrated in the computer unit 14 fulfills
`the receive/transmit functionality.
`
`APPL-1042 / IPR2018-00361
`Apple v. Uniloc / Page 27 of 34
`
`

`

`US 6,911,969 B1
`
`6
`
`TABLE 1-continued
`
`15
`
`S
`For example, in one implementation of the transmission
`network, a wireless local area network (LAN) may provide
`for operation in the 2.4 gigahertz (GHz) industrial, Scientific,
`and medical frequency band. The wireless LAN hardware
`may operate as a direct Sequence spread spectrum (DSSS)
`System. Range of transmission of network 25 is heavily
`influenced by the amount and type of obstructions between
`the antenna 29 of main access point 27 and antenna 17 of
`computer unit 14 worn by the user 11. Further, such range
`of communication will depend on the gain of Such antennas.
`Transmission networks 25 are known to those skilled in
`the art. The Summary description provided of transmission
`network 25 above is provided for illustration only and it will
`be recognized by one skilled in the art that various types of
`wireleSS transmission networks may be utilized in accor
`dance with the present invention.
`Generally, computer unit 14 may be any conventional
`wearable computer, suc