Europaisches Patentamt
`
`European Patent Office
`
`Office europeen des brevets
`
`@) Publication number:
`
`0 265 011
`A1
`
`EUROPEAN PATENT APPLICATION ·
`
`@ Application number: 87201959.1
`
`@ Int. Cl.4: GO6F 3/00 , G09B 21 /00
`
`@ Date of filing: 12.10.87
`
`@ Priority: 20.10.86 NL 8602624
`
`@ Date of publication of application:
`27.04.88 Bulletin 88/17
`
`@ Designated Contracting States:
`DE FR GB IT NL SE
`
`@ Applicant: Oce-Nederland B.V.
`Sl Urbanusweg 43
`NL-5914 CC Venlo(NL)
`
`@ Inventor: Mees, Martine
`Kolk 94
`NL-3011 MD Rotterdam(NL)
`Inventor: Valkenhoff, Andries Herman
`Van den Boschstraat 269
`NL-2595 AD Oen Haag(NL)
`
`•
`
`@ Representative: Hanneman, Henri W.A.M. et al
`Oce-Nederland B.V. Patents and Information
`Postbus 101
`NL-5900 MA Venlo(NL)
`
`@ Inputting device with tactile feedback.
`
`® A system comprising a computer, a screen, and
`an
`inputting device (10) for selection of specific
`areas on the screen and for moving a cursor on said
`screen, the inputting device (10) being coupled to
`the computer in which there is at least one screen(cid:173)
`controlled program, it being possible to bring the
`cursor to a selectable element by means of the
`inputting device (10), so that a sub-program symbol(cid:173)
`ized by said element can be selected by operating a
`selection key (13, 14) on the inputting device (10),
`the computer delivering a signal to the inputting
`device (10) when the position · of the cursor coincides
`with the position of a selectable element and the
`.- inputting device (10) being provided with a tactile
`C:C element (15, 16) which can be activated in response
`.-to said signal .
`
`... c::,
`Ln co
`N
`0
`Q. w
`
`FIG. 1
`
`Xerox Copy Centre
`
`Valve Exhibit 1040
`Valve v. Immersion
`
`

`

`0 265 011
`
`2
`
`Inputting device with tactile feedback
`
`This invention relates to a system comprisi~g a
`computer, a display unit comprising a screen, and
`an inputting device for selection of specific areas
`on the screen and for moving a cursor on said
`screen, the inputting device being coupled to the
`computer in which there is at least one screen(cid:173)
`controlled program (interactive program), a pro(cid:173)
`gram seeing to it that selectable elements are
`displayed on the screen at places corresponding to
`the specific areas, and it being possible to bring
`the cursor to a selectable element by means of the
`inputting device, so that a sub-program symbolized
`by said element can be selected by operating a
`selection key on the inputting device.
`A system of this kind is used in many places
`where
`information has
`to be processed on a
`screen, for example in CAD systems, word proces(cid:173)
`sors, or graphic workstations. Systems of this kind
`operate with mem:.1•controlled programs in which a
`number of commands selectable at a given time in
`the program are displayed at a special place on the
`screen. The user can move a cursor over the
`screen by means of an inputing sytem, and if the
`cursor is placed on a command or icon that com(cid:173)
`mand can be selected. The inputting unit is fre(cid:173)
`quently a so-caWed "mouse" coupled to the com(cid:173)
`puter system. By moving the mouse over a flat
`surface the cursor can be positioned to any re(cid:173)
`quired place on the screen. The mouse is also
`provided with one or more press-buttons for selec(cid:173)
`tion of a command. If the cursor position coincides
`with the position where a command is displayed
`pressure on the botton will start the sub-program to
`be performed under that command. This sub-pro(cid:173)
`gram may, for example, provide a new menu dis(cid:173)
`play or a so-called pop-up menu.
`Using such mouse-controlled menu programs
`requires high concentration visually of the user.
`Particularly if the areas for selection are small, it is
`rather difficult to get the cursor exactly in the
`required area and to actuate the selection button at
`the correct moment.
`The object of the invention is accordingly to
`provide a system according to the preamble in
`which this disadvantage is obviated. According to
`the invention, this object is attained in a system
`according to the preamble, in that when the posi(cid:173)
`tion of the cursor coincides with the position of a
`selectable element the computer delivers a signal
`to the inputting device, in that the inputting device
`is provided with a tactile element which can be
`activated in response to said signal and in that the
`tactile element is disposed in a selection key.
`
`Tactile feedback makes use of the touch re(cid:173)
`ceptors and/or pressure receptors in the hand, so
`that the amount of attention required particularly for
`the central part of the field of view of the eyes can
`be relieved. This means a more balanced distribu(cid:173)
`tion of the offered information over the various
`senses. Although extra attention is called for on the
`part of the touch receptors in the fingers, the
`intensively used eyes will become less tired and
`the -total strain will be reduced when using the
`system.
`The use of the simultaneous feedback of visual
`and tactile information also results in a form of
`reduncancy which benefits the error-sensitivity of
`the mouse usage. Reaction time can also be re(cid:173)
`duced by the tactile feedback. Thus tactile feed(cid:173)
`back eliminates retinal delay, which
`is involved
`when the visual channels are used. With the skin
`receptors there is a more rapid transmission of
`information to the brain. Tactile feedback can also
`result in a kind of reflex on the following lines:
`tactile element moves - actuate button, so that
`relatively rapid response times can be obtained.
`In the inputting of commands with which the
`situation on the screen changes considerably, it is
`sometimes necessary for the user to wait until the
`prompt re-appears as an
`indication that the in(cid:173)
`formation has been processed and the screen ad(cid:173)
`justed. This necessary wait is also due to the fact
`that the software for the image processor which is
`responsible
`for refreshing
`the screen
`requires
`some time to process new information. If the user
`receives a tactile signal at the instant that the
`information has been processed, but before the
`instant that the information processing for screen
`refreshment has started, the waiting time can be
`saved because the user cari continue operations
`before the screen is refreshed.
`These and other advantages of the invention will be
`explained with reference to the drawings wherein:
`Fig. 1 is an embodiment of a mouse as used
`in a system according to the invention,
`Fig. 2a to ! show the use of said mouse,
`Figs. 3a and £ show the use of the mouse in
`selecting one pixel thick lines and
`Figs. 4a en 2 represent a diagram showing
`the- principle of a tactile element as. used in a
`system according to the invention.
`Fig. 1 shows an inputting device of the system
`according·to tl'.)e invention. This inputting device or
`mouse 1 O has a baseplate 11, the base of which is
`provided with sensors which, when the mouse is
`moved over a flat surface, cause the cursor on the
`screen to move in a corresponding direction. The
`mouse is connected by a cord 12 to a computer in
`
`5
`
`10
`
`15
`
`20
`
`25
`
`so
`
`35
`
`40
`
`45
`
`50
`
`2
`
`

`

`3
`
`0 265 011
`
`4
`
`is running. The
`inter-active program
`which an
`mouse has two press-buttons (selection keys) 13,
`14,
`the operation of which will be described
`hereinafter. Each button contains a tactile element
`15 and 16 respectively in the form of a pin which
`under certain conditions can project slightly above
`the top of the button.
`The operation of this mouse will be described
`in detail with reference to Figs. 2a to 2f. In Fig. 2a,
`two specific areas 21, 22 for selection are shown
`on a screen 20. A cursor 23 in the form of a small
`arrow is also situated on this screen. Next to this
`screen 20 the buttons 14 and 13 of the mouse are
`shown diagrammatically in their initial positions. By
`moving the mouse in the correct direction the cur(cid:173)
`sor 23 is brought to the area 21 for selection. The
`user's fingers rest on the buttons 14, 13. When the
`tip of the cursor 23, i.e. the hot spot, enters the
`area 21 for selection (Fig. 2b) the tactile element
`16 in button 14 rises and signals to the user that
`the area 21 has been reached. The user then
`presses button 14 down (Fig. 2c) in order to select
`the field symbolized by area 21. After selection the
`user allows the button 14 to rise. By then pressing
`in the button 13 (Fig. 2d) it is possible to call up a
`so-called pop-up menu 25 which shows a number
`of possible operations to be performed on area 21.
`The cursor jumps to a standard position (default
`position) in this menu ('OPEN'). The tactile element
`15 of button 13 rises when the button 13 is pressed
`in. As long as button 13 is pressed in, the pop-up
`menu 25 is visible on the screen 20. The cursor 23
`can be brought to a different position in the menu
`by moving the mouse in order to select a different
`command required. If the cursor 23 is brought to
`the command 'CUT' (Fig. 2e), the tactile element
`15 will go down just before the cursor leaves a
`command, and then rises again when entering the
`next command. If the button 13 is released (Fig.
`2f) after reaching the command 'CUT' 27, this
`command is performed on the selected area 21 so
`that area 21 disappears from the screen 20. The
`tactile element 15 descends again beneath the
`surface of the button 13.
`When a command of this kind is performed, in
`the situation as shown in Fig. 2a the cursor 23 can
`be moved towards the area 21 without intensive
`use of the eyes. All that is necessary is for the user
`to know. the overall screen layout to be able to
`perform a positioning movement of this kind, since
`he can rely on information from the peripheral area
`of the eyes and the tactile feedback when area 21
`is reached. Thus there is no need for the user to fix
`his attention on cursor 23 and follow it. Once the
`situation is reached as shown in Fig. 2b, the user
`
`5
`
`15
`
`20
`
`25
`
`30
`
`can react automatically by immediately pressing in
`the button where the tactile element is perceptible
`so that the selection of the relevant command
`takes place.
`After the pop-up menu has been called up,
`giving the situation shown in Fig. 2d, a user familiar
`with this pop-up menu can basically find the com(cid:173)
`mand 'CUT' blindly. He knows that as soon as the
`pop-up menu has been called up the tactile ele-
`10 ment in button 13 becomes perceptible in the
`default position (pre-set command) and that there
`must then be another two perceptible stimuli on the
`upward movement of the cursor 23 before the
`command 'CUT' is reached. If after calling up the
`pop-up menu the user for the third time feels the
`tactile element against his finger (OPEN-COPY(cid:173)
`CUT) then the situation shown in Fig. 2e has been
`reached and by releasing the button .13 he can
`activate the command 'CUT' to give the situation
`shown in Fig. 2f.
`•
`The tactile feedback is particularly important if, for
`example, it is necessary to select a frame line
`having a thickness of one pixel. Figs. 3~ and E
`show the
`line 30 for selection displayed on a
`screen 31. Just one pixel of the cursor 23 (the
`extreme tip of the arrow in this case) governs the
`position of the cursor 23 as recorded by the com·
`puter and this one pixel must come exactly on the
`one-pixel thick line 30. Selection by means of a
`tactile feedback takes place by moving the cursor
`in the direction of the required object. When it
`becomes perceptible · that the cursor is on the
`frame line 30, an immediate response is possible
`by pressing in the button in which the tactile ele-
`35 ment becomes perceptible, so that the object can
`be selected with great accuracy.
`Figs. 4~ and E represent a diagram showing
`the principle of a tac tile element as used in the
`invention. A pin 40 is disposed on a compression
`in a button 44. The pin 40 is of mag(cid:173)
`spring 41
`netizable material. Disposed around the pin 40 is
`core 43 with a coil 42 therearound. By passing a
`d.c. through the coil the pin 40 remains in the core
`and hence beneath the surface of the button 44. By
`(temporarily) disconnecting the current the spring
`41 presses the pin 40 out of the surface of the
`button 44.
`A large number of embodiments are possible
`to give the same effect as described above using
`electromagnetism. For example, the spring 41 can
`be replaced by a leaf spring or the pin can be
`constructed as a hinged construction. The framing
`can also be such that the pin is raised if the coil is
`energized.
`
`4-0
`
`45
`
`50
`
`55
`
`3
`
`

`

`5
`
`0 265'011
`
`6
`
`ment (15, 16) wnich can be activated in response
`to said signal and in that the tactile element is
`disposed in a selection key (13, 14).
`2. A system according to claim 1 characterised
`in that the tactile element comprises a piezoelectric
`element.
`3. A system according to claim 1 characterised
`in that the tactile element comprises an electro(cid:173)
`·magnetic element.
`4. A system according to any one of the pre-
`ceding claims, characterised in that the tactile ele:
`ment comprises a pin (40) which, on activation of
`the element, projects above the surface (44) of the
`inputting device (10) or selection key (13, 14).
`5. A system according to any one of the pre-
`·ceding claims, characterised in that the tactile ele(cid:173)
`ment is vibrated for a given time during coinci(cid:173)
`dence of the positions.
`6. A system according to claim 5, charac-
`terised in that the vibration frequency is preferably
`between 10 Hz and 500 Hz.
`7. A system according to claim 6, charac(cid:173)
`terised in that the vibration frequency is 250 Hz.
`
`Another embodiment of the tactile element can
`be obtained by means of a piezoelectric element.
`This comprises a bimorphous rod on which the pin
`rests. By applying a voltage across the bimorphous
`rod a deflection occurs in the rod and can cause
`the pin to move.
`In yet another embodiment of the tactile ele(cid:173)
`ment, a piezoelectric disk is used which is dis(cid:173)
`posed on the selection button. This disk can be
`vibrated.
`•
`Instead of the tactile element being moved
`upwards only once during selection, it is possible
`to vibrate it at a frequency of between 10 Hz and
`500 Hz for the entire duration of the selection or
`while the cursor coincides to the special area. This
`is very advantageous particularly in embodiments
`in which a piezoelectric element is used. A fre(cid:173)
`quency of about 250 Hz gives an optimum for the
`feel at the user's fingertips.
`The invention is not restricted to the embodi(cid:173)
`ments described and a skilled addressee will be
`able to make numerous modifications. For exam(cid:173)
`ple, the entire selection button can be moved up(cid:173)
`wards during coincidence. The tactile elements
`may consist of more pins per button, which are
`activated uniformly. They can also be in the form
`of a plane or one or more linear elements.
`The mouse need not be connected to the computer
`system via an electrical conriection. Connection
`may also be made, for example, by an infra-red or
`ultrasonic element. The tactile feedback may also
`be the opposite to the operation described herein(cid:173)
`before. However, all these modifications and em(cid:173)
`bodiments will come under the following claims.
`
`Claims
`
`1. A system comprising a computer, a display
`unit comprising a screen, and an inputting device
`(10) for selection of specific areas on the screen
`and for moving a cursor on said screen, the input(cid:173)
`ting device (10) being coupled to the computer in
`which there is at least one screen-controlled pro(cid:173)
`gram, a program seeing to it that selectable ele(cid:173)
`ments are displayed on the screen at places cor(cid:173)
`responding to the specific areas, and it being pos(cid:173)
`sible to bring the cursor to a selectable element by
`means of the inputting device (10), so that a sub(cid:173)
`program symbolized by said element can be se(cid:173)
`lected by operating a selection key (13, 14) on the
`inputting device (10), characterised in that when
`the position of the cursor coincides with the posi(cid:173)
`tion of a selectable element the computer delivers
`a signal to the inputting device (10), in that the
`inputting device (10) is provided with a tactile ele-
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`/
`/s5
`
`. 4
`
`

`

`0 265 011
`
`FIG. 1
`
`
`
`FIG. 4 b
`
`FI G.4 a
`
`1-ill-8617
`
`

`

`0 265 011
`
`20
`
`23
`
`:-:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:-:-:-:-:,:-:-:-:-:-:•:•:•:•:•.·.···············=·=•:•
`-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:,:-:,:-:,:-:-:-:-:-:-:-:-:,:-:-:,:,:-:,:
`
`'lll..•.·············•:•·····•:•:•:•:•:•:•:•:•:-:,;-:,:•.•·······················•;•:iil
`
`FIG.2a
`
`:::::•:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::;:::;:;::::::
`•:•,•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•.···········=···=······························
`
`,:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-..... .. ........ -;-:,
`
`FIG.2b
`
`-····· ·················································-
`
`~-.•.•,•,•··································=·····=·=•:•:•:•:•:•:-:,:,:-:-:,•,•;•~
`
`Fl G. 2 c
`
`25 26
`
`21
`~-----23
`
`~:;;;;a.:._..:..,__--ll:llll~
`
`20
`
`..+;.;.;.l-._22
`
`14
`
`,~~~~~
`;::15 13
`~~~~~
`
`FIG. 2 d
`
`2-fil.;;.8617
`
`

`

`0265011
`
`25 27 23 21
`
`cu
`copy
`open
`properties
`move
`
`14
`
`\%~~"~~
`,'0--~~ k13
`
`15
`
`Fl G. 2 e
`
`23 I
`
`•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:❖:-:-:-:-:-:-:-:•:-
`.❖:-:-:-:-:-:-:-:­
`:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:-:-:-:-:,:-:-:-:-:-:-:-:-:-::~:-:•:•:•:•:::•:
`•:•:•:•:•:•:•:-:♦:♦:-:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:::•:•:•:::::•:::::-:-::::::
`:-:•:•:•:-:-:-:-:-:-:-:-:-:-:-:-:-:;:-:-:-:-:;:•:::•:::•::::::::::::::::::::::,:::,::
`-:-:-:-:-:-:-:-:•:•:-:-:-:-:-:-:-:,:-:-:-:-:-:-:•:•:•:•:•:::•:-:-:-:-:;:•::::::::::,:
`-:-:-:-:-:-:-:-:-:,:-:-:-:-:-:•:•:-:,:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:•:;:-:-:-:;:•::
`:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:•:-:-:::::::::-:
`
`- ··-·-•-•,•,•-·-•.•-•.•,•-•-· .. •.• ... · ................... ·~
`
`FIG. 2 f
`
`Fl G. 3 a
`
`Fl G.3 b
`
`3 -.ll.-8617
`
`

`

`European Patent
`Office
`
`EUROPEAN SEARCH REPORT
`
`Category
`
`•
`
`y
`
`DOCUMENTS CONSIDERED TO BE RELEVANT
`Citation of document with indication, where appropriate,
`of relevant passages
`IBM TECHNICAL DISCLOSURE BULLETIN, vol.
`28, no. 3, August 1985, pages
`1343-1344, New York, US; "Seeing eye
`mouse 11
`* Page 1344, last two paragraphs;
`figures 1,2 *
`
`Relevant
`to claim
`1
`
`Application Number
`
`EP 87 20 1959
`
`CLASSIFICATION OF THE
`APPLICATION (Int. Cl.4)
`
`G 06 F
`G 09 B
`
`3/00
`21/00
`
`A
`
`y
`
`A
`
`A
`
`Idem
`
`(F. LAPSON et al.)
`GB-A-2 166 627
`* Abstract; figure 14; claims 1-3 *
`
`IEEE TRANSACTIONS ON MAN-MACHINE •
`SYSTEMS, vol. MMS-11, no. 1, March
`1970, pages 58-65; J.C. BLISS et al.:
`"Opt i ca 1-to-tact il e image conversion
`for the blind"
`* Page 59, right-hand column*
`
`MEDICAL AND BIOLOGICAL ENGLNEERING,
`vol. 14i no. 7, July 1976, pages
`451-454; HIDETO:
`11 Vibrotactil e
`stimulation by using both the
`electrical and mechanical stimuli"
`* Whole document*
`
`3,4
`
`1
`
`2,5
`
`5-7
`
`TECHNICAL FIELDS
`SEARCHED (Int. Cl.4)
`
`G 06 K 11/06
`G 06 F 3/00
`G 09 B 21/00
`
`The present search report has been drawn up for all claims
`
`Place of searcll
`DIie of caa,pletloll of Ille se.-cll
`MOENS RA A
`19-11-1987
`THE HAGUE
`-
`!1 - - - - - - - - - - ' - - - - - - - - - - - ' - - - - - · _· - · - - - - - - - 1
`N .
`CATEGORY OF OTED DOCUMENTS
`T : theory or principle underlying the Invention
`a
`E : earlier patent document, but published on, or
`X : particularly relevant if taken alone
`after the filing date
`~
`Y : particularly relevant if combined with another
`D : document cited In the application
`L : document cited for other reasons
`document of the same category
`'1 c,:
`A : technological background
`~
`& : member of the same patent family, corresponding
`0 : non-written disclosure
`document
`P : intermediate document
`0
`II,
`111...__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __,
`
`