United States Patent
`
`[19]
`
`Bier
`
`5,561,811
`[11] Patent Number:
`[45] Date of Patent: Oct. 1, 1996
`
`
`
`lllllllllllllllll|l||||||||Illlllllllllllllllll||l|l|||||||||ll||||l|ll||||
`U3005561811A
`
`[54]
`
`METHOD AND APPARATUS FOR PER-USER
`CUSTOMIZATION OF APPLICATIONS
`SHARED BY A PLURALITY OF USERS ON A
`SINGLE DISPLAY
`
`[75]
`
`Inventor: Eric A. Bier, Mountain View, Calif.
`
`[73]
`
`[21]
`
`[22]
`
`[51]
`[52]
`
`[58]
`
`[56]
`
`Assignee: Xerox Corporation, Stamford, Conn.
`
`Appl. N0.: 974,044
`
`Filed:
`
`Nov. 10, 1992
`
`Int. Cl.6 ........................................................ G06F 3/00
`US. Cl.
`.......................... 395/825; 395/153; 395/155;
`395/893; 395/600; 395/650; 395/700; 395/800;
`364/DIG. l; 364/236.8; 364/237.2; 364/237.3;
`364/238; 364/286; 364/286.3
`Field of Search .................................. 341/20, 22, 23;
`345/115, 116, 162, 163, 167, 172, 145;
`395/200, 275, 600, 650, 700, 800, 153—155,
`200.04, 821, 828, 836, 882, 892—893; 364/188,
`189, 190, 474.22; 273/438
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`12/1977 Baxter ....................................... 341/22
`4,064,560
`.. 340/709
`4/1982 Akashi
`..............
`4,323,891
`
`..... 395/887
`2/1988 Endfield et al.
`..
`4,727,478
`
`5/1992 Kerr ........................ 395/600
`5,115,501
`
`8/1992 Belt et a1. .............. 395/887
`5,136,694
`
`5,157,384 10/1992 Greanias et a1.
`..
`..... 340/706
`
`1/1993 Aihara et a1. .............. 395/650
`5,179,700
`
`5,220,675
`6/1993 Padawer et a1.
`..
`..... 395/800
`
`2/1994 Gram .......................... 395/700
`5,287,514
`
`5,319,747
`611994 Gerrissen et a].
`..... 395/155
`
`............................. 395/153
`5,337,407
`8/1994 Bates et a1.
`
`
`..
`.
`. ..
`.. 395/800
`11/1994 San Soucie et al.
`5,369,778
`5,473,744 12/1995 Allen et al.
`............................. 395/154
`
`FOREIGN PATENT DOCUMENTS
`
`58142996
`6271453
`
`Japan.
`8/1983
`Japan.
`3/1987
`OTHER PUBLICATIONS
`
`Bier, E. A. et a1. “MMM: A User Interface Architecture for
`Shared Editors on a Single Screen,” Proceedings ofthe ACM
`Symposium on User Interface Software and Technology,
`Nov., 11—13, 1991, P. 79.
`Simpson, A. “Mastering WordPerfect S. 185.2 for Win-
`dows” CA, Sybex, 1993, fig. 2.2.
`WordPerfect for Windows version 5.1, WordPerfect Corpo-
`ration,. Apr. 30, 1992.
`
`Primary Examiner—Lance L. Barry
`Attorney, Agent, or Finn—Rosen, Dainow & Jacobs Limited
`Liability Partnership
`
`[57]
`
`ABSTRACT
`
`A multi-user multi-device system enables a plurality of users
`to control a single screen. Each user has one or more input
`devices, which can be used to control one or more stored
`applications. At any time, the system produces a consistent
`view of all applications on the single screen. The input from
`each user produces a response customized to the preferences
`of that user. Each user can inform the system of which
`devices that user is using. Inputs to the system initiate the
`building of Event Records that are queued and then directed
`to specific applications. The screen is updated, and the
`coordinates of child applications are determined, only dur-
`ing pauses in the operations of all applications.
`
`7 Claims, 13 Drawing Sheets
`
`
`
`Petitioners Ex. 1001 Page 1
`
`Petitioners Ex. 1001 Page 1
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 1 of 13
`
`5,561,811
`
`
`
`Petitioners Ex. 1001 Page 2
`
`Petitioners Ex. 1001 Page 2
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 2 of 13
`
`5,561,811
`
`Reci Ediiar
`Text Ediior
`
`.1111!-
`
`Primary device Area
`
`Secondary device Area
`
`F/G.3
`
`Petitioners Ex. 1001 Page 3
`
`Petitioners Ex. 1001 Page 3
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 3 of 13
`
`5,561,811
`
` Coopemie
`
`FIG/2
`
`feedback
`display area
`
`Petitioners Ex. 1001 Page 4
`
`Petitioners Ex. 1001 Page 4
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 4 of 13
`
`5,561,811
`
`
`F/G.9
`
`Petitioners Ex. 1001 Page 5
`
`Petitioners Ex. 1001 Page 5
`
`

`

`US. Patent
`
`,
`
`Oct. 1,1996
`
`Sheet 5 of 13
`
`5,561,811
`
`F / 6 . / 4
`
`F1
`
`..
`
`Done by the
`OS thread
`known as the
`Device Process
`
`Build-event record with
`time stamp d9ViC€
`identification and
`M device event type
`
`
`
`|00
`
`l0l
`
`l02
`
`|03
`
`Enqueue Event Record
`on System Queue
`
`instance
`Add user
`device states and
`modified event
`(including x-y coordinates)
`to the event record
`
`.
`Done by the
`Opemfing
`system thread
`known as the
`Notify Process
`
`l08
`
`l06
`
`Execute menu
`input algorithm
`
`evenItSin a
`home area
`7
`
`Y
`Execute home area
`input algorithm
`
`N
`
`
`
`Execute
`OPPI‘CO’HOH
`
`
`input algorithm
`|09
`
`.
`
`,
`
`Petitioners Ex. 1001 Page 6
`
`Petitioners Ex. 1001 Page 6
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 6 of 13
`
`5,561,811
`
`set .by the
`devrce process
`
`F7615
`
`se’r b the
`nofi y process
`
`
`
` Application AppIico’rion
`Queue
`
`Process
`
`Petitioners Ex. 1001 Page 7
`
`Petitioners Ex. 1001 Page 7
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 7 of 13
`
`5,561,811
`
`'50
`
`Lei U = The
`User Insionce
`Record of This
`Home Area
`
`[5|
`
`Y
`
`Previous owner
`-
`Of Device D
`
`‘
`
`
`
`
`152
`
`Remove previous
`owner from User
`Insionce in Device
`and Device to Use
`Insiunce fables.
`
`for deviceD
`
`F/G. /6
`
`'57
`
`
`
`'53
`
`Inspect lisi of devices
`curreniiy cioimed
`in user instance 10
`device iobie by
`user insiance U
`
`'55
`
`
`Add Dio ihe lisi
`156
`and ossocioieU
`I
`D
`d
`
`
`with o in the
`.thoirf.” fY
`
`
`'“
`0
`'3
`Device io Use
`
`(.9
`Insicnce Tobie
`
`
`
`
`
`Click oiiop
`
`Y
`
`'
`
`Record device
`
`.OfHomimary
`Record Das
`
`Petitioners Ex. 1001 Page 8
`
`Petitioners Ex. 1001 Page 8
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 8 of 13
`
`5,561,811
`
`“$8,553:
`
`
`
`Es;22m8:22.5:
`
`28$:o_§__&<
`
`”:2;25.soaE<w=cofio£om££cmDUMB{flue
`
`
`
`"WageHgfiwscgmémms
`
`
`
`ugésgzmefia8:8__&<so:
`
`28%22w8:22.5:
`
`anmaggozfié
`uEufiogmwefia
`€§§§83€
`
`"885533
`
`”$85
`
`E2303:5
`28mm8:8:&<
`
`DHH2%
`
`aszmsceuctog
`
`H321%
`
`“mtgggsa
`
`
`
`"2833122
`
`.
`
`'I'
`.
`. ‘cv-
`
`‘
`
`.
`
`'1
`
`
`
`
`
`-.. _P... .
`
`Petitioners Ex. 1001 Page 9
`
`Petitioners Ex. 1001 Page 9
`
`
`
`
`
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 9 of 13
`
`5,561,811
`
`FIG' 20
`
`Remove eveni
`E from queue
`U= User who
`generaied ihe even’r
`
`A irooi
`applica’rion
`
`|80
`
`l
`
`8|
`
`I82
`
`i8?)
`
`Le’r A=chiid I
`
`Direc’r evem‘
`Record to A
`
`Examine The user
`ins’rance s’ro’re
`records of A
`
`Find The user instance
`
`siaie record of
`
`U,call H R
`
`
`
`
`
`
`
`Lei child=R's poih to
`
`mouse focus opplicohon field
`
`
`
`
`Pass
`eveni
`
`fochnd
`
`
`
`
`find fronimos’r
`of A ’rhai has
`ihe mouse career
`
`I90
`
`
`
`
`Place eveni in
`mplicoiion
`queue ofA
`
`
`
`
`Does child
`exist
`
`Y
`
`the chiid
`
`I9I
`
`|92
`
`Petitioners Ex. 1001 Page 10
`
`Petitioners Ex. 1001 Page 10
`
`

`

`US. Patent
`
`Oct. 1, 1996
`
`Sheet 10 of 13
`
`5,561,811
`
`F/G.2/
`
`
`
`Pareni applica’rion
`
`A, wishes To pass a
`
`pointing eveni io
`
`
`chiid applicaiionC
`
`
`200
`
`20|
`
`Y
`
`N
`
`Has A
`205
`
`
`
`paused oi
`
`
`the reques’r
`
`of ano’rher
`I’OCGSS
`P
`Q
`
`209
`N
`
`204
`
`Y
`
`206
`
`202
`Transform
`
`coordinafes
`
`R
`
`ecor
`
`d A
`heaiihy
`
`as
`
`S
`
`
`
`C
`ei oniinue
`Okfiag 10
`false
`
`
`
`
`Y
`
`Discard
`even’r
`
`2'0
`
`Requesi A's app
`process to pause
`
`Wai’run’rii Apauses
`
`2H
`
`2'2
`
`Transform coordinaies
`
`2|3
`
`Res’rariL A's
`applicaiion process
`
`2'4
`
`207
`
`Transform
`0001111110183
`
`208
`
`. Se’r Coniinue
`Ok flag ’10
`
`free
`
`Petitioners Ex. 1001 Page 11
`
`Petitioners Ex. 1001 Page 11
`
`

`

`Oct. 1, 1996
`
`Sheet 11 of 13
`
`US. Patent
`'F/G‘. 24
`
`Request all applications
`
`to stop after processing
`current events
`
`
`
`5,561,811
`
`
`250
`
`25‘
`
`252
`
`
` Application paint
`
`
`request
`
`received
`
`Gather all
`point requests
`
`Compute rectangle
`that bounds all
`
`regions to change
`
`253 Request root
`
`
`application to redraw
`itself in the
`
`region, and recursively.
`
`ask child applic.
`to
`
`redraw themselves
`
`Restart
`all
`
`254
`
`
`
`applications
`
`Wait for next
`paint request
`
`Petitioners Ex. 1001 Page 12
`
`Petitioners Ex. 1001 Page 12
`
`

`

`U.S. Patent
`
`Oct. 1, 1996
`
`Sheet 12 of 13
`
`5,561,811
`
`
`
`322mg:52853?;am:2838hemofiucaweso
`
`mg:m:25
`
`hNGFk
`
`2:85
`+8.30
`
`2%
`
`$223
`$38
`
`838
`
`,‘/\$3:/_w\\Efimfiwo
`
`//123%run8.;\\\.Q
`
`
`...222822:58
`
`
`
`.\.N.#5:285285.31+Em,\E822251.2822th,,
`
`Petitioners Ex. 1001 Page 13
`
`Petitioners Ex. 1001 Page 13
`
`
`
`

`

`US. Patent
`
`Oct. 1,1996
`
`Sheet 13 of 13
`
`5,561,811
`
`Deliyering inpu’r and Refreshing the Display in lhe Presence of
`Mulhpe Users and Multiple Applicaiions
`
`' P
`
`etitioners Ex. 1001 Page 14
`
`Petitioners Ex. 1001 Page 14
`
`

`

`5,561,811
`
`1
`METHOD AND APPARATUS FOR PER-USER
`CUSTOMIZATION OF APPLICATIONS
`SHARED BY A PLURALITY OF USERS ON A
`SINGLE DISPLAY
`
`FIELD OF THE INVENTION
`
`This invention relates to a method and apparatus employ-
`ing a single display for a plurality of users, and is more in
`particular directed to a method and apparatus for establish-
`ing correspondence between the users and pointing devices
`in such a system. The invention is related to the invention -
`disclosed in US. patent application Ser. No. 07/974,036,
`filed on the same date as the present application, now US.
`Pat. No. 5,442,788, issued Aug. 15, 1995, the contents of
`which are incorporated by reference herein.
`
`BACKGROUND OF THE INVENTION
`
`Many modern groupware applications generally require
`that each of a number of participants have his or her own
`computer. There are many occasions, however, in which two
`or more people wish to collaborate in a single-computer
`situation. This may occur, for example, when one person
`visits another’s office, or when two or more people are using
`a large-screen computer, such as a Xerox LIVEBOARD®
`interactive display system, which is described in Proceed-
`ings of CHI,92, ACM Conference on Human Factors in
`Computing Systems, May 3—7, 1992, Monterey, Calif., pgs.
`599—607. for per-user customization. Those that do usually
`take advantage of the fact that each user has his or her own
`computer and is running a separate instance of an applica-
`tion. Prior systems have not permitted customization such
`that the same instance of the application and the same screen
`can be shared. The superscript identifications in the follow-
`ing discussion refer to the Bibliography at the end of this
`disclosure.
`
`Shared window systems1 in the past have allowed users to
`share standard applications over a network. Some systems of
`this type are of the What You See Is What I See (WYSY—
`WIS) type. Work on shared window systems has focused on
`sharing unmodified single—user applications, maintaining a
`networked conference, and floor control. These systems
`have experienced difliculties in maintaining consistent
`views between displays when local user preferences are
`honored.
`
`Networked conference-aware systems such as window
`systemsz, text editors3 and drawing tools4 allow users at
`different workstations to make changes simultaneously.
`These systems permit some types of local user preferencess.
`One simple system of this type is Commune4 which allows
`users to simultaneously draw with a stylus on the pages of
`a shared electronic notebook. This latter system shows the
`same image to all users and uses colored cursors and objects
`to distinguish among different users.
`SharedArk6 supports collaborative learning of physics by
`allowing users to simulate experiments. In this system, users
`move around a planar space that contains objects that they
`can manipulate. When users’ view of the plane overlap, the
`views are WYSIWIS in the shared region. Each user con-
`trols a pointer, labeled with the user’s name, that is visible
`on all conference screens. Any user can manipulate any
`object in his or her view. Buttons in this system can be
`applied to any object, and shared between users. This system
`also supports per-user pointers, shared buttons and objects,
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`lightweight locking of objects, and equal status of partici-
`pants.
`
`Several systems support meetings by allowing partici-
`pants, working at individual workstations, to control a large
`shared screen. Some use chalk-passing floor control, such as
`the Capture Lab7 Others allow simultaneous editing, such as
`Colabs. The Colab team, in reporting on the Cognoter tool,
`noted that simultaneous access to the screen improves
`brainstorming because users need not wait for others to
`complete an entry before making their own contribution.
`They also noted that shared user interfaces need to be
`simple, because there is little time for training in a group
`setting. In all of these systems, however, the number of
`active participants is limited by the number of machines and
`the participants must go to a special meeting room.
`
`SUMMARY OF THE MENTION
`
`The present invention is directed to the provision of a
`method and apparatus for sharing customizable software
`applications on a single display that overcomes the above
`discussed disadvantages of prior systems and that permits
`two or more persons to share the same instance of an
`application, exploying a common screen.
`The invention is directed to the solution of the following
`user interface problems:
`1. Customized simultaneous input. In a system where a
`plurality of users may invoke the same command in the
`same application by taking comparable actions using
`comparable, but distinct,
`input devices,
`it must be
`possible to vary the effect of that command depending
`on the preferences of the user who generated it.
`2. Registration. It must be possible to quickly register
`each input device with a user, and hence, with a set of
`preferences.
`3. Per-user feedback. The system must direct feedback to
`the correct user without unduly disturbing other users.
`In addition, the invention is directed to the solution of the
`following architectural problem:
`Replication. The data structures of each application must
`be replicated for each user to support per—user modes,
`selections, and preferences.
`In accordance with the invention, a special on-screen
`region, referred to herein as the “Home Area”, allows each
`user to claim input devices, such as mice, that move cursors.
`The user simply clicks on a Home Area with the cursor in
`order to establish the association between the user repre-
`sented by that Home Area and the input device. A user’s
`Home Area also provides a natural place to position the
`control panel for that user’s current application. These
`control panels are referred to herein as “Home Area menus”.
`In addition, the system must provide feedback to each
`user without confusing the other users. In accordance with
`the invention this result
`is achieved by displaying the
`feedback for each user in a different color, and at other times
`by displaying feedback for each user in a different region of
`the screen.
`
`The system in accordance with the invention supports
`multiple users, and also supports multiple applications. Such
`multiple applications can be spatially nested inside one
`another, forming a logical hierarchy where one application
`may be nested inside another. Users can act within this
`hierarchy at any level, with different users acting at different
`levels.
`
`In the system and method of the invention:
`
`Petitioners Ex. 1001 Page 15
`
`Petitioners Ex. 1001 Page 15
`
`

`

`3
`
`4
`
`5,561,811
`
`(1) Each user can have a different “current position”. For
`instance, one user can be adding text to a first paragraph
`of a document, while another user adds text to the third
`paragraph.
`(2) Each user can be operating in a different mode. For
`example, one user can be rotating one object while
`another user is creating new objects.
`(3) Each user can have different style settings. For
`example, when one user types, the new text appears in
`his or her chosen color or font. Another user can create
`text in a different font or color.
`
`(4) Feedback involving a particular user (e.g. what appli-
`cation that user is currently using, which input devices
`are associated with that user, what operation the user is
`performing, and what current modes, such as current
`color or current font, are in force) is displayed in a way
`that indicates which user that feedback is directed to.
`The user interface of the present invention is comprised
`of three visible components, namely:
`1. Home Areas for providing iconic representations of the
`users.
`
`2. Applications that allow the users to perform tasks, such
`as editing document media.
`3. Menus, which provide buttons that users can press to
`invoke commands.
`
`BRIEF FIGURE DESCRIPTION
`
`In order that the invention may be more clearly under-
`stood, it will now be disclosed in greater detail with refer-
`ence to the accompanying drawings, wherein:
`FIG. 1 is a block diagram of a system in accordance with
`the invention;
`FIG. 2 is a view of a screen display in accordance with
`one embodiment of the invention;
`FIG. 3 is a view of the Home Area of the screen display
`of FIG. 2;
`FIG. 4 is a view of the screen display of a Rectangle
`Editor;
`FIG. 5 is a view of the screen display of a Text Editor;
`FIG. 6 is a screen display illustrating a method for moving
`a rectangle in a rectangle editor, in accordance with the
`invention;
`FIG. 7 is a screen display illustrating resizing of a
`rectangle in a rectangle editor;
`FIG. 8 is a screen display illustrating the simultaneous
`resizing of a rectangle by manipulation of two pointing
`devices;
`
`FIG. 9 is a screen display illustrating the display of four
`editors on the screen display;
`FIG. 10 is a screen display illustrating a doubly selected
`rectangle;
`FIG. 11 is a screen display illustrating doubly selected text
`characters;
`FIG. 12 is a screen display illustrating a Home Area with
`rectangle creation color feedback;
`FIG. 13 is a screen display illustrating a Home Area with
`current font feedback;
`FIG. 14 is a flow diagram of the processing of an input
`from an external device such as a mouse;
`FIG. 15 illustrates the data structure of an Event Record;
`FIG. 16 is a flow diagram of processing of Event Records
`in the Home Area;
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`4o
`
`45
`
`50
`
`55
`
`60
`
`65
`
`FIG. 17 is a simplified diagram illustrating the interface
`between the Notify Process and the shared applications;
`FIG. 18 is a simplified illustration showing the updating
`of a screen, in accordance with the invention;
`FIG. 19 is a diagram illustrating the structure of an
`application;
`FIG. 20 is a flow diagram illustrating the handling of an
`Event Record by an application;
`FIG. 21 is a flow diagram illustrating the transformation
`of coordinates for a child editor;
`FIG. 22 illustrates a rectangle editor in which a rectangle
`contained in that editor and the editor itself are being
`simultaneously repositioned;
`FIG. 23 illustrates the results of the instructions of FIG.
`22, in the absence of the system of the invention; and
`FIG. 24 is a flow diagram of a Paint Process in accordance
`with the invention;
`FIG. 25 illustrates the involvement of user preferences in
`the processing of input events from two different users, both
`when an application is constructing a command and when
`the display is being updated; and
`FIG. 26 illustrates three applications that are receiving
`input from two input devices and updating a single display.
`
`DISCLOSURE OF PREFERRED
`EMBODIMENTS OF THE INVENTION
`
`Referring now to the drawings, FIG. 1 is a block diagram
`of a basic multi-user system that includes a computer 20,
`which may be a conventional microcomputer having RAM
`and program ROM, and a plurality of input devices 21, 22,
`23. The devices 21, 22 may be pointing devices, such as
`mouses with one or more mouse buttons, and the input
`device 23 may be a keyboard. It will be understood that the
`invention is not
`limited to this configuration of input
`devices, and that more input devices may be provided than
`herein illustrated. The computer system also is provided
`with a display screen 25 which may be a conventional
`computer monitor, or it may be a display device having a
`much larger surface area.
`The system is adapted to be employed by one or more
`users. For example, a single user may employ the two
`devices 21, 22, with the system reacting to inputs from the
`two devices in different manners. In this case, as discussed
`herein the two devices have different “user instances”.
`
`Alternatively, the two devices 21, 22 may be controlled by
`dilferent users, with the users controlling functions on the
`same application or in different applications. Further input
`devices may be assigned to other users, or be used for other
`user instances.
`
`The Display Screen
`
`A typical display that. may be provided on the display
`screen 25, in accordance with the invention, is illustrated in
`FIG. 2. This screen includes a command menu 30, a rect-
`angle editor 31 which may have a nested text editor 32, a text
`editor 33 and a Home Area 34. The illustrated editor screens
`are exemplary only, and other combinations of editor screens
`may be employed.
`In order to invoke any of the selections of the cormnand
`menu 30, the user controls his or her input device to place
`the cursor on the desired menu selection, and clicks or
`otherwise controls the device to make the selection.
`
`Petitioners Ex. 1001 Page 16
`
`Petitioners Ex. 1001 Page 16
`
`

`

`5
`
`6
`
`5,561,811
`
`The Rect Editor selection immediately turns all selected
`rectangles into rectangle editors, the Text Editor selection
`turns them into text editors, the Menu turns them into menus,
`the Color Menu turns them into color menus and the Delete
`selection deletes the selected rectangles. The New User
`selection creates a new Home Area for the user named by the
`selected text characters in the selecting user’s current appli—
`cation. The Print selection produces a file in a page descrip-
`tion language corresponding to the selecting user’s current
`application.
`A separate Home Area is employed for each user partici-
`pating in a session. Briefly, in order to join a session, a new
`user may enter his or her name into a keyboard or via a
`selection menu, followed by the clicking of the “New User”
`menu button. Other conventional techniques may of course
`be provided for creating a new Home Area. The system of
`the invention then displays a new Home Area that displays
`the new user’s name. The Home Area of a user may appear,
`for example, as illustrated in FIG. 3, wherein the user’s
`name is displayed at the top of the area. The user then moves
`the cursor to the name in the desired Home Area, as seen in
`FIG. 2, and clicks his or her mouse button or other input
`device. As a result, the system assigns the respective input
`device to that user, and any actions performed with the input
`device will take his or her preferences into account. The
`change of ownership of the input device may cause the
`cursor color to change to that of the user’s Home Area. A
`user may have more than one Home Area, which may be
`selected in the same manner as above discussed. Extra Home
`Areas allow users to switch back and forth between different
`
`sets of preferences. Each Home Area is said to belong to a
`different “user instance”.
`
`The applications may, for example only, be rectangle
`editors and text editors. These applications are discussed
`herein only to illustrate generally the function of an appli-
`cation, and the invention is not limited to these specific
`applications. The rectangle editor permits a user to create
`solid colored rectangles on the screen, to select and delete
`groups of rectangles, and to change their size, position and
`color. The text editor, on the other hand, allows users to
`place a starting point for a line of text, to enter characters,
`to choose a font and color, to move a line of text, to select
`a sequence of characters, and to delete or change the color
`of selected characters.
`
`Since the system of the invention is largely concerned
`with keyboardless interfaces, all editor commands, except
`for example text entries, are preferably activated by a
`pointing device such as a mouse. Command selection
`involves pointing to different parts (called “handles”) of
`graphical objects on the screen for the purpose of invoking
`difierent operations.
`For example, as illustrated in FIG. 4, the rectangle editor
`may display one or more rectangles, which may be nested.
`The rectangles may have a wide border or frame. The screen
`display of a text editor is illustrated in FIG. 5, wherein text
`may appear in desired fonts and colors.
`In order to move a rectangle, in the rectangle editor, from
`one position to another, without changing its size, the user
`places his or her cursor 61 on the border of the rectangle 60
`that is to be moved, as seen in FIG. 6, clicks and holds the
`input device button, and drags that border in any direction,
`until the desired position is reached. If, on the other hand, the
`user desires to change the size of a rectangle, the user places
`his or her cursor 61 on a comer of the rectangle 62, as seen
`in FIG. 7, clicks and holds the input device button, and drags
`that corner in a direction to change the lengths of the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`adjacent two sides of the rectangle, to thereby increase or
`decrease the size of the rectangle.
`A user may create a new rectangle on the screen by
`clicking his or her input device with the cursor on the
`background of the rectangle editor portion of the screen.
`Several users may simultaneously work in an application,
`performing different operations and using different modes.
`For example, in the rectangle editor, one user may drag one
`rectangle while another user resizes the rectangle, or while
`another user performs operations on another of the rect—
`angles. As seen in FIG. 8, two users via cursors 64, 65 may
`be simultaneously stretching the rectangle 66 in different
`directions. The rectangle editor keeps track of both opera-
`tions and updates the screen to show their progress. Sirni—
`larly, the users may employ the color menu 36 in order to
`control the color that rectangles will appear when created
`using the respective input device. For example, one user
`may choose blue as the current color for that editor while
`another user chooses red. The editor stores both users’
`
`modes and creates a rectangle of a color appropriate to the
`creating user.
`Users can work simultaneously on the same object. One
`user can add characters to a text string while another changes
`the color of existing characters. Very fine-grained sharing is
`also possible. One user can stretch a rectangle by one comer
`while another stretches its opposite comer, as seen in FIG.
`8, or one user can type a string of characters while another
`repositions the string.
`In FIGS. 6-8, the arrows indicate cursor motion; they are
`not visible on the screen.
`
`The system in accordance with the invention does not
`have a conventional window manager. Instead, the desktop
`is an instance of the rectangle editor. The rectangle editor
`allows other editors to be nested inside of it. Each editor
`presents a finite window onto an “infinite” plane that con-
`tains the objects and child editors that editor manages, which
`may overlap. The shape and position of this window is
`managed by the editor’s parent. Thus, FIG. 9 shows several
`rectangle editors A, B and D and a text editor C. The text
`editor C is partly hidden by sibling editor B and by a
`rectangle 70, and the rectangle editor D is partly outside of
`the porthole of its parent editor B. Each child application
`occupies a display sub-region of its parent’s display region.
`Thus, application A is the parent of child applications B and
`C, and application B is the parent of child application D.
`Because the window system of the invention is an editor,
`users do not need to learn both window manager and editor
`operations, unlike systems where selection of an editor is
`different from selection within an editor. Users can also
`
`place shared widgets in the document in which they are
`working; widgets do not need to remain at the top level.
`In single-user environments, graphical feedback is used to
`display aspects of system state, such as the current applica-
`tion mode, color, or insertion point. In accordance with the
`present invention, however, graphical feedback must show
`all of this information, as well as indicating the user to which
`the feedback applies. Color and spatial placement may be
`employed to indicate this correspondence.
`Each user has a color to identify his or her cursors,
`insertion points, and selections. As illustrated in FIG. 10,
`two users have selected a rectangle 72, 74 and each user’s
`selection is shown by highlighting a comer 75, 76 of the
`rectangle frame in that user’ s color. Similarly, FIG. 11 shows
`two selections in a text string, each marked by an underline
`77, 78 in one user’s color. The overlapping part of the
`selection is underlined twice.
`
`Petitioners Ex. 1001 Page 17
`
`Petitioners Ex. 1001 Page 17
`
`

`

`5,561,811
`
`7
`Likewise, a narrow band in the frame around a user’s
`current editor is set to that user’s color. Where several users
`
`are using the same editor, a band is colored for each user. If
`too many users have selected an object, such as a rectangle,
`text or editor, to identify them all in the frame, then only the
`most recent selections are shown.
`
`A user’s mode within his or her current editor is displayed
`in the lower half 40 of that user’s Home Area, as illustrated
`in FIG. 12. When a user works in a rectangle editor, his or
`her Home Area displays that user’s default color in that
`editor. Similarly, for a text editor, the current font and colors
`are illustrated in the users’s Home Area, as seen at reference
`numeral 41 in FIG. 13.
`
`In many desktop environments, menus are displayed once
`for each application window or at a unique location on the
`screen. For example, the Apple Macintosh pull-down menus
`are at the top of the screen. For a shared application on a
`single screen, however, menus displayed once per window
`take up much space and menus displayed at a fixed location
`only allow a single application to be used at any given time.
`In accordance with the invention, however, menus can be
`shared among users and editors, and may be positioned
`anywhere on the screen, even in documents. For example,
`the menus illustrated in FIG. 2 can be placed in a rectangle
`editor regardless of its nesting level, and then applied to
`objects in any user’s selected editor, regardless of that
`editor’s nesting level.
`‘
`Allowing people and editors to share menus reduces the
`screen real estate needed for control. Also, menus can be
`created or positioned where a user is working, avoiding user
`interference. Finally, users can group menus into a document
`to use as a customized control panel.
`Shared menus work well for commands, like changing
`color or deleting, that apply to several editors. Some func-
`tions, however, are specific to a particular editor. Menus of
`these functions need only be displayed when such an editor
`is in use. In the system of the invention, menus for a user’s
`selected editor are displayed in that user’s Home Area.
`These menus can be combined with feedback that shows
`
`current modes in the selected editor. For example, as illus-
`trated in FIG. 13, reference numeral 41, menus can display
`possible font choices with the user’s currently selected font
`highlighted.
`Users can simultaneously work at different levels in the
`application hierarchy. This capability reduces interference
`between users. One user can reposition or resize a document
`while another edits its contents. While some multi-level
`edits may disturb the user working at a lower lever, others
`work well. For example, a user may resize a rectangle editor
`to uncover an object beneath it without disturbing another
`user who is editing child rectangles that are far from the
`border being moved.
`The user interface of the invention requires a software
`architecture that is unusual
`in several respects. It must
`support a hierarchy of concurrently active applications. It
`must handle multiple devices and users, and make links
`between them. Each input event must be directed to the
`correct application, even though other input events may be
`modifying the application tree. Each application must store
`state for all of the users who have referenced it. Finally, the
`applications must coordinate their screen updates to produce
`a consistent image.
`The applications are arranged in a tree structure. The
`application at the root of the tree structure can have any
`number of child applications, each of which can have any
`number of children and so on. Each application has an
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`4o
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`
`application queue of input events and its own lightweight
`application process
`(or operating system “thread”)
`to
`dequeue and execute events. When an application receives
`an input event, it may act on the event immediately, place it
`on its queue for sequenced execution or pass it to a child
`application.
`At any moment, up to three applications are important to
`each user. A user’s selected application will receive his or
`her keyboard events and his or her commands generated by
`clicking on a menu. If a user performs an interactive
`operation, such as dragging a rectangle, for example, the
`application in which that operation began is the mouse focus
`application. The mouse focus application receives mouse
`motion and mouse button events until the interactive opera—
`tion completes, so it is possible to drag objects out of View.
`The most deeply nested visible applica