`
`
`
`
`
`
`
`
`
` of I98?)
`
`
`.FAIIT ATE
`'§$‘;:ER
`.
`.
`I
`PATEFT
`1
`71111111711?“
`
`NUMBER
`1531
`{Sal-F“
`m] L
`
`
`arm. uuunzh
`
`anemia unn-
`
`
`
`genie A. 315R, MDUNTRIN VIEW, tn.
`g1‘1‘CDNTINIJING 99112.:*ttaramxsmu‘gi/mssthsx
`_,=
`
`
`
`UERIFIED
`
`-
`
`;l
`
`
`
`
`
`
`
`**FDREIGN/F'CT RPPLICRTIUNS************
`VERIFIED
`
`
`
`' _Mff//j
`
`,-
`
`
`
`FfiflII'fl nrlnfllv fill?“
`
`i
`f
`_3§ USC 119 an?! m W
`Vprllln Inufiunowlufld
`lag , QAIND & JACOBS
`
`439 FIFTH AVENUE
`
`FOREIGN FILING LICENSE GRANTED- 12/01/92
`
`.’
`.gm7f.r .
`
`A‘I’T 0 RN EY ’S
`DOCKET NO.
`
`
`
`R
`
`..1'4—
`
`
`
`U5- DEPT-MBOMMM I W Offlfl—MLIM. ‘10-"!
`
` ..-. r1-m{m.W_-n. ...-. "'1'. _. ,
`
`
`
`
`I PARTS OF APPLICATION
`.f
`.
`1' FILED SEEARATELY
`
`'
`'
`
`NOTICE OF ALLOWANCE MAILEB
`
`FRET-“RED FOFI ISSUE
`
`‘ » '
`
`-
`
`CLAIMS ALLOWED
`
`
`‘ n
`omwma‘
`—
`A
`
`
`
`finlsIanlEx-minér
`‘61 0'} ,
`I
`.1
`
`5m .
`
` ISSUE CLASSIFICIRTION
`11:29
`
`'
`
`.
`
`n-W
`.
`
`
`WARNIHQ: Th3 lnlormatlon ditch-nod Mroln may be mtrlclad. Unaulhorhod dlaclosurn may bu
`
`
`pmhlhllod by Ihe United Silica Coda Tm. 35. Sections in. 181 and 36!.
`
`
`Penguin mama Illa v.5. PIIIM & Tradnmnrk omen Is mlrlcloa to sulhmlud all-Inlay“:
`and nonunion only.
`'
`
`
`
`m
`
`
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 1 of 134
`Page 1 of 134
`
`
`
`I‘-
`
`APPROVED Foa LICENSE D
`' m r'."
`.m- q ‘90 dd
`,n (p .r z.-
`INITIALS
`:
`
`Eritered
`'
`' CONTENTS -
`.
`.
`I Coflted
`
`I
`— 1. Appficafion #989916.
`
`Received
`Mamet!
`.. ”'5.[$1.1 ill-5h;
`1
`
`ii
`
`111/9140“
`__}\jL/lfl—1591999920
`
`.NPE'W
`
`
`
`
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 2 of 134
`Page 2 of 134
`
`
`
`ODE LA'E
`
`IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
`
`,
`
`us. PATENT APPLICATION
`
`SERiAL "uni
`
`APPLICANT
`
`07/97l+,ohl+
`
`11/10/92
`
`ERIC A. BIER, MOUNTAIN VIEW, CA.
`
`*fiCONTINUING DATAfiwkkkkwfl*kfiflflkfiflkflfifik
`.VERIFIED
`
`*kFORE|GN/PCT APPL|CAT|0NSR****k**R*#*
`VERIFIED
`
`
`
`FOREIGN FILING LICENSE GRANTED 12/01/92
`
`o
`-
`COUNTRY
`
`'
`
`’
`
`’
`
`r
`" ‘n'
`RECEIVED
`
`CA
`
`13
`
`$
`
`9110.00
`
`XRX—13h-(D/9
`
`ADDRESS
`
`ROSEN. DAINOW 5 JACOBS
`#89 FIFTH AVENUE
`NEW YORK, NY 10017
`
`METHOD AND APPARATUS FOR PER-USER CUSTOMIZATION OF APPLICATIONS
`SHARED BY A PLURALITY OF USERS ON A SINGLE DISPLAY
`
`This is to certify that annexed hereto is a true copy from the records of the United States
`Patent and Trademark Office of the application as filed which is identified above.
`
`the
`By authority of
`COMMISSIONER OF PATENTS AND TRADEMARKS
`
`Date
`
`Certifying Officer
`
`Nintendo Exhibit 1005
`
`Nintendo Exhibit 1005
`Page 3 of 134
`Page 3 of 134
`
`
`
`
`
`Docket No. XRx-134
`
`(D/92142)
`
`PATENT APPLICATION
`TRANSMITTAL LEI'TER
`
`To the commissioner of Patents and Trademarks:
`Transmitted herewith for filing is the patent application of
`R
`BI R
`
`
`STOMIZATION OF
`entitled METHOD AND APPARATUS FOR P R- SER
`
`APPLICATIONS SHARED BY A PLURR I 1' OF USERS ON A SINGLE DISPLAY
`
`M nclosed are:
`_;3____ sheet1s) of drawings
`an assignment of the invention to
`
`Xerox Corpgration
`
`ll“H<
`
`executed declaration of the inventorts)
`a certified copy of
`application No.
`the priority of which is hereby claimed.
`associate power of attorney
`a verified statement to establish small entity status under 37 CPR
`1.9 and 1.2?
`information disclosure statement
`preliminary amendment
`other:
`
`,
`
`llle
`
`CLAIRE AB FILED
`
`
`—m
`
`
`“M
`
`
`—_”u
`
`
`——-_n
`—mmm-
`
`
`*Numher extra must be zero or larger
`TOTAL
`Sgbfl
`
`
`It a
`iicant has small sntit
`stat. a under
`3}" CEB 1.9 and 1.23.
`than dialdo tgtll in by 2,
`and enter amount here.
`
`MRI!
`
`5
`
`MIT! TOTAL
`
`A check in the amount of 5
`enclosed.
`
`to cover the filing fee is
`
`x The Commissioner is hereby authorized to charge and credit Depoeit
`Account No. 24-003? as described below.
`I have enclosed a duplicate copy
`of this sheet.
`
`charge the amount of s 940-00
`3
`.3. Credit any overpayment
`x Charge any additional filing fees required under 37 CPR 1.16 and
`1.1?.
`
`as filing fee.
`
`({[jflfiqe
`Date
`
`
`
`Daniel M. Rose , Reg. No. 22,505
`ROSEN DAINOW f: JACOBS
`489 Fifth Avenue
`New York. NY 10017
`(212) 692-7000
`EXPRESS MAIL CERTIF Carr 3? CFR 1.10
`Express Hail Label No. EB 2&3 217 915 US
`Data of. DenomitllgJ 1'01 92
`so is being deposited
`I hereby certify L at.
`t.
`a paper or
`with the 13.3. Postal Ssrvice Express flail 2o Irfico LO Addrusss”
`service under 3? CH! 1.19 on the data indie
`. above and is
`
`addressed to Commissioner o! 9atsnts 5c Tra- -
`--
`
`
` 'ignaturn
`
`Thomas M. Webster
`Name 0! Person Malling
`
`10.392.
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 4 of 134
`Page 4 of 134
`
`
`
` Patent Application
`
`Attorney Docket No. D/XRX-134
`(D1292 142}
`
`em
`
`APPLICATION
`
`FOR
`
`UNITED STATES LETTERS PATENT
`
`Be it known that I, Eric A. Bier, a citizen of the United States of
`
`America have invented new and useful improvements in:
`
`Method and A are
`
`
`r-User Cgstpmization of Applications
`
`Shared by a Plurality of Users on a Single Display
`i...—._._———-——'_"'——_'_'——‘——-—-.._______
`
`of which the following is the specification:
`
`EXPRESS MAIL CERTIFICATE
`
`“Express Mai l" label number:
`
`RE 948 917 91 5 US
`
`Data of Deposit:
`
`November 10, 1992
`
`I hereby certify that the following attached paper or fee, an application entitled Method
`and Apparatus for Per-User Customization of Applications Shared by a Plurality of
`Users on a Single Display, is being deposited with the United States Postal Service “Express
`Mail Post Office to Addressee” service under 37 CFR 1.10 on the date indicated above and is
`addressed to the Commissioner of Patents and Trademarks, Washington. DC. 20231.
`
`Thomas M. Webster
`
`(Typed a)- printed name of person mailing paper or fee)
`
`
`
`
`Signature of person mailin paper or fee)
`
`
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 5 of 134
`Page 5 of 134
`
`
`
`
`
`"13997:! Odd/gt
`
`Attorney Docket 0/92142 - XRx134
`
`METHOD AND APPARATUS FOR PER-USER CUSTOMIZATION
`
`OF APPLICATIONS SHARED BY A PLURALITY OF
`
`USERS ON A SINGLE DISPLAY
`
`FIELD OF THE INVENTION
`
`This invention relates to a method and apparatus employing a
`
`single display for a plurality of users, and is more in particular
`
`directed to a method and apparatus for establishing correspondence
`
`between the users and pointing devices in such a
`
`system. The
`
`invention is related to the invention disclosed in U. 8. Patent
`
`Application Ser. NoW°i/?tyo3w yAxLQ;ngX_Donket_D%9£&4&—-waxlaaijL érfiéé
`filed on the same date as the present applicationthhe contents of
`which are/L
`rpo
`by reference herein.
`[5'
`BACKGROUND OF THE INVENTION
`Many mtdern groupware applications generally require that each
`
`15
`
`of a number of participants have his or her own computer. There are
`
`many occasions, however,
`
`in which two or more people wish to
`
`collaborate in a single-computer situation. This may occur,
`
`for
`
`example, when one person visits another‘s office, or when tw0 or
`
`more people are using a large-screen computer,
`We
`“a? m.
`
`
`
`such as a Xerox
`
`Many groupware computer systems do not provide any facilities
`
`
`
`Page 1
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 6 of 134
`Page 6 of 134
`
`
`
`for per—user customization. Those that do usually take advantage of
`
`the fact that each user has his or her own Computer and is running
`
`a separate instance of
`
`an application. Prior systems have not
`
`permitted customization such
`
`that
`
`the same
`
`instance
`
`of
`
`the
`
`application and the same screen can be shared. The superscript
`
`identifications
`
`in
`
`the
`
`following
`
`discussion
`
`refer
`
`to
`
`the
`
`Bibliography at the end of this disclosure.
`
`Shared window systemsl
`
`in the past haVe allowed users to
`
`share standard applications over a network. Some systems of this
`
`10
`
`type are of the What You See Is What I See (WYSYWIS)
`
`type. Work on
`
`shared window systems has focused on sharing unmodified single—user
`
`applications, maintaining a
`
`networked
`
`conference,
`
`and
`
`floor
`
`control. These systems have experienced difficulties in maintaining
`
`consistent views between displays when local user preferences are
`
`15
`
`honored.
`
`Networked conference-aware systems such as window systemsz'
`
`text
`
`editors3 and drawing tools4 allow users at different
`
`workstations to make changes simultaneously. These systems permit
`
`some types of local user preferences5. One simple system of this
`
`type is Commune4 which allows users to simultaneously draw with a
`
`stylus on the pages of a shared electronic notebook. This latter
`
`system shows the same image to all users and uses colored cursors
`
`and objects to distinguish among different users.
`
`SharedArk6 supports collaborative learning of physics
`
`by
`
`allowing users to simulate experiments. In this system, users move
`
`around a planar
`
`space
`
`that
`
`contains objects
`
`that
`
`they can
`
`20
`
`25
`
`Page 2
`
`."—)
`
`)
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 7 of 134
`Page 7 of 134
`
`
`
`manipulate. When users' view of the plane overlap,
`
`the views are
`
`WYSIWIS in the shared region. Each user controls a pointer,
`
`labeled
`
`with the user's name,
`
`that is visible on all conference screens.
`
`Any user can manipulate any object in his or her view. Buttons in
`
`this system can be applied to any object, and shared between users.
`
`This system also supports per-user pointers,
`
`shared buttons and
`
`objects,
`
`lightweight
`
`locking of objects,
`
`and equal
`
`status of
`
`participants.
`
`Several
`
`systems support meetings by allowing participants,
`
`working at
`
`individual workstations,
`
`to control
`
`a
`
`large shared
`
`screen. Some use chalk—passing floor control, such as the Capture
`
`Lab7. Others allow simultaneous editing, such as Colabs. The Colab
`
`team,
`
`in reporting on the Cognoter tool, noted that simultaneous
`
`access to the screen improves brainstorming because users need not
`
`wait
`
`for others to complete an entry before making their own
`
`contribution. They also noted that shared user interfaces need to
`
`be simple, because there is little time for training in a group
`
`setting.
`
`In all of these systems, however,
`
`the number of active
`
`participants
`
`is
`
`limited by
`
`the
`
`number of machines
`
`and
`
`the
`
`participants must go to a special meeting room.
`
`SUMMARY OF THE INVENTION
`
`The present invention is directed to the provision of a method
`
`and apparatus for sharing customizable software applications on a
`
`single display that overcomes the above discussed disadvantages of
`
`prior systems and that permits two or more persons to share the
`
`same instance of an application, exploying a common screen.
`
`Page 3
`
`10
`
`15
`
`20
`
`25
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 8 of 134
`Page 8 of 134
`
`
`
`The invention is directed to the solution of the following
`
`user interface problems:
`
`1.
`
`Customized simultaneous
`
`input.
`
`In a
`
`system where a
`
`plurality of users may invoke the same command in the
`
`same application by
`
`taking comparable actions using
`
`comparable, but distinct,
`
`input devices,
`
`it must be
`
`possible to vary the effect of that command depending on
`
`the preferences of the user who generated it.
`
`2.
`
`Registration.
`
`It must be possible to quickly register
`
`each input device with a user, and hence, with a set of
`
`preferences.
`
`3.
`
`Per-user feedback.
`
`The system must direct feedback to
`
`the correct user without unduly disturbing other users.
`
`In addition, the invention is directed to the solution of the
`
`following architectural problem:
`
`Replication.
`
`The data structures of each application
`
`must be replicated for each user to support per-user
`
`modes, selections, and preferences.
`
`In accordance with the invention, a special on—screen region,
`
`referred to herein as the "Home Area", allows each user to claim
`
`input devices,
`
`such as mice,
`
`that move cursors. The user simply
`
`clicks on a Home Area with the cursor in order to establish the
`
`association between the user represented by that Home Area and the
`
`input device. A user's Home Area also provides a natural place to
`
`position the control panel
`
`for that user's current application.
`
`These control panels are referred to herein as "Home Area menus".
`
`Page 4
`
`10
`
`15
`
`2O
`
`25
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 9 of 134
`Page 9 of 134
`
`
`
`In addition,
`
`the system must provide feedback to each user
`
`without consting the other users. In accordance with the invention
`
`this result is achieved by displaying the feedback for each user in
`
`a different color, and at other times by displaying feedback for
`
`each user in a different region of the screen.
`
`The system in accordance with the invention supports multiple
`
`users,
`
`and also supports multiple applications. Such multiple
`
`applications can be spatially nested inside one another,
`
`forming a
`
`logical hierarchy where one application may be nested inside
`
`another. Users can act within this hierarchy at any level, with
`
`different users acting at different levels.
`
`In the system and method of the invention:
`
`{1)
`
`Each user can have
`
`a different
`
`l'current position". For
`
`instance, one user can be adding text to a first paragraph of
`
`a document, while
`
`another user
`
`adds
`
`text
`
`to the third
`
`paragraph.
`
`(2)
`
`Each user can be operating in a different mode. For example,
`
`one user can be rotating one object while another user is
`
`creating new objects.
`
`(3}
`
`Each user can have different style settings. For example, when
`
`one user types,
`
`the new text appears in his or her chosen
`
`color or font. Another user can create text in a different
`
`font or color.
`
`(4}
`
`Feedback involving a particular user {e.g. what application
`
`that user
`
`is currently using, which
`
`input devices are
`
`associated with that user, what operation the user
`
`is
`
`[f
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 10 of 134
`Page 10 of 134
`
`
`
`performing, and what current modes, such as current color or
`current
`font,
`are din force)
`'is displayed in a way
`that
`
`indicates which user that feedback is directed to.
`
`The user interface of the present
`
`invention is comprised of
`
`three visible components, namely:
`
`1.
`
`2.
`
`Home Areas for providing iconic representations of the users.
`
`Applications that allow the users to perform tasks,
`
`such as
`
`editing document media.
`
`3.
`
`Menus, which provide buttons that users can press to invoke
`
`10
`
`commands.
`
`BRIEF FIGURE DESCRIPTION
`
`In order that the invention may be more clearly understood, it
`
`will now be disclosed in greater detail with reference to the
`
`15
`
`accompanying drawings, wherein:
`m/fig.
`1 is a block diagram of a system in accordance with the
`
`invention;
`is a view of a screen display in accordance with one
`2
`x/gig'
`emhodiment of the invention;
`
`yufig.
`
`3
`
`is a View of the Home Area of the screen display of
`
`20
`
`Fig. 2;
`
`*fig;
`Fig.
`
`4 is a View of the screen display of a Rectangle Editor:
`5 is a view of the screen display of a Text Editor:
`
`‘Fig.
`
`6 is a screen display illustrating a method for moving a
`
`rectangle in a rectangle editor,
`
`in accordance with the invention;
`
`25
`
`Fig.
`
`7
`
`is
`
`a
`
`screen display illustrating resizing of
`
`a
`
`rectangle in a rectangle editor;
`
`Page 6
`
`\
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 11 of 134
`Page 11 of 134
`
`
`
`/"
`
`/§ig.
`
`8
`
`is a screen display illustrating the simultaneous
`
`/
`resizing of a rectangle by manipulation of two pointing devices;
`
`fig.
`
`/
`
`9 is a screen display illustrating the display of four
`
`editors on the screen display:
`
`Eig.
`;/
`rectangle:
`
`10 is a screen display illustrating a doubly selected
`
`_Fig. 11 is a screen display illustrating doubly selected text
`
`//
`characters;
`
`Fig.
`
`12
`
`is a screen display illustrating a Home Area with
`
`rectangle creation color feedback;
`
`_Fig.
`
`13
`
`is a screen display illustrating a Home Area with
`
`current font feedback;
`
`,Fig.
`
`14 is a flow diagram of the processing of an input from
`
`an external device such as a mouse:
`
`Fig. 15 illustrates the data structure of an Event Record:
`
`Fig.
`
`16 is a flow diagram of processing of Event Records in
`
`the Home Area:
`
`' Fig.
`
`17 is a simplified diagram illustrating the interface
`
`between the Notify Process and the shared applications;
`/Fig. 18 is a simplified illustration showing the updating of
`a screen,
`in accordance with the invention:
`
`Ifig.
`
`19
`
`is
`
`a diagram illustrating the structure of
`
`an
`
`application:
`
`Fig.
`
`20 is a flow diagram illustrating the handling of an
`
`Event Record by an application:
`
`Fig. 21 is a flow diagram illustrating the transformation of
`
`Page 7
`
`10
`
`15
`
`20
`
`25
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 12 of 134
`Page 12 of 134
`
`
`
`coordinates for a child editor;
`
`//éig.
`
`22 illustrates a rectangle editor in which a rectangle
`
`contained
`
`in
`
`that editor
`
`and
`
`the editor
`
`itself
`
`are being'
`
`simultaneously repositioned;
`
`ffiia. 23 illustrates the results of the instructions of Fig.
`
`22,
`
`in the absence of the system of the invention: and
`
`/Ifig.
`
`24 is a flow diagram of a Paint Process in accordance
`
`with the invention:
`”Fig: 25 illustrates the involvement of user preferences in the
`
`processing of input QVEHtSAffiffi
`
`twa different users, both when an
`
`application is constructing a command and when the display is being
`
`updated§,and
`fig.
`26 illustrates three applications that are receiving
`input from two input devices and updating a single display.
`DISCLOSURE OF PREFERRED EMBODIMENTS OF THE INVENTION
`
`Referring now to the drawings, Fig. l is a block diagram of a
`
`basic multi-user system that includes a computer 20, which may be
`
`a conventional microcomputer having RAM and program ROM,
`
`and a
`
`plurality of input devices 21, 22, 23. The devices 21, 22 may be
`
`pointing devices, such as mouses with one or more mouse buttons,
`
`and the input device 23 may be a keyboard. It will be understood
`
`that the invention is not
`
`limited to this configuration of input
`
`devices, and that more input devices may be provided than herein
`
`illustrated. The computer system also is provided with a display
`
`screen 25 which may be a conventional Computer monitor, or it may
`
`be a display device having a much larger surface area.
`
`Page 8
`
`10
`
`15
`
`20
`
`25
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 13 of 134
`Page 13 of 134
`
`
`
`The system is adapted to be employed by one or more users. For
`
`example, a single user may employ the two devices 21, 22, with the
`
`system reacting to inputs
`
`from the two devices
`
`in different’
`
`manners.
`
`In this case, as discussed herein the two devices have
`
`different "user instances". Alternatively,
`
`the two devices 21, 22
`
`may be controlled by different users, with the users controlling
`
`functions on the same application or
`
`in different applications.
`
`Further input devices may be assigned to other users, or be used
`
`for other user instances.
`
`The Display Screen
`
`A typical display that may be provided on the display screen
`
`25,
`
`in accordance with the invention,
`
`is illustrated in Fig. 2.
`
`This screen includes a command menu 30, a rectangle editor 31 which
`
`may have a nested text editor 32, a text editor 33 and a Home Area
`
`34. The illustrated editor screens are exemplary only, and other
`
`combinations of editor screens may be employed.
`
`In order to invoke any of the selections of the command menu
`
`30,
`
`the user controls his or her input device to place the cursor
`
`on the desired menu selection, and clicks or otherwise controls the
`
`10
`
`15
`
`20
`
`device to make the selection.
`
`The Rect Editor selection immediately turns all
`
`selected
`
`rectangles into rectangle editors,
`
`the Text Editor selection turns
`
`them into text editors,
`
`the Menu turns them into menus,
`
`the Color
`
`Menu turns them into color menus and the Delete selection deletes
`
`25
`
`the selected rectangles. The New User selection creates a new Home
`
`Area for the user named by the selected text characters in
`
`Page 9
`
`KC
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 14 of 134
`Page 14 of 134
`
`
`
`the selecting User's current application. The Print selection
`
`produces a file in a page description language corresponding to the
`
`selecting user's current application.
`
`A separate Home Area is employed for each user participating
`
`in a session. Briefly,
`
`in order to join a session, a new user may
`
`enter his or her name
`
`into a keyboard or via a selection menu,
`
`followed by the clicking of
`
`the "New User" menu button. other
`
`conventional
`
`techniques may of course be provided for creating a
`
`new Home Area. The system of the invention then displays a new Home
`
`Area that displays the new user‘s name. The Home Area of a user may
`
`appear, for example, as illustrated in Fig. 3, wherein the user's
`name is displayed at the top of the area. The user then moves the
`
`cursor to the
`
`name in the desired Home Area, as seen in Fig. 2,
`
`and clicks his or her mouse button or other input device. As a
`
`result,
`
`the system assigns the respective input device to that
`
`user, and any actions performed with the input device will take his
`
`or her preferences into account. The change of ownership of
`
`the
`
`input device may cause the cursor color to change to that of the
`
`user‘s Home Area. A user may have more than one Home Area, which
`
`may be selected in the same manner as above discussed. Extra Home
`
`Areas allow users to switch back and forth between different sets
`
`of preferences. Each Home Area is said to belong to a different
`
`"user instance".
`
`The applications may,
`
`for example only, be rectangle editors
`
`and text editors. These applications are discussed herein only to
`
`illustrate generally the function of
`
`an application,
`
`and the
`
`10
`
`15
`
`2O
`
`25
`
`Page 10
`
`11/
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 15 of 134
`Page 15 of 134
`
`
`
`invention’
`
`is not
`
`limited to these specific applications. The
`
`rectangle editor permits a user to create solid colored rectangles
`
`on the screen,
`
`to select and delete groups of rectangles, and to
`
`change their siZe, position and color. The text editor, on the
`
`other hand, allows users to place a starting point for a line of
`
`text,
`
`to enter characters,
`
`to choose a font and color,
`
`to move a
`
`line of text,
`
`to select a sequence of characters, and to delete or
`
`change the color of selected characters.
`
`Since the system of the invention is largely concerned with
`
`keyboardless interfaces, all editor commands, except for example
`
`text entries, are preferably activated by a pointing device such as
`
`a mouse. Command selection involves pointing to different parts
`
`(called "handles") of graphical objects on the screen for
`
`the
`
`purpose of invoking different operations.
`
`For example, as illustrated in Fig. 4,
`
`the rectangle editor
`
`may display one or more rectangles, which may be nested. The
`
`rectangles may have a wide border or frame. The screen display of
`
`a text editor is illustrated in Fig. 5, wherein text may appear in
`
`desired fonts and colors.
`
`In order to move a rectangle,
`
`in the rectangle editor,
`
`from
`
`one position to another, without changing its size,
`(v1
`his or her cursorAOn the border of the rectanglekthat
`
`the user places
`
`is to be
`
`moved, as seen in Fig. 6, clicks and holds the input device button,
`
`and drags that border in any direction, until the desired position
`
`is reached. If, on the other hand,
`
`the user desires to change the
`(01
`the user places his or her cursor on a corner
`
`size of a rectangle,
`
`10
`
`15
`
`20
`
`25
`
`Page 11
`
`f/ z
`
`
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 16 of 134
`Page 16 of 134
`
`
`
`1
`
`b9
`of the rectangle! as seen in Fig. 7, clicks and holds the input
`
`device button, and drags that corner in a direction to change the
`
`lengths of the adjacent
`
`two sides of
`
`the rectangle,
`
`to thereby
`
`increase or decrease the size of the rectangle.
`
`5
`
`A user may create a new rectangle on the screen by clicking
`
`his or her input device with the cursor on the background of the
`
`rectangle editor portion of
`
`the
`
`screen. Several users may
`
`simultaneously work
`
`in an
`
`application,
`
`performing different
`
`operations and using different modes. For example,
`
`in the rectangle
`
`10
`
`editor, one user may drag one rectangle while another user resizes
`
`the rectangle, or while another user performs operations on another
`
`;
`
`of
`the
`rectangles.
`As
`seen
`in Fig.
`8,
`two users4 may
`be
`simultaneously stretching the rectanglejggn different directions.
`The rectangle editor keeps track of both operations and updates the
`
`15
`
`screen to show their progreSs. Similarly,
`
`the users may employ the
`
`color menu 36 in order to control
`
`the color that rectangles will
`
`appear when created using the respective input device. For example,
`
`one user may choose blue as the current color for that editor while
`
`another user chooses red. The editor stores both users' modes and
`
`20
`
`creates a rectangle of a color appropriate to the creating user.
`
`Users can work simultaneously on the same object. One user can
`
`add characters to a text string while another Changes the color of
`
`existing characters. Very fine—grained sharing is also possible.
`
`\\85
`
`One user can stretch a
`rectangle by one corner while another
`.
`.
`.
`.
`stretches its opposite corner, as seen in Fig. 8, or one user can
`
`‘30.”?
`
`P 0T’/2> type a string of characters while another repositions the string.
`/
`Page 12
`!’>
`
`I
`
`'
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 17 of 134
`Page 17 of 134
`
`
`
`The system in accordance with the invention does not have a
`
`conventional window manager. Instead, the desktop is an instance of
`
`the rectangle editor. The rectangle editor allows other editors to
`
`be nested inside of it. Each editor presents a finite window onto
`
`5"
`
`an "infinite" plane that contains the objects and child editors
`
`that editor manages, which may overlap. The shape and position of
`
`this window is managed by the editor's parent. Thus, Fig.
`
`9 Shows
`
`a.
`
`q\
`
`\10
`
`15
`
`20
`
`25
`
`several rectangle :dftfi field a text editor?” The text editor C is
`
`4
`
`7o
`partly hidden by sibling editor B and by a rectangle,
`
`and the
`
`rectangle iditor D is partly outside of the porthole of its parent
`editor B33"
`
`Because the window system of the invention is an editor, users
`
`do not need to learn both window manager and editor operations,
`
`unlike systems where selection of an editor is different
`
`from
`
`selection within an editor. Users can also place shared widgets in
`
`the document
`
`in which they are working: widgets do not need to
`
`remain at the top level.
`
`In single-user environments, graphical
`
`feedback is used to
`
`display aspects of system state, such as the current application
`
`mode, color, or insertion point.
`
`In accordance with the present
`
`invention,
`
`however, graphical
`
`feedback must
`
`show all of
`
`this
`
`information, as well as indicating the user to which the feedback
`
`applies. Color and spatial placement may be employed to indicate
`
`this correspondence.
`
`Each user has
`
`a color
`
`to identify his or her cursors,
`
`insertion points, and selections. As illustrated in Fig. 10,
`
`two
`
`Page 13
`
`L!
`
`5
`
`(I
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 18 of 134
`Page 18 of 134
`
`
`
`'12., 'T'i
`users have selected a rectangle, and each user's selection is shown
`151v
`by highlighting a cornergof the rectangle frame in that user's
`
`color. Similarly, Fig. 11 shows two selections in a text string,
`H? i?I
`each marked by an underlinefiin one user's color. The overlapping
`
`part of the selection is underlined twice.
`
`Likewise, a narrow band in the frame around a user‘s current
`
`editor is set to that user's color. Where several users are using
`
`the same editor, a band is colored for each user. If too many users
`
`have selected an object,
`
`such as a rectangle,
`
`text or editor,
`
`to
`
`identify them all
`
`in the frame,
`
`then only the most
`
`recent
`
`selections are shown.
`
`A user‘s mode within his or her current editor is displayed in
`
`the lower half 40 of that user‘s Home Area, as illustrated in Fig.
`
`12. When a user works in a rectangle editor, his or her Home Area
`
`displays that user's default color in that editor. Similarly,
`
`for
`
`a text editor,
`
`the current font and colors are illustrated in the
`
`users's Home Area, as seen at reference numeral 41 in Fig. 13.
`
`In many desktop environments, menus are displayed once for
`
`each application window or at a unique location on the screen. For
`
`example,
`
`the Apple Macintosh pull-down menus are at the top of the
`
`screen. For a shared application on a single screen, however, menus
`
`displayed once per window take up much space and menus displayed at
`
`a fixed location only allow a single application to be used at any
`
`given time.
`
`In accordance with the invention, however, menus can be shared
`
`among users and editors,
`
`and may be positioned anywhere on the
`
`10
`
`15
`
`20
`
`25
`
`Page 14
`
`/. {Sm
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 19 of 134
`Page 19 of 134
`
`
`
`screen, even in documents. For example,
`
`the menus illustrated in
`
`Fig.
`
`2 can be placed in a
`
`rectangle editor regardless of
`
`its
`
`nesting level, and then applied to objects in any user's selected
`
`editor, regardless of that editor‘s nesting level.
`
`Allowing people and editors to share menus reduces the screen
`
`real estate needed for control. Also, menus can be created or
`
`positioned where a user is working, avoiding user interference.
`
`Finally, users can group menus
`
`into a document
`
`to use as
`
`a
`
`customized control panel.
`
`10
`
`Shared menus work well for commands,
`
`like changing color or
`
`deleting,
`
`that apply to several editors. Some functions, however,
`
`are specific to a particular editor. Menus of these functions need
`
`only be displayed when such an editor is in use.
`
`In the system of
`
`the invention, menus for a user's selected editor are displayed in
`
`15
`
`that user's Home Area. These menus can be combined with feedback
`
`that shows current modes
`
`in the selected editor. For example, as
`
`illustrated in Fig. 13,
`
`reference numeral 41, menus can display
`
`possible font choices with the user's currently selected font
`
`highlighted.
`
`1
`
`Users can simultaneously work at different
`
`levels in the
`
`application hierarchy. This capability reduces interference between
`
`users. One user can reposition or resize a document while another
`
`edits its contents. While some multi—level edits may disturb the
`
`user working at a lower lever, others work well. For example,
`
`a
`
`user may resize a rectangle editor to uncover an object beneath it
`
`without disturbing another user who is editing child rectangles
`
`20
`
`25
`
`Page 15
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 20 of 134
`Page 20 of 134
`
`
`
`that are far from the border being moved.
`
`The user
`
`interface of
`
`the invention requires
`
`a
`
`software
`
`architecture that is unusual
`
`in several respects. It must support
`
`a hierarchy of concurrently active applications. It must handle
`
`5
`
`multiple devices and users, and make links between them. Each input
`
`event must be directed to the correct application,
`
`even though
`
`other
`
`input events may be modifying the application tree. Each
`
`application must
`
`store state for all of
`
`the users who have
`
`referenced it. Finally,
`
`the applications must coordinate their
`
`10
`
`screen updates to produce a consistent image.
`
`The applications are arranged in a
`
`tree structure. The
`
`application at the root of the tree structure can have any number
`
`of child applications,
`
`each of which can have
`
`any number of
`
`children and so on. Each application has an application queue of
`
`15
`
`input events
`
`and its own
`
`lightweight application process
`
`(or
`
`operating system "thread")
`
`to dequeue and execute events. When an
`
`application receives an input event,
`
`it may act
`
`on the event
`
`immediately, place it on its queue for sequenced execution or pass
`
`it to a child application.
`
`20
`
`At any moment, up to three applications are important to each
`
`user.
`
`A user's selected application will
`
`receive his or her
`
`keyboard events and his or her commands generated by clicking on a
`
`menu. If a user performs an interactive operation, such as dragging
`
`a rectangle, for example,
`
`the application in which that operation
`
`25
`
`began is the mouse focus application. The mouse focus application
`
`receives mouse motion and mouse button events until the interactive
`
`Page 16
`
`/‘127
`
`Nintendo Exhibit 1005
`Nintendo Exhibit 1005
`Page 21 of 134
`Page 21 of 134
`
`
`
`operation completes, so it is possible to drag objects out of view.
`
`The most deeply nested visible application that contains a user‘s
`
`cursor
`
`is his or her containing application.
`
`If no interactive
`
`operation is in progress,
`
`the containing application receives all
`
`mouse motion and mouse button events.
`
`The Processing of Input Events
`
`As illustrated in the flow diagram of Fig. 14, when an input
`device is controlled to provide an input (block4gégs to the system,
`i.e. to create an "event",
`the system initially starts building an
`
`Event Record (block 101)
`
`that includes the time of the event,
`
`the
`
`identification of the device that created the event, and the nature
`
`of the event. The nature of the event may be the pressing of a
`
`keyboard key,
`
`the movement of a mouse,
`
`the clicking of a mouse,
`
`etc. The above actions are taken by a thread called the device
`
`process. The device process then places the Event Record on system
`
`queue (block 102).
`
`The device process runs at a high priority so that the time-
`
`stamps will b