United States Patent 19
`Naef, III
`
`{54] METHOD AND APPARATUS FOR
`INTERACTIVE COMPUTER
`CONFERENCING
`
`(75]
`
`Inventor:
`
`Frederick E. Naef, Il, Reston, Va.
`
`[73] Assignee: Group Technologies, Inc., Arlington,
`Va.
`
`[21] Appl. No.: 393,876
`
`[22] Filed:
`
`Aug. 15, 1989
`
`[51] Ut, CVS cececcccccccccetcceseaseceesseeeeeseeee GOGF 15/16
`[52]
`UWS. CM. ececcccscssscconsseeseees 395/200; 364/DIG. 1;
`364/228.3; 364/228.4; 364/229; 364/259.9;
`364/259
`[58] Field of Search ... 364/200 MS File, 900 MSFile;
`395/200, 375
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`.
`
`4,414,621 11/1983) Bown et al. oo. 364/200
`4,441,162 4/1984 Lillie ...........
`ve 364/900
`eescseeeteeereees 379/54
`4,493,021
`1/1985 Agrawalet al.
`+ 364/200
`4,503,499
`3/1985 Masonet al.
`.
`+ 364/200
`4,551,720 11/1985 Levin ........
`1 340/825
`4,654,483
`3/1987 Imai et al...
`
`...
`4,686,698
`8/1987 Tompkinset al.
`379/53
`4,718,005
`1/1988 Feigenbaum et al.
`364/200
`4,768,150 8/1988 Changet al..
`364/300
`
`4,777,595 10/1988 Strecker et al.
`.. 364/200
`4,780,821 10/1988 Crossley .......
`. 364/200
`4,796,293
`1/1989 Blinken et al.
`ve 379/202
`4,814,984
`3/1989 Thompson....
`. 364/200
`
`4,825,354 4/1989 Agrawal etal. .
`1 364/200
`4,887,204 12/1989 Johnsonetal. ..
`v. 364/200
`
`4,897,781
`1/1990 Changetal. .....
`. 364/200
`4,937,784 6/1990 Masai et al.
`..........
`.- 364/900
`4,939,509 7/1990 Bartholomew et al.
`.
`ve 340/717
`
`4,949,248 8/1990 Caro oe .. 364/200
`
`+. 364/200
`4,949,254 8/1990 Shorter .
`4/1991 Bly et ab. oeascseree 364/900
`$,008,853
`OTHER PUBLICATIONS
`
`CRTTATA
`
`US005206934A
`(11) Patent Number:
`[45] Date of Patent:
`
`5,206,934
`Apr. 27, 1993
`
`Build a Network Painting Program”, MacTuior, vol. 5,
`No. 7, pp. 28-51 (Jul. 1989).
`J. Langowiski, “Forth Forum-Appletalk Protocol Han-
`dlers”, MacTutor, vol. 5, No. 7, pp. 52-57 (Jul. 1989).
`M. Stefik et al., “Beyond the Chalkboard: Computer
`Support
`for Collaboration and Problem Solving in
`Meetings”, Communications of the ACM, vol. 30, No. 1,
`pp. 32-47 (Jan. 1987).
`S. Sarin and I. Greif, “Computer-Based Real-Time
`Conferencing Systems”, Computer, vol. 18, No. 10, pp.
`33-45 (Oct. 1985).
`H. Forsdick et al., “Initial Experience with Multimedia
`Documents in Diamond”, Computer Message Service,
`Proceedings of IFIP 6.5 Working Conference, IFIP, pp.
`97-112 (1984).
`W. Pferd et al., “Interactive Graphics Teleconferenc-
`ing”, Computer, vol. 12, No. 11, pp. 62~72 (Nov. 1979).
`M. Brown and R. Sedgewick, “Techniques for Algo-
`rithm Animation”, JEEE Software, vol. 2, No. 1, pp.
`28-39 (Jan. 1985).
`
`Primary Examiner—Kevin A. Kriess
`Attorney, Agent, or Firm—Venable, Baetjer & Howard
`[57]
`ABSTRACT
`A method and apparatus for conferencing among a
`plurality of computers is described. A local user re-
`ceives inputs containing user actions and data forinter-
`pretation as to what the user action is with respect to
`the data and producesuser action outputs. An interpret-
`er/encoder interprets and encodes the user action out-
`puts into packets of encoded commands and data re-
`lated thereto. A communications device sends the en-
`coded information to at least one remote user and the
`local user itself. An interpreter decoder at each com-
`puter interprets and decodes the encoded information
`specifying an operation to be performed on the data.
`Theoperation is then executed at the local user and the
`remote users. The system is operating system indepen-
`dent and multiple documents and applications may be
`worked on at the same time by different users.
`
`E. Circenis and R. Magnuson, “Pascal Procedures--
`
`INTERPRETATION
`| aN AND. ENCODING
`
`34 Claims, 5 Drawing Sheets
`
`Petitioner Valve - Ex. 1013, Page 1
`Petitioner Riot Games,Inc. - Ex. 1013, p. 1
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 1
`
`Petitioner Valve - Ex. 1013, Page 1
`
`

`

`U.S. Patent
`
`Apr. 27, 1993
`
`Sheet 1 of 5
`
`5,206,934
`
`fo 112A 10
`|
`—
`
`! user Input|1-~
`|
`|
`14
`|
`16
`|
`1
`[i
`
`DATA
`
`INTERACT
`
`APPLICATION,
`
`|__|
`
`i|
`
`VISUAL ouTPuT||
`
`COMMUNICATIONS
`
`LINK (NETWORK,
`
`MODEM, ETC.)
`
`USER INPUT v
`2) eer
`|
`
`||
`
`|
`
`DATA
`
`| |
`
`VISUAL OUTPUT}!
`
` fomom
`ennee~~ 4
`
`FIG. 7
`
`Petitioner Valve - Ex. 1013, Page 2
`Petitioner Riot Games,Inc. - Ex. 1013, p. 2
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 2
`
`Petitioner Valve - Ex. 1013, Page 2
`
`

`

`U.S. Patent
`
`Apr. 27, 1993
`
`Sheet 2 of 5
`
`5,206,934
`
`
`
`—-WAODNOILVONdd¥
`
`
`
`JNIGOIS0SNOLLVOINA
`
`NOLVIGUdHaINI
`
`NOLWIGYdYAINI
`
`LY
`
`I_---_-_4______J|03-€ifatt30/2}=—+|“91gi€_uasnOF4001|———Srasnondy[ooOY
`
`eeTnOUherEYOvwainoy
`ONIGOOSGSNOLLWOINNNOUWI34duINI
`
`
`—-WAO9NOILVONdd¥INSAINOLOdd¥
`
`
`
`ONISSIO0ud>>|U#NOW||Z#NOL|]1#NOLLYZ#NOUVI}L#NOL|—ONdd¥||-oNMdav|]—ardd¥—Ndav|]—ONdd¥||
`ONSSIO0Udoo”LINIVd)
`s<-ERIORONEN]HGOORTo
`
`QNIGOONTCNYINonyornday|
`SNOLLWOINNWAOD|LAdLNO
`
`~1N09|
`SNOIL|—YOINN
`
`dowai!
`
`Petitioner Valve - Ex. 1013, Page 3
`Petitioner Riot Games,Inc. - Ex. 1013, p. 3
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 3
`
`Petitioner Valve - Ex. 1013, Page 3
`
`
`
`

`

`U.S. Patent
`
`Apr. 27, 1993
`
`Sheet 3 of 5
`
`5,206,934
`
`NOWVONddV¥
`
`
`
`4Q09JINGON
`
`INANND00
`
`YALN3GI
`
`d0Vd
`
`40SS3uCdv
`
`4OSSAYNddv
`
`YS0NIS
`
`YIAOIY
`
`404ZIS
`
`Vivd
`
`SNOIdO
`
`GNVNW09
`
`4q09
`
`Petitioner Valve - Ex. 1013, Page 4
`Petitioner Riot Games,Inc. - Ex. 1013, p. 4
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 4
`
`Petitioner Valve - Ex. 1013, Page 4
`
`

`

`U.S. Patent
`
`Apr. 27, 1993
`
`Sheet 4 of 5
`
`5,206,934
`
`REMOTE
`INPUT
`
`60
`
`“fT
`
`Lie
`
`cpu
`INTERRUPT
`PACKET
`INPUT LISTENER
`
`110
`
`112—T open[NETWORK|omnerJ''®
`108-1
`INPUT BUFFER
`PACKET La
`
`114
`
`18
`
`LOCAL USER
`INPUT
`
`
`
`
`NO
`
`LOCAL N—-
`PUT. PENDING?
`USER EVENT
`
`\.NO
`
`7
`GO 101
`
`40
`
`
` EVENT ROUTER:
`ROUTE 10 SPECIFIC
`APPLICATION}
`APP
`#3
`
`106
`REMOTE IN-\YES To 107,
`
`Y)
`PUT PENDING?
`AG, 45
`
`
`
`
`
` ASSEMBLE PACKET WITH
`
`om
`MODIFY DATA?
`120
`
`APP
`ft
`
`sat
`
`TO 48
`;
`Mo. ea
`
`EVENT
`APP
`fn
`
`42N
`
`428Y/ \
`
`/ x>-420~"MOVE"
`
`7
`"ORAW_\
`
`INTERPRET LOCAL
`USER EVENT
`
`ACTION-SPECIFIC INFORMATION
`
`PACKET
`
`FIG.4A
`
`SEND PACKET TO OTHER CON-
`PUTERS AND TO OURSELF
`
`PACKET
`
`COMMUNICATIONS OUTPUT
`|MODEM|NETWORK|OTHER
`
`Petitioner Valve - Ex. 1013, Page 5
`Petitioner Riot Games,Inc. - Ex. 1013, p. 5
`
`
`90 LOCAL OUTPUT
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 5
`
`Petitioner Valve - Ex. 1013, Page 5
`
`

`

`U.S. Patent
`
`Apr. 27, 1993
`
`Sheet 5 of 5
`
`5,206,934
`
`FROM 108,
`FIG. 4A
`FROM 106,
`FIG. 4A
`
`FROM 90,
`FIG. 4A
`
`PACKET
`
`RETRIEVE PACKET
`
`107
`
`
`COMMUNICATIONS ROUTER:
`ROUTE TO SPECIFIC
`APPLICATION
`
`FROM BUFFER
`
`
`APPLICATION DATA
`MANIPULATION FUNCTIONS
`ie.
`INSERT TEXT, PAINT
`
`
`APPLICATION COMMAND
`ROUTER:
`INTERPRET COM-
`MAND PACKET PROTOCOL
`
`
`
`Petitioner Valve - Ex. 1013, Page 6
`Petitioner Riot Games,Inc. - Ex. 1013, p. 6
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 6
`
`Petitioner Valve - Ex. 1013, Page 6
`
`

`

`1
`
`5,206,934
`
`METHOD AND APPARATUS FOR INTERACTIVE
`COMPUTER CONFERENCING
`
`BACKGROUND OF THE INVENTION
`
`5
`
`The invention relates to computer networks and com-
`munication systems. In particular, the invention relates
`to a method and apparatus incorporating a computer
`program for implementing collaborative conferencing
`between two or more users.
`In a traditional computer application, when a user
`performs an operation with an input device such as a
`mouse or keyboard, an event is generated by the operat-
`ing system and passed to the application program for
`execution. The standard application would interpret the
`event and perform operationsto changeits internal data
`based upon what the user requested. Similar applica-
`tions include, for example, performing a deletion step in
`a word processing program or creating a graphic ele-
`ment in a computer aided design (CAD) program.
`Whenthetraditional application is expanded from a
`single user into a multiple user environment, such as
`simulating a conference over a network or a modem,
`each user must be able to manipulate the same data and
`see all of the modifications madeto the data. To dothis,
`each machine must interpret and transmit information
`about whatthe user is doing and what theuseris seeing.
`There are three basic approaches to accomplish these
`results. These approaches correspond to three levels of
`data manipulation referred to as user input, data modifi-
`cation and visual display. Each approach intercepts and
`transmits information at the particular level. As a result,
`each approach hasspecific drawbacks and benefits.
`At the highest level, for example, namely the user
`input level, each machine intercepts its user input and
`transmits that input to other machines. For example, if
`the user clicked the mouse at a screen location (100,
`150), the machine would transmit a “mouse click” com-
`mand with those coordinates to other machines in the
`conference. The drawback of this approach is that it
`requires that all machines be set up in exactly the same
`mannerincluding identical hardware operating systems
`and applications. If one user has an icon or symbolin a
`different place on his screen than other users, then a
`mouse click on that icon on one machine will not be a
`mouse click on the icon on the other machines. As a
`result, communication between the people using the
`system quickly breaks down because they are each
`seeing different
`things on their
`respective screens,
`thereby making this method of conferencing unusable.
`At the lowest level, namely visual display, each ma-
`chine intercepts visual outputs to the screen and sendsit
`to all other users so that they see the same screen. A
`program which performs such a function is sometimes
`referred to as a remote contro] program. Under such
`schemes one machine acts as a host and contains all of
`the data that is manipulated. The other machines pass
`user inputs to the host machine and receive screen
`change commands back. A drawback of this method of
`conferencing is that it requires a large amountofinfor-
`mation to be transmitted in the form ofall or part of the
`screen memory and as a result it is relatively slow and
`inflexible. Also, because all the users in the conference
`share the same information, all users must possess suffi-
`Cient hardware and softwareto display the screen infor-
`mation including complex graphical representations.
`Both of the aforementioned methods manipulate data
`through one entry point. In other words, even though
`
`40
`
`45
`
`50
`
`a
`
`65
`
`2
`multiple users are working on a document, each user is
`seen as a single entity to the system. As a result, if one
`user selects text in a word processor and another user
`types the key, both operations will be considered as
`originating from one user, and the data may be acciden-
`tally destroyed. Thus, the user input and visual display
`systems are not truly collaborative wherein multiple
`users may modify the data concurrently.
`In mid-level conferencing, application primitives
`(units of information necessary to duplicate the user’s
`operation on remote machines) are sent from one ma-
`chine to the other. While this has been done to some
`extent at the research level, the drawback is that some
`centralized control is necessary to implement the ex-
`change of primitives. Another drawback is that high
`performance (speed) computers and expensive com-
`puter systems are necessary to implement such mid-
`level conferencingin real time. Further, with a central-
`ized architecture, performance limitations are incurred
`because at least two message transmission delays result
`between the originating work station and the controller
`and then back. While the time delay may be reduced by
`inter-work station communications, a controller work
`station is required and conflicting commands must be
`reversed or reexecuted.
`In summary then attempts have been made to pro-
`duce collaborative conferencing programsor architec-
`tures but these attempts have significant drawbacks.
`Someof the approaches rely on host, terminal] or master
`slave relationships among the various users. Some sys-
`tems only work with the same operating system. Some
`attempts merely allow one user at a time to access the
`data. In the case of remote systems, the controller re-
`quires interpretation of screen change commands which
`is slow because of the high data volume. In primitive
`passing systems the problem is alleviated but at the cost
`of central processing with high performance equipment.
`SUMMARYOF THE INVENTION
`
`Thepresent invention seeks to obviate the disadvan-
`tages and limitations of the described prior arrange-
`ments. In particular, the invention comprises a com-
`puter program and architecture for implementing con-
`ferencing among computers that can share information.
`The system includes means for each computer which
`controls communications and command routing. The
`system is such that any computer may communicate
`directly with any other connected computer. The com-
`mands from one computer are assembled as packets and
`are sent to itself and to other connected computers. The
`computer which is addressed receives the packets and
`converts and routes the commands to modify data and
`optionally to modify the user’s view of the data. Any
`connected computer maydirectly affect the other com-
`puters within the system. Data is distributed among
`each of the computers and notification of data changes
`is sent to other computers by packet messaging. Each
`computer has its own unique entry point or cursor for
`data entry and/or modification, and each has a copy of
`the entry point of each other computer. The computer
`which is addressed thus determines the proper entry
`point for the incoming data by determining who sent
`the packet.
`In a functional embodiment of the invention, each
`computer receives inputs from its local user which con-
`tains user actions and data and interprets what the user
`action is with respect to the data. This is manifested as
`
`Petitioner Valve - Ex. 1013, Page 7
`Petitioner Riot Games,Inc. - Ex. 1013, p. 7
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 7
`
`Petitioner Valve - Ex. 1013, Page 7
`
`

`

`5,206,934
`
`3
`user action outputs which are encoded into packets of
`commands and data. Encoded commandsand data are
`sent to at least one remote user and the local useritself
`for decoding in order to specify the operation to be
`performed on the data and to execute the operation.
`In an embodiment of the present
`invention, each
`computer has a conferencing program which includes
`an application program or module (sometimes simply
`referred to as application or module). An event routeris
`responsive to local user inputs for accessing a selected
`application and a documentsuch as a letter having data
`to be manipulated. An encoder interprets user com-
`mands for the application and assembles packets by
`identifying the local user, the application, the command
`and the document. A communications router responsive
`to the packet addresses the appropriate application and
`document. An application command router responsive
`to the packet interprets the packet command for manip-
`ulating the data in the selected application. The commu-
`nication router is also responsive to packets from other
`users for routing packets on to the appropriate applica-
`tion for execution.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The various objects, advantages and novel features of
`the present invention will be more readily understood
`from the following detailed description when read in
`conjunction with the appended drawings, in which:
`FIG. 1 is a simplified block diagram illustrating the
`general architecture of the conferencing system called
`Interact of the present invention;
`FIG. 2 is a schematic block diagram of the Interact
`conferencing system of the present invention employing
`a local and remote user in communication over a com-
`munications channel;
`information
`FIG. 3 is an illustration of a typical
`packet which is assembled and disassembled in accor-
`dance with the teachings of the present invention; and
`FIGS. 4A and 4B form a flow chart illustrating the
`operation of the Interact conferencing system illus-
`trated in FIG.1.
`.
`Throughout the drawings, like references numerals
`will be understood to refer to like parts or components.
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`
`FIG. 1 illustrates a general architecture of the confer-
`encing system 10 of the present invention. In the ar-
`rangement, each of a plurality of computers 12A-12N,
`which maybe similar or dissimilar, are coupled together
`over a common communications link 14 such as a net-
`work or a modem. Each computer 12A-12N,in addi-
`tion to having data processing capability in a CPU and
`random access memory (not shown), also has an Inter-
`act application 16 for managing the conference, a user
`input 18, a visual output and a body of identical but
`separate data 22. The Interact application 16 (sometimes
`referred to as Interact) also may include a word pro-
`cessing program, a graphics program and a paint pro-
`cessing program (sometimes referred to as applications
`or modules). CAD,spread sheet and other modules may
`also be provided. However, in order for conferencing
`to occurat least one application on each machine must
`be compatible with an application on another machine.
`For example, for conferencing to occur on a word pro-
`cessing level each machine must have a word process-
`ing module. The same is true for graphics or paint or
`computer aided design and the like. Also, each com-
`
`4
`puter may have its own operating system and different
`user inputs and visual outputs. The Interact application
`16 is specifically designed to handle each operating
`system and a variety of user inputs. However, even
`though it may be stored or formatted differently in
`different machines, the data 22 in each of the computers
`is conceptually equivalent but distinct.
`In accordance with the invention the user input 18 on
`the computer 12A to the Interact application 16 pro-
`duces a desired result on the data 22 in a selected docu-
`ment. For example,
`in the word processing module
`incorporated into the Interact application 16 a delete
`key stroke commands the application 16 to effect a
`delete command on the data 22 in the particular docu-
`ment. In another computer 12B, the user may be work-
`ing on another documentin a graphics module incorpo-
`trated into the Interact application 16. In such a case the
`user input 18 may be a mouse click in an icon displayed
`on the screen or visual output 20 of the user 12B. Each
`user input 18 in addition to being locally processed, that
`is sentto itself, is also communicated over the communi-
`cation link 14 as an input to the Interact application 16
`ofeach of the other-computers. The Interact application
`16 in each computer 12A-12N interprets the user input
`18 and performs the data manipulation. Similarly, a user
`input 18 effected in computer 12B is input to the Inter-
`act application 16 for manipulating the data 22 therein
`and likewise user inputs are communicated overthe link
`14 to the other computers in the form of remote inputs
`to their Interact applications 16 for manipulation of the
`data 22. All data manipulations assure that the resulting
`change is the same in each machine. Theeffect on the
`data 22 is either put in storage or may be immediately
`displayed on the corresponding visual output 20 of each
`computer 12A-12N.
`FIG. 2 is a more detailed illustration of the present
`invention in which a computer 12A,designated as the
`jocal user and a computer 12N,designated as the remote
`user, are shown coupled over the communication link
`14. In principle as few as two users as illustrated in FIG.
`2 or more may be accommodated if desired.
`Each computer 12A-12N is programmed with a set
`ofinstructions, i.e. the Interact program which manifest
`themselves in a variety of functions hereinafter de-
`scribed. It should be understood that the invention is
`not limited to a single operating system. Thus, each
`computer 12A-12N may have its own unique operating
`system such as MS-DOS, Unix or Macintosh OS and
`various machines may be used on the communication
`link for the conference as desired. Thus, each machine
`1ZA-12Nis programmed with a copy of Interact com-
`patible with its operating system in order to perform the
`various functions hereinafter described.
`All users in a conference have their own copy of the
`data 22 that they are manipulating, and each copy must
`be kept exactly the same as the other copies. As a result,
`if one user moves text or graphics on his machine, then
`the text or graphics must be moved on all other ma-
`chines in the conference. Also, one user may be moving
`text in a selected document while the other is moving
`graphics in another documentat the same time. In each
`case the data in the corresponding documentis changed
`or updated in each machinein the conferenceas a result
`of the user input. All information is passed between the
`machines 12A-12N concerning all the different data
`manipulations. When the users are working on the same
`documentat the same location within that document,
`the display of changes is shown to both users, otherwise
`
`0
`
`3
`
`20
`
`25
`
`3%
`
`35
`
`40
`
`45
`
`50
`
`35
`
`a
`
`65
`
`Petitioner Valve - Ex. 1013, Page 8
`Petitioner Riot Games,Inc. - Ex. 1013, p. 8
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 8
`
`Petitioner Valve - Ex. 1013, Page 8
`
`

`

`5,206,934
`
`5
`only data changes are made. As a result, multiple users
`can work on the sameor different documents using the
`same or different application modules in a collaborative
`environment.
`.
`An illustrative example could be characterized as
`follows. The user A at computer 12A wishes to work on
`a proposal in collaboration with the user B at computer
`12B. The proposal contains teat and graphics docu-
`ments. The present invention allows users A and B at
`the same time to enter changesin the text in a free forall
`collaboration. Similarly, they can work on graphics in
`the same way. They can each work independently
`within the conference on different parts of the text or
`graphics moving into and out of various application
`modules and documents at will. In the present inven-
`tion, then, user A can put a graph in the proposal and
`user B can add text to the graph or work on some other
`part of the proposal if he wishes. The users may also
`enter and leave the conference if desired.
`The present invention includes a numberoflevels of
`operation which have corresponding functionalities.
`For example, the event loop 30 is that level in which the
`users cause something to happen either locally or re-
`motely. The router 32 is that level which is responsive
`to an event loop 30 occurrence to be appropriately
`routed. The application layer 34 is responsive to the
`selection made at the router level 32 for interpreting the
`event and for appropriately encoding the event as a
`series of information packets. At the communications
`level 36, the packets are fed back to the event loop level
`30 of the local user 12A. In effect the local user sends
`the messageto itself. In addition, the packet is sent over
`line 14 to the remote users in the conference. In each
`case the local user and the remote users receive coded
`commands and data which are routed for execution and
`manipulation in each corresponding machine.
`In the event loop 30 at the local user 12A an event
`such as a key stroke or mouse click is sensed by the local
`user input 18. The input is then communicated to the
`Interact application 16. The event router 40 within the
`Interact application 16 interprets the event and selects
`an application module outof a plurality of such applica-
`tion modules 42A ... 42N. The particular application
`selected, e.g. 42N has an interpreter/encoder 44 (shown
`in exploded view) which interprets the event and en-
`codes the information. It should be understood that the
`other application modules 42A through 42N-1 have
`similar interpreter/encoders, but they are not shown in
`the drawing for clarity.
`In the example, the word processing application mod-
`ule 42N is incorporated into the Interact application 16
`and is designed specifically to operate with the commu-
`nication system of the present invention. The applica-
`tion 42N selected may, for example, interpret the key
`stroke or mouse click for a particular function such as a
`delete. The application event interpreter/encoder 44
`thus interprets the key stroke and produces encoded
`information to identify the application, the document,
`the command and command specific data such as the
`identity of the keys pressed and the data. An output of
`the interpreter/encoder 44 is coupled to a transaction
`manager 46 in the communications level 36. The trans-
`action manager 46 respondsto the encoded information
`and assembles it into a packet P for routing over line 47
`to a communication router 48 in the local user 12Aitself
`and to the communications block 50 in the communica-
`tions level 36. The communications block may include a
`modem 52, a network 54 or other appropriate communi-
`
`6
`cation means 56 (e.g. microwave) for connecting ma-
`chines whereby the communication link 14 communi-
`cates with one or more remote users 12N.In particular,
`the packets P are sent to the communication router 48
`of each remote user 12N via its respective remote input
`60.
`An exemplary packet P is shown in FIG. 3. Included
`therein are various forms of information necessary to
`allow the local user 12A and the remote user 12N to
`execute the input event. Thetitles for the various forms
`of information are indicated in bold uppercase letters.
`The ADDRESS OF RECEIVER,part of the packet,
`directs the information to the correct remote user. De-
`pending upon the numberofusers, a packetis prepared
`for each one.It is also within the teaching of the present
`invention to use a known multi-casting procedure in
`which a single packet is constructed with multiple re-
`mote user addresses. ADDRESS OF SENDERallows
`the receiver to know the source of the information so
`that it can determine the correct entry point, thatis,
`where in the document the data goes. DOCUMENT
`IDENTIFIERlocates the desired document for data
`entry. APPLICATION MODULE CODEidentifies
`_the application module to be used, for example, word
`processing. COMMAND CODEinstructs the applica-
`tion module to perform a certain function, for example,
`delete. SIZE OF DATA and DATAprovide com-
`mand-specific data used in performing the requested
`operation. OPTIONSis packet space made available for
`other desired enhancements that may be incorporated in
`the future. One possibility is a check sum which would
`allow the receiver to insure that no packet data was lost
`in the transmission. These codes are received by the
`communication router 48 of each user 12A-12N and are
`utilized as hereinafter described.
`The communication router 48 at the local user 12A is
`responsive to the packet P of information communi-
`cated by the transaction manager 46 to select or route
`the information to the application module 42N in accor-
`dance with the APPLICATION MODULE CODE.
`The application module 42N has an interpreter/decoder
`56 (shown in exploded view). Once the communication
`router 48 selects the application 42N for execution the
`interpreter/decoder 56 decodes and interprets the
`COMMAND CODEfor producing an output to the
`execution block 58 which manipulates the data in data
`block 22 and produces a visual output 20. Execution
`occurs in accordance with the decoded command and
`also in accordance with the ADDRESS OF SENDER
`information so that the execution occurs at the proper
`entry point. In this case, the sender and the receiver are
`one in the same, namely the local user 122A. SIZE OF
`DATA and DATA information is communicated so
`that the local user 12A can perform the operation.
`Thepacket P of information which is implemented at
`the local user 12A is also communicated over the link 14
`to the remote input 60 of the remote user 12N, which
`has a listener not shown but hereinafter discussed,
`which likewise includes a modem, a network or other
`communications device. The communication router 48
`at the remote user 12N responsive to the remote input
`60 performs a selection of the application 42N in the
`same wayas the local user 12A selected its application,
`namely in accordance with the APPLICATION MOD-
`ULE CODE.Similarly, documentselection, decoding
`and manipulation operations are performed in the same
`way that they were performed at the local user except
`that the remote user 12N is identified by the AD-
`
`10
`
`20
`
`30
`
`40
`
`45
`
`50
`
`35
`
`60
`
`65
`
`Petitioner Valve - Ex. 1013, Page 9
`Petitioner Riot Games,Inc. - Ex. 1013, p. 9
`
`Petitioner Riot Games, Inc. - Ex. 1013, p. 9
`
`Petitioner Valve - Ex. 1013, Page 9
`
`

`

`5,206,934
`
`8
`fected and other things necessary for housekeeping.
`Along with the protocol command, the PAINTappli-
`cation 42P adds data specific to that commandinto the
`packet. In the example, the packet contains information
`about where on the document
`the user clicked the
`mouse, and what fill pattern should be used to fill the
`screen.
`
`7
`DRESS OF RECEIVERportion of the packet P and
`the ADDRESS OF SENDERis the local user 12A.
`This allows the remote user 12N to correctly determine
`the entry point for the data. Thus, the data at the local
`user 12A and the data at the remote user 12N are ren-
`dered identical.
`The local user 12A may also receive a remote input
`60 from other users. This remote input 60 is likewise
`directed to the communication router 48, as shown, for
`subsequent
`interpretation decoding and execution.
`Thus, all remote inputs 60 and local
`inputs look the
`same to the machine and are handled in essentially the
`same manner.
`
`A more specific example follows in connection with
`an exemplary painting application called “PAINT”. In
`this example, the user input is an instruction to fill a
`region on the screen to thereby change the data in the
`document and display in the painting application mod-
`ule in the application layer 34 (FIG.2). The instruction
`is a mouse click somewhere in the painting document
`(i.e. at the entry point) coupled with a commandtofill
`the space around the entry point or cursor with a cer-
`tain fill pattern.
`At the event loop 30, the user input 18 indicates that
`the user has clicked the mouse. The event loop 30 is the
`main dispatcherofall operations in the system. Thus,all
`local user actions and incoming communications from
`other users are passed therethrough. The information
`represented by the local user input 18 is directed to the
`event router 40 whereit is interpreted and further dis-
`patched to the appropriate application, namely applica-
`tion 42P entitled “PAINT.” The event router 40 han-
`dles local user inputs such as mouse clicks and key
`strokes and passes the information on to be interpreted,
`encoded and transmitted to all the users includingitself.
`The communication router 48 handles the receipt of
`commands to modify the data. These commands can
`come from either the local user 12A or from remote
`users 12N. Thus, there is a logical separation between
`command interpretation codes and data manipulation
`codes.
`
`The event router 40 determines to which application
`42A ...42N the local user input 18 is directed. In the
`example, when the event router 40 receives the mouse
`click it decides that the click occurred in a painting
`documentandit passes that event to the painting mod-
`ule or (PAINT) application 42P. Using this hierarchical
`structure, applications can be separated from one an-
`other which thus simplifies the entire system and facili-
`tates the addition of new applications.
`The user command enters the painting module 42P of
`the application layer 34 where it is further interpreted,
`encoded, packeted and sent
`to the communication
`router 48 in a manner similar to the operations per-
`formed in the word processing module 42N described
`above.
`The communication router 48 of the local user 124
`receives and interprets the packet P which the local
`user 12A has produced and sent over the internal chan-
`nel 47. The packet is interpreted in order to identify the
`application module to which the packet is addressed,
`for example the PAINT module 42P, and to identify
`what operation or command the application module
`42P should perform on the data 22. In the example
`referred to herein,
`the packet application/command
`protocol will be (PAINT-application,fill-bucket). Also
`includedin the packet is information about whosent the
`command and to what document the commandis di-
`
`The assembled packet is then sent to the communica-
`tions layer 36 where it is transmitted to all the other
`10 users within the conference by means of the communi-
`cation block 50 appropriately coupled to computers
`12A-12N on the conference. The packet is sent over the
`channel 14 between the local user 12A and the remote
`users 12N by a connection hiding technique in which
`5 the various members of the conference are unaware of
`whether the system is using a network, a modem or
`other communication means. This allows various types
`of connections between machines to be added easily.
`The transaction manager 46 assembles and sends the
`20 packet to the communication router 48 of the local user
`12A and also to the communication router 48 of the
`remote user via the remote input 60. Thus