European Patent Office
`Office europeen des brevets
`
`^ ^ ^ ^ I ^ ^ ^ ^ ^ ^ II ^ ^ ^ II ^ II ^ ^ ^ ^ ^ ^ ^ ^ ^ I ^
`
`E P 0 9 3 9 5 1 6 A 2
`
`EUROPEAN PATENT APPLICATION
`
`(43) Date of publication:
`01.09.1999 Bulletin 1999/35
`
`(21) Application number: 99660008.6
`
`(22) Date of filing: 19.01.1999
`
`(51) |nt CI H04L 12/26, H04L 29/06,
`G06F 1 1/34, G06F 17/30
`
`(84) Designated Contracting States:
`AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU
`MC NL PT SE
`Designated Extension States:
`AL LT LV MK RO SI
`
`(72) Inventors:
`• Robinson, Michael
`Kingston on Thames, Surrey KT2 5JA (GB)
`• Pekkola, Samuli
`401 00 Jyvaskyla (Fl)
`
`(30) Priority: 19.01.1998 Fl 980096
`
`(71) Applicants:
`• Robinson, Michael
`Kingston on Thames, Surrey KT2 5JA (GB)
`• Pekkola, Samuli
`40100 Jyvaskyla (Fl)
`
`(74) Representative: Helke, Kimmo Kalervo
`Kespat Oy,
`P.O. Box 601
`40101 Jyvaskyla (Fl)
`
`(54)
`
`User communication and monitoring system for computer networks
`
`The present invention relates to user commu-
`(57)
`nication and monitoring system (UCMS) for computer
`networks, where a user connects her client computer
`through networks to the host computer chosen by her
`and an information connection is formed, which has fol-
`lowing features:
`
`the host computer has information as files and soft-
`ware for uploading a requested file to network ,
`the client computer has software for requesting the
`file from the host computer and downloading and
`generating at least one screen or page for browsing
`from the downloaded file,
`client and host computers has support for so called
`applet software, where a downloaded file can em-
`bedded by an applet program, which is launched by
`certain user actions on the browser, especially ac-
`tions on the browsed page, and which applet pro-
`gram is able to communicate with the host computer
`and to send starting information as well as periodic
`pulses so called heartbeats, and the host is adapted
`to record this information.
`
`Downloads
`page and applet
`
`Client software starts
`LOGIN-info to server
`I
`
`1
`
`J
`
`**" Heartbeat
`■4-
`Heartbeat stops
`
`.
`
`1
`
`*
`Client software
`shutdown
`LOGOUT-info to server
`
`Downloads of the
`new page
`Figure 2. The client software model.
`
`Printed by Jouve, 75001 PARIS (FR)
`
`CM
`<
`CO
`l o
`O)
`CO
`O)
`o
`a .
`LU
`
`IPR2020-00677
`Vudu Ex. 1008, Page 1
`
`

`

`1
`
`EP0 939 516 A2
`
`2
`
`Description
`
`[0001] This invention relates to a user communication
`and monitoring system for computer networks (UCMS)
`like the Internet.
`In a computerized information network like the
`[0002]
`Internet a user connects her client computer through
`networks to the host computer chosen by her and an
`information connection is formed, which has following
`features:
`
`the host computer has information as files and soft-
`ware for uploading a requested file to the network,
`the client computer has software for requesting the
`file from the host computer and downloading and
`generating at least one screen (or page) for brows-
`ing from the downloaded file,
`the client and host computers have support for so
`called applet software (applet software is defined
`as a program that does not need a permanent stor-
`age medium like disk or tape, although it may be so
`stored), where a downloaded file contains an em-
`bedded applet program, which is launched by cer-
`tain user actions on the client software, especially
`actions of downloading or actions on the download-
`ed screen or page, and which applet program is
`able to communicate with the host computer.
`
`In general, a page means a file, which has
`[0003]
`been downloaded from the server. In this document,
`when the term page is used to mean a user location and/
`or source for communication, it means any user location
`or set of locations. Even if not stated explicitly on each
`occasion, the term "page" includes and/or implies e.g.
`"a set of pages", "a server" or "servers".
`[0004] Current World Wide Web (WWW) Technology
`is able to detect the arrival of visitors ("hits") on a WWW
`site or page. This information is often used on the page
`itself (e.g. "Welcome. You are visitor number 489,234 to
`this page"). It is also used as one measure of the "suc-
`cess" of a page.
`[0005] Excluding the following WO-document there
`is, at the time of writing, no way of directly and continu-
`ously monitoring the continuing presence of a visitor on
`a page, or when a visitor leaves a page voluntarily (e.g.
`by changing pages, leaving the program, or disconnect-
`ing from the network) or involuntarily (e.g. because of a
`machine or network crash). This information is needed
`to make real-time communication between users on the
`same page possible, and for subsequent measurement
`of visit durations. An indirectway of achieving this result
`(used to terminate the transmission of information to a
`visitor who has left) is by requiring visitors to install a
`"plug in" before they can access the page. Once the
`"plug-in" is installed on the client machine by the user,
`the "plug-in" provides "leaving" information to the server.
`There is currently no direct way of achieving this result
`without actions to install special software on the client
`
`10
`
`is
`
`35
`
`45
`
`machine by the user.
`[0006] PCT-publication WO 98/43380 introduces a di-
`rect method to measure visit duration by applet calcula-
`tions on the client machine. This invention does not sup-
`5 port ongoing information as to a user(s)' presence, does
`not support information on a user leaving involuntarily
`or by termination of the connection, and does not sup-
`port or make possible any real-time communication be-
`tween the users.
`[0007] The object of the invention is to provide a user
`communication and monitoring system for Computer
`Networks (UCMS) and software that performs this func-
`tion. Another object of the invention is to provide a sys-
`tem for generating ongoing real-time user information
`(e.g. who is on the same page (including as already not-
`ed same set of pages or server(s)) at the same time)
`that is not vulnerable to interrupted communication (ma-
`chine crashes, network breakdowns, etc.).
`[0008] The UCMS introduces a direct method to
`20 measure visit duration by applet (or similar) pulses
`("heartbeats") on the client machine. The UCMS does
`support ongoing information as to a user(s)' presence,
`does support information on a user leaving involuntarily
`or by termination of the connection, and does support
`25 and make possible various modes of real-time commu-
`nication between users. As noted, these features are
`supported by a pulse, which is herein called "a heart-
`beat". This provides ongoing information on one or more
`users presence to the host, from which awareness of
`30 other users within the same page or same set of pages
`or server(s) can be generated, thus supporting the
`opening of communication in any medium between any
`users of whom the server (or linked server) is aware.
`[0009] The heartbeat information (and its cessation)
`received by the server can further be used to make an
`accurate calculation (external to and independent of the
`client) of the client leaving time and of visit duration. In
`summary, the heartbeat provides information to the
`server of visit duration, of leaving time, of which users
`40 are really on-line, and this can support awareness by
`each user about which others are in the same 'place',
`and can hence support the opening of communication
`in any medium between these users.
`In the following, the invention is illustrated by
`[0010]
`reference to the accompanying figures, which show the
`UCMS mechanism.
`[0011] Figure 1 shows an UCMS operative model.
`[0012] Figure 2 shows a model of an UCMS client
`software e.g. applet.
`[0013] Figure 3 shows a model of an UCMS server.
`[0014] Figure 4 shows the procedure of establishing
`connections between clients.
`[0015] Figure 5 shows the case where the communi-
`cation occurs through the server.
`[0016] Figure 6 shows the case where clients com-
`municate without passing messages through the server.
`[0017] UCMS is generic software with multiple uses,
`some of which are indicated in this paragraph. Currently
`
`so
`
`55
`
`2
`
`IPR2020-00677
`Vudu Ex. 1008, Page 2
`
`

`

`3
`
`EP0 939 516 A2
`
`4
`
`there is no way to detect whether there are other users
`in the same page or in the same set of pages. Many IRC
`(Internet Relay Chat) programs provide a place for chat-
`ting in WWW by importing IRC-programsfrom the Inter-
`net. However, those systems provide information about
`other users in the same communication specific loca-
`tion, but do not provide any information from other users
`if they are not in a communication specific location, or
`in the same server, so it is impossible to establish any
`communication channel generally with or between us-
`ers in the same or linked pages or set of pages or server
`(s).
`In addition, current WWW monitoring software
`[0018]
`only supports knowledge of "hits". It does not support
`knowledge of how long any visitors stay on a page. It
`does not support calculation by the host of aggregate
`statistical information or profiles on page usage such as
`mean or median length of stay, standard deviation of
`length of stay, shape of length of stay curve, etc. It does
`not support passing information to other users or man-
`agers about who is "on" a page at any given time. UCMS
`generates information and enables knowledge on arriv-
`al and departure time of visitors, and thus of duration of
`visit. This can be used by the host to generate: log files
`of arrival & departure times, and of length of stay for
`each visitor; aggregate statistical profiles of visit dura-
`tions; and information on who is "on" a WWW page at
`any given time that can in turn have many further uses.
`Individual and aggregate visit duration information is po-
`tentially more valuable and accurate in monitoring the
`"success" of WWW sites, and in redesigning layout and
`marketing strategy. Information on who (e.g. which cli-
`ent) is "on" a given page at a given time can be used to:
`provide managers with information about who is current-
`ly on a page, and hence the ability to open communica-
`tion with users if desired; provide users with information
`on who else is on the same page, and hence the ability
`to open communication with them if desired. The former
`use (supporting vertical communication) is an obvious
`commercial application of UCMS.
`It is anticipated that UCMS, when released, will
`[0019]
`be widely adopted, as there is currently no lightweight
`software available that supports mutual awareness by
`users, allows real-time communication between the us-
`ers and/or generates WWW (or other Internet informa-
`tion base) duration of visit information, and that does not
`require installation on the client machine (e.g. plug-ins).
`[0020] The basic mechanism of the UCMS is illustrat-
`ed in Figure 1, and a more precise description follows
`Figure 2. When a user arrives on a web page, the down-
`load process is launched. Together with the page, she
`receives a small Java applet which is able to communi-
`cate with the server side UCMS program. The messag-
`es the client sends are the starting and closing times of
`the applet, and regular pulse messages (heartbeat) to
`indicate that the user is on-line and able to communicate
`(from technical point of view) with other users if neces-
`sary. Whenever the server receives a starting or closing
`
`25
`25
`
`30
`30
`
`10
`
`message, or fails to receive a heartbeat, it updates a
`log-file.
`[0021] Heartbeat rate is kept in the server's memory
`for faster reaction to any demand for communication.
`5 The log-file consists minimally of information about the
`login and logout times of any web page (with identifier
`and page-indicator) to which the applet has been added.
`This file can be used for many different purposes, e.g.
`real time monitoring of users or collecting long-term sta-
`tistics or for security reasons.
`[0022] Figure 2 illustrates the sequence of operations
`when the UCMS applet is started. At first ("Download"-
`stage) the user types an URL-address and starts to
`download the page information to her own client corn-
`'s puter. With the HTML (Hyper-Text Modeling Language)
`code, she receives a client software (called an applet,
`which is e.g. programmed in Java) which is able to com-
`municate with the server side program. Once the down-
`load is completed, the applet starts ("Client software
`20 starts"-stage) and informs the server of its existence.
`With this information, the UCMS server is able to write
`the LOGIN-time and any other available information
`from the client machine to the log-file. Minimally, with
`the time information, the server saves some kind of iden-
`tifier and page-indicator.
`[0023] After the page is completely downloaded and
`the applet is running, there will be a period (long or short)
`of user actions ("User actions"-stage). She can read,
`view, or do whatsoever she is able and willing to do with
`the information on the page. In parallel to this, the applet
`regularly sends pulse message to the server ("Heart-
`beat"-stage). In the case of user disconnection, of net-
`work crashes, or other problems, this could cause the
`cessation of heartbeat signals to the server side. The
`35 server reacts to cessation of heartbeat pulses by as-
`35
`suming that the client is not on-line anymore, so it up-
`dates the log-file and refuses all connection attempts to
`that client. In the case that the user changes the page
`or shuts down the applet (i.e. the stop or destroy meth-
`40 ods of the applet are called) ("Client software shut-
`40
`down"-stage), LOGOUT information is send to the serv-
`er. This process is similar to the process of "Applet
`starf-stage, i.e. the applet transmits a notification mes-
`sage of shutdown to the UCMS-server, which then up-
`45 dates the log-file and writes the logout time with similar
`45
`information on identifier and page-indicator.
`[0024] Before the new web page is downloaded
`("Download of the new page"-stage), the applet is de-
`leted from the client. It is unnecessary for user to delete
`so any files or applications from her computer. If the new
`50
`page that the user enters also includes the applet, it is
`downloaded and played in the same way as described
`above. In general, the applet is installed and operates
`(i.e. sends information to UCMS) automatically, but it is
`55 activated and terminated entirely by the user actions.
`[0025] The operational model of the UCMS server is
`very similar to the UCMS client software model. Figure
`3 illustrates the principles of such a model. At first
`
`3
`
`IPR2020-00677
`Vudu Ex. 1008, Page 3
`
`

`

`5
`
`EP0 939 516 A2
`
`6
`
`s
`
`("Download request"-stage) the download request is re-
`ceived from the client. The server starts the page infor-
`mation transmission ("page and applet transmission"-
`stage). Together with the page (HTML-information), the
`applet is transmitted. When the download request has
`been received, the WWW-server software updates an
`ordinary log-file (this log-file update is a standard pro-
`cedure, so it is not explained). However, this original log-
`file can now be used as a comparison file to calculate
`the real number of visitors, (since all browsers do not 10
`currently support Java, such visitors will be missed by
`the UCMS mechanism).
`[0026] To return to the main flow, when the page in-
`formation and applet have been transmitted, the server
`shifts to the port listening stage ("listen to a port"-stage) 15
`with a previously set port. Once a start-up message from
`the client is received, the server updates both the log-
`file by writing the LOGIN-time to the file and its' own
`memory for possible demands for real-time communi-
`cation. Then the server keeps listening the port (actually 20
`the server listens to port all the time, all other procedures
`are being completed by the WWW-server itself). In par-
`allel to listening to the port, the server checks that it re-
`ceives the heartbeat message regularly from the client.
`After a while, when the user changes the page she is 25
`viewing and sends a logout message, or if the heartbeat
`pulse from the client is absent for a few cycles in a row,
`the server shifts to the "Client software acknowledge-
`ment-stage. In this stage, the server writes LOGOUT-
`time to the log-file and removes the user from its' data- 30
`base, so no attempts to communicate are accepted.
`Then the server continues monitoring other visitors (or
`the original visitor on a new page).
`[0027] Figure 4 illustrates the possibilities for human
`communication. Several clients are connected to the 35
`server (the clients could be either in the same page or
`in different pages or on different servers). If communi-
`cation through the server is wanted (from the manage-
`ment point of view, or to keep the client software sim-
`pler), a client sends a message to the server ("Message 40
`received from a client"-stage, see also figure 5). The
`message indicating required message type (e.g. text-
`message, audio or video stream etc.) includes an ad-
`dress-tag, so the server simply passes it to the appro-
`priate client. On the other hand, if a client wants to com- 45
`municate directly without using the server as a gateway,
`the client requests an address of the recipient from the
`server ("Address request"-stage, see also figure 6),
`which then replies and sends the address. Therefore the
`client can establish a communication channel directly to
`the appropriate client. The latter approach can be used
`when two clients want to communicate, while the former
`one is more appropriate for multiparty communication.
`[0028] To take into account cases when the client
`computer or network crashes, or the user terminates the 55
`connection, and for other reasons, the applet in the cli-
`ent updates its existence at intervals e.g. of 1 second
`(generally in the range 0,001 - 5 min). The lower part of
`
`so
`
`the range will be proper in the future e.g. in the asyn-
`chronous transfer mode (ATM). The interval must be rel-
`atively short in order to provide accurate information on
`which to base communication with or between users.
`For example, in a direct video or audio communication
`the interval range is preferably 0,001 - 1 sec, while in a
`indirect communication it is preferably 1 sec - 5 min.
`[0029] The data directly from the applet, or indirectly
`from the log-file can be used for many different purposes
`as noted earlier. By comparing the UCMS log-file to the
`type of log-file currently generated by web browser, it is
`possible to calculate the number of users incapable of
`running applets (i.e. those on whom UCMS has no in-
`formation). By the use of this comparison, it is possible
`to calibrate the statistical information from UCMS more
`precisely. With time, the proportion of clients that cannot
`receive applets (or be similarly enabled) can be expect-
`ed to decrease greatly.
`[0030] Especially, when Java is used, the file, which
`presents a downloadable page, includes one or more
`text variables for the applet on the page for use in sys-
`tem administration e.g. by a web master. There are the
`following useful variables:
`
`heartbeat intervals for real-time communication and
`robustness of server side information,
`page identification for starting and closing informa-
`tion,
`port number for communication between the applet
`in the client and the host program,
`image file for displaying on the page with the applet
`the applet provider's URL i.e. the address of the UC-
`MS-server.
`
`[0031] Because the Java binary code is universal, all
`client and server dependent variables such as heartbeat
`rate, page identifier and port number should be included
`on a WWW-page, not in the binary distribution. There-
`fore the same Java applet can be used with all pages
`each page containing unique identification variables.
`[0032] An image file and URL above are optional.
`[0033] A Java applet according the invention contains
`at least initialization, starting, heartbeat, stopping and
`destroying routines. The login code locates in the start-
`ing section and the logout code in the stopping section,
`respectively. The original Java software may be re-
`placed by Microsoft Java or other Java version, or even
`by some other software written for similar kind of pur-
`poses as original Java developed by Sun Microsystems.
`[0034] One of the uses of UCMS is as a communica-
`tion channel. For example, in electronic commerce (EC)
`applications, a on-line customer could ask questions in
`real-time of the vendor or producer while she is shop-
`ping in an electronic market place; or vice versa, the
`vendor may offer online help for the customer. Currently
`there are no such possibilities in EC applications. The
`communication possibility can also be used in many oth-
`er situations and applications in which human-human
`
`4
`
`IPR2020-00677
`Vudu Ex. 1008, Page 4
`
`

`

`7
`
`EP0 939 516 A2
`
`8
`
`interaction is needed.
`[0035] One of the uses of the log-file is as a real time
`user monitor, i.e. to find out who is looking at a particular
`page at the given moment. UCMS is designed to gen-
`erate real time information on a web page if necessary.
`Information display formats for UCMS real time user in-
`formation (who is on the same page at the same time)
`are being developed in both 2D and 3D. The display for-
`mats take into account the need for scalability, since the
`number of users on some commercial pages may be
`very high.
`[0036] For example, in the case of 3D presentation,
`UCMS is able to read the log-file and send the informa-
`tion to a client VRML browser, which changes the ap-
`pearance of the world using the instructions it receives
`from the server. There should be an automatic genera-
`tor, which creates the VRML file from the files located
`on a web server. The generator goes through the direc-
`tories the administrator has specified, and reads the UC-
`MS applet definitions from those pages. From this data,
`the generator is able to generate a 3D projection of web
`information. The Java code, which is later needed for
`the communication with the UCMS server, is added from
`another file. This process is completed each time when
`a new page with the UCMS applet is installed to provide
`accurate information about the server. The appearance
`of the world is a 3D model of a set of web pages or server
`(s) in which the online users (as detected by the UCMS)
`are represented individually or in aggregate as appro-
`priate.
`[0037] Log-files can also be used for monitoring users
`real-time in some 'groupware' software applications. E.
`g. with the BSCW system from GMD (German National
`Research Center for Information Technology) the users,
`who share a workspace, are known directly because of
`user identification and password protection. It is possi-
`ble to combine UCMS and GMD-BSCW to create a new
`identification file. On addition to ordinary UCMS log-file,
`this new file includes real user identification, i.e. login
`and -out times and IP number as usual, and, in addition,
`real name user id. Supporting such combined informa-
`tion is which requires merging or linking these two ap-
`plications, i.e. some modification either to the applica-
`tion itself or to the UCMS viewing program is needed.
`This topic is an application, not related to this specifica-
`tion.
`[0038] Communication channels for audio, video, text
`messages, and document transfer for users on the
`same page can be created from the UCMS files in sim-
`ilar ways to those discussed above for VRML and GMD-
`BSCW.
`[0039] Since the log-file, which is created by the UC-
`MS server, is formatted differently to the history record
`output, conversion is needed before output. The original
`log-file consists only of data on login and logout times.
`The time for which the user actually viewed the page (or
`the time the applet was running) can be calculated from
`those numbers. This information can be easily shown
`
`on a 2D interface and/or used to create 3D worlds for
`the users and/or used as a basis for audio, video, text
`messaging, or document transfer (see above). The orig-
`inal log-file should remain unchanged by any statistics
`that are calculated, since it may be needed in the future.
`
`5
`
`Generality
`
`10
`
`In the above text, the specific programming
`[0040]
`language Java, and the specific networking mechanism
`of the World Wide Web, including 'applet', have been
`used to exemplify and instantiate the UCMS server and
`client programs and operations. The UCMS client-serv-
`er mechanisms are intended to be entirely general, and
`to operate in any networking/program environment that
`is similar in the relevant respects.
`[0041] The Java programming language is disclosed
`in numerous books, e.g. Ken Arnold, James Gosling:
`The Java Programming Language; Addision-Wesley,
`20 1996.
`
`is
`
`Claims
`
`25 1.
`25 1. User communication and monitoring system (UC-
`MS) for computer networks, where a user connects
`her client computer through networks to the host
`computer chosen by her and an information con-
`nection is formed, which has following features:
`
`30
`30
`
`35
`35
`
`40
`40
`
`45
`
`so
`
`the host computer has information as files and
`software for uploading a requested file to
`network ,
`the client computer has software for requesting
`the file from the host computer and download-
`ing and generating at least one screen or page
`for browsing from the downloaded file,
`client and host computers has support for so
`called applet software, where a downloaded file
`can embedded by an applet program, which is
`launched by certain user actions on the brows-
`er, especially actions on the browsed page, and
`which applet program is able to communicate
`with the host computer and to send starting in-
`formation representing the starting time of the
`browsing of the page, and the host is adapted
`to record this information,
`
`characterized in that the applet software is adapted
`also to send periodic information (so called heart-
`beats) to the host computer, which is adapted to
`keep record of these heartbeats representing the
`user being active in the page.
`
`55 2. User communication and monitoring system ac-
`cording to claim 1 , characterized in that the host
`computer creates a separate file from the recorded
`information presenting a real-time view of the users
`
`5
`
`IPR2020-00677
`Vudu Ex. 1008, Page 5
`
`

`

`9
`
`EP 0 939 516 A2
`
`10
`
`on a certain page.
`
`3. User communication and monitoring system ac-
`cording to claim 1 or 2, where the applet software
`uses virtual ports for communication, characterized
`in that in each time that the page information and
`the applet have been transmitted to the client, the
`host shifts to the port listening stage with a previ-
`ously set port and starts to check heartsbeats from
`the client in a predetermined rate.
`
`s
`
`10
`
`4. User communication and monitoring system ac-
`cording to one of claims 1 - 3,
`characterized in that the host computer counts the
`browsing time of the page according to the heart- 15
`beats received from the client.
`
`5. User communication and monitoring system ac-
`cording to one of claims 1 - 4,
`characterized in that the applet software is adapted 20
`also to send closing information to the host compu-
`ter representing the finishing time of the browsing
`of the page.
`
`6. User communication and monitoring system ac- 25
`cording to one of claims 1 - 4,
`characterized in that the network is Internet and the
`UCMS code, the applet and/or server side pro-
`grams, are programmed in Java.
`
`7. User communication and monitoring system ac-
`cording to claim 4 or 5,
`characterized in that the file presenting a download-
`able page contains one or more variables for the
`applet in the same file, which variables belong to 35
`the following group:
`
`30
`
`page identification for starting and closing infor-
`mation,
`port number for communication between the 40
`applet in the client and the host program,
`heartbeat rate,
`image for displaying on the page with the ap-
`plet,
`url i.e. the address of the UCMS-server.
`
`45
`
`8. User communication and monitoring system ac-
`cording to one of claims 1 - 7,
`characterized in that the applet in the client updates
`its existence at intervals in the range 0,001 - 5 min.
`
`9. User communication and monitoring system ac-
`cording to claim 8 for video or audio communication,
`characterized in that the updating interval is in the
`range 0,001 - 1 second.
`
`so
`
`55
`
`6
`
`IPR2020-