(12) United States Patent
`Eintracht et al.
`
`USOO6687878B1
`(10) Patent No.:
`US 6,687,878 B1
`(45) Date of Patent:
`Feb. 3, 2004
`
`(54) SYNCHRONIZING/UPDATING LOCAL
`CLIENT NOTES WITH ANNOTATIONS
`PREVIOUSLY MADE BY OTHER CLIENTS
`IN ANOTES DATABASE
`(75) Inventors: Zvika Eintracht, Ramat Gan (IL);
`Alexander Ovsiankin, Lod (IL);
`Roman Dementiev, Givatayim (IL);
`O
`1
`O
`Gil Sideman, Raanana (IL)
`(73) Assignee: RealTime Image Ltd., Or Yehuda (IL)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/270,255
`(22) Filed:
`Mar 15, 1999
`7
`(51) Int. Cl." .................................................. GO6F 7/00
`
`
`
`4/2000 Abe et al. ................... 707/202
`6,052,695 A
`6,081,291. A * 6/2000 Ludwig, Jr. ..
`... 348/16
`6,081,829. A
`6/2000 Sidana ....................... 709/203
`(List continued on next page.)
`OTHER PUBLICATIONS
`ss
`66
`Trigg et al., “Supporting Collaboration in Notecards”, Pro
`ceedings of the 1986 ACM conference on Computer-Sup
`ported cooperative work, Dec. 1986, pp. 153–162.*
`Primary Examiner Stephen S. Hong
`S.A.C.C.C., "Yad Zaretsk
`y, Agent,
`y
`(57)
`ABSTRACT
`A System for collaborative document annotation whereby
`notes (i.e. annotations) associated with a document, Such as
`an image or text document, are Stored in a notes database on
`a central notes Server. The documents and associated anno
`tations are treated independently from each other whereby
`
`(52) U.S. C. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 715/512; 513/515
`
`Separate data StructureS are created for the documents and
`
`(58) Field of Search ................................. 707/501, 512,
`707/513,515; 715/501.1, 512, 513,515
`
`(56)
`
`2- -- 12
`
`CKOWIZ C a
`
`References Cited
`U.S. PATENT DOCUMENTS
`5,165,012 A 11/1992 Crandall et al. ............ 345/347
`5,220,657 A * 6/1993 Bly et al...
`... 711/152
`5. A 3.
`E. C. tal... 29,3.
`5,671,428 A
`9/1997 Muranaga et al. .......... 345/329
`5,806,079 A 9/1998 Rivette et al. .......
`707s
`5,821,931. A 10/1998 Berquist et al. ..
`... 345/346
`5,831,615. A 11/1998 Drews et al. ............... 345/344
`5,832,474. A * 11/1998 Lopresti et al................. 707/2
`5,838.914. A 11/1998 Carleton et al. ............ 709/204
`5,845,301 A * 12/1998 Rivette et al. .............. 707/512
`5,860,074 A
`1/1999 Rowe et al.
`...
`... 707/526
`5,870,547 A 2/1999 Pommier et al. ........... 709/204
`5,870,759 A * 2/1999 Bauer et al. ......
`... 707/201
`5,890,177 A * 3/1999 Moody et al. .............. 71.5/511
`5,966,512 A * 10/1999 Bates et al. ................. 709/205
`6,052,514 A
`4/2000 Gill et al. ................... 345/733
`
`for the associated annotations. A web server application O
`the Server Side functions to capture requests from one or
`more note client applications for creating, Storing, editing
`and retrieving annotations related to Specific documents
`stored on the notes server. On the client side, the notes client
`functions to display the document that the user wishes to
`annotate and provides the tools necessary to permit the user
`to create, edit, delete, retrieve and Store notes. A Synchro
`nization process transmits the annotations generated by the
`No. from
`noteS SE the
`i. In R A.
`notes server transmits back an acknowledgement along wi
`any new notes that other notes clients may have posted SCC
`the last Synchronization was performed thus enabling mul
`tiple notes clients to annotate a document asynchronously
`with respect to each other. When annotations are posted to
`the notes Server by a notes client, the State of the annotation
`database is Synchronized Such that all other notes clients can
`retrieve the current, up to date annotations associated with a
`document
`
`96 Claims, 13 Drawing Sheets
`
`Browse Docu-ENT
`
`CLEN SELECTS DOCENT URL TO BROWSE
`
`DISPLAY
`ERROR
`MESSAGE
`k
`118
`
`SERVER WALIDATES USER CREDENTIALS
`-
`-
`NO-K USER AUTHENFICATED 2
`YES
`SERVER SENS MAGE DATA WITH
`APPROPRIATE MIME TYPE
`-
`
`CLET LOADS NOTES CLIENT FLJ3-IN
`
`- 114
`
`IAGE CISPLAYE) BY NOTES CLEENT
`
`-
`
`13
`Y
`120
`
`-->
`
`122
`- Y
`
`124
`--
`ReRIEWEROES FROM THE SERWER
`-—
`server reaps oTEs Database For speciFED
`DocuMENT USING URLASKEY
`FILTER THENOTES
`
`SERIALIZE He OE RESULTS AND
`SORE IN RESPONSE SUFFER
`
`30
`
`- w
`
`132
`
`SEND RESPONSE BUFFER TO THE CLIENT -
`34
`
`cIENT UPDATES LOCAL OTES DATABASE
`
`-
`
`CENT DISPLAYS NOTES IN WINDO
`
`138
`
`138
`CLIENT CAN cREATE, ECIT ANDAOR DELETE NOTES
`
`END
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 001
`
`

`

`US 6,687,878 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`6,161,149 A 12/2000 Achacoso et al. ............. 710/4
`6,230,172 B1 * 5/2001 Purnaveja et al.
`... 707/512
`6,240,444 B1 * 5/2001 Fin et al. .........
`... 709/205
`6,308,199 B1 * 10/2001 Katsurabayashi ........... 709/204
`6,336,134 B1 * 1/2002 Varma .............
`... 709/105
`6,342,906 B1
`1/2002 Kumar et al. ............... 345/751
`
`6,353,851 B1 * 3/2002 Anupam et al. ............ 709/202
`6,360,250 B1 * 3/2002 Anupam et al. ............ 709/204
`6,493.731 B1 * 12/2002 Jones et al. .............. 715/501.1
`6,507.845 B1 * 1/2003 Cohen et al. ..
`... 707/100
`6,571,295 B1 * 5/2003 Sidana ..........
`709/246
`6,601,087 B1 * 7/2003 Zhu et al. ................... 709/205
`* cited by examiner
`
`
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 002
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 1 of 13
`
`US 6,687,878 B1
`
`
`
`FIG.1A
`
`FIG1B
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 003
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 2 of 13
`
`US 6,687,878 B1
`
`
`
`S
`
`i
`
`CA)
`H
`Z
`l
`H
`Z
`
`C
`
`2
`H
`
`L
`DS
`-CC
`2
`1.
`(/)
`D
`
`C
`H
`<C
`C
`
`e-cos ... 2
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 004
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 3 of 13
`
`US 6,687,878 B1
`
`SELON
`
`èJEME|/\
`
`
`
`
`
`
`
`
`
`
`
`0/
`
`Z9
`
`SELON
`
`09
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 005
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 4 of 13
`
`US 6,687,878 B1
`
`86
`
`
`
`
`
`99
`
`
`
`
`
`DISPLAY
`ERROR
`MESSAGE
`
`
`
`INTIALIZATION
`
`USER LAUNCHES WEB BROWSER
`
`URL PROVIDED BY CLIENT
`
`CLIENT TRANSMTS URL
`TO THE SERVER
`
`8O
`
`82
`
`84
`
`CLENT PREVIOUSLY
`AUTHENTICATED 2
`NO
`
`YES
`
`REQUEST AUTHENTICATION
`
`BROWSER SENDS USER SUPPLIED
`INFORMATION TO THE SERVER
`
`
`
`WEB SERVER CHECKS CREDENTIALS
`AGAINST THOSE DEFINED FOR
`THE REQUESTED URL
`
`USER NAME AND
`PASSWORD CORRECT 2
`YES
`
`SERVER RESPONDS WITH DRECTORY
`LISTING OF URL OR THE DOCUMENT
`CONTENT
`
`FIG.4
`
`CLIENT DISPLAYS THE
`DIRECTORY LISTING
`
`END
`
`96
`
`98
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 006
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 5 of 13
`
`US 6,687,878 B1
`
`
`
`
`
`
`
`EWWN (JEST LE5)
`
`SINBINOO
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 007
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 6 of 13
`
`US 6,687,878 B1
`
`BROWSE DOCUMENT
`
`CLIENT SELECTS DOCUMENT URL TO BROWSE
`
`11 O
`
`
`
`DISPLAY
`ERROR
`MESSAGE
`
`
`
`116
`
`SERVER VALIDATES USER CREDENTIALS
`
`114
`
`USER AUTHENTICATED 2
`YES
`SERVER SENDS IMAGE DATA WITH
`APPROPRIATE MIME TYPE
`
`CLENT LOADS NOTES CLIENT PLUG-IN
`
`IMAGE DISPLAYED BY NOTES CLIENT .
`
`RETRIEVE NOTES FROM THE SERVER
`
`SERVER READS NOTES DATABASE FOR SPECIFIED
`DOCUMENT USING URL AS KEY
`
`FILTER THE NOTES
`
`SERIALIZE THE NOTE RESULTS AND
`STORE IN RESPONSE BUFFER
`
`SEND RESPONSE BUFFER TO THE CLIENT
`
`CLIENT UPDATES LOCAL NOTES DATABASE
`
`CLIENT DISPLAYS NOTES IN WINDOW
`
`CLIENT CAN CREATE, EDIT AND/OR DELETE NOTES
`
`FIG.6
`
`END
`
`112
`
`118
`
`12O
`
`122
`
`124
`
`126
`
`128
`
`130
`
`132
`
`154
`
`136
`
`138
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 008
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 7 of 13
`
`US 6,687,878 B1
`
`
`
`
`
`
`
`
`
`SELON ONESTWOOT BIWOc}[^
`
`
`
`‘BSWEW IWO BION
`
`
`
`‘SELON AVTdSIG
`
`|NEW[\000 || 30
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 009
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 8 of 13
`
`US 6,687,878 B1
`
`NOTE
`SYNCHRONIZATION
`
`USER INITIATES SYNCHRONIZATION PROCEDURE
`
`CLIENT PREPARES BUFFER TO SEND
`TO NOTES SERVER
`
`CLIENT SERALIZES NOTES DATA
`
`CLIENT POSTS NOTES BUFFER TO THE
`SERVER USING ORIGINAL URL
`
`SERVER VALIDATES USER D AND PASSWORD
`
`SERVER MERGES NOTE, EVENT INFORMATION
`WITH THE NOTES DATABASE
`
`150
`
`152
`
`154
`
`156
`
`158
`
`16O
`
`NEW NOTES ASSIGNED UNIQUE NOTE ID AND PLACED
`IN A NEW RECORD IN THE NOTES DATABASE
`
`162
`
`THE TRANSACTION IS RECORDED IN THE
`NOTES LOG TABLE
`
`NOTE SERVER PREPARES A SERVER ANNOTATION
`RESPONSE DATA STRUCTURE WITH A LIST OF
`NOTES AND THEIR CONTENT
`
`SERVER ANNOTATION RESPONSE IS SENT TO
`THE SYNCHRONIZING CLIENT
`
`CLIENT MERGES RECEIVED LIST WITH ITS
`LOCAL NOTES DATABASE
`
`CLIENT REFRESHES THE DISPLAY
`
`164
`
`166
`
`168
`
`17O
`
`172
`
`END
`
`FIG.8
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 010
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 9 of 13
`
`US 6,687,878 B1
`
`
`
`YAAYSS{N30
`
`
`
`NOILLVWZINOYHONASSLON
`
`
`
`
`
`JSVavVivVd4LONSLlVddnN
`
`
`
`Y3SNONIMOSHOYdLIV
`
`SIWILNSd4Y9
`
`YAAuSSONS
`
`NOWLVLONNY
`
`SSNOd$4u
`
`
`
`SALONJuVdsud
`
`ONV&444Ng
`
`1SOd
`
`
`
`WOOTJlvadn
`
`
`
`“ISVEVIVSSLON
`
`
`
`‘SALONAV1dSIO
`
`
`
`LAdNISLON149
`
`Y3SNWOds
`
`6Old
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 011
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 011
`
`
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 10 of 13
`
`US 6,687,878 B1
`
`CLIENT ANNOTATION EVENT DATA STRUCTURE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`190
`
`192
`
`194
`
`196
`
`198
`
`2OO
`
`2O2
`
`DOCUMENT D
`
`
`
`NOTE #N
`
`FIG.10
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 012
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 11 of 13
`
`US 6,687,878 B1
`
`CLIENT ANNOTATION EVENT NOTE
`DATA STRUCTURE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SERIAL NUMBER
`
`ACTION
`
`ANCHOR
`
`STATUS
`
`COLOR
`
`TEXT LENGTH
`
`TEXT
`
`SHAPE
`
`SHAPE LINE CAP
`
`SHAPE LINE WIDTH
`
`MARKER TYPE
`
`NUMBER OF SHAPE VERTICES
`
`SHAPE VERTICES
`
`
`
`212
`
`214
`
`21 6
`
`218
`
`220
`
`222
`
`224
`
`226
`
`228
`
`230
`
`232
`
`234
`
`236
`
`210
`
`FIG 11
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 013
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 12 of 13
`
`US 6,687,878 B1
`
`SERVER ANNOTATION RESPONSE DATA STRUCTURE
`
`
`
`DOCUMENT ID
`
`LOCAL SERVER TIME
`
`DOCUMENT DIMENSIONS
`
`NUMBER OF NOTES
`
`NOTE #1
`
`242
`
`244
`
`246
`
`248
`
`250
`
`252
`
`240
`
`FIG.12
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 014
`
`

`

`U.S. Patent
`
`Feb. 3, 2004
`
`Sheet 13 of 13
`
`US 6,687,878 B1
`
`SERVER ANNOTATION RESPONSE
`NOTE DATA STRUCTURE
`
`SERIAL NUMBER
`
`NOTE TIME STAMP
`
`262
`
`264
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`260
`
`FIG.13
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 015
`
`

`

`1
`SYNCHRONIZING/UPDATING LOCAL
`CLIENT NOTES WITH ANNOTATIONS
`PREVIOUSLY MADE BY OTHER CLIENTS
`IN ANOTES DATABASE
`
`FIELD OF THE INVENTION
`The present invention relates generally to collaborative
`work Systems and more particularly relates to a System for
`collaborative document annotation based on the exchange of
`notes attached to web based documents, the System being
`Suitable for use over a network Such as the Internet based
`World Wide Web (WWW).
`BACKGROUND OF THE INVENTION
`Abasic function of WorldWide Web browsers in common
`use today, Such as Netscape Navigator and MicroSoft Inter
`net Explorer, is to allow web pages stored in remote Web
`server to be retrieved and viewed. The user initiates a
`Viewing Session on a web browser by Specifying a particular
`URL corresponding to a page to be viewed. The Web Server
`transmit the various elements of the requested page to the
`browsing application. Any special data formats, e.g., video,
`certain images, documents not written in HTML language,
`etc., are treated by one or more web browser Software
`plug-ins on the client computer.
`A common need is to be able to view the document, i.e.,
`images, etc., on the client computer and be able to add one
`or more annotations to the document. It is becoming more
`and more common for groups of two or more people to work
`collaboratively on a project. The members of the group may
`or may not be located in the same geographic area. The
`collaboration is often via a network Such as the global
`Internet. The members of the group may be located any
`where in the World as long they have a connection to the
`Internet.
`The ability to annotate documents displayed on Screen is
`very useful when information needs to be exchanged among
`more than one perSon. It would be beneficial if one perSons's
`comments could be shared among many others by attaching
`notes that travel with the underlying document being worked
`O.
`Currently, however, it is not possible to annotate docu
`ments on the client Side Such that other clients can See the
`notes. The entire document with the attached notes must be
`transmitted to other clients for them to see the notes.
`Transmitting the entire document from client to client
`through the Server in order to exchange annotation is
`burdensome, inefficient and costly in terms of time and
`bandwidth.
`The majority of prior art Systems that with image and page
`annotation are adapted to handle annotation only on the
`client Side. In these Systems, a user launches an editing tool
`to open a document locally on her/his computer. The docu
`ment is edited locally with the other clients not being able to
`easily view the changes or notes. Some of these editing tools
`permit attaching data information Such as annotations to the
`document. These Systems operate by Saving the annotations
`within the local file system that is also used to store the
`document.
`Also known in the prior art are Systems that permit
`collaboration that is based on the replication of the display
`at one client System to the display on another client System.
`These Systems, however, require that all parties to the
`collaboration Session be logged on to the Server at the same
`time in order to view the annotations made by one or more
`of the parties.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,687,878 B1
`
`2
`Thus, there is strong felt need for a System that would
`allow multipartly collaboration based on the asynchronous
`eXchange of annotations over a network Such as the Internet
`without the requirement that all parties wishing to collabo
`rate be simultaneously logged on to a Server.
`SUMMARY OF THE INVENTION
`The present invention is a System for collaborative docu
`ment annotation whereby notes or annotations associated
`with a document, Such as an image or text document, are
`Stored on a Web Server. The documents and associated
`annotations are treated independently from each other. Sepa
`rate data Structures are created for the documents and for the
`asSociated annotations thus permitting their independent
`management.
`The invention can be implemented as Software, a portion
`of which executes on the Server Side and a portion that
`executes on the client Side. The Server Side may comprise a
`plurality of Software applications running in parallel that in
`combination provide the server functionality of the inven
`tion. A web server application on the Server Side functions to
`capture Special requests from one or more client applications
`for creating, Storing, editing and retrieving, annotations
`related to Specific documents located in the Server. A notes
`Server functions to log all annotation activities along with
`information about the corresponding clients that create, edit
`and retrieve them.
`The notes information are transmitted between client and
`server applications via TCP/IP protocols over communica
`tions means Such as a LAN, WAN or the Internet. The
`invention may be adapted to operate over an Intranet, e.g.,
`LAN, or Extranet, e.g., WAN or the Internet.
`On the client Side a client Software application functions
`to display the document that the user wishes to annotate and
`provides the tools necessary to permit the user to create, edit,
`retrieve and Store notes. Note that the client Software appli
`cation can be implemented as a web browser plug-in mod
`ule. The plug-in contains the user interface for navigating
`within the document and for handling the notes.
`A Synchronization button is provided which, when
`pressed by the user, transmits the annotations generated by
`the user from the client to the Server using a particular
`protocol. In response, the Server transmits back an acknowl
`edgement along with any new notes that other clients may
`have posted Since the last Synchronization was performed.
`A user may annotate many documents at the same time by
`opening Several web browser windows. In addition, other
`clients can annotate either the same document or other
`documents at the same or a later time. In accordance with the
`invention, the annotations are transmitted from the Server
`independent of the data transmitted that is related to the
`Viewed document. At the client Side, the client application
`layers the annotations over the image (or document) in
`accordance with the coordinates of each.
`AS a result, the present invention has the advantage of
`permitting multiple clients to annotate a document that
`resides on a central web server in an asynchronous fashion.
`When annotations are posted to the server by a client, the
`State of the annotation database is Synchronized Such that all
`other clients can retrieve the current, up to date annotations
`asSociated with a document. AS an option, the Server can be
`adapted to trigger one or more alarms in response to a
`change being made to the database maintained by the Server.
`Those clients that wish to be notified in the event of a
`database change at the Server, can be notified by means of
`Visual indications, audio and/or e-mail. In addition, the
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 016
`
`

`

`3
`alarms can filtered in numerous ways. Examples of filtering
`include, but are not limited to, Specific documents, docu
`ment types, document folders, annotation contents and Spe
`cific clients.
`The system of the present invention further allows anno
`tations to be kept private. Annotations can be kept Secure
`from changes made by non authorized clients. The time an
`annotation was created is also tracked and Stored in terms of
`the clock in the server. When the time stamp of the anno
`tation is shown on the client, it is shown in terms of the clock
`on the client.
`There is provided in accordance with the present inven
`tion a System for annotating documents comprising a docu
`ment file located on a Server, the document file for Storing
`one or more documents, a notes database located on the
`Server, the notes database for Storing one or more notes, each
`note associated with a particular document, one or more
`notes clients coupled to a network, each notes client opera
`tive to locally display a representation of a document
`remotely Stored on the Server in the document file, the notes
`client adapted to permit a user to annotate the document with
`one or more notes, the notes client operative to Simulta
`neously display the one or more notes associated with the
`document over the displayed document Such that the docu
`ment is viewable along with the one or more notes, a notes
`Server coupled to the network, the notes Server operative to
`Store the documents in the document file Separately from
`notes Stored in the notes database, the notes Server receiving
`one or more notes associated with a particular document
`from the one or more notes clients and Synchronization
`means within the notes clients and the notes Server, the
`Synchronization means for updating the notes Server with
`any notes events processed by the notes clients and for
`updating the notes client with the results of Synchronization
`updates previously performed by other notes clients since
`the last Synchronization event.
`The notes database comprises a Notes Table for storing
`the current content of a note and a Notes Log Table for
`Storing a transaction history of all not events for notes
`asSociated with a particular document. The notes client
`comprises a display, network communication means, a pro
`ceSSor Suitably programmed to run a web browser and a
`client note plug-in adapted to run within the web browser.
`The network comprises the Internet, a TCP/IP network or
`an Intranet. The notes Server comprises network communi
`cation means, a processor Suitably programmed to provide a
`Web Server Services, provide notes agent Services, provide
`document and notes browsing and Synchronization Services
`to the one or more notes clients. The notes agent Services
`comprises an Internet Server Application Programming
`Interface (ISAPI).
`The System further comprises a notes administrator for
`providing one or more Universal Resource Locations (URL)
`aliases in managing and adminstrating the notes Server and
`a note viewer for generating reports on the history of
`annotation on a per user or a per document basis that have
`occurred over a period of time.
`The system permits two or more user to collaboratively
`work on the same document whereby each user is capable of
`Viewing annotations made by the other users upon Synchro
`nization with the notes Server. A Single page of a multipage
`document can be annotated on a notes client without requir
`ing the entire multipage document to be transmitted to the
`notes client.
`The System further comprises Security means for provid
`ing Universal Resource Locator (URL) translation and redi
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,687,878 B1
`
`4
`rection Services So as to provide user level Security for
`documents and notes Stored on the Server. The Security
`means comprises a Document Location Table and a Client
`Access Key Table.
`The document comprises an image or a document con
`Sisting of a page description language. The notes client is
`adapted to permit the user to annotate the document in
`parallel with the display of the document. The document
`comprises a plurality of pages whereby each page can be
`annotated independently of all the other pages. The Synchro
`nization means is user initiated or is triggered by the notes
`via the Signaling of Selected notes clients.
`There is also provided in accordance with the present
`invention method of annotating documents, the method
`comprising the Steps of getting a first request comprising a
`Universal Resource Locator (URL) associated with a docu
`ment to be viewed and annotated and transmitting the first
`request from a client to a Server, Sending a first response
`from the Server in response to the first request, the first
`response comprising the document type of and a represen
`tative of the requested document, getting a Second request
`comprising a document note URL corresponding to the notes
`asSociated with the document and Sending the Second
`request from the client to the Server, Sending a Second
`response from the Server in response to the Second request,
`the Second response comprising one or more notes associ
`ated with the document, displaying the representation of the
`document and the associated one or more notes locally on
`the client Such that both are viewable simultaneously, per
`mitting the user to perform annotation functions on the client
`So as to generate a third request comprising one or more note
`events related to the document that are transmitted from the
`client to the server and sending a third response from the
`Server to the client in response to the third request, the third
`response comprising an updated list of notes associated with
`the document, the list of notes incorporating any note events
`previously received from other clients.
`The Step of getting a first request comprises the Step of
`querying the user for a user name and password. The Step of
`Sending a first response comprises the Step of authenticating
`the user on the Server before Sending the first response. The
`method further comprises the Step of getting a fourth request
`comprising a folder URL associated with the folder com
`prising one or more documents to be viewed and annotated
`and transmitting the fourth request from the client to the
`SCWC.
`The method further comprises the Step of Sending a fourth
`response from the Server in response to receiving a request
`comprising a folder URL, the fourth response comprising a
`directory listing of the folder. The document comprises an
`image or a document represented using a page description
`language.
`BRIEF DESCRIPTION OF THE DRAWINGS
`The invention is herein described, by way of example
`only, with reference to the accompanying drawings,
`wherein:
`FIGS. 1A, 1B, 1C are illustrations of an example image
`displayed with and without multiple annotation displayed
`thereon;
`FIG. 2 is an illustration of an example annotation;
`FIG. 3 is a block diagram illustrating the collaborative
`document annotation System of the present invention;
`FIG. 4 is a flow diagram illustrating the initialization
`portion of the collaborative document annotation System of
`the present invention;
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 017
`
`

`

`US 6,687,878 B1
`
`6
`
`-continued
`
`S
`FIG. 5 is a message flow diagram illustrating the trans
`action Sequence between client and Server during initializa
`tion;
`FIG. 6 is a flow diagram illustrating the document brows
`ing portion of the collaborative document annotation System
`of the present invention;
`FIG. 7 is a message flow diagram illustrating the trans
`action Sequence between client and Server during document
`browsing,
`FIG. 8 is a flow diagram illustrating the note synchroni
`Zation portion of the collaborative document annotation
`System of the present invention;
`FIG. 9 is a message flow diagram illustrating the trans
`action Sequence between client and Server during note
`Synchronization;
`FIG. 10 is a diagram illustrating the Client Annotation
`Event Data Structure in more detail;
`FIG. 11 is a diagram illustrating the note data Structure of
`the Client Annotation Event Data Structure of FIG. 10 in
`more detail;
`FIG. 12 is a diagram illustrating the Server Annotation
`Response Data Structure in more detail; and
`FIG. 13 is a diagram illustrating the note data structure of
`the Server Annotation Response Data Structure of FIG. 12
`in more detail.
`
`Term
`
`Definition
`
`Note Alarms
`
`Note Anchor
`
`Note Client
`
`Notifications that the Notes Server received a Note
`Event.
`The specific location in a document associated with a
`note.
`The application that displays the annotated document
`and allows for annotation sessions.
`Note Contents The text or graphical information contained in the note.
`Note
`Identifier for documents and sets of aggregated
`Document ID documents organized under the same logical structure.
`Note Event
`A message related to a note that did not exist at the time
`of the last synchronization with the Notes Server, or a
`note that was modified or deleted since that time.
`Identifier used to differentiate between notes generated
`by various users.
`Mechanism whereby the Notes Clients saves the Note
`Events to the local file system, allowing for
`synchronization at a later time.
`A number uniquely identifying a note.
`
`1O
`
`15
`
`Note
`Owner ID
`Note
`Persistence
`
`Note Serial
`Number
`Note Server
`
`25
`
`Note
`Synchroniza-
`tion
`Note Time
`Stamp
`Notes
`Database
`Notes Log
`
`The application that provides the centralized
`role of document and note management.
`The update activity whereby one or more Note Events
`are exchanged between the Notes Server and
`the Notes Client.
`The time stamp saved on the Notes Server, stored
`using time zone of the Notes Server.
`The database maintained by the Notes Server holding
`all note related information.
`A transactional history of Notes Events.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`Notation Used Throughout
`The following notation is used throughout this document.
`
`Term
`
`Definition
`
`Greenwich Mean Time
`GMT
`Hypertext TextMarkup Language
`HTML
`Hypertext Transport Protocol
`HTTP
`Internet Information Server
`IIS
`Internet Protocol
`IP
`Internet Server Application Programming Interface
`ISAPI
`Local Area Network
`LAN
`Multipurpose Internet Mail Extension
`MIME
`Network File System
`NFS
`Portable Document Format
`PDF
`Random Access Memory
`RAM
`Red Green Blue
`RGB
`Simple Mail Transport Protocol
`SMTP
`Structured Query Language
`SOL
`Transport Control Protocol
`TCP
`Tag Image File Format
`TIFF
`Universal Resource Locator
`URL
`Wide Area Network
`WAN
`WWW World Wide Web
`
`The following terminology and definitions apply through
`out this document.
`
`Term
`
`Definition
`
`Annotation
`(Note)
`Annotation
`Session
`
`A portion of text or a graphical drawing that is associated
`with a specific location in a document.
`The time period during which a user creates, edits,
`retrieves and stores one or more notes associated with
`a document.
`
`General Description
`The present invention is a System for collaborative docu
`ment annotation whereby notes or annotations associated
`with a document are Stored on a web server. Examples of
`documents include, but are not limited to, images, text
`documents or documents expressed in a page description
`language such as PostScript or Adobe PDF. In addition, each
`document may contain more than one page, wherein each
`page is annotated independently of the other. The documents
`and associated annotations are treated independently from
`each other. Separate data Structures are created for the
`documents and for the associated annotations thus permit
`ting their independent management. The invention can be
`implemented as Software, a portion of which executes on the
`Server Side and a portion executes on the client Side. The
`Server Side may comprise a plurality of Software applications
`running in parallel that in combination provide the Server
`functionality of the invention. A web server application on
`the Server Side functions to capture Special request from one
`or more client applications for creating, Storing, editing and
`retrieving annotations related to Specific documents located
`in the Server. A notes Server functions to log all annotation
`activities along with information about the corresponding
`clients that create, edit and retrieve them.
`Illustrations of an example image displayed with and
`without multiple annotations displayed thereon are shown in
`FIGS. 1A, 1B and 1C. The system of the present invention
`is Suitable for use with a variety of document types, includ
`ing text and image based documents. For illustrative
`purposes, the following description uses an image type
`document to illustrate the principles of the invention. This is
`not meant to limit the scope of the invention. One skilled in
`the computer and Software arts can apply the System the
`invention to other document types as well without departing
`from the Spirit and Scope of the present invention.
`In FIG. 1A, the display 10 is shown comprising a single
`document window 12 in which an image is displayed. No
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Meta Platforms, Inc.
`Exhibit 1006
`Page 018
`
`

`

`US 6,687,878 B1
`
`15
`
`40
`
`45
`
`7
`annotations are displayed in this view. In FIG. 1B, the
`display 10 is shown comprising a Single document window
`14 in which an image is displayed with a plurality of
`annotations 16. The annotations 16 are displayed over the
`image but are not a part of the image itself. In FIG. 1C, the
`display 10 is shown comprising two portions: the image
`window 18 and a Note List window 20. The image window
`18 displays the image with the plurality of annotations 16 on
`top of the image. The Note List window 20 displays a list of
`all the annotations associated with the image shown in
`window 18. The user can optionally sort the Note List by
`Note Owner ID, Note Time Stamp, Note color label and
`Note Contents. The display may include Note Event
`description, Note Owner ID, user name, path, originator IP
`address, time and Note Contents. When a note is selected on
`the Note List, its counterpart icon in the document window
`frame is highlighted.
`An illustration of an example annotation is shown in FIG.
`2. The annotation, generally referenced 30, comprises a
`window having a title portion 32 and text portion 34. The
`window contains a note label (Sam), Note Owner ID or login
`name (Acme) and the data and time of the most recent
`Synchronization.
`A block diagram illustrating the collaborative document
`25
`annotation System of the present invention is shown in FIG.
`3. The annotation System, generally referenced 40, com
`prises one or more standard web browsers 42 labeled web
`browser #1 through web browser #N. Each web browser
`comprises a Note Plug-In component 44 that loads into the
`browser. The web browser 42 may comprise any suitable
`browser such as Netscape Navigator or Microsoft Internet
`Explorer. Each web browser in combination with the Note
`Plug-In is referred to as a Notes Client 41 and is the tool used
`for browsing the documents located on