United States Patent
`US 6,725,427 B2
`(10) Patent No.:
`(12)
`Freemanetal.
`(45) Date of Patent:
`Apr.20, 2004
`
`
`US006725427B2
`
`(54) DOCUMENT STREAM OPERATING SYSTEM
`WITH DOCUMENT ORGANIZING AND
`DISPLAY FACILITIES
`
`(75)
`
`ee
`
`.
`
`?
`
`Inventors: oeeaBani.Ceocr
`(US)
`.
`.
`.
`(73) Assignee: Mirror Worlds Technologies, Inc.,
`New Haven, CT (US)
`
`.
`
`?
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`US.C. 154(b) by 76 days.
`(21) Appl. No.: 10/013,150
`.
`(22)
`Filed:
`Dec. 10, 2001
`.
`oa.
`Prior Publication Data
`(65)
`US 2002/0046220 A1 Apr. 18, 2002
`
`Related U.S. Application Data
`
`5,287,448 A
`5,297,032 A
`5,402,526 A
`5,430,710 A
`5,448,729 A
`
`2/1994 Nicoletal.
`3/1994 Trojan et al.
`3/1995 Baumanetal.
`7/1995 Mueller et al.
`9/1995. Murdock
`
`6/1996 Tobias, II etal.
`5,530,859 A
`7/1996 Lamming
`5,535,063 A
`12/1996. Kneeetal.
`5,589,892 A
`3/1997 Lucus etal. we. 715/526
`5,613,134 A *
`4/1997 Cluts
`5,616,876 A
`4/1997 Zarmeretal.
`5,625,818 A
`7/1997 Reitz
`5,649,182 A
`12/1997 DeBey
`5,701,582 A
`11/1998 Kumar
`5,835,129 A
`3/1999 Moodyetal.
`5,890,177 A
`6/1999 Sciammarellaet al.
`5,912,668 A *
`6/2001 Manderetal.
`6,243,724 B1
`OTHER PUBLICATIONS
`
`..... 345/788
`
`Gelernter, David, The Cyber—-Road Not Taken, The Wash-
`ington Post, Apr. 3, 1994.
`Nelson, Theodor H., The Right Way To ThinkAbout Software
`Design, In The Art of Human—Computer Interface Design,
`Brenda Laurel, (Ed.), (1990): pp. 235-243.
`Landsdale, M., The Psychology Of Personal Information
`Management, Applied Ergonomics,(Mar. 1988): pp. 55-66.
`
`(62) Division of application No. 09/398,611, filed on Sep. 17,
`1999, now Pat. No. 6,638,318, which is a continuation of
`(List continued on next page.)
`application No. 08/673,255,filed on Jun. 28, 1996, now Pat.
`}
`;
`No. 6,006,227.
`Primary Examiner—Stephen S. Hong
`7
`(74) Attorney, Agent, or Firm—Cooper & Dunham LLP
`Tint. C0ec eeecneseneneceneneeenees GO06F 17/21
`(51)
`(52) US. C1. cece cescnseeeecnesenseecenseesenecassenees 715/526
`(58) Field of Search .ccccsssssssssssssseeeee 715/500, 511, ©
`ABSTRACT
`715/512, 526; 707/7, 2, 102
`A document stream operating system and method is dis-
`.
`closed in which: (1) documents are stored in one or more
`References Cited
`chronologically ordered streams; (2) the location and nature
`U.S. PATENT DOCUMENTS
`of file storage is transparent to the user; (3) information is
`organized as needed instead of at the time the documentis
`created; (4) sophisticated logic is provided for summarizing
`a large group of related documents at the time a user wants
`a concise overview; and (5) archiving is automatic. The
`documents can include text, pictures, animations, software
`programsor any other type of data.
`
`(56)
`
`5,060,135 A
`5,063,495 A
`5,140,676 A
`5,150,410 A
`5,159,669 A
`5,241,671 A
`5,247,437 A
`5,283,864 A
`
`10/1991 Levineet al.
`11/1991 MacPhail
`8/1992 Langelaan
`9/1992 Bertrand
`10/1992 Triggetal.
`8/1993. Reedet al.
`9/1993. Valeetal.
`2/1994. Knowlton
`
`39 Claims, 6 Drawing Sheets
`
`fSnEazPeasae efaeeeeeeeeeoseepeeepeeegae
`
`
`
`reams Substreams Summarize Personal Agents{‘| Thursday,[Dec 14/10:27 AM/1995 EST ||
`
`Stream, Eric:Freeman
`Wain stream.
`4701. Documents:
`Won-.12/04/96 to Wed: 12/06/95:
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 1
`
`iD}
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 1
`
`

`

`Malone, Thomas W., How Do People Organize Their
`Desks? Implications For The Design Of Office Information
`Systems., ACM Transactions On Office Systems, vol. 1, No.
`1 Jan. 1983): pp. 99-112.
`Gelernter, The Cyber-Road Not Taken, The Washington
`Post, Apr. 3, 1994, 9 pgs.
`Freeman, Lifestreams Project
`1994-1996.
`Freeman, Lifestreams for the Newton, Steve Mann’s Devel-
`oper Corner, Oct. 1995, 5 pgs.
`
`Home Page,
`
`3
`
` pgs.,
`
`US 6,725,427 B2
`Page 2
`
`OTHER PUBLICATIONS
`
`Steinberg, Lifestreams, WIRED 5.02, Feb. 1997, 11 pgs.
`
`Freemanet al, Lifestreams: Organizing your Electronic Life,
`AAAI Fall Symposium, Al Applications in Knowledge
`Navigation and Retrieval, Nov. 1995, Cambridge MA,6 pgs.
`
`Getting Results with Microsoft Outlook,
`Microsoft Corporation 1995-1996, pgs. 28-29.
`
`copyright
`
`Cowart, Mastering Windows 3.1, 992, chapter 12, pp.
`396-417.
`
`* cited by examiner
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 2
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 2
`
`

`

`U.S. Patent
`
`Apr. 20, 2004
`
`Sheet 1 of 6
`
`US 6,725,427 B2
`
`0¢
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 3
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 3
`
`

`

`U.S. Patent
`
`Apr. 20, 2004
`
`US 6,725,427 B2
`
`Sheet 2 of 6
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 4
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 4
`
`

`

`U.S. Patent
`
`Apr. 20, 2004
`
`Sheet 3 of 6
`
`US 6,725,427 B2
`
`Wed
`
`@ December 1995 >
`
`Tue
`
`FIG. 5
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 5
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 5
`
`

`

`U.S. Patent
`
`Apr. 20, 2004
`
`Sheet 4 of 6
`
`US 6,725,427 B2
`
`
`
`FIG. 6A
`
`Scott Fertig
`Ward Mullins
`Beth Freeman
`
`Tue Aug
`Tue Aug
`Tue Aug
`
`1
`1
`1
`
`12:05
`14:57
`10:22
`
`EDT 1995
`EDT 1995
`EDT 1995
`
`to PPC
`Port
`432-6433
`415 224-1912 Tel/Java discussion
`432-1287
`insurance
`
`FIG. 6B
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 6
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 6
`
`

`

`U.S. Patent
`
`Apr. 20, 2004
`
`Sheet 5 of 6
`
`US 6,725,427 B2
`
`
`
`Quote—O—Matic Stock Service for 5/16/95
`
`GVIL
`LMASX
`ODWA
`SPLS
`TSA
`lavtx
`
`14.00
`20.84
`18.50
`27.12
`19.25
`21.41
`
`FIG. 8A
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 7
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 7
`
`

`

`U.S. Patent
`
`Apr. 20, 2004
`
`Sheet 6 of 6
`
`US 6,725,427 B2
`
`a8“Sls
`
`shog
`
`t
`
`00°0S-
`
`t
`
`00°001-
`
`00°0S1-
`
`00°00e-
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 8
`
`' G6/Z1/g8
`
`40}AiDUUNSO}/OJHOd
`
`
`26reovevenpesosacestunetassrousosacaseasoO
`
`
`
`aggneeeenecernnmanenenannens[noqy}[AdoopsoH}|f/ qenenesenseseceeteOt8tOmereyes,esor9|
`
`pteecennnasorrr®tent
`
`edlqBulso|9
`
`-00°8f
`
`-00°9¢
`
`-00°7E
`
`-00°2¢
`
`-00°0£
`
`-00°82
`
`-00°92
`
`-00°¥2
`
`-00°22
`
`-00°02
`
`-00°81
`
`-00°91
`
`-00'F1
`
`-00°C1
`
`-00°01
`
`-00°8
`
`-00°9
`
`-00°7
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 8
`
`

`

`US 6,725,427 B2
`
`1
`DOCUMENT STREAM OPERATING SYSTEM
`WITH DOCUMENT ORGANIZING AND
`DISPLAY FACILITIES
`
`This application is a divisional of application Ser. No.
`09/398,611, filed Sep. 17, 1999, now US. Pat. No. 6,638,
`318 whichis a continuation of application Ser. No. 08/673,
`255, filed Jun. 28, 1996, now U.S. Pat. No. 6,006,227.
`FIELD OF THE INVENTION
`
`The present invention relates to an operating system in
`which documents are stored in a chronologically ordered
`“stream”.
`In other words,
`that
`is, as each document
`is
`presented to the operating system, the documentis placed
`according to a time indicator in the sequence of documents
`already stored relative to the time indicators of the stored
`documents.
`
`Within this application several publications are referenced
`by arabic numerals within parentheses. Full citations for
`these and other references may be found at the end of the
`specification immediately preceding the claims. The disclo-
`sures of all of these publicationsin their entireties are hereby
`incorporated by reference into this application in order to
`more fully describe the state of the art
`to which this
`invention pertains.
`BACKGROUND OF THE INVENTION
`
`Conventional operating systems frequently confuse inex-
`perienced users because conventional operating systems are
`not well suited to the needs of most users. For example,
`conventional operating systems utilize separate applications
`which require file and format
`translations.
`In addition,
`conventional operating systems require the user to invent
`pointless names for files and to construct organizational
`hierarchies that quickly become obsolete. Namedfiles are an
`invention of the 1950’s and the hierarchical directories are
`an invention of 1960’s.
`
`Some conventional operating systems employ a “desktop
`metaphor” which attempts to simplify common file opera-
`tions by presenting the operations in the familiar language of
`the paper-based world, that is, paper documents as files,
`folders as directories, a trashcan for deletion, etc. Also, the
`paper-based model is a rather poor basis for organizing
`information wherethestate ofthe artis still a messy desktop
`and where one’s choices in creating new information para-
`digmsis constrained [1].
`Thus, conventional operating systems suffer from at least
`the following disadvantages: (1) a file must be “named”
`when created and often a location in whichto store thefile
`must be indicated resulting in unneeded overhead;(2) users
`are required to store new information in fixed categories,
`that
`is directories or subdirectories, which are often an
`inadequate organizing device; (3) archiving is not automatic;
`(4) little support for “reminding” functions are provided; (5)
`accessibility and compatibility across data platforms is not
`provided and (6) the historical context of a documentis lost
`because no tracking of where, why and how a document
`evolves is performed.
`“Naming”a file when created and choosing a location in
`whichto place the file is unneeded overhead: when a person
`grabs a piece of paper and starts writing, no one demands
`that a name be bestowed on the sheet or that a storage
`location be found. Online, many filenames are not only
`pointless but useless for retrieval purposes. Storage loca-
`tionsare effective only as long as the user remembers them.
`Data archiving is an area where conventional electronic
`systems perform poorly compared to paper-based systems.
`
`2
`Paper-based systems are first and foremost archiving
`systems, yet data archivingis difficult in conventional desk-
`top systems. Often, users throw out old data rather than
`undertaking the task of archiving and remembering how to
`get the data back.If archiving and retrieval of documents is
`convenient, old information could be reused more often.
`Remindingis a critical function of computer-based sys-
`tems [2] [3], yet current systems supply little or no support
`for this function. Users are forced either to use location on
`their graphical desktops as rewinding cues or to use add-on
`applications such as calendar managers.
`A solution to these disadvantages is to use a document
`stream operating system. One such system is outlined in a
`1994 article [4]. However,this article fails to address many
`of the disadvantages of conventional operating systems.
`
`SUMMARYOF THE INVENTION
`
`invention is to provide a
`One object of the present
`documentstream operating system and method which solves
`many,if not all, of the disadvantages of conventional oper-
`ating systems.
`Another object of the present invention is to provide a
`document stream operating system in which documents are
`stored in one or more chronologically ordered streams.
`An additional object of the present inventionis to provide
`an operating system in which the location and natureoffile
`storage is transparentto the user, for example, the storage of
`the files is handled automatically and file names are only
`used if a user chooses to invent such names.
`
`A further object of the present invention is to provide an
`operating system which takes advantage of the nature of
`electronic documents. For example, a conventional paper
`document can only be accessed in one place, but electronic
`documents can be accessed from multiple locations.
`Another object of the present invention is to organize
`information as needed insteadofat the time the documentis
`
`10
`
`15
`
`20
`
`25
`
`30
`
`created. For example, streams may be created on demand
`and documents may belong to as many streams as seems
`reasonable or to none.
`
`40
`
`An additional object of the present inventionis to provide
`an operating system in which archiving is automatic.
`A further object of the present invention is to provide an
`operating system with sophisticated logic for summarizing
`or compressing a large group of related documents when the
`user wants a concise overview.In addition, this summarizing
`can include pictures, sounds and/or animations. Also, no
`matter how many documents fall into a given category, the
`operating system is capable of presenting an overview in a
`form so that all the documents are accessible from a single
`screen.
`
`Also, an object of the present operating system is to make
`“reminding” convenient.
`Another object of the present invention is to provide an
`operating system in which personal data is widely accessible
`anywhere and compatibility across platforms is automatic.
`Accordingly, this invention provides that computers using
`the operating system of the present invention need not be
`independent data storage devices, but also act as “view-
`points” to data store, and maintained on external systems
`such as the INTERNET. Thus,
`in accordance with the
`present invention users can access their personal document
`streams from any available platform such as a UNIX
`machine,
`a Macintosh or
`IBM-compatible personal
`computer, a personaldigital assistant (PDA), or a set-top box
`via cable.
`
`45
`
`50
`
`55
`
`60
`
`65
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 9
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 9
`
`

`

`US 6,725,427 B2
`
`3
`According to one embodiment of the invention a com-
`puter program for organizing one or more data units is
`provided. The computer program includes: (1) means for
`receiving one or more of the data units, each of which is
`associated with one or more chronological indicators; and
`(2) meansfor linking each ofthe data units according to the
`chronological indicators to generate one or more streams of
`data units. Other embodimentsof the invention also provide:
`(1) chronological
`indicators including past, present, and
`future times; and (2) means for displaying the streams,
`wherein respective indicia representing the data units are
`displayed and each data unit includes textual data, video
`data, audio data and/or multimedia data. The means for
`displaying the streams may further
`include displaying
`selected segments of the streams corresponding to selected
`intervals of time. The means for receiving may further
`include meansfor receiving data units from the Word Wide
`Web or from a client computer.
`According to another embodiment of the invention, a
`method of organizing one or more data units is provided
`including the steps of: (1) receiving on or more data units,
`each of which is associated with one or more chronological
`indicators; and (2) linking each of the data units according
`to the chronological
`indicators to generate one or more
`streams of data units. In other embodiments, the chrono-
`logical
`indicators may include past, present, and future
`times. The method may further include the steps of: (1)
`displaying the streams, wherein respective indicia represent
`each data unit and each of the data units may be textual data,
`video data, audio data and/or multimedia data. The step of
`displaying the streams may further include the steps of: (1)
`receiving from a user one or more values indicative of one
`or more selected segments of the streams corresponding to
`selected intervals of time; and (2) displaying the segments of
`the streams corresponding to the selected intervals of time.
`These and other advantages of the present invention will
`become apparent from the detailed description accompany-
`ing the claims and the attached figures.
`DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 shows a viewport
`present invention;
`FIG. 2 shows a substream menu in one embodimentof the
`
`in one embodiment of the
`
`present invention;
`FIG. 3 showsa list of summary types for the substream
`chosen in FIG. 2 of the present invention;
`FIG. 4 showsthe time display in one embodimentof the
`present invention;
`FIG. 5 shows a calendar-based dialog box in one embodi-
`ment of the present invention;
`FIG. 6a showsa dialog box in connection with a phone
`call in one embodimentof the present invention;
`FIG. 6b shows a summary of phone calls in one embodi-
`ment of the present invention;
`FIG. 7 shows a phone call record dialog box in one
`embodiment of the present invention;
`FIG. 8a showstext data used by one embodimentof the
`present invention; and
`FIG. 8b showsthe result of a summarize operation in one
`embodiment of the present invention.
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`This invention is a new model and system for managing
`personal electronic information which uses a time-ordered
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`stream as a storage model and stream filters to organize,
`locate, summarize and monitor incoming information.
`Together, streams and filters provide a unified framework
`that subsumes manyseparate desktop applications to accom-
`plish and handle personal communication, scheduling, and
`search and retrieval tasks. One embodiment of the present
`invention utilizes a machine-independent, client/server open
`architecture so that users can continue to use the same
`conventional document types, viewers and editors.
`A “stream” according to the present invention is a time-
`ordered sequence of documents that functions as a diary of
`a person or an entity’s electronic life. Every document
`created and every document send to a person or entity is
`stored in a main stream. The tail of a stream contains
`
`documents from the past, for example starting with an
`electronic birth certificate or articles of incorporation. Mov-
`ing away from thetail and toward the present and future, that
`is, toward head of the stream more recent documents are
`found including papers in progress or new electronic mail.
`A document can contain any type of data including but not
`limited to pictures, correspondence, bills, movies, voice mail
`and software programs. Moving beyondthe present and into
`the future, the stream contains documents allotted to future
`times and events, such as, reminders, calendar items and
`to-do lists.
`
`Time-based ordering is a natural guide to experience.
`Time is the attribute that comes closest
`to a universal
`
`skeleton-key for stored experience. Accordingly, streams
`add historical context
`to a document collection with all
`
`documents eventually becoming read-only, analogously as
`history becomes “set in stone”. The stream preserves the
`order and method of documentcreation. Also, like a diary,
`a stream records evolving work, correspondence and trans-
`actions because historical context can be crucial
`in an
`
`organizational setting.
`One embodimentof this invention allows for basic opera-
`tionsto be perform on a stream: new,clone, transfer, find and
`summarize.
`
`Users create documents by means of the new and clone
`operations. New creates a new, empty document and adds
`the document
`to the main stream. Clone duplicates an
`existing document and addsthe duplicate to the main stream
`at a new time point. Documents can also be created indi-
`rectly through the transfer operation. The transfer operation
`copies a document from one stream to another stream.
`Creation of a document if “transparent” because documents,
`by default, are added to the at
`the present
`time point.
`Internally, the documentis identified by a time indication so
`no name is required from the user for the document.
`Nevertheless, a user can optionally name a document is
`desired.
`
`Somestreams can be organized on the fly with the find
`operation. Find prompts for a search query, such as “all
`E-mail I haven’t responded to,” or “all faxes I’ve sent to
`Schwartz” and creates a substream. Substreams, unlike
`conventional, virtual or fixed directories which only list
`filenames, present
`the user with a stream “view” of a
`document collection. This view, according to the present
`invention contains all documents that are relevant to the
`
`search query. Also, unlike searches of conventional fixed
`directories, the substream is generated by default from all
`the documents in the main stream. Accordingly, individual
`substreams may overlap, that is, contain some documents
`that are the same and can be created and destroyed onthe fly
`without affecting the main stream or other substreams.
`The find operation creates a substream of the main stream
`or of another substream based on, for example, a boolean
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 10
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 10
`
`

`

`US 6,725,427 B2
`
`5
`attribute-and-keyword expression or a ‘chronological
`expression’, for example, “mylast letter to Schwartz”. Also,
`substreams may point to the future, for example, “my next
`appointment”.
`Once created, substreams operate dynamically, that is, if
`a user allows a substream to persist,
`the substream will
`collect new documents that match the search criteria as
`documentsarrive from outside the operating system or as the
`user creates the document. This dynamic operation provides
`automatic monitoring of information because the substream
`not only organizes the documentsas received but also filters
`for incoming information. For example, a substream created
`with the query “find all documents created by other people”
`would subsumea user’s mailbox and automatically collect
`all arriving mail from other users. A substream remains in
`existence until destroyed by the user and acts as a filter by
`examining each new documentthat enters the main stream.
`Although a document may belong to any number of
`substreams, the document also enters and remains on the
`main stream. A substream, in other words,is a “subset” of
`the main stream documentcollection. In other words, a way
`of looking at
`the main stream so as to exclude certain
`documents temporarily.
`The summarize operation “compresses” or “squish” a
`stream to generate one or more overview documents. The
`content of an overview document depends on the type of
`documentsin the stream. Forinstance, if the stream contains
`the daily closing prices of all the stocks and mutual funds in
`a user’s investmentportfolio, the overview documents may
`contain a chart displaying the historical performance of
`particular securities and the user’s net worth. As another
`example, if the substream containsa list of tasks a user needs
`to complete, the overview document mightdisplay a priori-
`tized “to-do”list. Thus, the summarize operation collapses
`a stream into a summary document. This summary docu-
`ment is a “live” document which is updated as additional
`documents are added to the main stream.
`
`The type of summary depends on the type of documents
`in the substream. In one embodimentof the present inven-
`tion at least one general “squish” function is provided no
`matter which stream is to be squished. Typically, however,
`the user will have a numberof different squishers to choose
`from, for example, one squisher might produce a summary
`in words, while another squisher might produce a graph.
`Also “custom squishers” may be supplied by third parties or
`created by the user.
`Another aspect of the invention is that applications
`execute “inside” a stream document leaving any output in
`that document. Thus, running an application is a variant of
`the new operation. For example, to run a spreadsheet appli-
`cation such as Lotus 1-2-3, a user creates a new document
`at the head of the main stream, specifically, a “live” spread-
`sheet document. The application itself is stored on the main
`stream, or located by meansof a calling card that points to
`another stream containing the application.
`A stream has three main portions: past, present, and
`future. The “present” portion of the stream holds “working
`documents”, which also includes the timepoint in the stream
`where new documents are created and where incoming
`documents are placed. As documents age and newer docu-
`ments are added, older documents pass from the user’s view
`and enter the “past portion”where the documents are even-
`tually “archived”. By disappearing from view old informa-
`tion is automatically cleared away so the old information
`will not clutter up the workspace. At some future point if
`documents in the past portion are needed, such documents
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`can be located with the find operation even if the past
`documenthas already been archived.
`The “future” portion of the stream allows documents to be
`created in the future. Future creation is a natural method of
`posting reminders, for example, meeting dates and sched-
`uling information. The system allows users to dial to the
`future by selecting a future timepoint for a document. The
`present invention keeps the document until that future time
`occurs. When the time of documents timepoint arrives the
`reminder documentis brought into view and the document
`enters the present portion of the stream.
`One embodimentof the present invention is implemented
`in a client/server architecture running over the Internet. The
`server is the workhorse of this embodiment handling one or
`more streams by storing all main stream and substream
`documents. Each view of a stream is implementedas a client
`of the server and provides the user with a “viewpoint”
`interface to documentcollections, that is, streams. The “look
`and feel” of the viewport may be different for different
`computing platforms but each viewport should support the
`basic operations.
`One embodiment of the present invention implements a
`client viewport using graphically based X Windows, another
`embodiment implementa client viewport solely with text in
`standard ASCII (American Standard Characters for Infor-
`mation Interchange) and yet another embodiment imple-
`ments a client viewport for the NEWTONpersonal digital
`assistant (PDA). The X Windowsviewport provides the full
`range of functionalities including picture and movie display.
`In contrast the text-only viewport has a mail-like interface
`althoughall the basic operationsare available. Also, because
`the NEWTONPDAlacks substantial internal memory and
`relies on slow external communications,
`that
`is low
`bandwidth, a minimal stream-access methodis provided.
`The X Windows viewport embodiment is shown in FIG.
`1. The interface is based on a visual representation of the
`stream metaphor 5. Users can slide the mouse pointer 10
`over the document representations to “glance” at each
`document, or use the scroll bar 20 in the lower left-hand
`corner to move through time, either into the past or into the
`future portion of the stream.
`Color and animation indicate important document fea-
`tures. A red border in one embodiment means “unseen” and
`
`a blue border means “writable”. Open doc documents may
`be offset to the side to indicate when the documentis being
`edited. In this embodiment incoming documents slide in
`from the left side and newly created documents pop down
`from the top and push the steam backwards by one document
`into the past.
`External applications are used to view and edit documents
`which the user can select by clicking on the documents
`graphical representation. The external applications speed the
`learning process significantly because new users can con-
`tinue to use familiar applications for example, conventional
`UNIX application such as emacs, xv, and ghostview,
`to
`create and view documents while using streams to organize
`and communicate the documents.
`
`The X Windowsinterface prominently displays the basic
`operations, that is, New 30, Clone 40, Xfer 50 (that is,
`transfer), Find 60, and Summarize 70 as buttons and/or
`menus. As discussed previously the New button creates a
`new document and adds the documentto the stream at the
`
`“present” timepoint. The Clone button duplicates an existing
`documentand places the copy in the stream. The Xfer button
`first prompts the user for one or more mail addresses and
`then forwards the selected document. The find operation is
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 11
`
`SPACETIME3D
`EXHIBIT 2007 - PAGE 11
`
`

`

`US 6,725,427 B2
`
`7
`supported through a text entry box 60 that allowsthe user to
`enter a boolean search query which results in a new sub-
`stream being created and displayed. The summarize menu
`70 generates a new document which displays information
`from documentsin a stream in a desired format, for example,
`a graph.
`The X Windowsinterface of this document also provides
`additional buttons. The Print button 80 copies a selected
`document
`to a printer where documents may be either
`printed conventionally or moved to a printer stream. A
`software agent which can be associated with the stream
`forwards each new documentto an appropriate printer. The
`Freeze button 90 makes a document read-only.
`Pulldown menus are used to select documents from
`streams or existing substreams, create summaries, initiate
`personal agents and change the clock.
`The Streams menu 110 allowsthe userto select from a list
`of locally available streams.
`FIG. 2 shows the Substreams menu 120 of one embodi-
`
`ment of the present invention. This menu is divided into
`three sections. The first section 130 contains a list of
`
`for
`operations that can be performed on substreams,
`example, remove. The second section 140 contains one
`menu entry labeled “Your Lifestream”, and causes the
`viewport to display a user’s main stream. Thethird section
`150 lists all of the user’s substreams. As indicated by this
`third section, substreams can be created in an incremental
`fashion,
`that
`is, one substream generated from another
`resulting in a nested set of menus. In this example the nested
`menus were created by first creating a substream
`“lifestreams and david” 160 from the main stream and then
`
`creating two substreams from this substream, “scenarios”
`and “ben” 170. Substream “scott” 180 was created from the
`
`“scenarios” substream. Semantically this incremental sub-
`streaming amounts to a boolean ‘and’ of each new query
`with the previous substream’s query.
`FIG. 3 shows the summarize menu 190 which lists the
`possible summary types. Choosing any of these menu
`options creates a substream summary and a new document
`containing the summary is placed on the stream.
`The Personal Agents menu 200lists a numberofavailable
`software agent types. Personal software agents can be added
`to the user interface in order to automate commontasks.
`
`The embodimentillustrated in FIG. 4 always displays the
`time in the upper right hand cornerof the viewport interface.
`This time display also acts as a “time” pull-down menu 190
`that allows the user to set the viewport time to the future or
`past via a calendar-based dialog box as illustrated in FIG. 5.
`Setting the viewport time causes the cursor to point to that
`timepoint position in the stream such that all documents
`forward of that timepoint, that is, towards the head of the
`stream have a future timestamp and all documents behind
`that timepoint, that is, towards the tail, have a past times-
`tamp. As time progresses,
`this cursor moves forward
`towards the head of the stream. When the cursor slips in
`front of the present timepoint “future” documents are added
`to the visible part of the stream in the viewpoint, just like
`new mailarrives.
`
`The effect of setting the time to the future or past is to
`reset the time-cursor temporarily to a fixed position desig-
`nated by the user. Normally the user interface displays all
`documents from the past up to the time-cursor. Setting the
`time-cursorto the future allows the user to see documentsin
`
`the future part of the stream. Creating a document in the
`future results in a document with a future timestamp. Once
`the user is finished time-tripping, the user can reset to the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`present time by selecting the “Set time to present” menu
`option in the time menu.
`invention “browse
`In one embodiment of the present
`cards” 100 are employed so that when the user touches a
`document in the stream-display with the cursor, a browse
`card appears. The purpose of the browse card is to help the
`user identify a document by providing the user someidea of
`the document’s contents in a small window. The content of
`browsecards is an abbreviated version of a document which
`as been compressed into an micro-documentlike an index
`card. In one embodiment, the browse card creation operation
`does header stripping so that the browse card displays the
`first non-trivial words in a document.
`In another
`embodiment, complex analysis is performed on the docu-
`ment contents so that
`‘most
`important’ words, pictures
`and/or sounds are presented.
`Another embodiment of the present invention provides
`“calling cards” which represent or point
`to a stream or
`substream Every stream has a calling card and the only way
`to reference a stream is via this calling card. In this embodi-
`ment the find operation performs as follows: (1) the user
`provides a search query; (2) an appropriate substream is
`generated; (3) the substream’s calling card is generated; and
`(4) the new calling card is deposited as a new documentat
`the head of the main stream. Every duplicated calling card
`bears on the face text, an icon or both. In the case of the find
`operation, the new calling card is marked with the argument
`supplied by the user for the search query, for example “from:
`Schwartz and Lifestreams”or “last letter from Piffel”. As a
`
`default in this embodiment, the interface will automatically
`display the new substream.
`invention allows
`Another embodiment of the present
`documents to be grouped explicitly into a substream. With
`this feature the user marks, that is, selects all documents to
`be included in the substream and groupsthe selected docu-
`ments into a substream by creating a