Docket No. 4138-4003
`
`U.S. PROVISIONAL PATENT APPLICATION
`
`FOR:
`
`METHOD AND APPARATUS FOR BROWSING USING
`
`MULTIPLE COORDINATED DEVICE SETS
`
`INVENTOR:
`
`RICHARD R. REISMAN
`
`
`
`MORGAN & FINNEGAN, LLP
`345 PARK AVENUE
`
`NEW YORK, NY 10154-0053
`(212)758-4800
`(212) 751-6849 (FACSIMILE)
`WWW.MORGANFINNEGAN.COM
`
`701516 V1
`
`EVDi=E'?'-I‘-H::l3US
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page1
`
`IPR2016-00047 Exhibit 2002 Page 1
`
`

`
`TITLE OF THE INVENTION:
`
`Method And Apparatus For Browsing Using Multiple Coordinated Device Sets.
`
`Docket No. 4138-4003
`
`Field of the Invention
`
`The present invention is directed generally to interactive television and similar interactive
`
`hypermedia such as from television or Internet sources, and more particularly to the provision
`
`and use of user interfaces that permit interaction using of multiple coordinated device sets.
`
`Background of the Invention
`
`While “convergence” of television (TV) and computer technology have been a major
`
`focus of innovation and commercial development since the early 1990s, particularly in the area
`
`of “interactive television” (ITV), there remains a huge gulf in the nature of the user experience of
`
`ITV and of computer-based media such as the World Wide Web. Convergence has taken hold in
`
`infrastructure technologies, with digital and computer-based TV (DTV) editing, production,
`
`distribution, transmission, and devices. At heart ITV is a matter of hypermedia browsing, the
`
`process of browsing linked media resources like the Web, differing only on its emphasis on
`
`video as the central medium.
`
`However, there remains a divide relating to the dramatic difference in how TV-centric
`
`and computer-centric media are used, and to the cultural divide between the TV production and
`
`distribution industry and the computer and Web industries that has prevented a convergence in
`
`user experience from developing or even being seen as possible and desirable. TV usage and
`
`directions are focused on its character as a lean—back, across-the-room, low resolution, and
`
`relatively passive, relaxed experience of couch potatoes viewing large, often shared TV screens
`
`5 2
`
`0
`
`701516 vl
`
`|PR2016—0O047
`
`Exhibit 2002
`
`Page 2
`
`IPR2016-00047 Exhibit 2002 Page 2
`
`

`
`Docket No. 4138-4003
`
`with simple remote controls. PC usage and directions are focused on its character as lean-
`
`forward, up-close, high resolution, and intensive, highly interactive experiences of individuals
`
`with PC-styles displays, keyboards, and pointing devices. Variant device sets and applications,
`
`such as PDAs, tablets, and video games, could be taken as suggestive of the desirability of
`
`selecting among alternative usage modes and form factors, but only very limited aspects of these
`
`suggestions have been recognized.
`
`The limitations of these radically disparate device set form factors have severely limited
`
`the appeal of ITV. ITV promises to greatly enrich the TV experience by allowing interactive
`
`features that can range from access to supplementary enhancement material such as background
`
`on programs, casts and players, sports statistics, polls, chat messaging, and interactive
`
`advertisements and purchase offers (“t-commerce”), and all manner of other tangential
`
`information, to ways to vary the core program content by acting on viewer input and choices as
`
`to camera angles or even alternative plots, as well as providing improved control of the core
`
`experience with electronic program guides (EPGS), personal video recorders (PVRs) and video
`
`on demand (VOD) and similar features.
`
`
`
`The problem is that these interactive features are not well served by the TV usage mode
`
`and form factor, and their use interferes with the basic TV experience. Rich interaction with a
`
`TV is inherently difficult. Presentation of information is limited by the poor capabilities of a TV
`
`screen for presenting text, menus, and navigations controls, and the crude input capabilities of a
`
`20
`
`remote control. The rich information and navigation functionality available on a Web browser or
`
`other PC-based user interface (e.g., UI, especially graphical user interfaces, GU15) must be
`
`“dumbed-down” and limited for use on a TV, and even use of high-definition TV (HDTV) may
`
`not significantly ease that—people do not like to read or do fine work from across-the-room, it is
`
`701516 vl
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 3
`
`IPR2016-00047 Exhibit 2002 Page 3
`
`

`
`Docket No. 413 8-4003
`
`just not comfortable ergonomics. Furthermore, the attempt to show interactive controls and
`
`enhancements on the TV interferes with viewing by the person interacting, as well as any other
`
`viewers in the room. Compounding these issues and slowing recognition of better solutions is
`
`the dominance of the cable TV industry, its struggles in developing and deploying the advanced
`
`set-top boxes (STBs) needed to offer meaningful ITV services of the form it envisions, and its
`
`orientation to closed, proprietary systems that do not fully exploit or adapt to advances in the PC
`
`and Internet world.
`
`
`
`The computer community has attempted to market PCs that include a TV tuner to support
`
`TV function in a PC-centric model, as promoted by the PC-DTV Consortium. However, these
`
`systems suffer from the converse problem, in that their form factors are not suited to the fact that
`
`most people do not want to watch TV at a PC, with its lean-forward, up-close form factor.
`
`Furthermore, such devices cannot effectively receive protected cable or satellite programming.
`
`And here, as with conventional TVs, the use of a single system forces technical, economical, and
`
`usage constraints on the inherently complex, multi-tasking, man-machine behavior that is desired
`
`in a rich hypermedia browsing experience.
`
`There has also been some recognition that PCs provide a way around the limited installed
`
`base of advanced STBs, but this is generally perceived only as a limited stopgap. So called
`
`Enhanced TV or Extended TV or “telewebbing” has emerged to exploit the fact that tens of
`
`millions of households have PCs in the same room as their TVs, and can surf related content on
`
`20
`
`the Web while watching TV. Some broadcasters such as ABC and PBS have exploited this to
`
`offer Web content synchronized to a TV program, but it is the user who must coordinate the use
`
`of the PC with the TV, by finding the appropriate Web site. In spite ofthe fact that the installed
`
`base for such open hardware is some ten times that of ITV-capable set-top boxes, the ITV
`
`701516 V1
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 4
`
`IPR2016-00047 Exhibit 2002 Page 4
`
`

`
`Docket No. 413 8-4003
`
`community generally views such “two-box” solutions as an unfortunate and awkward stopgap
`
`that may be desirably supplanted by advanced “one-box” systems whose wide deployment must
`
`be awaited. Some major reasons for this lack of acceptance are that this simplistic two-box
`
`model supports only very limited, pre-defined synchronization of the availability of TV and
`
`enhancement content that is built into a rigidly fixed two-box structure at the content source, and,
`
`even more importantly, that it completely fails to address any coordination of user activity at the
`
`two separate boxes.
`
`
`
`Across all of this, the key elements that are lacking are provision of a broadly flexible,
`
`powerful, selective, and simple user interface paradigm for browsing hypermedia across multiple
`
`device sets, whether they are integrated or not, with related methods for user and/or authoring
`
`control of such a U1, and provision of an effective method for independent systems to coordinate
`
`browsing activities to enable such a user interface to be employed across multiple independent
`
`systems. Further lacking across all of these aspects is delivery of these services in a way that
`
`provides the user with a smoothly integrated experience in which interactions on the multiple
`
`systems are coupled or decoupled to the degree appropriate to the task of the moment.
`
`Summary of Various Embodiments the Invention
`
`According to embodiments of the present invention there are provided systems and
`
`methods for navigating hypermedia using multiple coordinated input/output device sets.
`
`20
`
`Embodiments of the invention allow a user and/or an author to control what resources are
`
`presented on which device sets (whether they are integrated or not), and provide for coordinating
`
`browsing activities to enable such a user interface to be employed across multiple independent
`
`systems.
`
`701516 Vl
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 5
`
`IPR2016-00047 Exhibit 2002 Page 5
`
`

`
`Docket No. 41 38-4003
`
`Brief Description of the Drawings
`
`Further aspects of the instant invention will be more readily appreciated upon review of
`
`the detailed description of the preferred embodiments included below when taken in conjunction
`
`5
`
`with the accompanying drawings, of which:
`
`
`
`FIG. 1 is a block diagram of an exemplary assemblage of user systems, networks, and
`
`remote services for implementing certain embodiments of the present invention.
`
`FIG. 2 is a set of block diagrams of exemplary groupings of device sets and systems in
`
`the assemblage of FIG. 1.
`
`FIG. 3 is a schematic diagram of a number of exemplary user interface display layouts
`
`according to certain embodiments of the present invention.
`
`FIG. 4 is a schematic diagram of an exemplary structure for state information relating to
`
`systems within the assemblage of FIG. 1, relating to the coordination of a multimachine user
`
`interface according to certain embodiments of the present invention.
`
`FIG. 5 is a schematic diagram of an exemplary process, performed by the systems ofFIG.
`
`1, for transferring state data according to certain embodiments of the present invention.
`
`FIG. 6 is a flow chart of an exemplary process, performed by the systems of FIG. 1, for
`
`transferring state data according to certain embodiments of the present invention.
`
`20
`
`Detailed Description of the Invention
`
`As used herein, the term “hypermedia” is meant to refer to any kind of media that may
`
`have the effect of a non-linear structure of associated elements represented as a network of
`
`information-containing nodes interconnected by relational links. Hypermedia is meant to include
`
`701516 V1
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 6
`
`IPR2016-00047 Exhibit 2002 Page 6
`
`

`
`Docket No. 4138-4003
`
`“hypertext”, and the two may at times be used synonymously in the broad sense, but where
`
`stated or otherwise clear in context, “hypertext” can refer particularly to text content, and
`
`“hypermedia” to extend that to content that includes other formats such as graphics, video, and
`
`sound. The terminology used herein is meant of be generally consistent with that used in World
`
`Wide Web Consortium (W3 C) recommendations.
`
`The associations of elements may be specified as “hyperlinks” or “links,” such as
`
`described by the XLink (XML Linking Language), SMIL (Synchronized Multimedia Integration
`
`Language), HTML, XHTML, and similar W3C recommendations. Links define an association
`
`
`
`between a “starting resource,” the source from which link traversal is begun, and an “ending
`
`resource,” the destination, collectively referred to as “participating resources.” A “resource” is
`
`used to refer to any addressable unit of information or service and may at times refer to a
`
`resource portion rather than a whole resource, and a “content resource” to refer to any resource
`
`suited to presentation to a user. In the context of hypermedia, “node” may be used
`
`synonymously with resource. “Navigation” is meant to refer to the process of following or
`
`“traversing” links. Unless specifically indicated as “link navigation” or otherwise clear in
`
`context, navigation also is meant to include the control of presentation within a resource, such as
`
`scrolling, panning, and zooming, using VCR-like controls to play a continuous media resource,
`
`and the like. Addresses for Internet resources are typically in the form of Universal Resource
`
`Locators (URLs) or Universal Resource Names (URNs) or other Universal Resource Identifiers
`
`20
`
`(URIS), but may be based on any other suitable addressing mechanism. Hypermedia resources
`
`may contain content (also referred to as mediadata) and metadata (including hyperlinks), aspects
`
`of a resource may be declarative (such as markup) or procedural (such as embedded logic or
`
`program code elements) and may include embedded resources.
`
`701516 Vl
`
`IPR2016-00047
`
`Exhibit 2002
`
`Page 7
`
`IPR2016-00047 Exhibit 2002 Page 7
`
`

`
`Docket No. 4138-4003
`
`Links may have information about how to traverse a pair of resources, including direction
`
`and application behavior information, called an “arc,” and such information may include link
`
`“elements” having “attributes” that take on “values.” Behavior attributes include “show” to
`
`specify how to handle the current state of the presentation at the time the link is activated,
`
`“externa ” to specify whether the link is to be opened in the current application, or an external
`
`application, such as one suited to a special media type, “activate” to specify Whether the link is
`
`triggered by some event, typically user interaction, or automatically traversed when its time span
`
`is active, and “target” to specify either the existing display environment in which the link should
`
`be opened (e.g., a SMIL region, an HTML frame or another named window), or trigger the
`
`creation of a new display environment with the given name. It should be noted that the term
`
`target is sometimes also used in the art to refer to an ending resource as the target of a link, as for
`
`a “target resource” or “target page.”
`
`Links may be contained in the starting or ending resource, “outbound” or “inbound”
`
`respectively, or may be independently stored as “third-party” arcs. Standard HTML links are
`
`typically outbound, but inbound and third-party link arcs may be useful, such as for adding links
`
`that are external to read-only or third-party content. By providing such external, third-party
`
`links, resources not originally intended to be used as hypermedia can be made into hypermedia.
`
`Third-party links may be collected in “linkbases.” Linkbases may be directly associated with
`
`
`
`their starting resources by a resource that leads to both the starting resource and the linkbase,
`
`20
`
`referred to herein as “coupled” linkbases, such as a set of image map links in a Web page that
`
`has an embedded image link, or may be “decoupled” and obtained by other means.
`
`Where so indicated or clear in context, the term hypermedia may also be used to include
`
`“hypermedia-like” resources and systems that do not use coded links as such, but which support
`
`701516 vl
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page8
`
`IPR2016-00047 Exhibit 2002 Page 8
`
`

`
`Docket No. 413 8-4003
`
`functionally similar non-linear resource relationships using other more or less similar
`
`mechanisms, such as special coding and logic that implements structures such as menu structures
`
`that have a defined graph structure, transaction request forms that have an associated address or
`
`other process identifier for transaction submission, and selectable content elements having a
`
`defined relationship to other resources or actions. Use of VCR-like or audio recorder-like
`
`controls to add non-linearity to a linear medium (e.g., fast forward/reverse, and skip ahead), also
`
`referred to as “trick-play” functionality, is also considered as hypem1edia—like.
`
`According to embodiments of the invention, links may refer to specific portions of a node
`
`
`
`l”'.‘.;
`in";
`
`or resource, such a by an “anchor” that associates the link to a position in text (such as in a
`
`HTML “A element”), or an “area” or “region” that associates the link to a spatial portion of an
`
`object’s visual display, or to non-spatial portions, such as temporal subparts that may be defined
`
`by “begin” and “end” attributes, also referred to as “time positions” or together as a “time scope”
`
`or “time-span.” Similar facilities are provided by XPointer, which supports addressing into the
`
`internal structures of XML documents, and provides an “origin” function to enable addressing
`
`relative to third-party and inbound links. Unless otherwise indicated or clear in context,
`
`“anchor” may used herein to be synonymous with similar forms, such as origin and “area.”
`
`Hypermedia structures may also be understood in graph—theoretic terms, and modeled as
`
`a directed graph, consisting of a set of abstract “nodes,” the resources, joined by directional
`
`“edges,” the hyperlinks. In this usage, a linkbase defines a directed graph.
`
`20
`
`As used herein, and consistent with the Dexter Hypertext Reference Model, a
`
`“hypermedia system” allows users to create, manipulate, and/or examine hypermedia, and
`
`consists of a “run-time layer” that provides tools for accessing, viewing, navigating, and
`
`manipulating hypermedia, a “storage layer” that models the basic node/link or resource/link
`
`701516 vl
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 9
`
`IPR2016-00047 Exhibit 2002 Page 9
`
`

`
`Docket No. 4138-4003
`
`network structure of the hypermedia, and a “within component layer” that addresses the structure
`
`of components or resources of various given types. The storage layer, as used herein, includes
`
`media that may be streamed directly from a media capture device, such as a camera, microphone,
`
`or other sensor, and may not actually be stored. “Streaming” refers to this process of
`
`transmitting a resource representation, whether or not the resource is stored or not, and the
`
`representation may be in a format suited to storage, or one specifically suited to streaming. A
`
`data stream may itself contain multiple data streams, including both continuous media streams
`
`and other kinds of data or resources, including discrete resources, metadata, and the like.
`
`Depending on the particular embodiment, streams may contain channels, or channels may
`
`contain streams. Linkbases associated with streamed media may also take the form of
`
`continuous metadata streams, whether embedded with the mediadata stream or as an independent
`
`stream.
`
`As used herein, a “browser” or “media browser” is meant to include any kind of
`
`presentation system capable of presenting media, and is used synonymously with “user agent” as
`
`a process within a device that renders the presentation data for a resource into physical effects
`
`that can be perceived and interacted with by the user. A “hypermedia browser” includes
`
`browsers that support hypennedia, including standard Web browsers, SMIL players, interactive
`
`television presentation systems (including self-contained advanced TVs and TVs with set—top
`
`
`
`boxes), and the like, and specialized applications capable of presenting hypermedia, including
`
`20
`
`word processors, multimedia and video editors, virtual reality presentation systems, game
`
`players, and the like. “Player” or “viewer” may be used as synonymous with browser, and use of
`
`any media type descriptor as an adjective with “browser" refers to a browser capable of that
`
`media type. Thus any conventional TV set is included as a “browser” or a “TV browser,” and
`
`70l516vl
`
`10
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page10
`
`IPR2016-00047 Exhibit 2002 Page 10
`
`

`
`Docket No. 4138-4003
`
`music players and radios are also included as browsers unless otherwise indicated or clear in
`
`context. Cases where hypermedia functions are not used are referred to as “linear” or “simple”
`
`presentation, viewing, or listening. “Media player” is used to refer to all such players
`
`collectively. Similarly, “browsing” is used to refer to any kind of viewing or playing experience,
`
`inclusive of hypermedia browsing and simple or linear viewing (such as watching TV), unless
`
`otherwise indicated or clear in context.
`
`Web browsers are commonly limited to read-only use, except perhaps in use of forms,
`
`but other hypermedia systems are not so limited, and as used herein, unless otherwise stated,
`
`such as by the term “pure browser” or clear in context, “browser” is meant to include systems
`
`capable of resource creation and editing as well, including sound and Video editing. Key
`
`functions of a browser include, but need not be limited to, providing access to resources,
`
`presentation of resources to the user and navigation of hyperlinks under user control or as
`
`
`
`directed by the hypermedia resources and links.
`
`“Presentation” is meant to include any means of making a resource sensible to a human
`
`user, including visual display and audio, as well as any other sensible presentation such as used
`
`in current and future virtual or augmented reality systems affecting the sight, sound, touch,
`
`haptic, smell, taste, motion sensing, heat sensing, neural or other physiological interface, and the
`
`like. In addition to such “output,” presentation also includes recognizing and responding
`
`appropriately to user “input” and/or “signals” of any kind that may be provided for, including
`
`20
`
`keyboard, character recognition, touchpad, pointing device, haptic, microphone/speech, and
`
`camera, as well as more exotic inputs such as gesture, body movement, brain
`
`wave/electroencephalogram, neural or other physiological interface, and the like.
`
`701516 vl
`
`11
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 11
`
`IPR2016-00047 Exhibit 2002 Page 11
`
`

`
`Docket No. 413 8-4003
`
`“Media format” or, synonymously, “resource format,” as used herein refers to the format
`
`of a resource as retained, or potentially retained, as when streamed, in the storage layer and
`
`accessed by the browser, including access from local storage, via communications from a remote
`
`storage location or server or as streamed from storage or a live capture source. “Presentation
`
`format” refers to the format as rendered or otherwise processed by a browser or equivalent
`
`viewer or player or presentation system for actual presentation to a human user in sensible form.
`
`“Hypermedia system” as used herein refers broadly to all system elements comprising
`
`such a system, including the hardware, software, communications, and storage, including
`
`portions at a user location, portions at server/peer locations providing content and processing
`services, potentially including the entire Internet or any similar network to the extent that those
`
`elements are usable with a hypermedia presentation system and the resources that may be
`
`accessible to it. “User system” refers to the portions local to or controlled by an individual user
`
`or a group of users of a shared presentation system. “Server” or “server system” refers to any
`
`system, whether hardware or software, providing auxiliary services that may be supportive of a
`
`user system. “Remote servers” include content servers or repositories, application servers that
`
`
`
`may perform information processing, searching, e-commerce, or other transaction or support
`
`services remote from the user, including TV and video servers, audio servers, other storage
`
`servers, including storage area networks (SANS), network addressable storage (NAS), game
`
`servers, virtual reality servers, cable and satellite TV and ITV head-end systems, network servers
`
`20
`
`such as proxies and caching servers, and the like. “Head-end server” is meant to be inclusive of
`
`other remote servers that may be reached via the head-end, regardless of actual location or
`
`function. “Local servers” include analogous services that may be local to the user, including
`
`media servers, gateways, controllers, PCs, hubs, storage servers, storage area networks, DVRS
`
`701516 vl
`
`12
`
`IPR2016-00047
`
`Exhibit 2002
`
`Page 12
`
`IPR2016-00047 Exhibit 2002 Page 12
`
`

`
`Docket No. 4138-4003
`
`(also referred to a PVRs). Peer systems may also provide services in “peer-to-peer” (P2P)
`
`systems, and unless otherwise indicated or clear in context, the term server is meant to include
`
`peers acting in service provider roles.
`
`“User” as used herein refers to any human end-user of a system, and may include users of
`
`a shared system. Users may be private consumers or workers in an organization or enterprise.
`
`User and “viewer” may be used synonymously. Depending on context, “subscriber” may refer to
`
`a user of a subscription service or more loosely to any user. “User interaction session” or “user
`
`session” as used herein refers to a series of interactions with a hypermedia system by a user,
`
`especially a series having a degree of continuity and relationship in time and with regard to an
`
`activity workflow or series of workflows, including concurrent workflows that may be related by
`
`a multitasking user.
`
`According to embodiments of the invention, a user session may be composed of one or
`
`more “browser sessions,” and well as other “application sessions” with other applications. The
`
`relationship of such sessions with each other within a user session may vary with different
`
`embodiments and with the settings and circumstances. For example, with enhancements to a
`
`TV-centric browsing experience, it might normally be appropriate that the base TV program and
`
`the related enhancement session be considered as “linked sessions” or sub-sessions that are
`
`distinct from one another, so that a browser session transfer is understood to transfer the
`
`
`
`enhancement session, but not the base TV session. The terms “transfer” and “migrate” are used
`
`20
`
`synonymously to refer to the movement of the locus of work of a session, such as from one
`
`system or device set to another. The term “clone” is used to refer to a transfer that duplicates the
`
`current resource presentation of a session at a second device set. A migration that deactivates the
`
`session at the original device set is referred to as a “complete migration” or “terminal migration.”
`
`701516 vl
`
`13
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 13
`
`IPR2016-00047 Exhibit 2002 Page 13
`
`

`
`Docket No. 413 8-4003
`
`A user session may be local to the user system or may involve one or more
`
`“communications sessions” with remote server or eer s stems, where such communications
`Y
`
`sessions may be defined in accord with a communications protocol. A user session may be
`
`composed of multiple “client/server sessions” (or “peer sessions” or “client/server/peer
`
`sessions”, or collectively “remote sessions”), including concurrent such sessions. A “server
`
`session” refers to a series of activities performed by a server in support of a series of client/server
`
`service requests (and similarly for a “peer session” and “remote session”). Except where
`
`indicated otherwise or clear from context, references to peer-to-peer and client/server are meant
`
`to be inclusive of one another. Some protocols, such as HTTP for example, may be sessionless
`
`(based on request-response sets only), so that a remote HTTP communications session may
`
`strictl
`
`Y
`
`s eakin be com osed of multi 1e se arate communications interchan es at the rotocol
`8
`P
`P
`P
`g
`P
`
`level that are related by the server into a single server session, and this can be thought of as
`
`constituting a single virtual communications session. Unless otherwise stated or clear from
`
`context, communications session is meant to include such virtual sessions.
`
`“Shared sessions” or “multi-user sessions” are applicable to multi-user systems where
`
`users cooperate or collaborate in controlling an interactive session, are recognized as individuals,
`
`
`
`and retain their individual identity and state.
`
`“State” refers to the representation of the current state of a system relating to one or more
`
`tasks or sessions, usually in discrete values of some set of “state variables” that can be stored as a
`
`20
`
`“state record” sufficient to define the state fully enough to allow the current activity to be
`
`deactivated and then reactivated, such as in a context switch or shutdown, using the state record
`
`to reset it so that it then behaves as if never interrupted. “Session state” refers to the state of a
`
`user session, for a browser session typically including, depending on the granularity desired, a
`
`701516 vl
`
`14
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page 14
`
`IPR2016-00047 Exhibit 2002 Page 14
`
`

`
`Docket No. 4138-4003
`
`selection of such state variables as the user identity and related authentication information
`
`(including for example password and certificate information), the identification of active
`
`hypermedia resources and details of how they are currently rendered (such as window sizes and
`
`locations, and scrolling state), link are data for any link currently being traversed, the execution
`
`state of embedded logic components such as Java applets (including the state of a Java Virtual
`
`Machine, JVM), ActiveX controls, Javascript (or ECMAScript, or Jscript, or other scripts), or
`
`FLASH, or other plug-ins, or helper applications, or the like, navigation path history (the ordered
`
`list of resources back and forward from the current resource, corresponding those activated by
`
`the back and forward browsing controls, as well as, optionally, next and previous with regard to
`
`tree branches), selected interaction history, variable user preferences, status of communications
`
`and server/peer sessions (including addresses, ports, identities, and authentication information),
`
`and other current context regarding other internal and external resources, including such
`
`information as may be stored in cookies. Any or all of such information may be stored in a
`
`“state record.” State records may include details of user interactions not yet saved in the storage
`
`layer, including edits and forms field inputs not yet submitted. State may also include data on
`
`link arcs, including trigger data, and on resources, if such data must be transferred to establish
`
`
`
`state in a coordination embodiment in which such information carmot be obtained directly by the
`
`coordinated system. Sessions, software processes, and the like that are characterized by state
`
`variables are referred to as “stateful” and those that are not, as “stateless.”
`
`20
`
`“Software process state” refers to the program environment state of a software process as
`
`it runs on a system. A process typically runs with the support of an operating system, and its
`
`state typically includes the current values of the instruction counter, registers, dynamic memory,
`
`input/output activities, and open or assigned operating system, network, and hardware resources,
`
`701516 vl
`
`15
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page15
`
`IPR2016-00047 Exhibit 2002 Page 15
`
`

`
`Docket No. 4138-4003
`
`as well as active sessions with external systems, and is used synonymously with “task,” as an
`
`operating system concept that refers to the combination of a program being executed and
`
`bookkeeping information used by the operating system. Note however that “task” is more
`
`commonly used herein to refer to tasks at the user and/or session level. A software process is
`
`meant to include any of application software, middleware, and system software, and the case of a
`
`pure hardware, firmware, or dedicated implementation is also meant to be included in this usage.
`
`A “process instance” refers to a single process with its associated state information. It
`
`may be possible to run multiple browser process instances on a single computer, sharing some
`
`system resources, such as caches, persistent storage, network access, and the like, in common,
`
`and thus having some state elements in common. Depending on implementation, a browser
`
`instance may allow for multiple presentation windows to be open, each presenting a different
`
`resource (and, as for example in MSIE, each supported by a separate process thread within one
`
`browser process). In such cases, depending on context, browser state may refer to the entire set
`
`of state information for all active browsers or the information for one browser instance (also
`
`referred to as one browser), for all its active windows. The term “current state” may be used to
`
`denote more limited state information on the single window, or single browser instance that is
`
`
`
`currently in focus for user interaction.
`
`“Context” may be used as generally synonymous with state in referring to the
`
`information needed to allow a session to be interrupted, moved, copied, restarted, or otherwise
`
`20
`
`shifted without apparent loss of context beyond the intended change. Context may also be used
`
`to refer to broader aspects of state that go beyond and are external to the state of the application,
`
`hardware, software, and network, to include the user, both in regard to his session, and
`
`potentially to the broader situation and environment of the session, including aspects that may be
`
`701516 vl
`
`16
`
`|PR2016—00047
`
`Exhibit 2002
`
`Page16
`
`IPR2016-00047 Exhibit 2002 Page 16
`
`

`
`Docket No. 4138-4003
`
`sensed or inferred. This broad usage of context is defined (by Dey in “A Conceptual Framework
`
`and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Ap