miniliiHIMinnmumu1unimuinnmumuinninmiiin
`
`US 20130061273A1
`
`(i9) United States
`(12) Patent Application Publication no Pub. No.: US 2013/0061273 Al
`Mar. 7, 2013
`(43) Pub. Date:
`Reisman
`
`filed on Sep. 6, 2002, provisional application No.
`60/455,433, filed on Mar. 17, 2003.
`Publication Classification
`
`(51)
`
`(54) METHOD AND APPARATUS FOR BROWSING
`USING ALTERNATIVE LINKBASES
`
`(71) Applicant: Richard Reisman, New York, NY (US)
`
`(72)
`
`Inventor: Richard Reisman, New York, NY (US)
`
`(21) Appl . No.: 13/662,213
`
`(22) Filed:
`
`Oct. 26, 2012
`
`Related U.S. Application Data
`(63) Continuation of application No. 13/094,505, filed on
`Apr. 26, 2011, which is a continuation of application
`No. 10/434,032, filed on May 8, 2003, now Pat. No.
`7,987,491.
`(60) Provisional application No. 60/379,635, filed on May
`10, 2002, provisional application No. 60/408,605,
`
`(2011.01)
`
`725/86
`
`Int. Cl.
`H04N 7/173
`(52) U.S. Cl
`(57)
`ABSTRACT
`Systems and methods for navigating hypermedia using mul ¬
`tiple coordinated input/output device sets. Disclosed systems
`and methods allow a user and/or an author to control what
`resources are presented on which device sets (whether they
`are integrated or not), and provide for coordinating browsing
`activities to enable such a user interface to be employed
`across multiple independent systems. Disclosed systems and
`methods also support new and enriched aspects and applica ¬
`tions of hypermedia browsing and related business activities.
`
`SYSTEMS /
`DEVICE SETS
`
`130
`
`TV (ITV)
`
`PC
`
`140
`
`50
`
`PDA / PHONE
`
`100
`
`NTERNET
`
`124
`
`HOME
`NETWORK
`/ LAN
`
`128
`
`WIRELESS
`NETWORK
`
`126
`
`LOCAL
`STORAGE
`
`160
`
`SYSTEM ELEMENTS
`
`CONTENT /
`CONNECTIVITY
`110
`
`BROADCAST
`SATELLITE
`CABLE
`VIDEO ON DEMAND
`STREAMING MEDIA
`WEB
`WIRELESS PORTALS
`TRANSACTIONS
`AND THE LIKE
`
`IPR2016-00047 Exhibit 2003 Page 1
`
`

`
`CONTENT /
`CONNECTIVITY
`110
`
`BROADCAST
`SATELLITE
`CABLE
`VIDEO ON DEMAND
`STREAMING MEDIA
`WEB
`WIRELESS PORTALS
`TRANSACTIONS
`AND THE LIKE
`
`SYSTEMS /
`DEVICE SETS
`
`130
`
`TV (ITV)
`
`PC
`
`140
`
`150
`
`PDA / PHONE
`
`100
`
`INTERNET
`
`124
`
`HOME
`NETWORK
`/ LAN
`
`128
`
`WIRELESS
`NETWORK
`
`126
`
`LOCAL
`STORAGE
`
`160
`
`SYSTEM ELEMENTS
`
`FIG. 1
`
`Patent
`Application
`Publication
`
`Mar.
`7,2013
`Sheet1
`
`of
`10
`
`US2013/0061273A1
`
`IPR2016-00047 Exhibit 2003 Page 2
`
`

`
`Patent Application Publication
`
`Mar. 7, 2013 Sheet 2 of 10
`
`US 2013/0061273 A1
`
`200
`
`250
`
`128
`
`128
`
`DEVICE SET 1
`
`TV
`
`212
`
`RC
`
`214
`
`MONITOR
`222
`
`KB
`
`224
`
`DEVICE SET 2
`
`DEVICE SET 1
`
`262
`
`RC
`
`MONITOR
`272
`
`KB
`
`274
`
`\ STB
`(SYSTEM 1)
`/
`210
`
`\ PC
`(SYSTEM 2)
`/
`220
`
`FIG. 2a
`
`CONTROLLER
`(STB/PC)
`(SYSTEM 1)
`260
`
`DEVICE SET 2
`
`FIG. 2b
`
`IPR2016-00047 Exhibit 2003 Page 3
`
`

`
`Patent Application Publication
`
`Mar. 7, 2013 Sheet 3 of 10
`
`US 2013/0061273 A1
`
`TV
`
`ITV
`
`332
`333XJ
`
`PC
`(+ITV)
`
`300
`
`TYPICAL DISPLAYS
`
`FIG. 3
`
`312
`
`A
`(VIDEO)
`
`322
`
`MENU
`
`A
`(PIP)
`
`INTERACTIVE
`CONTENT
`
`314
`
`326
`
`324
`
`310
`
`320
`
`WINDOW n
`WINDOW 2
`
`336
`
`334
`
`338
`
`* MENU BAR
`FMENU
`DROP¬
`DOWN
`
`A
`(VIDEO)
`
`INTERACTIVE
`CONTENT
`
`T1 T2
`
`r
`
`330
`
`PDA/
`(PHONE)
`
`MENU
`SCREEN
`
`CONTENT
`SCREEN
`
`FUTURE
`VIDEO
`A
`
`MENU
`
`340
`
`342
`
`344
`
`346
`
`IPR2016-00047 Exhibit 2003 Page 4
`
`

`
`PORTABLE STATE
`430
`
`410
`
`TRANSFER STATE
`RECORD (FORM)
`450
`
`400
`
`SYSTEM
`A
`
`BASE STATE
`
`STATIC
`SETTINGS
`
`412
`
`SESSION A1
`TRANSIENT
`STATE
`
`414
`
`SESSION A2
`TRANSIENT
`STATE
`
`416
`
`:
`
`SUPPLEMENT A 418
`
`A
`
`STATIC
`452
`
`A1
`454
`
`STATIC
`432
`
`A
`
`A1
`
`iiisiiiii
`
`A2
`436
`
`l
`SUPPLEMENT A
`
`SYSTEM
`B
`
`STATIC
`SETTINGS
`
`422
`SESSION B1
`TRANSIENT
`STATE
`
`424
`
`SESSION B2
`TRANSIENT
`STATE
`
`426
`
`420
`
`440
`
`STATIC
`442
`
`B
`
`B1
`444
`
`B2
`446
`
`SUPPLEMENT B
`
`428
`
`SUPPLEMENT B
`
`448
`
`BASE STATE
`AFTER TRANSFER
`(B +A1')
`
`460
`
`STATIC
`SETTINGS
`B + A1'
`
`462
`
`SESSION B1
`TRANSIENT
`STATE
`
`464
`
`SYSTEM
`B
`
`SESSION B2
`TRANSIENT
`STATE
`
`466
`SESSION A1'
`TRANSIENT
`STATE
`
`466
`
`SUPPLEMENT B 468
`
`STATE DATA
`FIG. 4
`
`Patent
`Application
`Publication
`
`Mar.
`7,2013
`Sheet4
`
`of
`10
`
`US2013/0061273A1
`
`IPR2016-00047 Exhibit 2003 Page 5
`
`

`
`Patent Application Publication Mar. 7, 2013 Sheet 5 of 10
`
`US 2013/0061273 A1
`
`500
`
`TRANSFER
`STATE RECORD
`(SESSION A1)
`
`550
`
`2
`
`TRANSFER
`STATE RECORD
`(SESSION A1'
`TRACKING)
`
`555
`
`SESSION
`TRANSFER
`
`FIG. 5
`
`SESSION A1
`520
`
`CO_/
`Occ
`
`L2Oo
`
`530
`
`510
`
`SYSTEM
`A
`
`-*
`
`aL
`
`Uco=>
`
`STATE
`EXPORTER / IMPORTER /
`TRACKER
`
`540
`
`TRANSFERRED
`SESSION A1'
`570
`
`CO_l
`
`Oc
`
`tK2oo
`
`560
`SYSTEM
`B
`
`OL
`
`U
`CO
`
`3 *- -
`
`580
`
`STATE
`EXPORTER / IMPORTER /
`TRACKER
`
`590
`
`*
`
`IPR2016-00047 Exhibit 2003 Page 6
`
`

`
`Patent Application Publication
`
`Mar. 7, 2013 Sheet 6 of 10
`
`US 2013/0061273 A1
`
`600
`
`(USER
`INPUT)
`
`SYSTEM
`B
`UNRELATED ACTIVIES
`IN PROGRESS
`
`650
`
`IMPORT SESSION A1 ( ... Ai)
`STATE
`
`655
`
`1
`SETUP A1' (...Ai')
`ON SYSTEM B
`
`660
`
`IF TRACKING,
`SYNCHRONIZE SESSIONS A1
`AND A1' AND ECHO
`INTERACTION EVENTS
`665
`
`SYSTEM
`A
`INTERACTIVE SESSION
`IN PROGRESS
`
`1 f
`
`605
`
`TRANSFER REQUEST:
`SESSION A1( ... Ai)
`TO SYSTEM B
`
`610
`
`T
`ASSEMBLE TRANSFER STATE
`RECORD(S) FOR A1 (...Ai)
`615
`
`EXPORT TO SYSTEM B
`(OR CONTROLLER)
`
`620
`
`IF TRACKING,
`SYNCHRONIZE SESSIONS A1
`AND A1' AND ECHO
`INTERACTION EVENTS
`625
`
`IF NO TRACKING,
`DISPOSE OF A1 AS REQUESTED
`630
`
`SYSTEM FLOW
`FIG. 6
`
`IPR2016-00047 Exhibit 2003 Page 7
`
`

`
`Patent Application Publication
`
`Mar. 7, 2013 Sheet 7 of 10
`
`US 2013/0061273 A1
`
`Full Connectivity
`
`TV
`130
`t k
`
`PC
`
`140 *-
`
`700
`
`120
`
`FIG. 7a
`
`No Local Connection (e g.: Cable Relay)
`
`TV
`
`130
`
`„
`
`PC
`140
`
`W
`
`\
`
`120
`
`r
`
`1
`
`FIG. 7b
`
`No/Limited Back-channel (eg.: Satellite)
`
`TV
`
`130 -
`
`j k
`
`PC
`140
`
`120
`
`FIG. 7c
`
`IPR2016-00047 Exhibit 2003 Page 8
`
`X
`
`

`
`Device Set 1 - TV
`
`TV
`212
`
`RC
`214
`
`Monitor
`222
`
`Keyboard
`224
`
`Device Set 2 - PC
`
`Set-Top Box
`
`210
`
`PC'
`
`220
`TV Progr.
`A 2 TV Ad
`§
`EPG+
`U
`+++?.
`
`USER
`
`800
`
`Cable
`Head-end
`810
`
`i
`
`Session
`Coordination
`
`r
`Cable Operator
`Portal
`820
`
`Tv Content
`TV
`Ads
`Programming
`850
`
`830
`
`Enhancement Co itent
`Ads
`TV
`Programming
`660
`
`840
`
`Coordination/Relay Portal Service
`Fig. 8
`
`Patent
`Application
`Publication
`
`Mar.
`7,2013
`Sheet8
`
`of
`10
`
`US2013/0061273A1
`
`IPR2016-00047 Exhibit 2003 Page 9
`
`X
`
`

`
`900
`
`WINDOW n
`WINDOW 2
`
`920
`
`MENU BAR
`
`ONTROL PANEL
`
`MULTIPANE PAGE
`-CHAN A / PROG A
`IND. PAGE A1
`PROGR. PAGE A1
`-CHAN B / PROG B
`IND. PAGE B1
`PROGR. PAGE B1
`-CHAN A / AD A1
`IND. PAGE AD A1
`ADV. PAGE AD A1
`+CHANA / ADA2
`-CHAN B / AD 81
`IND. PAGE AD B1
`ADV. PAGE AD B1
`+CHAN A / AD B2
`
`EMBEDDED
`AD
`940
`
`T1
`
`T2
`
`PROGRAM A
`INDEPENDENT ENHANCEMENT PAGE
`
`ADA1
`INDEPENDENT ENHANCEMENT PAGE
`
`EMBEDDED
`AD
`945
`
`PROGRAM A
`PROGRAMMER ENHANCEMENT PAGE
`
`ADA 1
`PROGRAMMER ENHANCEMENT PAGE
`
`Patent
`Application
`Publication
`
`Mar.
`7,2013
`Sheet9
`
`of
`10
`
`330
`
`MULTICHANNEL ENHANCEMENTS
`
`FIG. 9
`
`\
`4
`
`930
`
`334
`
`US2013/0061273A1
`
`IPR2016-00047 Exhibit 2003 Page 10
`
`X
`
`

`
`Patent Application Publication
`
`Mar. 7, 2013 Sheet 10 of 10
`
`US 2013/0061273 A1
`
`FIG.10
`
`a:
`-2LU
`
`Oi
`
`0Q
`
`1000
`
`>
`
`£1
`
`LLl
`Q
`O
`
`O
`
`_i_
`
`/
`
`/
`
`/
`
`a:
`
`cd
`
`CO
`
`§
`
`CO
`
`\
`\
`\
`
`\
`
`Q_ c-)O 5
`2
`
`LU
`CD
`
`/
`
`/
`
`/
`
`/
`
`(
`
`/
`
`/
`
`/
`
`a
`
`r
`
`Ocu
`
`Q_ CM
`
`CM
`O CM
`o
`
`CsJ
`
`m
`
`IPR2016-00047 Exhibit 2003 Page 11
`
`X
`
`

`
`US 2013/0061273 A1
`
`1
`
`Mar. 7, 2013
`
`METHOD AND APPARATUS FOR BROWSING
`USING ALTERNATIVE LINKBASES
`[0001] This application claims the benefit of U.S. Provi ¬
`sional Application No. 60/379,635, filed May 10, 2002, U.S.
`Provisional Application No. 60/408,605, filed Sep. 6, 2002,
`and U.S. Provisional Application No. 60/455,433, filed Mar.
`17, 2003, all of which are incorporated herein by reference.
`
`FIELD OF THE INVENTION
`[0002] The present invention is directed generally to inter¬
`active television and similar interactive hypermedia such as
`from television or Internet sources, and more particularly to
`the provision and useof user interfaces that permit interaction
`using multiple coordinated device sets.
`
`BACKGROUND OF THE INVENTION
`[0003] While convergence of television (TV) and com¬
`puter technology have been a major focus of innovation and
`commercial development since the early 1990s, particularly
`interactive television (ITV), there remains a
`in the area of
`huge gulf in the nature of the user experience of ITV and of
`computer-based media such as the World Wide Web. Conver¬
`gencehas taken hold ininfrastructure technologies, with digi ¬
`tal and computer-based TV (DTV) editing, production, dis ¬
`tribution, transmission, and devices. At heart ITV is a matter
`of hypermedia browsing, the process of browsing linked
`media resources like the Web, differing only on its emphasis
`on video as the central medium.
`[0004] However, there remains a divide relating to the dra ¬
`matic difference in how TV-centric and computer-centric
`media are used, and to the cultural divide between the TV
`production and distribution industry and the computer and
`Web industries that has prevented a convergence in user expe¬
`rience from developing or even being seen as possible and
`desirable. TV usage and directions are focused on its charac ¬
`ter as a lean-back, across-the-room, low resolution, and rela ¬
`tively passive, relaxed experience of couch potatoes viewing
`large, often shared TV screens with simple remote controls.
`PC usage and directions are focused on its character as lean-
`forward, up-close, high resolution, and intensive, highly
`interactive experiences of individuals with PC-styles dis ¬
`plays, keyboards, and pointing devices. Variant device sets
`and applications, such as PDAs, tablets, and video games,
`could be taken as suggestive of the desirability of selecting
`among alternative usage modes and form factors, but only
`very limited aspects of these suggestions have been recog ¬
`nized.
`[0005] The limitations of these radically disparate device
`set form factors have severely limited the appeal of ITV. ITV
`promises to greatly enrich the TV experience by allowing
`interactive features that can range from access to supplemen¬
`tary enhancement material such as background on programs,
`casts and players, sports statistics, polls, chat messaging, and
`interactive advertisements and purchase offers (“ t-com-
`merce”), and all manner of other tangential information, to
`ways to vary the core program content by acting on viewer
`input and choices as to camera angles or even alternative
`plots, as well as providing improved control of the core expe¬
`rience with electronic program guides (EPGs), personal
`video recorders (PVRs) and video on demand (VOD) and
`similar features.
`[0006] The problem is that these interactive features are not
`well served by the TV usage mode and form factor, and their
`
`use interferes with the basic TV experience. Rich interaction
`with a TV is inherently difficult. Presentation of information
`is limited by the poor capabilities of a TV screen for present ¬
`ing text, menus, and navigations controls, and the crude input
`capabilities of a remote control. The rich information and
`navigation functionality available on a Web browser or other
`PC-based user interface (e.g., UI, especially graphical user
`interfaces, GUIs) must be dumbed-down and limited for
`use on a TV, and even use of high-definitionTV (HDTV) may
`people do not like to read or do
`not significantly ease that
`fine work from across-the-room, it is just not comfortable
`ergonomics. Furthermore, the attempt to show interactive
`controls and enhancements on theTV interferes with viewing
`by the person interacting, as well as any other viewers in the
`room. Compounding these issues and slowing recognition of
`better solutions is the dominance of the cable TV industry, its
`struggles in developing and deploying the advanced set-top
`boxes (STBs) needed to offer meaningful ITV services of the
`form it envisions, and its orientation to closed, proprietary
`systems that do not fully exploit or adapt to advances in the
`PC and Internet world.
`[0007] The computer community has attempted to market
`PCs that include a TV tuner to support TV function in a
`PC-centric model, as promoted by the PC-DTV Consortium.
`However, these systems suffer from the converse problem, in
`that their form factors are not suited to the fact that most
`people do not want to watchTV at a PC, with its lean-forward,
`up-close form factor. Furthermore, such devices cannot effec ¬
`tively receive protected cable or satellite programming. And
`here, as with conventional TVs, the use of a single system
`forces technical, economical, and usage constraints on the
`inherently complex, multi-tasking, man-machine behavior
`that is desired in a rich hypermedia browsing experience.
`[0008] There has also been some recognition that PCs pro ¬
`vide a way around the limited installed base of advanced
`STBs, but this is generally perceived only as a limited stop ¬
`gap. So called Enhanced TV or Extended TV or “ teleweb ¬
`bing” has emerged to exploit the fact that tens of millions of
`households have PCs in the same room as their TVs, and can
`surf related content on the Web while watching TV. Some
`broadcasters such as ABC and PBS have exploited this to
`offer Web content synchronized to a TV program; but it is the
`user who must coordinate the use of the PC with the TV, by
`finding the appropriate Web site. In spite of the fact that the
`installed base for suchopen hardware is sometentimes that of
`ITV-capable set-top boxes, the ITV community generally
`views such “ two-box” solutions as an unfortunate and awk¬
`ward stopgap that may be desirably supplanted by advanced
`“ one-box” systems whose wide deployment must be awaited.
`Some major reasons for this lack of acceptance are that this
`simplistic two-box model supports only very limited, pre¬
`defined synchronization of the availability of TV and
`enhancement content that is built into a rigidly fixed two-box
`structure at the content source, and, even more importantly,
`that it completely fails to address any coordination of user
`activity at the two separate boxes.
`[0009] Across all of this, the key elements that are lacking
`are provision of a broadly flexible, powerful, selective, and
`simple user interface paradigm for browsing hypermedia
`across multiple device sets, whether they are integrated or
`not, with related methods for user and/or authoring control of
`such a UI, and provision of an effective method for indepen¬
`dent systems to coordinate browsing activities to enable such
`a user interface to be employed across multiple independent
`
`IPR2016-00047 Exhibit 2003 Page 12
`
`

`
`US 2013/0061273 A1
`
`2
`
`Mar. 7, 2013
`
`systems. Further lacking across all of these aspects is delivery
`of these services in a way that provides the user with a
`smoothly integrated experience in which interactions on the
`multiple systems are coupled or decoupled to the degree
`appropriate to the task of the moment.
`
`SUMMARY OF VARIOUS EMBODIMENTS THE
`INVENTION
`[0010] According to embodiments of the present invention
`there are provided systems and methods for navigating hyper¬
`media using multiple coordinated input/output device sets.
`Embodiments of the inventionallow a user and/or an author to
`control what resources are presented on which device sets
`(whether they are integrated or not), and provide for coordi ¬
`nating browsing activities to enable such a user interface to be
`employed across multiple independent systems. Embodi ¬
`ments of the invention support new and enriched aspects and
`applications of hypermedia browsing and related business
`activities.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`[0011] Further aspects of the instant invention will be more
`readily appreciated upon review of the detailed description of
`the preferred embodiments included below when taken in
`conjunction with the accompanying drawings, of which:
`[0012] FIG. 1 is a block diagram of an exemplary assem¬
`blage of user systems, networks, and remote services for
`implementing certain embodiments of the present invention.
`[0013] FIGS. 2a and 2b are a set of block diagrams of
`exemplary groupings of device sets and systems in theassem¬
`blage of FIG. 1.
`[0014] FIG. 3 is a schematic diagram of a number of exem¬
`plary user interface display layouts according to certain
`embodiments of the present invention.
`[0015] FIG. 4 is a schematic diagram of an exemplary
`structure for state information relating to systems within the
`assemblage of FIG. 1, relating to the coordination of a mul¬
`timachine user interface according to certain embodiments of
`the present invention.
`[0016] FIG. 5 is a schematic diagram of an exemplary pro ¬
`cess, performed by the systems of FIG. 1, for transferring
`state data according to certain embodiments of the present
`invention.
`[0017] FIG. 6 is a flow chart of an exemplary process,
`performed by the systems of FIG.1, fortransferring state data
`according to certain embodiments of the present invention.
`[0018] FIGS. 7a, lb, and 7c are a set of block diagrams of
`exemplary alternative communication configurations in the
`assemblage of FIG.1.
`[0019] FIG. 8 is a block diagram of details of an exemplary
`portal facilitating session coordination linkage in the assem¬
`blage of FIG. 1.
`[0020] FIG. 9 is a schematic diagram portraying exemplary
`further detail of a user interface for a cross-program portal.
`[0021] FIG. 10 is a schematic diagram of an exemplary
`LiberatedSTB configuration.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`Overview
`[0022] The present invention may be described, in various
`embodiments, as a system and method for navigating hyper¬
`media using multiple coordinated input/output device sets. It
`
`provides a broadly flexible, powerful, selective, and simple
`user interface paradigm for browsing that allows the user
`(and/or an author) to control what resources are presented on
`which device sets (whether they are integrated or not), and
`provides an effective method for coordinating browsing
`activities to enable such a user interface to be employed
`across multiple independent systems.
`is, in the spirit of human-centered
`[0023] One aspect
`design, to anticipate and be responsive to the user s desires
`(and the author’s suggestions) as to what resources to present
`where, in order to make the best possible use of the hardware
`resources at a user’s disposal. Homes, offices, and other per¬
`sonal environments of the future will have a rich array of
`computer-based input output devices of various kinds, some
`general purpose, and some more or less dedicated to specific
`uses. The desire is to minimize constraints on what system
`resources can be used for a given task, to enable the most
`powerful browsing experience possible. Browsing of hyper¬
`media, such as in the case of ITV is a task in which the use of
`multiple devices might be valuable because it may be
`expected to be a dominant activity, if supported effectively,
`and because of the disparity of UI issues between watching
`extended video segments and doing intensive interactions
`(such as with Web media) that may be more or less closely
`coupled with such video segments.
`[0024] Prior work has generally not recognized that it is
`inherent in rich ITV and similarforms of video-centric hyper¬
`media browsing to be best served as two-box, multitasking
`experiences, at least much of the time, and the problem is not
`to squeeze it into one box (and fight over which box’s func ¬
`tionality and form factor is better), but to enable effective
`coordination of both boxes. While theTV vendors and the PC
`vendors might fervently wish to offer a single system that
`meets the needs of ITV users, that is not an effective solution.
`If one assumes that an ideal level of coordination among
`device sets can be enabled and explores usage scenarios, it
`can then be seen that different modes of viewing are best
`served by different device set form factors. These modes are
`not fixed for the duration of a session or task, but can blend,
`overlap, and vary as the flow of a set of linked tasks changes.
`What begins as a TV-centric browsing (or pure viewing)
`experience may shift to casual use of a PC for light interaction
`(such as looking at menus and options or doing a quick
`lookup) to intensive PC-centric activity (and then back
`again). The user may shift focus from the TV to both, to
`primarily the PC for a time, then become involved in the TV
`again. Conversely, an user at a PC may shift to immersion in
`a TV program or movie, then return to intensive use of the PC.
`While some broad usage patterns tend to favor video on the
`lean-back TV device set and interactivity on the lean-forward
`PC-type device set, other issues may relateto incidental view ¬
`ing of video from a PC centric phase of activity, and casual
`interactions with enhancements in a TV-centric experience,
`as well as a complex mix of secondary issues, such as quality-
`of-service factors, whether an alternative device set is at hand
`and ready for use, other activities, presence of other people,
`location/setting, mood, and the like.
`[0025] The point in a session at which a user may wish to
`shift device sets may depend not only on the immediate task,
`but the user’s expectation of where that task is leading, so an
`intensive task soon to end may not warrant a shift from TV to
`PC, but a less intensive task leading to deeper interaction may
`warrant an early shift. Varying form factors of different TV
`devices and of the range of PCs, PDAs, tablets, and Internet
`
`IPR2016-00047 Exhibit 2003 Page 13
`
`

`
`US 2013/0061273 A1
`
`3
`
`Mar. 7, 2013
`
`appliances may also affect what tasks a user wants to do on
`what device, with what UI. At the same time, to avoid bur¬
`dening the user with the complications of too much flexibility
`and too many choices, it may be desirable that both the user
`and the content author be able to pre-set affinities, prefer¬
`ences, and recommendations, relating to task types, content
`types, and device availabilities, that could automatically
`place elements on the device set or device set group that is
`presumably best suited to the apparent context, while leaving
`the user with the ability to recognize that expected targeting
`(based on conventions and/or unobtrusive cues) and to accept
`it with no further action, or override it if desired.
`[0026] Providing the desired flexibility can be viewed in
`terms of three interrelated issues, one of structuring an effec ¬
`tive and flexible multimachine user interface (MMUI) for
`browsing by a user, one of providing methods (such as
`markup) for the resource creator/author/producer to aid in
`exploiting that MMUI, and one of implementing such an
`interface on a wide range of hardware and software, including
`systems for which such usage may not be a primary mission
`(including both new systems and legacy systems).
`[0027] A general approach to a MMUI for browsing that
`provides both user control and authoring support may advan¬
`tageously build on the concept of targets for presentation of
`linked resources already present in hypermedia formats such
`as HTML (and XLink). In HTML, the link target attribute can
`be used to specify which of multiple frames a linked resource
`is to be presented in, with options that include the current
`frame, another existing frame, or a new frame. Coded speci¬
`fications within thelink aretypically set by authors/producers
`of content, and controls in the browser allow the user to
`override and alter these settings, such as (with MICROSOLT
`Internet Explorer, MSIE) by using a shift-click combination
`to indicate that a link should be opened in a new window.
`Extending this to an MMUI can be done by expanding the
`coding of target attributes and by adding new browser control
`options, such as control-click, to target a window on an alter¬
`nate device set.Additional control can be achieved by extend ¬
`ing the richer drop-down control that is invoked in MSIE by
`right-clicking on a link. That drop-down list can be extended
`to list windows on alternate device sets. This provides a very
`flexible, general, and simple way to shift activity from one
`device set to another. Similar controls can be provided on
`simpler devices, such as for example, with a TV remote
`control, instead of select to activate a link to an enhancement
`overlay on the TV, a combination such as exit-select could be
`used to activate that link to an associated PC, or a new control
`button could be provided. As with current browsers, varia ¬
`tions on such controls can also be defined to open the current
`resource at a second location (cloning).
`[0028] To implement such an interface on multiple inde¬
`pendent device sets, the ending system must be given infor¬
`mation to inform it when a link is to be activated, to what
`resource, with what browser attributes, and with what context
`information. A basic method is to transfer from the starting
`system to the ending system a link activation message that
`that includes a state record and contains relevant link arc
`information. The state record contains essential information
`onthe stateof the browser and related activities onthe starting
`system that can be used at the ending system to configure its
`browser and related context accordingly. A state exporter/
`importer/tracker component may be provided as an addition
`
`to a standard browser to provide these functions (with
`exporter/importer function being sufficient for simple appli ¬
`cations).
`In simple embodiments, export from the starting
`[0029]
`system and import at the ending system need be done only
`once per transfer of locus. In certain embodiments, hill event
`synchronization can be maintained, when desired, by the state
`tracker to provide ongoing collaborative functionality, as
`well. This is useful in the case of multiple users, and also can
`be useful for a single user that desires the ability to use both
`device sets in a hilly replicated mode. However an advantage
`of the proposed method over conventional collaboration and
`synchronization systems, is that such ongoing event synchro ¬
`nization is not needed for basic MMUI browsing by a single
`user, and the complications and overhead of continually log ¬
`ging, exporting and importing all events that may alter state
`can be avoided. Instead, state information need be assembled
`for transfer only when a transfer is actually invoked, and only
`at the necessary granularity. This simple, occasional, coarse¬
`grained transfer is readily added to any browser of existing
`architecture, unlike more fine-grained full synchronization
`approaches, which require either excessive tracking activity,
`display replication approaches, or rearchitecting of browsing
`to use model-view-controller architectures, such as in event
`replication approaches.
`[0030] Another key benefit of this method is that it is
`readily applied to heterogeneous systems with only simple
`addition of an exporter/importer and some new UI functions
`to each system s own native browser. This exploits the fact
`that the underlying resources being browsed can be common
`to all systems, and that at a high level, browsing state is
`relatively independent of system architecture. Thus the
`method is readily applied to both TV and PC-based systems,
`and could be added to existing or new systems by manufac ¬
`turers, integrators, distributors, service providers, or by end
`users themselves. The proposed methods are well suited to
`standardization, which could facilitate the inherent capability
`of themethods described hereto allow any suitably functional
`device sets and systems to be used together in the desired
`coordinated fashion, regardless of its internal software and
`hardware architecture, vendor, or provisioning. Use of XML,
`RDL, and related standards is suggested to facilitate this.
`These features for ad hoc provisioning and use of devices
`acquired for other purposes removes a major hurdle to the
`introduction of MMUIs for ITV and other hypermedia brows ¬
`ing applications.Thus, for example, a household need not buy
`a lean forward device for ITV, but can simply use an existing
`PC, PDA, tablet, or the like.
`[0031] As a further perspective on the range of ways to use
`a MMUI for interactive TV and similar hypermedia browsing
`as described herein it may be helpful, perhaps with regard to
`varied levels of multitasking and (correspondingly) of how
`closely enhancement resources relate to the viewing of a
`primary program , to consider the term interactivity. The
`term “ interactive TV” might tend to suggest that a viewer
`interacts with a TV device and/or with TV content. Such a
`view may be appropriate to many kinds of ITV interaction.
`However, in considering the embodiments of MMUI brows ¬
`ing described herein, it is noted that many cases of what might
`be broadly described in terms of “ interactive TV” could
`involve interactions that need not directly involve the TV
`device, or even theactual program contentthat is “ ontheTV” ,
`but that, for instance, involve other content perhaps more or
`less closely related to the program content that is on the TV.
`
`IPR2016-00047 Exhibit 2003 Page 14
`
`

`
`US 2013/0061273 A1
`
`4
`
`Mar. 7, 2013
`
`[0032] From such a standpoint, the term coactivity might
`be considered as useful to emphasize the possible distinction
`between what is interacted with and what is on TV. Thus, for
`example, in the case of a loosely coupled interactive sub-task
`on a PC that relates to a program on the TV, the interactivity
`that takes place as part of that sub-task might be described as
`“ coactivity” .
`[0033] The concept of coactivity could be useful, for
`instance, in clarifying certain motivations for using a MMUI.
`To the extent that one might think of a task as “ interacting
`with the TV”
`the idea of using another device set (for
`example, a PC) might seem odd and unnatural to the task.
`Flowever, by recognizing that many interactive tasks actually
`involve coactivity with content that might not be “ on theTV” ,
`but that relates to what is on the TV, the use of a separate
`device set might be more readily recognized as possibly being
`natural and appropriate. Accordingly, “ two-box” embodi ¬
`ments of the present invention could be seen as potentially
`well suited to the essential nature of ITV and similar hyper¬
`media browsing, and not as a “ stopgap” or “ work-around”
`embodiments. Development of this new paradigm for man-
`machine-media interaction affords enriched capabilities and
`supports new and enriched applications.
`[0034] As used herein, the term “ hypermedia” is meant to
`refer to any kind of media that may have the effect of a
`non-linear structure of associated elements represented as a
`network of information-containing nodes interconnected by
`relational links. Flypermedia is meant to include “ hypertext” ,
`and the two may at times be used synonymously in the broad
`sense, but where stated or otherwise clear in context, “ hyper¬
`text” can refer particularly to text content, and “ hypermedia”
`to extend that to content that includes other formats such as
`graphics, video, and sound. The terminology used herein is
`meant of begenerally consistent with that used in World Wide
`Web Consortium (W3C) recommendations.
`[0035] The associations of elements may be specified as
`“ hyperlinks” or “ links,” such as described by the XLink
`(XML Linking Language), SMIL (Synchronized Multimedia
`Integration Language), F1TML, XF1TML, and similar W3C
`recommendations. Links define an association between a
`“ starting resource,” the source from which link traversal is
`begun, and an “ ending resource, the destination, collectively
`referred to as “ participating resources.” A “ resource” is used
`to refer to any addressable unit of information or service and
`may at times refer to a resource portion rather than a whole
`resource, and a content resource” to refer to any resource
`suited to presentation to a user. In the context of hypermedia,
`“ node” may be used synonymously with resource. “ Naviga ¬
`tion” is meant to refer to the process of following or “ travers ¬
`ing” links. Unless specifically indicated as “ link navigation”
`or otherwise clear in context, navigation also is meant to
`include the control of presentation within a resource, such as
`scrolling, panning, and zooming, using VCR-like controls to
`play a continuous media resource, and the like. Addresses for
`Internet resources are typically in the form of Universal
`Resource Locators (URLs) or Universal Resource Names
`(URNs) or other Universal Resource Identifiers (URIs), but
`may be based on any other suitable addressing mechanism.
`Hypermedia resources may contain content (also referred to
`as mediadata) and metadata (including hyperlinks), aspects
`of a resource may be declarative (such as markup) or proce¬
`dural (such as embedded logic or program codeelements) and
`may include embedded resources.
`
`[0036] Links may have information about how to traverse a
`pair of resources, including direction and application behav ¬
`ior information, called an “ arc,” and such information may
`include link “ elements” having “ attributes” that take on “ val ¬
`ues.” Behavior attributes include “ show” to specify how to
`handle the current state of the presentation at the time the link
`is activated, “ external” to specify whether the link is to be
`opened in the current application, or an external application,
`such as one suited to a special media type, “ activate” or
`“ actuate” to s