as) United States
`a2) Patent Application Publication 10) Pub. No.: US 2008/0276279 Al
`
`(43) Pub. Date: Nov.6, 2008
`Gossweiler et al.
`
`US 20080276279A1
`
`(54)
`
`INTERACTIVE MEDIA DISPLAY ACROSS
`DEVICES
`
`(76)
`
`Inventors:
`
`Richard C. Gossweiler, Sunnyvale,
`CA (US); RohanSeth, Palo Alto,
`CA (US); Shumeet Baluja,
`Leesburg, VA (US)
`
`Correspondence Address:
`FISH & RICHARDSON P.C.
`PO BOX 1022
`MINNEAPOLIS, MN 55440-1022 (US)
`
`(21) Appl. No.:
`
`12/059,245
`
`(22)
`
`Filed:
`
`Mar.31, 2008
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/909,384, filed on Mar.
`30, 2007.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`GO6F 3/00
`
`(2006.01)
`
`(52) US. CMe cccseccsssssssssssssesseesssesssessssssssssnnneseeseees 725/46
`
`(57)
`
`ABSTRACT
`
`A computer-implemented method includes identifying a
`computer-based portable program module, automatically
`altering code in the portable program module to permit dis-
`play of the module on a television-based display so that the
`displayed module has a substantially similar appearance on
`the television-based display as on a computer display, and
`providing the altered code for execution on a processor con-
`nected to a television-based display.
`
`806 4:30p Thursday, March 18
`
`
`
`
`
`‘Content
`New From Friends (24):
`
`802
`
`804
`
`
`
`SAMSUNG 1038
`
`SAMSUNG 1038
`
`1
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 1 of 19
`
`US 2008/0276279 A1
`
`aorot
`
`
`
`FLA
`
`VlOld
`
`
`
`c08
`
`v08
`
`908
`
`2
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 2 of 19
`
`US 2008/0276279 Al
`
`Pe
`
`
`
`4:30pThursday,March15
`Content
`
`
`
`NewFromFriends(24):
`
`NewFromYou(12):
`
`<x
`«x
`NK
`3 *
`= *«a
`o 3
`A OS
`GD
`> €
`=< @&
`woSe
`
`©
`N &
`Qa!
`& 5o>
`
`So
`
`Recommended:
`
`FIG.1B
`
`3
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 3 of 19
`
`US 2008/0276279 A1
`
`
`SLLYor--‘OSPIASIU}Ul4N}s
`
`
`
`
`AzeJo9y}JNOyOsYDjueyoy
`
`-~Gqay
`SLyoy‘Aepsunyydog:yZUCUHLOG“SAUBAY]
`
`
`‘uouswoUsUdeqnNO,|eUuIBLOay}0}janbaspajyedionue-Ajjoyay
`
`
`
`
`
`
`
`
`
`
`1UBISueMJe}sjanbesJaqesyybi|||OM)UeLUYJOPJoqoImUeAL:sbe|Woo
`
`(sBulel67L'S)«yxmHJOOSUBWIOG-95:60
`WOUsYUgayU}ZpALMmm||79EL
`
`
`Ob“Ola
`
`vib
`
`4
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 4 of 19
`
`US 2008/0276279 Al
`
`
`
`
`
`
`a
`~
`Q
`Lo
`
`©N
`
`>[
`
`43]
`a
`£
`—l
`oO
`
`£—
`
`
`
`5
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 5 of 19
`
`US 2008/0276279 A1
`
`
`
`GLYEW‘AepsunuLdoey
`[wpa}(evoJ(ues)ee)||
`
`()FACSPUITBULL
`
`
`
`obeskep&SUI|UOISB]_Woo-s9R|dawosMgen
`
`
`
`
`
`9002‘JOqUeAONZzSoulspUuaLy
`
`CEL
`
`
`
`
`
`
`9EL
`
`SEL
`
`6
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 6 of 19
`
`US 2008/0276279 A1
`
`crl
`
`dbOld
`
`
`oos|ouel4uesdroesoaieae&))
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PUI]BULLie
`
`
`
`
`
`
`
`wa
`
`SoM
`
`
`
`7
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 7 of 19
`
`US 2008/0276279 A1
`
`Fare|SEESSee|Fae
`
`
`RRRKAYRAKEKAKAH
`RRRAOLRREESHAPERSEEAARE
`PAPASELLERAPES
`
`HRRRRREESEBERLEBREE
`RAPPERA?
`RRAKE(By)
`KARAKARRRR
`RRA
`
`FERRERERAPEC...._]
`
`eeeHRRRLRPREN
`FRRLtt
`
`
`
`
`
`
`
`
`
`
`PEELEPERLEEE
`
`RRRRARBREERS
`
`
`
`
`
`P|RE
`
`RHREESMEREES
`
`
`
`JLo=eae5-6
`i
`FRRRRPRRARRESLA
`Oe
` ®aeqoo,
`
`doigAresodwa
`39SuBUOY
`Z0Z\ooz
`
`GmNO,
`
`JlasunoAseapeoig
`
`FEB
`
`FERRE
`
`RE
`
`
`
`
`|
`
`602
`
`802
`
`Ole
`
`Lbé
`
`cle
`
`VeOla
`
`FEEEE
`
`ORERAE
`
`RBA
`
`ERKHE
`
`RBAK
`
`SRE
`
`
`
`
`
`
`PRK
`
`
`
`RRPLAREPPAEEE
`
`
`
`HRPAAAEPPAPALS
`
`HHERAARELAAPLEEY
`
`ERR
`
`
`
`LRRLELLALEEREEE
`
`
`
`BERBERARERR
`
`PEELERE
`
`SPEEAEE
`
`
`
`wee]Jawes
`
`a
`
`8
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 8 of 19
`
`US 2008/0276279 A1
`
`TooBee
`
`Temporary Drop
`
`202
`
`
`
`
`210
`
`211
`
`212
`
`Gamer Team
`
`FIG. 2B
`
`9
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 9 of 19
`
`US 2008/0276279 Al
`
` &
`
`Temporary Drop
`
`
`
`
`
`
`a
`
`Oh
`
`Rich Gossweiler (rcg@google.com)
`5 friends are online
`
`214
`
`212
`
`12/17 Miracle of Birth: You Tube the Goat- 04:39 - Rohan
`
`
`
`
`
`12/17 Condi Rice Raps
`- 02:04 - Rohan
`
`
`12/17 The Luke Johnson Phone Experi...
`- 01:12 - Bob
`
`12/16 Running Amok
`- 03:45 - Mary
`
`1-4 of 312
`
`Older >>
`
`
`
`
`Gamer Team
`
`Home TV Box
`
`
`FIG, 2C
`
`10
`
`10
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 10 of 19
`
`US 2008/0276279 A1
`
`VeOld90¢
`
`
`
`xoqdo}38SAL
`
` nl BeReRee
` aaeener|_|eee
`
`
`
`
`
`EEIJESSEE—OA=
`ARNEwees(OOEEEOE
`
`
`BeppeAEE
`
`ELENSSLERHEEEE
`
`C=BjeusuwesHleeveeeeeeee[al
`
`were[Lee]eeeJLeres
` SUILLUEL|]0@@6.8)
`
`
`aseeoopeee.TOF
` SERRETEP
`ateeeieenett
`OR
`EEOCEEY
`Penennntrans
`
`
`
`
`
`OLE
`
`\Cooe
`
`
`
`uunpdca]=980pad[h—panaeRy
`
`DsitE)
`
` 0D[e
`
`
`
`
`
`
`
`
`SIBAIOS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SEGREUDE
`
`
`
`wee]Jawes
`
`
`
`
`
`cO€
`
`11
`
`
`
`
`
`
`
`
`
`
`
`11
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 11 of 19
`
`US 2008/0276279 A1
`
`juaqUogAY
`
`Ore
`
`yoress
`
`synsey
`
`Jel00s
`
`SOeBLa}U|
`
`29PE
`
`
`
`UONEUUOJUY
`
`PPE
`
`yUEY
`
`ainpayos
`
`
`
`eyeq(Od3)
`
`GE
`
`OPre
`
`
`
`GOKE
`
`
`
`SOllqOSPlA
`
`deSls
`
`Od4
`
`
`
`JeBeueyjusju0y
`
`Cee
`
`bee
`
`Joyeseuesy
`
`Jainpsyos
`
`weibold9ze
`
`12
`
`12
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 12 of 19
`
`US 2008/0276279 A1
`
`356
`
`JBSMOIG
`
`
`
`JseBbeuewpueti4euibuz}dwageaer
`
`O2€Je1@pueyTWLH
`
`
`
`
`
`
`JBIayoeyje007
`
`
`
`JOjUODs]owSyY
`
`soBbeuey
`
`Jeyoyes
`
`
`
`Jeyeuel)Juaju0D
`
`
`
`JBAC|qCOPIA
`
`Oe“Old
`
`374
`
`354
`
`JaBbeueywsepAL
`
`cZe
`
`JebeuriIn
`
`SABLBAQ
`
`s]ugAg
`
`
`
`
`
`Jabeueyseajoys/Jabeueyjays
`
`352
`
`mos
`
`13
`
`13
`
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 13 of 19
`
`US 2008/0276279 Al
`
`400 ly
`
`PC Client
`402
`PC Client
`
`Ul Manager||Friend Manager Java Serviets
`Friend List
`Person Table
`Friend Deiails
`Friend Table
`Group Table
`Link Table
`
`408
`
`Television
`
`
`
`TV Client
`
`Fetcher
`Local Cache Manager
`
`UI Manager
`Browser
`Events
`Video Player (Flash)
`Overlays
`Javascript Engine
`4
`18
`HTML Renderer 420||Shelf Manager
`Watchdog
`Shelves Manager
`TV PPM Manager
`426
`
`419
`Remote Control
`Mgr & API 428
`
`06
`
`14
`
`14
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 14 of 19
`
`US 2008/0276279 Al
`
`‘zoom out / undo
`
`- zoom in / select
`
`FIG. 5
`
`600 tL
` 604
`
`automatically generated bumpers
`
`skip points jo
`
`FIG. 6
`
`15
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 15 of 19
`
`US 2008/0276279 A1
`
`
`
`Hesuno,Jseopeolg
`
`
`
`Wwe9P:2
`
`oosiouel4UBS
`
`4.6[ax]
`
`nun,
`
`/Old
`
`
`
`16
`
`16
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 16 of 19
`
`US 2008/0276279 A1
`
`“ne
`
`
`
`“nm
`
`Receive Request For
`Display of Friends List
`$22
`
`Identify Associated Users
`and Devices
`
`824
`
`Serve Data For Friends
`List
`
`G26
`
`
`
`Receive Selection
`Associating a Media
`Program With a Member
`or Members of theList
`
`828
`
`Verify The Friend and the
`Media Program
`
`830
`
`Cause an Indicator of The
`Media Program To Be
`Sent To Accounts
`Associated With Each
`Selected Friend
`
`832
`
`Identify Desktop Portable
`Program Module (PPM)
`802
`
`Parse DOMfor Desktop
`PPM
`
`804
`
`Identify Relevant
`Formatting Code and
`Translate
`306
`
`Identify Relevant Content
`and Reformat
`
`808
`
`Package Reformatted
`Code and Content
`810
`
`Provide Code and
`Content for Display on
`Television
`B12
`
`Receive Confirmation
`From TV User 814
`
`FIG. 8A
`
`FIG. 8B
`
`17
`
`17
`
`

`

`Nov. 6, 2008 Sheet 17 of 19
`
`US 2008/0276279 Al
`
`Patent Application Publication
`
`834(
`
`846
`
`Receive UserRequest
`
`For Display of Friends
`List
`836
`
`Transmit Request With
`User ID To Media Server
`838
`
`Receive Information On
`Friends And Their
`Related Media g4o
`
`Display Friends in User-
`Specified Format
`
`842
`
`Receive User Selection
`To Play Media Programs,
`And Play Programs With
`Inserted Bumpers
`
`844
`
`Receive User Selection
`To Share Media, And
`Transmit Information to
`Media Server
`
`FIG. 8C
`
`18
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 18 of 19
`
`US 2008/0276279 A1
`
`
`
`
`Request Media
`Connection Interface
`
`
`850
` Serve Media
`
`
`Connection Interface
` Display Acquaintances
`
`
`852
`And Devices
`
`
`
`854
`
`
`
`Receive Media Sharing
`
`
`Parse Media Location
`Commandand Identify
`
`
`
`And Recipients From
`Media And Recipient(s)
`
`Request
`
`
` 858
`
`856
`
`
`
`Verily Recipients And
`Media — Get Media
`
`
`
`
`
`
`
`
`
`
`Display Updated Page
`864
`
`
`
`
`Receive And Store
`
`
`Media
`872
`
`
`Play Media
`874
`
`
`Report Results
`
`860
`
`862
`
`
`
`
`876
`
`Schedule Delivery
`866
`
`Update EPG Data
`868
`
`Deliver Media
`
`870
`
`Select And Deliver Ads
`And OtherInformation
`
`TV Client
`
`Computer Client
`
`Server(s)
`
`FIG. 8D
`
`19
`
`19
`
`

`

`Patent Application Publication
`
`Nov. 6, 2008 Sheet 19 of 19
`
`US 2008/0276279 Al
`
`920
`
`Ved
`
`|
`
`20
`
`20
`
`

`

`US 2008/0276279 Al
`
`Nov. 6, 2008
`
`INTERACTIVE MEDIA DISPLAY ACROSS
`DEVICES
`
`request from a remote set-top box. In addition, altering the
`code in the portable program module may comprise generat-
`ing code responsive to navigation commandsfromatelevi-
`CROSS-REFERENCE TO RELATED
`sion remote control keypad. Generating code responsive to
`APPLICATIONS
`navigation commandscanalso include generating four-direc-
`tion code for navigating between input elements generated by
`the portable program module. The method may also comprise
`transmitting data in response to a request from an executed
`portable program module, and can also comprise encoding
`the data as XMLdata.
`
`[0001] This application claimspriority to U.S. Application
`Ser. No. 60/909,384, filed on Mar. 30, 2007, the contents of
`which are hereby incorporated by reference inits entirety.
`
`TECHNICAL FIELD
`
`[0002] This document discusses systems and methods for
`displaying and sharing media content and other objects for
`television viewing.
`
`BACKGROUND
`
`Inother aspects, the method also includes automati-
`[0007]
`cally altering the code in the portable program module, and
`requesting confirmation from a user for one or morealter-
`ations. The method can also include displaying an executed
`module on a computer monitoras part of a simulatedtelevi-
`sion display.
`[0008]
`In another implementation, a method of displaying
`[0003] As the PC andthetelevision continue to converge,
`content on a video monitor is disclosed, and comprises
`both in delivery models and interaction models, we face a
`receiving, from a viewer of a television, a command for
`numberof new challenges and opportunities. These devices,
`display of computer content on the television, generating a
`communicating through a network in a loosely coupled man-
`display from a portable program module with a display of
`ner, will serve different and overlapping roles in presenting
`media content on thetelevision, requesting over the internet,
`entertainment options. Commonly, the PC is used by the
`by execution ofthe portable program module,data for display
`individual, used for many non-entertainmenttasks and often
`with the portable program module. The command may be
`located away from the main entertainment room. In contrast,
`received from a television remote control. The portable pro-
`the television 1s in the centerofthe entertainmentspace, often
`gram module can comprise a gadget. Also, the portable pro-
`used by groups of people in various social contexts, has an
`gram module can comprise mark-up code, including JavaS-
`impoverished input device (the remote) and has a lowerreso-
`cript referencing code. The portable program module can also
`lution display. It lacks computational power, uses a different
`model for content distribution, and a different model for
`comprise mark-up code for navigation of input controls on a
`displayed module using four-direction navigation.
`media presentation (channel-based).
`[0004] Today we are seeing the beginnings of a federated
`[0009]
`Inother aspects, the method further comprises posi-
`model, where the PC is used to store content and the TV (or
`tioning the display from the portable program module in a
`set top box) uses the local internet system to access and
`pre-selected corner of a television screen. The method may
`present the content. As computers become less expensive,
`also comprise generating a plurality of displays from one or
`someof this will movedirectly onto the TV-small hard drives
`more portable program modules in an array, and the array
`and internet access will make TVs capable of getting at web
`may bealinear array. The linear array may be generated, for
`content without requiring a PC. Nonetheless, some of the
`example, along a side edge of the television screen. The
`interaction is likely to remain on the PC. You may receive
`method can also include changing the display of media con-
`recommendations for video, gather content links and share
`tent from a 16:9 aspectratio to a 4:3 aspect ratio, such as when
`links while using a PC. The TV and the PC will work in a
`portable program modules are displayed along one edge of
`loosely coupled fashion to support the Collect-Relate-Create-
`the display.
`Donate (CRCD) model.
`[0010]
`In another implementation, a method of sharing
`media content is disclosed. The method includes receiving,
`from a viewer of a television, a command for display of a
`friend list, displaying indicators for a plurality of friends
`associated with a user, and receiving a selection of one or
`more friends from the plurality of friends, and causing an
`indicator of a media program playing on thetelevision to be
`sent to accounts associated with the one or morefriends. The
`indicatorsfora plurality offriends may each comprise a name
`of a friend and an image associated with the friend. The
`method may further comprise contacting a social networking
`site to obtain information identifying theplurality of friends.
`Also, the indicator of a media program comprises a URL
`pointing to the a file of the program.
`[0011]
`In yet another implementation, a method of obtain-
`ing media content is disclosed. The method includes receiv-
`ing, from a viewerofa television, a commandfor display of
`recommended media content, identifying one or more friends
`associated with a user of a system anddisplaying representa-
`tions of content recommendedby the one or morefriends, and
`causing media content recommended by the one or more
`friends to be displayed on the television. The method may
`further comprise displaying indicators for a plurality of
`
`SUMMARY
`
`[0005] This documentdescribes various features and tech-
`niquesfor sharing digital content and for providing and modi-
`fying portable program modules such as widgets or gadgets.
`In one implementation, a computer-implemented method
`comprises identifying a computer-based portable program
`module, altering code in the portable program module to
`permit display of the module on a television-based display so
`that the displayed module has a substantially similar appear-
`ance on the television-based display as on a computerdisplay,
`and providing the altered code for execution on a processor
`connected to a television-based display. The computer-based
`portable program module may comprise a gadget, and the
`code in the portable program module may comprise mark-up
`code. Altering the code can comprise altering style sheet
`definitions for the module. The method may also include
`altering graphical elements referenced by the portable pro-
`gram module to be dimensionedfor the television-based dis-
`play.
`In some aspects, providing the altered code for
`[0006]
`execution comprises transmitting the code in response to a
`
`21
`
`21
`
`

`

`US 2008/0276279 Al
`
`Nov. 6, 2008
`
`friends associated with a user andreceiving a selection of one
`or more friends from the plurality of friends, and displaying
`representations of content recommendedonlybytheselected
`friends. The representations of content recommendedby the
`one or more friends can also be displayed in one or more
`shelves.
`
`In someaspects, the representations of content rec-
`[0012]
`ommendedby one or more friends are displayed in a single
`shelf, and other recommendation are displayed in one or more
`other shelves. Also, causing the media content to be displayed
`can include causing bumpercontent to be displayed between
`adjacent pieces ofmedia content, and the bumpercontent can
`comprise previewsof following media files. The method may
`further comprise receiving a skip input from a user while
`bumpercontent is being displayed, and in response, causing a
`mediafile associated with the bumper content to be skipped.
`Also, the bumper content may comprise advertisements. In
`other aspects, causing media content to be displayed can
`comprise accessing streaming media over the internet and
`providing the streaming media for display on the television.
`The details of one or more embodiments are set forth in the
`
`accompanying drawings and the description below. Other
`features, objects, and advantages will be apparent from the
`description and drawings, and from the claims.
`
`DESCRIPTION OF DRAWINGS
`
`FIG. 1A shows an example television display of
`[0013]
`shelves of media content organized by the source that recom-
`mendedthe content.
`
`FIG. 1B showsa screen shot of shelves 0 recom-
`[0014]
`mended content for a view, with oneparticular file selected.
`[0015] FIG.1C is a screen shot of an example of a bumper
`showing program details.
`[0016] FIG.1D isascreen shotofa friends gallery showing
`images associated with a plurality of friends.
`[0017]
`FIG. 1E is a screen shot of a details page for a
`selected friend.
`[0018]
`FIG. 1F is a screen shot ofan interface for selecting
`a friend orfriends with which to share a media program while
`watching the program.
`[0019]
`FIG. 2A showsa graphical flow diagram 200 of the
`gather of program links in a social media sharing application.
`[0020]
`FIG. 2B. showsa video gathering application that
`displays friends to which a user maydrag representations of
`media files for association of the files with the selected
`friends.
`
`FIG. 2C showsdetail information for a friend in the
`[0021]
`application of FIG. 2B.
`[0022] FIG.3Aisaschematic diagram showing the recom-
`mendation on a personal computer of a video file, and the
`subsequent delivery ofthe file to a television of the recom-
`menderorofa friend of the recommender.
`[0023]
`FIG. 3B is a schematic diagram of a system for
`sharing audio-visual program content.
`[0024]
`FIG. 3C isa block diagram of a media display man-
`agement system.
`[0025]
`FIG. 4 shows a schematic diagram of a system for
`sharing and displaying media content.
`[0026]
`FIG. 5 shows an example of four-direction naviga-
`tion controls for a television remote control.
`
`FIG. 6 graphically displays organization of media
`[0027]
`programsor shows, and intervening bumpers.
`[0028]
`FIG. 7 is a screen shot showing a TV widget super-
`impose on a videodisplay.
`
`FIG. 8A isa flow chart of a process for converting a
`[0029]
`portable program module, such as a gadget, from a computer
`format to a television format.
`[0030]
`FIG. 8B is a flow chart of a process for sharing
`audio-visual content among friends and/or devices.
`[0031]
`FIG. 8C is a flow chart of a process for sharing
`audio-visual content amongfriends.
`[0032]
`FIG. 8D is a swim lane diagram of a process for
`sharing audio-visual programs among devices.
`[0033]
`FIG. 9 shows an example of a generic computer
`device and a generic mobile computer device.
`[0034] Like reference symbols in the various drawings
`indicate like elements.
`
`DETAILED DESCRIPTION
`
`[0035] This document addresses, amongother things, three
`problemsrelated to the convergence and contentions between
`the PC and the TV. Thefirst problem relates to finding good
`content from the enormous amount of content that has sud-
`denly becomeavailable online. The second problem relates to
`the delivery of online content to the entertainment environ-
`ment, and the third problem relates to managing this content
`(viewing it, sharing it, and opening doors for other con-tent/
`interactions while watching television).
`[0036]
`Problem—Finding Good Content: YOUTUBEand
`other such services are filled with a vast and rapidly growing
`library of user-generated, commercial-generated, informal,
`and professional-level video content. The content not only
`spans a broad range of topics, but also a broad range of
`aesthetic, visual, and semantic quality. This large, ever-
`changing and varying set of topics makes finding “good”
`videos a challenge. The “what’s a good video about” discov-
`ery problem is further exacerbated by the difficulty people
`have in expressing what is interesting to them.
`[0037]
`To address this problem,researchers are working on
`systemsto automatically extract features and contenttags that
`can be used to aid users in searching through video content.
`Outside of contentanalysis, social networksare proving to be
`a powerful source of information for sifting through the vast
`amount of content. Not only are millions ofpeople a powerful
`parallel filtering system, but they can provide an invaluable
`source of tagging meta-data, clustering, recommendations,
`and popularity rankings.
`[0038] One of the common mechanisms through which
`users find content
`is through email. With YOUTUBE’s
`immense popularity and volumeof content, people typically
`share content by emailing each otherlinks. This points to the
`need to support easy sharing on the PC, and allows us to
`examine whether email is the best option. Emailis familiar,
`general, great for transport and readily available. But, for a
`task-specific goal of sharing links with a small set of people,
`email may be more complicated than necessary. Email
`requires several steps to compose, target, and deploy the
`links, as well as several steps and separate tools to extract the
`link, resolve it to video, and view it. Because email is used for
`a variety of information sharing tasks, people must sift
`through the email or jump from onetask to the next as they
`process email messages. Often people view videos individu-
`ally and this lengthy viewing interrupts the flow of other
`email-related tasks. Because of this interruption, users may
`be less inclined to explore other videos and content, and many
`instead return to the task of reducing their email queue.
`Problem: Delivering Content: Currently, the prevalent mode
`for watching internet content is streaming over the internet
`
`22
`
`22
`
`

`

`US 2008/0276279 Al
`
`Nov. 6, 2008
`
`onto a PC. This model provides a sub-optimal viewing expe-
`rience and limited range of content because of necessary
`bandwidth-to-latency trade-offs. We currently cannot experi-
`ence high-quality, feature length films delivered in real time.
`Instead, we usually click on content, wait someperiod oftime
`for a partial download, only to watch low fidelity content that
`is necessarily short and often temporally discontinuous
`(hitchy). For the viewer, this destroys any sense ofimmersion.
`For the contentcreator, this restricts the contentthatthat they
`are willing to produce, the poor quality of service for stream-
`ing video beget lower quality, shorter video, and little incen-
`tive to create more than quick clips. In contrast, independent
`film industries have repeatedly demonstrated that there is a
`large, untapped reservoir of longer playing, high quality
`(LPHQ) content ready for consumption, but little means for
`unhindered mass.
`
`gathered from today’s (or a longer recent time) recommen-
`dations, and can easily skip or start playing along a shelf.
`Recommendations come from friends(their social network),
`from content analysis, and from persistent searches. In par-
`ticular, a first shelf 102 shows programs from friends, as a
`horizontal row ofstill frames from the relevant content (audio
`content may show meta data about the audiofile such astitle,
`artist, and length) having a superimposed imageofthe friend
`that shared the content. The content on shelf 102 may be
`sorted orfiltered in a variety of manners, such as by friend,
`wherethe top friends may befriends that the user has identi-
`fied as people who share good content, or by comparing
`ratings that the user and the friends have given to common
`pieces of content and making a determination of co-variance
`in tastes between users (where higher correlation makes a
`friend’s recommendations appearhigherin the list of recom-
`mended programs).
`[0043]
`Shelf 104 shows programs that the user himself
`provided. As discussed elsewhere, the user may have dragged
`certain content onto their device while they were surfing the
`web ontheir desktop computer, and such content would show
`up on shelf 104. One particular program—Ryan vs. Dork-
`man2—is shownin an enlarged, highlighted state with a lit
`border 108 in the middle of the screen 110. In this example,
`the user has navigated to that program through the two-di-
`mensionalgrid of programs using a 4-direction controller of
`a television remote control.
`
`Problem: Managing Content on a TV: A third prob-
`[0039]
`lem revolves around where and when the user may want to
`view the content. Email typically delivers the link to a per-
`sonal computing environment rather than to a social enter-
`tainment environment such as the living room. Sifting
`through email, extracting the video and watchingit as part of
`an email-processing task is a very different experience than
`being in an entertain-me mindset, turning on a TV, watching
`it with a group of friends and using a remote control to
`navigate through the content. However, naively browsing on
`a TV will not work either. If the sea of internet-based, user-
`generated content is suddenly available on a TV (viaaset top
`[0044]
`Shelf 106 shows other recommendations, such as
`box), we will need new interaction models that deal with the
`recommendations made automatically by the system. Such
`amountof content and the desire to connect with the commu-
`recommendations may be made, for example, by comparing
`a user’s interests (as indicated by programsthe user watches,
`or that the user watches without skipping out of, or that the
`user rates highly) to those of others, and providing recom-
`mendations of programsthat similar users liked. In a similar
`manner, a user may establish persistent searches for content,
`and the recommendations may included the results of such
`persistent searches. For example, the user may provide a key
`wordlike “funny cat stunts,” and may then be provided with
`all new videos that show such stunts. The persistent search
`maykeep track of content the user has previously viewed or
`declined to view, or may only show content from a most
`recent time period, so that the user is presented only with fresh
`content.
`
`nities who share the content. This is a great opportunity for the
`UserInterface/Experience community to pioneer beyond the
`pick-and-play models of PVRstoday.
`[0040]
`To address someorall ofthese problems, we discuss
`an example system that integrates the PC andthe television
`via the internet. The hardware for the TV, in this example, is
`a set top box prototype that developers can construct from a
`low-end laptop and remote control. The example system is
`designed to gather content andplay it though an SVGAcable
`to an ordinary television-though other connections such as
`HDMIand DVI mayalso be employed. This means develop-
`ers can easily purchase oneandstart developing and extend-
`ing the system without any special hardware. This also gives
`researchers a concrete system for developing andtesting the
`complete experience. Other hardware solutions may also be
`used to perform similar functionality, including standard set
`top boxes and electronics integrated with a televisionitself.
`[0041] FIG.1A showsan example television display 100 of
`shelves of media content organized by the source that recom-
`mended the content, using the example implementation,
`while FIG. 1B shows a moredirect view ofthe display 100. In
`this example, a graphical user interface is displayed showing
`audio-visual programsthat have been associated with friends,
`or acquaintances, of the owner of the television, where the
`television has been logged onto an accountofthe television
`owner. The organization ofthe contentis in horizontal rows or
`shelves, where each program includesa still frame of video
`from the associated program, and can also include an image
`or other indicator of the friend from which the content was
`provided. Certain content may have been provided by the
`television owner himself, such as videos identified by the
`owner when he waspreviously using his desktop computer.
`[0042] The system provides a familiar remote controlinter-
`face. A user turns their device on, they see shelves of content
`
`[0045] The particular display shown in FIG. 1B is at a
`middle zoom level for displaying content on a television. If
`the user hits “back” on their remote control, they can be
`shown smaller view ofthe shelves. Such a “globally” zoomed
`view clusters shelves together. It relies on animation and
`pre-biasing to help recognition even when the imagesare very
`small. Whenthe userselects a particular piece of content, the
`system zoomsin on that piece of content to the highest zoom
`level of detail as shown in FIG. 1C.
`
`FIG. 1C isa screen shot of an example of a bumper
`[0046]
`111 showing program details. The bumper 111 can be
`reviewed by a user to see program detail, and can also be
`shown for a short period of time immediately before the
`corresponding program beginstoplay (e.g., when programs,
`such as adjacent programsona single shelf from FIG. 1B, are
`being played automatically in succession). The bumpergives
`the viewer a chance to pause orjump from bumper to bumper
`ina linear fashion. Hitting the back button while on a bumper
`zooms the viewer back outto the shelf overview.
`
`[0047] Bumpers, whether ofprogram information or adver-
`tisements or other information, may also be shown simulta-
`
`23
`
`23
`
`

`

`US 2008/0276279 Al
`
`Nov. 6, 2008
`
`neously with a program,such as in the form of a “bug”. A bug
`is a visible item that takes up less than a substantial portion of
`the display, and may provide a wide variety of information to
`a viewer. Thebig, in certain circumstances, may be caused to
`generate at various times during the playing of a program so
`as to act like an in-line commercial, and can be particularly
`generatedas part of a pop up programmable program module
`ofthe forms discussed below.
`
`[0053] There are several advantages to using a bookshelf
`metaphor. It allows a user to place various, heterogeneous
`objects onit (music, video, etc.), itis a familiar mimetic, and
`it has a close correlation or mapping to play lists and ranked
`search. Order is implicit from left to right in the design and
`hitting playstarts at the given item and continuesto the right,
`looping when the end has been reached.In addition, looping
`may occur on the sameshelf (e.g., repeat) or from one shelfto
`the next. Bookshelves can also represent other groupings, for
`[0048] The bumper 111 includesa title area 112 that shows
`example a persistent search query such as “surfing,” and
`the program title along with a short description of the pro-
`gatherall available surfing videos each day.
`gram. An image area 114 showsa still image from the pro-
`gram, though a bumper 111 for a program like an audio
`[0054] Bookshelves can be stacked andstacks can behier-
`program may simply showalogo or an album cover. Detail
`archical. This allows for scaling to large numbers of items.
`area 116 showsparticular relevant details about a program,
`Visually, the system can pre-bias the viewer so that even at
`such as the program length and a composite rating level the
`small sizes they will still recognize items. Shelves can also be
`program has received from other users, a URL for the pro-
`sorted andfiltered (e.g. by time, person, ranking). And rec-
`gram or a website associated with the program,andtags that
`ommendations/related videos can be highlighted on the shelf
`have been attached to the program. When the bumper is
`(or another shelf). The ability to define shelves explicitly and
`displayed as part of a queue of programs being displayed in
`implicitly (search, recommendation) and then sort them is a
`sequence, it may be displayed for a pre-determined period
`simple yet powerful tool. Shelves also allow people to create
`and then rolled up so that the corresponding program may be
`and share ata level above the individual content. For example,
`a user can “subscribe”to the Jon Stewart shelf and see what
`played.
`videos he recommends (presumably high-intelligence com-
`[0049] A commentarea 118 shows comments on the pro-
`edy work).
`gram from other users such as friends ofthe first user. The
`comments may be accompanied automatically by images of
`[0055] While watchingorlistening to a program, ifthe user
`the commenting user, in a familiar manner, to permit faster
`decides to share the content, pressing a button on their remote
`navigation of the comments (e.g., so that a user can more
`control can bring upa list of friends 140, as shownin FIG.1F,
`quickly locate comments from someonethey respect). Con-
`and the user can share the current content easily with them.
`trols may also be provided, such asfor the first user to leave
`The list of friends 140 may be obtained, for example, from a
`comments, or forthefirst user to complete a message to one
`list stored on the user’s local client device, but synchronized
`of their friends, such as a note playfully castigating a friend
`with a central server. The list 140 may bespecific to the
`for recommending a program that the first user thought was
`sharing application, or may be a more ge