United States P3128111
`
`[19]
`
`{I1} Patent Number:
`
`5,557,724
`
`Sampat et al.
`
`[45] Date of Patent:
`Sep. 17, 1996
`
`111111 llllllll Ill Illll ||||l lllll lllll Illll llll lllll lllll Illlll Ill Illll llll
`US005557724A
`
`5,392,223
`
`2/1995 Caci 3541514
`
`FOREIGN PATENT DOCUMENTS
`
`0529354
`6373416
`
`311993 European Pat. on.
`1011989
`Japan.
`
`........ G06F 15116
`
`OTHER PUBLICATIONS
`
`Bonet, Bernard A., “Progress in CD—ROM Publishing: a
`Look at New Tools and Titles”, Seyboid Report on Desktop
`Publishing, v.8 n8, p. 15, Apr. 4, 1994.
`Ripley, G. David, “DV1—a Digital Multimedia Technol-
`ogy”, Communications of ACM, v32, n7, p. 811, Jul. 1989.
`Cross, Jerry, “Playing Live Digital Video Under Windows",
`Windows—DOS Developer's Journal, v3, n3, p. 39, Mar.
`1992.
`Little, T. D. C., et a.l., “Network Considerations for Distrib-
`uted Multimedia Object Cornpostion and Communication,”
`IEEE Network, The Magazine of Computer Communica-
`tions, vol. 4, No. 6, Nov. 1990, New York, US, pp. 32-49,
`XP172741. See p. 39, right column, line 1—p. 43, left
`column, line 29.
`
`(List continued on next page.)
`
`Primary Examr'ner—~Mark R. Powell
`Assistant Examiner~—-V. Chauhart
`Attorney, Agent, or Firm.—Stcve Mendelsohrt; William H.
`Murray
`
`1571
`
`ABSTRACT
`
`A user interface is displayed on a computer system capable
`of processing one or more data streams. The user interface
`has one or more displayed representations, where each of the
`displayed representations corresponds to one of the data
`streams. A user of the computer system selectively adjusts
`the processing of each of the data streams using the corre-
`sponding displayed representation of the user interface. In a
`preferred embodiment, in which the computer system is a
`system for multicasting audio, video, and1or
`text data
`strearns, the user interface has a video window for display-
`ing the video stream, a set of audio controls for controlling
`the play of the audio stream, and a text reader bar for
`displaying the text stream.
`
`30 Claims, 21 Drawing Sheets
`
`[54]
`
`['15]
`
`1731
`
`1211
`
`[221
`
`[51]
`[521
`[53]
`
`[55]
`
`USER INTERFACE, METHOD, AND
`APPARATUS SELECTING AND PLAYING
`CHANNELS HAVING VIDEO, AUDIO,
`AND10R TEXT STREAMS
`
`Inventors: Ketan Sampat, Portland; John
`Kembel, Lake Oswego, both of Oreg.
`
`Assignee:
`
`Intel Corporation, Santa Clara, Calif.
`
`Appl. No: 133,614
`
`Filed:
`
`Oct. 12, 1993
`
`GIJGF 3114
`Int. Cl.“
`3951157; 3951155; 395116]
`US. Cl.
`Field of Search ................................... .. 3951155, 156,
`3951157, 158, 159, 161; 3641514
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`........................ .. 3401721
`511980 Bakula et a1.
`4,204,206
`4,794,465 1211988 Van Luyt et a1.
`.. 3581341
`4,819,160
`411989 Tanks. et a].
`.. 3951600
`4,914,571
`411990 Baratz et a1.
`.. 3951600
`4,942,574
`711990 Zelle ........
`370185.15
`4,974,149
`1111990 Valenti
`......
`3951650
`5,005,167
`411991 Art1'itu's etal
`.. 3591135
`5,089,956
`211992 Ma.t:Pha.il
`3951600
`5,179,556
`111993 Turner
`.. 370194-.1
`..... ..
`5,202,961
`411993 Mills ct a1.
`.. 3951159
`5,206,929
`411993 Langford et al.
`.
`.. 3951159
`5,216,427
`611993 Yon el al.
`.
`.. 3421352
`5,247,678
`911993 Littleton ...... ..
`3951700
`5,257,369
`1011993 Ske-en etal. ..
`.. 3951650
`5,287,530
`211994 Davisctsl.
`..
`. 3'10194.1
`5,291,477
`311994 Liew
`370154
`5,291,601
`311994 Sands
`.. 3951100
`5,305,311
`411994 Lyles
`370160
`5,313,454
`511994 Bustirti ct al.
`370113
`5,321,693
`611994 Perlman ........
`370135.13
`5,331,637
`711994 Francis eLa.l.
`...... .. 370154
`5,339,392
`811994 Risberg etal.
`.... .. 3951161
`5,341,474
`311994 Gelman et a1.
`. 395120009
`5,374,952 1211994 Flohr
`348112
`5,375,068 1211994 Palmer et al.
`.. 3641514
`5,388,l97
`211995 Rayner
`3951154
`
`
`
`.
`
`205
`
`210
`
`208
`
`.2129
`
`SAMSUNG 1004
`
`SAMSUNG 1004
`
`1
`
`

`
`5,557,724
`Page 2
`
`OTHER PUBLICATIONS
`
`Wm. H. M., et al., “Multimedia Conferencing in the Ether-
`phone Environment.” Computer, vol. 24, No. 10, Oct. 1991,
`Los Alarnitos, CA, US.
`Rangen, V. P. et a1., "Designing an 0n—Dema.ntl Multimedia
`Service." IEEE Communications Magazine, vol. 30, No. '1',
`Jul. 1992, US pp. 56-64. See page 61, line 9—line 51.
`Lirtie et al, “Synchronization and Storage Models for Mul-
`timedia Objects." IEEE Journal on Selected Areas in Com-
`munications, vol. 8, No. 3, Apr. 1990, pp. 413-427.
`Rangan et al, “Designing an On—Demand Multimedia Ser-
`vice." IEEE Communications Magazine, Jul. 1992, PP.
`56-64.
`
`Vakalopoulou, Maria, “Multimedia—E.ndgerate." Naehricht-
`enteehnik Elcktronik, vol. 42, No. 2, Mar. 1992, Berlin, pp.
`56-58.
`
`Fox, Edward A.. “Advances in Interactive Digital Multime-
`dia Systems." Computer, vol. 24, No. 10, Oct. 1991. Long
`Beach US. pp. 9-21.
`
`Gopal, 1., et al., “Multicasting to Multiple Groups over
`Multicast Channels." Computer Networking Symposium,
`Apr. 1988, USA. See page 79, left column, line 2]—page
`80, left column, line 5.
`
`Rangan, P. Verlkat, et al.. “Software Architectures for Inte-
`gration of Video Services in the Etlterphonc System." IEEE
`Journal on Selected Areas in Communication, vol. 9, No. 9,
`Dec. 1991, New York, US, pp. 1395-1404.
`
`Douglas A. Young, “X Window Systems Programming and
`Applications with X1," 1989 Prentice Hall, Englewood
`Cliffs. New Jersey 07632, Chapter 11 Interclient Commu-
`nications, pp. 265-283.
`
`Lauren Weiustcin, “Project Stargate,” 1985 The USENIX
`Association, pp. 79-80.
`
`Frequently Asked Questions on the Multicast Backbone
`(MBone) by Steve Casner, Jan. 16, 1993.
`
`2
`
`

`
`U.S. Patent
`
`Sep.17, 1995
`
`Sheet 1 of 21
`
`5,557,724
`
` éLm¢5}255
`
`F]G.I
`
`3
`
`

`
`U.S. Patent
`
`Sep.1‘7,1996
`
`Sheet 2 of 21
`
`5,557,724
`
`
`
` 200
`
`!§—Efl
`—guide...
`flptions
`
`202
`
`' 204
`
`206
`
` A password is
`
`
`—‘—*“—
`
`
`
`Password
`
`uired to
`' w the chem you have
`
`ected.
`
`
`
` " Password:
`" E::]
`‘P
`
`FIG. 4
`
`4
`
`

`
`3U
`
`4|.nm3
`
`7.
`
`%
`
`5,
`
`.4.2
`
`3,.mm,..o~,..~
`
`Pcan
`
`mgflwEaamoam
`
`mom
`
`
`
`E._!,ia_&...9.__zo;maoz
`
`w.ENew
`
`12:55...E3599%3mm“.3.20.8EooumoS3m3:3E3852:523newM:2:$55_._§”888mS...coommfieoo
`
`5
`
`
`

`
`U.S. Patent
`
`Sep. 17, 1995
`
`Sheet 4 of 21
`
`5,557,724
`
`Pay-Per-View
`
` H L
`
`have selected is 0 5*’
`The channel
`unnei. Pl
`Pay-Per-View
`e enter
`your credit card number
`ow.
`
`‘Q Credit Card Number:
`:: 1:
`
`FIG. 5
`
`
`
`6
`
`

`
`U.S. Patent
`
`Sep. 17,1996
`
`Sheet 5 of 21
`
`5,557,724
`
`Qefoult Window Size
`Hide Controls
`Always On _T_op
`
`900
`
`/
`
`J _\{ideo Window
`
`J gudio Controls
`
`J Bender Board
`
`Remote Control
`
`7
`
`

`
`U.S. Patent
`
`Sep.17,1996
`
`Sheet 5 of 21
`
`5,557,724
`
`8
`
`

`
`U.S. Patent
`
`Sep. 17, 1995
`
`Sheet 7 of 21
`
`5,557,724
`
`=
`
`Remote Control
`
`
`
`FIG. 13
`
`9
`
`

`
`U.S. Patent
`
`Sep. 17, 1996
`
`Sheet 3 of 21
`
`5,557,724
`
`NOH
`
`momfi
`
`vomfi
`
`momfi
`
`ma.w_m
`
`xmoxhwz
`
`mmizum
`
`mzxxwmom
`
`m==_umpH=ux¢
`
`ofimnx
`
`xm>_ma
`
`o_a=¢
`
`m:=_m¢u
`
`_zmzom=ou
`
`10
`
`10
`
`

`
`QMnu
`
`¢n.aDr
`
`¢u»nunu
`
`Sep. 17, 1996
`
`Sheet 9 of 21
`
`427,755,5
`
`
`
`
`
`mwMwmw.wAzmxvmmo¢=¢=mwuH,mmm¢_mm=
`
`
`
`
`
`Nfimw
`
`mw.fi
`
`.m._n—
`
`mum“fiE«%5
`
`
`
`
`wu¢_mmF=_u=_==¢mmo:mzo_h¢uHd¢¢¢mwwxzizmwu_»mwm‘Ham:
`
`
`
`wHmu6;:
`
`wu¢xo~mmm<;=o__¢uH4m¢¢mm}mmm
`
`
`EfimicpNowa
`
`woofi
`
`voefi
`
`wmmfi
`
`xzogpmz
`
`wu¢_xm_zH
`
`ohmm:
`
`bxw_
`
`x2_m
`
`oHo=<
`
`¥=_m
`
`aw:
`
`omgH>
`
`xzflw
`
`mm:
`
`
`
`vfimfimmwfi
`
`
`
`omofimfiofiofiwfi
`
`vfiwfi
`
`ow
`
`,¢4¢m_a
`
`mm>Hxa
`
`zommoh20¢;
`
`up=om_
`
`HxmpoHc;¢oHa=¢
`
`
`
`,¢_¢mHgomg_,
`
`mumzommw,_mnxm}H=a
`
`
`
`¢m>_xaummou
`
`ofimfi
`
`aomfi
`
`
`
`
`
`
`
`AHm¢mzmvwu¢¢¢mH=_uzmzxxmmozmzoHh¢uH4m¢<mmum>mwm«Ham:m=H»-4{mx
`
`
`
`
`
`
`
`ofimfi
`
`Nfiofi
`
`11
`
`11
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Sep. 17, 1995
`
`Sheet 10 of 21
`
`5,557,724
`
`S:
`
`o_n=¢
`
`xw>H¢a
`
`¢o_H=a:
`
`5¢_Ho_g
`
`»2mH4u
`
`mm¢;H¢om
`
`mx=»ump.=ux‘
`
`29¢;
`
`xmogpmz
`
`12
`
`12
`
`
`
`
`

`
`‘n.aPamU
`
`nu9»
`
`;n.
`
`Sep. 17, 1996
`
`Sheet 11 of 21
`
`427,755,5
`
`mU¢;mwp=_
`
`vfinfi
`
`xmozbmzmm:
`
`=ommxxflm
`
`mmwfi
`
`hxmh
`
`o_:={
`
`xzfim
`
`mm:
`
`wfimfi
`
`omgfi;
`
`mm:
`
`xz_m
`
`vfiwfi
`
`op
`
`,¢4¢m_n
`
`mm,Hxa
`
`voufl
`
`OM93‘
`
`xm>_xa
`
`ofinfi
`
`*¢4mmHa
`
`mw,_gg
`
`«end
`
`mfinfi
`
`
`
`Mw.“.mv_n_
`
`ofimfimsaa
`
`mgcmopmmm.=
`
`:2:2....E202539::25Now“
`
`
`
`
`
`
`
`wwwfiA:¢%a
`
`
`
`mum:ems$3.2;wfiam=3:
`
`
`
`
`
`
`
`
`
`mu¢_:wH=_a=H=z¢maomm=o_~«uHA¢¢¢xmo¢=¢=mmu_}mmm(Haw:
`
`
`
`
`
`
`
`
`
`
`
`
`
`Awmim=mvmu¢_xm_=Hazmszimoomgzo_F¢u_4mm¢mwuH:xwm«Ham:m=Hp-A¢mx
`
`
`
`
`
`
`
`
`
`vowfi
`
`vmwfi
`
`mama
`
`Ofiwfi
`
`13
`
`13
`
`
`
`
`
`

`
`0...3U
`
`..I.ne..lna
`
`Sep. 17, 1995
`
`Sheet 12 of 21
`
`5,557,724
`
`AVmHm4m=ou¢~¢amHHm;
`
`m
`
`AV¢mmm=mmmPmHomx
`
`
`
`mm:xzwm
`
`
`
`mm:mumsom
`
`moi
`
`N
`
`Av¢H¢mm}Hmumm
`
`
`
`
`
`mwuqzizmmuH:mmm‘Ham:
`
`3.O_n_
`
`~23
`
`AVm»m4m=oua=mm
`
`m
`
`0
`
`A.mm_m=ma=mm
`
`14
`
`14
`
`
`
`
`
`

`
`U
`
`tneflaP
`
`Nm
`
`P...0B
`
`M
`
`Son.0.m
`
`7.,§_E_§E2§_7_.,E.is5woom11% cE§8Ea:§
`
`m_Vm9.282%
`
`
`
`mm.$2.33
`
`NOON
`
`
`
`m$93.:395%:3...%cfimzamavoom
`
`%m
`
`Nwnvmwum=mm>Hmuwm
`
`15
`
`
`

`
`U.S. Patent
`
`Sep. 17, 1996
`
`Sheet 14 of 21
`
`5,557,724
`
`FIG. 21
`
`2100
`
`INPUT/OUTPUT
`
`
`
`2114
`
`TO AND FROM
`
`NETWORK
`
`INTERFACE
`
`16
`
`16
`
`

`
`U.S. Patent
`
`Sep.17,1996
`
`Sheet 15 of 21
`
`5,557,724
`
`FIG. 22
`
`2106
`
`2202
`
`2110
`
`DLM_BEGINSESSION
`
`DLM_ENDSESSION 2206
`
`
`
`PORT/SOCKET
`
`MANAGER
`
`
`
`DLM_REGISTERSOCKET
`DLH_UHREGISTERSOCKET
`
`U’! r112 I3 '3fl‘!(2rr1 I‘'\ U‘! ‘I-..J
`
`I I I
`I I I
`
`: : :
`- — -
`I I I
`I I I
`
`
`
`MESSAGE OUTPUT
`"AMER
`
`2210
`
`BL!-LDGSEND
`
`2212
`
`um SEND COMPLETE
`CMMK
`
`2216
`
`DLH DGPOST
`-
`
`3”
`
`FFER
`
`2218
`
`DLM MESSAGE
`RECEIVE CALLBACK
`
`SEND CU!-IPLETE
`
`HHNIJLER
`
`ASYNCHRONOUS
`
`FRAGMENTER
`
`
`RECEIVE QUEUE(S} J
`
`
`
`RECEIVE BUFFER — - —
`amen
`I I -
`I I I
`I I I
`I I I
`I I I
`
`- - -
`
`DE-FRAGMENTER
`
`ASYNCHRONOUS
`
`17
`
`17
`
`

`
`U.S. Patent
`
`Sep. 17, 1996
`
`Sheet 16 of 21
`
`5,557,724
`
`I‘ I (3 .
`
`2223
`
`2302
`
`O
`
`HDM_BEGINSESSION
`
`2304
`HDH_R£GiSTER
`
`HDH_UNREGISTER
`
`"""""""""""""""""""""""" -'
`
`HOM_ENOSESSION
`
`
`LINK PACKET
`
`OUTPUT HhNAGER
`
`EVENT CONTROL
`
`
`
`
`
`BLOCK HANAGER
`
`
`
`2312
`
`HDM_fiGCLEARTOSEND
`
`SEND PROCESS
`MANAGER
`FLOW CONTROL
`HANflGER
`
`
`
`2314
`
`
`___________ __
`
`”*NAGER
`
`a£cE1uE PROCESS
`
`
`
`____________________________________________-
`
`
`
`5E"Ehfi:E::ETE
`
`18
`
`non usssasz
`RECEIVE CALLBACK
`
`2318
`
`MDM SEND COMPLETE
`
`CALLBACK
`
`18
`
`

`
`U.S. Patent
`
`Sep. 17, 1996
`
`Sheet 17 of 21
`
`5,557,724
`
`F IG . 24 .
`
`AUDIO MSP
`
`VIDEO HSP
`
`TEXT MSP
`
`LEVEL 1
`
`LEVEL 2
`
`LEVEL 3
`
`LEVEL 4
`
`LEVEL 5
`
`MEDIA SERVICES MANAGER (MSH)
`
`DATA LINK MANAGER (DLM)
`
`MEDIA DEPENDENT MODULE (MUM)
`
`NETWORK INTERFACE
`
`T0 & FROH
`
`NETWORK
`
`19
`
`19
`
`

`
`U.S. Patent
`
`Sep. 17, 1996
`
`Sheet 13 of 21
`
`5,557,724
`
`‘
`
`FIG. 25
`
`2 was
`
`TIME srmp
`
`|
`
`
`I
`
`
`2043 ems
`
`AUDIO mm
`
`FIG. 26
`
`23 was
`
`'4 ewes I
`
`up To 13 KBYTES
`
`vmzo ron
`WW5 HEADER
`
`
`RESERVED
`
`
`
`vwao um
`
`I
`
`FIG. 27
`
`BOT BYTES
`
`TEXT DATA
`
`20
`
`I
`
`
`
`
`|
`
`20
`
`

`
`U.S. Patent
`
`Sep.1’7,1996
`
`Sheet 19 of 21
`
`5,557,724
`
`FIG. 28
`
`I
`
`24 BYTES
`
`I
`
`UP TO I476 BYTES
`
`I
`
`DLH HEhDER
`
`DLH DATE
`
`FIG. 29
`
`1
`
`I
`6 BYTES
`I BYTEI
`
`
`DEST
`
`PORT
`
`6 BYTES
`
`I
`
`I BYTEI 4 BYTESI BYTES
`
`2
`
`2
`
`2
`
`I BYTESI BYTESI
`
`
`SOURCE
`
`SRC
`
`HESSAGE OFFSET SIZE
`
`TOTAL
`
`SIZE
`
`ADDRESS
`
`PORT
`
`HUHBER
`
`FIG. 30
`
`I
`
`6B‘rTES
`
`I
`
`6 BYTES
`
`IZBYTES I
`
`up To 1500 BYTES
`
`DESTINATION
`
`SOURCE
`
`PACKET
`TYPE
`
`LINK PA
`
`CKE
`
`T
`
`21
`
`

`
`QMU
`
`..I.anr
`
`tHan
`
`Sep. 17, 1996
`
`Sheet 20 of 21
`
`427’755,5
`
`mofim
`
`AIIV
`
`xmozmz
`
`
`
`
`
`moan£82;:o8§___
`
`mwH=¢mm_4mammhmmaamm
`
`
`
`mh=_om,m_=moz‘
`
`rn
`___....____.....______._.
`
`fin.0_n_
`
`mgmu
`
`mmM¢¢xm_4mommpmmzamm
`
`
`
`mH=_om,mh=m=2:
`
`
`
`
`
`$03..=o:__a_:.a._.5§:=_._
`
`mE§_:asflaswx
`
`Nofim
`
`
`
`
`
`w»=Homrmpzmoz:muH»¢mm
`
`mmpmmzomm
`
`22
`
`ES
`
`A
`
`
`
`mggavm_m¢_=uuxm
`
`
`
`um=¢=*n.¢mo4a
`
`music;
`
`
`
`m~z~o¢ymhzm
`
`
`
`uH=¢=,n4¢ma4u
`
`
`
`Adamvxm¢¢od
`
`
`
`mp=_omymgzm
`
`SS
`
`22
`
`
`
`
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Sep. 17, 1996
`
`Sheet 21 of 21
`
`5,557,724
`
`FIG. 32
`
`LIBRARY B
`
`LIBRARY A
`
`IRITIRLIZE LIBRARY A
`
`LORD LIBRARY B
`
`CA L TO
`L
`
`INITIRLIZE LIBRRRY B
`
`INITIALIZE L[BR‘RY B
`
`RETURN
`4—j-—-———---5-.-_.——_—-—
`
`RETURN TO USER
`‘T35
`
`STRRT CLOSING LIBRARY A
`
`TIHE 1
`
`CELL TO CLOSE LIBRARY 3
` ._, START CLOSING LIBRARY B
`
`SYNCHRONOUS RETURN THAT
`
`CLOSING OF LIBRARY B IS STARTED
` ~*
`
`TIME 2
`
`SYNCHRONOU5 RETURN TO
`
`USER THAT CLOSING OF
`LIBRRRY A I5 STARTED
` r~
`
`ASYNCHRONOUS NOTIFICATION THAT
`
`CLOSE OF LIBRARY 5 IS COMPLETE
`«T-——-5-5
`
`CLOSE COMPLETE INTERRUPT
`
`TIME 3
`
`RSYNCHRONOUS NOTIFICRTION
`
`TO USER THAT CLOSING OF
`
`LIBRARY R IS COHPLETE
`§.....__..................._...._...
`
`RETURN FROM USER
`-
`
`TIME 4
`
`TIRE X
`
`RETURN FROH ASYNCHRONOUS NOTIFICATION
`.________H____________________________*
`
`23
`
`23
`
`

`
`I
`
`2
`
`5,557,724
`
`USER INTERFACE, METHOD, AND
`APPARATUS SELECTING AND PLAYING
`CHANNELS HAVING VIDEO, AUDIO,
`ANDIOR TEXT STREAMS
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to computer data processing,
`and, in particular, to user interfaces and methods for pro-
`cessing multiple data streams on a computer.
`2. Description of the Related Art
`In multieasting, one or more sources transmit a plurality
`of data signals for potential receipt by one or more receivers
`in a network. Only one copy of each data signal is trans-
`mitted. Each receiver selects which if any of the plurality of
`signals to receive and process.
`Mullicasting differs from point-to-point communication,
`tnultipoint communication without multicasting, and broad
`casting. In point-to—point communication, one copy of data
`is selectively transmitted from one source to one receiver. In
`multipoint communication without rnulticasting, data is cop-
`ied multiple times, one copy of which is transmitted to each
`of a set of multiple receivers. In broadcasting, each data
`signal is transmitted to every receiver in the network without
`giving the receiver the ability to select only a subset of those
`transmitted signals to be received.
`It is desirable to provide ntulticasting on a computer
`network. It is particularly desirable to provide a system For
`transmitting audio, video, and text data streams for selective
`receipt by one or more client computers of a computer
`network. For example, a user would be able to select a
`television channel comprising audio and video signals for
`play on the client computer. The user would also preferably
`be able to control certain aspects of the play of the selected
`signal. For example. the user would be able to control the
`volume of the audio component and the size of the display
`of the video component. Moreover, the user would be able
`to select a subset of the components of a selected channel for
`play (e.g., playing only the audio component of a television
`channel).
`
`20
`
`3|}
`
`35
`
`It is also desirable that the multieast system support data
`streams that are received from an external source (c.g., via
`air transmission or cable) or from a local source (e.g., a
`VCR]. When the client computer provides a windowed
`environment (such as that provided by Microsoft Windows),
`the multicast system preferably allows a user to work in one
`window while the selected video andfor text are displayed in
`one or more other windows.
`
`43
`
`50
`
`The Internet MBONE multicast backbone system is a
`semi—permanent multicast
`testbed. MBONE is a virtual
`network. It is layered on top of portions of the physical
`Internet to support routing of multicast packets since that
`function is not integrated into many production routers. The
`network is composed of islands that can directly support
`multicast, such as multicast local area networks (LANS) like
`Ethernet, linked by point-to—point links called “tunnels". The
`tunnel endpoints are typically workstation-class machines
`having operating system support for multicast and running
`the multicast routing daemon.
`However,
`the MBONE system does not provide high-
`quality multicasting. Audio signals are subject to unaccept-
`able delays that result in non-real-time play at the client
`computers. In addition, audio and video signals are not
`related. As a result, the play of audio signals is not synchro-
`
`65
`
`24
`
`nized with the play of video signals. The multicasting is
`therefore of low quality. Moreover, MBONE does not allow
`the user to select components and control aspects of the
`selected signal. Furthermore, MBONE does not support the
`play of a selected signal in a windowed environment.
`It is accordingly an object of this invention to overcome
`the disadvantages and drawbacks of the known art and to
`provide methods and apparatuses for rnulticasting multiple
`signals on a computer network.
`It is a further object of the present invention to provide
`high-quality multicasting of audio, video, and text data
`streams on a computer network.
`It is a further object of the present invention to provide
`multicasting on a computer network wherein a user may
`select components of a selected channel for play.
`It is a further object of the present invention to provide
`multicasting on a computer network wherein a user may
`control certain aspects of the play of a selected channel.
`It is a further object of the present invention to provide
`multicasting on a computer network having client computers
`that operate in a windowed environment.
`Further objects and advantages of this invention will
`become apparent from the detailed description of a preferred
`embodiment which follows.
`
`SUMMARY OF THE INVENTION
`
`According to a preferred embodiment. the present inven-
`tion is a user interface for display on a computer system
`capable of processing one or more data streams. The user
`interface comprises one or more displayed representations,
`where each of the displayed representations corresponds to
`one of the data stneams. A user of the computer system may
`selectively adjust the processing of each of the data streams
`using the corresponding displayed representation.
`According to an alternative preferred embodiment, the
`present invention is a method and system for controlling the
`processing of one or more data streams by a computer
`system. A user interface comprising one or more displayed
`representations is generated, where each of the displayed
`representations corresponds to one of the data streams. The
`processing of at least one of the data streams is selectively
`adjusted using the corresponding displayed representation.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Other objects, features. and advantages of the present
`invention will become more fully apparent from the follow-
`ing detailed description of the preferred embodiment, the
`appended claims, and the accompanying drawings in which:
`FIG. 1 is a representation of a multicast system for
`multicasting multiple, related data streams on a computer
`network, according to a preferred embodiment of the present
`invention;
`FIG. 2 shows a preferred embodiment of the user interface
`as displayed on the monitor of a client of the multicast
`system of FIG. 1;
`FIG. 3 shows an example of a preferred embodiment of
`the Program Guide window displayed when the user selects
`the Guide option in the channel controls of the user interface
`of FIG. 2;
`FIG. 4 shows a preferred embodiment of the Password
`window created when the user selects a channel that requires
`the entry of a password;
`
`24
`
`

`
`3
`
`4
`
`5,557,724
`
`FIG. 5 shows a preferred embodiment of the Pay—Per—
`View window created when the user selects a channel that
`requires payment;
`FIGS. 6. 7, and 8 show preferred embodiments of the user
`interface of FIG. 2 for selected channels consisting of only 5
`video, only audio, and only text, respectively;
`FIG. 9 shows a preferred embodiment of the Options
`menu created when the user selects the Options option in the
`chartnel controls of the user interface of FIG. 2;
`FIGS. 10, 11, and 12 show preferred embodiments of the 10
`user interface of FIG. 2 when video and text. video only, and
`text only, respectively, are selected for display with controls
`hidden;
`FIG. 13 shows a preferred embodiment of the remote
`control window that is created when the Remote Control
`item of the Options menu of the user interface of FIG. 2 is
`selected;
`
`15
`
`FIG. 14 shows a preferred embodiment of the configure
`window that is created when the Configure .
`.
`. item of the
`Options menu of the user interface of FIG. 2 is selected;
`FIG. 15 is a block diagram of the server subsystem of the
`multicast system of FIG. 1;
`FIG. 16 is a block diagram of the software architecture of
`the server subsystem of FIG. 15;
`FIG. 17 is a block diagram of the client subsystem of the 25
`multicast system of FIG. 1;
`FIG. 18 is a block diagram of the software architecture of
`the client subsystem of FIG. 17;
`
`FIG. 19 is a representation of the flow of data through the 30
`server software architecture of FIG. 16;
`FIG. 20 is a representation of the flow of data through the
`client software architecture of FIG. 13;
`FIG. 21 is a block diagram of the software architecture of
`the network inpulloutput (U0) driver of the server software 35
`architecture of FIG. 16 and the client software architecture
`of FIG. 18;
`
`FIG. 22 is a block diagram of the data link manager of the
`network I.-"O driver of FIG. 21;
`FIG. 23 is a block diagram of the media dependent
`module of the network U0 driver of FIG. 21;
`FIG. 24 is a representation of the data flow through each
`server and client of the multicast system of FIG. 1;
`FIGS. 25, 26, and 27 are representations of Level 1 audio,
`video, and text data packets, respectively, of the multicast
`system of FIG. 1;
`FIG. 28 is a representation of a Level 3 data packet of the
`multicast system of FIG. 1;
`FIG. 29 is a representation of the 24-—byte DLM header of 50
`the Level 3 data packet of FIG. 28:
`FIG. 30 is a representation of a Level 5 data packet of the
`multicast system of FIG. 1;
`
`45
`
`FIG. 31 is a block diagram of the software architecture of 55
`each of the server and clients ofthe multicast system of FIG.
`1 for loading and unloading of service libraries; and
`FIG. 32 is a diagram of the timing of function calls when
`a user openslcloses one module, which in turn opensfcloses
`another module, under the traditional method of using
`straight calls to the Windows LoadI..ibrary and FreeLibrary
`functions.
`
`DBSCRIPHON OF THE PREFERRED
`EMBODlMENT(S)
`
`Referring to FIG. 1, there is a representation of tnulticast
`system 100 for rnulticasting multiple, related data streams
`
`on a computer network, according to a preferred embodi-
`ment of the present invention. Multicast system 100 com-
`prises a single server 102 and multiple clients 104 linked by
`network 106. Server 102 captures and posts data on network
`channels, with any number of clients 104 independently
`selecting channels for receipt and play.
`Server 102 is capable of capturing analog audio and video
`signals from three difieient sources: (1) signals generated
`locally by camera 108, (2) signals received by antenna 110
`from a remote source, and (3) recorded signals from VCR
`112. In addition, server 102 may receive digital text signal
`from a remote source (not shown) {e.g., via modem). Server
`102 may receive multiple signals of each type (i.e., audio,
`video, or text) from one or more sources at the same time.
`For example, server 102 may receive via antenna 110 a
`first television program consisting of three signals: video,
`English language audio, and Spanish language audio. At the
`same time, server 103 may receive a second television
`program consisting of video and English language audio
`from VCR 112. Server 102 may also concurrently receive
`the audio signal for a radio station via antenna 110 and a text
`stream via modem.
`
`Server 102 digitizes the received analog audio and video
`signals to generate digital audio and video data streams.
`Server 102 selectively relates the digital audio, video, and
`text data streams together to form specified eharutcls. A
`channel is a logical representation of a specific collection of
`data streams transmitted over the network. For example, the
`video and English audio data streams of the first television
`program may be related together to form a first channel. That
`same video data stream may be related to the Spanish audio
`data stream to form a second channel. In addition, the video
`and English audio data streams of the second television
`program and the text data stream may be related to form a
`third channel. The audio data stream for the radio station
`may constitute a fourth channel by itself.
`Server 102 fragments each data stream into network data
`packets for transmission over network 106. Server 102
`transmits a single copy of each of the network data packets
`for all four channels over network 106 for potential receipt
`by clients 104. Each client 104 independently and optionally
`selects any one of the four channels. When a client 104
`selects a channel, the client may receive and process the
`network data packets corresponding to the data streams of
`the selected channel. Thus, system 100 is a multicasting
`system that provides multicasting of one or more channels
`from server 102 to one or more clients 104. A preferred
`embodiment of a user interface for multicast system 100 as
`well as the options provided to a user via that interface are
`described in further detail
`later in this specification in
`conjunction with FIG. 2.
`Server 102 and clients 104 may be any suitable computers
`and are preferably personal computers having an Intel®
`i486-based central processing unit (CPU) running Microsoft
`Windows. Server 102 and clients 104 are preferably sound
`enabled with a SoundBlaster Pro from Creative Labs, net-
`work enabled with an Intel® Ether Express 16 card, and
`video enabled with Intel® SmartVideo® Recorders (ISVRJ.
`Network 106 is preferably an Ethernet network.
`
`User Interface
`
`Referring now to FIG. 2, there is shown a preferred
`embodiment of the user interface 200 that is displayed on the
`monitor of a client 104 of the multicast system 100 of FIG.
`1. In a preferred embodiment, client 104 operates in a
`
`25
`
`25
`
`

`
`5
`
`6
`
`5,557,724
`
`windowed environment. such as that provided by Microsoft
`Windows. User interface 200 is a window frame comprising
`window controls 202, channel controls 204, video display
`206. audio controls 208, and text reader bar 210.
`is
`The video component [if any) of a selected channel
`displayed in video display 206 and the text component (if
`any) of the selected channel is displayed in text reader bar
`210. Preferably using a computer mouse, a user may use
`audio controls 208 to control the play of the audio compo-
`nent (if any) of the selected channel. Controlling the audio
`play includes increasing or decreasing the volume or muting
`the sound completely. Audio controls 208 also displays a
`volume meter for depicting the current volume level.
`Those skilled in the art will understand that a user may use
`window controls 202 to close [i.e., terminate the display of)
`user interface 200 and to control the size and position of user
`interface 200. User interface 200 may be moved around the
`display raster by dragging either window controls 202, video
`display 206, or text reader bar 210 using the mouse. Channel
`controls 204 provides the user with the ability to select a
`channel and to control certain aspects of the play of the
`selected channel.
`
`Multicasl system 100 supports three types of data streams
`(audio, video, and text}. A charutcl may comprise any
`combination of data strearns. The user is able to select how
`to configure the play of a selected channel (e.g., play only
`the audio component of a channel having both audio and
`video components). Moreover,
`the user may change the
`selected channel configuration and various aspects of the
`channel (e.g., size of video display 206 or volume of audio
`play} at any time. Certain channels may be marked as
`password protected andlor as pay-per-view. In those cases,
`the user would have to enter the correct password andfor a
`valid credit card number depending upon the nature of the
`channel.
`
`Program Guide of the User Interface
`
`Referring now to FIG. 3, there is shown an example of a
`preferred embodiment of the Program Guide window 300
`created when the user selects the Guide‘ option in channel
`controls 204 of user interface 200 of FIG. 2. Program Guide
`window 300 comprises a list 302 of the channels currently
`being transmitted over the computer network and a list 304
`of the channels to be transmitted over the computer network
`in the future. Program Guide window 300 also preferably
`displays the current time in clock 306.
`Each entry in lists 302 and 304 of Program Guide window
`300 identifies the date, start time, and narne (e.g., television
`channel name or program name} of the transmission. The
`entry also provides (in brackets) information about
`the
`components of the channel, where the letters A, V, and T
`indicate that the channel has audio, video, and text compo-
`nents, respectively.
`The letter F‘ indicates that the user must enter a special
`password in order to play the selected channel. Referring to
`FIG. 4,
`there is shown a preferred embodiment of the
`Password window created when the user selects a channel
`that requires the entry of a password. The user uses the
`Password window to enter the special password for the
`program.
`
`The symbol $ indicates that the user must pay in order to
`play the selected channel. Referring to FIG. 5,
`there is
`shown a preferred embodiment of the Pay-Per-View window
`created when the user selects a channel that requires pay
`ment. The user uses the Pay~Per-View window to enter a
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`35
`
`65
`
`26
`
`credit card number to which to charge the payment for the
`program.
`
`After the user selects a desired channel, the Program
`Guide window 300 is closed and user interface 200 is
`configured in accordance with the components of the
`selected channel. For example, referring now to FIGS. 6, 7,
`and 8, there are shown preferred embodiments of the user
`interface 200 for selected channels consisting of only video,
`only audio, and only text, respectively.
`
`Options Menu of the User Interface
`
`there is shown a preferred
`Referring now to FIG. 9,
`embodiment of the Options menu 900 created when the user
`selects the Options option in channel controls 204 of user
`interface 200 of FIG. 2. Options menu 900 provides controls
`for the user to customize the component configuration and
`other aspects of the window.
`When selected, the Pause Services item of Options menu
`900 pauses reception of all currently active data streams all
`the way down to the network level. When implemented in
`the preferred windowed environment, multicast system 100
`allows a client 104 to play a selected channel in one window,
`while the client 104 concurrently works in another window.
`Pause Services allows a user to suspend the multicasting
`functions performed by client 104 in order to accelerate a
`network, disk, or CPU intensive job also being handled by
`client 104.
`
`When Pause Services is selected. many of the charmel and
`audio controls are preferably disabled, although the user
`may change the position of the user interface and perform
`other window—related operations. The Pause Service menu
`item toggles the application back and forth between paused
`and unpaused states. A check mark is preferably displayed
`next to the menu item to indicate that service is paused.
`The user may shrink or enlarge video display 206 of user
`interface 200 by selecting and dragging a corner or side of
`video display 206 with the mouse. When selected, the
`Default Window Size item of Options menu 900 returns user
`interface 200 to its specified default window size as dictated
`by the default size for video display 206 (preferably 160
`pixels wide by 120 pixels high). The maximum size of video
`display 206 is preferably 320 pixels wide by 240 pixels high
`and the minimum size is preferably 120 pixels wide by 90
`pixels high. The aspect ratio of video display 206 is pref-
`erably always preserved.
`When selected, the Hide Controls item of Options menu
`900 hides (i.e., terminates the display of) window controls
`202, channel controls 204, and audio controls 208 of user
`interface 200. The controls are redisplayed by double click-
`ing on either video display 206 or text reader bar 210. As
`such, the Hide Controls menu item is only enabled when at
`least one of video display 206 and text reader but 210 is
`displayed. Referring now to FIGS. 10, 11, and 12, there are
`shown preferred embodiments of the user interface 200
`when video and text, video only, and text only, respectively,
`are selected for display with controls hidden.
`The Always On Top item of Options menu 900 toggles the
`application to and from being TopMost in the Microsoft
`Windows Z-Order. When a window is TopMost, it always
`remains in view on top of all other open windows. The user
`may select Always On Top when the user does not want the
`multicasting application to be buried by other windows. A
`check mark is displayed next to the menu item when the
`Always On Top item is selected.
`
`26
`
`

`
`5,557,724
`
`7
`The Video Window item of Options menu 900 is used to
`display or hide video display 206 of user interface 200. For
`example,
`the user may choose to play only the audio
`component of a selected channel having both video and
`audio components. A check mark is displayed next to the
`Video Window menu item when video display 206 is visible.
`The Audio Controls item of Options menu 900 is used to
`display or hide audio controls 208 of user interface 200.
`Audio controls 208 preferably ca.nnot be hidden when
`neither video display 206 nor text reader bar 210 is visible.
`since nothing would be visible other than the window frame.
`As depicted in FIG. 7, audio controls 208 preferably has a
`fixed height, but may be sized from a minimum width of 120
`pi