(12) United States Patent
`Graham
`
`USOO6732183B1
`(10) Patent No.:
`US 6,732,183 B1
`(45) Date of Patent:
`May 4, 2004
`
`(54) VIDEO AND AUDIO STREAMING FOR
`MULTIPLE USERS
`
`(75) Inventor: John J. Graham, Saratoga, CA (US)
`
`(73) ASSignee: Railway styles Inc.,
`uperuno,
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`(21) Appl. No.: 09/565,563
`9
`(22) Filed:
`May 4, 2000
`
`Related U.S. Application Data
`pp
`(63) Satyali in part of application No. 09/001,850, filed on
`(60) Pisional application No. 60/034,128, filed on Dec. 31,
`
`(51) Int. Cl. ................................................ G06F 15/16
`(52) U.S. Cl. ........................................ 709/231; 348/722
`(58) Field of Search ................................. 709/245, 203,
`709/231; 370/401, 352, 395, 229; 348/722;
`345/327
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`5228,859 A 7/1993 R
`5247,670 A o 1993 Rinaga
`5390,240 A 2/1995 Sensney
`5512938 A 4/1996 Ohno
`5,528.282 A 6/1996 Voeten et al.
`5,602.992 A 2/1997 Daneels
`5,606,693 A 2/1997 Nilsen et al.
`5,668.986 A 9/1997 Nilsen et al.
`5,668.996 A : 9/1997 Radinsky
`5,724,355 A 3/1998 Bruno et al. ................ 370/401
`5,751,280 A 5/1998 Abbott et al.
`5,764,750 A * 6/1998 Chau et al. ................. 379,229
`5,764,916 A 6/1998 Busey et al.
`5,778,181 A 7/1998 Hidary et al.
`
`5,802,526 A 9/1998 Fawcett et al.
`5,815,662 A 9/1998 Ong
`5,828,839 A 10/1998 Moncreiff
`5,842,015 A 11/1998 Cunniff et al.
`5,844,553 A 12/1998 Hao et al.
`5,861,881. A
`1/1999 Freeman et al. ............ 345/302
`5,864,555 A * 1/1999 Mathur et al. .............. 370/395
`5,880,792 A * 3/1999 Ward et al. ................. 348/722
`5,884,032 A 3/1999 Bateman et al.
`5,889,506 A 3/1999 Lopresti et al.
`5,889,942 A 3/1999 Orenshteyn
`5,898,780 A 4/1999 Liu et al.
`5,905,524. A
`5/1999 Sauer
`5,905,872 A * 5/1999 DeSimone et al. ......... 709/245
`5,922,054 A
`7/1999 Bibayan
`5,928,327 A
`7/1999 Wang et al.
`5,928,330 A 7/1999 Goetz et al.
`(List continued on next page.)
`OTHER PUBLICATIONS
`Paek et al, Video Server Retrieval Scheduling for Variable
`Bit Rate Scalable Video, IEEE 1996.*
`List continued
`t
`(List continued on next page.)
`Primary Examiner-Marc D. Thompson
`Assistant Examiner Thong Vu
`(74) Attorney, Agent, or Firm-Cooley & Godward LLP
`(57)
`ABSTRACT
`A computer System provides Streaming media to a plurality
`of clients over a network. The System includes a proxy
`module that forwards Streaming media from a first Streaming
`media Source to a network Server module that is communi
`cating to the plurality of client terminals. The proxy module
`is comunicatable with the network server module to be
`Signaled a Switch request. In response to the Switch request,
`the proxy module forwards Streaming media from a Second
`Streaming media Source to the network Server module while
`the network Server module is signaling Streaming media
`from the first Streaming media Source to the plurality of
`client terminals
`
`7 Claims, 7 Drawing Sheets
`
`-130
`-180
`?
`- Client Side-- Communications > Server Side
`io
`:
`78 Streaming Audio and WideoData
`vs.
`Stygia
`:
`138-
`f re 0.
`08
`Display
`76
`-20'ao
`
`s
`
`3.
`
`10 La
`
`O
`
`17-a-CE
`
`Browser
`
`2
`
`78
`
`Browser
`
`. N, w
`
`A
`
`2
`
`...' .
`
`104
`video
`Display
`
`108
`
`Internet
`
`GE
`
`
`
`
`
`Real-Time Server
`
`40
`
`44
`
`if
`f
`
`48
`
`9
`
`9
`
`
`
`Mike
`
`Line. In
`
`HTP Connection
`to Regte Server
`
`170 - 175
`Streaming Audio and
`Wideo data
`
`Netflix v. GoTV
`IPR2023-00758
`Netflix Ex. 1010
`
`

`

`US 6,732,183 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,930,493 A 7/1999 Ottesen et al.
`5,953,005 A
`9/1999 Liu
`5,956,716 A 9/1999 Kenner et al.
`5,960,173 A 9/1999 Tang et al.
`6,006.241 A 12/1999 Purnaveja et al.
`6,009,410 A 12/1999 LeMole et al.
`6,018,768 A 1/2000 UIlman et al.
`6,020,931 A * 2/2000 Bilbrey et al. .............. 34.8/584
`6,025,837 A 2/2000 Matthews, III et al.
`6,055,522 A 4/2000 Krishna et al.
`6,091,407 A * 7/2000 Boetje et al. ............... 345/327
`6,151,609. A 11/2000 Truong
`6,222,843 B1 * 4/2001 Mauger ...................... 370/397
`6.298,120 B1 10/2001 Civinlar et al.
`6,452.933 B1
`9/2002 Duffield et al.
`OTHER PUBLICATIONS
`Huang et al, The Mercuri Multimedia Laboratory at Hon
`eywell, IEEE 1996.*
`
`Liu et al., Perfomance of a Storage System for Supporting
`Different Video Types and Qualities, IEEE 1996.*
`Server Products User's Guide, Graham Technology Solu
`tions, Inc. www.graham.com/products/manual. 1998.*
`Paek et al. “Video Server Retrieval Scheduling for Variable
`Bit Rate Scalable Video", IEEE 1996, ps. 108-112.
`Huang et al., “The Mercuri Multimedia Laboratory at Hon
`eywell", IEEE 1996, pp. 72-77.
`Liu et al., Performance of a Storage System for Supporting
`Different Video Types and Qualities, IEEE 1996, pp.
`1314-1331.
`Instructions for using Netscape Navigator, downloaded from
`Netscape website/.../NetHelp/netscape/navigator/nav.html,
`printed on Dec. 14, 1997, 67 pages.
`Video Server Architectures: Performance and Scalability. -
`Bernhardt, et al. (1994) fp.eurecom.fr/ATM/papersEURE
`COM/PAPERS/brestBernhardt.ps.g2.
`* cited by examiner
`
`

`

`U.S. Patent
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`68. I88 I
`
`- - - - - - mn
`
`w
`
`
`
`pIBO oºp?A
`
`Ipp. I
`
`69. I
`
`------------------------------------------------
`
`801I I I
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 2 of 7
`
`US 6,732,183 B1
`
`Web Server Supplies Decoding Instructions
`
`220
`
`230
`
`Client Executes Instructions
`
`240
`
`250
`
`Client Makes Audio Request
`
`Client Makes Video Request
`
`242
`
`252
`
`Web Server Makes Audio Request
`
`Web Server Makes Video Request
`
`244
`
`246
`
`Yes
`
`
`
`248
`
`Yes
`
`254
`
`256
`
`258
`
`Client Decompresses Audio Data and
`Plays Audio
`
`Client Decompresses Video Data and
`Plays Video
`
`241
`
`251
`
`Continue?
`
`No
`
`Stop Audio
`
`FIG. 2
`
`Continue?
`
`No
`
`Stop Video
`
`

`

`
`
`
`
`YU]uonsspg
`
`Oo
`
`
`
`
`
`U.S. Patent
`U.S. Patent
`
`May 4, 2004
`May 4, 2004
`
`Sheet 3 of 7
`Sheet 3 of 7
`
`US 6,732,183 B1
`US 6,732,183 B1
`
`O3oOOOOoOoOooOrd
`
`
`JOSMOIGonGASenCpUl2'5OCie;
`]ZOEziia7;:::;UECeRaeRCLSGopsuyousmeyedyinp
`
`
`
`
`MOPUIA\
`_
`
`
`HOISSIWSUBITtanPAdtyssappayjo2}80jouy2a1STD
`
`POE.fouononsuouapaayY
`amesmoetaaWaaraeoeereaeeee
`==SS
`
`
`
`jasbgMolAdlgSOWagOsplAMAN,
`BINITEDUSBI$6SMOPULA,
`
`
`BISUIE)J3}US7)J9]{JO4OOY
`
`BiawWEDsJayenbpeaySLO
`
`BIOUTe)$6SMOPUTANWUaWeWINO],yen
`
`
`Bioure)JepeedPlasqe’]jeuoneNsuuosIy
`
`ALWOMINSANIeIaU]
`
`gioure7)due]eae]
`XOGINTAUTH
`
`(Gswo1pewoard
`xoqaynyo1pny
`
`DawOPeY2ArT
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 4 of 7
`
`US 6,732,183 B1
`
`} } }
`
`-
`A.
`All
`
`- 3. i
`
`Z08,
`
`
`
`(1.
`
`TO?E?TIJOS
`
`^ * * * * * * * * * _.',
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`May 4, 2004
`May 4, 2004
`
`Sheet S of 7
`Sheet 5 of 7
`
`
`
`oopla[eNsiq
`
`keidsiq
`
`Zos
`
`ayOOplA
`
`ors
`
`¢Dld
`§ 15)IH
`
`ayeyOIpny
`
`zis
`
`d01879OFPZ8)Clan
`
`add
`
`ava
`
`escS3>
`
`US 6,732,183 B1
`US 6,732,183 B1
`
`JOSMOIG]eeesearaeSeemsitiespteeeeregstLeaeae
`MOPUTAA-a
`J0}09]9§‘SOUNDoFMTG
`of9&&OORDA 10}93]2¢
`ne
`TESSOTECEon700BERSTEAYSIDAHwoNoApuoREpU2aLOGY
`
`CLSSeSEEESaECSYSHOUMOS=SNCALYdHODaWondvWausS0WwaG
`
`=eaeraweaSFaeOSolaehBPsaeOe
`
`
`
`zae),|
`
`oa.
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 6 of 7
`
`US 6,732,183 B1
`
`I
`
`
`
``-sco 0
`
`• /
`
`929
`
`pº 9829
`
`OJCI
`
`pº IBUIS
`
`IZ9
`
`ZZ9
`
`929
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 7 of 7
`
`US 6,732,183 B1
`
`710
`
`720
`
`730
`
`740
`
`PROVIDE AUDIENCE
`WITH STREAMING MEDIA
`
`PROXY SERVER
`RECEIVES COMMAND TO
`SWITCH
`
`PROXY SERVER MAKES
`SWITCH TO NEW
`STREAMING MEDIA
`SOURCE
`
`
`
`
`
`
`
`PROVIDE AUDIENCE
`WITH STREAMING MEDIA
`FROMNEW SOURCE
`
`FIG. 7
`
`

`

`1
`VIDEO AND AUDIO STREAMING FOR
`MULTIPLE USERS
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`This continuation-in-part appliction claims priority from
`the U.S. patent application Ser. No. 09/001,850, filed Dec.
`31, 1997, entitled Video And Audio Streaming For Multiple
`Users, which claims priority to U.S. Provisional Patent
`Application No. 60/034,128, filed Dec. 31, 1996, both of the
`aforementioned applications are herein incorporated by ref
`CCCC.
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`This invention relates to the field of network delivery of
`audio and Video information. In particular, the invention
`relates to a System for delivering audio and Video informa
`tion to multiple users.
`2. Background Information
`The Internet enables many different ways of communi
`cating. Originally, the Internet was used for the exchange of
`files and electronic mail. AS the capabilities of the Internet
`expand, other types of communications are enabled.
`Audio and Video transmissions are an important area of
`the communications that the Internet enables. For example,
`many technologies Support the transmission of digital video
`and/or audio signals over the Internet. An example of Such
`a technology is Quicktime TM, available from Apple
`Computer, Inc., of Cupertino Calif. Quicktime movies are
`files that can be transmitted across the Internet. Quicktime
`provides both audio and video displays. Many other file
`formats allow audio and Video to be displayed on people's
`computers.
`This paragraph describes an example use of a the Quick
`time technology. A user will have a browser application that
`resides on his/her computer. The computer, acting as a client
`under the direction of the browser application, will connect
`to various World Wide Web (web) servers. Each web server
`will typically serve hypertext markup language (HTML)
`files to the clients. The files may include text, graphics
`references, and references to specialized files. Some of these
`Specialized files can include audio and Video information in
`Quicktime form. The clients can then play these audio and
`Video files once they are downloaded using a Quicktime
`plug-in, a helper application, or Quicktime capabilities built
`into the browser application. A plug-in and a helper appli
`cation are described in greater detail below.
`Streaming audio and Video, as a Subset of all the types of
`audio and Video that can be transmitted over the Internet,
`allow people to broadcast long and/or live video and audio
`transmissions acroSS the Internet. Streaming Video and audio
`is Video and audio digital data that is transmitted on a
`continuous basis. A client can access the data Stream and
`regenerate the Video images and audio signal as they are
`being transmitted. Streaming technology is particularly
`helpful where the events are live, or where the files would
`be So large as to be a burden on the end users. Examples of
`where Streaming technology is particularly useful are for the
`display of conferences, Sporting events, radio broadcasts,
`television broadcasts, and the like.
`RealNetworks, Inc. of Seattle, Wash., provides a system
`for transmitting Streaming audio and Video signals to users
`over the Internet. RealNetworks supplies a server that allows
`multiple users to Simultaneously receive Streaming audio
`and Video.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,732,183 B1
`
`2
`The real audio System requires that not only the client
`have additional Software, but that the content provider have
`a separate Server from their normal Web Server. For a client
`to receive a real audio broadcast, the client typically con
`nects through their browser to a Web page with a reference
`to a real audio Server. The client then accesses its Separate
`real audio player program. The real audio player program
`then connects to the referenced real audio Server. A signifi
`cant drawback to Such an arrangement is that the user must
`download the real audio player program. This program must
`then be installed on the user's computer. This may cause a
`number of problems for the user. For example, if the user is
`behind a firewall, or Some Security program, the client may
`not be able to receive the broadcast from the server.
`Additionally, the installation of any program may have
`conflicts with other programs. The program has the disad
`Vantage of being platform Specific. This means that a dif
`ferent program must be developed and downloaded for each
`type of computer that is to be used to acceSS RealNetworks
`broadcasts. Additionally, the broadcasters of the Streaming
`audio and video need to use the RealNetworks server, which
`is separate from the broadcasters’ World Wide Web server
`(also referred to as the web server). This increases the
`broadcasters Security problems because now the broadcast
`erS must be concerned with two Separate Servers.
`Another example of a Video and audio System that uses
`Internet like communications is the MBone. The MBone is
`a specialized communication network that allows for the
`distribution of Streaming Video and audio signals to multiple
`users. A specialized network is Set up specifically to transmit
`MBone communications. A Significant drawback of this
`System is that users must be connected to the Specialized
`network. Additionally, users will be required to have spe
`cialized Software on their computers to listen to and watch
`MBone transmissions.
`A Streaming video System, not requiring a user to down
`load a separate program, was developed for a single user by
`John Graham of California. This single user broadcast
`technology allowed a Web Server to Serve a single Streaming
`Video signal to a Single client. Although the user did not need
`to download a plug-in to see the Video, only one user was
`allowed to access the Video Stream at a time. In this System,
`Video information was captured from a Video camera and
`digitized. The digital video information was then encapsu
`lated in a MIME encoded multipart data stream. The client
`received this data Stream and reconstructed frames of the
`digital Video.
`Therefore, what is desired is a platform independent video
`and audio Streaming System that does not require the user to
`download additional programs beyond the functionalities
`found in a browser.
`
`1. A SUMMARY OF THE INVENTION
`A System and method of providing Streaming audio and
`Video data to multiple users is described. In one
`embodiment, the System comprises a first client, a Second
`client and a Server. The first and Second clients are executing
`browsers. The Server can communicate with the two clients.
`The Server concurrently provides Streaming audio and Video
`data to both of the clients. Importantly, the server does not
`require the two browsers to use a plug-in or a helper
`application to receive and use the Streaming audio and Video
`data.
`In Some embodiments of the invention, a browser causes
`a client to request an HTML file from a web server. The
`client receives the HTML file. The HTML file includes an
`
`

`

`3
`HTML tag that directs the browser to load one or more
`applets from the Server. The browser executes the applets
`causing the browser to request Streaming audio and Video
`from the Web Server. That request may or may not include
`parameters giving information about the type of request
`being made. The Web Server associates a Server process with
`the request, given the parameters in the request. The web
`Server notifies real-time audio and Video process that Stream
`ing audio and Video information is needed. In response to the
`notification, the real-time audio and Video proceSS Stores
`encoded audio and Video data in a shared memory location.
`The Server process accesses the shared memory and inserts
`the audio and Video data into one or more data Streams. The
`client receives the data Streams and reconstructs the audio
`and Video Signals using only the capabilities of the browser.
`In Some embodiments, a separate Stream and Server proceSS
`is used for each of the audio and video data. These embodi
`ments allow multiple clients to Simultaneously receive the
`Same audio and Video data.
`Other embodiments of the invention include a web server
`that can Serve Streaming audio and Video information as well
`as perform more usual web server functions (such as,
`Serving web pages, performing file transferS, Supporting
`Secure communications). These embodiments have the
`advantage of allowing the broadcasters and the users to Set
`up their Security configurations for one web server, rather
`than two servers (a web server and a streaming audio and
`video server).
`Although many details have been included in the descrip
`tion and the figures, the invention is defined by the Scope of
`the claims. Only limitations found in those claims apply to
`the invention.
`2. A BRIEF DESCRIPTION OF THE DRAWINGS
`The figures illustrate the invention by way of example,
`and not limitation. Like references indicate Similar elements.
`FIG. 1 illustrates a system including one embodiment of
`a streaming audio and Video System for multiple users where
`client computers do not need plug-ins or helper programs.
`FIG. 2 illustrates an example method of Streaming audio
`and Video for multiple users.
`FIG. 3 illustrates a web page that a user can use to acceSS
`a streaming audio and Video broadcast.
`FIG. 4 illustrates a web page for selecting the bandwidth
`of the user's Internet connection.
`FIG. 5 illustrates a web page having Streaming audio and
`video.
`FIG. 6 is a block diagram under another embodiment of
`the invention, illustrating a proxy Switching System
`FIG. 7 is a flow chart for a process to use proxy Switching
`for providing Streaming media to client terminals, under an
`embodiment of the invention.
`3. THE DESCRIPTION
`The following sections describe embodiments of the
`invention. The first section provides definitions that will help
`in the understanding of the remaining Sections. The Second
`Section shows an example System that Supports various
`embodiments of the invention. The third section describes an
`example method of using the invention. The fourth Section
`illustrates an actual use of the Streaming audio and Video
`used in Some embodiments of the invention. The last Section
`reviews additional alternative embodiments of the invention.
`(a) Definitions
`The following definitions will help in the understanding
`of the following description.
`
`1O
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,732,183 B1
`
`4
`Client: a computer, process, program, or combination of
`computers, processes or programs, that can communicate
`with a server.
`Server: a computer, process, program, or combination of
`computers, processes or programs, that can communicate
`with a client. The Server and the client can be executing on
`the same computer(s).
`Web server: a server for serving at least Internet related
`requests. Example web servers can Serve HTML pages in
`response to HTTP requests from clients. Some web servers
`can serve many different kinds of requests, e.g., HTTPS,
`and/or FTP.
`Browser: an application, program, process, or combina
`tion of applications, programs or processes, that allow a
`client to make a request of a web server and process the
`results of the request. The browser may be part of a Stand
`alone application or a set of programs that are integrated into
`the operating System of the client.
`Plug-in: plug-ins are external Software programs that
`extend the capabilities of the browser in a specific way. For
`example, a plug-in can be used to give the browser the
`ability to play audio Samples or view Video movies from
`within the browser.
`Helper application: like plug-ins, helper applications are
`external Software programs. The browser redirects Some
`types of file to the helper applications. The helper applica
`tions allow clients to proceSS many different types of files on
`the Internet. When the browser encounters a Sound, image,
`or video file, the browser hands off the data to the helper
`applications to run or display the file.
`JavaScript: a Stand-alone programming language built
`into many browsers. Primarily an extension to the Internet
`standard HTML language.
`Java: a programming language whose programs (called
`applets) can travel over the Internet for use by clients.
`Whereas plug-ins and helper applications are tailored for a
`particular type of client, Java applets generally work acroSS
`the Internet regardless of the type of client. Many browsers
`include Java capabilities So Java applets require no instal
`lation. Except where noted, Java and JavaScript are inter
`changeable.
`
`(b) System
`FIG. 1 illustrates a system including one embodiment of
`the invention where audio and Video is Supplied over the
`Internet to multiple users. The following paragraphs first list
`the elements of FIG. 1, describe their interconnections, and
`then describe the various elements in greater detail.
`This paragraph lists the elements of FIG. 1. FIG. 1
`includes a System having three parts: a client Side 100, a
`communications interface 180, and a server side 130. The
`client side 100 includes a client 112 and a client 111. The
`client 112 includes a browser 102 having a video display
`area 103. The client 111 includes a browser 108 and a video
`display area 104. The communications interface 180
`includes the Internet 185. The serverside 130 includes a web
`server 131, a shared memory 135, and a real-time server
`140. The web server 131 includes two processes, a process
`138 and a process 139. The real-time server 140 includes a
`video module 144, an audio module 146, a video proxy 148,
`and an audio proxy 149. The real-time server 140 interfaces
`with a number of other elements. These elements include a
`video card 159, an input audio interface 162, and an HTTP
`connection to remote server 170. Various elements of FIG.
`1 have now been listed.
`
`

`

`US 6,732,183 B1
`
`15
`
`25
`
`35
`
`40
`
`S
`The following paragraphs describe the interconnections
`between the elements of FIG.1. Beginning on the serverside
`130, the video card 159 receives a video input 158 outputs
`a digital video Signal to the Video module 144. Similarly, an
`input audio interface 162 receives a mike input 164 and/or
`a line input 166 and outputs a digital audio signal to the
`audio module 146. The HTTP connection to remote server
`170 receives a streaming audio and video data 175. The
`HTTP connection to remote server 170 outputs the video
`data to the video proxy module 148 and the audio data to the
`audio proxy module 149. The real-time server 140 uses the
`data received by the various modules and Stores portions of
`that data in the shared memory 135, after some manipulation
`of the data. The shared memory 135 is accessed by the web
`server 131. The process 138 and the process 139 transmit
`and receive data to the communications interface 180. Thus,
`the couplings of the server side 130 have been described. In
`Some embodiments, the process 138 and the process 139
`correspond to HTTPD processes.
`The communications interface 180 allows the client side
`100 to communicate with the server side 130. The commu
`nications interface, and the Internet 185 in particular, Sup
`port many different types of connections by the client Side
`100 and the serverside 130. In the example communications
`interface 180, the communications interface 180 includes
`the Internet 185. In particular, in this example, the process
`138 is communicating streaming audio and video data 176
`to the Internet 185. Similarly, the process 139 is communi
`cating the streaming audio and video data 178 to the Internet
`185.
`On the client side 100, the client 112 is communicating
`the streaming audio and video data 176 with the Internet
`185. Similarly, the client 111 is communicating with the
`Internet 185 to receive and manage the Streaming audio and
`video data 178. The clients then communicate the informa
`tion to their respective browser applications. The browser
`applications the generate Video images in their respective
`Video display areas.
`Thus, the connections between the various elements of
`FIG. 1 have been described. Now the various elements will
`be described in greater detail in the following paragraphs.
`An example method of using these elements is described
`below in relation to FIG. 2.
`The server side 130 will be described first.
`The video input 158 represents a video signal that a user
`of Such a System wishes to broadcast to the clients on the
`client side 100. The video input 158 can include analog
`signals representing video information. The video card 159
`digitizes the video input 158 to produce a digital video
`image. Various embodiments of the invention include Sun
`video cards available from Sun Microsystems, of Mountain
`View, Calif., and Parallax XVideo Xtra TM video cards,
`available from Parallax Graphics, Inc., of Santa Clara, Calif.
`However, what is important is that the real-time server 140,
`and in particular the Video module 144, receives Some Sort
`of digitized video signal. The video module 144 is respon
`sible for providing the real-time server 140 with the video
`information that will be broadcast to the client side 100. The
`Video module 144 can convert the digitized Video signals to
`be of better use to the rest of the system. For example, the
`video module 144 may, if not done by the video card 159,
`convert digital video data into a Sequence of JPEG digital
`Video frames. In any case, what is important is that the
`real-time server 140 receives digital video information in a
`format that it can use (example formats include, JPEG,
`MPEG, GIF, and AVI).
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`Similarly, the input audio interface 162 allows for the
`input of analog audio signals and converts this input to
`digital audio signals. What is important is that the audio
`module 146 receives a digitized audio signal that can be used
`by the real-time server 140. The audio module 146 may
`convert the digitized audio Signal into any of a number of
`formats, corresponding to any of a number of audio trans
`mission rates (examples include G.711 and G.723 audio
`compression formats).
`The HTTP connection to remote server 170 represents an
`important advantage of one embodiment of the invention. In
`this embodiment, the HTTP connection to remote server 170
`allows the server side 130 to forward broadcasts of audio
`and Video signals from other Streaming audio and Video
`servers. In these uses, the server side 130 acts as a client to
`another server. The HTTP connection to remote server 170
`can receive Video and audio Signals being broadcast through
`the Internet 185 from another server. The HTTP connection
`to remote server 170 provides the digital video information
`from the other server to the video proxy module 148.
`Similarly, the HTTP connection to remote server 170 pro
`vides the audio data to the audio proxy module 149. The
`video proxy module 148 and the audio proxy module 149
`then Supply the respective Video and audio data to the
`real-time server 140.
`The real-time Server 140 represents an application, or Set
`of applications, executing on one or more computers, that
`prepares audio and Video data for broadcasting to multiple
`users through the web server 131. The real-time server 140
`takes the data from the various modules, processes the data,
`and stores the processed data in the shared memory 135. The
`real-time Server 140 can perform compression, and other
`manipulations of the data, to reduce the processing burden
`on the web server 131. For example, in some embodiments
`of the invention, the real-time server 140 receives digitized
`Video data and compresses that data into JPEG images.
`These JPEG images are Sequenced digital frames of Video.
`Similarly, for the audio data, the real-time server 140 breaks
`the audio information into one-half Second time periods of
`audio data (other embodiments use other time periods).
`These one-half Second time periods of data are Stored in the
`shared memory 135. The real-time server 140 can also
`compress the audio information into one of a number of
`various compressed audio signals (e.g., G.711 and/or G.723
`audio compression formats). In Some embodiments of the
`invention, the real-time Server can broadcast audio and Video
`from multiple Sources to multiple clients.
`The shared memory 135 represents a shared Storage area
`for use by the real-time Server to Store audio and Video data
`for access by the web server 131. In one embodiment the
`shared memory 135 has a locking and Semaphore usage
`scheme to ensure that the real-time server 140 is not writing
`data into the shared memory 135 while the web server 131
`is accessing that data. In Some embodiments, the Sema
`phores act as notifiers to indicate that new data in the shared
`memory 135 is available for use by the web server 131. In
`Some embodiments, the Video data and the audio data are
`Stored in different shared memory locations.
`The web server 131 communicates data over the Internet
`185 using one or more communications protocols. In Some
`embodiments of the invention, these protocols include
`HTTP (Hypertext Transfer Protocol), TCP (Transmission
`Control Protocol) and UDP (User Datagram Protocol). The
`Web Server 131 represents an application, including one or
`more processes, for communicating over the Internet 185. In
`one embodiment, the web server 131 includes an Apache
`web server. Each of the processes in the web server 131
`
`

`

`25
`
`7
`represents one or more processes for Serving Streaming
`audio and video data to the client side 100. In some
`embodiments, the web server 131 transmits the video data as
`a multipart MIME (multi-purpose Internet mail extensions)
`encoded file for decoding directly by the browser or as
`compressed Video information for decoding by an applet run
`in the browser. The web server 131 transmits the audio data
`as compressed audio data for decoding by an applet run in
`the browser.
`In some embodiments, the web server 131 initiates a
`Separate process for each audio and Video connection made
`from client side 100. Thus, for one client receiving stream
`ing audio and Video data, two processes would be started
`within the web server 131. The first process would supply
`Video data and the Second process would Supply audio data.
`The processes access the shared memory 135 and serve the
`data acroSS the Internet to the respective client.
`In some embodiments, the web server 131 initiates at least
`one process for each client. This provides important advan
`tages in Some embodiments of the invention. In particular,
`because the web server 131 is serving the data directly
`through processes it created, Server Side 130 users need not
`worry about Security issues beyond those already faced with
`their web server 131. Thus, these embodiments of the
`invention have a lower chance of interfering with client Side
`100 fire walls and have a lower chance of having a server
`side 130 security problem.
`Other embodiments of the invention include separate
`Common Gateway Interface (CGI) programs for audio and
`video. These CGI programs are used by the web server 131
`to Serve the Streaming audio and Video data. These CGI
`programs are not necessarily integrated as tightly to the web
`server 131 as the process 138 and the process 139. However,
`a CGI program allows for the easy extension of many
`different types of web servers.
`The communications interface 180 permits communica
`tions between the server side 130 and the client side 100. In
`this example, the Internet 185 Supports the communications.
`Other embodiments of the invention Support other commu
`nications interfaces. For example, the Internet 185 can be
`40
`replaced by a local area network, a wide are network, a
`proprietary telecommunications and networking
`infrastructure, or Some other communications interface.
`What is important is that the server side 130 can commu
`nicate with the client side 100. The communications inter
`45
`face 180 can also include combinations of the above
`described technologies. For example, the Internet 185 can
`include a web server to which the clients on the client side
`communicate through to access the Internet 185. The client
`side 100 can be on a local area network that is connected
`through a server, or router, to the Internet 185.
`The client side 100 represents the consumers of the
`Streaming audio and Video data. In this example, the two
`clients are receiving Separated Streaming audio and Video
`data Signals. Other embodiments of the invention Support
`many more clients.
`The client 112 represents a computer, such as a PC
`compatible computer, running a browser application 102.
`For video display, the browser application 102 can include
`a Netscape NavigatorTM or CommunicatorTM program for
`“multipart/xmixed-replace MIME type video,” or a
`Microsoft Internet ExplorerTM 3.0 or later for a Java based
`Video transmission. In Some embodiments, the Java based
`Video transmission applet parses the multipart/X-mixed
`replace MIME type video. For audio, the browser