USOO7007062B1
`
`US 7,007,062 B1
`(10) Patent N0.:
`(12) United States Patent
`
`Serenyi et al. Feb. 28, 2006 (45) Date of Patent:
`
`
`(54) METHODS AND APPARATUSES FOR
`TRANSFERRING DATA
`
`W0
`
`5/2000
`WO 00/27087 A1
`OTHER PUBLICATIONS
`
`(75)
`
`IHVCHIOFSI Denis Serenyi, MCHIO Park. CA (US);
`Chrls LeCroy, Mountain View, CA
`(US)
`
`(73) A551gnee: Apple Computer, Inc., Cupertmo, CA
`(US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 970 days.
`
`( * ) Notice:
`
`Carsten Griwodz, Alex Jonas, Michael Zink, “Affordable
`Infrastructure for Stream Playback in the Internet”, Darm-
`stadt University of Industrial Process and System Commu-
`nications
`(KOM), Dec.
`12,
`1999, Technical Report
`TR-KOM-1999-07, Darmstadt, Germany, (pp. 1-17).
`Henning Schulzrinne,
`Jonathan Rosenberg,
`“Internet
`Telephony: architecture and protocols—an IETF perspec-
`five,” Computer Networks, ElseV1er Sc1ence Publishers:
`Amsterdam, NL, vol. 31 No. 3, Feb. 11, 1999 (pp. 237-255).
`
`(22)
`
`Filed:
`
`Jan, 8, 2001
`
`Related US. Application Data
`
`Primary Examiner—Robert B. Harrell
`(74)Att0mey, Agent, or Firm—Blakely, Sokoloff, Taylor &
`Zafman LLP
`
`63
`
`)
`
`(
`
`Continuation-in- art of a
`filed on Jun. 22,p2000.
`
`pp
`
`lication No. 09/603,108,
`
`(57)
`
`ABSTRACT
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 13/00
`(52) us. Cl.
`...................................................... 709/203
`(58) Field of Classification Search .................. 705/14;
`709/231, 203
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`US PATENT DOCUMENTS
`5,918,020 A
`6/1999 Blackard et a1.
`6,263,371 B1
`7/2001 Geagan, III et a1.
`6,275,471 B1
`8/2001 Bushmitch et a1.
`6,415,327 B1
`7/2002 Beckerman et a1.
`6,505,169 B1 *
`1/2003 Bhagavath et a1.
`675467421 Bl
`4/2003 Wynbla“ 6t a1~
`6,625,258 B1
`9/2003 Ram et 211.
`6,629,138 B1
`9/2003 Lambert et 211.
`6,708,213 B1
`3/2004 Bommaiah et 211.
`6,771,644 B1
`8/2004 Brassil et a1.
`FOREIGN PATENT DOCUMENTS
`
`~~~~~~~~~~~ 705/14
`
`The present invention provides several methods and appa-
`ratuses for transmitting multimedia data using streaming
`media Protocols such as real-time transfer Protocols (RTP)
`and real-time streaming protocols (RTSP) in a computer
`network environment. A request for RTP data is sent from
`the caching proxy server to the server. The request may be
`for one specific type of data or multiple unrelated types of
`data. The server responds to the request
`indicating its
`support for the requested data. The caching proxy server
`determines Whether to proceed or terminate the data trans-
`mission process based on the response provided by the
`server. If it is determined to proceed With the data transmis-
`sion process, the caching proxy informs the server to send
`the requested data. The server sends the requested data in a
`body of a RTP packet. The RTP packet uses a RTP Meta-Info
`payload format, Which includes a body and a field header.
`The field header includes fields to identify the streaming
`media data, and the field body includes the requested stream-
`ing media data.
`
`W0
`
`WO 97/22201 A2
`
`6/1997
`
`19 Claims, 15 Drawing Sheets
`
`401
`
`DAGHING PROXY SERVER (GP)
`
`I'I ASSOCIATED WITH
`STREAMING MEDIA
`DATA 1
`
`FT ASSOCIATED WITH
`STREAMING MEDIA
`DATA 1
`
`IT ASSOCIATED WITH
`STREAMING MEDIA
`DATA 2
`
`FT ASSOCIATED WITH
`STREAMING MEDIA
`DATA 2
`
`
`
`Apple Exhibit 4428
`
`Apple v. SightSound Technologies
`CBM2013-00023
`
`Page 00001
`
`1101
`
`I102
`
`TRANSMISSION/RECEIPT
`CONTROL SOFTWARE
`(ES. FOR RTSP AND RTP
`1104
`STREAMING MEDIA
`STREAMING MEDIA
`DATA 1
`DATA 2
`(IN STREAMING
`(IN STREAMING
`MEDIA FORMAT)
`MEDIA FORMAT)
`
`
`
`Apple Exhibit 4428
`Apple v. SightSound Technologies
`CBM2013-00023
`Page 00001
`
`

`

`US 7,007,062 B1
`
`Page 2
`
`OTHER PUBLICATIONS
`Henning Schulzrinne, et al. “RTP: A transport protocol for
`real-time applications,” Network Working Group RFC 1889
`RTP, Jan. 1996 (pp. 1-75).
`
`PCT International Search Report for PCT Int’l Appln No.
`U801/20044’ mailed May 22’ 2002 (8 pages).
`
`* cited by examiner
`
`Page 00002
`
`Page 00002
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 1 0f 15
`
`US 7,007,062 B1
`
`DRIGINATING SERVER AND CACHING PROXY SERVER(CP)
`NEGOTIATE, THROUGH REAL-TIME STREAMING PROTOCOL
`(RSTP) FOR SERVER TO SEND AND CP SERVER TO RECEIVE
`STREAMING MEDIA DATA (FOR EXAMPLE: OUICKTIME
`STREAMING AUDIO DATA).
`
`CONTROLLED BY THE CP SERVER.
`
`CP SERVER RECEIVES STREAMING MEDIA DATA. USING REAL
`TIME TRANSFER PROTOCOL (RTP), IN STREAMING MEDIA
`FORMAT NOT THE SAME FORMAT AS STORED ON DRIGINATING
`SERVER AND OF SERVER STORES THE STREAMING MEDIA DATA.
`IN THE STREAMING MEDIA FORMAT, ON A STORAGE DEVICE
`
`CP SERVER RECEIVES REQUEST. THROUGH RSTP. FROM A
`CLIENT FOR THE STREAMING DEDIA DATA AND RETRIEVES THE
`DATA FROM THE STORAGE DEVICE AND TRANSMITS THE
`STREAMING MEDIA DATA TO THE CLIENT SYSTEM.
`
`FIG. 1A
`
`(Prior Art)
`
`Page 00003
`
`Page 00003
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 2 0f 15
`
`US 7,007,062 B1
`
`100
`
`\
`
`122
`
`102
`
`CLIENT
`
`SYSTEM
`
`104
`
`c MPU
`
`SYSTEM
`
`106
`
`108
`
`
`
`
`130
`
`132 INTERNET
`
`
`
`OF SERVER
`
`12
`
`
`COMPUTER m “2-CP“53"“
`
`
`
`
`-ATEWAY_YSTEM
`
`112
`
`126
`
`WEB
`
`SERVERSYSTEM
`
`
`
`114
`
`116
`
`NETWORK
`INTERFACE
`
`NETWORK
`INTERFACE
`
`11C
`
`120
`
`CLIENT
`COMPUTER
`SYSTEM
`
`CLIENT
`COMPUTER
`SYSTEM
`
`FIG. 13
`
`Page 00004
`
`Page 00004
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 3 0f 15
`
`US 7,007,062 B1
`
`252
`
`255
`
`PROCESSOR
`
`MEMORY
`
`
`
`
`l
`253
`262
`' 254
`
`
`
`
`
`
`
`DISPLAY
`CONTROLLER
`
`MASS
`MEMORY
`
`DISPLAY
`
`260
`
`DIGITAL PROCESSING
`SYSTEM 25.0
`
`CONTROLLER
`
`266
`
`W
`DEVICES
`
`MODEM 0R
`NETWORK
`INTERFACE
`
`FIG. 2
`
`Page 00005
`
`Page 00005
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 4 0f 15
`
`US 7,007,062 B1
`
`301
`
`302
`
`CLIENT(C)
`
`SERVER(S)
`
`RTSP 81 RTP
`
`FIG. 3
`
`301
`
`401
`
`302
`
`CLIENT(C)
`
`CACHING PROXY
`SERVER (CP)
`
`SERVER(S)
`
`RTSP & RTP
`
`FIG. 4
`
`Page 00006
`
`Page 00006
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 5 0f 15
`
`US 7,007,062 B1
`
`
`
`CP ASKS SETUP IN RTSP
`FOR AUDIO DATA AND SPECIFIES
`EXTENSIONS TO RTP DAT
`FOR AUDIO DATA
`
`501
`
`SERVER RESPONDS/NEGOTIATES
`
`502
`
`
`
`GP ASKS SETUP IN RTSP
`FOR VIDEO DATA AND SPECIFIES
`EXTENSIONS TO RTP DATA
`
`503
`
`SERVER RESPONDS/NEGOTIATES
`
`504
`
`GP ASKS PLAY IN RTSP
`
`SERVER RESPONDS WITH RTP DATA
`FOR ALL TRACKS WITH NEGOTIATED
`
`EXTENSIONS
`
`505
`
`506
`
`FIG. 5
`
`Page 00007
`
`Page 00007
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 6 0f 15
`
`US 7,007,062 B1
`
`momSq5m
`
`:33“5$>Ew
`
`bibs;
`
`EEMESE
`
`m3:
`
`HEEmioE<35
`
`wz=2<mEm0mmx8mm0mmP0mm0Now
`ozmmC..E.ttxNFo<55«53<55<205w
`
`
`
`5.55m52me5.55mEEEm
`
`newnoE.mumprw
`
`
`
`95on.203
`
`cd.”—
`
`Page 00008
`
`Page 00008
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 7 0f 15
`
`US 7,007,062 B1
`
` CP ASKS FOR TRANSMIT TIME
`
`INFORMATION FROM THE SERVER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`705
`
`
`
`706
`
`___________ ,__
`l
`I
`CLIENT MAY ALSO
`'
`I
`ASK FOR PLAY 0“
`I
`NI OTHER DATA DIRECTLY I
`711
`I
`PROM SERVER
`I
`----------- T -‘
`
` SERVER RECEIVES THE REQUEST FOR -----------------------------
`TRANSMIT TIME (TT)
`709
`
`
`SERVER DOES NOT RESPOND TO
`
`
`SERVER RESPONDS AND SENDS TT
`REQUEST. THIS INDICATES THAT
`
`IDENTIFIER CODE. WHICH
`SERVER DOES NOT SUPPORT THE
`
`
`CORRESPONDS. TD TT TYPE OF
`EXTENSION REQUESTED
`
`
`EXTENSION, IN AN EXTENSIBLE
`
`HEADER FORMAT - STREAMING DATA
`
`IS SENT WITH TT EXTENDED DATA
` CP MAY TERMINATE PROCESS
`
`710
`IF TI" IS CRITICAL ELEMENT
`
`
` -EXTENSIBLE HEADER FORMAT IS A
`OF THE CP OPERATION
`
`
`SUB-EXTENSION CONTAINING A SUB-
`
`
`EXTENSION ID, IDENTIFYING THE
`CONTENT OF THE INFORMATION.
`-TT SUB-EXTENSION ALSO
`CONTAINS A SINGLE 64-BIT
`
`
`UNSIGNED INTEGER REPRESENTING
`
`
`RECOMMENDED TRANSMISSION
`
`
`TIME OF RTP PACKET IN
`
`
`MILLISECONDS
`
`
`
`
`CP GETS THETI' DATA, REMOVES
`EXTENSION HEADER AND STORES THE
`DATA LOCALLY;TTT|IVIES ARE
`ASSOCIATED WITH APPROPRIATE
`
`
`STREAMING DATA
`
`.
`I
`
`CLIENT REQUESTS STREAMING DATA
`
`
`BY MAKING A PLAY REQUEST IN
`RTSP
`
`
`
`Page 00009
`
`Page 00009
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 8 0f 15
`
`US 7,007,062 B1
`
`CP ASKS FOR TRANSMIT TIME
`INFORMATION FROM THE SERVER
`
`SERVER RECEIVES THE REQUEST FOR —————————————————————————————
`FRAME TIME (FT)
`
`
`
`
`
`807
`
`
`SERVER DOES NOT RESPOND TO
`
`
`
`SERVER RESPONDS AND SENDS FT
`REQUEST THIS INDICATES THAT
`
`
`IDENTIFIER CODE, WHICH
`SERVER DOES NOT SUPPORT THE
`
`
`
`CORRESPONDS, TO FITYPE OF
`EXTENSION REQUESTED
`
`
`
`EXTENSION. IN AN EXTENSIBLE
`HEADER FORMAT - STREAMING DATA
`
`
`IS SENT WITH FT EXTENDED DATA
`CP MAY TERMINATE PROCESS
`
`808
`IF FT IS CRITICAL ELEMENT
`
`
`
`-EXTENS|BLE HEADER FORMAT IS A
`OF THE CP OPERATION
`
`
`
`SUB-EXTENSION CONTAINING A SUB-
`EXTENSION ID, IDENTIFYING THE
`
`
`CONTENT OF THE INFORMATION.
`FT SUB-EXTENSION ALSO
`
`CONTAINS 64-BIT UNSIGNED INTEGER
`
`
`VALUE WITH WELL-KNOWN FRAME
`TYPES SUCH AS O-UNKOWN FRAME,
`1-KEY FRAME, 2-B FRAME, AND
`
`
`3-P FRAME. FUTURE VERSIONS
`
`CAN ADD MORE FRAMES.
`CLIENT INFORMS
`
`
`CP THAT IT IS
`OVERLOADED
`
`.” ---------- 1— —.
`WITH
`
`I
`CLIENT MAY ALSO
`:
`INFORMATION
`
`
`
`I
`ASK FOR FT OR
`.
`CP GETS THE I—T DATA, REMOVES
`
`EXTENSION HEADER AND STORES THE
`N. OTHER DATA DIRECTLY I
`
`DATA LOCALLY; FI' FRAMES ARE
`810 I
`FROM SERVER
`,
`
`ASSOCIATED WITH APPROPRIATE
`'- —————————— T -1
`STREAMING DATA
`
`
` CP REVIEWS CLIENT REQUEST STOPS
`SENDING LEAST IMPORTANT FRAMES
`AND SENDS ONLY HIGHER
`IMPORTANCE FRAMES
`
`----------------------------
`
`
`
`FIG. 8
`
`Page 00010
`
`Page 00010
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 9 0f 15
`
`US 7,007,062 B1
`
`CP ASKS BY NAME FOR ONE OR MORE RTP
`SUB-EXTENSIONS USING RTSP PROTOCOL
`
`0
`9 1
`
`
`
`
`
`
`
`
`SERVER RESPONDS BACK TO CP INDICATING ITS
`SUPPORT FOR THE REQUESTED SUB-EXTENSIONS.
`SERVER ALSO TRANSMITS T0 OF AN IDENTIFIER (E.G.,
`NUMBER CODE) CORRESPONDING TO EACH NAME
`OF RTP EXTENSION. OF MAY LATER USE THE CODE
`WHEN RECEIVING EXTENDED DATA. RESPONSE
`MAY OR MAY NOT BE A COMPLETE RESPONSE.
`
`
`
`I
`CP RECEIVES SERVER'S RESPONSE
`
`CP DETERMINES WHETHER SERVER RESPONDED
`TO ALL REQUESTED SUB-EXTENSIONS
`
`903
`
`
`
`
`
`
`
`905
`
`907
`
`
`919
`
`NO
`SERVER DID NOT RESPOND TO ALL
`REQUESTED SUB-EXTENSIONS
`
`909
`
`
`
`911
`
`
`913
`
`
`
`YES
`SERVER RESPONDED TO ALL
`
`
`REQUESTED SUB-EXTENSIONS
`
`
`I
`
`9 1
`CP REQUESTS SERVER TO SEND ALL
`2
`ARE ANY OF THE MISSING SUB-
`
`
`
`SUPPORTED STREAMING MEDIA DATA
`EXTENSIONS CRITICAL TO CP
`
`
`ASSOCIATED THE RTP SUB-EXTENSIONS
`PROCESSES?
`
`
`
`
`
`SERVER SENDS ALL SUPPORTED RTP SUB-
`
`
`EXTENSIONS AND STREAMING MEDIA DATA
`
`
`
`ASSOCIATED WITH THE RTP SUB-
`CP DECIDES WHETHER
`
`EXTENSIONS
`TO TERMINATE OPERATIONS
`
`
`
`
`
`
`
`CP STORE SUB-EXTENSION DATA AND
`CP DECIDES TO PROCEEI
`CP
`
`
`
`STREAMING MEDIA ASSOCIATED WITH THE
`AND REQUEST THE
`TERMINATES
`
`
`
`
`OPERATIONS
`RTP SUB—EXTENSION LOCALLY
`REMAINING/SUPPORTED
`
`
`SUB-EXTENSIONS
`
`
`917
`915
`
`GP MAY REMOVE SUB-EXTENSION FIELDS
`
`927
`
`
`GP EVALUATES CLIENT REQUESTS AND
`PERFORMS FUNCTIONS ACCORDINGLY.
`
`FOR EXAMPLE FRAME THINNING OR USING
`
`TRANSMIT TIME OPTIONALLY TO TRANSMIT
`DATA
`
`
`
` 929
`
`931
`
`FIG- 9
`
`Page 00011
`
`Page 00011
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 10 0f 15
`
`US 7,007,062 B1
`
`New
`
`Ohnommczbz
`
`music”.2...;
`
`Ewfio
`
`smkuqm
`
`t§ma<E
`
`m3:
`
`2552:mummumEVE
`
`E?E5:§<Ewas:
`
`Emfio28525mw>5.bzo
`
`mQmemo
`
`S.d."—
`
`mm>mmw
`
`
`
`mEon.733
`
`noEmmmohm
`
`Page 00012
`
`Page 00012
`
`
`
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 11 0f 15
`
`US 7,007,062 B1
`
`401
`
`CACHING PROXY SERVER (CP)
`
`DATA 2
`
`TRANSMISSION/RECEIPT
`CONTROL SOFTWARE
`(E.G. FOR RTSP AND RTP
`
`STREAMING MEDIA
`DATA 2
`
`(IN STREAMING
`MEDIA FORMAT)
`
`TI" ASSOCIATED WITH
`STREAMING MEDIA
`DATA 2
`
`FTASSOCIATED WITH
`STREAMING MEDIA
`
`STREAMING MEDIA
`DATA 1
`
`(IN STREAMING
`MEDIA FORMAT)
`
`TT ASSOCIATED WITH
`STREAMING MEDIA
`DATA 1
`
`FT ASSOCIATED WITH
`STREAMING MEDIA
`DATA 1
`
`FIG. 11
`
`Page 00013
`
`Page 00013
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 12 0f 15
`
`US 7,007,062 B1
`
`301
`
`SERVER (ORIGINATING SERVER)
`
`EXTENSIONS
`
`TRANSCEIVER
`CONTROL SOFTWARE
`
`RTP EXTENSIONS
`
`NAME
`TT
`FT
`
`QQDE
`01
`02
`
`SOFTWARE FOR
`TRANSMISSION AND
`CONTROL OF RTP
`
`ORIGINAL DATA FILE
`FOR MEDIA
`(IN ORIGINAL
`DATA F0 R MAT)
`
`STREAMING MEDIA
`CONVERSION
`SOFTWARE (CONVERTS
`ORIGINAL FILE TO
`
`STREAMING FORMAT)
`
`FIG. 12
`
`Page 00014
`
`Page 00014
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 13 0f 15
`
`US 7,007,062 B1
`
`302
`
`CLIENT
`
`SOFHNARE
`
`130‘l
`
`STREAMING MEDIA
`DATA PLAYER
`SOFTWARE
`
`WEB BROWSER
`
`‘303
`
`FIG. 13
`
`Page 00015
`
`Page 00015
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 14 0f 15
`
`US 7,007,062 B1
`
`I
`field dataf 14“
`-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
`
`l
`I I I
`—+-+-+-+-+-+~+—+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+—+
`cl
`field name
`I
`field Ian
`I
`-+-+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
`field data
`I
`-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+«+-+-+-+—+-+-+-+«+
`
`--+—+—+——+—+
`
`FIG. 14A
`
`Page 00016
`
`Page 00016
`
`

`

`US. Patent
`
`Feb. 28, 2006
`
`Sheet 15 0f 15
`
`US 7,007,062 B1
`
`1450
`
`f
`
`1456
`
`1453
`
`0
`
`1454012
`
`1
`
`2
`
`3
`
`5678952345678912345678901
`
`“4+4 +-+-+-+—+-+- +-+-+-+-+-+-+-+-+-
`-+-+-+-+-+-+-+-+—+-+-+
`lclfield ID
`lfield Ian
`I
`f
`|
`+-+-+-+-+-+-+-+-+-+-+-+-+- -+-+-+-+-+-+-+-+-+-+-+-
`
`
`1452
`1451
`+-+-+-+-+-+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+
`|c| field ID
`1 field Ien
`|
`field data
`|
`I
`0..
`I
`
`-+-+-+-+-+-+
`
`FIG. 14B
`
`1452
`14 1
`.
`+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+
`
`3
`2
`1
`0
`01234567890123456789-012345678901
`+-+-+-+-+-+-+-+-+—+-+-+-+—+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+-+-+-+
`lclfield ID
`Ifield Ian
`I
`field data
`|
`+-
`-
`-+
`-+-+-+-+- -+-+-+-+-+-+-+-+-+-+-+- -+-+-+-+-+-+
`
`1401 { |c|
`field name
`I
`field Ien
`l
`1409{ I
`field data
`+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
`I
`DID
`I
`+-+-+-+-+-+-+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
`lcl field ID
`1 field Ian
`I
`field data
`|
`+-+-+-+-+-+-+-+-+-+-+—+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+—+~+-+ '
`
`+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+—+-+-+
`
`‘
`
`|
`
`FIG. 146
`
`Page 00017
`
`Page 00017
`
`

`

`US 7,007,062 B1
`
`1
`METHODS AND APPARATUSES FOR
`TRANSFERRING DATA
`
`This application is a continuation-in-part of prior US.
`patent application Ser. No. 09/603,108, which was filed Jun.
`22, 2000 entitled “Methods and Apparatuses for Transfer-
`ring Data” to Denis Serenyi, and commonly assigned to the
`assignee of the present invention, the disclosure of which is
`expressly and fully incorporated herein by reference.
`
`FIELD OF THE INVENTION
`
`The present invention relates to the field of multimedia
`data transmission. In particular, the present invention in one
`exemplary embodiment relates to multimedia data transmis-
`sion of real-time transfer protocol (RTP) packets using real
`time streaming protocol (RTSP) in a computer network
`environment.
`
`INTRODUCTION AND BACKGROUND OF THE
`INVENTION
`
`2
`mation allows a caching proxy server to provide smooth
`packet delivery to its client by knowing the time an RTP data
`packet was intended to be sent, and type of video frame that
`is being sent without knowing the specific payload format.
`Another problem with the current approach is that it is not
`able to provide multiple pieces of unrelated data in one
`delivery to the caching proxy server. Furthermore, packets
`from the server may be “lost” and never reach the caching
`proxy server.
`In addition,
`there is normally no way to
`recreate a complete “pristine” copy at the caching proxy
`server.
`
`Prior art servers communicate RTP information to the
`
`caching proxy server by sending information through a
`cache-control header.
`In one approach, a cache-control
`header contains normal header fields. In another approach,
`unrelated to cache control of RTP information, a single type
`of additional information has been added to the normal fields
`
`5
`
`10
`
`15
`
`20
`
`in a header extension format without specifying the type of
`additional information. In this approach only a single piece
`of RTP extension can be added to the normal field of the
`
`Methods of transmitting data are commonly known and
`performed today on a routine basis to send various multi-
`media data such as text, graphics, audio, video, images etc.
`across computer networks situated in various parts of the
`world. Generally the transmission process requires both
`hardware and software for performing its function. Typi-
`cally, the hardware includes various types of personal com-
`puters and hand held multimedia data sending or receiving
`devices. These devices run under the control of an operating
`system and utilize multimedia application software pro-
`grams. As is known in the art, streaming media data is data
`which is transmitted to a receiving computer system and
`presented (usually after buffering temporarily at the receiv-
`ing system) and then discarded (not stored) at the receiving
`system.
`in form of packets from one
`Currently, data is sent
`multimedia device to another. Alarge amount of information
`is required to be sent in a real-time manner in the data
`packets, which imposes a heavy load on the systems.
`Streaming media data, such as Real-Audio data in streaming
`media format specified by Real-Networks, is sent through
`the Internet is near real-time manner in many cases.
`In one approach, the components involved in data trans-
`mission of streaming media are known to be a server (which
`may be referred to as originating server), a caching proxy
`server and a client. These components in various combina-
`tions communicate with each other for transmitting data
`packets in real-time. The communication link that currently
`exists between the components uses real-time transfer pro-
`tocols (RTP) and real-time streaming protocols (RTSP) to
`communicate and send packets to each other. For this
`approach to work, a caching proxy server needs to commu-
`nicate with the system server, receive a stream of RTP data
`packets, and transfer the information contained within the
`RTP data packets to a client. FIG. 1a shows an example of
`a prior method in which a caching proxy server receives
`streaming media data and provides this data to a client. In
`order to perform its function properly and efficiently, the
`caching proxy server needs several pieces of information
`from the server to be able to cache an RTP stream easily and
`reliably.
`Aproblem with the current approach is that it is not able
`to provide some of the key required information such as data
`packet transmit time and video packet frame type informa-
`tion that a caching proxy needs to be efficient. This infor-
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`header and sent at any one time.
`limited, non-extensible
`A problem with using this
`approach is that a server is not able to attach multiple sets of
`unrelated data at a time to send to the caching proxy server.
`Another problem with this approach is that
`the header
`extension used in these methods are still not able to provide
`all the information a caching proxy server needs to cache a
`stream properly and to transmit the stream properly. Yet
`another problem with this approach is that there is no way
`to identify the particular extension independently of other
`possible extensions.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides several methods and appa-
`ratuses for transmitting multimedia data using streaming
`media protocols such as real-time transfer protocols (RTP)
`and real-time streaming protocols (RTSP) in a computer
`network environment. In one exemplary embodiment, a
`request for RTP data is sent from the caching proxy server
`to the server. The request may be for one specific type of
`data or multiple unrelated types of data. The server responds
`to the request indicating its support for the requested RTP
`data. The caching proxy server determines whether to pro-
`ceed or terminate the data transmission process based on the
`response provided by the server. If it
`is determined to
`proceed with the data transmission process,
`the caching
`proxy informs the server to send the requested and supported
`RTP data. The server sends the requested data in the body of
`a packet. The RTP packet uses a RTP Meta-Info payload
`format, which includes a body and a field header. The field
`header includes fields to identify the streaming media data,
`and the field body includes the requested streaming media
`data.
`
`In another embodiment, the caching proxy server requests
`and receives packet transmit time data and/or packet frame
`type data from the server. The caching proxy server uses the
`frame type data to communicate with the client and supply
`frames based on client’s capacity to handle loads at given
`times. Transmit time data is also used by the caching proxy
`to store packets locally and deliver these packets at appro-
`priate times to the client for a smooth packet delivery.
`Other features and advantages of the present invention
`will be apparent from the accompanying drawings, and from
`the detailed description, which follows below.
`
`Page 00018
`
`Page 00018
`
`

`

`3
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`4
`DETAILED DESCRIPTION
`
`US 7,007,062 B1
`
`The present invention is illustrated by way of example
`and not limited by the figures of the accompanying drawings
`in which like references indicate similar elements and in
`which:
`
`FIG. 1a is a flowchart which shows a method in the prior
`art for transferring streaming media data to caching proxy
`server and then to a client;
`FIG. 1b illustrates a network of computer systems in
`which media data may be exchanged and/or processed,
`according to one embodiment of the present invention;
`FIG. 2 illustrates a block diagram of an exemplary digital
`processing system, which may be used in accordance with
`one embodiment of the present invention;
`FIG. 3 illustrates one embodiment of a communication
`
`10
`
`15
`
`method between a server and a client using RTSP and RTP
`protocols;
`FIG. 4 illustrates another embodiment of a communica-
`
`20
`
`tion method between a server, caching proxy server and a
`client;
`FIG. 5 illustrates one embodiment of a RTSP, RTP nego-
`tiation process between a caching proxy and a server;
`FIG. 6 illustrates one embodiment of a relationship
`between the server, caching proxy, and client during a
`transfer of a Transmit Time (TT) sub-extension to the
`caching proxy server and its use of TT information in
`transmitting streaming data to a client;
`FIG. 7 illustrates one embodiment of process that takes
`place during transfer of a transmit
`time sub-extension
`between server and caching proxy server;
`FIG. 8 illustrates one embodiment of process that takes
`place during transfer of a frame type sub-extension between
`server, and caching proxy server;
`FIG. 9 is a flow diagram of one embodiment of an
`operation to provide various types of information to a
`caching proxy in an extensible header format;
`FIG. 10 illustrates one embodiment of a relationship
`between the server, caching proxy, and client during a
`transfer of a Frame Type sub-extension;
`FIG. 11 illustrates a block diagram of a machine readable
`medium which stores executable computer program instruc-
`tion for execution by an exemplary caching proxy server,
`which may be used in accordance with one embodiment of
`the present invention;
`FIG. 12 illustrates a block diagram of a machine readable
`medium which stores executable computer program instruc-
`tion for execution by an exemplary originating server
`(server), which may be used in accordance with one embodi-
`ment of the present invention;
`FIG. 13 illustrates a block diagram of a machine readable
`medium which stores executable computer program instruc-
`tion for execution by an exemplary client, which may be
`used in accordance with one embodiment of the present
`invention;
`FIG. 14a illustrates an exemplary RTP data packet includ-
`ing a standard field header and a field body in accordance
`with one embodiment of the present invention;
`FIG. 14b illustrates an exemplary RTP data packet includ-
`ing a compressed field header and a field body in accordance
`with one embodiment of the present invention; and
`FIG. 14c illustrates an exemplary format of a combination
`RTP data packet 1470 in accordance with one embodiment
`of the present invention.
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`A method and system for providing multimedia data
`transmission using real-time transfer protocol (RTP) and real
`time streaming protocol (RTSP) are described. For purposes
`of explanation, numerous specific details are set forth in
`order to provide a thorough understanding of the present
`invention. For example, various computer network system
`architectures and digital processing system architectures are
`provided for illustrative purposes rather than to be construed
`as limitations of the present invention. It will be evident,
`however, to one skilled in the art that the present invention
`may be practiced without these specific details. In other
`instances, well-known structures and devices are shown in
`block diagram form to facilitate explanation.
`FIG. 1b is a diagram of a network of computer systems in
`which media data may be processed, according to one
`embodiment of the present invention. As shown in FIG. 1b,
`a number of client computer system, one or more of which
`may represent one implementation of a receiving system, are
`coupled together through an Internet 122. It will be appre-
`ciated that the term “Internet” refers to a network of net-
`
`works. Such networks may use a variety of protocols for
`exchange of information, such as TCP/IP, ATM, SNA, SDI,
`RTP, RTSP etc. The physical connections of the Internet and
`the protocols and communication procedures of the Internet
`are well known to those in the art. Access to the Internet 103
`
`is typically provided by Internet service providers (ISPs),
`such as the ISP 124 and the ISP 126, which may also be
`connected with caching proxy servers 130 and 132. Users on
`client systems, such as the client computer systems 102, 104,
`118, and 120, generally obtain access to the Internet through
`Internet service providers, such as ISPs 124 and 126, which
`may also be connected through the internet with caching
`proxy servers 130 and 132. Access to the Internet may
`facilitate transfer of information (e.g., email,
`text files,
`media files, etc.) between two or more digital processing
`systems, such as the client computer systems 102, 104, 118,
`and 120 and/or a streaming media server system 128 which
`may be considered an originating server from which caching
`proxy servers receive streaming media data. For example,
`one or more of the client computer systems 102, 104, 118,
`and 120 and/or the streaming media server 128 may provide
`media data (e.g., video and audio, or video, or audio) to
`another one or more of the client computer systems 102,
`104, 118, and 120 and/or the streaming media server 128.
`Such may be provided in response to a request. As described
`herein, such media data may be transferred in the system 100
`according tracks. Such tracks, in one embodiment of the
`invention, may be created according to a specific format of
`the streaming media data and/or a specific data communi-
`cation (e.g., network) protocol(s).
`The streaming media server 128 is typically comprised of
`at least one computer system to operate with one or more
`data communication protocols, such as the protocols of the
`World Wide Web, and as such, is typically coupled to the
`Internet 122. Optionally, the streaming media server 128
`may be part of an ISP which may provide access to the
`Internet and/or other network for client computer systems.
`The client computer systems 102, 104, 118, and 120 may
`each, with appropriate web browsing software, access data,
`such as HTML documents (e.g., Web pages), which may be
`provided by the streaming media server 128. Such data may
`provide media, such as QuickTime movies or QuickTime
`streaming media data, which may be presented by the client
`computer systems 102, 104, 118, and 120.
`
`Page 00019
`
`Page 00019
`
`

`

`US 7,007,062 B1
`
`5
`The ISP 124 provides Internet connectivity to the client
`computer system 102 via a modem interface 106, which may
`be considered as part of the client computer system 102. The
`client computer system may be a conventional computer
`system, such as a Macintosh computer, a “networ ” com-
`puter, a handheld/portable computer, a Web TV system, or
`other types of digital processing systems (e.g., a cellular
`telephone having digital processing capabilities). Similarly,
`the ISP 126 provides Internet connectivity for the client
`computer systems 104, 118 and 120, although as depicted in
`FIG. 1b, such connectivity may vary between various client
`computer systems, such as the client computer systems 102,
`104, 118, and 120. For example, as shown in FIG. 1b, the
`client computer system 104 is coupled to the ISP 126
`through a modem interface 108, while the client computer
`systems 118 and 120 are part of a Local Area Network
`(LAN). The interfaces 106 and 108, shown as modems 106
`and 108, respectively, in FIG. 1b, may be an analog modem,
`an ISDN modem, a cable modem, a satellite transmission
`interface (e.g., “Direct PC”), a wireless interface, or other
`interface for coupling a digital processing system, such as a
`client computer system, to another digital processing sys-
`tem. The client computer systems 118 and 120 are coupled
`to a LAN bus 112 through network interfaces 114 and 116,
`respectively. The network interfaces 114 and 116 may be an
`Ethernet-type, Asynchronous Transfer Mode (ATM), or
`other type of network interface. The LAN bus is also
`coupled to a gateway digital processing system 110, which
`may provide firewall and other Internet-related services for
`a LAN. The gateway digital processing system 110, in turn,
`is coupled to the ISP 126 to provide Internet connectivity to
`the client computer systems 118 and 120. The gateway
`digital processing system 110 may, for example, include a
`conventional server computer system. Similarly, the stream-
`ing media server 128 may, for example, include a conven-
`tional server computer system.
`The system 100 may allow one or more of the client
`computer systems 102, 104, 118, and 120 and/or the stream-
`ing media server 128 to provide media data (e.g., video and
`audio, or video, or audio) to another one or more of the client
`computer systems 102, 104, 118, and 120 and/or the stream-
`ing media server 128. Such data may be provided, for
`example, in response to a request by a receiving system,
`which may be, for example, one or more of the client
`computer systems 102, 104, 118, and 120.
`FIG. 2 is a block diagram of an exemplary digital pro-
`cessing system which may be used in accordance with one
`embodiment of the present
`invention. For example,
`the
`digital processing system 250 shown in FIG. 2 may be used
`as a client computer system, a streaming media server
`system, a conventional server system, etc. Furthermore, the
`digital processing system 250 may be used to perform one
`or more functions of an Internet service provider, such as the
`ISP 124 or 126. The digital processing system 250 may be
`interfaced to external systems through a modem or network
`interface 268. It will be appreciated that the modem or
`network interface 268 may be considered as part of the
`digital processing system 250. The modem or network
`interface 168 may be an analog modem, an ISDN modem,
`a cable modem, a token ring interface, a satellite transmis-
`sion interface, a wireless interface, or other interface(s) for
`providing a data communication link between two or more
`digital processing systems.
`The digital processing system 250 includes a processor
`252, which may represent one or more processors and may
`include one or more conventional types of such processors,
`such as a Motorola PowerPC processor, an Intel Pentium (or
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`
`x86) processor, etc. A memory 255 is coupled to the pro-
`cessor 252 by a bus 256. The memory 255 may be a dynamic
`random access memory (DRAM) and/or may include static
`RAM (SRAM). The processor may also be coupled to other
`types of storage areas/memories (e.g., cache, Flash memory,
`disk, etc.), which could be considered as part of the memory
`255 or separate from the memory 255.
`The bus 256 further couples the processor 252 to a display
`controller 258, a mass memory 262, the modem or network
`interface 268, and an input/output (I/O) controller 264. The
`mass memory 262 may represent a magnetic, optical, mag-
`neto-optical, tape, and/or other type of machine-readable
`medium/device for storing information. For example, the
`mass memory 262 may represent a hard disk, a read-only or
`writable optical CD, etc. The display controller 258 controls
`in a conventional manner a display 260, which may repre-
`sent a cathode ray tube (CRT) display, a liquid crystal
`display (LCD), a plasma display, or other type of display
`device. The I/O controller 264 controls I/O device(s) 266,
`which may include one or more keyboards, mouse/trackball
`or other pointing devices, magnetic and/or optical disk
`drives, printers, scanners, digital cameras, microphones, etc.
`It will be appreciated that the digital processing system
`250 represents only one example of a system, which may
`have many different configurations and architectures, and
`which may be employed with the present invention. For
`example, Macintosh and Intel systems often have multiple
`busses, such as a peripheral bus, a dedicated cache bus, etc.
`On the other hand, a network computer, w