US0070391 16B1
`
`(12) United States Patent
`Zhang et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,039,116 B1
`May 2, 2006
`
`(54)
`
`METHODS AND APPARATUS FOR
`EMBEDDING AND FORMAT CONVERSION
`OF COMPRESSED VIDEO DATA
`
`(75)
`
`Inventors: Ji Zhang, San Jose, CA (US);
`Humphrey Liu, Fremont, CA (US)
`
`(73)
`
`Assignee: Cisco Technology, Inc., San Jose, CA
`(US)
`
`(*)
`
`(21)
`(22)
`(51)
`
`(52)
`(58)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 674 days.
`Appl. No.: 09/708,163
`
`Filed:
`
`Nov. 7, 2000
`
`Int. C.
`H04N 7/2
`
`(2006.01)
`
`U.S. Cl. .................. 375/240.26; 709/247; 348/441
`Field of Classification Search ........... 375/240.26,
`375/240.28, 240.01, 240.16; 348/441, 454,
`348/458,512; 382/100, 243; 370/257,328;
`709/247
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,467,139 A * 11/1995 Lankford .................... 348,512
`5,577,042 A * 11/1996 McGraw et al. ......
`... 370/257
`5,729,535 A * 3/1998 Rostoker et al. ......
`... 370,328
`5,768,535 A * 6/1998 Chaddha et al. ............ 709/247
`6,507,618 B1* 1/2003 Wee et al. ............. 375,240.16
`6,553,150 B1 * 4/2003 Wee et al. ........
`... 382.243
`6,553,566 B1 * 4/2003 Grant et al. .................. 725/28
`6,687,384 B1
`2/2004 Isnardi ....................... 382/100
`* cited by examiner
`Primary Examiner Gims Philippe
`(74) Attorney, Agent, or Firm Beyer Weaver & Thomas,
`LLP
`
`(57)
`
`ABSTRACT
`
`System and methods are provided for modifying a com
`pressed video data stream to match the available video
`decoding capability of a target decoder. Compressed video
`content can be embedded in another compressed bitstream to
`allow more efficient usage of available channel bandwidth.
`Format conversion allows flexible transmission of video
`content even if the original format and available decompres
`sion capabilities are mismatched.
`
`33 Claims, 10 Drawing Sheets
`
`27O
`Ya
`
`
`
`input
`compressed
`video data
`276
`
`272
`
`
`
`format
`Converter
`
`
`
`
`
`embedder
`apparatus
`
`transmitter
`
`
`
`
`
`275
`
`277
`format converted
`compressed
`video data
`
`278
`transmission
`bitstream
`
`
`
`280
`
`IPR2018-01413
`Sony EX1033 Page 1
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 1 of 10
`
`US 7,039,116 B1
`
`14
`
`17
`
`15
`
`21
`
`18
`
`22
`
`16
`
`24
`
`25
`
`OuickTime
`compressed
`Video data
`
`c. d
`20 . E dat
`WCeO Cata
`
`23 26
`
`MPEG-2
`compressed
`video data
`
`network
`device
`
`27
`
`1O
`
`32
`
`34
`
`36
`
`Figure 1
`
`IPR2018-01413
`Sony EX1033 Page 2
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 2 of 10
`
`US 7,039,116 B1
`
`
`
`IPR2018-01413
`Sony EX1033 Page 3
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 3 of 10
`
`US 7,039,116 B1
`
`250 Ya
`
`input
`Compressed
`video
`data
`256
`
`
`
`
`
`
`
`
`
`
`
`252
`
`embedder
`apparatus
`
`transmitt
`e
`
`27O
`
`
`
`input
`compressed
`video data
`276
`
`
`
`
`
`
`
`
`
`260
`
`258
`transmission
`bitstream
`
`Figure 3A
`
`272
`
`
`
`
`
`format
`Converter
`
`embedder
`apparatus
`
`transmitter
`
`
`
`275
`
`277
`format converted
`compressed
`Video data
`
`278
`transmission
`bitstream
`
`280
`
`Figure 3B
`
`IPR2018-01413
`Sony EX1033 Page 4
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 4 of 10
`
`US 7,039,116 B1
`
`
`
`388
`
`39
`
`FIG. 4
`
`IPR2018-01413
`Sony EX1033 Page 5
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet S of 10
`
`US 7,039,116 B1
`
`402 asses areassesssssssssssssssssssear-ses assasssssssa sea as a sess as surrors--arr svyer 416
`
`: -butter
`- but
`4Q6;
`41 Ob
`-bu?e
`412C Y-414C
`408
`N410c
`u?e
`Plbufer
`rocessor -> buffer
`buffer
`412d N414d
`|
`N410d
`---------------------------- S-9-------------
`
`4Q4
`
`410a
`
`412a
`
`414a
`
`4.18
`
`/419
`
`s
`
`386
`
`.
`
`412b
`
`414b
`
`is scrambler-stransmitter
`3 ||
`:
`
`f
`
`Figure 5A
`
`
`
`Converter
`
`
`
`Figure 5B
`
`IPR2018-01413
`Sony EX1033 Page 6
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 6 of 10
`
`US 7,039,116 B1
`
`:c:85833 234
`
`—————_—----———----——-——_ ——-—-----—----------—---—~---—----‘--—--
`
`436 438
`
`FIG. 50
`
`
`
`llllllllllllllllllllllllllllllllllllllllllllllllll
`
`452
`
`
`
`“E:mEcBEoo
`
`_—_—-_-..-__-___.._J
`
`|PR2018—O1413
`
`Sony EX1033 Page 7
`
`IPR2018-01413
`Sony EX1033 Page 7
`
`
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 7 of 10
`
`US 7,039,116 B1
`
`600
`
`
`
`606
`
`608
`
`layer
`601
`
`PES
`
`PES payload
`
`604
`
`elementary stream
`
`602
`
`Figure 6A
`
`transport
`
`605
`
`elementary
`stream
`layer
`
`
`
`
`
`9021-903-904-905-906 -907 -908
`910
`912
`SEQU.D. GOP U.D. P.U.D.PICTURE DATAPPICTUREDATA
`MPEG
`elementary
`Video
`bitstream
`900
`
`
`
`
`
`9021-903 - 904
`905
`906 -907 - 908
`910
`912
`SEQUDGOP COMPR. VIDEO PUDPICTURE DATAPPICTURE DATA
`transmission
`elementary
`bitstream
`901
`
`Figure 6B
`
`IPR2018-01413
`Sony EX1033 Page 8
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 8 of 10
`
`US 7,039,116 B1
`
`650
`
`receive multiple compressed bitstreams
`
`652
`
`Convertformat for next
`compressed video bitstream?
`YES
`
`654
`
`
`
`embed compressed video data in
`transmission bitstream
`
`658
`
`another compressed bitstream
`to be embedded?
`
`66O
`
`NO
`
`transmit transmission bitstream
`
`662
`
`Figure 7
`
`IPR2018-01413
`Sony EX1033 Page 9
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 9 of 10
`
`US 7,039,116 B1
`
`704
`
`
`
`encoder
`
`
`
`
`
`
`
`offline
`processing:
`
`Figure 8
`
`IPR2018-01413
`Sony EX1033 Page 10
`
`

`

`U.S. Patent
`
`May 2, 2006
`
`Sheet 10 of 10
`
`US 7,039,116 B1
`
`
`
`:
`5
`
`i
`
`IPR2018-01413
`Sony EX1033 Page 11
`
`

`

`US 7,039,116 B1
`
`1.
`METHODS AND APPARATUS FOR
`EMBEDDING AND FORMAT CONVERSION
`OF COMPRESSED VIDEO DATA
`
`BACKGROUND OF THE INVENTION
`
`2
`the reverse schemes is called a decoder or decoding appa
`ratus. In the above example of a set-top box (STB) located
`in a residential home, the set-top box is typically installed
`with a static decoding capability. More specifically, the
`set-top box only decodes video content according to the
`video decompression formats that the set-top box hardware
`and Software are initially configured to manage. Any video
`content outside of the established decoding capability of the
`set-top box results in inaccessible video content services and
`programs for the residential user. Video content and pro
`grams refer to compressed video data transmitted by a
`network source or Supplier for reception, decompression and
`viewing by an end-user.
`Currently, there is no synchronization between video
`content Suppliers with respect to which video compression
`format is to be globally used. As the decompression capa
`bility of the residential user is typically static, the burden of
`facing the kaleidoscope of public and proprietary video
`compression formats must then be overcome at Some point
`between the video content suppliers and the residential user.
`The decompression burden becomes more unruly when
`residential users carry different decoding capabilities. Dif
`ferent decoding capabilities are common when different
`local cable companies Supplying the set-top boxes imple
`ment different decoding capabilities, or when a set-top box
`strictly includes proprietary decoding capabilities—at the
`expense of other proprietary and public decoding capabili
`ties. Also, it is important to point out that compressed video
`bitstreams are usually generated by Video content providers
`using either real-time encoders or pre-compressed video
`server storage systems. Both are likely to be in a remote site,
`away from the network and end-user. This increases the
`difficulty in encoding the video signal with a format suitable
`or tailored to the decoding capability of an end-user.
`In addition to format incompatibility problems between
`Video content Suppliers and end-users, modern video service
`also includes multiple programs to be simultaneously
`received by the end-user. The multiple video programs may
`often be too large for the available bandwidth on the
`channels used to transmit the programs from the Suppliers to
`the users. Overloading the available bandwidth may lead to
`degradation in transmission quality or unavailability of
`programs to end-users.
`Based on the foregoing, it should be evident that there is
`a need for methods and systems of providing compressed
`digital video content over data channels that fit within the
`available bandwidth and that comply with an end-user's
`decoding capability.
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`The present invention relates generally to systems and
`methods for transmitting data. More specifically, the present
`invention relates to systems and methods for embedding
`compressed digital video data in a bitstream, potentially
`containing additional compressed digital video data of a
`different compression format, and transmitting the bitstream
`over communication channels.
`Video services and programs are provided by a spectrum
`of different video content suppliers. For example, residential
`digital video services may include digital television, video
`OnDemand, Internet video, etc.—each service having hun
`dreds of programs. A set-top box (STB) located in a resi
`dential home receives video services and programs from a
`number of different video content suppliers via assorted
`transmission channels. Transmission channels such as
`coaxial cable distribution networks, digital subscriber loop
`(DSL) access networks, ATM networks, satellite, or wireless
`digital transmission facilities are all well known. Preferably,
`all digital video programs provided by Video content Sup
`pliers are available for all residential users.
`Since transmission of video data with existing commu
`nication channels often requires excessive bandwidth, com
`pression is an approach that has been used to make digital
`Video images more transportable. Digital video compression
`formats allow digitized video frames to be represented
`digitally in much more efficient manner. Compression of
`digital video makes it practical to transmit the compressed
`signal using digital channels at a fraction of the bandwidth
`required to transmit the original signal without compression.
`There is an abundant number of proprietary and public video
`data compression formats currently in use. Popular propri
`etary video compression formats include the Microsoft
`streaming format, QuickTime, RealNetworks, etc. Public
`Video data compression formats include numerous interna
`tional standards built upon video data compression schemes
`such as MPEG-1, MPEG-2, MPEG-4, H.261, H.262, H.263,
`H.263+, wavelet, fractal, etc.
`These standardized and proprietary compression formats
`rely on several algorithm schemes to compress the video
`data. The MPEG-2 standard, for example, includes motion
`compensated transform coding (for example, DCT trans
`forms or wavelet/sub-band transforms), quantization of the
`transform coefficients, and variable length coding (VLC).
`The motion compensated encoding removes the temporally
`redundant information inherent in Video sequences. The
`transform coding enables orthogonal spatial frequency rep
`resentation of spatial domain video signals. Quantization of
`the transformed coefficients reduces the number of levels
`required to represent a given digitized video. The other
`factor contributing to the compression is the use of variable
`length coding (VLC) so that most frequently used symbols
`are represented by the shortest code word. In general, the
`number of bits used to represent a given image determines
`the quality of the decoded picture. The more bits used to
`represent a given image, the better the image quality.
`Abitstream refers to a continuous sequence of binary bits
`used to digitally represent compressed video, audio or
`computer data. The system that is used to compress digitized
`Video data in an uncompressed bitstream using a compres
`sion scheme is called an encoder or encoding apparatus. The
`system that is used to decompress a video bitstream using
`
`50
`
`55
`
`60
`
`65
`
`SUMMARY OF THE INVENTION
`
`The present invention overcomes the deficiencies and
`limitations of the prior art with systems and methods for
`embedding and/or format conversion of compressed video
`data to match the available video decoding capability of a
`target decoder. The present invention also allows video
`content embedding of one or more compressed video data in
`another compressed bitstream to better use available channel
`bandwidth between a video content Supplier and a target
`decoder. Embedding may be further advantageous for target
`digital receiver/decoders having multiple format decompres
`sion capabilities. This may allow target digital receiver/
`decoders having multiple format decompression capabilities
`to decompress the multiple compressed video bitstreams
`simultaneously. The present invention may also include
`format conversion of the video content based on the avail
`able video decoding capability of a target decoder. Format
`
`IPR2018-01413
`Sony EX1033 Page 12
`
`

`

`US 7,039,116 B1
`
`3
`conversion allows flexible transmission of video content
`even when the original video data format and available
`decompression capability are mismatched.
`In one aspect, the invention relates to a network device for
`providing first compressed video data onto a network. The
`network device comprises an embedder apparatus having a
`first embedder portion which embeds first compressed video
`data having a first compressed format in a transmission
`bitstream having a second compressed format. The network
`device also comprises a transmitter that transmits the trans
`mission bitstream having the second compressed format, the
`transmission bitstream including the first compressed video
`data having the first compressed format.
`In another aspect, the invention relates to a network
`device for providing compressed video data onto a network.
`The network device comprises a compressed format con
`verter that receives first compressed video data having a first
`compressed format and outputs the first compressed video
`data having a second compressed format. The network
`device also comprises an embedder which receives the first
`compressed video data and second compressed video data
`and embeds one of the first compressed video data and
`second compressed video data into the other of the first
`compressed video data and second compressed video data.
`The network device further comprises a transmitter config
`ured to transmit the first and second compressed video data.
`In yet another aspect, the invention relates to a method for
`providing compressed video data. The method comprises
`embedding compressed video data having a first compressed
`30
`format in a transmission bitstream having a second com
`pressed format. The method also comprises transmitting the
`transmission bitstream having the second compressed for
`mat and containing the compressed video data.
`In still another aspect, the invention relates to a network
`device for transmitting compressed video data. The network
`device comprises means for receiving the compressed video
`data in a first compressed format. The network device
`further comprises means for converting the compressed
`Video data from the first compressed format to a second
`compressed format. The network device also comprises
`means for embedding the compressed video data having the
`second compressed format in a transmission bitstream hav
`ing a third compressed format. The network device addi
`tionally comprises means for transmitting the transmission
`bitstream having the third compressed format, the transmis
`sion bitstream including the compressed video data having
`the second compressed format.
`In another aspect, the invention relates to a computer
`readable medium including instructions for method for
`providing compressed video data. The instructions compris
`ing instructions for embedding compressed video data hav
`ing a first compressed format in a transmission bitstream
`having a second compressed format. The instruction further
`comprising instructions for transmitting the transmission
`bitstream having the second compressed format and con
`taining the compressed video data.
`These and other features and advantages of the present
`invention will be described in the following description of
`the invention and associated Figures.
`
`4
`FIG. 2 is a high-level block diagram of the decoder of
`FIG. 1 in accordance with one embodiment of the present
`invention.
`FIG. 3A is a high level block diagram of a system which
`embeds compressed video data having a first compressed
`Video data format in a transmission bitstream having a
`second compressed video data format in accordance with
`one embodiment of the present invention.
`FIG. 3B is a high level block diagram of a system which
`embeds compressed video data having a first compressed
`Video data format in a transmission bitstream having com
`pressed video data in accordance with another embodiment
`of the present invention.
`FIG. 4 illustrates a system for transporting compressed
`digital video data in accordance with one embodiment of the
`present invention.
`FIG. 5A illustrates the network device of FIG. 4 in more
`detail in accordance with one embodiment of the present
`invention.
`FIG. 5B illustrates the processor of FIG. 5A in more detail
`in accordance with one embodiment of the present inven
`tion.
`FIG. SC illustrates the format converter of FIG. SB in
`more detail in accordance with one embodiment of the
`present invention
`FIG. 6 illustrates a transmission bitstream having
`MPEG-2 format in accordance with one embodiment of the
`present invention.
`FIG. 7 illustrates a process flow for embedding a first
`bitstream into a second bitstream and converting the format
`of one bitstream including video data in the compressed
`domain in accordance with one embodiment of the present
`invention.
`FIG. 8 illustrates a general-purpose computer for non
`real-time embedding in accordance with one embodiment of
`the present invention.
`FIG. 9 illustrates a high-level block diagram of an exem
`plary general-purpose router Suitable for implementing the
`present invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`The present invention will now be described in detail with
`reference to a few preferred embodiments thereof as illus
`trated in the accompanying drawings. In the following
`description, numerous specific details are set forth in order
`to provide a thorough understanding of the present inven
`tion. It will be apparent, however, to one skilled in the art,
`that the present invention may be practiced without Some or
`all of these specific details. In some instances, well known
`process steps and/or structures have not been described in
`detail in order to not unnecessarily obscure the present
`invention.
`In some multimedia delivery systems, compressed video
`programs are delivered to a target digital receiver/decoder
`from numerous video program Suppliers via one or more
`digital transmission channels. Each of the programs may be
`carried within bitstreams having a different video compres
`sion format. In Such situations, the target digital receiver/
`decoder, which has a static decompression capability based
`on installation or most recent upgrade, must be able to
`decode the video content regardless of the compression
`format used to transmit the video content. In these cases, the
`present invention may apply format conversion to an origi
`nally compressed video bitstream so that the format of the
`
`5
`
`10
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 illustrates a system where format conversion and
`compressed video data embedding of the present invention
`is particularly advantageous is described in accordance with
`one embodiment of the present invention.
`
`65
`
`IPR2018-01413
`Sony EX1033 Page 13
`
`

`

`US 7,039,116 B1
`
`10
`
`5
`compressed video data matches the decompression capabil
`ity of the target digital receiver/decoder.
`In addition to format conversion to improve end-user
`compressed video data service, the present invention may
`also combine multiple compressed video bitstreams into a
`single bitstream. Embedding video data in this manner may
`reduce the total bit rate of the video data and may be
`advantageous for target digital receiver/decoders limited in
`their service by low upstream channel bandwidth. Embed
`ding may be further advantageous for target digital receiver/
`decoders having multiple format decompression capabili
`ties. As the decompression capabilities of the target digital
`receiver/decoders are usually known at Some point in net
`work transmission of the video content, the multiple com
`pressed video bitstreams may be combined based on the
`15
`decompression capabilities of the target decoder. This may
`allow target digital receiver/decoders having multiple format
`decompression capabilities to decompress the multiple com
`pressed video bitstreams simultaneously.
`The multiple compressed video bitstreams are combined
`by embedding one compressed video bitstream into a second
`compressed video bitstream that is to be transmitted—
`referred to herein as a transmission bitstream. Upon receiv
`ing the transmission bitstream including the multiple com
`pressed video data, each bitstream having a different format
`25
`may be processed separately to expedite decompression of
`the multiple bitstreams, or to provide simultaneous multiple
`video content reception. Each of the compressed video
`bitstreams combined into the transmission bitstream may be
`format converted to comply with the known decompression
`capabilities of the target digital receiver/decoder.
`The invention relates to a set of network devices. In one
`embodiment, each of the network devices is adapted to
`convert an incoming compressed bitstream to a bitstream
`with a format that is appropriate for a decoder that receives
`the bitstream. For example, if the incoming compressed
`bitstream contains video data in a QuickTime format, and
`the target decoder is solely capable of decompressing
`MPEG-2, the network device converts the original com
`pressed video data to have a format that matches the decoder
`40
`capability, namely MPEG-2. Similarly, if a second decoder
`solely has a wavelet decompression capability, the network
`device converts the original compressed QuickTime bit
`stream to have a format that matches the decoder capability,
`namely wavelet.
`In another embodiment, a set of network devices is
`adapted to embed one or more incoming compressed video
`bitstreams into a single bitstream to be transmitted to the
`target decoder. For example, if two incoming compressed
`video bitstreams have an MPEG-4 format and an MPEG-2
`format, respectively, and a targeted decoder is capable of
`MPEG-2 and MPEG-4 decompression, the network device
`may embed the MPEG-4 compressed video bitstream into
`the MPEG-2 compressed video bitstream. The resulting
`combined bitstream is then transmitted from the network
`device to the target decoder where the video content in each
`format may be separated, separately decompressed, and
`provided to the user. In addition, if one or more incoming
`compressed video bitstreams do not match the decompres
`sion capabilities of the target decoder, then the one or more
`incoming compressed video bitstreams may be format con
`verted to match the decompression capabilities of a targeted
`decoder—before combining into a single bitstream. For
`example, if two incoming compressed video bitstreams have
`a QuickTime format and an MPEG-2 format respectively,
`and a targeted decoder is capable of MPEG-2 and MPEG-4
`decompression, the network device may convert the Quick
`
`50
`
`6
`Time compressed video bitstream into an MPEG-4 com
`pressed video bitstream before embedding it into the
`MPEG-2 compressed video bitstream. The resulting com
`bined bitstream is then transmitted from the network device
`to the target decoder where the video content in each format
`may be separated and independently decompressed.
`Video content embedding and format conversion to the
`decompression capability of the target digital receiver/de
`coder results in more robust video content transmission,
`regardless of video content Supplier compression format and
`end-user decompression capability. This allows end-users to
`receive video content regardless of their decompression
`capabilities and improves video content reception flexibility.
`Video content embedding also allows improved use of
`limited channel bandwidth between video content suppliers
`and end users. Format conversion according to the present
`invention also advantageously permits video content Sup
`pliers to reach end-users normally inaccessible due to a
`compressed video format incompatibility.
`The present invention relates to compressed video data
`transmission on communication channels, and for the pur
`poses of the present application a channel is defined broadly
`as a connection facility to convey properly formatted digital
`information from one point to another. A channel may
`include some or all of the following elements: 1) physical
`devices that generate and receive the signals (modulator/
`demodulator); 2) medium that carries the actual signals; 3)
`mathematical schemes used to encode and decode the sig
`nals; 4) proper communication protocols used to establish,
`maintain and manage the connection created by the channel;
`5) storage systems used to store the signals such as magnetic
`tapes and optical disks. The concept of a channel includes
`but is not limited to a physical channel, but also logical
`connections established on top of different network proto
`cols, such as xDSL, ATM, IP, wireless, HFC, coaxial cable,
`Ethernet, Token Ring, etc.
`The channel is used to transport a bitstream, or a con
`tinuous sequence of binary bits used to digitally represent
`compressed video, audio or data. The bit rate is the number
`of bits per second that the channel is able to transport. A
`multiplex is a scheme used to combine bitstream represen
`tations of different signals, such as audio, video, or data, into
`a single bitstream representation. A remultiplex is a scheme
`used to combine bitstream representations of different mul
`tiplexed signals into a single bitstream representation.
`One particular communication channel that is becoming
`increasingly common is the cable modem channel. Such as
`that defined by the DOCSIS cable modem standard by
`CableLabs and the DAVIC cable modem standard by DVB.
`In DOCSIS based cable data networks, operators and digital
`service providers use the radio frequency spectrum in the
`coaxial cable network to deliver digital data signal in a 6
`MHz frequency slot. The aggregate data bit rate capacity of
`the slot may be shared by multiple, possibly different, types
`of information services. Cable modem service is one such
`information service that allows service operators to provide
`a number of two-way interactive digital services to end
`users. One particular digital service delivers a compressed
`digital video signal for video streaming or video on demand.
`Another particular communication channel that is becom
`ing increasingly common is networks running on IP and/or
`Asynchronous Transfer Mode (ATM) network protocols.
`Both network protocols are based on different transport
`protocols that can be used to transport data or multimedia
`bitstreams with a pre-specified quality of service. For
`example, ATM protocol specifies how data is first packetized
`into fixed sized data units, called cells. It also specifies how
`
`30
`
`35
`
`45
`
`55
`
`60
`
`65
`
`IPR2018-01413
`Sony EX1033 Page 14
`
`

`

`US 7,039,116 B1
`
`10
`
`15
`
`25
`
`30
`
`7
`Such a cell stream can be multiplexed, de-multiplexed,
`switched and routed between different locations to support
`end-to-end connections at given bit rate and/or quality of
`service (QOS).
`In another communication channel example, integrated
`services (IntServ), differential services (DiffServ), and Mul
`tiple Protocol Label Switch (MPLS) are used to establish a
`QOS for different network traffic. These services may use
`the Common Open Policy Service (COPS) for admission
`control. Internet Group Management Protocol (IGMP) and a
`variety of multicast routing protocols are also used to
`establish and tear down service on a multicast tree, which is
`commonly used for video transmission. Strictly speaking,
`these QOS protocols are an additional layer of protocol on
`top of IP protocol. Therefore, they are not examples of
`communication channels themselves, but an additional
`attribute of other communication channels that support IP
`protocol.
`In embodiment, the present invention converts the format
`of compressed video data to comply with the allowable
`decompression capability of a targeted decoder. This allows
`flexible transmission of video content even when the origi
`nal video data format provided by the central office, stream
`ing server or network device transmitting the video content
`and available decompression capability of the target decoder
`are mismatched. The present invention is also suitable for
`flexible video content combining of multiple compressed
`video bitstreams by embedding first compressed video data
`into a bitstream having second compressed video data.
`Referring now to FIG. 1, a system where format conversion
`and compressed video data embedding of the present inven
`tion is particularly advantageous will be described.
`FIG. 1 illustrates a system 10 for transmitting multiple
`compressed bitstreams including compressed video data to
`target decoders 32, 34 and 36 in accordance with one
`35
`embodiment of the present invention. The multiple com
`pressed video data bitstreams are produced by encoders 14,
`15 and 16, each of which produce a compressed bitstream
`including video data having a different compression format.
`The encoder 14 receives uncompressed video data and
`compresses the video data to provide compressed video data
`having a first format, namely QuickTime. The QuickTime
`compressed video data is then received by the transmitter 17
`and output over a medium 18. The transmitter 17 can be any
`one of a variety of those known in the art for DSL networks,
`for example. In a specific embodiment, the communication
`path is a ATM based communication system where the
`transmitter 17 transmits the QuickTime compressed video
`data to receiver 20.
`The encoder 15 receives uncompressed video data and
`compresses the video data to provide compressed video data
`having a second format, namely MPEG-4. The MPEG-4
`compressed video data is then received by the transmitter 21
`and output over a medium 22. The transmitter 21 can be any
`one of a variety of those known in the art for IP networks,
`for example. In a specific embodiment, the medium 22 is an
`Internet network where the transmitter 21 transmits through
`a series of nodes which route the MPEG-4 compressed video
`data to a receiver 23.
`The encoder 16 receives compressed video data in a
`proprietary format Such as Microsoft streaming, decom
`presses the compressed video data (decoder not shown in
`FIG. 1), and re-encodes the video data to provide com
`pressed video data having a third format, namely MPEG-2.
`For purposes herein, re-encoding refers to the process of
`performing decoding and then encoding to a compressed
`bitstream, either fully or partially, and with the same or a
`
`55
`
`8
`different format. Re-encoding is also commonly referred to
`as recoding or transcoding. The MPEG-2 compressed
`video data is then received by the transmitter 24 and output
`over a medium 25. The transmitter 24 can be any one of a
`variety of those known in the art, for example. In a specific
`embodiment, the medium 25 is a coaxial cable network
`where the transmitter 21 transmits the MPEG-2 compressed
`video data to a receiver 26 according to the digital video
`broadcast (DVB) specification.
`The compressed video bitstreams sent over the mediums
`18, 22 and 25 are received by their respective receivers 20,
`23 and 26 and input to the network device 27. Based on a
`request one or more of the target decoders 32.34 and 36, the
`network device 27 transmits any one or a combination of the
`compressed bitstreams. For example, the target decoder 32
`receives compressed video data from the network device 27
`through a local network 28. The local network 28 is included
`in a larger network 29 that also includes local networks 30
`and 31, which service the target decoders 34 and 36,
`respectively. In one embodiment, the encoder 14 is a central
`office, the network device 27 is a headend and the local
`network 28 corresponds to a geographic service area of a
`cable network serviced by the headed. The decoder 32 may
`then correspond to a decoder included in a set-top box or
`personal computer located in a residential home. The