WWWWWMWMM
`
` WWMWMWM
`
`U5006172988B1
`
`US 6,172,988 B1
`(12) United States Patent
`(10) Patent Ne.:
`Jan. 9, 2001
`’l‘iernan et al.
`(45) Date of Patent:
`
`(54) METHOD FOR UNIVERSAL MESSAGING
`AND h’IUIJl‘II’l.,EXlNG 0F VIDEO, AUDIO,
`AND DATA STREAMS
`
`(75)
`
`Inventors: James C. Tiernan, Solzina Beach;
`Christopher .1. Bennett. San Diego,
`both of (.‘A (us)
`
`(73) Assignee: Tiernan Communications, Ine., San
`Diego, CA (US)
`
`(
`
`'t‘) Notice:
`
`Under 35 U.S.C. 154(1)), the term ofthis
`patent shall be extended for 0 days,
`
`(21) Appl. No: 08/675,558
`(32)
`I’iled:
`Jul. 3, 1996
`
`(60)
`
`Related U .8. Application Data
`Provisional application No. 60/010,873, filed on Jan. 31,
`1906, and provisional application No. 60!0'l(),l94, filed on
`Apr. 25, 1996.
`
`Int. Cl.7 ....................................................... 1104,} 3/24
`(51)
`(52) US. Cl.
`........................
`370/473; 370/465
`
`(58) Field of Search .
`370/366, 474,
`370/469, 467, 535, 536, 537, 395, 389,
`396, 51.7, 538, 307, 473, 353, 41.9, 465,
`466, 4-98, 539, 541; 348/461, 467
`
`(56)
`
`References Cited
`US. PAI‘EN'I‘ DOCUMENTS
`
`4,933,934
`5,319,707
`5,381,181
`5,400,347 '
`5,400,411.11
`5,420,806
`
`......................... 370/531)
`6/1990 Aikoh et al.
`6/1094 Wasilewski ct al.
`380/14
`
`1/1995 Deiss
`348/423
`3/1995 Lee .................................... 371/375
`
`3/1995 Wasilcwski et a1.
`380/9
`370/1101
`5/1995 Wasilewski
`
`(List continued on next page.)
`FOREIGN PATENT DOCUMENTS
`
`22:23::
`0753954
`116/37009
`
`iii/i832 Eng
`[/1907 (131’).
`1111090 (WO)
`
`OTHER PUBLICATIONS
`
`'l‘., "Description of a
`(1., and Wechselberger,
`.l,
`'l‘iernan,
`Universal Packet Multiplex 'l‘ransport for the 'l'ransrnission
`for Broadcast Applications of Digital Services,
`including
`Video, Audio, and Data Programs," 'l‘eehniea] Proposal,
`International Organitation l‘or Standarimtion Organization
`Internationale de Normalization, pp,
`'1 —-»'14 (Jan, 9, 1993),
`“Implementation Guidelines for the Use of MPEG—2 Sys
`tems, Video and Audio in Satellite and Cable Broadcasting,
`Applications in L-urope, DVD—$1.8((1eneva)"pp. l~—‘l4 (Jun,
`1994).
`"Guidelines on Implementation and Usage of Service Infor—
`mation," TM1324 Rev. I, Final Draft, pp. 1~3O (Mar. 17,
`1995),
`“Information Technology—Generic Coding of Moving Pic—
`tures and Associated Audio Informationv—Part ‘1: Systems”,
`Draft International Standard ISO/IBC DIS 13818—1, pp.
`x—xix, 1041, 36-47, 64~65, 86-95, 100413 (1995).
`
`(list continued on next page.)
`
`Primary Ivfxrzmincr—J)ouglas W. ()lms
`Assistant EXaminer—Phirin Sam
`(74) Attorney, Agent, or Firm—Hamilton, Brook, Smith &
`Reynolds, PC
`
`(57)
`
`ABSTRACT
`
`A method is disclosed for supplying arbitrary types of
`information in a digital signal eonlbrmant to the structures
`defined in ISO standard ISO/113C 13818—1, known generally
`as MI’EGZ Systems. The method extends Program Stream
`and Transport Stream structures by application of principles
`developed for standardization ol‘ bidirectional data commit-
`nications protocols to define a universal messaging syntax
`applicable to all types of information, including Internet data
`streams, V111 data streams, other types of data streams, as
`well as MPELQ video, audio, and control;
`to define a
`multiplexing structure that is independent oi“ the details of
`the MPEG2 signal structure; and to define a scheme for
`addressing such enhanced MI’EG'Z signals, or portions of
`signals, to individual decoders or to groups of decoders.
`
`40 Claims, 6 Drawing Sheets
`
`
`
`10
`
`t
`
`,
`|
`
`_-
`
`
`—
`24
`ENCODER _l_2__
`DECODER ”lg
`
`
`
`
`ES
`Output
`_,_.._
`- Stream 34
`
`
`
`PES
`PES
`1 . ”
`
`
`
`Packetizer
`-
`De-Pocketizer
`
`
`- as
`14
`22
`-—-
`——
`
`.32r’
`ES
`
`+
`
`-
`
`as '
`
`—
`
`Control
`r
`23
`
`
`
`
`
`
`
`
`
`
`DE/MUX
`£9
`
`
`_ Control
`'3:
`
`Message
`Streams
`
`L
`
`Megsnge
`Streams
`
`Broadeom v. Eriesson
`lPR2013—00602
`Broadeom 1025
`
`

`

`US 6,172,988 B1
`
`
`Page 2
`
`US. PATENT DOCUMENTS
`
`
`
`‘
`
`5,01 9,35 7
`5,623,693
`5,027,519;
`5,650,825
`5,703.877
`3,712,850
`5,720,989
`5,801,781
`
`0/1998 Boyce at al.
`.
`.
`5,805,762 "
`386/68
`
`3,832,256 * 11/1908 Kim
`395/559
`5.844.600 a< 12/1998 Kc”
`
` ‘ 370/260
`5184711089 95 12/1908 Ohira et al'
`370/352
`
`. 370/395
`5.890.388 1
`4/1999 Earnest
`
`. 380/198
`5,987,212 * 11/1999 Kim et a1.
`
`OTHER PUBLICATIONS
`
`"Digital Broadcasting, Systems for Television, Sound and
`Data Scrvices; Specification for Service Information (81) in
`Digital Video Broadcasting (DVB) Systems." European
`Tclccommtmicaiion Standard, prlj’l’s 300 408. pp. 1412
`(()cl..1994).
`"Information ’l‘cchnology—Gcneric Coding of Moving, Pic«
`111103 and Associated Audio: Systems Recommendation
`H.222.0,”, ISO/IEC 13818—1, International Organization for
`Standardization Organization 1nternaiionale dc Normaliza-
`tion, pp.
`1 146 (Nov 13, 1994).
`
`1‘ citcd by examiner
`
`*
`
`'
`'
`
`............................. 380/23
`(1/199). “’00 E1. 111.
`5,425,101
`.
`. 358/335
`10/1Q§H Fujinami 61 HI.
`5,455,084
`
`1. 370/395
`5,459,722 * 10/1995 Sherif
`
`.. 370/253
`5.4673512 * 11/1995 1.,0gston ct at.
`
`
`....................
`. ”1.927 *
`5/1096 Kim c-i £11.
`370/474
`
`.. 370/396
`I
`'
`='
`7/1996 Brunfilzld 81111.
`.
`
`*
`7/1996 Branstad ct 111.
`370/518
`
`8/1996 Bigham ct a1.
`. 370/5841
`.1 370/302
`1 1/1996
`Pang at 211.
`. 370/397
`12/1990
`Dart: 11'. ct 111.
`
`Snitch
`1’1007
`348/732
`Nubei‘ (:1 1-11.
`1/1997
`
`.. .. 386/83
`370/474
`4/1997
`NaimpziHy
`
`. 713/187
`*
`51/15/97
`:11.
`Rohatgi cl
`
`('onosucnli 01 ul
`.3 717/397
`"
`5/1907
`
`348/465
`’
`7/1997
`Naimpafly et all.
`. 370/395
`
`”“ 12/1997 Nubcr 01 al.
`. 3 70/5 38
`1/1908 13112: 61 n1.
`, 370/5119
`3/1998 Dokic ...................
`...... 370/474
`9/1998 Hiroshima c1211,
`
`

`

`US. Patent
`
`Jan. 9, 2001
`
`Sheet 1 of 6
`
`US 6,172,988 B1
`
`Mm
`
`wwa
`
`$283093
`
`mm
`
`xszmo
`
`.VmEomtm
`
`Sago
`
`[E5885_$085
`
`
`
`3033/.
`
`9:095
`
`3832
`
`2586
`
`_0E
`
`
`
`
`
`
`

`

`US. Patent
`
`Jan. 9, 2001
`
`Sheet 2 0f 6
`
`US 6,172,988 B1
`
`'ll‘.
`
`mmON_ESEJM0:
`
`
`835m:03chVN./\8;watoamco;
`
`
`>AwEowtwmm:\on:nmEomtm
`
`EEQEBEtoficmEBmv
`
`89:25II.lI!I.I.83.25
`
`
`mmaowaEmmaowazm:m:_mummmaN_m:
`
`IIIIIIIIIttIIJ__MmM3:63:36__M__mN_._o_.w835::4{WWWmMmRmoW{Iwu__muotmt:
`
`3332iiiiiIM3335
`
`82.25I825::
`2025wmnzz¢:2025ONES.
`
`cmchoowns.650umccmoiommz
`om_/\/E.
`$5852.Fkmmm:8%
`
`
`CopcmegmioEmEQmCUEmEmMm
`
`mEomzm3625mEom:m
`#1............................................J.
`_._\lllllllllllllllllllllllllllllllllll.._._.lllli!!flt1
`
` mwooomommoOozm
`oExm_E::E/\IL4|/
`:0.a563Ali
`
`
`Signsm
`
`.9595Son.
`
`{4.4K
`
`

`

`US. Patent
`
`m029;wJ
`
`3mh
`
`.6
`
`US 6,172,988 B1
`
`
`
`omoflwooing
`
`mm:mm_
`
`mt
`
`Smt
`
`
`
`mm.IIIIImm.
`
`pm30:3
`
`2526a
`
`06E
`
`NO.
`
`
`
`
`

`

`US. Patent
`
`U
`
`m008
`
`Mmm.nma.
`
`825ammagmas.\wt%X:
`
`m¢0Ew?<¢0.“.
`
`
`
`
`
`
`.O.oZtmmmo<m1\m0<mmmz
`
`
`
`mm.....\....mm:om_\\\
`
`
`
` 1m.095:252.N//H,LmuoomoEllllllll///mIwaY9312025m8//Jmm“.59.:22$51.95.1:llllllltwooozfiwfioawwaowQEV
`
`V9
`
`qummmi
`
`
`
`
`
`2325mma_|Ec€+mlm8>:&wmafiv
`
`Smm._mm:
`
`m_......mw......o.oz:m”a007mOON595453:0me08.535memm!W,
`
`
`
`
`
`
`
`89:525565-NON
`
`
`

`

`US. Patent
`
`Jan. 9, 2001
`
`Sheet 5 or 6
`
`US 6,172,988 B1
`
`’78
`
`I88
`
`1/32
`
`ISO
`
`H6
`
`Payload
`
`\
`
`I88
`
`I78
`
`I78
`
`488
`
`FIG6
`
`210
`
`Section
`PDU
`
`'62
`
`2K)
`
`Secfion
`PDU
`
`
`
`
`2K)
`
`
`Secfion
`PDU
`
`
`
`

`

`US. Patent
`
`Jan. 9, 2001
`
`Sheet 6 0f 6
`
`US 6,172,988 B1
`
`\8505200\OmEESuwkm9
`IIIII.I.835::mma
`53:954/19
`
`
` Om_ammmammma2329.5
`550$5$35508:29:328magma8a:mmagma8%Am8:953me
`6:26.256:96:30k
`
`
`
`
`
`
`
`
`
`203:0;owns.toamcoukmum—2
`
`x».OE
`
`€986€035
`
`V3r...m.lllllllllllllllllllllllllllllllJ.
`Illullllll88.25
`
`835m{03ka
`
`
`
`cozoEmEoummp
`
`
`OE9.28me29um0E
`
`Cozumm_mEoFShowcoxumm
`
`
`296am>o\own=2$35.6m>oxwwn=z202i
`
`
`
`mi.983503
`833:3:wk835%
`IIII|.|8395200/
`
`EcmtwCommas;
`
`
`
`
`

`

`US 6,172,988 Bl
`
`1
`METHOD FOR UNIVERSAL. MESSAGING
`AND M UIII‘II’LEXING OF VIDEO, AUDIO,
`AND DATA STREAMS
`
`RELATED APPLICATIONS
`
`This application claims priority to US. Provisional
`Application, Serial No. 60/0l0,873, filed Jan. 3], .1996 and
`US Provisional Application Serial No. 00/016,194, filed
`Apr. 25, 1996, both entitled “A Universal Messaging and
`Multiplexing System Compatible With MPEGE Digital
`Video and Audio Transmissions," by James (I. 'l‘iernan and
`Christopher J. Bennett, and claims the benefit thereof.
`BACKGROUND OF THE INVENTION
`
`The invention relates generally to the transmission of
`digital signals, and more particularly to the transmission of
`digital signals in systems containing digital video and audio
`signals conforming to packet-based multiplexed communi-
`cation standards.
`
`2
`PID (PID 0), used to carry a Program Association Table
`(PAT) listing all Programs in the- TS, and a PID associated
`with each Program which carries a Program Map Table
`(PMT) describing the components of the Program PSI
`syntax also permits the use of PIDs to transport private data
`in formats compatible with PSI, or in totally private formats.
`Other private control information may include Entitlement
`Control Messages (ECMs) or Entitlement Management
`Messages (EMMs) used for encryption and authorimtion,
`generally referred to as Conditional Access Information. In
`addition, the Digital Video Broadcasting (DVB) committee
`has defined a similar structure for transport of Service
`Information (SI).
`Private data signals can cover a wide variety of purposes,
`including signals strictly related to broadcast
`television
`applications, such as would be carried in the Vertical Blank-
`ing Interval (VHI) of an analog television signal; signals
`carrying services ancillary to broadcast
`television
`applications, such as Electronic Program Guides (EPG); and
`signals such as are used for transmitting voice and/or data in
`conventional telecommunications networks, in both asyn~
`chronous and synchronous modes.
`from
`It
`is apparent
`that
`the MPEG2 structures sut’r‘cr
`certain difficulties. One class of dilliculty emerges when one
`considers the possibility of building modular encoders and
`decoders that could be applicable to both 'I‘Ss and PSs.
`In practice, however, there are significant restrictions on
`such an approach.
`In the first
`instance,
`the treatment of
`control data is very diifcrcnt between the PS and the PS.
`While the control data used to describe a Program in it PS is
`in a PBS format,
`the control data used in a TS is in an
`incompatible section format. Thus, the interfaces between
`modules must necessarily be different for PS and TS encod-
`ers, This diITiculty also prevents storage of a Program in a
`form suitable for direct inclusion in a TS on a device adapted
`to the PS format.
`
`the PS structure
`Another significant concern is that
`imposes restrictions on the types and numbers of program
`elements, or components, which may be part of a Program.
`Such restrictions do not exist in the TS structure. As an
`example, a Program may contain multiple streams of sub
`titling information that are encoded according to lnterna»
`tional Radio Consultative Committee (CCIR) System B
`Teletext specifications. Each stream of subtitling informa-
`tion may be encoded as a PBS according to the procedures
`defined by the Digital Video Broadcasting (DVB) committee
`in DVB TMl.3.98 “DVB Subtitling System Working Draft
`1.0". Within the TS structure, each such PES of subtitling
`information may be assrgned a separate PID and may be
`associated with the Program. However, the PES structure is
`a Private Data PBS, and the PS specifications only allow one
`such PES to be associated with a Program.
`A second class of difficulty emerges when one considers
`the variations in the number of components that may be
`associated with a Program, and the variation in their data
`rates. The Main Profile Main Level of the MPEG2 Systems
`standard permits video elementary streams to be transmitted
`at rates of between '1 and IS Mbps. A 'I‘S may include
`multiple such streams. A decoding devree which selects such
`streams from the 'l“; by PID will
`therefore typically be
`implemented in high speed fieId-prograrrunable gate arrays
`(PPGAs) or application-specific integrated circuits (ASle)
`in which the PII)
`is held in. a register
`for comparison
`purposes. The number of 'PIDs which may be prr'icessed
`simultaneously therefore becomes a controlling cost element
`in the design ofsueh FPGAs or ASICs. However, since each
`
`Nm
`
`’4.)0
`
`Lu'Ji
`
`The International (..)rgani7,ation for Standardization (ISO)
`recently adopted a standard (ISO/IEC l3818»1)
`that
`addresses the combining of one or more "elementary
`streams” ofvideo and audio, as well as other data, into single
`or multiple streams suitable for storage or transmission. The
`ISO/[EC 13818—1 standard, hereinafter referred to as the
`“MPEGZ Systems" standard.
`is described in detail in the
`ISO draft document “Generic Coding of Moving Pictures
`and Associated Audio", ISO/IEC‘ .lrI‘Cl/SCBWWGII N080]
`(Nov. 13, 1994), which is incorporated herein in its entirely
`by reference. The MPEGZ Systems standard specifies syn.
`tactical and semantic rules for system level coding of the
`elementary streams using packet-oriented multiplexes. 'lhe
`MPEG2 Systems standard defines an individual coded
`video, audio or other coded bitstream as an "elementary
`stream".
`'I'he contents of an elementary stream may be
`broken into a sequence of discrete units, in which case the
`elementary stream is structured as a Packetizetl Elementary
`Stream (PIES). The individual units, or packets, are known as
`PljS packets, which can be of large and variable size. The _,
`MPEGZ Systems standard defines generic structures for PBS
`packet formats and specifies particular rules for creating
`PESs from digital video and audio elementary streams.
`Allowance is made for creating PESs for private data
`formats, ie., data formats not defined by the MPEGZ
`Systems standard.
`The MPEGZ Systems standard defines two methods of
`creating, a multiplex of PESs. In a Program Stream (PS), all
`components in the multiplex are assumed to belong to a
`single “Program“, that is a collection of elementary streams ‘
`which may sensibly be presented as a unity to a user, all
`components being referenced to a common time base,
`together with certain coordinating control information. PliS
`packets from component PESs are multiplexed by PliS
`packet. The PS is rigidly structured so that at most one. PES
`of a specified format may carry a private data elementary
`stream.
`
`In a Transport Stream (TS), the components of the mul-
`tiplcx may belong to many programs. Each PES is assigned
`a "packet identifier" (PID). The PES packets are broken into
`small. fixed-size units called transport packets, which may
`be multiplexed with transport packets from other PESs.
`Coordination and control of the PESs and the Programs in
`the TS is managed via control data called Program Specific
`Information (PSI). PSI is structured according to a section
`format which differs from and is incompatible with the PBS
`packet. PSI is distributed via several PIDs, including a base
`
`(it)
`
`

`

`US 6, I 72,988 B 1,
`
`3
`PID corresponds to a single PBS. the same register is used
`regardless ot‘ whether the component selected is it IS Mbps
`video stream or a 1200 bps asynchronous data component.
`The MPEGZ Systems standard scheme ol‘ making each PID
`carry a single component cart therefore induce significant
`inelliciencres.
`A third class of dilficulty emerges when one considers
`certain classes of application tor which addressabrlity is
`desirable. For example, it may he desired to address certain
`streams, including video and audio streams. to a decoder or
`a group of decoders. This need may exist without regard to
`the capabilities that may exist
`for such services in the
`underlying transmission media, for example because no
`such facilities exist in the medium. I-“or a further example, it
`may be desired to transmit data signals such as datagrants
`conforming to the Internet Protocol (IP) in a private data
`channel over an MPEG2 Transport Stream. However, a
`decoder capable of accepting such datagrams must first be
`provided with an Il’ address.
`In other broadcast systems
`faced with this problem, such as Ethemet,
`the decoder is
`provided with a native address by a manufacturer which may
`be bound to an Internet address by the systems administrator
`using protocols such as the Reverse Address Resolution
`Protocol (RARP). Such an approach is not possible in
`MPEGZ Systems because no such native address scheme is
`defined for MPEGZ decoders.
`
`SUMMARY OF THE INVENTION
`
`()nc object of the present invention is to separate pro-
`cessing of elementary streams from the multiplexing and
`dcmultiplexing of such streams into a Transport Stream or
`Program Stream by a common interface based on a universal
`message structure that
`is compatible with the PBS packet
`structure but does not share the limitations of the latter.
`Another object
`is to multiplex multiple elementary
`streams itito a format which permits the creation of Pro—
`grams that are compatible with a Program Strcartr. and
`which permits the use of a single I’ll) to identify multiple
`elementary streams in circumstances when such multiplex-
`ing, creates significant efficiencies in use ot‘ PII)s in Trans—
`port Strcam decoders.
`invention is to address
`A further object of the present
`individual signals to individual decoders or groups of
`decoders, without regard to any addressing facilities that
`may exist in the underlying transmission medium.
`In accordance with an aspect of the invention, a method
`of providing transport services to a plurality of first elemen-
`tary streams comprises segmenting the elementary streams
`into messages to form a plurality of first message streams in
`which each message stream is associated with a particular
`elementary stream, The first message streams are converted
`into first packetized elementary streams. The first packetized
`elementary streams are multiplexed into an output stream.
`The output stream can he a Program Stream, a Transport
`Stream, or one of other types of streams including Asyn'
`Chronous Transfer Mode.
`
`wm
`
`an
`
`According to another aspect of the invention, a plurality
`of second elementary streams are segmented into messages
`
`to form a plurality ofsecond mess“
`streams. Data channel
`identifiers are associated with the second message streams,
`such that each data channel identifier is associated with a
`particular message stream.
`the second message streams are
`multiplexed into a second packetixed elementary stream
`which is then multiplexed with the Iirst paeketixed elemenv
`tary streams into the output stream.
`According to another aspect of the invention, for 3 Trans"
`port Stream, multiplexing the first and second packetized
`
`(ti)
`
`fru‘
`
`4
`elementary streams is achieved by converting packctizcd
`elementary stream packets of certain paclrctizcd elementary
`streams into table sections and segmenting the sections and
`the remaining packetized elementary stream packets into
`'I‘ransport Stream packets The Transport Stream packets are
`multiplexed to form the Transport Stream.
`the
`invention.
`According to yet another aspect of the.
`elementary streams can include elementary streams which
`have dill‘crent data rates.
`
`BRIEF DESCRIPTION OF TIIE DRAWINGS
`
`The foregoing and other objects, features and advantages
`of the invention will be apparent from the following more
`particular description of preferred embodiments of the
`invention, as illustrated in the accompanying drawings in
`which like reference characters refer to the same parts
`throughout the different views.
`FIG. 1 is a schematic block diagram of an encoder/
`decoder system in accordance with the present invention.
`FIG. 2 is a llow diagram illustrating a transport layer in
`accordance with the present invention,
`I‘IG. 3 illustrates a Message Stream conlorming to the
`PIES format defined by MPEG2.
`FIGS. 4A and 4B illustrate the general content and
`arrangement of a message structure in accordance with the
`present invention.
`FIG. 5 illustrates Message Streams having a channel
`number in accordance with the present invention.
`HQ. 6 illustrates a multiplex of Message Streams forming
`a Private PBS in accordance with the invention.
`FIG. 7 is a flow diagram illustrating a Transport Stream
`convergence sublaycr in accordance with the present invert-
`tron for use in the transport layer of FIG. 2.
`MG. 8 i.lustratcs a Message Stream which includes sec-
`tion formatted information.
`
`FIG. 9 illustrates the format of private data descriptors for
`rise
`in the message structure of FIGS. 4A and JR in
`accordance with the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`invention provides for the adoption ol' the
`The present
`principles of protocol layering specified by the Open Sys
`terns Interconnection (OSI) Reference Model defined in
`standard ISO/IEC 7498 for use in systems providing
`MPEG2 P85 and "I‘Ss. The MPEG2 Systems protocol archi—
`tecture is positioned at
`the Transport. Layer ol‘
`the 081
`Reference Model. In the present invention, three sublaycrs
`are provided:
`In this sublayer, an elementary
`Segmentation suhlayer.
`stream Protocol Data Unit (PDU) is accepted across a
`Service Access Point (SAP) to the sublayer as a trans—
`port service Service Data Unit (SDU) and is segmented
`into units called Messages. A Message is a generaliza-
`tion of a PBS Packet. 'l'he Messages can be optionally
`addressed. The Message is the segmentation sublaycr
`PDU. For each elementary stream input to thc segmen-
`tation sublayer, a single I’lzzS is created Within the
`sublayer. The scgtrtcntation sublayer also permits the
`identificatit‘m of ‘l‘ransport Service Connections
`between the input SAP and the output SAP. called Data
`Channels, by means ot‘ a channel identifier.
`Multiplexing sublaycr. In this sublayer, a set of input Data
`Channels may optionally be multiplexed into a single
`
`

`

`US 6,172,988 B1
`
`m
`
`5
`PES. Messages are both the PDUs and SDUs of the
`multiplexing sublayer.
`Convergence sublaycr. In this sublaycr, the various PESs
`arc multiplexed using a format specific to the transport
`system being used.
`In the preferred embodiment, two variants of the conver
`genee sublayer are defined to support the PS and TS defined
`by MPEG2. Addrtit’tnal convergence sublayers could be
`defined for other contexts, e.g., for transport of PESs using
`Asynchronous 'l‘ransfer Mode (ATM). In the variant of the
`Convergence sublayer ttsed for a PS, referred to as the PS
`Convergence sublayer, PIES packets are multiplexed directly
`into at PS, as defined by the MPEGZ Systems standard. The
`variant of the Convergence sublayer used for a TS, referred
`to as the TS Convergence sublayer, is in turn comprised of .
`three sublayers: a format convergence strblayer for convert—
`ing certain PESs to the "Table Section formats required by
`MPEGZ Systems for certain purposes, notably transport of
`PSI; an additional segmentation sublayer for segmenting
`PISS packets into small, fixed size Transport Packets; and a
`multiplexing sublayer for multiplexing Transport Packets.
`The present
`invention is based on the following prin-
`ciples:
`l. Each elementary stream input to a transport service is
`output as an identical single elementary stream. The
`association between the input and output elementary
`streams defines a Data Channel. Each Data Channel
`uniquely identifies an association between the TSAP
`('l‘ransport SAP) at the source and the TSAI’ at each
`decoder. Elementary streams may be freely associated
`with Data Channels.
`
`N u;
`
`2. The Transport Layer is decomposed ittto sublayers. The
`interfaces between the sublayers is based on the PBS
`syntax defined by the MPEGZ Systems standard, since
`PIES is the common element linking MPEGZ elemen-
`tary streams with both the Transport Stream and Pro-
`gram Stream variants. An extension of the PIES packet
`is used to transport all data types in the system in a
`Universal Messaging Structure referred to earlier as a
`Message. The Message also provides the structures
`)
`required for addressability.
`3. The syntax applying to the contents of a Message
`depends on the type of elementary stream. Each Mes-
`sage therefore is assigned a Message Type identifying
`the message syntax. The message type acts as a selector
`for the Presentation I.aycr Entity needed to process the
`message (cg. video, audio, VBI).
`To the extent that a message type is present in a Message,
`the Message is self-descriptive regarding the presentation
`layer processing that is required. Since it is possible in the
`future that PESs may be processed by transport services
`having a non-MPEG2 structure, such as ATM, and may even
`be processed in isolation from other elementary streams,
`Messages should be self—descriptive with respect to prescri—
`tation layer parameters, where possible.
`Referring now to FIG. 1, a packet—based communication
`system is shown which embodies the principles of the
`present invention. The system '10 includes an encoder 12 and
`a decoder 18 connected by a transmission path 28. The
`transmission path 28 can be any transmission facility, such
`as a
`fiber optic link or
`a satellite transmission uplink/
`downlinlt. Blet:t:ter‘ttary streams 24 and control information
`23 input to the encoder .12 are converted to a single output
`stream 34 and transmitted over path 28 to the decoder '18.
`The decoder 18 reconvcrts the output stream to the constitu—
`ent elementary streams 32 and control information 31. The
`encoder 12 includes a Packetized Elementary Stream (PES)
`
`LuLI!
`
`4f1
`
`55
`
`()1)
`
`{D
`
`6
`paekctizer 14 and a multiplexer 16. The elementary streams
`24 and control information 23 input to the PES packctizer
`are converted to PBS streams 26. The multiplexer '16 mul-
`tiplexes the PES streams to form the output stream 34.
`The decoder 18 includes a demultiplexer 20 and a PIES
`depacketizer 22. The demultiplexer 20 demultiplexers the
`output stream 34 to extract PISS streams 30. The PES
`de-packetizer 22 converts the PISS streams to the constituent
`elementary streams 32 and control information 31. The PES
`packetizcr 14 and the PES tle—packetizer 22 operate in
`accordance with the MPEG2 Systems standard. In preferred
`cmbodin rents, the muxi‘demux 16, 20 operate with 'l'ransport
`Streams or Program Streams as the output stream 34. in an
`alternate embodiment. the. multiplexer and dcmultiplexer '16,
`20 can operate to provide non~MPEG streams, such as an
`ATM formatted stream for output stream 34.
`With reference to the concepts defined in ISO/113C stan-
`dard 7498, an MPEG2 encoder and decoder provides uni-
`directional broadcast transport services between the encoder
`and decoders. The transport services ensure transparent
`transport of elementary streams between the encoder and
`decoder. MPEG2 defines this process for video and audio
`elementary streams, FIG. 2 illustrates the generalized trans~
`port process embodied in the present invention.
`In FIG. 2, the transport layer of ISO/113C 7398 is used to
`provide unidirectional services only. The transport layer is
`decomposed into three sublayers in the encoder and decoder.
`FIG. 2 illustrates the first two of these sublaycrs only which
`correspond to functions performed by the PES paekctizer 14
`and PBS de—pacltetiLer 22 (FIG. 1). The encoding process is
`illustrated on the left side and the decoding process on the
`right side of FIG. 2.
`to the encoder is
`the input
`In the encoding process,
`provided as one or more elementary streams 102. The
`structure of these elementary streams is defined elsewhere.
`The coding of certain types of elementary streams, notably
`Video and audio, are defined by the MPL‘GZ video and audio
`standards specified in ITU-T Rec. 11.2621180/113C‘ 13818~2
`and ISO/lEC 13818-3. Each e ementary stream is provided
`to the encoder across the transport service interface 1(14.'111e
`'I‘SAP 106, 108 identifies the endpoint of a Data Channel.
`The first sublaycr of the transport layer is the segmenta-
`tion sublayer. 1n the segmen ation sublayer entity tn the
`encoder,
`the contents of the elementary streams are pro-
`cessed and divided into Messages. The structure of the
`Message in the present invert ion is considered in further
`detail below. Each elementary stream is thereby converted
`into a Message Stream.
`(Ier ain Message Streams 1103,
`representing the content of Data Channels carrying elemen-
`tary streams defined by MPEG2 are iden ical to MPEG2
`PESs; the identity of the associated Data Channel
`is not
`explicitly preserved. FIG. 3 ilustrates a Message Stream
`103 as defined by MPEGZ. An MPEGZ—de ined elementary
`stream 102, such as a compressed audio or Video stream, is
`there shown segmented into Messages 158. Each ritcssztge
`158 includes a PBS packet header 160 anc payload 162 as
`defined for the PES packet structure in the MPEG2 standard.
`Referring again to FIG. 2,
`in certain other Message
`Streams 109, representing the content of Data Channels
`carrying elementary streams not defined 3y MPEGZ,
`the
`identity ofthe Data Channel may optionally be preserved by
`an identifier called a channel number which is stored in a
`field in the Message format. In FIG. 2, sum Data Channels
`are shown With the identity of the Data (Ihanncl preserved.
`The Message Stream 109 comprises Messages having the
`structure illustrated in FIGS. 4A and 4B. A Message 178
`includes a PIES packet header 180 and a payload 182. The
`
`
`
`
`
`

`

`US 6, ,1 72,988 B1
`
`7
`Message I'Icadcr 180 conforms to an MPEG2-defined
`Private Stream 1 PES Packet
`format which includes a
`pes private data field 184. The structure of Message
`Ileader 180 is described in further detail below. It should be
`noted that
`a Message can also be based on a Private
`Stream 2 PES Packet format
`in which case private data
`forms the first part of the payload.
`It is important at this point in the discussion to note that
`a channel number can be stored in the pes private data
`field 184. FIG. 5 illustrates Message Streams 10917., 10%
`having the Message structure 178. Each Message includes a
`channel nttmber 188 unique to each Message Stream. (Tcr»
`tain Messages Streams, and certain Messages within indi-
`vidual Message Streams, may be addressed at this point.
`Referring again to FIG. 2,
`in the encoder, Messages are
`passed by the segmentation sublayer entity to the next
`sublayer,
`the Multiplexing sublayer, across the Message
`Interface 1,10.
`In this sublayer, all Message Streams are
`transformed into PESs by the Multiplexing sublaver entity.
`If the Messages in a given Message Stream do not retain the
`channel number, identifying their associated Data Channel,
`they cannot be multiplexed and are passed to the. Conver-
`gence sublayer, as shown in 112. Optionally, as shown in
`114, Messages from different Data Channels may be mul-
`tiplexed into a single PES, conforming to the syntax of a
`Private PES 116 as defined by MPEG2, which is then passed
`to the Convergence sublayer. FIG. 6 illustrates the structure
`of the. Private PES 116. The Private PES 116 is a stream of
`messages 178 multiplexed from Message Streams 109a and
`10.91). The messages 178 can be distinguished by the channel
`number 188.
`In FIG. 2, Message Streams are shown being multiplexed
`into a unique MPEG2 private PES. Nothing in the present
`invention precludes the creation of multiple MPEG2 Private
`PES. unless constrained by the capabilities of the Conver-
`gence sublayer. The structure of Convergence sttblayers are
`discussed in further detail below.
`It will be understood that
`the interface between the
`Multiplexing sublayer and the Convergence sublayer at all
`times consists of an MPEG2 PES. This is not
`trite for
`MPEG2 itself, which in particular defines a second class of
`interface, the so-called MPEG2 sections. for ttsc of transport
`of control information and private data in an MPEG2 TS. It
`is a principal object and a feature of the present invention
`that the interface to the Convergence sublaycr is at all times
`an MPEG2 PES, so that different Convergence sublayers
`may be substituted for each other, with the. only impact upon
`the Segmentation and Multiplexing sublayers, if any, being
`an adjustment of the set of Data Channels which are mul-
`tiplexed into a set of MPEGZ Private PESs.
`It will also be understood that
`the. data capacity of the
`MPEGZ Private PESs created in the Multiplexing sublayer
`of the encoder 11.6 will be approximately equal to the sum
`of the capacities of the individual Data Channels 108. In
`systems in which the transport service must carry a number
`of elementary streams of widely variant data rates,
`the
`invention may be used to aggregate a number of low-rate
`elementary streams into a single high-rate PES,
`thereby
`minimizing the resources required to identify the various
`low-rate data streams in ASK? or V1.81 implementations of
`components of the decoder.
`Each PES passed across the interface between the MUP
`tiplexing, sublayer and the Convergence sublaycr in the
`encoder .112 is recovered in the ('Ionvergcncc sublayer entity
`in the decoder and passed across the corresponding interface
`128 to the Multiplexing sublayer entity in the decoder. If the
`PES transports an MPEG-delined elementary stream 1.19
`
`8
`then it is passed directly to the Segmentation sublayer in the
`decoder. If an MPEG2 Private PES 118 transports a multi-
`plex of elementary streams created according to the present
`invention, as described above, the