Z
`
`/
`
`wo 97/33434
`
`PCT/US97/03525
`
`25
`
`time at which the particular program will be received by a first receiving
`
`means (16, 114); and
`
`storing the entered information.
`
`5
`
`9.
`
`The method of claim 8, further comprising the steps of:
`
`sending the stored uniform resource locators‘ at the time at which
`
`the particular program will be received by the first receiving means (16,
`
`114), directly over an Internet connection (94) to a second receiving means
`
`(106).
`
`10
`
`15
`
`10.
`
`A system for presenting integrated television programming and
`
`corresponding related Internet information segments obtained from Web sites on
`
`the Intemet (20), the system comprising:
`
`a television broadcaster data entry and broadcast means comprising:
`
`a means (70, 74) for accessing a service Web site (62) on the
`
`Internet (20), wherein a member broadcaster of television
`
`programming accesses the service Web site (62);
`
`a means (70, 74) for entering information into the service
`
`Web site (62), wherein the entered information is comprised of
`
`20
`
`uniform resource locators and a time at which a particular program
`
`will be broadcast (86) by the member broadcaster (66), wherein the
`
`uniform resource locators specify one or more Internet addresses
`
`(102) of the relevant Internet information pages which relate
`
`specifically to the content of the particular program being broadcast
`
`NTFX-1002 / Page 821 of 1867
`
`

`
`wo 97/33434
`
`Pcr/vssmsszs
`
`26
`
`by the member broadcaster (66);
`
`a means (70), connected to the entering means (70, 74), for
`
`storing the entered information;
`
`a first means (86,110) for sending the particular program being
`
`5
`
`broadcast by the member broadcaster (66) to a user (118), wherein the
`
`particular program contains a video signal and an audio signal;
`
`a second means (74), connected to the storing means (70), for
`
`sending, at the time at which the particular program will be
`
`broadcast, the stored uniform resource locators over a direct Internet
`
`10
`
`connection (94) to the user (118);
`
`a user terminal (16) comprising:
`
`a first means (16, 114) for receiving, from the first sending
`
`means (86, 110), the particular program, including the video and
`
`audio signals, being broadcast by the member broadcaster (66) to a
`
`15
`
`user (118);
`
`a second means (106) for receiving, from the second means
`
`(74) for sending, the stored uniform resource locators which
`
`correspond to the video and audio signals received by the first
`
`receiving means (16, 114);
`
`20
`
`a controller means (16), connected to the second receiving
`
`means (106), comprising:
`
`a means (12) for decoding the uniform resource locators
`
`to determine the specified Internet addresses;
`
`a means (98), connected to the decoding means (12), for
`
`NTFX-1002 / Page 822 of 1867
`
`

`
`wo 97/33434
`
`PCT/US97/03525
`
`27
`
`retrieving the one or more Internet information segments
`
`residing at the determined Internet addresses (102); and
`
`a display means (18, 114), connected to the controller (16) and
`
`the first and second receiving means (16, 106, 114), for presenting the
`
`5
`
`video and audio signals concurrently with the Internet information
`
`segments.
`
`
`
`NTFX-1002 / Page 823 of 1867
`
`

`
`wo 97/33434
`
`[ / 3
`
`PC!‘/US97/03525
`
`20
`
`VIDEO
`
`WITH
`
`URL?
`
`
`
`
`SUBSCRIBER SITE
`
`1 6
`
`
`
`
`
`LOCAL
`URL
`DECODER
`
`SERVER
`URL
`DECODER
`
`NTFX-1002 / Page 824 of 1867
`
`

`
`
`
`wo 97/33434
`
`PC!‘/U897/03525
`
`2 / 3
`
`
`
`
`
`URL
`SEEN
`BEFORE
`?
`
`
`
`SEND NETSCAPE
`GOTO COMMAND
`
`54
`
`SOFIWARE DESIGN
`
`FIG. 3
`
`SUBSTTFUTE SHEET (RULE 26)
`
`
`
`NTFX-1002 / Page 825 of 1867
`
`

`
`WO9‘II33-434
`
`PCTIUS97/03525
`
`:3
`
`<1“
`
`(5
`F--I
`L’:-.
`
`8
`
`It .
`
`E I
`1
`
`g
`*3
`:5
`
`8
`
`SUBSTITUTE SHEET (RULE 26)
`
`NTFX-1002 / Page 826 of 1867
`
`
`
`

`
`
`
`
`
` Intemational application No.
`INTERNATIONAL SEARCH REPORT
`PCT/U597/0357.5
`
`CLASSIFICATION OF SUBJECT MATTER
` A.
`
`
`|PC(6)
`:H04N 7/00; H04L 12100
`US CL :395/200.01, 200.02; 348/7, 906
`
`
`According to International Patent Classification (IPC) or to both national classification and [PC
`8.
`FIELDS SEARCHED
`
`
`Minimum documentation searched (classification system followed by classification symbols)
`
`
`
`U.S.
`
`:
`
`395/200.01, 200.02. 200.09, 327; 348/7, 8, 10, I2, 13. 461, 564, 906; 455I3.l, 5.1, 6.1, 6.3
`
`
`
`Electronic data base consulted during the international search (name of data base and. when pncticable, search team used)
`
`APS
`
`
`
`4-6.
`
`C.
`
`DOCUMENTS CONSIDERED TO BE RELEVANT
`‘
`Citation of document. with indication, when appropriate. of the relevant passages
`
`'
`
`Relevant to claim No.
`
`US. 5.589.892 A (KNEE ET AL) 31 DECEMBER 1996, cols 1-9
`
`Documentation searched other than minimum documentation to the extent that such documents are included in the fields seatehed
`
`
`
`
`
`US, 5,534,913 A IMAJETI AL) 09 JULY 1996, cols 1-3.
`
`US. 5,481,542 A (LOGSTON ET AL) 02 JANUARY 1996.
`abstract.
`
` US. 5.479.268 A (YOUNG ET AL) 26 DECEMBER 1995. cols
`1-3.
`
`
`
`US. 5,014,125 A (POCOCK ET AL) 07 MAY 1991, cols 1-2.
`
`
`
`'
`'
`mono’
`
`D Further documents are listed in the continuation of Box C. 9 See patent family annex.
`Spaehlcalegosiaolciddoeuxaanz
`hztdnuuaanplblfiaflerhfifixlfiliogdneor
`'1’
`doasnlatdefnbgthgunsalnataoflheanvhidtinaeouiiad
`uoneorpuuuimeevnoa
`
`. ‘
`
`A’
`
`-5-
`'1:
`
`'0'
`
`euuaaocmanptuamnuniuiuuaansoairihgdau
`«puma:-sagsmymgugaouaunpcsuiycaag-iwvaasa
`:",,-_,”£'::,,,,,'“?"°"“"°',;,,,,""°"°°"“”'"‘”" ‘Y’
`do&Inlef:rI"Is;IouoIaId'ncIoIl:.I-'..el.hhIt-Ia" ordkt
`%
`
`""
`
`"°°'."°"I|§LM .“l
`'*-'1-=‘°°'==-Iilhlnhne
`aoasuuotpn_:auuhnuuynan;_iaechaaasipvnsnaeunnaaa
`beingobvioubapaaoadilkdisthean.
`
`
`
`
`
`
`
`
`
`aocunaunuwuuaenmpauruny
`$ bhmrdm|flh%Mmm -r
`'P‘
`
`
`
` Date of mailing of the inte:-national search report
`Date of the actual completion of the intemational search
`24 J11!) 1997
`25 APRIL 1997
`
`
`
`
`Name and mailing address of the ISA/US
`Conuniuionet of Patents and Tradamath
`Box PC!‘
`
`wumngion. be 2023:
`Facsimile No.
`703 305-330
`
`
`Form PCT/ISAIZIO (second sheet)(JuIy l992)t
`
`NTFX-1002 / Page 827 of 1867
`
`

`
`
`
`am-jug
`
`(19)
`
`(12)
`
`Europaisches Patentamt
`
`' European PatentOttlce
`
`Office européen des brevets
`
`Illllllllflllllflllfllllillllfllflllflllllllllfllllllllflfllllflflllll
`
`(11)
`
`EP 0 805 598 A1
`
`EUROPEAN PATENT APPLICATION
`
`(43) Date of publication:
`05.1 1.1 997 Bulletin 1997/45
`
`(21) Application nurrber: 96106961.4
`
`(22) Date of filing: 03.05.1996
`
`(51) int. CL‘: HO4N 7/50, HO4N 7/08
`
`
`
`
`
`‘BESTAVAILABLECOPY...
`
` (84) Designated Contracting States:
`
`- Egawa, Ren
`Princeton, New Jersey 08540 (us)
`
`
`
`DE ES FR GB IT NL
`
`(71) Applicant:
`-
`. MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD.
`Kadoma-shl, Osaka 571 (JP)
`
`(74) Representative: Marx, Lothar, Dr.
`Patentanwfilte Schwabe, Sandmalr, Marx
`stuntzstrasse 16
`
`(72) Inventors:
`- Nalmpally, salprasad V.
`Langhorne, PA 19047 (US)
`
`’
`
`81677 Milnchen (DE)
`
`
`(54) Method and apparatus for replacing stuffing bits with supplementary data in an MPEG video
`data stream
`
`An apparatus and method. applicable to varia-
`(57)
`ble bit rate video and constant bit rate video. is dis-
`closed tor replacing "stuffing bytes‘ with private data.
`The invention takes advantage oi the otherwise wasted
`resources dedirated to 'stu1ling" in a data stream in
`order to insert private data. This is accorrplished by
`inserting useful private data in a Transport Stream
`instead of the stuffing bits That is. effectively. a I'e-t‘t‘lul-
`tiplexing operation occurs where. based on the exist-
`ence ot certain conditions in a Transport Packet (e.g..
`stulfing bytes exist).
`the irrtonnation necessary to
`replace stuliing bytes with private data yet still comply
`with established standards is accorrplished. This data
`» generally is, referred to as privatestuit data in order to
`distinguish it from typical private data which may other-
`wise be encoded into a Transport Stream The stuffing
`bytes removed from the Transport Packet may corrre
`from an adaptation field in the Transport Header or
`directly irorn the Transport Payload or both.
`
`EP0805598A1
`
`FlG.4A
`
`.j_._.__.._.—:___.._
`Prtrneobyi'tanlrxerur(UK)8ushessserviaas
`
`
`
`NTFX-1002 / Page 828 of 1867
`
`

`
`Description
`
`EP0805598A1
`I
`
`-,"
`
`'
`
`5
`
`FIELD OF THE INVENTION
`Thepresent invention relates generally todata storage andtransmission using MPEG standards and. more partic-
`ularly, the present invention relates to the established standards of transmitting ‘private data‘ and "smiling bytes" in a
`Transport Data Stream complying with MPEG standards
`
`BACKGROUND OF THE INVENTION
`
`I-figh Definition Television (HDTV) continues to make progress in its attenpts to replace conventional telwision.
`Paving the way for this progress are various oonpanies and associations working on standards to provide for a global
`market for HDTV.
`
`20
`
`2s
`
`as
`
`40
`
`rs
`
`One such group of corrpanies is known as the ‘Digital HDTV Grand Alliance” including menbers such as AT&T,
`15 David Sarnoff Research Center. Massachusetts institute of Technology and others A corrprehensive overview oi the
`strides made by this group are presented in an article by Robert Hopkins entitled ‘Digital Terrestrial HDTV tor North
`America: The Grand Alliance HDTV System‘ published in the IEEE Tiansactions on Constmer Electronics (Summer
`1994). This article is herein incorporated by reference for all of its teachings regarding the background and basics of
`HDTV systems including the rise of Program and Transport Packet Streams.
`In addition to the Grand Alliance. much effort has been expended by the Moving Pictures Expert Group (MPEG). a
`committee within the International Standards Organization (ISO). in attempts to establish various standards for the stor-
`age and transmission of HDTV data (e.g.. MPEG-2 standards - formats for Transport Packet Streams). Accepted stand-
`ards are periodically published such as the Video Section of lnforination Technology - Generic Coding oi Moving
`Pictures and Associated Audio ISOIIEC 13818-2 (Novenber 1994) (hereinafter ‘Video Section") and the Systems Sec-
`tion of Information Technology - Generic Coding of Moving Pictures and Associated Audio ISOIIEC 13818-1 (Noveriber
`1994) (hereinafter "Systems Section") both of which are herein incorporated by reference for their teachings regarding
`established standards and formats including "smiling" techniques.
`The syntax for the MPEG-2 standard defines several layers of data records which are used to convey both audio
`and video data For the sake of sirrplicity. the decoding of the audio data is not described herein. Encoded data which
`so describes a particular video sequence is represented in several nested layers. the Sequence layer. the Group of Pic-
`tures layer. the Picture layer, the Slice layer and the Macroblock layer. To aid in transmitting this intomtation. a digital
`data stream representing multiple video sequences is divided into several smaller units and each of these units is
`encapsulated into a resaective packetized elementary stream (PES) packet For transmission. each PES packet is
`divided. in turn. among a plurality of tixed-length Transport Packets. Each Transport Packet contains data relating to
`only one PES packet. The Transport Packet also includes a header which holds control intonnation, sometimes includ-
`ing an adaptation field. to be used in decoding the transport packet.
`When an MPEG-2 encoded image is received. a transport decoder decodes the Transport Packets to reassenble
`the PES packets. The PES packets, in turn. are decoded to reasserrble the MPEG-2 bit-stream which represents the
`irmge in the layered records. as described above. A given transport data stream may simultaneously convey multiple
`irriage sequences. tor exanple as interleaved transport packets. This flexibility also allows the transmitter to switch
`among formats providing material in 4 by 3 aspect ratio according to one standard and widescreen (16 by 9) material
`according to another standard
`'
`Turning to a system implementation for delivering HDTV using MPEG-2 standards to the consumer. in general. as
`illustrated in high-level block diagram of Figure 1. on the transmission side. video and audio signals are input to respec-
`tive encoders 110 and 112. buttered in buffers 1 14 and 116. delivered to the system coder/multiplexer 118. and stored
`in storage unit 120 or transmitted by transmitter unit 120. On the receiving side. the signals are received by a system
`decoder/demultiplexer 122. again bulfered in butters 124 and 126, then decoded by decoders 128 and 130 and output
`as the original video and audio signals
`‘ An irrportant aspect of the illustration of Figure 1 is that. although the intermediate stage buffering of the signals
`includes a variable delay. the overall delay from input to output of the signals is required to be substantially constant.
`This is accorrplished by monitored flow control and buffers
`As indicated in Figure 1. the delay from the input to the encoder to the output or presentation from the decoder is
`constant in this model, while the delay through each of the encoder and decoder butters is variable. Not only is the delay
`through each of these butters variable within the path of one elementary stream. the individual butler delays in the video
`and audio paths diller as well. Therefore. the relative location of coded bits representing audio or video in the contained
`stream does not indicate synchronization information. The relative location of coded audio and video is constrained only
`by a System Target Decoder (STD) model such that the decoder butters must believe properly; l.herelore. coded audio
`and video that represent sound and pictures that are to be presented simultaneously may be separated in time within
`the coded bit system by as much as one second, which is the maximum decoder buffer delay that is allowed in the STD
`
`.
`
`so
`
`55
`
`NTFX-1002 / Page 829 of 1867
`
`

`
`EP 0 805 598 A1
`
`model. Sirrilar to the STD model is a Video Buffering Verifier (VBV) which. as stated in the Video Section:
`
`Constant rate coded video bitstreams shall meet constraints imposed through a Wdeo Buffering Verifier (VBV)
`defined in this clause....
`
`The VBV is a hypothetical decoder} which is conceptually connected to the output of an encoder...Coded data is
`removed from the butter as defined below. it is required that a bitstream that conforms to this specification shall not
`cause the VBV to overllow. When low_delay equals 0. the bitstream shall not cause the VBV buffer to underflow...
`
`A high-level illustration of an exemplary STD model operating in conjuntion with an encoder is shown in Figure 2.
`The requirement that the VBV butter or STD model decoders not undenlow is very inportant as product quality is
`at stake. in order to maintain constant bitrate video. "stuffing" is implemented within various aspects of the system.
`"Stuliing" is the act of filling the data stream with "don't care‘ information sinply to maintain the required bit-rate.
`For Transport Stream packets carrying PES packets. stuffing is used when there is insufficient PES packet data to
`fill the Transport Stream packet payload bytes to a level tfat would support the transmitted data rate.
`Stuffing. for exanple. can be accomplished by defining an adaptation field longer than the sum of the lengths of the
`data elements in it. so that the payload bytes remaining after the adaptation field exactly accommodates the available
`PES packet data. The extra space in the adaptation field andlor payload can be filled with stuliing bytes.
`Figure 3 shows the format and field locations for a Transport Packet Stream where each Transport Packet includes
`a Header and a Payload. The header of a Transport Packet includes fields for indicating the existence and controlling
`the length and content of an adaptation field Within that adaptation tield. another field is designated as “stuffing bytes".
`Stuffing bytes are similarly used in the payload of the Transport Packets.
`'
`
`SUMMARY OF THE INVENTION
`
`30
`
`The present invention. in a system including variable bit-rate video data in the form of data packets which uses
`stuffing bytes tofill a data stream, is directed to a method and system for removing the stuffing bytes and using the addi-
`tional bandwidth to transmit private data (hereinafter ‘privatestutf data‘). The invention includes examining means for
`examining a data packet which includes an indication of whether stulting bytes are being used in the data packet and I
`determining if the data packet is eligible. according to predetermined
`to have the stuffing bytes removed: and
`re-multiplexing means. responsive to the eramining means. for removing the stuffing bytes from the data packet and
`adding predetermined privatestuff data to the data packet.
`in one aspect of the invention. the stuffing bytes are removed from a header portion of the data packet in order to
`gain additional transmission bandwidth.
`In another aspect of the present invention, the stuffing bytes are removed from a payload portion of the data packet
`in order to gain additional transmission bandwidth. in both aspects, however. the privatestutf data is inserted into the
`header portion of the data packet.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The invention is best understood from the following detailed description when read in connection with the accom-
`panying drawing, in which:
`I
`
`Figure 1 (prior art) shows a lrigh-level block diagram of an exerrplary digital multi-program transmission and receiv-
`ing system.
`
`Figure 2 (prior art) shows a high-level block diagram of an exemplary implementation of a STD model with portions
`of the system shown in Figure 1.
`
`Figure 3 (prior art) shows an exen-plary fonnat. including tield designations for a Transport Packet Stream used in
`conjunction with the system shown in Figures 1 and 2.
`
`figure 4A shows a high~level flowchart diagram illustrating the exemplary steps executed by the present invention
`to generally replace stuffing bytes with privatestufi data.
`‘
`
`NTFX-1002 / Page 830 of 1867
`
`

`
`EPo805598A1
`
`-
`
`,'-
`
`5
`
`10
`
`15
`
`25
`
`so
`
`as
`
`40
`
`45
`
`so
`
`Figure 48 shows a high-level flowchart diagram illustrating the exemplary steps executed by the present invention
`to replace stuffing bytes in the adaptation field with privatestufi data.
`
`Figure 5 shows a flowchart diagram illustrating the exemplary steps executed by the present invention to replace
`stuffing bytes in the packet payload with privatestufi data,
`
`Figure 6 shows a flowchart diagram illustrating an exemplary start code processing technique suitable for use with
`the embodiment shown in Figure 5.
`
`Figure 7 shows a flowchart diagram illustrating an exerrplary stuffing bytes searching technique suitable for use
`with the errbodiment shown in Figure 5.
`
`Figures 8A through 80 illustrate three examples of stuffing byte replacements as carried
`trated ln Figures 5-7.
`
`by the techniques illus-
`
`Figure 9 shows a high-level functional block diagram of an exemplary errbodiment of an encoder suitable for use
`with the present invention.
`
`DETAILED DESCRIPTION
`
`As mentioned in the BACKGROUND section. for Transport Stream packets carrying PES packets. stuffing is used
`when there is insuttioient PES packet data to fill the Transport Stream packet payload bytes to support the established
`data rate. One way stufiirig can be accorrplished is by defining an adaptation field that is longer than the sum of the
`lengths of the data elements in it. so that the payload bytes remaining after the adaptation tield exactly accommodates
`the available PES packet data. The extra space in the adaptation field is filled with stuffing bytes. Another way stuffing
`can be acoornplished is by filling unused portions of the Transport Payload with zeros.
`The present invention. generally applicable to variable bit-rate video. takes advantage of the otherwise wasted
`resources dedicated to 'stulfing' in order to insert private data. In the exenplary errbodiment of the present invention.
`to take advantage of these otherwise wasted resources. useful private data is inserted in the Transport Stream instead
`of the stuffing bits That is to say. effectively, a re-multiplexing operation occurs where. based on the existence of certain
`predetermined conditions in the fields of the Transport Stream (e.g.. stuffing bytes exist). the information necessary to
`replace stuffing bytes with primate data yet still conply with the standard is generated and appropriately inserted.
`It should be noted that. although the present invention is descrbed as being generally arplicable to variable bit rate
`video. it essentially is also applicable to constant bit rate video. That is to say. that. although. in the present invention.
`the modilied video will always be variable bit rate video. the original video to be processed and transmitted may be
`either constant or variable bit rate video.
`The data that is used to replace the stufling bytes generally will be referred to as 'privatestufl" data in order to dis-
`tinguish it from typical private data which may otherwise be encoded into a 1'ransport Stream.
`when privatestutt data is inserted in the Transport stream. it necessary. it can be sent with an individual program
`identification (PID) code indicating that the present Transport Packet includes privatestufl data As described in the Sys-
`tems Section. a PID is a 13-bit field in a Transport Stream Header. indicating the type of data stored In the packet pay-
`load. Some PID values are assigned and some are reserved. In the exerrplary errbodiment of the present invention.
`newly assigned PID values can be designated to indicate that the private data included in the particular Transport
`Packet is actually privatestutt data rather than normal private data. If a newly assigned PID is used. decoding of privat-
`estuti data may be easier on the receiving end.
`it should also be noted that. in addition to stuffing bytes. some Transport Padrets are designated NULL packets
`using a special NULL PID. Using the techniques described herein. the present invention could also take advantage of
`the wasted resources of a NULL packet by remultiplexing the packet to include privatestult data and all other appropri-
`ate fields (e.g.. adaptation and private data fields).
`Additionally. as the stufiing bytes are only used on an "as needed" basis. the privatestutl data is sent on a ‘bursty"
`basis. i.e. only when the video channel "wants" to send stuffing bytes. Exarrples of information which can be sent as
`privatestutt data include program reviews. program synopsis. etc for programs to be transmitted at a later time.
`As additional badrground. in the Systems Section, a syntax representation is provided for enoodingldecoding the
`adaptation field ot a Transport Header. This syntax is represarted below in Table l.
`'
`
`NTFX-1002 / Page 831 of 1867
`
`

`
`
`
`Mnemonic
`
`imsbf
`
`bslbf
`bslbf
`bslbf
`bslbf
`bslbf
`bslbf
`bslbf
`bslbt
`
`uimsbf
`bslbf
`uimsbf
`
`No.of
`Bits
`
`8
`
`9I
`
`-‘D40-4|-‘D-'9-5|-‘F-I
`
`33
`6
`
`33
`
`uimsbf
`
`EP0805598A1
`
`Table I -- Transpbrt Stream adaptation field
`
`Syntax
`
`(
`
`(
`adapcacion__fie1d ()
`adap:acion_£1e1d length
`if(adapca:iqn_£ie1d_1engch>o)
`discontinuity_indicacor
`random_access_indica:or
`elemencary_s:ream_priority;indicator
`PCR_f1ag
`=
`0PCR_£1ag
`sp1icing_poinc_£1ag
`transport_priva:e_data_£1ag
`adapcacion_£ie1d_extension_f1ag
`i£(PCR_£1ag == '1') (
`program_c1ocx_reference_base
`reserved
`
`p:ogram_c1ock_re£erence_gxtension
`
`} i
`
`.
`{
`f(OPCR__f1ag =3 '1’)
`origina1_program_c1ock_reference_base
`
`10
`
`30
`
`
`
`NTFX-1002 / Page 832 of 1867
`
`

`
`
`
`No.o£
`Bits mnemonic
`
`6
`
`9
`
`8
`
`.
`
`8
`
`8
`
`8
`1
`1
`1
`5
`
`1
`15
`
`2
`
`22
`
`4
`3
`1
`15
`1
`15
`1
`
`8
`
`a
`
`bslbf
`
`uimsbf
`
`tcimsbf
`
`uimsbf
`
`bslbf
`
`uimsbt
`bslbf
`bslbf
`bslbf
`bslbf
`
`bslbf
`uimsbf
`
`bslbf
`
`uimsbf
`
`bslbf
`belief
`bslbf
`bslbf
`bslbf
`bslbf
`be 1b£
`
`bslbf
`
`bslbf
`
`EP0805598A1
`
`Table I
`
`(continued)
`
`.
`Syntax
`
`'
`
`resenred
`
`original_progtam__c1ocl:_re£erence__excension
`
`} i
`
`f (sp1:i.cing_po:i.n:_£1ag --.-.
`sp1ice_’coun:down
`
`' 1')
`
`(
`
`)
`if (transport__private_dat:a_£1ag == ’ 1')
`transport_private_data._1ength
`for (5.-O; 1<transpor:_privat:e__data_1ength;i++) (
`privat:e_dat:a_by:e
`
`{
`
`}
`
`} i
`
`f (adapt:at:i.on_£ie1d_ext.ension_£1a.g == ' 1’ ) (
`adaptat:l.on_f i e1d_ext-.ension_1ength
`1t:w_£1ag
`p:i.ecewise_ra:e_f1ag
`seam1ess__splice_£1ag'
`reserved
`
`i£(ltv_f1a == '1')(
`1:-u_va.1id_f1ag
`1:w_o££set:
`
`} i
`
`}
`
`f (piecewise_rate_£1ag -- '1') {
`reserved
`
`piecewise__rate
`
`'1'){
`
`i£(seam1ess_sp1ice_f1ag I1:
`sp1:i.ce_type
`D'rS_next_a.u [32 . . 30]
`marl:er_bic
`D'rS__next__au [29 . . 151
`marke r_b.i. t;
`D‘!.'S__nex:__au [14 . . 01
`max-lce r_bi. :
`
`lf
`
`}
`
`or (i:0;i<N;i++){
`reserved '
`
`l f
`
`l
`
`or (i=0) :i<N;i++l{
`stuf£.i.ng_byte
`
`l
`
`}
`
`.
`As shown in Table I. stuffing bytes are placed into the Transport Header in the adaptation field as needed.
`Referring to the syntax. the adaptation_tield_length. listed in Table l and illustrated in Figure 3, is an 8 bit field spec-
`itying the number of bytes in the adaptationjield immediately following the adaptation_tield__length. For exanple. in the
`exemplary embodiment. the value 0 is used for inserting a single stufling byte in a Transport Stream packet.
`Moreover. when the edeptetion_field_oontrol value is '11‘. the value at the adeptation_tield_length shall be in the
`range of 0 to 182. when the adaptation_tield_oontnol value is '10’. the value of the ade.ptation_tield_length shall be 183.
`Astuffing_byte. torthe adaptation field. is afixed 8-bit value usually equalto'11111111'thatcan be inserted by
`
`5
`
`10
`
`'5
`
`20
`
`‘£5
`
`3°
`
`_
`
`as
`
`40
`
`‘5
`
`50
`
`ss
`
`
`
`NTFX-1002 / Page 833 of 1867
`
`

`
`the encoder. Once identified as stuliing bits. this 'don‘t care" infonnation is discarded on the reception and by the
`decoder.
`
`EP 0 805 598 A1
`
`5
`
`Confinuing with Table I, in addition to the stuffing bytes. the syntax of the standard for the adaptation field provides
`for private data. For exanple. as shown in Figure 3. the two fields immediately before the stuffing bytes field are desig-
`nated "5 flags‘ and ‘optional fields‘. The '5 flags‘ field indicates if the optional fields exist and. if so. the optional fields
`indicate the existence and length of ‘transport private data‘. This same interrelationship of the fields is also presented
`in syntactical format of Table l.
`in addition to the stuffing bytes used in the adaptation field of the Transport Header. as mentioned. stuffing bytes
`may also be used in the ‘Transport Payload. in the present invention. stuffing bytes from either the adaptation field or the
`10 Transport Payload can be removed to provide additional bandwidth for privatestuff data. it should be noted, however,
`whether stuffing bytes are removed from either the adaptation field or the Transport Payload or both. the privatestuff
`data added to the packet. in the exemplary embodiment of the present invention. is only added to an adaptation field in
`the ‘transport Header.
`Figure 4A shows a high-level flowchart illustrating exenplary steps executed for generally oonpleting a shifting
`byte removal and replacement operation. also known as a remultiplexing operation. This flowchart is intended to gen-
`erally illustrate stutling byte replacement in either the adaptation held or Transport Payload.
`As shown in Figure 4A. first, at step 402, a Transport Packet is captured from the Transport Stream and, then. at
`step 404. the Packet is examined. The examination includes determining if stuffing bytes exist. step 405. and. it so.
`where and how many. step 407. Then, using information obtained during the examination, at step 408. a remuttiplexing
`operation occurs. That is, the stuffing bytes are replaced with privatestuff data Additional details for the above steps
`are provided below with reference to Figures 48. 5-7 and 8A-8C.
`Figure 4B shows a high-level flowchart diagram illustrating the exenplary steps executed by the present invention
`to replace stuffing bytes in the adaptation field with privatestuff data. As shown in Figure 48, first, at step 410. a Trans-
`port Packet is captured from the Transport Stream amt. then, at step 412, the Packet is examined. The examination
`includes. initially. determining if an adaptation field exists. step 414. This can be acoorrplished by examining various
`fields. Next. it there is an adaptation field. then it is determined if private data exists in the adaptation field. step 416. In
`the exemplary errbooiment. this can be acconplished by examining the '5 flags‘ field and ‘transport private data
`length" field. lt private data exists. the process ends because this adaptation field. in the exemplary embodiment of the
`present invention. will not be used for privatestuff data. It should be noted. however. that although it is possible to insert
`privatestuff data when private data already exists. the present invention intentionally elects to not disturb an adaptation
`field wlich already contains private data.
`Continuing with the flowchart of Figure 4B. if private data does not exist. the location and nunber of stuffing bytes
`is determined. at step 418. using the information from the various fields and. at step 420. a renultiplexing operation
`occurs.
`
`15
`
`20
`
`25
`
`so
`
`-.
`
`as
`
`40
`
`45
`
`so
`
`55
`
`Now. proceeding to the remultiplexing operation. it is important to remenber that any modification of the adaptation
`field should adhere to established standards (i.e.. the forrrats shown in Figure 3 and the syntax listed in Table I). There-
`fore. it stuffing bytes in an adaptation field are to be replaced with privatestuff data. not only is the privatestuff data l‘tI.ll-
`tiplexed into the data stream but all of the appropriate bits in the appropriate fields are set accordingly. For exarrple if.
`in a particular adaptation field. no optional fields had existed in the initial examination. the '5 flags" field is modified to
`reflect that after the re-rnultiplex operation. the optional fields do exist Furthermore. the nunber of privatestuff bytes is
`added to the "transport private data length" field in order to property indicate the modification of the adaptation field.
`Being aware of the established standards such as those herein incorporated by reference. one of orcfinary skill in
`the art would appreciate the various conbinations of fields which may exist when attempting to replace stuffing bytes
`with privatestuffdata. thereby. understanding that all necessaryfields would have to be modified during the re-rnultiplex-
`ing operation to reflect the updated content of the adaptation field.
`Notwithstanding the abilities of one of ordinary skill in the art, by way of exarmle. Figures 5-7 and 8A-8C showtlow-
`charts and field replacement diagrams detaifing the steps necessary to detect and remove stuffing bytes from a single
`Transport Payload and utilize the consequent extra bandwidth to tr
`' privatestuff data in an adaption field at the
`transport layer.
`It should be noted that the method. illustrated in Figures 5-7 and 8A-8C.
`
`‘
`
`1) finds the minimum required number of video stuffing bytes within a transport packet based on the structure of its
`adaptation field (described above).
`
`2) locates the positiorls of video stuffing bytes within a transport payload and removes these bytes from the pay-
`load.
`
`3) inserts private data at the adaptation field. and
`
`NTFX-1002 / Page 834 of 1867
`
`

`
`EP0805598A1
`
`4) locates a picture header structure and replaces its VBV_DELAY value with oxFFFF.
`
`it should be further noted that the method illustrated in Figures 5-7 and 8A-8C assumes that
`
`1) the program association table (PAT) and program map table (PM1') have been processed and the PID for the tar-
`geted video elementary stream has been recognized and
`A
`
`2) the parsing of the payload of the Transport Payload containing the video elementary stream starts at or before
`the very first video sequence header and ends at the sequence end code.
`
`Turning to Figure 5-7. the illustrated processes examines many fields within the Transport Header and Payload as
`well as the tracking of some field sizes. For that purpose, variables are used throughout the process. In particular. the
`variable legends provided beiowfor each of Figures 5, 6 and 7. show the particular variables used for thatflowchart and
`their corresponding definitions:
`
`1!.” H .5 5
`
`Cnt__V__ZB
`StitCd_Fnd
`PiotStrtCd_Fnd
`MlN_V_SB
`Cnt_V_SB
`Cnt__PyId_B
`
`»
`
`as Counter of video zero bytes
`- Flag to indicate a finding of a start code
`- Flag to indicate a finding of a picture start code
`= Minimum nurriber of video stuffing bytes that need to exist in the TP packet
`is Counter of video stuffing bytes
`a counter of the TP payload bytes
`
`5
`
`10
`
`15
`
`20
`
`5
`
`PictStrtCd__Fnd
`StrtCd_Fnd
`Cnt_Pict_Hdr
`
`= Flag to indicate a finding of a picture start code
`= Flag to indicate a finding of a start code
`a counter of bytes in a picture header structure
`
` fl1@l
`
`Cnt_Pylii_B a counter of the TP payload bytes
`Cnt_V_SB
`= counter of video stuffing bytes
`35 Cnt_V_Z8
`= Counter of video zero bytes
`Cnt_Cur_SB = Counter of current video stuffing bytes
`StrtCd._Fnd
`= Flag to intficate a findi