United States Patent [19J
`Fujii et al.
`
`111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US005898695A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,898,695
`Apr. 27, 1999
`
`[54] DECODER FOR COMPRESSED AND
`MULTIPLEXED VIDEO AND AUDIO DATA
`
`[75]
`
`Inventors: Yukio Fujii, Yokohama; Masuo Oku,
`Kamakura, both of Japan
`
`[73] Assignee: Hitachi, Ltd., Tokyo, Japan
`
`........................... 348/423
`9/1997 Okada et a!.
`5,668,601
`5,677,980 10/1997 Naoe ....................................... 348/384
`2/1998 Kikuchi et a!. ......................... 348/466
`5,719,646
`
`FOREIGN PATENT DOCUMENTS
`
`0679028A2 10/1995 European Pat. Off .......... H04N 7/64
`0679035A2 10/1995 European Pat. Off .......... H04N 7/64
`
`[21] Appl. No.: 08/622,622
`
`[22] Filed:
`
`Mar. 27, 1996
`
`[30]
`
`Foreign Application Priority Data
`
`Mar. 29, 1995
`Mar. 29, 1995
`
`[JP]
`[JP]
`
`Japan .................................... 7-071131
`Japan .................................... 7-071132
`
`Int. Cl.6
`....................................................... H04N 7/12
`[51]
`[52] U.S. Cl. ............................. 370/464; 348/423; 348/10
`[58] Field of Search ..................................... 348/423, 467,
`348/385, 382, 7, 10, 714; 370/389, 464,
`522,503,428,457,509,535,536,537
`
`[56]
`
`References Cited
`
`OTHER PUBLICATIONS
`
`P. Stamminitz, et al, "Hardware Implementation of the
`Transport Stream Demultiplexer for the HDTV r Demonstra(cid:173)
`tor", Signal Processing of HDTV, VI., Proceedings of the
`Int'l Workshop on HDTV, 1995, pp. 435-441.
`van den Hurk, et al, "A Concept for Source Decoding in
`Digital Video Broadcast Applications", Int' l Conference on
`Consumer Electronics-Digest of Techni Papers, Jun. 7-9,
`1995, Conf. No. 14, IEEE, pp. 260-261.
`
`Primary Examiner-Benedict V. Safourek
`Assistant Examiner-Ken Vanderpuye
`Attorney, Agent, or Firm--Antonelli, Terry, Stout & Kraus,
`LLP
`
`U.S. PATENT DOCUMENTS
`
`[57]
`
`ABSTRACT
`
`5,475,688
`5,475,754
`5,521,918
`5,521,979
`5,559,999
`5,613,003
`5,635,979
`5,649,029
`5,666,293
`
`12/1995 Bridgewater et a!. .. ... ... .... ... .. 370/94.1
`12/1995 Bridgewater et a!. .. ... ... .... ... ... .. 380/20
`5/1996 Kim ........................................ 348/423
`5/1996 Deiss .. ... ... ... ... .... ... ... ... ... ... .... ... . 380/20
`9/1996 Maturi eta!. ........................... 370/389
`3/1997 Bridgewater et a!. .................... 380/20
`....................... 455/4.2
`6/1997 Kostreski et a!.
`7/1997 Galbi ....................................... 382/233
`9/1997 Metz eta!. ................................ 348/10
`
`An apparatus for filtering TS packets multiplexed with a
`plurality of programs and sending the filtered packets to
`decoders. A packet landing buffer is provided in a RAM used
`for a microprocessor for system control. After a channel is
`selected, the microprocessor filters video and audio data and
`performs a value added service process.
`
`38 Claims, 23 Drawing Sheets
`
`r
`
`1
`)
`
`I
`
`2
`)
`
`14
`
`TUNER ~ LA TOR
`
`DEMODU- ~ INTERFACE
`
`UNIT
`
`~
`
`-
`
`VIDEO
`DECODE
`BUFFER
`
`r---9
`
`t ~8
`
`VIDEO _.....
`DECODER
`
`c
`
`AUDIO
`DECODER
`
`r---10
`~
`
`12
`(
`
`MICRO-
`PROCESSOR
`
`0
`
`~ ~
`ClJ
`c
`(/)
`
`CLOCK
`4--------- GENERATOR 1--
`
`13
`(
`USER
`------z__-j INTERFACE
`UNIT
`
`RAM
`
`LANDING r 1
`
`PACKET
`
`..__-7
`7
`BUFFER ~
`SYSTEM CONTROL f-- ..__-7
`2
`PROGRAM+ DATA
`
`SYSTEM
`DECODE
`BUFFER
`
`3
`f.--- ..__-]
`
`AUDIO DECODE
`f.---
`+ DELAY BUFFER
`
`..__-7
`4
`
`SONY EX. 1004
`Page 1
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`
`"""" 0 ......,
`~ .....
`'JJ. =(cid:173)~
`
`'0
`'0
`""""
`'0
`~-..J
`N
`
`~ :-;
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`4
`
`,__.
`
`+ DELAY BUFFER
`AUDIO DECODE l---
`
`3
`
`,__.
`
`I.-
`
`BUFFER
`DECODE
`SYSTEM
`
`2
`
`,__.
`
`7
`1
`
`...---...._
`
`,_
`
`PROGRAM+ DATA
`SYSTEM CONTROL .--
`
`BUFFER
`LANDING
`PACKET
`
`RAM
`
`--
`
`~
`
`en
`c
`OJ
`
`---------10
`
`DECODER
`
`AUDIO
`
`I
`
`r--
`
`-z___. INTERFACE
`
`UNIT
`
`USER
`(
`13
`
`4-----
`
`4~ GENERATOR
`
`CLOCK
`
`PROCESSOR
`
`MICRO-
`
`~ ,
`
`12
`
`~
`
`r-...--8
`
`DECODER
`
`VIDEO
`
`..
`
`~
`
`--
`
`INTERFACE
`
`UNIT
`
`......
`
`LA TOR
`DEMODU-
`
`......
`
`TUNER
`
`r-...--9
`
`BUFFER
`DECODE
`VIDEO
`
`FIG.1
`
`SONY EX. 1004
`Page 2
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`0 ......,
`N
`~ .....
`'JJ. =(cid:173)~
`
`'0
`'0
`""""
`'0
`~-..J
`N
`
`~ :-;
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`~7
`
`1--+---73
`
`r----.-11
`
`BUFFER
`DECODE
`AUDIO
`
`~10
`
`J
`
`DECODER
`
`AUDIO
`
`....
`
`1-------B
`
`DECODER
`
`VIDEO
`
`BUFFER
`DECODE 1------.-9
`VIDEO
`
`1
`
`PROGRAM+ OAT A
`SYSTEM CONTROL 1-+---72
`
`•I
`
`~INTERFACE I•
`
`USER
`
`13
`)
`UNIT
`
`BUFFER
`DECODE
`SYSTEM
`
`RAM
`
`•I
`
`en I•
`CD c
`~
`
`0
`
`--..-
`
`6
`~
`BUFFER
`::0
`f:i ~
`LANDING
`"'0 r m
`PACKET
`Ci
`c
`~ ~
`0
`
`BUFFER
`LANDING
`PACKET
`~
`5
`
`r_.-4
`
`CLOCK
`
`GENERATOR
`
`FIG.2
`
`I
`
`•
`
`PROCESSOR 1 4
`
`MICRO-
`
`~ 12
`
`3~
`
`LA TOR
`DEMODU-r-.
`
`~
`2
`
`TUNER
`~
`1
`
`y
`
`SONY EX. 1004
`Page 3
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 3 of 23
`
`5,898,695
`
`FIG.3A
`
`TS PACKET
`
`1 188 1 188
`
`188 1 188 1 188
`
`188
`
`__ _..TIME
`
`PAYLOAD
`
`PACKET I D (PI D)
`
`PES PACKET I
`
`PES PACKET
`
`' ' ' '
`
`' '
`
`..
`. PES
`HEADER ...
`
`PROGRAM ELEMENT
`
`VIDEO,AUDIO,TEXT DATA OR THE LIKE
`
`TS PACKET
`
`188 1 188
`
`FIG.3B
`
`188 1 188 1 188
`'
`
`188
`
`--~TIME
`
`PAYLOAD
`
`PACKET ID (PID)
`
`SECTION
`
`SECTION
`
`SECTION
`
`SECTION
`HI:ADE:R
`
`PROGRAM/SERVICE INFOMATION
`
`PROGRAM MAP TABLE OR THE LIKE
`
`SONY EX. 1004
`Page 4
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 4 of 23
`
`5,898,695
`
`PID
`TSHEADER
`
`FIG.4
`
`PACKET LANDING TIME
`
`Video#1
`
`Video#2
`
`Video#1
`
`Audio#1
`
`PSI
`
`Video#2
`
`0
`
`1
`
`2
`
`3
`
`4
`
`5
`
`TS LANDING
`BUFFER ROW
`ADDRESS
`
`PACKET
`LANDING
`BUFFER
`
`I
`'.<'
`
`:.,::
`
`PSI
`
`Video#1
`
`188 BYTES
`
`~
`~
`~
`r-.
`
`N-2
`
`N-1
`
`SONY EX. 1004
`Page 5
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 5 of 23
`
`5,898,695
`
`FIG.5
`
`INTERFACE
`UNIT
`
`TS PACKET
`
`TSCLOCK
`
`TS HEADER
`DETECTOR
`
`t
`
`14
`f
`140
`
`;141
`
`TRANSFER
`BUFFER
`
`BUSCLOCK
`
`MICRO-
`PROCESSOR
`
`INTERRUPT
`SIGNAL
`
`DREQ
`
`DACK
`
`CJ)
`::J
`c::l
`<(
`
`'-
`12~
`
`TRIGGER f123
`4
`1
`)
`I TIMER I
`ONE LINE
`I
`ADDITION
`..__
`CLOCK
`WRITE
`GENERATOR
`ADDRESS
`COUNTER ~122
`
`1
`
`DMA
`CONTROLLER
`
`~~ ~ 121 ~
`
`REGISTER
`5
`124
`
`WRITE ADDRESS
`RAM
`7~
`
`PACKET LANDING
`BUFFER
`
`71
`
`PACKET
`LANDING
`TIME
`
`;ACKET,DATA
`
`Video#1
`Video#2
`
`'
`
`SONY EX. 1004
`Page 6
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 6 of 23
`
`5,898,695
`
`TS LANDING BUFFER
`ROW ADDRESS (_
`N-2
`
`FIG.6
`
`N-1
`
`0
`
`1
`
`PACKET READ
`
`2 &READADDRESS
`
`::::5!';: 3\
`
`PACKET NOT
`READ YET
`
`. 6\J
`/ VWRITE ADDRESS
`
`SONY EX. 1004
`Page 7
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 7 of 23
`
`5,898,695
`
`FIG.?
`
`PACKET FILTERING
`PROCESS (PROGRAM/
`ELEMENT FILTER)
`
`S1
`
`PI D#program#k
`
`CHECK
`PROGRAM
`#k
`
`PROGRAM
`REFERENCE MAP TABLE
`
`- - - - - - - - 1--t---n
`
`PID=program#k
`
`83 REFERENCE
`
`I
`CLOCK ~
`• '
`
`S4
`
`YES
`
`CLOCK
`RECOVERY
`PROCESS
`
`REFERENCE?
`(PCR_PID)
`
`NO
`
`REFERENCE
`
`S5
`
`PI D=Video#k
`
`ELEMENT/
`PSI FILTERING
`
`PID=PSI
`
`S6
`
`S7
`
`SB
`
`VIDEO PACKET
`TRANSFER
`PROCESS
`
`AUDIO PACKET
`TRANSFER
`PROCESS
`
`PSI PACKET
`PROCESS
`
`ROW ADDRESS++
`
`S9
`
`'
`'
`'
`'
`'
`'
`'
`'
`'
`'
`'
`'
`'
`'
`:uPDATE
`'
`'
`'
`' I
`'
`'
`' I
`
`I
`I
`
`'
`' I
`'
`'
`'
`'
`
`-~_I
`
`SONY EX. 1004
`Page 8
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 8 of 23
`
`5,898,695
`
`FIG.8
`
`PICTURE DATA
`
`DECODE INPUT 6 i
`
`Video
`
`DECODESTART~, ----~
`·
`DISPLAY START 1 - l - - - . -___ ___ J
`
`r - ' ____.;__ _
`
`__ ,
`
`vbv BUFFER DELAY Tvbv o------.
`
`DISPLAY TIMING DELAY
`
`EXTERNAL DEVICE DELAY
`
`Tdisp i - - - -.. ...::
`
`I
`I
`I
`
`Text:-----:
`
`I
`I
`
`I
`I
`I
`
`I VIDEO
`............_ __ __, OUTPUT
`
`I
`I
`I
`I
`
`VIDEO SYSTEM Tvid !-----~ioo!,
`DELAY
`
`I
`
`DELAY FOR A-V
`'Tadj 1
`. . I SYNC COMPENSATION
`
`•
`
`AUDIO SYSTEM T d t - - - . - !
`DELAY
`au
`
`DECODE STARTOb---fAUDIO FRAME DATA
`
`I
`I
`
`Audio
`
`I
`I
`
`DISPLAY START
`
`DISPLAY TIMING DELAY
`
`EXTERNAL DEVICE DELAY
`
`I
`I
`I
`I
`
`•
`
`TdiSp :-----'
`
`I
`I
`I
`
`'
`
`I
`I
`I
`
`•Text...-..:
`
`!-----~AUDIO OUTPUT
`
`I
`I
`I
`I
`I
`
`- - -.. TIMEt
`
`t=PTS(v)=PTS(a)
`
`SONY EX. 1004
`Page 9
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 9 of 23
`
`5,898,695
`
`FIG.9
`
`CLOCK
`RECOVERY
`PROCESS
`
`DIFcur<-PCR VALUE(cid:173)
`ARRIVAL TIME
`
`DIFFERENCE
`<-DIFcur-DIF re
`
`S4
`
`S11
`
`S12
`
`S13
`
`ERR>
`THRESHOLD
`
`YES
`>-------------~
`
`S14
`
`POSITIVE
`
`S15
`
`ACCELERATE
`CLOCK
`FREQUENCY
`
`ERR
`
`0
`
`NEGATIVE
`
`S16
`
`DECELERATE
`CLOCK
`FREQUENCY
`
`DIFpre<-DIFcur
`
`S17
`
`S18
`
`SONY EX. 1004
`Page 10
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 10 of 23
`
`5,898,695
`
`FIG.1 0
`
`PCR VALUE
`TIMER COUNTER VALUE
`
`,
`
`,
`
`_
`
`___,.,~ Tl ME
`
`SONY EX. 1004
`Page 11
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`
`'"""' 0 ......,
`'"""'
`~ .....
`'JJ. =(cid:173)~
`
`~ :-:
`
`'0
`'0
`'"""'
`'0
`~-..J
`N
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`+ DELAY BUFFER
`AUDIO DECODE H--.--7 4
`
`1---i---73
`
`BUFFER
`DECODE
`SYSTEM
`
`PROGRAM+ DATA
`SYSTEM CONTROL H---72
`
`PACKET lt1
`
`BUFFER
`LANDING
`
`RAM
`
`7
`
`Ill
`
`CD c
`• I)>
`~
`0
`
`(J)
`
`13
`~
`
`~INTERFACE I•
`
`UNIT
`
`USER
`
`I •
`
`PROCESSOR
`
`MICRO(cid:173)
`
`12~
`
`DEMODU-~
`
`2
`)
`LA TOR
`
`GENERATOR
`
`CLOCK
`
`~1
`
`TUNER
`
`T
`
`~10
`
`DECODER
`
`AUDIO
`
`~DECODER
`
`~a
`
`VIDEO
`
`.......-
`,...1 __. __
`
`~ INTERFACE
`
`UNIT
`~
`14
`
`BUFFER
`DECODE ~9
`VIDEO
`
`~4
`
`FIG.11
`
`FILTER
`PACKET
`PROGRAM
`
`15
`1
`
`t
`
`SONY EX. 1004
`Page 12
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 12 of 23
`
`5,898,695
`
`FIG.12A
`
`188
`
`188
`
`188
`
`188
`
`188
`
`---- I V-#j I V-#k I A-#i I V-#k I PSI
`
`188
`
`V-#j
`
`188
`
`188
`
`188
`
`V-#i 1----
`I V-#i I A-#k I
`
`FIG.12B
`
`188
`
`I V-#k I
`
`188
`
`188
`
`I V-#k I PSI I
`
`TIME
`
`TIME
`
`188
`
`I A-#k I
`
`FIG.13
`
`PID
`
`TS~HEADER
`
`'
`
`Video#k
`
`Video#k
`
`Video#k
`
`Audio#k
`
`PSI
`
`Video#k
`
`PSI
`
`Video#k
`
`188 BYTES
`
`0
`
`1
`
`2
`
`3
`
`4
`
`5
`
`N-2
`
`N-1
`
`PACKET
`LANDING
`BUFFER
`
`PROGRAM
`#k
`SELECTED
`
`TS LANDING
`BUFFER
`ROW
`ADDRESS
`
`SONY EX. 1004
`Page 13
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 13 of 23
`
`5,898,695
`
`FIG.14
`
`ERR
`
`_) 4
`CLOCK
`GENERATOR
`PROGRAM
`PACKET FIL TEA I
`•
`15~ '--1 PCR
`153
`
`TSP ACKET
`
`TS CLOCK
`
`t ~
`
`.__./ 154
`COMPA- ~
`COUNTER
`RATOR
`INTERFACE
`151_'\ TRIGGER 152'\. PCR
`UNIT
`PID
`TS HEADER
`FILTER ~ ~
`DETECTOR
`TRANSFER
`BUFFER
`
`14
`
`BUSCLOCK
`
`'""
`
`141
`
`(j)
`::J
`co
`<(
`I-
`<(
`0
`
`INERRUPT SIGNAL
`MICRO-
`PROCESSOR..--
`
`_/
`
`PID
`
`DREQ
`
`DACK
`
`DMA
`-1
`CONTROLLER
`
`123\
`121
`ONE
`REGISTER l
`12~ LINE
`ADDITION
`WRITE
`ADDRESS
`COUNTER
`
`~122
`
`ADDRESS
`RAM
`7~
`
`PROGRAM
`MAP TABLE
`
`PACKET LANDING BUFFER
`71
`Video#1 V
`
`.
`
`-..
`
`Video#2
`
`SONY EX. 1004
`Page 14
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 14 of 23
`
`5,898,695
`
`FIG.15
`
`ELEMENT
`FILTERING
`PROCESS
`
`S1
`--(cid:173)
`
`.. -...
`
`REFERENCE·-
`
`PID=Video#k
`
`ELEMENT/
`PSI FILTERING
`
`PID=PSI
`
`S6
`
`PI D=Audio#k
`
`S7
`
`sa
`
`VIDEO PACKET
`TRANSFER
`PROCESS
`
`AUDIO PACKET
`TRANSFER
`PROCESS
`
`PSI PACKET
`PROCESS
`
`ROW ADDRESS++
`
`S9
`
`!
`I
`'
`I
`I
`
`' I
`:UPDATE
`
`I
`I
`I
`I
`I
`I
`I
`
`-- _,
`
`SONY EX. 1004
`Page 15
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`0 ......,
`"""" Ul
`~ .....
`'JJ. =(cid:173)~
`
`~ :-:
`
`'0
`'0
`""""
`'0
`~-..J
`N
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`73
`
`,.....-
`
`BUFFER
`DECODE
`SYSTEM
`
`-
`
`r-
`
`2
`
`7
`1
`
`PROGRAM+DAT A
`SYSTEM CONTROL I-
`
`I--I'
`
`BUFFER
`LANDING
`PACKET
`
`RAM
`
`r-.---16
`
`--
`
`INTERFACE
`
`DISPLAY
`
`UNIT
`
`-
`
`--
`
`OJ c
`)>
`~
`0
`
`(fJ
`
`~
`
`.---
`
`-
`
`__.,
`
`"
`
`13
`
`[
`
`--
`
`-
`
`-
`
`--
`
`-----z__.. INTERFACE
`
`USER
`
`UNIT
`
`PROCESSOR -
`
`-
`
`-
`
`12~ MICRO-
`t
`
`~4
`
`FIG.16
`
`CLOCK
`
`GENERATOR
`
`INTERFACE
`
`~
`
`t
`
`15
`~
`FILTER
`PACKET ~ UNIT
`PROGRAM
`~
`14
`
`,....._.
`
`2
`~
`LATOR
`DEMODU-
`
`A
`
`r---1
`
`TUNER
`
`SONY EX. 1004
`Page 16
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`0 ......,
`'"""' 0'1
`~ .....
`'Jl =-~
`
`'0
`'0
`'"""'
`'0
`~-..J
`N
`:-:
`'t:l
`>
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`208
`
`207
`
`206
`
`I h
`
`DECODER
`
`VIDEO
`
`bus
`
`elk
`
`FIG.17
`
`DECODER
`
`AUDIO
`
`205
`I
`
`204
`
`203
`
`MEMORY
`PROGRA
`
`CESSOR
`PRO-
`I RAM I I MICRO-
`
`INT
`
`201
`
`200
`
`PLEXER
`I DEMODU-a CORRECTION c DEMULTI-
`CHANNEL
`
`LA TOR
`DEMODU-
`
`ERROR
`. -_/-__, b
`
`202
`
`I
`2
`
`LA TOR
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`I
`
`FROM TUNER
`
`SONY EX. 1004
`Page 17
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 17 of 23
`
`5,898,695
`
`~
`T'""
`f
`i
`
`CIJ
`::l
`..0
`
`C\1
`0
`C\1
`
`)
`
`Q)
`
`en
`I
`I
`;=:=::=:::::;1~1 (
`r-----L..L...I _ ____,
`1
`I
`w
`I <((.)
`J-<(
`<(u_
`I Oa:
`0~
`I a..:z
`I
`
`T"""
`
`0
`
`C\1 I a:
`
`~~ (.)<(
`oa::
`-IW oz
`w
`CJ
`
`I
`L __ _
`
`j
`-
`-
`-1-
`-
`- - - -1 - - - -
`
`-
`
`- -
`
`- - - - t - -
`
`.
`CJ
`LL
`
`ERROR FLAG
`~ ADDER
`~ L---,.-;;;,._-----1
`T'""
`
`C\1 .----1.....-------,
`,./
`MIXER
`cD'-r------.----r
`T"""
`C\1
`
`a:
`w
`~
`u..
`0
`a... I
`(/) 5 a...
`
`0
`T"""
`C\1
`
`a:
`w
`~
`u..
`a:
`(.)
`a...
`
`>-
`:5 w
`0
`(.) z
`>(cid:173)(/)
`
`I
`I
`I_
`
`\
`1.0
`
`..c
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`· - -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`--
`
`SONY EX. 1004
`Page 18
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 18 of 23
`
`5,898,695
`
`('I')
`I
`(.)
`
`......
`I
`(.)
`
`C\1
`I
`(.)
`
`·.·,•,·,•.•.•:-:-·:·:·.·.·!·:- ..... •.·
`..-
`I
`(.)
`
`:-:-:~-.· ·!·!·.·· ..... :·
`
`C\1
`I
`(.)
`
`......
`I
`(.)
`
`.
`CJ
`LL
`
`('I')
`I
`(.)
`
`......
`I
`(.)
`
`C\1
`I
`(.)
`
`......
`I
`(.)
`
`~
`~
`~ttS.···v~.~~ .... ·>:·~·······
`
`......
`I
`(.)
`
`.....
`I
`(.)
`
`., ..
`
`~-.· • •,•'•,-:• •,•.•,.•!•
`
`rr:
`L1J
`(/')
`ll..
`~l L1J
`l-2zzz:z.z:zl
`I
`
`0 -
`
`0
`N
`CJ
`LL
`
`C\1
`
`[]]
`
`0
`
`-
`
`-""0
`-
`
`SONY EX. 1004
`Page 19
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 19 of 23
`
`5,898,695
`
`FIG.21
`
`~
`
`FETCH DATA FROM CHANNEL
`DEMULTIPLEXER (DMAC)
`
`.....J
`0
`a:
`I-
`z
`0
`()
`~
`CJ)
`
`~
`
`-
`
`-
`
`SELECT AND SEND VIDEO
`AND AUDIO DATA
`
`1-
`w
`:::.::::
`Oa:::
`~w
`ANALYZE PSI DATA AND SEND PID
`0 ~
`W<(
`INFORMATION TO BE SELECTED
`Nw
`~I
`<( z
`
`<(
`
`t---
`
`ANALYZE Sl DATA AND
`GENERATE OSD DATA
`
`SONY EX. 1004
`Page 20
`
`

`
`U.S. Patent
`
`Apr. 27, 1999
`
`Sheet 20 of 23
`
`5,898,695
`
`CIJ
`::I
`.0
`
`a:
`<(W
`~u..
`0~
`co
`
`-
`
`CX>
`,.....
`C\1
`\,..,
`
`,.....
`C\1
`C\1
`
`I'-,.....
`C\1
`
`<0
`,.....
`C\1
`
`C\1
`C\1 .
`(!J
`LL
`
`C\1 I
`SELECTOR
`w w
`I ---~1-----1--~-------~<t: ~ <( ~u.. I-
`1- a: 0
`t------+---+-+-----+-1(§ 0 ~ ffi ~ 1---4-c..-
`~ ~
`ERROR FLAG
`ADDER
`
`C\1
`
`C"
`- 1-44--=-
`
`MIXER
`
`._1 u
`
`a:
`w
`~
`u:::
`Cl
`a..
`en
`) a..
`.----
`0
`,.....
`C\1
`
`I
`
`)
`,.....
`,.....
`C\1
`
`a:
`w
`~
`u..
`0
`a..
`I
`Cf)
`
`.----
`
`oa:
`a:
`ow
`w
`---
`~~ ~
`u..
`au..
`a:
`wo
`o-
`()
`a..
`-C..
`) >
`5
`.----
`C\1
`('t)
`,.....
`,.....
`C\1
`C\1
`
`.0
`
`u
`
`"0
`
`~
`
`C\1
`0
`C\1
`
`SONY EX. 1004
`Page 21
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`
`'"""' 0 ......,
`
`N
`~ .....
`'JJ. =(cid:173)~
`
`~ :-:
`
`'0
`'0
`'"""'
`'0
`~-..J
`N
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`FIG.23
`
`;218
`
`221
`
`217
`
`216
`
`21 0 '-"
`
`202 _____..._
`
`f.-+ m __..
`
`:D
`0
`~
`()
`r
`en m
`
`1..--..--
`G)
`:II~
`:0
`m"Tl
`m
`>< __., O:D
`oo
`s::
`:>JJ
`:D
`m
`
`:0
`m
`><
`s::
`
`PSI-PID FILTER I
`
`223 "-----
`
`VIDEO/AUDIO ~
`
`PID FILTER
`
`-
`
`224"'
`
`ERROR FLAG
`
`DELAY
`
`215"'
`
`SYNC DELAY
`
`PCR FILTER ~
`
`214"'
`
`213"'
`
`-
`
`.......
`
`PID FILTER
`VIDEO/AUDIO
`
`212""
`
`SI-PID FILTER I.-
`PSI-PID FIL TEA ~
`
`211 "'
`
`d
`
`c
`
`b
`
`q
`
`p
`
`226
`
`225
`
`INTERFACE
`STORAGE -----222
`
`UNIT
`__.. MEDIA
`DATA
`
`G)
`:II~
`m"Tl
`__.. O::II
`oo
`)>:D
`JJ
`m
`
`-
`
`e
`elk
`
`GENERATOR~220
`
`CLOCK
`
`-----219
`
`INTERFACE
`PID DATA -
`
`-..--
`
`bus
`
`BUFFER f--
`
`DMA
`
`SONY EX. 1004
`Page 22
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`0 ......,
`N
`N
`~ .....
`'JJ. =(cid:173)~
`
`'0
`'0
`'"""'
`'0
`~-..J
`N
`:-'l
`~
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`~-----OPCR
`
`L.....-----SPLICE COUNT-DOWN
`
`....____ __ PRIVATE DATA
`L_=-._ AF EXTENSION
`j_
`
`OPTION DATA* I STUFFING*
`
`PCR
`
`SOME CASES
`*ABSENT IN
`
`4 BYTES
`
`PRIORITY-------~
`
`RANDOM ACCESS_j
`I
`
`LENGTH TINUITY.
`
`IDISCON(cid:173)
`
`AF
`
`PAYLOAD
`
`AF*
`
`CONTROL COUNTER
`I CYCLIC
`
`AF
`
`PID
`
`ITY
`SYNC I ERROR I UNIT SCRAM-PRIOR-
`
`START
`
`BLE
`
`TS PACKET
`
`TS PACKET
`
`TS PACKET
`
`(188 BYTES)
`TS PACKET
`
`FIG.24
`
`SONY EX. 1004
`Page 23
`
`

`
`Ul
`\C
`0\
`....
`00
`\C
`00
`....
`Ul
`
`~
`N
`0 ......,
`~
`N
`~ .....
`'JJ. =(cid:173)~
`
`'0
`'0
`'"""'
`'0
`~-..J
`N
`;"l
`~
`
`~ = ......
`~ ......
`~
`•
`\Jl
`d •
`
`MAP TABLE
`PROGRAM
`PID= "Mk"
`
`MAP TABLE
`PROGRAM
`PID= "Mi"
`
`PMT·
`
`FIG.25
`
`TABLE 1\
`PID= "0" v PID= "Mj"
`
`MAP TABLE
`PROGRAM
`
`~
`
`ASSOCIATION
`
`PROGRAM
`
`SONY EX. 1004
`Page 24
`
`

`
`5,898,695
`
`1
`DECODER FOR COMPRESSED AND
`MULTIPLEXED VIDEO AND AUDIO DATA
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to decoding video and audio
`data which are multiplexed after compression coding, and
`more particularly to a receiver/decoder for receiving video
`and audio data compression encoded by high efficiency
`coding means and decoding the received encoded data.
`In order to reduce a transmission/record cost of a large
`amount of video data, the video data is compression encoded
`by high efficiency coding means which removes redundancy
`and thereafter the data is transmitted or recorded. An
`example of high efficiency coding means is the well known
`MPEG method standardized by ISO/SC29/WG11. In the
`MPEG method, an encoded data multiplexing method as
`well as the video and audio data encoding method are
`standardized as MPEG/Systems.
`In the fields of broadcasting and communications, in
`recent years, redundancy of moving picture video data is
`removed to compress the video data which is then transmit(cid:173)
`ted in the form of digital signals. For video data
`compression, discrete cosine transformation (DCT) and
`motion compensation prediction coding such as one stipu(cid:173)
`lated in MPEG specifications are generally performed. With
`high compression ratio of video data, a plurality of broadcast
`programs can be transmitted in a multiplex manner over a
`single transmission channel. A program means a set of video
`data and its associated audio and/or text data.
`Multiplexing of a plurality of programs according to the
`MPEG specifications is stipulated in ITU-T Rec. H.222.0,
`ISO!IEC13818-1, 1994 (E), "Information Technology(cid:173)
`Generic Coding of Moving Pictures and Associated Audio(cid:173)
`Part 1: Systems", pp. 9-21, wherein it is described that a
`transport stream (hereinafter abbreviated as TS) packet is
`used for such multiplexing which has a fixed length of 188
`bytes. The structure of an apparatus called a set top box is
`shown in the block diagram of FIG. 2. This apparatus
`separates TS packets formed in accordance with the MPEG
`specifications and supplied from a broadcast station or the
`like, and supplies them to a video decoder and an audio
`decoder to output decoded video and audio signals. Con(cid:173)
`ventional techniques will be described with reference to
`FIG. 2.
`A tuner 1 selects data of one channel transmitted from a
`communications medium such as a CATV and a communi(cid:173)
`cations satellite, and supplies the selected channel data to a
`demodulator 2. The demodulator 2 demodulates the channel
`data which was channel encoded by QAM, QPSK, or the
`like, performs an error correction process by using redun(cid:173)
`dancy codes, and supplies the processed data to a demulti(cid:173)
`plexer 3. The processed data is a bit stream data of the TS
`packet format which is shown in FIGS. 3A and 3B. The
`formats are classified into two types shown in FIGS. 3A and
`3B according to the contents of a TS packet. The format
`shown in FIG. 3Ais used for transmitting program elements,
`including video data, audio data, text data such as teletext,
`and other data. Each TS packet of 188 bytes is constituted by
`a transport stream header (abbreviated as TS header) and a 60
`payload containing elements described above. The TS
`header always contains a packet ID (abbreviated as PID)
`representative of the attribute of the TS packet, and some(cid:173)
`times contains a program clock reference (abbreviated as
`PCR) which is time information used for a decoder to 65
`recover a system clock used as a time reference when each
`element was encoded. The payload is part of a packetized
`
`2
`elementary stream (PES). The PES packet is a variable
`length packet constituting an element unit determined by
`each element, the type of data storage media, and the like.
`The PES packet is constituted by data of each element and
`5 a PES header. The PES header includes a stream ID for
`describing the element contents, time stamp information
`(PTS) for describing a PES packet length and a time when
`the element is displayed, and other pieces of information.
`The element unit designated by PTS is called an access unit.
`10 rr~:neex~~h~\~~t f~~c~:~i~s ~~~~~ef~~:~~0s~;~~ ~~dF~~~
`3B is used for program specific information (abbreviated as
`PSI) which is additional information used for the system
`control. The payload of the TS packet is part of PSI
`described in units of sections. Each section is constituted by
`15 a section header, PSI, and cyclic redundancy check (CRC)
`used for error correction. The section header indicates the
`attribute of the succeeding PSI and a section length. PSI is
`structured hierarchically and contains information necessary
`for the system control, such as program association table
`20 (PAT) describing program information (specifically, PID of
`PMT to be described later) contained in bit stream data
`transmitted as TS, and a program map table (PMT) describ(cid:173)
`ing a correspondence between the element and PID in each
`program. The demultiplexer 3 shown in FIG. 2 receives a TS
`25 packet, and supplies PSI data via a data bus to a system
`decode buffer allocated in a RAM 7, and video and audio
`data which are elements constituting a program selected by
`a user, respectively to a video decoder 8 and an audio
`decoder 10. The demultiplexer 3 samples time information
`30 from the header of the TS packet containing PCR, and
`supplies control signals to a clock generator 4 to recover the
`system clock. A microprocessor 12 decodes the contents of
`PSI data sent to the system decode buffer in RAM 7 and
`stores the results in a system decode buffer 73 in RAM 7 in
`35 the data format usable by system control software programs.
`Software programs are generally stored in the same RAM or
`in a dedicated ROM. The work area of software programs is
`generally developed in RAM 72. In response to an instruc(cid:173)
`tion entered by a user via a user interface unit 13, the
`40 microprocessor 12 supplies PID which is used for deriving
`the TS packet of a program, to the demultiplexer 3, by using
`the decoded PSI data, and also supplies a control signal
`which is used for selecting the program, to the tuner 1. The
`video decoder 8 and audio decoder 10 decode the video and
`45 audio data in cooperation with a video decode buffer 9 and
`an audio decode buffer 11. Data transmission speed on a
`transport channel is different from the bit rate used when
`each element was encoded, because of multiplexing of
`programs. Therefore, if data is supplied at the data trans-
`50 mission speed directly to the video decoder 8 and audio
`decoder 10, the video decode buffer 9 and audio decode
`buffer 11 may locally overflow or underflow and video and
`audio outputs are disturbed. It is therefore necessary to insert
`packet transport buffers 5 and 6 between the demultiplexer
`55 and decoders to convert the bit rates in accordance with the
`capacities of the decoder buffers, and thereafter to supply
`element data to the video decoder 8 and audio decoder 10.
`Multiplexing according to the MPEG specifications assume
`that a packet transport buffer having a capacity of 512 bytes
`per each element is provided.
`The present inventors have found out problems of an
`increased number of system components and an increased
`cost to be caused by the provision of the packet transport
`buffers 5 and 6 independently from the system memory.
`Even if the packet transport buffers are implemented in the
`demultiplexer 3, the increased circuit scale and an increased
`cost thereof are inevitable.
`
`SONY EX. 1004
`Page 25
`
`

`
`5,898,695
`
`3
`The MPEG specifications standardize not only a video
`and audio encoding method but also a coded data multi(cid:173)
`plexing method, as the MPEG/Systems.
`FIG. 24 shows the structure of a transport packet
`(hereinafter called a TS packet) adopted by the transport
`system which represents one multiplexing method of the
`MPEG/Systems.
`Each TS packet is a fixed length packet of 188 bytes
`including a 4-byte header and a 184-byte payload. The
`4-byte header is constituted by a sync byte, an error flag, a
`unit start flag, a scramble control flag, a priority flag, a set
`of PID data, an adaptation field control flag, and a cyclic
`counter. The contents of PID data identify the payload of the
`TS packet.
`The payload is constituted by an adaptation field
`(hereinafter abbreviated as AF) designated by the adaptation
`field control flag and a payload having video and audio
`encoded data and multiplexing information data. AF is
`constituted by AF length data, a discontinuity flag, a random
`access flag, a priority flag, a PCR flag indicating a presence/
`absence of AF optional data, an OPCR flag, a splice point
`flag, a private data flag, and optional data designated by
`these flags. Of the optional data, PCR data is used for clock
`synchronization between the transmission and reception
`sides, and the reception side generates a reference clock of
`27 MHz by using a phase locked loop.
`FIG. 25 illustrates selection of a particular channel of
`encoded video and audio data from a bit stream (TS packets)
`with a plurality of programs (hereinafter called channels)
`multiplexed. A program association table (hereinafter abbre(cid:173)
`viated as PAT) is always transmitted by a TS packet having
`PID="O". PAT contains multiplexing information of a plu(cid:173)
`rality of channels and supplies information on whether each
`channel program map table (hereinafter abbreviated as
`PMT) is transmitted by a TS packet having what PID value.
`FIG. 25 shows an example of multiplexing of three
`channels. In order to select a j -th channel and obtain its PMT,
`the TS packet with PID=Mj is captured. This PMT has
`information that the video data is transmitted by the TS
`packet with PID=Aj and the audio data is transmitted by the
`TS packet with PID= Vj. Therefore, encoded video data can
`be received by capturing the TS packet with PID= Vj and
`encoded audio data can be received by capturing the TS
`packet with PID=Aj.
`The MPEG/Systems described above, however, define
`only syntax of TS packets and the like and does not
`determine how a reception apparatus is actually configured.
`In addition, it does not provide particular definition of
`multiplexing of value added services such as program
`guidance, which definition is relied upon arbitrary decision
`on the side of broadcasting companies or program vendors.
`In conventional technique disclosed in JP-A-4-229464
`(corresponding to EP-0460751A2), each packet has time
`information which is used on the decode side as a reference 55
`of various timings including a decode timing.
`
`SUMMARY OF THE INVENTION
`
`It is an object of the present invention to provide a 60
`decoder for compressed and multiplexed video and audio
`data, wherein packet landing buffers are allocated in a RAM
`used by a CPU for the system control to thereby reduce the
`number of components and lower the cost of components.
`According to one aspect of the present invention, there is
`provided a decoder for compressed and multiplexed video
`and audio data for receiving a group of packets having a
`
`10
`
`20
`
`30
`
`4
`plurality of multiplexed packet sets each containing video
`data and associated audio data compressed by compression
`codes and packetized and outputting a set of video and audio
`signals, comprising: a video decoder for decoding the corn-
`s pressed video data; an audio decoder for decoding the
`compressed audio data; a first memory for sequentially
`storing the group of packets; a processor for executing a
`process in accordance with a stored program, the process
`including sequentially reading a packet from the first
`memory, filtering packets containing a set of particular video
`and audio data and a control packet containing attribute
`information of the group of packets, and supplying the
`particular video and audio data to the video and audio
`decoders; a second memory having a work area for at least
`the stored program; a third memory for storing the attribute
`15 information contained in the control packet; and an interface
`unit for transferring an external control signal to the
`processor, the external control signal being used for filtering
`a set of video and audio data, wherein the first to third
`memories are provided in the same memory element.
`A storage device of RAM used by a processor (CPU) as
`a main memory stores operating system software and is
`required to have a capacity of several mega bits. It is
`therefore easy to allocate the memory of which capacity is
`required 512 bytes for a packet landing buffer for each data
`25 element in the RAM. Therefore, the number of components
`is not increased.
`It is another object of the present invention to provide a
`receiver for compressed video and audio data capable of
`receiving a bit stream with a plurality of multiplexed
`channels, decoding the video and audio data, and also
`receiving value added service data flexibly.
`According to another aspect of the invention achieving
`the above object, there is provided an apparatus comprising:
`35 data selecting means for selecting an encoded stream of one
`channel (a group of packets), multiplexing information data,
`and additional information data, from a multiplexed stream
`(packets); bus access means for supplying the selected
`encoded stream and the like to a bus; a microprocessor
`40 connected to the bus; a random access memory; a program
`memory; means for processing the additional information
`data; means for decoding the video data; and means for
`decoding the audio data.
`The apparatus may further comprises: means for gener-
`45 ating a clock signal in accordance with clock information in
`the encoded stream; packet sync supply means for supplying
`a packet sync signal of the encoded stream to the micro(cid:173)
`processor; means for adding an error flag indicative of
`failure in correcting transmission errors to the encoded
`50 stream, and data storage media interface means.
`The encoded stream of one channel and other data
`selected by the data selecting means are stored via the bus
`access means and the bus into the random access memory,
`data read from the random access memory is analyzed and
`discriminated, and via the bus the video data is supplied to
`the video data decoding means, the audio data is supplied to
`the audio data decoding means, the additional information
`data or processed additional information data is supplied to
`the additional information data processing means.
`The video data decoding means expands and decodes the
`encoded video data and the audio data decoding means
`expands and decodes the encoded audio data to reproduce
`video and audio signals. The additional information data
`processing means allows, for example, a program guide, to
`65 be displayed on video data in an overlay manner.
`The clock signal generating means supplies a clock signal
`to the video data decoding means, audio data decoding
`
`SONY EX. 1004
`Page 26
`
`

`
`5,898,695
`
`6
`FIG.1 is a block diagram showing the structure of the first
`embodiment of the invention. A tuner 1 selects data of one
`channel transmitted from a communications medium such as
`a CATV and a communications satellite, and supplies the
`5 selected channel data to a demodulator 2. The demodulator
`2 demodulates the channel data which was channel encoded
`by QAM, QPSK, or the like, performs an error correction
`process by using redundancy codes, and supplies the pro(cid:173)
`cessed data to an interface unit 14. The processed data is a
`10 bit stream data of the TS packet format. In response to a
`control signal from a microprocessor 12, the interface unit
`14 transfers all TS packet data to a packet landing buffer 71
`in a main memory RAM 7. The internal structure of the
`packet landing buffer 71 is shown in FIG. 4. The packet
`15 landing buffer 71 is a first-in-first-out (FIFO). Only one
`packet is written in each line in the landing order, and read
`in the same order. Video#1 in FIG. 4 represents a packet
`containing video data of a program #1. The buffer is
`assigned with row addresses which are updated by one each
`20 time a packet is written, and after N packets are received, the
`address returns to 0. The line number N is set so as to satisfy:
`
`1