United States Patent [19J
`Fujisaki et al.
`
`I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US005899577 A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,899,577
`May 4, 1999
`
`[54] TRANSMISSION DEVICE AND
`TRANSMISSION METHOD
`
`5,583,652 12/1996 Ware ......................................... 386/75
`5,647,047
`7/1997 Nagasawa ................................. 386/52
`
`[75]
`
`Inventors: Naoki Fujisaki, Chiba; Nobuyuki
`Takasu, Tokyo, both of Japan
`
`[73] Assignee: Sony Corporation, Tokyo, Japan
`
`[21] Appl. No.: 08/629,276
`
`[22] Filed:
`
`Apr. 8, 1996
`
`[30]
`
`Foreign Application Priority Data
`
`Apr. 9, 1995
`
`[JP]
`
`Japan .................................... 7-109096
`
`Int. Cl.6
`..................................................... H04N 5/783
`[51]
`[52] U.S. Cl. ................................................. 386/68; 386/75
`[58] Field of Search .................................. 386/68, 80, 81,
`386/75, 52, 96, 105, 106, 111, 112, 125,
`126, 104, 46, 124; H04N 5/783
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`FOREIGN PATENT DOCUMENTS
`
`0 605 115
`
`7/1994 European Pat. Off ..
`
`Primary Examiner-Huy T Nguyen
`Attorney, Agent, or Firm-Frommer Lawrence & Haug,
`LLP.; William S. Frommer
`
`[57]
`
`ABSTRACT
`
`In a transmission device, an image signal and an audio signal
`are played back at a speed n times normal speed from a
`recording medium and the played back image signal is
`encoded. The played back audio signal is sampled using a
`sampling clock signal of a sampling frequency which is
`proportional to the speed n times normal speed. When n>l,
`1/n items of sampling data of the sampling data generated by
`sampling is encoded. When n<l, all of the sampling data
`generated by sampling is encoded. The encoded image
`signal and encoded audio signal are then transmitted to a
`terminal device.
`
`4,736,260
`
`4/1988 Wilkinson ................................. 386/96
`
`12 Claims, 6 Drawing Sheets
`
`4
`
`r -
`
`-
`
`-
`
`I
`
`.J.~_ -
`-
`13
`
`-
`
`-
`
`-
`
`- -I
`I 54
`\
`
`:
`
`8
`7
`6
`'
`;
`----~~---------------t----1
`'
`'
`I
`!-----~----~
`,'
`I
`1
`I
`I
`I
`~----l_ ____ l
`I
`I
`I
`
`I
`
`I
`
`:
`
`I
`
`I
`
`:
`
`terminal
`
`terminal
`
`terminal
`
`9A1
`
`9A2
`
`9A3
`
`terminal L
`'----~f'.-9AN
`
`~--
`2
`52A
`r-.L---_-_-_-',-i...-,..:-: _\_j_____L-r'--'-"=--1 56 :
`VTR
`
`I
`: 01
`5
`I
`____________ J
`
`51A,51B
`
`I
`I
`
`1
`
`:
`
`mediaunit
`
`I
`
`1
`
`I
`I
`I
`I
`l __________ _J
`
`I
`
`12A2
`
`12A3
`
`1
`
`12AM 1
`1
`I
`I
`1 __________ 1
`
`:
`1
`I
`:
`
`I
`: 02
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`1 ~ 1
`1 11A7:
`
`~---------------------------~
`
`IPR2016-01710
`UNIFIED EX1024
`
`

`
`.....::.
`.....::.
`Ul
`....
`\C
`\C
`00
`....
`Ul
`
`O'I
`
`'"""' 0 .....,
`~ .....
`'Jl =-~
`
`\C
`\C
`'"""'
`\C
`~,J;;..
`'-<
`~
`~
`
`~ = ......
`~ ......
`~
`•
`\JJ.
`d •
`
`j terminal ~9AN
`
`3
`
`termina~ gA
`
`terminal ~ 9A2
`
`_I
`I
`I
`I
`I
`I
`I
`
`02
`'
`I
`I
`I ,_
`
`)
`I
`
`I
`
`I
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`'~ I
`
`---
`1
`I
`2AM I
`:
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`-I
`
`-
`
`decoder
`
`-
`
`-
`
`I
`I
`I
`11A7 I
`:
`:
`
`-_J
`I
`I
`I
`
`I
`I
`I
`
`11A2
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`-
`
`-
`
`---
`
`--
`
`-
`
`-
`
`-
`
`1 "-9A1
`
`terminal
`
`t I
`
`.-
`
`I
`
`I
`I
`,--'
`
`52
`
`j 1
`
`I
`
`I
`
`I I -'---..J--
`
`1
`
`I
`
`I
`
`I
`
`I
`
`8
`
`I
`I
`I
`I
`I
`I
`r----L----1 I
`I
`I
`I
`I
`
`I
`12A1 I
`
`---
`
`I
`
`)
`
`7
`
`--
`
`--
`
`-
`
`I _____
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`L_ -
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`I
`I
`I
`I _____ L_____
`1
`r~:----~~---------------~----
`
`mArli::i
`
`10
`
`I
`I
`I
`I
`I
`
`I
`
`I
`I
`I
`
`6
`
`FIG. 1
`
`01
`
`\
`54
`
`56:
`
`I
`r
`I
`I
`I
`I
`1
`I
`I
`I
`I
`I
`
`controller
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`--'.-_ -
`
`-
`
`13
`
`I
`
`4
`
`/
`
`I
`.
`153
`I
`:
`I
`I
`I
`I
`r -
`
`-
`
`'\
`
`\
`
`.....;
`-1
`
`52A
`
`1
`
`51A,51B
`
`J
`I
`
`-
`
`-
`
`---
`5--'
`
`I
`I
`I
`
`encoder
`
`5281
`
`__ I
`I
`
`I
`I
`
`VTR
`
`1'
`
`--J -
`
`-
`
`,_
`
`3
`
`~ ~ -
`2
`
`-
`
`I,
`
`

`
`.....::.
`.....::.
`Ul
`....
`\C
`\C
`00
`....
`Ul
`
`O'I
`0 .....,
`N
`~ .....
`'Jl =(cid:173)~
`
`\C
`\C
`'"""'
`\C
`~,J;;..
`'-<
`~
`~
`
`~ = ......
`~ ......
`~
`•
`\JJ.
`d •
`
`t\22
`
`~
`
`5
`
`0 ....
`iD x
`;:::;: -a·
`c
`3
`
`---------------------------------------
`~a
`
`~o
`
`~:>
`
`processor
`
`coding
`
`1
`
`t
`
`converter
`
`input
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`I--
`
`23
`
`I 010
`
`I
`
`synchronization v 30
`
`controller
`
`image encoder
`
`~
`20
`
`FIG. 2
`
`S1A~
`
`I
`I
`1
`1
`1
`I
`I
`I
`I
`I
`1
`
`~
`0
`~
`0
`O
`'<
`Ill
`CD
`a.
`
`I
`0151
`I
`
`~-----------------------~
`I
`I
`24.J 1
`
`1
`
`generator
`detector &
`coding unit S10:
`I
`I
`I
`
`013
`
`I
`I
`1
`
`0141
`,
`
`-
`
`-
`
`1
`
`converter
`
`speed
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`012
`
`-
`
`-
`
`-
`
`/~---------------------------~~-------§~~-~!~J
`
`29
`T
`
`SS~
`
`S3~
`
`I
`
`interiace
`
`21
`
`~~~''--~
`524
`
`I
`
`r1 S23
`: S21,
`I
`011 I
`I
`
`520
`
`111
`!
`
`

`
`.....::.
`.....::.
`Ul
`....
`\C
`\C
`00
`....
`Ul
`
`O'I
`0 .....,
`~
`
`~ .....
`'Jl =(cid:173)~
`
`\C
`\C
`'"""'
`\C
`~,J;;..
`'-<
`~
`~
`
`~ = ......
`~ ......
`~
`•
`\JJ.
`d •
`
`I I
`
`I
`
`:
`
`1
`
`CL~SlB
`
`------)) (timeforoneframe) ))
`
`coding unit time
`
`))
`
`))
`~
`
`))
`
`I n/2 I
`
`'
`I
`
`I
`
`1
`
`FIG. 3K
`FIG. 3J
`FIG. 31
`FIG. 3H
`FIG. 3G
`FIG. 3F
`FIG. 3E
`FIG. 30
`FIG. 3C
`FIG. 38
`FIG. 3A
`
`1H
`i
`P=J
`I 1 I 2131 •I s I s I )) n-2ln-1I n I
`~00)) BB ))Do~p
`! 0 0 0 )) B G §1 )):J D D ~ D
`:
`llDl· .. ·11
`~ 012
`j 1 i 2131 •Is Is I )) n-2ln-11nin+tln+2I ),< [!11 I 1,,_,l2ni
`hmuum )) nJUU1fl ))
`:
`))
`))
`:
`((
`((
`! 1 I 2 I 3 I ))
`I )) I
`fl__n__rr_ )) LIUL ))
`
`llTil····ll i
`:
`I s T n
`i
`
`,-CL2
`
`012
`
`I
`
`:
`
`I
`
`I
`
`-
`
`EE)) I I I I l20
`
`))
`
`

`
`.....::.
`.....::.
`Ul
`....
`\C
`\C
`00
`....
`Ul
`
`O'I
`0 .....,
`.i;;..
`~ .....
`'Jl =-~
`
`\C
`\C
`'"""'
`\C
`~.i;;..
`'-<
`~
`~
`
`~ = ......
`~ ......
`~
`•
`\JJ.
`d •
`
`22
`
`01
`
`5
`
`29
`
`____ I S10
`
`I
`
`28
`
`I - I
`
`I
`I
`I
`I
`I
`I
`
`I
`
`-..
`0
`CD x
`;:;·
`c
`3
`
`-..
`CD
`Q..
`0 ;::. -..
`ct> ro-
`a.
`
`C")
`'<
`
`S10
`
`!
`
`I coding unit I
`
`generator
`detector &
`
`I 26
`
`41
`
`I
`
`I
`
`I S20
`
`I
`
`23
`
`I
`
`interface
`
`~S21, S23 27r
`.____
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`--1--
`
`-
`
`-
`
`L _
`
`S24
`
`converter
`
`processor
`adaptive ~ speed
`
`.
`
`I I c~~e~~er H input sampling coding
`
`015
`
`S1B :---------D1_1 _____________ o12 _________ o14 __ _
`
`010
`
`image encoder
`
`S1 A ~1
`
`20
`
`30
`
`synchronization
`
`controller
`
`FIG. 4
`
`21-I:
`
`I
`I
`I
`I
`I
`I
`I
`
`

`
`U.S. Patent
`
`May 4, 1999
`
`Sheet 5 of 6
`
`5,899,577
`
`m
`,....
`en
`
`Q)
`
`....... _
`~~ ·- ..._
`
`:!::::: Q.l
`Cc
`::i 0
`O> ..._ co
`:o~ 8E
`
`c
`C\J
`.......
`I c:
`N
`
`C\J + r::::
`.......
`+ r::::
`c
`
`T""
`
`I c
`C\J
`I c
`
`<D
`
`LO
`
`C\J
`
`Lt)
`•
`
`m
`Cl -LL
`
`0
`Lt)
`•
`
`Cl -LL
`
`Lt)
`•
`
`c
`Cl -LL
`
`w
`Lt)
`•
`Cl -LL
`
`• -LL
`
`

`
`.....::.
`.....::.
`Ul
`....
`\C
`\C
`00
`....
`Ul
`
`O'I
`0 .....,
`O'I
`~ .....
`'Jl =-~
`
`\C
`\C
`'"""'
`\C
`~,J;;..
`'-<
`~
`~
`
`~ = ......
`~ ......
`~
`•
`\JJ.
`d •
`
`coding processor
`
`I
`
`I
`
`sampling buffer
`.._________ selection input
`previous frame
`
`-52
`
`032
`
`I
`
`input sampling
`
`processor
`selection
`adaptive
`
`sampling buffer
`double speed input
`
`012
`
`033
`
`53
`
`51
`
`I 031
`
`030
`
`)
`50
`
`011
`
`FIG. 6
`
`

`
`5,899,577
`
`1
`TRANSMISSION DEVICE AND
`TRANSMISSION METHOD
`
`BACKGROUND OF THE INVENTION
`
`2
`transmitter for transmitting signals encoded at the first and
`second encoders to a terminal device.
`Further, in order to achieve the object of the present
`invention, a transmission device of the present invention for
`5 encoding and transmitting an image signal and an audio
`signal comprises a playback device for playing back the
`image signal and the audio signal from a recording medium
`at a speed n (n<l) times normal speed, a first encoder for
`encoding the image signal played back by the playback
`10 device a sampler for sampling the audio signal played back
`by the playback device using a sampling clock of a sampling
`frequency proportional to the speed n times the normal
`speed, a second encoder for encoding sampling data gener(cid:173)
`ated as a result of sampling at the sampler and a transmitter
`15 for transmitting signals encoded at the first and second
`encoders to a terminal device.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram showing an embodiment of an
`audio visual server system to which the present invention is
`applied;
`FIG. 2 is a block diagram showing an example of an
`encoder of the audio visual server system shown in FIG. 1;
`FIG. 3A to FIG. 3K are timing charts illustrating the
`operation of the encoder shown in FIG. 2;
`FIG. 4 is a block diagram showing another embodiment
`of an encoder of the audio visual server system shown in
`FIG. 1;
`FIG. SA to FIG. SE are timing charts illustrating the
`operation of the encoder shown in FIG. 4; and
`FIG. 6 is a block diagram showing an example of an
`adaptive coding processor for the encoder shown in FIG. 4.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`30
`
`The present invention relates to a transmission device and
`transmission method particularly suited for application in,
`for example, an audio visual server system.
`Conventionally, audio signal coding devices where base(cid:173)
`band audio signals comprising inputted analog waveforms
`are sampled at a number of defined sampling frequencies,
`code processed and then outputted have been widely used.
`On the other hand, there are currently various types of
`coding devices for coding image signals, but recently, image
`coding devices represented by using the MPEG method have
`come to be in general use.
`One of the characteristics of MPEG-defined image coding
`is that variable-speed playback of the image at the time of
`decoding is possible.
`However, with usual MPEG method encoding devices, 20
`the image signal and accompanying audio signal are inde(cid:173)
`pendently encoded and the encoding of the image signal
`takes into consideration the aforementioned variable-speed
`play-back. However, the encoding of the audio signal does
`not take into consideration this variable-speed playback and 25
`the played-back audio signal is therefore silenced at the time
`of variable-speed playing-back of the image signal.
`Generally, encoding taking this kind of variable-speed play(cid:173)
`back into consideration has been difficult with the existing
`MPEG method audio encoding devices.
`Therefore, with MPEG method encoding devices, with
`regards to the image signal, if the audio signal that accom(cid:173)
`panies this image signal can also be encoded so as to be
`played-back at a variable speed in synchronism with the
`variable-speed playback of the image signal, then the afore- 35
`mentioned kind of silencing of the audio signal can be
`avoided even during a specific playing-back and the audio
`signal can be perceived as being information that is in
`synchronism with the image signal.
`Further, the number of decoding devices produced is
`definitely greater than that of encoding devices and decoding
`devices have become particularly superior ones both in
`economical efficiency and maintainability. Therefore, for
`example, in a system where a plurality of decoding devices
`are connected to a single encoding device, it is much
`disadvantageous for the decoding device to have a special
`function such as the variable-speed playback of an audio
`signal in simplifying and downsizing the structure of the
`decoding device.
`It is therefore an object of the present invention to provide
`a transmission device and transmission method which are
`flexible and expandable.
`
`40
`
`The following is a description with reference to the
`drawings of one embodiment of the present invention.
`An embodiment of an audio visual (hereinafter referred to
`as "AV") server system to which the present invention is
`applied, will be described with reference to FIG. 1. An
`image signal SlAand an audio signal SlB outputted from an
`information-providing source 3, which comprises a video
`45 tape recorder (hereinafter referred to as "VTR") 2 etc.
`capable of recording and playing back operation, are pro(cid:173)
`vided to an encoder S of an encoder part 4.
`The encoder S encodes the image signal SlA and the
`50 audio signal SlB using a format that complies with the
`MPEG2 standard, forms encoded data Dl by multiplexing,
`and sends this out to the server 6.
`The server 6 comprises a recording/playback device 7 and
`a decoder part 8. The encoded data Dl is temporarily
`55 recorded at a prescribed recording medium by the recording/
`playback device 7 as well as being played back in response
`to a request signal S2 from the user. Then, the played-back
`encoded data Dl is, after having been decoded by the
`decoder 8, sent out to the one of terminals 9A1 to 11AN_7
`60 and 9AN comprising, for example, television receivers, from
`which there has been a request.
`The recording/playback device 7 comprises a media con(cid:173)
`troller 10 and a plurality of media units 11A1 to 11A7 .
`Encoded data Dl inputted at the media controller 10 is
`recorded on a hard disc in any one of the media units 11A1
`to 11A6 and 11A7 . The media units 11A1 to 11A6 and 11A7
`are driven in response to request signals S2 from each of the
`
`SUMMARY OF THE INVENTION
`In order to achieve the object of the present invention, a
`transmission device of the present invention for encoding
`and transmitting an image signal and an audio signal com(cid:173)
`prises a playback device for playing back the image signal
`and the audio signal from a recording medium at a speed n
`(n>l) times normal speed, a first encoder for encoding the
`image signal played back by the playback device, a sampler
`for sampling the audio signal played back by the playback
`device using a sampling clock of a sampling frequency
`proportional to the speed n times the normal speed, a second 65
`encoder for encoding 1/n items of sampling data of sampling
`data generated as a result of sampling at the sampler and a
`
`

`
`5,899,577
`
`3
`terminals 9A1 to 9AN_ 1 and 9AN for playing back the
`requested encoded data Dl recorded on the hard disc.
`A plurality of hard disc transmission devices are built into
`the media units 11A1 to 11A7 . A plurality of items of
`encoded data Dl can therefore be played back in parallel 5
`even when there are a large number of requests from a large
`number of users (terminal 9A1 to 9AN) at the same time
`because the recording/playback device 7 has a plurality of
`media units 11A1 to 11A7 with a plurality of hard disc
`transmission devices being built therein.
`The decoder part 8 is constructed from decoders 12A1 to
`2AM, the number of which corresponds to the number of
`terminals 9A1 to 9AN and the number of items of encoded
`data Dl capable of being played back at the same time by the
`recording/playback device 7. The encoded data Dl played 15
`back by the recording/playback device 7 is decoded and
`decoded data D2 is sent out to the terminals of 9A1 to 9AN_ 1
`and 9AN for which there has been a request.
`At the AV server system 1, the operation of outputting the
`image audio data SlA and SlB by the information-providing 20
`source 3, the encoding operation of the encoder S and the
`operation of recording and playing back the encoded data
`Dl by the media units 11A1 to 11A7 under the control of the
`media controller 10 are controlled by control signals S2A,
`S3 and S4 outputted from the controller 13 of the encoder 4. 25
`Further, the controller 13 monitors the operating conditions
`of the information-providing source 3 and the encoder S on
`the basis of a status signals S2B and SS and receives data
`instructing the operating conditions of the media controller
`10 and data instructing the playback speeds requested at 30
`each terminal via a control signal S6.
`The VTR 2 within the information-providing source 3
`plays back the image and audio signals at the specified speed
`based on data provided from the controller 13 instructing the
`playback speed and provides the played back image and
`audio signals to the encoder S.
`Here, in the case of the AV system 1, as shown in FIG. 2,
`the encoder S is constructed from an image encoder 20 for
`encoding an image signal SlA provided from an 40
`information-providing source 3, an audio encoder 21 for
`encoding an audio signal SlB and an interface 29.
`The image encoder 20 encodes the provided image signal
`SlA using a format conforming to the MPEG2 standard as
`described above and encoded data obtained in this way is 45
`outputted to a multiplexor 22 as image coded data DlO.
`On the other hand, at the audio encoder 21, an analog
`audio signal SlB provided from the information-providing
`source 3 is inputted to an input converter 23 and undergoes
`analog-to-digital conversion at this input converter 23. Digi- 50
`tal audio data Dll obtained in this way is successively
`provided to a coding processor 2S of a framing signal
`forming block 24.
`The coding processor 2S encodes the digital audio data
`Dll provided in real-time sequentially in prescribed coding
`units (corresponding to digital audio data Dll for single
`frame of the image signal SlA) in conformity with the
`MPEG2 standard. Coded audio data D12 obtained in this
`way has header data D13 attached at the leading position of
`each coded unit and is sequentially sent to the speed con(cid:173)
`verter 26. At this time, the coding processor 2S sends each
`item of header data D13 attached at the leading position of
`each coding unit sequentially to a coding unit detector and
`generator 27.
`The speed converter 26 sequentially stores the provided
`coded audio data D12 and sequentially reads them out at a
`predetermined rate, i.e. at a rate equal to the rate during
`
`4
`normal speed playback of the audio data. The sequentially
`obtained sequential coded audio data D14 is provided to the
`coding unit detector and generator 27 and a delay controller
`28.
`The coding unit detector and generator 27 sequentially
`detects the positions of each of sequential coded audio data
`D14 at the head of each coding unit from the sequential
`coded audio data D14 sequentially outputted from the
`sequential speed converter 26 on the basis of the provided
`10 header data D13. A framing signal SlO indicating the
`position of this sequential coded audio data D14 is then sent
`to the delay controller 28.
`The delay controller 28, under the control of a synchro-
`nization controller 30, adjusts the phases of the sequential
`coded audio data D14 and the framing signal SlO to be equal
`to that of the image coded data DlO by providing a pre(cid:173)
`scribed amount of delay corresponding to the relative time
`difference between corresponding items of the image coded
`data DlO. The phase-adjusted sequential coded audio data
`D14 and framing signal SlO are then sent to the multiplexor
`22 as audio coded data DlS and the framing signal SlO. The
`synchronization controller 30 obtains, from the image
`encoder 20, data indicating the timing of outputting the
`image coded data at the head of a unit frame after being
`coded. The amount of the delay at the delay controller 28 is
`then controlled on the basis of the data display this timing.
`The multiplexor 22 thus sequentially multiplexes one
`code unit of audio coded data DlS and one frame of image
`coded data DlO while sequentially identifying the audio
`coded data DlS at the head of each coding unit on the basis
`of the framing signal SlO. In this way, the multiplexor 22 is
`operated to form the aforementioned encoded data Dl which
`is thereafter sent to a media controller 10 (FIG. 1) of the
`recording/playback device 7 (FIG. 1).
`In the case of this embodiment, as is clear from FIG. 2, an
`interface 29 is provided at the encoder S.
`The interface 29 sends control signals S20, S21, S22 and
`S24 to the input converter 23, the coding processor 2S, the
`delay controller 28, and the image encoder 20, respectively,
`on the basis of various parameters in the control signal S3
`provided from the controller 13 (FIG. 1). In this way, the
`operations of the input converter 23, the coding processor
`2S, the delay controller 28 and the image encoder 20 are
`controlled.
`The interface 29 is supplied from the coding processor 2S
`with a status output which is further provided to the con(cid:173)
`troller 13 (FIG. 1) as a status signal SS. In this way, the
`controller 13 (FIG. 1) monitors the encoder Son the basis of
`this status signal SS.
`The controller 13 provides data indicating playback with
`a speed n times normal speed to the VTR 2 and the interface
`29 of the encoder S on the basis of data indicating the
`playback speeds requested from each of the terminals. The
`interface 29 then provides control data for the image encoder
`55 20, input converter 23 and coding processor 2S in such a
`manner that code processing is carried out in response to n
`times normal speed.
`In the case of this encoder S, the input converter 23 and
`the coding processor 2S are capable of coding the audio
`60 signal SlB provided at a variable speed (n times speed).
`Namely, during the first coding mode corresponding to
`high-speed playback (n>l), the input converter 23 samples
`a provided audio signal SlB as shown in FIG. 3A on the
`basis of a second clock CL2 (FIG. 3E) of a frequency of n
`65 times that of the usual (n=l) first clock CLl (FIG. 3B).
`In this way, the input converter 23 obtains a number of
`items of audio data Dll (FIG. 3F) that is n times the usual
`
`35
`
`

`
`5,899,577
`
`5
`number of digital audio data Dll (FIG. 3C) and that is
`sequentially provided to the coding processor 2S. In
`addition, FIG. 3E to FIG. 3H show the case where n=2.
`The coding processor 2S codes only the 1/n items of
`digital audio data Dll (FIG. 31) of the provided digital audio 5
`data Dll that are pre-defined in every code unit. In this way,
`the coding processor 2S can obtain coded audio data D12
`(FIG. 3D) compressed along the time axis by 1/n when
`compared with sampling using the usual clock CLl, as is
`clear from FIG. 3A to FIG. 31. In FIG. 31, there is shown a 10
`case where only the n items of digital audio data Dll at the
`first half of the digital audio data Dll is coded when n=2.
`On the other hand, during the second coding mode
`corresponding to low speed playback (O<n<l), the input
`converter 23 samples the provided audio signal SlB on the 15
`basis of a third clock of a frequency which is n times
`(O<n<l) the frequency of the first clock CLl. In this way, the
`input converter 23 obtains items of the digital audio data
`Dll 1/n times less than the usual and sequentially provides
`them to the coding processor 2S.
`In this case, the coding processor 2S adds dummy data to
`the items of digital audio data Dll by an amount corre(cid:173)
`sponding to the number of insufficient items of data before
`encoding each item of digital audio data Dll and dummy
`data. In this way, the coding processor 2S is able to obtain
`coded audio data D12 (FIG. 3G) extended n times along the
`time axis when compared with sampling using the usual
`clock CLl.
`With the above structure, at the encoder S, the provided
`audio signal SlB is sampled at the input converter 23 during
`the first coding mode corresponding to high-speed playback
`(n>l) at a first sampling frequency that is n times (n>l) the
`usual frequency. In this way, 1/n (n>l) items of digital audio
`data Dll are coded at the coding processor 2S from the 35
`obtained digital audio data Dll.
`On the other hand, during the second coding mode
`corresponding to low-speed playback (O<n<l), the provided
`audio signal SlB is sampled at the input converter 23 at a
`second sampling frequency that is n times (O<n<l) the usual
`frequency. This is then coded at the coding processor 2S
`after dummy data has been added to the provided digital
`audio data Dll by an amount corresponding to the number
`of insufficient items of data.
`In this case, audio coded data DlS formed with the first
`and second coding modes can be decoded in synchronism
`with the decoding of the image coded data DlO. The
`silencing of sounds during the specific playing-back can
`therefore be avoided and the sound can be perceived as
`information that is synchronized with the image.
`According to the above structure, during the first coding
`mode corresponding to high-speed playback (n>l), the pro(cid:173)
`vided audio signal SlB is sampled at the input converter 23
`at a first sampling frequency that is n (n>l) times the
`frequency of the usual frequency. In this way, 1/n (n>l) 55
`items of digital audio data Dll are coded at the coding
`processor 2S. Meanwhile, during the second coding mode
`corresponding to low-speed playback (O<n<l), the provided
`audio signal SlB is sampled at the input converter 23 at a
`second sampling frequency that is n times (O<n<l) the usual 60
`frequency and is then decoded at the coding processor 2S
`after dummy data has been added to the obtained digital
`audio data Dll by an amount corresponding to the number
`of insufficient items of data. As a result, the audio coded data
`DlS can be decoded in synchronism with the decoding of the 65
`image coded data DlO in a manner that meets variable-speed
`playback of the image code using the MPEG method. The
`
`6
`audio signal can therefore also be played back at a variable
`speed in synchronism with the variable speed playback of
`the image signal. Thus, an adaptive and expandable encoder
`S can be achieved.
`It is therefore not necessary for the decoders 12A1 to
`12AM to have any special functions because the coding for
`the specific playback is carried out on the side of the encoder
`S. Specific playback and equivalent variable speed playback
`is therefore possible with any decoding device.
`Further, there are also advantages with regards to various
`aspects such as expandability, economy, space factor and
`reliability when compared with the case for achieving the
`same functions with a conventional coding device.
`A further embodiment of the encoder S for the AV server
`system shown in FIG. 1 will now be described with refer(cid:173)
`ence to FIG. 4. Portions corresponding to those of FIG. 2
`will be given the same reference numerals and their descrip(cid:173)
`tion will be omitted.
`At the encoder S shown in FIG. 4, during the first coding
`mode corresponding to the high-speed playback (n>l), the
`provided audio signal SlB of FIG. SA is sampled at the input
`converter 23 at a clock signal CLl having a frequency n
`times (n>l) the usual one as shown in FIG. SB and the
`25 digital audio data Dll of sample number n as shown in FIG.
`SC are obtained which are provided to the input sampling
`adaptive coding processor 41.
`The adaptive coding processor 41 has the structure as
`shown in FIG. 6. The digital audio data Dll provided from
`30 the input converter 23 is divided into a front-halfn-items and
`a back half n-items via a double speed input sampling buffer
`SO and then inputted to an input sampling adaptive selection
`processor Sl as two systems of first and second digital audio
`data D30 and D31 (FIG. SD).
`The input sampling adaptive selection processor Sl
`selects, on the basis of the first or second digital audio data
`D32 which is for the precedently selected previous frame
`and is supplied at this time from a previous frame selection
`input sampling buffer S2, one of either first or second
`40 sampling data of the present frame D30 or D31 which has a
`stronger correlation to the digital audio data D32. The
`selected first or second sampling data D30 or D31 is then
`sent to the previous frame selection input sampling buffer S2
`and a coding processor S3 as adaptive digital audio data
`45 D33.
`The coding processor S3 forms the coded audio data D12
`as shown in FIG. SE by coding the provided adaptive digital
`audio data D33 and sends the coded audio data D12 to the
`sequential speed converter 26 (FIG. 2).
`In this way, the input sampling adaptive coding processor
`41 selects and encodes the appropriate one of the front half
`or the back half of the n items of the digital audio data Dll
`provided with the number of n (n>l) times the usual.
`According to the above structure, by dividing sampling
`data 512 outputted from the input converter 23 into a
`front-half n-items and a back-half n-items and selectively
`coding the one thereof having higher correlation with the
`previous frame, in the same way as for the first embodiment,
`the audio coded data DlS can be decoded in synchronism
`with the image coded data DlO in such a manner as to meet
`the variable speed displaying of the image code using the
`MPEG method, and thus an adaptive and extendible encoder
`S can be achieved.
`In the aforementioned embodiment and further
`embodiment, the case was described where the encoder S
`according to the present invention is applied to the AV
`
`20
`
`50
`
`

`
`5,899,577
`
`7
`system 1 shown in FIG. 1. However, the present invention
`is by no means limited thereto and may be well applied to
`coding devices that can be used in various other technologi(cid:173)
`cal fields of digital data communications.
`In the aforementioned embodiment and further 5
`embodiment, the number of items of digital audio data Dll
`corresponding to one frame of an image are taken as a
`coding unit at the encoder 5. However, the present invention
`is by no means limited thereto, and other coding units are
`also possible.
`Moreover, in the aforementioned further embodiment, the
`case is described where the coded audio data D12 outputted
`from the input converter 23 is divided into a front-half
`n-items and a back half n-items. However, the present
`invention is by no means limited thereto and the digital 15
`audio data Dll outputted from the input converter 23 is not
`just limited to be divided into a front half and a back half but
`may be divided into n series, with each series being provided
`to the input sampling adaptive selection processor 51, pro(cid:173)
`vided that the data is made to be compatible with high-speed 20
`playback of n-times.
`Moreover, in the aforementioned embodiment, the case is
`described where just the front n-items of the digital audio
`data Dll are coded for the case for n=2, as shown in FIG.
`31. However, the present invention is by no means limited 25
`thereto, and it is also possible to code just n items of digital
`audio data Dll every other one as shown in, for example,
`FIG. 3H and FIG. 3I, or just code the back half n-items of
`digital audio data Dll as shown in FIG. 3K.
`As described above, during a first coding mode corre-
`sponding to high-speed playback, the audio signal is
`sampled at a second sampling frequency which is higher
`than a standard first sampling frequency and the obtained
`sampling data is selectively coded in real-time with a 35
`number of items corresponding to the ratio of the first
`sampling frequency to the second sampling frequency every
`prescribed coding unit, and, during a second coding mode
`corresponding to the low-speed playback, the audio data is
`sampled at a third sampling frequency lower than the first 40
`sampling frequency and, after dummy data has been added
`every coding unit to the obtained sampling data by an
`amount corresponding to the ratio of the first sampling
`frequency to the third sampling frequency, the sampling data
`and dummy data are coded in real-time. The audio signal can 45
`therefore be played-back at a variable speed in synchronism
`with the variable-speed playing-back of the image signal and
`a flexible and expandable coding device and coding method
`can be provided. Further, by using this coding device, a
`transmission device can be obtained which is compatible 50
`with variable-speed playing back of the image signal.
`What is claimed is:
`1. A transmission device for encoding and transmitting an
`image signal and an audio signal comprising:
`playback means for playing back said image signal and
`said audio signal from a recording medium at a speed
`n (n>l) times normal speed;
`first encoding means for encoding said image signal
`played back by said playback means, said first encod(cid:173)
`ing means producing an encoded image signal which is 60
`delineated in frames;
`sampling means for sampling said audio signal played
`back by said playback means using a sampling clock of
`a sampling frequency propotional to said speed n times
`said normal speed, whereby a total number of audio 65
`samples are generated by said sampling means for each
`frame of said encoded image signal;
`
`8
`second encoding means for encoding, for each frame of
`said encoded image signal, 1/n of said total number of
`audio samples to produce an encoded audio signal; and
`transmission means for transmitting the encoded image
`signal and the encoded audio signal at