`
`
`
`11111111111111111111111111111I11111)11111111111111111111111111
`
`(12) United States Patent
`Imai et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 6,507,611 B1
`Jan. 14, 2003
`
`(54) TRANSMITTING APPARATUS AND
`METHOD, RECEIVING APPARATUS AND
`METHOD, AND PROVIDING MEDIUM
`
`(75)
`
`Inventors: Kenichi Imai, Tokyo (JP); Minoru
`Tsuji, Chiba (JP); Takashi Koike,
`Kanagawa (JP)
`
`(73)
`
`Assignee: Sony Corporation, Tokyo (JP)
`
`( * )
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/299,509
`
`(22) Filed:
`
`Apr. 26, 1999
`
`(30)
`
`Foreign Application Priority Data
`
`(JP)
`
`May 8, 1998
`
` 10-125633
` HO4B 1/38; 1104L 5/16
`(51) Int. C1.7
` 375/222; 375/220; 375/219
`(52) U.S. Cl.
` 375/219, 220,
`(58) Field of Search
`375/221, 222, 257; 370/537, 469, 453
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,931,250 A * 6/1990 Greszczuk
`5,687,157 A
`11/1997 Imai et al.
`5,848,384 A
`12/1998 Hollier et al.
`
` 375/222
` 369/124
` 704/831
`
`5,960,035 A * 9/1999 Sridhar et al.
`6,021,158 A * 2/2000 Schurr et al.
`6,236,676 B1 * 5/2001 Shaffer et al.
`
` 375/219
` 375/211
` 375/299
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`
`9-127989
`
`5/1997
`
`
`
` G10L/9/14
`
`* cited by examiner
`
`Primary Examiner—Phuong Phu
`(74) Attorney, Agent, or Firm—Sonnenschein, Nath &
`Rosenthal
`
`(57)
`
`ABSTRACT
`
`The invention intends to decode and reproduce digital audio
`signals in real time. A transmission rate of a transmission
`line is detected, and a selection instructing unit instructs, to
`an encoding selecting circuit, a coding method which can
`provide coded data having a bit rate corresponding to the
`detected transmission rate. In response to the instruction
`from the selection instructing unit, the encoding selecting
`circuit controls a switch so as to select one of a plurality of
`encoders for encoding an audio signal with different coding
`methods. Each frame of the audio signal cut out by a frame
`cutting circuit is supplied to the encoder selected by the
`switch for encoding thereof. Resulting coded data is output-
`ted after an ID representing the instructed coding method
`has been added to the coded data in a header inserting
`circuit.
`
`48 Claims, 17 Drawing Sheets
`
`SELECTION
`INSTRUCTING
`UNIT
`
`ENCODING
`SELECTING
`CIRCUIT
`
`55
`
`56
`
`531
`
`54
`
`51
`
`0
`
`)0- ENCODER
`
`
`
`AUDIO
`DATA
`
`FRAME
`CUTTING
`CIRCUIT
`
`52
`
`41 m
`
`HEADER
`INSERTING
`CIRCUIT
`
`TO MULTIPLIER 42
`
`ENCODER
`
`532
`
`ENCODER
`
`53N
`
`HULU LLC
`Exhibit 1006
`IPR2018-01170
`
`Page 1
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 1 of 17
`
`US 6,507,611 B1
`
`FIG. 1
`
`1
`S
`SERVER 4
`
`3
`S
`
`CLIENT
`TERMINAL
`
`CLIENT
`TERMINAL
`
`Page 2
`
`
`
`lualud *S*11
`
`LI JO Z lootIS
`
`Ill TT9`LOS`9 Sfl
`
`FIG. 2
`
`12
`S
`
`CPU
`Z\
`
`13s
`
`RAM
`
`1 /
`
`17
`
`COMMUNICATING 4
`APPARATUS
`
`>TO NETWORK 2
`
`INPUT
`DEVICE
`S
`14
`
`V
`OUTPUT
`DEVICE
`S
`15
`
`\ 7
`EXTERNAL
`STORAGE
`
`16
`
`11
`S
`
`ROM <
`
`Page 3
`
`
`
`lualud *S*11
`
`LI JO £ lootIS
`
`Ill TT9`LOS`9 Sfl
`
`FIG. 3
`
`22
`S
`
`CPU
`Z\
`
`23s
`
`RAM
`Z\
`
`21
`
`ROM <
`
`INPUT
`DEVICE
`S
`24
`
`OUTPUT
`DEVICE
`S
`25
`
`EXTERNAL
`STORAGE
`S
`26
`
`27
`
`COMMUNICATING <
`APPARATUS
`
`> TO NETWORK 2
`
`Page 4
`
`
`
`lualud *S*11
`
`LI Jo 17 lootIS
`
`Ill TT9`LOS`9 Sfl
`
`FIG. 4
`
`12
`
`42
`S
`
`CODING CIRCUIT
`
`j 411
`
`f41
`
`CODING
`UNIT
`
`CODING
`UNIT
`
`MULTI-
`PLEXER
`
`CODING
`UNIT
`
`31
`S
`
`AUDIO
`SIGNAL
`INPUT
`CIRCUIT
`
`AUDIO
`SIGNALS
`
`33
`S
`TRANSMITTING
`CIRCUIT
`
`TO NETWORK 2
`
`Page 5
`
`
`
`lualud *S*11
`
`LI JO S lootIS
`
`Ill TT9`LOS`9 Sil
`
`FIG. 5
`
`SELECTION
`INSTRUCTING ----55
`UNIT
`
`51
`
`AUDIO
`DATA
`
`),
`
`FRAME
`CUTTING
`CIRCUIT
`
`ENCODING
`SELECTING
`CIRCUIT
`
`0
`
`0
`
`52
`
`h 56
`
`531
`S
` ENCODER H
`
`54
`
`ENCODER H
`'-)
`532
`
`HEADER
`INSERTING
`CIRCUIT
`
`> TO MULTIPLIER 42
`
`ENCODER
`L,
`53N
`
`41m
`
`Page 6
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 6 of 17
`
`US 6,507,611 B1
`
`FIG. 6
`
`IDm
`
`CODED DATA OUTPUTTED
`FROM ENCODER 53m
`
`Page 7
`
`
`
`lualud *S11
`
`IN
`
`LI JO L
`
`Ill TT9`LOS`9 Sil
`
`FIG. 7
`
`62s
`
`DECODING CIRCUIT
`
`71
`5
`
`61
`c
`
`FROM
`NETWORK 2
`
`RECEIVING
`CIRCUIT
`
`SEPARA-
`TOR
`
`63s
`
`AUDIO
`SIGNAL
`OUTPUT
`CIRCUIT
`
`•
`
`AUDIO
`SIGNALS
`
`( 72'
`
`DECODING
`UNIT
`
`HDECODING
`
`UNIT
`
`722
`
`DECODING
`UNIT
`
`72M
`
`Page 8
`
`
`
`lualud *S*11
`
`LI Jo 8 lootIS
`
`Ill TT9`LOS`9 Sfl
`
`FIG. 8
`
`831
`
`0
`
`DECODER
`
`1
`
`CODED
`BIT STREAM
`
`HEADER
`EXTRACTING
`CIRCUIT
`
`S
`81
`
`82
`
`DECODER
`.
`•
`
` 0,-
`.
`832 .
`
`, TO AUDIO SIGNAL
`OUTPUT CIRCUIT 63
`
`0
`
`DECODER
`
`-,
`
`83N'
`
`/ 72m
`
`Page 9
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 9 of 17
`
`US 6,507,611 B1
`
`FIG. 9
`
`AUDIO DATA TRANSMITTING)
`PROCESS BY SERVER
`
`Si
`V
`CONNECTING PROCESS
`
`S2
`DETERMINE ABILITY OF
`CLIENT TERMINAL 3
`
`S3
`SET ENCODING SCHEDULE
`
`( UPDATE OF DECODER
`
`S4
`
`S5
`
`REQUIRED?
`
`Y
`
`S9
`CHANGE
`ENCODING
`SCHEDULE
`
`TRANSMIT UPDATE
`CONFIRMATION MESSAGE
`
`S6
`)
`
`S7
`
`UPDATE APPROVED?
`
`Y
`
`DECODER UPDATING
`PROCESS
`
`S8
`TRANSMIT AUDIO DATA
`
`( END
`
`)
`
`Page 10
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 10 of 17
`
`US 6,507,611 B1
`
`SERVER 1
`
`CLIENT TERMINAL 3
`ACQUIRE
`SYSTEM
`INFORMATION
`
`TRANSMIT
`SYSTEM
`INFORMATION
`
`FIG. 10A
`
`CLIENT PROCESSING
`ABILITY
`
`FIG. 10B
`
`SERVER 1
`TRANSMISSION
`OF "SYSTEM
`INFORMATION
`REQUEST"
`
`CLIENT TERMINAL 3
`
`REQUEST
`SYSTEM
`INFORMATION
`
`TRANSMIT
`SYSTEM
`INFORMATION
`
`ACQUIRE
`SYSTEM
`INFORMATION
`
`0
`CLIENT PROCESSING
`ABILITY
`
`FIG. 10C
`
`SERVER 1
`TRANSMISSION
`OF DUMMY
`DATA
`
`TRANSMIT
`DUMMY DATA
`
`TRANSMIT
`DECODING TIME
`
`0
`CLIENT PROCESSING
`ABILITY
`
`CLIENT TERMINAL 3
`
`DECODING
`
`PROCESS 1
`
`Page 11
`
`
`
`lualud *S*11
`
`P
`,..4k.
`
`N O O W
`
`LT JO II WIN
`
`Ill TT9`LOS`9 Sfl
`
`FIG. 11
`
`i A \ AUDIO DATA OF
`k ^ / SOME PART
`
`to
`
`ti
`
`t2
`
`t3
`I
`
`t4
`I
`
`t5
`
`t6
`
`x
`B )
`
`(
`
`ENCODING
`SCHEDULE
`
`E1
`
`E2
`
`E3
`
`E4
`
`E2
`
`E3
`
`\
`C )
`
`(
`
`ENCODING
`SCHEDULE
`
`E1
`
`Ei E3
`
`E4
`
`E1
`
`E3
`
`to
`
`ti
`
`t2
`
`t3
`
`t4
`
`t5
`
`t6
`—TIME
`
`Page 12
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 12 of 17
`
`US 6,507,611 B1
`
`FIG. 12
`
`eUDIO DATA RECEIVING
`
`PROCESS BY CLIENT TERMINAL)
`
`S11
`REQUEST CONNECTION
`
`S12
`
`TRANSMIT ABILITY
`
`( UPDATE CONFIRMATION
`
`S13
`)
`
`MESSAGE TRANSMITTED?
`Y
`
`S14
`
`NOT
`UPDATE ( DETERMINE APPROVAL /
`DISAPPROVAL OF UPDATE
`UPDATE
`
`S18
`'I
`TRANSMIT
`DISAPPROVAL
`OF UPDATE
`
`S15
`TRANSMIT UPDATE REQUEST
`
`S16
`
`UPDATING PROCESS
`
`><
`
`S17
`RECEIVE AND DECODE
`AUDIO DATA
`
`( END
`
`)
`
`Page 13
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 13 of 17
`
`US 6,507,611 B1
`
`CLIENT
`TERMINAL 3
`
`FIG. 13
`
`REQUEST
`FROM
`CLIENT
`
`REPLY FROM
`
`REQUEST
`FROM
`CLIENT
`
`COMMUNICATION BAND B
`= (n1+n2)/(t2-t1) [bps]
`
`SERVER 1
`
`RECEPTION TIME t1
`
`TRANSMITTED
`BACKET LENGTH n1
`
`RECEPTION TIME t2
`RECEIVED
`BACKET LENGTH n2
`
`Page 14
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 14 of 17
`
`US 6,507,611 B1
`
`FIG. 14
`
`Z
`0
`U)
`Cl)
`M
`CI)
`M w
`< i-
`CC <
`I- CC
`
`.4
`
`T
`T
`TIMING TO REASSESS
`COMMUNICATION STATUS
`
`i
`
`TRANSMISSION
`START TIME OF
`AUDIO SIGNAL
`
`I
`
`TIME
`
`Page 15
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 15 of 17
`
`US 6,507,611 B1
`
`CLIENT
`TERMINAL 3
`
`FIG. 15
`
`REPEATED
`S TIMES
`
`SERVER 1
`r
`
`\._
`
`REARRANGE
`ENCODING
`SCHEDULE
`
`DATA
`
`DATA
`
`ACK
`
`•
`
`DATA
`
`ACK
`
`DATA
`
`ACK
`
`Page 16
`
`
`
`lualud *S*11
`
`LT JO 91 lamIS
`
`Ill TT9`LOS`9 Sfl
`
`FIG. 16
`
`SELECTION INSTRUCTING
`UNIT
`
`55
`
`ENCODING SELECTING 1- 56
`CIRCUIT
`
`92-
`
`READING UNIT
`
`911
`
`912
`
`STORAGE
`
`•
`
`0
`
`STORAGE
`A
`
`531
`
`32
`
`91N
`L?
`STORAGE
`
`•
`
`53N
`
`ENCODER
`
`ENCODER
`
`ENCODER
`
`54
`
`HEADER
`INSERTING
`CIRCUIT
`
`TO MULTIPLEXER 42
`
`51
`C)
`FRAME
`CUTTING
`CIRCUIT
`
`I
`
`AUDIO
`DATA
`
`41m
`
`Page 17
`
`
`
`U.S. Patent
`
`Jan. 14, 2003
`
`Sheet 17 of 17
`
`US 6,507,611 B1
`
`AUDIO
`DATA
`
`CODED ,
`DATA
`'
`
`
`
`FIG. 17
`
`E1
`
`E2
`
`EN
`
`1
`
`----)-TIME
`
`Page 18
`
`
`
`US 6,507,611 B1
`
`1
`TRANSMITTING APPARATUS AND
`METHOD, RECEIVING APPARATUS AND
`METHOD, AND PROVIDING MEDIUM
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The present invention relates to a transmitting apparatus
`and method, a receiving apparatus and method, and a
`providing medium. More particularly, the present invention
`relates to a transmitting apparatus and method, a receiving
`apparatus and method, and a providing medium which are
`suitably used, for example, in the case of encoding and
`transmitting time-series digital signals such as audio signals,
`and receiving and decoding the digital signals to reproduce
`them in real time on the receiving side.
`2. Description of the Related Art
`For example, when digital signals such as digital audio
`signals are transmitted from the transmitting side to the
`receiving side via a network, e.g., Internet, it is customary
`that the digital signals are transmitted after being com-
`pressed and encoded into coded data on the transmitting
`side, and the coded data is decoded on the receiving side,
`because the data rate of the digital signals is higher than the
`transmission band (i.e., the transmission rate) of the net-
`work. Such transmission and reception of digital signals is
`performed on the premise that the receiving side includes a
`decoder corresponding to an encoder provided on the trans-
`mitting side.
`Coded data resulted from encoding made by an encoder
`on the transmitting side is transmitted to the receiving side
`via a network as mentioned above, but the transmission rate
`of the coded data in the network generally varies due to the
`amount of traffic, etc. Apart from the case where the coded
`data is all downloaded and then decoded on the receiving
`side, therefore, it may occur in the case of decoding the
`coded data while receiving the same that the transmission of
`the coded data becomes too late to decode and reproduce the
`audio signals in real time if the transmission rate of the
`network lowers below the data rate of the coded data.
`Also, although a system of Internet or any other network
`is designed so as to avoid the transmission of the coded data
`from becoming too late as far as possible, it is nevertheless
`sometimes difficult to decode and reproduce audio signals in
`real time due to, e.g., the ability of hardware and the
`decoding method of a decoder on the receiving side.
`More specifically, for example, when the coding method
`of an encoder on the transmitting side is complex, the
`decoding method of a decoder for decoding the coded data,
`which has been encoded by the encoder, is also usually
`complex. In such a case, if hardware on the receiving side
`has not a processing ability capable of coping with the
`complex decoding method, decoding of audio signals would
`be too late for the start of reproduction thereof. Further, the
`processing time required for the decoder to decode the coded
`data is greatly affected by the processing ability of hardware
`on the receiving side. Accordingly, if the processing ability
`of hardware on the receiving side is lower than expected by
`the transmitting side, the time taken for decoding the coded
`data would be longer than expected by the transmitting side,
`thus resulting in a difficulty in decoding and reproducing the
`audio signals in real time.
`
`SUMMARY OF THE INVENTION
`The present invention has been made in view of the state
`of art set forth above, and intends to decode and reproduce
`digital signals in real time.
`
`5
`
`2
`According to a first aspect of the present invention, in a
`transmitting apparatus for outputting coded data resulted
`from encoding a time-series digital signal, the transmitting
`apparatus comprises a plurality of coding means for encod-
`ing the digital signal with a plurality of coding methods and
`outputting the coded data, instructing means for instructing,
`from among the plurality of coding methods, one for encod-
`ing a part or the whole of the digital signal, selecting means
`for selecting the coded data obtained with the coding method
`10 instructed from the instructing means, adding means for
`adding, to the coded data selected by the selecting means,
`coding method information indicating the coding method
`used for obtaining the selected coded data, and outputting
`means for outputting the selected coded data added with the
`15 coding method information.
`According to a second aspect of the present invention, in
`a transmitting method for use in a transmitting apparatus for
`outputting coded data resulted from encoding a time-series
`digital signal, the transmitting apparatus comprises a plu-
`20 rality of coding means for encoding the digital signal with a
`plurality of coding methods and outputting the coded data,
`and the transmitting method comprises an instructing step of
`instructing, from among the plurality of coding methods,
`one for encoding a part or the whole of the digital signal, a
`25 selecting step of selecting the coded data obtained with the
`coding method instructed in the instructing step, an adding
`step of adding, to the coded data selected in the selecting
`step, coding method information indicating the coding
`method used for obtaining the selected coded data, and an
`30 outputting step of outputting the selected coded data added
`with the coding method information.
`According to a third aspect of the present invention, in a
`providing medium for providing a computer program for
`rendering a computer to execute processing to output coded
`35 data resulted from encoding a time-series digital signal, the
`computer comprises a plurality of coding means for encod-
`ing the digital signal with a plurality of coding methods and
`outputting the coded data, and the computer program
`includes an instructing step of instructing, from among the
`40 plurality of coding methods, one for encoding a part or the
`whole of the digital signal, a selecting step of selecting the
`coded data obtained with the coding method instructed in the
`instructing step, an adding step of adding, to the coded data
`selected in the selecting step, coding method information
`45 indicating the coding method used for obtaining the selected
`coded data, and an outputting step of outputting the selected
`coded data added with the coding method information.
`According to a fourth aspect of the present invention, in
`a receiving apparatus for receiving and processing coded
`so data resulted from encoding a part or the whole of a
`time-series digital signal with one of a plurality of coding
`methods, the receiving apparatus comprises extracting
`means for extracting coding method information added to
`the coded data and indicating the coding method used for
`55 obtaining the coded data, recognizing means for recognizing
`a decoding method for decoding the coded data based on the
`coding method information, and decoding means for decod-
`ing the coded data with the decoding method recognized by
`the recognizing means.
`60 According to a fifth aspect of the present invention, in a
`receiving method for receiving and processing coded data
`resulted from encoding a part or the whole of a time-series
`digital signal with one of a plurality of coding methods, the
`receiving method comprises an extracting step of extracting
`65 coding method information added to the coded data and
`indicating the coding method used for obtaining the coded
`data, a recognizing step of recognizing a decoding method
`
`Page 19
`
`
`
`US 6,507,611 B1
`
`4
`corresponding to the transmission rate of the transmission
`line, and a transmitting step of transmitting the coded data
`obtained with the coding method selected in the selecting
`step.
`According to an eleventh aspect of the present invention,
`in a receiving apparatus for receiving coded data resulted
`from encoding a digital signal via a predetermined trans-
`mission line, the receiving apparatus comprises receiving
`means for receiving the coded data encoded by a coding
`method capable of providing data having a bit rate corre-
`sponding to a transmission rate of the transmission line, and
`decoding means for decoding the coded data.
`According to a twelfth aspect of the present invention, in
`a receiving method for receiving coded data resulted from
`encoding a digital signal via a predetermined transmission
`line, the receiving method comprises a receiving step of
`receiving the coded data encoded by a coding method
`capable of providing data having a bit rate corresponding to
`a transmission rate of the transmission line, and a decoding
`step of decoding the coded data.
`According to a thirteenth aspect of the present invention,
`in a providing medium for providing a computer program
`for rendering a computer to execute processing to receive
`coded data resulted from encoding a digital signal via a
`predetermined transmission line, the computer program
`includes a receiving step of receiving the coded data
`encoded by a coding method capable of providing data
`having a bit rate corresponding to a transmission rate of the
`transmission line, and a decoding step of decoding the coded
`data.
`
`5
`
`10
`
`15
`
`30
`
`3
`for decoding the coded data based on the coding method
`information, and a decoding step for decoding the coded
`data with the decoding method recognized in the recogniz-
`ing step.
`According to a sixth aspect of the present invention, in a
`providing medium for providing a computer program for
`rendering a computer to process coded data resulted from
`encoding a part or the whole of a time-series digital signal
`with one of a plurality of coding methods, the computer
`program includes an extracting step of extracting coding
`method information added to the coded data and indicating
`the coding method used for obtaining the coded data, a
`recognizing step of recognizing a decoding method for
`decoding the coded data based on the coding method
`information, and a decoding step for decoding the coded
`data with the decoding method recognized in the recogniz-
`ing step.
`According to a seventh aspect of the present invention, in
`a providing medium for providing coded data resulted from
`encoding a time-series digital signal, the providing medium 20
`provides coded data and coding method information that are
`obtained by instructing, from among a plurality of coding
`methods, one for encoding a part or the whole of the digital
`signal, selecting the coded data resulted from encoding the
`digital signal with the instructed coding method, and adding, 25
`to the selected coded data, coding method information
`indicating the coding method used for obtaining the selected
`coded data.
`According to an eighth aspect of the present invention, in
`a transmitting apparatus for transmitting coded data resulted
`from encoding a digital signal via a predetermined trans-
`mission line, the transmitting apparatus comprises a plural-
`ity of coding means for encoding the digital signal with a
`plurality of coding methods and outputting the coded data,
`detecting means for detecting a transmission rate of the
`transmission line, selecting means for selecting one of the
`plurality of coding methods which is able to provide coded
`data having a bit rate corresponding to the transmission rate
`of the transmission line, and transmitting means for trans-
`mitting the coded data obtained with the coding method
`selected by selecting means.
`According to a ninth aspect of the present invention, in a
`transmitting method for use in a transmitting apparatus for
`transmitting coded data resulted from encoding a digital
`signal via a predetermined transmission line, the transmit-
`ting apparatus comprises a plurality of coding means for
`encoding the digital signal with a plurality of coding meth-
`ods and outputting the coded data, and the transmitting
`method comprises a detecting step of detecting a transmis-
`sion rate of the transmission line, a selecting step of select-
`ing one of the plurality of coding methods which is able to
`provide coded data having a bit rate corresponding to the
`transmission rate of the transmission line, and a transmitting
`step of transmitting the coded data obtained with the coding
`method selected in the selecting step.
`According to a tenth aspect of the present invention, in a
`providing medium for providing a computer program for
`rendering a computer to execute processing to transmit
`coded data resulted from encoding a digital signal via a
`predetermined transmission line, the computer comprises a
`plurality of coding means for encoding the digital signal
`with a plurality of coding methods and outputting the coded
`data, and the computer program includes a detecting step of
`detecting a transmission rate of the transmission line, a
`selecting step of selecting one of the plurality of coding
`methods which is able to provide coded data having a bit rate
`
`40
`
`45
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a diagram showing an exemplified construction
`of one embodiment of a transmission system to which the
`35 present invention is applied.
`FIG. 2 is a block diagram showing an exemplified con-
`struction of hardware of a server 1 in FIG. 1.
`FIG. 3 is a block diagram showing an exemplified con-
`struction of hardware of a client terminal 3 in FIG. 1.
`FIG. 4 is a block diagram showing an exemplified func-
`tional construction of the server 1 in FIG. 2.
`FIG. 5 is a block diagram showing an exemplified con-
`struction of a coding unit 41„, in FIG. 4.
`FIG. 6 is a diagram showing a format of data outputted
`from a header inserting circuit 54 in FIG. 5.
`FIG. 7 is a block diagram showing an exemplified func-
`tional construction of the client terminal 3 in FIG. 3.
`FIG. 8 is a block diagram showing an exemplified con-
`50 struction of a decoding unit 72„, in FIG. 7.
`FIG. 9 is a flowchart for explaining a transmitting process
`executed by the server 1.
`FIGS. 10A to 10C are diagrams each for explaining a
`process executed in step S2 of FIG. 9.
`FIGS. 11A to 11C are charts for explaining change of an
`encoding schedule.
`FIG. 12 is a flowchart for explaining a receiving process
`executed by the client terminal 3.
`FIG. 13 is a diagram for explaining a manner of deter-
`mining the transmission rate of a network 2.
`FIG. 14 is a chart for explaining the timing to change the
`encoding schedule.
`FIG. 15 is a chart for explaining the timing to change the
`65 encoding schedule.
`FIG. 16 is a block diagram showing another exemplified
`construction of the coding unit 41„, in FIG. 4.
`
`55
`
`60
`
`Page 20
`
`
`
`US 6,507,611 B1
`
`5
`FIG. 17 is a chart for explaining a method of transmitting
`coded data to the client terminal.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`Embodiments of the present invention will be described
`below.
`FIG. 1 shows an exemplified construction of one embodi-
`ment of a transmission system (the term "system" means a
`plurality of devices assembled together in a logical corre-
`lation regardless of whether the devices having their own
`constructions are all positioned in the same housing or not)
`to which the present invention is applied.
`In the transmission system, when a request for time-series
`digital signals, e.g., digital audio signals, is issued from a
`client terminal 3 to a server 1 via a network 2 such as
`Internet, ISDN (Integrated Service Digital Network) or
`PSTN (Public Switched Telephone Network), the server 1
`encodes the requested audio signals with a predetermined
`coding method, and resulting coded data is transmitted to the
`client terminal 3 via the network 2. After receiving the coded
`data from the server 1, the client terminal 3 decodes the
`coded data and reproduces the original audio signals in real
`time, for example, (so-called streaming reproduction).
`FIG. 2 shows an exemplified construction of hardware of
`the server 1 in FIG. 1.
`A ROM (Read Only Memory) 11 stores, for example, an
`IPL (Initial Program Loading) program and so on. A CPU
`(Central Processing Unit) 12 executes an OS (Operating
`System), which is stored (recorded) in an external storage
`16, in accordance with the IPL program stored in the ROM
`11, and also executes various application programs, which
`are stored in the external storage 16, under control of the OS,
`thereby carrying out a coding process of audio signals, a
`transmitting process of coded data resulted from the coding
`process to the client terminal 3, etc. A RAM (Random
`Access Memory) 13 stores programs, data and so on which
`are necessary for operation of the CPU 12. An input device
`14 is constructed of, e.g., a keyboard, a mouse, a microphone
`or an external interface, and is operated when necessary data
`or command is inputted. The input device 14 is also con-
`structed to function as an interface for accepting input of
`digital audio signals externally applied to the client terminal
`3. An output device 15 is constructed of, e.g., a display, a
`speaker or a printer, and displays or outputs necessary
`information. The external storage 16 comprises a hard disk,
`for example, and stores the OS and the application programs
`mentioned above. Program modules provided to the client
`terminal 3 and serving as decoders, described above, are also
`stored in the external storage 16. In addition, the external
`storage 16 stores other data including data necessary for
`operation of the CPU 22, etc. A communicating apparatus 17
`carries out control required for communication via the
`network 2.
`FIG. 3 shows an exemplified construction of hardware of
`the client terminal 3 in FIG. 1.
`The client terminal 3 is made up of illustrated components
`21-27, i.e., a ROM 21 to a communicating apparatus 27, and
`has a construction basically similarly to that of the server 1
`made up of the ROM 11 to the communicating apparatus 17.
`Unlike the server 1, an external storage 26 stores, as
`application programs, a program for decoding the coded
`data transmitted from the server 1, programs for executing
`later-described processing, and so on. A CPU 22 executes
`those application programs to, for example, perform a
`decoding process of the coded data.
`
`5
`
`20
`
`6
`FIG. 4 shows an exemplified functional construction of
`the server 1 in FIG. 2. The illustrated construction is
`basically realized upon the CPU 12 executing the applica-
`tion programs stored in the external storage 16.
`Audio signals to be provided to the client terminal 3 are
`supplied to an audio signal input circuit 31. In the audio
`signal input circuit 31, analog audio signals are A/D-
`converted into digital audio signals. Then, the audio signal
`input circuit 31 separates the digital audio signals for each
`10 of channels, for example, and supplies them to a coding
`circuit 32.
`More specifically, audio signals of a piece of music, for
`example, are separated for each of respective parts of guitar,
`piano, drum, song (words), etc. that constitute the piece of
`is music together, and are inputted to the audio signal input
`circuit 31 in a synchronous relation. The audio signal input
`circuit 31 outputs the audio signals of individual parts
`respectively as the audio signals of individual channels.
`Note that audio signals of two or more parts may be
`inputted to the audio signal input circuit 31 in a mixed
`condition. In this case, the audio signal input circuit 31
`outputs the audio signals in the mixed condition.
`Also, audio signals inputted to the audio signal input
`25 circuit 31 may be in the form of, e.g., MIDI (Musical
`Instrument Digital Interface) data (i.e., data for controlling
`a MIDI sound source) rather than the actual audio signals.
`The coding circuit 32 encodes the audio signals from the
`audio signal input circuit 31 at high efficiency.
`30 More specifically, in the embodiment shown in FIG. 4, the
`coding circuit 32 comprises a number M of coding units 411
`to 41N, and a multiplexer 42. The coding unit 41,„, (m=1,
`2, . . . , M) is supplied with the audio signal of each
`corresponding channel from the audio signal input circuit
`35 31. The coding unit 41,„, encodes the audio signal from the
`audio signal input circuit 31, and supplies resulting coded
`data to the multiplexer 42. The multiplexer 42 multiplexes a
`number M of coded data outputted from the coding units 411
`to 41M respectively into one line, and supplies them as
`40 multiplexed data to a transmitting circuit 33.
`The transmitting circuit 33 converts the multiplexed data
`into a format corresponding to the communication protocol
`adapted for transmitting it via the network 2, and transmits
`resulting data to the client terminal 3 via the network 2.
`FIG. 5 shows an exemplified construction of the coding
`unit 41,„, in FIG. 4.
`A frame cutting circuit 51 cuts the audio signal (audio
`data) from the audio signal input circuit 31 in units of frame
`so having a predetermined length (e.g., a length suitable for
`coding made by encoders 531 to 53,, or a length suitable for
`packet (network packet) transmission via the network 2),
`and then supplies resulting frames to a switch 52. The switch
`52 selects one of the encoders 531 to 53, under control of
`55 an encoding selecting circuit 56. Accordingly, each frame
`outputted from the frame cutting circuit 51 is supplied to one
`of the encoders 531 to 53N through the switch 52.
`The encoders 531 to 53N (N is two or more) are con-
`structed to encode the audio signal with different coding
`60 methods from each other (for example, linear PCM (Pulse
`Code Modulation), ADPCM (Adaptive Differential PCM),
`layers 1, 2, 3 of MPEG (Moving Picture Experts Group),
`ATRAC (Adaptive Transform Acoustic Coding), ATRAC 2,
`and LIVXC (Harmonic Vector Excitation Coding)). Stated
`65 otherwise, in the embodiment, the encoders 531 to 53, are
`prepared by using encoders which perform encoding of the
`audio signal with various coding methods, including a
`
`4 5
`
`Page 21
`
`
`
`US 6,507,611 B1
`
`7
`method which provides a relatively large (high) bit rate of
`the resulting coded data, but can reproduce an audio signal
`with relatively good reproducibility, a method which can
`provides a relatively small (low) bit rate of the resulting
`coded data, but reproduces an audio signal with relatively
`poor reproducibility, a method which requires a larger
`amount of computation for decoding (such a method usually
`also requires a larger amount of computation for coding), a
`method which requires a not so large amount of computation
`for decoding, and a method particularly suitable for coding
`a voice (human voice).
`One example of the coding method, which provides a
`relatively many bit rate of the resulting coded data, is the
`linear PCM (the coded data resulted with this coding method
`is the same as that obtained by outputting a digital audio
`signal after A/D conversion as it is). Example of the coding
`method, which provides a relatively less bit rate of the
`resulting coded data (i.e., which provides a high compres-
`sion rate), are MPEG layer 3 and ATRAC 2. One example
`of the coding method, which requires a not so large amount
`of computation for decoding, is ATRAC. Further, examples
`of the coding method suitable for a voice are HVXC and a
`method utilizing a linear estimation factor. Incidentally,
`HVXC is one of the methods previously proposed by the
`assignee of this application, and is disclosed in detail in, e.g.,
`Japanese Unexamined Patent Publication No. 9-127989
`(corresponding to U.S. Pat. No. 5,848,387).
`It is here assumed that even in the case of the encoders all
`employing, e.g., ATRAC 2 only, if bit rates are different from
`each other, this means the use of "different coding methods".
`In one example, therefore, all the encoders 531 to 53N
`perform encoding with ATRAC 2, while data rates of coded
`data outputted from the encoders are 64 Kbps, 32 Kbps, 24
`Kbps, . . . , respectively.
`A selection instructing unit 55 decides appropriate one
`from a plurality of coding methods corresponding to the
`encoders 531 to 53,, as described later, and then instructs the
`encoding selecting circuit 56 to select the decided coding
`method. In response to the instruction, the encoding select-
`ing circuit 56 controls the switch 52 so as to select the
`encoder which performs encoding with the instructed coding
`method. Accordingly, the frame outputted from the frame
`cutting circuit 51 is supplied to selected one of the encoders
`531 to 53N (referred to as the selected encoder hereinafter)
`through the switch 52.
`In the selected one of the encoders 531 to 53N, the frame
`supplied thereto is encoded with the predetermined coding
`method (referred to as the selected coding method herein-
`after because it is executed in the selected encoder). Coded
`data resulted from encoding made in the selected encoder is
`supplied to the header inserting circuit 54 where an ID
`(Identification, i.e., coding method information) represent-
`ing the selected coding method is added to the coded data.
`Thus, in the illustrated embodiment, the audio signal is
`encoded wit