United States Patent c191
`Stautner et al.
`
`[54) AUDIO COMPRESSION SYSTEM
`EMPLOYING MULTI-RATE SIGNAL
`ANALYSIS
`
`[75]
`
`Inventors: John P. Staatner, Wellesley Hills;
`William R. Morrell, Somerville;
`Srirani. Jayasfmha, Boston, all of
`Mass.
`
`[73] Assignee: Aware, Inc., Cambridge, M ass.
`
`[21) Appl. No.: 948,147
`
`[22) F iled:
`
`Sep. 21, 1992
`
`Int. Q.6 ................................................ GlOL 7/04
`[51)
`[52] U.S. CL ................................................... 395/ 2.14
`[58] Field of Search ............................... : .... 381/29-35;
`395/2.1, 2.38, 2.14, 2.1 5, 2.39; 364/724.01,
`724.1
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`3,947,827 3/1976 Dautremont, Jr. et al . ....... 395/2.15
`4,216,354 8/1980 Esteban et al. .................... 395/2.14
`4,622,680 11/1986 Zinser ................................ 395/ 2.14
`4,799,179 1/1989 Masson et al ..............•..•.. 364/724.1
`4,815,023 3/1989 Arbeiter ......................... 364/724.01
`4,896,356 1/1990 Millar .................................... 381/29
`4,896,362 1/1990 Veldhuis et al. .....•............. 395/2.38
`·4,949,383 8/1990 Koh et al ........................... 395/2.14
`4,972,484 11/1990 Theile et al ........................ 395/2.36
`
`I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US005408580A
`5,408,580
`[11] Patent Number:
`[45] Date of Patent: Apr. 18, 1995
`
`FORE IGN PATENT DOCUMENTS
`0400222 5/1990 European Pat. Off . ....... H04B 1/66
`
`OTHER PUBLICATIONS
`Vaidyarathar, "Quadrature Mirror Filter Banks, M(cid:173)
`Band Extensions and Perfect-Reconstruction Tech·
`niques," IEEE ASSP Magazine, Jul. 1987, pp. ~20.
`Primary Examiner-David D . Knepper
`Attorney. Agent. or Firm-McCubbrey, Bartels & Ward
`[57]
`ABSTRACT
`The compression system utilizes sub-band analysis fil(cid:173)
`ters whose bandwidths are chosen to approximate the
`critical bands of the human auditory system while
`avoiding the aliasing problems encountered in QMF
`filter banks designed to provide similar band splitting.
`One embodiment of the invention may be implemented
`on a digital computer. The computational requirements
`of the synthesis filters may be varied in response to the
`available computational resources of the computer,
`thereby allowing a single compressed audio signal to be
`played back in real time on a variety of platforms by
`trading off audio quality against available computa(cid:173)
`tional resources. Similar trade offs can be made in com·
`pressing an audio signal, thereby allowing a platform
`having limited computational capacity to compress a
`signal in real time.
`
`5 Claims, 11 Drawing Sheets
`
`400
`
`/
`
`J04
`
`I
`
`QUANTIZER
`
`0
`T
`ER
`COD
`,__
`
`406
`\
`
`PYSCC>-ACOUST!C ANAL Y2ER
`
`'
`
`I
`I
`
`~ ... ~
`C? 408
`
`408
`
`1 408
`
`;
`
`I
`
`'
`'
`
`'
`
`AUD
`10 :
`IN
`
`I
`:
`I
`
`-+
`: -r
`-r.
`.
`=:
`'
`; { .
`' ' ;
`~---'
`SUB-BAND
`ANALYSIS
`FILTER.BANK
`
`.
`
`:
`
`402
`
`I
`I
`I
`I
`I
`
`!
`i
`I
`
`FIGURE6
`
`IPR2016-01710
`UNIFIED EX1027
`
`

`
`00 =
`.a;:. = QC
`
`(.II
`....
`
`....
`(.II
`
`....
`e. ....
`....
`(!) ....
`00 =(cid:173)
`
`(1>
`
`U1
`\0
`~
`~
`
`~ ....
`
`"'C a. a
`
`•
`00
`~ •
`
`{PRIOR ART)
`
`FIGURE 1
`
`1.§.
`
`ANALYZER,_-~
`ACOUSTIC
`PSYCHO(cid:173)
`
`COMPRESSED
`
`SIGNAL
`
`QUANTIZER
`
`14
`
`,/
`
`10
`
`ANALYSIS
`SUB-BAND
`
`BANK
`FILTER
`
`12
`
`19
`
`lw I
`
`11
`
`SIGNAL
`INPUT
`
`

`
`HIGHEST
`
`•
`00
`L: •
`
`FIGURE2
`
`18-22
`BANDS
`
`31
`
`BANK
`FILTER
`
`SIGNAL
`INPUT
`
`'------,-'
`
`14-17
`BANDS
`
`BANDS
`
`9-13
`
`'----v----' '---,.----'
`
`BANDS
`
`1-8
`
`BANK
`FILTER
`
`BANK
`FILTER
`
`BANK
`FILTER
`
`\..
`
`34
`
`33
`
`32
`
`LOWEST
`
`~
`30
`
`

`
`U.S. Patent
`
`Apr. 18, 1995
`
`Sheet 3of11
`
`5,408,580
`
`BAND
`
`r -N :r:
`.:t:. ->-(.) z
`
`w
`:::::>
`0 w
`a:
`LL
`
`21
`
`20
`
`19
`
`18
`
`17
`
`16
`
`12
`11
`10
`9
`8
`
`1
`
`TIME(ms)
`FIGURE 3
`
`.....
`
`

`
`U.S. Patent
`
`Apr. 18, 1995
`
`Sheet 4 of 11
`
`5,408,580
`
`301
`
`_,..31 3
`i.-M .... .__ ____ __,
`
`FIGURE 4
`
`375
`
`376
`
`AUDIO
`SIGNAL - -.i
`
`377
`/
`LOW-FREQ.
`BANDPASS
`FILTER
`
`FILTERED
`AUDIO
`SIGNAL
`
`· MODULATION
`SIGNAL
`
`FIGURE S(a)
`
`350 \
`
`M
`AUDIO
`SAMPLES
`
`W-SAMPLE SHIFT REGISTER
`
`320
`
`325
`controller
`
`330
`
`clock
`
`GENERATE
`2M POL YPHASE
`COMPONENTS
`
`MATRIX MUL T.
`TO GENERATE
`M SUB-BAND
`COMPONENTS
`
`322
`
`324
`
`Si
`
`FIGURE S(b)
`
`

`
`U.S. Patent
`
`Apr. 18, 1995
`
`Sheet 5 of 11
`
`5,408,580
`
`400
`,/
`
`~04
`
`0
`T
`QUANTIZER COD
`ER
`
`-
`
`406
`\
`' I
`,,
`I
`
`\
`408
`
`Il l 111
`
`11II1111
`I
`' I
`
`408
`
`l PYSCO-ACOUSTIC ANALYZER
`... I
`I 11 II II
`\ l
`I
`\
`408
`
`r- - - ---,
`i
`I
`i
`'
`I
`l
`
`I
`
`-
`
`I
`I
`I
`I
`I
`I
`DIO I
`I
`I
`I
`I
`I
`I
`!
`I
`I
`I l
`!
`l
`
`AU
`IN
`
`I
`l
`
`I
`I
`
`l
`
`r
`
`i
`
`' l
`!
`i
`l
`i
`i
`!
`!
`
`!
`
`!
`
`t
`
`I
`
`r - l
`i
`I
`i
`
`'--
`-, ~
`{
`!
`I
`'- i
`-
`!
`
`L - !
`
`i
`I
`'
`
`-
`
`1 - -
`
`1 - -
`
`'--
`
`FIGURE 6
`
`SUB-BAND
`ANALYSIS
`FILTER BANK
`
`402
`
`

`
`430
`
`429
`
`DECOMPRESSED
`
`1----_. SIGNAL
`
`426
`
`.. /
`
`..
`
`'fJ1 •
`~ •
`
`FIGURE7
`
`0
`@
`::::>
`<("""'"
`z .....
`j::: .q-1
`ffi N
`
`Cl
`(_)'111\t w
`o..-
`~ w ONI
`
`\
`
`410
`
`COMPRESSED
`
`SIGNAL
`
`

`
`UI oc =
`...
`~ oc
`...
`UI
`
`00. •
`•
`c::
`
`FIGURE8
`
`xJ
`
`VALUES
`SIGNAL
`DOMAIN
`
`TIME
`
`CIRCUIT 1---.....
`SUMMING
`
`518
`
`WEIGHTED ARRA y
`r:-----1---~ 516
`
`I
`h~
`
`ARRAY
`
`r-_....J_ ___ ,, 514
`
`r---'---~...-513
`
`GENERATOR
`
`ARRAY
`
`2W SHIFf REGISTER
`
`MULTIPLIER
`
`512
`
`MATRIX
`
`500
`
`510
`
`

`
`r 00
`
`....
`....
`s,
`
`L: • rJ'J.
`
`•
`
`FIGURE9
`
`CLOCK
`
`CONTROLLER
`
`612
`
`606
`
`CONVERTERS
`
`DIA
`
`BUFFER
`_ _____, OUTPUT
`
`SYNTHESIZER
`
`612
`
`610
`
`608
`
`DE-QUANTIZER
`
`-+I DECODE~ I
`
`i
`
`604
`
`602
`
`~
`
`600
`
`

`
`705 708
`
`/
`
`704
`
`DECODERl'
`
`702
`
`c:: • 00
`
`•
`
`RlGHT
`•••
`DE-QUANTIZER
`
`~
`
`0 ..... ...
`
`\C
`.....
`00 =-
`
`(I)
`(I)
`
`FIGURE 10
`
`716
`
`717
`
`CONVERTER
`
`CONVERTER
`
`DIA
`
`DIA
`
`714
`
`710
`
`BUFFER
`OUTPUT
`
`715
`
`712
`
`BUFFER
`OUTPUT
`
`709
`
`713
`
`CONTROLLER
`
`SYNTHESIZER
`
`SYNTHESIZER
`
`LEFT
`
`706
`
`CLOCK
`
`720
`
`~'4
`
`700
`
`

`
`816
`
`817
`
`CONVERTER
`
`CONVERTER
`
`DIA
`
`DIA
`
`FIGURE 11
`
`814
`
`810
`
`815
`
`...,.,..1 BUFFER1----i BUFFER
`OUTPUT
`
`OUTPUT
`
`.__ ____
`
`807
`
`806
`
`812
`
`SWITCH
`
`----.i SYNTHESIZER.--
`
`CONTROLLER
`
`813
`
`~.-----~-r-'
`CLOCK 1-----• DE-QUANTIZER·,
`,---__ __._ __ ---. /
`
`820
`
`~ • 00
`
`•
`
`802
`
`~
`
`800
`
`805
`
`804
`
`ADDER
`SWITCW
`
`DECODER
`
`

`
`FIGURE 12
`
`,.-----862
`
`1 CLOCK r
`
`CONTROLLER
`
`~ ('"-'----1--_L__
`
`COM MUNI CATION
`
`LINK
`
`BUFFER r--_.
`
`860
`
`CODER
`
`QUANTIZER
`
`AND
`
`ANALYSIS
`SUB-BAND
`
`BANK
`FILTER
`
`858
`
`856
`
`BUFFER
`
`854
`
`~
`
`850
`
`

`
`1
`
`5,408,580
`
`AUDIO COMPRESSION SYSTEM EMPLOYING
`MULTI-RATE SIGNAL ANALYSIS
`
`FIELD OF THE INVENTION
`The present invention relates to audio compression
`and decompression systems.
`
`BACKGROUND OF THE INVENTION
`While digital audio recordings provide many advan- 10
`tages over analog systems, the data storage require(cid:173)
`ments for high-fidelity recordings are substantial. A
`high fidelity recording typically requires more than one
`million bits per second of playback time. The total stor(cid:173)
`age needed for even a short recording is too high for 15
`many computer applications. In addition, the digital bit
`rates inherent in non-compressed high fidelity audio
`recordings makes the transmission of such audio tracks
`over limited bandwidth transmission systems difficult.
`Hence, systems for compressing audio sound tracks to 20
`reduce the storage and bandwidth requirements are in
`great demand.
`One class of prior an audio compression systems di(cid:173)
`vide the sound track into a series of segments. Over the
`time interval represented by each segment, the sound 25
`track is analyzed to determine the signal components in
`each of a plurality of frequency bands. The measured
`components are then replaced by approximations re(cid:173)
`quiring fewer bits to represent, but which preserve
`features of the sound track that are important to a 30
`human listener. At the receiver, an approximation to the
`original sound track is generated by reversing the analy-
`sis process with the approximations in place of the origi(cid:173)
`nal signal components.
`The analysis and synthesis operations are normally 35
`carried out with the aid of perfect, or near perfect,
`reconstruction filter banks. The systems in question
`include an analysis filter bank which generates a set of
`decimated subband outputs from a segment of the sound
`track. Each decimated subband output represents the 40
`signal in a predetermined frequency range. The inverse
`operation is carried out by a synthesis filter bank which
`accepts a set of decimated subband outputs and gener(cid:173)
`ates therefrom a segment of audio sound track. In prac(cid:173)
`tice, the synthesis and analysis filter banks are imple- 45
`mented on digital computers which may be general
`purpose computers or special computers designed to
`more efficiently carry out the operations. If the analysis
`and synthesis operations are carried out with sufficient
`precision, the segment of audio sound track generated 50
`by the synthesis filter bank will match the original seg(cid:173)
`ment of audio sound track that was inputted to the
`analysis filter bank. The differences between the recon(cid:173)
`structed audio sound track and the original sound track
`can be made arbitrarily small. In this case, the specific 55
`filter bank characteristics such as the length of the seg(cid:173)
`ment analyzed, the number of filters in the filter bank,
`and the location and shape of filter response characteris(cid:173)
`tics would be of little interest, since any set of filter
`banks satisfying the perfect, or near-perfect, reconstruc- 60
`tion condition would exactly regenerate the audio seg(cid:173)
`ment.
`Unfortunately, the replacement of the frequency
`components generated by the analysis filter bank with a
`quantized approximation thereto results in artifacts that 65
`do depend on the detail characteristics of the filter
`banks. There is no single segment length for which the
`artifacts in the reconstructed audio track can be mini-
`
`2
`mized. Hence, the length of the segments analyzed in
`prior art systems is chosen to be a compromise. When
`the frequency components are replaced by approxima(cid:173)
`tions, an error is introduced in each component. An
`5 error in a given frequency component produces an
`acoustical effect which is equivalent to the introduction
`of a noise signal with frequency characteristics that
`depend on filter characteristics of the corresponding
`filter in the filter bank. The noise signal will be present
`over the entire segment of the reconstructed sound
`track. Hence, the length of the segments is reflected in
`the types of artifacts introduced by the approximations.
`If the segment is short, the artifacts are less noticeable.
`Hence, short segments are preferred. However, if the
`segment is too short, there is insufficient spectral resolu(cid:173)
`tion to acquire information needed to properly deter-
`mine the minimum number of bits needed to represent
`each frequency component. On the other hand, if the
`segment is too long, temporal resolution of the human
`auditory system will detect artifacts.
`Prior art systems also utilize filter banks in which the
`frequency bands are uniform in size. Systems with a few
`(16-32) sub-bands in a 0-22 kHz frequency range are
`generally called "subband coders" while those with a
`large number of sub-bands (~64) are called "transform
`coders". It is known from psychophysical studies of the
`human auditory system that there are critical band(cid:173)
`widths which vary with frequency. The information in
`a critical band may be approximated by a component
`representing the time averaged signal amplitude in the
`critical band.
`In addition, the ear's sensitivity to a noise source in
`the presence of a localized frequency component such
`as a sine tone depends on the relative levels of the sig(cid:173)
`nals and on the relation of the noise spectral compo(cid:173)
`nents to the tone. The errors introduced by approximat(cid:173)
`ing the frequency components may be viewed as
`"noise". The noise becomes significantly less audible if
`its spectral energy is within one critical bandwidth of
`the tone. Hence, it is advantageous to use frequency
`decompositions which approximate the critical band
`structure of the auditory system.
`Systems which utilize uniform frequency bands are
`poorly suited for systems designed to take advantage of
`this type of approximation. In principle, each audio
`segment can be analyzed to generate a large number of
`uniform frequency bands, and then, several bands at the
`higher frequencies could be merged to provide a de(cid:173)
`composition into critical bands. This approach imposes
`the same temporal constraints on all frequency bands.
`That is, the time window over which the low frequency
`data is generated for each band is the same as the time
`window over which each high-frequency band is gener(cid:173)
`ated. To provide accuracy in the low frequency ranges,
`the time window must be very long. This leads to tem-
`poral artifacts that become audible at higher frequen(cid:173)
`cies. Hence, systems in which the audio segment is
`decomposed into uniform sub-bands with adequate low(cid:173)
`frequency resolution cannot take full advantage of the
`critical band properties of the auditory system.
`Prior art systems that recognize this limitation have
`attempted to solve the problem by utilizing analysis and
`synthesis filter banks based on QMF filter banks that
`analyze a segment of an audio sound track to generate
`frequency components in two frequency bands. To
`obtain a decomposition of the segment into frequency
`components representing the amplitudes of the signal in
`
`

`
`5,408,580
`
`4
`3
`unattractive. In addition, the storage requirements of
`critical bands, these two frequency band QMF filters
`the multiple formats partially defeats the basic goal of
`are arranged in a tree-structured configuration. That is,
`reducing the amount of storage needed to store the
`each of the outputs of the first level filter becomes the
`audio material.
`input to another filter bank at least one of whose two
`outputs is fed to yet another level, and so on. The leaf 5
`Furthermore, the above discussion assumes that the
`computational resources of a particular playback plat-
`nodes of this tree provide an approximation to a critical
`form are fixed. This assumption is not always true in
`band analysis of the input audio track. It can be shown
`practice. The computational resources of a computing
`that this type of filter bank used different length audio
`system are often shared among a plurality of applica-
`segments to generate the different frequency compo-
`nents. That is, a low frequency component represents 10 tions that are running in a time-shared environment.
`the signal amplitude in an audio segment that is much
`Similarly, communication links between the playback
`longer than a high-frequency component. Hence, the
`platform and shared storage facilities also may be
`need to choose a single compromise audio segment
`shared. As the playback resources change, the format of
`length is eliminated.
`the audio material must change in systems utilizing a
`While tree structured filter banks having many layers 15 multi-format compression approach. This problem has
`may be used to decompose the frequency spectrum into
`not been adequately solved in prior art systems.
`Broadly, it the object of the present invention to
`critical bands, such filter banks introduce significant
`aliasing artifacts that limit their utility. In a multilevel
`provide an improved audio compression system.
`It is a further object of the present invention to pro-
`filter bank, the aliasing artifacts are expected to increase
`exponentially with the number of levels. Hence, filter 20 vide an audio compression system which utilizes a fre-
`quency decomposition system that has good frequency
`banes with large numbers of levels are to be avoided.
`Unfortunately, filter banks based on QMF filters which
`resolution at low frequencies and good temporal resolu-
`divide the signal into two bandlimited signals require
`tion at high frequencies without utilizing tree structured
`large numbers of levels.
`filter banks having large numbers of levels.
`It is yet another object of the present invention to
`Prior art audio compression systems are also poorly 25
`suited to applications in which the playback of the ma-
`provide an audio compression system that allows the
`terial is to be carried out on a digital computer. The use
`compressed material to played back on a variety of
`of audio for computer applications is increasingly in
`playback platforms with different computational capa-
`demand. Audio is being integrated into multimedia ap-
`bilities without maintaining multiple copies of the com-
`plications such as computer based entertainment, train- 30 pressed material.
`ing, and demonstration systems. Over the course of the
`It is a still further object of the present invention to
`next few years, many new personal computers will be
`provide an audio compression system in which the
`outfitted with audio playback and recording capability.
`bandwidth needed to transmit the audio material may
`In addition, existing computers will be upgraded for
`be varied in response to changes in the available band-
`audio with the addition of plug-in peripherals.
`35 width.
`These and other objects of the present invention will
`Computer based audio and video systems have been
`limited to the use of costly outboard equipment such as
`become apparent from the following detailed descrip-
`an analog laser disc player for playback of audio and
`tion of the invention and the accompanying drawings.
`video. This has limited the usefulness and applicability
`of such systems. With such systems it is necessary to 40
`provide a user with a highly specialized playback con(cid:173)
`figuration, and there is no possibility of distributing the
`media electronically. However, per~onal computer
`based systems using compressed audio and video data
`promise to provide inexpensive playback solutions and 45
`allow distribution of program material on digital disks
`or over a computer network.
`Until recently, the use of high quality audio on com(cid:173)
`puter platforms has been limited due to the enormous
`data rate required tier storage and playback. Quality has 50
`been compromised in order to store the audio data con(cid:173)
`veniently on disk. Although some increase in perfor(cid:173)
`mance and some reduction in bandwidth has been
`gained using conventional audio compression methods,
`these improvements have not been sufficient to allow 55
`playback of high fidelity recordings on the commonly
`used computer platforms without the addition of expen(cid:173)
`sive special purpose hardware.
`One solution to this problem -would be to use lower
`quality playback on computer platforms that lack the 60
`computational resources to decode compressed audio
`material at high fidelity quality levels. Unfortunately,
`this solution requires that the audio material be coded at
`various quality levels. Hence, each audio program
`would need to be stored in a plurality of formats. Differ- 65
`ent types of users would then be sent the format suited
`to their application. The cost and complexity of main(cid:173)
`taining such multi-format libraries makes this solution
`
`SUMMARY OF THE INVENTION
`The present invention comprises audio compression
`and decompression systems. An audio compression
`system according to the present invention converts an
`audio signal into a series of sets of frequency compo(cid:173)
`nents. Each frequency component represents an ap(cid:173)
`proximation to the audio signal in a corresponding fre(cid:173)
`quency band over a time interval that depends on the
`frequency band. The received audio signal is analyzed
`in a tree-structured sub-band analysis filter. The sub(cid:173)
`band analysis filter bank comprises a tree-structured
`array of sub-band filters, the audio signal forming the
`input of the root node of the tree-structured array and
`the frequency components being generated at the leaf
`nodes of the tree-structured array. Each of the sub-band
`filter banks comprises a plurality of FIR filters having a
`common input for receiving an input audio signal. Each
`filter generates an output signal representing the input
`audio signal in a corresponding frequency band, the
`number of FIR filters in at least one of the sub-band
`filter bank is greater than two, and the number of said
`FIR filters in at least one of the sub-band filters is differ(cid:173)
`ent than the number of FIR filters in another of the
`sub-band filters. The frequency components generated
`by the sub-band analysis filter are then quantized using
`information about the masking features of the human
`auditory system.
`A decompression system according to the present
`invention regenerates a time-domain audio signal from
`
`

`
`15
`
`5
`the sets of frequency components such as those gener(cid:173)
`ated by a compression system according to the present
`invention. The decompression system receives a com(cid:173)
`pressed audio signal comprising sets of frequency com(cid:173)
`ponents, the number of frequency components in each 5
`set being M. The decompression apparatus synthesizes
`M time domain audio signal values from each of the
`received set of frequency components. The synthesis
`sub-system generates 2M polyphase components from
`the set of frequency components. Then it generates a W 10
`entry array from the polyphase phase components and
`multiples each entry in the array by a corresponding
`weight value derived from a prototype filter. The time
`domain audio samples are then generated from the
`weighted array. The generated samples are stored in a
`FIFO buffer and outputted to a D/ A converter. The
`FIFO buffer generates a signal indicative of the number
`of time domain audio signal values stored therein. The
`rate at which these sample values are outputted to the 20
`DI A converters is determined by clock. The preferred
`embodiment of the decompression system includes a
`controller that uses the level indicator in the FIFO
`buffer or other operating system loading parameter to
`adjust the computational complexity of the algorithm 25
`used to synthesize the time domain samples. When the
`level indicator indicates that the number of time domain
`samples stored in the FIFO buffer is less than a first
`predetermined value, the normal synthesis operation is
`replaced by one that generates an approximation to the 30
`time domain samples. This approximation requires a
`smaller number of computations than would be required
`to generate the time domain audio signal values. The
`approximation may be generated by substituting a trun(cid:173)
`cated or shorter prototype filter or by eliminating the 35
`contributions of selected frequency components from
`the computation of the polyphase components. In ste(cid:173)
`reophonic systems, the controller may also switch the
`synthesis system to a monaural mode based on average
`frequency components which are obtained by averag- 40
`ing corresponding frequency components for the left
`and right channels.
`
`BRIEF DESCRIPTION OF THE ORA WINGS
`FIG. 1 is a block diagram of an audio compression 45
`system.
`FIG. 2 is a block diagram of a sub-band decomposi(cid:173)
`tion filter according to the present invention.
`FIG. 3 illustrate the relationship between the length
`0
`of the segment of the original audio signal used to gen- 5
`erate the frequency of each sub-band and the bandwidth
`of each band.
`FIG. 4 illustrates the relationship between successive
`overlapping segments of an audio signal.
`FIG. S(a) is a block diagram of an audio filter based
`on a low-frequency filter and a modulator.
`FIG. S(b) is a block diagram of a sub-band analysis
`filter for generating a set of frequency components.
`FIG. 6 illustrates the manner in which a sub-band 60
`analysis filter may be utilized to obtain the frequency
`information needed for psycho-acoustical analysis of
`the audio signal prior to quantization.
`FIG. 7 is a block diagram of an audio decompression
`system for decompressing the compressed audio signals 65
`generated by a compression system.
`FIG. 8 is a block diagram of a synthesizer according
`to the present invention.
`
`55
`
`5,408,580
`
`6
`FIG. 9 is a block diagram of an audio decompression
`system utilizing the variable computational load tech(cid:173)
`niques of the present invention.
`FIG. 10 is a block diagram of a stereophonic decom(cid:173)
`pression system according to the present invention.
`FIG. 11 is a block diagram of a stereophonic decom(cid:173)
`pression system according to the present invention
`using a serial computation system.
`FIG. 12 is a block diagram of an audio compression
`apparatus utilizing variable computational complexity.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`The manner in which the present invention obtains its
`advantages over prior art audio compression systems
`may be more easily understood with reference to the
`manner in which a conventional audio compression
`system operates. FIG. 1 is a block diagram of an audio
`compression system 10 using a conventional sub-band
`analysis system. The audio compression system accepts
`an input signal 11 which is divided into a plurality of
`segments 19. Each segment is analyzed by a filter bank
`12 which provides the frequency components for the
`segment. Each frequency component is a time average
`of the amplitude of the signal in a corresponding fre(cid:173)
`quency band. The time average is, in general, a
`weighted average. The frequencies of the sub-bands are
`uniformly distributed between a minimum and maxi(cid:173)
`mum value which depend on the number of samples in
`each segment 19 and the rate at which samples are
`taken. The input signal is preferably digital in nature;
`however, it will be apparent to those skilled in the art
`that an analog signal may be used by including an ana(cid:173)
`log-to-digital converter prior to filter bank 12.
`The component waveforms generated by filter bank
`12 are replaced by digital approximations by quantizer
`14. The number of bits assigned to each amplitude is
`determined by a psycho-acoustic analyzer 16 which
`utilizes information about the auditory system to mini(cid:173)
`mize the distortions introduced by the quantization. The
`quantized frequency components are then further coded
`by coder 18 which makes use of the redundancy in the
`quantized components to further reduce the number of
`bits needed to represent the coded coefficients. Coder
`18 does not introduce further errors into the frequency
`components. Coding algorithms are well known to
`those skilled in the signal compression arts, and hence,
`will not be discussed in more detail here.
`The quantization process introduces errors into the
`frequency coefficients. A quantization scheme replaces
`the amplitude of each frequency component by an inte(cid:173)
`ger having a finite precision. The number of bits used to
`represent the integers will be denoted by P. The inte(cid:173)
`gers in question are then transmitted in place of the
`individual frequency components. At the receiver, the
`inverse of the mapping used to assign the integer values
`to the frequency components is used to produce ampli(cid:173)
`tudes that are used in place of the original amplitudes
`for the frequency components. There are at most 2P
`distinct values that can be represented; hence, if there
`are more than 2Pdifferent frequency component values,
`at least some of the frequency components will not be
`exactly recovered. The goal of the quantization algo(cid:173)
`rithm is to minimize the overall effect of the quantiza(cid:173)
`tion errors on the listener.
`The errors introduced by the quantization algorithm
`affect the reconstructed audio track for a time period
`equal to the length of the segment analyzed to calculate
`
`

`
`5,408,580
`
`8
`7
`composition filter for carrying out this decomposition is
`the frequency components. The artifacts introduced by
`shown at 30 in FIG. 2. Filter 30 includes two levels of
`these errors are particularly noticeable in regions of the
`filter banks. The manner in which the filter banks are
`audio track in which the sound increases or decreases in
`constructed will be discussed in more detail below. For
`amplitude over a period of time which is short com-
`pared to the length of the segments being analyzed. 5 the purposes of the present discussion, it is important to
`Because of the rapid rise, the set of frequency compo-
`note that the decomposition is carried out with only
`nents of audio track in the segment will have a number
`two levels of filters, and hence, avoids the aliasing prob-
`of high-frequency components of significant amplitude
`lems inherent in QMF filter banks that require many
`which are not present in the segments on either side of
`levels. The aliasing problems encountered with QMF
`the segment in question. Consider a quantization error IO filter banks become significant when the number of
`in one of these high-frequency components. The error is
`levels exceeds 4.
`equivalent to adding noise to the original signal. The
`The first level of filter 30 consists of a filter bank 31
`amplitude of the noise will be determined by the quanti-
`which divides the input signal into eight sub-bands of
`zation error. This noise will be present for the entire
`equal size. The second level sub-divides the lowest
`length of the segment in the reconstructed audio track. 15 three frequency bands from filter bank 31 into finer
`The noise resulting from the quantization error com-
`sub-divisions. The second level consists of three filter
`mences at the boundary of the segment even though the
`banks 32-34. Filter bank 32 divides the lowest sub-band
`attack begins in the middle of the segment. The ampli-
`from filter bank 31 into 8 equal sub-bands. Filter bank 33
`tude of the noise in the early part of segment may be of
`and filter bank 34 divide the second and third sub-bands
`the same order of magnitude as the reconstructed audio 20 created by filter bank 31 into four sub-bands. The com-
`track; hence, the noise will be particularly noticeable.
`bination of the two levels generates 21 frequency sub-
`Since the noise precedes the actual rise in intensity of
`bands. The relationship between the length of the seg-
`the audio track, it is perceived as a "pre-echo". If the
`ment of the original audio signal used to generate the
`segment duration is long compared to the rise time of
`frequency and phase of each sub-band and the band-
`the audio signal, the pre-echo is particularly noticeable. 25 width of each band is shown schematically in FIG. 3.
`Hence, it would be advantageous to choose filter bands
`The lower frequencies, bands 1-8, have the finest fre-
`in which the high-frequency components are calculated
`quency resolution, but the poorest temporal resolution.
`from segments that are shorter than those used to calcu-
`The highest frequencies, bands 17-21, have the poorest
`late the low-frequency components. This arrangement
`frequency resolution, but the finest time resolution. This
`avoids the situation in which the segment used to com- 30 arrangement more nearly approximates the ear's sensi-
`pute high-frequency components is long compared to
`tivity than systems utilizing filter banks in which all
`the rate of change of the component being computed.
`bands have the same temporal resolution, while avoid-
`Low bit rate audio compression systems operate by
`ing the aliasing problems inherent in tree-structured
`distributing the noise introduced by quantization so that
`filters having many levels of filters.
`it is masked by the signal. The ear's sensitivity to a noise 35 While quantization errors in each of the amplitudes
`source in the presence of a localized frequency compo-
`still introduces noise, the noise spectrum obtained with
`nent such as a sine tone depends on the relative levels of
`this embodiment of the present invention is less objec-
`the signals and on the relation of the noise spectral
`tionable to a human listener than that obtained with
`components to the tone. The noise becomes signifi-
`prior art systems. As noted above, prior art systems
`cantly less audible if its spectral energy is within one 40 tend to have a noise spectrum which changes abruptly
`at the segment boundaries. In the present invention, the
`critical bandwidth of the tone. Hence, it would be ad-
`vantageous to choose filter bands that more closely
`amplitude of the quantization noise can switch more
`rapidly at higher frequencies. If the length of the low
`match the critical bands of the human auditory system.
`The present inven