5
`
`EXPLORATION OF TIMBRE BY
`
`ANALYSIS AND SYNTHESIS
`
`JEAN-CLAUDE RISSET
`
`Directeur de Recherche au CNRS
`
`Laboratoire de Mécanique et d’Acoustique
`Marseille, France
`
`Davip L. WESSEL
`
`Centerfor New Music and Audio Technologies
`Department of Music
`University of California, Bérkeley
`Berkeley, California
`
`I]. TIMBRE
`
`Timbrerefers to the quality of sound. It is the perceptual attribute that enables
`us to distinguish amongorchestral instrumentsthat are playing the same pitch and
`are equally loud. But, unlike loudness and pitch, timbre is not a well-defined per-
`ceptual attribute. Definitions tend to indicate what timbreis not rather than whatit
`is. Take as an example the following enigmatic definition provided by the Ameri-
`can National StandardsInstitute (1960, p. 45): “Timbreis that attribute of auditory
`sensation in terms of which a listener can judge that two sounds similarly pre-
`sented and having the same loudness and pitch are dissimilar.”
`The notion of timbral constancy or invariance is even vaguer than that sug-
`gested in the definitions of timbre as a basis for discrimination. It would seem that
`a form of timbral constancy is implied by the commonobservation that a sound
`source can be reliably identified over a wide variety of circumstances. For ex-
`ample, a saxophoneis readily identified as such regardlessof the pitch or dynamic
`it is playing. Furthermore, the saxophone remains a saxophone whetherit is heard
`over a distortion-ridden pocket-sized transistor radio or directly in a concert hall.
`Thus, the question arises as to the physical correlates of this constancy. Is there a
`physical invariant or a characteristic feature mediating a given timbre?
`The issue is not only academic: it has musical relevance, because electronic
`and computer technology promises access to an unlimited world of timbres. One
`
`—,—TT Copyright © 1999 by Academic Press.
`The Psychology ofMusic, Second Edition
`113
`All rights of reproduction in any form reserved.
`
`Sony Exhibit 1028
`Sony Exhibit 1028
`Sony v. MZ Audio
`Sony v. MZ Audio
`
`

`

`114
`
`JEAN-CLAUDE RISSET & DAVID L. WESSEL
`
`must, however, know how to evoke a given timbre; that is, how to describe it in
`terms of the physical structure of sound.
`
`H. TIMBRE AND THE FOURIER SPECTRUM:
`
`THE CLASSICAL VIEW
`
`Physicists have been analyzing musical instrument tones for some time. The
`goal of manyof these acoustical analyses is to determine the physicalcorrelates of
`tone quality.
`Manyresults of such analyses have been published (Culver, 1956; Meyer &
`Buchmann,1931; Miller, 1926; Olson, 1967; Richardson, 1954). The general con-
`clusion of such studies was that musical sounds are periodic and that the tone
`quality is associated solely with the waveshape, more precisely with the Fourier
`spectrum of the waveshape. These early analyses were strongly motivated by the
`theorem of Fourier, which states that a periodic waveshape is completely defined
`by the amplitudes and phases of a harmonicseries of frequency components (see
`Feynman,Leighton, & Sands, {963, chapters 21-25; Jenkins & Watts, 1968). But
`the claim, often known as Ohm’s acoustical law, is that the ear is phase deaf. Put
`more precisely, Ohm’s acoustical law states that if the Fourier representations of
`two sounds have the same pattern of harmonic amplitudes but have different pat-
`terns of phase relationships, a listener will be unable te perceive a difference be-
`tween the two sounds, even though the sounds mayhave very different waveforms
`(see Figure 1).
`;
`It has been argued that the ear is not actually phase deaf. It is indeed true that
`under certain conditions, changing the phase relationship between the harmonics
`of a periodic tone can alter the timbre (Mathes & Miller, 1947; Plomp &
`Steeneken, 1969); however, this effect is quite weak, and it is generally inaudible
`in a normally reverberant room where phase relations are smeared (Cabot, Mino,
`Dorans, Tackel, & Breed, 1976; Schroeder, 1975). One must remember, though,
`that this remarkable insensitivity to phase, illustrated by Figure 1, holds only for
`the phaserelationship between the harmonicsofperiodic tones.'
`Thus, it would appear that timbre dependssolely on the Fourier spectrum of the
`sound wave. The mostauthoritative proponentof this conception has been Helm-
`holtz (Helmholtz, 1877/1954). Helmholtz was aware that “certain characteristic
`particularities of the tones of several instruments depend on the mode in which
`they begin and end,” yet he studied only “the peculiarities of the musical tones
`which continue uniformly,” considering that they determined the “musical quality
`of the tone.” The temporal characteristics of the instruments were averaged out by
`'A varying phase can be interpreted as a varying frequency. Also, dispersive media (for which the
`speed of propagation is frequency dependent) cause inaudible phase distortion for periodic tones and
`objectionable delay distortion for nonperiodic signals (e.g., the high frequencies can be shifted by
`several sounds with respectto the low onesin a long telephone cable: this makes speech quite incom-
`prehensible).
`
`

`

`5. EXPLORATION OF TIMBRE BY ANALYSIS AND SYNTHESIS
`
`115
`
`NA MG
`
`| \Ay
`
`The waves 1 to 4 correspond to tones generated with the same spectrum but with
`FIGURE 1_
`different phase relations between the components: these tones with quite different waveforms sound
`very similar (Plomp, 1976).
`
`the early analyses (Hall, 1937); but because different instruments had different
`average spectra, it was believed that this difference in average spectrum wasut-
`terly responsible for timbre differences. This view is still widely accepted: a re-
`puted andrecenttreatise like the Feynmann Lectures on Physics gives no hint that
`there may be factors of tone quality other than “the relative amountof the various
`harmonics.”
`Actually, even a sine wave changesquality from the low to the high end ofthe
`musical range (KG6hler, 1915, Stumpf, 1926). In order to keep the timbreof a peri-
`odic tone approximately invariant when the frequency is changed, should the
`spectrum be transposed so as to keep the same amplitude relationship between the
`harmonics or should the absolute position of the spectral envelope be kept invari-
`ant? This question produced a debate between Helmholtz and Herman (cf. Winc-
`kel, 1967, p. 13). In speech, a vowel corresponds approximately to a spectrum
`with a given formantstructure. A formant is a peak in the spectral envelope that
`occurs at a certain frequency andis often associated with a resonancein the sound
`source. This is the case for voice sounds, and the formants can berelated to reso-
`nances in the vocaltract.
`Indeed, in many cases, a fixed formant structure (Figure 2) gives a timbre that
`varies less with frequency than a fixed spectrum—a better invariance for “sound
`color,” as Slawson (1985)calls timbre for nonchanging sounds (Plomp, 1976, pp.
`107-110; Plomp & Steeneken, 1971; Slawson, 1968).
`Certain characteristics of the spectrum induce certain timbral qualities. This
`can easily be demonstrated by modifying the spectrum with filters. Brightness (or
`sharpness)relates to the position of the spectral envelope along the frequency axis
`(see Section XVI). Presence appearsto relate to strong components around 2000
`Hz.
`The concept ofcritical bandwidth,’ linked to the spectral resolution of the ear
`(Plomp, 1966), may permit a better understanding of the correlation between
`spectrum and timbre. In particular, if many high-order harmonicslie close to-
`gether, that is, within the same critical bandwidth, the sound becomesvery harsh.
`
`?Thecritical bandwidth around a certain frequency roughly measures the range within which this
`frequency interacts with others. The width ofa critical band is about one third of an octave above 500
`Hz and approximately 100 Hz below 500 Hz (cf. Scharf, 1970). This important parameter of hearing
`relates to spectral resolution (Plomp, 1964, 1976).
`
`

`

`116
`
`JEAN-CLAUDE RISSET & DAVID L. WESSEL
`
`Harmonic
`
`4 number
`
`FREQUENCY
`
`INTENSITY
`
`ALTERNATIVE A
`
`
`ALTERNATIVE B
`
`FREQUENCY
`
`FIGURE 2 Thisfigurerefers to an experiment by Slawson (1968) comparing alternative predic-
`tions of invariance in timbre under octave increases in fundamental frequency. The experiment rules
`out alternative B, that of the relative pitch or overtonetheory, in favorofalternative A,that of the fixed-
`frequency or formanttheory.
`
`FREQUENCY
`
`Hence, for instance, antiresonances in the frequency response of string instru-
`ments play an important part in diminishing the roughnessof the tones. It may be
`more significant to display spectra modified so as to take critical bands into ac-
`count. This was done in some studies: the frequency axis is converted into so-
`called Bark units: 1 Bark corresponds to the width of onecritical band over the
`whole frequency range (cf. Grey & Gordon, 1978; Moore & Glasberg, 1981;
`Scharf, 1970; Zwicker, 1961).
`
`Il. THE SHORTCOMINGS OF THE CLASSICAL
`CONCEPTION
`
`So, for periodic tones, timbre depends upon spectrum.It has long been thought
`that musical tones are periodic, at least for most of their duration. Musical tones
`are often thought of as comprising three sections: attack, steady state, and decay.
`Helmholtz and his followers considered timbre to be determined by the spectrum
`of the steady state. However, this conception suffers from serious difficulties. As
`
`

`

`5. EXPLORATION OF TIMBRE BY ANALYSIS AND SYNTHESIS
`
`117
`
`wenoted at the beginning of this chapter, musical instruments can be recognized
`even from a very poor recording, despite the fact that their spectra are radically
`changed by such distortion (Eagleson & Eagleson, 1947).
`In fact, a normally reverberant room has an incredibly jagged frequency re-
`sponse, with fluctuations up to 20 dB, and this frequency responseis different at
`every point in the room (Wente, 1935). Hence, spectra are completely changed in
`ways that depend onthe specific location. However, when one movesin the room,
`the corresponding timbres are not completely upset as one would expect them to
`be if they depended only on the precise structure of the frequency spectrum.
`Also, various methods of sound manipulation show that temporal changes bear
`strongly on tone quality. Removing the initial segment of notes played by various
`instruments impairs the recognition of these instruments, as noted by Stumpf as
`early as 1910 (Stumpf, 1926). Subsequently, tape-recorder manipulation (George,
`1954; Schaeffer, 1966) has madeit easy to demonstrate the influence of time fac-
`tors on tone quality. For instance, playing a piano tone backwards gives a non-
`piano-like quality, although the original and the reversed sound have the same
`spectra. However, temporal factors were not taken into account in mostearly anal-
`yses (cf. Hall, 1937): the analysis process could not follow fast temporal evolu-
`tions.
`Recently, computer sound synthesis (Mathews, 1963, 1969) has madeit pos-
`sible to synthesize virtually any sound from a physical description of that sound.
`Efforts have been madeto use the results of analyses of musical instrument tones
`that are to be found in treatises on musical acoustics as input data for computer
`sound synthesis. In most cases, the sounds thus obtainedbear little resemblance to
`the actual tones produced by the instrument chosen; the tones thus produced are
`dull, lacking identity and liveliness (Risset & Mathews, 1969). Hence,the avail-
`able descriptions of musical instrument tones must be considered inadequate, be-
`cause they fail to pass the foolproof synthesis test. This failure points to the need
`for more detailed, relevant analyses and for a more valid conception of the physi-
`cal correlates of timbre. Clearly, one must perform somekind of “running” analy-
`sis that follows the temporal evolution of the tones.
`
`IV. ATTACK TRANSIENTS
`
`A few attempts have been made since 1930 to analyze the attack transients of
`instrument tones (Backhaus, 1932; Richardson, 1954). These transients constitute
`an important part of the tones—in fact, many toneslike those from the piano or
`percussion instruments have no steady state—yettheir analysis has not produced.
`much progress. The transients are intrinsically complex, and they are not repro-
`ducible from one tone to another, even for tones that sound very similar (Schaef-
`fer, 1966). Most analyses have been restricted to a limited set of tones, and the
`researchers have tended to make generalizations that may be inappropriate even
`for different samples collected from the same instruments. These shortcomings
`
`

`

`118
`
`JEAN-CLAUDE RISSET & DAVID L. WESSEL
`
`have produced many discrepanciesin the literature and cast doubt on the entire
`body of acoustic data.
`
`V. COMPLEXITY OF SOUNDS: IMPORTANCE OF
`CHARACTERISTIC FEATURES
`
`Soundsare often intrinsically complex. Musical instruments have a complex
`physical behavior (Benade, 1976); often the damping is low, and transients are
`long compared with note duration. Also, the tones are not generated by a standard-
`ized mechanical player, but by human musicians whointroduce intricacies both
`intentionally and unintentionally. Even if a human player wanted to, a human be-
`ing could not repeat a note as rigorously as a machine does.If the musician has
`good control of the instrument, he or she should be able to play two tones sound-
`ing nearly identical, but these tones can differ substantially in their physical struc-
`ture. More often the performer will not wantto play all notes the same way, and
`the performer’s interpretation of some markings depends on the performer’s sense
`of style and technique. All these considerations, which involve different disci-
`plines—physics, physiology, psychology, esthetics—certainly makeit difficult to
`isolate characteristic invariants in musical instrument sounds.
`This points out the needto extract significant features from a complex physical
`structure. Also, one must be able to control through synthesis the aural relevance
`of the features extracted in the analysis—to perform analysis by synthesis. Only
`recently has this been possible, thanksto the precision and flexibility of the digital
`computer.
`Weshall now review pioneering work on the exploration of timbre by computer
`analysis and synthesis.
`
`VI. INSTRUMENTAL AND VOCAL TIMBRES:
`ADDITIVE SYNTHESIS
`
`The study of trumpet tones performed in the mid-1960s by one of the authors
`(Risset, 1966; Risset & Mathews, 1969)illustrates some of the points just made.
`Wechose trumpet tones because we were experiencing difficulties in synthesizing
`brasslike sounds with the computer. The tones synthesized with fixed spectra de-
`rived from the analysis of trumpet tones did not evoke brass instruments.
`To obtain more data, we recorded musical fragments played by a professional
`trumpet player in an anechoic chamber. Sound spectrogramssuggestedthat, for a
`given intensity, the spectrum has a formantstructure; that is, the spectrum varies
`with frequency so as to keep a roughly invariant spectral envelope. The spectro-
`grams gave useful information, althoughit was not precise enough. Thus, selected
`tones were converted to digital form and analyzed by computer, using a pitch-
`synchronous analysis (PISA program, Mathews, Miller, & David, 1961). Pitch-
`
`

`

`5. EXPLORATION OF TIMBRE BY ANALYSIS AND SYNTHESIS
`
`119
`
`synchronousanalysis assumesthat the sound is quasi-periodic; it yields displays
`of the amplitude of each harmonicasa function of time (one point per fundamen-
`tal pitch period). The curved functions resulting from the analysis program were
`approximatedwith linear segments (Figure 3). These functions were then supplied
`to the MUSIC IV sound-synthesis program,andthe resulting synthetic tones were
`indistinguishable from the originals, even when compared by musically skilled
`listeners. Hence, the additive synthesis model, with harmonic components con-
`trolled by piecewise linear functions, captures the aurally importantfeatures of the
`sound.
`Conceptually, the model is simple. The pitch-synchronous analysis yields a
`string of snapshot-like spectra, hence a kindoftime-variant harmonic analysisthat
`is further reducedbyfitting the linear segments to the amplitude envelope of each
`component. Computationally, however, this model is not very economical. Figure
`3 showsthat the functions can be quite complex, and the parameters mustbeesti-
`mated for every tone. So further simplifications of the model were sought. By
`systematic variation of the various parareters— one at a time—therelative im-
`portance of the parameters was evaluated. Whereas some parameters were dis-
`missed as aurally irrelevant—for example, short-term amplitude fluctuations—a
`few physical features were found to be of utmost importance. These include the
`following: the attack time, with faster buildup of the low-order harmonics than the
`
`
`
` LINEAR
`
`AMPLITUDESCALE
`
`3
`
`
`O12
`TIME (SEC)
`
` O16 620
`
`FIGURE 3 Line-segmentfunctions that approximate the evolution in time of 13 harmonics ofa
`D,trumpettone lasting 0. 2 sec. Functionslike these, obtained by analysis ofreal tones, have been used
`to control the harmonic amplitudes of synthetic tones (Risset & Mathews, 1969).
`
`

`

`120
`
`JEAN-CLAUDE RISSET & DAVID L. WESSEL
`
`high-order ones; for certain tones, a quasi-random frequency fluctuation; and,
`most importantly, a peak in the frequency spectrum between 1000 and 1500 Hz
`and an increase in the proportion of high-order harmonics with intensity.
`In fact, the latter property permitted us to abstract a simplified model of brass-
`like tones. Here only the amplitude function forthe first harmonic wasprovided,
`and the amplitude functions for the other harmonics were deducedas fixed func-
`tions of this first harmonic amplitude, such that they increased at a faster rate
`(Risset, 1969). The specification was much more economical than the previous
`one and did not need to be precisely adjusted to yield the brasslike quality. Hence
`this property of an increase in spectral width with amplitude seemsto be the most
`salient physical correlate of brasstone quality. This showsthat, in addition to the
`way spectra vary overthe pitch range,the variation in time of the spectrum can be
`critical to determine timbre. In the case of the brass,it can be described in terms of
`a nonlinear characteristic that enriches the spectrum when the amplitude in-
`creases. During the short attack of a brass tone, lasting less than 50 msec, the ear
`interprets the increase of spectral width as a brassy onset, even though it cannot
`describe what happens.
`Beauchamp (1975) studied nonlinear interharmonic relationships in cornet
`tones and ascribed the brasslike character to the type of nonlinear relationship
`between the different harmonics, which are all functionsof the first one regardless
`of the general level. This relationship has been found to have an acoustical basis
`(Backus & Hundley,1971; Benade, 1976, pp. 439-447). This nonlinear property
`was used in the late sixties by Moog to produce brasslike sounds with his analog
`synthesizers: the cutoff frequency of a low-passfilter was made to go up with the
`amplitude, which was easy to achieve through voltage control. This characteristic
`has also been implemented in a very simple, satisfying way, using Chowning’s
`powerful technique of spectral generation by frequency modulation described
`later (see Section X; Chowning, 1973; Morrill, 1977).
`It was found in the trumpet-tone study that some factors may be important in
`some conditions and inaudible in others. For instance, details of the attack were
`more audible in long sustained tones than in brief tones. Also, it appeared that
`some listeners, when comparing real and synthetic tones, made their decision
`about whethera tone wasreal or synthetic on the basis of some particular property.
`Forinstance, they often assumedthat the real tones should be rougher, more com-
`plex than the synthetic ones. This suggests that by emphasizing roughness in a
`synthetic tone, one could cause the listeners to believe it was a real tone. In his
`striking syntheses of brassy tones, Morrill (1977) has simulated intonation slips
`that greatly enhancethe realistic human character of the tones. Similarly, in their
`study of string tones, Mathews, Miller, Pierce, and Tenney (1965, 1966) had in-
`cludedaninitial random-frequency component, which simulatesthe erratic vibra-
`tion that takes place whenthestring is first set in motion by the bow. When exag-
`gerated, this gives a scratchy sound strikingly characteristic of a beginning string
`player. Such idiomatic details, imperfections, or accidents (Schaeffer, 1966) are
`characteristic of the sound source, and the hearing sense seems to be quite sensi-
`
`

`

`5. EXPLORATION OF TIMBRE BY ANALYSIS AND SYNTHESIS
`
`121
`
`tive to them. Taking this into account might help to give stronger identity and
`interest to synthetic sounds. Indeed, a frequency skew imposed on even a simple
`synthetic tone can help strongly endow it with subjective naturalness andidentity.
`The pattern of pitch at the onset of each note is often a characteristic feature of a
`given instrument: the subtle differences between such patterns (e.g., a violin, a
`trombone,a singing voice) act for the ear as signatures of the source of sound.
`The paradigm for the exploration of timbre by analysis and synthesis followed
`in the latter study has been much more thoroughly pursued by Grey and Moorer
`(1977) in their perceptual evaluation of synthesized musical instrument tones.
`Grey and Moorerselected 16 instrumental notes of short duration played near E>
`above middle C. This pitch was selected because it was within the range of many
`instruments (e.g., bass clarinet, oboe, flute, saxophone, cello, violin); thus, the
`tones represented a variety of timbres taken from the brass, string, and woodwind
`families of instruments. The tones were digitally analyzed with a heterodynefilter
`technique, providing a set of time-varying amplitude and frequency functions for
`each partial of the instrumental tone. Digital additive synthesis was used to pro-
`duce a synthetic tone consisting of the superposition of partials, each controlled in
`amplitude and frequency by functions sampled in time. Each of the 16 instruamen-
`tal notes could appear in at least four of the five following conditions:(a) original
`tone; (b) complex resynthesized tone, using the functions abstracted from the
`analysis; (c) tone resynthesized with a line-segment approximation to the func-
`tions (4 to 8 line segments); (d) cut-attack approximation for some of the sounds;
`and (e) constant-frequencies approximation. In order to evaluate the audibility of
`these types of data reduction, systematic listening tests were performed with mu-
`sically sophisticated listeners. The tones were first equalized in duration, pitch,
`and loudness. An AA AB discrimination paradigm was used. On eachtrial four
`tones were played, three of them identical and the fourth one different; the listen-
`ers had to detect whether one note wasdifferent from the others, to tell in which
`pair it was located, and to estimate the subjective difference between this note and
`the others. The judgments were processed by multidimensional scaling tech-
`niques.
`The results demonstrated the perceptual closeness of the original and directly
`resynthesized tones. The major cue helping the listeners to make a better than
`chance discrimination was the tape hiss accompanying the recording of the origi-
`nal tones and notthe synthetic ones. The results also showedthat the line-segment
`approximation to the time-varying amplitude and frequency functionsfor the par-
`tials constituted a successful simplification, leading to a considerable information
`reduction while retaining most of the characteristic subjectivity (see Figure 4).
`This suggests that the highly complex microstructure in the time-varying ampli-
`tude and frequency functions is not essential to the timbre and that drastic data
`reduction can be performed with little harm to the timbre. The constant-frequen-
`cies approximation (for tones without vibrato) was good for some tones but dra-
`matically altered other ones. The importance of the onset pattern of the tones was
`confirmed by the cut-attack case.
`
`

`

`122
`
`JEAN-CLAUDE RISSET & DAVID L. WESSEL
`
`T
`
`relative
`
`amplitude
`
`frequency
`in hertz
`
`(B)
`
`time in seconds <3
`
`(A)
`
`
`
`
`t
`
`relative
`
`amplitude
`
`frequency
`in hertz
`
`time in seconds
`
` (A)Time-varying amplitude functions derived from heterodyne analysis from a bass
`FIGURE 4_
`clarinet tone, shown in a three-dimensional perspective plot. (B) Line-segment approximation to the
`function plotted in A. Both of these functions have been used to resynthesize the tone. Form B gives a
`considerable information reduction (Grey & Moorer, 1977).
`
`

`

`5. EXPLORATION OF TIMBRE BY ANALYSIS AND SYNTHESIS
`
`123
`
`A later study by Charbonneau (1979) has demonstrated that the simplification
`can go even further for most of the tones studied by Moorer and Grey(i.e., short
`tones of nonpercussive instruments). The various envelopes controlling each har-
`monic are replaced by a single averaged envelope; for each harmonic,this curveis
`weighted in order to preserve the maximum amplitude for this harmonic; it is also
`warped in time in order to preserve the times of appearance and extinction of the
`various harmonics. Althoughthis is not a proper modelforflute tones, it permits a
`good imitation for mostof the other instruments. We shall mention other examples
`of simplified resynthesis in the following paragraphs.
`Fletcher and his collaborators (Fletcher & Bassett, 1978; Fletcher, Blackham,
`& Christensen, 1963; Fletcher, Blackham, & Stratton, 1962; Fletcher & Sanders,
`1967) studied the timbre of several instruments by analysis and synthesis, using an
`additive synthesis model. (Theearlier of these studies did not use a computer but
`ad hoc analysis and synthesis devices). A study of the quality of piano tones (Flet-
`cheret al., 1962) indicated that the attack time must be less than 0.01 sec, whereas
`the decay time can vary from 20 sec for the lowest notes to less than 1 sec for the
`very high ones. The variation of partial level versus time during the decay was
`highly complex and not always monotonic—thepartials at times increase in inten-
`sity rather than decrease. However, the complexities of the decay pattern did not
`appear to be very relevant to the ear because the much simplified syntheses could
`sound similar to the original sounds—althoughit appearedin later studies that the
`dissimilarity between the behaviorofdifferentpartials is often linked to liveliness.
`The piano study provided a major insight. It ascribed subjective warmth to the
`inharmonicity of the partials. The frequencies of the successivepartials of a low
`piano toneare close to, but higher than, the frequencies of the harmonicseries, to
`the extent that the 15th partial frequency can be 16 times that of the lowest one
`(Young, 1952). Now this slightly inharmonic pattern gives rise to a complex pat-
`tern of beats that induces a peculiar lively and warm quality. This is an important
`feature for low piano tones (and also for organ tones;cf. Fletcher et al., 1963).
`Many analyses have been performed on piano sounds (Martin, 1947). They
`have been used to devise electronic pianos (Dijksterhuis & Verhey, 1969) whose
`tone quality (although not fully satisfying) depends on the simplified model ab-
`stracted from the analyses. Acoustic piano tones are extremely complex: low notes
`may comprise 100 or moresignificant spectral components, and the spectra fluctu-
`ate considerably. The quality of the acoustic piano may be hard to approach by
`synthesis. The issue is of practical significance, however, because digital pianos
`can be made much cheaperthan acoustic ones, and, even more important, they are
`much easier to insulate acoustically. Current digital pianos obtained by sam-
`pling—thatis, by recording of actual pianos tones—arenotsatisfactory:they fail
`to emulate the extremely responsive character of the acoustic piano, where the
`spectrum changes drastically with loudness, and also the interaction between
`strings occurring thanks to the soundboard.
`
`

`

`
`JEAN-CLAUDE RISSET & DAVID L. WESSEL
`
`124
`
`In a study of violin tones, Fletcher and Sanders (1967) investigated the slow
`frequency modulation (around 6 Hz) knownas vibrato, showingthatit also modu-
`lates the spectrum of the tone. They also pointed to two features that enhance
`naturalness if they are simulated in the synthetic tones: the bowing noise at the
`onsetof the tone and the sympathetic vibrations coming from the openstrings (the
`latter occur substantially only when certain frequencies are played).
`Clark, Luce, and Strong have also performed significant research on wind in-
`strument tones by analysis and synthesis. In a first study (Strong & Clark, 1967a)
`wind instrument tones were synthesized as the sum of harmonics controlled by
`one spectral envelope (invariant with note frequency) and three temporal enve-
`lopes. (A more specific model was also sought for brass instruments, cf. Luce &
`Clark, 1967). Listeners were tested for their capacity to identify the source of the
`tones. Their identification was nearly as good as for real instrument tones, which
`indicates that this model grasps the elements responsible for the difference be-
`tween the soundsof the different instruments. Incidentally, the probability of con-
`fusion between the tones of two instruments gives an indication of the subjective
`similarity between these tones; it has been usedto ascertain the perceptualbasis of
`the conventional instrument families (cf. Clark, Robertson, & Luce, 1964). The
`results suggest that some conventional families represent fairly well the subjective
`differentiations, especially the string and the brass family. A double reed family
`also emerged, comprising a tight subfamily (oboe and English horn) and a more
`remote member(the bassoon).
`
`VII. ADDITIVE SYNTHESIS: PERCUSSION
`INSTRUMENTS
`
`The aforementioned studies of timbre resorted to models of additive synthesis,
`whereby the sound was reconstituted as the superposition of a number of fre-
`quency components, each of which can be controlled separately in amplitude and
`frequency. Such models require much information specifying in detail the way
`each componentvaries in time: hence, they are not very economical in terms of the
`amountof specification or the quantity of computations they require. However,as
`wasstated, the information on the temporal behavior of the components can often
`be simplified. In addition, the development of the digital technology has madeit
`possible to build special processors with considerable processing power, for in-
`stance, digital synthesizers that can yield in real time dozens of separate voices
`with different envelopes (Alles & Di Giugno, 1977); so additive synthesis is a
`process of practical interest, considering its power and generality. It is not re-
`stricted to quasi-periodic tones; in fact, it can be used to simulate the piano and
`percussion instruments (Fletcher & Bassett, 1978; Risset, 1969).
`In percussion instruments, the partials are no longer harmonics: their frequen-
`cies, found from the analysis, are those of the modes of vibration excited by the
`percussion and can sometimes be predicted from consideration of theoretical
`
`

`

`
`5. EXPLORATION OF TIMBRE BY ANALYSIS AND SYNTHESIS
`
`125
`
`acoustics. The synthesis can correspond to a considerably simplified model and
`still be realistic, provided it takes into accountthe aurally salient features. Fletcher
`and Bassett (1978) have simulated bass drum tones by summing the contribution
`of the most important components detected in the analysis—-these were sine
`waves decaying exponentially, with a frequency shift downward throughout the
`tone. The simulation wasasrealistic as the recorded bass drum tones. The authors
`noted, however, that the loudspeakers could not render the bass drum tones in a
`completely satisfactory way.
`Timbre canoften be evoked by a synthesis that crudely takes into account some
`salient properties of the sound. Bell-like tones can be synthesized by adding to-
`gether a few sine wavesof properly chosen frequencies that decay exponentially at
`different rates—in general, the higher the frequency, the shorter the decay time.
`The frequency tuning of the first components is often critical, as it is in church
`bells, for instance: the frequencies of the first components approximate frequen-
`cies falling on a harmonicseries, so that a distinct pitch (the strike tone) can be
`heard, even though there is no componentat the corresponding frequency. The
`lowest component, which rings longer, is called the hum tone(cf. Rossing, 1990).
`Chinese bells have modes that tend to occur in pairs; these bells can emit two
`distinct notes depending on where they are struc