`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MS 1104
`MS 1104
`
`MS 1104 - Page 1
`
`MS 1104 - Page 1
`
`

`

`lnteractively Skimming
`Recorded Speech
`
`Barry Michael Arons
`
`B.S. Civil Engineering, Massachusetts Institute of Technology, 1980
`MS. Visual Studies, Massachusetts Institute of Technology, 1984
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`Submitted to the Program in Media Arts and Sciences
`in partial fulfillment of the requirements for the degree of
`Doctor of Philosophy at the
`Massachusetts Institute of Technology
`
`
`
`
`
`
`
`
`
`
`
`
`
`February 1994
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`"
`
`
`
`

`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(
`
`$
`
`'
`
`
`
`
`
`
`
`
`
`,
`
`
`
`)
`
`
`
`'
`
`
`
`
`
`*
`
`
`
`
`
`'
`
`
`
`+
`
`
`
`-
`
`
`
`.
`
`
`
`/
`
`!
`
`
`
`'
`
`
`
`
`'
`
`0
`
`
`
`1
`
`
`-
`
`
`
`
`
`%
`
`
`
`
`
`
`2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`'
`
`/
`
`'
`
`:
`
`2
`
`
`
`
`

`.
`
`
`0
`
`
`
`
`6
`
`
`
`)
`
`)
`
`
`
`5
`
`
`
`&
`3
`
`4
`
`
`
`
`
`
`0
`
`
`
`0
`
`
`
`4
`
`
`
`(
`
`
`
`-
`
`
`
`0
`
`7
`
`8
`
`0
`
`)
`
`)
`
`5
`
`)
`
`6
`
`'
`
`+
`
`9
`
`6
`
`(
`
`-
`
`8
`
`Author:
`
`
`
`
`
`
`
`
`
`
`
`<
`
`Certified by:
`
`Christopher M. Schmandt
`Principal Research Scientist
`Thesis Supervisor
`
`Program in Media Arts and Sciences
`January 7, 1994
`
`
`
`>
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`<
`
`Accepted by:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`$
`
`
`
`
`
`
`?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Stephen A. Benton
`Chair, Departmental Committee on Graduate Students
`Program in Media Arts and Sciences
`
`
`
`;
`"
`=
`"
`"
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`!
`!
`
`
`@
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`#
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`#
`#
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`<
`
`
`
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MS 1104 - Page 2
`
`MS 1104 - Page 2
`
`

`

`B
`
`#
`
`C
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`All rights reserved.
`
`
`
`© 1994 Massachusetts Institute of Technology.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MS 1104 - Page 3
`
`MS 1104 - Page 3
`
`

`

`lnteractively Skimming Recorded Speech
`
`3
`
`Barry Michael Arons
`
`Abstract
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`#
`
`Submitted to the Program in Media Arts and Sciences on January 7, 1994,
`in partial fulfillment of the requirements for the degree of
`Doctor of Philosophy.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`=
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`Listening to a speech recording is much more difficult than visually
`scanning a document because of the transient and temporal nature of
`audio. Audio recordings capture the richness of speech, yet it is difficult
`to directly browse the stored information. This dissertation investigates
`techniques for structuring, filtering, and presenting recorded speech,
`allowing a user to navigate and interactively find information in the
`audio domain. This research makes it easier and more efficient to listen
`
`D
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`!
`
`!
`
`
`
`
`
`
`
`
`!
`
`to recorded speech by using the SpeechSkimmer system.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`>
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`"
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`K
`
`
`
`
`H
`
`M
`
`
`
`G
`
`
`
`
`
`
`
`I
`
`
`
`J
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`First, this dissertation describes Hyperspeech, a speech-only hypermedia
`system that explores issues of speech user interfaces, browsing, and the
`use of speech as data in an environment without a visual display. The
`system uses speech recognition input and synthetic speech feedback to
`aid in navigating through a database of digitally recorded speech. This
`system illustrates that managing and moving in time are crucial in speech
`interfaces. Hyperspeech uses manually segmented and structured speech
`recordings—a technique that is practical only in limited domains.
`
`
`
`
`
`
`
`O
`
`
`
`!
`
`
`
`
`P
`
`
`
`O
`
`
`
`J
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`H
`
`Q
`
`
`L
`
`I
`
`I
`
`J
`
`P
`
`
`
`N
`
`J
`
`
`
`
`
`
`
`
`L
`
`T
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U
`
`
`!
`
`
`
`
`
`$
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`V
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`R
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`Second, to overcome the limitations of Hyperspeech while retaining
`browsing capabilities, a variety of speech analysis and user interface
`techniques are explored. This research exploits properties of spontaneous
`speech to automatically select and present salient audio segments in a
`time-efficient manner. Two speech processing technologies, time
`compression and adaptive speech detection (to find hesitations and
`pauses), are reviewed in detail with a focus on techniques applicable to
`extracting and displaying speech information.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`!
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`W
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`J
`
`
`
`
`
`!
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`W
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`X
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`F
`
`
`
`#
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`S
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`V
`
`
`
`
`
`
`
`$
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`J
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`_
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`O
`
`
`J
`
`
`
`
`
`L
`
`
`N
`
`Z
`
`
`
`
`
`]
`
`J
`
`Y
`
`K
`
`J
`
`
`
`Y
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`I
`
`K
`
`Z
`
`O
`
`
`S
`
`J
`
`S
`
`N
`
`[
`
`
`
`
`
`\
`
`]
`
`\
`
`Y
`
`N
`
`^
`
`J
`
`J
`
`J
`
`K
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
``
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Finally, this dissertation describes SpeechSkimmer, a user interface for
`interactively skimming speech recordings. SpeechSkimmer uses simple
`speech processing techniques to allow a user to hear recorded sounds
`quickly, and at several levels of detail. User interaction, through a
`manual input device, provides continuous real-time control of the speed
`and detail level of the audio presentation. SpeechSkimmer incorporates
`time-compressed speech, pause removal, automatic emphasis detection,
`and non-speech audio feedback to reduce the time needed to listen. This
`dissertation presents a multi-level structural approach to auditory
`skimming, and user interface techniques for interacting with recorded
`speech.
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`I
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`K
`
`M
`
`$
`$
`I
`
`H
`
`
`
`
`
`J
`
`K
`
`L
`
`H
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`Y
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`O
`
`N
`
`$
`
`J
`
`P
`
`T
`
`
`
`
`
`!
`
`
`
`
`P
`
`
`
`
`
`
`
`
`

`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`N
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Thesis Supervisor
`
`Christopher M. Schmandt
`Principal Research Scientist, Program in Media Arts and Sciences
`
`
`
`
`
`

`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`[
`
`
`
`
`
`
`
`
`
`
`
`
`U
`
`
`
`!
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`J
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`V
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`K
`
`Y
`
`$
`
`
`Z
`
`
`
`]
`
`J
`
`Y
`
`J
`
`

`
`
`
`
`
`
`
`
`
`
`
`!
`
`I
`
`O
`
`
`
`
`L
`
`
`
`
`
`N
`
`S
`
`S
`
`\
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`!
`
`$
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`N
`
`
`
`
`[
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U
`
`
`
`
`
`
`
`V
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`"
`
`
`
`
`
`Z
`

`
`
`
`
`
`
`
`
`
`$
`$
`G
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U
`
`
`U
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`J
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`This work was performed in the Media Laboratory at MIT. Support for
`this research was provided, in part, by Apple Computer, Inc., Interval
`Research Corporation, and Sun Microsystems, Inc. The ideas expressed
`herein do not necessarily reflect those of the supporting agencies.
`
`
`
`!
`
`
`
`
`
`!
`!
`
`
`
`
`
`?
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U
`

`
`
`
`"
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`MS 1104 - Page 4
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`
`
`
`G
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`O
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`G
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`E
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`$
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`S
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`#
`
`$
`
`$
`
`
`
`$
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`V
`
`$
`
`
`
`
`
`
`
`
`
`
`
`@
`E
`
`
`
`;
`
`
`
`
`
`$
`;
`!
`
`"
`
`?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`!
`$
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`!
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`$
`
`
`
`
`I
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MS 1104 - Page 4
`
`

`

`b
`
`MS 1104 - Page 5
`
`MS 1104 - Page 5
`
`

`

`Doctoral Dissertation Committee
`
`c
`
`Thesis Advisor:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`<
`
`Thesis Reader:
`
`Christopher M. Schmandt
`Principal Research Scientist
`Program in Media Arts and Sciences
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`$
`
`
`
`
`
`
`
`
`
`?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Nathaniel I. Durlach
`Senior Research Scientist
`
`
`
`
`
`
`
`
`
`?
`
`
`
`
`
`
`
`
`
`
`
`
`
`<
`
`Thesis Reader:
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`?
`
`
`
`Thomas W. Malone
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`@
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Department of Electrical Engineering and Computer Science
`
`
`
`$
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`e
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Patrick J. McGovern Professor of Information Systems
`Sloan School of Management
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`$
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`?
`
`
`
`
`
`
`
`
`
`
`
`
`
`<
`
`"
`"
`d
`@
`"
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`G
`
`
`
`=
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`"
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MS 1104 - Page 6
`
`MS 1104 - Page 6
`
`

`

`f
`
`MS 1104 - Page 7
`
`MS 1104 - Page 7
`
`

`

`Acknowledgments
`
`g
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`kept me motivated and busy over the years. Thanks for helping me get
`here.
`
`conversational computing field, an area of emerging importance that has
`
`me to think about how people operate and how we can get machines to
`
`together for over a decade. Chris is one of the founding fathers of the
`
`Chris Schmandt and I have been building interactive speech systems
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`communicate with them efficiently and seamlessly.
`
`Nat Durlach has been a friendly and supportive influence, challenging
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Tom Malone has provided much encouragement and insight into the way
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`humans, machines, and interfaces should work in the future.
`
`
`
`
`
`G
`
`
`
`
`
`
`since before this dissertation process even started.
`
`Don Jackson, from Sun Microsystems, provided support and friendship
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`$
`
`
`
`
`
`
`
`$
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Eric Hulteen, from Apple Computer, critiqued many of my early ideas
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and helped shape their final form.
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`!
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`

`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`G
`
`
`
`involved in what could be the best place since the ArcMac.
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`David Liddle and Andrew Singer, from Interval Research, got me
`
`
`
`
`
`
`
`
`
`
`!
`=
`
`
`
`
`G
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`$
`
`
`
`
`
`
`
`
`
`
`Lisa Stifelman provided valuable input in user interface design, assisted
`
`in the design and the many hours of carrying out the usability test, taught
`
`
`
`
`
`
`
`
`
`%
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`

`
`$
`
`!
`
`
`
`
`
`
`
`
`
`@
`
`
`
`
`
`
`
`
`W
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`!
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`$
`
`
`
`
`
`
`
`segmentation.
`
`
`
`
`
`
`
`
`me the inner workings of the Macintosh, and helped edit this document.
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`Meg Withgott lent her help and expertise on pitch, emphasis, and
`
`
`
`
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`F
`
`
`
`
`
`
`
`
`
`
`
`E
`
`
`
`
`
`
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`?
`
`
`
`
`
`#
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`!
`
`
`
`
`
`
`
`someday.
`
`!
`$
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`needed it most) beyond the call of duty. I hope to repay the favor
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Michele Covell and Michael Halle provided technical support (when I
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`