`MIT Media Lab
`E15355
`20 Ames St.
`Cambridge, MA 02139
`+16172535156
`
`Education
`
`M.I.T., Master of Science, Visual Studies (Computer Graphics), 1980
`
`M.I.T., Bachelor of Science, Computer Science, 1978
`
`Professional Experience MIT
`
`Media Laboratory, Principal Research Scientist, 1985Present
`Director, Speech + Mobility Group (formerly known as Speech Research Group)
`
`Architecture Machine Group, Research Associate, 19801984
`
`Architecture Machine Group, Research Assistant, 19791980
`
`Architecture Machine Group, Graphics Programmer, 19771979
`
`Departmental Undergraduate Research Opportunities Program Coordinator, 1984Present
`
`Laboratory Intellectual Property Committee 2001present, chair 20022008
`
`Departmental Committee on Graduate Studies, 19962001, 20052007
`
`Sponsored Research Activities
`
`Things That Think Research Consortium, MIT Media Lab, 2002Present
`
`Alerting and Mobile Messaging, Digital Life Consortium, MIT Media Lab, 1997Present
`
`Acoustical Cues to Discourse Structure, National Science Foundation, Principal Investigator, 19951998
`
`Parsing Radio, News in the Future Consortium, MIT Media Lab, 19932000
`
`Desktop Audio, SUN Microsystems, Inc., Principal Investigator, 19891996
`
`Voice Interaction in a Hand Held Computer, Apple Computer, Principal Investigator, 19911993
`
`Voice Interfaces for Network Services, AT&T, Principal Investigator, 19891991
`
`Back Seat Driver, NEC, Principal Investigator, 19881991
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 1
`
`
`
`Acoustic and Visual Cues for Speech Recognition, DARPA, coPrincipal Investigator, 19861988
`
`Personal Computers and Telephony, NTT Public Corporation, Principal Investigator, 19841989
`
`Home Telecomputing, Atari, Inc., Principal Investigator, 1983
`
`Professional Experience outside M.I.T
`
`Professional Experience Intellectual Property
`
`Coinventor on 6 U.S. Patents
`
`5,177,685 for "Automobile navigation system using real time spoken driving instructions"
`6,728,348 for "System for storing voice recognizable identifiers using a limited input device such as a
`telephone key pad"
`6,937,986 for "Automatic dynamic speech recognition vocabulary based on external sources of
`information"
`7,098,776 for "Methods and apparatus for vibrotactile communication"
`7,392,280 for "Method for summarization of threads in electronic mail"
`7,738,637 "Interactive voice message retrieval"
`7,443,283 for "Methods and apparatus for connecting an intimate group by exchanging awareness cues
`and text, voice instant messages, and twoway voice communications"
`7,865,560 for "System for summarization of threads in electronic mail"
`8,121,653 for "Methods and apparatus for autonomously managing communications using an
`intelligent intermediary"
`8,135,128 for "Animatronic creatures that act as intermediaries between human users and a telephone
`system"
`
`Coinventor on U.S. Patent Applications
`
`20020076009 "International dialing using spoken commands"
`20030023688 "Voicebased message sorting and retrieval method"
`20030081738 "Method and apparatus for improving access to numerical information in voice
`messages"
`20030144846 "Method and system for modifying the behavior of an application based upon the
`application's grammar"
`20030158903 "Calendar bar interface for electronic mail interaction"
`20060239362 "Interactive voice message retrieval"
`20070127704 "Methods and apparatus for autonomously managing communication using an intelligent
`intermediary"
`
`Media Laboratory Intellectual Property Committee 2001 present, chair 20022008
`
`Expert legal testimony and engagements within the last five years, affiliated party in italics:
`
`DDB v. Major League Baseball Advanced Media, expert, 20042006
`Collaboration Properties, Inc. v. Tandberg, expert, 20052007
`Ronald A. Katz v. Premiere Global Services et al., expert, 20072008
`Web Telephony v. AT&T, expert, 2008
`Klausner Technologies v. Vonage, expert, 2008
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 2
`
`
`
`Klausner Technologies v. Comcast, expert, 20082009
`Intellec Wireless v. Research in Motion, expert, 2009
`Klausner Technologies v. Cisco, expert, 2010
`Klausner Technologies v. Avaya, expert, 2010
`Phoenix Solutions, Inc v. West Interactive Corp, declaration, 2010
`Active Video v. Verizon, expert, 2011
`Electronic Frontier Foundation v. Personal Audio LLC, IPR expert, 2013
`Klausner Technologies v. Fonality, expert, 2014
`HTC v. Advanced Audio Devices, IPR expert, 2014
`Silver State Intellectual Technologies v. Foursquare IPR expert, 2014
`CallWave v. Fonality, expert, 2015
`Keker & van Nest, consultant
`Durie Tangrie, consultant
`Ostrow Kaufman, consultant
`Klarquist Sparkman, consultant
`Paul Hastings, consultant
`
`Professional Service
`
`Associate Editor, International Journal on Human Computer Studies
`
`Associate Editor, Journal of Personal and Ubiquitous Computing
`
`Editorial Board, International Journal of Speech Technology
`
`Editorial Board, Interactions (ACM)
`
`Recurring member of program commitees for CHI, UIST, CSCW, Ubicomp, Mobile HCI, and Pervasive
`
`Conference Chair, CSCW 1996
`
`Program Chair, UIST 1997
`
`Publications
`Most publications can be found at http://www.media.mit.edu/speech/sig_papers.html
`or http://living.media.mit.edu/publications.html
`
`Expanding social mobile games beyond the device screen (with Misha Sra) Journal of Personal and
`Ubiquitous Computing, 2015
`
`NailO: Fingernails as an input surface (with Cindy HsinLiu Kao, Artem Dementyev, Joseph Paradiso)
`CHI 2015
`
`MugShots: a Mug Display for Front & Back Stage Social Interaction in the Workplace (with Cindy Hsin
`Liu Kao) Proceedings of TEI (Tangible and Embedded Interaction) 2015.
`
`Mime: compact, low power 3D gesture sensing for interactionwith head mounted displays (with Andrea
`Colaco, Ahmend Kirmani, Hye Soo Yang, NanWei Gong, and Vivek Goyal) Proceedings of UIST 2013.
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 3
`
`
`
`Spotz: A locationbased approach to selfawareness (with Misha Sra) Proceedings of Persuasive 2013.
`
`Setting the stage for interaction: A tablet application to augment group discussion in a seminar class
`(with Drew Harry and Eric Gordon) Proceedings of CSCW 2012.
`
`Indoor Location Sensing using GeoMagnetism (with Jaewoo Chung, Matt Donahoe, IgJae Kim, Pedram
`Razavai and Micaela Wiseman) Proceedings of International Conference on Mobile Systems,
`Applications, and Services (Mobisys) 2011.
`
`My second bike: a TVenabled social and interactive riding experience (with Jaewoo Chung, Kuang Xu,
`Andrea Colaco, and Victor Li) Proceedings of IEEE Communications and Networking Conference, Jan
`2010. Are we there yet? a temporally aware media player (with Matt Adcock and Jaewoo Chung),
`Australian User Interface Conference (AUIC) 2008
`
`Physical embodiments for mobile communication agents (with Stefan Marti), UIST 2005
`
`Giving the caller the finger: collaborative responsibility for cellphone interruptions (with Stefan Marti)
`Extended Abstracts, CHI 2005
`
`Active Messenger: email filtering and delivery in a heterogeneous network (with Stefan Marti) Human
`Computer Interaction Journal (HCI) Volume 20 (2005)
`
`WatchMe: communication and awareness between members of a closelyknit group (with Natalia
`Marmasse) Proceedings of Ubicomp 2004
`
`Improving speech playback using timecompression and speech recognition (with Sunil Vemuri, Philip
`DeCamp, and Walter Bender) Proceedings of CHI 2004
`
`Impromptu: managing networked audio applications for mobile users (with Kwan Lee, Jang Kim, and
`Mark Ackerman), Proceedings of MobiSys 2004
`
`TalkBack: a conversational answering machine (with Vidya Lakshmipathy and Natalia Marmasse)
`Proceedings of UIST 2003
`
```ListenIn'' to domestic environments from remote locations (with Gerardo Vallejo) Proceedings of the
`2003 International Conference on Auditory Display (ICAD)
`
`Safe & Sound: a wireless leash (with Natalia Marmasse) Extended Abstracts, Proceedings of CHI 2003
`
`Mediated voice communication via mobile IP (with Jang Kim, Kwan Lee, Gerardo Vallejo, and Mark
`Ackerman), Proceedings of UIST 2002.
`
`The Audio Notebook: Paper and pen interaction with structured speech (with Lisa Stifelman and Barry
`Arons), Proceedings of CHI 2001
`
`Synthetic News Radio (with Keith Emnett) IBM Systems Journal, Vol. 39 Nos. 34, pp. 646659, 2000.
`
`Everywhere messaging (with Natalia Marmasse, Stefan Marti, Nitin Sawhney, and Sean Wheeler) IBM
`Systems Journal, Vol. 39 Nos. 34, pp. 660677, 2000.
`
`Locationaware information delivery wth comMotion (with Natalia Marmasse), Proceedings of the
`Second International Symposium on Handheld and Ubiquitous Computing, pp. 157171, Springer, 2000.
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 4
`
`
`
`Nomadic Radio: Scalable and contextual notification for wearable audio messaging (with Nitin
`Sawhney), Proceedings of CHI 1999.
`
`Speaking and listening on the run: Design for wearable audio computing (with Nitin Sawhney),
`Proceedings of International Symposium on Wearable Computing, 1998.
`
`Audio Hallway: A virtual acoustic environment for browsing, Proceedings of UIST 1998.
`
`Dynamic Soundscape: Mapping time to space for audio browsing (with Minoru Kobayashi), Proceedings
`of CHI 1997.
`
`CLUES: Dynamic personalized message filtering (with Matt Marx), Proceedings of CSCW 1996.
`
`Using acoustic structure in a handheld audio playback device (with Deb Roy), IBM Systems Journal,
`Vol 35, Nos. 3 and 4, 1996.
`
`Mailcall: Message presentation and navigation in a nonvisual environment (with Matt Marx),
`Proceedings of CHI 1996
`
`AudioStreamer: Exploiting simultaneity for listening (with Atty Mullins), short paper, CHI 1995.
`
`Multimedia momadic services on today's hardware, IEEE Network, September/October 1994.
`
`Putting people first: Specifying proper names in speech interfaces (with Matt Marx), proceedings of
`UIST 1994.
`
`Chatter: A conversational learning speech interface (with E. Ly) AAAI Workshop on Intelligent Multi
`Media MultiModal Systems, 1994.
`
`Voice Communication with Computers: Conversational Systems. New York: Van Nostrand Reinhold.
`1994.
`
`Capturing, structuring, and representing ubiquitous audio (with D. Hindus and C. Horner), ACM
`Transactions on Information Systems, Vol. 11, No. 4, October 1993.
`
`Speech Recognition Architectures for Multimedia Environments, (with E. Ly and B. Arons), Proceedings
`of the 1993 AVIOS Conference, September 1993.
`
`Phoneshell: the Telephone as Computer Terminal, Proceedings of the ACM Multimedia Conference,
`August 1993.
`
`Voicenotes: A Speech Interface for a HandHeld Voice Notetaker (with L. Stifelman, B. Arons, and E.
`Hulteen), Proceedings of INTERCHI'93, April 1993.
`
`From Desktop Audio to Mobile Access: Opportunities for Voice in Computing, book chapter in Advances
`in HumanComputer Interaction, Vol. 4, H.R. Hartson and D. Hix editors. 1992.
`
`Ubiquitous Audio: Capturing Spontaneous Collaboration (with D. Hindus), Proceedings of CSCW'92,
`November 1992.
`
`Integrating Audio and Telephony in a Distributed Workstation Environment (with S. Angebranndt, R.
`Hyde, D. Luong, and N. Siravara), Proceedings of the Summer 1991 USENIX Conference, June 1991.
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 5
`
`
`
`Augmenting a Window System with Speech Input (with M. Ackerman and D. Hindus), Computer, IEEE
`Computer Society, Vol. 23, No. 8, August 1990.
`
`Observations on Using Speech Input for Window Navigation (with D. Hindus, M. Ackerman, and S.
`Manandhar), Proceedings, HumanComputer Interaction, Interact '90, IFIP, August 1990.
`
`Phonetool: Integrating Telephones and Workstations (with S. Casner), Proceedings, GLOBECOM '89,
`IEEE Communications Society, November 1989.
`
`Desktop Audio (with B. Arons), UNIX Review, October 1989.
`
`Synthetic Speech for Real Time DirectionGiving (with J. Davis), IEEE Transactions on Consumer
`Electronics, IEEE, September 1989.
`
`The Back Seat Driver: Real Time Spoken Driving Instructions (with J. Davis), Proceedings, IEEE Vehicle
`Navigation and Information Systems Conference, IEEE, Toronto, Canada, September 1989.
`
`An Audio and Telephone Server for Multimedia Workstations (with M. McKenna), Proceedings, Second
`IEEE Conference on Workstations, IEEE, Palo Alto, CA., 1988.
`
`Employing Voice Back Channels to Facilitate Audio Document Retrieval, Proceedings, ACM Conference
`on Office Information Systems (COIS), Santa Clara, CA, 1988.
`
`Conversational Telecommunications Environments, Proceedings, Second International Conference on
`HumanComputer Interaction, 1987.
`
`Understanding Speech Without Recognizing Words, Proceedings, American Voice Input/Output Society
`Conference, AVIOS, 1987.
`
`A Robust Parser and Dialog Generator for a Conversational Office System (with B. Arons and C.
`Simmons), Proceedings, American Voice Input/Output Society Conference, AVIOS, Palo Alto, CA,
`1987.
`
`Integrated Messages and Network Services for a Personal Workstation, IEEE Workshop on Telematics
`and Message Handling Systems, IEEE, 1986.
`
`Voice Interaction in an Integrated Office and Telecommunications Environment, Proceedings, American
`Voice Input/Output Society Conference, AVIOS, San Francisco, CA, 1985.
`
`Voice Communication with Computers, book chapter in Advances in HumanComputer Interaction, H. R.
`Hartson ed., 1985.
`
`Speech Synthesis Gives Voiced Access to an Electronic Mail System, Speech Technology, Vol. 2, No. 3,
`Aug/Sept 1984.
`
`A Conversational Telephone Messaging System (with B. Arons), IEEE Transactions on Consumer
`Electronics, IEEE, Vol CE30, August 1984.
`
`Phone Slave: A Graphical Telecommunications Interface (with B. Arons), Proceedings, Society for
`Information Display International Symposium, SID, San Francisco, CA, June 1984.
`
`Input/Display Registration in a Stereoscopic Workstation, Displays, April 1984.
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 6
`
`
`
`Remote Access to Voice and Text Messages, Proceedings, American Voice Input/Output Society
`Conference, AVIOS, Washington D.C., 1984.
`
`Fuzzy Fonts: Analog Models Improve Digital Text Quality, Proceedings, National Computer Graphics
`Association Conference, Chicago, IL, 1983.
`
`Greyscale Fonts Designed From Video Signal Analysis, Society of Applied Learning Technology,
`Houston, TX, 1983.
`
`Spatial Input/Display Correspondence in a Stereoscopic Computer Graphic Work Station, Proceedings,
`ACM/SIGGRAPH, Detroit, MI, 1983.
`
`A Programmable Virtual Vocabulary Speech Processing Peripheral (with W. Bender), Proceedings,
`American Voice Input/Output Society Conference on Voice Data Entry Systems Applications, AVIOS,
`1983.
`
`The Intelligent Voice Interactive Interface (with E. A. Hulteen), Proceedings, Human Factors in
`Computer Systems, National Bureau of Standards/ACM, Gaithersburg, MD, 1982.
`
`Interactive ThreeDimensional Computer Space, Proceedings, SPIE Conference on Processing and
`Display of ThreeDimensional Data, SPIE, San Diego, CA, 1982, Vol. 367.
`
`Speech Communications, a Systems' Approach, Proceedings, American Voice Input/Output Society
`Conference on Entry Systems Applications, 1982.
`
`Voice Interaction: Putting Intelligence into the Interface, Proceedings, IEEE International Conference on
`Cybernetics and Society, IEEE, Seattle, WA, 1982.
`
`The Intelligent Ear: A Graphical Interface to Digital Audio, Proceedings, IEEE International Conference
`on Cybernetics and Society, IEEE, Atlanta, GA, 1981.
`
`Soft Typography, Information Processing 1980, IFIPS, S. Lavington ed., NorthHolland Publishing Co.,
`1980.
`
`Petitioner Microsoft Corporation, Ex. 1004, p. 7
`
`