`az United States Patent
`(10) Patent No.:
`Wesemannetal.
`May4, 2004
`(45) Date of Patent:
`
`
`US006731724B2
`
`(54) VOICE-ENABLED USER INTERFACE FOR
`VOICEMAIL SYSTEMS
`
`(73) Assignee: Pumatech, Inc., San Jose, CA (US)
`
`1/1999 Kalveram 0... cee 704/233
`5,864,804 A *
`5/1999 Tatchell et al. 0.0.0... 379/88.01
`5,905,774 A *
`
`.......... 379/88.25
`9/1999 Culbreth et al.
`5,953,393 A *
`atin ‘ . sey000 wel Cb ale sees 379/197
`Inventors: Darren L. Wesemann, North Salt
`(75)
`3
`ZH
`0.
`.
`7
`
`
`rakeOn STUseJo, wasn. Sal 6,044,133 A*3/2000 Furukawa etal. ....... 379/88.01
`
`
`ake City,UT(US); Jon ow 6,088,428 A * 7/2000 Trandal et al. ....c.sse- 379/189
`
`
`West Valley City, UT (US); Tae-Deok
`6,094,476 A *
`7/2000 Hunt et ale cece 379/88.04
`Kweon, Salt Lake City, UT (US);
`6,144,938 A
`11/2000 Suraceetal.
`Dong-Kyun Nam, Murray, UT (US)
`6,167,251 A * 12/2000 Segalet al. woe. 455/405
`6,173,042 Bl *
`1/2001 Wu oo. eee 379/67.1
`6,195,417 B1 *
`2/2001 Dams w..cceeeeccceeceeees 379/352
`6,198,947 B1 *
`3/2001 Barber... 379/88.02
`6,349,132 BL *
`2/2002 Wesemann etal. ...... 379/88.01
`6,389,398 Bl *
`5/2002 Lustgarten etal.
`......... 704/270
`6,501,832 B1 * 12/2002 Saylor et al. 0.0... 379/88.04
`2002/0001370 A1 *
`1/2002 Walker et al... 379/88.04
`
`(*) Notice:
`
`Subject to any disclaimer,the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 43 days.
`
`(21) Appl. No.: 09/887,215
`(22)
`Filed:
`Jun.22, 2001
`
`(65)
`
`Prior Publication Data
`
`OTHER PUBLICATIONS
`ISBN
`H. Newton, Newton’s Telecom Dictionary,
`1-57820-023-7, Mar. 1998, Flatiron Publishing, 14” cd.,
`pp. 72 and 73.*
`
`(60)
`
`Related U.S. Application Data
`Provisional application No. 60/263,469, filed on Jan. 22,
`20s
`Tint. C07 eeeeceeceeeensensneenesneneeneens HO4M 1/64
`(SV)
`(2) US.C
`379/88.16; 379/88.01:
`“— ~379/88.18:704/231: 70470 te 455563
`(58) Field of Search —
`,
`379188 01 88.03
`i
`i
`=
`_
`es ae G8 AE a
`"Se
`GRAS
`SeeRoeae 7=eeeHbat
`ee? neon 546 370
`“—
`.
`References Cited
`
`56
`68)
`
`U.S. PATENT DOCUMENTS
`eee ‘ : lotoos Bula) socmeesomeorenenre oss
`
`S471521 A * 11/1995 Minakamiet al. _. 379/88.01
`5,659,597 A *
`8/1997 Bareis et al.
`cecccssees. 379/88.01
`5,737,393 A
`4/1998 Wolf
`5,822,405 A * 10/1998 Astarabadi
`5,864,605 A
`1/1999 Keshav
`
`........000.0... 379/352
`
`Primary Examiner—Roland G.Foster
`(74) Altorney, Agent, or Firm—Workman Nydegger
`(57)
`ABSTRACT
`.
`.
`.
`A voice-enabled user interface and method for enabling a
`user that is providing audio and vocal input to access data
`from telephone service systems that are only responsive to
`dual tone multi-frequency (““DTMF”) signals. A user access
`the voice-enabled user interface with a telephone device and
`provides inputthat is translated into a DTMFtranslation that
`can be recognized by DTMEFtelephone service system. A
`template of the voice-enabled user interface maps menu
`P
`P
`states, prompts, and acceptable inputs of the DTMFtele-
`phoneservice system. The template also monitors the cur-
`rent state of the telephone service system at all times. The
`invention also enables a user to jump from one menustate
`to another menu state of the telephone service system
`without having to enter input for each menu state between
`the first and the second menuslates.
`
`28 Claims, 5 Drawing Sheets
`
`
`VOICE-ENABLEC
`
`
`USER INTERFACE
`
`160
`
`
`
`USER
`VOICEMAIL
`STATE
`STATE
`MANAGER
`MANAGER
`MODULE
`MODULE
`
`
`
`
`TELEPHONY
`NETWORK
`
`
`
`
`
`
`
`
`
`DATABASE
`
`1
`
`EX1004
`EX1004
`
`
`
`Sheet 1 of 5
`
`US 6,731,724 B2
`
`|Old
`
`ISVaVLVA
`
`U.S. Patent
`
`May4,2004
`
`n
`
`Ob
`
`QSTS¥NI'IOIOA
`
`
`
`JOVIYSLN|ISN
`
`
`
`0S}
`
`TIVWSOIOA
`
`yl¥d
`
`ANOHd3131
`
`WYOMIIN
`
`TIVNJOIOA
`
`I1V1§
`
`YJOVNYAN
`
`J1N00W
`
`Y3Sn
`
`I1VLS
`
`YIOYNVN
`
`J1NdOW
`
`ANOHd3131
`
`YYOMIIN
`
`2
`
`09}
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`May4, 2004
`
`Sheet 2 of 5
`
`US 6,731,724 B2
`
`USER STATE
`MANAGER MODULE
`160
`
`a
`TRANSLATOR
`
`930
`
`i eee
`
`
`
`
`
`
`
`VOICEMAIL STATE
`
`eeeNODULE STATE
`
`MANAGER
`
` SOUND
`ANALYZER 180
`
`3
`
`
`
`U.S. Patent
`
`May4, 2004
`
`Sheet 3 of 5
`
`US 6,731,724 B2
`
`USER
`
`410
`
`ACCESS
`voice-ENABLED
`USER INTERFACE
`
`ENTER
`
`it
`
`VOICE-ENABLED
`USER INTERFACE
`
`PRONPT USER
`FOR ACCOUNT
`INFORMATION
`
`
`
`
`
`
`
`AUTHENTICATE
`ACCOUNT
`
`INFORMATION
`
`
`
`ACCESS
`TELEPHONE
`SERVICE SYSTEM
`
`TELEPHONE SERVICE
`SYSTEM
`
`PROMPT FOR
`ACCOUNT
`
`- INFORMATION
`
`4480
`
`470
`
`
`
`
`
`I$ ACCOUNT
`INFORMATION STORED
`
`
`
`TRANSFER
`PROMPT TO
`
`USER
`
`
`
`GENERATE
`DTMF
`TRANSLATION
`OF ACCOUNT INFORMATION
`
` TRANSMIT
`
`
`AUTHENTICATE|4gp
`DTMF ACCOUNT
`
`INFORMATION
`USER ACCOUNT
`
`TO TELEPHONE
`INFORMATION
`SERVICE SYSTEM
`490
`
`478~J
`
`ENTER
`ACCOUNT
`INFORMATION
`
`RETRIEVE
`ACCOUNT
`INFORMATION
`
`419%
`
`I$ IT
`A OTHE
`SIGNAL ?
`
`
`
`
`
`PROVIDE
`WENU
`PROMPT
`
`44
`
`WAIT FOR|496
`USER INPUT
`
`FIG. 4
`
`4
`
`
`
`U.S. Patent
`
`May4, 2004
`
`Sheet 4 of 5
`
`US 6,731,724 B2
`
`W1WOLINSNYYL¥i¥0$INOILYTSNWYLSWLO||3IVLS3HLSONVHOOLMANYO4
`
`
`
`
`
`
`H3SNJHLOLaaisgn03yJNOHd3ISLJHLOL}|3NOHdIIILIHL40INAN!
`
`
`
`
`
`WALSASJOIANISSNOHAIIILJOVINILNIYISNGIIGYNS3OIOAyasn
`
`1g3nd3y=reeIHLLINSNYYL(1)
`;WIISASJDIAMISWALSAS3OIAYISbes
`
`
`
`
`
`
`HLLINSNVYLLMdLNO3LVY3N39Y3SNldWONd
`S3ONVHO019
`
`
`3AI30340IAONd
`
`INdNIInd
`
`4W1G31¥Y3N39Y3d0¥dLNANISI
`
`ossoss
`
`
`YOJLdWOud
`
`LNdLNO3A1503Y
`
`
`
`LAdNI¥3S/
`
`
`
`I1VL$SONVH)
`
`IWSWOVEL
`
`SJONVHO
`
`§“Old
`
`
`
`ritinalno
`
`
`
`WIISASJOIAYIS
`
`9¢¢
`
`
`INdNi
`
`¥3SNSHLiJONOILVISNWYLIVSANYYOd
`
`5
`
`
`
`J1VLSWOWYL
`
`
`
`Y3d04dLNENIS|
`
`
`
` éF1VLSLNINYNDYOs
`
`
`
`
`
`
`
`
`
`
`
`
`May4, 2004
`
`U.S. Patent
`
`009
`
`019ANINIV
`
`
`079AYOLOIYIG0&9029YilndW0d
`TINNOSYIdJOSaysLuOddNs
`
`019099059$99979
`
`
`
`
`
`
`
`NOISNSLX3TVACIAIONISSINISNGJNOKQJHSIGUNIIeSS3INISNGWOH
`
`
`
`
`
`Sheet 5 of 5
`
`US 6,731,724 B2
`
`
`
`
`
`p79Q3HSI9YNIIYQ3HSi8uns3yJNOH
`
`p19619p59059
`
`
`
`
`
`dQ1¥S30d0ld¥}dOLHS30
`
`
`
`
`
`$31¥SS31¥SSI1¥S
`
`799099
`
`
`
`SSINISNESS3NISNG
`
`
`
`dOLYS30d0ld¥1
`
`
`
`S31¥$S31V$
`
`9‘Old
`
`nsSUISWnN
`
`
`Y3LNdW0dYILAdWOd
`
`Y3LNdW0d
`
`$31¥$$31¥S
`
`
`$31¥SLUOddNls
`
`Y3LNdWOd
`
`YaldWOd
`
`LUOddNs
`
`6
`
`
`
`
`
`
`
`
`US 6,731,724 B2
`
`1
`VOICE-ENABLED USER INTERFACE FOR
`VOICEMAIL SYSTEMS
`
`RELATED APPLICATIONS
`
`This application claims the benefit of U.S. Provisional
`Patent Application Ser. No. 60/263,469, entitled “Universal
`Voicemail Gateway (UVG),”filed Jan. 22, 2001, which is
`incorporated herein by reference.
`BACKGROUND OF THE INVENTION
`
`1. The Field of the Invention
`
`The present invention relates to a voice-enabled user
`interface for accessing a telephone service system that
`respondsto dial tones. More particularly, the present inven-
`tion relates to methods and systems for enabling a user, with
`voice commands, to access and navigate a telephone service
`system that only recognizes dual tone multi-frequency sig-
`nals.
`2. The Prior State of the Art
`
`Through the years, it has become commonplace to use a
`telephone, not only to communicate, but also to perform
`electronic business and financial transactions, and to obtain
`prerecorded information.
`It
`is possible to perform these
`activities over a telephone, in part, because of the develop-
`mentof corresponding telephone service systems, which are
`used in virtually every industry and business environment.
`Voicemail is perhaps the most commontype of telephone
`service system. Voicemail,
`like an answering machine,
`enables an incoming telephone message to be recorded for
`review by an intended recipient at a later time. However,
`unlike an answering machine, voicemail can record an
`incoming message even when a corresponding telephoneis
`“off the hook.” Accordingly, voicemail is particularly useful
`when a person cannot answer an incoming telephone call
`because that person is already engaged in another telephone
`call.
`
`The prior art also includes various other types of tele-
`phone service systems that facilitate business and financial
`transactions and enable users to obtain prerecorded infor-
`mation. Essentially, a telephone service system includes any
`automated system that provides a user with data over a
`telephone device. Virtually every industry uses a telephone
`service system of one type or another, even if the telephone
`service system consists of only a customized telephone
`directory of available business services or personnel.
`Some of the more useful telephone service systems enable
`users using a telephone device to interact with a business
`networkto transact variousactivities, such as for performing
`home banking, purchasing consumer products, receiving
`customer support, and accessing news, entertainment,
`financial, and travel information. Some telephone service
`systems also enable a user to access information that is
`generated on-the-fly, such as when a telephone service
`system generates text-to-speech translations of dynamic
`information.
`
`Telephone service systems are particularly useful from a
`user’s perspective because they enable a user to access
`information and to make business and financial transactions
`when and where it is most convenient for the user. Telephone
`service systems are useful from a business’s perspective
`because they enable businesses to hire fewer employees to
`answer incoming calls and to perform other tasks that can be
`automated by telephone service systems.
`One problem with certain telephone service systems,
`however,
`is that
`they only recognize dual
`tone multi-
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`7
`
`2
`frequency signals (““DTMF”) signals. A DTMFtelephone
`service system is generally problematic because a user can
`only use the system by manually pressing the keys on a
`DTMFtelephone keypad. This is a problem becauseit is
`easy for a user to inadvertently press an incorrect key,
`sometimes without realizing it. Pressing an incorrect key
`may result in an undesirable consequence. For example, by
`pressing an incorrect key a user may cause the telephone
`service system to transition from one menu state to an
`undesired menu state. This may also require a user to return
`to the prior menu state before a desired selection can be
`made. This is not only inconvenient,it is a waste of time. By
`pressing incorrect keys, a user may also cause a telephone
`service system to perform an act that is not desired. For
`example, by pressing an incorrect key, a telephone service
`system may terminate a phone call, call a wrong extension
`number,or in the case of home banking,transfer an incorrect
`amount of money between accounts or to a wrong account.
`It should be appreciated that these examples are given by
`wayof illustration and not limitation.
`To overcome the problemsthat are associated with inad-
`vertently pressing a wrong key, many telephone service
`systems provide prompts that request a user to confirm
`certain entries before they are processed. This, however,
`does not resolve the underlying problem associated with
`DTMFtelephone service systems, namely, requiring a user
`to enter input by pressing keys on a telephone keypad.
`Another problem associated with DTMEFtelephone ser-
`vice systems is that a user is required to move a handheld
`telephone back and forth between a position that enables the
`userto listen to the voicemail system and to a position that
`allows the user to press keys on the telephone keypad.
`Having to do this is inconvenient. It can also be dangerous,
`particularly for a person whois driving a car because it may
`require that person to take his eyes off the road and his hands
`off the steering wheel.
`To overcome the problems associated with DTMFtele-
`phoneservice systems, the industry has developed telephone
`service systems that recognize speech so that a user can
`speak commands and other input without having to press
`keys on a telephone keypad. There are various methods for
`performing speech recognition, which are well knownin the
`art. Some examples include spectral analysis, dynamic time
`warping, neural networks, and recognition by discrete and
`continuous hidden Markov modeling.
`Although speech recognition telephone service systems
`provide a significant advantage over DTMFtelephoneser-
`vice systems,
`they do not resolve all of the associated
`problems with the prior art telephone service systems. In
`particular, a user must still expend the lime thal is necessary
`to move systematically through a hierarchy of levels or
`menustates that are associated with the menu structure of a
`
`telephone service system, even when a user already knows
`what the final menu state will be. For example, from a main
`menu a user may be required to transition through several
`“in between” menu states before finally reaching a final
`menu state that contains the information that is desired by
`the user. Many telephone service systems also require a user
`to return to the main menu before navigating to a second
`desired menu state. Navigation through a menu hierarchy is
`particularly time consuming whenthe hierarchy is complex,
`or has many menustates. This type of navigation through a
`telephone service system can also be expensive, particularly
`whenaccess to the telephone service system requires a user
`to incur costs associated with long-distance and/or a wireless
`telephone service plan.
`Another problem with speech recognition telephone ser-
`vice systems is that because of variations in speech it is
`
`
`
`US 6,731,724 B2
`
`3
`impossible to guarantee that speech recognition software
`will recognize user input. It is also expensive for businesses
`to modify existing DTMF telephone service systems to
`recognize and process audio and vocal data with speech
`recognition software.
`Accordingly, it would be beneficial to provide a method
`for enabling a user to use an existing DTMF telephone
`service system by speaking into a telephone device without
`requiring modification to the DTMFtelephone service sys-
`tem. It would also be beneficial to accomplish this while
`enabling a user to jump to any level or state in a menu
`hierarchy of a telephone service system without requiring
`the user to systematically navigate through numerous “in
`between” menustates.
`
`SUMMARYOF THE INVENTION
`
`The present invention is directed towards a voice-enabled
`user interface for enabling a user to speak over a telephone
`device to access data from a telephone service system thatis
`only responsive to dual tone multi-frequency (“DTMF”)
`signals.
`In one embodiment, a user accesses the voice-enabled
`user interface with a telephone device and enters authenti-
`cation information. The voice-enabled user interface verifies
`
`the identity of the user and accesses a telephone service
`system that has information that is desired by the user, but
`which is only responsive to DTMF signals. The user
`responds to prompts generated by the telephone service
`system by speaking into the telephone device. The voice-
`enabled user interface translates the user’s input into a
`DTMFtranslation that is recognized by the telephone ser-
`vice system. The voice-enabled user interface maps the
`hierarchy of menu states and corresponding prompts ofthe
`telephone service system within a template. During use, the
`voice-enabled user interface uses the template to determine
`what menustate the telephone scrvicc system is in at all
`times. Auser can jump from one menustate to another menu
`state by providing input that the voice-enabled user interface
`associates with a corresponding menu state. The voice-
`enabled user interface generates output
`that causes the
`telephone service system to transition to the menu state that
`corresponds with the user input. Once the telephone service
`system is in the appropriate menu state, the voice-enabled
`user interface transmits a DTMFtranslation of the user input
`to the telephone service system so that it can be processed.
`One advantage of the present invention is that it enables
`users using voice commands to access data from existing
`DTMF telephone service systems without requiring any
`modification to the existing systems. The present invention
`also enables users to navigate through a hierarchy of menu
`states of a telephone service system without requiring the
`users to enter input to transition through “in between” menu
`states. Instead, the invention enables a user to jump over “in
`between” menustates, from a first menu state to a second
`menu state with only a single user input.
`These and other objects, features, and advantages of the
`present invention will become more fully apparent from the
`following description and appended claims, or may be
`learned by practicing the invention as set forth below.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`A more extensive description of the present invention,
`including the above-recited features and advantages will be
`rendered with reference to the specific embodimentsthat are
`illustrated in the appended drawings. Because these draw-
`ings depict only exemplary embodiments,
`the drawings
`
`10
`
`15
`
`*
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`
`4
`should not be construed as imposing any limitation on the
`present invention’s scope. As such, the present invention
`will be described and explained with additional specificity
`and detail through use of the accompanying drawings in
`which:
`
`FIG. 1 is a block diagram showing one suitable network
`environment for practicing the invention that includes a
`telephone device,
`a voice-enabled user interface, and a
`telephone service system;
`FIG. 2 is a block diagram showing increased detail of the
`components that make up the user state manager module of
`the voice-enabled user interface of the present invention;
`FIG. 3 is a block diagram showing increased detail of the
`components that make up the voicemail state manager
`module of the voice-enabled user interface of the present
`invention;
`FIG. 4 is a flow chart illustrating an embodiment for
`enabling a user to access a telephoneservice system with the
`voice-enabled user interface of the invention;
`FIG. 5 is a flow chart illustrating an embodiment of the
`invention for enabling a user to access data from a telephone
`service system with the voice-enabled user interface of the
`invention;
`VIG. 6 is a block diagram illustrating one embodiment of
`a hierarchal menu structure of a telephone service system
`that can be accessed and navigated using the voice-enabled
`user interface of the invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`The present invention is directed to a voice-enabled user
`interface and methods for enabling users to access and use
`an existing telephone service system that is only responsive
`to dial tone signals. More particularly, the present invention
`enables users to speak into a telephone device to access data
`from an existing telephone service system that
`is only
`responsive to dual tone multi-frequency (““DTMF’”)signals.
`The term telephone service system, as used herein, should
`generally be construed to include any telephone service that
`is responsive to user input entered over a telephone device
`and is capable of providing data to a uscr. One type of
`telephone service system that can be accessed by using the
`present invention includes a voicemail system.
`Users can perform various functions with a voicemail
`system. These functions include, but are not
`limited to,
`listening to messagesthat are left by a third party, recording
`messages or greetings for a third party to listen to, forward-
`ing messages, deleting messages, and configuring voicemail
`answering and recording procedures. Users perform func-
`tions with a voicemail system by entering input
`into a
`telephone device that can be recognized and processed by
`the voicemail system. DTMF voicemail systems are one
`type of voiccmail systemsthat are only responsive to DTMF
`signals. DTMF voicemail systems cannot recognize audio or
`vocal input.
`In one embodimentof the invention, a voice-enabled user
`interface enables users to perform functions with a DTMF
`voicemail system by only speaking into a telephone device.
`It should be appreciated, however,
`that although certain
`embodiments of the present
`invention are described as
`applying to DTMEFvoicemail systems, the invention is not
`limited to any particular type of telephone service system.
`Similarly, a user can access and use the voice-enabled user
`interface of the invention with any type of telephone device,
`including, but not limited to wire boundtelephones, wireless
`
`
`
`US 6,731,724 B2
`
`5
`telephones, and
`telephones, cellular telephones, digital
`PDA’s and computers having telephony capabilities.
`Embodiments of the invention, as described herein, may
`comprise a special purpose or general-purpose computer
`comprising various computer hardware. Embodiments may
`also include computer-readable media having computer-
`executable instructions or data structures stored thereon.
`Such computer-readable media can be any available media
`that can be accessed by a gencral-purposce or special-purpose
`computer. By way of example, and not limitation, such
`computer-readable media can comprise RAM, ROM,
`EEPROM, CD-ROMorother optical disk storage, magnetic
`disk storage or other magnetic storage devices, or any other
`medium which can be used to store the desired executable
`instructions or data structures and which can be accessed by
`a general-purpose or special-purpose computer.
`When information is transferred or provided over a net-
`work or other communications connection to a computer, the
`computer properly views the connection as a computer-
`readable medium. Thus, such a connection is also properly *
`termed a computer-readable medium. Combinations of the
`above should also be included within the scope of computer-
`readable media. Computer-executable instructions
`comprise, for example, instructions and data which cause a
`general-purpose computer, special-purpose computer, or
`special-purpose processing device to perform a certain func-
`tion or group of functions. The computer-executable instruc-
`tions and associated data structures represent an example of
`program code meansfor executing the steps of the invention
`disclosed herein.
`
`10
`
`15
`
`25
`
`30
`
`The invention will be described in the general context of
`computer-executable instructions, such as program modules,
`being executed by a computer. Generally, program modules
`include routines, programs, objects, components, data
`structures, or the like that perform particular tasks or imple-
`mentparticular abstract data types. Morcover, those skilled
`in the art will appreciate that the invention may be practiced
`with other computer system configurations, including hand-
`held devices, multi-processor systems, microprocessor-
`based or programmable consumerelectronics, network PCs,
`minicomputers, mainframe computers, and the like. The
`invention may also be practiced in distributed computing
`environments where tasks are performed by remote process-
`ing devices that are linked through a communications net-
`work. In a distributed computing environment, program
`modules may be located in both local and remole memory
`storage devices.
`Turning now to T'IG. 1, one suitable environment 100 for
`practicing the method of the invention for enabling a user to
`access data from a telephoneservice system isillustrated. As
`shown, a voice-enabled user interface 110 is in communi-
`cation with a telephone device 120 and a telephone service
`system through telephony networks 130. The invention may
`be practiced in various telephony network environments,
`including, but not limited to Public Switched Telephone
`Networks (PSTN),
`Integrated Services Digital Network
`(ISDN) and Fiber Distributed Digital Interface (FDDI). In
`the present embodiment, the telephone service system com-
`prises voicemail system 140 which can be accessed by a user
`using telephone device 120.
`Voicemail system 140 comprises voicemail data 150 that
`can include any type of data that can be transmitted to a user
`over a telephone devicc.
`In one preferred embodiment,
`voicemail data 150 comprises a prerecorded message, such
`as, for example, a menu prompt, a telephone phone message,
`a greeting, etc. Voicemail data 150 may also include data
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`9
`
`6
`that is dynamicorthat is accessed from another network. For
`example, in one embodiment, voicemail system 140 com-
`prises a telephone service system of a financial institution
`and voicemail data 150 includes dynamic stock quote data
`that is obtained by the voicemail system 140 from another
`network. Voicemail data 150 may also include data that is
`generated on-the-fly. For example, stock quotes can be
`provided to voicemail system 140 in the form of text, such
`that during use voicemail system 140 generates a text-to-
`speech translation of the stock quotes on-the-fly.
`According to the present embodiment, voicemail system
`140 is responsive only to DTMFsignals that are generated
`by pressing the keys on a keypad of a DTMFtelephone
`device, such as telephone device 120. DTMF telephone
`signals are well knownin the art of telephone communica-
`tions. According to this embodiment, voicemail system 140
`cannot recognize audio or vocal input that is generated by a
`user speaking into telephone device 120. However, voice-
`enabled user interface 110 of the invention, enables a user to
`access voicemail data 150 from the voicemail system 140 by
`speaking into a telephone device 120, as will be described
`herein.
`Voice-enabled user interface 110 contains various com-
`ponents that enable a user to access voicemail data 150 from
`voicemail system 140. These components are generally
`described in reference to a user state manager module 160,
`a voicemail state manager module 170, and a database 180.
`FIG. 2 shows someof the basic components that make up
`user state manager module 160. User authentication 210
`verifies the identity of a user accessing the voice-enabled
`user interface 110 by comparing the user account informa-
`tion to account information 220 that is stored in database
`180. The act of verifying the identity of the user may require
`the user to enter a password, personal identification number,
`an account number, or any other information that can be
`verified by user authentication 210.
`According to one embodiment, and upon authenticating
`the identity of the user, user authentication 210 connects the
`voice-enabled user interface to a telephone service system
`that contains data that
`is desired by the user, such as
`voicemail system 140. According to this embodiment, user
`account information 220 includes information that is used to
`
`connect to and access voicemail system 140. For example,
`user account information 220 can include a telephone num-
`ber for calling voicemail system 140, and account informa-
`tion and a password for accessing voicemail system 140.
`According to another embodiment, user account informa-
`tion 220 includes a telephone numberfor calling voicemail
`system 140, but does not include account information or a
`passwordthat is necessary to access voicemail system 140.
`According to this embodiment, the user must provide the
`account information and password during use, after user
`authentication 210 has connected the voice-enabled user
`interface to the voicemail system 140.
`Speech recognition 230 interprets audio or voice data that
`is received from a telephone device, which is generated by
`a user speaking into the telephone device. Speech recogni-
`tion 230 can also interpret user input comprising telephone
`signals, such as DTMEFsignals. Techniques for implement-
`ing speech recognition 230 are well knownin the respective
`art, and include, but are not limited to spectral analysis,
`dynamic time warping, neural networks, and recognition by
`discrete and continuous hidden Markov modeling.
`Translator 240 receives user input comprising audio or
`voice data andtranslates the data into DTMEsignals that can
`be recognized by voicemail system 140. According to the
`
`
`
`US 6,731,724 B2
`
`7
`present embodiment, voicemail system 140 does not recog-
`nize audio or voice data and is only responsive to DTMF
`signals. The accuracy of making a translation of user input
`into a DTMEFsignal is enhanced by comparing user input
`with anticipated user input according to data thatis stored in
`template 232 of database 180. It should be noted that the
`translation of user input that is generated by translator 240
`is not a direct translation, but instead is a translation of the
`data stored in template 232 that most resembles the user
`input. Because template 232 has a finite number of accept-
`able inputs and responses,translator 240 is generally able to
`make an accurate translation of user input, which can vary
`from one user to another, by comparing the user input to the
`known data in template 232.
`A template, such as template 232, maps all of the menu
`states of a telephone menu system andall of the correspond-
`ing prompts and acceptable responses and inputs. A techni-
`cian prepares only a single template for each telephone
`service system that is accessible to users using the voice-
`enabled user interface of the invention. Accordingly, asingle °
`template is used for making translations of user input from
`multiple users accessing the same telephone service system.
`Communication switch 250 controls the transmission of
`
`10
`
`15
`
`data through the voicc-cnabled user interface that is trans-
`mitted between a telephone device 120 and voicemail sys-
`tem 140. In one embodiment, two simplex channels 252, 254
`connect the voice-enabled user interface to the telephone
`device 120 and two simplex channels 256, 258 connect the
`voice-enabled user interface to the voicemail system 140,
`each set of simplex channels comprising an inbound and an
`outbound simplex channel. A simplex channel is simply a
`one-way communication channel. It should be appreciated
`that other types of communication channels can also be
`used, such as a half-duplex channel.
`According to this embodiment, communication switch
`250 synchronizes all communication over the simplex chan-
`nels 252-258. During use, the communication switch 250
`connects the inbound simplex channel 256, from the voice-
`mail system 140, to the outbound simplex channel 252 that
`is directed to the telephone device 120. This enables the user
`to hear transmissions of data directly from the voicemail
`system 140. The communication switch 250 also directs
`transmissions of data from inbound simplex channel 254 to
`speech recognition 230, while directing all outboundtrans-
`missions of the voice-enabled user interface to voicemail
`
`system 140 through outbound simplex channel 258.
`The communication switch 250 also enables the voice-
`
`enabled user interface to interrupt the voicemail system 140,
`whenever appropriate, by disconnecting the inbound sim-
`plex channel 256 from the outbound simplex channel 252.
`One appropriate circumstance to interrupt the voicemail
`system 140 is to prevent the user from hearing promptsthat
`the user does not need to respond to. For example, after user
`authentication 210 connects the voice-enabled userinterface
`
`the voicemail system 140 may
`to voicemail system 140,
`transmit prompts for the user to enter a valid account number
`and password.
`the voice-enabled user interface
`In one embodiment,
`disconnects the inbound simplex channel 256 from the
`outbound simplex channel 252 while user authentication
`210 transmits the user’s account number and password to
`voicemail system 140 from database 180. According to this
`example,
`the communication switch 250 prevents the
`incoming transmission of data from the voicemail system
`140 from being transmitted over the outbound simplex
`channel 252 to the user.
`
`30
`
`35
`
`45
`
`50
`
`55
`
`60
`
`65
`
`10
`10
`
`8
`Another appropriate circumstance for interrupting the
`voicemail system 140 is when the voice-enabled userinter-
`face is causing the voicemail system to jump over “in
`between” menustates, from a menustate to final menustate.
`This will be described later in reference to FIG. 6.
`
`State manager 260 executes user requests and performs
`various general control functions. State manager 260 also
`coordinates the connectivity and operation of other
`components, suchas, for example enabling components 210,
`230, 240, and 250 to access database 180, and coordinating
`the transmission of data to and from components 210, 230,
`240, and 250 to simplex channels 252-258.
`FIG. 3 shows someof the basic components that make up
`voicemail state manager module 170, of FIG. 1. In one
`preferred embodiment, voicemail authentication 310 con-
`nects the voice-enabled user interface to voicemail system
`140 of FIGS. 1-2. According to this embodiment, voicemail
`authentication 310 retrieves account information 220 from
`database 180 and accesses voicemail system 140. Account
`information 220 can include a telephone numberforcalling
`voicemail system 140, and account voicemail information
`and password for accessing voicemail system 140.
`According to another embodiment, necessary account
`information 220 includes a telephone number for calling
`voicemail system 140, but does not include account infor-
`mation or a password for accessing voicemail system 140.
`According to this embodiment, a user must provide the
`account information and password during use, after voice-
`mail authentication 310 has accessed the voicemail system
`140.
`
`Keepalive 330 keeps a telephone service system in synch
`with the voice-enab