US 6,731,724 B2
`(0) Patent No.:
`«2 United States Patent
`Wesemann etal.
`(45) Date of Patent:
`May4, 2004
`
`
`US
`
`(00673172482
`
`3.8.80 A ©
`1/1990 Kalveram veecccsicccare TOM233
`5,905,774 A *
`379/88.01
`5/1999 Tatchell et al.
`
`5,953,393 A =
`STH /88. 25
`9/1999 Culbreth et al.
`
`cssstitceeee STONLYT
`ees “ . or oo CL AL,
`i
`a
`fel
`‘0
`6,044,133 A *
`3/2000 Furukawa et al.
`..,,... 379/98.01
`
`7/2000 Trandal et al.
`.
`ve ST9/189
`6.088.428 A *
` 379/88.04
`6.004.476 A *
`7/2000 Hunt et al,
`11/2000 Surace et al,
`6,144,938 A
`
`6,167,251 A * 12/2000 Sepal ef ab. occu. 455/405
`« ST96T.1
`6,173,042 Bl *
`1/2001 Wu .....
`
`
`we 379/352
`6195417 Bl
`*
`2/2001 Dans
`379/88.02
`6,198,947 Bl
`*
`3/2001 Barber
`
`
`
`2/2002 Wesemann et al,
`379/88.01
`6,349,132 Bl
`*
`
`se. 704/270
`5/2002 Lusigarten chal.
`6.359.598 Bl *
`
`379/88.04
`6,501,832 Bl * 12/2002 Saylor etal...
`
`« 374/88.04
`202AWOLST AT 1/2002 Walker et al,
`
`OTHER PUBLICATIONS
`ISBN
`H. Newton, Newton’s Telecom Dictionary,
`1-S7820-023-7, Mar. 1998, Flatiron Publishing, 14ed.,
`pp. 72 and 73.*
`* ciled by examiner
`
`Primary Examiner—Roland G, Foster
`(74) Avorney, Agent, or Firm—Workman Nydegger
`(57)
`ABSTRACT
`.
`;
`;
`re
`A voice-enabled user interface and method for enabling a
`user that is providing audigy and vocal input to access data
`from telephoneservice systems that are only responsive to
`dual tone multi-frequency (“DTMF") signals. A useraccess
`the voice-enabled user interface with a telephone device and
`provides input that is translated into a D'T'MFtranslation that
`can be recognized by DTMF telephone service system. A
`template of
`the voice-enabled user interface maps menu
`stales, prompts, and acceptable inputs of the DTMEFtele-
`phone service system. The template also monitors the cur-
`rent Slate of the telephone service system al all times. The
`invention also enables a user to jump fram one menu state
`to another menu state of the telephone service system
`without having to enter input for each menu state between
`the first and the second menu states.
`
`28 Claims, 5 Drawing Sheets
`
`(54) VOICE-ENABLED USER INTERFACE FOR
`VOICEMAIL SYSTEMS
`
`(75)
`
`Inventors: Darren L. Wesemann, North Salt
`ake
`T
`.
`<
`babeitesUTSton eel
`S
`ecg
`eat
`:
`a 5
`WestValley City, UT (US); Bae-Deok
`Kweon, Salt Lake City, UT (US):
`Dong-Kyun Nam, Murray, UT (US)
`
`(73) Assignee: Pumatech, Inc., San Jose, CA (US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 43 days.
`.
`
`(21) Appl. No.: 09/887,215
`(22) Filed:
`Jun. 22, 2001
`
`(65)
`
`Prior Publication Data
`US 2002/7848 Al Jal. 25, 2002
`
`Related U.S. Application Data
`Provisional application No, 60/263,469, filed on Jan, 22,
`ys
`(51)
`Int. CL...
`esses HOM 1/64
`
`(52) US. Ch ..
`_. 379/88.16; 379/88.01:
`es
`- "379/88.18: 704/231: F04/270.1: 455/563
`(58) Field of Search
`‘
`va
`379/88 i 58.03
`.
`379/98 04“98 09.88.1 388.25. 03.01 0326-93.28.
`518.01455/79. 563: s04,270) 17 “331.
`Fee eee peel a46 70
`ba
`alg
`References Cited
`ee
`U.S. PATENT DOCUMENTS
`
`abies
`;
`-
`!
`
`eee ; - Apect is BONE!ciel anieet
`471591 A * Lwos Minakami a),af : 579/88.011
`5,659,507 A *
`§/1997 Bareis ef al. cco. 379/88.01
`5,737,393 A
`4/1998 Wolf
`5822405 A * LO/L99R Astarabadi 0.0.2.0... 374/352
`5,804,605 A
`1/1999 Keshav
`
`%a
`
`(60)
`
`56
`(36)
`
`USER INTERFACE TELEPHOWY
`
`VOICE-ENABLED
`
`TELEPHONY
`HETWORK
`
`NETWORK,
`ee,
`
`PETITIONERS - EXHIBIT 1004
`PETITIONERS- EXHIBIT 1004
`
`IPR2022-00217
`
`IPR2022-00217
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 1 of 5
`
`US 6,731,724 B2
`
`0S}
`
`vLVd
`
`b‘Old
`
`3svavlvd
`
`TIVNSDIOA
`
`J1NGOW
`
`
` J1NGON
`
`ANOHd313L
`
`AYOMLIN
`
`IlVLS
`
`YJOVNYA
`
`ANQHd313L
`
`YYOMLIN
`
`Q378VNI-JOIOA
`
`
`
`JOVINILNIY3SN
`
`Ob
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 2 of5
`
`US 6,731,724 B2
`
`USER STATE
`
`
`
`24 nt WanewoDUL
`
` 2 50
`
` VOICEMAIL
`SYSTEM
`
`
`
`
`260
`
`
`
`
`STATE
`ANAGER
`
`M
`
`COMMUNICATION
`SWITCH
`
`180
`
`
`
`
`VOICEMAIL
`AUTHENTICATION
`
`
`
`maBAOE,NOEL
`
`
`STATE
`
`
`VOICEMAIL STATE
`
`MANAGER SOUND
`ANALYZER 180
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 3 of5
`
`US 6,731,724 B2
`
`USER
`
`410~J
`
`ACCESS
`VoICE-ENABLED
`USER INTERFACE
`
`ENTER
`
`tO7 ge)
`
`
`
`VOICE-ENABLED
`USER INTERFACE
`
`
`PROMPT USER
`FOR ACCOUNT
`4-420
`INFORMATION
`
`
`
`AUTHENTICATE
`
`ACCOUNT
`INFORMATION
`
`
`
`
` ACCESS
`SERVICE SYSTEM
`TELEPHONE
`
`TELEPHONE SERVICE
`SYSTEM
`
`PROMPT FOR
`ACCOUNT
`
`ik INFORMATION
`
`4-460
`
`
`
`
`
`
`
`
` TRANSMIT
`AUTHENTICATE|ago
`DTMF ACCOUNT
`
`
`INFORMATION
`USER ACCOUNT
`
`TO TELEPHONE
`INFORMATION
`SERVICE SYSTEM
`
`
`490
`
`470
`
`[$ ACCOUNT
`INFORHATION STORED
`
`
`TRANSFER
`
`
`PROMPT TO
`
`USER
`
`
`RETRIEVE
`ACCOUNT
`INFORMATION
`
`GENERATE
`OTMF
`TRANSLATION
`OF ACCOUNT
`INFORMATION
`
`478
`
`ENTER
`ACCOUNT
`INFORMATION
`
`I$ IT
`419% DTME
`SIGNAL ?
`
`PROVIDE
`WENU
`PROMPT
`
`44
`
`FIG, 4
`
`WAIT FOR
`USER INPUT
`
`496
`
`

`

`U.S. Patent
`
`May 4, 2004
`
`Sheet 4 of 5
`
`US 6,731,724 B2
`
`404LdWO¥d
`
`
`
`LndNi¥asn
`
`ogOss
`
`¢Old
`
`
`
`
`
`Alv1$JONVHOSJONVHO
`
`
`
`LMdLNO3N1393YALVLSWOVUL
`
`
`
`
`
`vivaLIWSNYYLvi¥0SiNOLLWISNWHL4WLO||3L¥LS3HLJONVHOOLKANHO4
`
`
`
`
`¥3SNIHLOLnaigsnoaINOHdITALJHLOL}|3NOHGITIL3HL40INdNl
`
`
`
`
`WALSASJOIAYISINOHdIISLJOVIYILN|YISNCIVIWNSIOIOA
`WHO4N4dSHLINSNVHL
`
`
`
`;WAISAS30INHISWALSAS3OINN3S
`
`
`
`
`JHLLINSNYYLLNdLNO31¥¥3N39y3Sf)LdWOUd
`
`
`
`
`
`L$3N034
`
`3H.OLLNdino
`
`JNOHd313L
`
`
`
`WILSAS39/ANIS
`
`
`
`JWW1SYOVUL
`
`SJONVHO
`
`9AI3038
`
`LNdNl
`
`
`
`Y3d0dLANISI
`
`
`
` abJIVLSLNIYYNDYOd
`
`JWG31VY3N39
`
`JONOIVISNVYL
`
`
`
`AN¢NIY3SNJHL
`
`
`
`Yad0ddLNdNISI
`
`(ALVISANY¥O3
`
`JOINOUd
`
`Lndn|
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May4, 2004
`
`Sheet 5 of 5
`
`US 6,731,724 B2
`
`a
`
`byg
`
`Q3HS!8UNI3e
`Q3HSIGUNIIY
`
`d01H$30
`
`$31V$
`
`d01d¥1
`
`$31¥$
`
`JNOH
`
`d01¥$30
`
`$iv$
`
`big
`
`019
`
`ps9
`
`059
`
`p99
`
`299
`
`SSINISNG
`
`d0LYS30
`
`$31¥$
`
`SSINISNG
`
`d0ld¥)
`
`$31¥S
`
`9‘Old
`
`0vg
`
`AY0LOIYIG
`
`TANNOSYId40
`
`089
`
`$318
`
`029
`
`Y3LNdWOd
`
`Lu0ddNs
`
`019
`
`099
`
`0S9
`
`Brg
`
`949
`
`fa)
`
`SUISWNN
`
`NOISNILX3TWNGIAIONI
`
`Q3HSIGUNIFY
`
`Y3LNdWOd
`
`S31VS
`
`SS3NISNg
`
`43LNdW0d
`
`$31¥$
`
`NOK
`
`Y3LNdWOd
`
`$31¥S
`
`SSINISNE
`
`Y3LNdNOd
`
`LuOddNs
`
`JNOH
`
`YaLNdNOd
`
`LuOddfs
`
`
`
`
`
`
`
`
`
`
`
`

`

`US 6,731,724 B2
`
`1
`VOICE-ENABLED USER INTERFACE FOR
`VOICEMAIL SYSTEMS
`
`RELATED APPLICATIONS
`
`oH
`
`This application claims the benelit of U.S. Provisional
`Pateot Applicaton Ser. No. 60/263.469, entitled "Universal
`Voicemail Gateway (UVG),” filed Jan. 22, 2001, which is
`incorporated herein by reference.
`BACKGROUND OF THE INVENTION
`
`1. The Field of the Invention
`
`invention relates to a voice-enabled user
`The present
`interface for accessing a telephone service system that
`responds to dial tones. More particularly, the present inven-
`tion relates to methods and systems for enabling a user, with
`voice commands, to access und navigatea lelephone service
`system that only recognizes dual tone multi-frequency siz-
`nals.
`2. The Prior State of the Art
`
`an
`
`it has become commonplace to use a
`Through the years,
`iclephone, not only to communicate, but also to perform
`electronic business andfinancial transactions, and to obtain
`prerecorded information.
`It
`is possible to perform these
`activiies over a telephone, in part, because of the develop-
`ment of corresponding telephone service systems, which are
`used in virtually every industry and business environment.
`Voicemail is perhaps the most common type of telephone
`service system. Voicemail,
`like an answering machine,
`enables an incoming telephone message to be recorded for
`review by an intended recipient al a laler time. However,
`unlike an answering machine, voicemail can record an
`incoming message even When a corresponding telephoneis
`“off the book.” Accordingly, voicemail is particularly useful
`when a person cannot answer an incoming telephone call
`because thal person is already engaged in another telephone
`call,
`
`‘The prior art also includes various other types of tele-
`phone service systems that facilitate business and financial
`transactions and enable users to obtain prerecorded infor-
`mation, Essentially, a telephone service system includes any
`automated system that provides a user wilh data over a
`telephone device. Virtually every industry uses a telephone
`service sysiem of one type or another, even if the telephone
`service system consists of only a customized telephone
`directory of available business services or personnel.
`Some of the more useful telephone service systems enable
`users using a telephone device to interact with a business
`network to transact various activities, such as for performing,
`home banking, purchasing consumer products, receiving
`customer supporl, and accessing news, entertainment,
`financial, and travel
`information. Some telephone service
`systems also enable a user to access information that
`is
`generated on-the-fly, such as when a telephone service
`sysiem generates lext-lo-speech translations of dynamic
`information.
`‘Telephone service systems are particularly useful from a
`user’s perspective because they enable a user to access
`information and to make business and financial transactions
`when and where il is most convenient for the user, Telephone
`service sysiems are uscful from a business’s perspective
`because they enable businesses to hire fewer employees to
`answer incominy calls and to perform other tasks thal can be
`aulomated by telephone service systems.
`One problem with certain telephone service systems,
`however,
`is that
`they only recognize dual
`tone multi-
`
`A
`
`ryon
`
`‘ol
`
`2
`frequency signals (“DTMEP”) signals. A DTMFtelephone
`service system is generally problematic because a user can
`only use the system by manually pressing the keys on a
`DTMPF telephone keypad. This is a problem because it is
`easy for 4 user to inadvertently press an incorrect key,
`sometimes without realizing il. Pressing an incorrect key
`may result in an undesirable consequence, For example, by
`pressing an incorrect key a user may cause the telephone
`service system to transition from one menu slate (o an
`undesired menu state. This may also require a user to return
`to the prior menu stale before a desired selection can be
`made, This is not only inconvenient,ii is a waste of time. By
`pressing incorrect keys, a user may also cause a iclephone
`service Ssyslem to perform an act that is not desired. Por
`example, by pressing an incorrect key, o telephone service
`system may terminate a phone call, call a wrong extension
`number, or in the case of home banking,transfer an incorrect
`amount of money between accounts or to a wrong account.
`It should be appreciated thal these examples are given by
`way ofillustration and not limitation.
`‘To overcome the problems that are associated with inmad-
`vertenily pressing a wrong key, many telephone service
`sysiems provide prompts thal
`request
`a user to confirm
`certain entries before they are processed. This, however,
`does not resolve the underlying problem associated with
`DITMF telephone service systems, namely, requiring a user
`to enter inpul by pressing keys on a telephone keypad.
`Another problem associated with DTMFtelephone ser-
`vice syslems is thal a user is required lo move a handheld
`telephone back and forth between a position that enables the
`user to listen to the voicenvail sysiem and to a position that
`allows the user to press keys on the telephone keypad.
`Having to do this is inconvenient. It can also be dangerous,
`particularly for a person whois driving a car because il may
`require thal person lo lake his eyesolf the road and his hands
`off the steering wheel.
`‘To overcome the problems associated with DTMF tele-
`phoneservice systems,the industry bas developed telephone
`service systems thal recognize speech so thal a user can
`speak commands and other input without having to press
`keys on a telephone keypad. There are various methods for
`performing speech recognilion, which are well knownin the
`urt. Some examples include spectrul analysis, dynamic lime
`warping, neural networks, and recognition by discrete and
`continuous hidden Markov modeling.
`Although speech recognition telephone service systems
`provide a significan! advantage over DTMPtelephone ser-
`vice systems,
`they do not
`tesolve all of the associated
`problems with the prior art telephone service systems. In
`particular, a user must still expend the time that is necessary
`lo move systematically through a hierarchy of levels or
`menu states thal are associated with the menu structure ofa
`iclephone service system, even when a user already knows
`whalthe final meou state will be. For example, from a main
`menu a user may be required to transition through several
`“in between” menu states before finally reaching a final
`menu stale that contains the information that is desired by
`the user. Many telephone servicesystemsalsw require a user
`to return to the main menu before navigating to a second
`desired meou state, Navigation through a menu hierarchy is
`particularly time consuming when the hierarchy is complex,
`or has many menuslates. This type of navigation through a
`iclephone service system can also be expensive, particularly
`when aecess to the telephone service system requires a user
`to incur costs associated with long-distance and/or a wireless
`telephone service plan.
`Another problem with speech recognition telephone ser-
`vice systems is thal because of variations in speech it is
`
`

`

`US 6,731,724 B2
`
`3
`impossible lo guarantee thal speech recognition software
`will recognize user inpul. [I is also expensive for businesses
`to modify existing DTMF telephone service systems to
`recognize and process audio and vocal data with speech
`recognition soltware.
`Accordingly, it would be benclicial to provide a method
`for enabling 4 user lo use an existing DTMF telephone
`service system by speaking into a telephone device without
`requiring modification to the DTMEtelephone service sys-
`tem. [| would also be beneficial to accomplish this while
`enabling a user to jump to any level or stale in a menu
`hierarchy of a telephone service system without requiring
`the user to systematically navigate through numerous “in
`between” menu states.
`
`on
`
`u
`
`SUMMARY OF THE INVENTION
`
`‘The present invention is directed towards a voice-enabled
`user inlerface for enabling a user to speak over a telephone
`device to access data from a telephone service system that is
`only responsive to dual
`tone multi-frequency (“DTMP”)
`signals.
`In one embodiment, a user accesses the voice-enabled
`user interface with a telephone device and enters authenti-
`cation information. The voice-enabled user interface verifies
`the identity of the user and accesses a lelephone service
`system (hat has information thal is desired by the user, but
`which is only responsive to DTME signals, The user
`responds to prompts generated by the telephone service
`system by speaking into the telephone device. The voice-
`enabled user interface translates the user's input
`into a
`DTMPF translation that is recognized by the telephone ser-
`vice system, The voice-enabled user interface maps the
`hierarchy of menu states and corresponding prompts ofthe
`telephone service system within a template. During use, the
`voice-enabled user interface uses the template to determine
`what menu state the telephone service system is in at all
`limes. A user can jump from one menu slate to another menu
`slate by providing input thal the voice-enabled user interface
`associaies with a corresponding menu state. The voive-
`enabled user interface generates output
`that causes the
`telephone service sysiem 10 transition to the menu siate that
`corresponds with the user input. Once the telephone service
`sysiem is in the appropriate menu state, the voicc-enabled
`user interface transmits a DTMEtranslation of the user input
`lo the lelephone service sysiem so thal il can be processed.
`One advantage ofthe present invention is that it enables
`users using voice commands to access data from existing
`DTMFtelephone service systems withoul requiring any
`modification to the existing systems. ‘The present invention
`also enables users to navigate through a hierarchy of menu
`states of a telephone service system without requiring the
`users to enter input to transition through “in between” menu
`states. Instead, the invention enables a user to jump over “in
`between” menu states, from a first menu slate to a second
`menu state with only a single user input,
`These and other objects, features, and advantages of ihe
`present invention will become more fully apparent from the
`following, description and appended claims, or may be
`learned by practicing the invention as set forth below,
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`invention,
`A more extensive description of the present
`including the above-recited features and advantages will be
`rendered with reference to the specilic embodimentsthal are
`illustrated in the appended drawings. Because these draw-
`ings depict only exemplary embodiments,
`the drawings
`
`4
`should not be construed as imposing any limitation on the
`present invention’s scope, As such, the present invention
`will be described and explained with additional specificity
`und detail
`through use of the accompanying drawings in
`which:
`
`FIG, 1 is a block diagram showing one suitable network
`environment for practicing the invention that includes a
`tclephone device, a voice-enabled user interface, and a
`telephone service system;
`FIG, 2 is a block diagram showing increased detail of the
`components that make up the user stale manager module of
`the voice-enabled user interface of the present invention;
`FIG, 3 is a block diagram showing increased detail of the
`components that make up the voicemail state manager
`module of the voice-enabled user interface of the present
`invention;
`ilustrating ao embodiment for
`PIG, 4 is a flow chart
`enabling a user fo access a telephone service system with the
`voice-enabled user interface of the invention;
`FIG. 5 is a flow chart illustrating an embodiment of the
`invention for enabling a user (o access dala from a telephone
`service sysiem wilh (he voice-enabled user interlace of the
`invention;
`FIG, 6 is a block diagram illustrating one embodiment of
`a hierarchal menu structure of a telephone service system
`that can be accessed und navigated using the voice-enabled
`user interface of the invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`40
`
`A
`
`60)
`
`The present invention is directed lo a voice-enabled user
`interface and methods for enabling users lo access and use
`an existing telephone service system that is only responsive:
`to dial tone signals. More particularly, the present invention
`enables users lo speak into a telephone device lo access dala
`from an existing telephone service system that
`is only
`responsive to dual tone multi-frequency (“DTME") signals.
`‘The term telephone service system, as used herein, should
`generally be construed to include any telephone service that
`is responsive lo user input entered over a telephone device
`and is capable of providing data to a user. One type of
`iclephone service system that can be accessed by using the
`present invention includes a voicemail system,
`Users can perform various functions with a voicemail
`system. These functions include, but are not
`limited to,
`listening to messages that are lef by a third party, recording
`messages or greetings for a third party to listen to, forward-
`ing messages, deleting messages, and configunng voicemail
`answering and recording procedures. Users. perform func~
`lions wilh a voicemail system by entering inpul
`into a
`telephone device that can be recognized and processed by
`the voicemail system. DTMIV voicemail systems are one
`type of voicemail systems that are only responsive to DTMF
`signals. DTMFvoicemail systems cannot recognize audio or
`vocal input,
`In one embodimentof the invention, 1 voice-enabled user
`interface enables users to perform functions with a DTME
`voicemail system by only speaking into a telephone device.
`It should be appreciated, however,
`thal although certain
`embodiments of the present
`invention are described as
`applying lo DTMEP veicemail systems, the invention is nol
`limited to any particular type of telephone service system.
`Similarly, a user can access and use the voice-enabled user
`interface of the invention with any type of telephone device,
`including, but not limited to wire bound telephones, wireless
`
`

`

`US 6,731,724 B2
`
`oH
`
`u
`
`5
`telephones, and
`telephones, cellular telephones, digilal
`PDA’s and computers having telephony capabilities,
`Embodiments ofthe invention, as described herein, may
`comprise a special purpose or general-purpose computer
`comprising Various computer hardware. Embodiments may
`also include computer-readable media having computer-
`exectilable instructions or data structures stored thereon.
`Such computer-readable media can be any available media
`thal can be accessed by a general-purpose or special-purpose
`compuler. By way of example, and not
`limilation, such
`computer-readable media can comprise RAM, ROM,
`EEPROM, CD-ROM or other optical disk storage. magnetic
`disk storage or other magnetic storage devices, or any olher
`medium which ean be used to store the desired executable
`insiructions or data structures and which can be accessed by |
`a general-purpose or special-purpose compuler,
`When information is transferred or provided over a nel-
`work or olher communications connection lo a compuler, the
`computer properly views the connection as a compuler-
`readable medium. Thus, such a connection is also properly
`termed a computer-readable medium, Combinations of the
`above should also be included within the scope of computer-
`readable media. Computer-executable instructions
`comprise, for example, instructions and data which cause a
`general-purpose compuler, special-purpose computer, or
`special-purpose processing device io perform a certain fune-
`tion or group offunctions. The compuler-executable insiruc-
`tions and associated data structures represent an example of
`program code means for executing the steps of the invention
`disclosed herein.
`
`The invention will be described in the general context of
`compuler-execulable instructions, such as program modules,
`being executed by a computer, Generally, program modules
`include routines, programs, objects, components, data
`structures, or the like that performparticular tasks or imple-
`ment particular abstract data lypes. Moreover, those skilled
`in the art will appreciate that the invention may be practiced
`wilh other computer system configurations, including hand-
`held devices, multi-processor systems, microprocessor-
`based or programmable consumerelectronics, network PCs,
`minicomputers, mainframe computers, and the like. The
`invention may also be practiced in distributed computing
`environments where tasks are performed by remote process-
`ing devices that are linked through a communications nel-
`work,
`In a distribuled compuling environment, program
`modules may be locaied in both local and remote memory
`storage devices.
`Turning now to FIG. 1, one suitable environment LOO for
`practicing the method ofthe invention for enabling a user to
`access data from a telephoneservice system is illustrated. As
`shown, a Voice-enabled user interface 110 is in communi-
`cation with a telephone device 120 and a telephone service
`sysiem through telephony networks 130, ‘The invention may
`be practiced in various telephony network environments.
`including, bul not
`limited to Public Switched ‘Telephone
`Networks (PSTN),
`Integrated Services Digital Network
`(ISDN) and Piber Distributed Digital Interface (PDD. In
`the present embodiment, the tlephone service system com-
`prises voicemail system 140 which can be accessed by a user
`using lelephone device 120,
`Voicemail system 140 comprises voicemail data 150 that
`can include any type of data that can be transmitted to a user
`over a
`telephone device.
`In one preferred embodiment,
`voicemail data 150 comprises u prerecorded message, such
`us, for example, a menu prompt, a telephone phone message,
`4 greeting, etc. Voicemail data 150 may also include data
`
`40
`
`St
`
`‘ola
`
`6
`thal is dynamic or that is accessed from another network. For
`example, in one embodiment, voicemail sysiem 140 com-
`prises a telephone service system ofa financial institution
`und voicemail data 150 includes dynamic stock quote data
`that is obtained by the voicemail system 140 from another
`network. Voicemail data 150 may also include data that is
`generated on-the-tly. Por example, stock quotes can be
`provided to voicemail system 140 in the form of text, such
`that during use voicemail system 140 generates a lext-lo-
`speech translation of the stock quotes on-the-lly.
`According to the present embodiment, voicemail system
`140 is responsive only to DTMF signals that are generated
`by pressing the keys on a keypad of a DTMEtelephone
`device, such as telephone device 120. DITMFtelephone
`signals are well known in theart of ielophone communica-
`tions. According to this embodiment, voicemail system 140
`cannot recognize audio or vacal input thal is generated by a
`user speaking into telephone device 120. However, voice-
`enabled user interface 110 of the invention, enables a user to
`access voicemail data 150 from the voicemail system 140 by
`speaking into a telephone device 120. as will be deseribed
`herein.
`Voice-enabled user interface 110 contains various com-
`ponents that enable a user lo access voicemail data 150 from
`voicemail system 14). These components are generally
`described in reference to a user stale manager module 160,
`a Voicemail stale manager module 170, and a database 180,
`FIG, 2 shows someolthe basic components thal make up
`user state manager module 160. User authentication 210
`verifies ihe identity of a user accessing the yoiee-cnabled
`user interface 110 by comparing the user account informa-
`tion to account information 220 that is stored in database
`180. The act of verifying the identity of the user may require
`the user to enter a password, personal identilication number,
`un account number, or any other information that can be
`verified by user authentication 210,
`According to one embodiment, and upon authenticating
`the identity of the user, user aulhenticalion 210 connects ihe
`voice-enubled user interface io 4 telephone service system
`that contains data that is desired by the user, such as
`voicemail system 140, According to this embodiment, user
`account information 220 includes information that is used to
`connect to and access voicemail system 140. For example,
`user account information 220 can include a telephone num-
`ber for calling voicemail system 140, and account informa-
`tion and a password for accessing voicemail sysiem 140.
`According to another embodiment, user account informa-
`tion 220 includes a telephone number for calling voicemail
`system 140, but does nol include account information or a
`password [hat is necessary to access voicemail system 140,
`According to this embodiment, the user must provide the
`account information and password during use, after user
`authentication 210 has connected the voice-enabled user
`interface to the voicemail system 140,
`Speechrecognition 230) interprets audio or voice data that
`is received from a telephone device, which is generated by
`a user speaking into the telephone device. Speech recogni-
`tion 230 can also interpret user inpul comprising telephone
`signals, such as DTMFsignals. Techniques for implement-
`ing speech recognition 230 are well known in the respective
`url, and include, bui are not
`limited to speciral analysis,
`dynamic time warping, ocural networks, and recognition by
`discrete and continuous hidden Markov modeling.
`‘Translator 240 receives user inpul comprising audio or
`voicedata and translates the data into DTME signals that can
`be recognized by voicemail system 140. According to the
`
`

`

`US 6,731,724 B2
`
`c
`
`20
`
`7
`presen! embodiment, voicemail system 140 does not recog-
`nize audio or voice dala and is only responsive to DOTMP
`signals, ‘The accuracy of making 4 translation of user input
`into a DTMF signal is enhanced by comparing user input
`with dnticipated user input according to data that is stored in
`template 232 of database 180. [t should be noted that the
`translation ofuser input that is generated by translator 240
`is not a direct translation, but instead is a translation of the
`dala stored in template 232 thal most resembles the user
`input. Because template 232 has a finite number of accept-
`able inputs and responses,translator 240 is generally able to
`make an accurate translation of user inpul, which ean vary
`from one user to another, by comparing the user inpul to the
`known data in template 232.
`Alemplate, such as template 232, maps all of the menu |
`slates of a telephone menu system and all of the correspond-
`ing prompts and acceptable responses and inputs. A techni-
`cian prepares only @ single lemplate for each telephone
`service system that
`is accessible to users using the voice-
`enabled userinterlace of the invention. Accordingly, a single
`template is used for making translations of user input from
`multiple users accessing the same telephone service system.
`Communication switch 250 controls the transmission of
`dala through the vaice-enabled user interlace thal is trans-
`mitted between a telephone device 120 and voicemail sys-
`tem 140. [n one embodiment, two simplex channels 252, 254
`connect
`the voice-enabled user interface to the telephone
`device 120 and two simplex channels 256, 258 connect the
`voice-coabled user interface to the voicemail system 140,
`each set of simplex channels comprising an inbound and an
`outbound simplex channel, A simplex channel is simply a
`one-way communication channel, lt should be appreciated
`ihat other types of communication channels can also be
`used, such asa ball-duplex channel.
`According to this embodiment, communication switch
`250 synchronizesall communication over the simplex chan-
`otls 252-258. During use, the communication switch 250
`connects the inbound simplex channel 256, from the voice-
`mail system 140, to the outbound simplex channel 252 that
`is directed to the telephone device 120. This enables the user
`to hear transmissions of data directly from the voicemail
`sysiem 140. The communication switch 250 also directs
`transmissions of data from inbound simplex channel 254 to
`speech recognition 230, while dirceling all outbound trans-
`missions of (he voice-enabled user interface to voicemail
`system 140 through outbound simplex channel 258.
`‘The communication swilch 250 also enables the voice-
`enabled user interface (o interrupt the voicemail system 140,
`whenever appropriate, by disconnecting the inbound sim-
`plex channel 256 from the outbound simplex channel 252,
`One appropriate circumstance to interrupt
`the voicemail
`sysiem 140 is to prevent the user from hearing prompts thal
`the user does nol need to respond to, Por example, afler user
`authentication 210 connects the voice-enableduser interface
`to voicemail system 140, the voicemail system 140 may
`transmil prompts for the user to coter a valid account number
`and password.
`the voice-enabled user interface
`In one embodiment,
`disconnects the inbound simplex channel 256 from the
`oulbound simplex channel 252 while user authentication
`210 (transmits the user's account number and password to
`voicemail system 140 from database 180, According to this
`example,
`the communication switch 250 prevents the
`incoming transmission of data from the voicemail system
`140 from being transmitted over the outbound simplex
`channel 252 to the user.
`
`43
`
`Al
`
`‘ol
`
`8
`Another appropriate circumstance for interrupting the
`voicemail system 140 is when the voive-enabled user inter-
`face is causing the voicemail syslom lo jump over “in
`between” menu sfales, from a menu stale to final menu slate,
`This will be deseribed later in reference to FIG. 6.
`State manager 260 executes user requests and performs
`various general control functions. State manager 260 also
`coordinates the connectivity and operation of other
`components, such as, for example enabling components 210,
`230, 240, and 250 to access database 180, and coordinating
`the transmission of data to and from components 210, 230,
`240, and 250 tosimplex channels 252-258.
`FLG, 3 shows someof(he basic components thal make up
`voicemail stale manager module L170, of FIG.
`[. In one
`preferred embodiment, voicemail authentication 310 con-
`neets ihe veice-cnabled user interface to voicemail system
`140 of PIGS. 1-2. According to this embodiment, voicemail
`authentication 310retrieves account information 220 from
`database 180 and accesses voicemail system 140. Account
`information 220 can include a telephone qumber for calling
`voicemail system 140, and account voicemail information
`and password for accessing voicemail system 140.
`According to another embodiment, necessary account
`information 220 includes a telephone number for calling
`voicemail system 140, but does not include account infor-
`mation or a password for accessing voicemail system 140,
`According to this embodiment, 4 user must provide the
`account information and password during use, afler yoice-
`mail authentication 310 has accessed the voicemail svstem
`140.
`Keep alive 330 keeps a telephone service system in synch
`with the voice-enabled user interface b