`Bogard
`
`I 1111111111111111 11111 111111111111111 1111111111 1111111111 1111111111 11111111
`US006757365Bl
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 6,757,365 Bl
`Jun.29,2004
`
`(54)
`
`INSTANT MESSAGING VIA TELEPHONE
`INTERFACES
`
`(75)
`
`Inventor: Travis A. Bogard, San Francisco, CA
`(US)
`
`(73) Assignee: Tellme Networks, Inc., Mountain View,
`CA(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 213 days.
`
`6,424,945 Bl * 7/2002 Sorsa ...................... 704/270.1
`6,430,602 Bl * 8/2002 Kay et al. ................... 709/206
`6,430,604 Bl * 8/2002 Ogle et al. .................. 709/207
`6,463,142 Bl * 10/2002 Kilp ........................ 379/88.11
`OTHER PUBLICATIONS
`
`Diament et al. Method an apparatus for telephony-enabled
`instant messaging Jun. 13, 2002. *
`Diament et al., Method and Apparatus for Telephony-en(cid:173)
`abled Instant Messaging Jun. 13, 2002. *
`Myers, Telephone Based Access to Intant Messaging, May
`17, 2001.*
`
`(21) Appl. No.: 09/691,606
`
`(22) Filed:
`
`Oct. 16, 2000
`
`Int. Cl.7 ................................................. H04M 1/64
`(51)
`(52) U.S. Cl. ................................ 379/88.17; 704/270.1;
`709/206
`(58) Field of Search ........................... 379/88.13, 88.01,
`379/88.04, 88.14, 88.16, 88.22, 67.1, 265.01;
`704/270.1; 709/206
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,721,763 A * 2/1998 Joseph et al. ............ 379/88.04
`5,799,063 A
`8/1998 Krane
`6,301,609 Bl * 10/2001 Aravamudan et al.
`...... 709/206
`6,324,569 Bl * 11/2001 Ogilvie et al. .............. 709/206
`6,377,944 Bl * 4/2002 Busey et al. ........... 379/265.01
`6,405,035 Bl * 6/2002 Singh ......................... 455/414
`6,424,647 Bl * 7/2002 Ng et al.
`.................... 370/352
`
`* cited by examiner
`
`Primary Examiner-Fan Tsang
`Assistant Examiner-Gerald Gauthier
`(74) Attorney, Agent, or Firm-Bever, Hoffman & Harms,
`LLP; Jeanette S. Harms
`
`(57)
`
`ABSTRACT
`
`A method and apparatus for enabling users of a phone based
`speech activated system such as a voice portal to commu(cid:173)
`nicate with users of an Internet based instant messenger (IM)
`service is described. Phone based users are able to send and
`receive IMs. Incoming messages can cause an asynchronous
`notification in the user's current voice application and the
`user can (if they desire) switch contexts to hear the IM and
`respond. Sent messages may be expeditiously sent to users
`of the GUI as a hypertext link to a recorded audio. Other
`sending formats are also possible; similarly, buddy lists can
`be supported.
`
`7 Claims, 5 Drawing Sheets
`
`ci
`
`Wireless
`Telephone
`301
`
`DI--
`
`Telephone
`300
`
`D
`
`Computer
`302
`
`Telephone Gateway
`307
`
`Voice Portal
`310
`
`Instant Messenging
`Server
`308
`
`Page 1 of 14
`
`
`
`U.S. Patent
`
`Jun.29,2004
`
`Sheet 1 of 5
`
`US 6,757,365 Bl
`
`Message History
`202
`
`. :•.
`
`• LB4"dies < 114)1
`Buddy1
`• Family (Dill)
`• Co .Workers (J/12) i f
`. I:
`Buddy2
`·. ,,
`Buddy3
`Buddy4
`► Of:line (1 :, 1 G)
`
`User1: Hey Joe
`Buddy1: Hey Tom
`
`Want to catch a movie tonight
`
`Send Instant
`Message
`102
`
`Over
`Internet
`Voice
`Chai 104
`
`Message
`Composition 204
`
`Figure 1
`(Prior Art)
`
`Over Internet
`Voice Chat--~--
`104
`
`Figure 2
`(Prior Art)
`
`Page 2 of 14
`
`
`
`U.S. Patent
`U.S. Patent
`
`Jun.29,2004
`Jun. 29, 2004
`
`Sheet 2 of 5
`Sheet 2 of 5
`
`US 6,757,365 Bl
`US 6,757,365 Bl
`
`('I)
`
`¢ainbi4
`
`E
`::::s
`C> ·-LL
`
`
`
`JENOgad10A,
`
`Ole
`
`
`
`BuiBuassayjue}su]
`
`J8Nlas
`
`80€
`
`JaWS}UI
`
`90€
`
`Sudydayja| aeSTTTULAS
`
`Aemayesauoydaja)
`
`WOMAN
`
`L0€
`
`vO
`
`=LETTTN
`
`en Q)
`(/) §
`LJ (i; o...O
`Q) ..c: ..-
`LO€
`s: Q)
`.!:::: Q) CV)
`
`auoydale|SSOJOIIN
`
`I-
`
`Q)
`C
`0 0
`Q0€
`..c: 0
`a}M
`
`auoydaja|Hl
`
`Q)
`I-
`
`□.
`
`>-
`$
`::IN
`COE
`O.c,
`EC")
`0
`(_)
`
`sayndwo4y
`
`Page 3 of 14
`
`Page 3 of 14
`
`
`
`
`i,-
`~
`(It
`a-...
`~
`-...,l
`(It
`-...,l
`_,.a-...
`rJ'J.
`e
`
`~
`
`Ul
`0 ....,
`~ ....
`'JJ. =(cid:173)~
`
`~
`
`,i;;..
`0
`0
`N
`~~
`N
`
`~
`
`~ = ......
`~ ......
`~
`•
`r:JJ.
`d •
`
`Figure 5
`
`End
`
`Figure 4
`
`End
`
`message text format
`
`506
`
`using IM server and instant
`received text to voice; all
`
`reader to translate
`
`translate voice to text; Use
`Use speech recognition to
`
`recipient in IM format
`voice connection w/
`Use IM server to open
`
`504
`
`Identify recipient
`
`502
`
`messaging application
`
`500
`
`Access instant
`
`Respond verbally
`
`406
`
`404
`buddy
`
`Query IM server for
`
`or receive input
`
`402
`
`Select from buddy list
`
`messaging application
`
`400
`
`Access instant
`
`Start
`
`Initiate Messaging ...
`
`Start
`
`Is a user online?
`
`Page 4 of 14
`
`
`
`i,-
`~
`(It
`a-...
`~
`-...,l
`(It
`-...,l
`_,.a-...
`rJ'J.
`e
`
`Ul
`0 ....,
`~ ....
`'JJ. =(cid:173)~
`
`,i;;..
`
`~
`
`,i;;..
`0
`0
`N
`~~
`N
`
`~
`
`~ = ......
`~ ......
`~
`•
`r:JJ.
`d •
`
`Figure 7
`
`'
`
`End
`
`, Ir
`
`,r
`
`Figure 6
`
`audio form
`
`704
`
`playback to user in
`
`information and
`
`Access IM server for
`
`,
`
`Construct query
`
`702
`
`messaging application
`
`700
`
`Access instant
`
`, .
`
`Start
`
`'\..
`
`'
`
`End
`
`, Ir
`
`r
`
`needed)
`
`604
`
`text-to-speech as
`with audio (including
`Play back messages
`
`, '
`
`stored messages
`Query IM Server for
`
`602
`
`, Ir
`
`600
`
`messaging application
`
`Access instant
`
`''
`
`Start
`
`'-
`
`Query IM Server for Information, etc ...
`
`Retrieving Stored Messages ...
`
`Page 5 of 14
`
`
`
`i,-
`~
`(It
`O'I
`~
`-....,l
`(It
`~
`O'I
`rJ'J.
`e
`
`Ul
`0 ....,
`Ul
`~
`
`'JJ. =(cid:173)~
`
`~
`
`,i;;..
`0
`0
`N
`~~
`N
`
`~
`
`~ = ......
`~ ......
`~
`•
`r:JJ.
`d •
`
`Figure 9
`
`.....
`
`End
`
`1 •
`
`r
`
`Figure 8
`
`messaging mode
`Allow user to enter
`
`904
`
`,.
`
`current application
`audible signal into
`asynchronously, e.g.
`
`902
`
`Notify user
`
`, '
`
`900
`server
`
`of messages via IM
`Listen for notification
`
`, I'
`
`~
`
`Start
`
`\...
`
`.....
`
`End
`
`, Ir
`
`/
`
`messages
`notification of
`program for
`
`804
`
`Start background
`
`, I'
`
`user profile
`
`802
`
`using information in
`Sign on to IM server
`
`1 •
`
`messaging
`
`800
`
`activation of instant
`
`User requests
`
`1
`
`~
`
`Start
`
`\...
`
`Asynchronous notification ...
`
`Register presence ...
`
`Page 6 of 14
`
`
`
`US 6,757,365 Bl
`
`1
`INSTANT MESSAGING VIA TELEPHONE
`INTERFACES
`
`BACKGROUND OF THE INVENTION
`
`2
`of the traditional computer based IM service to be notified
`when a buddy is signed in by way of the phone and message
`that buddy, and vice versa. Similarly, communication
`between phone based users by way of the IM service should
`5 be supported.
`
`SUMMARY OF THE INVENTION
`
`A method and apparatus for enabling users of a phone
`based speech activated system such as a voice portal to
`10 communicate with users of an Internet based instant mes(cid:173)
`senger (IM) service is described. Phone based users are able
`to send and receive IMs. Incoming messages can cause an
`asynchronous notification in the user's current voice appli(cid:173)
`cation and the user can (if they desire) switch contexts to
`15 hear the IM and respond.
`Sent messages may be expeditiously sent to users of the
`GUI as a hypertext link to a recorded audio. Other formats
`may include textual representations of speech, e.g. through
`the results of speech recognition as well as initiation of a
`voice communication in the format of the IM protocol.
`Buddy lists can be provided, e.g. phone based users can
`indicate those users they want to know the online/offline
`status for. The buddy list might be presented verbally.
`Notably, the IM functionality changes the general
`
`1. Field of the Invention
`This invention relates to the field of information services.
`In particular, the invention relates to technologies for
`improving voice-based access to instant messaging services
`over a telephone interface.
`2. Description of the Related Art
`Instant Messaging services such as the popular ICQ(™)
`and AOL INSTANT MESSANGER(™), also referred to as
`AIM, both operated by America Online, Inc., Dulles, Va.
`(AOL), have risen in popularity in the last few years and
`shown explosive growth. Older protocols and services such
`as Internet Relay Chat (IRC), see RFC 1459, and the even
`older talk program (primarily found on UNIX(™)-type
`computers) have quickly been eclipsed. Competitors to AOL 20
`such as Microsoft Corporation, Redmond, Wash., and
`Yahoo!, Inc., Santa Clara, Calif., have introduced competing
`instant messenger products that operate in a similar overall
`fashion to AIM which shall be used as a reference herein.
`Turning to prior art FIGS. 1-2, exemplary screenshots 25
`from AIM operating under the Windows(™) operating sys(cid:173)
`tem are shown. FIG. 1 shows the buddy list 100. The buddy
`list 100 allows a user of AIM to see which buddies ( other
`users of interest to our particular user, e.g. friends,
`co-workers, family members) are signed on, e.g. Buddyl, 30
`Buddy2, Buddy3, and Buddy4 in this example. When the
`user desires to instant message, or IM, with a buddy, she
`simply clicks on the send instant message button 102 and a
`window such as the buddy chat window 200 of FIG. 2
`appears. The buddy chat window 200 allows a user to see a 35
`message history 202 of previous messages in an IM session
`and compose additional messages in the message composi(cid:173)
`tion areas 204.
`Additionally, if both users have sufficient computer
`equipment, e.g. microphones, speakers, fast enough Internet 40
`connections, etc., the newer versions of the AIM client
`software allow computer-to-computer voice communica(cid:173)
`tions over the packet switched Internet backbone, e.g. by
`clicking on the over Internet voice chat button 104. Clicking
`on the button will bring up a window for monitoring 45
`performance and, in half duplex mode, controlling who
`speaks when.
`Previous IM systems do not provide a mechanism for
`allowing users of a basic telephone ( or wireless telephone)
`to send and receive instant messages. Further, the existing
`systems are not adapted to handle voice only users, e.g. users
`who do not have a graphical user interface (GUI) for
`reviewing buddy lists and sending/receiving text messages.
`On the telephone side, several types of "party lines" have 55
`been offered, frequently of the pay variety (900 number in
`the United States). However, these services have never been
`integrated with an IM service and further these services do
`not have an "appearance"/"buddy" concept to allow specific
`users to contact each other. Rather, they are simply large 60
`conference calls.
`Lastly, previous systems have not allowed two telephone
`users to be connected by way of a computer based identity
`such as an instant message appearance.
`Accordingly, what is needed is a method and apparatus for 65
`allowing users with telephones to access IM services. The
`system should support a number of features that allow users
`
`50
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`FIG. 1 illustrates a prior art graphical user interface based
`buddy list.
`FIG. 2 illustrates a prior art graphical user interface based
`buddy chat.
`FIG. 3 illustrates a system including embodiments of the
`invention used to provide instant messaging service to users
`of telephones.
`FIG. 4 is a process flow diagram for determining whether
`another user is online in the instant messaging (IM) service.
`FIG. 5 is a process flow diagram for initiating and sending
`an IM.
`FIG. 6 is a process flow diagram for retrieving stored IMs.
`FIG. 7 is a process flow diagram for querying an IM
`server for information.
`FIG. 8 is a process flow diagram for registering a tele(cid:173)
`phone user's presence with an IM service.
`FIG. 9 is a process flow diagram for asynchronous noti(cid:173)
`fication of incoming IMs.
`
`DETAILED DESCRIPTION
`
`A Introduction
`A voice portal supporting electronic commerce over a
`telephone interface is described. The voice portal allows
`users of telephones, including wireless telephones, to access
`a voice portal by dialing a phone number to purchase goods
`and services, interact with applications, and access IM
`services. The information provided over the voice portal
`may come from the World Wide Web (WWW), databases,
`third parties, and/or other sources.
`The voice portal can receive dual-tone multi-frequency
`(DTMF or touch-tone) commands as well as spoken com(cid:173)
`mands to further control the content presented and direct
`commerce transactions as well as the manner of presenta(cid:173)
`tion. The term audio request, or input, is used to refer to
`either a voice or touch-tone input, or a combination of the
`two types of input.
`Embodiments of the invention can use telephone identi(cid:173)
`fying information to personalize caller interactions with the
`
`Page 7 of 14
`
`
`
`US 6,757,365 Bl
`
`3
`voice portal. This allows the system to present highly
`customized information to each caller based on a personal
`profile the system associates with the telephone identifying
`information. Additionally, since a single user can access the
`voice portal from a number of telephones, embodiments of 5
`the invention may construct user profiles that allow the
`telephone identifying information from multiple telephones
`to be associated with a single user profile. In some
`embodiments, the telephone identifying information may be
`used to access authenticating information from a user profile
`for provision to an IM service, e.g. the IM service username
`and password might be stored in the user profile.
`The invention will be described in greater detail as
`follows. First, a number of definitions useful to understand(cid:173)
`ing the invention are presented. Then, the hardware and 15
`software architecture for one embodiment of a voice portal
`presented. Next, features provided by embodiments of the
`invention are discussed in greater detail.
`
`B. Definitions
`1. Telephone Identifying Information
`For the purposes of this application, the term telephone
`identifying information will be used to refer to ANI
`information, CID information, and/or some other technique
`for automatically identifying the source of a call and/or other
`call setup information. For example, telephone identifying
`information may include a dialed number identification
`service (DNIS). Similarly, CID information may include text
`data including the subscriber's name and/or address, e.g.
`"Jane Doe". Other examples of telephone identifying infor- 30
`mation might include the type of calling phone, e.g.
`wireless, pay phone, and/or hospital phone.
`Additionally, the telephone identifying information may
`include wireless carrier specific identifying information, e.g.
`location of wireless phone now, etc. Also, signaling system 35
`seven (SS7) information may be included in the telephone
`identifying information.
`2. User Profile
`A user profile is a collection of information about a
`particular user. The user profile typically includes collec(cid:173)
`tions of different information as shown and described more
`fully in connection with FIGS. 3 and 4. Notably, the user
`profile contains a combination of explicitly made selections
`and implicitly made selections.
`Explicitly made selections in the user profile stem from
`requests by the user to the system. For example, the user
`might add business news to the main topic list. Typically,
`explicit selections come in the form of a voice, or touch-tone
`command, to save a particular location, e.g. "Add to my
`favorites", "Remember this", "Bookmark it", "shortcut
`this", pound(#) key touch-tone, etc., or through adjustments
`to the user profile made through the web interface using a
`computer.
`Additionally, the user profile provides a useful mecha(cid:173)
`nism for associating telephone identifying information with
`a single user, or entity. For example, Jane Doe may have a
`home phone, a work phone, a cell phone, and/or some other
`telephones. Suitable telephone identifying information for
`each of those phones can be associated in a single profile for
`Jane. This allows the system to provide uniformity of 60
`customization to a single user, irrespective of where they are
`calling from.
`In contrast, implicit selections come about through the
`conduct and behavior of the user. For example, if the user
`repeatedly asks for the weather in Palo Alto, Calif., the 65
`system may automatically provide the Palo Alto weather
`report without further prompting. In other embodiments, the
`
`4
`user may be prompted to confirm the system's implicit
`choice, e.g. the system might prompt the user "Would you
`like me to include Palo Alto in the standard weather report
`from now on?"
`Additionally, the system may allow the user to customize
`the system to meet her/his needs better. For example, the
`user may be allowed to control the verbosity of prompts, the
`dialect used, and/or other settings for the system. These
`customizations can be made either explicitly or implicitly.
`10 For example if the user is providing commands before most
`prompts are finished, the system could recognize that a less
`verbose set of prompts is needed and implicitly set the user's
`prompting preference to briefer prompts.
`3. Topics and Content
`A topic is any collection of similar content. Topics may be
`arranged hierarchically as well. For example, a topic might
`be business news, while subtopics might include stock
`quotes, market report, and analyst reports. Within a topic
`different types of content are available. For example, in the
`20 stock quotes subtopic, the content might include stock
`quotes. The distinction between topics and the content
`within the topics is primarily one of degree in that each
`topic, or subtopic, will usually contain several pieces of
`content.
`25 4. Cookie
`The term cookie, as used herein, refers to a structured data
`element formatted according to the general principles of
`IETF RFC 2109 and/or some other state management stan(cid:173)
`dard.
`A brief review of RFC 2109 may be useful. The core
`structure of a cookie is a name-value pair. The name is a
`token for identifying the cookie, e.g. "Customer", and the
`value is the value of that corresponding token, e.g. "Jane
`Doe".
`Implicitly, each cookie is associated with a sending
`domain on the World Wide Web. According to RFC 2109,
`the implicitly set domain is the originating domain to which
`an HTTP request was sent. For example, if an HTTP GET
`request is sent to the request host "www.example.com", then
`40 the cookie set in response to that request would be implicitly
`associated with "www.example.com"
`Additionally, a number of optional fields can be set, for
`example: a different domain for which the cookie is valid
`(Domain); a time to live (Max-Age); a version string
`45 (Version); etc. The phrases in parenthesis correspond to the
`RFC 2109 standard field names for the options.
`5. Demographic and Psychographic Profiles
`Both demographic profiles and psychographic profiles
`contain information relating to a user. Demographic profiles
`50 typically include factual information, e.g. age, gender, mari(cid:173)
`tal status, income, etc. Psychographic profiles typically
`include information about behaviors, e.g. fun loving,
`analytical, compassionate, fast reader, slow reader, etc. As
`used in this application, the term demographic profile will be
`55 used to refer to both demographic and psychographic pro(cid:173)
`files.
`6. Locale
`As used in this application, the term locale refers to any
`geographic area. The geographic area may be a
`neighborhood, a city, a county, a metropolitan region, a state,
`a country, a continent, a group of countries, and/or some
`other collection of one or more geographic areas, e.g. all
`United State major metropolitan areas.
`For this reason, a single user of the system may be
`considered to be in several locales. For example, a caller
`from Palo Alto, Calif., might be in the Palo Alto locale, a
`Silicon Valley locale, a San Francisco Bay Area locale, a
`
`Page 8 of 14
`
`
`
`US 6,757,365 Bl
`
`5
`Northern California locale, a California state locale, and a
`United States locale.
`Thus, the telephone identifying information for a single
`telephone number can be mapped to a number of system(cid:173)
`defined locales.
`
`6
`Additional information regarding voice portal 310 and
`various components interfacing with voice portal 310 are
`discussed in further detail in U.S. patent application Ser. No.
`09/426,102, entitled "Method and Apparatus for Electronic
`5 Commerce Using a Telephone Interface", filed on 22 Oct.
`1999, which is incorporated by reference herein.
`
`30
`
`C. System Overview
`First, the hardware and software architecture of a system
`including an embodiment of the invention will be described
`with reference to FIG. 3. FIG. 3 illustrates a system includ(cid:173)
`ing embodiments of the invention used to provide IM 10
`services to users of telephones. The system of FIG. 3 can be
`used to allow users of standard telephones and wireless
`telephones to access a voice portal.
`The following lists the elements of FIG. 3 and describes
`their interconnections. FIG. 3 includes a telephone 300, a 15
`wireless telephone 301, a computer 302, a telephone net(cid:173)
`work 304, an Internet 306, a telephone gateway 307, an IM
`server 308, and a voice portal 310. The wireless telephone
`301 and the telephone 300 are coupled in communication
`with the telephone network 304. The telephone network 304 20
`is coupled in communication with the telephone gateway
`307. The telephone gateway 307 is coupled in communica(cid:173)
`tion with the voice portal 310. The computer 302 is coupled
`in communication with the Internet 306. The Internet 306 is
`coupled in communication with the web server 308. The 25
`voice portal 310 and the web server 308 are coupled in
`communication with the shared database 312.
`The following describes each of the elements of FIG. 3 in
`greater detail. The use of each of the elements will be
`described further in conjunction with the sections describing
`the personalization features.
`The telephone 300 and the wireless telephone 301 are two
`different telephone interfaces to the voice portal 310. The
`telephone 300 and the wireless telephone 301 may be any
`sort of telephone and/or wireless telephone. For example the 35
`telephone 300, or the wireless telephone 301, may be a land
`line phone, a PBX telephone, a satellite phone, a wireless
`telephone, and/or any other type of communication device
`capable of providing voice communication and/or touch(cid:173)
`tone signals over the telephone network 304. However, any 40
`audio signal carrying interface could be used.
`The telephone network 304 may be the public switched
`telephone network (PSTN) and/or some other type of tele(cid:173)
`phone network. For example, some embodiments of the
`invention may allow users with a voice over Internet Pro- 45
`tocol (IP) phone to access the voice portal 310. The tele(cid:173)
`phone network 304 is coupled to the telephone gateway 307
`that allows the voice communications and/or touch-tone
`signals from the telephone network 304 to reach the voice
`portal 310 in usable form. Similarly, the telephone gateway 50
`307 allows audio signals generated by the voice portal 310
`to be sent over the telephone network 304 to respective
`telephones, e.g. the telephone 300. The telephone network
`304 generally represents an audio signal carrying network.
`The computer 302 is a computer such as a personal 55
`computer, a thin client computer, a server computer, a
`handheld computer, a set top box computer, and/or some
`other type of visual web browsing device. The computer 302
`is coupled in communication with the Internet 306, e.g. by
`a dial-up connection, a digital subscriber loop (DSL), a cable 60
`modem, and/or some other type of connection. This allows
`the computer 302 to communicate with the IM server 308.
`The computer 302 typically provides a visual interface to the
`WWW and the IM service, by way of IM server 308, using
`web browsing software and IM software such as Internet 65
`Explorer(™) from Microsoft Corporation, Redmond, Wash.,
`and AIM.
`
`D. Instant Messaging Functionality
`
`Overview
`First the usage scenarios are considered. Then, the basic
`changes to the voice portal 310 to support IM functionality
`will be discussed. Finally, the process flow/implementation
`for those scenarios is described.
`Usage Scenarios
`It is helpful to understand how the IM functionality will
`be made available to users of the voice portal 310 by
`considering a few usage scenarios. The usage scenarios can
`easily be divided into two primary categories: initiating and
`receiving. In terms of initiating messages four primary
`sub-areas can be identified: (1) determining if user X is
`online; (2) sending text and/or voice messages to a user; (3)
`retrieving stored messages (if supported by underlying IM
`service); and ( 4) getting information, e.g. user info, etc.
`From the receiving side there are four similar issues: (1)
`registering your presence on the phone with the IM service;
`(2) receiving notification of messages; (3) allowing partici(cid:173)
`pating in messaging; ( 4) posting information/registering.
`These usage scenarios dovetail nicely into the implementa-
`tion issues.
`Platform Changes
`The voice portal 310 includes one or more programs for
`interpreting voice applications, e.g. VoiceXML (or VXML)
`programs, colloquially these programs for running VXML
`programs for multiple phone users together with the asso(cid:173)
`ciated functionalities are sometimes referred to as the "plat(cid:173)
`form". Although some shared messaging capabilities may
`have been available through the platform and voice portal
`310 through dedicated applications, those features were
`application specific, e.g. message board, chat room (voice
`based user discussion), etc. In some embodiments, those
`specialized features can be generalized (and implemented)
`through the instant messaging functionality, e.g. channel
`features of an IM service.
`The platform in normal operation supports the execution
`of a single VXML application per user. For example, if the
`user is accessing weather information using a weather
`application, e.g. weather.vxml, then only that application
`would be running. The platform can be modified to support
`concurrent execution of multiple programs for users, for
`example both the weather application and, in the
`"background", an instant messaging application.
`Additionally, mechanisms for switching between running
`applications must be provided, this mechanism should allow
`preservation of state (where the user is, variables, dialogue
`point, etc.) when the user switches applications, e.g. to
`respond to an instant message or send an instant message.
`Similarly, one or more "universal" commands, dual-tone
`multi-frequency (DTMF), or switch hook signals, may be
`provided to switch the running application. For example, in
`one embodiment, the word "Message" might be recognized
`to switch to the messaging application while preserving state
`in the other application.
`According to some embodiments, the voice portal 310
`allows users to control their experience. The system reacts to
`commands the user says (or doesn't say in the allotted time)
`in a synchronous fashion. Since incoming messages may
`
`Page 9 of 14
`
`
`
`US 6,757,365 Bl
`
`8
`
`7
`come at any time the voice portal 310 may also provide an
`2. Initiating Messaging
`asynchronous notification mechanism, e.g. a distinctive tone
`FIG. 5 is a process flow diagram for initiating and sending
`or beep, to alert the user to switch to messaging mode.
`an IM. This could be used for example after the process of
`These underlying architectural changes will be made
`FIG. 4 (or in conjunction with the process of FIG. 4) to send
`clearer in the following discussion.
`5 a message to a logged in user. In the case that the particular
`Implementations
`IM service supports stored messages, then the process of
`The implementations will be discussed in greater detail
`FIG. 5 could be used to send stored messages as well.
`with reference to FIGS. 4-9.
`First, at step 500, the caller to the voice portal 310
`1. Is a User Online?
`accesses the IM application. This process can be done as in
`FIG. 4 is a process flow diagram for determining whether
`10 step 400.
`Next, at step 502, the caller identifies one or more
`another user is online in the instant messaging (IM) service.
`This could be used by a user to learn which of her/his
`recipients or, when supported by the underlying IM service,
`buddies are signed into the IM service, learn if a specifically
`a chat room name. This can be done according to the process
`of step 402.
`designated user is signed in, etc.
`At step 400, the user accesses the instant messaging 15
`The process than branches to either step 504 or 506,
`application. This may be accomplished by providing a signal
`though some embodiments may allow a caller to use steps
`over the telephone interface to the voice portal 310. In some
`504 and 506 concurrently in the same IM session, may allow
`the user to select between step 504 and 506, or may
`embodiments of the invention, a keyword such as "Instant
`Messanger" or "Messenger" is provided to allow access to
`automatically determine which of steps 504 and 506 are
`IM functionality within specific locations, e.g. main menu, 20 used based on the capabilities of the message recipient. In
`of the voice portal 310. Additionally, in some embodiments
`either case, the messaged buddy may not be available, e.g.
`one or more universal commands are provided, e.g. "*IM"
`because they just signed off, etc. In this case, the voice portal
`(e.g., key press on"*" followed by "4" followed by "6" on
`310 can verbally tell the user that the message recipient was
`United States-style telephones) or one or more keywords. A
`unavailable.
`Use of step 504 requires that the underlying IM service
`universal command would typically be available in most, if 25
`not all, running applications contexts. In some
`offer a voice connection functionality. In this step, the voice
`portal 310 will send one or more requests to the IM server
`embodiments, when a universal command is provided to the
`308 to request initiation of a voice connection with the
`voice portal 310, the presently running application is sus(cid:173)
`pended ( and its state saved) until the user exits from the IM
`identified recipient, e.g. Buddyl. Typically, this may cause
`30 the EM server 308 to send one or more requests to the
`application. In some embodiments, the same universal com(cid:173)
`mand is used to toggle between the IM application and the
`remote user, e.g. on their computer or perhaps logged into
`other presently running application, e.g. "**", "##''. "00",
`the IM service via the voice portal 310 on a different
`"*IM", etc. According to this embodiment, the user can
`telephone. The IM software being used by Buddyl may
`easily switch back and forth between the IM functionalities
`allow Buddyl to select whether to allow the IM session, if
`and her/his other activities on the voice portal 310.
`35 approved, the connection will then be established (possibly
`Next, at step 402, an IM user is selected from the caller's
`through the IM server 308) between the IM software used by
`Buddyl ( e.g. the prior art AIM(™) client of FIG. 1) and the
`buddy list (as maintained on the voice portal 310 or on the
`IM application executing on the voice portal 310. Once so
`IM server 308) or by direct entry of the username(s).
`According to some embodiments, the IM user name may be
`connected, the caller and Buddyl can talk over that connec-
`40 tion with the IM application on the voice portal 310 trans(cid:173)
`entered using a voice keyboard technique as described in
`lating the telephone audio into the IM voice protocol format
`U.S. patent application Ser. No. 09/655,277 entitled
`(which may be a standard VoIP format) for transmission to
`"Method and Apparatus for Voice Keyboard" filed 5 Sep.
`the remote IM software client ( and performing the reverse
`2000, and assigned to the assignee of the present application.
`The user may be able to query for a list of available users
`translation on received audio).
`Step 506 in contrast is may used when the recipient cannot
`from the buddy list with a spoken command or the name of 45
`a buddy e.g. "Who's online", "Buddyl", etc.
`(or will not) accept, or their IM software or connection
`Next, at step 404, the voice portal 310 sends one or more
`cannot support, voice communications. In this configuration,
`the underlying IM service text formats are used to send
`queries to the IM server 308 to obtain the request informa(cid:173)
`messages. Messages received from the remote user can be
`tion about the selected IM user(s). For example, according
`50 played back using text-to-sp