(12) United States Patent
`Bogard
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 6,757,365 B1
`Jun. 29, 2004
`
`US006757365B1
`
`(54) INSTANT MESSAGING VIA TELEPHONE
`INTERFACES
`
`(75) Inventor: Travis A. Bogard, San Francisco, CA
`(Us)
`(73) Assignee: Tellme Networks, Inc., Mountain View,
`CA (US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 213 days.
`
`6,424,945 B1
`6,430,602 B1
`6,430,604 B1
`6,463,142 B1
`
`*
`
`*
`
`*
`
`*
`
`7/2002 Sorsa .................... .. 704/270.1
`
`8/2002 Kay et al. . . . . . .
`
`. . . . .. 709/206
`
`8/2002 Ogle et al. . . . . .
`. . . . .. 709/207
`10/2002 Kilp ...................... .. 379/88.11
`
`OTHER PUBLICATIONS
`
`Diament et al. Method an apparatus for telephony—enabled
`instant messaging Jun. 13, 2002*
`Diament et al., Method and Apparatus for Telephony—en
`abled Instant Messaging Jun. 13, 2002*
`Myers, Telephone Based Access to Intant Messaging, May
`17, 2001.*
`
`(21) Appl. No.: 09/691,606
`(22) Filed:
`Oct. 16, 2000
`
`(51) Int. Cl.7 ............................................... .. H04M 1/64
`(52) US. Cl. .............................. .. 379/8817; 704/270.1;
`709/206
`(58) Field of Search ......................... .. 379/88.13, 88.01,
`379/8804, 88.14, 88.16, 88.22, 67.1, 265.01;
`704/2701; 709/206
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,721,763 A * 2/1998 Joseph et a1. .......... .. 379/88.04
`5,799,063 A
`8/1998 Krane
`6,301,609 B1 * 10/2001 Aravamudan et a1. .... .. 709/206
`6,324,569 B1 * 11/2001 Ogilvie et a1. ............ .. 709/206
`6,377,944 B1 * 4/2002 Busey et a1.
`379/265.01
`6,405,035 B1 * 6/2002 Singh .................... .. 455/414
`6,424,647 B1 * 7/2002 Ng et a1. .................. .. 370/352
`
`* cited by examiner
`
`Primary Examiner—F an Tsang
`Assistant Examiner—Gerald Gauthier
`(74) Attorney, Agent, or Firm—Bever, Hoffman & Harms,
`LLP; Jeanette S. Harms
`(57)
`ABSTRACT
`
`A method and apparatus for enabling users of a phone based
`speech activated system such as a voice portal to commu
`nicate With users of an Internet based instant messenger (IM)
`service is described. Phone based users are able to send and
`receive IMs. Incoming messages can cause an asynchronous
`noti?cation in the user’s current voice application and the
`user can (if they desire) sWitch contexts to hear the IM and
`respond. Sent messages may be expeditiously sent to users
`of the GUI as a hypertext link to a recorded audio. Other
`sending formats are also possible; similarly, buddy lists can
`be supported.
`
`7 Claims, 5 Drawing Sheets
`
`Wireless
`Telephone
`
`Telephone
`300
`
`Computer
`302
`
`Telephone Network
`304
`
`Telephone Gateway
`307
`
`Voice Portal
`31 0
`
`Internet
`306
`
`Instant Messenging
`Sewer
`308
`
`Facebook's Exhibit No. 1007
`Page 1
`
`

`

`U.S. Patent
`
`Jun. 29, 2004
`
`Sheet 1 of5
`
`US 6,757,365 B1
`
`Message History
`202
`
`v Family (811])
`
`A m h c F
`
`w w e.
`
`Figure 1
`(Prior Art)
`
`Message
`102
`
`Over
`lnte met
`a
`‘ /_ Voice
`Chat 104
`
`
`
`Message Composition 204
`
`Want to catch a movie l?nignt
`
`Over Internet
`Voice Chat
`104
`
`Figure 2
`(Prior Art)
`
`Facebook's Exhibit No. 1007
`Page 2
`
`

`

`U.S. Patent
`
`Jun. 29, 2004
`
`Sheet 2 of5
`
`US 6,757,365 B1
`
`m mSmE
`
`52mm
`
`
`
`msmcmmwmz E82:
`
`mom
`
`6525
`
`8m
`
`
`
`atom 86>
`
`o 3
`
`
`
`$538 @6523
`
`8m
`
`
`
`vtoéoz 2258. E
`
`$22;
`
`29323
`
`Sm
`
`@8523
`
`8m
`
`Facebook's Exhibit No. 1007
`Page 3
`
`

`

`U.S. Patent
`
`Jun. 29, 2004
`
`Sheet 3 of 5
`
`US 6,757,365 Bl
`
`
`
`
`
`0}uoN|uBooesyosadsasj}
`
`
`
`jUe]SU!SSa00"
`
`
`
`uoneaiddeBuibessow
`
`00S
`
`
`
`quaidioasAjquap|
`
`20S
`
`
`
`JEWJO}1x9}afessow
`
`90S
`
`v0
`
`aye}Sued}0}Japeal
`
`ASN){}XS}0}BOIOAaye\SUeR
`
`
`uado0}JGAJ9SI]Sf)
`
`]Ue\SUIpuJaAJESJ]Bulsn
`I|2{SO10A0}Xa}panieoa.
`
`
`JEWJOYWijUlJUs!dioal
`
`/MUONOBUUCDBd10A
`
`sgoinbiy
`
`7ainBbi4
`
`
`
`JUe}SUISSAD0'/
`
`
`
`uoyeoddeBulbessow
`
`00¥
`
`
`
`ysi|Appngwioyyoajag
`
`
`
`ynduldaig09uJO
`
`cOV
`
`Jo}JanesW]Asent)
`
`Appng
`
`b0r
`
`
`
`Ajpequeapuodsay
`
`90P
`
`Facebook's Exhibit No. 1007
`Page 4
`
`
`
`“Bulbessayyayeniuy
`
`9ul|uoJasneS|
`
`Facebook's Exhibit No. 1007
`Page 4
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 29, 2004
`
`Sheet 4 of 5
`
`US 6,757,365 Bl
`
`
`
`jue\sulssa00y
`
`
`
`uonjeoyddeBulbessow
`
`002
`
`
`
`Asanbjonysuo09
`
`ZOL
`
`JO}J8AIaS|$8890
`
`PUVONEWUJOjU!
`
`ulJasn0}yoeqheld
`
`
`
`WO}olpne
`
`p02
`
`Zeinbi4
`
`9ainbi4
`
`
`
`JUB}SU]SS800V7
`
`
`
`uonjeoyddeBuibessow
`
`009
`
`Jo}JAAIaS||ent
`
`
`
`safessowpaiojs
`
`209
`
`
`
`safessewyoegAe|4
`
`
`
`Bulpnjoul)oipneyjm
`
`seyoaads-0}-}x9}
`
`(papsau
`
`709
`
`Facebook's Exhibit No. 1007
`Page 5
`
`
`
`"919‘UONPUOJU]JO}JBAJESJAI]AlaNH
`
`
`
`
`
`
`
`
`
`““saBessayypaioj}sBuinsujay
`
`Facebook's Exhibit No. 1007
`Page 5
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 29, 2004
`
`Sheet 5 of 5
`
`US 6,757,365 Bl
`
`UOHEdYOU10}Ud}SI7
`
`
`
` JeAias|elAsaBessawjo
`
`006
`
`JasnAjHON
`
`
`
`‘B'a‘Ajsnouoiyoudse
`
`
`
`
`
`OjuljeuBlsajqipne
`
`
`
`uojeaddejuauna
`
`206
`
`Jaqua0}JasnMolly
`
`apowBulbessew
`
`p06
`
`6aunbi4
`
`gainbi4
`
`
`
`sjsenbalasp
`
`JUe}SUIJOUOIEAOe
`
`Bulbessaw
`
`008
`
`JOAI9S||0}UoUBIS
`
`uluoWeWwojUIBulsn
`
`aloudJasn
`
`208
`
`punosByoeqWes
`
`Jo}weJGoud
`
`JOuoJeoynou
`
`
`v08
`
`sobessow
`
`Facebook's Exhibit No. 1007
`Page 6
`
`
`
`““YOIeoyOUSNOUIOUASY
`
`
`
`“-gouasaidJa}siboy
`
`Facebook's Exhibit No. 1007
`Page 6
`
`
`
`
`
`

`

`US 6,757,365 B1
`
`1
`INSTANT MESSAGING VIA TELEPHONE
`INTERFACES
`
`BACKGROUND OF THE INVENTION
`
`2
`of the traditional computer based IM service to be noti?ed
`When a buddy is signed in by Way of the phone and message
`that buddy, and vice versa. Similarly, communication
`betWeen phone based users by Way of the IM service should
`be supported.
`
`SUMMARY OF THE INVENTION
`
`A method and apparatus for enabling users of a phone
`based speech activated system such as a voice portal to
`communicate With users of an Internet based instant mes
`senger (IM) service is described. Phone based users are able
`to send and receive IMs. Incoming messages can cause an
`asynchronous noti?cation in the user’s current voice appli
`cation and the user can (if they desire) sWitch contexts to
`hear the IM and respond.
`Sent messages may be expeditiously sent to users of the
`GUI as a hypertext link to a recorded audio. Other formats
`may include textual representations of speech, eg through
`the results of speech recognition as Well as initiation of a
`voice communication in the format of the IM protocol.
`Buddy lists can be provided, e.g. phone based users can
`indicate those users they Want to knoW the online/offline
`status for. The buddy list might be presented verbally.
`Notably, the IM functionality changes the general
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`FIG. 1 illustrates a prior art graphical user interface based
`buddy list.
`FIG. 2 illustrates a prior art graphical user interface based
`buddy chat.
`FIG. 3 illustrates a system including embodiments of the
`invention used to provide instant messaging service to users
`of telephones.
`FIG. 4 is a process How diagram for determining Whether
`another user is online in the instant messaging (IM) service.
`FIG. 5 is a process How diagram for initiating and sending
`an IM.
`FIG. 6 is a process How diagram for retrieving stored IMs.
`FIG. 7 is a process How diagram for querying an IM
`server for information.
`FIG. 8 is a process How diagram for registering a tele
`phone user’s presence With an IM service.
`FIG. 9 is a process How diagram for asynchronous noti
`?cation of incoming IMs.
`
`DETAILED DESCRIPTION
`
`A. Introduction
`
`A voice portal supporting electronic commerce over a
`telephone interface is described. The voice portal alloWs
`users of telephones, including Wireless telephones, to access
`a voice portal by dialing a phone number to purchase goods
`and services, interact With applications, and access IM
`services. The information provided over the voice portal
`may come from the World Wide Web (WWW), databases,
`third parties, and/or other sources.
`The voice portal can receive dual-tone multi-frequency
`(DTMF or touch-tone) commands as Well as spoken com
`mands to further control the content presented and direct
`commerce transactions as Well as the manner of presenta
`tion. The term audio request, or input, is used to refer to
`either a voice or touch-tone input, or a combination of the
`tWo types of input.
`Embodiments of the invention can use telephone identi
`fying information to personaliZe caller interactions With the
`
`1. Field of the Invention
`This invention relates to the ?eld of information services.
`In particular, the invention relates to technologies for
`improving voice-based access to instant messaging services
`over a telephone interface.
`2. Description of the Related Art
`Instant Messaging services such as the popular ICQ(TM)
`and AOL INSTANT MESSANGER(TM), also referred to as
`AIM, both operated by America Online, Inc., Dulles, Va.
`(AOL), have risen in popularity in the last feW years and
`shoWn explosive groWth. Older protocols and services such
`as Internet Relay Chat (IRC), see RFC 1459, and the even
`older talk program (primarily found on UNIX(TM)-type
`computers) have quickly been eclipsed. Competitors to AOL
`such as Microsoft Corporation, Redmond, Wash., and
`Yahoo!, Inc., Santa Clara, Calif., have introduced competing
`instant messenger products that operate in a similar overall
`fashion to AIM Which shall be used as a reference herein.
`Turning to prior art FIGS. 1—2, exemplary screenshots
`from AIM operating under the WindoWs(TM) operating sys
`tem are shoWn. FIG. 1 shoWs the buddy list 100. The buddy
`list 100 alloWs a user of AIM to see Which buddies (other
`users of interest to our particular user, e.g. friends,
`co-Workers, family members) are signed on, eg Buddyl,
`Buddy2, Buddy3, and Buddy4 in this example. When the
`user desires to instant message, or IM, With a buddy, she
`simply clicks on the send instant message button 102 and a
`WindoW such as the buddy chat WindoW 200 of FIG. 2
`appears. The buddy chat WindoW 200 alloWs a user to see a
`message history 202 of previous messages in an IM session
`and compose additional messages in the message composi
`tion areas 204.
`Additionally, if both users have suf?cient computer
`equipment, eg microphones, speakers, fast enough Internet
`connections, etc., the neWer versions of the AIM client
`softWare alloW computer-to-computer voice communica
`tions over the packet sWitched Internet backbone, eg by
`clicking on the over Internet voice chat button 104. Clicking
`on the button Will bring up a WindoW for monitoring
`performance and, in half duplex mode, controlling Who
`speaks When.
`Previous IM systems do not provide a mechanism for
`alloWing users of a basic telephone (or Wireless telephone)
`to send and receive instant messages. Further, the existing
`systems are not adapted to handle voice only users, e. g. users
`Who do not have a graphical user interface (GUI) for
`revieWing buddy lists and sending/receiving text messages.
`On the telephone side, several types of “party lines” have
`been offered, frequently of the pay variety (900 number in
`the United States). HoWever, these services have never been
`integrated With an IM service and further these services do
`not have an “appearance”/“buddy” concept to alloW speci?c
`users to contact each other. Rather, they are simply large
`conference calls.
`Lastly, previous systems have not alloWed tWo telephone
`users to be connected by Way of a computer based identity
`such as an instant message appearance.
`Accordingly, What is needed is a method and apparatus for
`alloWing users With telephones to access IM services. The
`system should support a number of features that alloW users
`
`10
`
`15
`
`25
`
`35
`
`40
`
`45
`
`55
`
`65
`
`Facebook's Exhibit No. 1007
`Page 7
`
`

`

`US 6,757,365 B1
`
`3
`voice portal. This allows the system to present highly
`customized information to each caller based on a personal
`pro?le the system associates With the telephone identifying
`information. Additionally, since a single user can access the
`voice portal from a number of telephones, embodiments of
`the invention may construct user pro?les that alloW the
`telephone identifying information from multiple telephones
`to be associated With a single user pro?le. In some
`embodiments, the telephone identifying information may be
`used to access authenticating information from a user pro?le
`for provision to an IM service, e.g. the IM service username
`and passWord might be stored in the user pro?le.
`The invention Will be described in greater detail as
`folloWs. First, a number of de?nitions useful to understand
`ing the invention are presented. Then, the hardWare and
`softWare architecture for one embodiment of a voice portal
`presented. Next, features provided by embodiments of the
`invention are discussed in greater detail.
`
`B. De?nitions
`1. Telephone Identifying Information
`For the purposes of this application, the term telephone
`identifying information Will be used to refer to ANI
`information, CID information, and/or some other technique
`for automatically identifying the source of a call and/or other
`call setup information. For example, telephone identifying
`information may include a dialed number identi?cation
`service (DNIS). Similarly, CID information may include text
`data including the subscriber’s name and/or address, e.g.
`“Jane Doe”. Other examples of telephone identifying infor
`mation might include the type of calling phone, e.g.
`Wireless, pay phone, and/or hospital phone.
`Additionally, the telephone identifying information may
`include Wireless carrier speci?c identifying information, e.g.
`location of Wireless phone noW, etc. Also, signaling system
`seven (SS7) information may be included in the telephone
`identifying information.
`2. User Pro?le
`A user pro?le is a collection of information about a
`particular user. The user pro?le typically includes collec
`tions of different information as shoWn and described more
`fully in connection With FIGS. 3 and 4. Notably, the user
`pro?le contains a combination of explicitly made selections
`and implicitly made selections.
`Explicitly made selections in the user pro?le stem from
`requests by the user to the system. For example, the user
`might add business neWs to the main topic list. Typically,
`explicit selections come in the form of a voice, or touch-tone
`command, to save a particular location, e.g. “Add to my
`favorites”, “Remember this”, “Bookmark it”, “shortcut
`this”, pound (#) key touch-tone, etc., or through adjustments
`to the user pro?le made through the Web interface using a
`computer.
`Additionally, the user pro?le provides a useful mecha
`nism for associating telephone identifying information With
`a single user, or entity. For example, Jane Doe may have a
`home phone, a Work phone, a cell phone, and/or some other
`telephones. Suitable telephone identifying information for
`each of those phones can be associated in a single pro?le for
`Jane. This alloWs the system to provide uniformity of
`customiZation to a single user, irrespective of Where they are
`calling from.
`In contrast, implicit selections come about through the
`conduct and behavior of the user. For example, if the user
`repeatedly asks for the Weather in Palo Alto, Calif., the
`system may automatically provide the Palo Alto Weather
`report Without further prompting. In other embodiments, the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`user may be prompted to con?rm the system’s implicit
`choice, e.g. the system might prompt the user “Would you
`like me to include Palo Alto in the standard Weather report
`from noW on?”
`Additionally, the system may alloW the user to customiZe
`the system to meet her/his needs better. For example, the
`user may be alloWed to control the verbosity of prompts, the
`dialect used, and/or other settings for the system. These
`customiZations can be made either explicitly or implicitly.
`For example if the user is providing commands before most
`prompts are ?nished, the system could recogniZe that a less
`verbose set of prompts is needed and implicitly set the user’s
`prompting preference to briefer prompts.
`3. Topics and Content
`Atopic is any collection of similar content. Topics may be
`arranged hierarchically as Well. For example, a topic might
`be business neWs, While subtopics might include stock
`quotes, market report, and analyst reports. Within a topic
`different types of content are available. For example, in the
`stock quotes subtopic, the content might include stock
`quotes. The distinction betWeen topics and the content
`Within the topics is primarily one of degree in that each
`topic, or subtopic, Will usually contain several pieces of
`content.
`4. Cookie
`The term cookie, as used herein, refers to a structured data
`element formatted according to the general principles of
`IETF RFC 2109 and/or some other state management stan
`dard.
`A brief revieW of RFC 2109 may be useful. The core
`structure of a cookie is a name-value pair. The name is a
`token for identifying the cookie, e.g. “Customer”, and the
`value is the value of that corresponding token, e.g. “Jane
`Doe”.
`Implicitly, each cookie is associated With a sending
`domain on the World Wide Web. According to RFC 2109,
`the implicitly set domain is the originating domain to Which
`an HTTP request Was sent. For example, if an HTTP GET
`request is sent to the request host “WWW.example.com”, then
`the cookie set in response to that request Would be implicitly
`associated With “WWW.example.com”
`Additionally, a number of optional ?elds can be set, for
`example: a different domain for Which the cookie is valid
`(Domain); a time to live (Max-Age); a version string
`(Version); etc. The phrases in parenthesis correspond to the
`RFC 2109 standard ?eld names for the options.
`5. Demographic and Psychographic Pro?les
`Both demographic pro?les and psychographic pro?les
`contain information relating to a user. Demographic pro?les
`typically include factual information, e.g. age, gender, mari
`tal status, income, etc. Psychographic pro?les typically
`include information about behaviors, e.g. fun loving,
`analytical, compassionate, fast reader, sloW reader, etc. As
`used in this application, the term demographic pro?le Will be
`used to refer to both demographic and psychographic pro
`?les.
`6. Locale
`As used in this application, the term locale refers to any
`geographic area. The geographic area may be a
`neighborhood, a city, a county, a metropolitan region, a state,
`a country, a continent, a group of countries, and/or some
`other collection of one or more geographic areas, e.g. all
`United State major metropolitan areas.
`For this reason, a single user of the system may be
`considered to be in several locales. For example, a caller
`from Palo Alto, Calif., might be in the Palo Alto locale, a
`Silicon Valley locale, a San Francisco Bay Area locale, a
`
`Facebook's Exhibit No. 1007
`Page 8
`
`

`

`US 6,757,365 B1
`
`5
`Northern California locale, a California state locale, and a
`United States locale.
`Thus, the telephone identifying information for a single
`telephone number can be mapped to a number of system
`de?ned locales.
`
`C. System OvervieW
`First, the hardWare and softWare architecture of a system
`including an embodiment of the invention Will be described
`With reference to FIG. 3. FIG. 3 illustrates a system includ
`ing embodiments of the invention used to provide IM
`services to users of telephones. The system of FIG. 3 can be
`used to alloW users of standard telephones and Wireless
`telephones to access a voice portal.
`The folloWing lists the elements of FIG. 3 and describes
`their interconnections. FIG. 3 includes a telephone 300, a
`Wireless telephone 301, a computer 302, a telephone net
`Work 304, an Internet 306, a telephone gateWay 307, an IM
`server 308, and a voice portal 310. The Wireless telephone
`301 and the telephone 300 are coupled in communication
`With the telephone netWork 304. The telephone netWork 304
`is coupled in communication With the telephone gateWay
`307. The telephone gateWay 307 is coupled in communica
`tion With the voice portal 310. The computer 302 is coupled
`in communication With the Internet 306. The Internet 306 is
`coupled in communication With the Web server 308. The
`voice portal 310 and the Web server 308 are coupled in
`communication With the shared database 312.
`The folloWing describes each of the elements of FIG. 3 in
`greater detail. The use of each of the elements Will be
`described further in conjunction With the sections describing
`the personaliZation features.
`The telephone 300 and the Wireless telephone 301 are tWo
`different telephone interfaces to the voice portal 310. The
`telephone 300 and the Wireless telephone 301 may be any
`sort of telephone and/or Wireless telephone. For example the
`telephone 300, or the Wireless telephone 301, may be a land
`line phone, a PBX telephone, a satellite phone, a Wireless
`telephone, and/or any other type of communication device
`capable of providing voice communication and/or touch
`tone signals over the telephone netWork 304. HoWever, any
`audio signal carrying interface could be used.
`The telephone netWork 304 may be the public sWitched
`telephone netWork (PSTN) and/or some other type of tele
`phone netWork. For example, some embodiments of the
`invention may alloW users With a voice over Internet Pro
`tocol (IP) phone to access the voice portal 310. The tele
`phone netWork 304 is coupled to the telephone gateWay 307
`that alloWs the voice communications and/or touch-tone
`signals from the telephone netWork 304 to reach the voice
`portal 310 in usable form. Similarly, the telephone gateWay
`307 alloWs audio signals generated by the voice portal 310
`to be sent over the telephone netWork 304 to respective
`telephones, eg the telephone 300. The telephone netWork
`304 generally represents an audio signal carrying netWork.
`The computer 302 is a computer such as a personal
`computer, a thin client computer, a server computer, a
`handheld computer, a set top box computer, and/or some
`other type of visual Web broWsing device. The computer 302
`is coupled in communication With the Internet 306, eg by
`a dial-up connection, a digital subscriber loop (DSL), a cable
`modem, and/or some other type of connection. This alloWs
`the computer 302 to communicate With the IM server 308.
`The computer 302 typically provides a visual interface to the
`WWW and the IM service, by Way of IM server 308, using
`Web broWsing softWare and IM softWare such as Internet
`Explorer(TM) from Microsoft Corporation, Redmond, Wash.,
`and AIM.
`
`1O
`
`15
`
`25
`
`35
`
`40
`
`45
`
`55
`
`65
`
`6
`Additional information regarding voice portal 310 and
`various components interfacing With voice portal 310 are
`discussed in further detail in US. patent application Ser. No.
`09/426,102, entitled “Method and Apparatus for Electronic
`Commerce Using a Telephone Interface”, ?led on 22 Oct.
`1999, Which is incorporated by reference herein.
`
`D. Instant Messaging Functionality
`OvervieW
`First the usage scenarios are considered. Then, the basic
`changes to the voice portal 310 to support IM functionality
`Will be discussed. Finally, the process ?oW/implementation
`for those scenarios is described.
`Usage Scenarios
`It is helpful to understand hoW the IM functionality Will
`be made available to users of the voice portal 310 by
`considering a feW usage scenarios. The usage scenarios can
`easily be divided into tWo primary categories: initiating and
`receiving. In terms of initiating messages four primary
`sub-areas can be identi?ed: (1) determining if user X is
`online; (2) sending text and/or voice messages to a user; (3)
`retrieving stored messages (if supported by underlying IM
`service); and (4) getting information, eg user info, etc.
`From the receiving side there are four similar issues: (1)
`registering your presence on the phone With the IM service;
`(2) receiving noti?cation of messages; (3) alloWing partici
`pating in messaging; (4) posting information/registering.
`These usage scenarios dovetail nicely into the implementa
`tion issues.
`Platform Changes
`The voice portal 310 includes one or more programs for
`interpreting voice applications, eg VoiceXML (or VXML)
`programs, colloquially these programs for running VXML
`programs for multiple phone users together With the asso
`ciated functionalities are sometimes referred to as the “plat
`form”. Although some shared messaging capabilities may
`have been available through the platform and voice portal
`310 through dedicated applications, those features Were
`application speci?c, e.g. message board, chat room (voice
`based user discussion), etc. In some embodiments, those
`specialiZed features can be generaliZed (and implemented)
`through the instant messaging functionality, e.g. channel
`features of an IM service.
`The platform in normal operation supports the execution
`of a single VXML application per user. For example, if the
`user is accessing Weather information using a Weather
`application, eg Weather.vxml, then only that application
`Would be running. The platform can be modi?ed to support
`concurrent execution of multiple programs for users, for
`example both the Weather application and, in the
`“background”, an instant messaging application.
`Additionally, mechanisms for sWitching betWeen running
`applications must be provided, this mechanism should alloW
`preservation of state (Where the user is, variables, dialogue
`point, etc.) When the user sWitches applications, eg to
`respond to an instant message or send an instant message.
`Similarly, one or more “universal” commands, dual-tone
`multi-frequency (DTMF), or sWitch hook signals, may be
`provided to sWitch the running application. For example, in
`one embodiment, the Word “Message” might be recogniZed
`to sWitch to the messaging application While preserving state
`in the other application.
`According to some embodiments, the voice portal 310
`alloWs users to control their experience. The system reacts to
`commands the user says (or doesn’t say in the allotted time)
`in a synchronous fashion. Since incoming messages may
`
`Facebook's Exhibit No. 1007
`Page 9
`
`

`

`US 6,757,365 B1
`
`10
`
`15
`
`7
`come at any time the voice portal 310 may also provide an
`asynchronous noti?cation mechanism, e. g. a distinctive tone
`or beep, to alert the user to sWitch to messaging mode.
`These underlying architectural changes Will be made
`clearer in the folloWing discussion.
`Implementations
`The implementations Will be discussed in greater detail
`With reference to FIGS. 4—9.
`1. Is a User Online?
`FIG. 4 is a process How diagram for determining Whether
`another user is online in the instant messaging (IM) service.
`This could be used by a user to learn Which of her/his
`buddies are signed into the IM service, learn if a speci?cally
`designated user is signed in, etc.
`At step 400, the user accesses the instant messaging
`application. This may be accomplished by providing a signal
`over the telephone interface to the voice portal 310. In some
`embodiments of the invention, a keyWord such as “Instant
`Messanger” or “Messenger” is provided to alloW access to
`IM functionality Within speci?c locations, e.g. main menu,
`of the voice portal 310. Additionally, in some embodiments
`one or more universal commands are provided, e.g. “*IM”
`(e.g., key press on “*” folloWed by “4” folloWed by “6” on
`United States-style telephones) or one or more keyWords. A
`universal command Would typically be available in most, if
`25
`not all, running applications contexts. In some
`embodiments, When a universal command is provided to the
`voice portal 310, the presently running application is sus
`pended (and its state saved) until the user exits from the IM
`application. In some embodiments, the same universal com
`mand is used to toggle betWeen the IM application and the
`other presently running application, e.g. “**”, “##”. “00”,
`“*IM”, etc. According to this embodiment, the user can
`easily sWitch back and forth betWeen the IM functionalities
`and her/his other activities on the voice portal 310.
`Next, at step 402, an IM user is selected from the caller’s
`buddy list (as maintained on the voice portal 310 or on the
`IM server 308) or by direct entry of the username(s).
`According to some embodiments, the IM user name may be
`entered using a voice keyboard technique as described in
`U.S. patent application Ser. No. 09/655,277 entitled
`“Method and Apparatus for Voice Keyboard” ?led 5 Sep.
`2000, and assigned to the assignee of the present application.
`The user may be able to query for a list of available users
`from the buddy list With a spoken command or the name of
`a buddy e.g. “Who’s online”, “Buddyl”, etc.
`Next, at step 404, the voice portal 310 sends one or more
`queries to the IM server 308 to obtain the request informa
`tion about the selected IM user(s). For example, according
`to some embodiments the IM service is implemented using
`one or more IM application protocols on top of a TCP/IP
`implementation. In such an embodiment, the voice portal
`310 Will generate suitable query packets according to the IM
`application protocols and send them over the Internet 306 to
`the IM server 308 for a query response. In some implemen
`tations of 308 the IM server Will have already told the voice
`portal 310 of the presence of buddies already on the list and
`the voice portal 310 can check against a local data store.
`Finally, at step 406, the voice portal 310 receives and
`decodes the IM application protocol packets received in
`response to its query or gets the result from its local data
`store and plays back information to the caller verbally. For
`example, if the caller at step 402 asked if “Buddyl” Was
`online, then at step 406, the voice portal 310 Would playback
`a human understandable version of the query results, e.g.
`“Buddyl is presently online, but is aWay” or “Buddyl is not
`logged in”, etc.
`
`8
`
`2. Initiating Messaging
`FIG. 5 is a process How diagram for initiating and sending
`an IM. This could be used for example after the process of
`FIG. 4 (or in conjunction With the process of FIG. 4) to send
`a message to a logged in user. In the case that the particular
`IM service supports stored messages, then the process of
`FIG. 5 could be used to send stored messages as Well.
`First, at step 500, the caller to the voice portal 310
`accesses the IM application. This process can be done as in
`step 400.
`Next, at step 502, the caller identi?es one or more
`recipients or, When supported by the underlying IM service,
`a chat room name. This can be done according to the process
`of step 402.
`The process than branches to either step 504 or 506,
`though some embodiments may alloW a caller to use steps
`504 and 506 concurrently in the same IM session, may alloW
`the user to select betWeen step 504 and 506, or may
`automatically determine Which of steps 504 and 506 are
`used based on the capabilities of the message recipient. In
`either case, the messaged buddy may not be available, e.g.
`because they just signed off, etc. In this case, the voice portal
`310 can verbally tell the user that the message recipient Was
`unavailable.
`Use of step 504 requires that the underlying IM service
`offer a voice connection functionality. In this step, the voice
`portal 310 Will send one or more requests to the IM server
`308 to request initiation of a voice connection With the
`identi?ed recipient, e.g. Buddyl. Typically, this may cause
`the EM server 308 to send one or more requests to the
`remote user, e.g. on their computer or perhaps logged into
`the IM service via the voice portal 310 on a different
`telephone. The IM softWare being used by Buddyl may
`alloW Buddyl to select Whether to alloW the IM session, if
`approved, the connection Will then be established (possibly
`through the IM server 308) betWeen the IM softWare used by
`Buddyl (e.g. the prior art AIM(TM) client of FIG. 1) and the
`IM application executing on the voice portal 310. Once so
`connected, the caller and Buddyl can talk over that connec
`tion With the IM application on the voice portal 310 trans
`lating the telephone audio into the IM voice protocol format
`(Which may be a standard VoIP format) for transmission to
`the remote IM softWare client (and performing the reverse
`translation on received audio).
`Step 506 in contrast is may used When the recipient cannot
`(or Will not) accept, or their IM softWare or connection
`cannot support, voice communications. In this con?guration,
`the underlying IM service text formats are used to send
`messages. Messages received from the remote user can be
`played back using text-to-speech (TTS), or similar technolo
`gies. Outgoing messages may be entered using speech
`recognition (limited quality for unrestricted natural language
`utterances With present generation voice recognition
`systems),