(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2007/0043687 A1
`(43) Pub. Date:
`Feb. 22, 2007
`Bodart et al.
`
`US 2007.0043687A1
`
`ASSISTANT
`VIRTUAL
`
`(52) U.S. Cl. .................................................................. 707/1
`
`(54)
`(75)
`
`(73)
`(21)
`(22)
`
`Inventors:
`
`Andrew J. Bodart, New York, NY
`(US); Chris Ernest Condon, Jesery
`City, NJ (US)
`Correspondence Address:
`BANNER & WITCOFF, LTD.
`ATTORNEYS FOR CLIENT NO. O05222
`10 S. WACKER DRIVE, 30TH FLOOR
`CHICAGO, IL 60606 (US)
`Assignee:
`Accenture LLP, Palo Alto, CA (US)
`Appl. No.:
`11/206,829
`Aug. 19, 2005
`
`Filed:
`
`(57)
`
`ABSTRACT
`
`Systems and methods for assisting a user with a variety of
`tasks are provided. A virtual assistant has access to a user's
`contacts, calendar, and location. The virtual assistant also is
`able to access information about weather, traffic, and mass
`transit, and is able to adjust the time of for alerting a user
`about an upcoming appointment. The virtual assistant also
`has a rules engine enabling a user to create rules for handling
`incoming calls and instant messages, rerouting calls based
`on their caller identification. The virtual assistant also has a
`query engine enabling a user to find a document and to work
`with it, including sending it to a contact in the user's address
`book. Interfaces to virtual assistant may include installed
`software client, web browser, SMS/instant message, as well
`as an interactive voice response system.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`G06F 17/30
`
`(2006.01)
`
`131
`
`c
`
`133
`
`85
`
`U101
`
`135
`
`e
`Go
`
`132
`
`an
`S- t s
`
`(SN
`VV
`
`134
`
`hus
`
`101
`
`FUNCTIONALITY 110
`RULES ENGINE
`111
`MESSAGE ICALL ROUTING
`112
`SPEECH PROCESSING
`113
`QUERY & RESPONSE
`114
`
`CALENDAR -
`121
`RULES
`122
`FILES
`123
`ACCOUNTS
`124
`
`USER INTERFACES 140
`
`
`
`141
`
`147
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 1 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 1 of 7
`
`US 2007/0043687 A1
`
`
`
`101
`
`
`
`USER DATA 120
`
`CALENDAR -
`
`- CONTACTS -
`
`141
`
`147
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 2 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 2 of 7
`
`US 2007/0043687 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`CALL
`ROUTING
`202
`
`MESSAGE
`ROUTING
`214
`
`SPEECH
`AUDIO
`INTERFACE PROCESSING
`203
`204
`
`INSTANT
`MESSAGE
`205
`WEB
`SERVER
`206
`
`SOFTWARE
`CLIENT
`207
`
`IPTV
`INTERFACE
`208
`
`OTHER USER
`INTERFACES
`209
`
`PERSONAL
`NFORMATION
`MANAGER
`212
`
`
`
`VIRTUAL
`ASSISTANT
`101
`
`WEB
`SERVICES
`211
`
`
`
`213
`
`RULES
`ENGINE
`111
`
`FILES, PREFS
`ACCOUNTS
`210
`
`OUERY
`ENGINE
`114
`
`FIG. 2
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 3 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 3 of 7
`
`US 2007/0043687 A1
`
`FIG. 3
`
`VIRTUAL
`ASSISTANT
`101
`
`
`
`
`
`
`
`
`
`START
`
`DETERMINE
`NITIAL
`ALARM TIME
`
`404
`
`ADJUST
`ALARM TIME
`FOR CALENDAR
`
`FIG. 4
`
`402
`
`
`
`405
`
`RECEIVE
`CONTEXT
`INFORMATION
`
`403
`
`ADJUST
`ALARM TIME
`FOR TIME ZONE
`
`
`
`
`
`
`
`RAVEL
`INVOLVED?
`
`
`
`
`
`
`
`
`
`ADJUST
`ALARM TIME
`FOR TRAFFIC,
`WEATHER, AND
`TRANSIT
`
`407
`
`TRIGGER
`ALARMAT
`ADJUSTED TIME
`
`
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 4 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 4 of 7
`
`US 2007/0043687 A1
`
`
`
`VIRTUAL
`ASSISTANT
`101
`
`201
`
`
`
`RECEIVE
`INCOMING
`COMMUNICATION
`
`DETERMINE
`CONTACT
`PRIORITY
`
`PRIORITY?
`
`ROUTE TO
`APPROPRIATE
`DEVICE
`
`ALLOW TO
`LEAVE MESSAGE
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 5 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 5 of 7
`
`US 2007/0043687 A1
`
`
`
`
`
`
`
`
`
`VIRTUAL
`ASSISTANT
`101
`
`215
`
`201
`
`
`
`
`
`
`
`
`
`RECEIVE
`SEARCH
`COMMAND
`
`SEARCH
`FOR FILE
`
`
`
`
`
`
`
`FIG. 8
`
`ALERT
`FILE SENT
`
`2
`
`705
`
`ALERT
`FILE FOUND
`
`RECEIVE
`SEND
`COMMAND
`
`SEND
`FILE TO
`CONTACT
`
`
`
`ALERT
`FILE NOT
`FOUND
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 6 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 6 of 7
`
`US 2007/0043687 A1
`
`143
`
`Y
`
`201
`
`NY
`
`903
`
`FIG. 9
`
`VIRTUAL
`ASSISTANT
`101
`
`& S ASE
`A
`7,"
`
`4N
`
`as
`& 142
`
`F.G. 10
`
`2O1
`
`
`
`VIRTUAL
`ASSISTANT
`101
`
`
`
`
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 7 of 17
`
`

`

`Patent Application Publication Feb. 22, 2007 Sheet 7 of 7
`
`US 2007/0043687 A1
`
`
`
`PROCESSOR
`1202
`
`MEMORY
`1203
`
`NETWORK
`ADAPTER
`1205
`
`FIG. 12
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 8 of 17
`
`

`

`US 2007/0043687 A1
`
`Feb. 22, 2007
`
`VIRTUAL ASSISTANT
`
`FIELD OF THE INVENTION
`0001. The invention relates generally to personal produc
`tivity systems. More particularly, the invention provides for
`the integration of personal and public information to enable
`more productive individuals by automating routine tasks.
`
`BACKGROUND
`0002 Personal productivity software has helped to
`streamline and simplify the role of information workers.
`Beginning with basic email clients, productivity Software
`has grown to include a variety of other "desktop' applica
`tions, replacing paper calendars, rolodexes, and task lists
`with their software equivalents. Hybrid programs sometimes
`referred to as personal information managers (PIMs) have
`Succeeded somewhat in combining these disparate programs
`into a single interface. Not only are such applications able to
`track appointments, to do’s, contacts, and so forth, but they
`can combine the functions, such that setting up a meeting
`merely requires adding an appointment to your calendar and
`adding contacts to the appointment. Some applications have
`taken personal information managers a step further, enabling
`new interface methods. Such as having a user's email read to
`her by phone.
`0003 Having all this relevant information available in
`one place may have enhanced user productivity, but these
`PIMs have failed to take full advantage of the information.
`For example, when a user creates a new appointment, she
`must still discern how long of a lead time will be needed for
`a reminder, or provide one default value that is used for all
`reminders. Furthermore, if the user is not at her desk when
`the reminder is triggered, then she may forget the appoint
`ment, and the reminder is wasted. Alternatively, when mul
`tiple devices are synchronized, duplicative reminders may
`be unnecessarily sent, e.g., to a user's computer PIM (e.g.,
`Outlook(R) brand PIM), to a user's PDA PIM (e.g.,
`GoodLinkR) brand PIM), and to a user's watch PIM (e.g.,
`Microsoft Smart Watch). When a user receives an incoming
`message, she must decide whether to take the call or respond
`to the instant message based on whether she recognizes the
`identifying phone number or screen name. Such little deci
`sions and annoyances can be time consuming. Ultimately,
`PIMs and their users do not take full advantage of the
`information available to them to further enhance productiv
`1ty.
`
`SUMMARY
`0004 The following presents a simplified summary in
`order to provide a basic understanding of some aspects of the
`invention. The summary is not an extensive overview of the
`invention. It is neither intended to identify key or critical
`elements of the invention nor to delineate the scope of the
`invention. The following Summary merely presents some
`concepts of the invention in a simplified form as a prelude
`to the more detailed description below.
`0005. A first illustrative embodiment may be directed to
`a system having an information sources Software module for
`receiving data from a plurality of information providers. The
`system may also include a user interfaces software module
`for receiving input from and providing output to a plurality
`of heterogeneous user devices. A user data module may store
`
`data about a specific user, where the data about the specific
`user includes calendar appointment information, contact
`information, rules information, and preferences information.
`A functionality module may control overall operation of the
`system, and may include Submodules Such as a rules engine
`for analyzing and executing rules based on the rules infor
`mation, and a real-time communication component. The
`real-time communication component may direct an incom
`ing communication to the user. The routing may be deter
`mined at least in part based on a recognized contact status as
`determined by a comparison of a call identifier with the
`contact information, and optionally at least in part based on
`information received from one or more of the plurality of
`information providers via the information sources software
`module. The recognized contact status may recognize the
`caller as a human caller or an automated caller, such as a
`PIM associated with the user.
`
`OVERVIEW OF THE FIGURES
`0006. A more complete understanding of the present
`invention and the advantages thereof may be acquired by
`referring to the following description in consideration of the
`accompanying drawings, in which like reference numbers
`indicate like features, and wherein:
`0007 FIGS. 1 and 2 depict components of a virtual
`assistant according to one or more illustrative aspects of the
`invention;
`0008 FIG. 3 depicts a virtual assistant controlled alarm
`according to one or more illustrative aspects of the inven
`tion;
`0009 FIG. 4 is a flow chart illustrating a method for
`adjusting an alarm according to one or more illustrative
`aspects of the invention;
`0010 FIG. 5 depicts communication routing using a
`virtual assistant according to one or more illustrative aspects
`of the invention;
`0011 FIG. 6 is a flow chart illustrating a method for
`routing an incoming communication according to one or
`more illustrative aspects of the invention;
`0012 FIG. 7 depicts forwarding a file using a virtual
`assistant according to one or more illustrative aspects of the
`invention;
`0013 FIG. 8 is a flow chart illustrating a method for
`finding and forwarding a file according to one or more
`illustrative aspects of the invention;
`0014 FIG. 9 depicts controlling a virtual assistant using
`a remote controlled television according to one or more
`illustrative aspects of the invention;
`0015 FIG. 10 depicts sending data to an automobile
`using a virtual assistant according to one or more illustrative
`aspects of the invention;
`0016 FIG. 11 depicts controlling functions within a
`house according to one or more illustrative aspects of the
`invention; and
`0017 FIG. 12 is an illustrative operating environment in
`which one or more illustrative aspects of the invention may
`be implemented.
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 9 of 17
`
`

`

`US 2007/0043687 A1
`
`Feb. 22, 2007
`
`DETAILED DESCRIPTION
`0018. In the following description of the various embodi
`ments, reference is made to the accompanying drawings,
`which form a part hereof, and in which is shown by way of
`illustration various embodiments in which the invention
`may be practiced. It is to be understood that other embodi
`ments may be utilized and structural and functional modi
`fications may be made without departing from the scope and
`spirit of the present invention.
`0019. Throughout this description, various components
`and functional blocks are described as being connected to or
`in communication with each other. It should be understood
`that these components and functional blocks may be either
`directly or indirectly connected and may be in communica
`tion through any one of many methods, including wireless
`and/or wired technologies.
`0020 FIG. 1 depicts one possible set of components of a
`virtual assistant 101 according to one or more aspects of the
`invention. The term virtual assistant 101 is used to describe
`Software designed to simplify, streamline and enhance the
`often repetitive activities which occupy the daily lives of
`many people. Such software may interact with a user via any
`number of interfaces 140. Furthermore, such software is able
`to retrieve data from a variety of information sources 130, in
`part to augment its ability to make decisions on the user's
`behalf. Virtual assistant 101 software further may include
`the ability to access user data 120, combined with technical
`functionality 110, to assist the user and make the most
`accurate decisions possible. The functionality depicted in
`FIG. 1 need not necessarily be part of a single device or
`system. Virtual assistant 101 functions may be performed by
`multiple devices in communication with each other.
`0021. User data 120 may be available to virtual assistant
`101 either through its own user interface and data storage, or
`by electronic interface with the data and functions of other
`existing Software packages. For example, user calendar 121
`may be a component built into virtual assistant 101, or it may
`be a separate program (e.g., Microsoft Outlooks) which
`stores a user's calendar information and with which virtual
`assistant 120 can interface. Other user data may include
`contact information 125, the equivalent of a software rolo
`dex, where users can store contact email addresses and
`phone numbers, along with other information. User data 120
`may include a set of rules for use with rules engine 111,
`setting forth the logic used by virtual assistant to aid the user.
`In addition, there may be preference 126 data for individual
`application or operating system features. These preferences
`126 may include preference settings for the virtual assistant
`Software, or may include preferences for any other number
`of Software programs or websites.
`0022. User data 120 may also include a collection of user
`files 123, possibly including documents, presentations, pho
`tographs, web pages, and so forth. These files may be
`designated by a user, or may include all the files on a local
`computer system. Furthermore, user data 120 may include
`the emails 127 that a user has received or sent. Accounts 124
`may include personal financial account information for the
`user, possibly including bank login information. Accounts
`124 may also include information for other types of user
`accounts, including email passwords, website passwords,
`and so forth. Finally, user data 120 may include any voice
`mail messages 128 left for or sent by a user.
`
`0023. As discussed, each of these types of user data 120
`may be accessed and stored directly using specialized soft
`ware components (e.g., interfaces and/or objects) which may
`be a part of virtual assistant 101. Alternatively, different user
`data 120 may be manipulated or created using software
`which is not a part of virtual assistant 101, and then be
`Subsequently accessed by virtual assistant 101. Accessing
`data from other applications may require interfacing directly
`with another piece of Software (e.g., through an application
`programming interface or API). It also may require access
`ing the data directly through files stored in a memory,
`whether volatile (e.g., dynamic random access memory or
`DRAM) or non-volatile (e.g., a hard drive). The types of
`user data 120 displayed in the figure is not exhaustive and
`other data may be made available to virtual assistant 101.
`0024 Rules engine 111 is one portion of the functionality
`found in virtual assistant 101. A rules engine may be
`compared with a software compiler which translates user
`input (i.e. computer source code) into a series of instructions
`which a computer processor can understand (i.e. computer
`object code). Rules engine 111 takes user-created rules,
`which may be represented as a series of textual if-then-else
`statements, and translates them into a series of actions. Like
`computer Source code, the rules created for rules engine 111
`may be portable and translatable on multiple computer
`platforms. Rules engine 111 may additionally infer new
`rules based on the rules already in place.
`0025 Message & call routing 112 is another portion of
`the functionality found in virtual assistant 101. Technologies
`such as voice over internet protocol (VoIP) may enable call
`routing functionality 112, where telephone conversations are
`converted to binary data which can be routed via a computer
`network Such as the Internet. Message & call routing 112
`may enable virtual assistant 101 to intercept incoming
`telephone calls, instant messages, and other real time com
`munication requests. Based on the rules set forth for rules
`engine 111, calls may be selectively routed to a user based
`on the current context of date, time, user location, and so
`forth.
`0026 Speech processing 113 functionality can enable
`virtual assistant 101 to comprehend spoken language com
`mands. Interactive voice response (IVR) technologies may
`enable speech processing 113 abilities for virtual assistant
`101. Using this functionality, a remote user can call into his
`or her virtual assistant and provide Voice commands. These
`can be interpreted into commands which control the func
`tioning of virtual assistant 101.
`0027 Query and response 114 abilities may further
`enhance virtual assistant 101. Using a keyboard or speech
`processing, a user may query virtual assistant with regard to
`various aspects of user data 120 and/or information sources
`130. For example, a user may request that virtual assistant
`find a particular contact, by typing or saying, "Find Contact
`John Doe.” Based on the result of a query, virtual assistant
`101 may provide a response (e.g., whether the contact was
`found) as an audible or visual message. Similar queries may
`search user files 123, emails 127, calendar 121 appoint
`ments, and so forth. Additional functionality 110 may be
`implemented as a part of virtual assistant 101, including for
`example, the ability to initiate telephone calls and fax calls.
`0028. In addition to utilizing user data 120, virtual assis
`tant 101 may receive information from various sources 130
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 10 of 17
`
`

`

`US 2007/0043687 A1
`
`Feb. 22, 2007
`
`using one or more computer networks. Each information
`source may provide different specialized data for use by
`query & response 114, rules engine 111, and other software
`components. These information sources 130 may reside on
`computer servers located within the same entity housing
`virtual assistant 101, or they may reside on remote networks,
`interconnected for example, via the Internet. Information
`may be pushed by the information sources 110 to virtual
`assistant 101, or it may be received in response to a request.
`The format used to send information may vary by informa
`tion source, or sources may use a common format, Such as
`extensible markup language (XML) or really simple syndi
`cation (RSS).
`0029. Among information sources 130, individual
`Sources of information may include date & time information
`131, weather information 132, user geographic location
`information 133, road traffic information 134, mass transit
`information 135, and finance information 136. Each of these
`information Sources may supply timely information about
`their respective categories of data. Virtual assistant 101
`receiving information from these sources 130 may utilize the
`information in combination with user data 120 and rules
`engine 111 to make decisions on behalf of a user. Examples
`of these types of decisions are set forth in some detail below.
`Other information sources may also or alternatively be used.
`0030. Other components of virtual assistant 101 include
`one or more user interfaces 140 which allow a user to
`interact with virtual assistant 101. These may include a
`home interface 141 (e.g., a home wired for digital control),
`an auto interface 142 (e.g., a navigation system), interactive
`television 143, computer 144, cell phone 145 and/or per
`Sonal digital assistant (not shown), a conventional telephone
`146, and even an alarm clock 147. Each interface may be
`directly associated with the equipment within which virtual
`assistant 101 is housed, or may be remotely located and in
`communication with virtual assistant 101 using a computer
`network or other electronic communication scheme. Via
`user interfaces 140, the virtual assistant 101 can communi
`cate with many different types of heterogeneous user
`devices, and the virtual assistant is not limited to commu
`nication with a homogenous set of devices. Other interfaces
`may also or alternatively be used.
`0031
`FIG. 2 depicts one possible component configura
`tion of a virtual assistant 101 according to one or more
`aspects of the invention. The configuration may include the
`same or similar components as those shown in FIG. 1. The
`functional blocks present one possible implementation of
`virtual assistant 101. Components may be removed, rear
`ranged, and other components may be added.
`0032 Here, virtual assistant 101 is comprised of a num
`ber of functional components which enable the overall
`functionality described. Each component may work in con
`cert directly with the other components, or virtual assistant
`101 may be developed using software so as to integrate the
`functions. For example, virtual assistant 101 may be prima
`rily written in the computer language Java, and designed to
`interact with and integrate the various components, each of
`which may also share a public interface using Java, or some
`interface definition language (IDL).
`0033) User 201 may interact with virtual assistant 101
`through one of many interfaces, as previously described.
`Interfaces may require Some combination of visual, touch, or
`
`audible communications between user 201 and virtual assis
`tant 101. For example, a user may control virtual assistant
`101 using an audio interface 203 using a conventional
`telephone, a cell phone, or even a microphone and speaker
`attached to a computer. A user may use push button tones or
`another audio electronic interface to make selections from a
`menu. A user may also use natural language and/or spoken
`commands via audio interface 203. Speech processing 204
`may then process spoken commands and translate them into
`text or another format which virtual assistant 101 can
`understand. Software components or systems which perform
`Such speech processing may be referred to as interactive
`voice response (IVR) systems.
`0034) A visual interface may include those provided by
`instant messenger 205, web server 206, software client 207,
`or any other electronic interface. User 201 may interact
`using a computer, cell phone, or any other device which can
`receive and send commands. For instant message interface
`205, a user may be able to send messages using an instant
`message client Such as America Online (AOL) Instant
`Messenger, Microsoft Network (MSN) Messenger, and so
`forth. Alternatively, using a cell phone, user 201 may send
`messages using Short Message Service (SMS), multimedia
`message service (MMS) or some other message standard.
`Commands may be sent to virtual assistant 101 and
`responses received by the user using the same interface.
`Using the example of a virtual assistant 101 implemented
`using Java, SMS messages may be received and sent using
`a Java SMS Software Development Kit (SDK), or another
`set of software classes which ease the development process
`of SMS communications. Other programming languages
`may have similar SDKs to enable easy programming with
`SMS.
`0035) A web server 206 may supply an interface from
`virtual assistant 101 to any computer, cell phone, or other
`electronic device having a web browser via a computer
`network Such as the Internet. Such an interface may provide
`pages formatted using Hypertext Markup Language
`(HTML) or Some other markup language to provide infor
`mation to user 201, and also to receive information from the
`user. The HTML pages may be created using a standard
`programming language such as Java Servlets or Java Server
`Pages (JSPs), Microsoft .NET, PHP, and so forth.
`0036) An installed software client 207 may provide a
`more rich and customizable interface for user 201. Such an
`interface may be an add-on to an existing piece of software
`(e.g., Microsoft Outlook), or a standalone computer appli
`cation. Such an interface may be written using any number
`of programming languages, and may be installed on the
`same computer as virtual assistant 101 or communicate with
`the virtual assistant via a network connection.
`0037 Another alternative interface for virtual assistant
`101 is an Internet Protocol Televeision (IPTV) interface 208.
`IPTV is not a particular standard and rather is a term used
`to refer generally to interactive services delivered to users
`over television broadband networks. IPTV may require that
`the user have a separate set top box connected to a television,
`although the functionality may be included as part of a
`television, or other video component. User 201 may receive
`information from virtual assistant 101 through her televi
`Sion. Likewise, user 201 may access and control virtual
`assistant 101 using a television remote or other input device.
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 11 of 17
`
`

`

`US 2007/0043687 A1
`
`Feb. 22, 2007
`
`0038. Additional user interfaces 209 may interact with
`the devices mentioned, or with other devices and systems.
`For example, an interface may enable virtual assistant 101 to
`communicate with a home automation system 141 over an
`existing network connection. As another example, virtual
`assistant 101 may communicate using a wireless network
`with an automobile system 142 (e.g., a navigation system).
`Another example of a possible interface is virtual assistant
`101 communicating with a user by controlling an alarm
`clock 147 connected to a home network. For each interface,
`an existing networking standard (e.g., HTML, XML, SMS,
`etc.) may be customized to enable communication with the
`appropriate device, and using the appropriate Software calls.
`0.039
`Virtual assistant 101 includes a rules engine 111 as
`described above, capable of providing a user a simple way
`to automate the control of virtual assistant. One possible
`implementation of rules engine 111 is the use of the Java Jess
`application programming interface (API). Jess provides a
`generalized format for creating rules in a portable format,
`essentially a high-level declarative programming language.
`These rules can be run based on information from the
`various interfaces and information sources accessible by
`virtual assistant 101. The rules may be stored as user data
`and transported between multiple systems. Other types of
`rules engines may be used, each of which provides a way to
`make conclusions and inferences based on user data and
`contextual information.
`0040. A personal information manager (PIM) may form
`the basis of user data to be used by virtual assistant 101.
`Well-known PIMs include Microsoft Outlook, Microsoft
`Entourage, Lotus Notes, ACT, GoodLink, and so forth.
`Virtual assistant 101 may use the data associated with each
`of these well-known applications, or alternatively, virtual
`assistant may provide its own PIM functionality and store
`the data in a format of its choosing. The user data to which
`a PIM may provide access includes calendar appointments,
`address book contacts, to do list items, email messages,
`notes, and so forth. As opposed to one single application,
`each of these types of PIM data may alternatively be
`managed by individual applications (e.g., iCal. Address
`Book, and Mail on an Apple Computer). In either case,
`virtual assistant 101 may access each type of PIM data for
`use by user 201.
`0041 Web services 211 provide virtual assistant 101 a
`method of providing up-to-date contextual information for
`purposes of evaluating rules and responding to queries,
`among other uses. Web services may pull information from
`the Internet 213, including weather, traffic, mass transit, user
`location (e.g., via GPS in a user's cell phone), and so forth.
`This information may be provided using a simple request
`and response format, possibly polled on an hourly basis.
`Alternatively, the information may be requested on a “when
`needed' schedule, or the information may be pushed by
`outside servers when updated rather than awaiting a request.
`The data may be stored on a single server or on multiple
`servers throughout the Internet. In addition, virtual assistant
`101 may pull the same information from multiple sources in
`order to average or otherwise determine the most correct
`information. Information may be pulled using XML or RSS,
`or some other standard or non-standard data format. Alter
`natively, information may be “scraped from standard
`HTML web pages, where virtual assistant 101 may know
`exactly what part of a page contains the information needed.
`
`0042 Virtual assistant 101 may also access user files,
`preferences and account information 210 stored either
`locally on the same machine, or on remote servers and
`devices. Access may simply involve access to the item when
`a directory location or namespace is known, or it may
`include access to all the contents of all the items, possibly in
`the form of a complete index of item contents. Preferences
`may include application or operating system preferences,
`which may be transportable to other computers accessible to
`virtual assistant 101. Accounts may include passwords for
`specific websites, or possibly to bank accounts and the like.
`Information Such as account balances (when authorized)
`may be useful in the types of decisions that virtual assistant
`101 may make.
`0043 Virtual assistant 101 may include query engine
`114, which may or may not work in conjunction with rules
`engine 111. Query engine 114 provides virtual assistant a
`method for finding files, items, data, and so forth within the
`realm of information available, including data gathered via
`web services 211. Query engine 114 may search an index
`structure for text matching a particular string or combination
`of strings. Query engine 114 may also perform natural
`language queries, which do not necessarily require a specific
`Syntax or lexicon of commands. The responses provided by
`query engine may similarly be in the form of natural
`language rather than structured and limited. Speech process
`ing 204 and audio interface 203 may work in conjunction
`with query engine 114 to provide a spoken input ability for
`queries. Furthermore, responses may be provided using
`audio interface 203 to produce a spoken output.
`0044) Virtual assistant 101 may include the ability to
`receive and direct calls and messages directed to user 201.
`Such ability includes call routing 202 where an incoming
`telephone call from contact 215 is routed based on the user's
`current status as well as whether or not the person making
`the call is recognized. Likewise, incoming messages includ
`ing instant messages and/or SMS messages, and even email
`messages, may similarly be routed using message routing
`214. By routing calls and messages, user 201 does not have
`to screen her calls. Rather, virtual assistant 101 is able to
`recognize an identity of the sender (via caller ID or some
`other identification method) and decide whether or not to
`route the call to voice mail 128 or some other system. Using
`speech processing 204, virtual assistant 101 may further
`translate a voice mail into a text message to be stored or
`forwarded to user 201.
`0045 FIG. 3 depicts an alarm controlled by virtual assis
`tant 101 according to one or more aspects of the invention.
`Although a conventional bedside alarm clock 302 is
`depicted here, any sort of audible, visual, or other alarm may
`be utilized in this fashion, including a computer alarm, a cell
`phone alarm, and so forth. Virtual assistant 101 may be
`incorporated into alarm 302, or it may be remotely located,
`utilizing a network or direct data connection in order to
`inform the alarm when to alert user 201. Virtual assistant 101
`may have access to the location of user 302 via a cell phone
`or other location sensor. By knowing the user's location,
`virtual assistant 101 can automatically adjust the time Zone
`and, if necessary, any calendar appointments for the user.
`Also, by having access to the calendar of user 201, virtual
`assistant can adjust an alarm (e.g., a wake-up alarm) based
`on any impending appointments.
`
`Ex. 1005
`CISCO SYSTEMS, INC. / Page 12 of 17
`
`

`

`US 2007/0043687 A1
`
`Feb. 22, 2007
`
`0046) If virtual assistant 101 not only knows the calendar
`of user 201, but also the current location of user 201, and the
`location of the next appointment on the calendar, virtual
`assistant may calculate a length of time for user 201 to travel
`between her current location and her destination, and also
`send the alarm to the proper location (e.g., when the user
`owns multiple devices capable of producing an alarm). By
`calculating this time (e.g., by using a