Parus Exhibit 2010
`Google LLC, et al. v. Parus Holdings, Inc.
`IPR2020-00846
`Page 1 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 1 of 12
`
`6,157,705
`
`Be
`
`
`
`Parus Exhibit 2010, Page 2 of 25
`
`Parus Exhibit 2010, Page 2 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 2 of 12
`
`6,157,705
`
`Network Dvr 22
`
`OS 20
`
`
`
`
`
`
`
`
`
`
`
`IVR App 42
`
`
`Resources
`
`Web App 34
`
`HTTP Server 32
`
`Os 40
`
`
`
`Speech
`Recognizer
`
`16
`
`
`
`
`
`
`
`12
`
`Identifier/URL
`Table 50
`
`
`
`Fig, 10
`
`Parus Exhibit 2010, Page 3 of 25
`
`Parus Exhibit 2010, Page 3 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 3 of 12
`
`6,157,705
`
`Oeb
`
`ANWdNOO
`
`401450
`
`ANOHdSTAL
`
`MYOMLAN
`
`onand
`
`zl
`
`NIVAI
`
`AOVYOLS
`
`SIAC
`
`901
`
`AYOWSW
`
`ght
`
`YOSIAYSdNS
`
`bh
`
`TWNIWAL
`
`JOV4Y3LNI
`
`Lb
`
`MYOMLAN
`
`SOVIYSLNI
`
`Parus Exhibit 2010, Page 4 of 25
`
`Parus Exhibit 2010, Page 4 of 25
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 4 of 12
`
`6,157,705
`
`Recognizer
`
`Parus Exhibit 2010, Page 5 of 25
`
`Parus Exhibit 2010, Page 5 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 5 of 12
`
`6,157,705
`
`
`Resources
`
`
`
`
`
`Telephony Server
`
`Web App 34
`
`HTTP Server 32
`
`Voice File 52
`
`
`
`
`
`
`
`
`11
`
`OS 30
`
`
`
`
`
`
`
`
`Identifier/URL
`Speech
`Recognizer
`Table 50
`
`
`16
`
`Fig. 298
`
`Parus Exhibit 2010, Page 6 of 25
`
`Parus Exhibit 2010, Page 6 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 6 of 12
`
`6,157,705
`
`WELCOMETOE*TRADE
`
`oO
`Lu
`LL
`
`——L
`
`u
`oO
`
`Hig.8A
`
`Parus Exhibit 2010, Page 7 of 25
`
`z ow
`
`a
`wv
`Lu
`wa
`ja
`Lu
`
`e=L
`
`u
`Lu
`a“
`<<
`Lu
`—!
`ou
`
`Parus Exhibit 2010, Page 7 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 7 of 12
`
`6,157,705
`
`[EVTRADE |
`
`202
`Welcorne to the
`
`206
`
`smarterwayto invest /
`
`7 212b
`Customerlog on
`
`|__—Canada|
`‘Join
`C7 Canadian?
`Need Help?
`,
`Click Here
`
`E*TRADE
`
`Account Benefits
`Investment Tools
`Commissions
`Margin Rates
`Security
`Customer Service
`FAQs
`208
`
`Open an
`accoun
`
`Online Application
`Request by Mail
`Open an IRA
`
`The E*TRADE Demo
`Free Quotes
`Play the Stock Game
`
`today
`
`212a
`Open an E*TRADEaccount.If you make
`your own investment
`decisions,it's time to
`get empowered with E*TRADE's Web
`trading and portfolio management system.
`We'vesetthe standard for unparalleled
`innovation, control, convenience,service
`and value.
`
`
` 214a
`;
`a
`
`in the news
`
`214b
`Find out whére we're making headlines.
`
`Current News
`R!, Esquire, May 1997.
`R
`E*TRADEis profiled as one of the companies
`"bringing aboutnothing less than the
`democratization of Wall Street."
`
`E*TRADE's CEO,Christos Cotsakos, is named
`oneof the top 10 visionary CEOsby
`Communications Week.
`
`Click here for access
`to the smarter way to
`investin Canadal
`218
`
`In just a few clicks,
`youl learn how easy
`itis to place orders,
`track the markets and
`manage your
`portfolio online.
`Do your homework
`
`compare,
`
`ogageLae} Onceyou
`you | see
`why we've become a
`leaderin online
`investing.
`Customer quotes
`
`Getthe real story.
`Our customers share
`their experiences
`with E*TRADE.
`
`The Company
`Press Releases
`Job Opportunities
`Wo
`210 {quotes | demo|commissions |contact us | help
`204
`©Copyright 1996 E* TRADESecurities, Inc. All Rights Reserved.
`
`
`
`200
`
`220
`
`222
`
`224
`
`226
`
`228
`
`Fig. 86
`
`Parus Exhibit 2010, Page 8 of 25
`
`Parus Exhibit 2010, Page 8 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 8 of 12
`
`6,157,705
`
`E*TRADE
`
`|°
`
`visitor
`center
`
`230x
`
`Customerlog on
`
`E*TRADE's Free Quotes
`
`Need Help?
`
`E*TRADE
`
`Account Benefits
`Investment Tools
`Commissions
`Margin Rates
`Security
`Customer Service
`FAQs
`
`|Openan___|
`accoun
`
`Online Application
`Request by Mail
`Open an IRA
`
`The E*TRADE Demo
`Free Quotes
`Play the Stock Game
`
`The Company
`Press Releases
`Job Opportunities
`WV
`204
`
`»
`get
`PointCast £
`
`Quotes are delayed 20 minutes during market hours.
`
`Quotes
`For a free quote, please enter the symbol:
`Lex
`
`232
`
`234
`
`236
`
`Symbols
`To find a symbol, please enter the name andtype below:
`
`[| Sek)nt
`
`238
`
`234
`
`239
`
`Learn more about E'TRADEbytrying our Trading Demo.
`
`210
`
`quotes | demo |commissions |contact us |help
`
`©Copyright 1996 E * TRADE Securities, Inc. All Rights Reserved.
`
`Fig. 86
`
`Parus Exhibit 2010, Page 9 of 25
`
`Parus Exhibit 2010, Page 9 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 9 of 12
`
`6,157,705
`
`E* TRADE
`
`®
`
`ws
`
`center
`
`240a
`
`Take advantage of E*TRADE'S low commissionsandsimplefee structure.
`For more information, see our full commission schedule. You mayalso
`wantourtrading demo.
`
`242
`
`BANKAMERICA CORP
`246
`248
`
`Price: [71 16
`
`Change:| |Open:|71 15/164-1/8}
`
`
`
`
`All quotes are delayed 20 minutes during market hours.
`52-Week High: 07/25/97 72 3/4
`52-Week Low: 09/03/96 37 7/8
`
`Return to the Free Quotes page.
`
`
`quotes | demo| commissions |contact us |help
`©copyright 1996 E *TRADESecurities, Inc. All Rights Reserved.
`
`210
`
`Fig. 83D
`
`Parus Exhibit 2010, Page 10 of 25
`
`Parus Exhibit 2010, Page 10 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 10 of 12
`
`6,157,705
`
`E+ TRADE} Stock,
`
`®
`
`260a
`
`of shares:
`
`symbol:
`
`Account: 962
`Vv Transaction:
`@ Buy
`© Sell
`
`© Sell Short
`O Buy to Cover
`
`Switch Accounts
`Vv Number
`Vv Stock
`~ \sismsmbol
`
`For help, click on any V
`¥ Price:
`O Limit:
`
`© Market
`
`© Stop: |
`O Stop Limit: [—F
`
`
`¥ Term:|Good forday lw] Optional: W Allornone (300shares and
`
`Trading password: [___|_ Review theorder carefully
`[Cancel
`
`over
`
`To place mutual fund orders, use our new EXTRADE Mutual Fund Center.
`
`main
`
`x
`
`the markets
`
`trading
`
`your account
`
`Accounts Balances
`Enter Stocks Order
`Investor Tools
`mutual fund center
`
`
`
`messaqesRecordsMarkets ata Glance Open Stocks Order Accounts
`9
`Quotes/News/Charts
`Options Trading
`Portfolio Summary
`Stock Watch
`Transaction History
`Switch Accounts
`
`help
`
`
`
` © Copyright 1997 E*TRADE Securities, Inc. All Rights Reserved.
`
`Parus Exhibit 2010, Page 11 of 25
`
`Parus Exhibit 2010, Page 11 of 25
`
`

`

`U.S. Patent
`
`Dec.5, 2000
`
`Sheet 11 of 12
`
`6,157,705
`
`400
`
`Establish Data
`Communication
`Channel
`
`
`
`402
`
`Establish Voice
`Communication
`Channel
`
`404
`
`
`
`Synchronize Data and
`Voice Communication
`Channels
`
`406
`
`Receive Voice
`Command
`
`408
`
`440
`
`412
`
`Associate Voice
`Commandwith Natural
`Language Phrase
`
`
`
`Associate Natural
`Language Phrase
`with Network Address
`
`Retrieve Document
`Identified By Network
`Address
`
`414
`
`416
`
`Transmit Document over
`Data Communication
`
`Channel
`
`(coe)
`418
`
`Transmit Response over
`Voice Communication
`Channel
`
`Siy. 4
`
`Parus Exhibit 2010, Page 12 of 25
`
`Parus Exhibit 2010, Page 12 of 25
`
`

`

`U.S. Patent
`
`Dec. 5, 2000
`
`Sheet 12 of 12
`
`6,157,705
`
`8d
`
`0€s
`
`YAAYAS
`
`JADVYOLS
`
`SJOIAAC
`
`OLS
`
`
`
`S
`n
`
`eS
`
`1SOH
`
`veS
`
`0¢S
`
`YOMLIN
`WO01
`MYOMLAN
`
`ANU
`
`NOILLVOINNAIWOD
`
`JOVAYSLNI
`
`v0
`
`YOSS300Ud
`
`AV1dSId
`
`cls
`
`LAdNI
`
`SOAS
`
`vhs
`
`YOsHNo
`
`TOXLNOOD
`
`StS
`
`Parus Exhibit 2010, Page 13 of 25
`
`Parus Exhibit 2010, Page 13 of 25
`
`
`
`
`
`
`

`

`6,157,705
`
`1
`VOICE CONTROL OF A SERVER
`
`FIELD OF THE INVENTION
`
`invention generally relates to computer
`The present
`systems,and relates in particular to mechanismsthat provide
`voice control of a server.
`
`BACKGROUND OF THE INVENTION
`
`Avast amountof information is available using computer
`servers. Servers, mainframe computers, and other computer
`storage devices on networks provide a warehouse of infor-
`mation and services. However, accessing information and
`initiating processes or services on such servers is difficult
`using presently available mechanisms. In particular, mul-
`tiple commands using a keyboard or mouse are usually
`required for a user to navigate through the file directory
`structure of a server to locate desired information. The data
`processing field has failed to develop systems that provide
`voice control of a remote server from a local point, so that
`a local user can command the remote server by voice to
`display visual information at the local point or carry out a
`desired process.Past systems provide incomplete solutions
`to this problem. For example, interactive voice response
`(IVR) systemsare used to deliver stored information over a
`telephone line to an end user. [VR systems are specialized
`computer systems that have a processor, a storage device
`such as a hard disk, hardware for interfacing the processor
`to the public switched telephone network (PSTN) and an
`IVR application program that runs on the processor.
`Generally, the end user connects to the IVR system using a
`telephone. The end user takes the telephone handset off hook
`and dials a pre-determined telephone numberthat identifies
`the IVR system. The telephone call is delivered over the
`PSTN to one of several trunk lines connected to the IVR
`system. The IVR system answersthe call by seizing one of
`the trunk lines and playing a pre-recorded greeting to the
`caller. Typically the greeting is a voice recording stored
`digitally on a storage device that provides the end user with
`a menu of processing options that can be selected using
`telephone dial pad keys. Some IVR systems include voice
`recognition software or processors, so that an end user can
`select system options by speaking a short word or phrase
`such as a number.
`
`Example IVR applications include automated receptionist
`services, various customer service or “help desk”
`applications, and airline reservations systems.
`IVR systems can be configured to carry out a voice
`dialogue with the end user. The dialogue comprises a series
`of questions from the IVR system and answers from the end
`user until the desired service is provided to the end user by
`the IVR system. However,
`IVR systems are unable to
`display visual information, such as pre-formatted text or
`graphics, or dynamically created custom graphic
`information, in conjunction with the dialogue. The absence
`of visual information from present IVR systems is a major
`limitation, and represents the loss of a powerful medium for
`conveying information.
`Client-server computer systems also provide an incom-
`plete solution. In a client-server system, a client at a local
`point is connected by a data connection to a server at a
`remote point. The client can be a computer or a combination
`of a computer and software running on the computer. The
`data connection can be a cable, a local area network (LAN),
`a wide area network, or another type of network. The data
`connection can be the global network, operating according
`to standard protocols, knownas the Internet. The server can
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`be a file server of the LAN,or a server notaffiliated with the
`client. For example,
`the server can be a server that
`is
`publicly accessible using anonymousfile transfer protocol
`(FTP) over the Internet. Using the Internet and certain wide
`area network technologies, a client can connectto, “log on”
`to, request and use a distant server.
`One popular technology enjoying wide use with the
`Internet is known as the World Wide Web. The World Wide
`Web enables a computer to locate a remote server using a
`server name in an agreed-upon format that is indexed at a
`central Domain Name Server (DNS). The local computer or
`client runs a browser program. Using the browser,the client
`locates the remote server using the DNS, and connects to the
`remote server. The client requests information from the
`server using a communication protocol called the Hypertext
`Transfer Protocol
`(HTTP), by providing a Uniform
`Resource Locator (URL) that uniquely identifies a page of
`information stored on the server. A URL is a form of network
`address that identifies the location of information stored in
`a network and represents a reference to a remote World Wide
`Web server, known as a website. The pages of information
`are files prepared in the Hypertext Markup Language
`(HTML). Thus, a Web client-server system can be used to
`request and display information stored on a remote server.
`URLsgenerally are formatted according to the following
`syntax:
`<protocol id>://{<server>} <second level domain><top
`level domain>{<directory>} {<file>}
`The <protocol id> identifies the transmission protocol to be
`used. For example, in the case of the Web it is <http>, and
`in the case of an anonymousfile transfer protocol transaction
`it is <ftp>. The <server> element is an optional server name
`such as <www.>. The <server> element mayalso identify a
`directory on a storage device of the Web server that contains
`HTMLdocuments. The <second level domain> elementis
`the name of the server domain as found in the DNStable,
`such as <etrade>. The <top level domain> elementidentifies
`the type of the second level domain, and must be an item
`selected from a finite set of globally recognized top level
`domains, such as “com,” “org,” “mil,” “edu,” “gov,” and
`others. The <directory> element is an optional name of a
`directory within the <server>, such as <DocumentRoot/>.
`The <file> element is an optional nameofa file, document
`or image to beretrieved, such as <Index.-html>. Thus, a URL
`serves as one type of a network address to locate a document
`anywhere in a network.
`However, client-server systems and World Wide Web
`systems cannot respond to voice commands and cannot
`deliver visual or graphic information that is coordinated with
`a voice dialogue. These systems also do not enable a user to
`select or initiate computation processes in coordination with
`a voice dialogue. World Wide Web systems can include
`panels or pages that are dynamically generated by the
`systems, and can include internal or external computation
`processes rather than static documents or pages. However,
`there is no wayto select such processesor locate them using
`voice interaction.
`
`In addition, when a user wishes to obtain specific infor-
`mation or activate a specific process using a Web server, but
`the URLof the information or process is unknown,the user
`must follow the hypertext links of numerousirrelevant pages
`until the desired information is located. Navigation through
`this control structure is clumsy.It is especially inconvenient
`at the slow data transfer speeds that presently characterize
`most use of the Web.
`One approach to these issues is to integrate speech
`recognition software in a computer program or computer
`
`Parus Exhibit 2010, Page 14 of 25
`
`Parus Exhibit 2010, Page 14 of 25
`
`

`

`6,157,705
`
`3
`remote from a server. The computer is connected through an
`interface to a microphone. The computer runs a speech
`recognition program that converts voice commandsreceived
`by the microphone into keystrokes or commands under-
`standable by the program. This is knownas “local control”
`or client-side speech recognition because recognition of
`voice commandsis carried out in a local computer. The local
`computer is separated from a server that stores Web pages
`and runs application programs that serve the client. The
`speech recognition controls only the program of the local
`computer, not
`the server or applications running in the
`server. However, such an approach has several disadvan-
`tages.
`First, high-quality speech recognition is expensive both in
`the commercial sense and in terms of computing resources
`and powerneeded to provide acceptable results. Currently,
`high-quality speech recognition software is in very limited
`use and is not generally available at a reasonable price to the
`vast majority of home or business computer users.
`Also, local control allows an end user to access only the
`information structure presented by the program currently
`running in the local computer. The voice commands are
`limited to the command set of the current program. Local
`control cannot provide flexible shortcuts through a Website,
`and cannot enable the Web site to identify, during a voice
`dialogue with the end user, suitable Web pagesto present to
`the end user.
`Thus, there is a need for a system that enables a local
`client to rapidly retrieve information from a remote server
`using voice commands.
`Thereis also a need for a system that enables a local client
`to carry out a voice dialogue with a remote server and
`receive or retrieve visual and graphic information that is
`coordinated with the voice dialogue.
`There is also a need for an arrangement with which voice
`commandsor a voice dialogue can be used to locate, select,
`activate or initiate a computing process or service that is
`available at the server; to locate information in a database,
`and to execute trades in a securities trading system.
`SUMMARYOF THE INVENTION
`
`These and other needs are fulfilled by the present
`invention, which comprises, in one embodiment, a method
`of controlling a remote server by a voice command issued
`from a location local to a client, comprising the steps of
`establishing a voice communication channel between the
`location local to said client and the remote server; estab-
`lishing a data communication channel associated with the
`voice communication channel between the client and the
`
`remote server; receiving the voice command by the voice
`communication channel; associating the voice command
`with a resource identifier; selecting a server resource based
`on the resource identifier; and delivering the resource from
`the remote server to the client by the data communication
`channel.
`
`One feature of the invention is establishing the voice
`communication channel integrated with the data communi-
`cation channel. Another feature is delivering a voice mes-
`sage over the voice communication channel in coordination
`with delivery of the server resource. Yet another feature is
`processing the voice command using a speech recognition
`process to recognize the natural language phrase.
`One aspect of this embodimentis associating the natural
`language phrase with a network address by performing the
`steps of: identifying the natural language phrase in a table of
`the remote server that maps natural language phrases to
`network addresses; and looking up the network address in
`
`4
`the table. A feature of this aspect is loading said document
`identified by the network address from a storage device
`coupled to the remote server.
`Another aspect of the invention is establishing a data
`communication channel between the client and the remote
`
`server configured to communicate data between the client
`and the remote server; and establishing a voice communi-
`cation channel between the client and the remote server
`
`10
`
`configured to communicate sound information including the
`voice command between the client and the remote server.
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`One feature of this aspect is establishing a telephone con-
`nection from the client to a voice recognizer that 1s coupled
`to the remote server. Another feature is establishing a
`telephone connection from the client to an interactive voice
`response (IVR) system coupled to the remote server. Still
`another feature is establishing an Internet telephony con-
`nection from the client to a voice recognizer that 1s coupled
`to the remote server.
`
`Still another aspect of the invention is recognizing the
`natural language phrase in the voice command at the IVR
`system;
`transmitting the natural
`language phrase to the
`remote server; and transmitting a voice response from the
`IVR system to the client.
`The invention also encompasses a computer system and a
`computer program product configured in accordance with
`the foregoing aspects and f,eatures.
`Additional objects, advantages and novel features of the
`invention will be set forth in part in the description which
`follows, and in part will become apparent to those skilled in
`the art upon examination of the following or may be learned
`by practice of the invention. The objects and advantages of
`the invention may be realized and attained by meansof the
`instrumentalities and combinations particularly pointed out
`in the appended claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is illustrated by way of example,
`and not by wayof limitation, in the figures of the accom-
`panying drawings and in whichlike reference numerals refer
`to similar elements and in which:
`
`FIG. 1A is a block diagram showing an embodiment of
`the invention.
`
`45
`
`FIG. 1B is a block diagram of a software architecture of
`the embodimentof FIG. 1A.
`
`FIG. 1C is a block diagram of hardware elements of an
`interactive voice response (IVR) system.
`FIG. 2A is a block diagram showing an alternate embodi-
`ment of the invention.
`
`50
`
`FIG. 2B is a block diagram of software architecture of the
`embodiment of FIG. 2A.
`
`FIG. 3A is a diagram of an exemplary screen display
`involved in coordinating data and voice communication
`channels.
`
`55
`
`FIG. 3B is a diagram of an exemplary screen display of
`a Websecurities brokerage application that can be used with
`the invention.
`
`60
`
`FIG. 3C is a diagram of an exemplary screen display that
`provides a securities quote entry facility.
`FIG. 3D is a diagram of an exemplary screen display
`showing a securities quote.
`FIG. 3E is a diagram of an order page in the brokerage
`application of FIGS. 3B-3D.
`FIG. 4 is a flow diagram of an embodimentof a process
`for voice control of a server.
`
`Parus Exhibit 2010, Page 15 of 25
`
`Parus Exhibit 2010, Page 15 of 25
`
`

`

`6,157,705
`
`5
`FIG. 5 is a block diagram of a computer system that can
`be used to operate the present invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`
`A method and apparatus for voice control of a server is
`described. In the following description, for the purposes of
`explanation, numerousspecific details are set forth in order
`to provide a thorough understanding of the present inven-
`tion. It will be apparent, however, to one skilled in the art
`that the present invention may be practiced without these
`specific details. In other instances, well-known structures
`and devices are shown in block diagram form in order to
`avoid unnecessarily obscuring the present invention.
`VOICE CONTROL OF A SERVER
`
`FIG. 1A is a block diagram of a system that provides voice
`control of a server. An end user 2 has a computer 4 and a
`telephone 10 located at a local point 1. The computer 4 is
`coupled through a network 6 to a server 8 at a remote point
`3. In one embodiment, the network6 is a local area network,
`and the server 8 is separated from the end user 2. In the
`preferred embodiment,
`the network 6 is the Internet,
`the
`server 8 is configured as a World Wide Webserver, and the
`remote point 3 is geographically remote from the local point
`1. The connection from the computer 4 to the server 8 is a
`data communication channel.
`
`The telephone 10 is coupled through the public switched
`telephone network (PSTN) 12 to an interactive voice
`response (IVR) system 14. The connection between the
`telephone 10 and the IVR system 14 is a voice communi-
`cation channel. The IVR system 14 is coupled to a speech
`recognizer 16 and to the server 8.
`In the preferred
`embodiment, the IVR system 14, the speech recognizer 16,
`and the server 8 are in the same location, althoughthisis not
`required. The speech recognizer 16, IVR system 14, and the
`server 8 can be connected over a network.
`
`FIG. 1B is a block diagram of the software architecture of
`the system of FIG. 1A. The client computer 4 runs an
`operating system (OS) 20. Exemplary operating systems
`include Windows, MacOS, and Solaris. In known manner,
`the OS 20 provides an interface to the hardware elements of
`the computer 4 that can be used by application programs;
`controls and supervises the use of system resources; and
`loads, supervises and controls the operation of application
`programs. A network driver program 22 runs on top of the
`OS 20 and underits control. The network driver program 22
`functions as an interface between the OS 20 and the network
`
`6. An example of a network driver program 22 is Novell
`NetWare. An example of a client computer 4 is a Compaq
`personal computer, a workstation such as a SPARCstation
`made by Sun Microsystems, Inc., or a network computer
`made by Network Computer,Inc.
`The client computer 4 also runs a browser 24. The
`browser 24 is an application program that can read HTML
`documents, interpret HTML code stored in the documents,
`and display visual information based on the interpretation of
`the HTMLcode on a display of the client computer 4. The
`user 2 can use computer 4 to command browser 24to control
`its display. Examples of suitable programsthat can be used
`for the browser 24 are Netscape Navigator® and Microsoft®
`Explorer.
`The server 8 runs an OS 30 that generally performs the
`same functions described above in connection with OS 20.
`The OS 30 may be the same as OS 20 or may bea different
`operating system. In the preferred embodiment, the OS 30 is
`
`10
`
`15
`
`25
`
`30
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`an operating system that is optimized to perform server
`functions, such as Microsoft Windows NT.
`Undercontrol of OS 30, the server 8 runs an HTTP server
`32. The HTTP server 32 is an application program that
`enables the server 8 to send and receive information using
`the HTTPprotocol. Examplesof available programsthat can
`be used as HTTPserver 32 include the Apache HTTPserver
`and Microsoft Internet Information Server. The server 8 may
`be a computer, or the combination of a computer and a
`suitable operating environment.
`The server 8 also runs a Web application 34. Resources 60
`are available in association with the server 8. “Resources”
`that are “available” on server 8 in this context means
`documents stored on server 8 or on a network device or
`system that is accessible through server 8, or computation
`functions or processes provided by or through server 8.
`“Resources”also is intended to encompass HTMLpagesor
`other electronic information generated dynamically or as
`output from a program. “Resources” is also intended to
`include databases accessible through a server; documents,
`programs, and data accessible using one or more hyperlinks
`from a first document; and client-server application
`programs, such as a securities trading system. Such
`resources may be physically stored on the server 8, or may
`be physically stored elsewhere and accessed by or using the
`server 8. For example, the Resources 60 are a set of HTML
`files stored on a hard disk of the server 8. Such documents
`
`are merely an example ofthe type offiles that can be stored
`in association with the server 8 and delivered to the com-
`
`puter 4 using the invention; other stored information also
`can be delivered. Alternatively, resources 60 can be pro-
`grams such as Java applets, ActiveX controls, or other
`processes, facilities or functions provided by server8.
`FIG. 1C is a block diagram of the IVR system 14. A bus
`102 acts as an internal data connection for all the elements
`of IVR system 14 and provides a central pathway for data
`and instructions. Processor 104 executes instructions of an
`
`operating system and application programs and controls
`overall operation of the system.
`Instructions, data, and
`stored programs are stored in one or more storage devices
`110, for example, in a hard disk drive or floppy disk drive.
`Anon-volatile memory 108 such as a ROM is usedto store
`bootstrap instructions and other constant
`information.
`Instructions and data for execution and use by processor 104
`are loaded from storage device 110 into a volatile local
`high-speed memory 106, such as RAM.
`A terminal interface 114 coupled to the bus 102 provides
`communications between processor 104 and an external
`display terminal, which may be another computer. A net-
`work interface 116 couples bus 102 to an external data
`network such as LAN. The processor 104 can load instruc-
`tions or data from devices coupled to the external data
`network.
`
`One or more port interfaces 112 connect bus 102 and
`processor 104 to telephone trunks 120 that are connected to
`the public telephone network 122. The port interfaces 112
`can seize a call on a trunk, answertelephone calls arriving
`on the trunk 120, place outboundcalls, and manage multiple
`calls on a single line or trunk. The port interfaces 112 can
`also send and receive call placement and setup control
`information in the voice band or out-of-band. Each port
`interface 112 is uniquely identified within IVR system 14 by
`a port number that is used to select and address the port
`interface. “Port” refers to a logical connection between IVR
`systems 14 and one or more telephonelines of a trunk 120.
`A telephony port supervisor 118 provides electronics for
`controlling the port interfaces 112. The port supervisor 118
`
`Parus Exhibit 2010, Page 16 of 25
`
`Parus Exhibit 2010, Page 16 of 25
`
`

`

`6,157,705
`
`8
`7
`of commercially available speech recognition equipment
`can select an available port interface and route commands or
`and software include Dragon Systems, IBM, Applied Lan-
`data from processor 104 to an available port interface 112.
`guage Technologies and Nuance.
`The public telephone network 122 connects calls to or
`from trunks 120 through one or more telephone company
`In the preferred embodiment, the speech recognizer 16
`offices 130 and to or from remote telephones or other
`carries out continuous, speaker-independent speech
`equipment.
`recognition, meaning that the speech recognizer will recog-
`The network interface 116 or terminal interface 114 also
`nize words that are spoken continuously without pausesthat
`separate the words, and will recognize any speaker without
`can connect an external computer, such as server 8, to IVR
`a training session. Also, in the preferred embodiment, the
`system 14.
`FIG. 1B also showsthe software architecture of the IVR
`speech recognizer 16 associates an utterance or voice phrase
`spoken by the user 2 and received over the voice commu-
`system 14. The IVR system 14 is a computer that runs an OS
`nication channel with a resource identifier. In this context,
`40. The OS 40 can be the same operating system as OS 20,
`“resource identifier” means a name, label or numberthat
`30, or a different operating system. Because the IVR system
`non-uniquely identifies a resource, or provides a mapping of
`14 is expected to respondto telephonecalls arriving from the
`15
`an utterance to a resourceor class of resources. For example,
`PSTN12 onalive, real-time basis,it is advantageous for OS
`when the utterance is “stocks,” the resource identifier could
`40 to be optimized to handle real-time tasks. An example of
`be “stock-functions.” The same resource identifier could be
`suchareal-time optimized OSis the QNX operating system,
`used for an utterance of “bonds,” so that utterances of
`which can be used as OS 40. IVR systems are commercially
`“stocks” or “bonds” lead to the same resource or group of
`available from InterVoice, Octel Corporation, and others.
`resources.
`The IVR system 14 also runs an IVR application program
`42 under control of OS 40. In cooperation with the OS 40,
`the IVR application program 42 is responsible for detecting
`incoming calls, answering incoming calls, presenting pre-
`recorded greetings and messagesto the caller, receiving and
`interpreting caller inputs, and responding to caller inputs.
`The IVR application program 42 defines features and func-
`tions available to the end user 2 for controlling the IVR
`system 14. The IVR application 42 also provides an inter-
`face to an IVR system operator, such as a command line
`interface, graphical user interface, or terminal interface. An
`operator can connect a computer or “dumb terminal” to
`program the IVR system 14 or monitor its operation.
`The IVR application program 42 can also control com-
`munication between the IVR system 14 and the server 8. In
`one embodiment, the IVR system 14 and the server 8 each
`haveserial data ports that are connected by an RS-232serial
`data connection. The IVR application program 42 can read
`from or write to the serial data port of IVR system 14.
`Accordingly, the IVR application program 42 can command
`the server 8 to generate, load, and/or send Resources 60
`through the network 6 to browser 24.
`The IVR system 14 is coupled to a speech recognizer 16
`and an Identifier/URL table 50. In an alternate embodiment,
`the speech recognizer 16 is integrated into the IVR system
`14.
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`Another function of the IVR application program 42is to
`associate a natural language phrase or resource identifier
`detected by the speech recognizer 16 with a URL of a Web
`document 60 stored in association with the server 8. The
`Identifier/URL table 50 stores associations of resource iden-
`tifiers and URLs of Resources 60. In one embodiment, the
`Identifier/URL table 50 is a table of a relational database
`system having columnsfor an index value, a phrase code, a
`natural language phrase or resource identifier, and a URL.
`For example, the Identifier/URL table 50 has a record that
`has the values 101, 100, “home page”, and a URL valuethat
`identifies a home pageof the resource. This record indicates
`that the natural language phrase “home page”is associated
`with a Web document 60 having the specified address.
`Alternatively, the resource identifier “home-page” is stored
`and associated with the same resource.
`
`In operation, the user 2 connects the client computer 4 to
`the server 8 over the network 6 using the browser 24. For
`example, the user 2 enters a URL that identifies a known
`home page of the server 8 into a URLfield of the browser
`24. The browser commandsthe network driver program 22
`to establish an HTTP con