PATENT NUMBER
`
`9:1?t106
`
`6721708
`
`],, ] i
`
`PTO-2040
`12t99
`
`j
`
`;", "
`
`l';,ri.
`,, i
`"-'^''7'! ;"f
`'
`'-tl
`I
`
`UTILITY Pa
`
`z F EU
`
`)
`
`u)Io
`llJf
`a)
`U)
`
`v,
`
`@
`
`,ffhr
`.+
`N U'og
`
`!'r .
`t:\l
`
`',
`
`.,!
`
`ii:l .1. ,,;i: :l i.ji f"i r:::i:it:: l . ;.:r !,..i r- ,:r:j ::r t.r:-.! ?
`- r-' ;f i, ;..i :.r. r:::,
`.,.i.,.' i,,,, 1
`.i: :ir
`
`i;:,.'..'r!.'i:...;,'..,:.i:....i:::'i:i.|:..:l;::.l.--,i..,.;.,i.1,..'1.1,.';.-
`
`ISSUING CLASSIFICATION
`cRoss REFERENCE(S)
`
`SUBCLASS (ONE SUBCLASS PER BLOCK)
`
`TERMINAL
`DISCLAIMER
`
`a
`
`LJ Tne term of this patent
`
`Ll The term of this patent shall
`not extend beyond the expiration date
`of U,S Patent. No.
`
`SIJS,qNMOFAMDET
`PHIMAFiY FXAMINE$4
`
`L_l The terminal __months of
`this patent have been disclaimed.
`
`Continued on lssue Slip Inside File Jacket
`
`Print Fig.t
`
`Print Claim for O.G.
`
`NOTICE OF ALLOWANCE MAILED
`
`t
`
`tt
`t- /
`t
`I
`
`/ '(\ t
`r-t
`'a
`'/'
`f
`
`Alnount Due
`
`frffi:
`
`ISSUE BATCH NUMBER
`
`The inlormation disclosed herein may be restricled. unauthorized disclosure may be prohibited by the united states code Title 35, seclions 122, 1g1 and 36g.
`Possession outside the u.s. Palent & Trademark otfice is restricted to authorized employees and conlractors only.
`FILED wrrH: I orsx (cRF) f] FtcHE I co-norr,r
`Form PTO-436A
`(Rev. 6/99)
`
`(Attached in pocket on righl insid€
`
`ftap)
`
`J:
`
`(FACE)
`
`r:i
`
`t
`
`lr.:
`
`!ffib:i.;,;:
`
`i".
`^
`,
`?-*'
`*
`t.* ,,.*
`
`!l
`
`.'
`
`t f
`
`l
`!f
`
`fr
`
`f;
`
`&',
`
`Google Ex 1024 - Page 1
`
`

`

`r i--
`
`i
`
`ATION
`
`iltl
`
`i
`
`;ffi,iil'f;lifil,itnf;fi
`02/05/01
`
`i,r'i
`
`INil'IALS
`
`,,|
`
`o
`
`i, .r..,.
`
`CONTENTS
`l'lnte Received
`(tncl. C. of M.)
`or
`,, Date nitalted
`
`Date Received
`(lncl. C. of M.)
`or
`Date Mailed
`
`It
`
`,t5ll ':i Int*-
`V { l
`I Jy' ;l .
`r"r"-l
`
`3.
`
`4.
`";fr.\
`.,1 \j
`5.:)
`6. i.l,,i.Ji, ' '
`
`8.
`
`{
`
`.i,i
`lrirr g.
`it.
`i. 10.
`i;!lili
`ir,'t t.
`'
`tt. {)
`::ltl1+
`,:'lill
`
`12.
`
`,,, ,15'
`:,:.' 1 6,
`
`20.
`
`21.
`
`26.
`' 27.
`28,
`, 29.
`30.
`
`31.
`
`32.
`
`33.
`
`i
`
`' 34.
`. 35.
`' 36.
`37.
`" 38.
`I 39.
`40.
`
`42.
`
`43.
`
`44.;
`
`45.
`
`46.
`
`47.
`
`48.
`
`50.
`
`51.
`
`52.
`
`53.
`
`54.
`
`55.
`
`56.
`
`57.
`
`58.
`
`59.
`
`60.
`
`61.
`
`62.
`
`63.
`
`64.
`
`65.
`
`66.
`
`67.
`
`68.
`
`69.
`
`70.
`
`71.
`
`72.
`
`73.
`
`74.
`
`7l
`
`lf"
`'.,*r,
`
`78.
`
`79.
`
`80.
`
`81.
`
`82,
`
`'r
`
`,l
`
`I
`
`(LEFT OUTSTDE)
`
`Google Ex 1024 - Page 2
`
`

`

`PATENT APPLICATION SERIAL NO.
`
`U.S. DEPARTMENT OF COMMERCE
`PATENT AND TRADEMARK OFFICE
`FEE RECORD SHEET
`
`".'1.''
`,', i'.i!
`
`l:,
`
`PTO-1556
`(s/87)
`.U.S. GPO: 2000468'987/39595
`
`Google Ex 1024 - Page 3
`
`

`

`Page I of1
`
`UNITED STATES
`
`CoMMIssIoNEn FoR PATENTS
`.PATENT
`AND TRADEMARK OFFICE
`\ AsHrNG]oN, O.C. eOeSl
`www.usplo.gor'
`
`CONFIRMATION NO.4924
`
`GROUP ART UNIT?K
`
`ATTORNEY
`DOCKET NO.
`47242-00027USPT
`
`ir
`
`UNrrnn Srerrs PnrnNr eNo TiuonMARK OrHcu
`
`tffirilffiililllilillilllilllllilllilllllillillllff llilllilllil
`Bib Data Sheel
`
`SERIAL NUMBER
`09/776.996
`
`FILING DATE
`02t05t2401
`
`RULE
`
`CLASS
`704
`
`PPLICANTS
`Alexander Kurganov, Buffalo Grove, lL;
`Valery Zhukoff, Deerfield, lL;
`coNTINUING DATA *************************
`THIS APPLN CLAIMS BENEFIT OF 60/180,344 O2IO4I2OOO
`AND CLATMS BENEFIT OF 60/233.068 09/15/2000
`
`/
`
`,* FOREIGN APPLICATIONS ********************
`
`tF REQUIRED, FOREIGN FtLtNG LTCENSE GRANTED"* SMALL ENTtTy *"
`* 0311412001
`:oreign Priority claimed I y"* 6l no
`15 USC 119 (a-d) conditions E u". El no [ ,u, uft",
`
`*i*rorr*, ffi'u{
`
`Acknowledsed Exa
`{DDRESS
`)0223
`
`STATE OR
`GOUNTRY
`IL
`
`SHEETS
`DRAWING
`4
`
`TOTAL
`CLAIMS
`4
`
`NDEPENDbN
`CLAIMS
`4
`
`rITL E
`
`lobust voice browser svstem and voice activated device controller
`
`FILING FEE
`RECEIVED
`460
`
`:EES. Authority has been given in Paper
`\o.
`to charge/credit DEPOSIT ACCOUNT
`\o.
`for following:
`
`E ntt Fees
`E t.lo Fees ( Filing )
`E t.tz re"s ( Processing Ext. of
`time )
`E t.ta Fees ( lssue )
`E otnet
`El credit
`
`Google Ex 1024 - Page 4
`
`

`

`Page I of i
`
`I
`
`/I
`
`L'OMMISSIONEFI
`UNTTED STATES PATENT ANDTRADI
`
`OFFIcE
`D.C, 20231
`wwwusplogo/
`
`NO.4924
`
`GROUP ART UNIT
`
`ATTORNEY
`DOCKET NO.
`47242-00027USPT
`
`Srerns Pc'TENT
`
`lilililtflilt ililt tilil ililt flilt ililililil ilfi tffit iltil ililt il ilil
`Bib Data Sheet
`
`SERIAL NUMBER
`09/776.996
`
`Alexander Kurganov, Buffalo Grove, lL;
`Valery Zhukoff, Deerfield, lL;
`- coNTINUING DATA *************************
`THIS APPLN CLAIMS BENEFIT OF 60/180,344 O2IO4I2OOO
`AND CLATMS BENEFTT OF 60/233,068 09/15/2000
`
`FOREIGN AppLlcATloNs ********************
`
`F REQUIRED, FOREIGN FILING LICENSE GRANTED **
`03t14t2001
`Priority claimed E y", fll'po"
`USC 119 (a-d) conditions fl yu, El no fl M"r rftr,
`Allowance
`
`fied and
`
`ENKENS & GILCHRIST
`
`445 Ross Avenue
`las ,TX 75202-2799
`
`st voice browser system
`
`voice activated device controller
`
`FILING FEE
`RECEIVED
`460
`
`Authority has been given in paper
`1-
`tq charge/credit DEPOSIT ACCOUNT
`o._
`for following:
`
`E t.tO Fees ( Fiting )
`J t.ll Fees ( Processing Ext. of
`time )
`fl t.ta Fees ( tssue )
`
`Google Ex 1024 - Page 5
`
`

`

`d*3=E
`E:1:3
`
`\
`
`'.\
`fl1,- o6" ol
`vtr \'/
`
`A
`
`Patent Application
`Docket No. 47 242-00027USPT
`Customer No. 23932
`
`IN THE UNITtrD STATES PATBNT AND TRADEMARK OFFICE
`In re patent application of: Alexander Kurganov et al.
`FoT: ROBUST VOICE BROWSER SYSTEM AND VOICE ACTIVATED
`DEVICE CONTROLLER
`
`BOX PATENT APPLICATION
`Assistant Commissioner
`for Patents
`Washington, D.C. 20231
`
`MAILINGBY E)GRESS MAIL
`
`'E)(PRESS MAIL" Mailirg Label No. EK5066l6995US
`
`Date ofDeposit: February 5,2001
`
`I hereby certify that this paper or fee is being deposited with the U.S. Postal
`Sewice "Express Mail Post Office to Addressee" seruice uder 37 CFR l.l0
`on the date indicated above and is addressed to the Assistant Comissioner for
`Patents, WashingtorL D.C. 20231
`
`Type orPrint Nme
`
`PATENT APPLICATION TRANSMITTAL LETTER
`
`Transmitted herewith for filing, please find the following:
`
`Specification, claims and abstract of the above-referenced patent application
`(total of ).J-pages)
`
`4 sheet(s) of drawing(s) (4 formal/ _ informal).
`
`Combined Declaration and Power of Attornev .
`
`An Assignment of the invention to:
`
`Applicant(s) claim(s) small entity status under 37 CFR L9 and 1.27.
`
`Other (specify): Acknowledgment postcard .
`
`This application is a:
`
`Continuation
`Divisional
`Continuation-In-Part
`
`of prior copending parent application Serial No.
`now pending
`
`filed on
`
`Sir:
`
`X
`
`X X
`
`X X
`
`Google Ex 1024 - Page 6
`
`

`

`Please amend the application to insert the following line in the beginning of
`the specification:
`
`--This application is a Continuation of prior application Serial No.
`filed on _,
`now pending.--
`
`In the event that a petition to extend time under 37 CFR 1.136 is necessary in
`the parent application to maintain copendency for this application, a petition for an
`extension of the necessary time to maintain copendency is hereby requested for the
`parent application and the Commissioner is hereby authorized to debit our Account
`Number L0-0447 for the necessary fees.
`
`The filing fee has been calculated as shown below:
`
`NO. T'ILEI)
`
`NO. EXTRA
`
`RATE
`
`BASIC FEI
`
`TOTAL CLAIMS
`
`INDEPENDINT CLAIMS
`
`MULTIPLE DEPENDENT
`cLArM(S) PRESnNTED
`
`TOTAL FEES:
`
`-20
`
`-3
`
`Deduct One-Ilalf for Small Entitv Status
`
`Assignment Recordal Fee
`
`$40
`
`TOTAL AMOUNT DUE:
`
`$09
`
`$40
`
`$270
`
`X
`
`X
`
`Please charge my Deposit Account No. 10-0447/
`
`in the amount of
`
`$
`
`Check(s) in the amount of $ 395.00 is attached. Please charge any deficiency
`or credit any overpayment to Deposit Account No. 10-
`0447 I 47 242-00021U SPT .
`
`The Commissioner is hereby authorized to charge payment of the following
`fees during the pendency of this application or credit any overpayment to
`D eposit Account No. I 0-044 7 I 47 242-00027USPT.
`X
`
`Any patent application processing fees under 37 CFR 1.17 and under
`37 CFR 1.20(d).
`
`The issue fee set in 37 CFR 1 18 at or before mailing of the Notice of
`AlloWance, pursuant to 37 CFR 1.31 1(b).
`
`CHICAGO I 55876v1 47242-00027
`
`X'EE
`
`$355
`
`0
`
`40
`
`$395.00
`
`$39s.00
`
`Google Ex 1024 - Page 7
`
`

`

`Any filing fees under 37 CFR 1.16 including fees for presentation of
`extra claims.
`
`Russell J. Genet
`Registration No. 42,57 1
`JENKENS & GILCHRIST
`3200 Fountain Place
`1445 Ross Avenue
`Dallas, Texas 7 5202-2799
`3T21425-8516
`2r4l85s-4300 (Fax)
`
`CHICAGO 1 55876v1 4'1242-00027
`
`Google Ex 1024 - Page 8
`
`

`

`Patent Application
`Docket N o.: 47 242-00027USPT
`Customer Number 23932
`
`APPLICATION FOR UNITED STATES LETTERS PATENT
`
`for
`
`ROBUST VOICE BROWSER SYSTBM AND
`
`VOICE ACTIVATED DEVICE CONTROLLER
`
`ii:i I
`
`INVENTORS:
`
`Alexander Kurganov
`
`Valery Zhukoff
`
`NUMBER: s K .ro''iTi"-i#'#i'ili:T:'
`"F& br*".,t! -f . ;A a&'!
`sgs
`l hereby certify that this papCr or fee fs being depoiited with the United States Postal Service
`"EXPRESS MAIL POST OFFICE TO ADDRESSEE" service under 37 C.F.R. 1.10 on the date
`zoz:t <Q**1.**!*"* E
`indicated above and is addressed to: Assistant Commissioner for Patents, Washington D.C.
`
`CIiICAGO 1 5 5909v1 47 242-0002'1
`
`Google Ex 1024 - Page 9
`
`

`

`ROBUST VOICE BROWSER SYSTEM AND
`
`VOICE ACTIVATED DEVICE CONTROLLER
`
`CROSS.RJFERENCE TO RELATED APPLICATIONS
`This application claims the benefit of priority from United States Provisional
`Application Serial No. 60/18A344, filed February 4,2000 entitled "Voice-Activated
`Information Retrieval System" and United States Provisional Application Serial No.
`6A1233,A68, filed September 15,2000 entitled "Robust Voice Browser System and Voice
`Activated Device Controller" .
`
`"
`
`FTELD OF THE INVENTION
`
`The present invention relates to a robust and highly reliable system that allows
`users to browse web sites and retrieve information by using conversational voice
`commands. Additionally, the present invention allows users to control and monitor other
`
`systems and devices that are connected the Internet or any other network by using voice
`
`commands.
`
`BACKGROUND OF THE INVENTION
`
`Currently, three options exist for a user who wishes to gather information from a
`web site accessible over the Internet. The first option is to use a desktop or a laptop
`computer connected to a telephone line via a modem or connected to a network with
`Internet access. The second option is to use a Personal Digital Assistant (PDA) that has
`the capability of connecting to the Internet either through a modem or a wireless
`connection The third option is to use one of the newly designed web-phones or web-
`pagers that are now being offered on the market. Although each of these options provide
`a user with access to the Internet to browse web sites, each of them have their own
`
`drawbacks.
`
`Desktop computers are very large and bulky and are difficult to transport. Laptop
`computers solve this inconvenience, but many are still quite heavy and are inconvenient
`
`CHICAGO 1 55909v1 47242-00027
`
`Google Ex 1024 - Page 10
`
`

`

`z
`
`to carry. Further, laptop computers cannot be carried and used everywhere a user travels.
`For instance, if a user wishes to obtain information from a remote location where no
`electricity or communication lines are installed, it would be nearly impossible to use a
`laptop computer. Oftentimes, information is needed on an immediate basis where a
`cornputer is not accessible. Furthermore, the use of laptop or desktop computers to access
`the Internet requires either a direct or a dial-up connection tan an Internet Service
`Provider (ISP). Oftentimes, such connections are not available when a user desires to
`
`connect to the Internet to acquire information.
`" The second option for remotely accessing web sites is the use of PDAs. These
`devices also have their own set of drawbacks. Fifst, PDAs with the ability to connect to
`the Internet and access web sites are not readily available. As a result, these PDAs tend
`to bp very expensive. Furthermore, users are usually required to pay a special service fee
`to enable the web browsing feature of the PDA. A further disadvantage of these PDAs is
`that web sites must be specifically designed to allow these devices to access information
`on the web site. Therefore, a limited number of web sites are available that are accessible
`by these web-enabled PDAs. Finally, it is very common today for users to carry cell
`phones, however, users must also carry a separate PDA if they require the ability to
`gather information from various web sites. Users are therefore subjected to added
`expenses since they must pay for both cellular telephone service and also for the web-
`enabling service for the PDA. This results in a very expensive alternative for the
`c0nsumer.
`
`The third alternative mentioned above is the use of web-phones or web-pagers.
`These devices suffer many of the same drawbacks as PDAs. First, these devices are
`expensive to purchase. Further, the number of web sites accessible to these devices is
`limited since web sites must be specifically designed to allow access by these devices.
`Furthermore, users are often required to pay an additional fee in order to gain wireless
`web access. Again, this service is expensive. Another drawback of these web-phones or
`web-pagers is that as technology develops, the methods used by the various web sites to
`allow access by these devices may change. These changes may require users to purchase
`new web-phones or web-pagers or have the current device serviced in order to upgrade
`the firmware or operating system stored within the device. At the least, this would be
`
`inconvenient to users and may actually be quite expensive.
`
`Therefore, a need exists for a system that allows users to easily access and browse
`
`CHICAGO I 55909v1 4?242-00027
`
`Google Ex 1024 - Page 11
`
`

`

`3
`
`the Internet from any location. Such a system would only require users to have access to
`
`any type of telephone and would not require users to subscribe to multiple services.
`
`In the rapidly changing area of Internet applications, web sites change frequently.
`
`The design of the web site may change, the information required by the web site in order
`
`to perform searches may change, and the method of reporting search results may change.
`
`Web browsing applications that submit search requests and interpret responses from these
`web sites based upon expected formats will produce errors and useless responses when
`such changes occur. Therefore, a need exists for a system that can detect modifications to
`web sites and adapt to such changes in order to quickly and accurately provide the
`information requested by a user through a voice enabled device, such as a telephone.
`
`When users access web sites using devices such as personal computers, delays in
`
`receiving responses are tolerated and are even expected, however, such delays are not
`expected when a user communicates with a telephone. Users expect communications
`over a telephone to occur immediately with a minimal amount of delay time. A user
`attempting to frnd information using a telephone expects immediate responses to his
`search requests. A system that introduces too much delay between the time a user makes
`a request and the time of response will not be tolerated by users and will lose its
`usefulness. Therefore, it is important that a voice browsing system that uses telephonic
`communications selects web sites that provide rapid responses since speed is an important
`factor for maintaining the system's desirability and usability. Therefore, a need exists for
`
`a system that accesses web sites based upon their speed of operation.
`
`SUMMARY OF THE INVtrNTION
`It is an object of an embodiment of the present invention to allow users to gather
`information from web sites by using voice enabled devices, such as wireline or wireless
`
`telephones.
`
`An additional object of an embodiment of the present invention is to provide a
`system and method that allows the searching and retrieving of publicly available
`information by controlling a web browsing server using naturally spoken voice
`commands.
`
`It is an object of another embodiment of the present invention to provide a robust
`voice browsing system that can obtain the same information from several web sites based
`
`CHICAGO I 5 5909v1 4'7 242-00027
`
`Google Ex 1024 - Page 12
`
`

`

`4
`
`upon a ranking order. The ranking order is automatically adjusted if the system detects
`that a given web site is not functioning, is too slow, or has been modified in such a way
`
`that the requested information cannot be retrieved any longer.
`
`A still further object of an embodiment of the present invention is to allow users
`
`to gather information from web sites from any location where a telephonic connection can
`
`be made
`
`Another object of an embodiment of the present invention is to allows users to
`browse web sites on the Internet using conversational voice commands spoken into
`wireless or wireline telephones or other voice enabled devices.
`
`An additional object an embodiment of the present invention is to provide a
`system and method for using voice commands to control and monitor devices connected
`to a network.
`It is an object of an embodiment of the present invention to provide a system and
`method which allows devices connected to a network to be controlled by conversational
`voice commands spoken into any voice enabled device interconnected with the same
`
`network.
`
`The present invention relates to a system for acquiring information from sources
`on a network, such as the Internet. A voice browsing system maintains a database
`containing a list of information sources, such as web sites, connected to a network. Each
`
`of the information sources is assigned a rank number which is listed in the database along
`with the record for the information source. In response to a speech command received
`from a user, a network interface system accesses the information source with the highest
`
`rank number in order to retrieve information requested by the user.
`
`The a preferred embodiment of the present invention allows users to access and
`browse web sites when they do not have access to computers with Internet access. This is
`accomplished by providing a voice browsing system and method that allows users to
`browse web sites using conversational voice commands spoken into any type of voice
`enabled device (i,e., any type of wireline or wireless telephone, IP phone, wireless PDA,
`or other wireless device). These spoken commands are then converted into data messages
`by a speech recognition software engine running on a user interface system. These data
`messages are then sent to and processed by a network interface system. This network
`
`interface system then generates the proper requests that are transmitted to the desired web
`
`CHICAGO I 5 5909v1 47 242-0002'7
`
`Google Ex 1024 - Page 13
`
`

`

`5
`
`site over the Internet. Responses sent from the web site are received and processed by the
`
`network interface system and then converted into an audio message via a speech synthesis
`engine or a pre-recorded audio concatenation application and finally transmitted to the
`
`user's voice enabled device.
`
`A preferred embodiment of the voice browser system and method uses a web site
`polling and ranking methodology that allows the system to detect changes in web sites
`and adapt to those changes in real-time. This enables the voice browser system of a
`preferred embodiment to deliver highly reliable information to users over any voice
`enabled device. This ranking system also enables the present invention to provide rapid
`responses to user requests. Long delays before ,receiving responses to requests are not
`tolerated by users of voice-based systems, such as telephones. When a user speaks into a
`
`telephone, an almost immediate response is expected. This expectation does not exist for
`
`non-voice communications, such as'email transmissions or accessing a web site using a
`personal computer. In such situations, a reasonable amount of transmission delay is
`acceptable. The ranking system of implemented by a preferred embodiment of the
`present invention ensures users will always receive the fastest possible response to their
`
`request.
`
`An alternative embodiment of the present invention allows users to control and
`monitor the operation of a variety of household devices connected to a network using
`speech commands spoken into a voice enabled device.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a depiction of the voice browsing system of the first embodiment of the
`
`present invention;
`
`FIG. 2 is a block diagram of a database record used by the first preferred
`
`embodiment of the present invention;
`
`FIG.3isablock
`
`diagram of a media server used by the preferred embodiment;
`
`FIG.4isablock
`
`diagram of a web browsing server used by the preferred
`
`embodiment, and
`
`FIG. 5 is a depiction of the device browsing system of the second embodiment of
`
`the present invention.
`
`CHICAGO 155909v1 47242-0002'l
`
`Google Ex 1024 - Page 14
`
`

`

`DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
`
`A first embodiment of the present invention is a system and method for allowing
`users to browse information sources, such as web sites, by using naturally spoken,
`conversational voice commands spoken into a voice enabled device. Users are not
`required to learn a special language or command set in order to communicate with the
`voice browsing system of the present invention. Common and ordinary commands and
`iq required for a user to operate the voice browsing system. The voice
`phrases are all
`that
`browsing system recognizes naturally spoken voice commands and is speaker-
`independent; it does not have to be trained to recognize the voice patterns of each
`individual user. Such speech recognition systems use phonemes to recognize spoken
`words and not predefined voice patterns.
`The first embodiment allows users to select from various categories of
`information and to search those categories for desired data by using conversational voice
`commands. The voice browsing system of the first preferred embodiment includes a user
`interface system referred to as a media server. The media server contains a speech
`recognition software engine. This speech recognition engine is used to recognize natural,
`
`conversational.voice commands spoken by the user and converts them into data messages
`based on the available recognition grammar. These data messages are then sent to a
`network interface system. In the first preferred embodiment, the network interface
`system is referred to as a web browsing server. The web browsing server then accesses
`the appropriate information source, such as a web site, to gather information requested by
`
`the user.
`
`Responses received from the information sources are then transferred to the media
`
`server where speech synthesis engine converts the responses into audio messages that are
`transmitted to the user. A more detailed description of this embodiment will now be
`provided.
`
`Referring to FIG. 1, a database 100 designed by Webley Systems Incorporated is
`connected to one or more web browsing servers 102 as well as to one or more media
`servers 106 The database may store information on magnetic media, such as a hard disk
`drive, or it may store information via other widely acceptable methods for storing data,
`such as optical disks. The database 100 contains a separate set of records for each web
`
`CHICAGO I 55909v1 47242-00027
`
`Google Ex 1024 - Page 15
`
`

`

`site accessible by the system. An example of a web site record is shown in FIG. 2. Each
`web site record 200 contains the rank number of the web site 202, the associated Uniform
`Resource Locator (URL) 204, and a command that enables the appropriate "extraction
`agent" 206 that is required in order to generate proper requests sent to and to format data
`received from the web site. The database record 200 also contains the timestamp 208
`indicating the last time the web site was accessed. The extraction agent is described in
`more detail below. The database 100 categorizes each database record 200 according to
`the type of information provided by each web site. For instance, a first category of
`database records 200 may correspond to web sites that provide "weather" information.
`The database 100 may also contain a second catggory of records 200 for web sites that
`provide "stock" information. These categories may be further divided into subcategories.
`For instance, the "weather" category may contain subcategories depending upon type of
`weather information available to a user, such as "current weather" or "extended forecast".
`Within the "extended forecast" subcategory, a list of web site records may be stored that
`provide weather information for multiple days. The use of subcategories may allow the
`web browsing feature to provide more accurate, relevant, and up-to-date information to
`the user by accessing the most relevant web site. The number of records contained in
`each category or subcategory is not limited. In the preferred embodiment, three web site
`
`records are provided for each category.
`
`Table 1 below depicts two database records 200 that are used with the preferred
`embodiment. These records also contain a field indicating the "category" of the record,
`which is "weather" in each of these examples.
`
`TABLE 1
`
`t^reathef
`n:1-annrrz.
`r-nn r-r-tm / r--:'i -
`TTRT,: IIRT,=h ttn'
`/ / rni
`uvrrl/
`uY!
`ULI/.
`/ | vY-.
`vrrrr.
`bin/weather / redirect? z:-p: zLp
`rank:
`web_di spatch . pl weather_cnn
`command:
`l -l
`r,rnnrf:
`hrnt^rc'innQarrrar.
`v!vvvJ!rrYUg!
`wy\Jr uctII
`vc!
`.
`browsingserverBackup: wportaL2
`dateTime: Dec 27 2000 2: l-5PM
`
`1
`
`category: weather
`
`CHICAGO I 55909v1 47242-00027
`
`Google Ex 1024 - Page 16
`
`

`

`8
`
`URL: URL=http : / /weather . lycos . com/wcfiveday. asp?city:zip
`rank:
`2
`command: web_dispat.ch.pl weather_lycos
`l-rrnr^rqinrr(orrror.
`r^rnnrl_:l
`hrows i noServerBackrrn: wDortaL?
`dateTime: Dec 2L 2000 1:45PM
`
`.
`
`1
`
`"t
`
`The database also contains a listing of pre-recorded audio files used to create
`concatenated phrases and sentences. Further, database 100 may contain customer profile
`information, system activity reports, and any other data or software servers necessary for
`
`the testing or administration of the voice browsing system.
`
`The operation of the media servers 106 will now be discussed in relation to FIG.
`3. . The media servers 106 function as user interface systems. In the preferred
`embodiment, the media servers 106 contain a speech recognition engine 300, a speech
`synthesis engine 302, an Interactive Voice Response (IVR) application 304, a call
`processing system 306, and telephony and voice hardware 308 required to communicate
`with the Public Switched Telephone Network (PSTN) 116. In the preferred embodiment,
`each media server is based upon Intel's Dual Pentium III 730 MHz microprocessor
`system.
`
`The speech recognition function is performed by a speech recognition engine 300
`
`that converts voice commands received from the user's voice enabled device 112 (i.e.,
`any type of wireline or wireless telephone, Internet Protocol (IP) phones, or other special
`wireless units) into data messages. In the preferred embodiment, voice commands and
`audio messages are transmitted using the PSTN 116 and data is transmitted using the
`TCP/P communications protocol. However, one skilled in the art would recognize that
`other transmission protocols may be used for either voice or data. Other possible
`transmission protocols would include SIP/VoIP (Session Initiation Protocol/Voice over
`IP), Asynchronous Transfer Mode (ATM) and Frame Relay. A preferred speech
`recognition engine is developed by Nuance Communications of 1380 Willow Road,
`Menlo Park, California 94025 (www.nuance.com). The Nuance engine capacity rs
`measured in recognition units based on CPU type as defined in the vendor specification.
`The natural speech recognition grammars (i.e., what a user can say that will be recognized
`
`by the speech recognition engine) were developed by Webley Systems.
`
`Table 2 below provides a partial source code listing of the recognition grammars
`
`CI'IICAGO I 55909v1 47242-00027
`
`Google Ex 1024 - Page 17
`
`

`

`9
`
`used by the speech recognition engine of the preferred embodiment for obtaining weather
`
`information.
`
`TABLE 2
`
`?WIIAT_IS ?the weather ?[info information report conditions]
`? ( (?tike in )
`
`USgitles : n
`
`t
`
`{ <paraml $n. zip> (param2 $n. city> (param3
`$n . state> )
`( (area code) AREA_CODE:n ) {<paraml $n>i
`( AREA_CODE:n (area code) ) {<paraml $n>i
`( (zip ?code) ZIP_CODE:n )
`{<paraml $n>}
`( ZIP_CODE:n (zip ?code) )
`{(paraml $n>i
`
`l
`
`)
`
`) {<menu 1-94>}
`
`The media server 106 uses recognition results generated by the speech recognition
`
`engine 300 to retrieve a web site record 200 stored in the database 100 that can provide
`the information requested by the user. The media server 106 processes the recognition
`result data identifying keywords that are used to search the web site records 200
`contained in the database 100 For instance, if the user's request was "What is the weather
`in Chicago?", the keywords "weather" and "Chicago" would be recognized. A web site
`record 200 with the highest rank number from the "weather" category within the database
`100 would then be selected and transmitted to the web browsing server 102 along with an
`identifier indicating that Chicago weather is being requested.
`
`The media servers 106 also contain a speech synthesis engine 302 that converts
`the data retrieved by the web browsing servers 102 into audio messags that are
`transmitted to the user's voice enabled device I i2. A preferred speech synthesis engine
`is developed by Lernout and Hauspie Speech Products, 52 Third Avenue, Burlington,
`
`Massachusetts 0 1 803 (www.lhsl.com).
`
`A further description of the web browsing server 102 will be provided in relation
`to FIG. 4. The web browsing servers 102 provide access to any computer. network such
`
`CHICAGO I 5 5909v I 47 242-00027
`
`Google Ex 1024 - Page 18
`
`

`

`l0
`
`as the Internet 110. These seryers are also capable of accessing databases stored on Local
`Area Networks (LANs) or Wide Area Networks (WANs). The web browsing servers
`receive responses from web sites and extract the data requested by the user. This task is
`also known as "content extraction." The web browsing servers 102 also perform the task
`of periodically polling or "pinging" various web sites and modifying the ranking numbers
`of these web sites depending upon their response and speed. This polling feature is
`further discussed below. The web browsing server 102 is comprised of a content
`extraction agent 400, a content fetcher 402, a polling and ranking agent 404, and the
`content descriptor files 406. Each of these are software applications and will be discussed
`
`below.
`
`Upon receiving a web site record 200 from the database 100 in response to a user
`request, the web browsing server 102 invokes the "content extraction agent" command
`206 contained in the record 200. The content extraction agent 400 allows the web
`browsing server 102 to properly format requests and read responses provided by the web
`site 114 identified in the URL fieId 204 of the web site record 200. Each content
`extraction agent command 206 invokes the content extraction agent and identifies a
`content description file associated with the web page identified by the URL 204. This
`content description the directs the extraction agent where to extract data from the
`accessed web page and how to format a response to the user utilizing that data. For
`example, the content description for a web page providing weather information would
`indicate where to insert the "city" name or ZIP code in order to retrieve Chicago weather
`information. Additionally, the content description file for each supported URL indicates
`the location on the web page where the response information is provided. The extraction
`agent 400 uses this information to properly extract from the web page the information
`
`requested by the user.
`
`Table 3 below contains source code for a content extraction asent 400 used bv the
`
`preferred embodiment.
`
`TABLE 3
`
`CFIICAGO 1 5 5909v I 47242-00027
`
`Google Ex 1024 - Page 19
`
`

`

`rirF4l
`
`r,, i
`
`i:i
`
`i,i ii
`
`ll
`
`ll A ?i
`
`# ! /usr / local /www/b:n/sybperl5
`ff)Heaoer:
`/usr / local/cvsroot/webley/aqents/service/web dispatch.pl, v
`16
`# Disnafr-hps
`
`all
`
`r^ral.r ra^rroStS
`
`-
`ll
`
`(1nt)L
`^1.^1,----,,T
`-J
`
`#http: / /wcorp. itn.net/ cqL/ flstat?carrier=ua&fliqht no:155emo
`cl-l
`^-
`(l-L O(L.rC1 Lq:-
`6istamp=OhLN-PdbuuE* itn/ord, :.tn/cblsprint hd
`#http : / / cg t. cnnfn . com/ f I i ghtvi ew/ r Im? ai rl ine:amt &number= 3 0 0
`
`r6drrira
`
`tt nnnf
`
`i n
`
`fmn nl//
`urlly
`. yt
`
`.
`f
`
`# check narameters
`die "Usage: $0 service [params]\n'r if $#eRgv < 7;
`#print STDERR GARGV;
`
`ff geL parameLers
`.i+ ^^.1-
`n-
`*--^F,
`my ( $service, Gparam ) : GARGV;
`# check service
`m\z ?Qarrzi
`cac
`
`I\
`
`weather_cnn :>'webget.pl weather_cnn'
`weather Iycos :) 'webget.pl
`weat.her_weather :) ' webget . p1
`tWebget.pl
`r^ro:i-hor eneD :>
`
`weather_infospace ->'webget.pl
`stockQuote yahoo =) 'webget.pl stock',
`fliqhtstat[s
`itn =) 'webget.pl
`yellowPages yahoo =) 'yp data.pl',
`yellowPages yahoo -> 'yp data.pl',
`newsHeaders_newsreal- :> 7rret".Pf ' ,
`newsArticle_newsreal- =) 'news . pl' ,
`
`) = $date :- /\s+ (\w{3}\s+\d{I,2} )\s+7;
`weather_cnn :>' 60053',
`weather lycos :) '60053',
`weather-weath