voGOvollllllllllllllllllllllllllllllllllllll‘ Old'9009191 PTO/SBl05 modified
`
`PATENT APPLICATION
`TRANS MITTAL
`
`First Inventor
`Title
`
`ALex Kurganov
`Robust Voice Browser System and Voice
`Activated Device Controller
`
`UTILITY
`
`Attorne Docket No.
`
`47242-00027U501
`
`(Only for new nonpmvisional applications under 37 CFR 1.53(b))
`
`Express Mail Label No
`
`E T
`L
`T
`L
`APP ICA IONE EM N S
`See MPEP chapter 600 concerning utility patent application contents.
`
`.
`ADDRESS TO.
`
`EV 284714429 US
`Commlssloner tor Patents
`a
`en
`rélgtfzasto tAplecatlon
`Alexa ndrla VA 223134450
`
`.
`
`Fee Transmittal Form (e.g.. PTO/SB/17)
`(Submit an original, and a duplicate for fee processing)
`Applicant claims small entity status.
`See 37 CRF 1.27.
`[Total Pages fl ]
`Specification
`(preferred arrangement set forth below)
`- Descriptive title ofthe invention
`- Cross References to Related Applications
`- Statement Regarding Fed sponsored R 8. D
`- Reference to sequence listing. a table,
`or a comp"15' program "sung append"
`- Background of the Invention
`- Brief Summary of the Invention
`- Brief Description of the Drawings (if filed)
`— CI
`'
`- Detailed Description
`_ “Emil om... Discmum
`[Total Sheets 3]
`4' E Drawrng(s) (35 USC 113)
`[Total Sheets 5]
`5‘ 03%” Declaration
`_
`.
`a
`NEW executed (ongmal 0r CODY)
`b. [X] Copy from a prior application (37 CFR 1.63(d))
`(for continuation/divisional with Box 18 completed)
`T
`N
`t
`i
`'_
`. D DELE ION OF I VENTOR S
`name in the prior application, see 37 CFR
`Signed statement attached deleting inventor(s)
`1,63(d)(2) and 1.33(b).
`
`e. D: Application Data Sheet. See 37 CFR1.7S
`
`.
`
`3.
`
`7, E] CD-ROM or CD-R in duplicate, large table or
`Computer Program (Appendix)
`Nucleotide and/or Amino Acid Sequence Submission
`(if applicable, all necessary)
`a.
`[3 Computer Reader Form (CRF)
`b.
`Specification Sequency Listing on:
`.
`'
`[:1 CD ROM or CD-R (2 Copies); or
`I.
`_,
`"- E] Paper
`. D Statement verifying identity of above copies
`ACCOMPANYING APPLICATION PARTS
`
`9. D Assignment Papers (cover sheet & document(s))
`10. El 37 CFR 3.730)) Statement
`[:I Power of
`(when there is an assignee)
`Attorney
`English Translation Document (ifapplicable)
`11.
`12.
`Information Disclosure
`D Copies ofIDS
`Statement (IDS)/PTO-1449
`Citations
`13.
`Preliminary Amendment
`14.
`Return Receipt Postcard (MPEP 503)
`(Should be specifically itemized)
`\
`.
`.
`.
`~
`Certified Copy of Priority Document(s)
`15.
`(if foreign priority is claimed‘I
`Nonpublication Request under 35 U.S.C. 122
`(b)(2)(B)(i). Applicant must attach form PTO/SB/So
`or its equivalent.
`
`16.
`
`_
`
`specification following the title, or in an Application Data Sheet under 37 CFR 1.76:
`
`of priorapplication No:
`U Continuation-in-parI(ClP)
`El Divisional
`E Continuation
`Art Unit:
`2641
`Examiner Susan McFadden
`Prior application information:
`For CONTINUATION OF DIVISIONAL APPS only; The entire disclosure of the prior application, from which an oath or declaration is supplied under Box 5b,
`is considered a part of the disclosure of the accompanying continuation or divisional application and is hereby incorporated by reference.
`The incorporation can only be relied upon when a gottIon has been inadvertently omitted from the submitted a - - . lication arts.
`19. CORRESPONDENCE ADDRESS
`
`u
`-
`..
`,
`-
`. -
`E Customer N mber or Bar Code Label
`‘
`‘
`'
`_30223
`OR [:1 Corres ondence address below
`
`
`p (insert Customer No. or 'Aitach bar code label here) ‘
`Russell J. Genet
`
`Address
`Jenkens & Gilchrist
`
`._99_/_7.Z§J_99_E§ ______________
`
`Other: 18. If a CONTINUING APPLICATION, check appropriate box and supply the requisite information below and in the first sentence of the
`
`Cit
`
`225 West Washington Street, Suite 2600
`Chica-o
`
`
`60606-3418
`312 425-3909
`
`Name (Print/Type)
`
`312 425-8516
`Telephone
`42,571
`'egistration No.
`Russell J. Genet
`._
`s A Attorne /Aent
`.
`
`_mfl_m_ April 9, 2004
`This collection of information is required by 37 CFR, .53(b). The information is required to obtain or retain a benefit by the public which IS to file (and by the USPTO to process)
`an application. Confidentiality Is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete. including gathering. preparing. and
`submitting the completed application form to the USPTO, Time will vary depending upon the individual case, Any comments on the amounl of time you require to complete this
`form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, US. Patent and Trademark Office, US. Department of Commerce. PO. Box
`1450. Alexandria. VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS SEND TO: Mail Stop Patent Application, Commissioner tor
`Patents, P.O. Box 1450, Alexandria, VA 223134450.
`If you need assistance in completing the form, call 1-800-PTO-9199 and select option 2.
`
`CHICAGO 28077v1 99999-0000]
`
`5
`
`|PR2020-00686
`
`Apple EX1002 Page 1
`
`IPR2020-00686
`Apple EX1002 Page 1
`
`

`

`
`
`FEE TRANSMITTAL
`
`for FY 2004
`Effective 01/01/2003. Patent fees are subject to annual revision.
`
`
`
`
`
`
`
`E Applicant claims small entity status. See 37 CFR 1.27
`TOTAL AMOUNT OF PAYMENT
`$ 385.00
`
`
`
`METHOD OF PAYMENT (check all that apply)
`
`Order
`|:| Credit card |:| Money |:] Other [I None
`IE Check
`I] Deposit Account:
`Deposit
`10-0447/47242-00027USC1
`Account
`Number
`
`Ap-lication Number
`Filing Date
`First Named Inventor
`Examiner Name
`Art Unit
`
`PTO/SB/17 (modified)
`
`Complete if Known
`Unknown
`
`Audi 9, 2004
`Alex Kurganov
`Unknown
`Unknown
`
`47242-00027USC1
`
`
` Attorney Docket No.
`FEE CALCULATION (continued)
`3. ADDITIONAL FEES
`
`Fee
`Code
`
`Fee
`(5)
`
`Fee
`Code
`
`Fee
`(5)
`
`Fee Description
`
`Surcharge - late filing fee or oath
`Surcharge - late provisional filing fee or
`cover sheet.
`
`Non-English specification
`For filing a request for ex parte reexamination
`Requesting publication of SIR prior to
`Examiner action
`
`Requesting publication of SIR after
`Examiner action
`
`Extension for reply within first month
`Extension for reply within second month
`Extension for reply within third month
`Extension for reply within fourth month
`Extension for reply within fifth month
`Notice of Appeal
`Filing a brief in support of an appeal
`Request for oral hearing
`Petition to institute a public use proceeding
`Petition to, revive - unavoidable
`Petition to revive - unintentional
`Utility issue fee (or reissue)
`Design issue fee
`Plant issue fee
`Petitions to the Commissioner
`Processing fee under 37 CFR 1.17(q)
`Submission of Information Disclosure Stmt
`Recording each patent assignment per
`property (times number of properties)
`Filing a submission after final rejection
`(37 CFR 1‘129(a))
`For each additional invention to be
`examined (37 CFR 1.129(b))
`Request for Continued Examination (RCE)
`Request for expedited examination
`of a design application
`
`
`
`Deposit
`Account
`Name
`Jenkens 8. Gilchrist
`The Commissioner is authorized to: (check all that apply)
`
`25
`
`1051
`1052
`
`130
`50
`
`2051
`2052
`
`65
`
`I:I Credit any overpayments
`I:I Charge fee(s) indicated below
`|:i Charge any additional feels) during the pendancy of this application
`El Charge fee(s) indicated below. except for the filing fee
`to the above-identified deoosit account.
`FEE CALCULATION
`1. BASIC FILING FEE
`Large Entity Small Entity
`Fee
`Fee
`Code
`(5)
`2001
`385
`2002 170
`2003 265
`
`Fee Description
`
`Utility filing fee
`Design filing fee
`Plant filing fee
`Reissue filing fee
`
`Provisionalfilingfee |:]
`so
`SUBTOTAL“)
`‘
`-
`2. EXTRA CLAIM FEES FOR UTILITY AND REISSUE
`Fee from
`.
`below
`
`Extra Claims
`
`Fee Paid
`
`2004 385
`
`2005
`
`Independent
`
`Multiple Dependent
`Large Entity Small Entity
`Fee
`Fee
`
`I:l = l:l
`
`Fee Description
`
`9
`43
`
`43
`
`Claims in excess of 20
`Independent claims in excess of 3
`Multiple dependent Claim, if not paid
`" Reissue independent claims
`over original patent
`“ Reissue claims in excess of 20
`“ and over original patent
`summer
`"or number previously paid, if greater; For Reissues, see above
`
`9
`
`1053
`1812
`
`1053
`1812
`
`"304
`
`920'
`
`1804
`
`1805 1.840'
`
`1805
`
`1 251
`1 252
`1 253
`1 254
`' 1255
`1401
`1402
`1403
`'1451
`1452
`1453
`1501
`1502
`1503
`1460
`1807
`1806
`6021
`
`1809
`
`1510
`
`1801
`1802
`
`1 10
`420
`950
`1,480
`2.010
`330
`330
`290
`1,510
`110
`1.330
`1,330
`480
`640
`1 30
`50
`180
`40
`
`2251
`2252
`2253
`2254
`2255
`2401
`2402
`2403
`1451
`
`2452
`2453
`2501
`2502
`2503
`1 460
`1607
`1806
`8021
`
`2809
`
`2810
`
`2801
`1802
`
`Other fee (specify) __
`
`' Reduced by Basic Filing Fee Paid
`
`SUBTOTAL (3) <5)—
`
`Name (PrinVType)
`Signature
`
`Russell J. Genet
`
`Attorne /A-ent
`Registration No
`
`42,571
`
`Telephone
`
`31342543515
`APRIL 9. 2004
`
`WARNING: Information on this form may become public. Credit card information should not
`be included on this form. Provide credit card information and authorization on PTO-203B.
`This collection of information is required by 37 CFR 117 and 1.27, The information is required to obtain or retain a benefit by the public which is to file (and by the
`USPTO to process) an application. Confidentiality is governed by 35 use 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete.
`including gathering, preparing. and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on
`the amount of time you require to complete this form and/or suggestions for reducing this burden. should be sent to the Chief Information Officer. US. Patent and
`Trademark Office, US. Department of Commerce, PO. Box 1450. Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS
`ADDRESS. SEND TO: Commissioner for Patents, PO. Box 1450. Alexandria, VA 22313-1450.
`If you need assistance in completing the form, call 1-800-PTO-9199 and select option 2.
`
`f‘l "r‘ A r‘n 10n'ln..l nnnnn nnnnl
`
`|PR2020-00686
`
`Apple EX1002 Page 2
`
`IPR2020-00686
`Apple EX1002 Page 2
`
`

`

`Patent Application
`Docket No.: 47242-00027USC1
`Customer Number 30223
`
`APPLICATION FOR UNITED STATES LETTERS PATENT
`
`for
`
`ROBUST VOICE BROWSER SYSTEM AND
`
`VOICE ACTIVATED DEVICE CONTROLLER
`
`INVENTORS:
`
`Alexander Kurganov
`
`Valery Zhukoff
`
`NUMBER:
`
`EXPRESS MAIL MAILING LABEL
`EV 284714429 US
`
`Box 1450, Alexandria. VA 22313-1450.
`
`'
`April 9 2004
`DATE:
`i hereby certify that this paper or fee is being deposited with the United States Postal Service
`"EXPRESS MAIL POST OFFICE TO ADDRESSEE" service under 37 C.F.R. 1.10 on the date
`indicated above and is addressed to: MS Patent Application, Commissioner for Patents, PO.
`
`CHICAGO 283215vl 47242-OOOOI
`
`I
`IPR2020-00686
`
`Apple EX1002 Page 3
`
`IPR2020-00686
`Apple EX1002 Page 3
`
`

`

`Patent Application
`Docket No.: 47242—00027USCI
`Customer Number 30223
`
`ROBUST VOICE BROWSER SYSTEM AND
`
`VOICE ACTIVATED DEVICE CONTROLLER
`
`CROSS-REFERENCE TO RELATED APPLICATIONS
`
`[001] This application is a continuation of U.S. Patent Application No. 09/776,996, filed
`
`February 5, 2001, now allowed, which claims priority to U.S. Provisional Application
`
`Serial No. 60/180,344,
`
`filed February 4, 2000 entitled “Voice-Activated Information
`
`Retrieval System” and U.S. Provisional Application Serial No. 60/233,068,
`
`filed
`
`September 15, 2000 entitled “Robust Voice Browser System and Voice Activated Device
`
`Controller” and these applications are incorporated by reference in their entirety into the
`
`present application.
`
`FIELD OF THE INVENTION
`
`[002] The present invention relates to a robust and highly reliable system that allows users
`
`to browse web sites and retrieve information by using conversational voice commands.
`
`Additionally, the present invention allows users to control and monitor other systems and
`devices that are connected the Internet or any other network by using voice commands.
`
`BACKGROUND OF THE INVENTION
`
`[003] Currently, three options exist for a user who wishes to gather information from a web
`
`site accessible over the Internet. The first option is to use a desktop or a laptop computer
`
`connected to a telephone line via a modem or connected to a network with Internet access.
`
`The second option is to use a Personal Digital Assistant (PDA) that has the capability of
`
`connecting to the Internet either through a modem or a wireless connection. The third
`
`option is to use one of the newly designed web-phones or web-pagers that are now being
`
`offered on the market. Although each of these options provide a user with access to the
`
`Internet to browse web sites, each of them have their own drawbacks.
`
`[004] Desktop computers are very large and bulky and are difficult to transport. Laptop
`
`computers solve this inconvenience, but many are still quite heavy and are inconvenient to
`
`carry. Further, laptop computers cannot be carried and used everywhere 3 user travels. For
`
`instance, if a user wishes to obtain information from a remote location where no electricity
`
`or communication lines are installed,
`
`it would be nearly impossible to use a laptop
`
`computer. Oftentimes, information is needed on an immediate basis where a computer is
`
`CHICAGO 283215vl 47242-00001
`
`|PR2020-00686
`
`Apple EX1002 Page 4
`
`IPR2020-00686
`Apple EX1002 Page 4
`
`

`

`Patent Application
`Docket No.2 47242-00027USCI
`Customer Number 30223
`
`not accessible. Furthermore, the use of laptop or desktop computers to access the Internet
`
`requires either a direct or a dial-up connection tan an Internet Service Provider (ISP).
`
`Oftentimes, such connections are not available when a user desires to connect
`
`to the
`
`Internet to acquire information.
`
`[005] The second option for remotely accessing web sites is the use of PDAs. These
`
`devices also have their own set of drawbacks. First, PDAs with the ability to connect to the
`
`Internet and access web sites are not readily available. As a result, these PDAs tend to be
`
`very expensive. Furthermore, users are usually required to pay a special service fee to
`
`enable the web browsing feature of the PDA. A further disadvantage of these PDAs is that
`
`web sites must be specifically designed to allow these devices to access information on the
`
`web site. Therefore, a limited number of web sites are available that are accessible by
`
`these web-enabled PDAs. Finally, it is very common today for users to carry cell phones,
`
`however, users must also carry a separate PDA if they require the ability to gather
`
`information from various web sites. Users are therefore subjected to added expenses since
`they must pay for both cellular telephone service and also for the web-enabling service‘for
`
`the PDA. This results in a very expensive alternative for the, consumer.-
`
`[006] The third alternative mentioned above is’the use of web-phones or web-pagers. These
`
`devices suffer many of the same drawbacks as PDAs. First; these devices are expensive to
`
`purchase. Further, the number of web sites accessible to these devices is limited since web
`
`sites must be specifically designed to allow access by these devices. Furthermore, users are
`
`often required to pay an additional fee in order to gain wireless web access. Again, this
`
`service is expensive. Another drawback of these web-phones or web-pagers is that as
`
`technology develops, the methods used by the various web sites to allow access by these
`
`devices may change. These changes may require users to purchase new web-phones or
`
`web-pagers or have the current device serviced in order to upgrade the firmware or
`
`operating system stored within the device. At the least, this would be inconvenient to users
`
`and may actually be quite expensive.
`
`[007] Therefore, a need exists for a system that allows users to easily access and browse the
`
`Internet from any location. Such a system would only require users to have access to any
`
`type of telephone and would not require users to subscribe to multiple services.
`
`[008] In the rapidly changing area of Internet applications, web sites change frequently.
`
`The design of the web site may change, the information required by the web site in order to
`
`CHICAGO 2832 l 5v| 47242-0000l
`
`|PR2020-00686
`
`Apple EX1002 Page 5
`
`IPR2020-00686
`Apple EX1002 Page 5
`
`

`

`Patent Application
`Docket No.: 47242-00027USC1
`Customer Number 30223
`
`perform searches may change, and the method of reporting search results may change.
`
`Web browsing applications that submit search requests and interpret responses from these
`
`web sites based» upon expected formats will produce errors and useless responses when
`
`such changes occur. Therefore, a need exists for a system that can detect modifications to
`
`web sites and adapt
`
`to such changes in order to quickly and accurately provide the
`
`information requested by a user through a voice enabled device, such as a telephone.
`
`[009] When users access web sites using devices such as personal computers, delays in
`
`receiving responses are tolerated and are even expected, however, such delays are not
`
`expected when a user communicates with a telephone. Users expect communications over
`
`a telephone to occur immediately with a minimal amount of delay time. A user attempting
`
`to find information using a telephone expects immediate responses to his search requests.
`
`A system that introduces too much delay between the time a user makes a request and the
`
`time of response will not be tolerated by users and will lose its‘usefulness. Therefore, it is
`important that a voice browsing system that uses telephonic communications selects web
`
`= sites that provide rapid responses since speed is an important factor for maintaining the
`system’s desirability and usability. Therefore, a need exists for a system that accesses web
`
`.|
`
`sites based upon their speed of operation.
`
`SUMMARY OF THE INVENTION
`
`[0010] It is an object of an embodiment of the present invention to allow users to gather
`
`information from web sites by using voice enabled devices, such as wireline or wireless
`
`telephones.
`
`[0011]An additional object of an embodiment of the present invention is to provide a system
`
`and method that allows the searching and retrieving of publicly available information by
`
`controlling a web browsing server using naturally spoken voice commands.
`
`[0012] It is an object of another embodiment of the present invention to provide a robust
`
`voice browsing system that can obtain the same information from several web sites based
`
`upon a ranking order. The ranking order is automatically adjusted if the system detects that
`
`a given web site is not functioning, is too slow, or has been modified in such a way that the
`
`requested information cannot be retrieved any longer.
`
`[0013] A still further object of an embodiment of the present invention is to allow users to
`
`gather information from web sites from any location where a telephonic connection can be
`
`CHICAGO 2832l5vl 47242-0000]
`
`|PR2020-00686
`
`Apple EX1002 Page 6
`
`IPR2020-00686
`Apple EX1002 Page 6
`
`

`

`Patent Application
`Docket No.: 47242-00027USCl
`Customer Number 30223
`
`made.
`
`[0014] Another object of an embodiment of the present invention is to allows users to browse
`
`web sites on the Internet using conversational voice commands spoken into wireless or
`
`wireline telephones or other voice enabled devices.
`
`[0015]An additional object an embodiment of the present invention is to provide a system
`
`and method for using voice commands to control and monitor devices connected to a
`
`network.
`
`[0016] It is an object of an embodiment of the present invention to provide a system and
`
`method which allows devices connected to a network to be controlled by conversational
`
`voice commands spoken into any voice enabled device interconnected with the same
`
`network.
`
`[0017] The present invention relates to a system for acquiring information from sources on a
`
`network, such as the Internet. A voice browsing system maintains a database containing a
`
`list of information sources, such as web sites, connected to a network. Each of the
`
`information sources is assigned a rank number which is listed in the database along with
`
`the record for the information source.
`
`In response to a speech command received from a
`
`user, a network interface system accesses the information source With the highest rank
`
`number in order to retrieve information requested by the user.
`
`[0018] The a preferred embodiment of the present invention allows users to access and
`
`browse web sites when they do not have access to computers with Internet access. This is
`
`accomplished by providing a voice browsing system and method that allows users to
`
`browse web sites using conversational voice commands spoken into any type of voice
`
`enabled device (i.e., any type of wireline or wireless telephone, IP phone, wireless PDA, or
`
`other wireless device). These spoken commands are then converted into data messages by
`
`a speech recognition software engine running on a user interface system. These data
`
`messages are then sent to and processed by a network interface system. This network
`
`interface system then generates the proper requests that are transmitted to the desired web
`
`site over the Internet. Responses sent from the web site are received and processed by the
`
`network interface system and then converted into an audio message via a speech synthesis
`
`engine or a pre-recorded audio concatenation application and finally transmitted to the
`
`user’s voice enabled device.
`
`[0019]A preferred embodiment of the voice browser system and method uses a web site
`
`CHICAGO 2832 I 5v| 47242-00001
`
`|PR2020-00686
`
`Apple EX1002 Page 7
`
`IPR2020-00686
`Apple EX1002 Page 7
`
`

`

`Patent Application
`Docket No: 47242-00027USCI
`Customer Number 30223
`
`polling and ranking methodology that allows the system to detect changes in web sites and
`
`adapt to those changes in real-time. This enables the voice browser system of a preferred
`
`embodiment to deliver highly reliable information to users over any voice enabled device.
`
`This ranking system also enables the present invention to provide rapid responses to user
`
`requests. Long delays before receiving responses to requests are not tolerated by users of
`
`voice-based systems, such as telephones. When a user speaks into a telephone, an almost
`
`immediate response is expected.
`
`This expectation does not exist
`
`for non-voice
`
`communications, such as email transmissions or accessing a web site using a personal
`
`computer.
`
`In such situations, a reasonable amount of transmission delay is acceptable.
`
`The ranking system of implemented by a preferred embodiment of the present invention ‘
`
`ensures users will always receive the fastest possible response to their request.
`
`[0020] An alternative embodiment of the present
`
`invention allows users to control and
`
`monitor the operation of a variety of household devices connected to a network using
`
`speech commands spoken into a voice enabled device.
`
`BRIEF DESCRIPTION OF THE DRAWINGS i
`
`[0021] FIG. 1 is a depiction of the voice browsing system of the first embodiment of the
`
`present invention;
`
`[0022] FIG. 2 is a block diagram of a database record used by the first preferred embodiment
`
`of the present invention;
`
`[0023] FIG. 3 is a block diagram of a media server used by the preferred embodiment;
`
`[0024] FIG. 4 is a block diagram of a web browsing server used by the preferred
`
`embodiment; and
`
`[0025] FIG. 5 is a depiction of the device browsing system of the second embodiment of the
`
`present invention.
`
`DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
`
`[0026] A first embodiment of the present invention is a system and method for allowing users
`
`to browse information sources, such as web sites, by using naturally spoken, conversational
`
`voice commands spoken into a voice enabled device. Users are not required to learn a
`
`CHICAGO 283215vl 47242—0000]
`
`|PR2020-00686
`
`Apple EX1002 Page 8
`
`IPR2020-00686
`Apple EX1002 Page 8
`
`

`

`Patent Application
`Docket No.: 47242-00027USCI
`Customer Number 30223
`
`special language or command set in order to communicate with the voice browsing system
`
`of the present invention. Common and ordinary commands and phrases are all that is
`
`required for a user to operate the voice browsing system. The voice browsing system
`
`recognizes naturally spoken voice commands and is speaker—independent; it does not have
`
`to be trained to recognize the voice patterns of each individual user.
`
`Such speech
`
`recognition systems use phonemes to recognize spoken words and not predefined voice
`
`patterns.
`
`[0027] The first embodiment allows users to select from various categories of information
`
`and to search those categories for desired data by using conversational voice commands.
`
`The voice browsing system of the first preferred embodiment includes a user interface
`
`system referred to as a media server. The media server contains a speech recognition
`
`software engine.
`
`This
`
`speech recognition engine is used to recognize natural,
`
`conversational voice commands spoken by the user and converts them into data messages
`
`based on the available recognition grammar. These data. messages are then sent to a
`
`network interface system.
`
`In the first preferred embodiment, the network interface system
`
`is referred to as a web browsing server. The web browsing server then accesses the
`
`appropriate information source, such as a web site, to gather information requested by the
`USCI‘.
`
`[0028] Responses received from the information sources are then transferred to the media
`
`server where speech synthesis engine converts the responses into audio messages that are
`
`transmitted to the user. A more detailed description of this embodiment will now be
`
`provided.
`
`[0029] Referring to FIG. 1, a database 100 designed by Webley Systems Incorporated is
`
`connected to one or more web browsing servers 102 as well as to one or more media
`
`servers 106. The database may store information on magnetic media, such as a hard disk
`
`drive, or it may store information via other widely acceptable methods for storing data,
`
`such as optical disks. The database 100 contains a separate set of records for each web site
`
`accessible by the system. An example of a web site record is shown in FIG. 2. Each web
`
`site record 200 contains the rank number of the _web site 202, the associated Uniform
`
`Resource Locator (URL) 204, and a command that enables the appropriate “extraction
`
`agent“ 206 that is required in order to generate proper requests sent to and to format data
`
`received from the web site. The database record 200 also contains the timestamp 208
`
`CHICAGO 283215vl 47242-0000]
`
`|PR2020-00686
`
`Apple EX1002 Page 9
`
`IPR2020-00686
`Apple EX1002 Page 9
`
`

`

`Patent Application
`Docket No.: 47242-00027USC1
`Customer Number 30223
`
`indicating the last time the web site was accessed. The extraction agent is described in
`
`more detail below. The database 100 categorizes each database record 200 according to
`
`the type of information provided by each web site. For instance, a first category of
`
`database records 200 may correspond to web sites that provide “weather” information. The
`
`database 100 may also contain a second category of records 200 for web sites that provide
`
`“stock” information. These categories may be further divided into subcategories. For
`
`instance,
`
`the “weather” category may contain subcategories depending upon type of
`
`weather information available to a user, such as “current weather” or “extended forecast”.
`
`Within the “extended forecast” subcategory, a list of web site records may be stored that
`
`provide weather information for multiple days. The use of subcategories may allow the
`
`web browsing feature to provide more accurate, relevant, and up-to-date information to the
`
`user by accessing the most relevant web site. The number of records contained in each
`
`category or subcategory is not limited. In the preferred embodiment, three web site records
`
`are provided for each category.
`
`[0030] Table 1 below depicts two database records 200 that are used with. the preferred
`
`embodiment. These records also contain a field indicating the “category” of the record;
`
`if“!
`
`~ which is “weather” in each of these examples.
`
`TABLE 1
`
`weather
`category:
`URL: URL=http://cgi.cnn.com/cgi—
`bin/weather/redirect?zip=_zip
`rank:
`1
`
`web_dispatch.pl weather_cnn
`command:
`browsingServer: wportall
`browsingServerBackup:
`wporta12
`dateTime: Dec 21 2000
`2:15PM
`
`category: weather
`URL: URL=http://weather.lycos.com/wcfiveday.asp?city=zip
`rank:
`2
`
`command: web_dispatch pl weather_lycos
`browsingServer: wportall
`browsingServerBackup:
`wporta12
`dateTime: Dec 21 2000
`1:45PM
`
`[0031] The database also contains a listing of pre—recorded audio files used to create
`
`CHICAGO 2832 | Svl 472420000|
`
`|PR2020-00686
`
`Apple EX1002 Page 10
`
`IPR2020-00686
`Apple EX1002 Page 10
`
`

`

`PakntAppHcafion
`Docket No.: 47242-00027USC1
`Customer Number 30223
`
`concatenated phrases and sentences. Further, database 100 may contain customer profile
`
`information, system activity reports, and any other data or software servers necessary for
`
`the testing or administration of the voice browsing system.
`
`[0032] The operation of the media servers 106 will now be discussed in relation to FIG.
`
`3.
`
`The media servers 106 function as user interface systems.
`
`In the preferred
`
`embodiment, the media servers 106 contain a speech recognition engine 300, a speech
`
`synthesis engine 302, an Interactive Voice Response (IVR) application 304, a call
`
`processing system 306, and telephony and voice hardware 308 required to communicate
`
`with the Public Switched Telephone Network (PSTN) l 16.
`
`In the preferred embodiment,
`
`each media server is based upon Intel’s Dual Pentium III 730 MHz microprocessor
`
`system.
`
`[0033] The speech recognition function is performed by a speech recognition engine 300
`
`that converts voice commands received from the user’s voice enabled device 112 (i.e.,
`
`any type of wireline or wireless telephone, Internet Protocol (IP) phones, or other special
`wireless units) into data messages.
`'In the preferred embodiment, voice commands and
`audio messages are transmitted using the PSTN’ 116 and data is transmitted using the
`
`TCP/IP communications protocol. However, one skilled in the art would recognize that
`
`other transmission protocols may be used for either voice or data. Other possible
`
`transmission protocols would include SIP/VolP (Session Initiation Protocol/Voice over
`
`IP), Asynchronous Transfer Mode (ATM) and Frame Relay. A preferred speech
`
`recognition engine is developed by Nuance Communications of 1380 Willow Road,
`
`Menlo Park, California 94025 (www.nuance.com).
`
`The Nuance engine capacity is
`
`measured in recognition units based on CPU type as defined in the vendor specification.
`
`The natural speech recognition grammars (i.e., what a user can say that will be recognized
`
`by the speech recognition engine) were developed by Webley Systems.
`
`[0034] Table 2 below provides a partial source code listing of the recognition grammars
`
`used by the speech recognition engine of the preferred embodiment for obtaining weather
`information.
`
`TABLE 2
`
`?WHAT_IS ?the weather ?[info information report conditions]
`?
`(
`(?like in )
`[
`
`CHICAGO 233215v1 47242-00001
`
`|PR2020-00686
`
`Apple EX1002 Page 11
`
`IPR2020-00686
`Apple EX1002 Page 11
`
`

`

`Us c i t: i e s 2 n
`
`Patent Application
`Docket No: 47242—00027USC1
`Customer Number 30223
`
`<paraml $n.zip> <param2
`
`$n.city> <param3
`
`{<param1 $n>}
`(area code) AREA_CODE:n )
`AREA_CODE:n (area code)
`) {<param1 $n>}
`(zip ?code)
`ZIP_CODE:n )
`{<param1 $n>}
`ZIP_CODE:n (zip ?code)
`)
`{<param1 $n>}
`
`)
`
`{<menu 194>}
`
`[0035]
`
`The media server 106 uses recognition results generated by the speech recognition
`
`engine 300 to retrieve a web site record 200 stored in the database 100 that can provide
`
`the information requested by the user. The media server 106 processes the recognition
`
`result data identifying keywords that are used to search the web site records 200
`
`contained in the database 100 For instance, if the user’s request was “What is the weather
`in‘Chicag07”, the keywords “weather” and “Chicago” would be recognized. A web site
`record 200 with the highest rank number from thev“weather” category within the database 3'
`100 would then be selected and transmitted to the web browsing server 102 along with an
`identifier indicating that Chicago weather is being requested.
`.
`
`[0036] The media servers 106 also contain a speech synthesis engine 302 that converts
`
`the data retrieved by the web browsing servers 102 into audio messags that are
`
`transmitted to the user’s voice enabled device 112. A preferred speech synthesis engine
`
`is developed by Lernout and Hauspie Speech Products, 52 Third Avenue, Burlington,
`
`Massachusetts 01803 (www.1hsl.com).
`
`[0037] A further description of the web browsing server 102 will be provided in relation
`
`to FIG. 4. The web browsing servers 102 provide access to any computer network such
`
`as the Internet 1 10. These servers are also capable of accessing databases stored on Local
`
`Area Networks (LANs) or Wide Area Networks (WANs). The web browsing servers
`
`receive responses from web sites and extract the data requested by the user. This task is
`
`also known as “content extraction.” The web browsing servers 102 also perform the task
`
`of periodically polling or “pinging” various web sites an