PTO/AIA/15 (10-17)
`Approved for use through 11/30/2020. 0MB 0651-0032
`U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless it displays a valid 0MB control number.
`I 149369-00103
`Attorney Docket No.
`UTILITY
`I Mark Catchpole
`First Named Inventor
`PATENT APPLICATION
`TRANSMITTAL
`
`PROCESSORS
`
`ie
`
`T(tl I SPEECH RECOGNITION CIRCUIT USING PARALLEL
`Priority Mail Express" I
`
`(Only for new nonprovisional applications under 37 CFR 1.53/b))
`
`Label No.
`
`APPLICATION ELEMENTS
`See MPEP chapter 600 concerning utility patent application contents.
`
`1. □ Fee Transmittal Form
`(PTO/SB/17 or equivalent)
`
`2. · 0 Applicant asserts small entity status.
`
`See 37 CFR 1.27
`□ Applicant certifies micro entity status. See 37 CFR 1.29.
`Applicant must attach form PTO/SB/15A or B or equivalent.
`
`3.
`
`4. 0 Specification
`5. 0 Drawing(s) (35 U.S.C. 113)
`
`[Total Pages
`Both the claims and abstract must start on a new page.
`(See MPEP § 608.01/a) for information on the preferred arrangement)
`6
`1
`[Total Pages
`6. Inventor's Oath or Declaration
`(including substitute statements under 37 CFR 1.64 and assignments
`serving as an oath or declaration under 37 CFR 1.63/e))
`
`[Total Sheets
`
`19
`
`l
`
`l
`l
`
`a. D Newly executed (original or copy)
`b. 0 A copy from a prior application (37 CFR 1.63(d))
`7. 0 Application Data Sheet
`
`• See note below.
`
`8.
`
`See 37 CFR 1.76 (PTO/AIN14 or equivalent)
`CD-ROM or CD-R
`in duplicate, large table, or Computer Program (Appendix)
`
`D Landscape Table on CD
`
`ADDRESS TO:
`
`Commissioner for Patents
`P.O. Box 1450
`Alexandria, VA 22313-1450
`ACCOMPANYING APPLICATION PAPERS
`10. D Assignment Papers
`
`(cover sheet & document(s))
`
`Name of Assignee
`
`•
`
`(when there is an assignee)
`
`(PTO/SB/06 or PTO-1449)
`
`11 D 37 CFR 3.73(c) Statement
`0 Power of Attorney
`12. D English Translation Document (if applicable)
`13. 0 Information Disclosure Statement
`D Copies of citations attached
`14. 0 Preliminary Amendment
`15. D Return Receipt Postcard
`16. D Certified Copy of Priority Document(s)
`17 D Nonpublication Request
`
`(MPEP § 503) (Should be specifically itemized)
`
`(if foreign priority is claimed)
`
`•
`
`Under 35 U.S.C. 122(b)(2)(B)(i). Applicant must attach form PTO/SB/35
`or equivalent.
`
`18. Oother:
`
`9, Nucleotide and/or Amino Acid Sequence Submission
`(if applicable, items a. - c. are required)
`
`ii. 0Paper
`
`a. D Computer Readable Form (CRF)
`b. D Specification Sequence Listing on:
`i. D CD-ROM or CD-R (2 copies); or
`c. D Statements verifying identity of above copies
`
`0 The address associated with Customer Number:
`Name Stephen A. Soffen
`BLANK ROME LLP
`Address 1825 Eye Street, NW
`WashinQton
`City
`
`I
`*Note: (1) Benefit claims under 37 CFR 1.78 and foreign priority claims under 1.55 must be included in an Application Data Sheet (ADS).
`(2) For applications filed under 35 U.S.C. 111, the application must contain an ADS specifying the applicant if the applicant is an
`assignee, person to whom the inventor is under an obligation to assign, or person who otherwise shows sufficient proprietary
`interest in the matter. See 37 CFR 1.46(b).
`19. CORRESPONDENCE ADDRESS
`24998
`
`I
`
`OR D Correspondence address below
`
`State I
`Telephone I
`' us
`L\k
`.,,_ - ,-
`Ct✓\L,-~ __.,,---·
`.
`'
`I
`Stephen A. Soffen
`
`f)
`
`DC
`(202) 420-2200
`Date
`
`I Zip Code
`I Email
`
`Registration No.
`(Attorney/Agent)
`
`20006-5403
`
`February 4, 2019
`
`31,063
`
`Country
`
`Signature
`
`Name
`(Print/Type)
`
`149369.00101 /117148002v.1
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 1
`
`

`

`Application Data Sheet
`
`Inventor Information
`
`Inventor Number::
`
`Given Name::
`
`Family Name::
`
`City of Residence::
`
`Country of Residence::
`
`Street of mailing address::
`
`City of mailing address::
`
`Country of mailing address::
`
`1
`
`Mark
`
`Catchpole
`
`Prickwillow
`
`United Kingdom
`
`Malahax, Ely Road
`
`Prickwillow
`
`United Kingdom
`
`Postal or Zip Code of mailing address::
`
`CB? 4UJ
`
`Correspondence Information
`
`Correspondence Customer Number::
`
`24998
`
`Application Information
`
`Application Type::
`
`Subject Matter::
`
`CD-ROM or CD-R?::
`
`Sequence submission?::
`
`Computer Readable Form (CRF)?::
`
`Title::
`
`Regular
`
`Utility
`
`None
`
`None
`
`No
`
`SPEECH RECOGNITION CIRCUIT
`
`USING PARALLEL PROCESSORS
`
`149369.00101/117147630v.1
`
`Page# 1
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 2
`
`

`

`Attorney Docket Number::
`
`149369-00103
`
`Request for Early Publication?::
`
`Request for Non-Publication?::
`
`Total Drawing Sheets::
`
`Small Entity?::
`
`Petition included?::
`
`No
`
`No
`
`6
`
`Yes
`
`No
`
`Portions or all of the application associated with this No
`Application Data Sheet may fall under a Secrecy
`
`Order pursuant to 37 CFR 5.2::
`
`Representative Information
`
`Representative Customer Number::
`
`24998
`
`Domestic Priority Information
`
`Application::
`
`Continuity Type::
`
`Parent Application::
`
`This Application
`
`Continuation of
`
`15/392,396
`
`15/392,396
`
`Continuation of
`
`14/309,476
`
`Continuation of
`
`13/253,223
`
`Continuation of
`
`12/554,607
`
`Continuation of
`
`14/309,476 (now U.S.
`Patent No. 9,536,516)
`
`13/253,223 (now U.S.
`Patent No. 8,768,696)
`
`12/554,607 (now U.S.
`Patent No. 8,036,890)
`
`10/503,463 (now U.S.
`Patent No. 7,587,319)
`
`Parent Filing
`Date::
`
`12/28/2016
`
`06/19/2014
`
`10/05/2011
`
`09/04/2009
`
`05/24/2005
`
`10/503,463
`
`371 National
`Stage of
`
`PCT /GB2003/000459
`
`02/04/2003
`
`149369.00101/117147630v.1
`
`Page# 2
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 3
`
`

`

`Foreign Priority Information
`
`Country
`
`Application No::
`
`Filing Date::
`
`United Kingdom
`
`0202546.8
`
`02/04/2002
`
`Priority
`Claimed::
`
`Yes
`
`Applicant Information
`
`Applicant Number::
`
`Applicant Type::
`
`Organization Name::
`
`Street of mailing address::
`
`City of mailing address::
`
`1
`
`Assignee
`
`Zentian Limited
`
`17 Paddock Row
`
`Elsworth
`
`Cambridge
`
`Country of mailing address::
`
`United Kingdom
`
`Postal or Zip Code of mailing address::
`
`CB23 4JG
`
`149369.00101/117147630v.1
`
`Page# 3
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 4
`
`

`

`Authorization or Opt-Out of Authorization to Permit Access
`
`When this Application Data Sheet is properly signed and filed with the application, applicant has provided written
`authority to permit a participating foreign intellectual property (IP) office access to the instant application-as-filed (see
`paragraph A in subsection 1 below) and the European Patent Office (EPO) access to any search results from the instant
`application (see paragraph B in subsection 1 below).
`
`Should applicant choose not to provide an authorization identified in subsection 1 below, applicant must opt-out of the
`authorization by checking the corresponding box A or B or both in subsection 2 below.
`
`1
`
`NOTE: This section of the Application Data Sheet is ONLY reviewed and processed with the INITIAL filing of an
`application. After the initial filing of an application, an Application Data Sheet cannot be used to provide or rescind
`authorization for access by a foreign IP office(s). Instead, Form PTO/SB/39 or PTO/SB/69 must be used as appropriate.
`
`1. Authorization to Permit Access by a Foreign Intellectual Property Office(s)
`
`A. Priority Document Exchange (POX) - Unless box A in subsection 2 (opt-out of authorization) is checked, the
`undersigned hereby grants the USPTO authority to provide the European Patent Office (EPO), the Japan Patent Office
`(JPO), the Korean Intellectual Property Office (KIPO), the State Intellectual Property Office of the People's Republic of
`China (SIPO), the World Intellectual Property Organization (WIPO), and any other foreign intellectual property office
`participating with the USPTO in a bilateral or multilateral priority document exchange agreement in which a foreign
`application claiming priority to the instant patent application is filed, access to: (1) the instant patent application-as-filed
`and its related bibliographic data, (2) any foreign or domestic application to which priority or benefit is claimed by the
`instant application and its related bibliographic data, and (3) the date of filing of this Authorization. See 37 CFR 1.14(h)
`(1 ).
`
`B. Search Results from U.S. Application to EPO - Unless box B in subsection 2 (opt-out of authorization) is checked,
`the undersigned herebv grants the USPTO authority to provide the EPO access to the bibliographic data and search
`results from the instant patent application when a European patent application claiming priority to the instant patent
`application is filed. See 37 CFR 1.14(h)(2).
`
`The applicant is reminded that the EPO's Rule 141 (1) EPC (European Patent Convention) requires applicants to submit a
`copy of search results from the instant application without delay in a European patent application that claims priority to
`the instant aoolication.
`
`2. Opt-Out of Authorizations to Permit Access by a Foreign Intellectual Property Office(s)
`
`D application-as-filed.
`
`A. Applicant DOES NOT authorize the USPTO to permit a participating foreign IP office access to the instant
`If this box is checked, the USPTO will not be providing a participating foreign IP office with
`any documents and information identified in subsection 1A above.
`
`B. Applicant DOES NOT authorize the USPTO to transmit to the EPO any search results from the instant patent
`
`D application. If this box is checked, the US PTO will not be providing the EPO with search results from the instant
`
`application.
`
`NOTE: Once the application has published or is otherwise publicly available, the USPTO may provide access to the
`application in accordance with 37 CFR 1.14.
`
`149369.00101/117147630v.1
`
`Page# 4
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 5
`
`

`

`Signature:
`
`NOTE: This Application Data Sheet must be signed in accordance with 37 CFR 1.33(b). However, if this Application
`Data Sheet is submitted with the INITIAL filing of the application and either box A or B is not checked in
`subsection 2 of the "Authorization or Opt-Out of Authorization to Permit Access" section, then this form must
`also be signed in accordance with 37 CFR 1.14(c).
`This Application Data Sheet must be signed by a patent practitioner if one or more of the applicants is a juristic
`entity (e.g., corporation or association). If the applicant is two or more joint inventors, this form must be signed by a
`patent practitioner, fill joint inventors who are the applicant, or one or more joint inventor-applicants who have been given
`power of attorney (e.g., see USPTO Form PTO/AIA/81) on behalf of all joint inventor-applicants,
`See 37 CFR 1.4(d) for the manner of making signatures and certifications.
`
`Signature
`
`Date (YYYY-MM-DD)
`
`2019-02-04
`
`Name
`
`Stephen A. Soffen
`
`Registration Number
`
`31,063
`
`149369.00101/117147630v.1
`
`Page# 5
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 6
`
`

`

`Docket No.: 149369-00103
`(PATENT)
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`In re Patent Application of:
`Mark Catchpole
`
`Application No.: Not Yet Assigned
`
`Confirmation No.: NIA
`
`Filed: Concurrently Herewith
`
`Art Unit: NIA
`
`For: SPEECH RECOGNITION CIRCUIT USING
`PARALLEL PROCESSORS
`•
`
`Examiner: Not Yet Assigned
`
`FIRST PRELIMINARY AMENDMENT UNDER 37 C.F.R. 1.115
`
`MS Amendment
`Commissioner for Patents
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`Dear Madam:
`
`INTRODUCTORY COMMENTS
`
`Prior to examination on the merits, please amend the above-identified U.S. patent
`
`application as follows:
`
`Amendments to the Specification begin on page 2 of this paper.
`
`Remarks/ Arguments begin on page 3 of this paper.
`
`149369.00101/117148204v.1
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 7
`
`

`

`Application No. Not Yet Assigned
`First Preliminary Amendment
`
`DocketNo.: 149369-00103
`
`AMENDMENTS TO THE SPECIFICATION
`
`Page 1, after the title, please insert the following:
`
`This is a continuation of Application No. 15,392,396, filed December 28, 2016, which is
`
`a continuation of Application No. 14/309,476, filed June 19, 2014, now U.S. Pat. No. 9,536,516,
`
`which is a continuation of Application No. 13/253,223, now U.S. Pat. No. 8,768,696, filed October
`
`5, 2011, which is a continuation of Application No. 12/554,607, now U.S. Pat. No. 8,036,890, filed
`
`September 4, 2009, which is continuation of Application No. 10/503,463, now U.S. Pat. No.
`
`7,587,319, filed May 24, 2005, which is a 371 oflnternational Application No.
`
`PCT/GB2003/000459, filed February 4, 2003, which claims foreign priority to United Kingdom
`
`Application No. 0202546.8 filed February 4, 2002, the disclosures of which are incorporated herein
`
`by reference in their entireties.
`
`149369.00101/117148204v. I
`
`2
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 8
`
`

`

`Application No. Not Yet Assigned
`First Preliminary Amendment
`
`DocketNo.: 149369-00103
`
`REMARKS
`
`This application, which is a continuation, has been amended to add a reference to the
`
`parent applications in accordance with 35 U.S.C. 120.
`
`Dated: February 4, 2019
`
`Respectfully submitted,
`
`By.~
`Stephen A. Soffen
`Registration No.: 31,063
`DICKSTEIN SHAPIRO LLP
`1825 Eye Street, NW
`Washington, DC 20006-5403
`(202) 420-2200
`Attorney for Applicant
`
`149369.00101/117148204v.1
`
`3
`
`DSMDB-3174234 vl
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 9
`
`

`

`1
`
`SPEECH RECOGNITION CIRCUIT USING PARALLEL PROCESSORS
`
`The present invention generally relates to a speech recognition circuit which uses
`parallel processors for processing the input speech data in parallel.
`
`Conventional large vocabulary speech recognition can be divided into two processes:
`front end processing to generate processed speech parameters such as feature vectors,
`followed by a search process which attempts to find the most likely set of words spoken
`from a given vocabulary (lexicon).
`
`The front end processing generally represents no problem for current processing
`systems. However, for large vocabulary, speaker independent speech recognition, it is
`the search process that presents the biggest challenge. An article by Deshmukh et al
`entitled "Hierarchical Search for Large-Vocabulary Conversational Speech
`Recognition" (IEEE Signal Processing Magazine, September 1999, pages 84 to 107),
`the content of which is hereby incorporated by reference, discusses the general concepts
`oflarge vocabulary speech recognition. As discussed in this paper, one algorithm for
`perfonning the search is the Viterbi algorithm. The Viterbi algorithm is a parallel or
`breadth first search through a transition network of states of Hidden Markov Models.
`An acoustic model for words in a lexicon are represented as states-of Hidden Markov
`Models. These states represent phones or n phones in a phone model of the words. The
`search requires the evaluation of possible word matches. It is known that such a search
`is computationally intensive.
`
`In order to speed up the processing performed during such a search in a speech
`recognition system, parallel processing has been explored. In an article by
`MK Ravishankar entitled "Parallel Implementation of Past Beam Search for Speaker(cid:173)
`Independent Continuous Speech Recognition" (Indian Institute of Science, Bangalor,
`India, July 16, 1993) a multi-threaded implementation of a fast beam search algorithm is
`disclosed. The multi-threading implementation requires a significant amount of
`communication and synchronization among threads. In an MSC project report by
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 10
`
`

`

`2
`
`R Dujari entitled "Parallel Viterbi Search Algorithm for Speech Recognition" (MIT,
`February 1992) the parallel processing of input speech parameters is disclosed in which
`a lexical network is split statically among processors.
`
`It is an object of the present invention to provide an improved circuit which can perform
`parallel processing of speech parameters.
`
`In accordance with a first embodiment of the present invention, a speech recognition
`circuit comprises an input port such as input buffer for receiving parameterized speech
`data such as feature ve_ctors. A lexical memory arrangement is provided which contains
`lexicon data for word recognition. The lexical data comprises a plurality of lexical tree
`data structures representing a plurality oflexical trees. Each lexical tree data structure
`comprises a model of words having common prefix components and an initial
`component which is unique as an initial component for lexical trees. A plurality of
`lexical tree processors are connected in parallel to the input port and perform parallel
`lexical tree processing for word recognition by accessing the lexical data irr the lexical
`memory arrangement A results memory arrangement is connected to the lexical tree
`processors for storing processing results from the lexical tree processors and lexical tree
`identifiers to identify lexical trees to be processed by the lexical tree processors. A
`controller controls the lexical tree processors to process lexical trees identified in the
`results memory arrangement by performing parallel processing of a plurality of lexical
`tree data structures.
`
`Thus in accordance with this embodiment of the present invention, the processing in
`order to perform word recognition is distributed across the processors by controlling the
`processors to perform processing on different lexical trees. The controller controls the
`processor by the processes to provide for efficient process management by distributing
`lexical processing to appropriate processors.
`
`The lexical tree data structure can comprise a phone model of words, wherein the
`components comprise phones. For reduced storage, the lexical tree data structure can
`comprise a mono phone lexical tree. The mono phone lexical tree can be used to
`generate context dependent phone models dynamically. This enables the use of context •
`dependent phone models for matching and hence increased accuracy whilst not
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 11
`
`

`

`increasing memory requirements. Alternatively, the lexical tree data structure can
`comprise context dependent phone models.
`
`3
`
`The processing performed by each processor in one embodiment comprises the
`comparison of the speech parameters with the lexical data, e.g. phone models or data
`derived from the lexical data (e.g. dynamically generated context dependent phone
`models) to identify words as a word recognition event and to send information
`identifying the identified words to the results memory as the processing results. In this
`embodiment a language model processor arrangement can be provided for providing a
`
`language model output for modifying the processing results at a word recognition event
`by a lexical tree processor. The modification can either take place at each lexical tree
`processor, or at the language model processing arrangement.
`
`In one embodiment each lexical tree processor determines an output score for words in
`the processing results at word recognition events. Thus in this embodiment the
`language model processing arrangement can modify the score using a score for a
`language model for n preceding words, where n isan integer.
`
`In one embodiment the controller instructs a lexical tree processor to process a lexical
`tree by passing a lexical tree identifier for the lexical tree and history data for a
`recognition path associated with the lexical tree from the results memory. The history
`data preferably includes an acewnulated score for the recognition path. This enables a
`score to be determined based on the scor-e for the recognition path to accumulate a new
`score during recognition carried out using the lexical tree data structure. The scores can
`be output in the processing results to the results memory during the processing of the
`speech parameters so that the scores can be used for pruning.
`
`In one embodiment of the present invention, each lexical tree processor operates on
`more than one lexical tree at the same time, e.g~ two lexical trees represented by two
`different lexical tree data structures, or two lexical trees represented by the same data
`structure but displaced in time (which can-be termed to instances of the same lexical
`tree).
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 12
`
`

`

`4
`
`At word recognition events, the controller determines new Jexical tree identifiers for
`storing in the results memory for words identified in the results memory for respective
`word events. In order to reduce the processing, the controller can prune the new lexical
`tree identifiers to reduce the number ofJexical trees which are required to be processed.
`This pruning can be achieved using context dependant n phones to reduce the number of
`possible next phones. The number can be further reduced by using a language model
`look ahead technique.
`
`In one embodiment of the present invention, the lexical tree processors are arranged in
`groups or clusters. The lexical memory arrangement comprises a plurality of partial
`lexical memories. Each partial lexical memory is connected to one of the groups of
`lexical tree processors and contains part of the lexical data. Thus a group of lexical tree
`processors and a partial lexical memory form a cluster. Each lexical tree processor is
`operative to process the speech parameters using a partial lexical memory and the
`controller controls each lexical tree processor to process a lexical tree corresponding to
`partial lexical data in a corresponding partial lexical memory.
`
`In another embodiment of the present invention the lexical memory arrangement
`comprises a plurality of partial lexical memories. Each partial lexical memory being
`connected to one of the lexical tree processors and containing part of the lexical data.
`Each lexical tree processor processes the speech parameters using a corresponding
`partia1 lexical memory and the controller is operative to control each lexical tree
`processor to process a lexical tree corresponding to partial lexical data in a
`corresponding partial lexical memory.
`
`In one embodiment of the present invention the lexical memory arrangement stores the
`lexical tree data structures as Hidden Markov Models and the lexical tree processors are
`operative to perfonn the Viterbi search algorithm using each respeetive lexical tree data
`structure. Thus in this way, this embodiment of.the present invention provides a
`parallel Viterbi lexical tree search process for speech recognition.
`
`The first aspect of the present invention is a special purpose circuit built for performing
`the speech recognition search process in which there are a plurality of processors for
`performing parallel lexical tree processing on individual lexical tree processors.
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 13
`
`

`

`5
`
`In another aspect of the present invention a speech recognition circuit comprises an
`input port such as-an input buffer for reGeiving parameterized speech data such as
`feature vectors. A plurality of lexical memories are provided which contain in
`combination complete lexical data for word recognition. Each lexical memory contains
`part of the complete lexical data. A plurality of processors are provided connected in
`parallel to the input port for processing the speech parameters in parallel. The
`processors are arranged in groups in which each group is connected to a corresponding
`lexical memory to form a cluster. A controller controls each processor to process the
`speech parameters using partial lexical data read from a corresponding lexical memory.
`The results of processing the speech parameters are output from the processors as
`recognition data.
`
`Thus this aspect of the present invention provides a circuit in which speech recognition
`processing is performed in parallel by groups of processors operating in parallel in
`which each group accesses a common memory of!exical data. This aspect of the
`present invention provides the advantage of parallel processing of speech parameters
`and benefits from a limited segmentation of the lexical data. By providing a plurality of
`processors in a group with a common memory, flexibility in the processing is provided
`without being bandwidth limited by the interface to the memory that would occur if
`only a single memory were used for all processors. The arrangement is more flexible
`than the parallel processing arrangement in which each processor only has access to its
`own iocal memory and requires fewer memory interfaces (i.e. chip pins). Each
`processor within a group can access the same lexical data as any other processor in the
`group. The controller can thus control the parallel processing of input speech
`parameters in a more flexible manner. For example, it allows more than one processor
`to process input speech parameters using the same lexical data in a lexical memory.
`This is because the lexical data is segmented into domains which are accessible by
`multiple processors.
`
`In a preferred embodiment this aspect of the present invention is used in combination
`with the first aspect of the present invention. In such an arrangement each processor
`performs lexical tree processing and the lexical data stored in each lexical memory
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 14
`
`

`

`comprises lexical tree data structures which each comprise a model of words having
`common prefix components and an initial component thatis unique.
`
`6
`
`In preferred embodiments of the second aspect of the present invention, the preferred
`embodiments of the first aspect of the present invention are incorporated.
`
`Embodiments of the present invention will now be described withTeference to the
`accompanying drawings in which:
`
`Figure 1 is a diagram of a speech data processing circuit for generating parameterized
`speech data (feature vectors);
`
`Figure 2 is a diagram of a speech recognition circuit in accordance with an embodiment
`of the present invention;
`
`Figures 3a and 3b are schematic diagrams illustrating lexical tree structures;
`
`Figure 4 is a flow diagram illustrating the process perfonned by a lexical tree processor
`to determine a temporary lexical tree score in accordance with an embodiment of the
`present invention;
`
`Figure 5 is a flow diagram illustrating the process perfonned by the lexical tree
`processor for processing the input feature vectors in accordance with an embodiment of
`the present invention; and
`
`Figure 6 is a flow diagram illustrating the process perfonned by the controller in
`accordance with an embodiment of the present invention.
`
`Figur-e 1 illustrates a typical circuit for the parameterization of input speech data. In this
`embodiment the parameters generated are speech vectors.
`
`A microphone 1 records speech in an analogue form and this is input through an anti(cid:173)
`aliasing filter 2 to an analogue-to-digital converter 3 which samples the speech at
`48 kHz at 20 bits per sample. The digitized output signal is normalized (4) to generated
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 15
`
`

`

`7
`
`a 10 millisecond data fra."lle every 5 milliseconds with 5 milfrseconds overlap (5). A
`pre-emphasis operation 6 is applied to the data followed by a hamming window 7. The
`data is then fast Fourier transfonned (FF1) using a 512 point fast Fourier transform (8:)
`before being filtered by filter bank 9 into 12 frequencies. The energy in the data frame
`5 is also recorded (13) as an additional feature and together with the 12 frequency
`outputs of the filter bank 9, 13 feature vectors (10) are thus produced and these are
`output as part of the 39 feature vectors 14. First and second derivatives (11 and 12) are
`taken of the 13 feature vectors 10 to complete the generation of the 39 feature vectors
`14.
`
`The arrangement illustrated in Figure 1 is purely given for illustration. The present
`invention encompasses any means by which speech and data can be parameterized to a
`suitable form for input to the search process as will be described in more detail
`hereinafter.
`
`Figure 2 is a schematic diagram of a speech recognition circuit in accordance with an
`embodiment of the present invention for performing the search process. The
`parameterized speech data, which in this embodiment comprise feature vectors, are
`input to a feature vector buffer 20. The feature vector buffer 20 is provided to buffer the
`incoming feature vectors to allow lexical tree processors 21 to read and process the
`feature vectors in the buffer 20 via a feature vector bus 24. A plurality k of lexical tree
`processors 21 are arranged in a respective lexical tree processor cluster 22. Each lexical
`tree processor cluster 22 has an acoustic model memory 23 in which is stored lexical
`data for use by the lexical tree processors 21 within the lexical tree processor cluster 22.
`Each lexical tree processor 21 in the lexical tree processor cluster 22 is connected to the
`acoustic model memory 23 within the lexical tree processor 22. There are N lexical tree
`processor clusters and thus there are Nk lexical tree processors 21 connected by the
`feature vector bus 24-to the feature vector buffer 20. Bach lexical tree processor21 is
`capable of processing a different lexical tree and thus Nk lexical trees can be processed
`in parallel. The acoustic model memories 23 store as a whole a complete set of lexical
`data, i.e. lexical tree data structures for use in the lexical tree processing by the lexical
`tree processors 21. Each acoustic model memory 23 contains part or a segment of the
`lexical tree data. Since lexical tree processors 21 in a lexical tree processor cluster 22
`access the same acoustic model memory 23, it is possible for more than one lexical tree
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 16
`
`

`

`8
`
`processor 21 to process the same lexical data. This provides for some degree of
`flexibility in the controlling of the processing by the lexical tree processors 21. -.Further,
`the acoustic model memories 23 need not contain only one copy of the lexical data. It is(cid:173)
`possible to build in a redundancy in the data to further enhance the flexibility. This
`avoids any bottleneck in the processing due to the search processing focusing on a small
`number oflexical trees.
`
`A results memory 25 is provided for storing processing results from the lexical tree
`processors 21 which are received over the path score and history bus 26. The results
`memory 25 also stores information on lexical trees to identify which lexical trees are to
`be processed. A search controller 27 is provided to control the processing performed by
`the lexical tree processors 21 in dependence upon a program and data stored in program
`and data memory 28. The search controller reads the path scores and lexical tree
`identifiers from the results memory and controls the le:irical tree processors accordingly.
`A language model processor 29 is provideff which is connected to each lexical tree
`processor 21 by a language model bus 30. The language model processor 29 accesses a(cid:173)
`language model memory 31 to read language model data for provision to lexical tree
`processors 21 in response to language model data requests. External control of the
`language model memory 31 is provided by a word constrains input. The language
`model processor 29 determines a score for a word occurring following N previous
`words using N grams. When a lexical tree processor requires a language model score a
`request is sent to the language model processor 29 over the language model bus 30
`identifying the current-word and the N-1 previous words. A language model score for
`the N gram can be returned to the lexical tree processor 21 for the modification of the
`score at the end of a branch of lexical tree processing. The lexicaLtree processor can
`modify the score in accordance with the language model and output a score to the
`results memory 25 for a word at the end of a branch of the lexical tree processing. Thus
`the results memory stores the results as an ordered list of scores for words together with
`their histories.
`
`The results memory 25 stores the foJlowing data:
`
`Initial lexical tree data. This comprises pointers to an initial set of lexical trees.
`1.
`No history data is associated with the initial set of lexical trees. The initial set of lexical
`
`Amazon / Zentian Limited
`Exhibit 1002
`Page 17
`
`

`

`9
`
`trees is predetermined and stored in the results memory 25 based on the most likely
`initial phones of an utterance. This initial lexical tree data is required to initialize the
`search process.
`
`2.
`
`History data for search results. This comprises a record of a recognition path
`
`through the lexical tree recognition process perform