I 1111111111111111 11111 111111111111111 1111111111 1111111111111111 IIII IIII IIII
`
`
`
`
`
`US007979277B2
`
`c12) United States Patent
`Larri et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 7,979,277 B2
`Jul. 12, 2011
`
`(54) SPEECH RECOGNITION CIRCUIT AND
`METHOD
`
`(75)
`
`Inventors: Guy Larri, Cambridge (GB); Mark
`Catchpole, Cambridge (GB); Damian
`Kelly Harris-Dowsett, Coventry (GB);
`Timothy Brian Reynolds, Cambridge
`(GB)
`
`(73) Assignee: Zentian Limited, Cambridge (GB)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1043 days.
`
`(21) Appl. No.:
`
`11/662,704
`
`(22) PCT Filed:
`
`Sep.14,2005
`
`(86) PCT No.:
`
`PCT /GB2005/003554
`
`§ 371 (c)(l),
`(2), ( 4) Date: Mar. 14, 2007
`
`(87) PCT Pub. No.: WO2006/030214
`
`PCT Pub. Date: Mar. 23, 2006
`
`(65)
`
`Prior Publication Data
`
`US 2008/0255839 Al
`
`Oct. 16, 2008
`
`(30)
`
`Foreign Application Priority Data
`
`Sep. 14, 2004
`
`(GB) ................................... 0420464.0
`
`(51)
`
`Int. Cl.
`(2006.01)
`GJ0L 15100
`(52) U.S. Cl. ....................................................... 704/238
`(58) Field of Classification Search ........................ None
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,881,312 A
`3/1999 Dulong
`7,328,153 B2 *
`2/2008 Wells et al ....................
`
`704/231
`
`OTHER PUBLICATIONS
`
`Glinski et al., "Spoken Language Recognition on a DSP Array Pro(cid:173)
`cessor" IEEE Transactions on Parallel and Distributed Systems, vol.
`5, No. 7, pp. 697-703, Jul. 1, 1994.
`
`* cited by examiner
`
`Primary Examiner - Susan McFadden
`(74) Attorney, Agent, or Firm - Dickstein Shapiro LLP
`
`(57)
`
`ABSTRACT
`
`A speech recognition circuit comprising a circuit for provid(cid:173)
`ing state identifiers which identify states corresponding to
`nodes or groups of adjacent nodes in a lexical tree, and for
`providing scores corresponding to said state identifiers, the
`lexical tree comprising a model of words; a memory structure
`for receiving and storing state identifiers identified by a node
`identifier identifying a node or group of adjacent nodes, said
`memory structure being adapted to allow lookup to identify
`particular state identifiers, reading of the scores correspond(cid:173)
`ing to the state identifiers, and writing back of the scores to the
`memory structure after modification of the scores; an accu(cid:173)
`mulator for receiving score updates corresponding to particu(cid:173)
`lar state identifiers from a score update generating circuit
`which generates the score updates using audio input, for
`receiving scores from the memory structure, and for modify(cid:173)
`ing said scores by adding said score updates to said scores;
`and a selector circuit for selecting at least one node or group
`of adjacent nodes of the lexical tree according to said scores.
`
`16 Claims, 27 Drawing Sheets
`
`107
`
`102
`
`100
`
`101
`
`D.:9:t~~
`DigH~l
`!~.t~~ik: D:::ta A;.Jr!-:o Oat::~
`
`I
`I
`I
`Disto:nr:es
`C;-f Pft)bcth1:-:t:n;:: Of
`Uk~':-:,l:!·if)(:-d:::.
`
`R.:,,,;ry,r;js,::,:j
`T:::xt
`
`■ Functicms pertor-rned by th~ ,:1,x:d€~rator am shown in wey
`0 FtineHons pe,forrrn,,(l by other hardwam or t:<inv,3re t;y~,!em~; are ~;hown in >Nhit,~
`
`IPR2023-00034
`Apple EX1001 Page 1
`
`

`

`0 ....
`....
`('D .....
`rJ'1 =(cid:173)
`
`('D
`
`-....J
`N
`
`....
`0 ....
`~ = :-' ....
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`■-1,, m
`CD
`C: ..,
`.,,
`
`(C
`-■
`
`flash memory
`storage e.g.
`non volatile
`
`main memory
`
`debug
`display,
`host,
`
`model(s)
`grammar
`
`models
`speech
`
`level
`grammar
`
`j/f
`
`front end
`
`audio
`
`i/f
`
`codec
`audio
`
`level
`system
`
`CODEC
`
`audio
`AC97
`
`.. I
`
`input
`audio
`
`generator
`~ I graph
`Word
`
`word lattice I
`
`one best
`n best list
`word graph
`
`Word Engine
`
`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`rJl.
`d
`
`figure 1b
`Joins to
`
`___
`
`I
`
`I s@a vame:
`word lat_ node _id
`worcl history
`
`..
`
`A
`
`word_;d • I
`
`score
`
`start_frame
`mph
`prev _ wordlat_node _id
`
`word level
`
`IPR2023-00034
`Apple EX1001 Page 2
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`N
`('D .....
`rJJ =(cid:173)
`
`('D
`
`-....J
`N
`
`•
`from path merge unit
`
`in wave front
`
`next phone no
`last {Jhone_no
`
`token -
`
`...,__
`
`PINE 1
`
`,... ________
`
`Level
`State'---~----(cid:173)
`
`fnter oord token passing
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`C'"
`~
`
`(I)
`C -,
`tC
`"T1 -·
`
`figure 1a
`-+--------,Joins to
`
`(E)
`
`(C)(D}
`
`B}
`
`prev_pho1!._~no'"
`grammar.Ji/
`g~/11mar_nods~Jd .
`next}eflmp/l_jd.
`current iwrdlat node id,
`currentLT..:..node_jd
`
`-
`
`RAM
`
`phone record
`wave front
`
`manager
`Front
`Wave
`
`IPR2023-00034
`Apple EX1001 Page 3
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`Joins to Figure 2b
`
`word/at_node_id = 5
`
`neither(word_id = 3556)
`
`word/at node id =6
`
`-
`
`-
`
`(word_id = 0)
`
`0 ....
`('D .....
`rJJ =(cid:173)
`
`-....J
`N
`
`~
`
`('D
`
`which (word_id = 238)
`
`wore lattice
`
`silence < wordlat_node_id = 4
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`r»
`I\)
`C'D
`C: -,
`"Tl -·
`(C
`
`LT_id=4
`grammar_node_id = 9
`word_id= O
`silence
`9
`
`g
`
`LT_
`gra
`won
`neit,
`25
`
`7
`
`LT_id=4
`grammar_node_id = 7
`word_id= 0
`silence
`7
`
`LT_id=3
`grammar_node_id = 17
`word_id =45
`house
`
`LT_id= 3
`grammar_node_id = 23
`word_id = 238
`which
`
`grammar id = 2
`
`level
`
`s;Jrc1mmar
`
`IPR2023-00034
`Apple EX1001 Page 4
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`rJl.
`d
`
`-...J
`N
`
`0 ....
`('D .....
`rJJ =(cid:173)
`
`.i;...
`
`('D
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`C"
`~
`al
`C
`"Tl -·
`(C
`
`trans /m_factor
`
`0
`
`0
`
`1
`
`0
`
`3456 456 7112
`
`345
`
`320
`
`on wavefront
`
`indictates phone is
`
`237
`
`22
`
`prev_phone no wordlat_node_id trl'l_ns lm_factor
`
`phone no st id1 st id2 st id3 bac/{_front empty end
`
`phone instance RAM
`
`"either" ______ _
`
`-@_-
`
`-o_
`
`"either"
`
`-@-
`
`al -0
`
`"neither"
`
`-@ -
`
`"neither"
`
`D -@ -
`
`n < I -D
`
`al-
`
`"-z-"house"
`
`"house"
`
`s-
`
`............ "which"
`
`~"witch•
`
`h-au<
`
`w-J-+S
`
`sil
`
`lexical trees
`
`Joins to Figure 2a
`
`phone level
`
`2
`
`567
`45
`
`grammar_id I gra,timar:.:_nodeJif LT{/ci,:if
`.:_ ,,.l~fv'lc> ?
`'.'
`ptione r~cotd ;
`·•wave ft6tf.)1<ii!'.
`
`phone_flO
`
`phone CAM
`wavefront
`
`IPR2023-00034
`Apple EX1001 Page 5
`
`

`

`U.S. Patent
`
`Figure 3a
`
`Jul. 12, 2011
`
`Sheet 5 of 27
`
`US 7,979,277 B2
`
`phone_no
`
`phone_no
`
`Write
`data
`
`Write
`address
`
`phone Instance
`CAM
`Kwords deep
`
`next
`phone
`request
`
`next
`phones
`
`next_empty
`address
`
`next phone's
`phone_no
`
`22b
`
`13b
`
`13b
`
`13b
`
`1b
`
`phone_no
`
`next phone's
`phone_no
`
`mod N counter
`generates sequence of all __
`state ids
`
`_.
`
`mod N counter
`is synchronous
`to MHD bus
`
`repeat count
`fifo
`
`MHDBus
`N state output probabillities per frame
`
`tokens requests
`from other PINE
`
`tokens to other
`.. PWEs
`
`global beam width
`
`~..,
`
`.. .,_ ______________________
`
`~~~-
`◄ .. 1-'=~~~;.;.._---------------K
`
`_.;;;;st:;;;a.:.;;te_t_~a_n_si-tio_n_m_o_d_e_Is .... _________________________
`
`CAM abbreviations
`phone_no - unique phone instance indentifier
`st_id1 - state_identifier1
`back(wave_back_flag) - phone is at back of wave
`front(wave_front_flag) - phone is at the front of wave
`end(word_end_f/ag) - phone is the last in a word
`oc - occupy - indicates that location has valid phone
`instance (it may not have any tokens)
`
`output liklihood
`
`-10
`• 0
`0
`~©
`
`RAM abbreviations
`st_dum - dummy state to hold incoming token
`st_ 1 - location for token in state 1, includes updated flag
`st 2 - location for token in state 2, includes updated flag
`st.3 - location for token in state 3, includes updated flag
`tmp - location for temporary storage of token
`prev_phone_no - pointer to previous phone instance
`trans (trans_vec_id) - pointer to state transition vector in
`state transition RAM
`LM_factor - language model factor
`end - word end phone
`
`IPR2023-00034
`Apple EX1001 Page 6
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 6 of 27
`
`US 7,979,277 B2
`
`Figure 3b
`
`® pointer to previous phone
`
`st_dum
`
`st_1
`
`st_2 st_3
`
`11,p prev_phone_no wordlat_node_id
`
`trans LM_factor end
`
`st_dum
`
`s1_1
`
`s1_2 st_3
`
`wordlat_node_id
`
`trans
`
`new model
`
`Write
`data
`
`phone Instance RAM
`model instances
`Kwords deep
`
`new
`model
`address
`
`encoded
`address
`output
`
`17b
`
`17b
`
`17b 16b
`
`22b
`
`16b
`
`6b
`
`16b
`
`1b
`
`address
`
`Read Port
`
`match
`fifo
`
`updated
`token
`scores
`
`output
`liklihood
`
`H J---------IW
`0..------!._
`
`trans
`
`state
`transition ------(cid:173)
`RAM
`
`state
`transition
`liklihood
`
`path score accumulator
`
`____________
`
`_
`
`@1------------------------:,
`
`Pruning logic
`
`IPR2023-00034
`Apple EX1001 Page 7
`
`

`

`~ = ~
`.,,
`
`~
`~
`~
`•
`00
`~
`
`(D
`C: -,
`-■ cc
`
`0 ....
`--..J
`('D .....
`rJJ =(cid:173)
`
`('D
`
`--..J
`N
`
`....
`0 ....
`
`~
`
`N
`N
`
`:-' ....
`2'
`
`I gramrr1er_id lword_id lwd_hist1 lwd_hist2 I st_frame
`
`next_leftmdh_id I LT _node_idlgrammer_node~!d
`
`RAM
`maps to
`address
`
`CAM
`
`wordlat_node_id
`
`single match
`Kwordsdeep
`
`binary
`
`CAM
`phone
`
`wave front
`
`data
`match
`
`1
`
`phone_no
`
`fife
`
`phone_na
`
`nexl_empty_address
`
`I
`
`empty count
`
`manager
`occupancy
`CAM RAM
`
`control
`
`word_end_flag
`
`..
`
`1
`
`request
`phone
`next
`word end flag
`word/at node id
`phone no
`
`I
`
`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`wd-=_hist2
`wd hist1
`word id
`to path merge unit
`
`st_frame
`word id
`word/at node id
`to word engine
`
`-
`
`grammar_id
`grammar_ node _id
`to grammar engine
`
`next_leflmp/J_id
`LT node id
`to phone_book
`
`wordTat_node_id(not stricUy needed)
`word id
`grammar_id
`grammac._node_id
`LT node id
`
`phone_no
`word id
`grammar_id
`grammar:node _id
`LT node id(cid:173)
`next leftmph id
`from phone book
`
`10b
`
`20b
`
`20b
`
`20b
`
`4b
`
`10b
`
`20b
`
`13b
`
`~
`
`kinds of phones -word end phones and intra word phones
`
`-possibly reducing memory requirements
`
`NB The CAM-RAM structure can be split to handle to
`
`Kwordsdeep
`
`wave front phone record RAM
`
`Write data
`
`address
`
`I I wrfte data L:I
`
`-
`
`i 1b
`
`19b
`
`write address
`
`IPR2023-00034
`Apple EX1001 Page 8
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 8 of 27
`
`US 7,979,277 B2
`
`Figure 5
`
`situation at frame m
`
`word
`
`(0 0 0) • • • 0 0 Q14---link--~.....---.-~
`
`first
`trip hone
`in word
`
`last
`triphone
`in word
`
`node
`
`word
`end
`
`situation at frame n
`(n>m)
`
`word
`end
`
`word
`start
`
`(0 00)
`
`triphone
`
`word k
`
`• • • 000
`
`word k (with identical word history)
`
`(0 0 0) • • • co O 0)
`
`triphone
`
`0
`00
`•
`•
`•
`
`0
`00
`•
`•
`•
`
`IPR2023-00034
`Apple EX1001 Page 9
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 9 of 27
`
`US 7,979,277 B2
`
`Figure 6
`
`word id
`wd Filst1
`
`__ w,_d;;,.-~h,1,;is.,;t2------
`
`B
`
`.. match data
`
`Path Merge
`CAM
`
`link node id
`
`encoded
`address
`output
`
`Path Merge
`RAM
`Kwordsdeep
`
`address
`
`CAM RAM
`occupancy
`manager
`empty_count
`next_empty
`a==a:d:d:re:s:s=J-J-.,➔11 write address
`
`Kwordsdeep
`single match
`binary
`
`---------------;::~L,;lin~k;:;;,;_node_~
`
`from wave front manager
`word id
`wd filst1
`wd-hist2
`prev _phone_ no
`
`counter
`
`to Path Merge Unit in
`PINEs
`Link node id
`prev _phone :no
`
`IPR2023-00034
`Apple EX1001 Page 10
`
`

`

`-....l = N
`
`-....l
`N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`('D a ....
`rJ'1 =(cid:173)
`
`0
`
`-....J
`N
`
`....
`0 ....
`~ = :-....
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`----1
`CD
`C: -,
`-· cc
`.,,
`
`* state_ids(x3), transvec_id, word_end_f/ag, ful/_trans_flag
`
`prev_phone_no, phone_no, wordlat_node_id
`
`I
`
`phone model
`
`l---------==------------,
`
`manager
`•vavefront
`
`word/at_node_id I to
`
`•
`phone no
`
`word id
`grammar_,
`d.d
`
`transvec_id
`
`phone_no
`
`word_end_flag
`prev_phone_no
`
`3 x 13 bits= 39 bits
`state_ids (x3)
`
`data
`
`(counter)
`generator
`
`phone number
`
`address to l
`
`lookup RAM
`
`state id
`
`deep
`64k
`
`16 bits
`address
`
`deep
`
`address 11 256k
`
`data
`RAM
`lookup
`
`to
`
`triphone
`
`'{raW?f!/l,r~~ode _i
`
`I I
`
`B
`
`6,6,6 = 18 bits
`triphone_id
`
`memory
`
`transvec_id
`
`monophone id
`central
`
`phone generator
`
`word id
`
`-
`
`next_leftmph_id
`prev _phone_no
`current grammar_id
`current grammar noife id
`current word/at node id
`current LT node id
`
`-
`
`instruction: address offset
`data: monophone _id/word _id
`
`(lexical tree address)
`LT_node_id
`
`grammar_id
`grammar_node _id
`LT_node_id
`RAM abbreviations_
`phone_no -unique phone instance indentifier
`CAM abbreviations
`
`data_
`
`address
`
`data fields
`
`instruction and
`
`ROM with
`
`micro-coded
`lexical tree
`
`current grammar_node _id
`current grammar _id
`
`..
`
`grammar_node_id
`grammar_id
`
`IPR2023-00034
`Apple EX1001 Page 11
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`....
`....
`('D .....
`rJ'1 =(cid:173)
`
`('D
`
`-....J
`N
`
`....
`0 ....
`N
`~
`:-' ....
`2'
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`.,,
`
`tu
`00
`(I)
`C: -,
`(C
`-■
`
`not used and is indicated by "O".
`relative addr:6. The third available branch is
`The second branch (to "dd") is given as
`branch (here to "cc") is implied at addr+1.
`instruction (at addr:1) such that the first
`The "01" branch is a single element
`
`00 r:
`
`d
`b
`
`7
`
`cc
`bb
`
`2
`
`01
`
`1
`
`bb
`aa
`00
`
`0
`
`aa
`
`aa -bb -dd
`/ cc
`
`"01" -two branches.
`
`??
`
`... -I
`c_p -
`?
`bb
`00
`
`bJJ
`aa
`00
`
`sequence implied.
`Next element in
`
`aa
`
`aa -bb-cc
`
`2
`
`0
`
`"00" -Single branch, ie. mono-phone to mono-phone.
`
`W4
`
`11
`
`17
`
`mm
`kk
`00
`
`16
`
`w3
`
`11
`
`15
`
`jj Yo kk
`
`II
`
`kk
`
`00
`
`14
`
`01
`
`13
`
`00
`
`12
`
`ii
`ii
`00
`
`11
`
`ii
`hh
`00
`10
`
`~
`
`hh
`bb
`00
`9
`
`~ 3,J
`
`-word end.
`
`11 vertices).
`10 -Mbranch (more than 3 exit
`01 -branch (up to 3 exit vertices).
`00 -monophone and unigram weight.
`
`W4
`
`W3
`
`w2
`
`w1
`
`s...,.
`
`5
`
`w2
`
`11
`
`8
`
`gg
`ff
`00
`
`7
`
`ff
`bb
`00
`6
`
`w1
`
`11
`
`5
`
`ee
`dd
`00
`
`4
`
`sil
`
`'mm-
`
`II -sil
`
`jj -kk-
`sil
`
`Lexical Tree RAM
`
`~
`dd
`cc
`00
`
`00
`
`01
`
`3
`
`2
`
`1
`
`~c
`
`--
`--, aa Ya bb
`
`bb
`
`00
`0
`
`0,
`
`Sil
`
`cc -dd -ee -
`
`(monophone tree)
`
`ii -
`ff -9g -
`
`'hh-
`
`aa -bb -
`/
`
`IPR2023-00034
`Apple EX1001 Page 12
`
`

`

`--...l = N
`
`--...l
`'N
`\0
`--...l
`\0
`'-"--...l
`d r.,;_
`
`0 ....
`N
`....
`('D .....
`rJJ =(cid:173)
`
`('D
`
`-....J
`N
`
`....
`0 ....
`N
`j'-J
`:-' ....
`2'
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`,,
`
`C"'
`00
`(D
`C: -,
`-■ cc
`
`is complete.
`
`the Mbranch instruction
`branch instruction, "O" implies that
`
`to addr:4. The final branch is to
`
`10 I 00 I l7o1 ffA ~ Rn addr:27, and the other available
`27 l The Mbranch directs the first branch
`
`_
`b
`
`b
`
`ff
`
`e
`b
`
`21
`
`15
`
`7
`
`4
`
`3
`
`1 Hffi o
`
`b
`
`0
`
`5 (odd)
`
`aa
`
`gg
`
`~ff
`
`-ee
`?dd
`
`cc
`
`aa -bb
`
`"10" -Mbranch: 5 (odd)
`
`that this is the end of the Mbranch.
`order of the given addresses is a code to indicate
`and gives rel branch addr 19 and 13. The (reverse)
`Mbranch instruction continues at addr+1 (addr:2),
`the second, in this example, at addr:6. The
`is at addr+1, so it is given ("cc" at rel addr:2). And
`The Mbranch can not assume that the first branch
`
`21 -00 -bb
`
`ff
`
`15 -00 -bb
`
`ee
`
`00
`
`7 -
`
`00
`3
`
`10
`2
`
`10
`1
`
`bb
`aa
`00
`0
`
`aa
`
`4 (even)
`
`cc
`
`ff
`"ee
`~
`aa -bb-dd
`/
`
`"10" -Mbranch: 4 (even)
`
`"01" instruction) branch is given as relative addr:14.
`as relative addr:6. The third (and final branch in the
`implied at addr+1. The second branch (to "dd") is given
`addr:1) such that the first branch (here to "cc") is
`(at
`The "01" branch is a single element instruction
`
`~ ~ b
`
`d
`b
`
`15
`
`7
`
`cc
`bb
`00
`3
`
`01
`
`1
`
`bb
`aa
`00
`0
`
`aa
`
`ee
`
`bb-dd '
`
`/
`
`cc
`
`aa-
`
`Lexical Tree RAM continued
`
`"01" -three branches.
`
`IPR2023-00034
`Apple EX1001 Page 13
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 13 of 27
`
`US 7,979,277 B2
`
`Figure 9
`data structure for word link record to enable word n-best
`decoding
`
`memory address
`word id
`pointer mph start frame end frame score
`
`• • •
`
`pointer mph start frame end frame score
`
`mph is the monophone
`id for the right context
`of the last triphone of
`the current word .
`
`0
`Sil
`null sil 1 3 0 0
`
`2
`
`seven
`0 t 4 12 -20
`0 t 4 13 -20
`0 t 4 14 -20
`
`4
`two
`2 f 11 19 55
`2 f 11 20 60
`
`3
`two
`f 13 17 -64
`
`5
`ten
`1f1317-60
`1 th 13 18 -55
`1f1519-56
`
`6
`five
`4 s 20 27 -75
`3s1827-80
`
`7
`five
`5s1827-79
`5 s 20 27 -82
`
`8
`three
`5 s 19 27 -79
`
`Notes
`
`1. each record represents the passage of a single token through a word with a particular right monophone
`context.
`2. each word's acoustic model with a particular right context monophone can only generate one token per
`frame. Thus frame number can be used to distinguish trace back paths with the same end_frame
`numbers but different right context monophones
`3. the end_frame doesn't need to be passed to the PINEs because it is simply the current frame number
`4. scores may be absolute or relative
`5. the address is referred to as the wordlat_node_id elsewhere
`
`Traceback
`
`for the n-best tokens
`1. using the associated wordlat_node_ids look up each word link record
`2. look at associated records and use the end_frame number and rnonophone id to select associated
`pointer and score.
`3. trace back recursively and select n-best paths
`
`IPR2023-00034
`Apple EX1001 Page 14
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 14 of 27
`
`US 7,979,277 B2
`
`Figure 10
`
`word id
`20b
`
`I head wlr ptr I
`I
`I
`13b
`
`spare
`
`pointer prev worldat node id mph start frame end frame score
`10b
`6b
`10b
`10b
`16b
`13b
`
`--
`war dlat_node id
`- translation
`
`logical to
`physical
`address
`
`memory
`
`physical
`address
`of word
`-
`record ~ address
`
`word link record memory
`
`Bk deep ----------..
`)
`
`next spare
`location
`
`t
`
`control
`
`empty count
`
`~ two
`pointer
`~
`pointer
`pointer
`
`-.:::
`
`I head wlr ptr I
`2
`4
`2
`
`t
`t
`t
`
`spare
`
`11
`12
`12
`
`17
`17
`18
`
`-55
`-55
`-55
`
`word/at_ node_ id
`
`h
`
`: counter I
`
`~ -
`
`add new record
`to existing record:
`new record:
`prev_wordlat_node_id word_id
`mph
`head_reaord_ptr
`start_frame
`end_frame
`score
`
`, '
`
`read record during traceback
`or read when adding WLR to
`existing word record
`
`IPR2023-00034
`Apple EX1001 Page 15
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`r.,;_
`d
`
`0 ....
`Ul
`....
`('D .....
`=-
`
`('D
`
`rJJ
`
`-....J
`N
`
`I
`
`~
`C'D
`-,
`C =
`(C
`-■ ~
`~
`"Tl ~
`•
`00
`~
`
`~
`
`~
`
`....
`....
`:-' ....
`~ ~ =
`
`0
`N
`N
`
`~
`
`RAM
`
`LM factor
`
`...... ~
`output ""
`address
`encoded
`
`1 ~ address
`
`I
`
`-
`
`I
`
`LM_factor
`
`encoder
`
`-address
`
`RAM
`maps to
`address
`CAM
`
`I
`
`II
`
`empty_count
`
`binary
`
`single match
`1 k words deep
`
`CAM
`
`LM factor
`
`I
`
`CAM RAM occupancy I
`
`manager
`
`B
`
`outputs
`line
`k match
`
`-w--,-,
`
`•wd_hist1 wd_hist2
`
`-
`
`-match data
`
`wd_hist2
`wd hist1
`LT_node_id
`triphone_id
`
`-
`
`I 20b
`
`write data
`
`I
`
`LM_factor
`wd_hist2
`wd_hist1
`LT_node_id
`triphone_id
`fromARM9
`
`I 18b
`next_empty address I j I • 1 write address
`
`~ r 1 K words deep
`~
`
`16b
`
`I
`
`I
`
`~ I Write data
`
`I
`
`~ I I
`.------1 11
`
`LM_factor
`to Phone Bookl
`
`I
`11 I
`
`-
`
`I 20b I 20b I 1b
`
`IPR2023-00034
`Apple EX1001 Page 16
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 16 of 27
`
`US 7,979,277 B2
`
`Figure 12
`
`start
`
`one
`
`two
`
`zero
`
`•••
`
`lipsmack ..,_ _____
`
`_
`
`breath
`
`sil
`
`IPR2023-00034
`Apple EX1001 Page 17
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 17 of 27
`
`US 7,979,277 B2
`
`Figure 13
`
`grammar
`memory
`
`address
`
`j
`
`..
`
`grammar_node_id
`
`data
`grammar_id
`1, grammar_node id
`
`current grammar_id
`current grammar node id -
`....
`
`control
`
`next grammar_node_id(s)
`next word_id(s)
`1 ' next L T_id
`next LT_node_,d(s)
`
`IPR2023-00034
`Apple EX1001 Page 18
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0
`
`1
`
`2
`
`3
`
`n.n.ru,_
`
`16
`
`17
`
`19 18
`
`n.n.ru,_
`
`(12-288 MHz, 81.4ns)
`
`BIT CLK
`
`SDATA IN
`
`0 ....
`....
`('D .....
`rJJ =(cid:173)
`
`QO
`
`('D
`
`-....J
`N
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`(D
`-,
`C
`-■ cc
`,,
`
`I~
`
`12
`►
`
`I I I I
`
`I I I
`
`I I I
`
`I
`
`11
`
`10
`
`9
`
`8
`
`7
`
`6
`
`5
`
`4
`
`3
`
`2
`
`1
`
`20.8 us (48 kHz), 12 x 20 bit slots+ 16 bit tag slot= 256 bits
`
`one codec :frame
`
`I t
`
`ltladdldat
`
`( codec output)
`SDATA IN
`
`(codec input)
`SDATA OUT
`
`0
`◄
`
`n
`
`SYNC
`
`Slot#
`
`IPR2023-00034
`Apple EX1001 Page 19
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`....
`('D .....
`rJJ =(cid:173)
`
`1,0
`
`('D
`
`-....J
`N
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`(J1
`.Ji..
`
`I
`
`I
`:
`I
`
`I
`
`1
`I
`I
`
`~ = ~
`.,,
`
`~
`~
`~
`•
`00
`~
`
`(I)
`C: -,
`-■ cc
`
`idle
`
`frame N
`
`l
`
`j8192j
`I
`
`feature vectors
`
`token passing
`
`!
`I
`I
`I
`I
`I
`:
`pruning thresholds calculated tot
`ax model, word end and beani
`
`next frame
`
`LM factors used during creation of phone
`
`word link records added
`
`return next phone responses
`receive next phone requests
`
`1
`
`RAM depth
`Branch Processor
`
`beam prune, token score update
`
`:
`
`instance RAM location
`Branch Processor model -
`
`:
`,
`
`....
`,8-19-2+I------id-le-------1I
`
`!3538!
`
`MHD processing latency
`
`IO I 1 I 2 I !1271
`
`:-
`I
`:
`
`digits management
`
`resource
`!3538!
`
`connected
`
`memory to MHD transfer latency
`
`front end to MHD transfer latency
`
`scale
`not to
`
`frame 1
`
`! Io j 1 ! 2 I l121j
`
`I
`:
`,
`
`I
`a-
`
`+-+i
`I
`:
`:
`
`I
`
`I
`
`j f
`
`i-
`11
`
`LM 1/F
`
`word engine
`
`phone book
`
`PINE array
`
`MHD engine
`
`_.
`
`_.
`
`mean & covar reg
`
`feature vector reg
`
`frame_count
`
`push_ to_ dictate_ button
`
`IPR2023-00034
`Apple EX1001 Page 20
`
`

`

`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`0)
`~
`
`(C
`~
`
`~,s
`
`C ;
`
`mocle!s
`\!ark)U8
`
`'107
`
`Ad:ive st.ate iist
`
`108
`
`Acoustic rnnde!
`
`104
`
`103
`
`105
`
`·102
`
`Tt:Xt
`
`Rt::icognised
`
`Ukelihoods
`
`or ProbabHities or
`
`[JistatK::es
`
`* t
`
`l
`
`\/ ~;}chJrs
`F i.::,,.1ctUf(t
`
`*
`f
`l
`
`~
`
`m
`
`~tHfJrc.h t3tage
`
`Front End
`
`/.1,,lKiio Data
`
`l\w:iio Data
`
`rngii.a!
`
`O!gita!
`
`+ .1 I
`
`I
`§
`
`i
`i
`l
`
`♦ I Buffer I
`
`Data
`Audio
`
`AOC
`
`+
`
`/~Ge
`
`101
`
`·mo
`
`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`rJl.
`d
`
`0 ....
`0
`N
`('D .....
`rJJ =(cid:173)
`
`('D
`
`-....J
`N
`
`. .:-
`••1 M.,t-,{¼f~
`
`'-!'-.:'"<:: ;;?-.· '.f•:;fi,L
`
`,:.,•h{)l<l.tfi
`
`-..;.~ 1v-n-....;.~ "W'tf,-..
`
`,::,n:,
`
`□ F'•"'"Ci'or0•• r,c:,~·c,,r;--,.v;;,·I ~.)·,v o· t!-J$:>·r· h·•rr4•H•>rf'< or C-<"1_.ts.-~,-:-:.r,:::. <n'C'.'t··O<'•'!';:,
`l:j:ji~i~I
`
`'v.~~~
`
`....... y~;,;,
`
`-.;❖,;,. .. H\.-i<;❖ ,,. •
`
`..
`
`_,,;t".~ ~..t¥¥"t'.-~ ~.,:,
`
`:-w-
`
`.... -...t t .}!
`
`t .... •··•~;-3~~,.J ~fh
`
`)¢">;
`
`,A~,:~-··~
`
`Functions performed by the acce!eratcn are sho1.vn in grey
`
`IPR2023-00034
`Apple EX1001 Page 21
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 21 of 27
`
`US 7,979,277 B2
`
`FIGURE 17
`
`Interface to CPU subsystem and/or DSP
`subsystem
`
`A single bus has been shown for
`simplicity. In reality there will be data,
`address, and control signals.
`
`212
`
`_________________________
`
`/200
`
`l
`
`___
`
`_
`
`203
`
`Memory
`A
`
`Memory
`B
`
`Control,
`Sequencing
`and
`Address
`Generation
`
`Distance
`Calculation
`Eugine
`
`202
`
`211
`
`208
`
`209
`
`FV
`Memory
`A
`
`FV
`Registers
`and
`optional
`FIFO
`
`210
`
`206
`
`CRC
`Check
`
`Decompressor
`
`207
`
`Acoustic
`Model Memory
`
`205
`
`Distance Accelerator Subsystem
`L----------------------------------------------
`
`IPR2023-00034
`Apple EX1001 Page 22
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`N
`N
`('D .....
`rJJ =(cid:173)
`
`('D
`
`-....J
`N
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`00
`...Ji.,
`
`using dists(FV5)
`for data frame 5
`Search Stage
`
`using dists(FV4)
`for data frame 4
`Search Stage
`
`using dists(FV3)
`for data frame 3
`Search Stage
`
`using dists(FV2)
`for data frame 2
`Search Stage
`
`using dists(FV1)
`for data frame 1
`Search Stage
`
`Dists(FV5) -> mem A
`Cale all dist. for FV5
`
`Dists(FV4) -> mem B
`Cale all dist. for FV4
`
`Dists(FV3) -> mem A
`Cale all dist. for FV3
`
`Dists(FV2) -> mem B
`Cale all dist. for FV2
`
`dists(FV1)
`
`Dists(FV1) -> mem A
`Cale all dist. for FV1
`
`Data Frame 5 -> FV5
`
`Data Frame 4 -> FV4
`
`Data Frame 3 -> FV3
`
`Data Frame 2-> FV2
`
`FV1
`
`Data Frame 1 -> FV1
`
`0::
`0. (/) 1--
`Q) +-'
`= a, E
`~ c.. Q)
`0::
`0. en 1--
`= a, E
`c:: c.. Q)
`Q)
`a:
`0. en I--
`c:: Q. Q) = a, E
`a:
`~00 i=
`= ai E
`c:: Q. Q)
`Q)
`a:
`~00 i=
`= ai E
`c:: Q. Q)
`Q)
`
`·-
`
`·-
`
`Q) +-'
`
`·-
`
`Q) +-'
`
`Q)
`
`0::
`0.. en 1--
`= ai E
`c:: 0.. Q)
`Q)
`a.
`.9-
`I-(cid:173)
`Q) 00 ·(cid:173)
`= ai E
`c:: Q. Q)
`Q)
`
`Q) +-'
`
`·-
`
`ciJ
`~ = ~
`C:
`-· cc
`.,,
`
`~
`~
`~
`•
`00
`~
`
`Search stage on CPU
`
`Distance Cale Accelerator
`
`Front end on DSP
`
`Time
`
`IPR2023-00034
`Apple EX1001 Page 23
`
`

`

`.., =
`
`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`r.,;_
`d
`
`....
`
`0
`~
`N
`('D .....
`rJJ =- ('D
`
`-....J
`N
`
`....
`....
`
`~
`
`0
`N
`N
`
`....
`=
`
`:-'
`~
`
`~
`
`~
`~
`~
`~
`•
`00
`~
`
`(0
`~
`CD
`C:
`(.Q
`-■
`'Tl
`
`Dists(FV6) -> mem B ~ Read dists from mem A
`Cale all dist. for FV6
`
`1!1
`
`I
`
`u'"'"V-vu 1
`
`Release memory B
`Read dists from mem 8
`Search Stage 6
`Release memory A
`1
`Search Stage 5
`
`18 Release memory B
`
`release@
`
`;-----
`
`memory B release
`@ awaiting result
`Acceier'ator stalled
`
`awaiting distances
`
`Read dists from mem B
`
`I Search Stage 4
`© Search stalled
`
`Release memory B
`Read dists from mem B
`Search Stage 2
`
`I Search Stage 3
`
`Release memory A
`Read dists from mem A
`
`awaitin distances
`
`0 Search stalled
`
`Release memory A
`Read dists from mem A
`Search Stage 1
`
`I OISlSlt"VO)
`I Cale all dist. for FV4 I C11StS(t-V4)
`e Accelerator stalled
`
`ready
`
`-are
`
`Dists(FV5) -> mem A
`I Cale all dist. for FV5
`
`ready
`
`,1
`
`~
`
`Dists(FV4) -> mem B
`
`awaiting next FV
`
`I=\/~
`
`Data Frame 6 -> FV6 I _
`
`C:\/&:
`
`Data Frame 5 -> FV5 I _
`
`FV4
`
`Dists(FV3)-> mem A
`ia{esl --Cale all dist. for FV3
`
`'v"
`
`Data Frame 4 -> FV4 I
`
`Cale all dist. for FV2 I dists(FV2)
`
`ro!:!rh,
`
`~ro
`
`-
`
`Dists(FV2) -> mem B
`
`Dists(FV1) -> mem A
`Cale all dist. for FV1
`
`Stall delivering FV3
`Data Frame 3 -> FV3
`Data Frame 2 -> FV2
`Data Frame 1 -> FV1
`
`Search stage on CPU
`
`Distance Cale Accelerator
`
`Front end on DSP
`
`Time
`
`IPR2023-00034
`Apple EX1001 Page 24
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`.i;...
`N
`('D .....
`rJJ =(cid:173)
`
`('D
`
`-....J
`N
`
`....
`0 ....
`N
`~
`:-' ....
`2'
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`0
`l\l
`(1)
`C: -,
`(.Q
`-■
`-n
`
`dists(FV6)
`
`~
`
`I Search Stage, data frame 6 , ...
`
`using dists{FV6)
`using dists{FV6)
`
`ge, data frame 6
`
`Cale all dist. for FV6
`
`"" dists(FV5)
`
`using dists(FV5)
`search Stage, data frame 5
`
`...
`
`FV6
`
`FE Data Frame 6 -> FV6
`
`Cale all dist. for FV5
`
`"' dists(FV4)
`
`~
`
`...
`
`FV5
`
`Cale all dist. for FV4
`
`~
`
`...
`
`FV4
`
`-
`
`~ dists(FV3)
`
`Cale all dist. for FV3
`
`Cale all dist. for FV2
`
`... dists(FV2)
`
`...
`
`FV3
`
`... dists(FV1)
`
`using dists(FV4)
`Search Stage, data frame 4
`
`FE Data Frame 5 -> FV5
`
`using dists{FV3)
`Search Stage, data frame 3
`
`FE Data Frame 4 -> FV4
`
`using dists(FV2)
`Search Stage, data frame 2
`
`FE Data Frame 3 -> FV3
`
`using dists(FV1)
`Search Stage, data frame 1
`
`...
`
`FV2
`
`FE Data Frame 2 -> FV2
`
`Cale all dist. for FV1
`
`...
`
`, __
`
`FE Data Frame 1 -> FV1
`
`I
`
`o::
`l-
`0..CI)
`--
`Q) -
`= Q) E
`C 0. Q)
`Q)
`
`0::
`g_u5 i=
`= a, E
`0.. Q)
`
`C
`Q)
`
`0::
`g_u:i i=
`= Q) E
`C 0.. Q)
`Q)
`
`o:
`l-
`--
`= Q) E
`C 0.. Q)
`Q)
`
`0..CI)
`Cl) -
`
`ii:
`I-
`0..CI)
`--
`Q) -
`= a, E
`~ 0.. Q)
`
`a:
`0.. Cl) l-
`--
`Q) -
`= cu E
`
`C 0.. Q)
`Q)
`
`ii:
`g_ u:i i=
`~ 0.. Q) = Q) E
`o:
`= a> E
`C 0.. Q)
`Q)
`
`0.. (/)
`Q) -
`
`l-
`--
`
`Distance Cale Accelerator
`
`Search stage on CPU
`
`Front end and
`
`Time
`
`IPR2023-00034
`Apple EX1001 Page 25
`
`

`

`U.S. Patent
`
`Jul. 12, 2011
`
`Sheet 25 of 27
`
`US 7,979,277 B2
`
`FIGURE 21
`
`Time
`
`Software on CPU
`
`Accelerator
`
`Front End
`Data Frame 1 ➔ FVl
`
`Front End
`Data Frame 2 ➔ FV2
`Prep search stage 1
`
`Search stage 1
`Read distances from
`memA
`
`Front End
`Data Frame 3 ➔ FV3
`
`Search stage 2
`Read distances from
`memB
`
`Front End
`Data Frame 4 ➔ FV 4
`Search stage 3
`Read distances from mem A
`Release memorv A
`
`Front End
`Data Frame 5 ➔ FV5
`
`Prf':n Sf':~rr.h st~crf': 4
`
`Search stage 4
`Read distances from
`memB
`
`0 I
`
`0 I
`
`Cale all dist. for FVl
`Dists(FV 1) ➔ mem A
`
`Cale all dist. for FV2
`Dists(FV2) ➔ mem B
`
`Cale all dist. for FV3
`Dists(FV3) ➔ mem A
`
`Cale all dist. for FV 4
`Dists(FV 4) ➔ mem B
`
`Cale all dist. for FV5
`Dists(FV5) ➔ mem A
`
`IPR2023-00034
`Apple EX1001 Page 26
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`O'I
`N
`('D .....
`rJJ =(cid:173)
`
`('D
`
`-....J
`N
`
`....
`0 ....
`:-' ....
`2'
`
`N
`N
`
`~
`
`using dists(FV5)
`for data frame 5
`Search Stage
`
`using dists(FV4)
`for data frame 4
`Search Stage
`
`using dists(FV3)
`for data frame 3
`Search Stage
`
`using dists(FV2)
`for data frame 2
`Search Stage
`
`using dists(FV1)
`for data frame 1
`Search Stage
`
`dists(FV5)
`
`Accelerator I,""
`
`dist result
`Transfer
`
`from
`
`dists(FV4)
`
`Accelerator I,""
`
`dist result
`Transfer
`
`from
`
`dis~V3)
`
`from
`
`dist result
`Transfer
`Accelerator ,
`dist result
`Transfer
`
`dis~V2)
`
`from
`
`'-"--
`
`I
`
`Accelerator II"
`
`dist result
`Transfer
`
`from
`
`V" ~
`
`'
`
`,..
`...
`.,.
`Accelerator ,, ..
`,, ...
`..
`...
`---
`-..
`
`r
`dists(FV5)
`
`dists(FV4)
`
`Ir
`
`dists(FV3)
`
`Ir
`
`dists(FV2)
`
`Dists(FV5) -> mem A
`Cale all dist. for FV5
`
`Dists(FV4) -> mem B
`Cale all dist. for FV4
`
`Dists(FV2) -> mern B
`Cale all dist. for FV2
`
`Dists(FV1) -> mem A
`Cale all dist. for FV1
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`Search stage on CPU
`
`interface
`
`or conflicted
`
`results across slow
`Transfer distance
`
`Accelerator
`
`~
`
`FV5
`
`Accelerator i,
`
`FV to
`Transfer
`
`Dists(FV3) -> rnern A
`Cale all dist. for FV3
`
`...
`
`FV4
`
`... Accelerator ,
`
`FV to
`Transfer
`
`f:_V5
`
`"'1111
`
`FV3
`
`FV to
`Transfer
`
`F=_V4
`
`... Accelerator 11
`• Accelerator " ..
`... Accelerator " ..
`FV1 ..
`
`FV2
`
`FV1
`
`FV to
`Transfer
`
`FV to
`Transfer
`
`£Y3
`
`EY2
`
`Data Frame 2 -> FV2 .,.
`
`i,
`
`Data Frame 1 -> FV1
`
`g
`
`0::
`0.. Cl)
`C: 0. =w Ql ....
`
`Q)
`
`Data Frame 5 -> FV5 ,
`
`i,
`
`V
`
`Data Frame 4 -> FV4
`
`Data Frame 3 -> FV3
`
`Ql - 0.. CJ) a::
`C 0. = Q)
`a::
`a. en l-
`Ql -
`•
`C 0. = Q)
`
`QJ
`
`QJ
`
`0::
`~U)
`= Ql
`o:
`l-
`0...U)
`w ...... ·-
`= a> E
`C 0.. Ql
`Ql
`
`i-
`C C.. W
`Ql
`
`o:
`0... Cl) l-
`a, +-J
`= a> E
`C 0.. Q)
`Ql
`
`·-
`
`a:
`~mi=
`=a>E
`C C.. Ql
`Ql
`
`o:
`c.. U)
`l-
`(l) +-"'
`= Ql E
`C 0.. Ql
`Ql
`
`·(cid:173)
`
`o:
`c.. en l-
`= a> E
`~ 0... Ql
`
`Q) +-"
`
`-(cid:173)
`
`Distance Calculation
`
`interface
`
`slow or conflicted
`Transfer FV across
`
`Front end on DSP
`
`Time
`
`IPR2023-00034
`Apple EX1001 Page 27
`
`

`

`-....l = N
`
`-....l
`'N
`\0
`-....l
`\0
`-....l
`d r.,;_
`
`0 ....
`--..J
`N
`('D .....
`rJJ =(cid:173)
`
`('D
`
`--..J
`N
`
`....
`0 ....
`N
`~
`:-' ....
`2'
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`,, -·
`
`I\) w
`~
`C
`(C
`
`I Search Stage, data frame 61 ...
`
`using dists(FV6)
`
`-
`
`dists(FV6)
`
`<-Xfer
`
`using dists(FV5)
`Search Stage, data frame 51 ""dists(FV5)
`
`-
`
`-
`
`results across slow Distance Cale Accelerator
`Transfer distance
`
`or conflicted
`
`interface
`
`interface
`
`slow or conflicted
`Transfer FV across
`
`FE Data Frame 1 -> FV1 I
`
`stage on CPU
`
`Front end and search
`
`ii:
`Cl.. Cl) I(cid:173)
`Q)
`......... 4-
`= Cl) E
`c:: C. Q)
`Cl)
`
`Time
`
`dists(FV5) Cale all dist. for FV6
`
`<-Xfer
`
`FV6.
`
`dists(FV4) Cale all dist. for FV5
`"
`
`-
`
`FV5
`
`dists(FV3) Cale all dist. for FV4
`.
`
`~
`
`FV4.
`
`<-Xfer
`
`Xfer ->
`
`<-Xfer
`
`Xfer ->
`
`"dists(FV2) Cale all dist. for FV3
`
`<-Xfer
`
`Xfer ->
`
`~
`
`FV3.
`
`dists(FV1) Cale all dist. for FV2
`..
`
`FV2.
`
`Cale all dist. for FV1
`
`FV1.
`
`<-Xfer
`
`Xfer->
`
`Xfer->
`
`Xfer->
`
`. -
`
`.
`. -
`
`.
`
`.
`
`.
`
`Search Stage, data frame 4 .. dists(FV4)
`
`-
`
`using dists(FV4)
`
`""" dists(FV3)
`
`using dists(FV3)
`Search Stage, data frame 3
`
`FV6
`
`FE Data Frame 6 -> FV6
`
`""" dists(FV2)
`,_ -
`
`FV5
`
`.. dists(FV1)
`
`using dists(FV2)
`Search Stage, data frame 2
`
`FE Data Frame 5 -> FV5
`
`using dists(FV1)
`Search Stage, data frame 1
`
`FV4
`
`FE Data Frame 4 -> FV4
`
`FV3
`
`FE Data Frame 3 -> FV3 I
`
`FV2
`
`FE Data Frame 2 -> FV2 I
`
`a:
`:.: a> E ~w i=
`~ a.(])
`
`ii
`~UJ i=
`:.: a, E
`~ Q. Cl)
`
`o::
`0.. Cl) l-
`Q) +-'
`·-
`= a, E
`C: 0.. Cl)
`Cl)
`
`ii
`~u5 i=
`= a, E
`C: 0.. Q)
`Cl)
`
`a:
`0.(/)
`'a5 .El.§
`~ 0. Q)
`
`l(cid:173)
`
`ii
`0.. Cl) I-
`..,_. ·-
`0)
`= Cl) E
`
`c:: 0.. Cl)
`Cl)
`
`IPR2023-00034
`Apple EX1001 Page 28
`
`

`

`1
`SPEECH RECOGNITION CIRCUIT AND
`METHOD
`
`US 7,979,277 B2
`
`2
`lookahead requires about a quarter of the states compared
`with unigram language model lookahead.
`Although these prior art documents provide improvements
`to speech recognition in embedded mobile technology, fur-
`ther improvement is still needed to provide a larger vocabu(cid:173)
`lary and better accuracy.
`One aspect of the present invention provides a speech
`recognition circuit including a circuit for providing state
`identifiers which identify states cor