(12) United States Patent
`US 6,289,138 B1
`(10) Patent N0.:
`(45) Date of Patent:
`Yip et al.
`Sep. 11, 2001
`
`USOO6289138B1
`
`(54) GENERAL IMAGE PROCESSOR
`
`(75)
`
`Inventors: Dominic Yip, Lindfield; Ian Gibson,
`Coogee; Mark Pulver, Enmore, all of
`(AU)
`
`(73) Assignee: Canon Kabushiki Kaisha, Tokyo (JP)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/025,725
`
`(22)
`
`(30)
`
`Filed:
`
`Feb. 18, 1998
`
`Foreign Application Priority Data
`
`Apr. 30, 1997
`Apr. 30, 1997
`Apr. 30, 1997
`Apr. 30, 1997
`APr- 30; 1997
`Apr. 30, 1997
`:pr. :8, 133;
`AP: 30’ 1997
`A; 30’ 1997
`.
`,
`
`(AU) ................................................... PO6480
`
`(AU)
`...... PO6481
`...... PO6482
`(AU)
`
`...... P06484
`(AU)
`
`
`------ P06485
`(AU)
`(AU) ................................................... P06488
`(:3) ................................................... £82233
`
`
`(AU)
`""" PO6491
`""" PO6492
`(AU)
`...................................................
`
`7
`
`(51)
`
`Int. Cl.
`
`....................................................... G06K 9/60
`.
`............................................. 382/307, 382/308
`(52) US. Cl.
`(58) Field of Search ..................................... 382/307, 308,
`382/233; 712/248, 36; 708/190
`
`(56)
`
`References Cited
`
`Re. 31,736
`Re. 32,493
`Re. 34,850
`
`US PATENT DOCUMENTS
`11/1984 Mueller et al.
`...................... 364/900
`
`9/1987 Matsumoto et a1.
`...... 364/200
`.................. 395/800
`2/1995 Murakami et al.
`
`OTHER PUBLICATIONS
`
`Evans et al., “Systolic Algorithms for Digital Image Filter-
`ing”, Parallel Computing, vol. 21, Issue 1, pp. 109—119, Jan.
`1995*
`
`Evans et al, “Systolic Algorithms for Digital Image Filter-
`ing”, Parallel Computing, vol. 1994, Issue 3, p. 12, Aug.
`1994*
`
`Huang et al., “Novel Gray Image Processing in a Single
`Operation Optical Morphological Processor”, Optics Com-
`munications, vol. 130, Issue 1—3, pp. 143—152, Sep. 1996*
`
`Primary Examiner—Jose L. Couso
`Assistant Examiner—Anh Hong Do
`(74) Attorney, Agent, or Firm—Fitzpatrick, Cella, Harper &
`Scinto
`
`(57)
`
`ABSTRACT
`
`The present invention relates to an image processor (242)
`comprising a control register block (1470), a decoding block
`(1468), a data object processor (1462), and flow control
`logic. The control register block (1470) stores all the rel-
`evant information about the image processing operation.
`The decoding block (1468) decodes the information into
`configuration signals, which configure an input data object
`interface (1460). The input data object
`interface (1460)
`accepts and stores data objects from outside, and distributes
`these data objects to the data object processor (1462). For
`some image processing operations,
`the input data object
`interface (1460) may also generate addresses for data
`objects, SO that the source Of these data objects can provide
`the correct data objects. The data object processor (1462)
`er orms ar1 me 1c 0 era lons on
`e
`a a o
`ec s rece1ve .
`pf
`thtpt
`thdtbjt
`d
`The flow control logic controls the flow of data objects
`Within the data object processing logic (1462). More
`particularly, the data object processor (1462) can comprise
`a number of identical data object sub-processors, each of
`which processes part of an incoming data object.
`
`(List continued on next page.)
`
`40 Claims, 135 Drawing Sheets
`
`FOREIGN PATENT DOCUMENTS
`0588726
`3/1994 (EP)
`............................. GO6F/15/332
`
`Microfiche Appendix Included
`(2 Microfiche, 101 Pages)
`
`1454
`
`242
`\
`1464
`
`Instmction
`Word
`Register
`
`|
`l
`p-
`CS table valhes,
`:23: 22:4,
`&I&Vvaluee 3:” value&
`
`1451
`
`1452
`
`1453
`
`
`1471
`
`Instruction
`Word
`Input Interface
`
`Decoder
`1461
`
`Control
`1460
`
`Signal
`
`
`Register
`
`Image Data
`Processor
`
`
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0001
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0001
`
`

`

`US 6,289,138 B1
`
`Page 2
`
`
`
`US. PATENT DOCUMENTS
`340/146 1 R
`5/1975 F
`k
`~
`“in
`~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`--------------------- 2(3)?ng
`gfigzg Eggfirearneftalal-
`
`10/1981 Mayer et al.
`........................ 364/900
`5/1982 Pratt et al.
`........................... 364/515
`
`.. 364/725
`5/1983 Wildergren et al.
`..
`................ 364/200
`7/1984 Christopher et al.
`10/1984 Kanayama ........................... 364/900
`.. 340/347
`8/1985 Weaver
`
`10/1985 Bechtolsheim ....................... 364/200
`5/1986 Rodman ............................... 364/200
`11/1986 Atkinson .
`.. 340/747
`
`2/1987 Bledsoe ................................ 340/347
`7/1987 Hester et al.
`........................ 364/200
`
`10/1987 Bledsoe ..........
`.. 340/347
`.. 364/518
`1/1988 Guttag et a1.
`..
`1/1988 Kobayashietal.
`382/41
`1/1988 Chambers
`382/42
`
`4/1988 Chabert
`' 382/41
`"""
`6/1988 Mischleret a
`382/41
`
`'
`NN364/900
`10/1988 Asai et a1
`N 358/133
`10/1988 Daly et a1.
`
`12/1988 Liou et al. N
`N 364/725
`1/1989 Amitai
`'
`N 364/900
`3/1989 Fedele ..
`375/27
`4/1989 Lumelsky etal.
`.. 364/521
`
`6/1989 Urushibata .....
`.. 364/518
`
`8/1989 Mukheljee
`341/65
`3/1990 Guiliano etal.
`364/72801
`
`4/1990 Hatori etal. ......
`.. 358/433
`4/1990 Murakamietal.
`.. 364/200
`
`5/1990 Boreland ........
`712/36
`.. 358/427
`6/1990 Sam et a1.
`..
`
`6/1990 Malinowski
`.. 364/724
`9/1990 Neustaedter
`.. 364/200
`10/1990 Tokuume .......
`.. 364/200
`12/1990 Kimata et al.
`.
`382/44
`
`.. 364/521
`1/1991 Houwitz etal.
`
`.. 340/799
`“1991 Carrick .......
`2/1991 Callemyn .......
`.. 364/518
`
`6/1991 Murakami et al
`382/56
`'
`""364/725
`7/1991 Uetani
`
`9/1991 Watanagg‘g‘tzlw
`" 358/433
`.
`"
`‘
`10/1991 Frledlander et a1.
`.. 364/725
`.
`............
`10/1991 Arbelter
`.. 375/122
`
`.. 395/275
`4/1992 Kubota et al.
`4/1992 Beausoleil et a1.
`.. 395/400
`
`6/1992 Kerr et al
`382/47
`.
`.
`'
`6/1992 Phllhps ..........
`.. 395/400
`.
`8/1992 Sakagaml et a1.
`.. 358/432
`
`............
`11/1992 Uetani
`382/56
`
`“1993 Miyazaki
`364/725
`2/1993 N
`" 358%
`
`g """""""""
`2/1993 Edenfield etal.
`. 395/425
`
`.. 395/130
`..
`2/1993 Alcorn et al.
`
`
`.........
`364/72801
`3/1993 Hsu etal.
`.
`
`3/1993 Balkanskletal.
` .. 358/433
`.
`.. 364/725
`3/1993 Cucchletal.
`.
`
`4/1993 Wang etal.
`. 364/754
`.
`5/1993 Grlbertetal.
`....................... 358/298
`6/1993 Tanaka etal.
`....................... 358/426
`
`.. 358/133
`“993 Stone et a1.
`7/1993 Barry et al.
`............................ 341/65
`8/1993 Pollmann et a1.
`..................... 341/67
`.. 395/162
`8/1993 Stattetal.
`
`......................... 307/449
`8/1993 Small et a1.
`
`3 883 847
`,
`,
`3733333 *
`4,296,476
`4,330,833
`4,385,363
`4,460,958
`4,475,174
`4535320
`4,550,368
`4587 610
`4,622,545
`4,646,061
`4680 700
`4,700,175
`4,718,024
`4,718,091
`4720871
`4,736,440
`4’754’491
`4’779’223
`4,780,761
`4,791,598
`4,797,850
`4,813,056
`4,823,286
`4839 826
`4’853’696
`4,907,182
`4,920,426
`4,920,480
`4926 355 *
`4’935’821
`4,937,774
`4,956,771
`4,965,722
`4,975,976
`4,982,343
`4,983,958
`4991 112
`5025482
`5’029’122
`5’051’840
`9
`9
`5,053,985
`5,060,242
`5,109,333
`5 109 496
`5’125’042
`9
`9
`5,125,085
`5,142,380
`5 163 103
`5,181,183
`5185661
`’
`’
`5,185,694
`5 185 856
`’
`’
`5,195,050
`5,196,946
`5,197,021
`5,204,830
`5,212,559
`5,216,516
`5223 926
`5,227,789
`5,233,348
`5,237,655
`5,241,222
`
`5,243,414
`5,249,146
`5,253,053
`5,253,078
`23:32;
`’
`’
`3732:9223
`7
`7
`572707832
`5,283,866
`592997027
`573039058
`5,303,349
`573079451
`573139577
`573177717
`5,321,806
`573259092
`5,325,215
`5,333,297
`5,337,319
`5,341,318
`5,349,348
`5,349,651
`5,351,067
`2233?:
`7
`7
`573927038
`5,394,515
`37:33:33;
`7
`7
`594369734
`574407404
`5,446,854
`37:27:32
`7
`7
`594679088
`5,488,487
`574839475
`,
`,
`5748557
`5,485,568
`5,485,589
`5,502,804
`5502 824
`’
`’
`5504 842
`7
`7
`5506 944
`,
`,
`5,509,115
`5,509,137
`5513 335
`,
`,
`5 515 296
`’
`’
`575249075
`5,528,238
`5,528,628
`5528 764
`7
`7
`575309823
`5530944
`,
`,
`5 535 291
`7
`7
`5539 865
`,
`,
`5544 290
`7
`7
`5544342
`7
`7
`5 557 733
`7
`7
`5,561,761
`575619772
`5,570,432
`
`9/1993 Dalrymple et a1.
`.................. 358/500
`9/1993 Uramoto et a1.
`..................... 364/725
`10/1993 Chu etal.
`............................ 358/133
`10/1993 Balkanski et al.
`................... 358/426
`12433: in“: etal'
`”'1'"""""""""""32:14::
`6‘” ergereta'
`‘
`/
`/
`
`128333 (Tioffigm 1
`‘ £323:
`
`
`511° lya? a‘
`‘
`”/1993 Balkan“? etal‘
`‘ 358/432
`
`2/1994 Kumagal
`.....
`. 395/164
`3/1994 Nakamura etal'
`' 358/403
`“994 FUkufla etal‘ “““
`‘ 358/261
`
`4/1994 Warrlner et a1.
`. 395/275
`“994 Chirk """"""""
`‘395/127
`
`‘ 395”“
`“994 Memerth 9‘ al'
`
`~ 395/425
`“994 cmleretaL
`. 395/162
`6/1994 Meinerth et a1.
`
`“994 All?“ et al' “““
`341/65
`..
`. 358/479
`6/1994 Shlbata et a1.
`. 395/500
`7/1994 Lemaire et a1.
`
`8/1994 Furukawa et a1.
`.
`371/11.1
`8/1994 Balkanski et a1.
`.
`. 364/725
`
`9/1994 Anderson et a1.
`341/51
`9/1994 Hertherington et a1.
`.
`. 395/400
`9/1994 Lumelsky et a1.
`. 345/191
`$33: 8?" ~~~~~~~~~
`~ 3323;:
`""""" :
`~
`2/1995 Bhanda“ et al'
`341/67
`
`2/1995 Lentz etal.
`.....
`. 395/115
`
`‘ 332/133
`3133: Sun?” et al'
`la 1
`
`7/1995 Yamaucmetal‘
`‘ 358/448
`8/1995 019910.“) “““““
`‘358/432
`. 395/401
`8/1995 Khahdlet al.
`..
`31335 $0M: et al'
`~
`~ 3:33;:
`Fe“
`““““
`‘
`“/1995 Kmoucmet al‘
`341/65
`13133: Eben {Ni~~~~~~
`~ 323%?
`1 1996 Kang e a' """""""""""""" 364 725
`ao ......................................
`/
`/
`
`“1996 saw et al'
`‘ 395/129
`1/1996 Venable et a1.
`...................... 395/155
`1/1996 Kocisetal.
`.................... 395/421.03
`
`3/1996 Butterfield etal.
`395/147
`3/1996 H611 ...................................... 395/293
`.
`4/1996 Gentlle ................................. 395/114
`.
`. 395/114
`4/1996 Gentlle ........
`
`4/1996 Butterfield et a1.
`.................. 395/147
`4/1996 Itomitsu et a1.
`..................... 395/495
`.395/457
`4/1996 McClure .....
`
`............................... 364/514
`5/1996 Agarwal
`
`
`
`
`
`382302
`“996 Rfmsseau et al'
`
`341/67
`6/1996 Nlckerson .......
`6/1996 Park etal.
`........................... 375/240
`6/1996 Heil ...................................... 395/293
`.
`
`“996 Tsucmya et al'
`‘ 395/417
`6/1996 Stones .................................. 395/494
`7/1996 Spencer et a1.
`...................... 382/254
`.
`. 395/115
`7/1996 Gentlle .....
`
`.
`8/1996 Gentlle ................................. 395/115
`8/1996 Dean .................................... 395/446
`.
`9/1996 chok et a1.
`395/162
`
`.
`.................... 395/183.06
`10/1996 chok et al.
`“”1996 D0191“ et al' """"""""""" 395/281
`10/1996 K0]1ma ................................. 382/254
`
`* cited by examiner
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0002
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0002
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 1 of 135
`
`US 6,289,138 B1
`
`226
`
`.l
`
`032
`
`, host
`ram
`
`202
`
`Host
`CPU
`
`204
`
`207
`
`PClbus
`interface
`
`I I I I I I I I I I
`
`220 \-
`
`225
`
`223
`
`Local
`Memory
`
`Peripheral
`Interface
`
`227
`
`224
`
`Raster Image
`Coprocessor
`
`Generic
`External
`interface
`
`: I I I | I I I I I
`
`PCI bus
`I" _________________ ‘l
`
`I
`
`:
`
`I
`
`I
`209
`I
`e
`I
`I
`
`21°
`
`Oh
`
`t er
`
`PCIb
`
`us
`
`211
`
`212
`
`'206
`
`I
`
`:
`
`I
`
`I
`I
`.
`I
`I
`
`201
`
`Fig.1
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0003
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0003
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 2 of 135
`
`US 6,289,138 B1
`
`224
`
`External Cbus Interface
`
`II
`
`Diagnostic H
`and II
`
`Miscellaneous
`
`" Clocks and
`II
`ll global
`
`II
`
`231
`to all
`modules
`
`230
`
`Data
`_’ Cache
`
`
`
`Test Ports”
`‘ ’ll signals
`
`
`II
`239
`235
`
`
`
`External
`'
`'
`Input
`Data
`Interface
`Interface
`.
`Cache
`
`
`Controller
`SW'tCh
`Controller
`
`
`::
`PCIbus :'
`(1':
`206
`
`
`
`u
`II
`Generic n
`Interface u
`|
`
`227 ll
`
`LII—I.
`
`248
`
`
`Page Mode ::
`L
`l
`
`
`D
`or "
`Mecn‘ISry
`
`Synchronous "
`Controller
`
`
`DRAM ::
`
`
`
`
`
`
`Organlser C
`
`I I
`
`
`Centronics "
`Peripheral
`Interface
`Interface ::
`
`or Video II
`Controller
`
`Interface "
`
`
`= = = = Represents ASIC pads
`
`External interface
`
`
`Control Bus (Cbus)
`----- Input Bus (Ibus)
`—--—--- Operand Bus (Obus)
`
`- — — - Partial Result (Pbus)
`
`I I I I I I I I I Result Bus (RbUS)
`_ ..... _ Interface Bus (AbUS)
`_ _ — Internal RAM interface
`
`249
`
`'
`
`Flg 2
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0004
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0004
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 3 of 135
`
`US 6,289,138 B1
`
`Peripheral Interface Controller
`
` External Interface Controller
` Local Memory Controller
`
`
`
`
`
`
`
`
`Pixel Organiser
` Operand Organiser B
`
`
`
`Operand Organiser C
`
`
`
`Main Data Path
`
`Result Organiser
` PClbus Configuration Space Alias
`
`Input Interface Switch
`
`reserved
`
`Miscellaneous
`
`Instruction Controller
`
`Data Cache Controller
`
`261
`
`JPEG Coder
`
`Coprocessor
`module
`
`registers
`
`262
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0005
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0005
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 4 of 135
`
`US 6,289,138 B1
`
`224
`
`Co-Processor
`
`MEMORY 203
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0006
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0006
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 5 of 135
`
`US 6,289,138 B1
`
` 1030
`
`Instruction
`
`Generator
`
`1031
`
`1032
`
`
`
`
`
`
`Memory
`Queue
`Manager
`Manager
`
`
` 224
`
`Coprocessor
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0007
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0007
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 6 of 135
`
`US 6,289,138 B1
`
`202
`
`
`
`1040
`
`1041
`
`
`
`IIIIII
`IIIIII
`Pending
`Completed
`
`Instructions
`Instructions
`Queue
`(Cleanup Queue)
`
`
`Graphics
`Coprocessor
`
`
`224
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0008
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0008
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 7 of 135
`
`US 6,289,138 B1
`
`1 040
`
`'\
`
`Co-Processor IP
`
`Queue Manager
`
`last instruction added
`
`Fig. 7
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0009
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0009
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 8 of 135
`
`US 6,289,138 B1
`
`cleanup
`
`finished
`
`interrupt
`
`todo
`
`270
`
`Stream II-III II-I-l
`
`
`
`cleanup
`
`finished
`
`interrupt
`
`todo
`
`II-III
`
`
`
`lI-I-II 1
`
`START
`
`276
`Register update
`Instruction stream
`
`
`complete
`to fetch next
`
`Instruction pending?
`
`Which stream?
`
`
`
`
`d
`b
`
`
`Upoate
`Fetch
`
`
`
`Instruction
`Reglsters
`
`
`
`277
`
`c
`
`
`
`Decode and
` Instruction
`
`Execute Instruction
`Instruction
`complete
`
`275
`
`Fig. 9
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0010
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0010
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 9 of 135
`
`US 6,289,138 B1
`
`280
`
`Ma
`
`31
`
`281——
`
`lnstructlon Opcode
`
`282— Operand and Result types
`
`0
`
`
`
`4— Instruction Word
`
`<— Data Word
`
`
`
`
`
`
`283— Operand A address or data <- Operand A Word
`
`284—— Operand B address or data 4— Operand B Word
`
`285— Operand C address or data <- Operand C Word
`
`286——
`
`287
`
`Result address
`
`For use by Host
`
`Application
`
`
`
`
`‘- Result Word
`
`Fig. 10
`
`0
`
`
`
`\
`
`290
`
`Fig. 11
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0011
`
`31
`
`28 27
`
`201918171615
`
`minor
`
`major
`
`
`—
`-I_ME
`297
`
`
`294
`
`296
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0011
`
`

`

`US. Patent
`
`Sep.11,2001
`
`Sheet10 0f135
`
`US 6,289,138 B1
`
`com33mmvm
`
`85:88m85:33055:0wam56:0me<A|VWHn-lugan-IEEHn-IEEHn-In
`
`
`
`
`
`SmE9mmwm
`
`55536EBaromgm_53:8va<IVEm-Hnégan-IEEIIIIE
`
`
`
`
`NF.9”.
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0012
`
`
`
`
`
`”$262553:880fiwm.Ev03...
`
`225fine2:hoHuston.
`
`
`
`
`
`6:26:mescewao3E...
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0012
`
`
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 11 of 135
`
`US 6,289,138 B1
`
`(AM
`
`235
`
`I
`
`307
`
`
`
`Input
`Interface I- ' - -
`Switch
`
`306
`
`
`Instruction
`
`Decoder
`
`
`
` I
`
`Miscellaneous
`Module
`
`Execution
`
`Controller
`
`Miscellaneous
`
`Module
`
`I. —————————————
`I
`i
`
`'
`I
`
`I
`I
`I
`I
`
`I
`
`Cbus
`Control
`Interface Register
`
`
`
`I
`I
`'
`I
`I Register. I
`I
`File
`|
`I
`
`I
`| | I I I l | | | | L
`
`§.' \'
`
`I
`
`I
`
`I
`
`External
`Interface
`
`Controller
`
`308
`
` Arbitrator
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0013
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0013
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 12 of 135
`
`US 6,289,138 B1
`
`Prefetch Buffer
`
`Instruction
`
`Controller
`
`Decoder
`
`
`
`
`Overlapped
`Instruction
`Decoder
`Prefetch
`Interface
`Instruction
`
`Logic
`Logic
`
`
`
`Logic
`
`
`Management
`
`I
`
`IC CBus
`
`Interface:"'""
`
`
`
`
`Register
`
`
`
`
`
`Miscellaneous Module
`
`Fig. 14
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0014
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0014
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 13 of 135
`
`US 6,289,138 B1
`
`312
`
`
`
`reset —>
`
`Decide
` /315
`
`
`whether to
`
`execute-an
`
`1nstruct10n
`
` FETCH *
`
`
`Fetch
`Instruction
`
`
` tll
`
` WAIT 0*
`Wait for
`overlapped
`1nstruct10n to
`finlsh
`
`
`
`
`
`th
`
`Wait for
`decoding to
`complete
`
`PREFETCH
`
`
`
` 313
`Prefetch
`1nstruct10n
`
`
`
`
`
` Wait for
`1nstruct10n
`
`to “finish”
`
`UPDATE *
`
`
`
`Update
`Registers
`
`
`Fig. 15
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0015
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0015
`
`

`

`US. Patent
`
`Sep. 11, 2001
`
`Sheet 14 of 135
`
`US 6,289,138 B1
`
`Prefetch Buffer Controller
`
`Execution
`Controller
`
`Sequencer
`
`CBus
`Dispatcher
`
` ID
`
`
`
`CBUS
`.
`Arbitrator
`
`306
`
`Fig. 16
`
`Instruction Controller Cbus Interface
`
`Instruction
`
`Prefetch Buffer
`Controller
`Execution
`Controller
`CBus Interface
`Controller
`-
`-
`- _I _ -
`- _
`- -
`
`
`
`Overall
`Sequencing
`Control
`
`
`
`
`
`Per Module
`(there are 8 different
`Per Module
`
`Configuration
`Conflguratlon Per Module Configuration
`
`
`
`Sequencer
`sequencer
`Sequencers)
`
`
`
`
`321
`
`
`
`CBus Dispatcher
`
`Fig. 17
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0016
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0016
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 15 of 135
`
`US 6,289,138 B1
`
`IA operation
`Organizer
`
`'p
`Interface
`Controller
`
`Configure
`Result
`Organiser for
`
`
`
`Configure
`Operand
`‘
`orgamzer B
`
`330
`
`Wait for
`start
`
`
`
`Configure
`Pixel
`
`
`
`
`
`Configure
`Data Cache
`
`Controller
`
`
`
`Configure
`JPEG
`Coder
`
`
`
`
`
`
`
`MDP *
`
`
`
`Configure
`Main Data
`Path
`
`Configure
`Operand
`Organizer C
`
`
`
`Fig. 18
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0017
`
`
`
`Configure
`Result
`
`
`Organizer
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0017
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 16 of 135
`
`US 6,289,138 B1
`
`335
`I
`
`
`Prefetch
`I
`Buffer
`
`
`
`Instruction
`Decoder
`
`Inigffie I
`lbus
`PBC
`
`
`3
`Controller
`'
`
`
`I
`equencer
`SWItch
`
`Execution
`Controller
`
`|
`
`307
`
`Address
`
`Manager
`
`337
`
`-
`Instruction Controller
`
`Cbus Interface
`
`Fig. 19
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0018
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0018
`
`

`

`US. Patent
`
`Sep. 11, 2001
`
`Sheet 17 of 135
`
`US 6,289,138 B1
`
`_________
`
`<3.9“.
`
`
`
`
`
`as._Bo;I33_IE_ABEImmmcoochcooNoccoo
`
`
`
`Bo;_I93_IE_Irmccoormzcoo
`
`mm_:uoEn:w550Ewm
`
`29$—I23—553N8Sm.u.I"IN_EEoo«65:8IN_9Eoo“v__m>mou__m>o___I295—l#wbmw_22:82928—Izozcoo
`
`
`
`______u"Smumlo.EIEmUon
`ovmuonmIIIJIIIIIIIIIIIIIIUIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII‘IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII__I
`
`
`
`mom6also8msmwm.23
` Emcee rIIIIIIIIIIIIIIIIIIIIIIIIIIII._IE39
`
`InmxmmEc:B5930mmmHmmmzlo
`
`
`.95ESE?Emumm.ommmvm
`
`
`
`vow\.2m2me2:520920660..Sou.xom93m.wflflflmm
`
`CV”.UCN#5C—va
`
`:30meEmulaam..
`
`9=2<umom
`
`5:95:00
`
`E0025
`
`
`
`.I.m_Io
`
`soI.:_Iu__m>mIo
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0019
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0019
`
`
`

`

`US. Patent
`
`Sep.11,2001
`
`Sheet18 0f135
`
`US 6,289,138 B1
`
`
`
` 9999:wow__.__mmmmmmE5330_nun-runnluni-unua-u-Iulu-uunun-nu..uuuuI.nun-unnuunnnn-nnnun-nunnnnn-nn-unuuurn...-nn-n-nnnInn-III...I.nun-nuuununum.Pmuoam.
`
`
`
`_II__2%toxmmEca__85556__5:0umxmecz
`
`
`
`mom
`
`m8.9".
`
`.950
`
`mom
`
`9%,96d9%EIoSmEIo
`
`_
`
`N8
`
`1IIVnnnnnnnnnnnnnnnnnnnnnnnnnnn48m3m___AcomIEthv
`528050EomJEItmvEomJEItmv__.29050
`
`
`
`EEEEEotm_vamlowwmmtoI__.SoIc_Iu__m>mIo_Smwmm_I.c_Smuwo_.otolwmm_u__m>mo.ufllo_umxmmEcz_SmulmE
`86550_SoII_33:95:95EstmEEotm.
`
`.v__m>Io
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0020
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0020
`
`
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 19 of 135
`
`US 6,289,138 B1
`
`Type A
`
`28 27
`
`24 23
`
`2019
`
`1615
`
`O
`
`IMMfl“
`
`Type B
`
`31
`
`28 27
`
`24 23
`
`Type C
`
`370
`
`O
`
`371
`
`28 27
`
`2423
`
`2019
`
`1615
`
`870
`
`Inmn-n
`
`\372
`
`Fig. 21
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0021
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0021
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 20 of 135
`
`US 6,289,138 B1
`
`input
`5
`381\§ external
`5 format
`
`I
`'
`
`
`
`I
`.
`.
`
`247
`
`I
`I..._..
`
`:
`248 5
`
`
`
`input
`organisers
`
`Operand
`Operand
`Pixel
`Organiser Organiser B Organiser C
`
`
`
`246
`:
`382\§ input
`L
`:internal
`format
`
`‘.._.._.._:....
`JrlIi-II-II—II.
`i
`"T— -"T__—":
`:
`:
`
`§
`
`
`
`
`
`main
`functional
`units
`
`241
`
`output
`383 \ internal
`format
`
`384
`
`5 output
`\1 external
`-
`format
`
`
`Data path
`
`'
`I
`L — _ _.:
`f _ _ J
`
`..............
`output .......................... Result
`
`Organiser
`organiser
`249
`
`
`
`E
`5
`1
`
`380
`
`242
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0022
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0022
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 21 of 135
`
`US 6,289,138 B1
`
`387\--- b te address
`
`31 \ j 0
`
`0 this data
`
`388
`
`
`
`
`
`—:<-W
`
`
`
`
`386
`
`3’91
`
`Emma?
`,/
`
`31
`
`\
`
`0
`
`396
`
`b te address
`0 this data
`
`395
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0023
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0023
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 22 of 135
`
`US 6,289,138 B1
`
`byte address
`of this data
`
`400
`
`/
`
`Fig. 29
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0024
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0024
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 23 of 135
`
`US 6,289,138 B1
`
`410—» External
`Formats
`
`packed
`stream
`
`unpacked
`bytes
`
`other
`
`411—,
`
`Internal
`Formats
`
`packed
`bytes
`
`pixels
`
`unpacked
`bytes
`
`other
`
`Fig. 30
`
`Internal
`412 —’ Formats
`
`packed
`bytes
`
`_
`pixels
`
`unpacked
`bytes
`
`413 _> External
`Formats
`
`packed
`stream
`
`unpacked
`bytes
`
`Fig. 31
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0025
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0025
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 24 of 135
`
`US 6,289,138 B1
`
`from input modules
`
`external
`unpacked
`bytes
`
`41 7
`
`external
`packed
`stream
`
`422
`
`external
`Other
`
`41 8
`
`unpacked
`normalisation
`
`packed
`normalisation
`
`421
`
`416
`
`419
`
`423
`
`l
`t
`_
`153m:
`bytes
`
`425
`
`4bytes
`
`internal
`unpacked
`bytes
`
`internal
`pixels
`
`component
`selection
`
`su bstltutlon
`
`component
`
`pixels
`
`5 MCU/MUV f4 ---------------
`handling
`if _______________
`
`/
`
`446
`
`-
`
`Flg ' 32
`
`to main
`
`functional units
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0026
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0026
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 25 of 135
`
`US 6,289,138 B1
`
`417 \
`
`
`
`unpacked
`normalisation
`
`
`
`418
`
`419 \
`
`8
`
`O
`
`— Fig. 33
`
`422
`
`3 channel objects,
`
`2 bits per channel 421
`
`
`
`packed_
`normalisation
`
`423
`
`data item 1
`
`data item 2
`
`etc
`
`Fig. 34
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0027
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0027
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 26 of 135
`
`US 6,289,138 B1
`
`
`
`425
`
`pack
`
`
`
`unpack
`
`426
`
`434-\1__
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0028
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0028
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 27 of 135
`
`US 6,289,138 B1
`
`data item 1
`
`data item 2
`data item 2
`
`data item 1
`
`31
`
`data item 1
`
`data item 2
`
`internal data register
`
`n-IEI— —’
`
`441
`
`0
`
`
`
`subseitution
`
`com onent
`
`
`
`0
`31
`data item 1 nun
`data item 2 nil—“-
`I
`n
`.
`I
`.
`.
`.
`|
`
`44o
`
`442
`
`Fig. 38
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0029
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0029
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 28 of 135
`
`US 6,289,138 B1
`
`0
`
`0
`
`31
`
`data item
`
`data item 2
`
`31
`
`data item 1
`
`data item
`
`lane
`
`Fig. 39
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0030
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0030
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 29 of 135
`
`US 6,289,138 B1
`
`31
`
`data item1
`
`ch3
`
`data item 2 .--ss_
`
`h1
`
`0
`
`chO
`
`ChO
`
`IIHHHHHEiII4
`
`
`
`(replication factor = 1)
`
`446
`
`
`
`Fig. 41
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0031
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0031
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 30 of 135
`
`US 6,289,138 B1
`
`from main functional units
`
`383
`
`\ internal
`internal
`packed unpacked internal
`bytes
`bytes
`pixels
`
`iMCU/MUVé------------------
`
`
`handling 5‘ ..................
`
`450
`
`/
`
`
`
`lane swap
`
`424
`
`
`
`
`internkald & external
`inteLnaJ & externkald
`packed
`unpac e
`unpac e
`pac e
`
`
`
`bytes
`stream
`bytes
`bytes
`
`
`
`
`
`
`
`com onent
`subsgitution
`
`
`deselection
`
`component
`
` byte
`453
` 454
`
`addressing
`
`Fig. 42
`
`to output modules / 384
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0032
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0032
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 31 of 135
`
`US 6,289,138 B1
`
`data item 1
`
`data item 2 456
`
`data item 1
`
`data item 2
`
`Fig. 43
`
`Fig. 44
`
`data item 1
`
`data item 2
`
`
`2 bits per channel
`
`3 channel objects,
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0033
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0033
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 32 of 135
`
`US 6,289,138 B1
`
`O
`
`addressing
`
`453
`
`address of _
`output strearfi‘m~
`
`31
`
`"‘1‘
`
`0
`
`31
`
`3 channel objects,
`
`2 bits per channel
`
`
`
`address of
`
`453
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0034
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0034
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 33 of 135
`
`US 6,289,138 B1
`
`31
`data item 1—-— cho
`——— cho
`data item 2
`ch3
`ch2
`
` data item1
`
`///_chO
`////_l
`
`data item2
`
`byte not
`written
`
`Fig. 47
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0035
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0035
`
`

`

`US. Patent
`
`Sep.11,2001
`
`Sheet34 0f135
`
`US 6,289,138 B1
`
`0252:?
`
`oF
`
`——mfi—_
`
`NmvmmnwmoFFFNFvaFmFoFNFmeFONFNNNMNVNmNoNNNwNmNomFm
`
`
`
`——_——_$—_——__———__————_“——_——————————————m$m$$$“——————$$$$fi——=__“————_m————mm—_——mm—_
`
`oF
`
`NmvmonwmoFFFNFmFVFmF©FNFwFmFONFNNNMNVNmNoNNNwNmNomFm
`
`wmzmctoc
`
`we.9". nmEo
`——————————“————$mmmm_$—_———_“"—n——_——““—————m—_——mm————mm——
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0036
`
`
`
`”CEUIEV$66mecozfizaEms.Ema38:0
`
`
`
`
`
`
`
`
`
`”CE—glooo0cmLEUInoo.LEUIOQV@9231cormSQEmEEmaSac.
`
`
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0036
`
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 35 of 135
`
`US 6,289,138 B1
`
`Instructions
`
`Input Data Word
`
`1061
`
`/
`
`1063
`
`Programming
`
`Agent
`
`Data
`
`Normalizer
`
`Engine
`
`Central Graphics
`
`Output Data
`
`Fig. 49
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0037
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0037
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 36 of 135
`
`US 6,289,138 B1
`
`E
`
`n[2._0]
`
`Input Data word
`1069
`
`1073
`
`fifo_empty
`
`FIFO_RD
`
`First-ln-First—Out
`
`
`
`Buffer (FIFO)
`
`/ 1062
`
`flfO_fU”
`
`1086
`
`FlFO_WR
`
`1085
`
`1074
`
`Control
`Unit
`
`Input Register (REG1)
`
`1078
`
`REGLEN
`
`
`x[31..0]
`
`
`in-bit[4..0]
`Normalization
`Multiplexors
`
`
`
`OUt—bytel150] MUX
`MUX MUX
`
`31
`1
`0
`
`
`
`1075
`
`n[2..0]
`
`' ' '
`
`y[31..O]
`
`Output Register (REG2)
`
`1081
`
`1076
`
`c[1..0]
`
`b[2..0]
`
`Pixel Data
`
`Fig. 50
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0038
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0038
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 37 of 135
`
`US 6,289,138 B1
`
`464
`\ source
`
`462
`
`/
`
`467
`
`operand
`
`blend
`
`generator
`
`455
`
`attenuation
`
`466
`
`470
`
`471
`
`optional
`
`multiplication
`
`optional .I
`
`X
`
`pre-
`multiplication
`
`469
`
`8
`
`46
`
`472
`
`473
`
`instruction
`
`475
`
`476
`
`477
`
`F'
`19.
`
`51
`
`clamp/wrap
`
`/
`
`optional
`un-pre-multiplication
`
`destination
`
`(—
`
`463
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0039
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0039
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 38 of 135
`
`US 6,289,138 B1
`
` 2:3 222121122 112222115
`
`
`
`31
`
`G
`
`11212
`
`
`
`:
`
`212221122 31212132921112 52221222291112 Startmend staffing 11121122 131W 11:2
`K
`“‘1 1322211111122 ‘32
`1121211111822.
`1221211121211“:
`621321218222
`p.
`'
`'
`D
`212211212111221122222 211122231222 82111232131112 end
`s
`-
`3
`f” 132131111122...-‘3
`1121211111212
`2213211122 ‘2 12213111112112?
`4Y2 -
`(22222131112 {3 address (:11 data
`
`112:2 B W911i
`
`.
`..
`,
`«Ia—=0 e -
`38‘
`2:1
`% 21%— 132316131112 ‘3 WGfd
`
`
`
`Hg. 22
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0040
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0040
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 39 of 135
`
`US 6,289,138 B1
`
`,,. 4‘55
`/
`
`flies: address
`
`«saw Operand A Word
`
`Hg. M
`
`we stem Offset
`
`;
`
`tiie Eength - 1
`
`,fim 0119mm E3 Wan:
`4m
`
`3
`
`Qparand C addregs m data
`
`4am Open-and :23 Wurfi
`
`the iength (MES) {W8}
`
`
`we start Qfis-et {WES} {@E’?)
`
`
`
` IIIIIIIIIIIIIIIIIII
`
`1113 address; {byte address) (4%)
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0041
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0041
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 40 of 135
`
`US 6,289,138 B1
`
`481
`ColourValue o o o o 010 o /
`
`:90
`
`Interval Table (4 bit) ;82
`
`Fractional Table (8 bit) [$83
`
`0
`
`1
`
`2
`485
`3
`4
`\ 5
`6
`
`7
`
`8
`
`9
`
`10
`11
`
`0
`
`0
`
`O
`0
`o
`0
`0
`
`1
`
`1
`
`1
`
`1
`1
`
`
`
`249
`
`250
`
`251
`
`252
`253
`254
`
`255
`
`O
`
`486
`
`1
`
`.'
`
`14
`
`0
`
`1
`
`2
`3
`4
`5
`6
`
`7
`
`8
`
`9
`
`10
`11
`
`249
`
`250
`
`251
`
`252
`253
`254
`
`255
`
`
`
`
`.
`I
`
`I
`
`I
`
`I
`I
`
`I
`
`I
`I
`
`/256
`
`/256
`
`/256
`/256
`/256
`/256
`/256
`
`/256
`
`/256
`
`I256
`
`/256
`/256
`
`/256
`
`I256
`
`I256
`
`/256
`I256
`/256
`
`/256
`
`.
`I
`
`I
`
`l
`
`l
`I
`
`I
`
`I
`I
`
`Fig. 56
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0042
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0042
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 41 of 135
`
`US 6,289,138 B1
`
`/250
`
`/488
`
`’/489
`
`/490
`
`4-bit
`
`8-bit
`
`4-bit
`
`8-bit
`
`4-bit
`
`8-bit
`
`H4——-> H‘——> {—94—}
`
`
`
`Interval and
`
`Interval and
`
`Interval and
`
`fraction values
`
`fraction values
`
`fraction values
`
`for channel 2
`
`for channel 1
`
`for channel 0
`
`Fig. 57
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0043
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0043
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 42 of 135
`
`US 6,289,138 B1
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0044
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0044
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 43 of 135
`
`US 6,289,138 B1
`
`
`
`Input Value
`
`Fig. 59
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0045
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0045
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 44 of 135
`
`US 6,289,138 B1
`
`
`Memory Bank 0
`
`
`512 x 8 bit
`
`entries
`
`Memory Bank 1
`
`352
`
`——————
`
`Stage 1 I Frac_r - 550
`
`551
`
`553
`
`Stage 2:
`
`Frac_g - -------
`
`Stage 3:
`
`FraC-b - -----
`
`Fig. 60
`
`Output Colour
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0046
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0046
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 45 of 135
`
`US 6,289,138 B1
`
`Memory Address
`
`entries
`
`52 128
`
`Fig. 61
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0047
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0047
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 46 of 135
`
`US 6,289,138 B1
`
` Memory Bank 0
`
`128 x 32 bit
`entries
`
`Stage 3:
`
`Mac}? 7>
`
`- _ _ _
`
`Fig. 62
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0048
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0048
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 47 of 135
`
`US 6,289,138 B1
`
`580581
`582
`
`578
`
`575
`‘1
`
`Ream;
`
`B”
`
`WW
`
`2bit
`tag address
`
`7bit
`line address
`
`587
`
`<— - 32 Bits
`
`- ->
`
`585
`
`O
`
`C I M I Y I K
`—4-—h—P—
`
`
`
`586
`
`127
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0049
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0049
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 48 of 135
`
`US 6,289,138 B1
`
`fl'
`(0
`I
`.9
`Ll.
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0050
`
`O
`
`LG
`nor
`[1
`oo\—
`
`O)
`
`NE 0'0
`<1-
`E8
`
`o
`
`U,
`
`c S
`
`La:
`of).
`NI ED-
`
`
`
`N 4—:
`
`N %
`
`- J)
`El 0".._,c)
`(50
`E E3
`ESE!
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0050
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 49 of 135
`
`US 6,289,138 B1
`
`31
`
`Input RGB pixels
`
`Lookup interval &
`fractional tables
`
`Concatenate 4 bit
`
`
`
`intervals to form
`12 bit address
`
`
`
`
`Convert 12 bit address
`
`to 8 x 9 bit bank address
`
`
`Remap 9 bit addresses
`
`
`
`Compare 9 bit
`addresses with tag
`
`Read output colour
`values from external
`
`
`
`
`
`
`
`
`
`
`memory into cache
`
`Look up output colour
`values from cache
`
`
`
`lnterpolate
`
`Output CYMK
`colour values
`
`Fig. 65
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0051
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0051
`
`

`

`US. Patent
`
`Sep. 11, 2001
`
`Sheet 50 of 135
`
`US 6,289,138 B1
`
`vw(v15+v16y2
`
`
`U15(U15+U16)]2(V1+V2)l2
`
`
`
`
`
`
`V1
`
`
`
`U1(u1+u2y2
`ifnofiltering:
`
`iffiltering:
`
`—§
`
`[39.66
`
`601
`
`inMUV
`{—
`
`
`
`
`(ON
`C’)
`\—l\
`GOV
`VOCO
`EHEIIIII
`0003K)
`EHEIIIII
`NGDVI'
`EHEIIIII
`v—'F~CO
`EHEIIIII
`OCDN
`FNVr
`LO!—
`NV
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`x—FC‘O
`IiiIIIII
`
`
`EEEIEIII
`N
`w
`(‘00)
`HHHIIIII
`NW
`EHHIIIII
`FN
`HHHIIIII
`V'NCO
`IEHIIIII
`LO
`0")
`0")
`
`oo<r
`NFC“)
`
`
`600
`
`
`
`604
`
`603
`
`602
`
`
`
`
`
`
`EIIIIIII
`
`
`
`EIIIIIII
`601
`IIIIIIII
`
`
`IIIIIIII
`
`
`IIIIIIII
`IIIIIIII
`
`
`IIIIIIII
`
`
`IIIIIIII
`
`
`EIIIIIII
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0052
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0052
`
`

`

`US. Patent
`
`Sep. 11, 2001
`
`Sheet 51 of 135
`
`US 6,289,138 B1
`
`v15(V15+V16)/2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IIIIIIII
`FIIIIIII
`>_
`
`
`IIIIIIII EIIIIIII
`
`
`
`IIIIIIII
`
`
`
`
`
`
`> IIIIIIII
`IIIIIIII
`
`
`
`IIIIIIII
`IIIIIIII
`
`
`
`QIIIIIII C)IIIIIIII
`
`
`
`IIIIIIII
`IIIIIIII
`
`
`
`613
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`612
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0053
`
`
`
`.0.)
`C
`'C
`”
`
`2h
`
`: o
`
`V1iffiltering:(u1+u2)/2(U15+U16)I2(v1+v2)/2
`
`
`
`U15
`
`U1
`
`
`
`C H
`
`:
`
`_+
`
`
`
`IHZIIIII
`
`
`NCO
`NO“)
`
`[x
`EEHIIIII
`
`OCD
`aaalllll
`
`LO
`EEHIIIII
`
`CD?
`EIHIIIII
`
`CO
`HEEIIIII
`
`
`
`
`
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0053
`
`

`

`US. Patent
`
`Sep. 11,2001
`
`Sheet 52 of 135
`
`US 6,289,138 B1
`
`Pbus interface to PO
`
`Cbus interface to IC
`
`620
`
`632
`
`621
`