a2, United States Patent
`US 6,289,138 B1
`(10) Patent No.:
`Yip etal.
`(45) Date of Patent:
`Sep. 11, 2001
`
`
`US006289138B1
`
`(54) GENERAL IMAGE PROCESSOR
`(75)
`Inventors: Dominic Yip, Lindfield; Ian Gibson,
`Coogee; Mark Pulver, Enmore,all of
`AU
`(AU)
`(73) Assignee: Canon Kabushiki Kaisha, Tokyo (JP)
`:
`:
`.
`Loy
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`US.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/025,725
`“44.
`Filed:
`(22)
`Feb. 18, 1998
`(30)
`Foreign Application Priority Data
`
`OTHER PUBLICATIONS
`Evanset al., “Systolic Algorithmsfor Digital Image Filter-
`ng Parallel Computing,vol. 21, Issue 1, pp. 109-119,Jan.
`:
`Evanset al, “Systolic Algorithms for Digital Image Filter-
`ing”, Parallel Computing, vol. 1994, Issue 3, p. 12, Aug.
`1994.*
`Huang et al., “Novel Gray Image Processing in a Single
`Operation Optical Morphological Processor”, Optics Com-
`munications, vol. 130, Issue 1-3, pp. 143-152, Sep. 1996.*
`
`Primary Examiner—Jose L. Couso
`Assistant Examiner—Anh Hong Do
`(74) Attorney, Agent, or Firm—Fitzpatrick, Cella, Harper &
`Scinto
`
`(57)
`
`ABSTRACT
`
`(AU) cesssssssssssssssssssssssssssssssssssessesssesseee Po6480
`Appr. 30, 1997
`
`(AU) cesssssssssssssssssssssssssssssssssssessssssesseee PO6481
`Apr. 30, 1997
`The present invention relates to an image processor (242)
`(AU) cecscesseesssessssssesssesstecstecnseencerseenneene PO6482
`Apr. 30, 1997
`
`
`comprising a control register block (1470), a decoding block
`(AU) vesececsscseeresseesscenssseesneneeses PO6484
`Apr. 30, 1997
`
`(1468), a data object processor (1462), and flow control
`(AU) vesesssssssseeeeeeeesenennnnnnnsseeceeeeetenns PO6485
`Apt. 30, 1997
`logic. The control register block (1470) stores all the rel-
`(AU) hess esecsesessreneseeeeneneeenes PO6488
`Apr. 30, 1997
`evant information about the image processing operation.
`(AU) veeecccececseensssenssesesssesenesenenes PO6489
`Apr. 30, 1997
`
`The decoding block (1468) decodes the information into
`(AU) Terre
`“Pr ay hoo
`es Ponto!
`
`configuration signals, which configure an input data object
`(AU) eee PQ6492
`‘Apr 30. 1997
`
`
`
`© BO,1997(AU)veeeececsseestsesescnecnsceneceneestesseeneees interface (1460). The input data object interface (1460)
`
`
`accepts and stores data objects from outside, and distributes
`7
`(51)
`Tint, Cd eeeesessncsnsesseenteeneeneeneees G06K 9/60
`these data objects to the data object processor (1462). For
`:
`some image processing operations,
`the input data object
`(52) U.S. C1. eeescceseseecseecssscneceneeseess 382/307; 382/308
`interface (1460) may also generate addresses for data
`(58) Field of Search 0... 382/307, 308,
`objects, so that the source of these data objects can provide
`382/233; 712/248, 36; 708/190
`the correct data objects. The data object processor (1462)
`performsarithmetic operations on the data objects received.
`The flow control logic controls the flow of data objects
`within the data object processing logic (1462). More
`particularly, the data object processor (1462) can comprise
`a number of identical data object sub-processors, each of
`which processes part of an incoming data object.
`40 Claims, 135 Drawing Sheets
`
`(56)
`
`Re. 31,736
`Re. 32,103
`eC.
`5
`
`References Cited
`U.S. PATENT DOCUMENTS
`11/1984 Mueller et al. oe 364/900
`
`iloss Matsumoto ct al. ve
`xeon
`
`cece ecee eee eee eee
`UTAKAMIL CL AL.
`(List continued on next page.)
`FOREIGN PATENT DOCUMENTS
`
`0588726
`
`3/1994 (EP) wcecceceeeceeeenes G06F/15/332
`
`Microfiche Appendix Included
`(2 Microfiche, 101 Pages)
`
`242
`“a
`
`1454
`
`Imagepixel,
`matrix coeff.
`&1&V values
`
`Pixel values,
`CStable values,
`matrix value &
`ate.
`
`1451
`
`1452
`
`1453
`
`1464
`
`Instruction
`
`Word
`Register
`1471
`
`Instruction
`Word
`
`Decoder
`
`Control
`1460
`
`
`Signal
`Register
`
`Image Data
`Processor
`
`
`
`
`1475
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0001
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0001
`
`

`

`US 6,289,138 B1
`
`Page 2
`
`sereceeereeeeeenneesensscacs
`
`9/1993 Dalrymple et al... 358/500
`5,243,414
`U.S. PATENT DOCUMENTS
`9/1993 Uramoto et al.eee 364/725
`5,249,146
`340/146.1 R
`5/1975 Frank
`3.983.847
`10/1993 Chu et al.
`ececceccescessessennen 358/133
`5,253,053
`:
`PAINE
`20"
`10/1993 Balkanski et al.
`....sssssssseen 358/426
`5,253,078
`stove Speleeral sesesensseeesenene welse
`3 .
`
`nvho03 Nt et all ernseas
`eeeoa
`10/1981. Mayer et ale
`cesssssssssensenen 364/900
`4,296,476
`/
`ewbergeret al.
`-
`304/
`22985
`5/1982 Pratt et al.
`vccccsccesesesssssssseeeees 364/515
`4,330,833
`
`een D190 rnd ee
`sey
`5/1983 Wildergren etal. ..
`. 364/725
`4,385,363
`
`
`
`.
`suchiya ef al.
`208,
`......sscs0000 364/200
`7/1984 Christopheret al.
`4,460,958
`12/1993 Balkanski etal.
`- 358/432
`5,270,832
`10/1984 Kanayama ecsssssseseeseeneen 364/900
`4,475,174
`
`5,283,866
`2/1994 Kumagai.....
`. 395/164
`340/347
`8/1985 Weaver
`4,535,320
`
`3/1994 Nakamuraetal.
`- 358/403
`5,299,027
`10/1985 Bechtolsheim ..seessssssseeeseee 364/200
`4,550,368
`
`4/1994 Fukudaetal......
`- 358/261
`5,303,058
`5/1986 Rodman sresecccsssssesssessenen 364/200
`4,587,610
`. 395/275
`5,303,349
`4/1994 Warrineretal.
`. 340/747
`1/1986 Atkinson .
`4.622.545
`
`4/1994 Clare ssssesseeevn
`5,307,451
`» 395/127
`cacesscssscseesseesssese 340/347
`2/1987 Bleds0€
`4,646,061
`
`- 395/166
`5,313,577
`5/1994 Meinerth et al.
`7/1987 Hester et ale
`sccsssssssseeusseeen 364/200
`4,680,700
`
`....
`- 395/425
`5,317,717
`5/1994 Cutleret al.
`10/1987 Bledsoe «.......
`340/347
`4,700,175
`
`
`. 395/162
`5,321,806
`6/1994 Meinerth et al.
`1/1988 Guttag etal. ..
`364/518
`A718.024
`
`6/1994 Allen etal. ...
`w. 341/65
`5,325,092
`1/1988 Kobayashi et al
`383/41
`4718091
`..
`. 358/479
`5,325,215
`6/1994 Shibata et al.
`1/1988 Chambers
`382/42
`470871
`
`
`. 395/500
`5,333,297
`7/1994 Lemaireetal.
`.........
`4/1988 Chabert
`382/41
`4736440
`
`8/1994 Furukawaetal. .
`5,337,319
`371/111
`6/1988 Mischler et a
`382/41
`4754491
`
`
`8/1994 Balkanski et al.
`.
`. 364/725
`5,341,318
`,
`10/1988 Asaiet al
`364/900
`4779993
`
`
`9/1994 Andersonetal. .....
`... 341/51
`5,349,348
`10/1988. Daly et all a
`358/133
`4.780.761
`9/1994 Hertherington etal. .
`. 395/400
`5,349,651
`12/1988 Liouet al. “
`364/725
`4.791.508
`. 345/191
`5,351,067
`9/1994 Lumelskyetal.
`1/1989 Amitai
`,
`~ 364/900
`4.797850
`ee sit008 Coto ston
`x0n078
`3/1989 Fedele ..
`vee 375/27
`4,813,056
`
`4,823,286 anne||UCMOMEhed :4/1989 Lumelskyetal. .. 364/521 ,
`
`
`
`
`6/1989 Urushibata.....
`. 364/518
`s+ S41/67
`4,839,826
`5,392,038
`2/1995 Bhandari et al.
`
`8/1980 Mukherjee
`3a6s
`2/1995 Lentz etal. .....
`. 395/115
`4.853.696
`5,394,515
`
`
`364/728.01
`4,907,182
`3/1990 Guilianoet al.
`nee eito0s cumaget ct al.
`ae
`
`”
`4/1990 Hatorietal. ......
`.. 358/433
`4,920,426
`w20
`Zak)
`
`4/1990 Murakamietal.
`364/200
`7/1995 Yamauchietal.
`- 358/448
`4,920,480
`5,436,734
`
`5/1990 Boreland .....
`8/1995 Okamoto........
`4.926.355 *
`.. 712/36
`5,440,404
`- 358/432
`. 395/401
`4035321
`G/1990 Samo et al
`358/427
`5,446,854
`8/1995 Khalidi etal. ..
`
`6/1990 Malinowski...
`4,937,774
`364/724
`oe oyt008 Kopp et al.
`.
`s4e384
`.
`4,956,771
`9/1990 Neustaedter
`364/200
`oO?
`TEDT
`vesesecrees
`10/1990 Tokuume .......
`4,965,722
`364/200
`5,467,088
`11/1995 Kinouchiet al.
`+ 3A1/65
`12/1990 Kimata etal. .
`4,975,976
`382/44
`oes too ven ae
`sens
`
`1/1991 Hourvitz etal.
`4,982,343
`364/521
`2483475
`«1/1906 Kae OE A sesseceenesnseeseeaeecerees 5e4 N05
`
`4,983,958
`1/1991 Carrick .......
`340/799
`5483,
`/
`AO .eeccsecssncceeceeseeceeeceneeeeeees
`/
`2/1991 Callemyn.......
`4,991,112
`364/518
`5,485,557
`1/1996 Satoetal....
`- 395/129
`
`
`6/1991 Murakamiet al
`1/1996 Venable et al. oe 395/155
`5025482
`382/56
`5,485,568
`3000122
`7/1901 Uetani
`364/125
`5,485,589
`1/1996. Kocis et al.
`sacseessssssseeee 395/421.03
`
`9/1991 Watanabeetal.
`we 395/147
`5051840
`358/433
`5,502,804
`3/1996 Butterfield et al.
`
`3/1996. Heil cecccsccssssssssssessecsessseeeseen 395/293
`oes
`5,502,824
`.
`10/1991 Friedlanderetal.
`5,053,985
`364/725
`me
`.
`
`;
`5,504,842
`4/1996. Gentile vaevcccsssssssssssssssseeseeen 395/114
`
`10/1991 Arbeiter0
`5,060,242
`375/122
`204,
`. 395/114
`5,506,944
`4/1996 Gentile.......
`
`5,109,333
`4/1992 Kubotaetall.
`....
`395/275
`06,
`4/1992 Beausoleil et al.
`4/1996 Butterfield et al. oe 395/147
`5,509,115
`5 109.496
`395/400
`
`5125.042
`6/1992 Kerr et al
`382/47
`5,509,137
`4/1996 Itomitsu et al...eee 395/495
`eos
`TE
`Cf
`al
`oon
`5,513,335
`. 395/457
`4/1996 McClure.....
`
`5,125,085
`6/1992 Phillips ..........
`395/400
`als,
`....cceccceeeeceeeees 364/514
`:
`5,515,296
`5/1996 Agarwal
`5,142,380
`8/1992 Sakagamietal.
`358/432
`ne
`
`11/1992 Uetani wsecseen
`5,524,075
`6/1996 Rousseau et al.
`382/302
`5,163,103
`382/56
`
`.. 341/67
`5181.183
`1/1993 Miyazaki
`364/725
`5,528,238
`6/1996 Nickerson.......
`
`ccescsecesssssssesseeseee 375/240
`3195661
`2/1993 N.
`358/75
`5,528,628
`6/1996. Park et al.
`
`6/1996 Heil cecccccccsscsscsscssssssstserseeeeeee 395/293
`B crecreereereereee
`7S
`5,528,764
`2/1993 Edenfieldetal.
`5,185,694
`395/425
`228;
`
`
`2/1993 Alcorn etal. ..
`. 395/130
`5,185,856
`5,530,823
`6/1996 ‘Tsuchiyaetal.
`- 395/417
`
`6/1996 Stones aecscresrssssessssssssseeeeeen 395/494
`ees
`5,530,944
`
`........
`364/728.01
`5,195,050
`3/1993 Hsuetal.
`30,
`:
`5,535,291
`7/1996 Spenceret al. veces 382/254
`
`3/1993 Balkanskiet al.
`.. 358/433
`5,196,946
`22>;
`;
`. 395/115
`;
`5,539,865
`7/1996. Gentile.....
`
`.. 364/725
`3/1993 Cucchiet al.
`3),
`5,197,021
`.
`
`8/1996. Gentile vaevecssssssssssssssssseeeeeen 395/115
`5,544,290
`5,204,830
`4/1993 Wangetal...
`. 364/754
`a4,
`‘
`5,544,342
`8/1996 Dea sescevesssrssssessseseesseeneeen 395/446
`5,212,559
`5/1993. Gilbert et al. vssessessssseeeen 358/298
`4,
`w. 395/162
`5,557,733
`9/1996 Hicoketal.
`
`6/1993. Tanaka et al. vssesssssseeen 358/426
`5,216,516
`2h
`
`caesesessssssseeee 395/183.06
`358/133
`5,561,761
`10/1996 Hicok et al.
`2593026
`-G/1993. Stone et al.
`7/1993. Barry et ale cccsssssssssssessnee 341/65
`5,227,789
`3,561,772
`10/1996 Dormier et al.
`sesvesvreeresrens 395/281
`5,233,348
`8/1993 Pollmann et al.
`cecccccccssssseseee 341/67
`5,570,432
`10/1996) Kojima oe ccceseeeeeee 382/254
`.. 395/162
`5,237,655
`8/1993 Statet al.
`.....
`
`8/1993 Small et al. wc 307/449
`5,241,222
`
`
`
`
`
`* cited by examiner
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0002
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0002
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 1 of 135
`
`US 6,289,138 B1
`
`032
`
`226
`
`a
`
`| I
`
`202
`
`Host
`
`CPU
`
`204
`
`| host
`
`ram
`
`207
`
`PClbus
`interface
`
`I I I I I I
`
`220 ~
`
`225
`
`223
`
`Local
`Memory
`
`Peripheral
`Interface
`
`227
`
`224
`
`Raster Image
`Coprocessor
`
`Generic
`External
`interface
`
`I I I | I I I
`
`PCI bus
`rr-"-—cenr oor r OrPeOe ee r= ee ere er er er ee a
`
`PClb
`
`us
`
`ther
`
`1° 206
`
`0
`
`|
`
`1
`
`1
`|
`
`1
`
`211
`
`212
`
`!
`a0 Other
`
`'
`
`I
`
`i
`
`U 1
`
`201
`
`Fig. 1
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0003
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0003
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 2 of 135
`
`US 6,289,138 B1
`
`224
`
`External Cbus Interface
`
`
`Diagnostic 11
`it Clocks and
`"
`and Wl
`i global
`
`rl
`Test Ports
`PCIbus !!
`"
`
`I
`
`A
`i
`
`
`Data
`Generic I
`External
`[~~~ "
`Input
`Data
`=I cache
`Interface n>}
`Interface
`F--»}
`Interface
`Cache
`
`
`
`
`1|Controller [*""5 Switch Controller
`
`
`
`
`iT]
`i
`I 246 DAT :
`7
`'
`227 "
`
`|
`
`231
`
`to all
`modules
`
`230
`
`ae
`
`248
`
`Organiser C
`
`
`
`2|
`
`206
`
`
`
`
`
`:
`Page Mode" —{
`DRAM or
`Memory a |
`
`
`Controller,
`Synchronous"
`= 4
`
`
`DRAM !
`
`
`
`
`:
`:
`237 ¢
`
`i
`:
`
`Centronics I
`Peripheral]!
`Interface
`lqamp
`Interface i
`
`Controller]
`or Video
`§
`Interface,,
`. 250
`
`= = = = Represents ASIC pads
`
`External interface
`
`Control Bus (Cbus)
`--—-= Input Bus (Ibus)
`==s=e= Operand Bus (Obus)
`
`- — — - Partial Result (Pbus)
`
`SuGEBeEEe Result Bus (Rbus)
`mses - Interface Bus (Abus)
`= m= = Internal RAM interface
`
`
`249
`
`j
`
`Fig. 2
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0004
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0004
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 3 of 135
`
`US 6,289,138 B1
`
`External Interface Controller
`
`Local Memory Controller
`
`Peripheral Interface Controller
`
`Miscellaneous
`
`Instruction Controller
`
`Data Cache Controller
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Pixel Organiser
`
`
`
`Operand Organiser C
`
`
`Main Data Path
`
`
`
`Result Organiser
`
`Input Interface Switch
`
`reserved
`
`
`261
`
`Operand Organiser B
`
`JPEG Coder
`
`PClbus Configuration Space Alias
`
`Coprocessor
`module
`registers
`
`262
`
`Fig. 3
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0005
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0005
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 4 of 135
`
`US 6,289,138 B1
`
`224
`
`
`
`Co-Processor
`
`MEMORY203
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0006
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0006
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 5 of 135
`
`US 6,289,138 B1
`
` 1030
`
`Instruction
`
`Generator
`
`
`
`1031
`1032
`
`
`Memory
`Queue
`
`Manager
`Manager
`
` 224
`Coprocessor
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0007
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0007
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 6 of 135
`
`US 6,289,138 B1
`
`202
`
`Pending
`Instructions
`Queue
`
` Completed
`
`Instructions
`(Cleanup Queue)
`
`
`Graphics
`Coprocessor
`
`
`204A
`
`Fig. 6
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0008
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0008
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 7 of 135
`
`US 6,289,138 B1
`
`1040
`
`\
`
`Co-ProcessorIP
`
`Queue Manager
`last instruction added
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0009
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0009
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 8 of 135
`
`US 6,289,138 B1
`
`cleanup
`
`finished
`
`interrupt
`
`todo
`
`cleanup
`
`finished
`
`interrupt
`
`todo
`
`Stream A
`
`Stream B
`
`START
`
`
`
`Register update
`Instruction stream
`complete
`to fetch next
`
` Instruction pending?
`
`Which stream?
`
`
`d
`
` Fetch
`
`Update
`Instruction
`Registers
`
`
`
`
`Decode and
`
` Instruction
` Instruction
`Execute Instruction
`complete
`Fig. 9
`
`c
`
`270
`
`a
`
`271
`
`a!
`
`275
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0010
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0010
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 9 of 135
`
`US 6,289,138 B1
`
`280
`
`co~
`31
`0
`
`281——
`
`Instruction Opcode
`
`<@-— Instruction Word
`
`282———_| Operand and Result types|««- Data Word
`
`283———| Operand A addressor data|<q Operand A Word
`
`284——_| OperandBaddressor data|<q Operand B Word
`
`
`
`
`
`
`
`
`285-——| Operand C addressor data|«q- Operand C Word
`
`286——
`
`287.
`
`Result address
`
`For use by Host
`
`Application
`
`
`
`
`~<@- Result Word
`
`Fig. 10
`
`31
`
`28 27
`
`20 19 18 17 16 15
`
`0
`
`major
`
`minor
`
`
`pede]opeode|fpofn]u] lena
`
`foC(i‘C
`\290
` opcode
`291
`P
`292
`293
`294
`
`296
`
`Fig. 11
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0011
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0011
`
`

`

`Sheet 10 of 135
`
`U.S. Patent
`
`DION,&}eG9U}JO}eEWIOS
`
`
`
`
`
`‘SUOIONISUIPueJOdOBOUL
`
`Sep. 11, 2001
`
`
`
`i”oaals]»jslewlrfo|s|»afew)atols|»Jalew)atols|»[200¢S|OL€%v2
`
`Jo\duosapyJo}duosap9JoyduosepgJo\diosapVW>
`
`
`
`
`
`
`
`‘SUOHONJJSUIpuesadoO(SS9a]JO)OM
`
`
`
`
`
` <>~2[uls]_»[a]___ssonpesofo}si»[aLO€GbOL€2¥2_[rewlfo}s|_#Jatewl
`
`
`
`
`
`
`
`
`
`
`
`
`US 6,289,138 B1
`
`ZL“Bis
`
`JoyduosepyJo}duosapqJO}duoOsep
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0012
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0012
`
`
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 11 of 135
`
`US 6,289,138 B1
`
`235
`
`a 1
`
`Interface --—-
`Switch
`
`306
`
`
`Instruction
`
`Decoder
`
`307
` Input
`
`
`
`Miscellaneous
`Module
`
`External
`Interface
`Controller
`
`i
`
`I I
`
`Execution
`
`Module
`
`poorer eee ee KKK 4
`
`I I;_C
`
`bus|Control
`308
` Cbus
`Interface Register}
`1
`File
`
`Controller Miscellaneous
`
`
`Arbitrator
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0013
`
`
`
`I I I
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0013
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 12 of 135
`
`US 6,289,138 B1
`
`Prefetch Buffer
`Controller
`
`instruction
`Decoder
`
`
`
`
`Overlapped
`Instruction
`Decoder
`Prefetch
`Interface
`Instruction
`
`
`
`Logic
`Logic
`Logic
`
`
`
`
`
`Register
`Central
`Management
`Brain
`
`
`
`i
`
`IC CBus
`
`Interface-
`
`
`
`Miscellaneous Module
`
`Fig. 14
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0014
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0014
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 13 of 135
`
`US 6,289,138 B1
`
`
`reset ——p>
`
`
`Decide
`
`whetherto
`executean
`instruction
`
`
`312
`
`FETCH *
`
`ye
`
`
`
`Waitfor
`
`
`overlapped
`instruction to
`
`finish
`
`
`Wait for
`
`decoding to
`complete t10
`
`
`
`
`UPDATE*
`Update
`Registers
`
`
`t11
`
`PREFETCH
`
`
`313
`
`Prefetch
`instruction
`
`
` Wait for
`instruction
`to “finish”
`
`Fig. 15
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0015
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0015
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 14 of 135
`
`US 6,289,138 B1
`
`Prefetch Buffer Controller
`
`Execution
`Controller
`
`? —=_——or ree 7
`
`Arbitrator
`
`CBus
`Dispatcher
`
`CBus
`
`306
`
`Instruction Controller Cbus Interface
`
`Fig. 16
`
`Instruction
`Controller
`Execution
`CBus Interface
`Controller
`
`
`Prefetch Buffer
`Controller
`
`Overall
`Sequencing
`Control
`
`
`
`
`
`Per Module
`(there are 8 different
`Per Module
`
`Configuration
`Configuration} Per Module Configuration
`
`
`
`Sequencer
`Sequencer
`Sequencers)
`
`
`321
`
`
`
`CBus Dispatcher
`
`Fig. 17
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0016
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0016
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 15 of 135
`
`US 6,289,138 B1
`
`Wait for
`
` Configure
`
`Pixel
`
`Configure
`Peripheral
`Interface
`Controller
`
`Configure
`Result
`Organiser for
`IA operation
`
`
`
`Configure
`Operand
`7
`Organizer B
`
`330
`
` start
`Organizer
`
`
`
`
`Configure
`Data Cache
`Controller
`
`
`Coder
`
`
`
`
`
`
`
`
`
` Configure
`
`JPEG
`
`Configure
`Result
`Organizer
`
`MDP*
`
`Configure
`Main Data
`Path
`
`Configure
`Operand
`Organizer C
`
`
`
`Fig. 18
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0017
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0017
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 16 of 135
`
`US 6,289,138 B1
`
`335
`
`I
`
`Prefetch
`Buffer
`
`
`
`PBC
`ntteoe |
`Ibus
`
`
`
`Controller
`i
`
`
`Switch |
`Sequencer
`
`Instruction
`Decoder
`
`Execution
`Controller
`
`i
`
`307
`
`Address
`
`Manager
`
`337
`
`instruction Controller
`Cbus Interface
`
`Fig. 19
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0018
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0018
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 17 of 135
`
`US 6,289,138 B1
`
`COE
`
`vOeJ
`
`Ove
`
`\dnJ
`
`Lve
`
`O9€
`
`voz‘big
`
`-19}U!il
`~
`il
`
`walbocceeeeeenccenceeeccecesecetgeecaceseceeseeceeseuseceesossesoveceetePensseeceneneoreseneseedeOucasneeeeees|-
`
`
`
`
`
`~peysewunjo}ndjnooseIjaseloOIno]fulpeasoO
`Joasu9}sibai;eyepBailOsesve
`
`
`js]9S|I‘no‘pela~gjoyuool|}Z1043u09]}|Zjo1ju00;peasof|‘pyenoO
` 1S]9S PJOMzByuoo
`
`
`piomIEa}Aq
`_plomayAq
`|.|04jU09]||.[04}U09/]|1;ou]U0D
`TTTTTeeeyep7ans}US
`
`
`
`
`$.19}si6adoly,jo9jqeua‘okqpegNs‘aqoljse19)s/694
`
`
`Joyndjnopueyndu|
`PJOM,6yuo09i
`
`UuOnoesS‘;s19}siboujnop‘apippe
`
`
`_9yAq_4a,6yuoo,6yu0o
`SainpOwWGnNsJ38yIOree
`
`
`I
`
`SJUAA/PESY
`
`49}|01}U0D)
`
`(uooMI)
`
`
`
`Byeps9‘ulByeps9
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0019
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0019
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 18 of 135
`
`US 6,289,138 B1
`
`sng9
`
`
`
`
`
`€SEII;|7086SEjoyndyjno;eesebannabssommesemeeeeeeBemeeseemoceeeeeeenewseeeeeeeeeeleeeeemeeeeeeeeeeesohoee5643p9q
`
`
`
`
`Jouydnw9}U|/OUg|eyeps9II‘jnoILSE9SEI—|1019—BSE|PHeAS©
`
`
`
`ydnwayuj(ue6Jur49)IIJO}BIaUaS)JOJEIBUSBSJO}BJBUBS|}no~!ydnw9}Uj/J04
`
`c9ENeeeee|ose
`|peysewun|eyepBeas!i———||(uahweysewun)|{JoyesauasI1JOpeysewuy
`sia\sibau
`I|l(usByur9)(uoByurwa)
`coeg0z‘614
`6re~.8PE\|ue9LvE~|wl9|dxe™o
`
`e
`
`cOe
`
`‘pIIeA9
`
` jase!9‘ulpeas9‘ul
`
`eyeps9‘pero
`
`Sve
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0020
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0020
`
`
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 19 of 135
`
`US 6,289,138 B1
`
`Type A
`
`
`
`2827=24.23 20 19 16 15
`
`
`
`ome[beA371
`
`se[eel|Yo
`
`Type B
`
`
`
`2827=24 23 0
`
`Type C
`
`28 27
`
`24 23
`
`2019
`
`1615
`
`8 7
`
`0
`
`ome[om[roi]ve[ome|
`S372
`
`Fig. 21
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0021
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0021
`
`

`

`
`
`
`
`
`Main
`Data path
`
`
`output
`i
`!
`383 “4 internal rr
`
`@ --- Hahn
`|
`I Fr comer ae® 1
`““@7
`aT FO TTT ag
`
`380
`
`242
`
`: 246
`382 ~ input
`‘
`internal
`:
`format
`
`main
`;
`functional
`
`244
`
`:
`
`format
`
`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 20 of 135
`
`US 6,289,138 B1
`
`input
`:
`381~_: external
`: format
`
`;
`'
`
`;
`'
`247
`
`'
`L..-..
`
`248 :
`
`input
`organisers
`
`Operand
`Organiser B}
`
`Operand
`{Organiser C
`
`
`
`output resut (
`
`Organiser
`organiser
`249
`
`
`
`:
`:
`Vv
`
`384
`
`output
`™ external
`:
`format
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0022
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0022
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 21 of 135
`
`US 6,289,138 B1
`
`387——~}cH2]ci1|cH0]byte address
`
`this data
`
`oO
`
`386
`
`~
`
`ofthisdat
`aN LL
`
`x0
`
`
`
`396
`
`byte address
`of
`this data
`
`395
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0023
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0023
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 22 of 135
`
`US 6,289,138 B1
`
`byte address
`
`400
`
`of this data
`
`
`
`404. —r
`
`Fig. 29
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0024
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0024
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 23 of 135
`
`US 6,289,138 B1
`
`410—» External
`Formats
`
`packed
`stream
`
`unpacked
`bytes
`
`other
`
`411—» Internal
`Formats
`
`packed
`bytes
`
`pixels
`
`unpacked
`bytes
`
`other
`
`Fig. 30
`
`unpacked
`;
`packed
`Internal
`
`412 —>Formats bytes pixels bytes
`
`
`
`413 —» External
`Formats
`
`packed
`stream
`
`unpacked
`bytes
`
`Fig. 31
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0025
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0025
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 24 of 135
`
`US 6,289,138 B1
`
`from input modules
`
`external
`unpacked
`bytes
`
`417
`
`418
`
`unpacked
`normalisation
`
`external
`packed
`stream
`
`422
`
`external
`other
`
`packed
`normalisation
`
`421
`
`A16
`
`internal
`pixels
`
`internal
`packed
`bytes
`
`internal
`
`419
`
`423
`
`
`
`MCU/MUV ¢-eeereeeeeeeees
`handling
`—
`
`446
`
`Fig. 32
`
`to main
`functional units
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0026
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0026
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 25 of 135
`
`US 6,289,138 B1
`
`417 we
`
`
`
`unpacked
`normalisation
`
`
`
`419
`
`a
`
`0
`
`418
`
`Fig. 33
`
`422
`
`31
`
`423
`
`~~
`
`data item 1
`
`N
`
`3/4[%2 |3/[1|2 bits per channel eet st
`3 channelobjects,
`
`
`
` 421
`packed
`normalisation
`
`
`Fig. 34
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0027
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0027
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 26 of 135
`
`US 6,289,138 B1
`
`425
`
`pack
`
`
`
` Fig. 36
`
`unpack
`
`426
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0028
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0028
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 27 of 135
`
`US 6,289,138 B1
`
`data item 1
`
`data item 2
`
`
`31
`
`data item 1 j_ch2|chi Fig. 37
`
`
`
`
`
`
`data item 2 | ch2|cht [cho|"
`
`
`
`31
`
`data item 1
`
`data item 2
`
`0
`
`>
`
`component
`substitution
`
`440
`
`internal data register,-441cote
`
`
`
`Qo
`31
`data item 1[ “DTcha|B[ch
`dataitem2{__D_|ch2_|BT cho|
`
`442
`
`Fig. 38
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0029
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0029
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 28 of 135
`
`US 6,289,138 B1
`
`0
`
`0
`
`31
`
`data item 1
`
`data item 2
`
`31
`
`data item 1
`
`data item 2
`
`lane
`
`Fig. 39
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0030
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0030
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 29 of 135
`
`US 6,289,138 B1
`
`31
`ch3
`
`data item1[
`
`data item 2
`
`0
`
`cho
`
`cho
`
`data item 1
`
`data item 1
`
`=
`
`data item 2
`
`data item 2
`
`
`
`(replication factor = 1)
`
`446
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0031
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0031
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 30 of 135
`
`US 6,289,138 B1
`
`from main functional units
`
`383 ‘ internal
`internal
`packed unpacked_internal
`bytes
`bytes
`pixels
`beceeeeeseeees
`450
`i MCU/MUV3----2-----22-220+
` 424
`
`
`
`
`external
`internal &.| unpacked
`
`component
`packe
`packe
`
`bytes
`substitution
`stream
`bytes
`
`handling q............--..--
`
`
`
`
`lane swap
`
`/
`
`internal
`unpacke
`bytes
`
`external
`
`
`
`
`component
`
`4951
`
`452
`
`
`
`deselection
`
`byte
`addressing
`
`453
` 454
`
`Fig. 42
`
`to output modules << 384
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0032
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0032
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 31 of 135
`
`US 6,289,138 B1
`
`
`
`
`
`data item 1 a
`“| ch2_ [cht
`data item 2
`
`
`'
`:
`'
`:
`
`
`
`0
`
`
`
`component
`deselection
`
`
`451
`
`Fig. 43
`
`2 bits per channel
`
`data item 1
`
`456
`
`data item 2
`
`data item 1
`
`data item 2
`
`Fig. 44
`
`3 channel objects,
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0033
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0033
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 32 of 135
`
`US 6,289,138 B1
`
`31
`
`
`
`byte
`addressing
`
`addressof |
`output stream--._
`
`453
`
`
`
`0
`
`
`
`“Toy
`
`0
`
`31
`
`3 channel objects,
`
`2 bits per channel
`453
` address of
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0034
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0034
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 33 of 135
`
`US 6,289,138 B1
`
`0
`31
`data item 1) ch3|ch2|chi|ch0
`data item 2} ch3|ch2|ch1|cho
`
` data item 1
`
`YYa2cho|data item 2
`Vl
`"byte not
`
`written
`
`Fig. 47
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0035
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0035
`
`

`

`jsscum|weafon|is]ar|os¢€bG9L86OLLLCLELHLGLOLZL8bGL02Le22€%beGZ9%Le82BZOELE ‘(dupOJ)
`9SI|BWIOU jsean:|wea|on|as]an|os¢€bGF9286OLLLZiGLHIGbOLLIBI6)Ozlzzz€ZHzSZOZLZBZOZOFLe
`
`
`
`(wp900pukeJWpgoo‘iwpod)suajsiBayuonejndiueyeyegyndu|
`
`
`
`
`
`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 34 of 135
`
`US 6,289,138 B1
`
`@SIBWJOU
`
`qswo
`
`8p‘bi4
`
`0 L
`
`
`
`
`
`JaysIBayUONe;ndiueyeyegjndjno
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0036
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0036
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 35 of 135
`
`US 6,289,138 B1
`
`Instructions
`
`Input Data Word
`
`Programming
`Agent
`
`Data
`Normalizer
`
`1061
`
`1063
`
`Engine
`
`Central Graphics
`
`Output Data
`
`Fig. 49
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0037
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0037
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 36 of 135
`
`US 6,289,138 B1
`
`E
`
`nf2..0]
`
`Input Data Word
`
`1069
`
`1073
`
`
`
`First-In-First-Out
`Buffer (FIFO)
`
`fifo_empty
`
`FIFO_RD
`
`1062
`
`fifo_full
`
`1086
`
`FIFO_WR
`
`1085
`
`1074
`
`REG1_EN
`
`Control
`Unit
`
`
`
`x[31..0]
`
`
`
`in-bit[4..0]
`Normalization
`Multiplexors
`n[2..0]
`
`
`
`out_byte[1:0]
`MUX]
`|MUX
`
`
`
`
`
`317°°" 0
`
`
`Input Register (REG1)
`
`1078
`
`1075
`
`
`
`y[31..0]
`
`Output Register (REG2)
`
`1076
`
`Pixel Data
`
`1081
`
`Fig. 50
`
`c[1..0]
`
`b[2..0]
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0038
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0038
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 37 of 135
`
`US 6,289,138 B1
`
`464
`Aw
`
`source
`
`467
`
`operand
`
`465
`
`blend
`
`generator
`
`470
`
`471
`
`462
`
`x
`
`attenuation
`
`466
`
`optional
`multiplication
`
`pre-
`
`optional 4pre-
`
`x
`
`multiplication
`
`469
`
`473
`
`475
`
`476
`
`clamp/wrap
`
`Fi
`ig.
`
`54
`
`477
`optional
`un-pre-multiplication
`
`/
`
`destination <
`
`463
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0039
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0039
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 38 of 135
`
`US 6,289,138 B1
`
` minor
`
`opcede
`
`opcode
`
`34
`
`0
`
`470
`
`
`
`
`|.
`| blend start] blend start| blend start] blendstart) 4 Operand A Word
`“~4 channel 3| channel 2 | channel 4 | channel 0 |
`peran
`°
`
`“hlendendl blend end! blend end|blendend |
`oe Ger
`
`i
`a«ag— Operand 6 Word
`i channel 3 channel 2} channel 1 |channel 0
`pers
`Are |
`Operand C addressor data
`| <— Operand C Word
`
`Fig. 53
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0040
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0040
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 39 of 135
`
`US 6,289,138 B1
`
`2 475
`z!
`
`tiie address
`
`~<_ Operand A Word
`
`Fig. 54
`
`tile start offset
`
`|
`:
`Operand C address or data
`
`tile length - 1
`
`@— Operand B Word
`“~~ 478
`|
`| ~<e— Operand C Word
`
`tile length (bytes) (478)
`
`
`
`
`tile start offset (bytes) (477)
`
`
`tile address (byte address) (476)
`
`Fig. 55
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0041
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0041
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 40 of 135
`
`US 6,289,138 B1
`
`Colour Value] 00000100
`
`481
`a
`
`480
`x
`
`482—Fractional Table (8bit) a
`/256
`
`0
`1
`2
`3
`486
`Sa 4
`5
`6
`7
`8
`9
`10
`11
`
`/256
`/256
`/256
`/256
`/256
`/256
`/256
`/256
`/256
`/256
`/256
`/256
`
`/256
`/256
`/256
`/256
`/256
`/256
`
`Interval Table (4 bit)
`
`0
`1
`2
`3
`485
`‘a 4
`5
`6
`7
`8
`9
`10
`11
`
`0
`0
`0
`0
`0
`0
`0
`1
`1
`1
`1
`1
`
`0
`
`4
`
`I
`
`249
`250
`251
`252
`253
`254
`255
`
`249
`250
`251
`252
`253
`254
`255
`
`14
`
`Fig. 56
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0042
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0042
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 41 of 135
`
`US 6,289,138 B1
`
`/
`Sf f
`
`4-bit
`
`8-bit
`
`4-bit
`
`f-
`
`4-bit
`
`8-bit
`
`8-bit
`
`Interval and
`fraction values
`for channel 2
`
`Interval and
`fraction values
`for channel 1
`
`Interval and
`fraction values
`for channel 0
`
`Fig. 57
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0043
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0043
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 42 of 135
`
`US 6,289,138 B1
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0044
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0044
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 43 of 135
`
`US 6,289,138 B1
`
`510
`
`Output
`Value
`
`(”
`
`Input Value
`
`Fig. 59
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0045
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0045
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 44 of 135
`
`US 6,289,138 B1
`
`Memory Bank 1
`
`
`Memory Bank 0
`
`
`512 x 8 bit
`entries
`
`235
`
`aa
`
`Stage 1: Fracr P| 550
`
`551
`
`953
`
`
`
`Stage 2: Fracgyey4a
`057
`558
`559
`
`
`
`Stage 3: Fracsmi[----
`
`Fig. 60
`
`Output Colour
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0046
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0046
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 45 of 135
`
`US 6,289,138 B1
`
`
`
`4x8bit fo
`
`533
`
`536
`
`52 128
`
`Memory Address
`
`entries
`
`Fig. 61
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0047
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0047
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 46 of 135
`
`US 6,289,138 B1
`
`128 x 32 bit
`entries
`
` Memory Bank 1 Memory Bank 0
`
`
`Stage 3:
`
`Frac_b a
`
`592
`
`ae
`
`595
`
`Fig. 62
`
`CMYK
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0048
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0048
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 47 of 135
`
`US 6,289,138 B1
`
`580
`582
`
`575
`
`578
`a!
`
`PROC THT TY
`
`SoOOo-OOm”
`2 bit
`7 bit
`tag address
`line address
`
`585
`
`587
`
`~—«@ - 32Bits - >»
`
`127
`
`
`
`
`
`
`
`
`
`CiMItiYtk
`—-aAa--l--rF-
`I
`l
`|
`-A--i--F-
`I
`I
`I
`-A--I--F-
`|
`I
`I
`1
`|
`|
`I
`I
`|
`|
`l
`|
`|
`I
`I
`1
`|
`|
`i
`{
`|
`1
`|
`I
`|
`|
`I
`
`
`
`I
`I
`I
`I
`
`
`
`1
`l
`I
`I
`
`586
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0049
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0049
`
`

`

`Sheet 48 of 135
`
`US 6,289,138 B1
`
`a
`5
`LL.
`
`o
`
`O°
`28a.
`
`Oo
`
`@
`OU
`oO
`OO
`ES
`Q
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0050
`
`U.S. Patent
`
`Sep. 11, 2001
`
`oO
`
`Le)
`coz
`S
`covr
`
`oOre
`
`rae)
`
`N
`=|]
`
`oD
`
`~~
`
`A w
`
`l
`
`—
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0050
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 49 of 135
`
`US 6,289,138 B1
`
`S4
`
`Ss
`
`2
`
`S4
`
`Ss
`
`S6
`
`S8
`
`Sg
`
`Input RGB pixels
`
`Lookupinterval &
`fractional tables
`
`Concatenate 4 bit
`intervals to form
`12 bit address
`
`
`
`
`Convert 12 bit address
`to 8 x 9 bit bank address
`
`Remap9 bit addresses
`
`Compare9 bit
`addresseswith tag
`
`
`
`Read output colour
`values from external
`
`
`memory into cache
`
`Look up output colour
`values from cache
`
`
`Interpolate
`
`S10
`
`0 utput CYMK
`
`.Fig. 65
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0051
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0051
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 50 of 135
`
`US 6,289,138 B1
`
`os
`z
`
`<©
`
`
`o>
`e
`
`= fore
`wa” LLL
`aw LLL
`
`
`sf tr
`SL
`
`
`.
`x fT
`© AT
`22ee
`2Coc
`U1(U1+U2)/2
`{ifnofiltering:
`iffiltering: Y
`
`
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0052
`
`Fig.66
`
`601
`
`
`
`inMUV
`<+—
` DINS||tT|
`
`|rot|||
`= |iCOSee
`
`
`
`
`
`Oo
`oO
`c
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0052
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 51 of 135
`
`US 6,289,138 B1
`
`610
`>a
`
`V15(V15+V16)/2
` V1|O[VIUTY|611
`
`
`ifnofiltering:U15iffiltering:(U1+U2)/2(U15+U16)/2(V1+V2)/2
`
`
`U1
`
`
`
`colsNes[CED
`
`
`
`
`h/09reeLT|
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0053
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0053
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 52 of 135
`
`US 6,289,138 B1
`
`Pbusinterface to PO
`
`31
`
`620
`
`Cbus interface to IC
`
`standard
`Cbus
`interface
`
` 632
`
`to all JPEG
`modules
`
`621
`
`622
`
`623
`
`624
`
`
`
`
`
`
`| Obus
`
`Obus
`interface
`unit
`
`I
`
`interface
`to OOB
`
`625
`
`
`multiplexer
`
`
`
`f Pbusinterface to RO
`241
`
`internal
`Pbus
`
`Fig. 68
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0054
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0054
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 53 of 135
`
`US 6,289,138 B1
`
`Jf 622
`
`dct submodule
`
`dct
`
`640
`
`operand
`organiser
`B
`
`Obus
`interface
`
`multiplier
`
`cc
`interface
`
`645
`
`address
`2
`control
`mux
`
`647
`
`
`
`642
`
`to all
`modules
`
`interface Pbusinterface
`
`
`
`
`
`
`coefficient coder 648
`
`Pbus interface
`
`
`
`—= =
`
`control
`
`Fig. 69
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0055
`
`Patent Owner Monterey Research, LLC
`Ex. 2002, 0055
`
`

`

`U.S. Patent
`
`Sep. 11, 2001
`
`Sheet 54 of 135
`
`US 6,289,138 B1
`
`AINGOWENSDH
`
`vc9
`
`ndo
`
`C997SOBL9IUI
`
`,43d0030|!|L.a=os—_=olmwES
`
`TOYLNOO
`
`
`
`
`
`QO-S9bY94Uul9-302L9}uUI
`
`
`
`SARISSNGdJaysewsngd
`
`
`
`
`
`OZ‘biz
`
`
`
`Od-99bp9}u!ON-20ep9}uI
`
`Pat