throbber
a2) United States Patent
`US 6,862,373 B2
`(10) Patent No.:
`Mar. 1, 2005
`(45) Date of Patent:
`Enomoto
`
`US006862373B2
`
`(54)
`
`IMAGE PROCESSING METHOD AND
`APPARATUS
`
`6/2001 Norita et al... 356/337
`6,243,165 Bl *
`6,476,869 B1 * 11/2002 Sekine et al. 0.0.0.0... 348/335
`
`(75)
`
`Inventor:
`
`Jun Enomoto, Kanagawa (JP)
`
`(73) Assignee: Fuji Photo Film Co., Ltd., Kanagawa
`(JP)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`US.C. 154(b) by 2 days.
`
`(21) Appl. No.: 10/201,447
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`
`6-311425
`9-281613
`
`11/1994
`10/1997
`
`* cited by examiner
`
`Primary Examiner—Thomas D Lee
`Assistant Examiner—Stephen Brinich
`(74) Attorney, Agent, or Firm—Sughrue Mion, PLLC
`
`(57)
`
`ABSTRACT
`
`(22)
`
`(65)
`
`Filed:
`
`Jul. 25, 2002
`
`Prior Publication Data
`US 2002/0196472 Al Dec. 26, 2002
`
`Related U.S. Application Data
`
`
`
`The image processing method and apparatus acquire input
`image data from the image recorded optically with a taking
`lens, acquire the information about the lens used to record
`the image and perform image processing schemes on the
`input image data using the acquired lens information, pro-
`vided that the type of the lens used is identified from the
`acquired lens information and the intensity of sharpness
`(62) Division of application No. 09/301,538, filed on Apr. 29,
`enhancementof the corresponding imageis altered in accor-
`1999, now Pat. No. 6,603,885.
`dance with the identified lens type. The characteristics of the
`taking lens may also be acquired from the lens information
`Foreign Application Priority Data
`(30)
`and using the obtained lens characteristics as well as the
`Apr. 30, 1998
`(IP)
`vessessesssessesssessssssesssesssssseseees 10-121263
`position information for the recorded image, the input image
`Aug. 27, 1998
`(JP)
`...
`. 10-241495
`data is subjected to aberration correction for correcting the
`Sep. 16, 1998
`(JP)
`...
`. 10-261304
`deterioration in image quality due to the lens characteristics,
`Sep. 16,1998
`GIP)
`...
`. 10-261305
`provided that the information about the focal length of the
`Sep. 30, 1998
`(IP) woe eee cee ceesseeeecnneecnnaeeenes 10-276679
`taking lens effective at
`the time of image recording is
`(SU)
`Inte C17 aa cceccseescesseestesteneesssessn HON 5/225
`additionally used, or image processing schemesare per-
`(52) US. Che cccccecnsscnesseecnesenenees 382/263; 382/260
`formed in two crossed directions of the recorded image, or
`58) Field of Search
`382/260-269
`
`(58)FieldofSearch«ose 382/276 parameters for correcting aberrations in the imaging plane of
`
`/
`the taking lens are scaled such that they are related to the
`(56)
`References Cited
`output image data on a pixel basis. High-quality prints
`reproducing high-quality images can be obtained from origi-
`U.S. PATENT DOCUMENTS
`nal images that were recorded with compact cameras, digital
`250/550
`4806,774 A *
`2/1980 Linet al
`cameras and other conventional inexpensive cameras using
`5,103,490 A *
`4/1992 McMillin .....cssesssssse 382/284
`[OW-Performancelenses.
`5,956,184 A *
`9/1999 Sato w..ceceecccseeeceeee 359/683
`6,097,430 A *
`8/2000 Komiya et al.
`.......... 348/218.1
`
`12
`
`16 Claims, 24 Drawing Sheets
`
`0
`SCANNER
`
`640
`ad
`
`
`
`DATA
`.
`PRESCAN a ‘|—crOTHER IMAGE]|°&_Tata<i
`
`[Eee me| ts]pon/
`|
`64 PROCESSING
`
`52
`SUBSECTION
`paescANNED Pa,
`18
`12
`DATA
`LOG
`PROCESSING
`(MAGE
`
`:
`74
`
`‘ Seon|Aces
`
`
`LENS TYPE|:
`CONDITION
`
`SETTING
`IDENTIFYING
`|
`if]
`SUBSECTION
`|
`SECTION 62°
`
`
`20
`
`
`
`
`
` A6
`
`56 ABERRATION|,[OTHER IMAGE
`
`PRINTER
`|:
`FINE
`“ISCAN
`
`CORRECTING||PROCESSING
`CONVERTING +!
`MEMORY| |
`SUBSECTION||
`PORTION_|[BLOCK
`
`“GOFINE SCANNED
`DATA PROCESSING
`SECTION
`
`50
`
`68IMAGE
`PROCESSING
`SUBSECTION
`
`)
`68A
`
`PETITIONERS EX1024
`Page 1
`
`PETITIONERS EX1024
`Page 1
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 1 of 24
`
`US 6,862,373 B2
`
` 34
`
`32
`
`fo
`Ay
`fT
`
`fie" 30
`
`i: p==[- an CONV
`
`(
`
`/D
`
`A
`
`ERTER |
`
`36
`
`14
`
`16
`
`L
`PROCESSING
`aK)
`
`IMAGE
`
`APPARATUS
`
`6
`
`DISPLAY
`
`20
`
`26
`
`
`
`
`
`RECORDING|
`MEDIUM
`DRIVE
`
`
`UNIT
`
`
`|:
`KEY.
`:
`(ole)
`|
`:
`| cohopo
`18 18a
`!
`18
`MANIPULATING UNIT
`
`
`
`
` 29
`IMAGE
`
`RECORDING
`
`MEDIUM
`
`
`PETITIONERS EX1024
`Page 2
`
`PETITIONERS EX1024
`Page 2
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 2 of 24
`
`US 6,862,373 B2
`
`FIG.2A
`
`
`
`FIG.2B
`
`
`
`34R==34G 34B
`
`PETITIONERS EX1024
`Page 3
`
`PETITIONERS EX1024
`Page 3
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 3 of 24
`
`OL
`
`US 6,862,373 B2
`
`ONLLOSIYNOD]|SNISSIOONOILD3SLslanvaveKDIin|[USLWBANOD
`
`
`
`O¢VivaNVOSsud
`
`91NOLLOaSEns||:poSNLLaS
`
`J90¥donioss|:|SNLLWINdINVIN
`
`
`AV1dS10ONILYSANODINISSAIONdAYOWANY,
`herZLgt!glaGaNNV2Saud
`
`ONISSIOOUdVLGv89ONISSIOOUd
`
` GANNVOS
`3NI409._ADVNI89
`
`NOLSNOLLOSSENS
`
`
`NOLLOASSNS19014
`
`N8sOLLOASENSanLas
`
`
`
`
`i43dALSN3T]i:[NOIasSans]}SNILVNIGUOOD}-1\eee‘[NOIOaSans]'
`
`
`ONIASILNAG][0p-----neneneepene“< NOLLIGNOD
`
`39|LSDVWIY3HLO
`
`
`€‘OI
`
`YSLNidd
`ONILYFANOD
`
`NOILO3S8Ns
`
`Viva
`
`AYOWAWN
`
`NVOS
`
`EINE
`
`INISSADOUd+9
`
`NOILOASAENS
`
`PETITIONERS EX1024
`Page 4
`
`PETITIONERS EX1024
`Page 4
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 4 of 24
`
`US 6,862,373 B2
`
`
`ADVWIYSHLO
`
`INIONGAY/ONIDNVINA
`
`ALVNIGYOOD
`
`SSa00ud
`
`Pp’Sia
`
`1201eelssap0ud
`
`ONIDNGAY/ONIDYVINAONILOFNYOD22ONINYOISNVYL
`
`(NOLLW10dYSLNI)SSI0OUdONINMOISNVELL
`
`N/41:SASSANAVa+V+44]
`
`-(NOILV10dY4S.LNI)SSI9OUd
`ONIONAIY/ONIDYWINA3LWNIGHOOD
`
`
`oNNaLanvavd|AiddNs:| ONIASLLNSGI; MALANVaWd|WUWHO:IdALSNF
`LuvdNOILYOdLuvd
`
`NoiLSasans|:—Nowoasansi|Nnouwoasans
`9oZLbae_—bl
`d¢Z)NOILVYuNsaVW2ZALVNIGdOOD
`
`
`ONINYOISNVYLL
`
`ONISSAIONd(}
`NOLLW10dY¥S.LNI)SSIOOUd
`ONINYOASNVYL
`
`
`
`N/5]::SaSSauav
`
`a+61=6
`
`
`
`
`
`N/dl:S3SS3YdV
`
`ag+9Vv+qI=qI
`
`SSADO8d
`
`
`
`ONIONGSY/ONIDYVINA
`
`JLVNIGYOOOD
`
`PETITIONERS EX1024
`Page 5
`
`PETITIONERS EX1024
`Page 5
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 5 of 24
`
`US 6,862,373 B2
`
`G°*SId
`
`
`
`19'19‘Ty
`
`06(8:9"u)sS
`
`c6
`
`
`
`SINILLAdWODJONVNINM
`
`aLlL°O+96S°0+HEO=A
`
`O=0=1
`
`XLW
`
`96
`
`
`
`Hg‘HIS‘HWY
`
`
`
`1419'TY
`
`(a°9"u)3S
`
`PETITIONERS EX1024
`Page 6
`
`PETITIONERS EX1024
`Page 6
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 6 of 24
`
`US 6,862,373 B2
`
`46
`
`bagogjo00000
`
`FIG.6A
`
`46
`
`46
`
`FIG.6B
`
`FIG.6C
`
`FIG.6D
`
`PETITIONERS EX1024
`Page 7
`
`PETITIONERS EX1024
`Page 7
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 7 of 24
`
`US 6,862,373 B2
`
`S1
`
`FIG./7
`
`FC
`
`PETITIONERS EX1024
`Page 8
`
`PETITIONERS EX1024
`Page 8
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 8 of 24
`
`US 6,862,373 B2
`
`LNIYdO.LOHdSe
`
`OLL
`
`WLINIQ~~_”
`
`8°Old
`
`NOILLOASNs
`
`
`
`QNISSSOOUd+9dns}:
`JVM!
`
`
`
`
`
`OINISSIOOUdVIVAINISSADOUd
`
`
`
`NOILOASNOILOASANS
`
`
`
`
`
`
`
`OGANNVOSSNIA09..V89ADVWI89
`
`YALNldd
`OINILYFANOD
`
`NOILOASaNs
`
`Vivd
`
`ONLLIFYYOO
`
`NOILYOd
`
`
`
`AYOWAWN
`
`NVOS
`
`INIS
`
`QVYVHD|SNLVUWddV¥
`
`ONISSIOOUdNOLLD3SAINAL|YSaLYIANOD
`
`
`
`AYOWIW|ONISSADOUdO2VivaNVOSAudViv
`
`J\AwidsiaQNILYFANOD
`NOLLDAS9NOILOasans||’voNSKAdNS
`
`OVNIONISSIDOUd=vLONLLWINdINV901
`rlViva;!St
`
`
`aapoervsnnececaecceceeeaeeat
`
`
`GaNNV
`
`
`osaudNOLLOASANsNOILOaSans|:
`
`
`
`
`
`
`
`
`[oipenseeeneneeeeefeseeeteetersfl,
`
`
`
`cl
`
`NOILOAS
`
`PETITIONERS EX1024
`Page 9
`
`PETITIONERS EX1024
`Page 9
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 9 of 24
`
`US 6,862,373 B2
`
`02
`
`YILNIdOLOHd
`
`WAIDIO
`
`O2l
`
`NOLLOaSans
`
`ONISSIDOUNd8SNOILOASENS
`
`Viva*.INISSSADOUd£9
`
`SNISSIOOUd|421921ONLINdINV901
`
`
`
`SALVUVddV‘|WALaWVuvdINAL|WaLYSANOD
`eaepoonerofeosesessens NOLLOASOLNowoasans}iios,|29ONILLISforsesenooseseonenesessesesneeed'
`
`
`
`
`
`eckNOLLOSAS
`
`AWdsIdONILYSANODeeeidviva
`
`ADVNIztSLgLBL
`
` GANNVOS
`
`
`ONISSADOUdVLVGONISSIOOUd
`WELWWY[beeenfeefo'INOLLDaYYOONOLLDaNNOD|NOLLSSEns][NOlLoasans]:bone
`
`
`
`PoensPoccnessenonenesesnePons!SNILVNIGUOOD
`
`
`
`GANNVOSTUdae_fuginbecinetn
`JANIS09-.v9JOVNI8Y
`!viva
`WaLAWvuvd
`
`NOLLOaSENS:NLLYIANODINILITHYOS:96
`
`
`MddNsONINYOISNVUL“~~NOLLIGNOD
`
`
`
`
`
`NOILDASNOILOASans
`
`UaLNIdd
`
`
`
`
`
`NOILUOd:
`
`cl
`
`
`
`PETITIONERS EX1024
`Page 10
`
`PETITIONERS EX1024
`Page 10
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 10 of 24
`
`US 6,862,373 B2
`
`ONILLYIANOD
`
`vivd
`
`YAHLO4O4
`
`JOVANI
`
`
`YaLNIdOLOHdrria-
`
`
`
`ONISSADONdNOLLDASONILLISNOLLSS8ns
`
` NOILOaSans]|ONILDAYYOD|:SNLVYVddvAlay!
`olOEFO
`YALNIdd'ONLINOSONISSIDOUdQNINNOINadNOILYOd
`
`
`OZNOLLYOd
`WIIG.———_NOLLOASENS6p|SaNNYS
`
`
`
`
`
`
`fon53sans]:IDYWNINOILD35AN3ID144300|||SNLLYNIGUOODbl—|onissao0udVivaNOHLOFHHOS|[LHL]!
`
`
`;|Awtasial.|:|NOtLoasansS3W3HOS
`
`CANNWOSANIg¢ADYWIOSL
`
`'TNouSAWSHOSNOILLYOd
`QANNVIS3ud|gel;dNLas]:
`:ONISSIDOUd
`Vel
`
`
`VIVdADVINOLLVuadOONILOANHOD
`VLWO
`9S1pee
`
`
`
`
`YSHLOYO4SJOVWIXLW/LAI
`OT°SIaONISSIOONd“JOVIél
`
`
` NOILOAS
`|_|YSLYSANOD
`
`NOILDaSONISS300ud
`
`NOLLOaSENS
`ONISSAI0Ud
`
`PETITIONERS EX1024
`Page 11
`
`907
`
`NOILIGNOOD
`
`\
`
`
`
`29ONILLASjOS
`
`
`
`NVOS3ud
`AYOWIAN
`INISSIDONd
`
`NOILO3S
`
`Vivd
`
`
`
`PETITIONERS EX1024
`Page 11
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 11 of 24
`
`US 6,862,373 B2
`
`pre nnn ee ee ee ee nee eee ees
`
` NEGATIVE FILM|—
`
`
`SCANNER
`READBAR |)
`CODE
`
`|
`
`YES
`
`‘[
`‘|
`
`!
`
`NO |
`
`RECOGNIZE:
`NG
`;
`
`‘
`
`:
`
`CORRECTION
`COEFFICIENT
`
`RECOGNIZE OR SETTING SECTION
`Po
`"1388[CAPTURE LENS TYPE INFORMATION] C
`138A
`{cn
`a ban nn ben nnneen nnn en ne fe nn pe eee eee eee

`
`'} CORRECTION COEF.
`
`CORRECTION PARAMETER
`COMPUTING
`‘FICIENT STORTING
`
`
`i!
`LOAD
`CORRECTION 1 SUBSECTION
`SUBSECTION
`
`COEFFICIENT f=
`CORRECTION
` PRINT SIZE AND
`COEFFICIENT
`
`
`
`OTHER PARAMETERS
`
`DETECT IMAGE
`
`|
`
`
`
`
`
`‘bag
`wee ee eee ene eee de
`a
`,
`rt
`t
`Cc
`
`
`: |[COMPUTE.1|
`OMPUTE|i p---zzszeeeefeeeeeeseo ‘
`
`'! CENTER |)!|AMOUNT OF
`FINAL '!|CORRECTION
`
`
`:! POSITION 1:1:]PIXEL LOSS
`COEFFICIENT|
`‘|
`COEFFICIENT
`'! COMPUTING|{i
`
`
`oe ‘ CORRECTION COEF-+|'j
`“FICIENT SUPPLY
`|:SUBSECTION|
`
`
`
`
`
`moa oF3%Po
`
`=>uU0
`
`54
`
`
`
` 20 :
`
`
`
`
`
`INFORMATION
`ABOUT CENTER
`
`
`POSITION
`
`
`138C
`
`a@ K TIMES
`
`K TIMES
`
`LENS ABERRATION
`CORRECTING PART
`THROUGH
`
`
`
`CORRECT CHROMATIC ABERRATION
`OF MA
`GNIFICATION
`

`THROUGH
`
`
`
`!.b2oss02250
`ENLARGING/
`REDUCING PART
`
`
`
`CALCULATE FINE
`
`|}
`
`
`‘| ADJUSTMENT
`
`| COEFFICIENT
`
`
`OTHER IMAGE PROCESSING
`MONITOR
`
`
`|:
`DISPLAYON
`
`DATA CONVERTING[~66(70)
`MONITOR
`
`PERFORM ELECTRONIC SCALING
`
`||
`
`148(156)
`
`INDEX PRINT
`
`
`
`PRINT
`
`16
`
`PETITIONERS EX1024
`Page 12
`
`PETITIONERS EX1024
`Page 12
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 12 of 24
`
`US 6,862,373 B2
`
`FIG.12
`
`CORRECT
`CHROMATIC ABERRATION OF
`MAGNIFICATION AND
`DISTORTION
`<MAIN SCANNING DIRECTION>
`
`<AUXILIARY SCANNING DIRECTION>
`
`PERFORM
`ELECTRONIC SCALING
`<MAIN SCANNING DIRECTION>
`
`CORRECT
`CHROMATIC ABERRATION OF
`MAGNIFICATION AND DISTORTION
`
`
`
`CORRECT
`
`COLOR DIVERGENCE IN SCANNER
`
`
`<AUXILIARY SCANNING DIRECTION>
`
`
`
`
`PERFORM
`
`ELECTRONIC MAGNIFICATION
`
`
`<AUXILIARY SCANNING DIRECTION>
`
`
`
`PETITIONERS EX1024
`Page 13
`
`PETITIONERS EX1024
`Page 13
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 13 of 24
`
`US 6,862,373 B2
`
`
`
`((°A'OX)AG+0A'((0AOX)AQ+240K)xQ-F0X)Cy
`
`
`
`
`
`
`
`(NOLLOAUICONINNVDSAYVIIXN)x
`
`
`
`AGHAOX)+OX)CY (COAX)A404‘(94OK)xQ+0X)LY]((°A‘X)AG+24'((oAOX
`
`
`
`
`
`
`
`
`
`
`((°AX)AG+24‘(9,40X)xQ+0X)Ly
`
`
`
`GOHLAWONISS3D0UdJOVI
`
`(OAOX)OV:LUVYOMd4O
`
`
`
`((°A‘xAt,ox)2
`
`
`(0,'0X)OV:NOLLNSANILNIS3Yd4O
`
`
`GOHLAWONISSIIONdJOVI
`
`
`
`
`
`(NOLLDJYIGONINNVOSNIVIDA
`
`el°Ola
`
`PETITIONERS EX1024
`Page 14
`
`PETITIONERS EX1024
`Page 14
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 14 of 24
`
`US 6,862,373 B2
`
`O22812Pile
`
`PI°OLld
`
`
`
` SNWALSASAYOLVEOREVTWISIOL2
`
`9L2
`
`
`
`QNIGUOD3Y)
`
`GSNNVO2S)
`
`
`
`YOSSADONYSLNIddONISSSDOUdann
`LINNYANNVOS
`
`Yyasv71FOV!do9
`
`LINNLIND
`
`
`
`(VLVdJ9VNI
`
`
`
`(VLVdJ9OVNI
`
`
`
`(314JOVWI)(VLVGS9VWI314)
`
`
`
`
`
`
` S3NIHOWW
`
`
`
` ONISSAD0¥d|' NOILVWYOANIYSHLO1YOVIGAWONIGYODSY;
`
`PETITIONERS EX1024
`Page 15
`
`PETITIONERS EX1024
`Page 15
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 15 of 24
`
`
`
`US 6,862,373 B2
`
`PETITIONERS EX1024
`Page 16
`
`PETITIONERS EX1024
`Page 16
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 16 of 24
`
`US 6,862,373 B2
`
`
`ONISSIOOUdJOVI|q99ANN
`
`
` NOISN3LX3 YaAING:||C|CuWOBAaDIwoud|||sng|jbleeZOLE|gZE:89eirIiSIO|lAyowan||Add|[|AWOWSNJeve|Z9E”OIE|||Pauowan[|
`fuassao0ud!LINNMSLNRd|_|veei||YaSVI]"|ocg|avi|YATIOULNOD|||o/||NOLLYOd
`
`JOVI||812:!iFoOEE
`
`||WNOSuWdC4:0|||a||:|——-OSEJWOU||WY|ti
`
`iae|||ponteNOILDASa|gQce|}OUVOSAaYTOULNOS||
`atelojnee|bSEn86,OPESEEE|SS|;|AYOWSW
`||\u2ze|b1Z
`91274
`Y3SS3IOOUd
`fya‘||PPEanionaanias-oinyfj|4
`c-a=---=ae
`
`
`
`
`rwow|[wve][nao]
`
`
`ASNOWO/|yaldyvo
`99¢a4wisBEe
`
`YOLIATASa¢’e
`
`9T°OI
`
`LINN
`
`YANNWOS
`
`
`
`
`
`
`
`PETITIONERS EX1024
`Page 17
`
`PETITIONERS EX1024
`Page 17
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 17 of 24
`
`US 6,862,373 B2
`
`
`
`LT°Ord
`
`
`
`lNOLLYOdONISSIDOYdJDVWITWNOISNAWIG-JNO
`
`
`
`
`
`
`(4aTNOYLNOD
`OALOL): YOSSADOUd
`
`ADVI
`
`JAILO3454
`VNOISNSWIC-3NO
`
`MYNOISNAWIG-3NO
`
`NOILVOIZINDVN
`
`NOILWYuNagEV
`
`NILOIUNOO
`
`DILWANOUHD
`
`LNNOWV
`
`
`
`ONLLVINDWd
`
`(4)Luvd
`
`“IWNOISNSWIG-3NO
`
`NOLLVOLINSVA
`
`NOLLVEYSEV
`
`NILDINNOD
`
`OLLWWOYHD
`
`LNNOWY
`
`ONILWAINDAWO
`
`(a)Luvd
`
`Zve
`
`AYOWSIN
`
`awed
`
`
`
`gee~_|YaTIOULNODANOWSWN
`
`
`
`|||||||||
`
`YOL9373S
`
`WOud
`
`OVNI
`
`
`_NOILLIGNOO
`
`“TeQNISSIIOUd
`
`
`
`(ANIDNIdA.L3s
`
`-O.LNVYWOYS)
`
`PETITIONERS EX1024
`Page 18
`
`PETITIONERS EX1024
`Page 18
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 18 of 24
`
`US 6,862,373 B2
`
`FIG.18A
`
`COORDINATE SYSTEM
`
`”
`
`DIRECTION OF
`f ~~ TRANSPORT OF
`PHOTOGRAPHIC FILM
`
`ORIGINE (0,0)
`
`OOOOUOUd
`LO
`x
`
`
`
`CENTER OF IMAGE
`
`apes
`
` FILM IMAGE
`
`OOOOUOOOO
`\ PHOTOGRAPHIC
`
`FILM
`
`FIG.18B
`
`Xp Yo COORDINATE SYSTEM
`
`DIRECTION OF
`TRANSPORT OF
`PHOTOGRAPHIC FILM
`
`OOOOROOOO
`
` ORIGINE (0,Q)
`
`
`(=CENTER
`OF IMAGE)
` Yp
`
`OOOUQRUVRUUOE
`
`FILM IMAGE
`
`PHOTOGRAPHIC
`FILM
`
`PETITIONERS EX1024
`Page 19
`
`PETITIONERS EX1024
`Page 19
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 19 of 24
`
`US 6,862,373 B2
`
`FIG.19
`
`ELECTRONIC SCALING
`RATIO COMPUTING ROUTINE
`
`400
`
`N
`
`iS THERE
`LE IMAGE DATA?
`
`402
`
`Y
`
`POSITION OF IMAGE
`
`IDENTIFY AND STORE
`LENS TYPE USED TO TAKING IMAGE
`
`406
`
`CAPTURE DISTORTION
`CORRECTING DATA CORRESPONDING
`TO IDENTIFIED LENS TYPE
`
`CALCULATE AMOUNT OF
`PIXEL LOSS A x IN x-DIRECTION
`AND AMOUNTOF PIXEL LOSS
`Avy IN y -DIRECTION BASED ON
`
`DISTORTION CORRECTING DATA
`
`408
`
`410
`
`412
`
`CALCULATE PERCENTAGES OF VIGNETTING
`N x-AND y-DIRECTIONS
`2*Ax/XX).
`(2: Ay/YY
`
`414
`
`416
`
`( y -DIRECTION IS GREATER THAN
`
`THAT IN x -DIRECTION)
`
`PERCENTAGES
`
`OF VIGNETTING I" x -AND
`
`( x-DIRECTION IS
`
`GREATER y-DIRECTION)
`CALCULATE FINE ADJUSTMENT
`CALCULATE FINE ADJUSTMENT
`COEFFICIENT OF
`COEFFICIENT OF
`ELECTRONIC SCALING A m
`
`ELECTRONIC SCALING A m
`Am=YY/(YY—2*A
`
`
`
`418
`
`CORRECT ELECTRONIC
`SCALING RATIO
`mx '=mx-:Am
`my
`'=my:-Am
`
`CALCULATE CORRECTED
`AMOUNTSOF PIXEL LOSS
`Ax', Ay'
`
`420
`
`422
`
`CALCULATE PROCE“SING CONDITION
`OF OTHER IMAGE PROCESSING
`
`424
`
`PETITIONERS EX1024
`Page 20
`
`PETITIONERS EX1024
`Page 20
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 20 of 24
`
`US 6,862,373 B2
`
`FIG.20
`
`ELECTRONIC SCALING
`RETIO COMPUTING ROUTINE
`
`430
`
`434
`
`
`
`
`IS APS FILM
`INSERTRD INTO FILM
`
`CARRIER?
`
`
`
`Y
`
`
`CAPTURE PRINT SIZE
`ax ANDay
`
`
`432
`
`
`
`
`
`
`CALCULATE ELECTRONIC
`SCALING RATIO
`
`
`mx, my
`
`INPUT PRINT SIZE a x AND a y
`
`DOES INPUT END?
`
`438
`
`RETURN
`
`FIG.23A
`x -DIRECTION
`-
`
`es
`
`310°
`/
`_
`
`y -DIRECTION
`
`
`
`PERCENTAGE OF REDUCTION
`: x DIRECTION > y DIRECTION
`
`ORIGINAL IMAGE
`
`OUTPUT IMAGE
`
`x BA
`
`—__>
`
`y -DIRECTION {
`
`FIG.23B
`
`310.
`
`4
`
`.
`
`310
`
`reese ence
`
`PERCENTAGE OF ENLARGEMENT
`- y DIRECTION > x DIRECTION
`
`
`
`ORIGINAL IMAGE
`
`[aT
`OUTPUT IMAGE
`
`PETITIONERS EX1024
`Page 21
`
`PETITIONERS EX1024
`Page 21
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 21 of 24
`
`US 6,862,373 B2
`
`FIG.21A
`
`ORIGINAL IMAGE
`
`
`
`FIG.21B
`
`IMAGE THAT HAS BEEN
`CORRECTED FOR LF ABERRATIONS
`AND SUBJECTED TO ELECTRONIC
`SCALING IN y -DIRECTION
`
`YY (=Y-my)
`SCALING x -DIRECTION
`
`FIG.21C
`
`IMAGE THAT HAS BEEN
`CORRECTED FOR LF ABERRATIONS
`AND SUBJECTED TO ELECTRONIC
`
`PETITIONERS EX1024
`Page 22
`
`PETITIONERS EX1024
`Page 22
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 22 of 24
`
`US 6,862,373 B2
`
`FIG.22A
`
`FIG.22B
`
`ORIGINAL IMAGE
`
`IMAGE SUBJECTED TO LF ABERRATION
`CORRECTION AND ELECTRONIC
`SCALING IN y -DIRECTION
`
`
`
`f
`
`\
`
`PL Ay! LnyYY
`
`To yy!
`Xx
`(=Y -my')
`
`FIG.22C
`
`FIG.22D
`
`IMAGE SUBJECTED TO LF ABERRATION
`CORRECTION AND ELECTRONIC
`SCALING IN x -DIRECTION
`
`IMAGE AFTER REMOVAL
`OF PIXEL LOSSES
`IN y -DIRECTION
`
`YY
`
`
`
`YY
`
`
`
`(=X-mx')
`
`FIG.22E
`
`OUTPUT IMAGE
`(IMAGE AFTER REMOVAL OF
`PIXEL LOSSES IN x -DIRECTION)
`
`YY
`
`~—_
`
`PETITIONERS EX1024
`Page 23
`
`PETITIONERS EX1024
`Page 23
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 23 of 24
`
`US 6,862,373 B2
`
`(YITIOULNOD|
`
`O/|OL)
`
`|||
`
`YOSSdD0Nd
`
`Ove
`
`|||
`=
`
`JAIVNWI
`
`
`INNOWY|ONILIIYUOD8S2|NOLLVuuaaV
`
`JILWWONHO]«=HOY|NOLLVI4INDVW||L
`||(u)Luvd
`
`
`
`NOLLYOdSNISSIIOUdJOVITWNOISNJWIG-3NO
`
`
`gee-~_]YSTIOULNOODANOWAN
`TWNOISN3WIG-3NO
`“IWNOISNAWNIG-ANO
`
`9S2
`
`Do°OLd
`
`eve
`
`AYOWAWN
`
`dWwvVds
`
`
`
`ONILWINDIW)
`
`NOILVIISINDVN
`ONILVINDWO
`
`VNOISNAWIG-JNO
`
`NOILYOLSIC
`
`ONILOAWYOD
`
`LNNOWV
`
`NOLLWudaav
`
`ONILOSNNOD
`
`DILVWOYHD
`
`LNNOWY
`
`
`
`ONILVANDO
`
`(a)Luwd
`
`Luvd
`
`YOL9373S
`
`WOU
`
`ONISSADOUd
`
`JDVWI
`
`“onyWOU)
`NOILIGNOD
`
`|
`
`
`
`(ANIDNJdALas
`
`PETITIONERS EX1024
`Page 24
`
`PETITIONERS EX1024
`Page 24
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Mar.1, 2005
`
`Sheet 24 of 24
`
`US 6,862,373 B2
`
`aeee
`
`FIG.25B
`
`G
`FIG.25A
`
`PETITIONERS EX1024
`Page 25
`
`PETITIONERS EX1024
`Page 25
`
`

`

`US 6,862,373 B2
`
`1
`IMAGE PROCESSING METHOD AND
`APPARATUS
`
`Thisis a division of application Ser. No. 09/301,538filed
`Apr. 29, 1999; now U.S. Pat. No. 6,603,885 the disclosure
`of which is incorporated herein by reference.
`BACKGROUND OF THE INVENTION
`
`2
`apparatus) that illuminates the film with reading light and
`captures the projected light to read the image on the film
`photoelectrically; an image processing apparatus that per-
`formsspecified image processing on the image data captured
`with the scanner or the image data supplied from a digital
`cameraor the like, thereby producing image data for image
`recording and exposing conditions; a printer (image record-
`ing apparatus) that scans and exposures a light-sensitive
`material to record a latent image on it, for example, by
`This invention relates to the field of digital image pro-
`scanning with optical beams in accordance with the image
`ecessing method and apparatus technologics. More
`data output from the image processing apparatus; and a
`particularly, the invention relates to image processing meth-
`processor (developing apparatus) that performs develop-
`ods and image processing apparatus which can applythis
`ment processing on the printer exposedlight-sensitive mate-
`image processing methods for typical use with digital pho-
`rial to produce a print reproducing the input image.
`toprinters that read film images photoelectrically to produce
`Users in general who intend to take ordinary pictures
`prints (photographs) reproducing the images and which are
`seldom use expensive, high-performance cameras such as a
`capable of achieving one of the following results: producing
`single-lens reflex camera but they normally usc so-called
`a high-quality image even if the input image is taken with
`“compact cameras” that are inexpensive and which are
`low-performance lenses as in films with lens, inexpensive
`capable of automatic exposing and focusing. Most recently,
`compact cameras and low-cost digital cameras; correcting
`there are a lot of users whoprefer using so-called “films with
`aberrations such as chromatic aberration of magnification
`lens” by the reason of easy handling.
`and distortion that develop in the images taken with those
`In cameras such as a single-lens reflex camera that needs
`low-performance lenses; producing a high-quality image
`a cost to some extent, high-precision lens elements are used,
`free from the image deterioration due to lens aberrations
`and a plurality of lens elements are combined to record
`even if the input image is taken with those low-performance
`images of very high quality.
`lenses; and particularly performing image processing on the
`In contrast, films with lens and inexpensive compact
`image data representing the image recorded on an image
`cameras cannotafford the use of costly lenses and only one
`recording medium.
`or two lens elements may be adopted. With such lens design,
`Heretofore, the images recorded on photographic films
`images of adequate quality cannot be taken and the image
`such as negatives and reversals (which are hereunder
`reproduced on prints does not necessarily have high quality.
`referred to simply as “films”) have been commonly printed
`If the image recorded on films is deteriorated in quality,
`on light-sensitive materials (photographic paper) by means
`there is a case that the quality of the output image on prints
`of direct (analog) exposure in which the film image is
`cannot be adequately improved by the aforementioned cor-
`optically projected onto the light-sensitive material
`to
`rections. Major causes of the deterioration of the image
`achieve its areal exposure.
`reproduced from films to be output on prints are lens
`A printer which relies upon digital exposure has recently
`aberrations such as “chromatic aberration of magnification”
`been commercialized as a digital photoprinter, in which the
`and “distortion” that originate from the low performance of
`image recorded ona film is read photoelectrically, converted
`the lenses mounted in the camera used to take the input
`to digital signals and subjected to various image processing
`image.
`operations to produce image data for recording purposes;
`Color images are formed of three primary colors, for
`recording light that has been modulated in accordance with
`example, red (R), green (G) and blue (BR). The refractive
`the image data is used to scan and exposealight-sensitive
`index (imaging magnification) ofa lens, evenif it is a single
`material to record a latent image, which is subsequently
`element, varies subtly with wavelength and differing refrac-
`developed to produce a finished print (photograph).
`tive indices occur with R, G and B lights. In other words,
`In the digital photoprinter, images are handled as digital
`even though the same position in a particular scene, a
`image data and the exposing conditions for printing can be
`focused position on a film are slipped off and differ among
`determined by image (data) processing. Hence, processing
`the R, G and B lights. This is the phenomenon generally
`operations suchas the correction of washed-out highlights or
`called “chromatic aberration of magnification” and the
`flat (dull) shadows due to the taking of pictures with rear
`image reproduced from the film has a definite color diver-
`gence.
`light or an electronic flash and sharpness enhancement
`(sometimes referred to simply as “sharpening”) can be
`to obtain a satisfactory and appropriately
`In order
`effectively performed to produce prints of the high quality
`recorded image, a plane of a scene of interest
`that
`is
`that has not been attainable by the conventional direct
`perpendicular to the optical axis must be focused on the
`exposure technique. In addition, the synthesizing of images
`sameplane as the imaging plane perpendicularto the optical
`and characters can be accomplished by image (data) pro-
`axis. In fact however, ordinary lenses have the imaging
`cessing and, as a result, prints can be output after editing
`plane displaced along the optical axis and the resulting
`and/or processing operations have been performedfreely in
`displacement of the focused position in the axial direction
`accordance with specific uses.
`causes a distortion of the focused object. As a natural
`the digital
`Aside from the images recorded on films,
`consequence, the reproduction of the image on the film is
`distorted.
`photoprinter can also output prints of images recorded with
`digital cameras or processed with computers. Other than
`being output as prints, images can be supplied to computers
`and so forth or stored in recording media such as floppy
`disks; hence, the image data can be put to various non-
`photographic uses.
`Having these features, the digital photoprinter is basically
`composed of the following units: a scanner (image reading
`
`Other causes of the image deterioration are the reduction
`of the brightness at the edge of image field which means a
`phenomenonin whichthe peripheral area of the image looks
`darker than the central area whichiscloserto the optical axis
`corresponding to the performanceof the lens used, and the
`point spread function (PSF) whichis attributable to differing
`focal positions in the plane of the film.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`PETITIONERS EX1024
`Page 26
`
`PETITIONERS EX1024
`Page 26
`
`

`

`US 6,862,373 B2
`
`3
`As noted above,if one uses a camera such as a single-lens
`reflex camera that needs a cost
`to some extent, high-
`precision lens elements may be used and a plurality of lens
`elements combined to correct various aberrations including
`chromatic aberration of magnification, distortion, deteriora-
`tion of marginal lumination and PSF and an appropriate
`image can be recorded on the film.
`However, cameras such as films with lens and compact
`cameras required to be a low cost can not use high-cost
`lenses and aberrations will develop in the images recorded
`on films. As a result, the images reproduced on prints will
`eventually have color divergence and distortion.
`To deal with this problem of image deterioration involv-
`ing the difficulty in improving the quality of output images
`onprints, techniques have been proposed in connection with
`image processing methods and apparatus that correct image
`aberrations in accordance with the characteristics of lens
`aberrations that are obtained via certain image acquisition
`means and two typical examples of such technology are
`disclosed in Unexamined Published Japanese Patent Appli-
`cation (kokai) Nos. 311425/1994 and 281613/1997,
`the
`latter being assigned to the present Applicant. According to
`these patents, the proposed technology can correct aberra-
`tions due to lenses and prevent the deterioration of image
`quality in the marginalarea, thereby ensuring the production
`of high-quality imagesat all times.
`Specifically, Unexamined Published Japanese Patent
`Application (kokai) No. 281613/1997 proposes a process of
`correcting the problem of deterioration of marginal lumina-
`tion in a photographic processing apparatus and method. In
`the process,the quantity oflight f(i,j) on an image in a given
`pixel position (i,j) is multiplied by a correction coefficient
`g(i,j) based on lens characteristics and the obtained product
`f' (i,j) is substituted as the corrected quantity oflight in the
`pixel position (i,j). To make the correction for the entire part
`of the image, j is first moved with i held constant and then
`i is moved, or alternatively,
`i is first moved with j held
`constant and then j is moved;in either way, the whole image
`can be corrected. When correcting distortion and chromatic
`aberration of magnification by this method, the position of
`the subject in an image ofinterest is changed so that a huge
`frame memoryis required to store the information about all
`pixel positions of not only before correction but also after
`the correction. In addition, a circuit is necessary that per-
`forms two processing schemesfor i and j as described above
`and this not only increases the cost of the apparatus but also
`causes a significant drop in the correction speed, thus there
`is a problem the practical use of the apparatus is difficult.
`Onthe other hand, unexamined Published Japanese Patent
`Application (kokai) No. 311425/1994 discloses an image
`correcting apparatus capable of rapid image correction in
`accordance with the characteristics of lens aberrations. In
`
`this apparatus, the subjects of the correction are the amount
`of defocusing, the decrease in the quantity of light, and the
`degree of unsharpness in the hue and chromasof a color
`image. According to the disclosure, a quantity of deteriora-
`tion in each of these correction subjects increases as it goes
`from the center of the imageto the peripheral area sothat the
`data of specified patterns that are increased as going from the
`center toward the peripheral in an imagearea that is formed
`by concentric circles or squares extending radially outward
`from the center of the imageare only used for each of given
`lens characteristics as correction enhancement coefficients
`
`for correcting these correction subjects. This approach is
`capable of rough correction but not image corrections aber-
`rant from the patterns that is prepared beforehand. Hence,
`there is a problem that it is impossible to perform appropri-
`ate corrections according to the characteristics of individual
`taking lenses.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`If one wants to accomplish the appropriate correction by
`this technology, correction enhancement coefficients must
`be made available for all patterns that are predicted for the
`given lens characteristics and to meet this need, a memory
`of sufficiently large capacity is required. Whatis more, if the
`available patterns are not simple concentric circles or
`squares whose center coincides with that of the image, an
`increased amount of data has to be calculated to correct
`matrices and the overall image processing speed is lowered.
`Thesepractical problems with cost and processing speed are
`particularly serious in the case of print services that involve
`the volume reproduction of images.
`As mentioned hereinabove, an image processing system is
`conventionally knownthat is able to perform various image
`processing schemes on the image data obtained by reading
`the film image recorded on photographic films or on the
`image data input from a digital camera or the like and which
`then outputs an image in various output modes such as by
`recording the image on recording materials such as photo-
`graphic paper or storing the image data in information
`recording media. Compared to a conventional photographic
`processing system that records the film image by areal
`exposure, the image processing system just described above
`can control the output image quality by image processing
`onto the image data, thereby output images of high quality
`are realized.
`
`Speaking of films with lens, the lens is usually composed
`of an inexpensive plastic lens that inevitably suffers from
`great amounts of aberrations such as distortion and chro-
`matic aberration of magnification. Hence, the film image
`recorded on a photographic film by exposure with the film
`with lens has a comparatively high level of geometric
`distortion according to the lens distortion (so-called
`“pincushion-type distortions) as typically shown in FIG.
`25A (FIGS. 25A and 25Billustrate how an image consisting
`of multiple lines in a grid pattern appearsif it is recorded on
`a photographic film by shooting with a film with lens); at the
`same time, a color divergence occurs at a comparatively
`high level due to chromatic aberration of magnification. To
`deal with this problem, distortion correction for correcting
`the geometric distortion of an image due to the distortion of
`the lens on the film with lens and magnification chromatic
`aberration correction for correcting the color divergence in
`an imagedueto the chromatic aberration of magnification of
`the same lens are being review in order to ensure that the
`image processing system described above can produce an
`output image of high quality from the original (input) image
`recorded with the film with lens.
`
`With a view to increasing the speed of the various
`processing schemes to be performed with the above-
`mentioned image processing system, the contents of image
`processing and the value of a parameter to be applied in a
`specific image processing schemehave beenset identical in
`specified directions, such as vertical and horizontal, of the
`image represented by the image data. For instance, if the
`image processing to be done is electronic scaling,
`the
`electronic scaling ratio is set identical in both the vertical
`and horizontal directions of the image represented by the
`image data to be processed.
`What is unique about image data is that its volume is
`tremendousand that it represents an image having a two-
`dimensional extent. Hence, image processing schemes such
`as ones for correcting distortion and chromatic aberration of
`magnification are so much complicated in contents that they
`not only take prolonged time but also require storage means
`of large capacity. As a result, the image processing section
`capable of performing image processing schemessuchas the
`PETITIONERS EX1024
`Page 27
`
`PETITIONERS EX1024
`Page 27
`
`

`

`US 6,862,373 B2
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`5
`correction of distortion and chromatic aberration of magni-
`fication is considerably complicated in configuration and,
`what
`is more,
`the processing performance of the image
`processing system is eventually deteriorated.
`Consider, for example, the correction of distortion. First,
`distortion correcting data representing the direction and the
`amountofthe shift in the position of each of the pixels in the
`original film image due to the distortion of a lens, as
`referencedto the inherentposition of each pixel (the position
`of its grid point) composingthe film image, are preliminarily
`measured and stored for each of the lens types used; given
`the image data to be processed,the distortion correcting data
`associated with the lens type used in actual shooting are
`captured; on the basis of the captured distortion correcting
`data, the positions of

This document is available on Docket Alarm but you must sign up to view it.


Or .

Accessing this document will incur an additional charge of $.

After purchase, you can access this document again without charge.

Accept $ Charge
throbber

Still Working On It

This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.

Give it another minute or two to complete, and then try the refresh button.

throbber

A few More Minutes ... Still Working

It can take up to 5 minutes for us to download a document if the court servers are running slowly.

Thank you for your continued patience.

This document could not be displayed.

We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.

You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.

Set your membership status to view this document.

With a Docket Alarm membership, you'll get a whole lot more, including:

  • Up-to-date information for this case.
  • Email alerts whenever there is an update.
  • Full text search for other cases.
  • Get email alerts whenever a new case matches your search.

Become a Member

One Moment Please

The filing “” is large (MB) and is being downloaded.

Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!

If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document

We are unable to display this document, it may be under a court ordered seal.

If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.


Access Government Site

We are redirecting you
to a mobile optimized page.





Document Unreadable or Corrupt

Refresh this Document
Go to the Docket

We are unable to display this document.

Refresh this Document
Go to the Docket