Synthetic Simulation and Modeling of
`Image Intensified CCDs (IICCD)
`
`Emmett J. IentilucciA.A.S. Monroe Community College (1989)B.S. Rochester Institute of Technology (1996)A thesis submitted in partial fulfillment of therequirements for the degree of Master of Sciencein the Center for Imaging Science in the College of ScienceRochester Institute of TechnologyMarch 31, 2000Signature of Author______________________________________________________________Emmett J. IentilucciAccepted by____________________________________________________________________Coordinator, M.S. Degree ProgramDate
`
`Magna 2027
`TRW v. Magna
`IPR2015-00436
`
`0001
`
`

`
`Synthetic Simulation and Modeling of Image Intensified
`CCD’s (IICCD)
`
`iiTHESIS RELEASE PERMISSIONROCHESTER INSTITUTE OF TECHNOLOGYCOLLEGE OF SCIENCE______________________________________________________________________________
`______________________________________________________________________________I, Emmett J. Ientilucci, hereby grant permission to the Wallace Memorial Library of RIT toreproduce my thesis in whole or in part. Any reproduction will not be for commercial use or profit.Signature of Author______________________________________________________________Emmett J. IentilucciDate__________________________________________________________________________
`
`0002
`
`

`
`iiiCHESTER F. CARLSONCENTER FOR IMAGING SCIENCECOLLEGE OF SCIENCEROCHESTER INSTITUTE OF TECHNOLOGYROCHESTER, NEW YORKCERTIFICATE OF APPROVAL___________________________________________________________________________________M.S. DEGREE THESIS___________________________________________________________________________________The M.S. Degree of Emmett J. Ientiluccihas been examined and approved by the thesis committeeas satisfactory for the thesis requirement for theMaster of Science degree in Imaging Science________________________________Dr. John R. Schott, Thesis Advisor________________________________Mr. Rolando V. Raqueño________________________________Dr. Jeff B. Pelz________________________________Date
`
`0003
`
`

`
`iv
`
`0004
`
`0004
`
`iv
`

`
`Abstract
`
`Image intensifying cameras have been found to be extremely useful in low-light-level (LLL)scenarios including military night-vision and civilian rescue operations. These sensors utilize theavailable visible region photons and an amplification process to produce high-contrast imagery.Today’s image intensifiers are usually attached to a CCD and incorporate a microchannel plate(MCP) for amplification purposes. These devices are commonly referred to as image intensifiedCCDs (IICCD).To date, there has not been much work in the area of still-frame, low-light-levelsimulations with radiometric accuracy in mind. Most work has been geared toward real-timesimulations where the emphasis is on situational awareness. This research proposes that a highfidelity simulation environment capable of producing radiometrically correct multi-band imageryfor low-light-level conditions can be an extremely useful tool for sensor design engineers and imageanalysts. The Digital Imaging and Remote Sensing (DIRS) laboratory’s Image Generation(DIRSIG) model has evolved to respond to such modeling requirements.The presented work demonstrates a low-light-level simulation environment (DIRSIG)which incorporates man-made secondary sources and exoatmospheric sources such as the moonand starlight. Similarly, a user-defined IICCD camera model has been developed that takes intoaccount parameters such as MTF and noise.
`
`0005
`
`v
`

`
`Acknowledgments
`
`I would personally like to acknowledge all those who helped in the development and completion ofthis research. First off, I would like to thank all the participants who helped in the ground truthcollection: Rolando Raqueño, Bryce Nordgren, my nephew Justin Ientilucci, and Mary EllenMiller. Thanks for staying on the roof with me till 3am. Also Andy Silano and Jeff Pelz for theuse of their secondary sources. Garrett Johnson for his help with the spectral radiometer andoutdoor spectral data collections. I would also like to thank all those at RIT’s physical plant forturning off the lights on one side of campus and letting my have access to their blueprints ofvarious buildings and surrounding pieces of real estate. Jason Gibson for the use of the radiometerand indoor data collections. My girlfriend Lori for supporting me and putting up with all thoseendless nights of not being able to go out. Finally, thanks to Scott Brown, Lee Sanders, and ErichHernandez-Baquero for all their helpful insights and problem solving skills.
`
`0006
`
`vi
`

`
`Dedication
`
`LOVE LIVES ONThose we love remain with us for love itself lives on,And cherished memories never fadeBecause a loved one’s gone…Those we love can never be more than a thought apartFor as long as there is memory, they’ll live on in the heart.This work is dedicated to my sister who I know would have been extremely proud to see me finishthis body of work.
`
`0007
`
`vii
`

`
`Table of Contents
`
`List of FiguresxiList of Tablesxvii1. Introduction11.1 Overview...........................................................................................................................11.2 Objectives (Statement of Work)..........................................................................................42. Background52.1 Intensifiers.........................................................................................................................52.1.1 Basic Image Intensifier Operation..............................................................................72.1.2 Image Intensifier Focusing Techniques......................................................................82.1.2.1 Proximity Focused.........................................................................................82.1.2.2 Electrostatically Focused.............................................................................112.1.2.3 Magnetically Focused..................................................................................122.1.3 Microchannel Plate Technology (MCP)...................................................................132.1.3.1 Ion Feedback...............................................................................................152.1.3.2 Stacking Microchannel Plates......................................................................172.1.4 Image Intensifier Types...........................................................................................182.1.4.1 Photon Counting Devices (MCP, no CCD)..................................................222.1.4.2 Image Intensified CCD (MCP and CCD).....................................................272.1.4.3 Solid State Detectors (CCD, no MCP).........................................................282.1.4.4 Photocathode Considerations.......................................................................302.2 Applications and State of the Technology.........................................................................332.3 Image Modeling...............................................................................................................342.3.1 Governing Radiance Equation, Energy Paths and the Big Equation..........................342.3.2 RIT’s Synthetic Image Generation Model (DIRSIG)................................................382.3.3 MODTRAN...........................................................................................................413. Experimental Approach423.1 Image Acquisition............................................................................................................42
`
`0008
`
`viii
`

`
`3.1.1 Scenes Imaged ...................................................................................................... ..43
`3.1.2 Use of the Ephemeris Utility ................................................................................. .. 44
`3.2 Image Intensified CCD Characterization ......................................................................... .. 45
`3.3 Image Modeling in DIRSIG ........................................................................................... .. 45
`3.3.1 HCCD Sensor Modeling ........................................................................................ ..46
`3.3.2 LLL Sensor Analysis ............................................................................................ ..47
`
`48
`. Ground Truth Collection
`4.1 Data Collection Scenario ................................................................................................ ..48
`4.2 Creation of the Truth Scene ............................................................................................ .. 51
`
`4.3 Equipment Used ............................................................................................................. .. 56
`4.3.1 Modifications to the HCCD Camera ...................................................................... .. 58
`4.4 Data Collected ............................................................................................................... .. 59
`
`4.5 Noise Averaging ............................................................................................................ .. 60
`
`63
`. DIRSIG Environment
`5.1 SIG Scene Creation ........................................................................................................ .. 63
`
`5.2 DIRSIG Input Files ........................................................................................................ .. 67
`5.2.1 AutoCAD and the GDB ........................................................................................ .. 70
`5.2.2 Materials and Emissivities ..................................................................................... .. 72
`5.2.3 Dview and Scene Node Files ................................................................................. .. 75
`5.2.4 Card Decks and Radiance Files ............................................................................. .. 76
`5.2.5 Weather Files ........................................................................................................ .. 77
`
`5.2.6 Sensor Responsivity Files ...................................................................................... .. 78
`5.2.7 Secondary Source Contribution ............................................................................. .. 78
`5.2.8 DIRSIG Batch Files .............................................................................................. .. 79
`
`5.3 Lunar and Sky/Starlight Contribution ............................................................................. .. 80
`
`ix3.1.1 Scenes Imaged........................................................................................................433.1.2 Use of the Ephemeris Utility...................................................................................443.2 Image Intensified CCD Characterization...........................................................................453.3 Image Modeling in DIRSIG.............................................................................................453.3.1 IICCD Sensor Modeling..........................................................................................463.3.2 LLL Sensor Analysis..............................................................................................474. Ground Truth Collection484.1 Data Collection Scenario..................................................................................................484.2 Creation of the Truth Scene..............................................................................................514.3 Equipment Used...............................................................................................................564.3.1 Modifications to the IICCD Camera........................................................................584.4 Data Collected.................................................................................................................594.5 Noise Averaging..............................................................................................................605. DIRSIG Environment635.1 SIG Scene Creation..........................................................................................................635.2 DIRSIG Input Files..........................................................................................................675.2.1 AutoCAD and the GDB..........................................................................................705.2.2 Materials and Emissivities.......................................................................................725.2.3 Dview and Scene Node Files...................................................................................755.2.4 Card Decks and Radiance Files...............................................................................765.2.5 Weather Files..........................................................................................................775.2.6 Sensor Responsivity Files........................................................................................785.2.7 Secondary Source Contribution...............................................................................785.2.8 DIRSIG Batch Files................................................................................................795.3 Lunar and Sky/Starlight Contribution...............................................................................806. IICCD Modeling866.1 System Modeling: A Chain Approach...............................................................................866.2 The II Module and CCD Imager.......................................................................................886.2.1 Quantum Efficiency (QE) and Spectral Matching....................................................906.2.2 MCP and CCD Imager...........................................................................................916.2.3 Blooming and Auto Gain Control (AGC).................................................................926.2.4 Propagation of a single photon................................................................................946.3 Component MTFs............................................................................................................956.3.1 Optics MTF............................................................................................................966.3.2 II Tube MTF..........................................................................................................976.3.3 Fiber Optic MTF....................................................................................................996.3.4 CCD MTF............................................................................................................1006.4 IICCD System Noise......................................................................................................1026.4.1 Shot Noise............................................................................................................1046.4.2 Pre-Amplifier Noise..............................................................................................1077. IICCD Characterization and Implementation1097.1 Measured MTF..............................................................................................................1097.1.1 Lab set-up and results...........................................................................................110
`
`86
`. IICCD Modeling
`6.1 System Modeling: A Chain Approach ............................................................................. .. 86
`6.2 The H Module and CCD Imager ..................................................................................... .. 88
`6.2.1 Quantum Efficiency (QE) and Spectral Matching .................................................. .. 90
`6.2.2 MCP and CCD Imager ......................................................................................... .. 91
`6.2.3 Blooming and Auto Gain Control (AGC) ............................................................... .. 92
`6.2.4 Propagation of a single photon .............................................................................. .. 94
`6.3 Component MTFs .......................................................................................................... .. 95
`6.3.1 Optics MTF .......................................................................................................... ..96
`6.3.2 II Tube MTF ........................................................................................................ .. 97
`
`6.3.3 Fiber Optic MTF .................................................................................................. .. 99
`6.3.4 CCD MTF .......................................................................................................... .. 100
`
`6.4 HCCD System Noise.................................................................................................... .. 102
`6.4.1 Shot Noise .......................................................................................................... .. 104
`
`6.4.2 Pre—Amplifier Noise ............................................................................................ .. 107
`
`109
`. IICCD Characterization and Implementation
`7.1 Measured MTF ............................................................................................................ .. 109
`
`7.1.1 Lab set—up and results ......................................................................................... .. 110
`
`0009
`
`0009
`
`

`
`7.2 Radiance Calibration Results ....................................................................................... .. 111
`
`7.3 Calibration Summary ................................................................................................... .. 113
`7.4 Implementation of the Sensor Model ............................................................................. .. 114
`7.4.1 Filters in one dimension....................................................................................... .. 114
`7.4.2 Filters in Two—Dimensions .................................................................................. .. 118
`
`7.5 Digitizer ...................................................................................................................... .. 122
`
`8. Radiance Predictions
`
`125
`
`8.1 Illumination from a Secondary Source .......................................................................... .. 125
`8.1.1 “Pick a Target in SIG Image and Compare Radiance Values” .............................. .. 126
`8.1.2 Limiting Apertures .............................................................................................. .. 132
`8.2 Sources of Lunar Illumination ...................................................................................... .. 133
`
`8.3 Analysis of Illumination from Lunar Sources ................................................................ .. 135
`8.4 Spectral Output of make_adb ....................................................................................... .. 139
`8.5 Radiance Prediction Summary ...................................................................................... .. 142
`
`164
`10. Conclusions and Recommendations
`10.1 SIG Enviromnent Conclusions and Recommendations ................................................. .. 164
`10.2 Calibration Conclusions and Recommendations .......................................................... .. 169
`
`10.3 Camera Modeling Conclusions and Recommendations ................................................ .. 170
`10.4 Recommendation Summary and Future Efforts ........................................................... .. 171
`10.5 Addendum: Preliminary Work on Image Fusion .......................................................... .. 172
`
`174
`11. Appendix A
`11.1 HCCD Specifications ................................................................................................. .. 174
`
`176
`12. Appendix B
`12.1 Details of ICCD modifications ................................................................................... .. 176
`
`143
`9. SIG Image Results
`9.1 SIG Images with Background and Extraterrestrial Sources ........................................... .. 143
`9.2 SIG Imagery with Secondary Sources ........................................................................... .. 148
`9.2.1 Tungsten—Halogen Source Off—Axis ..................................................................... .. 155
`9.2.2 Street Lamp Scenario .......................................................................................... .. 157
`9.3 Additional LLL SIG Imagery Examples ....................................................................... .. 160
`9.3.1 CCD Streaking ................................................................................................... .. 161
`9.4 Aerial Resolution ......................................................................................................... .. 162
`
`x7.2 Radiance Calibration Results.........................................................................................1117.3 Calibration Summary.....................................................................................................1137.4 Implementation of the Sensor Model...............................................................................1147.4.1 Filters in one dimension.........................................................................................1147.4.2 Filters in Two-Dimensions....................................................................................1187.5 Digitizer........................................................................................................................1228. Radiance Predictions1258.1 Illumination from a Secondary Source............................................................................1258.1.1 “Pick a Target in SIG Image and Compare Radiance Values”................................1268.1.2 Limiting Apertures................................................................................................1328.2 Sources of Lunar Illumination........................................................................................1338.3 Analysis of Illumination from Lunar Sources..................................................................1358.4 Spectral Output of make_adb.........................................................................................1398.5 Radiance Prediction Summary........................................................................................1429. SIG Image Results1439.1 SIG Images with Background and Extraterrestrial Sources.............................................1439.2 SIG Imagery with Secondary Sources.............................................................................1489.2.1 Tungsten-Halogen Source Off-Axis.......................................................................1559.2.2 Street Lamp Scenario............................................................................................1579.3 Additional LLL SIG Imagery Examples.........................................................................1609.3.1 CCD Streaking.....................................................................................................1619.4 Aerial Resolution...........................................................................................................16210. Conclusions and Recommendations16410.1 SIG Environment Conclusions and Recommendations...................................................16410.2 Calibration Conclusions and Recommendations............................................................16910.3 Camera Modeling Conclusions and Recommendations..................................................17010.4 Recommendation Summary and Future Efforts.............................................................17110.5 Addendum: Preliminary Work on Image Fusion............................................................17211. Appendix A17411.1 IICCD Specifications...................................................................................................17412. Appendix B17612.1 Details of ICCD modifications.....................................................................................17613. Appendix C17813.1 Day 1 Collection (Aug 19th - 20th).................................................................................17813.2 Day 2 Collection (Sept. 1st - 2nd)...................................................................................18314. Appendix D18714.1 AutoCAD and the GDB...............................................................................................18714.2 Material files...............................................................................................................18914.3 Emissivity Curves........................................................................................................19414.4 CONTROL7 Output for Creation of Card Deck...........................................................202
`
`178
`13. Appendix C
`13.1 Day 1 Collection (Aug 19”‘ — 20”‘) ............................................................................... ..178
`13.2 Day 2 Collection (Sept. 1“ — 2”“) ................................................................................. .. 183
`
`187
`14. Appendix D
`14.1 AutoCAD and the GDB ............................................................................................. .. 187
`14.2 Material files ............................................................................................................. .. 189
`
`14.3 Emissivity Curves ...................................................................................................... .. 194
`14.4 CONTROL7 Output for Creation of Card Deck ......................................................... .. 202
`
`0010
`
`0010
`
`

`
`xi
`
`14.5 Weather file ............................................................................................................... .. 205
`
`14.6 Sensor spectral responsivity files ................................................................................ .. 206
`14.7 Derivation of spectral source distributions .................................................................. .. 208
`14.8 DIRSIG Batch Files ................................................................................................... .. 213
`
`214
`15. Appendix E
`15.1 Spectral Radiometer ................................................................................................... .. 214
`
`215
`16. Appendix F
`16.1 Modified Ephemeris Code .......................................................................................... .. 215
`
`218
`17. Appendix G
`17.1 Starlight spectral conversions ..................................................................................... .. 218
`
`222
`18. Appendix H
`18.1 Electronic MTF ......................................................................................................... .. 222
`
`18.2 Digitizer MTF ........................................................................................................... .. 225
`18.3 Manufacture MTF ..................................................................................................... .. 227
`18.4 HCCD Radiometric Calibration .................................................................................. .. 228
`
`xi14.5 Weather file.................................................................................................................20514.6 Sensor spectral responsivity files..................................................................................20614.7 Derivation of spectral source distributions....................................................................20814.8 DIRSIG Batch Files.....................................................................................................21315. Appendix E21415.1 Spectral Radiometer.....................................................................................................21416. Appendix F21516.1 Modified Ephemeris Code............................................................................................21517. Appendix G21817.1 Starlight spectral conversions.......................................................................................21818. Appendix H22218.1 Electronic MTF...........................................................................................................22218.2 Digitizer MTF.............................................................................................................22518.3 Manufacture MTF.......................................................................................................22718.4 IICCD Radiometric Calibration....................................................................................22819. Appendix I23319.1 Inverse Square Law Predictions (no aperture)...............................................................23319.1.1 Bulb Efficiency...................................................................................................23419.1.2 Use Planck Equation to Reproduce PR-650 Data.................................................23519.1.3 Include and Sensor Response and Correct Bandpass............................................23719.1.4 Compute as Intensity Instead of Exitance.............................................................23919.2 Radiance Including Aperture........................................................................................24019.2.1 Recreate Aperture Data From PR-650.................................................................24319.2.2 Include Sensor Response and Bandpass to Correct DIRSIGs Output....................24519.2.3 Compute as Intensity Instead of Exitance, again…...............................................24619.3 Secondary Source Radiance Prediction Summary.........................................................24720. Appendix J24920.1 ICCD Simulator...........................................................................................................24921. Appendix K25521.1 Card deck used for MODTRAN run to generate lunar scattering...................................25522. Appendix L25822.1 CCD Streaking............................................................................................................25823. Appendix M26123.1 VIS/IR Fusion of DIRSIG Simulated Imagery..............................................................26123.2 Using SIG for Fusion Algorithm Development..............................................................26324. References26424.1 References...................................................................................................................264
`
`233
`19. Appendix I
`19.1 Ir1verse Square Law Predictions (no aperture) ............................................................. .. 233
`19.1.1 Bulb Efficiency ................................................................................................. .. 234
`19.1.2 Use Planck Equation to Reproduce PR—650 Data ............................................... .. 235
`19.1.3 Ir1clude and Sensor Response and Correct Bandpass .......................................... .. 237
`19.1.4 Compute as Intensity Ir1stead of Exitance........................................................... .. 239
`19.2 Radiance Ir1cluding Aperture ...................................................................................... .. 240
`19.2.1 Recreate Aperture Data From PR—650 ............................................................... .. 243
`19.2.2 Ir1clude Sensor Response and Bandpass to Correct DlRSIGs Output .................. .. 245
`19.2.3 Compute as Intensity Ir1stead of Exitance, again ................................................ .. 246
`19.3 Secondary Source Radiance Prediction Summary ....................................................... .. 247
`
`249
`20. Appendix J
`20.1 ICCD Simulator......................................................................................................... .. 249
`
`255
`21. Appendix K
`21.1 Card deck used for MODTRAN run to generate lunar scattering ................................. .. 255
`
`258
`22. Appendix L
`22.1 CCD Streaking .......................................................................................................... .. 258
`
`261
`23. Appendix M
`23.1 VIS/IR Fusion of DIRSIG Simulated Imagery ............................................................ .. 261
`23.2 Using SIG for Fusion Algorithm Development ............................................................ .. 263
`
`264
`24. References
`24.1 References .................................................................................................