`
`
`
`
`
`
`
`United States Patent
`(12)
`US 8,462,209 B2
`(10) Patent No.:
`
`
`
`
`
`
`
`
`(45) Date of Patent:
`Jun. 11, 2013
`Sun
`
`
`
`US008462209B2
`
`
`
`
`
`
`
`
`(54) DUAL-SWATH IMAGING SYSTEM
`.
`.
`.
`
`
`
`
`
`
`Inventor. Xluhong Sun, Windham, NH (US)
`(75)
`
`
`
`
`
`
`
`MD
`S
`C
`H
`73 A .
`K
`t'
`
`
`
`
`
`
`(
`551gnee.
`eyw orpora ion,
`anover,
`(U )
`)
`
`
`
`
`
`
`
`
`Subject to any disclaimer, the term of this
`( * ) Notice:
`
`
`
`
`patent is extended or adjusted under 35
`
`
`
`
`U.S.C. 154(b) by 851 days.
`
`
`
`
`
`(21) Appl. No.: 12/492,458
`.
`Flled:
`
`(22)
`
`
`
`
`
`
`
`Jun. 26, 2009
`
`
`
`
`
`
`JP
`JP
`
`
`
`6,735,348 B2 *
`6,778,211 B1
`
`
`6,798,984 B2 *
`
`
`
`
`7,009,638 B2
`
`
`7,075,735 B2
`
`
`
`
`
`
`
`
`
`..................... 382/293
`5/2004 Dial et a1.
`8/2004 Zimmermann et a1.
`
`
`
`9/2004 Antikidis ........................ 396/13
`
`
`
`
`
`
`3/2006 Gruber et a1.
`
`
`7/2006 Nozawa et a1.
`
`(Continued)
`
`
`FOREIGN PATENT DOCUMENTS
`
`
`
`8313825
`11/1996
`
`
`
`
`6/1998
`10155104
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`Savopol, F., Chapman, M., Boulianne, M., A Digital Multi CCD
`
`
`
`
`
`
`
`Camera System for Near Real-Time Mapping; 2000; International
`
`
`
`
`
`
`
`Archives of Potogrammetry and Remote Sensing; v01. XXXIII Part
`
`
`B1 pp. 266-271.*
`
`
`
`
`
`
`
`
`
`
`
`
`(Continued)
`
`
`
`
`Primary Examiner 7 Mehrdad Dastouri
`
`
`
`
`Assistant Examiner 7 Jeremaiah C Hallenbeck-Huber
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm 7 Morrison & Foerster LLP
`
`
`
`ABSTRACT
`(57)
`.
`.
`.
`.
`
`
`
`
`
`
`
`.
`.
`.
`.
`.
`.
`.
`Aportable, aerial, dual-swathphotogrammetric imaging sys-
`
`
`
`
`
`
`
`
`
`tem comprismg twm nadir pomtmg CCD cameras for Simul-
`
`
`
`
`
`
`
`
`taneously acquiring twin adjacent digital images for merging
`
`
`
`
`
`
`into a large panorama. A pair of lens shifters symmetrically
`
`
`
`
`
`
`
`
`
`
`shift twin images to the left and right sides of the focal points
`
`
`
`
`
`
`
`
`
`oftwo parallel lenses to extend imaging swathwidth. The twin
`
`
`
`
`
`
`
`
`adjacent images of the imaging system have a strip of narrow
`
`
`
`
`
`
`
`
`
`overlap at the center of the whole scene that are reserved for
`
`
`
`
`
`
`
`photogrammetric processing and stitching twin images into a
`
`
`
`
`
`
`seamless panorama. Each camera is connected to an embed-
`
`
`
`
`
`
`
`ded computer which controls imaging data acquisition,
`
`
`
`
`
`
`attaches GPS/IMU measurements, generates KML metadata
`
`
`
`
`
`
`
`
`
`files for its snapshots, and stores acquired images and meta-
`
`
`
`
`
`
`
`data into removable SSDs. Direct geo-referenced panoramic
`
`
`
`
`
`
`
`digital stills are immediately registered on to GoogleTM Earth
`
`
`
`
`
`
`
`
`precisely. Its images can be further processed for advanced
`
`
`
`
`
`
`mapping, change detection and GIS applications.
`
`
`
`
`16 Claims, 6 Drawing Sheets
`
`
`
`
`
`
`
`SECOND DAS/CPU h
`
`
`
`
`
`
`
`(65)
`
`(51)
`
`
`
`
`
`(56)
`
`
`
`
`Prior Publication Data
`
`
`
`
`US 2010/0328499 A1
`Dec. 30, 2010
`
`
`Int. Cl'
`
`H041V ”18
`
`
`
`(52) U'S' Cl'
`
`
`
`USPC ........................................... 348/144; 348/159
`
`
`
`
`(58) Field of Classification Search
`
`
`
`
`
`USPC ................................... 348/36, 42, 144, 218.1
`
`
`
`
`
`
`pp 1ca ion
`e or comp e e searc
`is ory.
`1 t
`hh' t
`See a
`fil f
`1.
`t'
`
`
`
`
`
`
`
`
`
`
`
`
`(200601)
`
`
`
`
`
`
`
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`
`
`5,166,789 A * 11/1992 Myrick ......................... 348/144
`5,499,051 A
`3/1996 Suda et a1.
`
`
`
`
`
`5,602,584 A
`2/1997 Mitsutake et a1.
`
`
`
`
`
`
`
`
`
`................ 348/144
`5,604,534 A *
`2/1997 Hedges et a1.
`
`
`
`
`5,668,595 A
`9/1997 Katayama et a1.
`
`
`
`
`5,682,198 A
`10/1997 Katayama et a1.
`
`
`
`
`5,699,108 A
`12/1997 Katayama et a1.
`5,760,884 A
`6/1998 Yahashi et a1.
`
`
`
`
`
`
`
`
`
`5,798,791 A
`8/1998 Katayama et a1.
`5,860,912 A
`1/1999 Chiba
`
`
`
`
`5,889,553 A *
`3/1999 Kino et a1.
`................. 348/218.1
`
`
`
`
`5,937,212 A
`8/1999 Kurahashi et a1.
`
`
`
`
`
`
`
`
`
`6,097,430 A
`8/2000 Komiya et a1.
`
`
`
`6,141,036 A
`10/2000 Katayama et a1.
`
`
`
`
`
`6,236,748 B1
`5/2001 ijima et a1.
`6,323,858 B1
`11/2001 Gilbert et a1.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1°\
`CAMERA TRIGGER
`PULSES 23
`
`
`
`\
`
`
`
`
`
`
`
`
`
`ll
`
`342
`
`
`
`
`
`
`
`28
`
`
`
`APPL-1031 / Page 1 of 15
`Apple v. Corephotonics
`
`
`
`
`FlRSTDAS/CPU a
`
`
`
`
`
`REMOTE
`TERMINAL
`
`
`
`
`INSTRUMENTMODULE
`
`
`(lMU/GPSAND
` .l
`
`
`TRIGGER/SYNC)
`
`
`___MB§.5_M_EMQB.Y.--
`
`
`T
`B
`A
`
`
`25
`
`
`APPL-1031 / Page 1 of 15
`Apple v. Corephotonics
`
`

`

`
`
`US 8,462,209 B2
`
`
`Page 2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U.S. PATENT DOCUMENTS
`
`
`10/2006 Lee et al.
`7,126,630 B1
`
`
`
`
`
`........................ 348/42
`3/2007 Abe et a1.
`7,190,389 B1*
`
`
`
`
`
`5/2007 Matsumoto et a1.
`7,221,364 B2
`
`
`
`
`
`
`8/2007 Suda
`7,262,799 B2
`
`
`
`
`
`1/2008 Ouchiet a1.
`7,324,135 B2
`
`
`
`
`2/2008 Frazier
`7,333,725 B1
`
`
`
`
`4/2008 Rohlfet al.
`7,353,114 B1
`
`
`
`2/2009 Inaba
`7,493,037 B2
`
`
`
`
`7,856,180 B2* 12/2010 Chishima ...................... 396/324
`
`
`
`
`
`
`
`7,924,311 B2 *
`4/2011 Yoshida et a1.
`............... 348/159
`
`
`
`
`
`
`2004/0264763 A1
`12/2004 Mas et a1.
`
`
`
`
`
`2005/0207672 A1 *
`9/2005 Bernardo et al.
`............. 382/284
`
`
`
`
`
`
`2008/0143842 A1
`6/2008 Gillard et a1.
`
`
`
`
`
`
`2010/0073371 A1*
`3/2010 Ernst et a1.
`.................... 345/428
`
`
`
`
`
`
`
`2010/0265329 A1* 10/2010 Doneker
`....................... 348/144
`
`
`
`
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`
`
`Gruber, M. et a1., “Digital Surface Models From Ultracam-X
`
`
`
`
`
`
`
`
`Images,” Stilla U et al (Eds) PIA07; International Archives of
`
`
`
`
`
`
`
`Photogrammetry, Remote Sensing and Spatial Information Sciences,
`
`
`
`
`
`
`
`36 (3/W49B), Munich, Germany, Sep. 19-21, 2007.
`
`
`
`
`
`
`
`Leberl, Franz W. et a1., “Novel Concepts for Aerial Digital Cameras”,
`
`
`
`
`
`
`
`ISRS Archives, vol. 34, Part 1, Proceedings of the ISPRS Commis-
`
`
`
`
`
`
`sion 1 Symposium, Denver, Colorado, Nov. 2002.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Leberl, Franz W., et al., “Flying the New Large Format Digital Aerial
`
`
`
`
`
`
`Camera Ultracam” , Proceedings of the Photogrammetric Week,
`
`
`
`Stuttgart, Germany, 2003.
`
`
`
`
`
`
`
`
`Petrie, Gordon, “Systematic Oblique Aerial Photography Using Mul-
`
`
`
`
`
`
`tiple Digital Frame Cameras”, Photogrammetric Engineering &
`
`
`
`
`
`
`Remote Sensing, Feb. 2009, pp. 102-107.
`
`
`
`
`
`
`
`Sun, X.,W Chen, J. J. Baker, D. E. Florence, R. L. Fischer, J. G. Ruby,
`
`
`
`
`
`
`
`J. C. Eichholz (2006b) “A Precision Geo -referenced Digital Airborne
`
`
`
`
`
`
`
`Camera System” Presented on 2006 IEEE International Geoscience
`
`
`
`
`
`
`and Remote Sensing Symposium & 27th Canadian Symposium on
`
`
`
`
`
`
`
`Remote Sensing, Denver, Colorado, USA, Jul. 31-Aug. 4, 2006.
`Sun, X., W. Chen, B. C. Patterson, D.E. Florence, R.L. Fischer, M.
`
`
`
`
`
`
`
`
`Jones, J.C. Eichholz, J. E. Richards, P. Shu, M. Jhabvala, a. La, D.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Kahle, and J Adams (2007), “An Advanced Airborne Multisensor
`
`
`
`
`
`
`
`
`
`Imaging System for Fast Mapping and Change Detection Applica-
`
`
`
`
`
`
`
`
`tions,” Proceedings of 2007 IEEE International Geoscience and
`
`
`
`
`
`
`
`
`
`Remote Sensing Symposium, Barsalona, Spain, Jul. 23-28, 2007, pp.
`600-605.
`
`
`
`
`
`
`
`
`
`Sun, X., and W. Chen (2008), “A Portable Airborne Multi-sensor
`
`
`
`
`
`
`Imaging System,” Proceedings of 2008 IEEE International
`
`
`
`
`
`
`Geoscience and Remote Sensing Symposium, Boston, Massachu-
`setts, USA, Jul. 10-15, 2008.
`
`
`
`
`
`
`
`
`
`
`
`
`* cited by examiner
`
`
`
`APPL-1031 / Page 2 of 15
`
`APPL-1031 / Page 2 of 15
`
`

`

`U.S. Patent
`
`1021,1
`
`648,
`
`2B9
`
`mm
`
`m295%ozoomm
`
`EoQEquS
`
`mmmmmsm
`
`Z
`
`m.29FGE
`
`Smm...wmUEmlfm...................................
`
`ommagmam3n,.........
`_ow‘2;ngQua/55%m2mmmmmw:
`mE252EMQSEQE"nnmgmmamm_"N58_w"Emgé
`562%$42A_u"1wI:mmm1azimwogEmm2num_m_
`
`_2502:allhe"HamillJ:
`
`APPL-1031 / Page 3 of 15
`
`APPL-1031 / Page 3 of 15
`
`
`
`

`

`
`U.S. Patent
`
`
`
`
`
`3mmHJ
`
`
`
`
`,m2mhS
`6
`
`SUV
`
`
`
`2,
`
`
`
`2R”
`
`00:illlnY
`
`
`
`mwm04§_.
`
`
`onmm§:mmw<§_ooomw<§_mww<§_ooo
`mfl
`
`
`vwm©<fi_
`
`
`
`Mmpg/momEmaamt/mam(zéozéflmzmomE4
`
`
`
`
`
`
`
`m8.GEmm.om<N.0:
`
`APPL-1031 / Page 4 of 15
`
`APPL-1031 / Page 4 of 15
`
`
`

`

`
`US. Patent
`
`
`
`
`
`Jun. 11,2013
`
`
`
`
`Sheet 3 of6
`
`
`
`US 8,462,209 B2
`
`
`
`,\
`
`
`
`4—0
`
`FIG.3
`
`APPL-1031 / Page 5 of 15
`
`
`
`OX
`
`
`
`
`54
`
`APPL-1031 / Page 5 of 15
`
`

`

`
`U.S. Patent
`
`
`
`Um
`
`6f
`
`902,2
`
`2
`
`
`
`M:Emaowmmamgxmma509%no25$sms”:598%Do
`
`Bw.OE
`
`%wmoswemoEEm8,E.was
`
`E<_mn_om&<EESEwax/(m9%egg/Ema
`
`
`
`zoEmmzmoNEE$4232a299:SE
`
`Emamg:51
`
`
`\
`
`mmmas/1:7:
`
`SE28$2:
`
`
`
`
`1,$3@584$3$5843M.S:5ES@502mamag32%wmama83.8m832:fig?w32:80hwait;
`
`
`MNEEEEssa;MNEEZ
`
`E
`
`
`
`
`
`
`
`
`
`APPL-1031 / Page 6 of 15
`
`APPL-1031 / Page 6 of 15
`
`
`
`

`

`U.S. Patent
`
`Jun. 11, 2013
`
`Sheet 5 of 6
`
`US 8,462,209 B2
`
`a
`
`mm
`
`
`
`20:.5200>bm<mzwjooEm:
`
`AN?EEK:H95
`
`ME9$90383RE?
`
`8mod;to$8$5
`
`om
`
`Ikm<wmjooow
`
`10$;me
`
`mm
`
`m.
`
`mm
`
`m2;BEm8E:a);
`
`2_gmmo<§Em:
`
`
`
`52,30;0mm;
`
`7:mmm0<§WIGE
`
`
`
`EEmOmOmaw
`
`
`
`m®<§_FIQEmom“BE.6?
`
`mm
`
`vm
`
`mm
`
`
`
`zoEozoo>mm<mz:._ooEoE
`
`
`
`”AN?“PIGEVA:uh
`
`33
`
`ME9$90mags.E59%
`
`mmmg;nomeowEm:
`
`m.9;
`
`3E
`
`mm®<_2_
`
`cm
`
`23m
`
`mun:Owmo<§
`
`mmkmm>zoo
`
`mm
`
`fiwémfi<mu
`
`wwdm
`
`@052:
`
`$29
`
`mm
`
`22mg
`
`29%83
`
`gwm
`
`APPL-1031 / Page 7 of 15
`
`APPL-1031 / Page 7 of 15
`
`
`
`
`
`
`
`
`
`
`
`

`

`
`US. Patent
`
`
`
`
`
`Jun. 11,2013
`
`
`
`
`Sheet 6 0f6
`
`
`
`US 8,462,209 B2
`
`
`
`APPL-1031 / Page 8 of 15
`
`(D
`(Y)
`
`
`
`O
`—_m “53
`"In-l 1
`.
`o
`Inn-[fl
`“—
`
`£13
`no
`
`91
`
`1.
`
`<u
`
`:
`
`5.2
`LL
`
`
`
`a
`
`
`
`
`
`%
`
`k CD
`
`{‘0
`
`
`
`APPL-1031 / Page 8 of 15
`
`

`

`
`1
`DUAL-SWATH IMAGING SYSTEM
`
`
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`
`
`10
`
`15
`
`25
`
`30
`
`35
`
`1. Field of the Invention
`
`
`
`
`
`
`
`
`
`
`This invention relates to photogrammetric imaging sys-
`
`
`
`
`
`
`tems and, in particular to a dual-swath electro-optical imag-
`
`
`
`
`
`
`
`
`
`ing system for use in fixed wing aircrafts, helicopters, and
`
`
`
`
`
`
`unmanned aerial vehicles (UAV) to acquire large-format,
`
`
`
`
`
`
`wide-swathwidth color imagery with centimeter-level reso-
`lution.
`
`
`
`
`
`2. Description of Related Art
`
`
`
`
`
`
`from natural disaster
`Remote
`sensing applications
`
`
`
`
`
`
`response to warfighting and peace keeping operations require
`
`
`
`
`
`
`
`
`an advanced airborne imaging system to provide fast and
`
`
`
`
`
`
`
`large format geo-referenced and orthorectified image maps.
`
`
`
`
`
`
`
`
`Many airborne digital frame camera systems have come
`
`
`
`
`
`
`
`
`into use for mapping, reconnaissance, and surveillance appli-
`
`
`
`
`
`
`
`cations. A major limitation is the size of the CCD and CMOS
`20
`
`
`
`
`
`
`
`
`area arrays currently available for use in such systems. As a
`
`
`
`
`
`
`
`result, widespread use of digital aerial photographic systems
`
`
`
`
`
`
`
`
`has occurred featuring multiple arrays, multiple lenses or
`
`
`
`
`
`
`
`multiple cameras in order to provide large ground coverage
`
`
`
`
`
`
`
`
`and big image format at a high ground resolution by a single
`
`flight.
`
`
`
`
`
`
`
`In an article by Gordon Petrie entitled “Systematic Oblique
`
`
`
`
`
`
`
`Aerial Photography Using Multiple Digital Frame Cameras”,
`
`
`
`
`
`published February 2009 in Photogrammetric Engineering
`
`
`
`
`
`
`
`
`
`and Remote Sensing, various twin cameras systems, and mul-
`
`
`
`
`
`
`
`
`tiple camera systems are described. Twin camera systems
`
`
`
`
`
`
`
`include “DiMAC” Wide from DiMAC Systems in Belgium, a
`
`
`
`
`
`
`
`Dual-DigiCAM system from IGI in Germany and a Rollei
`
`
`
`
`
`
`
`
`Metric (now Trimble) in Germany. All three of these twin
`
`
`
`
`
`
`
`
`camera units have a rather similar specification with each
`
`
`
`
`
`
`
`
`individual camera module having a digital image sensor back
`
`
`
`
`
`
`that produces images of 7.2K><5.4K:39 megapixels in size.
`
`
`
`
`
`
`
`Their camera shutters are synchronized to operate simulta-
`
`
`
`
`
`
`
`neously to obtain two oblique photos on either side of the
`
`
`
`
`
`
`
`
`
`flight line. After the rectification and the stitching together of
`40
`
`
`
`
`
`
`
`
`the two rectified images,
`the final merged (near-vertical
`
`
`
`
`
`
`image is 10.5K><7.2K:75 megapixels in size. However, the
`
`
`
`
`
`
`
`
`tilted photographs acquired by oblique cameras have fan
`
`
`
`
`
`
`
`
`shaped imaging areas with variable resolutions across the
`
`
`
`
`
`
`
`different parts of the images. Those tilted photographs need
`45
`
`
`
`
`
`
`
`
`
`extra work to be converted into the rectified images for Geo-
`
`
`
`
`
`
`
`graphic Information System (GIS) application needs. Extra
`
`
`
`
`
`
`
`cost is associated with these extra tilted-photograph pro-
`
`
`
`
`
`
`
`
`
`cesses, including to correct the fan shaped images with vari-
`
`
`
`
`
`
`
`able resolution to rectangular images with data interpolation,
`
`
`
`
`
`
`
`
`which wastes the precious image resources, introduces arti-
`
`
`
`
`
`fices, and sacrifices image quality.
`
`
`
`
`
`
`
`
`US. Pat. No. 7,009,638 issued Mar. 2, 2006 to Michael
`
`
`
`
`
`
`
`
`Gruber et al. and assigned to Vexcel Imaging GmbH, dis-
`
`
`
`
`
`
`
`
`closes a large format digital camera system exposing multiple
`
`
`
`
`
`
`
`
`detector arrays with one or more single lens systems to
`
`
`
`
`
`
`
`
`acquire sub-images of overlapping sub-area of large area
`
`
`
`
`
`
`
`
`objects. The sub-images are stitched together to form a large
`
`
`
`
`
`
`
`format, digital macroimage which can be colored. However,
`
`
`
`
`
`
`
`
`
`
`this camera system typically uses four (4) lenses and fifteen
`60
`
`
`
`
`
`
`
`
`(15) 35 mm format area sensor arrays, and is a fairly compli-
`
`
`
`
`
`
`
`
`cated system. The hardware is massive and heavy. It is com-
`
`
`
`
`
`
`
`plicated to build, calibrate, operate, and maintain. With 15
`
`
`
`
`
`
`critical non-redundant sub-systems, it means a higher failure
`
`
`
`
`
`
`
`
`
`rate. Its lens set is fixed and unlikely to have an interchange-
`
`
`
`
`
`
`
`able option. Its composite image is asymmetrically over-
`
`
`
`
`
`
`
`
`lapped and stitched. The patented processing solutions sug-
`
`
`
`
`
`
`gested many
`compromised
`options which means
`
`50
`
`55
`
`65
`
`
`
`US 8,462,209 B2
`
`
`2
`
`
`
`
`
`problematical. This system is unlikely to be widely accepted
`
`
`
`
`
`
`
`by the aerial photogrammetric and GIS industry even though
`
`
`
`
`
`
`
`the system manufacture has been sold to a large software
`manufacturer.
`
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Accordingly, it is therefore an object of this invention to
`
`
`
`
`
`
`provide a portable dual-swath photogrammetric imaging sys-
`
`
`
`
`
`
`
`tem for airborne applications to acquire wide swathwidth
`
`
`
`
`
`
`large-format color photogrammetric images with centimeter-
`level resolution.
`
`
`
`
`
`
`
`
`
`
`It is another object of this invention to provide a highly
`
`
`
`
`
`
`
`
`reliable simple two camera imaging system which simulta-
`
`
`
`
`
`
`
`
`
`neously captures a pair of left and right nadir shooting digital
`
`
`
`
`
`
`
`
`aerial photos having a strip of narrow overlapping field at the
`
`
`
`
`
`
`center of its composed panoramic frame or the edges of the
`
`
`
`
`
`
`
`left and right images from each camera.
`
`
`
`
`
`
`
`It is another object of this invention to provide a nadir
`
`
`
`
`
`
`shooting twin symmetrical lens imaging system that produces
`
`
`
`
`
`
`
`simultaneously captured left and right images with a center
`
`
`
`
`
`narrow overlapping strip that comprises radiometrically
`
`
`
`
`
`
`
`matched and digitally-registered image layers from the join-
`
`
`
`
`
`
`
`
`ing edges ofthe left and right images to form a seamless aerial
`
`
`panoramic image.
`
`
`
`
`
`
`
`
`It is another object of this invention to provide a nadir
`
`
`
`
`
`
`
`
`
`shooting dual camera system that has a native uniform pixel
`
`
`
`
`
`
`
`
`resolution across its aerial freeze frame panoramic image
`when terrain is flat.
`
`
`
`
`
`
`
`
`
`
`
`It is a further object of this invention to provide a dual
`
`
`
`
`
`
`
`camera system that acquires dual-swathes of two adjacent
`
`
`
`
`
`
`
`
`images with a narrow overlapped vertical strip at center. This
`
`
`
`
`
`
`
`
`narrow center overlap consists of a stereo pair that is acquired
`
`
`
`
`
`
`
`
`by two simultaneously exposed cameras with a fixed length of
`
`
`
`
`
`
`
`
`
`stereo baseline (distance between two shots) and the same
`
`
`
`
`
`
`
`
`attitude induced by aircraft roll, pitch, and yaw dynamics.
`
`
`
`
`
`
`
`
`
`This narrow strip of overlapped images can be more easily
`
`
`
`
`
`processed by simplified photogrammetric algorithms to pro-
`
`
`
`
`
`
`
`
`vide a software altimeter or range finder function for this
`
`
`
`dual-swath imaging system.
`
`
`
`
`
`
`
`It is another object of this invention to provide a portable
`
`
`
`
`
`
`
`airborne camera system using only two 6 cm by 4.5 cm format
`
`
`
`
`
`
`
`frame transfer CCD sensors to provide industrial leading
`
`
`
`
`
`
`
`
`large frame size options of78 megapixels and 120 megapixels
`
`
`
`
`
`
`
`
`
`today and even higher pixel count options in the future.
`
`
`
`
`
`
`
`It is another object of this invention to provide a portable
`
`
`
`
`
`
`
`
`airborne camera system using merely two sensors to provide
`
`
`
`
`
`
`
`the leading edge 14,200 pixel swathwidth ground track for 78
`
`
`
`
`
`
`
`
`
`megapixel option and 18,800 pixel swathwidth for 120 mega-
`
`
`pixel option.
`
`
`
`
`
`
`It is another obj ect ofthis invention to integrate a GPS/IMU
`
`
`
`
`
`
`
`
`device for this twin camera imaging system that provides
`
`
`
`
`
`
`
`GPS time reference, direct image geo-reference, and direct
`
`
`
`
`
`
`image mosaic capabilities to all of its captured images.
`
`
`
`
`
`
`
`It is yet another object of this invention to provide a dual
`
`
`
`
`
`
`camera system that acquires quickly deliverable precisely
`
`
`
`
`
`
`geo -referenced nadir shooting panoramic images rather than
`
`
`
`
`
`oblique shooting fan shaped images.
`
`
`
`
`
`
`
`
`These and other objects are further accomplished by a dual
`
`
`
`
`
`
`
`swath imaging system comprising a first CCD camera
`
`
`
`
`
`
`
`
`mounted on a remote sensing platform, the first CCD camera
`
`
`
`
`
`
`
`
`
`comprises a first CCD imager and a nadir pointing first optical
`
`
`
`
`
`
`
`
`lens, a first lens shift mount attached between a body of the
`
`
`
`
`
`
`
`
`
`
`
`first camera and the first optical lens for shifting the first
`
`
`
`
`
`
`
`
`optical lens a predetermined distance left with respect to the
`
`
`
`
`
`
`
`
`
`
`first camera body and first CCD imager, a second CCD cam-
`
`
`
`
`
`
`
`
`era mounted on the platform adjacent to the first CCD camera,
`APPL-1031 / Page 9 of 15
`
`
`
`
`
`
`
`
`
`APPL-1031 / Page 9 of 15
`
`

`

`
`
`US 8,462,209 B2
`
`
`3
`
`
`
`
`
`
`
`
`
`the second CCD camera comprises a second CCD imager and
`
`
`
`
`
`
`
`
`
`
`a nadir pointing second optical lens, a second lens shift mount
`
`
`
`
`
`
`
`attached between a body of the second CCD camera, and the
`
`
`
`
`
`
`
`
`
`second optical lens for shifting the second optical lens a
`
`
`
`
`
`
`
`predetermined distance right with respect to the body of the
`
`
`
`
`
`
`
`
`
`
`second CCD camera and the second CCD imager, the first
`
`
`
`
`
`
`
`
`CCD camera comprises the first CCD imager positioned on a
`
`
`
`
`
`
`
`
`
`
`right side of a first focal point of the first optical lens for
`
`
`
`
`
`
`
`
`
`capturing a left scene image, and the second CCD camera
`
`
`
`
`
`
`
`
`comprises the second CCD imager positioned on a left side of
`
`
`
`
`
`
`
`
`
`
`the second focal point ofthe second optical lens for capturing
`
`
`
`
`
`
`
`
`
`a right scene image. The dual-swath imaging system further
`
`
`
`
`
`
`comprises means for forming a dual-swath panoramic scene
`
`
`
`
`
`
`
`
`
`by narrow overlapping of the left scene image and the right
`
`
`
`
`
`
`
`
`
`
`scene image at adjacent edges. The first lens shift mount shifts
`
`
`
`
`
`
`
`the first optical lens a predetermined distance up to halfwidth
`
`
`
`
`
`
`
`
`
`
`ofthe first CCD imager, and the second lens shift mount shifts
`
`
`
`
`
`
`
`the second optical lens a predetermined distance up to half-
`
`
`
`
`
`
`
`
`
`width of the second CCD imager. The system comprises
`
`
`
`
`
`
`
`means for generating a trigger pulse for simultaneously acti-
`
`
`
`
`
`
`
`
`
`
`vating the first camera and the second camera to acquire the
`
`
`
`
`
`
`
`left scene image and the right scene image.
`
`
`
`
`
`
`The objects are further accomplished by a photogrammet-
`
`
`
`
`
`
`
`
`ric imaging system comprising a first CCD camera mounted
`
`
`
`
`
`
`
`
`
`on a platform having a nadir pointing first optical lens, and a
`
`
`
`
`
`
`
`
`second CCD camera mounted on the platform adjacent to the
`
`
`
`
`
`
`
`
`
`first CCD camera and having a nadir pointing second optical
`
`
`
`
`
`
`
`
`
`lens, the first CCD camera comprises a first CCD imager
`30
`
`
`
`
`
`
`
`
`positioned on a right side of a first focal point of the first
`
`
`
`
`
`
`
`
`
`
`optical lens for capturing a left scene image, the second CCD
`
`
`
`
`
`
`
`camera comprises a second CCD imager positioned on a left
`
`
`
`
`
`
`
`
`
`
`side of a second focal point of a second optical lens for
`
`
`
`
`
`
`
`capturing a right scene image, means for geo -referencing the
`
`
`
`
`
`
`
`
`
`left scene image, means for storing the geo-referenced left
`
`
`
`
`
`
`
`
`scene image, means for geo-referencing the right scene
`
`
`
`
`
`
`
`
`image, means for storing the geo-referenced right scene
`
`
`
`
`
`
`
`
`image, and means for merging the geo-referenced left scene
`
`
`
`
`
`
`
`
`image and the geo-referenced right scene image to form a
`
`
`
`
`
`
`
`dual-swath panorama. The system comprises means for form-
`
`
`
`
`
`
`
`
`ing a panoramic scene by narrow overlapping ofthe left scene
`
`
`
`
`
`
`
`
`
`
`image and the right scene image at adjacent edges. The first
`
`
`
`
`
`
`
`
`
`
`optical lens attaches to a first lens shift mount and the second
`
`
`
`
`
`
`
`
`
`optical lens attaches to a second lens shift mount, the first lens
`45
`
`
`
`
`
`
`
`
`
`
`shift mount being attached to the first CCD camera and the
`
`
`
`
`
`
`
`
`
`second lens shift mount being attached to the second CCD
`camera. The first shift mount and the second shift mount shift
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the first optical lens and the second optical lens whereby the
`
`
`
`
`
`
`
`
`
`first CCD imager and the second CCD imager are shifted on
`
`
`
`
`
`
`
`
`
`
`
`right and left focal points of the first optical lens and the
`
`
`
`
`
`
`
`
`second optical lens. The system comprises means for gener-
`
`
`
`
`
`
`
`
`ating a trigger pulse for simultaneously activating the first
`
`
`
`
`
`
`
`
`
`camera and the second camera to acquire the left scene image
`
`
`
`
`
`
`
`
`
`and the right scene image. The left scene geo-referencing
`
`
`
`
`
`
`
`
`means comprises means for attaching real time GPS/IMU
`
`
`
`
`
`
`
`metadata measurements
`to each image captured and
`
`
`
`
`
`
`
`recorded. The right scene geo-referencing means comprises
`
`
`
`
`
`
`
`
`means for attaching real time GPS/IMU metadata measure-
`
`
`
`
`
`
`
`
`ments to each image captured and recorded, the imaging
`
`
`
`
`
`
`
`system wherein the means for forming a panoramic scene by
`
`
`
`
`
`
`
`
`
`narrow overlapping ofthe left scene image and the right scene
`
`
`
`
`
`
`
`image comprises means for generating digital elevation mod-
`
`
`
`
`
`
`
`els wherein a measured width of the overlapping field pro-
`
`
`
`
`
`
`
`
`
`
`vides a real time above ground level height and the imaging
`
`
`
`
`
`
`
`
`system wherein the system comprises minimal size and
`
`
`
`
`
`
`
`
`weight for portability and use in light manned and unmanned
`aerial vehicles.
`
`
`
`10
`
`15
`
`20
`
`25
`
`35
`
`40
`
`50
`
`55
`
`60
`
`65
`
`
`
`
`
`
`
`
`
`The objects are further accomplished by a method for
`
`
`
`
`
`
`
`providing a dual-swath imaging system comprising the steps
`
`
`
`
`
`
`
`
`of mounting a first CCD camera on a remote sensing plat-
`
`
`
`
`
`
`
`
`
`
`form, the first CCD camera comprises a first CCD imager and
`
`
`
`
`
`
`
`
`
`
`a nadir pointing first optical lens, attaching a first lens shift
`
`
`
`
`
`
`
`
`mount between a body of the first camera and the first optical
`
`
`
`
`
`
`
`
`
`lens for shifting the first optical lens a predetermined distance
`
`
`
`
`
`
`
`
`
`
`with respect to the first camera body and first CCD imager,
`
`
`
`
`
`
`
`mounting a second CCD camera on the platform adjacent to
`
`
`
`
`
`
`
`
`
`the first CCD camera, the second CCD camera comprises a
`
`
`
`
`
`
`
`
`
`second CCD imager and a nadir pointing second optical lens,
`
`
`
`
`
`
`
`attaching a second lens shift mount between a body of the
`
`
`
`
`
`
`
`
`
`second CCD camera and the second optical lens for shifting
`
`
`
`
`
`
`
`
`the second optical lens a predetermined distance with respect
`
`
`
`
`
`
`
`
`
`
`to the second CCD camera and the second CCD imager,
`
`
`
`
`
`
`
`
`
`capturing a left scene image by the first CCD camera having
`
`
`
`
`
`
`
`
`the first CCD imager positioned on a right side of the first
`
`
`
`
`
`
`
`
`
`
`focal point ofthe first optical lens, and capturing a right scene
`
`
`
`
`
`
`
`
`
`image by the second CCD camera having the second CCD
`
`
`
`
`
`
`
`imager positioned on a left side of the second focal point of
`
`
`
`
`
`
`
`
`
`the second optical lens. The method wherein the method
`
`
`
`
`
`
`
`comprises the step of forming a dual-swath panoramic scene
`
`
`
`
`
`
`
`
`
`by narrow overlapping of the left scene image and the right
`
`
`
`
`
`
`
`
`
`scene image at adjacent edges, the step of attaching the first
`
`
`
`
`
`
`
`
`
`
`lens shift mount and, shifting the first optical lens a predeter-
`
`
`
`
`
`
`
`mined distance includes providing the predetermined dis-
`
`
`
`
`
`
`tance to be up to a halfwidth of the first CCD imager, and the
`
`
`
`
`
`
`
`
`
`
`step of attaching the second lens shift mount and shifting the
`
`
`
`
`
`
`
`second optical lens a predetermined distance includes provid-
`
`
`
`
`
`ing the predetermined distance to be up to a halfwidth of the
`
`
`
`second CCD imager.
`
`
`
`
`
`
`
`Additional objects, features and advantages of the inven-
`
`
`
`
`
`
`
`
`
`tion will become apparent to those skilled in the art upon
`
`
`
`
`
`
`
`consideration of the following detailed description of the
`
`
`
`
`
`
`
`preferred embodiments exemplifying the best mode of carry-
`
`
`
`
`
`
`ing out the invention as presently perceived.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`
`4
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The appended claims particularly point out and distinctly
`
`
`
`
`
`
`
`
`
`claim the subject matter ofthis invention. The various objects,
`
`
`
`
`
`
`
`
`advantages and novel features of this invention will be more
`
`
`
`
`
`
`
`
`fully apparent from a reading of the following detailed
`
`
`
`
`
`
`description in conjunction with the accompanying drawings
`
`
`
`
`
`
`
`
`
`in which like reference numerals refer to like parts, and in
`which:
`
`
`
`
`
`
`
`FIG. 1 is a block diagram of a dual-swath imaging system
`
`
`
`
`according to the present invention.
`
`
`
`
`
`
`FIGS. 2A, 2B and 2C illustrate the imaging principle ofthe
`
`
`present invention.
`
`
`
`
`
`
`
`FIG. 3 illustrates left and right images provided by the two
`
`
`
`
`
`
`
`cameras ofthe imaging system with overlapping field accord-
`
`
`
`
`ing to the present invention.
`
`
`
`
`
`
`FIG. 4 is a block diagram of the data acquisition software
`
`
`
`
`
`
`
`program for operation of the dual-swath imaging system of
`FIG. 1.
`
`
`
`
`
`
`FIG. 5 is a block diagram of an automatic image batch
`
`
`
`
`
`
`processing method for geo-referencing, and stitching the
`
`
`
`
`
`overlapped dual-swath aerial images of FIG. 3.
`
`
`
`
`
`
`
`
`
`
`
`FIG. 6A, FIG. 6B, and FIG. 6C show the front, top and side
`
`
`
`
`
`
`
`views of a lens shift mount (LSM) for attaching a lens to a
`
`
`
`
`
`
`
`
`camera body and shifting the lens a predetermined distance
`
`
`
`perpendicularly to its optical axis.
`DESCRIPTION OF ILLUSTRATIVE
`
`EMBODIMENT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Referring to FIG. 1 a block diagram of a dual-swath elec-
`
`
`
`
`
`
`tro-optical (EO) aerial imaging system 10 according to the
`
`
`
`
`APPL-1031 / Page 10 of 15
`
`APPL-1031 / Page 10 of 15
`
`

`

`
`
`US 8,462,209 B2
`
`
`6
`TABLE 1
`
`
`
`5
`
`
`
`
`
`
`
`present invention is shown comprising two cameras 12, 14
`
`
`
`
`
`
`
`
`having two identical parallel mounted nadir pointing optical
`
`
`
`
`
`
`
`
`
`lenses 16, 18 with two identical CCD imagers 11, 13 mounted
`
`
`
`
`
`
`
`
`
`on a pair of specially configured camera bodies 17, 19. The
`
`
`
`
`
`
`
`
`camera bodies 17, 19 are attached to optical lenses 16, 18 with
`
`
`
`
`
`
`
`
`
`
`a pair of mechanical lens shift mounts (LSM) 34-1 and 34-2
`which are mounted between camera 12 elements 16 and 17
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and between camera 14 elements 18 and 19 with appropriate
`
`
`
`
`
`
`
`
`orientations to have the lens optical axes symmetrically
`
`
`
`
`
`
`
`
`shifted to left and right a predetermined distance to take dual
`
`
`
`
`
`images simultaneously (see FIGS. 2A and 2C).
`
`
`
`
`
`
`
`
`Referring to FIG. 1, FIG. 6A, FIG. 6B and FIG. 6C, FIGS.
`
`
`
`
`
`
`
`
`
`
`6A, 6B and 6C show the front, top and side views of a lens
`
`
`
`
`
`
`
`
`
`shift mount 34 which are used to symmetrically shift the twin
`
`
`
`
`
`
`
`lenses 16, 18 a predetermined offset to their original optical
`axis locations. The lens shift mount 34-1 as shown in FIG. 1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`has a first end 38 (FIG. 6B) that attaches to the camera body
`
`
`
`
`
`
`
`
`
`
`17 and a second end 36 for attaching to the optical lens 16.
`
`
`
`
`
`
`
`
`
`This lens shift mount 34 shifts the lens a predetermined dis-
`
`
`
`
`
`
`
`tance 39 (FIG. 6B) up to a half width of the CCD imagers 11,
`
`
`
`
`
`
`
`13, which is determined by the width of the center overlap
`
`
`
`
`
`
`
`
`
`strip (for example, a 200 pixel width of the overlap when
`
`
`
`
`
`
`
`focused at infinity in the present embodiment) and results in
`
`
`
`
`
`
`
`the CCD imagers 11, 13 being symmetrically shifted on the
`
`
`
`
`
`
`
`
`
`left and right focal points of the lenses 16, 18 by approxi-
`
`
`
`
`
`
`mately 23.17 mm each in the present embodiment. FIG. 6A
`and FIG. 6B show the orientation of the lens shift mount 34-2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`for attaching to camera body 19. For attaching to camera body
`
`
`
`
`
`
`
`
`
`
`
`17, the lens shift mount 34-1 is rotated 180 degrees from the
`
`
`
`
`position shown in FIG. 6B.
`
`
`
`
`
`
`
`The two cameras 12 and 14 are triggered simultaneously by
`
`
`
`
`
`
`
`
`a common open/close trigger pulse 23 from a precision ana-
`
`
`
`
`
`
`
`
`log switch wired inside instrument module 26 when receiving
`
`
`
`
`
`
`
`
`
`a programmed CMOS-logic level pulse input from the main
`
`
`
`
`
`
`
`DAS/CPU unit 20. When triggered, camera 12 acquires digi-
`
`
`
`
`
`
`
`
`
`tal images and sends image data 27 to a first DAS/CPU
`
`
`
`
`
`
`
`module 20 which includes a first data acquisition system
`
`
`
`
`
`
`
`
`(DAS) for interfacing with a first central processing unit
`
`
`
`
`
`
`
`
`(CPU) wherein both the DAS and CPU are integrated in the
`
`
`
`
`
`
`
`stackable module 20. Camera 14 acquires digital images
`
`
`
`
`
`
`
`simultaneously with Camera 12 and sends image data 29 to a
`second DAS/CPU module 22 which includes a second DAS
`
`
`
`
`
`
`
`45
`
`
`
`
`
`
`
`
`for interfacing with a second CPU inside a second DAS/CPU
`
`
`
`
`
`
`
`
`module 22. The DAS/CPU module 20 processes and sends
`
`
`
`
`
`
`
`imaging data 27 of camera 12 to a removable solid state disk
`
`
`
`
`
`
`(SSD) 24 inside an IMU/GPS and mass memory multi-func-
`
`
`
`
`
`
`
`tion instrument module 26 via a computer Serial ATA (SATA)
`
`
`
`
`
`
`
`data cable, which is a part of inter-module connection buses
`
`
`
`
`
`
`
`
`28. The DAS/CPU module 22 processes and sends imaging
`data 29 of camera 14 to another removable solid state disk
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(SSD) 25 inside the multi-function instrument module 26 via
`another SATA cable in bus 28. Meanwhile the CPUs of DAS/
`
`
`
`
`
`
`
`
`CPU module 20 and DAS/CPU module 22 attach real time
`
`
`
`
`
`
`
`
`GPS/IMU metada