Ul’llted States Patent [19]
`Parulski et al.
`
`llllllllllllll|||llllllllllllllllllllllllllllllllllllllllllllllllllllllllll
`US005506617A
`[11] Patent Number:
`5,506,617
`[45] Date of Patent:
`Apr. 9, 1996
`
`[54] ELECTRONIC CAMERA INCORPORATING
`A COMPUTEKCOMPATIBLH BUS
`INTERFACE
`
`4,545,068 10/1985 Tabata et a1. ........................... .. 382/41
`4,587,633
`5/1986 Wang ............ ..
`364/900
`4,589,020 5/1986 Akatsuka ...... ..
`358/160
`5,138,459
`8/1992 Roberts et a1. ..
`..... .. 358/209
`
`
`
`Inventors: Kenneth Parulski; Raymond J. Bouvy DavidA smith 8110f
`
`5,231,501
`
`
`
`........... .. 7/1993 Sakal . . . . . . . . . . .
`
`
`
`.. . . . . . .. 358/209
`
`Roches’ter_ John'J Acell’o East
`
`5,258,859 11/1993 Wada et a1. .
`
`358/487
`
`Rochester’ an of NY'
`[73] Assignee: Eastman Kodak Company, Rochester,
`NY.
`
`[21] APPL NO-I 988,560
`[22] Filed.
`Dec 10 1992
`'
`'
`’
`...... .. H04N 5/225
`[51] Int. Cl.6
`[52] US. Cl. .......................... .. 348/207; 348/552
`[58] Field Of Search ...................... .. 354/75, 76; 358/209,‘
`358/903, 906, 909; 364/513, 514; 348/552,
`207, 373, 375
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`Primary Examiner-Wendy R. Greening
`Attorney’ Agent’ or F'rm_'Dav1d M‘ Woods
`
`ABSTRACT
`[57]
`An electronic camera is provided as a module that attaches
`to the signal bus of a PC-compatible computer. The camera
`includes a minimum of components, particularly an image
`sensor and an A/D converter, and a PC-compatible interface
`connector for mating with a bus extension connector on the
`computer- By directly presenting digitized data from the
`camera to the signal bus of the computer through the bus
`connector, the camera can be kept relatively simple and the
`computer can be relied upon to perform image processing,
`storage, and display.
`
`4,425,586
`
`1/1984 Miller .................................... .. 358/335
`
`26 Claims, 10 Drawing $118815
`
`,20
`
`28
`
`f
`
`\
`
`............................ v.
`
`34
`
`a - . . , _
`
`
`
`,,,,,,,,,, ~ 14
`
`QUENCHED
`
`FLASH 1
`\16
`
`7/36
`
`_
`
`,
`
`_
`
`_ ,1
`
`C00
`
`IMAGER \42
`
`__
`
`"
`
`z
`
`42A
`
`2
`
`z
`2 000
`\ :
`:
`44 ;
`; DR'VERS
`f ..........................
`
`4a
`W“ I’)
`
`40
`
`+
`—
`
`A D
`/
`
`uP
`
`SHUTTER
`RELEASE
`SWITCH
`
`\22
`
`STEPPER
`DRIVERS \50
`1
`1
`
`*
`
`46
`
`SIGNAL
`_ PROCESSOR
`E2
`
`O
`I‘— 2
`:
`‘U
`I
`5
`IT‘
`54
`I’
`
`_
`
`001
`
`Facebook/Instagram Ex. 1004
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheet 1 of 10
`
`5,506,617
`
`FIG. 1
`
`002
`
`Facebook/Instagram Ex. 1004
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheet 2 of 10
`
`5,506,617
`
`?/ $22
`
`f.
`
`00.0
`
`
`
`w ,1. 1 229m iii.
`
`m I I /
`
`
`
`Wllm 1 mowmwooma % Q\<
`
`C + $
`
`K mm % E?DIw
`
`.W @E NN/ 10:26
`3 ww?dm
`
`952%
`............................ .. g g 9/ 12:
`
`i/ m?zmo S10E30 Q8 9., /I M
`
`3/ EH55 3
`
`
`
`003
`
`Facebook/Instagram Ex. 1004
`
`

`

`US. Patent
`
`9sm
`
`%
`
`mm
`
`3
`
`05,5
`
`7
`
`wom
`
`
`
`Amama92:mmjoEzoo
`
`E052328mama92:
`
`mn
`
`"mc55
`wmmmmmonz
`
`mm2<m
`
`004><4¢m5mNH
`
`
`
`><..n_m_o>m02m2
`
`OT
`
`m.6”:on
`
`REgon
`
`m.6,mDE
`
`3&0
`
`<20
`
`mijmHzoo
`
`mm
`
`on
`
`+5
`
`mo<mmm._.z_
`
`mum:
`
`._.3n_z_
`
`4<mmzmw
`
`wmommnn.
`
`mum:
`
`HDQZ.
`
`004 Facebook/Instagram Ex. 1004
`
`004
`
`Facebook/Instagram Ex. 1004
`
`
`
`
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheet 4 0f 10
`
`5,506,617
`
`-|/o CH CK
`s07
`SD6
`s05
`$04
`$03
`502
`$01
`SDO
`l/O CH RDY
`AEN
`SA19
`sms
`SA17
`SA16
`SA15
`SA14
`SA13
`SA12
`SA11
`SA10
`5A9
`8A8
`SA7
`SAS
`8A5
`5A4
`8A3
`8A2
`5A1
`SAO
`
`VVVVVVVV'
`
`VVVVVVVVVVVVVVVVVVVV
`
`GROUND
`RESET
`+5 VOLTS
`IRQ2
`-5 VOLTS
`DRQ1
`-12 VOLTS
`-ows
`+12 VOLTS
`GROUND
`—SMEMW
`—SMEMR
`-1ow
`—-|OR
`-DACK3
`DRQIS
`—DACK1
`DRQ1
`—-DACKO
`CLK
`IRQ7
`mos
`IRQS
`IRQ4
`[R03
`—DACK2
`T/C
`BALE
`+5 VOLTS
`osc
`GROUND
`
`A AAAAA
`AAA AA VVVVV
`
`B1
`B2
`B3
`B4
`B5
`B6
`B7
`B8
`B9
`B10
`B1 1
`B12
`B13
`B14
`B15
`B16
`B17
`B18
`B19
`B20
`B21
`B22
`B23
`B24
`B25
`B26
`B27
`B28
`B29
`B30
`B31
`
`A1
`A2
`A3
`A4
`A5
`A6
`A7
`A8
`A9
`A10
`A1 1
`A12
`A13
`A1 4
`A15
`A1 6
`A1 7
`A1 8
`A1 9
`A20
`A21
`A22
`A23
`A24
`A25
`A26
`A27
`A28
`A29
`A30
`A31
`
`FIG. 6 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
`
`005
`
`Facebook/Instagram Ex. 1004
`
`

`

`V VVVVV
`
`A A A A
`
`—MEM C816 >
`-|/o C816 >
`|RQ1O
`IRQ11
`IRQ12
`IRQ15
`IRQ14
`--DACKO
`DRQO
`—DACK5
`DRQ5
`—DACK6
`DRQ6
`—DACK7
`DRQ7
`+5 VOLTS
`—MASTER
`GROUND
`
`>
`
`US. Patent
`
`Apr. 9, 1996
`
`Sheet 5 0f 10
`
`5,506,617
`
`SBHE
`LA23
`LA-22
`LA21
`LA2O
`LA19
`LA18
`LA17
`—MEMR
`—MEMW
`SD08
`SD09
`sm 0
`$01 1
`$012
`sms
`$014
`$015
`
`
`
`AAAAAAAA AAAAAAA
`
`VVVVVVVVVVVVVVVVVV
`
`D1
`D2
`D3
`D4
`D5
`D6
`D7
`D8
`D9
`D10
`D11
`D12
`D13
`D14
`D15
`D16
`D17
`D18
`
`C1
`C2
`C3
`C4
`C5
`C6
`C7
`C8
`C9
`C10
`C11
`C12
`C13
`C14
`C15
`C16
`C17
`C18
`
`FIG. 7
`
`006
`
`Facebook/Instagram Ex. 1004
`
`

`

`U
`
`6
`
`S%EnWWMmSPLP.MTMWm_SRAL_I|III.wWMWE_L_H._9.,DcwVJ.m“mmmzmo_mAmP.08n__mmmm¢mmm"--__I.mo20%....:o\<izom_moi:.m2oS<z<_ooo_P__.rIIIIILS”mIIIIIIIIIIIIIIIIIIIIIIIIIIIL/(mm
`tIIIIILm_I
`___0.03ozE:_m_nma.
`
`m.6,L<mo:
`1111111111J0IIIIIIIIIIIIIIIIIIIIIIIIII5,xood555,.cm_5_m_
`
`<mw2<o
`
`mommmoommomoi
`
`mo<mmm._.z_
`
`mufl<zo_m
`
`007 Facebook/Instagram Ex. 1004
`
`007
`
`Facebook/Instagram Ex. 1004
`
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheet 7 of 10
`
`5,506,617
`
`#N
`
`PC BUS
`
`E
`
`N_
`
`mM
`
`ll.mwwmu'58ll6_6%III.6_II._
`o<55_.58"m5.8_mmmmmogrllllllllllllllllllllllllllllllllllll<
`|M_io-H<I_
`nNu_1—5“.
`
`
` _mEDEEZEfimlmz:.a
`
`_oi<4_B.on.250u_mamrug-24‘
`
`
`
`mmmmaeqwmmmonzmzzlkxmzlmflmzéh02.2:
`
`JoEzoom
`
`
`
`_.........mmbg.....................6_xood$532
`
`
`
`mmooomo0.00..
`
`008 Facebook/Instagram Ex. 1004
`
`008
`
`Facebook/Instagram Ex. 1004
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheets of10
`
`5,506,617
`
`<m.OE6llllllllllllllllllllllllllllllllllll4xoodmEméom
`
`D
`
`ADDRESS MSBS
`
`00.226.
`
`4<zo_m
`
`mommmoowi
`
`”.552.
`
`mom2mm
`
`LINE_CLAMP
`
`PIXEL_SAMPLE
`
`PIXEL__CLAM P
`
`3
`
`
`
`0.004Q23:m._<20.m
`
`(Mm2(0
`
`mommmoomaomoi
`
`mo<umm.rz_
`
`009 Facebook/Instagram Ex. 1004
`
`009
`
`Facebook/Instagram Ex. 1004
`
`
`
`
`
`
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheet 9 of 10
`
`5,506,617
`
`PC BUS COMPATIBLE CONNECTOR
`
`_____________
`
`mwmmoo<
`
`qomhzoo
`
` _E“em
`
`mwmmoo<
`
`m3H<Hm
`
`mmzj
`
`mm.O_l._
`
`.1llllllllllllllllllllllllllllllllllllL_530E5;
`
`O10 Facebook/Instagram Ex. 1004
`
`010
`
`Facebook/Instagram Ex. 1004
`
`
`
`
`

`

`US. Patent
`
`Apr. 9, 1996
`
`Sheet 10 of 10
`
`5,506,617
`
`HDOQ<mm4<HZONEOI000ME9.2.
`
`
`
`m2:mo<2_z<mmmmwz<mh<1m2<0
`mEDn‘EOo20msz.mum:
`
`..>o<mmlmz_u_:<wmz.<mmzmoQz<mgmamm
`hmnmmmFZ
`
`
`
`
`
`mOkOIn.ozEPEfiE2<moomm
`
`
`
`
`
`ZO_._.<O_I_nE<wmz.<>_bo<mumsA
`
`
`
`hxmzmmumz<mhzmmDmm_mmhaazoo
`
`
`
`
`
`
`
`
`
`KMHDQEOOMI...OHZ_<F<Q1.5m2:mzo
`
`.u_Ommmmz<mh<55<mEMOmemoz<
`
`<mm§<om=.=.n5mmmgom
`
`
`
`
`
`mmmmoo<<1m2<0OH02(2200..m_z_n_
`
`
`
`
`
`IO_I>>.Im<._u._n5mn__.._l..mmm:
`
`
`
`>m02m§2<m
`
`mm;
`
`
`
`Qmommzh:Im<._|._mmom<IoQz<
`
`
`
`
`
`.ZOHPDmExam/‘0:mmmmmmmmum:
`
`
`
`
`
`|E>m4H163mo<mm<mm2<o
`
`
`
`
`
`.mmFHDImmZmEO<mm2<o
`
`
`
`.DmommzEIm<._u._wmmE
`
`
`
`mmtblmmmmOAQ‘Qz‘q
`
`
`
`
`
`QZ<mo<$=mmmmmoomn.mmfianzzoo
`
`
`
`mo.mmmmmmafioo.m><n_am_o
`
`meQm<IMIR20.Emmmokm
`
`OP.O_n_
`
`O11 Facebook/Instagram Ex. 1004
`
`011
`
`Facebook/Instagram Ex. 1004
`
`
`
`
`
`
`
`
`
`
`
`

`

`1
`ELECTRONIC CAMERA INCORPORATING
`A COMPUTER-COMPATIBLE BUS
`INTERFACE
`
`FIELD OF INVENTION
`
`This invention pertains to the ?eld of electronic imaging
`and, more particularly, to an image acquisition peripheral
`operated as an input device to a portable computer.
`
`BACKGROUND OF THE INVENTION
`
`It is well known to use a video camera with a computer
`frame grabber. A typical system employs a video motion
`camera (such as the CCD 4000 RGB Flash~Sync Camera
`manufactured by Eastman Kodak Co., Rochester, NY.) and
`a frame grabber board (such as a TARGA‘TM) frame store
`board manufactured by True Vision, Inc., Indianapolis, Ind.)
`attached to the PC bus of a personal computer. The camera
`provides the timing to interface with the video frame store
`board by activating the frame acquire line of the frame store
`board whenever an external voltage input to the camera is
`dropped low (e.g., by dropping the “Camera Acquire In” line
`to the CCD 4000 camera). It is customary to design software
`to activate the camera’s frame acquire line from the com
`puter. To capture an image into the computer in such a
`customary application, the operator frames the subject while
`observing the live camera output on a video monitor, and
`then interacts with the computer keyboard at the proper
`moment.
`Another approach to computer image acquisition is
`described in US. Ser. No. 805,220, entitled “Hand-Manipu
`lated Electronic Camera Tethered to a Personal Computer,”
`which was ?led Dec. 11, 1991 in the names of K. A.
`Parulski, R. H. Hamel, and J. J. Acello, and assigned to the
`assignee of the present application. In this system an elec»
`tronic camera is coupled to a personal computer through a
`computer interface. In particular, a digital interface standard
`may be used, and images from the camera are input to the
`computer through a Small Computer System Interface
`(SCSI). The camera is preferably linked, or tethered, to the
`computer with a cable, thus allowing a certain amount of
`mobility for the camera independent of the computer.
`Both of these known interfaces have drawbacks. The
`NTSC signal is an analog signal subject to noise, and
`additionally requires a special frame grabber card in the
`computer to decode and digitize the signal. The SCSI signal
`has a relatively low data rate and a complicated protocol,
`requiring an expensive SCSI interface integrated circuit in
`the camera. Notwithstanding such drawbacks, the system
`described in Ser. No. 805,220 provides a low cost electronic
`still camera which attaches to a personal computer that
`provides image processing, storage, and display. By relying
`on the computer to perform these tasks, the camera cost can
`be greatly reduced.
`A customized high speed bus connection is described in
`U.S. Pat. No. 4,196,450 for transferring data between a
`scanner and a specialized processing system. A special
`architecture is required, including an interprocessor link
`between a low speed section, having a standard Intel 8080
`microprocessor connected to a low speed data bus, and a
`high speed section having a high speed bus and high speed
`microprocessor for handling scan data from the scanner. As
`the scanner is moved across an image page, scan data is
`supplied to the high speed data bus. Communication
`between the high speed and the low speed sections is
`through the interprocessor link. The shortcoming of this
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`5,506,617
`
`2
`arrangement is that a custom computer, including a non
`compatible bus, must be built just to handle, process, and
`store images.
`In all of these systems, there is need for a low cost means
`of digitally interfacing a small camera peripheral to a-por—
`table computer at a rate higher than that provided by a SCSI
`interface or an RS-232 input, but without the attendant
`di?iculties of either an analog frame grabbing process or a
`specialized, non-compatible computer architecture.
`
`SUMMARY OF THE INVENTION
`
`In accordance with the invention, an electronic camera
`operates in conjunction with a portable computer of the type
`having a plurality of digital devices, including a central
`processing unit (CPU) interconnected by a standardized,
`industry-compatible signal bus. The electronic camera
`includes an image sensor and means for digitizing image
`data output from the sensor. Means are provided for enabling
`attachment of the camera to the portable computer. The
`camera module further includes bus-compatible connection
`means for engaging the signal bus, and means for providing
`the digitized image data to the signal bus through the
`bus-compatible connection means when the camera is
`attached to the portable computer. In a preferred embodi
`ment, the signal bus is a PC compatible bus and the
`connector means is a PC-compatible connector.
`The advantage of the invention is that the camera can be
`designed as an inexpensive “clip-on” accessory that mates
`with a portable computer through its bus slot. The computer
`can be thus relied upon to perform image processing,
`storage, and display, and the cost of the camera may be
`accordingly reduced. The bus connection provides a high
`speed parallel interface for rapidly transferring image data
`from an image sensor to a memory device in the personal
`computer. In particular, by using an ISA (Industry Standard
`Architecture) bus, existing portable computer designs can be
`used, rather than requiring the development of a special
`computer incorporating a specialized bus architecture.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The invention will be described in relation to the drawings
`in which
`FIG. 1 is an illustration showing an electronic still camera
`attached to the bus extension connector of a pen-based
`computer in accordance with the invention;
`FIG. 2 is an illustration of the electronic still camera
`shown in FIG. 1;
`FIG. 3 is an end elevation of the camera of FIG. 1 taken
`along the lines 3—3 of FIG. 2;
`FIG. 4 is a block diagram of the major elements of the
`camera of FIG. 2;
`FIG. 5 is a simpli?ed block diagram of a typical PC
`compatible computer;
`FIG. 6 is the pin-out diagram of the bus connector for a
`PC-compatible PC/XT bus;
`FIG. 7 shows the pin-out diagram of the bus connections
`which together with FIG. 6 make up a PC-compatible PC/AT
`bus;
`FIG. 8 is a more detailed diagram of the signal processor
`subsystem shown in FIG. 4;
`FIG. 9 shows a second embodiment of the signal proces
`sor subsystem shown in FIG. 4; and
`FIG. 10 is a ?ow diagram of the image capture process.
`
`012
`
`Facebook/Instagram Ex. 1004
`
`

`

`3
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`5,506,617
`
`4
`inside the camera module 2, and requires only a few optical
`elements. The camera module 2 also includes a capture
`switch 22 (i.e., a “shutter button”) for taking the picture. The
`camera module 2 further includes an electrical interface
`connector 24 designed to mate with the bus connector 3 on
`the computer 4, thereby directly attaching the camera mod
`ule 2 to the PC-compatible bus of the computer 4. This
`permits control of the camera module 2 from the computer
`4, and provides a high speed parallel interface for rapidly
`transferring image data into the computer 4. In practice, it is
`desirable to physically attach the camera module 2 to the
`computer 4. Depending on the design chosen, the interface
`connector 24 may be sufficiently sturdy to provide mechani
`cal as well as electrical attachment. Preferably, however, the
`camera module 2 ‘would be secured to the computer by
`means of screws 26 secured through the holes 26a to
`threaded holes (not shown) on the computer 4.
`FIG. 4 shows the subsystems of the camera module 2. An
`optical system 28 includes the lens 14, a diaphragm 34, a
`shutter 36, and an infrared blocking ?lter 38. To keep the
`system inexpensive, the lens 14 is ?xed focus, and a close-up
`diopter lens 40 is optionally moved into the optical axis for
`close-up pictures. Image light is directed by the optical
`system 28 upon an image sensor 42, which is a charge
`coupled device (CCD) sensor, such as the full frame sensor
`KAF-400 manufactured by Eastman Kodak Company,
`Rochester, NY. The timing of the read-out of the image
`sensor 42 through its horizontal read-out register 42a is
`controlled by a CCD clock driver circuit 44.
`The sub-systems of the camera 20 are controlled by a
`microprocessor 46. In particular, the exposure conditions of
`the image are input to the microprocessor 46 from a pho
`tometer 48 and shutter release is initiated by the capture
`switch 22. Accordingly, the microprocessor 46 instructs a
`pair of stepper drivers 50 to set the diaphragm 34 and to
`operate the shutter 36, and, as needed, to ?re the ?ash 16.
`The microprocessor 46 further interfaces with a signal
`processor 52, which controls the read-out clocking of the
`image sensor 42 and processes the image signal generated
`by the image sensor 42. The signal processor 52 provides the
`processed image signals to the computer 4 via the bus
`compatible electrical interface connector 24.
`A simpli?ed block diagram of the computer 4, in particu
`lar, a PC-compatible computer, is shown in FIG. 5. The key
`component of the computer is the central processing unit, or
`CPU, 54. In a pen-based computer, the CPU can be, for
`example, a low power 16 bit CHMOS microprocessor such
`as the Intel 80C86A, manufactured by Intel Corp, Santa
`Clara, Calif. Alternatively, other processors, including CPUs
`compatible with the Intel 8088 CPU, Intel 80C286 CPU, or
`Intel 80386 or 80486 CPU families could be used. The CPU
`54 connects to a standardized, industry-compatible bus 58,
`which is a set of shared lines exchange of three types of
`digital signals: data signals, address signals, and control
`signals. When an 8086 or 8088 compatible processor is used,
`the bus is known as the PC/XT bus, which utilizes 54 signal
`lines and 8 power/ ground lines, including eight bits of data
`and twenty bits of address. When an 80286, 80386, or 80486
`family CPU is used as the CPU in a “PC compatible”
`computer, the bus 58 is known as the PC/AT bus or ISA
`(Industry Standard Architecture) bus. The PC/AT bus is an
`enhancement of the PC/XT bus. An additional connector
`carries an extra eight bits of data, four bits of address, and
`extra control lines. FIG. 6 shows the 62 pin connector used
`in the PC/XT bus. The AT bus uses both the connector shown
`in FIG. 6 and the connector shown in FIG. 7. The connector
`pin names for the PC/XT bus are described in table 1, as
`follows.
`
`Since electronic still cameras and computers, including
`pen-based computers, are well known, the present descrip
`tion will be directed in particular to elements forming part
`of, or cooperating more directly with, apparatus in accor
`dance with the present invention. Elements not speci?cally
`shown or described may be selected from those known in the
`art.
`Referring to FIG. 1, a camera module 2 is shown attached
`to a bus connector 3 of a portable computer 4, which
`provides image processing, storage, and display. (The bus
`connector 3 is shown in broken line to indicate that it is
`hidden from view due to its location between the module 2
`and the computer 4.) The invention is not limited to a
`particular computer, although a preferred computer is a
`small, battery operated IBM compatible personal computer
`(PC). One type of computer which is especially suitable is
`a “pen-based” computer, which is a small, portable unit that
`uses a pen-type stylus 6, rather than a keyboard and/or a
`“mouse” input device, for controlling the computer and
`inputting data. Menus are selected and data is entered using
`the stylus 6 to point to selections displayed on a pressure
`sensitive LCD display 8.
`Appropriate pen-based computers that could be adapted
`for use with the invention include the NCR System 3000
`Model 3125 NotePad computer manufactured by the NCR
`Corporation, Dayton, Ohio or the GridPad SL computer
`manufactured by Grid, a division of Tandy Corporation, Fort
`Worth, Tex. Both of these computers are compatible with the
`PC/AT bus (sometimes called the Industry Standard Archi
`tecture). The PC/AT bus uses a compatible enhancement of
`the original PC bus, the PC/XT bus. The NCR 3125 uses a
`CPU compatible with the Intel 386 microprocessor family,
`while the GridPad uses a CPU compatible with the Intel
`8086 microprocessor family. Since the vast majority of
`personal computers in use today are compatible with the
`IBM PC/AT design, it is advantageous to design the camera
`module to connect to PC/AT compatible computers.
`The computer system includes optional “pod” attach
`ments, which include electronics for speci?c applications. In
`FIG. 1, a rechargeable battery pack 10 constitutes one pod,
`while the camera module 2 constitutes a second pod. In such
`a system, the camera module 2 can be attached when needed
`for a particular application. The camera captures images and
`stores the images in the memory contained within the
`pen-based computer. When not needed, other pods may be
`attached, such as an RF transmitter pod (not shown) for
`allowing data to be transferred between the computer 4 and
`a central receiver (not shown). An exemplary imaging
`application is the automobile claims process, where there is
`a need to record data, including pictorial data, on vehicle
`damage in order to provide a repair estimate. In such an
`application, the computer 4 would store a set of claim forms,
`and the camera module 2 would be attached to the computer
`4 to obtain still images which can immediately be incorpo-.
`rated into the forms, thereby creating “smart forms” includ
`ing both images and the data needed for a particular appli
`cation.
`Referring to FIGS. 2 and 3, a front 12 of the camera
`module 2 includes a lens 14, which is covered with a ?ip-up
`?ash 16 when not in use to protect the lens 14. A back 18 of
`the camera module includes an optical view?nder 20, which
`slides out of (or alternatively, ?ips away from) a recess 20a
`on the back 18 of the camera. The slide-out design for the
`view?nder 20 is advantageous because it takes up little room
`
`10
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`013
`
`Facebook/Instagram Ex. 1004
`
`

`

`5,506,617
`
`5
`
`TABLE 1
`
`IBM PC/XT BUS SIGNALS
`
`SIGNAL NAME PIN #
`
`FUNCTION
`
`A3l-Al2
`SAO-SA19
`A9-A2
`SDO-SD?
`B14
`IOR
`B13
`IOW
`B12
`SMEMR
`Bll
`SMEMW
`All
`AEN
`B4, B25-B2l
`IRQZ-IRQ7
`B2
`RESET
`B18, B6, B16
`DRQl-DRQ3
`DACKO-DACK3 B19, B17, B26,
`B15
`B28
`B20
`Al
`A10
`B30
`B27
`B8
`Bl, B10, B31
`B3, B29
`B9
`B5
`B7
`
`BALE
`CLK
`IIO CH CK
`110 CH RDY
`OSC
`T/C
`OWS
`GND
`+5 V DC
`+12 V DC
`—5 V DC
`—l2 V DC
`
`address (AC-A15 for I/O)
`data
`[/0 read strobe
`I/O write strobe
`memory read strobe
`memory write strobe
`address enable
`interrupt request
`power-on reset
`DMA request
`DMA acknowledge
`
`address latch enable
`CPU clock
`l/O error
`pull LOW for wait states
`oscillator (3 X CPU clk)
`DMA terminal count
`Zero wait state
`signal & power grid
`+5 V supply
`+12 V supply
`—5 V supply
`—12 V supply
`
`These plus include power pins, which allow the plug-in
`board, and therefore the camera module 2, to be powered by
`the power supply (not shown) of the computer 4. The PC/AT
`connector, besides additional data and address lines,
`includes additional interrupt request, direct memory access
`(DMA) request, DMA acknowledge, and other special lines
`for 16-bit control. Note that these standard connectors are
`not available on the outside of the case enclosing the
`computer 4. Instead, the case must be opened to insert the
`connector 24, which may be elongated for this purpose, into
`an empty slot in the computer bus 58.
`In some CPUs, for example, the Intel 8086 CPU, the data
`and address signals are typically time multiplexed on the
`same microprocessor pins, and control lines indicate
`whether the signals on these pins contains an address or data.
`The control lines are used to latch the data and address
`signals onto the separate address and data bus lines of the
`signal bus 58. The control lines carry a group of “interrupt
`request” signals, which are used by peripherals when they
`need attention from the CPU 54. The address lines include
`two diiferent address spaces known as memory address
`space and I/O address space. The most signi?cant bit (MSB)
`address lines are typically used only for memory address
`space. Therefore, these MSBs can be used to represent
`different commands, when addressing a device using I/O
`address space.
`The PC-compatible bus 58 is connected to both program
`mable read only memory (PROM) 60 and random access
`memory (RAM) 62. The PROM 60 stores the BIOS (Basic
`Input/Output System) portion of the PC operating system
`software. The BIOS includes the bootstrap loader, which is
`executed when the computer is ?rst powered on. The rest of
`the PC operating system is normally stored on a hard disk
`drive 64, which is connected to the bus 58 via a hard drive
`controller 66. The hard drive 64 is also used to store
`application programs and data, including the digital images
`captured by the camera module 2. Portions of the disk
`operating system (DOS), application programs, and data are
`typically downloaded to the RAM memory 62 before pro
`gram execution. Since the access time of the RAM memory
`62 is generally much less than the time required to access the
`
`6
`hard drive 64, this decreases the program execution time. A
`direct memory access (DMA) controller 68 is typically used
`to rapidly move data between the hard drive 64 and the
`RAM memory 62. The DMA controller 68 transfers data
`between the memory address space and the I/O address
`space. The computer system further includes the LCD
`display 8, which is normally connected to a display RAM
`memory 72 that is loaded by the CPU 54 via the bus 58, and
`at least one general purpose user input device 74, such as a
`keyboard, mouse, or the pen stylus 6. The user input device
`74 is connected to the bus 58 via a user input interface 76,
`which may include a separate microprocessor.
`The computer 4 includes the PC-compatible bus connec
`tor 3, which is accessible from the outside of the computer
`case. By attaching the camera module 2 to the connector 3,
`the camera module 2 is able to access the PC-compatible bus
`58 in order to transfer image data from the CCD imager 42
`to the computer 4, and in general to allow the camera module
`2 to be controlled using the general purpose user input 74.
`PC-compatible computers ordinarily include internal “slots”
`with a standard bus connection, to allow users to purchase
`plug-in boards having additional memory, modems, etc., in
`order to expand the capabilities of their computer. This
`interface provides what is known as the “I/O bus?, or “I/O
`channel”, using the standard connectors shown in FIGS. 6
`and 7. In order to connect the camera module 2 to such
`computers, the interface connector 124 may include a plug
`in extender board for inserting into the bus connection “slot”
`on the computer 4. (Although not speci?cally shown, the
`“slot” would enclose the connector 3 shown in FIG. 1.)
`In one embodiment of the invention utilizing the PC/AT
`bus, the camera includes a 62 pin card edge connector and
`a 36 pin card edge connector, providing the connections
`shown in FIGS. 6 and 7, while the computer includes the
`appropriate female connectors accessible from the exterior
`of the case, to allow the camera module 2 to attach to the
`computer 4. As shown earlier, the camera module 2 is
`attached to the computer 4 using the screws 26. When the
`computer 4 is used without the camera module 2, a plate (not
`shown) is screwed into the computer, to prevent the con?
`nector 3 from being abused. In other embodiments, other
`types of physical connectors can be used. Furthermore, the
`connectors 3 and 24 may have fewer than 98 pins, since
`some of the bus signals may not be required by the camera.
`Details of one embodiment of the signal processor 52
`(FIG. 4) are shown in FIG. 8. A logic circuit 80 (such as
`Model EPS464 or Model EPM7096, integrated circuits
`manufactured by Altera Corp., San Jose, Calif.) provides the
`timing signals to control the image sensor 42 and the various
`parts of the signal processor 52. In particular the logic circuit
`80 provides the horizontal and vertical clocking signals
`H1,H2,V1,V2 to read an image signal from the sensor 42
`and the timing signal RESET to initiate each pixel read~out
`period. The output of the sensor 42 is initially processed by
`an analog signal processor 82 incorporating, e.g., a gain
`stage and a correlated double sampling circuit, and con
`verted to a digital signal by an analog-to-digital (A/D)
`converter 84. The digitized signal is then processed by an
`EPROM look-up table (LUT) 86 that stores the white
`balance and gamma correction curveshapes, and information
`about the camera.
`The LUT 86 can also store a camera serial number,
`location of sensor defects, the structure of a color ?lter array
`used on the sensor 42, color matrix coe?icients optimized
`for a particular image sensor, etc. Moreover, the LUT 86
`may also store the computer program which is used (by the
`computer) to operate the camera and to process the images
`
`5
`
`10
`
`15
`
`20
`
`25
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`014
`
`Facebook/Instagram Ex. 1004
`
`

`

`5,506,617
`
`10
`
`15
`
`20
`
`25
`
`7
`from the sensor color ?lter array to obtain a full resolution,
`color corrected image. Such data is programmed into a
`portion of the EPROM memory 86, which is not used to
`provide the white balance and gamma correction look~up
`tables, when the camera module 2 is manufactured. The data
`is downloaded from the EPROM memory 86 in the camera
`module to the RAM memory 62 in the computer. To down
`load the information, a multiplexer 85 is used to allow the
`computer to address the EPROM address least signi?cant bit
`(LSB) lines which are normally provided by the A/D con~
`verter 84, while the timing logic 80 provides MSB address
`values which contain the required data, instead of the MSB
`values which provide the white balance and gamma correc
`tion curveshape tables.
`The digitized signal processed by the LUT 86 is then
`provided to two sets of octal latches, 88a, 88b, and 90a, 90b.
`Each octal latch stores one pixel value. While one set (88a,
`88b or 90a, 90b) of latches is being loaded, the pixel values
`stored in the other set from the previous two pixels are read
`into the computer 4 through the 16-bit data lines D0-D15 on
`the bus-compatible connector 24. The camera module 2 is
`{controlled by signals generated by the computer 4 and
`applied to the address lines, which are decoded in an address
`decoder 92, and to the control lines of the bus 58.
`The operation of the image capture system of FIGS. 4, 5
`and 8 proceeds as shown in the ?ow chart of FIG. 10, in
`particular for a direct memory access (DMA) mode of
`operation. The user begins by connecting the camera module
`2 to the computer 4 via the connectors 3 and 24, and turns
`on the computer 4. This activates an application program.
`The user then opens a “camera” application program by
`using the general purpose user input 74 and selects capture
`parameters from a menu displayed on the LCD display 8. A
`typical application might utilize an electronic “smart form”,
`which requires data to be entered via the LCD display 8 and
`the stylus 6, and which also requires a picture. The capture
`parameters may include color (8, 16, or 24 bit) or mono
`chrome (1 or 8 bit), full resolution or subsampled, and store
`direct to disk, compress before storing, or convert to a
`standard metric before storing. Therefore, depending on the
`capture parameters selected, the data is either stored directly
`from RAM 62 to the hard disk 64 (or other non-volatile
`computer memory), compressed before storage using con
`ventional methods, or converted before storage to a full
`resolution color image.
`To take the picture, the user ?ips up the ?ash 16 and slides
`down the view?nder 20. This closes a hidden switch (not
`shown) which powers up the camera. The camera, using the
`photometer 48, measures the ambient light level and charges
`the ?ash 16 if the ambient light level is low. The user frames
`the image using the optical view?nder 20, and depresses the
`capture button 22. This begins a sequence which actuates the
`diaphragm 34, opens the shutter 36, ?res the ?ash 16 if
`required, and closes the shutter 36. One line of the image is
`then transferred from the vertical register of the CCD imager
`42 to the horizontal readout register 42a. Next, the timing
`logic 80 generates a “line-ready” interrupt of the computer’s
`CPU using a PC 110 bus interrupt line.
`In response to the interrupt, the CPU 54 instructs the
`DMA controller 68 to begin transfer of one line of data from
`60
`the camera module 2 over the bus 58 to the computer’ s RAM
`memory 62. The DMA transfer is controlled by the control
`line signals on the bus 58. The data is provided from the
`camera in groups of two 8 bit pixels at a time, to rapidly
`transfer the image from the CCD imager 42 to the computer
`RAM memory 62. At the end of each line of the image (768
`pixels for the Kodak KAF-0400 sensor), the timing logic 80
`
`30
`
`40
`
`45
`
`50
`
`55
`
`65
`
`8
`transfers another image line into the horizontal readout
`register 42a and then generates another “line-ready” inter
`rupt.
`In a second embodiment of the signal processor 52 shown
`in FIG. 9, the camera supplies data using only 8 bits out of
`the 16 data bits available on the PC/AT connector. In
`addition, a DRAM buffer memory 94, capable of storing one
`entire image from the CCD imager 42, is inserted between
`the EPROM 86 and the interface connector 24 to substitute
`for the four octal latches in FIG. 8. This sub