'
`
`I
`
`WV
`
`»9
`
`PTorsBns (3-9?)
`Please type a pine sign (+} inside this box .9
`Approved for use through 1i’31i'9B. OMB O65!-O03?
`Patent and Trademark Office: U_.S. DEPARTMENT OF QOMMEHCE
`,
`Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of infonnation unless it displays a
`vatid OMB control number.
`
`PRO VISIONAL APPLICA TION FOR PA TENT C0 I/EH SHEET
`This is a request tor iiiing a PROVISIONAL APPLICATION FOFI PATENT under 37 CFR 1.53 (b)(2).
`
`GWEN Name W51 and middle [If aniiii
`
`Famity Name or Surname
`
`INVENTOR(3]
`
`
`
`
`Residence
`(City and either State or Foreign Country]
`
`Boulder, COLORADO
`Boulder, COLORADO
`Arvada, COLORADO
`
`Ao‘o'rir'onar riwentom are being named on me }_ separarery numbered sheets ariamed rrerero
`TITLE OF THE INVENHON 230 characters max
`
`INTEGRATED INTERNET CAMERA
`
`
`
`
`
`
`
`Specification NumberofPages El Smaii Entity Statement
`
`Drawingis) Number ofshsers
`B Other (specify) |
`
`
` A check or money order is enclosed to oover the tiling tees
`
`The Commissioner is hereby authorized to charge iiting
`fees or credit any overpayment to Deposit Account Number. 19-0039
`
`Ml.-Tl'HOD OF PAYMENT OF FILING FEES FOR THIS PROVISIONAL APPLICATION FOP. PATENT (check one)
`FIUNG FEE
`AMOUNT S
`
` $150.00
`
`
`
`
`The invention was made by an agency oi the United States Government or under a contract with an agency of the
`United States Govemrnent.
`
`m No.
`D Yes. the name oi the LLS. Govemment agency and the Government contract number are-
`
`
`3
`EE
`73} 4 1'?)
`Date
`
`TYPED or PFHNTED NAME Bruce H‘ Ber” tein
`”_EG'STR"?T‘°” “'0'
`29 ’ 027
`./if appropriate)
`Docket Number:
`
`Hespecrfuriy sub ' 90’.
`
`SIGNATURE
`
`TELEPHONE
`
`(703) 716-1191
`
`M5223
`
`USE ONL Y FOR FILING A PRO VISIONAL APPLICA TION F0!-7 PA TENT
`Burden Hour Statement: This Form is estimated to take 0.2 hours to complete. Time will vary depending u on the needs of the individual case.
`Any oomrrients on the amount of time you are required to complete this form should be sent to the
`hiei‘ lnionriaiion Ohicer. Patent and
`Trademark Office. Washington, DC 20231. DO NOT SEND FEES OH COMPLETED FORMS TO THIS ADDRESS. SEND TO: Box Provisionai
`
`I Application. Assistant Commissioner for Patents. Washington, DC 20231.
`
`0001
`0001
`
`Apple/Twitter
`Apple/Twitter
`Ex. 1004
`Ex. 1004
`IPR1 of U.S. Pat. No. 7,765,482
`IPR1 of U.S. Pat. No. 7,765,482
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Direct at‘! correspondence to:
`Customer Number
`
`CORRESPONDENCE ADDRESS
`
`""—'—*
`
`Place ‘C-iusrornar Number
`33, Code Labs; here
`
`7055
`Type Customer Number here
`GREENBLUM 5. BERNSTEIN, P.L.C.
`1941 Roland Clarke Place
`
`0g
`,F,,*.';,'f",i.‘,’[,’a. Name
`Address
`
`Address
`9-2*
`
`com
`
`
`
`716-1191
`ENCLOSED APPLICATION PARTS (check ail mar appiy;
`
`20191
`716-1180
`
`
`
`
`
`
`
`
`
`
`
`A
`
`iiiiiiiiiiiiiiiiiiiiiiiiiiii
`
`12/U4/97
`
`
`
`
`
`
`

`
`PROVISIONAL APPLICATION COVER SHEET
`Additional Page
`
`+
`
`PTO/SB/16 (3-97)
`Approved for use through 1/31/98. OMB 0651-0037
`Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
`Under the Paperwork Reduction AC1 of 1995, no persons are required to respond to a collection of information unless it displays a
`valid OMB control number.
`
`I Docket Number I Vl 6223
`
`lType a plus sign (+) .1 +
`inside this box
`--?
`
`INVENTOR(S)/APPLICANT(S)
`
`Given Name (first and middle [1! any])
`
`Family or Surname
`
`Yoshiyuki
`Richard
`
`ARAKI
`HELTON
`
`Residence
`(C1ty and either State or Foreign Country)
`
`Westminster, COLORADO
`Littleton, COLORADO
`
`.
`
`Number 2
`
`of
`
`2
`
`+
`
`0002
`
`

`
`TITLE OF THE INVENTION
`
`INTEGRATED INTERNET CAMERA
`
`INVENTORS
`
`Rob CREAMER
`
`Walter KNAPP
`
`Mark KOCH
`
`Yoshiyuki ARAKI
`
`Richard HELTON
`
`0003
`
`

`
`Vl6223.S01
`
`INTEGRATED INTERNET CAMERA
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`5
`
`The present invention relates to a digital camera, and more particularly, a camera
`
`capable of transmitting images over the Internet.
`
`2. Description of Background Information
`
`As the Internet (i.e., the worldwide inter-network, currently operated under TCP!IP:
`
`Transmission Control Protocol/Internet Protocol) gains more participants and becomes more
`
`10
`
`consumer-oriented, the demand for simplified ways of providing access to various media
`
`increases. A large portion ofthe new participants seek access to the "World Wide Web" (i.e.,
`
`a hypertext-driven global multimedia system, hereinafter the "Web"). Archives of digital
`
`images (photographs and motion video) are now ubiquitous. The demand for real-time or live
`
`video, whether motion video or still video, has different requirements, but has also become
`
`15
`
`strong. Needs in entertainment, advertising, education, security, traffic monitoring, weather
`
`monitoring, child care monitoring, and surveillance, as well as general consumer usage, have
`
`driven the creation of an initial wave of systems able to place a real-time image, or series of
`
`images, on the Internet and on the Web.
`
`However, the prior systems are complex and expensive, requiring the use of a general
`
`20
`
`purpose personal computer and a host of peripheral devices to place an image on the Internet
`
`or Web. The systems are typically large and lack portability.
`
`An example of such a prior system is shown in Fig. 1. A video camera 110 connects
`
`to a "frame grabber" peripheral card 112, hosted by the parallel bus 114 of a personal
`
`computer 122. The frame grabber card 112 decodes a frame of the analog video signal from
`
`25
`
`the video camera 110 into a digital image, and makes the digital image available to purpose(cid:173)
`
`designed software running on the computer 122. Typically, the purpose-designed software
`
`-1-
`
`0004
`
`

`
`V16223.S01
`
`eventually compresses the digital image into main memory using the main microprocessor
`
`of the personal computer 122. In order to upload the image to the Internet, the computer 122
`
`requires a serial port 118 and attached modem 120, which are hooked to the public telephone
`
`system 124. The personal computer 122 uses further software programs running in main
`
`5
`
`memory, which include at least a modem driver, telephone transmission protocol (e.g.,
`
`TCPIIP) driver, a telephone transmission protocol (e.g., PPP: Point-to-Point Protocol) driver,
`
`and an file transfer protocol (e.g., FTP: File Transfer Protocol) application, to connect to the
`
`modem 120, through the telephone system 124, and to an ISP (Internet Service Provider)
`
`128. Thereafter, the personal computer 122 may upload the compressed image to a shell
`
`10
`
`account available at the ISP 128.
`
`Costs for such a system may run to several thousand dollars. The computer 122 must
`
`be on-site, i.e., relatively close to the camera 110, and is large and relatively immobile. Since
`
`the system is an assembly of general-purpose components, and the computer 122 is usually
`
`dedicated to serving the camera 110, the system is redundant and has excess capabilities. In
`
`particular, multiple microprocessors/controllers, power supplies, and communication lines
`
`are necessary to operate the separate parts of the system. Moreover, such systems include
`
`many opportunities for error because of the many interfaces and communication links
`
`between discrete devices. Such error may occur as difficulties in setup and configuration and
`
`f~
`
`incompatibility between devices in operation.
`
`20
`
`3.Acronyms
`
`The following acronyms and abbreviations are used throughout the specification. For
`
`25
`
`brevity, the definitions are summarized as follows:
`
`xDSL
`ATM
`CCD
`CCTV
`DNS
`DNSs
`ExCA
`
`- (generic) Digital Subscriber Line
`-Asynchronous Transfer Mode
`- Charge Coupled Device
`-Closed Circuit Television
`-Domain Naming System
`-Domain Name Server
`- Exchangeable Card Architecture
`
`-2-
`
`0005
`
`

`
`V16223.SOI
`
`FTP
`HTML
`IrDA
`ISA
`ISDN
`ISP
`JPEG
`MIME
`NTSC
`PAL
`PCMCIA
`POTS
`ppp
`SMTP
`TCPIIP
`UDPIIP
`URL
`
`-File Transfer Protocol
`-Hypertext Markup Language
`- Infrared Data Association
`-Industry Standard Architecture
`-Integrated Services Digital Network
`- Internet Service Provider
`-Joint Photographic Experts Group
`-Multipurpose Internet Mail Extension
`-National Television System Committee
`- Phase Alternating Line
`-Personal Computer Memory Card International Association
`-Plain Old Telephone Service
`-Point-to-Point Protocol
`-Simple Mail Transfer Protocol
`- Transmission Control Protocol!Intemet Protocol
`-User Datagram Protocol/lnternet Protocol
`-Uniform Resource Locator
`
`5
`
`10
`
`15
`
`SUMMARY OF THE INVENTION
`
`Accordingly, it is an object of the invention to provide an inexpensive and efficient
`
`camera having all necessary components and functionality for transmission of real-time
`
`digital images to the Internet in a single, portable standalone apparatus (i.e., an embedded
`
`system), without requiring the use of an external controlling apparatus such as a personal
`
`computer.
`
`25
`
`It is a further object of the invention to provide a portable, standalone camera that
`
`may initiate and independently control transmission of digital images to the Internet, where
`
`the images become available to any authorized user on the Internet.
`
`The above objects are attained by providing an integrated Internet camera for
`
`transmitting digital images to an Internet address, including an image pickup, an optical
`
`30
`
`system for forming an image on the image pickup, and an image capturing circuit for
`
`capturing digital images from the image pickup. A network interface device connects to
`
`the Internet for transmission of the digital image files to the Internet, and a file transfer
`
`device communicates via the network interface device, with a destination shell account at
`
`-3-
`
`0006
`
`

`
`V16223.S01
`
`a predetermined Internet address and transfers the digital image files to the destination
`
`shell account according to a predetermined file transfer protocol. The digital image files
`
`in the destination shell account are then available to users accessing the Internet. A
`
`transport control device packetizes the digital image files according to a predetermined
`
`5
`
`Internet transport control protocol, and controls addressing of the packetized digital image
`
`files to the predetermined Internet address, while a transmission initiating device initiates
`
`a connection with the Internet via the transport control device and the network interface
`
`device. A first scheduling device, including timers, schedules transfer of the digital image
`
`files to the destination shell account by the transport control device and the file transfer
`
`10
`
`device. A microcontroller controls operations and communication between each of the
`
`recited devices, and a camera body houses therein all of the recited devices and the
`
`microcontroller.
`
`In another aspect of the invention, an Internet camera system for transmitting digital
`
`images via the Internet includes a destination shell account having a user directory at a
`
`15
`
`predetermined Internet address and an accessing device for accessing the user directory
`
`of the destination shell account via the Internet. As part of the system, an integrated
`
`Internet camera is housed in a camera body. The camera body contains an image capturing
`
`system, a network interface device, a file transfer device, a transport control device, and
`
`a transmission initiating device. The image capturing system captures digital images, and
`
`20
`
`the network interface device is connectible to the Internet for transmission of the digital
`
`image files to the Internet. The file transfer device communicates, via the network
`
`interface device, with the destination shell account and transfers the digital image files to
`
`the user directory of the destination shell account according to a predetermined file
`
`transfer protocol. The digital image files in the user directory of the destination shell
`
`25
`
`account are then available to the accessing device accessing the Internet. A transport
`
`control device packetizes the digital image files according to a predetermined Internet
`
`-4-
`
`0007
`
`

`
`Vl6223.S01
`
`transport control protocol, and controls addressing of the packetized digital image files to
`
`the predetermined Internet address. The transmission initiating device initiates a
`
`connection with the Internet via the transport control device and the network interface
`
`device.
`
`5
`
`In this manner, the portable, standalone integrated Internet camera may initiate and
`
`independently control scheduled connections to the Internet and transmission of real-time
`
`digital images to the Internet, without requiring the use of an external controlling
`
`apparatus such as a personal computer or server, and the images become available to any
`
`authorized user on the Internet. As part of a system, the portable, standalone integrated
`
`10
`
`Internet camera may initiate and independently control scheduled connections to a
`
`destination shell account having a user directory at a predetermined Internet address and
`
`transmission of real-time digital images to the user directory, without requiring the use of
`
`an external controlling apparatus such as a personal computer or server, and the images
`
`become available to any authorized user on the Internet via the accessing device.
`
`15
`
`The network interface device may include a modem for connecting to a telephone
`
`system connected to the Internet. In this case, the transmission initiating device includes
`
`a telephone conversion device that initiates a telephone connection with the Internet via the
`
`modem according to a predetermined telephone transmission protocol, and that converts
`
`between the predetermined telephone transmission protocol and the predetermined Internet
`
`20
`
`transport control protocol. Accordingly, the integrated Internet camera may perform the
`
`recited functions over a public or private telephone network, or any network or connection
`
`using telephone transmission protocols or analog data transmission.
`
`The integrated Internet camera may include a second scheduling device, including
`
`timers, for scheduling image captures by the image capturing circuit. Accordingly, image
`
`25
`
`captures and image transmission may be scheduled at different times. In this case, the
`
`digital image files may include information representing a status of one or more timers.
`
`-5-
`
`0008
`
`

`
`V16223.S01
`
`The integrated Internet camera may further include a character generator for
`
`generating textual information in the captured digital images, wherein the character
`
`generator generates textual information in the captured digital images. In this case, the
`
`generated textual information may represent a status of one or more timers.
`
`5
`
`The integrated Internet camera may include a serial interface adapted to connect to
`
`a setup device, the serial interface receiving commands for controlling the integrated
`
`Internet camera from the connected setup device. In this manner, the integrated Internet
`
`camera may by controlled or configured by another device.
`
`Optionally, one or more of the transport control device and file transfer device
`
`10
`
`further includes a network authentication device for providing network login authentication
`
`for connecting to the predetermined Internet address via the network interface device. In
`
`this manner, the integrated Internet camera may access and transmit files to networks
`
`having security and authorization provisions.
`
`The integrated Internet camera may further include a configuration device, which
`
`15
`
`includes a configuration information retrieving device and a configuration setting device.
`
`The configuration information retrieving device retrieves configuration information from
`
`the destination shell account, while the configuration setting device sets operational
`
`parameters of one or more of the image capturing circuit, the network interface device,
`
`the file transfer device, transport control device, the transmission initiating device, and the
`
`20
`
`first scheduling device, according to the configuration information.
`
`Further, the file transfer device may further include a directory selecting device for
`
`setting and transmitting a destination directory and filename for transferring digital image
`
`files to the destination shell account, allowing the transmission of digital image files to one
`
`or more specific directories of a destination shell account.
`
`25
`
`The image pickup may include a color component system for forming a color
`
`-6-
`
`0009
`
`

`
`Vl6223.S01
`
`image, in which case the integrated Internet camera may further include a color adjusting
`
`circuit for adjusting color properties of the captured digital images.
`
`The integrated Internet camera may include an image compression circuit that
`
`generates compressed digital image files from the captured digital images, so that the file
`
`5
`
`transfer device transfers the compressed digital image files to the destination shell account
`
`and the transport control device packetizes the compressed digital image files according
`
`to the predetermined Internet transport control protocol.
`
`In one modification of the system, the predetermined Internet transport control
`
`protocol does not detect errors or retransmit erroneous data, thereby increasing a rate of
`
`10
`
`image transfer by the file transfer device.
`
`In another modification the integrated Internet camera further includes an E-mail
`
`transmission device and E-mail message assembler. The E-mail message assembler
`
`assembles E-mail messages representing a status of the camera and the E-mail transmission
`
`device transmits the E-mail messages to a predetermined E-mail address via the transport
`
`15
`
`control device and the transmission initiating device.
`
`In this case, the E-mail message assembler may assemble E-mail messages
`
`including the digital image files. Accordingly, the E-mail transmission device may
`
`transmits the E-mail message including the digital image files to a predetermined E-mail
`
`address via the transport control device and the transmission initiating device.
`
`20
`
`The integrated Internet camera may further include a trigger device linked to the
`
`camera and/or the microcontroller. In response to triggering of the trigger device, the
`
`camera initiates an image capture and transfer of the digital image files to the destination
`
`shell account via the file transfer device, the transport control device, and the transmission
`
`initiating device.
`
`25
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is further explained in the description which follows with
`
`-7-
`
`0010
`
`

`
`Vl6223.S01
`
`reference to the drawings, illustrating, by way of non-limiting examples, various
`
`embodiments of the invention, with like reference numerals representing similar parts
`
`throughout the several views, and wherein:
`
`Fig. 1 is a block diagram of a prior art system capable of transmitting digital images
`
`5
`
`to the Internet;
`
`Fig. 2 is a perspective view of an integrated Internet camera according to a first
`
`embodiment of the invention;
`
`Fig. 3 is a block diagram of the integrated Internet camera shown in Fig. 2;
`
`Figs. 4A and 4B are schematic diagrams of the integrated Internet camera of Fig.
`
`10
`
`2 connected to the Internet;
`
`Fig. 5 is a block diagram showing a menu and parameter storage structure of Fig.
`
`2·
`'
`
`Fig. 6 is a flow chart of an initialization routine of the integrated Internet camera
`
`shown in Fig. 2;
`
`l5
`
`Fig. 7 is a flow chart of a main routine of the integrated Internet camera shown in
`
`Fig. 2;
`
`Fig. 8 is a flow chart of an image capture routine of the integrated Internet camera
`
`shown in Fig. 2;
`
`Fig. 9 is a flow chart of an image transmit routine of the integrated Internet camera
`
`20
`
`shown in Fig. 2;
`
`Fig. 10 is a flow chart of a disconnect routine of the integrated Internet camera
`
`shown in Fig. 2;
`
`Fig. 11 is a flow chart of an file transfer connect routine of the integrated Internet
`
`camera shown in Fig. 2;
`
`25
`
`Fig. 12 is a flow chart of a telephone connect routine of the integrated Internet
`
`camera shown in Fig. 2;
`
`-8-
`
`0011
`
`

`
`V16223.S01
`
`Fig. 13 is a flow chart of a write file routine of the integrated Internet camera
`
`shown in Fig. 2;
`
`Fig. 14 is a flow chart of a batch upload routine of the integrated Internet camera
`
`shown in Fig. 2;
`
`5
`
`Fig. 15 is a flow chart of a reporting routine of the integrated Internet camera
`
`shown in Fig. 2;
`
`Fig. 16 is a flow chart of a setup routine of the integrated Internet camera shown
`
`in Fig. 2;
`
`Fig. 17 is a block diagram of a second embodiment of an integrated Internet camera
`
`10
`
`according to the invention;
`
`Fig. 18 is a block diagram of an addendum to the menu and parameter storage
`
`structure of Fig. 2 for the second embodiment of Fig. 17;
`
`Fig. 19 is a flowchart addendum to image capture routine of Fig. 8 for the second
`
`embodiment shown in Fig. 17;
`
`15
`
`Fig. 20 is a block diagram of a third embodiment of an integrated Internet camera
`
`according to the invention; and
`
`Fig. 21 is a block diagram of a fourth embodiment of an integrated Internet camera
`
`according to the invention.
`
`DESCRIPTION OF THE PREFERRED EMBODIMENTS
`
`20
`
`Fig. 2 shows a first embodiment of the present invention. All of the electronic,
`
`mechanical and optical components of the integrated Internet camera 1 are housed within a
`
`camera body 201. Accordingly, in the context of this specification, "integrated" is equivalent
`
`to "self-contained", such that all the noted components are supported on or situated within
`
`the body or casing. As shown in Fig. 2, the camera 1 may be connected to the Internet via a
`
`25
`
`network interface device 236 (comprising, e.g., a modem or network card) and a connection
`
`cable 23 7 (which may be a telephone wire connected to the public network or a network
`
`-9-
`
`0012
`
`

`
`V16223.S01
`
`cable connected to a local or wide area network).
`
`A viewfinder 244 allows the operator to view a scene corresponding to, or identical
`
`to, an image formed on an image pickup (shown in Fig. 3) of the camera 1 via an image(cid:173)
`
`forming optical system (shown in Fig. 3). A display (e.g., an LCD) 218, preferably an
`
`5
`
`inexpensive multi-line text display, displays the results of user interaction, automatic
`
`reporting, and status reporting to the user. The user may input appropriate directions to the
`
`camera 1 via a button/switch input 214, in the first embodiment preferably including up and
`
`down buttons 214a and 214b, a "menu" button 214d for switching between and activating
`
`interaction menus, an "item" button 214c for indicating a selection in an active interaction
`
`10
`
`menu, a release button 214e for initiating the capture of an image in a manual mode and other
`
`specific functions (described later).
`
`g
`
`Fig. 3 is a block diagram of the first embodiment of the integrated Internet camera 1.
`
`The camera 1 is preferably operated in an "always-on" state, i.e., although it may appear to
`
`an operator that the camera is unpowered when the camera is "turned off' by, e.g., an "on(cid:173)
`
`off'' switch, the camera 1 remains responsive to control signals and inputs even when "turned
`
`off'. As shown in Fig. 3, the camera 1 is preferably controlled by an integrated
`
`microcontroller 200, which includes: a main processor or microprocessor 201; a parallel
`
`(e.g., 16 bit ISA) bus 234 (which connects to components outside the microcontroller 200);
`
`a slot (e.g., PCMCIA) controller 202 for controlling a slot (e.g., PCMCIA) interface 232 on
`
`20
`
`the parallel bus 234, a memory (DRAM) controller 204 for controlling a general purpose
`
`(GP) memory (DRAM) 228 on the parallel bus 234, a display (e.g., LCD) controller 206 for
`
`controlling display functions of the display 218 connected thereto, a real-time clock
`
`(clock/timer) 208 against which timing and interval functions are measured, a serial/IrDA
`
`port 210 (e.g., serial interface) for connecting an external peripheral or computer (setup
`
`25
`
`device) with the microcontroller 200, interrupt controllers 213, and a keyboard controller 212
`
`for scanning the button/switch input 214.
`
`-10-
`
`0013
`
`

`
`V16223.S01
`
`The microcontroller 200 also integrates a plurality of general-purpose input/outputs
`
`219 (GPIO pins) and trigger inputs 211, each communicating with the main processor 201.
`
`As shown in Fig. 3, the GPIO pins 219 may be connected to various inputs and outputs, for
`
`example, an audio input 221 (MIC). Furthermore, the trigger inputs 211 may be connected
`
`5
`
`to external triggering devices 215 (e.g., motion sensors or trip switches) which send a
`
`"manual" release signal to the microcontroller 200 (as described below). It should be noted
`
`that the GPIO pins 219, since they may receive input signals, are capable of acting in the
`
`same manner as the trigger inputs 211. One integrated microcontroller suitable for use in the
`
`camera 1 is the Vadem VG330, an x86 compatible single-chip microcontroller having the
`
`10
`
`above-described components, available from Vadem, Inc., 1960 Zanker Rd., San Jose, CA.
`
`fs
`
`The Vadem microcontroller may run under an operating system incorporating the transport
`
`control protocol (e.g., TCPIIP) stack discussed herewithin. It should be noted that a
`
`microcontroller having a lower level of integration may be utilized, with any of the above(cid:173)
`
`noted integrated components provided off-chip. One suitable microcontroller with a lower
`
`level of integration is the H8/3437 (available from Hitachi Semiconductor (America), Inc.,
`
`6431 Longhorn Dr., Irving, Texas, 75063), used, for example, in combination with a Fuji
`
`MD8501 PCMCIA controller (with direct memory access), available from Fujifilm
`
`Microdevices Co., Ltd., 1-6, Matsusakadaira, Taiwa-cho, Kurokawa-gun, Miyashi, Japan
`
`981.
`
`20
`
`The slot (PCMCIA) controller 202 and interface 232, in combination, may be
`
`configured to handle at least PCMCIA 2.1 and ExCA standard cards, supporting a network
`
`interface device 236, as described below, as well as hot swapping and memory cards.
`
`As noted, the display 218 is an inexpensive multi-line display capable of displaying
`
`character or text information, and of responding to the control of the display controller 206.
`
`25
`
`The real-time clock 208 has both clock and timer functions, maintaining the current date and
`
`time, as well as responding to queries by returning the date and/ or time, starting and stopping
`
`-11-
`
`0014
`
`

`
`V16223.S01
`
`one or more interval timers, or returning the status of a given timer.
`
`The serialllrDA port 210 is provided with one or both of an infrared transceiver
`
`operating under the IrDA standard, or a serial interface (e.g., an RS-232C interface). The
`
`serial!IrDA port 210 is connectible to a portable computer 216 or setup device via cable or
`
`5
`
`infrared transceiver. The interrupt controllers 213 process interrupts from, e.g., the keyboard
`
`controller 212, memory controller 204, slot controller 202, serial/lrDA port 210, GPIO pins
`
`219, trigger inputs 211, or the parallel bus 234.
`
`The parallel bus 234 connects to: the microcontroller 200 for transferring control
`
`instructions and data; to a compression engine 224 for compressing captured images (further
`
`10
`
`connected to an image memory 220); the general purpose (GP) memory (DRAM) 228 used
`
`by the microcontroller 200 as storage and application space; a boot ROM 230 for booting the
`
`microcontroller 200 (i.e., self-test and 0/S retrieval); a color adjusting circuit 256 for
`
`performing image processing on a stored digital image; a character generator 254 for
`
`superimposing text information on a stored digital image; and the slot interface 232. The GP
`
`memory 228 is preferably at least 2 MB, and the image memory is preferably at least 512K.
`
`f5
`
`The compression engine 224 implements image compression in hardware, freeing the
`
`main processor 201 to perform other tasks. Preferably, the compression engine 224 performs
`
`image compression under a JPEG standard, but may be alternatively arranged to output other
`
`image formats (e.g., TIFF, GIF) and/or other compression schemes (e.g., Huffman, wavelet,
`
`20
`
`fractal). When JPEG is used as the standard, the compression engine 224 is able to encode,
`
`decode, and recode JPEG image files with any suitable JPEG compression level at 8-bit
`greyscale or 24-bit color (8 bit * 3 color planes).
`The color adjusting circuit 256 is preferably a dedicated circuit for performing image
`
`data manipulation of an image stored in the image memory 220. The color adjusting circuit
`
`25
`
`preferably includes: a color adjusting (gamma) module for performing a color correction on
`
`the stored image, e.g., to compensate for the color spectral characteristics (linearity) of the
`
`-12-
`
`0015
`
`

`
`V16223.S01
`
`image pickup (CCD); a brightness module for increasing or decreasing the overall brightness
`
`of the stored image; a contrast module for increasing or decreasing the overall contrast to the
`
`stored image; a scaling module for interpolating or resampling the stored image to increase
`
`or decrease the size of the stored image, including adjustment of an aspect ratio of the image;
`
`5
`
`a hue/saturation/luminance module for increasing or decreasing hue, saturation, and
`
`luminance of the stored image. Each of these modules may use a conventional algorithm to
`
`perform the desired correction or function.
`
`Although the color adjusting calculations are performed by the color adjusting circuit
`
`256, the color adjusting calculations may alternatively be performed by the compression
`
`10
`
`engine 224, or by the microcontroller 200 in combination with appropriate color adjusting
`
`applications, e.g., loadable from the NVRAM 242 into the GP memory 228.
`
`The character generator 254, upon receiving a character string (e.g., a date and/or time
`
`and/ or message string), generates bitmap characters according to an internally stored font,
`
`f5
`
`and changes values of memory positions in the image memory 220 (corresponding to colors
`
`of image coordinates within a stored image) to superimpose the text information on a stored
`
`digital image. The character generator may be set to invert all the pixels corresponding to the
`
`bitmap character in the stored image (to ensure the characters are visible), or to change all
`
`the pixels corresponding to the bitmap characters in the stored image to the same value
`
`(providing characters of a uniform color).
`
`20
`
`The microcontroller 200 is further connected to a serial controller 238 (e.g., an
`
`EEPROM controller) having a serial bus 240. A rewritable non-volatile memory (NVRAM)
`
`242 (e.g., an EEPROM), preferably at least 64K, is provided on the serial bus. The NVRAM
`
`stores system firmware, parameters, and applications for the camera 242, and is accessed by
`
`the microcontroller 200 at least according to the boot ROM 230, e.g., when the
`
`25
`
`microcontroller 200 is initialized. Preferably, the NVRAM 242 stores at least: a user
`
`interface/operating system application for controlling the microcontroller 200; an exposure
`
`-13-
`
`0016
`
`

`
`V16223.S01
`
`control application with automatic gain control (AGC) for controlling an exposure taken by
`
`an image pickup circuit 250; a transport control protocol stack for Internet access (e.g., a
`
`TCP/IP stack); a file transfer application (e.g., FTP application); and a driver (e.g., modem
`
`driver, network adapter driver) for the network interface device 236 connected to the slot
`
`5
`
`interface 232. One example of a suitable NVRAM 242 is a serial EEPROM of the NM24cxx
`
`series, available from National Semiconductor, Inc., 2900 Semiconductor Dr., Santa Clara,
`
`CA, 95051.
`
`The transport control protocol stack, as controlled by the microcontroller 200,
`
`packetizes all data transmitted under the transport control protocol (e.g., TCP liP) connection,
`
`10
`
`and inserts header information (including addressing information) into each packet.
`
`Accordingly, when the camera 1 is connected to the Internet via the network interface device
`
`236 under the transport control protocol, all transmissions, including those of image files, are
`
`packetized and addressed according to the transport control protocol.
`
`t5
`
`As previously discussed, one example of a standard protocol which may be provided
`
`in the transport control protocol stack is TCPIIP, a connection-oriented protocol that offers
`
`error reporting, prioritizing of data, and retransmission of lost or erroneous packets. In this
`
`model, the TCP layer accepts and segments data streams and passes the segments to IP for
`
`routing, and accepts segmented data from the IP layer, resolves error conditions, and
`
`resequences segments as necessary. The IP layer routes segmented data, resolves e