I|||||||||||||||||||||||||||||||||l|||||||||ll|||||||||||||||||||||||||||||
`
`USO069307U9I3l
`
`(13)
`
`United States Patent
`Creamer et al.
`
`(10) Patent No.:
`
`(45) Date of Patent:
`
`US 6,930,709 B1
`Aug. 16, 2005
`
`(54)
`
`INTEGRATED INTERNET,n"lN'l‘RANET
`CAMERA
`
`(75)
`
`Inventors: Rob Creamer, Boulder, (TO (US);
`Walter Knapp, Boulder, (TO (US);
`Mark Koch, Broomfield, CO (US);
`Yoshlyuki Araki. Sztitama-ken (JP);
`Richard Helton, Boulder, CO (US)
`
`(73)
`
`Assigneez PENTAX of America, Inc., Montvale,
`NJ (us)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term oftltis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(31)
`
`Appl. No.: 09,I'204,289
`
`(33)
`
`Filed:
`
`Dec. 3, 1998
`
`(60)
`
`(51)
`
`(53)
`
`(58)
`
`(56)
`
`Related U.S. Application Data
`Provisional application No. n‘30t'£|85_.585_. filed on May 15,
`1998, and provisional application No. 6Dt'D67_.31{1. filed on
`Dec. 4, I991
`
`U.S. CI.
`
`Int. CL?
`
`H04N 5,n’232; lI04N SL225;
`I-I04N IIKOO
`34812113; 3483207.];
`348552
`34-8f2U'?.l, 211.3,
`Field of Search
`34¢‘?-t'211.1, 211.8, 211.12, 552, 231.2, 231.6,
`14.01-l4.16, 36, 211.6, 154, 155, 158;
`3S8;’4()3; 7041203
`
`References Cited
`
`U.S. PA'l'EN'l‘ DOCUMENTS
`
`5,185.66? A
`5,283,655 A
`5,477,264 A
`5_.4?9_.2[)6 A ‘*
`5_.6l2_.732 A *
`5,f)33_.6't'8 A "
`5,635,983 A
`3.646.684 A *
`5,724,155 A "'
`
`M1993 Zintntermann
`2.n‘l994 Maeno
`12t’l99S Sarbadhijrari et al.
`I2.-"1?-J95 Ucno ct al.
`3.-‘I99? Yuyanta el al.
`5.31997 Parul-ski cl al.
`I').iI‘J<)7 Oltmori
`7t‘l‘J‘.)'i' Nishizawa el al.
`3.n"l‘)98 Saito
`
`348;-‘2ti..5
`348314.01
`348t'23l.5
`
`348!231.S
`3531402
`
`M1998 Takizawa el all.
`5_.'i'34,4l‘fi A
`8Jt998 Gullichsen etal.
`5,795,425 A
`5_.8l8_.53".7 A "‘ 1Ut"1998 Enokida ct al.
`5,8”i'tJ,I35 A
`2r19<J9 (ilatt ct al.
`5,887,144] A "
`3.31999
`Ilsumi elal.
`5,011,044 A * M1999 Lo ct al.
`5,9‘)(J,94t A
`ll."1‘J9<J
`Jackson el al.
`6_.E)(J5.6l1 A
`12t’1999 Gnllichsen et al.
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`3431441
`
`'.r'[)9f225
`‘i'()9;‘2t.I3
`
`JP
`JP
`
`tJ8lU2837 A “
`08171691 A "‘
`
`4;-‘I996
`7,-‘I996
`
`I104-NIUI/32
`......... ..GO8Br‘25;-‘(II
`
`OTHER PUBLICATIONS
`
`3.3 Connectionless Transport: UDP, Ross and Kurose; httptff
`wvvw—net.cs.u masseclutku roset’transp0rti"U DP.htmI;
`199 6-2000. ‘
`
`(Continued)
`
`Primer‘): Examiner—Wendy R. Garber
`Asnistant Exariiiner—John M. Villecco
`(74) Attorrtqv, Agent, or Firm—Greenblum & Bernstein,
`P.L.C.
`
`(57)
`
`ABSTRACT
`
`An integrated Internet camera includes, as embedded com-
`ponents contained within the camera body and controlled by
`a mieroeontroller, at
`least a network interface device for
`connecting to the Internet, a transport control device for
`packetizing according to Internet protocols, a file transfer
`device for communicating with a destination user directory
`on the Internet, and a
`transmission initiating device for
`initiating the connection and transfer operations of the file
`transfer device and transport control device. The network
`interface device may be a modem, network adapter, or
`adapter for connection to the Internet. Upon capturing the
`digital
`image,
`the camera initiates a connection to the
`Internet, connects to the destination user directory, and
`uploads the digital images. Thereafter, the digital images are
`available to authorized (or any) user having access to the
`Internet.
`
`91 Claims, 22 Drawing Sheets
`
`24.5
`
`
`
`Apple/Twitter
`Apple/Twitter
`Ex. 1003
`Ex. 1003
`IPR3 of U.S. Pat. No. 7,765,482
`IPR3 of U.S. Pat. No. 7,765,482
`
`0001
`
`

`
`US 6,930, 709 Bl
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`3/2000 Whiting eta!.
`6,034,716 A
`6,038,296 A * 3/2000 Brunson et a!.
`....... 379/100.11
`6,043,837 A
`3/2000 Driscoll, Jr. et a!.
`6,047,264 A * 4/2000 Fisher et a!. .................. 705/26
`6,061,502 A * 5!2000 Ho et a!.
`................... 358/1.15
`6,065,062 A * 5!2000 Periasamy et a!. .......... 709/242
`6,067,571 A * 5!2000 Igarashi et a!. ............. 709/232
`6,094,221 A
`7/2000 Andersion
`6,134,606 A
`10/2000 Anderson et a!.
`6,147,598 A * 11/2000 Murphy eta!. ........ 340/426.19
`6,167,469 A * 12/2000 Safai et a!.
`................... 710/62
`6,188,431 B1 * 2/2001 Oie ......................... 348/211.5
`6,195,511 B1
`2/2001 Harada
`6,204,877 B1 * 3/2001 Kiyokawa ................ 348/211.3
`6,208,426 B1 * 3/2001 Saito et a!.
`................ 358/1.15
`6,226,449 B1 * 5/2001 Inoue et a!.
`................ 386/120
`6,256,059 B1
`7/2001 Fichtner
`6,278,481 B1 * 8/2001 Schmidt .. ... .. ... ... ... ... .. . 348/64
`6,300,976 B1
`10/2001 Fukuoka
`6,331,869 B1
`12/2001 Furlan et a!.
`6,360,362 B1
`3/2002 Fichtner et a!.
`6,389,464 B1 * 5!2002 Krishnamurthy eta!. ... 709/220
`6,407,752 B1 * 6/2002 Harnett ....................... 345/744
`6,438,587 B2 * 8/2002 Kitamura .................... 709/217
`6,441,924 B1 * 8/2002 Matsui ....................... 358/403
`6,452,629 B1
`9/2002 Aizawa et a!.
`
`6,525,761 B2 * 2/2003 Sato eta!. ............... 348/14.04
`3/2003 Driscoll, Jr. et a!.
`6,539,547 B2
`6,567,122 B1 * 5!2003 Anderson et a!. ........ 348/211.3
`5!2003 Savitzky et a!.
`6,571,271 B1
`6,591,279 B1
`7/2003 Emens eta!.
`6,594,032 B1 * 7/2003 Hiroki et a!.
`6,603,502 B2
`8/2003 Martinet a!.
`6,624,846 B1
`9/2003 Lassiter
`6,636,259 B1
`10/2003 Anderson et a!.
`6,720,987 B2
`4/2004 Koyanagi et a!.
`2001!0024232 A1
`9/2001 Suzuki
`2003/0025803 A1
`2/2003 Nakamura et a!.
`2003/0208567 A1
`11/2003 Gross
`2004/0012811 A1
`1!2004 Nakayama
`
`.............. 358/1.15
`
`OTHER PUBLICATIONS
`
`P.M. Corcoran et al., "Internet Enabled Digital Photogra(cid:173)
`phy," IEEE, pp. 84-84.
`"NEC Picona digital camera, Instruction Manual for
`PC-DC200 and PC-DC200K", dated Feb. 1997.
`A verified English language translation of "NEC Picona
`digital camera, Instruction Manual for PC-DC200 and
`PC-DC200K", dated Feb. 1997.
`
`* cited by examiner
`
`0002
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 1 of 22
`
`US 6,930, 709 Bl
`
`..... ___ ,
`
`?a
`
`C'\.1
`C'\.1
`,..-
`
`I
`
`1-(cid:173)a:
`,.- <(
`a:
`0
`a:
`a...
`
`u..
`
`0 .....
`.....
`
`c:::l 0
`c:::l 0
`
`c:::l 0 Do
`ooD
`
`I ooDo
`000 oB
`I c:::l 0
`
`DDo
`
`c:::l 0
`c:::l 0
`
`.....
`ex:>
`.....
`
`C'\1
`
`,..-.....
`
`0003
`
`

`
`U.S. Patent
`
`Aug.16, 2005
`
`Sheet 2 of 22
`
`US 6,930,709 Bl
`
`.
`CJ
`u..
`
`\
`
`,..-
`0
`C\.1
`
`,..-
`C\.1
`
`0
`"<:::~"
`,..-
`C\.1
`
`0004
`
`

`
`CHARACTER
`GENERATOR ~-
`254
`
`COLOR
`256...- ADJUST
`~
`
`GP DRAM
`
`I
`_,
`228
`
`BOOT
`ROM
`
`)
`~
`230
`
`232
`rs=
`:; 1---------,
`
`,--,
`
`200
`\.___/
`
`NETWORK
`M
`t\ c
`INTERFACE
`DEVICE
`\
`I
`(POTS/ISDN/ 1--
`...,/ A
`V S CABLE MODEM;
`NIC) I
`L
`
`0 / ] ~
`236,...-
`
`T
`
`L-
`
`- - - - - - .
`POWER
`
`-
`
`(
`
`SUPPLY
`
`217
`ri'-.o'
`
`,
`'
`
`~ 18
`LCD
`DISPLAY~ 2
`
`d •
`\Jl
`•
`~
`~ ......
`~ = ......
`
`~
`~
`'""" ~~
`N c c
`
`Ul
`
`'JJ. =(cid:173)~
`~ .....
`
`~
`0 ......,
`N
`N
`
`e
`rJ'l
`-..a-..
`~
`Q
`~
`Q
`\0
`~
`1--"
`
`p
`f!
`
`I } 1?
`~~
`~__?01
`JI
`
`Ill
`I~
`ri;jl"---216
`Ill
`j
`E=siil
`lrDA!SERIAL to PC
`
`FIG 3
`
`•
`
`, 220
`
`IMAGE
`MEMORY
`
`CONTROLLER
`
`'
`
`2M
`
`\
`
`246
`
`1 r (226
`~ I DRAM
`)J
`L;1
`
`COMPRESSION
`ENGINE
`
`1 [ J..
`
`~
`2~4
`
`A
`
`C
`IMAGE
`Cf--1 PICKUP
`CIRCUIT ~ \1
`D
`.c.50
`•
`r1
`1
`~
`
`248
`
`,..--
`L (
`
`245
`
`\
`
`)
`
`I
`
`V
`
`I
`
`\
`
`/
`
`CCD
`
`DRIVER
`-
`
`1/
`
`t
`
`252
`
`PARALLEL CONTROUDATA BUS
`
`, r
`..J..j...
`
`~
`
`~02
`\ PCMCIA
`CONTROLLER
`
`20,...6
`LCD
`\
`CONTROLLER
`
`INTEGRATED MICROCONTROLLER
`21 0
`_( 204
`f}
`DRAM )
`CONTROLLER
`~·
`
`SERIAL
`PORT/irDA
`
`KEYBOARD
`CONTROLLER
`
`r
`I
`I
`
`CLOCK!
`, .... :_~M~~--·,
`238
`i T1 i i T4 i
`~·
`INTERRUPT
`'"'"< ·------
`_l
`.---SE-R-IA-L--)--,1 ~ ~-!~,) .------.
`CONTROLLER
`J .
`,
`i T3 i i Tn i ,..1\
`CONTROLLER
`. I '--208
`L213
`·------
`•------
`1 r
`lUlL UUUUU
`-:) ~~
`211 219
`
`SERIAL
`CONTROL
`BUS
`.------"'-------,
`NV RAM
`(UI, STACK,
`FTP,
`cARD DRIVER)
`__[__
`. -;!
`242
`
`BUTTON/
`SWITCH INPUT
`(
`
`TRIGGER
`(e.g., MOTION
`SENSOR,
`r~ EXTERNAL
`215
`DEVICE)
`
`)
`214
`
`MIC
`221
`
`0005
`
`

`
`FIG. 4A
`
`FIG. 48
`
`liEf
`
`~
`1
`
`C>
`C>
`
`- - -&310
`
`BROWSER
`
`~10
`
`BROWSER
`
`~-- -·---
`I
`I
`SHELL
`I
`#1
`I
`I
`L-,---1
`' .. - 306
`
`~ 310
`
`312
`
`SHELL
`#2
`
`I
`I
`I
`I
`I
`I
`I
`L:c----1
`314
`
`~10
`,......._______ ~10
`
`BROWSER
`
`BROWSER
`------1
`I
`1
`SHELL
`I
`I
`#2
`-L 314
`~312 I
`L-----1
`
`~---·-- -1
`306 _.( SHELL
`:
`I
`I
`#1
`I
`I
`L _____ l
`
`310 g
`
`BROWSER
`
`~ •
`00
`• ;p
`......
`~ = ......
`
`~
`~
`"""" ~~
`N
`~
`
`rJJ. ::r
`~ ....
`
`~
`
`~
`0 .......
`N
`N
`
`e r;r;.
`
`-..C1'1
`~
`-..~
`-.....1
`~
`'1.0
`~
`"""""
`
`0006
`
`

`
`FIG. 5
`liMAGE FILES j}j
`············· .... ._..,
`~
`...
`
`J
`
`I
`
`MENU & PARAMETERS
`
`I
`I MISCOPTION Jn
`
`I
`I COMMUNICATIONS Th
`
`REPORTING 1,,
`
`FILE 9]
`
`FILE DEFINITION
`-file name
`- dest. directory
`- overwrite {YIN)
`- retries, Interval (#, l)
`- start, stop memory
`address, slz:e
`-thumbnail (YIN, #)
`
`FILE DEFINITION
`-filename
`- desl. directory
`_- overwrlle (YIN)
`- retries, Interval (#, l)
`- start, stop memory
`address, slz:e
`-thumbnail (YIN,#)
`
`H~PLOAD
`- lmmedialelbalch(IIB)
`
`-{UPLOAD
`- lmmedlatelbalch(IIB)
`
`STAMPING
`• type, slz:e, position
`- Annotallon(texl)
`• Headerllmage
`
`e.-
`
`STAMPING
`• type, slz:e, position
`- Annolallon(texl)
`- Header/Image
`
`IMAGE ADJUST
`- gamma (param)
`- brlghlness(param)
`• conlrast(param)
`- scale(param)
`- hue(param)
`- saturellon(param)
`- compression(param)
`- resolullon(param)
`- grey/color(param)
`- crop (param)
`
`IMAGE ADJUST
`• gamma (param)
`• brlghlness(param)
`• conlrast(param)
`- scale(param)
`-- hue(param)
`- saturallon(param)
`- compresslon(param)
`resoluUon(param)
`- grey/color(param)
`- crop (param)
`
`1
`HARDWARE SEITINGS
`- currenlllme/dale
`- aclive triggers (param)
`- Image source (param)
`
`l
`MANUFACTURER
`- firmware version (text)
`- serial no. (text)
`- debug settings (param)
`
`.•
`
`L
`
`RESETS
`-Interval (param. list)
`-event (param. list)
`• reset list (param. list)
`
`ADAPTIVE
`·change compression If
`data rate slow (param)
`- change to DNS 2 if data
`rate slow (param)
`- change to tel. no. 2 If
`data rate slow (parem)
`
`1
`
`BATCH
`-interval (Start;DIH:M:S)
`- dale/lime (Off)
`
`l
`
`TELEPHONE
`- primary (lei. no., rei string)
`- secondary (lei. no., ret string)
`- retries(#)
`- good connection speed
`- opllons (param)
`
`CAMERA ADDRESS
`- dynamlclslallc (D/S)
`• local IP(IP address)
`- SMTP address (param)
`- SMTP userldlpassword
`
`TRANSMISSION
`-primary DNS(IP address)
`-second DNS(IP address)
`-network security {YIN)
`- network userld, password
`- conf/dlalup (C/D)
`- retries wl Interval (#, l)
`-timeout (l)
`-return strings (#, text)
`
`•·
`
`MODEM/LAN
`- setup string(strlng)
`- selftesl(Y/N)
`- LAN options (param)
`
`ADDRESS
`- destination address
`(email address)
`- delall(brlef/verbose)
`
`l
`
`REPORTS
`- uploads(Y /N)
`- errors(YIN)
`-parameter change (YIN)
`• retries wl Interval (#, T)
`-attachment (params)
`
`ERROR REPORTING
`- login failure(Y/N)
`- rale too slow(ratef'(IN)
`- gen flO failure(Y/N)
`- FTP error(YIN)
`-modem failure(Y/N)
`- reset(Y /N)
`-pan/lilt error(YIN)
`
`TIMER
`-lnlervai(Slart;DIH:M:S)
`- date/lime (DIT list)
`- manual slot (YIN)
`
`TIMER
`c...~· lntervai(Slart;D/H:M:S)
`-date/lime (DIT list)
`-manual slot (YIN)
`
`AUTOCONFIGURE
`-gel selup;when(Y/N; BIF)
`- setup file directory
`
`FILE TRANSFER
`- host address(IP or URL)
`-file transfer useriD,passwd
`- retries w/lnlerval (#, l)
`
`d •
`\Jl
`•
`~
`~ .......
`
`~ = .......
`
`~
`~
`'""" ~~
`N c c
`
`Ul
`
`'JJ. =(cid:173)~
`~ .....
`Ul
`0 ......,
`N
`N
`
`e
`\Jl
`-..a-..
`~
`Q
`~
`Q
`\0
`~
`"'"""
`
`0007
`
`

`
`U.S. Patent
`
`Aug. 16, 2005
`
`Sheet 6 of 22
`
`US 6,930, 709 Bl
`
`FIG. 6
`
`s 10
`
`ON
`,,
`BOOT/
`INITIALIZE
`
`81 2
`
`~,
`
`MAIN
`
`~,
`
`OFF
`
`0008
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 7 of 22
`
`US 6,930, 709 Bl
`
`FIG. 7
`
`(
`
`MAIN
`
`)
`
`S14
`
`ENABLE USER
`INTERVENTION
`
`.____N-< SIG~16
`~
`
`ILOCAUREMOTE
`~COMMAND
`
`LOCAUREMOTE
`SETUP SlGNAL
`,.-,----L-T'"'"'"1
`
`LOCAUREMOTE
`RELEASE SIGNAL
`
`BATCH
`SIGNAL
`
`ERROR
`
`BATCH
`UPLOAD
`
`RECORD
`ERROR
`
`S20
`
`COMMAND
`
`SETUP
`
`CAPTURE
`
`S19
`
`S18
`
`l
`
`S22
`
`TRANSMIT
`
`S23
`
`REPORTING
`
`S24
`
`RESET
`IF NECESSARY
`
`S27
`
`S25
`
`OFF?
`
`y
`
`RETURN)
`
`N
`
`0009
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 8 of 22
`
`US 6,930, 709 Bl
`
`FIG. 8
`
`T
`
`S28
`
`ASSIGN
`IMAGE SLOT
`ACCORDING
`TO
`TIMER
`
`ASSIGN
`NEXT
`AVAILABLE 30
`EVENT
`IMAGE SLOT
`
`GET
`EXPOSURE
`
`S32
`
`ceo
`CAPTURE TO
`IMAGE MEMORY
`
`834
`
`IMAGE
`ADJUST
`
`S36
`
`JPEG COMPRESS S3B
`TOGP DRAM
`
`0010
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 9 of 22
`
`US 6,930, 709 Bl
`
`FIG. 9
`
`ASSIGN
`FILENAME 842
`
`FTP
`CONNECT
`
`844
`
`FTP
`WRITE
`
`DISCONNECT
`
`0011
`
`

`
`U.S. Patent
`
`Aug. 16, 2005
`
`Sheet 10 of 22
`
`US 6,930, 709 Bl
`
`FIG. 10
`
`DISCONNECT
`
`852
`DISCONNECT TCP/IP,
`PPP, MODEM AS
`NECESSARY
`
`0012
`
`

`
`U.S. Patent
`
`Aug. 16, 2005
`
`Sheet 11 of 22
`
`US 6,930, 709 Bl
`
`FIG. 11
`
`TCP/IP
`Y
`~--< CONNECTED?
`
`854
`
`TCP/IP (PPP)
`CONNECT/
`LOGIN, COUNT
`RETRIES, ERROR &
`RETURN IF
`RETRIES
`EXCEEDED
`
`N
`
`S58
`
`FTP
`CONNECT, COUNT
`RETRIES, ERROR &
`RETURN IF
`RETRIES
`EXCEEDED
`
`N
`
`RETURN
`
`0013
`
`

`
`U.S. Patent
`
`Aug. 16, 2005
`
`Sheet 12 of 22
`
`US 6,930, 709 Bl
`
`FIG. 12
`
`ppp
`
`ENABLE MODEM,
`DIAUREDIAL
`
`S64
`
`N
`
`ppp
`CONNECT
`
`S70
`
`RETURN
`
`0014
`
`

`
`U.S. Patent
`
`Aug. 16, 2005
`
`Sheet 13 of 22
`
`US 6,930, 709 Bl
`
`FIG. 13
`
`WRITE FILE
`
`,,
`
`872
`
`GET FILE INFO FROM
`SERVER
`
`WRITE/OVERWRITE
`FILE FROM SLOT N
`ACCORDING TO FTP
`DIRECTORY & ATIRIBUTE
`
`S?4
`
`~
`DOWNLOAD NEW
`SETUP FILE
`IF NECESSARY &
`FTP LOGOUT/DISCONNECT
`
`S75
`
`RETURN
`
`0015
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 14 of 22
`
`US 6,930, 709 Bl
`
`FIG. 14
`
`FTP
`CONNECT
`
`S76
`
`ASSEMBLE BATCH LIST S78
`
`WRITE/OVERWRITE
`FILES FROM BATCH LIST
`ACCORDING TO FTP
`DIRECTORY &ATIRIBUTE
`
`S80
`
`DOWNLOAD NEW
`SETUP FILE
`IF NECESSARY
`
`S81
`
`DISCONNECT
`
`0016
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 15 of 22
`
`US 6,930, 709 Bl
`
`FIG. 15
`
`N
`
`N
`
`RETURN
`
`S94
`
`N
`TCP/IP
`CONNECT/LOGIN
`(WI RETRIES)
`
`S96
`
`N
`
`y
`ASSEMBLE SMTP
`MESSAGE FROM
`REPORTING BUFFER
`
`S98
`
`SEND SMTP MESSAGE S1 00
`
`c
`
`D
`DISCONNECT TCP/IP,
`PPP, MODEM AS
`NECESSARY
`
`S104
`
`0017
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 16 of 22
`
`US 6,930, 709 Bl
`
`FIG.
`16A
`
`SETUP
`
`8106
`
`SETUP
`SIGNAL?
`
`BUTION/SWITCH
`
`REMOTE
`
`...
`RECEIVE
`NEW
`PARAMETERS/
`FIRMWARE
`
`8120
`
`WRITE
`NEW
`PARAMETERS/
`FIRMWARE
`
`8122
`
`8100~
`
`WHICH
`BUTTON?
`
`.
`
`MENU
`
`ITEM
`
`UP/ON
`
`REL
`
`DISPLAY &
`CHANGE
`MENU ITEM
`
`DISPLAY &
`CHANGE
`MENU
`LEVEL
`
`DISPLAY &
`CHANGE
`PARAMETER
`
`STORE
`
`8110
`
`8112
`
`S114
`
`S118
`
`RETURN
`
`0018
`
`

`
`U.S. Patent
`
`Aug. 16,2005
`
`Sheet 17 of 22
`
`US 6,930, 709 Bl
`
`FIG. 168
`
`(
`
`COMMAND
`
`READ COMMAND &
`DATA
`
`8130
`
`...
`
`8132
`
`N~AND?
`
`PARAMETER SET
`
`TASK EXECUTION
`
`ERROR
`
`1 8134
`SET
`PARAMETER
`
`8136
`
`8138
`
`EXECUTE
`
`RECORD
`ERROR
`
`(RETURN)
`
`0019
`
`

`
`248 A
`i
`,...--L----, h L
`~0 (
`ZOOM I I FOCUS II APERTURE
`
`IMAGE
`CH PICKUP
`
`g
`
`CIRCUI~250
`
`270
`
`268
`
`266
`
`(I
`
`DRIVE
`
`DRIVE
`
`DRIVE
`
`26l t 264/.
`
`I
`
`STROBE
`DRIVE
`
`272
`
`FIG.
`17
`
`I J
`IMAGE
`220-----) MEMORY
`
`• GENERATOR J254
`
`I CHARACTER
`~T COl
`
`256~DJUST
`2''1111110
`
`I GP DRAM I
`(
`228
`
`'i_
`
`BOOT
`ROM
`
`~
`~
`230
`
`I
`
`COMPRESSION
`1
`ENGINE
`
`~ ~ ' "
`
`PARALLEL CONTROUDATA BUS
`
`I
`
`I--
`
`308
`
`Internet
`
`CCD
`DRIVER
`I
`
`25{
`
`CLOCK/
`TIMER
`i"i'i''! !"i'4"!
`rT2-1 ·······
`c~~:1
`lUU) UUUUUI
`
`211 (219
`
`SERIAL
`CONTROL
`BUS 215
`2
`(TRIGGER
`(e.g., MOTION
`SENSOR,
`EXTERNAL
`DEVICE}
`
`242
`
`I)
`
`NV RAM
`(UI,TCP/IP STACK,
`FTP,
`CARD DRIVER}
`
`-, r
`
`BUTTON/
`SWITCH INPUT
`..i
`
`li
`D
`
`LCD
`DISPLAY
`
`I /'\
`
`'-Y218'
`
`I. ~216
`
`lrDNSERIAL to PC
`
`MIG
`
`215
`
`(_214
`
`d •
`\Jl
`•
`~
`~ ......
`~ = ......
`
`~
`~
`""" ~~
`N c c
`
`Ul
`
`'JJ. =(cid:173)~
`~ .....
`""" 00
`0 .....,
`N
`N
`
`e
`rJ'l
`-..a-..
`~
`Q
`~
`Q
`\0
`~
`1--"
`
`0020
`
`

`
`U.S. Patent
`
`Aug. 16, 2005
`
`Sheet 19 of 22
`
`US 6,930, 709 Bl
`
`,.-
`C")
`(/J
`
`{
`
`)
`w
`D:l
`0
`-
`-a:
`!--
`_j~!-
`1:::0(1)
`ZO>-
`~No
`a..
`<(
`w
`0:::
`
`N
`
`M en
`0
`1-
`
`Q)
`,.-
`
`~
`lL
`
`0
`
`M en
`~
`0
`~
`LL.
`
`co
`.,--
`
`CJ
`LL
`
`en w
`...J
`u..
`w
`(.!)
`~
`:!:
`
`0021
`
`

`
`FIG.20
`
`CHARACTER
`GENERATOR I 254
`mlnR ITi
`
`GP DRAM
`
`232
`
`COMPRESSION
`ENGINE
`
`--1--._
`
`I I <-C...U
`
`I I 230
`
`224
`
`PARALLEL CONTROUDATA BUS
`
`222
`
`NTSC/PAL
`TO
`YCbCr
`
`POWER
`SUPPLY
`
`217
`
`308
`
`LCD
`DISPLAY
`
`218
`
`212
`
`216
`
`BUTTON/
`SWITCH INPUT
`
`21ll-
`
`lrDAISERIAL to PC
`
`242
`
`NV RAM
`(UI,TCP/IP STACK,
`FTP,
`CARD DRIVER)
`
`215
`
`TRIGGER
`(e.g., MOTION
`SENSOR,
`EXTERNAL
`DEVICE)
`
`d •
`\Jl
`•
`~
`~ ......
`~ = ......
`
`~
`~
`"""' ~~
`N c c
`
`Ul
`
`'JJ. =(cid:173)~
`~ .....
`N c
`0 ......,
`N
`N
`
`e
`rJ'l
`-..a-..
`~
`-..=
`""-l
`Q
`\0
`~
`1--"
`
`0022
`
`

`
`FIG. 21
`
`220
`
`245 ()
`
`230
`
`BOOT
`ROM
`
`PARALLEL CONTROUDATA BUS
`
`234
`
`INTEGRATED MICROCONTROLLER
`202
`
`r. 2,....04 __ ___,
`
`I
`
`SERIAL
`CONTROL
`BUS
`
`242
`
`NV RAM
`(UI,TCP/IP STACK,
`FTP,
`CARD DRIVER)
`
`215
`
`TRIGGER
`(e.g., MOTION
`SENSOR,
`EXTERNAL
`DEVICE)
`
`232
`
`236
`
`NElWORK
`INTERFACE
`DEVICE
`(POTS/ISDN/
`CABLE MODEM;
`NIC)
`
`308
`
`2164
`
`2166
`
`PC
`
`I
`
`d •
`\Jl
`•
`~
`~ ......
`~ = ......
`
`~
`~
`'"""' ~~
`N c c
`
`Ul
`
`'JJ. =(cid:173)~
`~ .....
`N
`
`'"""' 0 ......,
`
`N
`N
`
`e
`
`rJ'l
`0'1
`
`~ Q
`
`~
`Q
`\0
`~
`J-ooO
`
`0023
`
`

`
`FIG.
`22
`
`TO SERIAL CONTROLLER
`
`CAMCORDER OR CCTV
`
`248
`
`~--··
`~
`: 246 I
`
`2...
`
`I
`
`J
`
`PARALLEL CONTROUDATA BUS
`
`232
`
`236
`
`NETWORK
`INTERFACE
`DEVICE
`(POTS/ISDN/
`CABLE MODEM;
`NIC)
`
`219
`
`~ 238
`
`242
`
`NV RAM
`(UI,TCP/IP STACK,
`FTP,
`CARD DRIVER)
`
`TELEVISION
`2 f6 OR CCTV NETWORK
`
`d •
`\Jl
`•
`~
`~ ......
`~ = ......
`
`~
`~
`"""" ~~
`N c c
`
`Ul
`
`'Jl =(cid:173)~
`~ .....
`N
`N
`0 ......,
`N
`N
`
`e
`rJ'l
`-..a-..
`~
`Q
`~
`Q
`\0
`~
`1--"
`
`0024
`
`

`
`US 6,930,709 Bl
`
`1
`INTEGRATED INTERNET/INTRANET
`CAMERA
`
`The present application claims the benefit of U.S. Pro(cid:173)
`visional Application No. 60/067,310, filed Dec. 4, 1997, and
`U.S. Provisional Application No. 60/085,585, filed May 15,
`1998, which are expressly incorporated herein by reference
`in their entireties.
`
`BACKGROUND OF THE INVENTION
`
`5
`
`10
`
`2
`functions and excess capabilities. In particular, multiple
`microprocessors/controllers, power supplies, and communi(cid:173)
`cation lines are necessary to operate the separate parts of the
`system. Moreover, such systems include many opportunities
`for error because of the many interfaces and communication
`links between discrete devices. Such error may occur as
`difficulties in setup and configuration and incompatibility
`between devices in operation.
`3. Acronyms
`The following acronyms and abbreviations are used
`throughout the specification. For brevity, the definitions are
`summarized as follows:
`xDSL---(generic) Digital Subscriber Line
`ATM-Asynchronous Transfer Mode
`CCD-Charge Coupled Device
`CCTV-Closed Circuit Television
`DNS-Domain Naming System, Domain Name Server
`ExCA-Exchangeable Card Architecture
`FTP-File Transfer Protocol
`HTML-Hypertext Markup Language
`IrDA-Infrared Data Association
`!SA-Industry Standard Architecture
`ISDN-Integrated Services Digital Network
`ISP-Internet Service Provider
`JPEG-Joint Photographic Experts Group
`MIME-Multipurpose Internet Mail Extension
`NTSC-National Television System Committee
`PAL-Phase Alternating Line
`PCMCIA-Personal Computer Memory Card Interna-
`tional Association
`POTS-Plain Old Telephone Service
`PPP-Point-to-Point Protocol
`SLIP-Serial Link Interface Protocol
`SMTP-Simple Mail Transfer Protocol
`TCP liP-Transmission Control Protocol/Internet Proto-
`col
`UDP/IP-User Datagram Protocol/Internet Protocol
`URL-Uniform Resource Locator
`USE-Universal Serial Bus
`
`SUMMARY OF THE INVENTION
`
`15
`
`1. Field of the Invention
`The present invention relates to an integrated camera for
`connecting to the Internet and transmitting images over the
`Internet.
`2. Description of Background Information
`As the Internet (i.e., the worldwide inter-network, cur(cid:173)
`rently operated under TCP/IP: Transmission Control
`Protocol/Internet Protocol) gains more participants and
`becomes more consumer-oriented, the demand for simpli- 20
`fied ways of providing access to various media increases. A
`large portion of the new participants seek access to the
`"World Wide Web" (i.e., a hypertext-driven global multi(cid:173)
`media system, hereinafter the "Web"). Archives of digital
`images (photographs and motion video) are now ubiquitous. 25
`The demand for real-time or live video, whether motion
`video or still video, has different requirements, but has also
`become strong. Needs in entertainment, advertising,
`education, security, traffic monitoring, weather monitoring,
`child care monitoring, and surveillance, as well as general 30
`consumer usage, have driven the creation of an initial wave
`of systems able to place a real-time image, or series of
`images, on the Internet and on the Web.
`However, the prior systems are complex and expensive,
`requiring the use of a general purpose personal computer 35
`and a host of peripheral devices to place an image on the
`Internet or Web, as well as attendance by a qualified opera(cid:173)
`tor. The systems are typically large and lack portability.
`An example of such a prior system is shown in FIG. 1. A
`video camera 110 connects to a "frame grabber" peripheral 40
`card 112, hosted by the parallel bus 114 of a personal
`computer 122. The frame grabber card 112 decodes a frame
`of an analog video signal from the video camera 110 into a
`digital image, and makes the digital image available to
`purpose-designed software running on the computer 122. 45
`Typically, the purpose-designed software eventually com(cid:173)
`presses the digital image into main memory using the main
`microprocessor of the personal computer 122. In order to
`upload the image to the Internet, the computer 122 requires
`a serial port 118 and attached modem 120, which are hooked 50
`to the public telephone system 124. The personal computer
`122 uses further software programs running in main
`memory, which include at least a modem driver, network
`transmission protocol (e.g., TCP/IP) driver, a telephone
`transmission protocol (e.g., PPP: Point-to-Point Protocol) 55
`driver, and an file transfer protocol (e.g., FTP: File Transfer
`Protocol) application, to connect to the modem 120, through
`the telephone system 124, and to an ISP (Internet Service
`Provider) 128. Thereafter, the personal computer 122 may
`upload the compressed image to a shell account available at 60
`the ISP 128.
`Costs for such a system may run to several thousand
`dollars. The computer 122 must be on-site, i.e., relatively
`close to the camera 110, and is large and relatively immo(cid:173)
`bile. Since the system is an assembly of general-purpose
`components, and the computer 122 is usually dedicated to
`serving the camera 110, the system has numerous redundant
`
`Accordingly, it is an object of the invention to provide an
`inexpensive and efficient camera having all necessary func(cid:173)
`tionality for transmission of real-time and stored digital
`images to the Internet in a single, portable standalone
`apparatus (i.e., an embedded system), without requiring the
`use of an external controlling apparatus such as a personal
`computer.
`It is a further object of the invention to provide a portable,
`standalone camera that may initiate and independently con(cid:173)
`trol scheduled transmission of digital images to the Internet,
`where the images become available to any authorized user
`on the Internet.
`The above objects are attained by providing an integrated
`Internet camera for transmitting digital images to an Internet
`address, including an image pickup, an optical system for
`forming an image on the image pickup, and an image
`capturing circuit for capturing digital images from the image
`65 pickup. A network interface device connects to the Internet
`for transmission of the digital image files to the Internet, and
`a file transfer device communicates via the network interface
`
`0025
`
`

`
`US 6,930,709 Bl
`
`20
`
`3
`device, with a destination shell account at a predetermined
`Internet address and transfers the digital image files to the
`destination shell account according to a predetermined file
`transfer protocol. The digital image files in the destination
`shell account are then available to users accessing the 5
`Internet. A transport control device packetizes the digital
`image files according to a predetermined Internet transport
`control protocol, and controls addressing of the packetized
`digital image files to the predetermined Internet address,
`while a transmission initiating device initiates a connection 10
`with the Internet via the transport control device and the
`network interface device. A first scheduling device, includ(cid:173)
`ing timers, schedules transfer of the digital image files to the
`destination shell account by the transport control device and
`the file transfer device. A microcontroller controls opera- 15
`tions and communication between each of the recited
`devices, and a camera body houses therein all of the recited
`devices and the microcontroller.
`In another aspect of the invention, an Internet camera
`system for transmitting digital images via the Internet
`includes a destination shell account having a user directory
`at a predetermined Internet address and an accessing device
`for accessing the user directory of the destination shell
`account via the Internet. As part of the system, an integrated
`Internet camera is housed in a camera body. The camera 25
`body contains an image capturing system, a network inter(cid:173)
`face device, a file transfer device, a transport control device,
`and a transmission initiating device. The image capturing
`system captures digital images, and the network interface
`device is connectible to the Internet for transmission of the 30
`digital image files to the Internet. The file transfer device
`communicates, via the network interface device, with the
`destination shell account and transfers the digital image files
`to the user directory of the destination shell account accord(cid:173)
`ing to a predetermined file transfer protocol. The digital 35
`image files in the user directory of the destination shell
`account are then available to the accessing device accessing
`the Internet. A transport control device packetizes the digital
`image files according to a predetermined Internet transport
`control protocol, and controls addressing of the packetized 40
`digital image files to the predetermined Internet address. The
`transmission initiating device initiates a connection with the
`Internet via the transport control device and the network
`interface device.
`In this manner, the portable, standalone integrated Inter- 45
`net camera may initiate and independently control scheduled
`connections to the Internet and transmission of real-time
`digital images to the Internet, without requiring the use of an
`external controlling apparatus such as a personal computer
`or server, and the images become available to any authorized 50
`user on the Internet. As part of a system, the portable,
`standalone integrated Internet camera may initiate and inde(cid:173)
`pendently control scheduled connections to a destination
`shell account having a user directory at a predetermined
`Internet address and transmission of real-time digital images 55
`to the user directory, without requiring the use of an external
`controlling apparatus such as a personal computer or server,
`and the images become available to any authorized user on
`the Internet via the accessing device.
`The network interface device may include a modem for 60
`connecting to a telephone system connected to the Internet.
`In this case, the transmission initiating device includes a
`telephone conversion device that initiates a telephone con(cid:173)
`nection with the Internet via the modem according to a
`predetermined telephone transmission protocol, and that 65
`converts between the predetermined telephone transmission
`protocol and the predetermined Internet transport control
`
`4
`protocol. Accordingly, the integrated Internet camera may
`perform the recited functions over a public or private
`telephone network, or any network or connection using
`telephone transmission protocols or analog data transmis(cid:173)
`sion.
`The integrated Internet camera may include a second
`scheduling device, including timers, for scheduling image
`captures by the image capturing circuit. Accordingly, image
`captures and image transmission may be scheduled at dif(cid:173)
`ferent times. In this case, the digital image files may include
`information representing a status of one or more timers.
`The integrated Internet camera may further include a
`character generator for generating textual information in the
`captured digital images, wherein the character generator
`generates textual information in the captured digital images.
`In this case, the generated textual information may represent
`a status of one or more timers.
`The integrated Internet camera may include a serial
`interface adapted to connect to a setup device, the serial
`interface receiving commands for controlling the integrated
`Internet camera from the connected setup device. In this
`manner, the integrated Internet camera may by controlled or
`configured by another device.
`Optionally, one or more of the transport control device
`and file transfer device further includes a network authen(cid:173)
`tication device for providing network login authentication
`for connecting to the predetermined Internet address via the
`network interface device. In this manner, the integrated
`Internet camera may access and transmit files to networks
`having security and authorization provisions.
`The integrated Internet camera may further include a
`configuration device, which includes a configuration infor(cid:173)
`mation retrieving device and a configuration setting device.
`The configuration information retrieving device retrieves
`configuration information from the destination shell
`account, while the configuration setting device sets opera(cid:173)
`tional parameters of one or more of the image capturing
`circuit, the network interface device, the file transfer device,
`transport control device, the transmission initiating device,
`and the first scheduling device, according to the configura-
`tion information.
`Further, the file transfer device may further include a
`directory selecting device for setting and transmitting a
`destination directory and filename for transferring digital
`image files to the destination shell account, allowing the
`transmission of digital image files to one or more specific
`directories of a destination shell account.
`The image pickup may include a color component system
`for forming a color image, in which case the integrated
`Internet camera may further include a color adjusting circuit
`for adjusting color properties of the captured digital images.
`The integrated Internet camera may include an image
`compression circuit that generates compressed digital image
`files from the captured digital images, so that the file transfer
`device transfers the compressed digital image files to the
`destination shell account and the transport control device
`packetizes the compressed digital image files according to
`the predetermined Internet transport control protocol.
`In one modification of the system, the predetermined
`Internet transport control protocol does not detect errors or
`retransmit erroneous data, thereby increasing a rate of image
`transfer by the file transfer device.
`In another modification the integrated Internet camera
`further includes an E-mail transmission device and E-mail
`message assembler. The E-mail message assembler
`
`0026
`
`

`
`US 6,930,709 Bl
`
`5
`assembles E-mail messages representing a status of the
`camera and the E-mail transmission device transmits the
`E-mail messages to a predetermined E-mail address via the
`transport control device and the transmission initiating
`device.
`In this case, the E-mail message assembler may assemble
`E-mail messages including the digital image files.
`Accordingly, the E-mail transmission device may transmits
`the E-mail message including the digital image files to a
`predetermined E-mail address via the transport control 10
`device and the transmission initiating device.
`The integrated Internet camera may further include a
`trigger device linked to the camera and/or the microcontrol(cid:173)
`ler. In response to triggering of the trigger device, the camera
`initiates an image capture and transfer of the digital image 15
`files to the destination shell account via the file transfer
`device, the transport control device, and the transmission
`initiating device.
`Optionally, the integrated Internet camera further includes
`a video input for receiving a standard video signal, and the
`image capturing circuit captures the digital images from the
`video input instead of from the image pickup. In this
`manner, a camcorder or other video source (tuner, CCTV