(12) United States Patent
`Kalajan
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,639,943 B1
`Dec. 29, 2009
`
`USOO7639943B1
`
`6.256,059 B1* 7/2001 Fichtner .................. 348.222.1
`6,278,466 B1
`8/2001 Chen .......................... 345/473
`6,751,454 B2* 6/2004 Thornton ...
`... 455,412.1
`6,784,925 B1* 8/2004 Tomat et al. ........... 348,207.11
`6,847,334 B2
`1/2005 Hayhurst et al.
`6,995,789 B2 *
`2/2006 McIntyre et al. ......... 348,207.1
`7,034,880 B1 * 4/2006 Endsley et al. ......... 348,333.11
`7,170,551 B2* 1/2007 Fichtner ........
`... 348,207.1
`7,188,307 B2 * 3/2007 Ohsawa ...................... 715/205
`7,239,346 B1* 7/2007 Priddy .......
`... 348,220.1
`7,343,049 B2 * 3/2008 Butterworth ................ 382,284
`2002/0108118 A1
`8, 2002 Cohen et al.
`2004/0012811 A1* 1/2004 Nakayama ................. 358,115
`2004/006 1667 Al
`4/2004 Sawano ....................... 345.30
`2004/0174434 A1* 9, 2004 Walker et al. ..
`... 348,211.3
`2004/0264542 A1* 12/2004 Kienitz ....................... 374f120
`2005.0036034 A1* 2, 2005 Rea et al. .......
`... 348,207.1
`2006/0105806 A1* 5/2006 Vance et al. ............. 455,556.1
`2007/0008321 A1* 1/2007 Gallagher et al. ........... 345/473
`* cited by examiner
`Primary Examiner W. B. Perkey
`Assistant Examiner—Linda B Smith
`
`(57)
`ABSTRACT
`A computer-implemented system and method for automati
`cally capturing a set of images with a camera-enabled mobile
`device and for uploading the set of images to a network server
`is disclosed. The present invention further includes creating
`an animation of two or more images of the set of images
`automatically captured by the camera-enabled mobile device.
`
`29 Claims, 23 Drawing Sheets
`
`(54) COMPUTER-IMPLEMENTED SYSTEMAND
`METHOD FOR AUTOMATED IMAGE
`UPLOADING AND SHARING FROM
`CAMERA-ENABLED MOBILE DEVICES
`
`(*) Notice:
`
`(76) Inventor: Kevin E. Kalajan, 14077 Gochine Dr.,
`Nevada City, CA (US) 95959
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 418 days.
`(21) Appl. No.: 11/281,193
`
`Nov. 15, 2005
`
`(22) Filed:
`(51) Int. Cl.
`(2006.01)
`GO3B 29/00
`(52) U.S. Cl. ....................... 396/429; 709/203; 709/230;
`709/219
`(58) Field of Classification Search ................. 396/429,
`396/247, 264,374; 348/207. 1, 207.11, 211,
`348/211.6; 455/83, 84,90.3, 78; 709/230
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`4,876,597 A 10/1989 Roy et al.
`5,553,609 A
`9, 1996 Chen et al.
`5,737.491 A * 4, 1998 Allen et al. ................. 704/27O
`6,038,295 A * 3/2000 Mattes .................... 379,9325
`6,122,526 A
`9, 2000 Parulski et al.
`
`
`
`GoPro/Garmin
`EX. 1028, Page 001
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 1 of 23
`
`US 7,639,943 B1
`
`Figure 1
`(Prior Art)
`
`
`
`56
`
`
`
`
`
`Figure 2
`(Prior Art)
`
`GoPro/Garmin
`EX. 1028, Page 002
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 2 of 23
`
`US 7,639,943 B1
`
`
`
`Cellular Transceiver
`
`Control Processing Unit
`
`Camera Module
`
`Telephone Camera
`Control Program
`-O
`
`Figure 3
`(Prior Art)
`
`GoPro/Garmin
`EX. 1028, Page 003
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 3 of 23
`
`US 7,639,943 B1
`
`454
`
`/-
`
`Cellular Transceiver
`
`-466-
`
`Control Processing Unit
`
`. -482-
`
`Carera Module
`
`-468
`
`Display Screen
`-456-
`
`Keypad
`458
`
`
`
`Telephone Camera
`Control Program
`-470
`
`Auto-Capture and Upload
`Control Program
`475
`
`Micro-3
`-472
`
`r
`
`Figure 4
`
`GoPro/Garmin
`EX. 1028, Page 004
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 4 of 23
`
`US 7,639,943 B1
`
`
`
`Server
`-520
`
`Figure 5
`
`GoPro/Garmin
`EX. 1028, Page 005
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 5 of 23
`
`US 7,639,943 B1
`
`
`
`Proprietary Network
`
`Server
`-620
`
`Figure 6
`
`GoPro/Garmin
`EX. 1028, Page 006
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 6 of 23
`
`US 7,639,943 B1
`
`
`
`Figure 7
`
`GoPro/Garmin
`EX. 1028, Page 007
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 7 of 23
`
`US 7,639,943 B1
`
`
`
`GoPro/Garmin
`EX. 1028, Page 008
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 8 of 23
`
`US 7,639,943 B1
`
`
`
`Subscriber 1
`
`Subscriber 2
`
`Server
`
`Figure 9
`
`GoPro/Garmin
`EX. 1028, Page 009
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 9 of 23
`
`US 7,639,943 B1
`
`
`
`Figure 10
`
`GoPro/Garmin
`EX. 1028, Page 010
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 10 of 23
`
`US 7,639,943 B1
`
`
`
`initial Service Ordering-Client Side
`-1110
`
`Request Automatic Capture and image Upload Service
`-112
`
`Receive Download of Device Oualification Client Software
`-1114
`
`Perfor in Cert Device Qualification Procedure
`1116
`
`Does the Client Device pass the
`Qualification Procedure?
`-1118
`
`Receive Download of Automatic Capture and image Upload
`Service Software
`-1120
`
`Figure 11
`
`GoPro/Garmin
`EX. 1028, Page 011
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 11 of 23
`
`US 7,639,943 B1
`
`
`
`initial Service Activation - Client Side
`-1210
`
`Activate Automatic Capture and image Upload Service
`Software
`-1212
`
`Perform initial Configurations and Preferences Selection.
`Establish Server-resident User Account.
`-1214
`
`Activate Automatic image Capture and Upload Process
`-1216
`
`The Automatic Capture and image Upload Service
`Software begins to capture and upload images from the
`client's camera-enabled portable device to the server via a
`network, tmages are automatically captured at the user
`specified rate and image size.
`-1218
`
`Figure 12
`
`GoPro/Garmin
`EX. 1028, Page 012
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 12 of 23
`
`US 7,639,943 B1
`
`
`
`initial Service Activation - Server Side
`-1310
`
`Receive a User Request to Open a User Account and Start
`the Automatic Capture and image Upload Service
`-1312
`
`Open a User Account and Create a Server-Resident User
`image Storage Area.
`-134
`
`Start Receiving Automatically Captured images from the
`Client Device and Pre-Process the Received images for
`Storage into the Corresponding Server-Resident User's
`timage Storage Area.
`-136
`
`Figure 13
`
`GoPro/Garmin
`EX. 1028, Page 013
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 13 of 23
`
`US 7,639,943 B1
`
`image Pre-Processing - Server Side
`14.10
`
`Receive an Automatically Captured Emage from the Client
`Device
`-142
`
`Walidate Username and Password
`-1414
`
`Walidate image Content is a Valid image (e.g. JPEG).
`-1416
`
`Perform Partial image Processing (if Upload did not
`Complete Property)
`-1418
`
`Walidate Upload Frequency Not Exceeded
`-1420
`
`implement Motion Detection Process and Buffering
`-1422
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Figure 14
`
`GoPro/Garmin
`EX. 1028, Page 014
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 14 of 23
`
`US 7,639,943 B1
`
`Continue
`A
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Optionally Adjust Brightness and Contrast Levei
`-150
`
`Optionally Rotate image and/or perform landscape/portrait
`detection
`-1512
`
`Optionally Convert image to Monochrome
`-1514
`
`Optionally Add DateTime Stamp in User-Specified
`Locationard Font Size
`-1516
`
`Determine if Voice-Over Should Be Sent
`-1518
`
`Optionally Walidate incoming IPAddress. Determine the
`upload transfer speed. Decode multiple image uploads.
`-1520
`
`Optionally Encrypt image Prior to Storing
`-1522
`
`Determine Appropriate Subscriber Virtual Camera Account
`To Which This timage Should be Added
`-1524
`
`Update image Upload Data and Client Control Interface
`information. Walidate that the Usef Account has Available
`Disk Storage Space. Add image to the Appropriate
`Subscriber Virtual Camera Account and the Subscriber's
`Live Window. Add image to Subscriber's Activity Timeline
`-1526
`
`Figure 15
`
`GoPro/Garmin
`EX. 1028, Page 015
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 15 of 23
`
`US 7,639,943 B1
`
`
`
`Login as the Subscriber to View, Configure, and Control a
`Set of Automatically Captured and Uploaded images.
`-1612
`
`Optionally Create and Configure a Group of Preference
`Paratheters.
`-1614
`
`Optionally View and Configure One of More Virtual Camera
`Accounts
`-1616
`
`Optionally invite Guests to View the Subscriber's image Set
`by Creating an invitation Link that is sent to the invited
`Guest(s).
`-618
`
`Optionally Manage Guest Access
`-1620
`
`Optionally Manage Library Content and Access
`-1822
`
`Optionally Receive a Voiceover interaction from a Guest
`-1622
`
`Figure 16
`
`GoPro/Garmin
`EX. 1028, Page 016
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 16 of 23
`
`US 7,639,943 B1
`
`invited Guest Processing
`-1710
`
`
`
`
`
`Login as an invited Guest to view a a Particular
`Subscriber's set of Automatically Captured and Uploaded
`Images. Activate a link sent to the Guest from a Subscriber.
`1712
`
`invited Guest's CEient System Obtains information from the
`Server to Update and Display a Cient Control interface.
`The Live Window, Activity Timeline, Recent Activity
`Window, and the Control Panel of the Client Control
`fnterface are Updated and Displayed on the Guest Client
`System.
`-174
`
`if Requested by the Guest via the Client Control Interface,
`Create and Display an image Animation.
`-1716
`
`if Requested by the Guest via the Client Control interface,
`Create and display artmage Library, Save images and/or
`Animations in the trmage library, or Retrieve timages and/or
`Animations from the image Library.
`-1718
`
`if Requested by the Guest via the Client Control Interface,
`Create and Activate a Voiceovef interaction with the
`Corresponding Subscriber.
`-1720
`
`if Requested by the Guest via the Client Control Interface,
`Create and Activate a Matrix View of Images from Multiple
`Subscribers.
`-1722
`
`
`
`
`
`
`
`
`
`
`
`
`
`Figure 17
`
`GoPro/Garmin
`EX. 1028, Page 017
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 17 of 23
`
`US 7,639,943 B1
`
`Subscriber's Guest Access Management
`Processing
`-1810
`
`Login as the Subscriberto View, Configure, and Control
`Guest Access to Subscriber images.
`-1812
`
`
`
`Optionally invite Guests,
`-1814
`
`Optionally View Guest Status and/or Time of Last Guest
`Access to Subscriber's images
`-1816
`
`Optionally Block/Unblock Guest Access to Subscriber's
`images or Cancel Guest Access to Subscriber's images.
`-1818
`
`Figure 18
`
`GoPro/Garmin
`EX. 1028, Page 018
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 18 of 23
`
`US 7,639,943 B1
`
`Subscriber's Library Access Management
`Processing
`-1910
`
`Login as the Subscriberto View, Configure, and Control
`Subscriber image Library.
`-1912
`
`
`
`
`
`Optionally Delete, Hide, or Restrict One or More Images
`and/or Animations from View or Access by All or Specific
`Guests.
`-1914
`
`Optionally Web-Enable a Subscriber's Image or Animation
`-1916
`
`Optionally Email a Subscriber's image of Animation or Link
`Thereto to a Desired Recipient.
`-1918
`
`Optionally Add an Audio Track or Annotation to a
`Subscriber's image or Animation.
`-1920
`
`Figure 19
`
`GoPro/Garmin
`EX. 1028, Page 019
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 19 of 23
`
`US 7,639,943 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Invited Guest Animation Generation
`Processing
`-2010
`
`Login as an invited Guest to view a a Particular
`Subscriber's set of Automatically Captured and Uploaded
`Images. Activate a tink sent to the Guest from a Subscriber.
`-2012
`
`tnvited Guest's Client System Obtains information from the
`Server to Update and Display a Client Control triterface.
`The Live Window, Activity Timeline, Recent Activity
`Window, and the Control Panel of the Client Control
`interface are Updated and Displayed on the Guest Client
`System.
`-2014
`
`f Requested by the Guest via the CEient Control interface,
`Initiate Creation of an Image Animation.
`-2015.
`
`Guest Clicks Any Location on the Activity Timeline. Guest
`can Optionally Request an Animation Preview. Guest
`Specifies Animation Duration and Quality Parameters.
`-2018
`
`Images Previously Captured and Stored at Times Within
`the Specified Time Range Selected on the Activity Timeline
`are Extracted and Appended Together in an Animation
`Sequence. An Appropriate Video Object is Thereby
`Created Depending on the Guest's Browser Type.
`-2020
`
`
`
`Once the Animation is Created, the Guest can Select the
`Viewing Speed, Select image Size, Add Captions
`Comments, Save the Video Clip locally, Save the Video
`Clip to the Library, Step Through the Video Clip One Frame
`at a Time, Email the Video Clip to Others
`-2022
`
`Figure 20
`
`GoPro/Garmin
`EX. 1028, Page 020
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 20 of 23
`
`US 7,639,943 B1
`
`
`
`tnvited Guest Voiceover Interaction Processing
`-2110
`
`
`
`Login as an invited Guest to view a a Particular
`Subscriber's set of Automatically Captured and Uploaded
`Images. Activate a link sent to the Guest from a Subscriber,
`-2112
`
`Envited Guest's Client System Obtains information from the
`Server to Update and Display a Client Control tinterface.
`The Live Window, Activity Timeline, Recent Activity
`Window, and the Control Panel of the CEient Control
`Interface are Updated and Displayed on the Guest Client
`
`if Requested by the Guest via the Client Control interface,
`initiate Voiceover interaction with Subscriber.
`-2116
`
`Guest Records a Short Audible Message for the Subscriber
`of Employs a Text-to-Speech Engine to Generate an
`Audible Message for the Subscriber
`-21 18
`
`
`
`The Recorded Audible Message is Delivered From the
`Guest to the Subscriber Upon the Next image Upload From
`the Subscriber.
`-2120
`
`Optionally, the Voiceover Message may be Pre-pended
`with an Audio Advertisement
`-2122
`
`Figure 21
`
`GoPro/Garmin
`EX. 1028, Page 021
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 21 of 23
`
`US 7,639,943 B1
`
`
`
`GoPro/Garmin
`EX. 1028, Page 022
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 22 of 23
`
`US 7,639,943 B1
`
`
`
`Figure 23
`
`GoPro/Garmin
`EX. 1028, Page 023
`
`

`

`U.S. Patent
`
`Dec. 29, 2009
`
`Sheet 23 of 23
`
`US 7,639,943 B1
`
`
`
`Figure 24
`
`GoPro/Garmin
`EX. 1028, Page 024
`
`

`

`US 7,639,943 B1
`
`1.
`COMPUTER-IMPLEMENTED SYSTEMAND
`METHOD FOR AUTOMATED IMAGE
`UPLOADING AND SHARING FROM
`CAMERA-ENABLED MOBILE DEVICES
`
`BACKGROUND
`
`2
`provided for in home patient health care from a remote loca
`tion via ordinary telephone lines. A number of master moni
`toring computers are linked to a control center and are acces
`sible by a corresponding number of healthcare professionals.
`A slave monitoring computer is located within the home of a
`plurality of patients and may be linked via telephone modems
`to any of the master monitoring computers. Audio/visual
`equipment at both locations, permits real-time two-way com
`munications during an in-home visit to a patient by a health
`care professional from a remote location. However, the 609
`patent does not describe a system in that uses camera-enabled
`mobile devices to upload images to a networked server. Fur
`ther, the referenced patent does not describe a way to archive
`and automatically animate (convert images to movies) the
`automatically uploaded images.
`U.S. Pat. No. 4,876,597 describes a system wherein a scene
`to be monitored (e.g., at a remote location or at a later time) is
`captured as a series of still images. These images are digitally
`encoded and stored in a digital memory for Subsequent
`retrieval and viewing. If the scene includes a moving object
`(e.g., the side of a moving freight train), the images may be
`taken so that they are substantially mutually exclusive and
`collectively exhaustive of the moving object. During play
`back, two or more adjacent images may be displayed adjacent
`to one another so that any feature partly present in adjacent
`images is visible as a whole. If the observer is at a location
`remote from the point at which the images are taken, various
`techniques can be used to facilitate transmission of the image
`information (preferably in digital form) via relatively low
`cost transmission links such as voice-grade telephone lines.
`The 597 patent is further described as an invention that will
`be fully understood from an explanation of its use in identi
`fying moving railroad freight cars, especially (although not
`necessarily) at locations remote from the observer required to
`make the identification. Thus, the 597 patent does not
`describe the use of a mobile device for capturing images.
`Further, the referenced patent does not describe a way to
`archive and automatically animate (convert images to mov
`ies) the automatically uploaded images.
`U.S. Pat. No. 6,122,526 describes a combined telephone/
`camera unit that includes a camera module for generating
`electronic image data representative of a scene to be imaged,
`a memory unit for storing the electronic image data generated
`by the camera module, a display screen for displaying the
`electronic image data stored in the memory unit, a mechanism
`for selecting which of the plurality of base units is to receive
`the digital image data, and a cellular transceiver for transmit
`ting the digital image data to the base units selected by the
`selection mechanism. However, the 526 patent does not
`describe a system in that uses camera-enabled mobile devices
`to automatically upload images to a networked server on a
`periodic basis. Further, the referenced patent does not
`describe away to archive and automatically animate (convert
`images to movies) the automatically uploaded images.
`Thus, a computer-implemented system and method for
`automatically capturing images with a camera-enabled
`mobile device and for uploading the images to a network
`server is needed.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is illustrated by way of example and
`not limitation in the figures of the accompanying drawings, in
`which
`FIGS. 1-3 illustrate a prior art camera-enabled cellular
`telephone.
`
`1. Related Application
`This patent application is related to U.S. patent application,
`Ser. No. 1 1/321,154; filed on Dec. 28, 2005; and filed by the
`same applicant as the present patent application.
`2. Field
`Embodiments of the invention relate to the field of camera
`enabled mobile devices and computer network applications
`and networked communications.
`3. Related Art
`Camera-enabled mobile devices continue to show an
`increasing popularity. Typical user paradigms include user
`initiated image or video capture with operator-specific post
`processing options, including for example, save locally,
`e-mail, send via MMS, upload to a web site, etc. Much of the
`mobile device industry is focused on delivering content to
`mobile devices. Clearly, the delivery of content to mobile
`devices is critical and useful. However, as mobile devices
`increase in power and functionality, these devices become
`increasingly important as sources of content in addition to
`being consumers of content. One important source of content
`on these devices is the image or video capture components
`provided with camera-enabled mobile devices.
`U.S. Patent Application Publication No. 2002/0108118A1
`describes a digital data storage and transmitting device par
`ticularly suited for use with a digital camera, digital cam
`corder, or other image/video capturing device. The digital
`data storage and transmitting device is configured into upload
`ortransmit picture information to a remote server using any of
`a number of communication protocols or techniques, includ
`ing wireless or cell phone technologies. However, the device
`described in the referenced patent application is a wireless
`digital camera adapter and not a camera-enabled mobile tele
`phone.
`U.S. Pat. No. 6,847,334 describes a mobile telecommuni
`cation device for simultaneously transmitting and receiving
`Sound and image data. The described mobile telecommuni
`cation unit provides two-way remote viewing of images and
`real-time audio. The self-contained portable unit communi
`cates directly with a similar unit at a remote location across
`POTS lines, cellular, ISDN, T-1, and satellite communica
`tions links. In one implementation of the system described in
`the referenced patent, a portable, mobile telecommunication
`unit is designed to transmit voice, and still image data simul
`taneously to a compatible device on the other end of a tele
`phone connection. One unit is placed into contact with the
`second, compatible device by a phone call. To capture an
`image the user takes a picture with a digital camera provided
`with the unit. Through a camera interface, the digital camera
`transfers an image to the local unit. The local unit can then
`transmit the image to the remote unit. The device described in
`the 334 patent does not automatically capture images with a
`camera-enabled mobile device and does not upload images to
`a network server. Further, the referenced patent does not
`describe away to archive and automatically animate (convert
`images to movies) the automatically uploaded images. Addi
`tionally, the referenced patent does not describe the automatic
`archiving and sharing of the transmitted image data.
`U.S. Pat. No. 5,553,609 describes an intelligent remote
`visual monitoring system for home healthcare service. In this
`system, a computer-based remote visual monitoring system is
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`GoPro/Garmin
`EX. 1028, Page 025
`
`

`

`US 7,639,943 B1
`
`3
`FIG. 4 illustrates a camera-enabled mobile device support
`ing the functionality of the present invention.
`FIG. 5 illustrates a system architecture for uploading
`images to one or more servers via the Internet.
`FIG. 6 illustrates a system architecture for uploading
`images to one or more servers via a proprietary network.
`FIG. 7 illustrates the image upload operation of one
`embodiment.
`FIG. 8 illustrates one embodiment of the image download
`process of the present invention.
`FIG. 9 illustrates one embodiment of the image download
`process of the present invention for use with animations.
`FIG. 10 illustrates one embodiment of the image download
`process of the present invention for use with various client
`devices.
`FIG. 11 is a flow diagram illustrating processing for initial
`service ordering.
`FIG. 12 is a flow diagram illustrating processing for initial
`service activation, client side.
`FIG. 13 is a flow diagram illustrating processing for initial
`service activation, server side.
`FIGS. 14-15 are flow diagrams illustrating image pre-pro
`cessing.
`FIG. 16 is a flow diagram illustrating subscriber process
`ing.
`FIG.17 is a flow diagram illustrating invited guest process
`ing.
`FIG. 18 is a flow diagram illustrating processing for sub
`scriber guest access management.
`FIG. 19 is a flow diagram illustrating processing for sub
`scriber library access management.
`FIG.20 is a flow diagram illustrating processing for invited
`guest animation generation.
`FIG.21 is a flow diagram illustrating processing for invited
`guest Voiceover interaction.
`FIGS. 22-24 illustrate various embodiments of the client
`control user interface.
`
`DETAILED DESCRIPTION
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`4
`cellular transceiver 66. Prior art cellular telephones can also
`be used to establish a wireless Internet connection using a
`micro browser 72 stored in memory unit 64. In this manner, a
`user of prior art camera-enabled cellular telephone 48 can
`also manually capture images and transfer the images wire
`lessly to a remote user or to a webpage via the Internet.
`Hand held or battery-operated products such as cellular/
`mobile phones, two-way pagers, personal digital assistants
`(PDAs), and personal organizers can communicate with
`other devices or servers by way of a wireless link. In one
`implementation, the Java programming environment devel
`oped by Sun MicrosystemsTM Inc. enables software develop
`ers to write device-independent wireless applications for
`accessing, downloading, or updating information over the
`wireless link. SunTM provides a JavaTM 2 Platform, Micro
`Edition (J2ME) that can be used with a Mobile Media API
`(MMAPI). The MMAPI extends the functionality of the
`J2ME platform by providing audio, video and other time
`based multimedia Support to resource-constrained mobile
`devices. MMAPI allows Java developers to gain access to
`native multimedia services available on a given device. Nev
`ertheless, it will be apparent to those of ordinary skill in the art
`that embodiments of the present invention can be developed
`without using a JavaM platform. Other programming para
`digms (other than Java) include: Microsoft Windows Mobile,
`Symbian, Qualconmm's BREW, and Palm's PalmOS.
`The present invention leverages these prior art technolo
`gies and extends the functional capabilities to provide a com
`puter-implemented System and method for automatically
`capturing images with a camera-enabled mobile device and
`for uploading the images to a network server.
`Referring to FIG. 4, a camera-enabled mobile device 410
`Supporting the functionality of the present invention is illus
`trated. As in a conventional camera-enabled mobile device,
`mobile device 410 includes a cellular transceiver 466, control
`processing unit 462, a camera module 468, a display Screen
`456, a keypad 458, and memory unit 464. Mobile device 410
`includes a telephone camera control program 470 stored in
`memory unit 464. Control program 470 controls the basic
`conventional functions of device 410. Device 410 can also
`include micro browser 472 to enable a user of mobile device
`410 to access the Internet via a wireless connection.
`In one embodiment, mobile device 410 also includes an
`auto capture and upload control program 475. Control pro
`gram 475 includes executable software and related data for
`implementing and controlling various features of the present
`invention, as will be described in more detail below. Control
`program 475 can be fetched and executed by control process
`ing unit 462 via bus 460.
`The present invention provides a network-based system
`and service using software that executes partly on camera
`enabled mobile devices and partly on a server. Using the
`present invention, a collection of images captured by the
`camera-enabled mobile device of an originating user (de
`noted hereinas the subscriber) can be uploaded to a server and
`processed by the server for Subsequent access and viewing by
`other authorized users (denoted herein as guests). Referring
`to FIG. 5, a camera-enabled mobile device 510 embodying
`the functionality of the present invention can automatically
`capture one or more images using the camera of mobile
`device 510. The captured images can then be wirelessly trans
`ferred via the Internet to one or more load-balanced servers
`520. Conventional interfaces and protocols exist for wire
`lessly transferring images to a server via the Internet. How
`ever, no conventional technologies provide a means for auto
`matically capturing images with a camera-enabled mobile
`device and for uploading the images to a network server in the
`
`A computer-implemented system and method for auto
`matically capturing images with a camera-enabled mobile
`device and for uploading the images to a network server is
`disclosed. In the following description, numerous specific
`details are set forth. However, it is understood that embodi
`ments of the invention may be practiced without these spe
`cific details. In other instances, well-known processes, struc
`tures and techniques have not been shown in detail in order
`not to obscure the understanding of this description.
`Referring to FIGS. 1-3, a prior art camera-enabled cellular
`telephone 48 is illustrated. The top end of the combined unit
`48 typically includes a lens 50 and an antenna 54. Although
`not used much in camera phones today, one embodiment may
`also have a flip up flash unit 52. The front face of unit 48 is
`typically provided with a liquid crystal or plasma display
`screen 56 and a telephone keypad 58, both of which are
`electrically coupled to an internal bus 60 (as shown in FIG.3).
`Referring to FIG. 3, camera-enabled telephone 48 includes a
`cellular transceiver component 66, a control processing unit
`62, a camera module 68, a display screen 56, a keypad 58, and
`memory unit 64. Each of these components is interconnected
`via bus 60. Memory unit 64 can be used to store an application
`Software component or telephone/camera control program 70
`used to control the basic conventional features of camera
`enabled telephone 48. In operation, a user can activate a
`65
`shutter button (not shown) to capture an image through cam
`era module 68 and send the image to a remote receiver via
`
`45
`
`50
`
`55
`
`60
`
`GoPro/Garmin
`EX. 1028, Page 026
`
`

`

`US 7,639,943 B1
`
`10
`
`15
`
`5
`manner described and claimed herein. As will be described in
`more detail below, server 520 can process the set of uploaded
`images for viewing by other authorized users.
`Referring to FIG. 6, it will be apparent to those of ordinary
`skill in the art that computer networks other than the Internet
`can be used for the transfer of images between camera-en
`abled mobile device 610 and server 620. Proprietary net
`works, peer-to-peer networks, or other networks can be used
`for the transfer of images between camera-enabled mobile
`device 610 and server 620.
`Referring to FIG. 7, the image upload operation of one
`embodiment is illustrated. Using auto capture and upload
`control program 475 (shown in FIG. 4), a series of automatic
`Snapshot images are captured by control program 475
`through camera module 468. Note that no manual user acti
`vation of a shutter button on mobile device 610 is required. In
`one embodiment, the auto capture and upload control pro
`gram 475 of the present invention automatically captures
`images in the following manner. The Subscriber typically
`places the camera-enabled mobile device in a tripod or other
`stable configuration (though not a requirement) and activates
`the auto capture and upload control program 475 in a conven
`tional manner. The auto capture and upload control program
`475 may begin by presenting a menu of options to the user
`(e.g. Configuration settings or help Screens) or may, option
`ally, immediately begin Snapping images (at a specified rate)
`and uploading the images to the network servers 620. The
`capture of an image is done under program control, typically
`by registering a request to the mobile device that the auto
`capture and upload control program 475 wants control of the
`camera. After this request is Successfully satisfied, the auto
`capture and upload control program 475 of the present inven
`tion makes requests to the device's operating system 470 for
`an image of a specific size, quality, and/or format
`According to a preconfigured time frequency parameter,
`mobile device 610 automatically captures Snapshot images
`on a periodic basis. This series of images is depicted in FIG.
`7 as images 625. Images 625 represent Snapshots automati
`cally taken by mobile device 610 at regular intervals. Again,
`other than an initial set up and configuration, no action is
`required by the user of mobile device 610 to capture images
`625. Images 625 can be encoded in any of a variety of con
`ventional image formats, including JPEG, TIFF, bitmap.
`PDF, etc. Once encoded, the images can be transferred to
`server 620 via a network. Server 620 receives the uploaded
`images 625 and stores the images in persistent server memory
`after performing all of the upload server processing steps
`described below.
`In one embodiment, the conventional HTTP POST proto
`col is used to upload images from the mobile device to Inter
`net-connected servers. This is one of a many ways that image
`content could be delivered to servers. Other alternative
`embodiments include (but are not limited to): SMTP (email),
`FTP (file transfer protocol), HTTP PUT method, or TCP/IP
`“sockets' combined with a custom/proprietary application
`level protocol.
`The HTTP POST approach has many benefits compared to
`other alternatives and was chosen for the preferred embodi
`ment. FTP requires two connections for a transfer and thus is
`not efficient. FTP also requires FTP usernames and pass
`words to be managed and has no built-in way to embed meta
`data with the image (HTTP POST has this via HTTP HEAD
`ERS). SMTP is a store-and-forward protocol and thus images
`could arrive completely out of sequence or be very delayed.
`HTTP PUT is not well implemented and has security issues.
`Using low-level TCP/IP sockets suffers from difficulty of
`implementation on the mobile device, as many mobile opera
`
`6
`tor networks are not designed or optimized for Such traffic,
`which is the reverse case for HTTP POST.
`A particularly novel element of the present invention is the
`automated, rapid uploading of images versus the prior art of
`“streaming.” Streaming refers to the isochronous (fixed rate)
`transmission of content in real-time. The approach used with
`images, in the present invention, affords a number of inter
`esting and valuable benefits. First, mobile op