throbber
C O V E R F E A T U R E
`
`Expanding the
`Digital Camera’s
`Reach
`
`Chandra
`Narayanaswami
`M.T.
`Raghunath
`IBM T.J. Watson
`Research Center
`
`Today’s digital cameras and large-capacity portable storage devices could
`soon be integrated into compact cell phones that establish symbiotic
`relationships with stationary devices in the environment, providing users
`with the ability to view and share images in many new settings and
`enabling the creation of several novel applications.
`
`O ver the past decade, consumer digital cam-
`
`eras have evolved from expensive and
`bulky devices that provide low-resolution
`images to affordable compact units capa-
`ble of recording high-resolution pictures.
`Recently, cell phones that integrate digital cameras
`have far outsold regular digital cameras. We expect
`this trend to continue because this combination
`provides great value.
`First, the cell phone’s voice communication capa-
`bility makes it the most ubiquitous portable device.
`Second, people enjoy the convenience of capturing
`high-resolution digital images using a device they
`already carry. Third, this integration relieves people
`from having to make a conscious decision to take a
`camera in anticipation of taking pictures. Some dig-
`ital cameras even offer integrated Wi-Fi capabilities
`for direct image transfer. Recent trends in portable
`storage devices indicate that cellular camera phones
`can integrate several gigabytes of storage.
`More than a straightforward replacement for film
`cameras, digital cameras allow new and different
`uses. People tend to take more pictures in more sit-
`uations with digital cameras than they do with film
`cameras. For example, because digital images
`require little physical space, some users have con-
`verted their children’s space-consuming art projects
`into compact yet accessible digital albums. Others
`have recorded a skin rash’s progress or a dog bite so
`that they can show their doctor the digital image.
`Digital cameras have also become memory aids
`
`and transcription devices, creating a class of ephem-
`eral images with shorter lifetimes than images taken
`with film cameras. People take pictures of where they
`parked their car at the airport so that they needn’t
`spend time searching the lot when they return.
`Others have captured the license plate number of
`erratic drivers and reported them to authorities.
`Digital cameras have become input devices as well.1,2
`More applications like these will appear as high
`portability, minimal per-picture cost, and generous
`storage increase the digital camera’s popularity.
`Despite the increasing image resolution, storage
`capacity, and wireless connectivity of cellular cam-
`era phones, human preferences dictate their physi-
`cal size, which essentially limits the integrated
`display’s size. The advent of flexible displays could
`change this balance, but the size available in a truly
`portable form factor will limit the rolled-up display’s
`area. Another alternative, projection technologies,
`consume too much power at the present time to be
`practical for most portable devices. Therefore, for
`the next few years, the camera’s display size will
`remain constrained while the number of images that
`can be stored in the camera itself will incease dra-
`matically. Although immediately viewing pictures
`on the integrated display is a valuable feature, it can
`show only a limited amount of detail.
`To address this limitation, we recently described
`a futuristic scenario in which mobile computers
`establish symbiotic relationships with stationary
`devices in the environment to offer users a combi-
`
`0018-9162/04/$20.00 © 2004 IEEE
`
`P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y
`
`December 2004
`
`65
`
`GoPro/Garmin
`EX. 1024, Page 001
`
`

`

`Table 1. Typical display and maximum possible resolution according to human visual acuity limitations.
`
`Display type
`
`User distance
`(inches)
`
`Typical width
`(inches)
`
`Typical width
`(pixels)
`
`Typical
`resolution
`(dpi)
`
`Maximum
`resolution
`(dpi)
`
`Maximum
`width
`(pixels)
`
`Cell phone panel
`PDA display
`Laptop display
`Desktop monitor
`Meeting room screen
`
`10
`12
`16
`20
`230
`
`1.5
`2
`10
`20
`80
`
`150
`300
`1,200
`2,000
`1,200
`
`100
`150
`120
`100
`15
`
`350
`291
`218
`175
`15
`
`525
`582
`2,180
`3,500
`1,200
`
`Table 2. Some examples of camera specifications evolution, 1995-2003.
`
`Camera
`
`Ricoh RDC-1
`Kodak DC210
`Kodak DC290
`Sony DSC-P10
`Samsung
`VGA1000 cell phone
`
`Year
`
`1995
`1997
`1999
`2003
`2003
`
`Introductory
`retail
`price
`
`$1,800
`$600
`$800
`$600
`$299
`
`Image
`resolution
`
`Typical
`image size
`
`768 x 480
`1,152 x 864
`1,792 x 1,200
`2,592 x 1,944
`640 x 480
`
`100 Kbytes
`300 to 400 Kbytes
`600 to 800 Kbytes
`2 Mbytes
`100 Kbytes
`
`Display
`size
`(mm)
`
`51 x 31
`37 x 27
`41 x 28
`31 x 23
`34 x 42
`
`Pixels
`
`72 K
`
`123 K
`20.4 K
`
`Maximum
`memory
`available
`
`24 Mbytes
`64 Mbytes
`128 Mbytes
`2 Gbytes
`
`Image
`capacity
`
`240
`160
`160
`1,000
`
`nation of both systems’ best attributes: large, easy-
`to-read, high-quality displays and content person-
`alization through mobile computers.3
`In this scenario, environmental displays become
`intelligent network objects that offer their services
`the same way today’s network printers do. Mobile
`computers discover such displays, communicate
`with them to ascertain their characteristics, and
`securely transmit information.
`
`SYMBIOTIC DISPLAYS IN THE ENVIRONMENT
`Many researchers have explored the idea of using
`environmental displays to supplement portable
`devices’ display capabilities.3,4 They base this work
`on the key observation that every display has intrin-
`sic limitations that create usage barriers. For exam-
`ple, human visual acuity imposes an upper bound
`on display resolution. Even people with perfect
`vision cannot resolve details smaller than one
`minute of visual arc angle for prolonged durations
`at comfortable brightness levels. Thus, increasing
`display resolution beyond that point does not con-
`tribute significantly to improvements in the dis-
`played information’s readability.
`Table 13 shows the typical resolution of current
`displays and the maximum meaningful resolution
`as a function of display size. The table shows that
`technological advances will not improve portable
`displays enough to make viewing a large amount
`of detail easy. The improvements in picture quality
`resulting from better capture resolutions are sel-
`dom observable on a digital camera’s integrated
`display. To provide an extended set of applications,
`
`we expect future digital cameras to establish on-
`demand symbiotic relationships with large envi-
`ronmental displays.
`In addition to cameras, we expect several other
`mobile devices to leverage the services of network-
`attached displays to view documents and other
`types of content that cannot be viewed easily on
`small displays. We believe that a new class of intel-
`ligent network-connected displays will emerge to
`support such symbiotic relationships. These displays
`will have some or all of the following attributes:
`
`• direct connection to the network infrastruc-
`ture and network addressability;
`• support for content formats such as ASCII,
`HTML, PDF, PostScript, JPEG, GIF, MPEG,
`and Flash, plus the ability to negotiate with
`other network devices, much as browsers
`express accept tags to Web servers;
`• the ability to express their capabilities to
`mobile devices in terms of pixel resolution and
`dimensions;
`• support for direct user interaction via a key-
`board, mouse, touch-sensitive screen, or other
`forms of gesture recognition, so that users can
`perform simple operations such as scrolling
`and pausing video;
`• downloadable code support, such as Java-
`Script and Java applets, for richer user inter-
`activity;
`• support for a short-range wireless network
`interface to communicate with mobile devices
`and make it easy for mobile devices to discover
`
`66
`
`Computer
`
`GoPro/Garmin
`EX. 1024, Page 002
`
`

`

`just those displays in the immediate vicinity;
`and
`• support for one or more wired interfaces such
`as USB or FireWire to connect directly to
`mobile devices.
`
`Although network-attached displays have much
`in common with network printers, we believe these
`displays will be less expensive to operate because
`they use no consumables such as ink and paper.
`We already see trends in this direction with the
`addition of Ethernet and Wi-Fi interfaces to pro-
`jectors, some of which even come with software to
`display content from a remote PC. We expect to
`see network displays become standard appliances
`eventually, deployed in various environments.
`Over time, we anticipate that several different
`types of network displays will become available,
`all supporting standard software and hardware
`interfaces. The prices of such displays will likely
`drop as well, widening their deployment further.
`We have started prototyping such network-
`connected displays by enhancing standard projec-
`tors. Recently we demonstrated an application
`scenario in which our WatchPad wristwatch com-
`puter5 and the Everywhere Display research pro-
`totype establish a symbiotic session.6
`
`APPLICATION ENVIRONMENTS
`We expect digital cameras to be used symbioti-
`cally in several environments, including homes,
`offices, shops, cars, airplanes, and trains. This intro-
`duces several challenges because some of these envi-
`ronments are friendlier than others, some have
`better access to power, some have organization fire-
`walls, and some are more private. This variability
`affects both the camera’s design and required infra-
`structure.
`
`Camera image capacity
`Table 2 shows that the size of the images cap-
`tured by digital cameras has increased at a steady
`pace in the past few years, thanks to higher-reso-
`lution imaging chips. The number of images that
`could be stored in the camera did not increase sig-
`nificantly during these early years because increas-
`ing image resolution neutralized the increased
`storage capacity. Figure 1 shows that as technol-
`ogy improved, several cameras became physically
`smaller, shrinking display size accordingly.
`The human eye’s limitations make it unlikely that
`the integrated camera’s display size and resolution
`will continue increasing. While professional pho-
`tographers may want higher capture resolutions,
`
`Figure 1. Cameras from 1995 through 2003. As technology improved, cameras
`became physically smaller, shrinking display size accordingly.
`
`Figure 2. Portable-storage devices with capacities ranging from 512 Mbytes to
`40 Gbytes.
`
`many consumers find the current image capture res-
`olutions adequate because they can already print
`high-quality enlargements. Although larger cam-
`eras can include sophisticated optics, consumers
`often prefer the convenience of smaller devices.
`In contrast, we can expect camera storage capac-
`ities to continue doubling almost every year, effec-
`tively increasing the number of pictures the camera
`can store. IBM’s introduction of its MicroDrives,
`with a 340-Mbyte capacity in a compact flash form
`factor, began this trend. Atomic-force-microscope-
`based data storage technologies such as Millipede7
`can have a data storage density of 125 Gbytes per
`square inch—10 times higher than the densest mag-
`netic storage available today.
`This difference in the growth rates of image and
`storage size will eventually let users retain many
`images in their cameras. For example, the 40-Gbyte
`storage device shown in Figure 2 can hold 20,000
`5-megapixel images. A VGA resolution thumbnail
`takes about 100 Kbytes, so thumbnails for 10,000
`
`December 2004
`
`67
`
`GoPro/Garmin
`EX. 1024, Page 003
`
`

`

`Display
`
`User’s
`camera
`
`Short-range
`wireless
`link
`
`Figure 3. Image transfer via direct symbiosis. This approach transfers the image
`directly from the camera to the high-resolution display.
`
`Local
`network
`
`Internet
`
`Display
`
`Short-range
`wireless
`link
`
`Long-range
`wireless
`link
`
`User’s
`camera
`
`Web
`repository
`
`Figure 4. Image transfer via indirect symbiosis. When images are stored in a Web
`repository, if the bandwidth between the display and the Web server is greater
`than the direct link between the camera and display, letting the display fetch
`them directly can save the camera’s battery power.
`
`images will take only 1 Gbyte. Assuming that a user
`takes about 10 pictures a day, the storage device
`can hold five years’ worth of images. If the camera
`retains only thumbnails when the actual image is
`transferred from the camera, a 40-Gbyte storage
`device could hold 400,000 VGA-resolution thumb-
`nails.
`Soon then, some cameras will be able to retain
`images taken over several years, along with anno-
`tations. Others will be able to retain thumbnails
`and annotations for all pictures captured over sev-
`eral years. Users will likely replace the camera
`before they run out of space. The function provided
`by auxiliary portable devices that help offload
`images from digital cameras will be integrated into
`the cameras themselves. As users upgrade to newer
`cameras, they could copy the entire contents of
`their current camera to a new one.
`Videos captured by the camera will likely fill up
`even these large built-in storage capacities. Until
`storage capacities increase further, users will be
`forced to offload video content to auxiliary devices
`or storage media. This offloading of video content
`reserves ample storage capacity for still images.
`Regardless of camera capacity, we expect users
`will often copy their images to home computers or
`Web repositories, primarily to safeguard against
`the camera’s loss. Already, several providers—such
`
`68
`
`Computer
`
`as Ofoto, Yahoo, and Shutterfly—offer network-
`accessible storage and photo printing services. Such
`repositories also let users share their pictures with
`others and obtain prints when necessary.
`Just as MP3 players let music lovers carry huge
`collections of their favorite music, technology
`trends will let consumers personally carry a large
`image collection. To unleash the full potential of
`these collections, users must be able to view and
`share images easily.
`
`Direct symbiosis
`How users view their images on intelligent envi-
`ronmental displays will vary depending on where
`the image is stored. If the image resides on the cam-
`era itself, the simplest approach transfers it directly
`from the camera to the display, as Figure 3 shows.
`However, to retain privacy and security in less
`secluded settings, a user can preview images on the
`camera display before sending them to the larger
`display.
`To do this, once the camera discovers the display
`and its capabilities, the camera and display estab-
`lish a session, and, if required, the session completes
`a mutual authentication. With the built-in camera
`display, the user scrolls through the image thumb-
`nails and picks one to view on the environmental
`display. The camera sends the higher resolution
`image to the display. If the display does not sup-
`port the high-resolution image, the camera can
`resize the image to reduce the amount of data being
`transferred.
`The connection between the camera and the envi-
`ronmental display could be made over a short-
`range wireless link or through a wired connection
`such as USB. From a communication semantics per-
`spective, the camera is the master and the display
`the slave, even though physical cabling can make
`the camera a USB slave.
`
`Indirect symbiosis
`When images reside in a Web repository, letting
`the display fetch them directly from the repository
`can save the camera’s battery power. This approach,
`shown in Figure 4, may be faster if the bandwidth
`between the display and the Web server is greater
`than the direct link between the camera and display.
`We expect that the user will still view the thumb-
`nails privately on the camera display before decid-
`ing which images to view on the large display. Image
`selection will thus still be a problem when dealing
`with hundreds or thousands of thumbnails.
`Once the user selects an image, the camera can
`direct the display to fetch it from the repository. To
`
`GoPro/Garmin
`EX. 1024, Page 004
`
`

`

`use this approach, the camera must be able to
`authorize the image transfer. It can do so by access-
`ing the Web repository via its long-range wireless
`connection and requesting that the repository cre-
`ate a freshly computed ephemeral token. To pro-
`vide suitable security, the token could be derived
`based on the time, user’s password, location, name
`and address of the display, and sequence numbers
`sent by the camera. The camera then hands off the
`token to the environmental display over the short-
`range wireless interface. The environmental display
`obtains the image from the Web repository on the
`camera’s behalf, proving its authority by supplying
`the token. The repository rejects any requests with-
`out valid tokens.
`The entire mechanism can be simplified if the cam-
`era can send a URL to the display that includes both
`the image identifier and the authorization token. The
`Web repository usually cannot initiate the image
`transfer to the display because the display may reside
`behind a firewall. The camera’s short- and long-
`range wireless capabilities work in conjunction to
`download and view selected images without com-
`promising the user’s entire image collection.
`This approach can also be used to share images.
`For example, a user could receive a message from
`a friend who has uploaded an image collection he
`wishes to share. The message might include thumb-
`nails that arrive at the user’s mobile device. While
`scrolling through the message’s thumbnails, the user
`can instruct the mobile device to display selected
`thumbnails on the larger display. At this point, the
`user’s camera delegates its authority to the display
`so that it can fetch the images from the friend’s Web
`repository. Optionally, the display could transfer a
`copy of the image to the camera over the short-
`range link.
`
`CHALLENGES
`Digital camera designers must confront several
`challenges, including issues of security, usability,
`power management, wireless interface range, and
`storage type.
`
`Image management
`Cameras that store images must deal with the
`challenge of managing large amounts of data. This
`differs from managing thousands of songs stored
`on an MP3 player because songs have descriptive
`metadata such as names and artists, whereas
`images must be user annotated.
`Given that thousands of images can be stored in
`one place, easy and effective naming schemes must
`be developed to replace sequential numbering.
`
`Thousands of images
`can be stored in one
`place, so effective
`naming schemes
`must be developed
`to replace
`sequential
`numbering.
`
`Voice-based input could be used for naming
`picture groups. Although people typically
`avoid annotating images because doing so is
`cumbersome, several approaches can make
`this task simpler,8-11 such as allowing the addi-
`tion of voice labels at the time of capture that
`PCs and similar devices can later convert to
`searchable text. Automatically recording a
`picture’s context at the time of capture could
`be essential to building organized albums.
`Cameras can easily be enhanced to auto-
`matically record several parameters, such as
`location and the photographer’s identity.
`Researchers have studied the benefits of using
`location coordinates to tag and organize digital
`images.12,13 The camera could use such location tags
`to organize images into viewlists associated with
`specific locations. Or annotations could be built on
`another device and transferred to the camera.
`Many of today’s cameras have no notion of accept-
`ing image content or metadata from other devices,
`so this shortcoming must be addressed as well.
`Because users can keep copies of images on other
`devices and also edit image albums on them, the
`camera’s stored images may not be synchronized
`with those on the other device. Thus, standard syn-
`chronization techniques have to be adapted for dig-
`ital cameras. If images are deleted from the Web
`repository, for example, the corresponding thumb-
`nails must be deleted on the camera as well. Given
`that some images will be ephemeral in nature, a
`mechanism must be provided to specify when those
`images can be automatically deleted.
`Users often replace cameras every few years, and
`they will need to transfer images from the old cam-
`era to the new one just as we transfer data between
`computers today. Adjustments may have to be
`made to account for the differences in the cameras,
`such as image resolution and viewfinder size and
`resolution. Techniques to manage large collections
`of images on PCs14 can be applied here as well.
`
`Quick search and retrieval
`Clearly, as the amount of storage in cameras
`increases, quick image retrieval and image man-
`agement will become a problem. Several ap-
`proaches can facilitate image retrieval. First and
`preferred, the input controls and software on the
`environmental display can be used to compose a
`query and overcome the camera’s input limitations.
`The camera could send JavaScript code to the envi-
`ronmental display and present a form from which
`the user selects search criteria. The criteria would
`include all the metadata captured with the image,
`
`December 2004
`
`69
`
`GoPro/Garmin
`EX. 1024, Page 005
`
`

`

`For maximum
`flexibility, the
`camera must include
`wired, short-range
`wireless, and
`long-range wireless
`communication
`capabilities.
`
`such as ranges for date, time, location, anno-
`tation subtext, aperture, focus and flash set-
`tings, photographer identity, privacy levels,
`and retention settings. The environmental
`display would then send the encoded query
`back to the camera to complete the search.
`Other criteria, such as “Find images simi-
`lar to the one selected,” can be provided
`using research in image similarity.15 User
`feedback to query responses can be used to
`improve image annotations and future
`searches.8
`Another approach uses embedded voice
`recognition technology to specify the query.
`A limited vocabulary recognition engine can han-
`dle most parameters. If the camera incorporates a
`phone, it would be logical to use voice for interac-
`tions.
`The fallback option uses the camera input con-
`trols to search for the images. For example, the user
`could position the dials used to specify image cap-
`ture modes to filter the images to ones that match
`the settings. Additional controls could be used to
`specify date ranges.
`In our experience, the most difficult parameter
`to specify is location. For our purposes, a map
`interface with zoom capability, similar to the one
`available on world clocks to select time zones,
`works best to specify a location.
`
`Security and privacy issues
`Given that cameras can have thousands of per-
`sonal images, it may be necessary to encrypt the
`images or at least use passwords or biometric
`means to protect them from unauthorized access.
`Borrowing from USB storage devices, which
`include fingerprint sensors and partition the flash
`into public and private sections, we can divide the
`camera storage into public and private areas. The
`private section can be accessed only when the user
`provides the right biometric or password. It might
`be useful to indicate the privacy level associated
`with each image at the time of capture.
`Because the camera will interact with unknown
`displays, it must be able to execute challenge-and-
`response sequences to establish the display’s iden-
`tity and authenticity. The camera’s display serves
`two purposes:
`
`• to preview images privately before deciding to
`view them on the larger public display, and
`• to show information from the environmental
`display that verifies its credentials and trust-
`worthiness.
`
`Network-attached displays will also need to
`enforce access controls, verifying that only autho-
`rized users can access the display. If the display has
`a short-range wireless interface or a direct wired
`connection, anyone who can connect to the display
`could use it. If the display is on an access-controlled
`network, anyone who has permission to use the
`network could use its services.
`The displays will also need to support the notion
`of a user session in which one authorized user con-
`trols the display and anyone else wanting to use it
`must wait until the session ends. Users must have
`the same confidence that such displays will not
`retain their content after use that they have with
`printers and audio speakers today.
`
`Network and power issues
`For maximum flexibility, the camera must
`include wired, short-range wireless, and long-range
`wireless communication capabilities. The advan-
`tages of a wired connection include higher speed,
`lower power, and simplicity because device dis-
`covery is easier. However, wired connections are
`prone to hardware malfunction caused by incor-
`rect attachment or mechanical fatigue. Moreover,
`with some displays, such as ceiling-mounted pro-
`jectors, it may be difficult to make wired connec-
`tions.
`Short-range wireless is useful for establishing
`trust relationships and transferring images from
`the camera to the display. With a long-range wire-
`less channel, the camera can fetch images from
`other sources and send them to the display over
`the short-range wireless channel. It also helps the
`camera obtain a fresh ephemeral token that grants
`permission to fetch and display images from the
`user’s network-accessible image repository.
`Long-range wireless channels consume a fair
`amount of power for large images and are slow.
`The camera can use long- and short-range chan-
`nels to establish trust between it and the display
`even if the camera cannot implement high-strength
`security algorithms. The camera can use the long-
`range channel to fetch a response to a challenge by
`communicating with a trusted proxy.
`
`CAMERA ARCHITECTURE
`To meet the requirements for different settings,
`designers must augment current digital cameras
`with features from other mobile devices. Our hypo-
`thetical cell phone camera, designed for effective
`symbiosis, integrates
`
`• a high-resolution imager,
`
`70
`
`Computer
`
`GoPro/Garmin
`EX. 1024, Page 006
`
`

`

`• a phone with a wide-area 3G capability,
`• a short-range wireless interface,
`• low-power storage,
`• high-density storage, and
`• a location-tracking mechanism.
`
`The camera also includes a wired USB connector
`for fast data transfer while powered by a host
`device. Figure 5 shows the architecture that incor-
`porates these features.
`The storage segments have a public and a private
`area. The camera shares the contents of the private
`area only when a user provides suitable authenti-
`cation. In addition to optional passwords, a fin-
`gerprint sensor authenticates each user. This sensor
`also records the photographer’s identity. The loca-
`tion-sensing mechanism records where the camera
`captured the image. In addition, the image captures
`several parameters, such as the aperture, light con-
`dition, and flash status.9
`The camera includes a high-density drive with a
`capacity of more than 100 Gbytes and its own private
`and public area. This drive functions as a secondary
`storage device so that the camera can use the flash
`memory as the primary storage device. To save
`power, the drive may be turned off most of the time.
`When the camera is plugged into an external power
`source, images can be moved from the flash memory
`to the drive. Additional dials record an image’s level
`of privacy, which can determine the section where
`the camera stores the image. Possible choices for who
`can view these images include immediate family,
`friends and family, or everyone. Another dial on the
`camera indicates how long the user expects to keep
`the image. If necessary, the privacy and longevity val-
`ues can be changed after the picture is taken.
`Although designers ensure that the camera’s dis-
`play is as large as its form factor allows, the reso-
`lution will still be substantially smaller than the
`resolution of the captured images. The camera
`includes voice input for recording annotations and
`embedded voice recognition technology for access-
`ing images.
`One downside to an integrated device is that if the
`user turns it off, all its functions become unavailable.
`For example, when a cell phone must be turned off
`in an airplane, the camera function also is unavail-
`able. Providing independent power controls for the
`camera, phone, storage, and other subsystems could
`overcome this limitation and save energy.
`
`APPLICATIONS
`The overall architecture we have described can
`be applied in several ways.
`
`Phone subsystem
`
`Long-range wireless
`Short-range wireless
`I/O controls
`
`Camera
`subsystem
`Lens
`Shutter
`Flash
`CCD
`I/O controls
`
`Display
`
`Other functions
`
`MP3 player, PDA, Wired USB,
`Location (GPS)
`
`Fingerprint
`sensor
`
`Longevity
`Privacy
`dial
`dial
`Low-power storage
`Public
`Private
`area
`area
`
`High-density storage
`Public
`Private
`area
`area
`
`Figure 5. Hypothetical digital cell phone and camera components. Designed to
`function in different settings, this camera incorporates components such as
`a high-resolution imager, a phone with wide-area capability, and a location-
`tracking mechanism.
`
`Collaborative image viewing
`Consider, for example, sharing an image album
`with guests at a party. The camera serves as a
`remote control for displaying the images, which are
`stored on a home PC connected to a projector
`through a high-speed link. In this scenario, the cam-
`era must display the selected images without reveal-
`ing any pictures the host chooses not to show.
`An initial phase connects the camera, PC, and
`projector. This approach simplifies network bound-
`ary and authentication issues because the devices
`function in a friendly environment where they trust
`each other. In this situation, the camera has a
`thumbnail view of the images along with the anno-
`tations and links to the images on the home PC.
`Even though the camera may store the complete
`images, they can be fetched faster from the home
`PC, which helps sustain audience interest.
`The user can look at the thumbnails on the cam-
`era, decide if an image is suitable for public view-
`ing, then tap the thumbnail. The application then
`shows any notes the user might have associated
`with the image such as the date and circumstances
`when the image was taken. Private audio annota-
`tions can be delivered to the camera headphone
`directly. While the projector shows the image, the
`user can advance the camera’s viewer and queue up
`other images. The user can skip any images deemed
`unsuitable for this audience.
`To simplify the image selection process, the user
`can create viewlists in advance. In this case, the sys-
`tem prefetches the content, if possible, to reduce
`image loading delays. Based on comments received
`from the audience, the camera’s viewlist can be
`updated. If someone in the audience requests a copy
`of an image and the camera’s address book has an
`entry for that person, a simple action e-mails the
`image from the home PC.
`
`December 2004
`
`71
`
`GoPro/Garmin
`EX. 1024, Page 007
`
`

`

`Figure 6. Shopper
`assistance. Digital
`camera images can
`help shoppers
`purchase the
`correct parts for
`their home
`hardware, such
`as these furnace
`controls.
`
`In-vehicle symbiosis
`Those who travel by car, train, or plane could
`find new uses for digital cameras. Many cars today
`have navigation consoles that show maps and other
`information. Further, several high-end cars have
`rear-view cameras to assist drivers while reversing.
`Similarly, some cars have front-view cameras that
`monitor road markers and signal when the driver
`strays from the lane. Others have started including
`Bluetooth wireless communication within the car
`to facilitate hands-free cell phone operation, while
`still others have begun offering voice-based opera-
`tion of some subsystems. Digital camera users can
`exploit all these features.
`For example, in 1997, we built a prototype sys-
`tem that annotated pictures captured with a Ricoh-
`RDC1 digital camera with GPS location metadata
`to augment textual directions with turn-by-turn
`images.9 We sought to provide pictures of signifi-
`cant points along the route, such as important inter-
`sections, landmarks, exit signs, entry and exit
`ramps, and so on, along with text directions. To do
`so, we captured the images ahead of time, manu-
`ally. We then added these images to a database and
`annotated them with GPS coordinates and direc-
`tional information. When the system computed
`directions between two places, it fetched appro-
`priate images from the database and included them
`as hyperlinks.
`Given current automotive developments, our
`camera could store images of appropriate intersec-
`tions, signs, and other features before a trip, then
`supply them to the onboard display during the jour-
`ney. The short-range wireless interface transfers the
`images from the camera phone to the navigation
`console at the trip’s start. The navigation system
`then shows the pictures at appropriate junctures,
`as Figures 7a and 7b show. The limited amount of
`detail visible in Figures 7c and 7d make the point
`that we need larger displays.
`To compile our database of intersection images,
`we can draw on the experience gleaned from the
`thousands of users who created CDDB, a

This document is available on Docket Alarm but you must sign up to view it.


Or .

Accessing this document will incur an additional charge of $.

After purchase, you can access this document again without charge.

Accept $ Charge
throbber

Still Working On It

This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.

Give it another minute or two to complete, and then try the refresh button.

throbber

A few More Minutes ... Still Working

It can take up to 5 minutes for us to download a document if the court servers are running slowly.

Thank you for your continued patience.

This document could not be displayed.

We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.

You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.

Set your membership status to view this document.

With a Docket Alarm membership, you'll get a whole lot more, including:

  • Up-to-date information for this case.
  • Email alerts whenever there is an update.
  • Full text search for other cases.
  • Get email alerts whenever a new case matches your search.

Become a Member

One Moment Please

The filing “” is large (MB) and is being downloaded.

Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!

If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document

We are unable to display this document, it may be under a court ordered seal.

If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.


Access Government Site

We are redirecting you
to a mobile optimized page.





Document Unreadable or Corrupt

Refresh this Document
Go to the Docket

We are unable to display this document.

Refresh this Document
Go to the Docket