as) United States
`a2) Patent Application Publication co) Pub. No.: US 2008/0306921 Al
`(43) Pub. Date: Dec. 11, 2008
`
`Rothmulleret al.
`
`US 20080306921A1
`
`(54) DIGITAL MEDIA MANAGEMENT
`APPARATUS AND METHODS
`
`(76)
`
`Inventors:
`
`Kenneth Rothmuller, Santa Rosa,
`CA (US); Laurie Vertelney, Palo
`Alto, CA (US); Bernard L. Peuto,
`San Francisco, CA (US); Michael
`Slater, Sebastopol, CA (US)
`
`Correspondence Address:
`FISH & RICHARDSON P.C.
`P.O. Box 1022
`
`MINNEAPOLIS, MN 55440-1022 (US)
`
`(21) Appl. No.:
`
`12/190,515
`
`(22)
`
`Filed:
`
`Aug. 12, 2008
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 10/198,618, filed on
`Jul. 17, 2002, now Pat. No. 7,415,662, which is a
`continuation of application No. 10/052,213, filed on
`Jan. 16, 2002, now abandoned, which is a continua-
`tion-in-part of application No. 09/774,523, filed on
`Jan. 31, 2001, now abandoned.
`
`(60) Provisional application No. 60/261,897, filed on Jan.
`16, 2001, provisional application No. 60/179,379,
`filed on Jan. 31, 2000.
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 17/30
`(52) US. CMe coeeccccssccsssssssssssssenseeseeses 707/3; 707/E17.014
`
`(57)
`
`ABSTRACT
`
`Methods and apparatus for managing,finding and displaying
`objects such as digital images. Objects are tagged (“associ-
`ated”) with descriptive textual and numeric data (“meta-
`data”), and stored in a relational database from which they can
`be selected, sorted, and found. Tags can be defined by name,
`tag type, and associated attributes. Objects can be tagged by
`dropping a tag onto the object, or relating a database record
`for the tag to a database recordfor the object. Tagged objects
`can be searched for and displayed according to the degree to
`which their metadata matches the search criteria. Visual cues
`
`can indicate whether displayed objects match all, some but
`notall, or none ofthe search criteria. Database object distri-
`butions can be displayed as histograms or scatter plots,
`including timelines, calendars or maps. Object distributions
`can be used to search for objects or to limit search results for
`a previous search.
`
`FINDER
`Timeline
`View Controls
`Search Criteria
`DRAWER
`
`210 GevView—Print 530Day Select 50
`
`
`
`
`520110° Albums 540
`200
`
`
`
`Nournal-Michgel"'s Photo Co
`220
`
`show
`550
`
`start a Slide
`
`
`+ (23 tags
`Tag
`Racentty Used
`9 Favorite
`Lp
`Pioni
`Efi Mary Jane
`& Kenya 96
`
`By Landscapes
`Efi Lon
`Ey animate
`
`Search status 260
`Make new tag 340
`
`Recently
`used tags 290
`
`Quick scroll 310
`
`TAG 300
`KEEPER
`Type tags 320 ,
`
`0 selected tags to 0 selected photos
`
`(+1714 Photosin: Kenya 1993...
`
`Image Area 100
`
`Project Area 400
`
`SAMSUNG 1033
`
`SAMSUNG 1033
`
`1
`
`

`

`Patent Application Publication
`
`Dec. 11, 2008 Sheet 1 of 6
`
`US 2008/0306921 Al
`
`
`
`MOUsmo
`
`
`
`
`
`
`
`bBiz
`
`OOrBauyyealoid001Bauyabewy
`
`
`
`BoaoeeeaheThWNOserr
`
`sunongsunaiyigMaINNODoszosyoeresArgOLZ00¢
`
`meeipo010Uda02S.OL
`
`
`S0}OUd,Itee/{|{srrah[reall0zz
`
`
`
`
`SU/;OUWELLSJONUODMOIA,BHOIDYoieagYAamMvud
`
`
`wee)BRBREE)(|(Greounenespex@weaovie0ud
`sedeospue)EyeyuoaeyoO
`
`sewuyDyquerMewwb.shiz}pasn
`
`uo}aedeeAnuasey
`
`
`pesnAnueve|OveBe)mouoyey
`
`Q9ZshyeysYyouras
`
`Aepo,-96,uerusemjeguaye{\josey“UMAR
`
`
`
`lege8661
`
`6,eAuey&)
`
`MSONId
`
`
`
`
`
`OLEjpor9s4OINH
`
`ooeSVL
`
`2
`
`
`

`

`Patent Application Publication
`
`Dec. 11, 2008 Sheet 2 of 6
`
`US 2008/0306921 Al
`
`
`
`
`
`Asewquny:woiyposeys
`
`foues
`
`
`
`
`
`SOJON010dwudg6:zWYbov6z/zhUO}OUS
`
`(gy9zZ)OBZXBPOZ
`
`
`
`sojoyd/LOLZe}a/sojoyds:abew)
`
`
`
`puejoBey:moysepiisuy
`
`“wnalyul
`
`
`
`
`
`“"*LOVZ8‘LOYOL/B‘UOPell
`
`[e][iS]BuOHOHOHE
`
`
`
`
`
`Apuy‘sis:yimpezeus
`
`w
`
`=
`}eAGI
`
`HE|ep
`
`fe] i)
`
`ehe
`
`[e]
`
`=3u
`
`3
`
`

`

`Patent Application Publication
`
`Dec. 11,2008 Sheet 3 of 6
`
`US 2008/0306921 Al
`
`250 CL
` LXif
`
`
`
`VMEZZ
`
`
`251
`
`Fig.3
`
`
`
`iy Y 251 woa
`
`
`
`250
`
`Li
`
`—~
`
`~
`
`re|
`
`4
`
`

`

`Patent Application Publication
`
`Dec. 11,2008 Sheet 4 of 6
`
`US 2008/0306921 Al
`
`View
`<>
`
`
`
`
`
`
`
`A > 5N >
`
`qJune
`
`June18,20013:45pm(CloseMatch)
`
`Fig.4
`
`5
`
`

`

`Patent Application Publication
`
`Dec. 11, 2008 Sheet 5 of 6
`
`US 2008/0306921 Al
`
` ;,euyeusynbounn
`
`
`demsasnoyH»
`Sp;doysysomLE“fby|ojoudAler}
`wv3
`
`AeMION
`
`S$Sid
`
`ueequuey
`
`asina
`
`
`
`UlS,OpeAA
`
`Oo1xa;w
`
`6
`
`
`

`

`Patent Application Publication
`
`Dec. 11, 2008 Sheet 6 of 6
`
`US 2008/0306921 Al
`
`00,Auewsa5‘aubBojo9
`
`
`
`
`00,AuRewued‘aubojo4
`
`
`
`SpuNnos}@al]S
`
`9‘Bis
`
`069wnavOL)
`
`029moysapis[Ll
`
`
`0L
`
`
`
`2opny(Chl
`
`7
`
`
`
`
`
`

`

`US 2008/0306921 Al
`
`Dec. 11, 2008
`
`DIGITAL MEDIA MANAGEMENT
`APPARATUS AND METHODS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application claims the benefit of priority to
`USS. application Ser. No. 10/198,618, filed Jul. 17, 2002,
`which claimsthe benefit of Ser. No. 10/052,213, filed Jan. 16,
`2002, which claimsthe benefit of priority to U.S. application
`Ser. No. 09/774,523, filed Jan. 31, 2001, which claimsprior-
`ity to U.S. provisional application Ser. No. 60/261,897,filed
`Jan. 16, 2001, U.S. provisional application Ser. No. 60/179,
`379,filed Jan. 31, 2000, and U.S. provisional application No.
`60/334,516 filed Oct. 31, 2001, the disclosure of which is
`incorporated by reference.
`
`BACKGROUND
`
`[0002] With the advent of digital photography and the
`world-wide-web, there has been an exponential growth in the
`creation and storage of digital photographic images. As the
`numberofdigital photographs taken and stored has grown,so
`too has the need for a convenient methodofarchiving,cata-
`loguing, searching, and retrieving them. Modern methodsof
`archiving andstoring digital images typically require users to
`remember large amounts of information merely to locate
`photos that are of particular interest to them. For example,
`many users currently store their digital images in the hierar-
`chical, directory-based file system structure that is native to
`personal computers. To find particular photos stored in such a
`hierarchical directory tree or structure, users must know the
`full pathnametothe directory in which their photographsare
`stored.
`
`[0003] There are other disadvantagesto storing digital pho-
`tographs in a hierarchical, directory-based file system. For
`example, cataloguing and storing groups ofphotos by catego-
`ries such as vacation photos or wedding photos requires cre-
`ating different directories for each of the desired categories.
`This further increases the amountof information that must be
`
`rememberedin order to locate desired photos. In addition, in
`order to store photos in two or more overlapping categories,
`such as photos that include yourfavorite aunt and photos from
`your cousin’s wedding, users musteither store duplicate pho-
`tographs, or master the concepts of directory trees andfile
`pointers. While these are not difficult concepts for sophisti-
`cated computer users,
`they can be troublesome for less
`sophisticated users, thereby limiting the useful ways these
`users can store andretrieve digital photographs and photo-
`graphic information.
`
`SUMMARY
`
`[0004] The present invention relates to methods and appa-
`ratus for storing, cataloguing, managing, organizing, finding
`and displaying objects such as digital images. The invention
`includes methods for associating (“tagging”) fields of text
`and numeric data (“metadata”) with individual objects such
`as imagesor photos, storing the objects and associated meta-
`data as records ina relational database, and selecting, sorting,
`organizing andfinding the objects based on their tagged meta-
`data content.
`
`[0005] Default metadata tags can be specified, and new
`metadata tags can be defined and created througha tag editor
`by namingthe tag, selecting its tag type, optionally selecting
`a graphical icon that represents the tag, and filling in any
`
`remainingfields orattributes that are unique to and define the
`tag type. Tags can be readily associated with an object by
`adding a record containingthe tag information or metadata to
`a database, and relating the tagged metadata record to a data-
`base record containing the object or a pointer to the object.
`Tags can also be graphically associated with an object by, for
`example, dragging and dropping a graphical icon represent-
`ing the tag onto a graphical representation ofthe object. In the
`latter case, database records containing the tag metadata are
`automatically created and related to the database record con-
`taining the target object or a pointer to the target object.
`[0006] Once objects have been tagged with metadata, they
`can be searched for according to one or more tagged search
`criteria. When the objects to be search for are photos, these
`search criteria can include, but are notlimitedto, the date and
`time the photos were taken, textual information that is asso-
`ciated with the photos such as the namesofthe people whoare
`in the photos or the places or events where the photos were
`taken, designations of the photos as favorite photos, and des-
`ignation ofthe photosas photos that have been printed, shared
`with others, or archived on a certain date.
`[0007] When a database is searched for objects that match
`one or more tagged search criteria, the matching objects can
`be viewed or arranged according to the degree to which they
`have associated metadata that matches the searchcriteria. In
`particular, objects that match all of the search criteria can be
`displayedfirst, followed by objects that match one or more of
`the search criteria, and finally by objects that match none of
`the search criteria. Objects in the different match groups can
`be differentiated from one another in the display area by
`visual cues, such as being displayed in front of different
`background colors or patterns. Thus, objects matching all of
`the search criteria can be displayed in front of a white back-
`ground, while objects matching someofthe search criteria
`can be displayed in front of a blue background, and objects
`matching noneofthe searchcriteria can be displayed in front
`of a gray background.
`[0008]
`Thedistribution ofthe objects stored in the database
`can be displayed as a histogram along a timeline. Time bands
`can be set along the timelineto indicate a time period that can
`be used to search for matching objects in the database, or to
`limit the search results for a given tag search to objects having
`temporal metadata within the indicated time period. When the
`timeline is used to limit the search results for a tag search, the
`timeline displays not only the temporal distribution ofall
`objects in the database overthe indicated time period, but also
`the temporaldistribution ofall objects in the database match-
`ing the specified tag search criteria over the indicated time
`period.
`Inaddition to timelines, the temporaldistribution of
`[0009]
`objects in the database can be represented in a calendar view
`such that the days of the calendar indicate the number of
`objects having metadata associated with a given day of the
`week in a given week of the month. The calendar view can
`also be used to limit the search results for a tag search, in
`which case the calendar view will indicate all of the days of
`the month associated with objects that matchall of the tagged
`search criteria, match someof the tagged search criteria, and
`match noneof the tagged search criteria.
`[0010] The details of one or more embodiments of the
`invention are set forth in the accompanying drawings and the
`description below. Other objects, features, and advantages of
`the invention will be apparent from the description and draw-
`ings, and the claims.
`
`8
`
`

`

`US 2008/0306921 Al
`
`Dec. 11, 2008
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1illustrates one embodimentofa userinterface
`[0011]
`for a computer program product in accordance with the
`present invention.
`[0012]
`FIG.2 illustrates an image displayed with its asso-
`ciated metadata, including its tags, in accordance with the
`present invention.
`[0013]
`FIG. 3 illustrates a timeline view of the data in
`accordance with the present invention.
`[0014]
`FIG. 4 illustrates a calendar view of the data in
`accordance with the present invention.
`[0015]
`FIG.5 illustrates a map view of the data in accor-
`dance with the present invention.
`[0016]
`FIG. 6 illustrates the display of different media
`types that are stored in accordancewiththe presentinvention.
`
`DETAILED DESCRIPTION
`
`[0017] The present invention provides a methodfor users to
`organize andfind digital images and photos by tagging them.
`Before being tagged, photos must be imported into a database
`where photographic metadata or information about the pho-
`tos can be stored. While entire photos can be stored in the
`database, it is generally more efficient to store pointers to
`photos in the database rather than the photos themselves.
`Photos can be imported into the database from any of a
`numberof devices or sources including, but not limited to, a
`digital camera, a flash memory device, a hard disk drive, a
`floppy drive, a CD-ROM,or a networked computerorfile
`server. Once imported into the database, the photos can be
`tagged with one or more objects containing metadata that
`identifies the unique or important properties ofthe photo such
`as when or where the photo was taken, or who or whatis the
`subject of the photo.
`[0018] Asshownin FIG.1, in one embodimenttags 350 can
`be applied to photos by dragging and dropping graphical
`icons representing the tags onto one or more photos 1-4 that
`are displayed in an image area 100. Whena tag is dropped
`onto a photo, the database record that contains a pointerto the
`photo is updated to contain or point to metadata that is asso-
`ciated with the tag that has been dropped onto the photo. This
`metadata can include whenthe photo was taken, whereit was
`taken, the nature ofthe event at which it was taken, the subject
`of the photo, and whetherthe user considers the photo one of
`his or her favorites. Once tagged, photos with specific tags or
`combinationsof tags can be readily foundin the database by
`searching the database for all records that contain the same
`metadata as the metadata that is associated with the one or
`moresearch tags.
`[0019] Tags, and the metadata they contain, can be created
`and modified in a tag editor. The tag editor allows a user to
`specify a tag nameandtag type, and to enter metadata in the
`form oftag attributes that can be stored in tags ofthe specified
`tag type. For convenience, tags can be divided into one or
`moretag categories. For example, in one embodimenttags are
`divided into people, events, places and miscellaneous tag
`categories. Tags in the different tag categories generally have
`different tag attributes to distinguish between themselves and
`tags in other tag categories. In general, a tag’s attributes do not
`need to befilled in to associate a tag with a photo. The tag
`itself is a form of metadata that can be associated with the
`photo, regardless of whether the tag’s possible attributes are
`also associated with the photo. However, when a tag’s
`
`attributes are completely orpartially filled in, more metadata
`is associated with the tagged photo, thereby making the photo
`easier to search for andfind.
`
`[0020] The people tag category includes default tag types
`for family and friends, and can be customizedto include other
`groups of people such as business associates, classmates,
`co-workers, and neighbors, and particular individuals such as
`a spouse, daughter, or friend. Tags in the people category can
`contain attributes such as a person’s name, sex, birthdate,
`anniversary, postal and/or email address(es), phone number
`(s), a sharing profile specifying which if any pictures can be
`shared with the people associated with the tag, and the rela-
`tionships between the people associated with the tag and
`other tagged individuals.
`[0021] The events tag category includes default tag types
`for parties and vacations, and can be customized to include
`tag types for particular types of events such as concerts, plays,
`showsand sporting events, and for particular events such as
`the 2002 Boston Marathon. In addition, tags in the events
`category can include pre-defined calendar events such as New
`Years Eve, and customized calendar events such as birthdays
`and anniversaries. Tags in the event tag category can contain
`attributes corresponding to the names, locations, and dates of
`the underlying events associated with the tags.
`[0022] The places tag category can be customized to
`include tag types for particular places such as a home, an
`office, an art museum,or a vacation destination. Tags in the
`places tag category can contain attributes corresponding to
`specific locations that are associated with photos, including
`the name of the location (e.g., The Metropolitan Opera
`House), the namesofthecity, state, country and region ofthe
`world in which the photos were taken or whichare the subject
`of the photos, and the geographical coordinates (e.g., longi-
`tude andlatitude) for those places.
`[0023]
`Finally, the miscellaneous tag category is as a cus-
`tomizable catchall for tags that cannot be easily groupedinto
`a meaningful global category with other tags. Examples of
`miscellaneous tag types include tags for an apartment or
`homesearch,tags forartistic or photos, andtags for particular
`cars or types of cars. Miscellaneous tags can contain
`attributes corresponding to the name of the subject of the
`photo, and where and whenthe photo wastaken.
`[0024] As shownin FIG.2, the metadatathat is associated
`with a photo can be viewed and edited directly by displaying
`the photo together with its associated metadata. FIG. 2 shows
`a photo entitled “Lori on the road at Legoland”associated
`with a customized people tag, Lori R., and a customized
`places tag, San Diego. The tags andtitle indicate this is a
`photo of Lori R. taken on a trip to Legoland in San Diego,
`Calif. This photo can be retrieved from the database in any
`numberof different ways, together with different photos that
`are related to this photo in different ways, as discussed below.
`[0025]
`In general, photos in the database that have been
`tagged with one or more tags can be searched for and sorted
`by queryingthe databasefor all photos having tags that match
`one or moresearch tags or the metadata contained within the
`one or more search tags. These metadata can include, but are
`not limited to, data indicating whether photos are favorites;
`frequently viewed; similar to currently selected photos;
`untagged; taken ona particular day or recurring event; shared
`with or received from certain people; imported from certain
`places; and printed or exported on certain dates. In addition,
`the metadata can include the subject of the photo, whether a
`person, place, or event; as well as the place and/or event at
`
`9
`
`

`

`US 2008/0306921 Al
`
`Dec. 11, 2008
`
`image area 100. For example, the timeline 250 can be used by
`whichthe photo wastaken. For example, the photo of Lori R.
`itself to find all photos taken between Jan. 1, 2000 and Feb.
`in Legoland can be retrieved from the database by querying
`28, 2000 by moving the adjustable time bands 251 to these
`the database for all photos tagged with a Lori R. tag. This
`two respective dates. The photos in the database that have
`search will pull upall photos of Lori R., including the Lego-
`been tagged with a timestampfalling between these two dates
`land photo, regardless of where the photos were taken. Alter-
`can beretrieved from the database, and displayedin the “best
`natively, the Legoland photo canberetrieved by searching the
`match”section of image area 100.
`database for all photos tagged with a San Diego tag. This
`search will pull up all photos taken in or of San Diego,
`[0030]
`In addition to finding photos according to their
`including the Legoland photo, regardless of who is in the
`timestamp, the timeline 250 can be used with other metadata
`photo. Finally, the Legoland photo can be retrieved by search-
`to limit search tag results. For example, if the adjustable time
`ing the databasefor all photos tagged with both a Lori R.tag
`bands 251 of timeline 250 indicate the period of interest
`and a San Diego tag. This search will pull up all photos taken
`extends from Jan. 1, 2000 to Feb. 28, 2000, searching the
`in or of San Diegothat include Lori R, including the Legoland
`databasefor all photos having a San Diego tag will return the
`photo.
`photo “Lori on the road at Legoland”as a “best match”photo,
`[0026] The database search for photos that match certain
`and display the photo in imagearea 100, only ifthe photo was
`tags or groups of tags can be graphically constructed by
`taken sometime between Jan. 1, 2000 and Feb. 28, 2000. Ifthe
`dragging various icons representative of tags 350 into a
`photo was taken outside of this time period, it would only
`graphical query builder or lens 220, and searching the data-
`appear as a “close match”photo in image area 100. Whentag
`base for records with matching tags or metadata. When search
`searches are conducted in conjunction with timeline 250, the
`criteria are applied to the photos in the database, the order in
`timeline displays the total number of photos in the database
`which the photos are displayed is updated so that “best
`per unit time period inafirst color which may beasolid color,
`match” photos or photos that match all of the search criteria
`and the total numberof photos in the database that match the
`are displayedat the top of an image area 100 in front ofafirst
`taggedsearchcriteria as “best” or “close” matches in a second
`backgroundcoloror pattern, while “close match” photos that
`color which may be a hatched pattern or color.
`match one or more but notall of the search criteria are dis-
`[0031]
`In one embodiment,the timeline 250 shownin FIG.
`played after the “best match” photos andare visually distin-
`3 does not display the exact numberof photos taken during a
`guished from them by, for example, being displayed in front
`given period of time, but rather displays a vertical bar graph
`of a second background color or pattern, and “no match”
`with bar heights that are representative of the number of
`photos that fail to match any of the search criteria are dis-
`photos taken during a given period of time normalized to the
`played at the bottom of the image area in front of a third
`average numberof photos taken during all such similar peri-
`backgroundcoloror pattern.
`ods oftime in the database. For example, for a given period of
`[0027]
`Perhaps the easiest search to conduct on tagged
`time,the displayedvertical bar can have a height of O when no
`photosis a search for photos taken on a certain date, or within
`photos have been taken during that period; 1 when oneto five
`acertain period oftime. As previously mentioned, among the
`photos have been taken during that period; 2 when the nor-
`metadata that can be stored with a photo is information indi-
`malized numberof photos taken during that period was up to
`cating the date and time a photo was taken. This information
`50% of the average numberof photos taken during all time
`is often automatically associated with a photo whenthe photo
`periods; 3 when the normalized numberof photos taken dur-
`is created or whenthe photo is scannedinto a digital scanner.
`ing that period was between 50% and 80% of the average
`If the photo is created on a digital camera, the camera will
`numberof photos taken during all time periods; 4 when the
`generally tag the photo with the date and time the photo was
`normalized number of photos taken during that period was
`taken. If the photo is scannedinto a digital scanner, the scan-
`between 80% and 120% of the average number of photos
`ner will generally tag the photo with the date and time it was
`taken duringall time periods; 5 when the normalized number
`scanned. If for any reasons neither the digital camera nor
`of photos taken during that period was between 120% and
`digital scannertags the photo with the date and time informa-
`150% of the average numberof photos taken duringall time
`tion, the database will tag the photo with the information
`periods; 6 when the normalized numberof photos taken dur-
`whenitis first imported.
`ing that period was between 150% and 200% ofthe average
`numberof photos taken during all time periods; and 7 when
`[0028] As shown in FIG. 3, when photos are imported into
`the normalized numberofphotos taken during that period was
`a database, the temporal metadata associated with the photos
`more than 200% of the average numberof photos taken dur-
`can be usedto present a histogram of photos in the form ofa
`timeline 250 as shown in FIG. 1. The timeline 250 can show
`ing all time periods.
`[0032] As shown in FIG. 4, in addition to timeline 250,
`photos taken on a particular day or during a particular month
`can also be foundby displaying the photos in a 2-D histogram
`or scatter plot such as the calendar view showninthefigure.
`The calendar view displays all of the photos that have been
`taken, scanned, or imported into the database on any day in a
`given month as a function of the day of the week the photos
`were taken, and the week in the month. Ifa particular day of
`the month is selected in the calendar view,all photos taken on
`that day can beretrieved from the database as “best match”
`photos. For example, FIG. 4 showsthat during the month of
`June 2001 twosets ofphotos were taken. Thefirst set contains
`a single photo taken on June 8, while the secondset contains
`10 photos taken on June 18. Byselecting the June 18 calendar
`
`the number of photos taken as a function of time over some
`period of time that can range from the timethefirst photo in
`the database was takento the present. The timeline 250 can be
`used byitself, or with other tags 350 to specify the criteria
`used to search for matching photos. The timeline includes
`adjustable time bands 251 that can be moved to allow timeline
`250 to specify the time period that is used to find matching
`photos.
`[0029] Whenthe timeline 250 is used by itselfto search for
`matching photos, the adjustable time bands 251 can be moved
`to find all photosin the database that are tagged with a date or
`timestamp that falls within the range indicated by the adjust-
`able time bands 251. Photos falling within this range are
`designated “best match”photos, and can be viewed as such in
`
`10
`
`10
`
`

`

`US 2008/0306921 Al
`
`Dec. 11, 2008
`
`tions containing one or more photos in the “close match”
`day, the 10 photos taken on June 18 are selected as the “best
`group and no photos in the “best match” group can be pre-
`match”photos, and can be displayed in image area 100.
`sented as a blue area, and locations containing no photos in
`[0033] The calendar view can also display the results of a
`either the “best match” or “close match” groups can be pre-
`tag search in the month-at-a-glance mode. When so used,
`sented as a gray area.
`each day in the calendar can indicate not only whether any
`[0037] The map view can be varied in size and shape to
`photos were taken on that day, but whether the photos taken
`accommodate the geographic extent of the photos that are
`on that day fall into the “best match”, “close match”, or “no
`either in the database or that are responsive to a tag search
`match” group with respect to the tagged search criteria. For
`conducted on the photos in the database. Thus, if a map view
`example, ifthe Legoland photo described in FIG. 2 was one of
`is used to display or further search among database photos
`ten photos of Lori R. taken in San Diego on Jun. 18, 2001, and
`having a North American tag, the map view can be limited to
`a search were doneforall photos having a San Diegotag, then
`a view of the North American continent. The map view can
`the Jun. 18, 2001 square in FIG. 4 would indicate that day as
`also bevaried in size and shapeby selecting particular regions
`having photosin the “best match” group. If, however, a search
`of a map, such as the western region of the United States, or
`were donefor all photos having a New Yorktag, the Jun. 18,
`by zoomingin and out of the currently displayed map region.
`2001 square in FIG. 4 would indicate that day as having
`Photos corresponding to particular locations within a map
`photosin the “no match”group. Finally, ifa search were done
`view, like San Diego, can be geographically found directly
`for all photos having a New York tag and a Lori R.tag, the Jun.
`from the map view.
`18, 2001 square in FIG. 4 would indicate that day as having
`photos in the “close match” group.
`[0038] Once photos matching a givenset of tags are found,
`they can be selected as a group, and various photo manage-
`[0034] The particular group into whichaset ofphotos taken
`ment functions such as printing, sharing, or exporting the
`on a given calendar day falls can be indicated on the calendar
`photos to a slide show orto a photo album can be performed
`using the samecolor based indication schemeusedto indicate
`on the group. As a default, all photos in the “best match”
`matching photo groupsthat are displayed in the viewing area.
`group are selected when selecting photos that match a given
`Thus, calendar days containing one or more photos in the
`set oftag search criteria. However, the default can be changed
`“best match” group can be presented as white squares, while
`such that all photos in both the “close match” and “best
`calendar days containing one or more photos in the “close
`match”groups are selected when selecting photos that match
`match” group and no photos in the “best match” group can be
`a given set of tag search criteria.
`presented as blue squares, and calendar days containing no
`[0039]
`It should be noted that while the invention has been
`photosin either the “best match”or “close match” groups can
`be presented as gray squares.
`described in terms of managing, cataloguing, searching, and
`finding digital images and photographs, the invention can
`[0035] As shownin FIG.5,in addition to the timeline 250
`equally be used to manage, catalogue, search for and find
`and calendar views, data can be searched for and displayed in
`other types of digital media such as videofiles, audiofiles,
`an alternate 2-D histogram orscatter plot such as a map view.
`photo slide shows, and photo albums. These different types of
`In the map view,the place tag metadata is used to display the
`media can be distinguished from one anotherwithatag oftag
`geographic distribution of photos in the database. Like the
`type media. The media tag, when applied to a media object,
`timeline 250 and calendar views, the map view can be used to
`can graphically indicate the type ofmedia objectthat is stored
`search for photos eitherby itself, or in conjunction with one or
`in the database.
`more tag searches. If the map view 1s usedbyitself to search
`for photos, icons representing the places where photos in the
`database have been taken are displayed on a map such as the
`world map shown in FIG. 5. When a location on the mapis
`selected, photos taken in that location can be retrieved from
`the database as photos in the “best match” group. For
`example, if the location of Kenya on the map in FIG. 5 is
`ponentpart ofa slide show or photo album,and can be played
`selected, photos taken in Kenya can be selected from the
`as a soundtrack wheneverthe slide show or photo album is
`database, and can be displayed in image area 100 as “best
`viewed. Slide shows such as slide show 620, and photo
`match” photos.
`albums such as photo album 630 can also be stored in the
`[0036] The map view canalso be used in conjunction with
`database, and iconically identified as shown FIG. 6. Each of
`atag search. Whenso used, the map view will display not only
`these objects can be tagged, searched for, and manipulated
`the distribution of photos as a function of geographic loca-
`using the same tools that are used to tag, search for, and
`tion, but whether the photos taken at the various geographic
`manipulate digital photos, as previously discussed.
`locations fall in the “best match”, “close match”, or “no
`[0041] While the invention has been described as a com-
`match” group with respect to the tagged search criteria. For
`puter program or algorithm, the invention can be imple-
`example, if a search for all photos having an African tag were
`mented in digital electronic circuitry, or in computer hard-
`displayed in the map view, the map view wouldindicate that
`ware, firmware, software, or in any combinations of them.
`photos taken from the KenyaSafari fall into the “best match”
`Apparatusofthe invention can be implementedin a computer
`group, whileall of the other photos shownin FIG.5 fall into
`the “no match”group. As with the calendar and timeline 250
`program product tangibly embodied in a machine-readable
`
`views, the particular group into whichaset of photos taken storage device for execution by a programmable processor;
`from a given location falls can be indicated on the map using
`and method steps of the invention can be performed by a
`the same color based indication scheme used to indicate
`programmable processor executing a program ofinstructions
`to perform functions of the invention by operating on input
`data and generating output. The invention can be imple-
`mented advantageously in one or more computer programs
`
`[00