(12) United States Patent
`Sharpe et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7.461,099 B1
`Dec. 2, 2008
`
`USOO7461099B1
`
`6,438,539 B1* 8/2002 Korolev et al. ................. 707/3
`FOREIGN PATENT DOCUMENTS
`O 637 807
`2, 1995
`O 637 807 A2
`2, 1995
`O 678 816 A2 10, 1995
`Continued
`(Continued)
`OTHER PUBLICATIONS
`Shneiderman et al. (Direct Annotation: A Drag-and-Drop Strategy
`for Labeling Photos, Aug. 2000.
`(Continued)
`Primary Examiner John Breene
`Assistant Examiner—Baoquoc N. To
`(74) Attorney, Agent, or Firm Kenyon & Kenyon LLP
`(57)
`ABSTRACT
`A method and system for archiving and retrieving digital
`media items based on episodic memory of predefined asso
`ciated groups of one or more people comprises: receiving an
`input identifying a group to which the user belongs; receiving
`user archiving input identifying a digital media item to be
`archived for the group, selecting Zero or more group event
`types from a predetermined plurality of group event tapes for
`the group, selecting Zero or more persons in the group, and
`selecting a time; generating index information using the
`received user archiving input; Storing the indexed information
`in association with the identified digital item; repeating the
`reception of user archiving input, the generation of the index
`information and the storing of the index information for a
`plurality of digital media items; receiving a user retrieval
`input selecting or automatically selecting: Zero or more
`groups of event types from the predetermined plurality of
`group event types for the group, Zero or more persons in the
`group, and a time period; and using the selections and the
`identified group to retrieve and output digital media items that
`match the selection.
`
`ck
`
`-
`
`-
`
`-
`
`EP
`EP
`EP
`
`(54) METHOD AND SYSTEM FOR ARCHIVING
`AND RETREVING TEMIS BASED ON
`Epic MEMORY OF GROUPS OF
`
`(75) Inventors: Elizabeth Sharpe, Edinburgh (GB);
`John Leslie Fraser, Edinburgh (GB)
`
`(73) Assignee: 6S, Limited, Edinburgh (GB)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 862 days.
`
`(21) Appl. No.: 09/670,635
`(22) Filed:
`Sep. 26, 2000
`(51) Int. Cl.
`(2006.01)
`G06F 7/30
`(52) U.S. Cl. .............................. 707/204; 707/3; 707/10
`(58) Field of Classification Search .............. 707/104.1,
`707/204, 1, 3, 4, 204.1, 10
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`1/1996 Astle ............................. 707/1
`5,485,611 A *
`2/1996 Balogh et al. ............... 395/600
`5,493,677 A
`6/1996 Krause ....................... 395/600
`5,526,520 A
`5/1997 Braden-Harder et al. .... 707/102
`5,630,121 A *
`1/1998 Dwyer et al. ............... 395,349
`5,706,457 A
`2, 1998 Wolfe ......................... 395/605
`5,715,445 A
`5,828,904 A * 10/1998 Batson et al. ............... TO9,102
`5,835,667 A 1 1/1998 Wactlar et al. ..
`... 386.96
`5,842,009 A * 1 1/1998 Borovoy et al. ................ 707/1
`5,943,670 A * 8/1999 Prager ........................... 707/5
`5.991,758 A 1 1/1999 Ellard ........................... TO6/6
`6,119,133 A
`9/2000 Nusbickel et al. ........... 707,205
`
`37 Claims, 8 Drawing Sheets
`
`2
`
`r
`
`asserATE
`INTIFERS a
`FORMWTEMS
`
`STORE New
`ITEMs
`
`75
`
`STOREIGH
`NT
`INFORMAN
`
`HGPON
`
`YES->
`
`NK
`
`<G) so
`
`STORE TRA
`
`71
`
`73
`
`74
`
`s
`
`Nes
`
`78
`
`ERMNEWHA
`Associate
`INFORMAT
`APPS
`
`AABASE
`of SRUPS
`FPEFE
`
`80
`
`S- TABLE OF
`
`WNTypes
`
`SS
`
`STORs Associated
`INFORMATIONAS
`NEXEMFORATION
`
`
`
`
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 001
`
`

`

`US 7461,099 B1
`Page 2
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`EP
`WO
`WO
`WO
`WO
`
`6, 1996
`0 717346 A2
`5, 2000
`1 OO4967 A1
`2, 2001
`1 O74923 A2
`4f1999
`WO99, 19816
`5, 2000
`WOOO.3OOOO
`8, 2000
`WOOOf 51342
`WOO1,80080 A2 10, 2001
`
`WO
`
`WOO1,80081 A2 10, 2001
`OTHER PUBLICATIONS
`Alexander C. Loui and Mark D. Wood, A Software System For
`Automatic Albuming Of Consumer Pictures, pp. 159-162.
`K. Priyantha Hewagamage and Masahito Hirakawa, Augmented
`Album: Situation-dependant System For A Personal Digital Video/
`Image Collection, pp. 323-326.
`* cited by examiner
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 002
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 1 of 8
`
`US 7461,099 B1
`
`
`
`
`
`
`
`GROUP
`REGISTRATION
`
`CALENOAR
`
`TABLE OF
`EVENT TYPES
`
`DAABASE
`OF GROUPS
`OF PEOPLE
`
`
`
`
`
`
`
`
`
`
`
`6
`
`
`
`MULTIMEDIA
`TEMS
`
`
`
`STORAGE
`PROCESS
`
`MULTIMEDIA
`DATABASE
`
`RETRIEVAL
`PROCESS
`
`ig 1
`F
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 003
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 2 of 8
`
`US 7461,099 B1
`
`10
`
`18
`
`MEDIA
`LEBRARY
`
`
`
`CONTROLLER
`
`MULTIMEDIA
`DATABASE
`
`15
`
`Fig 2
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 004
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 3 of 8
`
`US 7461,099 B1
`
`31
`
`30
`
`
`
`PROCESSOR
`
`
`
`DISPLAY
`
`EM
`GUI REE MODIFYING
`APPLICATION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PONTER CD
`PROGRAMMEMORY
`DEVICE
`MULTIMEDIA
`DATABASE
`
`GU CODE
`DATABASE INTERFACE
`CODE
`TEMMODIFYING
`APPLICATION CODE
`
`32
`
`
`
`34
`
`
`
`35
`
`Fig 3
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 005
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 4 of 8
`
`US 7.461,099 B1
`
`
`
`57
`
`- 58
`HGH POINT
`
`DDMMYY
`
`60
`
`61
`
`ARCHIVE
`
`RETREVE
`
`NOSTALGIC
`RERIEVA
`
`82
`
`Fig. 4
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 006
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 5 of 8
`
`US 7461,099 B1
`
`GENERATE
`IDENTIFFERS
`FOR NEW TEMS
`
`STORE NEW
`ITEMS
`
`71
`
`
`
`73
`
`
`
`
`
`
`
`
`
`
`
`DETERMINE WHAT
`ASSOCIATED
`NFORMATION
`APPLIES
`
`STORE ASSOCATED
`INFORMATIONAS
`NDEX INFORMATION
`
`72
`
`MULTIMEDIA
`ITEMS
`
`Fig 5
`
`STORE HGH
`POINT
`NFORMATION
`
`77
`
`STORE TRAIL
`INFORMATION
`
`79
`
`80
`
`DATABASE
`OF GROUPS
`OF PEOPLE
`
`TABLE OF
`EVENT TYPES
`
`H 81
`L
`
`CALENDAR
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 007
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 6 of 8
`
`US 7461,099 B1
`
`GROUP IDENTIFIER
`
`PEOPLE
`
`
`
`GROUP DENTIFIER
`
`
`
`
`
`ITEM DENTIFFER
`
`
`
`
`
`TEM DENIFER
`
`HGH PONT FLAG
`
`
`
`
`
`Fig 6c
`
`TRAL IDENTIFIER
`
`
`
`
`
`FIRST TEM
`
`SECOND TEM
`
`THIRD ITEM
`DENTIFER
`
`
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 008
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 7 of 8
`
`US 7.461,099 B1
`
`90
`
`
`
`91
`
`
`
`USER SELECTS
`RETRIEVAL
`PARAMETERS
`
`IDENTIFY TEMS
`WTH ASSOCATED
`PARAMETERS
`
`92
`
`
`
`DISPLAYA
`PREDETERMINED
`NUMBER OF THE
`TEMS
`
`NO
`
`ONE OF TH
`TEMS S A TRAIL
`TEM2
`
`DSPLAY TRAL
`OF TEMS
`
`93
`
`
`
`95
`
`SEREND
`SESSION?
`
`
`
`96
`
`W
`
`Fig 7
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 009
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 8 of 8
`
`US 7.461,099 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`focus g event
`" E
`V
`|
`|
`|
`|
`" " " " |
`|
`|
`|
`|
`" EE" |
`|
`|
`|
`|
`|
`" Sint-N" |
`d discs 98
`video
`S.
`v is pron
`put
`to
`son
`Misson. In
`)g -
`sounds () H
`-1N
`Chery
`s s
`focus changes onto a period;
`14 Mar
`image
`Sonia,
`r Cheryl,
`- 18
`same person
`H
`5 A. p C 2. N s
`Mar 97
`Tara
`vacation ET |
`|
`|
`|
`image
`Sonia
`3 Apr
`
`93
`
`party
`
`Sis
`
`Ar 97
`
`sse
`
`aspring 97 Any
`
`M
`
`severy
`
`C
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`al
`Y
`
`Sonia
`sist
`
`at
`
`focus changes onto an event
`date
`type; saic pcrson
`S these 3 items are a high point
`no special order)
`
`-
`N
`s P any
`
`s
`
`s
`
`S
`
`
`
`
`
`
`
`focus changes onto a person
`and a time period
`
`these 4 items in this
`sequence are a trail
`
`
`
`29 Jan
`96
`
`together
`
`audio
`
`inage
`
`gets
`Sonia, S 29 Jan
`Cheryl, --96 togeth
`Cain, N
`
`Sonia,
`Cheryl,
`
`30 Jan
`96
`
`school
`outing
`
`onia,
`ryl
`
`any
`
`
`
`
`
`s
`
`Jan 96 TYet-
`together
`
`this focus is narrow (people,
`time and event typicall
`
`any
`
`focus can be broad (people or
`tre or ever type
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 010
`
`

`

`US 7,461,099 B1
`
`1.
`METHOD AND SYSTEM FOR ARCHIVING
`AND RETREVING TEMIS BASED ON
`EPISODIC MEMORY OF GROUPS OF
`PEOPLE
`
`10
`
`15
`
`25
`
`30
`
`35
`
`2
`user friendly method of archiving material since this is based
`on the way people remember things. The application of these
`indices to a group with shared experience limits the number of
`people and event types needed for indexing or retrieving the
`digital data items. This facilitates a simple user interface Such
`as a pull down menu on a Graphical User Interface (GUI) to
`allow selection of a person and event type from all possible
`people and event types and to allow selection of a particular
`time. The time used for indexing can comprise not just a
`specific time Such as a date, but a time period Such as a range
`of dates e.g., 17th to 24 Jul. 2000 or August 2000.
`Because the collection and indexing of the digital media
`items is based on the episodic memory of the group, i.e. they
`have chosen the material and indexed it according to its rel
`evance to them, the retrieval and browsing through data digi
`tal items are attuned to the memories of the user. The aim of
`retrieval is not to retrieve a specific digital media item but
`instead to retrieve any digital media items relating to a memo
`rable episode. Thus the indexing system does not uniquely
`identify digital media items, but replaces them within a highly
`personal framework. For example, even if a specific photo
`graph were required, it would be remembered through the
`event and hence retrieved by searching on the event or the
`person. Thus the archive may contain many commonly
`indexed images taken at the same time period involving the
`same people at the same event.
`When agroup is setup, a group identifier is assigned so that
`all digital media items archived for that group can be associ
`ated to the group in the archive. A user setting up a group can
`select a number of group members and a number of group
`event types which are relevant to the episodic memory of the
`group. Members of the group can be changed and event types
`can be changed thus modifying the indexing of the archived
`digital media items. This allows flexibility since members of
`Social groups do not always remains fixed. The event types for
`different Social groups can be determined prior to use depend
`ing on the typical shared experiences of the group, e.g. family,
`friends, sports team or club, or sales organisation.
`The digital media items to be archived can be already
`archived digital media items to which the index information
`can be associated. Alternatively, the digital media items could
`be retrieved from another stored location for example on a CD
`ROM or on a web site or the digital media item could be
`generated e.g., by typing text which provides a text item,
`recording Sound to provide an audio item, using a drawing
`package to create an image item, talking or scanning pictures
`to generate image items or inputting digital video images to
`provide video items. In a preferred embodiment, a user is
`provided with the capability to obtain digital media items of
`any media type from any location.
`The user may also be provided with the ability to manipu
`late the digital media item e.g. edit the text item, or modify an
`image using an image processing application.
`In one embodiment the user is given an option to designate
`a particular digital media item as comprising a “high point.
`This enables a user to flag the digital media item as being
`associated with a particularly memorable event. The digital
`media item is thus flagged as being of an important category.
`Judicious use of this feature will allow these particularly
`significant items of digital media to be retrieved more readily.
`As a further aid to retrieval, in one embodiment, the media
`type of multimedia digital media items is used as a further
`index. The media type can either be identified automatically
`or the user can select the media type. This enables a user to
`specify a media type of items which are to be retrieved.
`In another embodiment of the present invention, a user may
`define a “rail'. A trail is a logical or meaningful route through
`
`The present invention generally relates to the archiving and
`retrieval of digital media items wherein the archiving and
`retrieval process is performed based on episodic memory of
`predefined groups of people.
`Storage and archiving techniques for digital media and
`corresponding retrieval techniques are well known. However,
`the work in this field is concentrated on facilitating the
`retrieval of a particular digital media item i.e., one which
`matches the user's requirements exactly. The prior art
`archiving and retrieval systems are based on a semantic
`memory approach. Such an approach is Suited to the location
`of items relevant to a specific requirement.
`The present invention provides a different approach to the
`archiving and retrieval of digital media items of any type. The
`invention is based on the fact that groups of people now
`readily establish archives of digital media items that reflect
`their activities, relationships and interests. The archiving and
`retrieval system of the present invention is designed for use by
`strong Social groups, such as those formed by teenage girls,
`families, work mates, or sports teams or clubs. The invention
`can also be used by individuals to archive and retrieve digital
`media items relating to a group of people. The invention is
`based on the realisation that people in creating these digital
`archives, are establishing group “memory and indulging in
`group nostalgia. Tools bases on episodic memory rather than
`semantic memory will be more effective and desirable for
`these purposes.
`In accordance with a first aspect of the present invention,
`there is provided a method and apparatus for archiving and
`retrieving digital media items in which the archiving and
`retrieval process is based on the common episodic memory of
`a strong Social group. To archive digital media items, a user
`identifies a group to which the user belongs. If it is used by an
`individual, the individual may set up a group but only the
`individual uses the system. In this case the archiving and
`retrieval is based on the episodic memory of the individual in
`relation to the activities of the group. Once the user has
`identified the group to which the user belongs, one or more
`digital media items to be archived for the group can then be
`identified. The digital media item to be archived may either
`already have been stored in an archive or may get stored in the
`archive as part of the archiving process. In order to archive the
`digital media item, a user can select to index the digital media
`item using finite sets of enumerate parameters. More specifi
`cally these comprise group event types to which the media
`item relates, persons associated with the digital media item,
`and a time period associated with the digital media item. The
`user wishing to archive the digital media item can thus select
`Zero or more group event types, Zero or more persons in the
`group and a time which can be defaulted to a default “no time'
`or to the current date. Using these three parameters index
`information for the digital media items is generated and
`stored in association with the digital media items. For
`retrieval, a user can manually select the three parameters or
`they can be automatically generated. Thus a user can select
`Zero or more group event types, Zero or more persons and a
`time period. The manually or automatically selected param
`eters are then used to retrieve digital media items. The auto
`matic generation of parameters enables random reminiscence
`through archived digital media items by group members.
`The use of time, people and event types as the three fun
`damental indices for archiving and retrieval provides a simple
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 011
`
`

`

`US 7,461,099 B1
`
`10
`
`15
`
`3
`an archive of digital media items as perceived by the user. The
`user is thus able to select digital media items and select the
`order of the digital media items in a trail. The fact that the
`digital media items belong to the trail and the order of the
`digital media items in the trail are stored as part of the index
`for the digital media items. Retrieval of a trail of digital media
`items enables a user to experience a particular memory trail.
`For example, a particular digital media item comprises a
`digital photograph of a party. At the party a particular song
`was played a large number of time and came to be closely
`related with that party in the minds of those who attended.
`Consequently, when the item is retrieved it will strengthen the
`recollection about that party if a trail is provided to a digital
`media file containing for example the chorus of that particular
`Song. The retrieval and outputting of digital media items in
`the trail can either be performed automatically i.e., providing
`the user with all digital media items in a trail if one digital
`media item in a trail is selected, or manually i.e., by indicating
`to a user that one of the retrieved digital media items is part of
`a trail to allow the user to follow that trail. In this way the
`nostalgic experience provided by the episodic retrieval sys
`tem is enhanced.
`An embodiment of the present invention additionally pro
`vides the ability to provide the user with a serendipitous
`nostalgic experience. A user may select the automatic
`retrieval and browsing of digital media items. As an initial
`query for retrieval, a time, period, one or more event types,
`and one or more people are automatically randomly selected.
`This forms an initial “focus' for a search. The focus is analo
`gous to that on a camera whereby a wide focus draws in many
`items while a narrow focus reveals a smaller number of items.
`Having performed an initial retrieval based on the initial
`focus, the focus can be shifted e.g. by simply changing one of
`the parameters to retrieve different digital media items and
`thus take the user on a nostalgic trip. Alternatively, the focus
`can be broadened or narrowed. Since the focus indicates the
`breadth of indexing, the indexing fields for time, people and
`event types can be broadened or narrowed to broaden or
`narrow the focus of the search. For example a wide focus
`might comprise a time range of the year 1999, a people range
`40
`ofeverybody in the group, an event range of parties or gigs. A
`narrower focus may comprise a time range of August 1999,
`either of two people in the group, and an event type compris
`ing a school outing.
`The present invention is not just applicable to the archiving
`and retrieval of digital media items for leisure purposes. The
`present invention can be used in a business context. For
`example it can be used as a brain-storming aid by an indi
`vidual or a group. The system will be useful in trying to
`retrieve digital media items related to a memorable episode
`Such as a particular meeting which took place with particular
`people on a particular date. The meeting may have e-mails,
`letters, and presentations associated with it. The system will
`allow members of the group to archive and retrieve digital
`media items relating to Such a meeting using parameters
`which mimic their memory process.
`A second aspect of the present invention provides a method
`and apparatus for archiving a digital media item in which
`index information is generated by allowing a user to identify
`a time, at least one person from a predetermined plurality of
`people, and an event type from a predetermined plurality of
`event types; and storing the index information in association
`with the digital media item.
`The index information may be stored together with the
`digital media item for example as a “header. Alternatively,
`the digital media items may be stored separately with unique
`identifiers such as a file name or index. The index information
`
`30
`
`4
`then carries a reference to this unique identifier. The index
`information may be stored separately from the digital media
`items in a faster storage medium. This allows the digital
`mediabase to be searched quickly but does not use the expen
`sive, fast access storage for space consuming digital media
`files.
`In accordance with a third aspect, the present invention
`provides a method of retrieving a digital media item from a
`database, the method comprising identifying a digital media
`item within the database; automatically identifying another
`digital media item within the database; and retrieving that
`other digital media item. The other digital media item can be
`identified via a previously stored association with the first
`mentioned digital media item, or the other digital media item
`can be identified at random. The previously stored association
`can indicate that the digital media items have at least one
`common characteristic. The further digital media item can be
`retrieved automatically or can be identified to the user that
`there is another digital media item which can then be selected
`by the user.
`The present invention can be implemented on a standalone
`processing apparatus or over a network. The database of
`index information and digital media items may not be physi
`cally located with the processing apparatus which is used by
`a user. In one embodiment the present invention is imple
`mented over the Internet using the World WideWeb. A server
`provides the functionality to interface the digital media item
`database and the index information database to users. The
`users can access the server using a conventional web browser
`running on a computer or a Web TV. Alternatively, the user
`may use a mobile device using a suitable protocol Such as the
`Wireless Application Protocol (WAP) for accessing the
`server. The digital media items can be input to the user's
`device e.g. by typing or reading digital media files. These can
`then be uploaded to the server for archiving. Alternatively, the
`user may access digital media items over the Internet for
`archiving.
`Conveniently the present invention can be implemented on
`any suitable processing apparatus by processor readable
`code. The Code can be provided to the processing apparatus
`on any suitable carrier medium such as a storage device (e.g.
`floppy disk, CD ROM, programming memory device, or tape
`device), or signal. Such as an electronic signal carried over a
`network such as the Internet. Thus the processor readable and
`executable code can be provided to a suitable processing
`apparatus on Such a carrier medium in order to implement the
`method and system of the present invention.
`Embodiments of the present invention will now be
`described with reference to the accompanying drawings in
`which:
`FIG. 1 is a block diagram giving an overview of the present
`invention,
`FIG. 2 is a schematic block diagram of a network arrange
`ment of processing apparatuses implementing embodiments
`of the present invention,
`FIG. 3 is a schematic block diagram of a stand alone
`embodiment of the present invention,
`FIG. 4 is a diagram of the graphical interface in accordance
`with an embodiment of the present invention,
`FIG. 5 is a flow diagram of the archiving process in accor
`dance with an embodiment of the present invention,
`FIGS. 6a, 6b, 6c, 6d and 6e illustrate the tables of the index
`information stored in the database in accordance with an
`embodiment of the present invention,
`FIG. 7 is a flow diagram of the retrieval process in accor
`dance with an embodiment of the present invention, and
`
`25
`
`35
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 012
`
`

`

`US 7,461,099 B1
`
`10
`
`15
`
`5
`FIG. 8 is a table illustrating the serendipitous nostalgic
`process in accordance with an embodiment of the present
`invention.
`FIG. 1 shows a block diagram of an overview of an embodi
`ment of the present invention in which groups of users 1 are
`registered by a group registration process 2 in a database 3. It
`is assumed that the members of the private group work
`together (or an individual works independently but uses digi
`tal media items related to other users) to identify, collect,
`translate or create digital media items in different media e.g.,
`favorite phrases, photographs, verses of poetry, recordings of
`their own voices, clips of home videos, or clips from their
`favorite music. These items collectively represent the culture
`of the group or individual. The items may all be in the same
`medium, e.g. photographs or music recordings. The items can
`be archived and at a later time individuals in the group or the
`group as a whole can reminisce and review the contents of the
`archive.
`The group registration process 2 is preferably only per
`formed oncealthough it is possible to update the groups over
`time. It is preferred to keep this activity to a minimum how
`ever to ensure a database meaningful to the group is con
`structed. An example of a necessary modification to the group
`is when a new baby joins a family. The storage process 7 is
`responsible for associating, with each item to be stored, an
`identifier and any other associated information for the index.
`The storage process 7, which will be described in more detail
`hereinafter, comprises identifying a group of people from the
`database 3, identifying one or more multimedia items to be
`archived 6 with index information, selection of an event type
`from a table of possible event types for the group of people 4
`and selecting a date from the calendar 5. Also individuals
`within the group identified on the database 3 are selected
`using the database 3 for association with the selected item or
`items. Thus the storage process 7 indexes one or more mul
`timedia items in accordance with the group, members of the
`group, one or more event types, and a date. Also, the storage
`process 7 can further index the multimedia items to give
`details of high points and trials as will be described in more
`detail hereinafter.
`The result of the storage process is stored or archived
`multimedia items, high points, trails and the index of associ
`ated information. The multimedia database can be structured
`so that the multimedia items are stored separately to the index
`data.
`The retrieval process 9 allows the digital media items to be
`retrieved and displayed as will be described in more detailed
`hereinafter.
`FIG. 2 is a schematic diagram of a network demonstrating
`embodiments of the present invention. The embodiments of
`the present invention are implemented using the Internet 12 to
`communicate between the user device 10, 13 or 14 and the
`remote server 15. The user's devices 10, 13 and 14 are con
`nected to the server 15 over the Internet 12. The user's device
`can comprise a general purpose computer 10 provided with a
`scanner 11 and a disk drive 10a (e.g. floppy, CD or DVD).
`This enables a user to input media items into the computer 10
`either by scanning images or text or by inputting text, audio,
`image or video files on a disk using the disk drive 10a.
`Alternatively, the user's device can comprise a web-TV 13 or
`a WAP enabled mobile device such as a mobile phone or
`personal digital assistant (PDA) 14.
`The server 15 comprises a controller 16 and the multimedia
`database 17. The controller comprises a web server to provide
`the necessary web interface for the users, and a database
`interface for accessing the multimedia database 17.
`
`6
`Another source of media items for a user is a media library
`18 such as a web site which is connected to the internet 12. A
`user can thus access media items at the media library 18 over
`the Internet 12 and archive these in the multimedia database
`17 at the server 15 for later retrieval.
`The multimedia database 17 can archive single medium or
`multimedia media items. The content comes from the
`medium or media which Surround the members of the group
`e.g. films, music, mimicry and sayings (from actors, the group
`members etc.), TV adverts, TV programmes, books, plays,
`books, photographs from personal events like dances, School
`trips, holiday and parties. The user's can create Some of the
`items themselves and obtain some items from on-line librar
`ies such as the media library 18. Examples of library material
`a.
`music samples
`advertising (not necessarily targeted at the users)
`film and TV including clips from classic films, TV series
`and soap operas
`books such as text from drama, fiction and poetry
`mimicry, impersonations of any of the above.
`It is envisaged that the service will be most applicable to
`groups ofteenage girls, women and families who will main
`tain and access the group archives over long periods of time.
`However, it is also applicable to individuals. An individual or
`a small group with access to a single machine may use a local
`storage medium such as a hard disk drive.
`FIG.3 shows a schematic diagram providing local storage
`as mentioned hereinabove. The diagram illustrates Schemati
`cally the architecture of a general purpose computer. The
`computer is provided with a display 30 for displaying options
`to a user and for displaying the media items. A keyboard 32 is
`provided for inputting text and a pointing device Such as a
`mouse 33 is provided to allow user selections. A multimedia
`database 34 is provided stored on a Suitable storage medium
`such as a hard disk drive or CDROM. Programme memory 35
`is provided storing computer code for implementation by a
`processor 31. The programme memory 35 stores graphical
`user interface code for implementation by the processor 31
`for providing a graphical user interface on the display 30.
`Database interface code is stored for providing a database
`interface to the multimedia database 34 for the retrieval and
`storage of media items. Also the programme memory 35
`stores media item modifying application code for implement
`ing a media item modifying application to allow a user to
`modify media items for archiving all the components. All the
`components of the computer are interconnected by a control
`and data bus 37.
`The present embodiment of the present invention is
`described with reference to groups of teenage girls. The
`present invention is also Suitable for groups of women and
`families. Further, the invention is also expected to be useful
`for other types of private social groups such as work mates,
`sports teams or clubs and even individuals. Each type of
`group will have different sets of event types that are relevant
`to them. For example, for teenage girls the event types can
`comprise: party, holiday, dance, gig, show, School trip, date.
`FIG. 4 is a diagram of a graphical user interface 50 which
`is displayed to the user in this embodiment. An area of the
`display is provided as a work space 51. In this work space
`media items such as images 52 and 53 and text item 54 can be
`placed using known computing techniques. Thus the media
`items represented in the work space 51 can be archived. The
`work space 51 can also be used for displaying retrieved media
`items. Beneath work space 51 are user controls for generating
`index information for archiving or for generating a query
`when retrieving. A drop down box 55 is provided for selecting
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Meta Platforms, Inc.
`Exhibit 1005
`Page 013
`
`

`

`US 7,461,099 B1
`
`5
`
`10
`
`15
`
`35
`
`45
`
`7
`any of a number of people within the group. A drop down box
`56 is provided for identifying one of a number of event types.
`A date entry 57 is provided to enable a user to enter a date. (A
`second date entry, not shown in FIG. 4, is also provided for
`retrieval of media items, to allow the entry of both the start
`and end dates of a time period.) If no date is entered, either
`today’s date or a default “no date' is used for the query. Also
`the date entry field can be used incomplete. For example is no
`day is specified, month and year only will be used. Also if no
`day or month is specified then only the year will be used for
`the query. A checkbox 58 is also provided to enable selection
`of a high point and a similar check box 59 is provided to
`enable to designation of a trail. When the interface is used for
`archiving, any of the media items on the work space 51 can be
`selected e.g. by pointing and clicking, and if the high point
`checkbox 58 is checked and the archive button 60 is selected,
`the media item will be archived with the selected people,
`event type and date selection and will be marked as a high
`point. Similarly the trail check box 59 is checked and a
`number of media items are selected in the work space 51, if
`the archive button 60 is selected, a user will be given an option
`to identify the sequence of media items in the trail and then
`the media items will be archived with indexes indicating the
`order in the trail.
`A retrieve button 61 is also provided to enable retrieval of
`25
`items in acco