The Digital Album: A Personal File-tainment System
`
`Yuichi Yagawa, Noiiyula Iwai, Kunihiro Ymgi, Keiji Kojima
`Systems Development Laboratory, Hitachi, Ltd.
`1099 Ohzenji, Lbao, Kawasaki, Kanagawa 215 Japan
`{ yagawa, iwai, k-yanagi, k-kojima}@sdl.hitachi.co.jp
`
`and Kiyoshi Matsumoto
`Data Storage 8z R.etrieval System Division, Hitachi, Ltd.
`(Presently with Graduate School of Engineering, 'The University of Tokyo)
`
`Abstract
`We describe the Digital Album: a personal File-tainment system
`that allows users to enjoy multimedia data as well cis file and
`retrieve them. This system incorporates an album metaphor which
`imitates a photo album and features pseudo-3D page-turning
`animation. Stories described in this album metaphor entertain
`users. We also adopt a multiview classijkation scheme in which
`each data item is registered to any nodes in multiple clissijkation
`hierarchies. The system is suitable for filing and retrieving
`multimedia data visually and bibliographically. Moreover, users
`can enjoy creating their own albums based on the class@cation
`easily and dynamically. The album metaphor is used as a
`PhotoCD user inlegme in Hitachi's multimedkproductx
`
`1. Introduction
`Recently, computers are not just thought of as a business on
`scientific tool but are being considered sources of entei-tainment,
`and the main interest of users has been shifting from efliciency to
`enjoyment. Multimedia systems are expected to satisfy these new
`requirements. For example, some multimedia titles puldished on
`CD-ROM specifically aim at entertainment and, as a rf:sult, they
`can delight and impress users. We enjoyed "Just Grandma and
`Me"[l] very much, and also "I Photograph to Remember"[2] is
`one of the most impressive multimedia titles we have seen.
`At the same time, the need for personal multimedia database
`systems is gradually increasing because users can make personal
`use of much more multimedia data than ever before. Onie factor in
`this trend is the progress in hardware technologies, such as input,
`output, and storage devices; another is the standardization of
`multimedia data formats, such as P E G (Joint Photographic
`Experts Group) and MPEG (Motion P i c t u ~ Experts Group). Such
`standardization allows even novice users to use computers to
`handle digital photographs and video. Moreover, CD-ROM has
`come into wide use as a distribution methcd, and DVD (Digital
`
`Video Disc) or the NII (National Information Enfrastructue) may
`be available in the near future, making it much easier for users to
`access multimedia data.
`However, most of the current multimedia database systems
`lack entertainment value, even though many users would like to
`have fun while using multimedia databases. For example, today's
`multimedia database products, which generally adopt a user
`interface that shows small thumbnail images in a window or that
`show each data item with detailed information on a card, are not
`suitable for describing stories or scenarios in a way that creates a
`feeling of pleasure. For paonal use, a multimedia database system
`should be a balance of easy filing, efficient retrieval, and fun.
`In this paper, we focus on a personal File-tainment system
`that lets users not only file and retrieve, but also enjoy multimedia
`data, such as pictures, video scenes, and so on. Because a real
`phobalbum is a medium in which we can file, browse, and enjoy
`photographs in the real world, we call this system the Digital
`Album. Our approach is to imitate both the external appearance
`and the hidden structure of the real album as much as possible. In
`Sec. 2, we show the conceptual model of the Digital Album. In Sec.
`3, we explain how the Digital Album imitates the external
`appearance of the real album (later, we define this as the album
`metaphor), and in Sec. 4, we describe the database design as the
`hidden strucbure of the Digital Album. Finally in Sec. 5, we discuss
`the user evaluation of the Digital Album and show other
`applications of the File-tainmeni system.
`
`2. Conceptual Model of The Digital Album
`One factor that increases the entertainment value of
`multimedia titles is a story (We use this word to mean the same as
`a scenario in this paper). For example, in "I Photograph to
`Remember", photographs are shown in a pre-defined sequence,
`which organizes the data into a meaningful overall story. As in
`"Just Grandma and Me", the more complicated the story is, the
`
`0-8186-7436-9/96 $5.00 0 1996 IEEE
`Proceedings of MULTIMEDIA '96
`
`433
`
`Authorized licensed use limited to: University of Maryland College Park. Downloaded on July 31,2022 at 12:54:15 UTC from IEEE Xplore. Restrictions apply.
`
`Meta Platforms, Inc.
`Exhibit 1015
`Page 001
`
`

`

`more user intewtim is possible.
`Before we look at the Digital Album, we would like to
`discuss how a story is organized in a real album. Generally
`spealung, an album might be considered to consist of only visible
`elements, such as photographs and comments that accompany
`them. However, especially in a well-organized album, meaningful
`relationships between the photograph exist apart from the external
`appearance and people can easily understand these relationships.
`For example, when people place photographs in an album,
`they arrange related photographs on the same pages. Even though
`most people dislike troublesome filing, they put photographs into
`an album at least in the order they were taken and this time
`sequence often produces a meaningful story. Also, when they look
`for photographs, they usually expect related photographs to be
`close to each other. When looking through the album, they enjoy
`not only individual photographs but also the overall feeling that is
`often due to the photogmphs' layout.
`These examples show that there are internal semantics
`hidden within the extemal appearance and these semantics
`organize the story and the layout which we see in the album. The
`examples also indicate that the real photo album satisfies the
`definition of theBle-tainment system explained in Sec. 1.
`With this in mind, we designed the conceptual model of the
`Digital Album to consist of two parts (Fig. 1): one is the extemal
`a p a n c e , which we call the album metaphor, and the other is the
`intemal semantics, which we call the multiview classification[3].
`These two parts are also related to each other. Here, we briefly
`explain the two parts.
`The album metaphor is a user interface that imitates the look
`and feel of a real photo album in order to organize a story in the
`Digital Album and to entertain users. Moreover, the album
`metaphor is effective for visual filing and retrieving multimedia
`data. How this is achieved is discussed in Sec. 3.
`The multiview classification scheme was first introduced in
`Hitachi's document management software, "BibliothecaflS
`(hf&hare)''[3], as a data model based on orthogonal classification
`hierarchies. In this model, each hiemhy represents a classification
`from one viewpoint and each data item can be registered into any
`nodes in multiple classification hierarchies. This means that users
`can class@ or retrieve data from multiple viewpints.
`We adopted this technology to represent the internal
`semantics in the Digital Album because, generally speaking, filing
`is based on classification. Even in a real photo album, people
`classify and arrange photographs. Also, multimedia data contain
`detailed information not seen by users. For example, in the case of
`photographs, there is information about subjects, photographers,
`places where the photos were taken, and so on. So, this detailed
`information clearly has multiple categories. Moreover, once
`
`multimedia d a t a are classified into the multiview
`dasdication, users can enjoy creating their own albums
`and stories and going through them at any time (We call
`ths feature a s dynamic album creation). Users select
`nodes in the herarches and the appropriate multimedm
`data are retrieved. The data are arranged on the pages
`by ranking classification hierarchies. How this is
`acheved is explamed in Sec. 4.
`
`Multiview
`Classification
`(internal
`semantics)
`
`Album
`Metaphor
`(external
`appearance)
`
`I
`
`I
`
`
`
`I l l I
`
`4 /
`
`
`
`Fig. 1 Conceptual Model of Digital Album
`Note that the internal semantics are described as
`classification hierarchies and that each data item may be
`connected to several nodes in different hierarchies.
`3. Album Metaphor
`As we indicated before, the album metaphor entertains users
`and helps users file and retrieve multimedia data visually. In this
`section, we explain how this is achieved.
`
`3.1 Interface Design
`The interface design of the album metaphor is quite simple
`and similar to a real album as Fig. 2 shows. The album interface
`consists of two pages spread open, page titles, two areas indicating
`the album's thickness (which we call the "Thumb Button"), no
`more than two thumbnail images on each page, thumbnail titles,
`and page numbers. Also, some pages contain Book Marks on
`themselves. The following paragaphs descrik the charactehtics
`of this user interface.
`We designed the album interface to be attractive so users will
`enjoy seeing it; for example, it has well-proportioned spaces and a
`favorable backgound color on the pages. We also defined simple
`layout rules according to the design: the system resizes thumbnail
`images to fit the pre-defined spaces, puts them on pages in order,
`and arranges them on a page according to the balance of each
`thumbnail's size. For example, if two thumbnails on a page are
`vertically long, they are horizontally shifted, and elsewhere, they
`are vertically aligned, as Fig. 2 shows. This means that users are
`released from the troublesome task that they arrange thumbnails on
`pages for themselves.
`
`434
`
`Authorized licensed use limited to: University of Maryland College Park. Downloaded on July 31,2022 at 12:54:15 UTC from IEEE Xplore. Restrictions apply.
`
`Meta Platforms, Inc.
`Exhibit 1015
`Page 002
`
`

`

`Together, these pages make up a story. In this case, it is very
`important that users can feel the continuity of the story by tuming
`pages. Also, the page-turning action should be as realistic as the
`design of album metaphor is. To represent continuity and realism,
`we implemented page-turning animation[4] and its operation
`methods, discussed in detail below.
`
`3.2 Page-turning Animation
`Visual browsing with page-turning animation is both
`enjoyable and effective for retrieving multimedia data. As we
`mentioned, the thumbnail layout in the album interface
`corresponds to relationships among the data; therefore, there is a
`kind of a story created in the album. Following the stoy by turning
`pages helps users find the data. Moreover, a previous study
`indicates that page-turning animation is useful during retrieval
`because people are good at memorizing objects spatially and page-
`turning animation stimulates this spatial memory[51.
`The idea of page-turning animation was introduced in the
`"Media Room"[4], in which the system used the realistic animation
`to turn pages. Also, this idea was implemented in several systems
`such as the desktop environment[6], the user interface of the
`Digital Library[7], and so on. However, these systems required
`graphic accelerators to represent reality, and, as a result, the
`implementation costs were high as a personal system. Although,
`some other systems, such as "Book Window"[S], implemented
`page-turning animation as software only, the reality of the
`animation was not so favorable.
`There is a trade-off between an animation's realism and its
`implementation cost. We implemented the Digital Album as
`personal computer software. To achieve rich realism while
`avoiding heavy transactions, we usxi an optical illusion to good
`account
`
`Principle
`Fig. 3 illustrates the principle of our page-turning animation
`by showing a book from above. The system creates a page-tuming
`illusion by mapping the pages onto a projection surface while the
`pages are beiig tumed over. In this figure, 8 is the page angle
`(Rad, 0 <= (3<= n) and ais the tuning speed (Rad/=), which
`users can change at any time.
`The geometric definitions of the mapping functions are:
`
`where k is the distance between the light source and the projection
`surface, and (x, y ) and ( X , Y ) are the coordinates of each pixel on
`a page in the case of beig still and tunning, respectively. For every
`
`Page Number
`Thumbnail
`Title
`Fig. 2 Interface Design of Album Metaphor
`Album design consists of two pages and four thumbnail
`images. The page size is 320x480 pixels and the thumbnail
`size is around 180x120 pixels, which is large enough for
`users to clearly see what the icon represents. Each page
`represents a meaningful scene because the thumbnails on the
`same page are related to each other.
`
`Although each thumbnail image is small, it is large enough so
`that its contents can be enjoyed. Here, a thumbnail iimage is a
`reduced image of an original picture, a clipped image of a video
`scene, or a related image to a sound, such as a CD jaciket. While
`having more thumbnail images on a page increases browsing
`efficiency, there is a trade-off between efficiency and im,rige clarity.
`We considered clarity more important than efficiency, so we set
`the page size to 320x480 pixels and the thumbnail size to around
`180x120 pixels, which is sufficient for users to clearly recognize
`the image content. Of course, the higher the display reschtion will
`become in the future, the more thumbnail images can be displayed
`Also, since the thumbnails are encapsulated as ohjects with
`detailed information such as o r i d data and attributes, users can
`easily access this information through the album interf,lce. In the
`current version of the Digital Album, users see original data by
`clicking a thumbnail image, and users access to attributes by
`clicking a thumbnail's title.
`The thumbnails on the two facing pages are often related, so
`the overall impression is of one meaningful scene. For example, a
`user may include pictures and video scenes from the same
`occasion on the same facing pages. A few years later, when the
`U= views them again, the whole atmosphere of the facing pages
`may bring to mind that occasion and create powerful meinones for
`the user.
`A set of pages may also represent one meaningful scene.
`
`435
`
`Authorized licensed use limited to: University of Maryland College Park. Downloaded on July 31,2022 at 12:54:15 UTC from IEEE Xplore. Restrictions apply.
`
`Meta Platforms, Inc.
`Exhibit 1015
`Page 003
`
`

`

`behind the page being turned.
`STEP 4: Set the width of the Thumb Buttons. This function is
`also qlained below.
`STEP 5: Continue the functionsfrom STEP 1 to 4 unless a
`user‘s commund to stop turning p g e s is detected.
`Here, each original page width is denoted by pw, the ori@
`thumbnail width by iw, and the thumbnail location by x and y.
`Each width of the page being turned is denoted by PW, the width
`of the tuming thumbnail by ZW, and its location by X and Y.
`We estimated that this method can cut more than 98% of the
`steps compared to the original method.
`
`1
`f l 1
`Projection surface i
`
`W L i g h t source
`8 : the page angle
`(Rad, 0<=8<=7c)
`a : the tuming speed
`(Rad/sec)
`
`Turning page
`
`frame in the animation, these functions are calculated for all pixels
`on the tuming page and the mapped image is also compounded
`with the images of still pages.
`Although this page-turning illusion is quite realistic, it is very
`difficult for a personal computer to execute the projection
`transaction in real time. Even if the system has a pre-defmed
`mapping table, the system wastes as much memory as all pixels of
`a page upon only the user interface. Therefore, we invented a new
`method that is executable on a personal computer in real time but
`still offers reasonable realism.
`Assuming that the light source is put on the vanishing pint in
`Fig. 3, we have the following formulas approximated to the
`formulas (1) and (2) respectively:
`
`(3)
`(4)
`because k = k - x x sin ( e) is given. This method uses orthographic
`projection and cuts the cost of vertical projection. Moreover,
`horizontal reduction is substituted for horizontal projection.
`This technique reduces the processing burden, but the
`animation realism is not adequate, mainly because users seem to
`feel some incongruity since there is neither change nor movement
`in the vertical direction. Therefore, we mcdified the methcd so that
`the thumbnail images would move in the vertical direction to
`simulate projection in the vertical direction.
`
`Algorithm
`When the system detects a user’s command to tum over the
`pages, it executes the following algorithm.
`STEP 1 : Set the turning speed and direction according to user’s
`opeidon which is expluined below.
`STEP 2: Load thumbnails for the next or the previous page,
`depencllng on the tuming direction.
`STEP 3: Calculate the number offrames in one page-tuming,
`which is in inverse proportion to the turning speed,
`and execute the following for eachfrm.
`( a ) Reduce the width of the page being turned
`according to the expression PW= pw x cos (4.
`(b) Execute the following functions for each thumbnail
`on the page being turned.
`1 ) Reduce the thumbnail width according to the
`expression IW = iw x cos (8,.
`2 ) Move the thumbnail in the horizontal direction
`uccording to the eqression X =x x cos (9.
`3) Move the thumbnail in the vertical direction
`according to the currentframe number.
`(c)Show the page and thumbnails which were hidden
`
`Still page
`Fig. 3 Principle of Page-turning Animation
`This shows a book above. The system creates a page-turning
`animation by projecting pages onto a surface as the page turns.
`
`33 Operation Methods
`Fig. 4 shows an operation method of the page-turning
`function. Users turn pages by clicking the Thumb Button on either
`side. clicking the right button turns to the next page and clicking the
`left button tums back to the previous page. The point at which the
`button is clicked determines the page-tuming speed according to a
`=A x n + N. Here,A is the maximum speed n is the distance from
`the clicking point to the page bottom, and N is the page height. If
`the cursor is moved up, the speed haeases and if it is moved down,
`thespeeddecreaseS.
`Also, users can jump to pre-deiined pages by clicking Book
`Marks. In the current version of the system, one Book Mark
`corresponds to one page and the place of the Book Mark
`corresponds to the page number, so users can roughly guess which
`page they are going to jump. In Fig. 4, the horiwntal points of the
`Book Marks are in proportion to the page numbers. About the way
`to put Book Marks, a user chooses pages and puts Book Marks, or
`the system automatically puts Book Marks to pages according to
`the classification when a user dynamically creates an album based
`on the multiview classification. Of course, the user can remove any
`Book Marks at any time. We also developed page jump aperation
`by clicking a title in a table of contents. Here, the table of contents
`is shown in the menu or in another window.
`
`436
`
`Authorized licensed use limited to: University of Maryland College Park. Downloaded on July 31,2022 at 12:54:15 UTC from IEEE Xplore. Restrictions apply.
`
`Meta Platforms, Inc.
`Exhibit 1015
`Page 004
`
`

`

`Moreover, the width of the 'Ihumb Buttons indicates the
`number of pages. The left Thumb Button corresponds to the
`current page number and the right Thumb Button corresponds to
`the number of remaining pages. When the user is viewing the first
`page, there is no left 'Ihumb Button, except for the operation arm.
`As the user turns pages forward, the left Thumb Button widens and
`the right Thumb Button narrows.
`Album editing functions are also available from the menu.
`The functions are used to arrange thumbnail images on the pages,
`or, in other words, for visually filing multimedia data. Users can
`place, move, or delete thumbnails on any page or at any place they
`want by dragging and dropping the thumbnails. Cut-and-paste and
`copy-and-paste functions are also available. In this case, the system
`grids the thumbnails automatically to maintain the balance and
`beauty of the layout.
`
`Example of operation
`Thumb Button Book Marks Thumb Button /-"-\
`
`Thirdly, in the hierarchies, each node has bi-direcfional links
`to multimedia data. When a node is selected, all multimedia data
`connected lo the selected node as well as all nodes below the
`selected nale are collected through an inheritance mechanism.
`Users can simultaneously select any number of nodes and the
`system will retrieve a subset of multimedia data collected from all
`selected nodes. In Fig. 1, when a user selects node A and node G,
`the result of the retrieval will be data 2. This search mechanism is
`also used for dynamic album creation explained in the following
`section.
`Furthermore, we also developed the tool which visualize the
`multiview classification. This tool displays a list of viewpoints, a
`hierarchy of the selected viewpoint, and thumbnail images of
`multimedia data which the selected nodes contain together.
`Therefore, in order to retrieve multimedia data, users just select
`views and nodes on the display.
`In the case of filing, thumbnail images of available data are
`displayed in another window, so users select appropriate Bhumbnail
`images, andl drag and drop them to the selected nodes. Editing
`functions are also available for modifying views andhiemhies
`The Digital Album is based on an object-oriented database.
`Fig. 5 shows this object model. Classes are organized in
`hierarchical structure to implement the multiview classification.
`Cursor '-.--/
`Each multimedia datum is managed as an object encapsulated with
`a = A x n + N (A: Maximum Speed)
`a title, a thumbnail image, an original datum and attributes. Also,
`N: Page Height, n: Clicked Point's Height
`an album consists of a title, a table of contents, and several pages,
`each of which also Consists of a title and links to objects. When the
`Fig. 4 Operation Method of Page-turning
`Function
`album metaphor shows an album's contents, objects belong to
`each page are collected according to the page-turning. Moreover,
`we define the mapping rules from the database to the album
`metaphor, which enable users to create an album dynamically.
`
`4. Multiview Classification
`In this section, we briefly explain how the niultiview
`classification achieves easy filing and efficient retrievitl, and we
`show the mechanism of the dynamic album creatilon which
`:stimulates users' enjoyment.
`
`4.1 Overview
`The multiview classification is a data model that irepresents
`the internal semantics of the Digital Album. The main
`characteristics are as the following.
`First, as Fig. 1 shows, the structure is based on orthogonal
`classification hierarchies and each hierarchy represents a
`classification from one viewpoint. Users can define and edit the
`h i m h i e s at any time.
`Secondly, users can register a multimedia data item into any
`number of nodes in multiple hierarchies. For example, in Fig. 1,
`data 1 is connected to both node B and node C, each of which
`belongs to a different classification hierarchy. This means that
`users can classify multimedia data from multiple viewpoints.
`
`437
`
`Fig. 5 Object Model of the Digital Album
`
`Authorized licensed use limited to: University of Maryland College Park. Downloaded on July 31,2022 at 12:54:15 UTC from IEEE Xplore. Restrictions apply.
`
`Meta Platforms, Inc.
`Exhibit 1015
`Page 005
`
`

`

`4.2 Dynamic Album Creation
`Once multimedia data are classified into the multiview
`classification, users can enjoy creating their own albums and going
`through them. Freshness in new albums stimulate users’ enjoyment,
`especially when users look into other users’ classifications. This is
`similar to the feeling which we feel when we see the friends’ photo
`albums in a real world The algorithm of this Operation is explained
`below.
`When a user chooses nodes and creates a new album,
`multimedia data items belonging to all selected nodes or their
`lower nodes are retrieved fist. After that, their thumbnail images
`are arranged on pages by ranking classification hierarchies. Here,
`we adopted the following layout rules.
`Rule 1: Follow the order of user’s selections ifthe user selected
`several nodes in the different hierarchies. For example,
`the system arranges thumbnails by ranking the classi-
`jicaiion hierarchy whose top node is selected veryfirst.
`Rule 2: Operatefiom the selected node to its lower nodes in the
`hierarchy by ordering the depth-first search. On the
`same level, search order deperm5 on a node‘s amibute
`(e.g. node name or time of creation).
`Rule 3: Place the thumbnail images that belong to the same
`node on the same page or its neighborhood. In the same
`node, layout order depends on a d a d s attribute.
`Also, the system automatically puts an album title, page titles
`and Book Marks on pages, and makes a table of contents by
`ranking the classification hierarchy.
`
`5. Discussions
`System Requirements
`We implemented the Digital Album as personal computer
`software. The system requires only a 486DX2-66 MHz CPU to
`achieve smooth page-turning animation and a 64kalor video card
`to enrich the image quality. A video capture card a sound card, and
`a scanner are also recommended for inputting video scenes, sounds,
`and pictures.
`
`User Evaluations
`We have showed the Digital Album to many people in
`private or at the several exhibitions. Fht, most of the people were
`impressed with the reality of the album metaphor and its page-
`turning animation. After a while, they were engrossed in the
`contents of digital albums, such as photographs of Japanese
`scenery and seasons, our private pictures, funny movies, and so on.
`This indicates that the album metaphor satisfies the user interface
`transparency which means users can enjoy contents without being
`disturbed by user interface.
`We made the user evaluation experiment and got the same
`
`results. Based on some criteria, the test users compared the album
`metaphor to the ordinary user interface that displays thumbnail
`images all together in a window. As a result, the album metaphor
`gained higher marks in “more natural”, “less fatiguing”, “less
`boring”, “more enjoyable”, “more friendly”, a n d “more
`CCIIII~OI-~~W~.
`
`Book Metaphor VS Album Metaphor
`Related works are well known as the book metaphors
`[4][5][6][7]. We distinguish our album metaphor from the usual
`book metaphors in the following two points. First, the album
`metaphor is the user interface to display multimedia data, not text
`data which the book metaphors mostly concem. In the album
`metaphor, text works only as a title or a caption; still more, the
`page-tuming animation does not show the text at all to reduce the
`operation cost. Secondly, our page-tuming animation achieved
`favorable reality at low cost. So far, the book metaphors required
`extra hardware such as graphic accelerators to implement pseudo-
`3D page-tuning animation. However, we achieved this on
`paofial computer due to the algorithm explained in Sec. 3.
`
`Other Applications
`Moreover, we developed three other applications of the File-
`tainment system which use the technologies introduced in the
`previous sections. One of them has already been produced.
`
`(1) A Digital Album on a multimedia television
`Hitachi brought “VideoCD Nextage”, a new type of
`television (TV) in which a CD-ROM drive is installed, to market
`November in 1994. Because this TV can transcribe a CD-ROM
`whose data format is VideoCD, PhotoCD, CD-DA, or CD-G,
`users can enjoy watching or listening to the contents of the CD-
`ROM as well as watch TV programs. In this product, the album
`metaphor was adopted as the user interface for PhotoCD. When a
`user inserts a CD-ROM, the TV automatically arranges
`photographs on pages. Although users cannot change the order of
`the photographs, they can delete or rotate selected photographs.
`Also, they can enjoy the photographs as if looking through a real
`photo album and can zoom-in on individual photographs.
`
`(2) A Digital Album on a multimedia personal computer
`Like the multimedia TV, the album metaphor can be adopted
`as the user interface for PhotoCD in a multimedia personal
`computer which can transcribe a CD-ROM. Also, in the current
`version of our prototype, the album metaphor is not only a
`PhotoCD viewer but also a platform that lets novice users file,
`browse, and use multimedia data. This is similar to the use of
`scraphks in the real world
`
`Authorized licensed use limited to: University of Maryland College Park. Downloaded on July 31,2022 at 12:54:15 UTC from IEEE Xplore. Restrictions apply.
`
`Meta Platforms, Inc.
`Exhibit 1015
`Page 006
`
`

`

`(3) A Digital Album working with WWW
`The idea of the File-tainmentcan apply to group uses as well
`as personal uses. We implemented the multimedia digital library to
`share pictures and video scenes between group members. This
`system consists of a server computer and client computers
`connected on a LAN (local area network) or the Internet. On the
`server computer, a WWW(Wor1d Wide Web)[8][91 server
`program and a database server program work. The data base m e r
`manages about 500 pictures and video scenes with their attributes,
`such as subjects, photographers, places, and dates. Also, we
`developed the gateway program based on CGI (Common
`Gateway Interface) to make the database server be aware of the
`WWW server.
`A client computer coIlsists of a WWW browser arid a Digital
`Album software. To make the WWW browser be aware of the
`Digital Album as the external viewer, we defined a new MIME
`(Multipupse Intemet Mail Extensions) file type and n:gistered it
`in the browser. This file format is the set of file paths to thumbnail
`images, original data, and attributes. The Digital Album collects
`these files through a network file system on the LAN. Also, a new
`version of the system which collects files using the HTTP
`(HyperText Transfer Protocol) has been developed, but it has not
`achieved practical access speed yet.
`One example of our results is shown in Fig. 6. Users input
`some queries on the WWW browser, and get the result as an
`album. This album's layout is based on the rules explairied in Sec.
`4. In Fig. 6, a user asked to collect pictures whose titles involve the
`word "fI0wd7. Also, the system has pre-defined albums and
`shows them as a clickable map of a book shelf, in which each back
`cover with its title represents an album. When a user clicks a back
`cover, a msponding album will appear.
`
`Input queries based on attributes
`Fig. 6 Digital Album working with the VVWW
`
`6. Conclusion
`In this paper, we described a personal File-tainmenl system,
`the Digital Album. The main objective of this system is to allow
`users to enjoy multimedia data as well as file and retrieve them. To
`achieve that objective, we introduced the album metaphor and the
`multiview classification into the Digital Album. The album
`metaphor allows users to enjoy the multimedia data as if they were
`looking through a real photo album. The multiview classification
`enables users to create their own albums and stories based on any
`categories.
`In the future, we plan to evaluate the Digital Album,
`especially to determine how well the system satisfies user demands
`for entertainment. Also, we will add hypertext links to the Digital
`Album to enrich the representation of the internal semantics. By
`using the image recognition technique, we will also experiment
`with automatic linking or indexing.
`
`Acknowledgments
`We wish to thank Katsuhiko Yuura, Kazuto Senda, and
`Hiromi Ukai for their valuable coments and Mi0 Koriyama for
`developing the current version of the system.
`
`References
`[l] Merser Meyer: " Just Grandma and Me", Broderbund, 1991. (CD-
`ROM)
`[2] Pedro Meym "I F'hotogaph to Remember", The Voyager Company,
`1991. (CD-ROM)
`[3] Hirolnchi Fujisawa, Kanji Kato, Keiiji Kojima, Shuu, T m o b and
`Satoshi Wakayama: "Concept-cente& Document Management and
`Full-text .c;earCh for Infomation Sharing.", Proc. of 47th htemational
`Federation for I n f a " and Documentation, Tokyo, October 1994.
`[4] Kchad A. Bolt 'The Human Intaface", Van Noshand Reinhold, New
`Yolk, 1984.
`[5] Ken-icki Okada, KaoIu h h i t a and Yutaka Matsushita: "scrolling or
`Leafing Through: Book Window", Proc. of 1st Moscow Intematid
`Ha91 Wokshop, August 1991.
`[6] Hajime K m t a , Yu Minakuchi, Kaesutoshi Yano and Toshimitsu
`Suzuki "A Study on Human Interface for B-ISDN Communication
`Workstation", E K E Spring Me