`
`
`
`
`Aeg
`Hn
`
`
`4R
`
`aA
`
`/f - psu
`
`4
`
`—_—
`tt ———
`og —=
`PTO/SB/05 (11-00)
`=
`this bo
`inside
`sign (+)
`a plus
`Pleasetype
`gf —
`Approvedfor use through 10/31/2002. OMB 0651-0032
`n=.
`
`
`
`—~>S—== 1 typeaplussign (+) mside U.S.Patent and TrademarkOffice; U.S. DEPARTMENT OF COMMERCE=iggy ===ease this Box >
`
`
`
`— Under tne Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unlessit displays a valid OMB control number.
`o— —
`
`
`—— Vv
`me =S=ca
`— a
`UTILITY
`Attorney Docket No. 76045.911
`sot =~
`
`
`
`—
`an =
`.
`PATENT APP LICATION
`aEen
`First Inventor
`Mark Frigon
`
`
`
`
`ASS
`"
`Title Method and Apparatus for Obtaining Information Relating.. o> =~
`TRAN SMITTAL
`
`
`
`nm
`>=
`Express Mail Label No.
`|EL705172267US
`
`for new nonprovisionalapplications under 37 CFR 1.53(b
` ‘Only
`
`
`Assistant Commissionerfor Patents
`
`.
`APPLICATION ELEMEN
`
`Box Patent Application
`ADDRESSTO:
`TS
`
`
`See MPEP chapter 600 concerningutility patent application contents.
`Washington, DC 20231
`
`
`
`1.
`(Submitanoriginalandaduphcateforfeeprocessing)
`CJ Computer Program (Appendix}
`g
`Fee Transmittal Form (e.g., PTO/SB/17)
`CD-ROMor CD-R in duplicate, large table or
`7.
`2.
`Applicant claims small entity status.
`8. Nucleotide and/or Amino Acid Sequence Submission
`
`
`se 37 CFR 1.27.
`1
`(ifapplicable, all necessary)
`ecification
`‘otal Pages
`
`
`3.
`(eefonedarrangementsetforth bee
`a CL] Computer Readable Form (CRF)
`ot
`b. Specification Sequence Listing on:
`- Descriptivetitle of the invention
`
`
`- Cross Reference to Related Applications
`.
`.
`- Statement Regarding Fed sponsored R & D
`i CJ
`cD-RoM or CD-R(2 copies); or
`
`- Reference to sequencelisting, a table,
`.
`/
`or a computerprogram fisting appendix
`Ut. Cl
`paper oo,
`;
`
`
`
`- Background of the Invention
`C. [| Statements verifying identity of above copies
`
`
`
`
`
`
`
` ACCOMPANYING APPLICATION PARTSAPPLICATION~ Brief Summary oFte tra[ACCOMPANYINGPARTS_|- Brief Description ofthe Drawings (ffiled)
`
`
`
`- Detailed Description
`9. || Assignment Papers (cover sheet & dacument(s))
`
`
`
`- Claim(s)
`37 CFR 3.73(b) Statement
`Powerof
`
`
`
`- Abstractof the Disclosure
`10. (when there is an assignee)
`Attorney
`
`
`
`
`
`4. { Total Sheets lo|]Drawing(s} (35 U S.C. 113) 11.[_] English Translation Document(ifapplicable)
`
`
`
`
`
`Information Disclosure
`Copies of IDS
`5, Oath or Declaration
`[ TotalPages
`]
`2f |
`[|
`Statement (IDS)/PTO-1449
`Citations
`
`
`lv Newly executed (original or coypy)
`13. [-] Preliminary Amendment
`
`a.
`
`
`
`
`503)
`b. CL)
`fonontinuatoniawisional with Box 18 completed)
`etun Receipt Postcard (MPEP
`MPEP 503)
`Copy from a prior application (37 CFR 1.63 (d))
`R
`Receipt
`Post
`
`
`(Should be specifically itemized)
`Certified Copyof Priority Document(s)
`
`i. [_] DELETION OF INVENTOR(S)
`
`
`Signed statement attached deleting inventor(s)
`(if foreign priority Is claimed)
`
`Request and Certification under 35 U.S.C. 122
`named in the prior application, see 37 CFR
`
`
`
`1,63(d)(2) and 1.33(b).
`(b)(2)(B)(i). Applicant mustattach form PTO/SB/35
`
`
`
`orits equivalent.
`
`
`Other:
`6. | Application Data Sheet. See 37 CFR 1.76
`
`
`
`
`
`
`and supply the requisite information below andin a preliminary amendment,
`48, fa CONTINUING APPLICATION,check appropriate box,
`
`or in an Application Data Sheet under 37 CFR 1.76:
`
`of prior application No.Jo
`| Continuation
`CO
`Drsional
`CO
`Continuation-in-part (CIP)
`
`
`
`
`Group Art Unit
`Prior appitcation information
`Exammer
`For CONTINUATION OR DIVISIONAL APPSonly: The entire disclosure of the prior application, from which an oath or declaration is supplied under
`
`
`Box 5b, is considered a part of the disclosure of the accompanying continuation ordivisional application and is hereby incorporated by reference.
`
`
`
`The incorporation can_only be relied upon when a portion has been inadvertently omitted from the submitted application parts.
`19. CORRESPONDENCE ADDRESS
`
`
`
`
`
`
`city—diLosAngelesStato[California__|_zipCove190067
`
`
`
`
`countryfuga/_ Fax|(310)286-0488)Telephone |(310)286-0377
`
`
`
`
`
`
`[simareLAAae [11-15-01
`Q
`parha
`Burden Hour Statement. This form 1 estipratedfortake bTgurs to complete. Time will vary depending upon the needs of the individual case Any comments on
`the arnount of time you are required to complete this form’should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, Washington, DC
`20231. DO NOT SEND FEES QR COMPLETED FORMS TO THIS ADDRESS. SEND TO:Assistant Commissioner for Patents, Box Patent Application,
`
`Washington, DC 20231
`é
`
`7
`
`|
`
`
`
`
`
`
`
`
`
`
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 001
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 001
`
`

`

`UNITED STATES PATENT APPLICATION
`
`FOR
`
`METHOD AND APPARATUS
`FOR OBTAINING INFORMATION
`RELATING TO THE
`EXISTENCE OF AT LEAST ONE
`OBJECT IN AN IMAGE
`
`INVENTOR:
`
`MARK FRIGON
`
`PREPAREDBY:
`
`
`
`THE HECKER LAW GROUP
`1925 Century Park East
`Suite 2300
`Los Angeles, CA 90067
`
`(310) 286-0377
`
`76045.911
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 002
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 002
`
`

`

`BACKGROUNDOEFTHE INVENTION
`
`This non-provisional application takes priority from U.S. Provisional
`
`Application Number 60/248,994 filed on November 15, 2000.
`
`
`
`
`
`
`FIELD OF THE INVENTION
`
`The present invention relates to computer software. Moreparticularly,
`
`the invention relates to a method and apparatusfor storing and sharing images
`
`such as photographs via a communications network and for permitting the
`
`identification of objects and the location of the objects within the images. The
`
`invention enables users to supply and/or receive information aboutthe existence
`
`of objects within images.
`
`Portions of this patent document contain material that is subject to
`
`copyright protection. The copyright ownerhas noobjection to the facsimile
`
`reproduction by anyoneof the patent documentorthe patent disclosure asit
`
`appears in the Patent and TrademarkOffice file or records, but otherwise
`
`reservesall copyrights whatsoever.
`
`BACKGROUND
`
`Digital photography allows people to quickly and easily take, print,
`
`20
`
`and/or share photos without film and the associated film developing costs. The
`
`76045.911
`
`2
`
`Express Mai! # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 003
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 003
`
`

`

`popularity of digital photographyhas increased dramatically dueto the
`
`increased availability of digital cameras and the widespread acceptance of the
`
`Internet. Many Internet users have created websites that organize their digital
`
`photographsinto online photo albumsthat can be accessed by otherInternet
`
`users. These web pages permit a large numberof people to view the photos and
`
`allow the author to both add and remove photos from an album.
`
`Althoughcreating web pages for online photo albumsoffer advantages
`
`overtraditional photo albums,their creation requires programming expertise
`
`such as skill in Hypertext Markup Language (HTML). Many people do not have
`
`the technical knowledge and/or timeto build their own online photo albums.
`
`Sharing of photos from online photo albumsis also problematic and requires that
`
`each photo be manually sent, e-mailed, or downloaded.If the recipient wishes to
`
`use the photos in their own web pages, each photo must be then uploadedto an
`
`additional site. These steps are time consuming, duplicative, and also require
`
`
`
`
`programming knowledge.
`
`To address these problems, several websites that specialize in online
`
`photo albumshave been created. These websites allow users without
`
`20
`
`programmingskills to quickly and easily create and maintain online photo
`
`albums. Users are only required to upload photosthat they wish to add to an
`
`online photo album. Thesites, through programming on the webserver, or
`
`"server-side scripting”, then create all necessary HTML programmingto add the
`
`76045.911
`
`3
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 004
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 004
`
`

`

`uploaded photosto the users’ photo albums. Manyof thesesites also offer image
`
`enhancementcapabilities (e.g. sharpen, brighten, reduce "red-eye”, etc.), as well
`
`as photo printing and developing functions. Photo-specific sites have also eased
`
`the process of sharing photos by providing a meansfor users to quickly and
`
`easily e-mail links to photos or photos themselves to one anotherand, in some
`
`cases, contribute photosdirectly to other's photo albums.
`
`While websites that specialize in the creation of online photo albumsoffer
`
`many advantages, they suffer from severallimitations. For example, existing
`
`websites do not offer users the ability to identify objects within photos. Ifa user
`
`is viewing a photo and does not know certain individuals in the picture, the user
`
`is unable to use current systems to determine their identities. Typically, photo
`
`album sites offer the ability to describe uploaded photos throughthe use of
`
`captions orotherdescriptive fields. These fields may be used to describe or
`
`nameindividuals within the photo. For example, a user may upload a photo and
`
`give it a caption of "Stephanie rollerblading with Tauny”. This caption aloneis
`
`notsufficient information to ensure the viewer can correctly determine the
`
`identities of the individuals within the photo, because to do so would require
`
`prior knowledge of which person is “Stephanie” and which is “Tauny”. A
`
`viewer who does not know these individuals may havedifficulty or be unable to
`
`
`
`20
`
`correctly identify the individuals within the photo. Furthermore, less descriptive
`
`captions such as "The Family at the beach” would offerlittle help unless the
`
`viewer knowseach individual in the photo. Thus, in order to providesufficient
`
`76045.911
`
`4
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 005
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 005
`
`

`

`information required to ensure a viewercan identify individuals within a photo,
`
`copious descriptions must be input and maintained by users. Typing such
`
`captions or descriptions is very time consuming andtedious, especially when a
`
`large numberof photos are uploaded. Furthermore, when the same person
`
`appears in multiple photos, the submitteris still required to retype the
`
`identification information into a caption for each photo.
`
` 10
`
`Processes have been developed to identify individuals within photos. For
`
`example, one process requires people to wear badges containing identification
`
`information such as unique color codings when their photos are taken. When a
`
`computer processes a photo containing a badge, the computer can access a
`
`database to determine which badge wasassigned to which person and then
`
`identifies the person asexisting in this photo. This process has many drawbacks
`
`including the dependence on physical badges. If the badges cannot be seen in
`
`the photos, the people wearing the badges cannotbe identified. Moreover, this
`
`processis not functional for the vast majority of photographs: any photos that
`
`were taken outsideofthis tightly controlled environment.
`
`Another problem with photo-specific websites is that they have no search
`
`capabilities that permit users to search for photosof specific individuals. Users
`
`wishing to view all photographs that contain a particular person must manually
`
`20
`
`accessall possible photo albumsto find the desired photos. For example,if a
`
`user wishesto find all pictures of a particular friend, the user may have to
`
`76045.911
`
`5
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 006
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 006
`
`

`

`browse throughseveral of his/her own albums wherethat friend may appear
`
`(e.g. "High Schoolpictures”, "Reunion Weekend”,etc.), as well as access and
`
`browse through eachof the friend’s albums,or any third-party’s albums where
`
`the friend may appear. With a reasonable numberof photos, this becomes time-
`
`5
`
`prohibiting if not impossible. Search features could be designed to search
`
`aforementioneddescriptive fields (e.g. a caption), however, as there is no naming
`
`standard, people are described differently by different people (e.g. first name,
`
`last name, full name, nickname,etc.) and many names would be used to describe
`
`multiple people (e.g. Mom, Dad, John,etc.). As a result, it would be difficult to
`
`have a oneto one relationship between descriptions of people and people
`
`themselves. Consequently, searches would be unreliable, likely returning many
`
`photosoflittle relevance and failing to return photos that would be of relevance.
`
`
`
`For example, a search for the term “John Smith” wouldfail to return photosin
`
`which this person wasreferred to as “John”or, perhaps, “Dad”. However,this
`
` samesearch likely would return photos of other individuals who share the same
`
`name “John Smith”.
`
`Whilereliable database searchability for digital images over the Internetis
`
`available, it has not been implemented with photo albums. Particularly, several
`
`companies which sell and license images have putcatalogsof digital
`
`20
`
`photographsonthe Internet. Such websites generally describe images in terms
`
`of themes(e.g. sunset, dark or sad) or broad descriptions of objects therein (e.g.
`
`beach, farmland, or water), without identifying the objects explicitly. For
`
`76045.911
`
`6
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 007
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 007
`
`

`

`example, a picture of a man and womanonthe beachat sunset, which could be
`
`licensed to a beach resort for print or web use, could be identified as "beach”,
`
`"sunset”, and/or "man and woman”. While such descriptive terms can provide a
`
`reliable mechanism for searching for images matching the terms, they do not
`
`uniquely identify any objects in the images, nor do they havethe capability of
`
`identifying where these objects appear within the images. For example, a search
`
`for the term "clown”is likely to return many pictures of clowns, however,it is
`
`
`
`very unlikely that every clown pictured would be the sameperson,just as a
`
`search for the term "beach” is unlikely to return pictures of the same beach. Even
`
`a search for the term "President” will return several photos ofdifferent people.
`
`Furthermore, the objects that appear within a photo are not identified as being in
`
`a particular area within the photo, they merely are described as being in the
`
`photo. For example, websites that return a photograph from a "President” search
`
`do not have the capability of identifying that the Presidentis in, perhaps, the
`
`lower right cornerof the picture. Such technologies have the capability to
`
`describe the nature or themeof a photo butnot to uniquely identify who or what
`
`is in the photo or where in the photoit appears.
`
`Anotherrelated limitation of prior art websites is that the captions
`
`described above contain static information only, such as names, and do not
`
`20
`
`provide a wayto obtain additional information about a person or object such as
`
`contact information or to locate additional photosof the person or object.
`
`76045.911
`
`7
`
`Express Mail # £L705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 008
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 008
`
`

`

`Another problem with photo-specific websites is that users wishing to
`
`share their photos must proactively e-mail or otherwise send their photos to
`
`others(i.e., a photo is not automatically shared whena user is described in a
`
`photo). For example, even if a user describes a photosas containing
`
`5
`
`“Jane@doe.com” and “John@doe.com”, existing website will not share the photos
`
`with these individuals. Through a separate process, the user muststill email this
`
`photo or otherwise share the photo with, in this case, Jane and John. Although
`
`photo-specific websites ease this process,it is still time consuming for users with
`
`large photo albums and/ora large numberoffriends to send their photostoall
`
`such persons.
`
`
`
`
`76045.911
`
`8
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 009
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 009
`
`

`

`SUMMARYOF THE INVENTION
`
`An embodimentof the invention comprises a system and method for
`
`identifying and displaying objects (in mostcases, a person) within a digital
`
`image over a system ofinterconnected computers(e.g. client/server, peer-to-
`
`peer, etc...). For example, a user can identify where a person exists in an image
`
`by selecting the person’s name(e.g. fromalist) and clicking on the person’s
`
` 10
`
`15
`
`location in the image. By identifying people within images, the system is able to
`
`search for images where a specified person(s) appears as well as generate output
`
`that recognizes where in the image a person appears. This outputis also capable
`
`of creating hyperlinks and calls to client-side scripts from the otherwise “static”
`
`image file. In an embodimentofthe invention the system can also utilize artificial
`
`intelligence algorithmsto locate other images wherethe identified object has not
`
`yet been defined by a userof the system.
`
`While, one embodimentof this invention utilizes the Internet as the
`
`medium for communication and web pagesfor displaying data, the scope of this
`
`invention is not limited specifically to the Internet and/or to web pages. The
`
`invention contemplates the use of any display output in a computer environment
`
`(e.g. any types of document). The terms person, people, and/or object are used
`
`20
`
`interchangeably and meantto refer to any object(s) (e.g. a pet, a possession, a
`
`place, or a thing) a user wishes to identify within a digital image. This most
`
`frequently will, in fact, be a person. When shown within the boundsof an image,
`
`76045.911
`
`9
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 010
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 010
`
`

`

`a person is considered akin to any type of object. Thus, the reader should note
`
`that the invention can be utilized to identify any type of object within an image.
`
`Depending on the computer environment,different media will be used for
`
`output and displayto the client. On the Internet, the most accessible and most
`
`common output is Hypertext Markup Language (“HTML”). However, the
`
`functionality of HTML can be duplicated or configured in other non-HTML
`
`media. For the sake of both brevity and clarity, I will outline an embodimentof
`
`the present inventionthat utilizes HTML. This embodimentof the present
`
`invention (i.e. utilizing the Internet and HTML)is logical due to the broad reach
`
`of the Internet and wide acceptance of HTML. Howeverthe scopeof the
`
`invention is not intended to be limited to the use HTMLorthe Internet.
`
`
`
`76045.911
`
`10
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 011
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 011
`
`

`

`DESCRIPTION OF THE DRAWINGS
`
`An embodimentof the invention is described in detail below with
`
`reference to the attached drawing figures, wherein:
`
`Figure 1 is a schematic diagram of computer and communications
`
`equipmentthat may be used to implement an embodimentof the present
`
`invention;
`
`Figure 2 is a schematic diagram of databases that may be used to
`
`implementcertain aspects of the invention;
`
`Figure 3 is a flow chart illustrating a high-level overview of the steps of an
`
`embodimentof the invention.
`
`Figure 4 is a schematic diagram of an identifying page that may be used
`
`whenidentifying objects within an image in accordance with an embodimentof
`
`the invention;
`
`
`
`76045.911
`
`il
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 012
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 012
`
`

`

`Figure 5 further describes the general details relating to the processfor
`
`requesting, generating, and transmitting an identifying page to the requestor
`
`(e.g., the client computer).
`
`Figure 6 is a flow diagram illustrating a general overview of the process
`
`for using the identifying page to obtain identifying information(e.g., information
`
`associated with object(s) in an image) from the user.
`
`Figure 7 is a schematic diagram of a display page that may be used to
`
`display images and information aboutobjects that have been identified within
`
`the image in accordance with an embodimentof the invention.
`
` 15
`
`Figure8is a flow chartillustrating a high-level view of the process for
`
`providing display page to a requesting user in accordance with an embodiment
`
`of the invention.
`
`Figure 9 illustrates various waysto display identification information,
`
`including the various states a user interface configured to operate in accordance
`
`with an embodimentof the invention may have.
`
`20
`
`Figure 10 is a flow chart describing the process for generating a set of
`
`search results in response to a request.
`
`76045.911
`
`12
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 013
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 013
`
`

`

`The figures do notlimit the invention to the specific embodiments
`
`disclosed and described herein. The drawings are not necessarily to scale,
`
`emphasis instead being placed uponclearly illustrating the principles of the
`
`5
`
`invention.
`
`
`
`76045.911
`
`13
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 014
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 014
`
`

`

`DETAILED DESCRIPTION
`
`The invention described below is a method and apparatusfor obtaining
`
`information relating to the existence of at least one object in an image. In the
`
`following description, numerousspecific details are set forth to provide a more
`
`thorough description of the present invention. It will be apparent, however, to
`
`one skilled in theart, that the present invention may be practiced without these
`
`specific details. In other instances, well known features have not been described
`
`in detail so as not to obscure the present invention.
`
`General System Architecture
`
`Embodiments of the invention can be implemented in hardware, software,
`
`firmware, or a combination thereof. In one or more embodiments, however, the
`
`invention is implemented with one or more computer programsthat operate one
`
`or more pieces of computer equipment broadly referred to by the numeral 101 in
`
`Figure 1. Such computer equipment 101 typically includes a server computer 200
`
`(also referred to as a host computer) and any number of user computers 100 (e.g.,
`
`client computers) that may access the host computer 200 via a communications
`
`network 150. The computer programs and equipmentillustrated and described
`
`herein are merely examples of programs and equipment that may be used to
`
` 10
`
`20
`
`implement an embodimentof the invention and may be replaced with other
`
`software and computer equipment without departing from the scopeof the
`
`76045.911
`
`14
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 015
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 015
`
`

`

`invention. For example, any computational device configured to display image
`
`data may be configured to implement one or more aspects of the invention.
`
`Host computer 200 may be any computing device such as a network
`
`computer running Windows2000, Novel Netware, Unix, or any other network
`
`operating system. The host computer 200 maybe connectedto a firewall
`
`computerat the boundaries of network 150 to prevent tampering with
`
`information stored on or accessible by the host computer 200 and to another
`
`computing device that may operated by an administrator of the host computer
`
`200. If the invention is implemented with the Internet, the host computer 200
`
`may include conventional web hosting operating software, an Internet
`
`connection such as a modem, DSL converter or ISDN converter, and be assigned
`
`an IP address and corresponding domain namesothat the website hosted
`
`thereon can be accessed via the communications network 150.
`
`Client computer(s) 100 may be used by anyone wishing to upload and/or
`
`view photographsvia host computer 200. The client computers 100 may be any
`
`device or group of devices that permits access to host computer 200 via
`
`communications network 150, including, but notlimited to, personalor laptop
`
`computers such as those manufactured and sold by Dell™, Compaq™,
`
`
`
`20
`
`Gateway™, or any other computer manufacturer, handheld personalassistants
`
`such as those operating with Palm’s PalmOS™operating software or those
`
`operating with Microsoft’s PocketPC™operating software, mobile phones, or
`
`76045.911
`
`15
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 016
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 016
`
`

`

`even Internet appliances that are specifically designed for accessing the Internet
`
`such as WebTV™. Each client computer 100 mayalso include or can access a
`
`conventional Internet connection such as a modem, Digital Subscriber Line (DSL)
`
`converter, or Integrated Service Digital Network (ISDN) converter and a web
`
`browserthat permits it to access and view data overtheInternet.
`
`
`
`The communications network 150 may be the Internet but can also be any
`
`other communications network suchasa local area network, a wide area
`
`network, a wireless network, an intranet ora virtual private network.
`
`Hereinafter, the term Internet or communication networkrefers to any type of
`
`interconnection fabric that provides computers coupled to the network with a
`
`mechanism for transmitting and/or receiving data.
`
`The computer program or programs embodying one or more aspects of
`
`the invention are stored in or on computer-readable medium residing on or
`
`accessible by host computer 200 and provide a mechanism for instructing host
`
`computer 200 to operate the invention as described herein. The computer
`
`programstypically comprise orderedlistings of executable instructions for
`
`implementing logical functions in host computer 200 and user computers 100
`
`coupled with host computer 200. The computer programs can be embodied in
`
`20
`
`any computer-readable medium for use by or in connection with an instruction
`
`execution system, apparatus, or device, such as a computer-based system,
`
`processor-containing system, or other system that can fetch the instructions from
`
`76045.911
`
`16
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 017
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 017
`
`

`

`the instruction execution system, apparatus, or device, and execute the
`
`instructions. In the context of this application, a “computer-readable medium”
`
`can be any meansthat can contain, store, communicate, propagate or transport
`
`the program for use by or in connection with the instruction execution system,
`
`apparatus, or device. The computer-readable medium canbe, for example, but
`
`notlimited to, an electronic, magnetic, optical, electro-magnetic, infrared,or
`
`semi-conductor system, apparatus, device, or propagation medium. Examples of
`
`the computer-readable medium would include the following: an electrical
`
`connection having one or more wires, a portable computer diskette, a random
`
`access memory (RAM), a read-only memory (ROM),an erasable, programmable,
`
`read-only memory (EPROM or Flash memory), an optical fiber, and a portable
`
`compact disk read-only memory (CDROM). The computer-readable medium
`
`could even be paper or another suitable medium upon which the program is
`
`printed, as the program canbeelectronically captured, via for instance, optical
`
`scanning of the paper or other medium, then compiled, interpreted, or otherwise
`
`processed in a suitable manner,if necessary, and then stored in a computer
`
`
`
`
`
`memory.
`
`In an embodimentof the invention, host computer 200 comprises server
`
`engine 220 which is programmedto operate or host a website and serve as a
`
`20
`
`repository for images and identification information for objects within the
`
`images as described in more detail below. The images may be photographs,
`
`graphics, artwork, or any other digital image that contains or depicts one or more
`
`76045.91]
`
`17
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 018
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 018
`
`

`

`objects. The objects within the images mayincludepeople, animals, plants,
`
`buildings, places, or anything else shown in images. In one embodimentof the
`
`invention, the images and objects are referred to (but notlimited to) as
`
`photographs and people, respectively. The reader should note that the term
`
`object as used herein includes any identifiable entity within the image data.
`
`Server engine 220 can access web page templates 210, databases 230, 240,
`
`and 250, and internal or external memory. Web page templates 210 can be any
`
`technology used to transcribe data from a data source into a web pageor
`
`otherwise provide "dynamic”or customized documents, including but not
`
`limited to, Active Server Pages, Java Server Pages, Cold Fusion, PHP, CGI
`
`scripts, or evenclient-side scripts. Server engine 220 receives information and
`
`requests from user computer(s) 100 via communications network 150, processes
`
`web page templates 210 by transcribing data from data sources, including
`
`databases 230, 240, and 250, into templates 210, and sendsthe resulting web page
`
`or documentto requesting client computer 100 as described in moredetail
`
`
`
`below.
`
`In order to implementthe invention host system 200 will access a server
`
`system or database system configured to support the functionality described.
`
`20
`
`Referring back to Figure 2, one or more databases could be utilized to store the
`
`data utilized by the system. For instance, the system mayutilize a users database
`
`76045.911
`
`18
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 019
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 019
`
`

`

`230, Identifications database 240, and Images database 250 as depicted in Figure
`
`2.
`
`Users database 230receives, stores, and provides information about
`
`people or any other objects identified within photos, including various people
`
`whoaccess host computer 200. Users database 230 can be populated to include,
`
`for each userorclient system, a user identifier that is uniqueto that user or client
`
`system, and other information such as the user’s name, e-mail address, home
`
`page address, and/ora list of contacts. The list of contacts is not necessary and
`
`may be stored in a separate database.
`
`Images database 250 receives and stores information about photos(e.g.,
`
`image data) uploadedor otherwise madeaccessible to the network by users.
`
`Images database 250 can be populated as described below to include, for each
`
`photo, a photo identifier that is unique to that photo, and the location of the
`
`imagefile on the network. The Images database 250 may include descriptive
`
`information about the photo suchasa caption orthe date the photo wastaken.
`
`While it is preferred to have photos uploaded to and stored on the host computer
`
`200, it is possible to have the photos stored elsewhere, being still accessible to
`
`users on the network, such as another website or a personal computer connected
`
`to the network. In an embodimentof the invention, both of these processeswill
`
`
`
`
`
`20
`
`be referred to hereinafter as uploading.It is also possible to store the images
`
`76045.911
`
`19
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 020
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 020
`
`

`

`directly into the databaseitself as binary objects, rather than simply storing
`
`information aboutthe location of the image elsewhere on the network.
`
`Identifications database 240 mayreceive, store, and provide information
`
`aboutrelationships between users and photos. Accordingly, in one embodiment
`
`5
`
`of the invention Identifications database 240 links information in Users database
`
`230 with information in Images database 250. Identifications database 240 may
`
`be populated as described below to contain fields specifying what kind of
`
`relationship exists between a photo anda user(e.g. the user is in the photo,the
`
`user took the photo, the photo has been shared with the user, etc.). In one
`
`embodimentof the invention, Identifications database 240 also can store
`
`information aboutthelocation within a photo, or the "coordinates”, that a user or
`
`other person appears.
`
`For example,a user, John Doe, may access and sign-up with host
`
`computer 200 and be assigned a user I.D. of “007” in the Users database. The
`
`
`
`~ 15 Users database 230 could then be populated to include the following record:
`
`
`
`
`
`
`Reem
`
`
`
`[efTne Ui
`
`The user, John Doe, may then upload a photo, “my_image.jpg”, which
`
`may be assigned an image ID. of “ABCD”in Images database 250.
`
`Images
`
`76045.911
`
`20
`
`Express Mail # EL705172267US
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 021
`
`Meta Platforms, Inc.
`Exhibit 1034
`Page 021
`
`

`

`database 250 could then be populated to include the following record:
`
`
`
`
`
`the beach
`
`C:\dir\my_image.jpg|John and Jane at 007
`
` ABCD
`
`To identify the user, John D