SNAP INC. v. BLACKBERRY LIMITED
`IPR2019-00715
`
`Snap Inc. Ex. 1014 Page 0001
`
`

`

`US 10,454,995 B2
` Page 2
`
`Int. Cl.
`GO6F 16/74
`7
`HOAW 4/21
`
`(51)
`
`(56)
`
`(2019.01)
`:
`(2018.01)
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`11/2000 Piccionelli et al.
`6,154,172 A
`9/2003 Naidoo
`6,629,136 Bl
`8,055,271 B2* IL/2011 Nair... sccssssecessans HO4W 4/02
`455/418
`
`JP
`JP
`JP
`KR
`KR
`WO
`wo
`Wo
`WO
`WO
`Wo
`
`2003-209779 A
`2003-233555
`2006-33273
`1020060095601 A
`10-2007-0002159 A
`WO 98/54896
`WO O2/17130 AB
`2003007542 Al
`O3L01139 AL
`WO 2004/090903 Al
`2006018712 Al
`
`7/2003
`8/2003
`2/2006
`9/2006
`1/2007
`W2/1998 eee HO4N 5/91
`2/2002
`cevsveerseare GOIS 5/14
`1/2003
`12/2003
`10/2004 oe GIB 27/10
`2/2006
`
`2005/0086087 AL*
`
`2006/0181546 Al*®
`
`as
`Pact
`8,548,424 BL* 10/2013 Mikan etal.
`OTHER PUBLICATIONS
`2004/0054665 Al
`3/2004 Yoo et al.
`5“
`"
`seus
`yee
`es
`.
`2005/0004986 AL*
`1/2005 AOE vccccseesreerens GOG6F9/543
`James Fallows, Spy's-eyve View, The Atlantic (Mar. 2006), available
`709/206
`for downloadat hitp://www.theatlantic.com/magazine/archive/2006/
`4/2005 Razza vecceece G06Q 10/025
`03/spy-s-eye-view/304620/."
`;
`7035/5
`Rick Vidallon, “How to Embed Your Youtube Video into Google
`ee a Cae a et Qo sestesaneneetSG
`ey
`tf
`int
`leapt
`TANAIM Cb AL. cece
`*
`a
`!
`é
`Sef
`a
`é
`Maps”, 2006, 5 pages. http://www.visionefx.net/articles/how-to-
`;
`;
`
`
`
`
`| (2006Johnson.2 Al* .........0006 ri / ; 1
`
`
`
`S2OGION2204S: A
`212008 Jolson
`eesaoe
`embed-youtube-video-in-google-maps.htm.
`2006/0154674 Al*
`7/2006 Landschaft
`........ HO4M 1/72572
`Alex Chitu, “Google Operating System Unofficial news and tips
`455/456.6
`about Google”, Jun. 23, 2006. 3 pages. http://googlesystem.blogspot.
`8/2006 Jung ccc GOIC21/3667
`com/2006/06/google-maps-youtube-video-map.html.
`345/619
`Alex Chitu Google Operating System Unoflicial News and Tips
`2006/0240847 AL™ 10/2006 Choi ......ccccc 455/456.2
`about Google, Jun. 23, 2006, 2 pages. http://googlesystem.blogspot.
`AGDE eee Al* 12/2006 Ikeda et al.
`...ccccccsceenene 386/95
`com/2006/06/panoramio-google-maps-with-photos.html.
`zaON eet a -
`5/2007 Adam ct al.
`0.
`/
`Author Unknown, 2 pages, hitp://vidteq.com, date unknown.
`SEEDSTOS 2
`MSGR SABIEDE seceenrsseee Hi ne AuthorUnknown,http://Atlas.freshlogicstudios.com, copyright 2010,
`i
`/
`+e
`oe
`2 pages.
`ir
`ay
`2009/0197612 Al*
`8/2009 Kiiskinen ........... HO4W 64/00
`—
`.
`455/456.1
`Author Unknown, http://Clipglobe.com, 2 pages. Copyright 2007.
`Author Unknown, http://Virtualtourist.com, 5 pages Copyright 1994-
`2013.
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`
`2000-235576
`2001-292389
`
`8/2000
`10/2001
`
`* cited by examiner
`
`Snap Inc. Ex. 1014 Page 0002
`SnapInc. Ex. 1014 Page 0002
`
`

`

`U.S. Patent
`
`Oct. 22, 2019
`
`Sheet 1 of 3
`
`US 10,454,995 B2
`
`
`
`
`UPLOAD2 COMPONENT 13
`PONDEN
`
`
`Seeye DIGITAL VIDEO
`
`INFORMATION
`CAMERA
`DETECTOR
`
`
`
`
`
`FIG, 2
`
`Snap Inc. Ex. 1014 Page 0003
`SnapInc. Ex. 1014 Page 0003
`
`

`

`U.S. Patent
`
`Oct. 22, 2019
`
`Sheet 2 of 3
`
`US 10,454,995 B2
`
`
`
`
`RECEIVE
`
`DETECT ITEM OF
`GEOGRAPHIC
`
`Soue
`INFORMATION
`
`
`=
`RELATING TO
`
`
`
`CONTENT
` 42
` ASSOCIATE
`
`
`
`
`ITEM OF
`CONTENT WITH
`RELATED
`GEOGRAPHIC
`INFORMATION
`44
`
`
`
`
`
` STORE
`
`
`
`ASSOCIATED
`DATA
`46
`
`
`UPLOAD
`
`
`ASSOCIATED
`DATA
`47
`
`FIG..3
`
`FIG. 4
`
`
`
` RECEIVE
`ASSOCIATED
`
`
`DATA
`
`48
`
`
`
` DISPLAY
`
`
`
`PLURALITY OF
`
`
`GEOGRAPHIC
`LOCATIONS
`52
`
`
`
`RECEIVE INPUT
`TO SELECT ONE
`
`OF THE
`PLURALITY
`54
`
`
`
`
`
`
` DISPLAY
`
`CONTENT
`
`
`INDICATORS
`
`
`
`
`ASSOCIATED
`WITH
`SELECTION
`56
`
`
`
`
`
`RECEIVE INPUT
`
`TO SELECT ONE
`RENDER
`
`
`OF THE
`INDICATED
`
`
`
`
`CONTENT
`
`CONTENT
`
`INDICATORS
`62
`
`58
`
`
`
`Snap Inc. Ex. 1014 Page 0004
`SnapInc. Ex. 1014 Page 0004
`
`

`

`U.S. Patent
`
`Oct. 22, 2019
`
`Sheet3 of 3
`
`US 10,454,995 B2
`
` USER RECORDS
`
`
`
`
`AND UPLOADS
`VIDEO CONTENT
`74
`
`
`
`
`
`
`FRIENDS OR
`SHARE GROUP
`INVITED TO
`VIEW CONTENT
`76
`
`
`
`
`
`
`
`
`
`DISPLAY ONE
`OR MORE
`GEOGRAPHIC
`LOCATIONS
`78
`
`
`
`
`
`RECEIVE INPUT
`
`RENDER
`TO SELECT ONE
`
`INDICATED
`
`GEOGRAPHIC
`
`
`86
`CONTENT
`
`
`PLURALITY
`
`82
`
`
`
`
`
`
`
`
`RENDER
`
`
`
`INDICATED
`DISPLAY VIDEO
`CONTENT
`88
`
`
`
`84
`
`
`
`FIG. 5
`
`UPLOAD
`ASSOCIATED
`DATA TO FILE
`SHARING
`SERVER
`64
`
`FILE SHARING
`SERVER
`ORGANIZES
`DATA,E.G., VIA
`
`72
`
`DEFINE SHARE
`GROUP
`68
`
`SHARE GROUP
`ACCESSES AND
`VIEW CONTENT
`
`FIG. 6
`
`Snap Inc. Ex. 1014 Page 0005
`SnapInc. Ex. 1014 Page 0005
`
`

`

`US 10,454,995 B2
`
`1
`SYSTEM AND METHOD FOR OBTAINING
`AND SHARING CONTENT ASSOCIATED
`WITH GEOGRAPHIC INFORMATION
`
`FIELD OF THE INVENTION
`
`The invention relates to content recording, geographic
`location tagging of the recorded content, and sharing ofthe
`recorded and tagged content.
`
`BACKGROUND
`
`Prior art references purport to disclose GPS video map-
`ping systems as well as systems that use wireless devices to
`create, access, and share geographical positioning data.
`Otherprior art references purport to disclose location-tagged
`data provision and display systems, and methods for posi-
`tion-stamping photos or video clips taken with a digital
`camera.
`
`Suchprior art disclosures suffer from various drawbacks,
`however. For example, they lack the convenience and por-
`tability of a unitary full-featured system. They further lack
`convenient sharing capabilities, including web-based shar-
`ing capabilities.
`
`SUMMARY
`
`Systems and methods are provided for combining content
`files with geographical data and for sharing these files. The
`combination can then be used to create enhanced user
`
`30
`
`interfaces for organizing, sharing, navigating and playing
`back the contentfiles.
`In one embodiment, a digital video camera is combined
`with a source of geographical data (such as a GPSreceiver)
`and a connection to a computer, generally as an integral
`component. Video clips captured by the camera are manu-
`ally or automatically tagged with appropriate geographic
`location data from the geographical data source, e.g., as
`metadata. The tagged video files are uploaded to the com-
`puter via an easy-to-use upload component. The connection
`to a computer may be an internet connection to a server or
`a direct connection to personal computer. The connection
`may be wired or wireless, such as a WiFi or other such
`connection. In the embodiment employing a wireless con-
`nection to the Internet, the tagged data files may be auto-
`matically uploaded whenever an available wireless connec-
`tion Is obtained or available.
`A corresponding software application, which may be
`resident on the user’s computer or which may be web-based,
`uses the video clips and geographical data to create an
`interactive user interface to browse and watch the video
`clips. Details vary by embodiment, but one such user
`interface appears as an interactive map with clickable links
`situated at locations where the video files were captured or
`as an interactive map that allows end-users to play video
`files associated with locations on the map. The system may
`also allow an indication ofthe time-sequence of the video
`clips to be displayed, such as via an arrow from point-to-
`point. In this way, reconstruction of, e.g., a trip may be put
`in proper sequential order. Clicking the links renders or
`plays back the videofiles. A link may contain information
`about the video such as a title and image thumbnail. For
`example, a video image thumbnail is displayed on the map
`and is clickable to play the video.
`In web-based or other shared systems, the software appli-
`cation may limit those who can view the video clips to only
`one or more authorized groups of people or may leave the
`
`40
`
`50
`
`60
`
`2
`clips accessible to anyone. As noted, the application can be
`desktop installed software or a web application, but may also
`be resident on the camera (in which case a wired or wireless
`link may link the camera to a display monitor). The web
`application can be accessible from any terminal that has an
`Internet connection, thoughthe contentfiles may be limited
`to only the one or more authorized groups.
`Sharing of the content files may be accomplished via
`“one-click” sharing using a dedicated physical or touch-
`screen-type button on the camera. An alternative embodi-
`ment
`is automatic sharing,
`for example, uploading and
`sharing without a user trigger whenever the camera is
`connected to the Internet.
`In one aspect, the invention is directed towards a device
`for detecting content and associating that content with
`geographic information. The device includes a detector
`configured to receive an item of content, a detector config-
`ured to receive information corresponding to a geographic
`location relating to the item of content; a correspondence
`module configured to associate the item of content with the
`geographic location information; and an upload component
`configured to transmit the associated item of content and
`geographic location information to a remote system on a
`network.
`
`Implementation of the invention may include one or more
`ofthe following. The content detector may be a videocam-
`era, a still camera, or a microphone. The geographic location
`information detector may be a GPS receiver or a mobile
`phone triangulation radio-based system. The correspon-
`dence module may associate the item of content to the
`geographic location information using metatags or metadata.
`The upload component may further include a wireless
`transmitter. The network may be the Internet.
`In another aspect,
`the invention is directed towards a
`method for associating an item of content with geographic
`information relating to the item of content. The method
`includes the steps of receiving an item ofcontentin a device,
`receiving geographic location information relating to the
`item of content in the device, associating the item of content
`with the geographic location information, and uploading the
`associated item of content and geographic location informa-
`tion to a remote system on a network.
`Implementation ofthe invention may include one or more
`of the following. The step of associating may occur auto-
`matically or manually. If automatically, the associating may
`occur in a manner such as: every time an item ofcontentis
`stored, periodically, or a combination ofthese.
`The step of uploading may occur automatically or manu-
`ally. [fautomatically, the uploading may occur every time an
`item ofcontent is stored, every time a network connection
`is established, periodically, or a combination ofthese. If
`manually, the uploading may occur upon user operation of
`a dedicated input device.
`In another aspect,
`the invention is directed towards a
`computer readable mediumcontaining instructions for caus-
`ing a computer to implement the above method.
`In yet a further aspect, the inventionis directed towards,
`in a computer system having a graphical user interface
`including a display and a selection device, a method of
`providing and selecting from a menu onthe display, the
`menu organizing and displaying content
`items having
`related geographic location information. Steps ofthe method
`include: retrieving a set of menuentries for the menu, each
`of the menu entries representing a geographic location;
`displaying the set of menu entries on the display; receiving
`a menu entry selection signal
`indicative of the selection
`device pointing at a selected menu entry from the set of
`
`Snap Inc. Ex. 1014 Page 0006
`SnapInc. Ex. 1014 Page 0006
`
`

`

`US 10,454,995 B2
`
`3
`menuentries; in response to the signal, displaying a set of
`content
`items corresponding to the geographic location;
`receiving a content item selection signal indicative of the
`selection device pointing at a selected content item from the
`set of content items; and in responsetothe signal, rendering
`the selected content item.
`Implementations of the invention may include one or
`more of the following. The graphical user interface may
`form part of a system such as a videocamera, a computer, or
`a web-based browser. The step of displaying the set of menu
`entries on the display may include displaying the set of menu
`entries on a map, with each menu entry disposed at a
`location corresponding to its geographic location. Each
`menu entry may be displayed along with a nameofa city or
`state near
`the corresponding geographic location. The
`method may further comprise displaying arrows betweenthe
`displayed menu entries,
`the arrows indicating a time-se-
`quence ofthe menuentries. The step ofdisplaying the set of
`menu entries on the display may include displaying the set
`of menuentries in a list. The content items may be video
`clips.
`In a related aspect, the invention is directed towards a
`computer readable medium containing instructions for caus-
`ing a computer to implement the above method.
`In another aspect,
`the invention is directed towards a
`methodofsharing an item of content with a group, including
`the steps of creating a share group having at least one
`member, receiving an item of content in a device, receiving
`geographic location information relating to the item of
`content in the device, associating the item of content with
`the geographic location information, uploading the associ-
`ated item of content and geographic location information to
`a remote system on a network, and notifying each member
`of the share group about the uploaded item of content.
`Implementations of the invention may include one or
`more ofthe following. The notifying step may be accom-
`plished by sending each member of the share group an
`instant message or an email about the uploaded item, where
`the instant message or email contains a clickable link which
`when clicked displays the item of content. The item of
`content may be displayed as a video overlayed above a map.
`The step of creating a share group may include inputting at
`least one contact
`into a share group list or analyzing a
`personal information manager to determine contacts. The
`instant message or email may be sent automatically when-
`ever a content item is uploaded.
`In a related aspect, the invention is directed towards a
`computer readable medium containing instructions for caus-
`ing a computer to implement the above method.
`In another aspect, the invention is directed towards a
`methodofsharing an item of content with a group, including
`steps of creating a share group having at least one member,
`receiving an item of content
`in a device, receiving geo-
`graphic location information relating to the item of content
`in the device, associating the item of content with the
`geographic location information, uploading the associated
`item of content and geographic location information to a
`remote system on a network, and notifying each memberof
`the share group about the uploaded item of content. When a
`memberofthe share group attempts to access the uploaded
`item of content, then further steps of the method include
`retrieving a set of entries, each of the entries representing a
`geographic location, displaying the set of entries on a
`display, and receiving an entry selection signal indicative of
`the selection device pointing at a selected entry from the set
`of entries. Further steps include, in response to the signal,
`displaying a set of content
`items corresponding to the
`
`4
`geographic location, receiving a content item selection sig-
`nal indicative ofthe selection device pointing at a selected
`content item from the set of content items, and in response
`to the signal, rendering the selected content item.
`Advantages of the invention may include one or more of
`the following. The content files with location data are easily
`uploadable to a computer, server or other storage. The same
`are navigable and may be viewed via an interactive user
`interface where the same are displayed as clickable links,
`such as by the use of thumbnails, icons or textual represen-
`tations, on a map correspondingto their geographic location.
`The invention provides an easy-to-use and convenient pro-
`cess for camera users to share video with geo-data and for
`others to watch the video over the Internet. For example,
`family members may watch videos ofa trip done by another
`family member and see where each video was shot on a map.
`Other advantages will be apparent from the description
`that follows, including the drawings and claims.
`
`o
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 illustrates a schematic diagram of a system accord-
`ing to a first embodimentof the invention.
`FIG. 2 illustrates a portion of a map-based user interface
`according to a second embodimentofthe invention.
`FIG. 3 is a flowchart illustrating a method for obtaining
`and uploading content items and geographic location infor-
`mation according to a third embodiment of the invention.
`FIG. 4 is a flowchart illustrating a method for displaying
`content and geographic location data according to a fourth
`embodiment of the invention.
`FIG.5 is a flowchart illustrating a method for displaying
`content and geographic data accordingtoa fifth embodiment
`of the invention.
`FIG. 6 is a flowchart illustrating a method for sharing
`content and geographic location data with a group ofusers.
`
`DETAILED DESCRIPTION
`
`The following terms, defined here, are used throughout
`ithe specification and attached figures.
`“Uploading” an item of content or geographic location
`information or both refers to the act of transferring a file
`containing such information from a device such as a video
`camera that is equipped with an upload component. The
`uploading is often accomplished wirelessly, but may also be
`accomplished using a wired connection, directly to a PC or
`network server.
`
`“Storing” data or a file refers to the act ofplacing data in
`permanent or temporary memory or storage, such as in a
`type of RAM or disk- or card-based storage. The storing act
`could be very transitory. For example, data may be obtained,
`stored in a buffer, and uploaded immediately. On the other
`hand, data could be stored indefinitely on, for example, a
`mini-DVD.
`“Geographic Location” correspondsto a location at which
`or near where content data was obtained. “Geographic Data”
`correspondsto data constituting the geographic location. An
`example of geographic location informationis the latitude,
`longitude, and altitude information obtained by a GPS
`system. Geographic location, in this context, can further
`refer to information such as camera direction and orienta-
`
`tion. An “Indication of a Geographic Location”refers to a
`shorthand notation for geographic information displayed in
`a user interface. For example, an indication of geographic
`information may be an icon, a display of latitude and
`
`40
`
`45
`
`60
`
`Snap Inc. Ex. 1014 Page 0007
`SnapInc. Ex. 1014 Page 0007
`
`

`

`US 10,454,995 B2
`
`5
`longitude coordinates, or the name of a corresponding city
`and state that represents the geographic location.
`“Content Item” refers to a collection of data obtained by
`a detector. Typically, a content item is a continuous sequence
`of data,
`for example, a video clip obtained by a video
`camera. “Content Data” refers to the underlying data that
`constitutes the content item. In some cases, a content item
`refers to one or more related video clips, all corresponding
`to a single (or set of adjacent or nearby) geographic
`location(s). An “Indication of the Content Item”refers to a
`shorthand notation for the content item displayed in a user
`interface. For example, an indication of a content item may
`be an icon, thumbnail, or filenamethat represents the content
`item. Content items may also be audio files, digital still
`photographs, or the like.
`“Associating” a content item with a geographic location
`refers to the act of connecting one with another, such as by
`placing data correspondingto onein the file of the other. For
`example, associating a content
`item with a geographic
`location mayrefer to placing metatags or metadata about the
`geographic location in the file of the content item.
`“Rendering” refers to the act of playing back an item of
`content. For example, rendering a content
`item that is a
`video clip means to play back the video clip in a media
`player. The media player may be any content playback
`device that is capable of playing back the selected type of
`content.
`
`A“share group”refers to a defined groupof users that are
`a target audience ofa shared file.A share group is typically
`defined by a user, and corresponds to that user’s group of
`friends or other defined cohort. A share group may also be
`constructed automatically, such as via an analysis of user
`settings in a separate application, such as a social network-
`ing service. A share group may also be created on a user
`system using, for example, contacts information suchas is
`stored in Microsoft Outlook®or using a buddy list in an
`Instant Messaging application. A share group may also be
`defined separately and users may join such a group using a
`software application or a website. For example, one such
`share group may be the ‘Lake Tahoe Videomakers’ Group.
`A group may also consist of only one person. In this case,
`sharing can be performed without specifying a group first.
`For example, sharing may be performed with various con-
`tacts without any formation of a group. In this example, the
`friend may receive an email with a link to access the video.
`Content Detection System, for Example, a Videocamera
`Referring to FIG. 1, a content detection system 10 is
`shown according to a first embodiment of the invention,
`which may be an integrated videocamera system with an
`upload capability. The system 10 includes a digital video-
`camera 12 having a detector 13, the detector in this case
`including a lens and microphoneto record videoclips.
`The system 10 also includes a detector of geographic
`information 14. The geographic information detector 14
`often has an antenna 15, which may beinternal or external.
`The geographic information detector
`14 may be,
`for
`example, a GPS receiver. However, alternative geographic
`information detectors may be employed, including mobile-
`phonetriangulation radio-based systems or other such sys-
`tems. The geographic information detector 14 usually
`receives data such as latitude and longitude, althoughalti-
`tude data may also be retrieved, along with information
`corresponding to the orientation and direction of system 10.
`The system 10 further includes a correspondence module
`16, which may be implemented in hardware, software, or a
`combination of both. The correspondence module 16 asso-
`ciates the item of content with its related geographic infor-
`
`30
`
`40
`
`50
`
`60
`
`6
`mation. For example, the correspondence module 16 may
`place metatags or metadata in the file of the content item,
`these indicating the related geographic information. In this
`embodiment, it is important to note that the correspondence
`module need not be a separate circuit or algorithm, e.g.,
`implemented as a separate circuit or algorithm in hardware,
`software firmware, or the like. Rather, the correspondence
`module may form an integral part of the camera firmware
`and may be integrated with a module that saves a digital
`videofile to local storage. The system 10 also incorporates
`an upload component 18. The upload component 18 orga-
`nizes the transmission of data corresponding to geographic
`information, the content item, and their association.
`It is
`noted that in certain embodiments, as described below, the
`geographic location data 1s placed into the digital videofile.
`In this case, and in others like it, the upload would be ofthe
`data itself, and there would not necessarily be a separate
`upload of geographic location information or association
`data. The upload component 18 may be wireless, in which
`case an antenna 22 may be employed. As with antenna 15,
`the antenna 22 may be internal or external. The upload
`component 18 may also be wired,
`in which case,
`for
`example, a USBor other such port may be provided.
`In a wireless configuration, the upload may occur to a
`wireless access point, wireless community network, WiFi
`hotspot, or any other such wireless network.
`A button 19 or other user-operable device may be pro-
`vided to allow the user to upload the associated content and
`geographic information conveniently. For example, the but-
`ton 19 may allow a “one-click” upload capability. In this
`case, depression ofthe button causes an immediate upload
`ofall or a portion of the content files, e.g., whenever a
`network connection is available. The button 19 may also be
`virtual, for example, the same may be a button on a touch
`screen or other user interface of the camera. For example,
`when viewing captured video clips on a camera’s LCD, one
`of the applicable commands for a video may be ‘share
`online’.
`Alternatively, the system 10 may automatically upload the
`associated content and geographic information whenever a
`network connection is established, for example, whenever a
`wireless connection is obtained, particularly where the wire-
`less connection, such as a WiFi “hotspot”, is an unsecured
`network.
`
`The system components described above may be imple-
`mented in a single device, i.e., a unitary full-featured inte-
`grated video camera system with a data upload capability.
`In use,
`the system 10 may be taken from location to
`location, filming content at each location, and uploading the
`same to a server or personal computer. Referring to FIG. 2,
`which indicates a portion ofa user interface according to a
`second embodimentofthe invention, and which is operable
`to display such content, the system 10 may be taken from
`location 26 to location 28, and then from location 28 to
`location 32, and likewise to location 34 and location 36. In
`each case content may be obtained, andlater the content may
`be viewed by selection of any location indicator and further
`by the choice of content indicators for content obtained for
`that location. Indicators such as the arrows shownin FIG. 2
`may be employed to indicate a time-sequence of when the
`content was obtained. In this way, a trip may be recon-
`structed and the sequence oflocations visited known.
`A method according to a third embodimentofthe inven-
`tion is illustrated by the flowchart of FIG. 3. The method
`starts (step 37) and an item of content is detected,
`i.e.,
`recorded (step 38). For example, a video clip is filmed.
`Geographic location information is received relating to the
`Snap Inc. Ex. 1014 Page 0008
`SnapInc. Ex. 1014 Page 0008
`
`

`

`US 10,454,995 B2
`
`7
`item (step 42). For example, GPS data may be
`content
`received and used as geographic location information. The
`geographic location information may be obtained periodi-
`cally, continuously, or at arbitrary times as designated by the
`user.
`
`Geographic information may also be obtained by use of
`detected IP address information. In particular, geolocation
`software may be employed to examine the IP address and
`determine what country, organization, or user it has been
`assigned to, and then an estimate may be made as to the
`user’s location based on that information. Such an estimate
`would be particularly accurate if the user were continuously
`connected to the Internet. However, if uploads are made only
`periodically, e.g., at the end of a day of video recording, then
`the geographic location estimated would only be correspond
`to the upload location, not each individual recording loca-
`tion.
`The item of contentis then associated with the geographic
`information (step 44). For example, metadata are added to 2
`the content
`item file to indicate the geographic location
`where the content item was obtained. In another example, a
`set ofpaired pointers are stored, with one pointer pointing to
`the content item and its pair pointing to the memory or
`storage location of the geographic information. As another
`example, the data corresponding to the content item and the
`data corresponding to the geographic location information
`may be combined into an entirely new datafile.
`The association may occur automatically or manually. In
`many cases, an automatic associationwill take place, and the
`geographic information as metadata, or may include data
`system will automatically associate each content item with
`such as paired pointers, content
`items, and geographic
`its related location data. In some instances, manual control
`information as described above.
`may be desired, and in this case the user can decide when to
`Once obtained,
`the geographic locations are displayed
`attach geographic information with content
`items. This
`(step 52). In one implementation, the locations are displayed
`alternative may be preferable when the user desires to
`as situated on a map. In another, they are displayed asalist
`control the “granularity” of the data. For example, the user
`of locations. The locations may be displayed via icons,
`may wishforall video clips taken within a 10-mile radius to
`textual data such as city name,
`thumbnails,
`latitude and
`be associated with the same central city. Similarly, a hybrid
`longitude coordinates, or via other means.
`association may be provided, in which automatic associa-
`The user may then select one ofthe plurality ofdisplayed
`tions occur, but within the confines ofthe granularity set by
`geographic locations (step 54). Upon selection, if only one
`the user, as described above. Alternatively, an interactive
`content item is associated with the geographic location, the
`map maybe displayed on the camera LCD, allowing the user
`content item may be rendered automatically. Alternatively,
`to pin-point the exact location of a content item.
`an indicator of the content
`item may be displayed.
`If
`A further implementation is to associate geographic data
`multiple content items are associated with a single geo-
`with content items at an arbitrary time, for example,after all
`graphic location, then a set of indicators of such content
`items are displayed (step 56). These indicators may be
`content items are recorded, at the time of upload, or at the
`thumbnails, icons, file names, or other such content indica-
`time of display. One way of accomplishing this implemen-
`tation is to record and store geographic information peri-
`tors. Upon selection by the user (step 58),
`the indicated
`odically, for example, every 15 minutes, as well as the time
`content itemis rendered (step 62). The rendering may occur
`at which the geographic information was obtained. Content
`by playing back the video clip in a media player appropriate
`items may then be recorded and time stamped. When the
`to the video content
`file type.
`In an embodiment,
`the
`association occurs, the geographic location information that
`playback is a streaming video playback, where the videoclip
`content is streamed from a network server.
`has a time stamp closest-in-time to that of the content item
`is associated with the content item. In enhancementsofthis
`implementation, extrapolation algorithms can be employed
`to deduce or approximate geographic locations for content
`items whose time stamps are between the time stampsofthe
`geographic location information.
`The data association may then be stored (step 46). If the
`association occurs via placing oneset of data into the other,
`such as via placing metadata about the geographic location
`into the content item, any superfluous or redundant geo-
`graphic location data may then be discarded. For example,
`there may be no need to retain geographic location data in
`a separate storage location once the same has beenstored as
`metadata in the item of content.
`
`In one exemplary embodiment, a user may record video
`content in a location and upload or share the same (step 74).
`One or more friends, which may or may not be members of
`a share group, then receive an email message or aninstant
`message whenthe video is available for watching (step 76).
`They may click a link in the message to access it.A map-like
`userinterface may then be displayed withthe location of the
`video indicated (step 78). The video maystart playing (step
`84) when a user clicks a locationor icon on the user interface
`(step 82) or alternatively may immediately start playing
`when the mapis displayed (step 86). In one implementation,
`the video is displayed (step 88). In one embodiment, the
`video is displayed above the map as an overlay.
`
`8
`Upload of Content from the Content Detection System
`The next step is to upload the associated data (step 47).
`This upload may occur wirelessly, such as via a WiFi
`connection, or via a wired link, for example, a PC connec-
`tion such as a USB cable directly or indirectly to a computer
`or network. In the case of a WiFi connection, the connection
`may occur automatically. The upload may also occur over a
`network such as a LAN or WLAN.
`The upload, like the association, can occur automatically
`or manually, the former having the potential of occurring
`whenever the system is connected to a network, such as a
`WiFi network. In some systems, the upload occurs periodi-
`cally, after a predetermined period of time, for example,
`every 15 minutes. In other systems, the upload may occur
`manually, for example, via user selection and use of button
`19. The upload of data may occur for all stored content items
`or for only a subset ofall stored content items. The upload
`may be directly to the user’s computer only, or may occur to
`a server. If the a