SNAP INC. v. BLACKBERRY LIMITED
`IPR2019-00715
`
`Snap Inc. Ex. 1014 Page 0001
`
`

`

`US 10,454,995 B2
` Page 2
`
`Int. Cl.
`GO6F 16/74
`7
`Hal“)? 4/“ 1
`
`(51]
`
`(56)
`
`(201901)
`'
`(20] 8'01)
`References Cited
`
`113 PATEN'I‘ IK)CUMI£N'1‘S
`
`6154172 A
`6.629.136 Bl
`8,055.2?1 132 *
`
`1112000 Piccinnelli el :0.
`912003 Naiduu
`1132011 Nair ........................
`
`['104W 4502
`455.:‘4l8
`
`8,548.424 Bl "
`200410054665 Al
`2005100049815I Al“
`
`l0.-"20l3 Mikan et al.
`3.2004 Yoo ct a].
`l-"2005 Aoki
`.......................
`
`(30:51I 91543
`709.5206
`412005 Razza ................ GUGQ10.-"025
`200510086087 01*
`_
`20515
`.
`am 0 a .
`.
`..
`1
`.1
`-1"
`_
`1'
`-'
`3&213323; 1:}: 3633: R1822” 5‘ 131‘] ----------32:23:69;
`.
`.
`1"
`2
`I
`It
`3'2
`................
`'
`.-'
`2006 002 048 \1
`2 006 Johnson
`11010504910322?
`200610154674 Al *
`212006 Landschaft
`...... 1104M 1122522
`45514566
`812006 Jung ................ (501C 2113667
`345.16”
`.......................... 45514562
`[012006 Chni
`1.2.1.2006 chda 131 a].
`..................... 380595
`[31333; Qda-m c-t 31‘
`110413 732643
`f
`ugltanl """"""""
`1703780
`-
`..
`-
`"
`"“.-
`812009 Kusklncn ............. 1104W 64:00
`4551‘456.l
`
`200610181546 Al *
`
`200610240847 Al "
`200010291805 A1 ’1‘
`g$gr321211§§§é ii :1:
`“
`f
`I
`.-
`20090197612 Al *
`
`FOREIGN PATENT DOCUNLENTS
`
`JP
`JP
`
`2000-2355261
`2001-292389
`
`81‘2 000
`[0:2 001
`
`2003-209??9 A
`JP
`2003-233555
`JP
`2006—33273
`J'P
`1020060095601 A
`KR
`10—2007—0002159 A
`KR
`WO 98354896
`W0
`wo 02117130 A3
`wo
`2003007542 Al
`WO
`03101 I39 Al
`W0
`W0 W0 20042090903 Al
`WU
`2006018212 Al
`
`7.32003
`812003
`212006
`9.32006
`11200?
`12.11998
`2.12002
`122003
`1212003
`101'2004
`2.-’2006
`
`.............. lI04N 5’91
`.............
`(3013 5.14
`
`............. GllB 271'10
`
`..
`.
`..
`()1[[1.LR PUBLICAI IONS
`.
`,
`_
`,_
`.
`.
`_
`James l-allows. Spy s—eye View. [he Atlantlc (Mar. 2006), avallable
`for download athtrp:-".*'wwwthcatlamiocmn’magazine’mhlave-“2006:
`031'spy—s—eye—viewfl04620f.“
`n
`--
`_-
`-
`-
`-
`Rick Vidallon. “How to Embed Your Youtubc Video into Google
`Maps . 2006. 5 pages. hllpL’IWWWA-151onefx.neL'anlcles-‘me—lo—
`.
`.
`embed—youlube—wdeu—In—g00gle—rnaps.hlm.
`“1°" Chim- “50°31” ”mating 53’0““ Ummd‘“ “W5 and “PS
`about (50090". Jun. 23. 2006, 3 pages. htlp:.-"fgooglcsystcm.blogspot.
`cum“2006105100001HMPS-wulube-videU-manhlml-
`Alex Chilu Gtmgle Operaling Syslern Unofficial News and Tips
`about Googlc. Jun. 23. 2006. 2 pages. http:.-".-"googlcsystcm.blogspol.
`colnflU061:06.-"pa_norajnjo—googlc—lnaps-wiLh-photog‘htm[,
`Author Unknown. 2 pages. hllp:.-’.I"vidteq.corn. dale unknown.
`Alller Unknown. 11l1p:.-'.-’Allas.l’reshlogicslutfiusxom. cupyrighl 2010.
`2 pages.
`.
`.
`_
`Allflwr Unknown. http:1'-’C]1pg]obc.com. 2 pages. Copyrlght 200?.
`Alllhor Unknown. hllpL-“x'WrlualltJLIrisl.C()1n. 5 pages Copyrighl 1994—
`2013.
`
`* cited by examiner
`
`Snap Inc. Ex. 1014 Page 0002
`Snap Inc. Ex. 1014 Page 0002
`
`

`

`US. Patent
`
`Oct. 22, 2019
`
`Sheet] of3
`
`US 10,454,995 32
`
`10
`
`\\
`19
`
`
`18
`
`
`
`
`
`
`16
`
`UPLOAD COMPONENT
`'I‘I‘
`
`GEOGRAPHIC DIGITAL VIDEO
`
`INFORMATION
`CAMERA
`DETECTOR
`
`
`14
`
`FIG. 1
`
`
`
`FIG. 2
`
`Snap Inc. Ex. 1014 Page 0003
`Snap Inc. Ex. 1014 Page 0003
`
`

`

`US. Patent
`
`Oct. 22, 2019
`
`Sheet 2 of 3
`
`US 10,454,995 82
`
`
`
`RECEIVE
`
`
`DETECT ITEM 0F
`GEOGRAPHIC
`
`
`CONTENT
`INFORMATION
`
`
`33
`RELATING TO
`
`
`
`CONTENT
`42
`
`
` ASSOCIATE
`
`
`
`
`ITEM OF
`CONTENT WITH
`RELATED
`GEOGRAPHIC
`
`INFORMATION
`44
`
`
`
` STORE
`
`ASSOCIATED
`
`DATA
`
`46
`
`
`
`UPLOAD
`
`ASSOCIATED
`
`
`DATA
`4?
`
`
`
`FIG. 3
`
`
`
` RECEIVE
`ASSOCIATED
`
`
`DATA
`
`
`48
`
`
`
`FIG. 4
`
` DISPLAY
`
`PLURALITY OF
`
`
`GEOGRAPHIC
`
`
`LOCATIONS
`52
`
`
`
`
`
`
`RECEIVE INPUT
`
`
`TO SELECT ONE
`
`OF THE
`PLURALITY
`54
`
` DISPLAY
`
`CONTENT
`
`
`INDICATORS
`
`
`ASSOCIATED
`WITH
`SELECTION
`56
`
`
`
`
`
`
`
`RECEIVE INPUT
`
`
`TO SELECT ONE
`RENDER
`
`
`
`OF THE
`INDICATED
`
`
`
`CONTENT
`CONTENT
`
`
`62
`INDICATORS
`
`
`53
`
`
`
`Snap Inc. Ex. 1014 Page 0004
`Snap Inc. Ex. 1014 Page 0004
`
`

`

`US. Patent
`
`Oct. 22,2019
`
`Sheet 3 of3
`
`US 10,454,995 82
`
` USER RECORDS
`
`
`
`
`AND UPLOADS
`VIDEO CONTENT
`74
`
`
`
`FRIENDS OR
`SHARE GROUP
`
`INVITED TO
`VIEW CONTENT
`76
`
`
`
`
`
`
`
`
`DISPLAY ONE
`OR MORE
`
`GEOGRAPHIC
`
`LOCATIONS
`
`78
`
`
`
`
`RENDER
`
`INDICATED
`
` CONTENT
`
`85
`
`
`
`
`RECEIVE INPUT
`TO SELECT ONE
`
`GEOGRAPHIC
`
`PLURALITY
`82
`
`
`
`
`
`U PLOAD
`ASSOCIATE D
`DATA TO FILE
`5 HARI NG
`SERVER
`64
`
`FILE SHARING
`SERVER
`ORGANIZES
`
`DATA. E.G.. VIA
`MAP
`66
`
`DEFINE SHARE
`GROUP
`68
`
`72
`
`SHARE GROUP
`ACCESSES AND
`VIEW CONTENT
`
`
`
`
`RENDER
`
`INDICATED
`DISPLAY VIDEO
`
`CONTENT
`88
`
`84
`
`
`
`
`FIG. 5
`
`FIG. 6
`
`Snap Inc. Ex. 1014 Page 0005
`Snap Inc. Ex. 1014 Page 0005
`
`

`

`US 10,454,995 32
`
`1
`SYSTEM AND METHOD FOR OBTAINING
`AND SHARING CONTENT ASSOCIATED
`WITH GEOGRAPHIC INFORMATION
`
`l"[lil.l) ()1: 'l‘lIl‘l INVENTION
`
`m
`
`15
`
`The invention relates to content recording. geographic
`location tagging of the recorded content, and sharing of the
`recorded and tagged content.
`
`BAC KGR€ )UNI)
`
`Prior art references purport to disclose GPS video map—
`ping systerns as well as systems that use wireless devices to
`create. access, and share geographical positioning data.
`Other prior art references purport to disclose location-tagged
`data provision and display systems, and methods for posi—
`tion—stamping photos or video clips taken with a digital
`CHIHCI'a.
`
`Such prior art disclosures squer from various drawbacks.
`however. For example, they lack the convenience and per—
`tability of a unitary full—featured system. They further lack
`convenient sharing capabilities, including web—based shar—
`ing capabilities.
`
`SUMMARY
`
`Systems and methods are provided for combining content
`files with geographical data and for sharing these files. The
`combination cart then be used to create enhanced user
`
`30
`
`interfaces for organizing, sharing, navigating and playing
`back the content files.
`In one embodiment. a digital video camera is combined
`with a source of geographical data (such as a UPS receiver)
`and a connection to a computer. generally as an integral
`component. Video clips captured by the camera are mantl—
`ally or automatically tagged with appropriate geoyaphic
`location data from the geographical data source, e.g.. as
`metadata. The tagged video flles are uploaded to the C011]-
`puter via an easy—to—use upload component. The connection
`to a computer may be an intemet connection to a server or
`a direct connection to personal computer. The connection
`may be wired or wireless, such as a Wilii or other such
`cormection. In the embodiment employing a wireless con-
`nection to the Internet. the tagged data files may be auto—
`matically uploaded whenever an available wireless connec—
`tion is obtained or available.
`
`A corresponding software application. which may be
`resident on the user‘s computer or which may be web—based,
`uses the video clips and geographical data to create an
`interactive user interface to browse and watch the video
`clips. Details vary by embodiment. but one such user
`interface appears as an interactive map with clickable links
`situated at locations where the video files were captured or
`as an interactive map that allows end—users to play video
`files associated with locations on the map. The system may
`also allow an indication of the time-sequence of the video
`clips to be displayed. such as via an arrow from point-to-
`point. In this way, reconstruction of, cg, a trip may be put
`in proper sequential order. Clicking the links renders or
`plays back the video files. A link may contain inlonnation
`about the video such as a title and image thumbnail. For
`example, a video image thumbnail is displayed on the map
`and is clickable to play the video.
`In web—based or other shared systems. the soitware appli—
`cation may limit those who can view the video clips to only
`one or more authorized groups of people or may leave the
`
`40
`
`50
`
`60
`
`2
`
`clips accessible to anyone. As noted. the application can be
`desktop installed software or a web application, but may also
`be resident on the camera (in which case a wired or wireless
`link may link the camera to a display monitor). The web
`application can be accessible from any temlinal that has an
`Internet connection, though the content files may be limited
`to only the one or more authorized groups.
`Sharing of the content files may be accomplished via
`“one-click” sharing using a dedicated physical or touch-
`screen-type button on the camera. An altemative embodi-
`ment
`is automatic sharing,
`for example, uploading and
`sharing without a user trigger whenever the camera is
`connected to the Internet.
`
`In one aspect. the invention is directed towards a device
`for detecting content and associating that content with
`geographic information. The device includes a detector
`configured to receive an item of content, a detector config—
`ured to receive information corresponding to a geographic
`location relating to the item of content; a correspondence
`module configured to associate the item of content with the
`geographic location information; and an upload component
`configured to transmit the associated item of content and
`geographic location infonnation to a remote system on a
`network.
`
`Implementation ofthe invention may include one or more
`of the following. The content detector may be a videocam—
`era, a still camera, or a microphone. The geographic location
`information detector may be a (i133 receiver or a mobile
`phone triangulation radio-based system. The correspon-
`dence module may associate the item of content to the
`geographic location information using metatags or metadata.
`The upload component may further include a wireless
`transmitter. The network may be the Internet.
`In another aspect,
`the invention is directed towards a
`method for associating an item of content with geographic
`information relating to the item of content. The method
`includes the steps of receiving an item ofcontent in a device,
`receiving geographic location information relating to the
`item of content in the device, associating the item of content
`with the geographic location information. and uploading the
`associated item of content and geographic location infonna—
`tion to a remote system on a network.
`Implementation of the invention may include one or more
`of the following. The step of associating may occur auto-
`matically or manually. If automatically, the associating may
`occur in a manner such as: every time an item of content is
`stored. periodically. or a combination of these.
`The step of uploading may occur automatically or manu-
`a lly. Ifautomatically, the uploading may occur every time an
`item of content is stored. every time a network connection
`is established, periodically, or a combination of these. If
`manually, the uploading may occur upon user operation of
`a dedicated input device.
`In another aspect,
`the invention is directed towards a
`computer readable medium containing instructions for caus—
`ing a computer to implement the above method.
`In yet a further aspect. the invention is directed towards,
`in a computer system having a gaphical user interface
`including a display and a selection device, a method of
`providing and selecting from a menu on the display. the
`menu organizing and displaying content
`items having
`related geographic location information. Steps of the method
`include: retrieving a set of menu entries for the menu. each
`of the menu entries representing a geographic location:
`displaying the set of menu entries on the display: receiving
`a menu entry selection signal
`indicative of the selection
`device pointing at a selected menu entry front the set of
`
`Snap Inc. Ex. 1014 Page 0006
`Snap Inc. Ex. 1014 Page 0006
`
`

`

`US 10,454,995 32
`
`3
`menu entries; in response to the signal. displaying a set of
`content
`items corresponding to the geographic location;
`receiving a content item selection signal indicative of the
`selection device pointing at a selected content item from the
`set of content items: and in response to the signal. rendering
`the selected content itetn.
`Implementations of the invention may include one or
`more of the following. The graphical user interface may
`form part of a system such as a videocamera, a computer. or
`a web-based browser. The step ofdisplaying the set of menu
`entries on the display may include displaying the set ofmenu
`entries on a map. with each menu entry disposed at a
`location corresponding to its geographic location. Each
`menu entry may be displayed along Willi a name of a city or
`state near
`the corresponding geographic location. The
`method may further comprise displaying arrows between the
`displayed menu entries.
`the arrows indicating a time—se—
`quence ofthe menu entries. The step of displaying the set of
`menu entries on the display may include displaying the set
`of menu entries in a list. The content items may be video
`clips.
`In a related aspect. the invention is directed towards a
`computer readable medium containing instructions for caus-
`ing a computer to implement the above method.
`In another aspect.
`the invention is directed towards a
`method of sharing an item of content with a group. including
`the steps of creating a share group having at least one
`membe _. receiving an item of content in a device. receiving
`geographic location information relating to the item of
`content in the device. associating the item of content with
`the geographic location information. uploading the associ—
`ated item of content and geographic location information to
`a remote system on a network, and notifying each member
`of the share group about the uploaded item of content.
`Implementations of the invention may include one or
`more of the following. The notifying step may be accom—
`plished by sending each member of the share group an
`instant message or an email about the uploaded item, where
`the instant message or email contains a cl ickable link which
`when clicked displays the item of content. The item of
`content may be displayed as a video overlayed above a map.
`The step of creating a share group may include inputting at
`least one contact
`into a share group list or analyzing a
`personal information manager to determine contacts. The
`instant message or email may be sent automatically when—
`ever a content item is uploaded.
`In a related aspect. the invention is directed towards a
`computer readable medium containing instructions for caus-
`ing a computer to implement the above method.
`In another aspect. the invention is directed towards a
`method of sharing an item of content with a group. including
`steps of creating a share group having at least one member.
`receiving an item of content
`in a device. receiving geo-
`graphic location information relating to the item of content
`in the device. associating the item of content with the
`geographic location infonnation. uploading the associated
`item of content and geographic location infonnation to a
`remote system on a network. and notifying each member of
`the share group about the uploaded item of content. When a
`member of the share grotip attempts to access the uploaded
`item of content. then further steps of the method include
`retrieving a set” of entries. each of the entries representing a
`geographic location, displaying the set of entries on a
`display. and receiving an entry selection signal indicative of
`the selection device pointing at a selected entry from the set
`of entries. liunher steps include, in response to the signal.
`displaying a set” of content
`items corresponding to the
`
`m
`
`15
`
`30
`
`40
`
`50
`
`60
`
`65
`
`4
`
`geographic location. receiving a content item selection sig-
`nal indicative of the selection device pointing at a selected
`content item from the set of content items. and in response
`to the signal. rendering the selected content item.
`Advantages of the invention may include one or more of
`the following. The content files with location data are easily
`uploadable to a computer. server or other storage. The same
`are navigable and may be viewed via an interactive user
`interface where the same are displayed as clickable links.
`such as by the use of thumbnails. icons or textual represen—
`tations. on a map corresponding to their geographic location.
`The invention provides an easy-to-use and convenient pro-
`cess for camera users to share video with goo-data and for
`others to watch the video over the Internet. For example.
`family members may watch videos ofa trip done by another
`family member and see where each video was shot on a map.
`Other advantages will be apparent from the description
`that follows. including the drawings and claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`l“ [G. 1 illustrates a schematic diagram ofa system accord-
`ing to a first embodiment of the invention.
`FIG. 2 illustrates a portion ofa map-based user interface
`according to a second embodiment of the invention.
`FIG. 3 is a flowchart illustrating a method for obtaining
`and uploading content items and geographic location infor-
`mation according to a third embodiment of the invention.
`FIG. 4 is a flowchart illustrating a method for displaying
`content and geographic location data according to a fourth
`embodiment of the invention.
`
`FIG. 5 is a flowchart illustrating a method for displaying
`content and geographic data according to a fifth embodiment
`of the invention.
`FIG. 6 is a flowchart illustrating a method for sharing
`content and geographic location data with a group of users.
`
`Dli'l‘AliJil) DESCRIPTION
`
`The following terms. defined here. are used throughout
`the specification and attached figures.
`“Uploading” an item of content or geographic location
`information or both refers to the act of transferring a file
`containing such infonnation from a device such as a video
`camera that is equipped with an upload component. The
`uploading is often accomplished wirelessly. but may also be
`accomplished using a wired connection, directly to a PC or
`network server.
`
`“Storing" data or a file refers to the act of placing data in
`permanent or temporary memory or storage. such as in a
`type ofRAM or disk- or card-based storage. The storing act
`could be very transitory. For example. data may be obtained,
`stored in a bufler. and uploaded immediately On the other
`hand. data could be stored indefinitely on. for example. a
`mini-DVD.
`“Geographic Location“ corresponds to a location at which
`or near where content data was obtained. “Geographic Data"
`corresponds to data constituting the geographic location. An
`example of geographic location information is the latitude.
`longitude. and altitude infon'nation obtained by a GPS
`system. Geographic location. in this context. can further
`refer to information such as camera direction and orienta—
`
`tion. An “Indication ofa Geographic Location” refers to a
`shorthand notation for geographic information displayed in
`a user interface. l"or example. all indication of geographic
`information may be an icon. a display of latitude and
`
`Snap Inc. Ex. 1014 Page 0007
`Snap Inc. Ex. 1014 Page 0007
`
`

`

`US 10,454,995 32
`
`5
`longitude coordinates. or the name ofa corresponding city
`and state that represents the geographic location.
`“Content Item" refers to a collection of data obtained by
`a detector. Typically. a content item is a continuous sequence
`of data.
`for example. a video clip obtained by a video
`camera. “Content Data" refers to the underlying data that
`constitutes the content item. In some cases, a content item
`refers to one or more related video clips. all corresponding
`to a single (or set of adjacent or nearby) geographic
`location(s). An “Indication of the Content Item“ refers to a
`shorthand notation for the content item displayed in a user
`interface. For example, an indication of a content item may
`be an icon. thumbnail. or filename that represents the content
`item. ('ontent items may also be audio files. digital still
`photographs. or the like.
`“Associating" a content item with a geographic location
`refers to the act of connecting one with another. such as by
`placing data corresponding to one in the file of the other. For
`example. associating a content
`iteln with a geographic
`location may refer to placing metatags or metadata about the
`geographic location in the file of the content item.
`“Rendering" refers to the act of playing back an item of
`content. For example. rendering a content
`item that is a
`video clip means to play back the video clip in a media
`player. The media player may be any content playback
`device that is capable of playing back the selected type of
`content.
`
`A “share group“ refers to a defined group of users that are
`a target audience of a shared file. A share group is typically
`defined by a user. and corresponds to that user‘s group of
`friends or other defined cohon. .A share group may also be
`constructed automatically. such as via an analysis of user
`settings in a separate application. such as a social network-
`ing service. A share group may also be created on a user
`system using. for example. contacts information such as is
`stored in Microsoft Outlookifi'» or using a buddy list in an
`Instant Messaging application. A share group may also be
`defined separately and users may join such a group using a
`software application or a website. For example, one such
`share group may be the ‘Lake Tahoe Videomakers’ Group.
`A group may also consist of only one person. In this case.
`sharing can be performed without specifying a group first.
`For example. sharing may be performed with various con-
`tacts without any fonnalion of a group. In this example. the
`friend may receive an email with a link to access the video.
`Content Detection System. for Example. a Wdeocamera
`Referring to FIG. 1. a content detection system 10 is
`shown according to a first embodiment of the invention.
`which may be an integrated videocainera system with an
`upload capability. The system 10 includes a digital video—
`camera 12 having a detector 13. the detector in this case
`including a lens and microphone to record video clips.
`The system 10 also includes a detector of geographic
`information 14. The geographic information detector 14
`often has an antenna 15. which may be intemal or external.
`The geographic information detector
`14 may be.
`for
`example. a GPS receiver. Ilowever. alternative geographic
`information detectors may be employed. including mobile-
`phone triangulation radio—based systems or other such sys—
`tems. The geographic information detector 14 usually
`receives data such as latitude and longitude. although alti-
`tude data may also be retrieved. along with infonnalion
`corresponding to the orientation and direction of system 10.
`The system 10 further includes a correspondence module
`16. which may be implemented in hardware. software. or a
`combination of both. The correspondence module 16 asso-
`ciates the item of content” with its related geographic infor-
`
`in
`
`IS
`
`30
`
`40
`
`50
`
`60
`
`6
`mation. For example, the correspondence module 16 may
`place inetatags or metadata in the file of the content item.
`these indicating the related geographic information. In this
`embodiment. it is important to note that the correspondence
`module need not be a separate circuit or algorithm. e.g..
`implemented as a separate circuit or algoriliu'n in hardware,
`software firmware. or the like. Rather. the correspondence
`module may form an integral part of the camera firmware
`and may be integrated with a module that saves a digital
`video file to local storage. The system 10 also incorporates
`an upload component 18. The upload component 18 orga-
`nizes the transmission of data corresponding to geographic
`information, the content item. and their association.
`It is
`noted that in cer1ain embodiments. as described below. the
`geographic location data is placed into the digital video file.
`In this case. and in others like it. the upload would be of the
`data itself. and there would not necessarily be a separate
`upload of geographic location information or association
`data. The upload component 18 may be wireless, in which
`case an antenna 22 may be employed. As with antenna 15.
`the antemia 22 may be internal or external. The upload
`component 18 may also be wired.
`in which case.
`for
`example, a USB or other such por1 may be provided.
`In a wireless configuration. the upload may occur to a
`wireless access point. wireless community network. WiFi
`hotspot. or any other such wireless network.
`A button 19 or other user—operable device may be pro—
`vided to allow the user to upload the associated content and
`geographic infonnalion conveniently. For example. the but-
`ton 19 may allow a “one—click“ upload capability. In this
`case. depression of the button causes an immediate upload
`of all or a portion of the content files. e.g._. whenever a
`network connection is available. The button 19 may also be
`virtual. for example. the same may be a button on a touch
`screen or other user interface of the camera. For example.
`when viewing captured video clips on a camera’s LC D. one
`of the applicable commands for a video may be ‘share
`online’.
`
`Altemalively. the system 10 may automatically upload the
`associated content and geographic information whenever a
`network connection is established. for example, whenever a
`wireless connection is obtained. particularly where the wire-
`less connection, such as a Wilii “hotspot”, is an unsecured
`network.
`
`The system components described above may be imple—
`mented in a single device. i.e.. a unitary full—featured inte—
`grated video camera system with a data upload capability.
`In use.
`the system 10 may be taken from location to
`location. filming content at each location. and uploading the
`same to a server or personal computer. Referring to FIG. 2.
`which indicates a portion of a user interface according to a
`second embodiment of the invention. and which is operable
`to display such content. the system 10 may be taken from
`location 26 to location 28. and then from location 28 to
`location 32. and likewise to location 34 and location 36. In
`each case content may be obtained. and later the content may
`be viewed by selection of any location indicator and further
`by the choice of content indicators for content obtained for
`that location. Indicators such as the arrows shown in FIG. 2
`
`may be employed to indicate a time—sequence of when the
`content was obtained. In this way. a trip may be recon-
`structed and the sequence of locations visited known.
`A method according to a third embodiment of the invert—
`tion is illustrated by the flowchart of FIG. 3. The method
`starts (step 37) and an item of content is detected.
`i.e..
`recorded (step 38). For example, a video clip is filmed.
`Geographic location information is received relating to the
`
`Snap Inc. Ex. 1014 Page 0008
`Snap Inc. Ex. 1014 Page 0008
`
`

`

`US 10,454,995 32
`
`7
`item (step 42). For example. GPS data may be
`content
`received and used as geographic location information. The
`geographic location information may be obtained periodi—
`cally. continuously. or at arbitrary times as designated by the
`user.
`
`Geographic information may also be obtained by use of
`detected 1? address information. in panicular. geolocation
`software may be employed to examine the IP address and
`determine what country. organization. or user it has been
`assigned to, and then an estimate may be made as to the
`user’s location based on that information. Such an estimate
`
`would be particularly accurate if the ttser were continuously
`connected to the Internet. Ilowever. if uploads are tnade only
`periodically. e.g.. at the end of a day of video recording, then
`the geographic location estimated would only be correspond
`to the upload location. not each individual recording loca—
`li()l].
`
`The item ofcontent is then associated with the geographic
`information (step 44). For example, metadata are added to
`the content
`item file to indicate the geographic location
`where the content item was obtained. In another example. a
`set of paired pointers are stored. with one pointer pointing to
`the content item and its pair pointing to the memory or
`storage location of the geographic information. As another
`example. the data corresponding to the content item and the
`data corresponding to the geographic location information
`may be combined into an entirely new data file.
`The association may occur automatically or manually. In
`many cases. an automatic association will take place. and the
`system will automatically associate each content item with
`its related location data. In some instances. manual control
`may be desired. and in this case the user can decide when to
`attach geographic information with content
`items. This
`alternative may be preferable when the user desires to
`control the “granularity“ of tlte data. For exatnple. tlte user
`may wish for all video clips taken within a lO-mile radius to
`be associated with the same central city. Similarly. a hybrid
`association may be provided. in which automatic associa—
`tions occur. but within the confines ofthe granularity set by
`the user. as described above. Alternatively. an interactive
`map may be displayed on the camera I .C[). allowing the user
`to pin-point the exact location of a content item.
`A further implementation is to associate geographic data
`with content items at an arbitrary time. for example, alter all
`content items are recorded. at the time of upload. or at the
`time of display. One way of accomplishing this implemen-
`tation is to record and store geographic infonnation peri—
`odically. for example. every 15 minutes. as well as the time
`at which the geographic information was obtained. Content
`items may then be recorded and time stamped. When the
`association occurs, the geographic local ion information that
`has a time stamp closest—in—time to that of the content item
`is associated with the content item. In enhancements of this
`implementation. extrapolation algorithtns can be employed
`to deduce or approximate geographic locations for content
`items whose time stamps are between the time stamps of the
`geographic location information.
`The data association may then be stored (step 46). If the
`association occurs via placing one set of data into the other,
`such as via placing lnetadata about the geographic location
`into the content item. any superfluous or redundant geo—
`graphic location data may then be discarded. For example.
`there may be no need to retain geographic location data in
`a separate storage location once the same has been stored as
`metadata in the item of content.
`
`8
`Upload of Content from the Content Detection System
`The next step is to upload the associated data (step 47).
`This upload may occur wirelessly. such as via a WiFi
`connection. or via a wired link. for example. a PC connec-
`tion such as a U SI} cable directly or indirectly to a computer
`or network. In the case of a WiFi connection. the connection
`may occur automatically. The upload may also occur over a
`network such as a LAN or WIAN.
`The upload. like the association. can occur automatically
`or manually. the former having the potential of occurring
`whenever the system is connected to a network. such as a
`Wil“i network. In some systems. the upload occurs periodi-
`cally. after a predetermined period of time. for example.
`every 15 minutes. in other systems. the upload may occur
`manually. for example. via user selection and use of button
`19. The upload ofdata may occttr for all stored content items
`or for only a subset of all stored content items. The upload
`may be directly to the user’s computer only. or may occur to
`a server. If the associated data is stored such that content
`items include geographic information metadata. then only
`the content items need be stored. If the associated data is
`
`stored as paired pointers. then the paired pointers. content
`items. and geographic information may all be uploaded.
`User Interface for Sharing Content and Geographic Infor-
`mation
`
`in
`
`IS
`
`Referring to FIG. 4. a flowchart is shown indicating a
`method of displaying the associated data. A diagram of the
`user interface displaying the data is indicated in FIG. 2. FIG.
`4 shoWs the first step being receiving the associated data
`(step 48). This receipt of associated data may be receipt of
`content items in the case where the content items include
`
`30
`
`geographic information as metadata. or may include data
`such as paired pointers. content
`items. and geographic
`informal ion as described above.
`()nce obtained.
`the geographic locations are displayed
`(step 52). In one implementation. the locations are displayed
`as situated on a map. In another. they are displayed as a list
`of locations. The locations may be displayed via icons.
`texttlal data such as city name.
`thumbnails.
`latittlde and
`longitude coordinates. or via other means.
`The user may then select one of the plurality of displayed
`geographic locations (step 54). Upon selection. if only one
`content item is associated with the geographic location. the
`content iteln may be rendered automatically. Alternatively,
`an indicator of the content
`item may be displayed. If
`multiple content items are associated with a single geo—
`graphic location. then a set of indicators of such content
`items are displayed (step 56). These indicators may be
`thumb