(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2013/0178257 A1
`Langseth
`(43) Pub. Date:
`Jul. 11, 2013
`
`US 2013 0178257A1
`
`(54) SYSTEMAND METHOD FOR INTERACTING
`WITH VIRTUAL OBJECTS IN AUGMENTED
`REALITIES
`Inventor: Justin Langseth, Great Falls, VA (US)
`
`(75)
`
`(73)
`
`Assignee: Augaroo, Inc., Great Falls, VA (US)
`
`(21)
`
`Appl. No.: 13/345,189
`
`(22)
`
`Filed:
`
`Jan. 6, 2012
`
`Publication Classification
`
`(51)
`
`Int. C.
`A63F 9/24
`G06T I5/00
`G09G 5/00
`
`(2006.01)
`(2011.01)
`(2006.01)
`
`(52) U.S. CI.
`USPC ................................ 463/4; 34.5/633: 345/419
`ABSTRACT
`(57)
`The system and method described herein may be used to
`interact with virtual objects in augmented realities. In particu
`lar, the system and method described herein may include an
`augmented reality server to host software that Supports inter
`action with virtual objects in augmented realities on a mobile
`device through an augmented reality application. For
`example, the augmented reality application may be used to
`create and deploy virtual objects having custom designs and
`embedded content that can be shared with other users to any
`suitable location, similarly interact with virtual objects and
`embedded content that other users created and deployed
`using the augmented reality application, participate in games
`that involve interacting with the virtual objects, obtain incen
`tives and targeted advertisements associated with the virtual
`objects, and engage in Social networking to stay in touch with
`friends or meet new people via interaction with the virtual
`objects, among other things.
`
`
`
`970
`
`Physical Reality Image
`
`975
`Goal indicator -/
`
`995
`
`Goal Distance
`
`Out Of Bounds
`
`Time Remaining
`
`O:47
`
`940a
`
`960
`
`930
`
`940
`
`Niantic's Exhibit No. 1006
`Page 001
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 1 of 10
`
`US 2013/01 78257 A1
`
`
`
`190
`N-
`
`Application
`Marketplace
`
`User / Mobile
`Device Data
`
`Wirtual Object
`
`Processor
`
`175
`
`ARSoftware
`
`18O
`
`Incentive
`Data
`
`Advertising
`Data
`
`Augmented Reality Server
`
`12O
`N
`
`Camera
`
`Media
`
`125
`
`130
`N
`
`AR App
`
`Other Apps
`
`135
`
`140
`N
`
`Processor
`
`User Interface
`
`Mobile Device
`
`Figure 1
`
`Niantic's Exhibit No. 1006
`Page 002
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 2 of 10
`
`US 2013/01 78257 A1
`
`
`
`Figure 2
`
`260
`
`210
`
`220
`
`240
`
`230
`
`Niantic's Exhibit No. 1006
`Page 003
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 3 of 10
`
`US 2013/01 78257 A1
`
`3 O
`
`
`
`370
`
`340
`
`360
`
`Niantic's Exhibit No. 1006
`Page 004
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 4 of 10
`
`US 2013/01 78257 A1
`
`400A
`
`
`
`Physical Reality Image
`
`Virtual Objects
`
`Live View
`
`Figure 4A
`
`450
`
`440
`
`460
`
`430
`
`Niantic's Exhibit No. 1006
`Page 005
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 5 of 10
`
`US 2013/01 78257 A1
`
`
`
`aO
`
`Physical Reality Image
`
`Figure 4B
`
`Niantic's Exhibit No. 1006
`Page 006
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 6 of 10
`
`US 2013/01 78257 A1
`
`500
`
`
`
`570
`
`Virtual Objects
`
`Figure 5
`
`510
`
`520
`
`540
`
`560
`
`590
`
`530
`
`Niantic's Exhibit No. 1006
`Page 007
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 7 of 10
`
`US 2013/01 78257 A1
`
`6 O
`
`
`
`s
`
`Comments
`
`Contents
`
`User Name
`
`User Image
`
`Comment Text
`
`Comment Post Time
`
`User Name
`
`User Image
`
`CommentText
`
`Comment Post Time
`
`User Name
`
`User Image
`
`CommentText
`
`Comment Post Time
`
`Wirtual Objects
`
`Figure 6
`
`610
`
`62O
`
`640
`
`630
`
`Niantic's Exhibit No. 1006
`Page 008
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 8 of 10
`
`US 2013/01 78257 A1
`
`
`
`Content
`Thumbnail
`
`Content
`Thumbnail
`
`Expanded Content Display
`
`Wirtual Objects
`
`Figure 7
`
`710
`
`720
`
`740
`
`730
`
`Niantic's Exhibit No. 1006
`Page 009
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 9 of 10
`
`US 2013/01 78257 A1
`
`8 O
`
`
`
`Free Spirit
`Kick anywhere in the world and interact with people
`
`810
`
`Geocache
`Create a virtual object for others to find and put content in
`
`Treasure Hunt
`Create a treasure or scavenger hunt
`
`Kickball / Interactive Game
`Create a game that allows bump-kicks or virtual object
`interaction, with or without Scoring
`
`More Virtual Object Applications ...
`Create and find virtual vending machines, incentives,
`events, games, and more...
`
`Virtual Objects
`
`Figure 8
`
`Niantic's Exhibit No. 1006
`Page 0010
`
`

`

`Patent Application Publication
`
`Jul. 11, 2013 Sheet 10 of 10
`
`US 2013/01 78257 A1
`
`
`
`970
`
`Physical Reality Image
`
`995
`
`Goal Distance
`
`Time Remaining
`
`O:47
`
`Figure 9
`
`Niantic's Exhibit No. 1006
`Page 0011
`
`

`

`US 2013/01782.57 A1
`
`Jul. 11, 2013
`
`SYSTEMAND METHOD FOR INTERACTING
`WITH VIRTUAL OBJECTS IN AUGMENTED
`REALITIES
`
`FIELD OF THE INVENTION
`0001. The invention generally relates to a system and
`method for interacting with virtual objects in augmented
`realities, and in particular, to enabling users to create and
`deploy virtual objects having custom visual designs and
`embedded content or other virtual items that can be shared
`with other users to any suitable location, interact with virtual
`objects and embedded content or other virtual items that other
`users created and deployed into certain locations, participate
`in treasure or scavenger hunts to locate and/or collect the
`virtual objects, obtain special offers, coupons, deals, incen
`tives, and targeted advertisements associated with the virtual
`objects, play games that involve interaction with the virtual
`objects, and engage in Social networking to stay in touch with
`friends or meet new people via interaction with the virtual
`objects, among other things.
`
`BACKGROUND OF THE INVENTION
`0002 Augmented reality generally refers to a field in com
`puter technology that relates to combining computer-gener
`ated data and real-world data, which typically involves over
`laying virtual imagery over real-world imagery. For example,
`many television sports broadcasts now incorporate aug
`mented reality applications to superimpose or otherwise over
`lay virtual images over real-world game action to provide
`viewers additional information about the broadcast or other
`wise enhance the viewer experience (e.g., football broadcasts
`often superimpose a virtual “first down marker to show the
`distance that the offense has to cover to continue a current
`drive, baseball broadcasts often superimpose a virtual “strike
`Zone' marker to indicate whether a certain pitch was a ball or
`strike, etc.). However, augmented reality systems have his
`torically required Substantial computing resources, a require
`ment which has interfered with the ability to deliver aug
`mented reality applications to everyday users that would
`otherwise benefit from having virtual imagery provide to
`better interact with real-world environments. Further, devel
`oping augmented reality applications that the common con
`Sumer can use has tended to be restrained due to difficulties in
`Suitably tracking user viewpoints that applications need to
`know in order to properly render virtual imagery based on
`where the user may be looking in the real-world.
`0003 More recently, substantial increases in the comput
`ing resources associated with many consumer electronics
`have brought about new opportunities to deliver augmented
`reality applications to everyday users. For example, common
`features in many (if not all) Smartphones and computers avail
`able in the marketplace today include built-in cameras, video
`capabilities, location detection systems, high-resolution dis
`plays, and high-speed data access, among others. As such,
`many modern consumer electronics can now have capabili
`ties that can suitably overlay virtual imagery over real-world
`imagery, which may support new tools to enhance how users
`experience physical reality. For example, using the built-in
`camera and the built-in location detection system on a mobile
`device to sense the viewpoint and physical location associ
`ated with a user, an augmented reality application may add
`virtual imagery associated with the sensed physical location
`to a display that represents the sensed viewpoint to visually
`
`represent additional information about the physical reality
`visible through the camera lens. Accordingly, augmented
`realities have significant potential to change how users view
`the real-world in many ways because augmented realities can
`show more information about user Surroundings than would
`otherwise be available through the physical senses alone.
`
`SUMMARY OF THE INVENTION
`
`0004. According to one aspect of the invention, the system
`and method described herein may support interacting with
`virtual objects in augmented realities. In particular, the sys
`tem and method described herein may generally host aug
`mented reality Software on an augmented reality server,
`which may support the interaction with the virtual objects in
`augmented realities on mobile devices (e.g., a Smartphone,
`augmented reality glasses, augmented reality contact lenses,
`head-mounted displays, augmented realities directly tied to
`the human brain, etc.). In one implementation, the augmented
`reality interaction Supported in the system and method
`described herein may generally enable users to create and
`deploy virtual objects having custom visual designs and
`embedded content or other virtual items that can be shared
`with other users to any suitable worldwide location, interact
`with virtual objects that other users created and deployed in
`addition to any content or other virtual items that may be
`embedded therein, participate in treasure hunts, Scavenger
`hunts, or other interactive games that involve interaction with
`the virtual objects, obtain special offers, coupons, incentives,
`and targeted advertisements associated with the virtual
`objects or the interaction therewith, and engage in Social
`networking to stay in touch with friends or meet new people
`via interaction with the virtual objects, among other things.
`0005 According to one aspect of the invention, to use the
`system and method described herein, a user associated with a
`mobile device may download an augmented reality applica
`tion over a network from an application marketplace, wherein
`the augmented reality application may be free, associated
`with a one-time fee, or available on a Subscription basis.
`Alternatively (and/or additionally), certain features associ
`ated with the augmented reality application may be free and
`the user may be required to pay to upgrade the augmented
`reality application or activate additional features associated
`therewith (e.g., a purchasing option in the augmented reality
`application may enable the user to buy virtual objects having
`certain types, buy certain designs that can be applied to the
`virtual objects, upload custom designs that can be applied
`thereto, etc.). In one implementation, the augmented reality
`server may therefore make the augmented reality application
`available to the mobile device via the application market
`place, which may share collected revenue associated with any
`fees charged to the user associated with the mobile device
`with an entity that provides the augmented reality server.
`Alternatively, in one implementation, the augmented reality
`server may make the augmented reality application directly
`available to the mobile device without employing the appli
`cation marketplace to collect or otherwise charge any fees to
`the user associated with the mobile device. Moreover, in one
`implementation, one or more companies or other Suitable
`entities may sponsor certain virtual objects for advertising or
`promotional purposes, in which case revenue associated
`therewith may be shared with or paid to the entity that pro
`vides the augmented reality server, used to reduce or elimi
`
`Niantic's Exhibit No. 1006
`Page 0012
`
`

`

`US 2013/01782.57 A1
`
`Jul. 11, 2013
`
`nate fees that may be charged to the user, or otherwise cus
`tomize certain features or contractual arrangements
`associated with the system.
`0006. According to one aspect of the invention, to support
`the augmented reality application, the mobile device may
`include a processor to execute the augmented reality applica
`tion, location sensors to sense information relating to a cur
`rent location, position, and/or orientation associated with the
`mobile device (e.g., a GPS sensor, compass, accelerometer,
`gyroscope), location data that relates to the current location,
`position, and/or orientation associated with the mobile device
`and other worldwide location-dependent information, a cam
`era to sense a physical reality that represents a current view
`point associated with the mobile device, and a user interface
`that shows the physical reality sensed with the camera and any
`virtual objects that may be present therein on a display. In one
`implementation, the physical reality that may be combined
`with virtual reality or virtual objects in any particular aug
`mented reality described herein need not be limited to any
`particular geography, in that the physical reality may be on
`water, at a certain altitude in the air or above ground, or even
`in space, on the moon, on other planets, or any other location
`in the world or the universe to the extent that current or future
`technologies may permit sensing and tracking location infor
`mation associated therewith. In addition, the mobile device
`may include one or more other applications functionality
`integrated with the augmented reality application (e.g., a
`Social networking application that the augmented reality
`application may use to interact with other users). In one
`implementation, the mobile device may further include a
`database or another suitable repository that contains text,
`pictures, graphics, audio, video, icons, games, Software, or
`other content or other virtual items that may be embedded in
`or otherwise associated with virtual objects that the user
`creates or interacts with via the augmented reality applica
`tion, wherein content or other virtual items to embed in the
`virtual objects may be uploaded to the augmented reality
`server (e.g., to make the embedded content or other virtual
`items available to other users).
`0007 According to one aspect of the invention, the aug
`mented reality server may include a processor, augmented
`reality Software, and various databases or data repositories to
`Support interaction with the virtual objects in the augmented
`realities via the augmented reality application installed on the
`mobile device, wherein the various databases or data reposi
`tories may contain user and mobile device data, virtual object
`content, incentive data, and advertising data. In particular, the
`augmented reality server may initially register the user asso
`ciated with the augmented reality application in response to
`the augmented reality application having been installed on
`the mobile device and used to initiate communication with the
`augmented reality server, wherein registering the user may
`include the augmented reality server obtaining personal data
`associated with the user, identifying data associated with the
`mobile device, or any other information that may suitably
`relate to using the augmented reality application to access
`services, content, virtual items, or other data via the aug
`mented reality server. As such, the augmented reality server
`may store the personal data associated with the registered
`user, the identifying data associated with the mobile device,
`and any other Suitable information relating to the user and/or
`the mobile device in the user and mobile device data reposi
`tory. In addition, the user may create a personal profile page
`associated with the augmented reality application and Subse
`
`quently post, add, link, or otherwise Submit information to
`customize the personal profile page, wherein the information
`associated therewith may be further stored in the user and
`mobile device data repository. In one implementation, addi
`tional information stored in the user and mobile device data
`repository may include payment information that the user
`Submits to the augmented reality server, usage data associated
`with the augmented reality application, and records that relate
`to the location associated with the mobile device, among
`other things.
`0008 According to one aspect of the invention, the virtual
`object content may include content or other virtual items that
`the user submits in relation to virtual objects that the user
`created, collected, or otherwise interacted with via the aug
`mented reality application. For example, the virtual object
`content may include any text, pictures, graphics, audio, video,
`icons, games, Software, virtual currency, real currency that
`can be exchanged for actual cash or electronic credits, maps,
`offers, coupons, or other content or virtual items embedded in
`or otherwise associated with the virtual objects and any
`designs or other customizations that have been applied
`thereto. For example, the user may choose the design to apply
`to any particular virtual object from defaults available via the
`augmented reality application, upload a custom design to the
`augmented reality server, or take a picture to create the design
`to apply to the virtual object, and in each case the design
`chosen by the user may be applied to a surface associated with
`the virtual object (e.g., wrapped around a three-dimensional
`surface associated with the virtual object). In addition, the
`virtual object content may include data to represent the virtual
`objects that the user and/or other users have created and
`deployed into various worldwide locations, which may be
`associated with GPS coordinates, compass headings associ
`ated with rotational orientations, or other suitable location
`data that indicates the worldwide locations where the virtual
`objects have been deployed. In one implementation, the aug
`mented reality server may dynamically update the GPS coor
`dinates, compass headings, or other Suitable location data
`associated with the virtual objects in response to one or more
`users finding and/or moving the virtual objects to a new
`location (i.e., to reflect the new locations where the virtual
`objects may have been moved).
`0009. According to one aspect of the invention, the incen
`tive data may generally include content or other virtual items
`relating to deals, special offers, coupons, or other incentives
`that may be available to users associated with the augmented
`reality application. For example, various third-parties may
`Submit the deals, special offers, coupons, or other incentives
`to the augmented reality server and specify certain locations
`where the deals, special offers, coupons, or other incentives
`may be available via the augmented reality application. Thus,
`in one implementation, the incentive content relating to the
`deals, special offers, coupons, or other incentives may be
`associated with virtual objects that can be found in the speci
`fied locations via the augmented reality application, and the
`augmented reality server may deliver the deals, special offers,
`coupons, or other incentives to the augmented reality appli
`cation on the mobile device in response to the user finding and
`interacting with the associated virtual objects in the specified
`locations. In one implementation, the advertising data may
`similarly include advertisement content that third-parties
`submit to the augmented reality server, which may be deliv
`ered to the augmented reality application on the mobile
`device in a manner targeted to the user associated therewith
`
`Niantic's Exhibit No. 1006
`Page 0013
`
`

`

`US 2013/01782.57 A1
`
`Jul. 11, 2013
`
`(e.g., based on the personal data associated with the user,
`friends associated with the user, the identifying data associ
`ated with the mobile device, the location data associated with
`the mobile device, etc.). In one implementation, the advertis
`ing data may similarly be associated with virtual objects that
`can be found in certain locations, whereby the advertisements
`may be delivered to the augmented reality application in
`response to the user finding and interacting with the associ
`ated virtual objects. Alternatively (or additionally), the adver
`tisements and the deals, special offers, coupons, or other
`incentives may not necessarily be associated with any par
`ticular virtual objects and instead delivered to the augmented
`reality application in response to certain conditions or crite
`18.
`0010. According to one aspect of the invention, in one
`implementation, the augmented reality server may therefore
`maintain and utilize the user and mobile device data, the
`virtual object content, the incentive data, and the advertising
`data to Support interaction among the augmented reality
`application and other users associated with other mobile
`devices that have the augmented reality application installed
`thereon. For example, the user may grant the augmented
`reality application access to social networking or other third
`party applications installed on the mobile device that relate to
`the other users or the other mobile devices, whereby the
`augmented reality application may access the Social network
`ing or other third-party applications to Support the interaction
`among the augmented reality application and other users
`associated with other mobile devices that have the augmented
`reality application installed thereon. In addition, the aug
`mented reality server may use the user and mobile device
`data, the virtual object content, the incentive data, and the
`advertising data to deliver information to the augmented real
`ity application that relates to targeted advertisements, incen
`tives, special offers, coupons, and new features associated
`with the augmented reality application. Furthermore, the aug
`mented reality server may store cookies or other state data on
`the mobile device to preserve settings associated with the
`augmented reality application, or the user may have the
`option to disable or otherwise decline to store the cookies or
`other state data on the mobile device, in which case certain
`features associated with the augmented reality application
`that require the cookies or other state data may be disabled.
`0011. According to one aspect of the invention, in
`response to having registered the user associated with the
`augmented reality application and Suitably populating the
`user and mobile device data, the virtual object content, the
`incentive data, and the advertising data, the augmented reality
`application may be used to interact with virtual objects in the
`augmented realities. For example, in one implementation, the
`location sensors associated with the mobile device may con
`tinuously obtain location data that represents the current loca
`tion, position, and/or orientation associated with the mobile
`device, which the augmented reality application may continu
`ously communicate to the augmented reality server. As such,
`the augmented reality server may use the current location data
`associated with the mobile device to derive real-world coor
`dinates that represent the viewpoint associated with the cam
`era on the mobile device. For example, in one implementa
`tion, the augmented reality server may use image registration,
`image recognition, visual odometry, or other Suitable tech
`niques to detect interest points, fiduciary markers, or optical
`flow information from the location data to detect real-world
`features that represent the viewpoint associated with the cam
`
`era (e.g., corners, edges, or other real-world features in a
`scene that represents the camera viewpoint). The augmented
`reality server may then map geometry associated with the
`real-world features that represent the camera viewpoint to
`construct real-world coordinates that represent the scene cor
`responding to the camera viewpoint, which may be correlated
`to the coordinates associated with the virtual objects managed
`on the augmented reality server.
`0012. According to one aspect of the invention, in
`response to identifying any virtual objects in the virtual object
`content repository having coordinates that are present within
`the camera viewpoint, the augmented reality server may
`deliver the virtual object coordinates and any custom designs,
`embedded contentor other virtual items, or other suitable data
`relating thereto to the augmented reality application on the
`mobile device, which may render the identified virtual objects
`on the user interface associated therewithin combination with
`the scene corresponding to the camera viewpoint. For
`example, the augmented reality application may cause the
`user interface to Superimpose the virtual objects over a real
`world image that represents the camera viewpoint, thereby
`generating an augmented reality that combines the virtual
`objects and the scene that represents the camera viewpoint.
`Further, as noted above, the augmented reality application
`may continuously communicate current location, position,
`and/or orientation data associated with the mobile device to
`the augmented reality server, which may use the current loca
`tion, position, and/or orientation data to continuously derive,
`map, or otherwise determine the current viewpoint associated
`with the camera. As such, based on the current camera view
`point, the augmented reality server may determine whether
`any virtual objects are present in the current camera view
`point, whereby the augmented reality application on the
`mobile device may use data that the augmented reality server
`determines in relation thereto to continually refresh the aug
`mented reality shown in the user interface to reflect move
`ments or changes in the current camera viewpoint. For
`example, the augmented reality application may refresh the
`location in the augmented reality where the virtual objects
`exist to reflect the current viewpoint, remove the virtual
`objects from the augmented reality shown in the user inter
`face if virtual objects that were previously displayed therein
`are no longer present in the current viewpoint, or otherwise
`refresh the augmented reality shown in the user interface
`based on any virtual objects that may or may not be present in
`the current camera viewpoint. Accordingly, the user may
`simply point the camera associated with the mobile device at
`real-world Surroundings, and the augmented reality applica
`tion may transparently communicate with the augmented
`reality server in a Substantially continuous manner to refresh
`the augmented reality shown in the user interface to reflect
`whether or not any virtual objects are present in the surround
`ings where the camera currently points, and further to prop
`erly orient and re-orient the virtual objects with respect to
`distances, positions, and rotations associated therewith and/
`or other virtual objects relative to where the camera currently
`points in Substantially real-time.
`0013. Other objects and advantages of the invention will
`be apparent to those skilled in the art based on the following
`drawings and detailed description.
`BRIEF DESCRIPTION OF THE DRAWINGS
`0014 FIG. 1 illustrates an exemplary system for interact
`ing with virtual objects in augmented realities, according to
`one aspect of the invention.
`
`Niantic's Exhibit No. 1006
`Page 0014
`
`

`

`US 2013/01782.57 A1
`
`Jul. 11, 2013
`
`0015 FIGS. 2-9 illustrate exemplary user interfaces that
`may support interacting with virtual objects in augmented
`realities, according to various aspects of the invention.
`
`DETAILED DESCRIPTION
`0016. According to one aspect of the invention, FIG. 1
`illustrates an exemplary system 100 for interacting with vir
`tual objects in augmented realities. In particular, the system
`100 shown in FIG. 1 may generally include an augmented
`reality server 150 that hosts augmented reality software 180
`to support interaction with the virtual objects in augmented
`realities on a mobile device 110, which may include a smart
`phone, augmented reality glasses, augmented reality contact
`lenses, a head-mounted display, augmented realities directly
`tied to the human brain, or any other Suitable computing
`device that can incorporate Substantially similar components
`and capabilities to the mobile device 110 shown in FIG. 1 and
`described herein. In one implementation, as will be described
`in further detail herein, the augmented reality interaction that
`the system 100 supports may enable users to create virtual
`objects having custom visual designs, embed content or other
`virtual items into the virtual objects that can be shared with
`other users (e.g., text, photos, videos, etc.), deploy the virtual
`objects to any suitable worldwide location, interact with the
`virtual objects and embedded content or other virtual items
`that other users created and deployed into the world, partici
`pate in treasure or Scavenger hunts to locate and/or collect
`virtual objects, obtain special offers, coupons, and other
`incentives associated with the virtual objects, play games that
`involve interacting with the virtual objects, and engage in
`Social networking to stay in touch with friends or meet new
`people via interacting with the virtual objects, among other
`things.
`0017. In one implementation, in order to use the system
`100 and interact with virtual objects in augmented realities, a
`user associated with the mobile device 110 may download an
`augmented reality application 130 over a network from an
`application marketplace 190 (e.g., iTunes, Android Market,
`etc.), wherein the augmented reality application 130 may be
`free, associated with a one-time fee, or available on a Sub
`Scription basis. Furthermore, in one implementation, certain
`features associated with the augmented reality application
`130 may be free and the user may be required to pay one or
`more fees to upgrade the augmented reality application 130
`and activate one or more additional features (e.g., a purchas
`ing option in the augmented reality application 130 may
`enable the user to buy virtual objects having certain types, buy
`certain designs that can be applied to the virtual objects,
`upload custom designs that can be applied thereto, etc.). In
`one implementation, the augmented reality server 150 may
`therefore make the augmented reality application 130 avail
`able to the mobile device 110 via the application marketplace
`190, which may share collected revenue associated with any
`fees charged to the user associated with the mobile device 110
`with an entity that provides the augmented reality server 150.
`Alternatively, in one implementation, the augmented reality
`server 150 may make the augmented reality application 130
`directly available to the mobile device 110 without employ
`ing the application marketplace 190 to collect or otherwise
`charge any fees to the user associated with the mobile device
`110. Moreover, in one implementation, one or more compa
`nies or other Suitable entities may sponsor certain virtual
`objects for advertising or promotional purposes, in which
`case revenue associated therewith may be shared with or paid
`
`to the entity that provides the augmented reality server 150,
`used to reduce or eliminate fees that may be charged to the
`user, or otherwise customize certain features or contractual
`arrangements associated with the system 100.
`0018. In one implementation, to support executing the
`augmented reality application 130, the mobile device 110
`may generally include a processor 140 to execute the aug
`mented reality application 130, location sensors 115a (e.g., a
`GPS sensor, compass, accelerometer, gyroscope, etc.) to
`sense information relating to a current location, position,
`and/or orientation associated with the mobile device 110,
`location data 115b that relates to maps, points of interest, or
`other location-dependent information in any suitable world
`wide location, a camera 120 to sense a physical reality that
`represents a current viewpoint associated with the mobile
`device 110, and a user interface 145 that shows the physical
`reality sensed with the camera 120 and any virtual objects that
`may be present therein on a display associated with the
`mobile device 110. In one implementation, the physical real
`ity that may be combined with virtual reality or virtual objects
`in any particular augmented reality described herein need not
`be limited to any particular geography. As such, the aug
`mented reality application 130 may be used to interact with
`virtual realities or virtual objects in physical realities located
`on w