Join the discussion @ p2p.wrar.com
`
`Programming for junaio, Layar, and Wikitude
`
`Niantic's Exhibit No. 1016
`Page 001
`
`Niantic's Exhibit No. 1016
`Page 001
`
`

`

`PROFESSIONAL
`AUGMENTED REALITY BROWSERS FOR SMARTPHONES
`
`INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
`
`(cid:2)
`PART I
`CHAPTER 1
`
`INTRODUCTION
`Introducing Augmented Reality (AR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
`
`CHAPTER 2 Natural-Feature Tracking and Visual Search . . . . . . . . . . . . . . . . . . . . . . .13
`
`CHAPTER 3
`
`Introduction to AR Browsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
`
`CHAPTER 4
`
`Latitude, Longitude, and Where to get POIs . . . . . . . . . . . . . . . . . . . . . . . 53
`
`PART II WIKITUDE
`(cid:2)
`Building Worlds with KML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
`CHAPTER 5
`
`CHAPTER 6
`
`Building Worlds with ARML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
`
`PART III LAYAR
`(cid:2)
`Building Layar Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
`CHAPTER 7
`
`CHAPTER 8
`
`Creating Filters and 2D Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
`
`CHAPTER 9 Using Layar Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177
`
`(cid:2)
`PART IV JUNAIO
`CHAPTER 10 Creating junaio Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
`
`CHAPTER 11 Natural-Feature Tracking and Visual Search with junaio . . . . . . . . . . . 235
`
`(cid:2)
`PART V THE NEXT STEPS
`CHAPTER 12 Adding Advanced Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
`
`CHAPTER 13 Taking Your Application to Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
`
`CHAPTER 14 The Future of AR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
`
`Continues
`
`ffirs.indd i
`
`5/5/11 1:51:50 PM
`
`Niantic's Exhibit No. 1016
`Page 002
`
`

`

`APPENDIX A Wikitude Support and ARML Parameters . . . . . . . . . . . . . . . . . . . . . . . . 301
`
`APPENDIX B Layar Support and Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
`
`APPENDIX C Junaio Support and Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
`
`INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .319
`
`ffirs.indd ii
`
`5/5/11 1:51:52 PM
`
`Niantic's Exhibit No. 1016
`Page 003
`
`

`

`PROFESSIONAL
`Augmented Reality Browsers
`for Smartphones
`
`ffirs.indd iii
`
`5/5/11 1:51:52 PM
`
`Niantic's Exhibit No. 1016
`Page 004
`
`

`

`PROFESSIONAL
`Augmented Reality Browsers
`for Smartphones
`PROGRAMMING FOR JUNAIO, LAYAR, AND WIKITUDE
`
`Lester Madden
`
`ffirs.indd v
`
`5/5/11 1:51:52 PM
`
`Niantic's Exhibit No. 1016
`Page 005
`
`

`

`Professional Augmented Reality Browsers for Smartphones: Programming for junaio,
`Layar, and Wikitude
`
`This edition fi rst published 2011
`
`©2011 John Wiley & Sons,
`
`Ltd Registered offi ce
`
`John Wiley & Sons Ltd,
`The Atrium, Southern Gate,
`Chichester, West Sussex,
`PO19 8SQ, United Kingdom
`
`For details of our global editorial offi ces, for customer services and for information about how to apply for permission to
`reuse the copyright material in this book please see our web site at www.wiley.com.
`
`The right of the author to be identifi ed as the author of this work has been asserted in accordance with the Copyright,
`Designs and Patents Act 1988.
`
`All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form
`or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright,
`Designs and Patents Act 1988, without the prior permission of the publisher.
`
`Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in
`electronic books.
`
`Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product
`names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The
`publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accu-
`rate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher
`is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a
`competent professional should be sought.
`
`Wikitude content reproduced with permission of Wikitude
`
`metaio content reproduced with permission of metaio
`
`junaio content reproduced with permission of junaio
`
`Layar content reproduced with permission of Layar
`
`978-1-119-99281-3
`978-1-119-99286-8 (ebk)
`978-1-119-99287-5 (ebk)
`978-1-119-99479-4 (ebk)
`
`A catalogue record for this book is available from the British Library.
`
`ffirs.indd vi
`
`5/5/11 1:51:54 PM
`
`Niantic's Exhibit No. 1016
`Page 006
`
`

`

`CONTENTS
`
`INTRODUCTION
`
`PART I: INTRODUCTION
`
`CHAPTER 1: INTRODUCING AUGMENTED REALITY (AR)
`
`My Augmented Reality Journey
`What is AR?
`Why AR Is Useful?
`Summary
`
`CHAPTER 2: NATURAL-FEATURE TRACKING AND VISUAL SEARCH
`
`Introducing Natural-Feature Tracking
`How Natural-Feature Tracking Works
`Scenarios for Natural-Feature Tracking
`Introducing Visual Search
`Shopping
`Translating Languages
`Identifying Objects
`Marketing AR-Enabled Apps
`Summary
`
`CHAPTER 3: INTRODUCTION TO AR BROWSERS
`
`AR Browser Basics
`The Growth of AR Browsers
`Anatomy of a Browser
`Wikitude World Browser
`Overview
`Development Choices
`Wikitude Worlds
`Creating Your First World
`Layar Reality Browser
`Overview
`Development Choices
`Functionality
`Layers
`Creating Your First Layer
`
`xxi
`
`3
`
`3
`4
`10
`11
`
`13
`
`13
`14
`15
`16
`17
`17
`18
`19
`19
`
`21
`
`22
`23
`24
`25
`26
`27
`28
`34
`36
`36
`37
`38
`38
`42
`
`ftoc.indd xv
`
`5/4/11 4:26:42 PM
`
`Niantic's Exhibit No. 1016
`Page 007
`
`

`

`CONTENTS
`
`junaio
`Overview
`Development Choices
`Channels
`Testing a junaio Demo
`Browser Accuracy
`GPS and Compass Accuracy
`Mapping Accuracy
`Summary
`
`CHAPTER 4: LATITUDE, LONGITUDE AND WHERE TO GET POIS
`
`An Overview of Latitude/Longitude
`Working with Points of Interest (POIs)
`Working with POI Databases
`Summary
`
`PART II: WIKITUDE
`
`CHAPTER 5: BUILDING WORLDS WITH KML
`
`Using the Wikitude Dashboard
`Developing with KML
`Creating KML With Google Earth (For Non Developers)
`Creating a World
`Testing
`Simulating Locations
`Creating KML With Google Earth (For Developers)
`Testing
`Understanding KML’s Limitations
`Summary
`
`CHAPTER 6: BUILDING WORLDS WITH ARML
`
`Understanding Augmented Reality Markup Language (ARML)
`What’s New With ARML?
`Creating a World With ARML
`Adding the POIs
`Completing the World
`Creating the ARML World
`Testing on the Device
`Summary
`
`43
`44
`45
`45
`47
`47
`49
`50
`52
`
`53
`
`53
`57
`63
`65
`
`69
`
`70
`71
`72
`74
`78
`80
`81
`84
`84
`84
`
`85
`
`86
`86
`87
`92
`94
`98
`99
`100
`
`xvi
`
`ftoc.indd xvi
`
`5/4/11 4:26:43 PM
`
`Niantic's Exhibit No. 1016
`Page 008
`
`

`

`PART III: LAYAR
`
`CHAPTER 7: BUILDING LAYAR LAYERS
`
`Creating Your Layar Account
`Creating a Layer
`Creating the Layer on the Publishing Site
`Testing in the Client
`Preparing the Database
`Creating the Table
`Adding POIs to the Database
`Creating a Web Service
`Viewing the mylayar.php Code in Full
`Testing the Layer
`Customizing Your Layer
`Creating a More Compelling Listing
`Changing POI Colors
`Creating Icon Sets
`Adding Layar Actions
`Changing to Version 4.0
`Creating an Actions Table
`Fetching the Actions Function
`Adding Actions
`Adding Audio and Video
`Adding Triggers
`Summary
`
`CHAPTER 8: CREATING FILTERS AND 2D OBJECTS
`
`Using Filters
`Creating the Real Estate Database
`Creating the Filters
`Connecting the Filters
`Using SQL Queries
`Preparing the SQL
`Testing the Real Estate Layer
`Troubleshooting
`Finishing the Layer
`Experimenting with 2D Objects
`Changing Dimensions
`Summary
`
`CONTENTS
`
`103
`
`104
`105
`105
`107
`108
`109
`112
`114
`121
`125
`128
`128
`131
`132
`135
`135
`135
`137
`140
`142
`143
`144
`
`145
`
`147
`148
`153
`157
`158
`162
`163
`163
`164
`165
`173
`176
`
`ftoc.indd xvii
`
`xvii
`
`5/4/11 4:26:43 PM
`
`Niantic's Exhibit No. 1016
`Page 009
`
`

`

`CONTENTS
`
`CHAPTER 9: USING LAYAR TOOLS
`
`Launching Layers
`Using Layar Intent
`The Layar Shortcut Tool
`Requirements
`Using the Shortcut Tool
`Hoppala
`Using Hoppala Augmentation
`BuildAR
`Using BuildAR
`Skaloop
`Confi guring Skaloop
`Summary
`
`PART IV: JUNAIO
`
`CHAPTER 10: CREATING JUNAIO CHANNELS
`
`Understanding the Requirements
`Setting up the Apache Server
`Adding Your API Key
`Creating Your First Channel
`Creating the Client Listing
`Testing Your Server Confi guration
`Setting up the POI
`Creating Multiple POIs
`Including Optional Parameters
`Understanding the [name:string] Error
`Adding Images, Sound, and Video
`Adding Images
`Playing Sounds
`Playing Videos
`Creating 3D Content
`Debugging 3D
`Scaling 3D Content
`Using Animation
`Using OBJ fi les
`Creating 3D Content
`Importing MD2 and OBJ Files
`Summary
`
`177
`
`177
`178
`179
`180
`180
`182
`182
`184
`185
`185
`185
`186
`
`189
`
`190
`190
`192
`192
`192
`196
`199
`204
`206
`208
`210
`210
`211
`212
`214
`216
`220
`225
`229
`230
`231
`233
`
`xviii
`
`ftoc.indd xviii
`
`5/4/11 4:26:44 PM
`
`Niantic's Exhibit No. 1016
`Page 0010
`
`

`

`CHAPTER 11: NATURAL-FEATURE TRACKING AND VISUAL
`SEARCH WITH JUNAIO
`
`Natural-Feature Tracking for Non-Developers
`Creating Your First GLUE Channel
`Adding Images and Video
`Experimenting with Natural-Feature Tracking
`Gluing 3D Objects to an Image
`Natural-Feature Tracking for Developers
`Creating a Channel
`Building a Channel from Scratch
`Using Visual Search
`Overlaying Videos (Movie Textures)
`Encoding Movie Textures
`Image Requirements for Natural-Feature Tracking
`Summary
`
`PART V: THE NEXT STEPS
`
`CHAPTER 12: ADDING ADVANCED FUNCTIONALITY
`
`Working with Dedicated XML Files
`Creating Advanced Interactions
`Adding Interactions
`Using LLA Markers
`Confi guring an LLA
`Retrieving Data from a Database
`Summary
`
`CHAPTER 13: TAKING YOUR APPLICATION TO MARKET
`
`Marketing Your Content
`Listing Your Content
`Generating Excitement
`Making Money from AR
`Summary
`
`CHAPTER 14: THE FUTURE OF AR
`
`Using AR in Marketing
`Using AR for Translation Services
`Using AR for Interactive TV
`
`CONTENTS
`
`235
`
`236
`237
`241
`242
`242
`247
`248
`250
`253
`257
`259
`261
`263
`
`267
`
`267
`270
`270
`274
`274
`276
`277
`
`279
`
`280
`280
`283
`287
`290
`
`291
`
`291
`292
`293
`
`ftoc.indd xix
`
`xix
`
`5/4/11 4:26:44 PM
`
`Niantic's Exhibit No. 1016
`Page 0011
`
`

`

`CONTENTS
`
`Using AR in Diminishing Reality
`Using AR in Advertising
`Using AR in Books and Print
`Using AR in Gaming
`Using AR in Hardware
`Summary
`
`APPENDIX A: WIKITUDE SUPPORT AND ARML PARAMETERS
`
`Support
`ARML Parameters
`
`APPENDIX B: LAYAR SUPPORT AND PARAMETERS
`
`Support
`Request Parameters
`
`APPENDIX C: JUNAIO SUPPORT AND PARAMETERS
`
`Support Channels
`junaio Certifi cation Program
`junaio Parameters
`Troubleshooting Guide
`Failure of Validation Test 1 - Check Callback URL
`Failure of Validation Test 2 - Check pois/search
`Test 3 - Check pois/search return value error
`Failure of Validation Test 5 - Check pois/search
`Status Codes
`
`INDEX
`
`294
`295
`296
`296
`298
`299
`
`301
`
`301
`301
`
`305
`
`305
`305
`
`311
`
`311
`311
`312
`316
`316
`316
`317
`317
`318
`
`319
`
`xx
`
`ftoc.indd xx
`
`5/4/11 4:26:44 PM
`
`Niantic's Exhibit No. 1016
`Page 0012
`
`

`

`3I
`
`ntroduction to AR Browsers
`
`WHAT’S IN THIS CHAPTER?
`
`An overview of the clients you will be developing content for
`
`Examples of AR content created by other developers
`
`A fi rst look at the functionality provide by AR browsers
`
`Why browsers are not accurate
`
`➤ ➤ ➤ ➤
`
`For the course of this book we will focus on building content for the three leading AR browser
`platforms. Taking this approach presents us with some benefi ts and, of course, the inevitable
`drawbacks. Before you begin, you’ll need an iPhone or an Android phone and the latest
`versions of junaio, Layar, and Wikitude.
`
`Benefi ts of using the three leading AR browser platforms include:
`
`It’s easier to build content with little or no programming knowledge required. In many
`cases, you can access a database or provide XML to reference your content.
`
`Your content appears to an audience already interested in AR and actively seeking
`your content.
`
`It’s in the platform providers’ interests to actively promote their applications to users,
`so that means less marketing on your part.
`
`You can take advantage of richer features as the platform provider makes the func-
`tionality available.
`
`You don’t have to worry about camera input, screen layout, or GPS APIs.
`
`The same content will work on Android, iPhone, and Symbian/Windows Phone/bada
`OS/MeeGo when the clients are released.
`
`It’s becoming increasingly common for AR browsers to be preinstalled on mobile
`devices. This provides even more discoverability opportunities for your content.
`
`➤
`
`➤
`
`➤
`
`➤
`
`➤ ➤
`
`➤
`
`c03.indd 21
`
`5/4/11 3:57:49 PM
`
`Niantic's Exhibit No. 1016
`Page 0013
`
`

`

`22
`
` ❘
`
` CHAPTER 3 INTRODUCTION TO AR BROWSERS
`
`Drawbacks include:
`
`Users must launch another application before fi nding your content.
`
`Your content can lay undiscovered in the platform provider’s client.
`
`You don’t have full control over the user interface.
`
`➤ ➤ ➤
`
`Mobile AR is still very much in its infancy, so unless you have a specialist need and want to provide
`functionality beyond some of the benefi ts highlighted previously, your best option — for now,
`at least — is to build content for one of the leading platforms. These companies are investing
`millions of dollars into making their platforms a success; they have a marketing budget we can
`only dream of. It’s better to ride their wave than try to fi ght the tide ourselves.
`
`Another reason not to go it alone is the fact that on the Android and the iPhone, there are almost
`100 AR browsers all fi ghting for user attention. Many are simple; they offer nothing more than the
`ability to view POIs in the camera window. A handful offer developers the ability to build their own
`content, but only three are known all over the world and used by millions of users. With that in
`mind, we will focus on building content for Wikitude, Layar, and junaio.
`
`Each browser offers unique functionality. So before we get down and dirty, we’ll take a look at
`each browser in detail to learn more about each platform and what kind of content developers are
`building.
`
`AR BROWSER BASICS
`
`While I said in Chapter 1 that you need a camera and a screen to experience AR, it may surprise you
`to know that the minimum requirement for an AR browser is simply a smartphone that has a GPS.
`Why no camera? Well, it is less common, but some developers have taken the approach of building
`applications that provide an AR experience through augmenting your hearing rather than using a
`visual method. Toozla is an application from a Russian company that detects the users’ location via
`GPS. When users come into range of a POI, an audio fi le begins to play, giving users an overview
`of their current surroundings. If users remain at the POI to listen, the application provides more
`detailed commentary. Like most AR applications, Toozla uses the Wikipedia API to provide some
`of its content, but Toozla also uses professionally recorded material to give users a richer tour guide
`experience. We are not going to cover building content for Toozla, but they do have an API if you
`are interested in producing audio content. You will fi nd them at www.toozla.com
`
`AR browsers work by using GPS to detect the user’s current location while the compass detects the
`direction the device is facing. With the user’s current location and their direction known, the nearby
`POIs can be displayed in the camera window. As the user moves the mobile device around, the
`accelerometer detects the elevation while the compass continues to detects the direction the user is
`currently facing. This combination enables the application to use the map data to build the AR view.
`In Figure 3-1, you can see POIs from the Qype world for the Wikitude browser shown on a map,
`with the map indicating their actual location. In Figure 3-2 the same POIs are drawn in the AR
`browser window, providing a visual indication on their locations.
`
`c03.indd 22
`
`5/4/11 3:57:51 PM
`
`Niantic's Exhibit No. 1016
`Page 0014
`
`

`

`AR Browser Basics
`
`❘ 23
`
`©2011 Google, Map Data
`©2011 Tele Atlas
`
`FIGURE 3-1: POIs shown on a map
`
`FIGURE 3-2: POIs shown in the AR
`browser
`
`Many of today’s mobile devices are equipped with accelerometers and
`magnetometers, both of which are put to good use with providing an AR experience.
`The accelerometer is used to sense the orientation of a device (for example, when the
`screen view on the device is moved from landscape to portrait) as well as detect the
`acceleration or how quickly a user moves the device. The accelerometer is also used
`to detect the angle of the device, enabling an application to know if the camera is
`being pointed at the sky or at the ground. In addition, the magnetometer —
`or digital compass, if you prefer — can track changes in the Earth’s magnetic
`fi eld and determine with considerable accuracy which direction the user is facing
`and thereby determine North, South, East or West. Hidden Sky for the iPhone is
`an AR application that enables users to track the location of satellites, stars, and
`planets in the sky. The user simply points his iPhone at the sky; the application
`uses the accelerometer to determine the height the device is being pointed and the
`magnetometer determines the direction. In this example, GPS determines the user’s
`latitude and longitude position and then presents him with the relevant POIs.
`
`The Growth of AR Browsers
`Since the fi rst AR browser fi rst appeared in 2008, the genre has grown to around 300 applications
`available for the iPhone alone. At last count, an average of 20 new iPhone applications that
`exhibited AR browser behavior were being added to the iPhone App Store each month. Many of
`these applications combine geo-location and the camera with freely available APIs from Wikipedia,
`Twitter, Flickr, Google Search, or an untold number of other sources to enable users to visualize the
`
`c03.indd 23
`
`5/4/11 3:57:52 PM
`
`Niantic's Exhibit No. 1016
`Page 0015
`
`

`

`24
`
` ❘
`
` CHAPTER 3 INTRODUCTION TO AR BROWSERS
`
`locations of POIs around them. There are applications that, for example, help you see who
`is tweeting around you, help you fi nd your nearest train station, or help you fi nd a place to eat.
`
`Since this book is about building for the popular platforms rather than building applications from
`scratch, you can relax a little because a lot of the plumbing is taken care of. You won’t need to
`worry about APIs necessary to detect the user’s location, what design is best for building a user
`interface, or even how to fi gure the movement of the user’s devices. In most cases, you need only to
`supply the XML that contains the coordinates of the POI along with relevant icons and text. The
`diffi cult choice is determining which platform to build for.
`Anatomy of a Browser
`Before you delve into the various platforms, however, you should learn what all the platforms have
`in common so you have a better understanding of what is available to you. All platforms offer
`similar functionality but they all name their elements differently. So for the time being, we will
`refer to the components of the browser as follows (see Figure 3-3):
`
`Radar
`
`Info bubble
`
`Information bar
`
`Range
`
`Map
`
`App Store
`
`➤ ➤ ➤ ➤ ➤ ➤
`
`Range
`
`Radar
`
`Map
`
`Info bubble
`
`Information bar
`
`© 2009 Web Measure Pty Ltd
`FIGURE 3-3: Anatomy of a browser
`
`c03.indd 24
`
`5/4/11 3:58:03 PM
`
`Niantic's Exhibit No. 1016
`Page 0016
`
`

`

`Wikitude World Browser
`
`❘ 25
`
`Radar
`The radar provides a visual location of the direction where the POIs are located and essentially
`tells the user which way the device should be pointed.
`
`Info Bubbles
`POIs are displayed as info bubbles, which is an icon that visually indicates what it represents
`(for example, a knife and fork for food or a house that indicates a structure).
`
`Information Bar
`Once a user selects the info bubble, a short description about that POI is displayed in the
`information bar, offering the user more information about the POI active in the info bubble. If a
`user selects the text in the information bar, they are taken to a longer description about the POI.
`Depending on the browser, this can be a web page, a fi le, or text.
`
`Range
`The ability for the user to set the search range enables the application to limit the distance for the
`POIs that are returned. If you are in London looking for the nearest hotel, you probably won’t want
`to see hotels that are 50 miles away. The range feature enables you to control the maximum distance
`you want content returned, making the content more relevant to your location and giving you more
`control over what content is displayed in the camera view.
`
`Map
`While AR is great when it comes to helping you visualize the location of the POI, it is not always
`useful in helping you navigate. Fortunately, most browsers include a map so you can see the POIs in
`a traditional map view. Included with the map is often the ability to get turn-by-turn instructions to
`the destination.
`
`App Store
`As you learn more about the individual browsers, you’ll notice that developers have already been
`building an amazing amount of AR content. Naturally, to fi nd content, you need an application
`store as part of the browser. This is where your content will be listed and found by users.
`
`Now that you have a basic understanding of the features of each browser, it is time to meet the key
`players in more detail.
`
`WIKITUDE WORLD BROWSER
`
`Application name: Wikitude World Browser
`Platforms: Android, iPhone, Symbian. Bada
`Developer: Mobilizy
`Home page: www.wikitude.org/
`
`c03.indd 25
`
`5/4/11 3:58:04 PM
`
`Niantic's Exhibit No. 1016
`Page 0017
`
`

`

`26
`
` ❘
`
` CHAPTER 3 INTRODUCTION TO AR BROWSERS
`
`Originally appearing for Android devices in 2008, Wikitude has the honor of being one of the
`fi rst AR browsers released. Since those early days, Wikitude has gone on to appear on the iPhone,
`Symbian and more recently, Samsung Bada devices. Wikitude has also won many industry awards,
`including being voted the best Augmented Reality Browser in the Augmented Planet Readers Choice
`Award in 2009 and 2010 Wikitude is also the easiest of the
`browsers to build content for, enabling anyone regardless of
`their development experience to create content through a Google
`Maps interface without having to write a single line of code.
`
`Overview
`The Wikitude client has undergone a number of changes since
`it was fi rst created. The current version (see Figure 3-4) shows
`a list of Worlds that can be sorted by Featured (staff picks),
`alphabetically (A-Z), or by Distance from your current location.
`
`Worlds can be loaded individually or you can select multiple Worlds
`to load. This can be useful if you are looking for tourist attractions
`and are not sure which World the content lives inside or if you want
`the widest possible range of results. In addition, users can search
`for content with the search icon and perform a search across all the
`Worlds to return relevant content.
`
`Once a World has been selected, users can view the content in
`either the AR view, in a map, or in a list.
`
`FIGURE 3-4: The Wikitude client
`
`Wikitude Drive
`Mobilizy is also pushing the boundaries with AR and recently released the world’s fi rst AR
`navigation system for Android devices. Aptly named Wikitude Drive (see Figure 3-5), the application
`displays driving instructions over the live camera feed. Although it’s beyond the scope of this book,
`it does show the incredible investment that platform providers are placing in their products and the
`future direction that AR will take.
`
`FIGURE 3-5: Wikitude Drive (Android)
`
`c03.indd 26
`
`5/4/11 3:58:05 PM
`
`Niantic's Exhibit No. 1016
`Page 0018
`
`

`

`Wikitude World Browser
`
`❘ 27
`
`Development Choices
`In Wikitude speak, developers build Worlds. Once these worlds are submitted and approved, they
`appear directly inside the Wikitude client. Building Worlds for Wikitude is probably the simplest of
`the platforms we’ll cover in this book. Also, it offers the greatest variety in terms of development
`methods. Developers can create Worlds by using:
`
`The Google Maps interface
`
`Keyhole Markup Language (KML)
`
`Augmented Reality Markup Language (ARML)
`
`WebServices
`
`APIs
`
`➤ ➤ ➤ ➤ ➤
`
`Google Maps Interface
`Developers and non-developers alike can add simple POIs through a Google Maps interface.
`Content created this way appears in the Wikitude.me World. This is incredibly useful if you want to
`geo-tag just a few locations, such as your favorite shop, bar, or restaurant.
`
`Keyhole Markup Language (KML)
`Keyhole Markup Language (KML) is an XML-based schema that describes geographical
`information. It was originally created by Keyhole, Inc. (since acquired by Google) and is the format
`used by Google Earth to describe locations, coordinates, and so on. As a common standard, it
`provides Wikitude developers with access to a wide range of content without them needing to
`format content into a new structure.
`
`Augmented Reality Markup Language (ARML)
`Augmented Reality Markup Language (ARML) is a standard XML schema proposed by Mobilizy
`(the creators of Wikitude) as a standard format for building cross-platform or cross-device
`AR browsers. In theory, if the XML is in the same format, then the same fi le can be used by
`developers in other browsers without having to tweak it. ARML offers you richer functionality
`over KML it enables you to include features such as phone numbers, addresses, and other helpful
`information.
`
`WebServices
`Both the KML and ARML fi les which describe the POIs are hosted on the Wikitude server. From
`a developer’s perspective, you need only to create the XML fi le (either in KML or ARML format)
`and enter a simple form to submit it to the Wikitude client. This is why developing for Wikitude
`is so incredibly simple and attractive to many developers. If, however, you maintain a database of
`geo-coded data, you can chose to continue to host that data and provide a web services interface to
`enable Wikitude to access the data.
`
`c03.indd 27
`
`5/4/11 3:58:05 PM
`
`Niantic's Exhibit No. 1016
`Page 0019
`
`

`

`28
`
` ❘
`
` CHAPTER 3 INTRODUCTION TO AR BROWSERS
`
`APIs
`If you don’t like the thought of not having control over the UI, or if you want to have your own
`launch icon for your content, the Wikitude APIs can be embedded in your own applications, thereby
`giving you more control over your products. This is especially useful when you want to create an
`experience that is tightly integrated and offer richer functionality.
`
`API Example
`In general, I am not a huge fan of developing standalone AR applications, particularly if the
`application offers nothing more than the ability to view a limited number of POIs in an AR view.
`
`For example, developing an application that does nothing more than show users the location of the
`nearest airports is unlikely to give you very much reward on your investment. Building a World
`would be a much better option. If, however, you wanted to extend an existing application that
`provided users with fl ight arrival and departure information as well as locations of the airport
`shops, hotels, and car-rental agencies, then extending the application to support AR is a much better
`solution than trying to integrate that functionality into someone else’s product.
`
`An excellent example of this is the IBM Seer
`application shown in Figure 3-6, which created a
`compelling standalone experience for tennis fans at
`the Wimbledon Tennis Open. Users could view the
`scores of all the matches taking place across the
`various courts, monitor the wait time for the taxis,
`fi nd amenities, and even use AR to get live information
`about the length of the wait time at the food counter.
`
`The IBM Seer application was built using the
`Wikitude API and offered a much richer experience
`than an application displaying only POIs. Thus, it
`made more sense than trying to create a World.
`
`FIGURE 3-6: The IBM Seer application
`
`IBM®
`
`Wikitude Worlds
`For a product that has Wiki in its name, you might expect that it would include Wiki-based
`content — and you’d be correct. Initially, Wikitude was entirely based upon Wikipedia data and
`was primarily used as a city guide that enabled users to identify tourist attractions. Since Wikitude
`became a fully fl edged platform with an open API, more than 500 Worlds have been developed
`by developers all over the world, enabling users to fi nd AR content for everything from shopping
`venues to any YouTube videos that have been shot in the area.
`
`If you haven’t done so already, install Wikitude from either the Android Market or
`the Apple App Store and try out some of the numerous Worlds that are available.
`
`The list of Worlds available to you might differ from those available to me because Worlds are
`relative to your location. Wikitude does an excellent job of only displaying content that is within
`
`c03.indd 28
`
`5/4/11 3:58:06 PM
`
`Niantic's Exhibit No. 1016
`Page 0020
`
`

`

`Wikitude World Browser
`
`❘ 29
`
`a reasonable range of your phone so searching for pizza doesn’t produce results for pizza joints
`3,000 miles away. As you explore the client, be sure to:
`
`Test the search icon and search for nearby businesses.
`
`Click on POIs to explore the data in the info bubbles and the descriptions.
`
`Test the routing and calling functionality.
`
`➤ ➤ ➤
`
`This section covers some Worlds that you may want to try.
`
`Wikipedia
`The Wikipedia World is pretty much the catch-all World providing users with locations of train
`stations, rivers, hotels, tourist attractions, and more. Because the World takes its data from
`Wikipedia, it can produce a great deal of content for users to navigate. Nevertheless, it’s an
`excellent World for discovering your surroundings. Figure 3-7 shows Wikipedia data in an AR
`view; Figure 3-8 shows the same data in a list view. (Wikipedia® is a registered trademark of the
`Wikimedia Foundation, Inc., a non-profi t organization.)
`
`FIGURE 3-7: Wikipedia data in the
`AR view
`
`FIGURE 3-8: Wikipedia data in a
`list view
`
`YouTube
`YouTube features millions of videos uploaded by users from around the world. Have you ever
`wondered if any of that content is geo-tagged for your local area? I don’t live in a particularly
`exciting place, but I am always amazed at how much YouTube content has been fi lmed and uploaded
`from around my neighborhood. Without AR, I never would have discovered this content.
`Figure 3-9 shows YouTube videos in an AR view; Figure 3-10 shows YouTube videos that have
`been shot around my neighborhood.
`
`c03.indd 29
`
`5/4/11 3:58:12 PM
`
`Niantic's Exhibit No. 1016
`Page 0021
`
`

`

`30
`
` ❘
`
` CHAPTER 3 INTRODUCTION TO AR BROWSERS
`
`FIGURE 3-9: YouTube videos in
`the AR view
`
`© 2011 Google, Map Data
`© 2011 Tele Atlas
`
`FIGURE 3-10: YouTube videos
`fi lmed around my neighborhood
`
`Webcams.travel
`The Webcams.travel World contains publicly accessible webcams from around the world and is
`another excellent example of how AR leads to discoverability of interesting content. Figure 3-11
`shows nearby webcams and Figure 3-12 shows links I can access to watch live feeds. In 2010,
`Augmented Planet extended the Readers Choice Awards to cover third-party applications developed
`for Layar, junaio and Wikitude. Webcams.travel won the award for best third-party content.
`
`FIGURE 3-11: Webcams located
`near me
`
`FIGURE 3-12: Link to watch the
`live feed
`
`c03.indd 30
`
`5/4/11 3:58:12 PM
`
`Niantic's Exhibit No. 1016
`Page 0022
`
`

`

`Wikitude World Browser
`
`❘ 31
`
`Qype
`If you are looking for a reliable business or a good place to eat, the Qype World is a social
`networking site that contains reviews for thousands of local business around the world. Using this
`World, you can read real opinions of services near your current location. Figure 3-13 shows Qype
`reviews in an AR view. Figure 3-14 shows reviews of nearby businesses.
`
`FIGURE 3-13: Qype reviews in the
`AR view
`
`FIGURE 3-14: Nearby business
`reviews
`
`Foursquare Venues
`Foursquare Venues (see Figure 3-15) is a social city guide with around eight million users
`worldwide. Users can fi nd local venues and see where their friends are. Active users can earn
`discounts from their favorite places. Figure 3-15 shows several Foursquare venues near my home,
`including