IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`
`)
`)
`)
`)
`)
`
`
`
`
`
`
`
`
`
`
`
`In re Inter Partes Review of:
`U.S. Patent No. 9,445,251
`Issued: September 13, 2016
`Application No.: 14/633,804
`
`
`For: Method to Provide Ad Hoc and Password Protected Digital and Voice
`Networks
`
`FILED VIA E2E
`
`
`
`DECLARATION OF DR. BENJAMIN B. BEDERSON IN SUPPORT OF
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 9,445,251
`
`
`
`
`
`
`Apple Inc.
`Exhibit 1002
`Page 001
`
`

`

`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`I.
`II.
`
`V.
`
`TABLE OF CONTENTS
`INTRODUCTION AND QUALIFICATIONS ................................. 4
`SUMMARY OF MATERIALS REVIEWED AND
`CONSIDERED ................................................................................ 12
`III. LEVEL OF SKILL IN THE ART AND PERSPECTIVE APPLIED
`IN THIS DECLARATION .............................................................. 14
`IV. THE ’251 PATENT ......................................................................... 16
` An Explanation of Georeferencing ........................................ 16
`Brief Description.................................................................... 23
`
`Prosecution History Of The ʼ251 Patent ................................ 25
`
`CLAIM CONSTRUCTION ............................................................. 30
`“georeferenced map” ............................................................. 31
`VI. UNDERSTANDING OF RELEVANT LEGAL PRINCIPLES ..... 35
`VII. THE PRIOR ART AND BACKGROUND EVIDENCE ................ 37
`Prior Art Relied Upon For Obviousness Combinations ........ 37
`’724 Patent (Ex. 1008) ................................................. 38
`
`VIII. THE ’724 PATENT IS NOT INCORPORATED INTO THE ’410
`APPLICATION ................................................................................ 38
`IX. CERTAIN FEATURES OF THE ’251 PATENT CLAIMS ARE
`NOT DISCLOSED BY THE ’410 APPLICATION, NOR THE ’728
`PATENT .......................................................................................... 39
`Requesting and Receiving Second Georeferenced Map from a
`Server ..................................................................................... 41
`Groups .................................................................................... 51
`The ’410 application does not describe receiving a message
`from a second device and, based on that message,
`participating in the group ....................................................... 54
`The ’410 application does not describe “anonymous
`communications” ................................................................... 57
`
`
`
`
`
`
`
`
`
`
`
`
`i
`
`Apple Inc.
`Exhibit 1002
`Page 002
`
`

`

`
`X.
`
`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`ii
`
`
`
`
`THE DISCLOSURES OF THE ’724 PATENT RENDER
`OBVIOUS ALL CLAIMS OF THE ’251 PATENT ....................... 65
`A.
`Independent claims 1 and 54 ................................................. 66
` Motivation to combine ................................................ 67
`Preambles and initial clause ........................................ 71
`
`1 and 24 [a] joining a group’s network ....................... 72
`
`1 and 24 [b] based on the message, participating in the
`
`group by exchanging location information via a first
`server ........................................................................... 73
`1 and 24 [c] presenting a georeferenced map and
`symbols ........................................................................ 77
`1 and 24 [d], [e] requesting and receiving second
`georeferenced map from a second server .................... 81
`1 and 24 [f] presenting a second georeferenced map and
`a second set of symbols ............................................... 84
`1 and 24 [g] selecting symbols and sending data ........ 91
`1 and 24 [h]: first device does not have access to IP
`addresses of second devices ........................................ 97
`Dependent Claims .................................................................. 99
`Claims 2 and 25: send SMS, text, image, or video ..... 99
`
`Claims 3, 9, 26, and 32: first device is a PDA, PC,
`
`smartphone ................................................................ 101
`Claims 4 and 27: second georeferenced map includes
`satellite image or aerial photograph .......................... 103
`Claims 5 and 28: update location based on time or
`distance travelled ....................................................... 105
`Claims 6 and 29: initiating phone call by selecting
`symbol ....................................................................... 107
`Claims 7 and 30: 251 claim 7 and 30: sending an SMS
`message ...................................................................... 109
`Claims 8 and 31: exchanging status information
`(battery level, etc.) ..................................................... 111
`Claims 10 and 33: Group identifier ........................... 114
`Claims 11 and 34: send voice recording ................... 115
` Claims 12 and 35: sending GPS location using Internet
`Protocol ...................................................................... 117
` Claim 13: identify user-selected symbol based on
`spatial coordinates corresponding to selected location
`on display and map .................................................... 119
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Apple Inc.
`Exhibit 1002
`Page 003
`
`

`

`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
` Claim 14: use database to identify symbol by
`location ...................................................................... 122
` Claims 15 and 16: adding a new entity with the user-
`specified location and symbol ................................... 125
` Claim 17: new entity information ............................. 129
` Claim 18: add new entity’s spatial coordinates to
`database ..................................................................... 132
` Claim 19: determine new entity’s spatial coordinates
`based on selected location on displayed map ............ 134
` Claims 20 and 21: entity database on device, server 137
` Claim 22: spatial coordinates include latitude and
`longitude .................................................................... 139
` Claim 23: initiating a VoIP or data call by selecting
`symbol ....................................................................... 141
`CONCLUSION ........................................................................................ 144
`
`
`iii
`
`Apple Inc.
`Exhibit 1002
`Page 004
`
`

`

`
`I.
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`INTRODUCTION AND QUALIFICATIONS
`
`I have been retained by Apple Inc. (“Petitioner”) to provide my
`
`opinion concerning the validity of U.S. Patent No. 9,445,251 (attached to the
`
`accompanying Petition as Ex. 1001 and henceforth referred to as “the ’251 patent”)
`
`in support of this Petition for Inter Partes Review of U.S. Patent No. 9,445,251.
`
`
`
`I have not previously been retained by Petitioner. I am simultaneously
`
`engaged by Petitioner to provide declarations for IPRs challenging U.S. Patent
`
`Nos. 8,213,970; 9,408,055; and 9,467,838.
`
`
`
`I received a Bachelor of Science degree in Computer Science with a
`
`minor in Electrical Engineering from Rensselaer Polytechnic Institute (“RPI”) in
`
`1986. I received a Master of Science degree and a Ph.D. in Computer Science from
`
`New York University (“NYU”) in 1989 and 1992, respectively.
`
`
`
`Since 1998, I have been a Professor of Computer Science at the
`
`University of Maryland (“UMD”), where I have joint appointments at the Institute
`
`for Advanced Computer Studies and the College of Information Studies
`
`(Maryland’s “iSchool”). I am also Associate Provost of Learning Initiatives and
`
`Executive Director of the Teaching and Learning Transformation Center. I am a
`
`member and previous director of the Human-Computer Interaction Lab (“HCIL”),
`
`the oldest and one of the best known Human-Computer Interaction research groups
`
`in the country. I was also co-founder and Chief Scientist of Zumobi, Inc. from 2006
`
`4
`
`
`Apple Inc.
`Exhibit 1002
`Page 005
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`to 2014, a Seattle-based startup that is a publisher of content applications and
`
`advertising platforms for smartphones. I am also co-founder and co-director of the
`
`International Children’s Digital Library (“ICDL”), a web site launched in 2002 that
`
`provides the world’s largest collection of freely available online children’s books
`
`from around the world with an interface aimed to make it easy for children and
`
`adults to search and read children’s books online. I am also co-founder and Chief
`
`Technology Officer of Hazel Analytics, a data analytics company whose product
`
`sends alerts in warranted circumstances. In addition, I have for more than 15 years
`
`consulted for numerous companies in the area of user interfaces, including
`
`Microsoft, the Palo Alto Research Center, Sony, Lockheed Martin, and NASA
`
`Goddard Space Flight Center.
`
`
`
`For more than 30 years, I have studied, designed, and worked in the
`
`field of computer science and human-computer interaction. My experience
`
`includes 30 years of teaching and research, with research interests in human-
`
`computer interaction and the software and technology underlying today’s
`
`interactive computing systems. This includes the design and implementation of
`
`software applications on mobile devices, including smart phones and PDAs, such
`
`as my work on DateLens, LaunchTile, and StoryKit described below.
`
`
`
`At UMD, my research is in the area of Human-Computer Interaction
`
`(“HCI”), a field that relates to the development and understanding of computing
`
`5
`
`
`Apple Inc.
`Exhibit 1002
`Page 006
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`systems to serve users’ needs. Researchers in this field are focused on making
`
`universally usable, useful, efficient, and appealing systems to support people in
`
`their wide range of activities. My approach is to balance the development of
`
`innovative technology that serves people’s practical needs. Example systems
`
`following this approach that I have built include PhotoMesa (software for end users
`
`to browse personal photos), DateLens (2002 software for end users to use their
`
`mobile devices to efficiently access their calendar information), LaunchTile (2005
`
`“home screen” software for mobile devices to allow users to navigate apps in a
`
`zoomable environment), SpaceTree (2001 software for end users to efficiently
`
`browse very large hierarchies), ICDL (as described above), and StoryKit (a 2009
`
`iPhone app for children to create stories).
`
`
`
`LaunchTile led to my creation of Zumobi in 2006, where I was
`
`responsible for investigating new software platforms and developing new user
`
`interface designs that provide efficient and engaging interfaces to permit end users
`
`to access a wide range of content on mobile platforms (including the iPhone and
`
`Android-based devices). For example, I designed and implemented software called
`
`“Ziibii,” a “river” of news for iPhone, software called “ZoomCanvas,” a zoomable
`
`user interface for several iPhone apps, and iPhone apps including “Inside Xbox”
`
`for Microsoft and Snow Report for REI. At the International Children’s Digital
`
`Library (ICDL), I have since 2002 been the technical director responsible for the
`
`6
`
`
`Apple Inc.
`Exhibit 1002
`Page 007
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`design and implementation of the web site, www.childrenslibrary.org (originally
`
`at www.icdlbooks.org). In particular, I have been closely involved in designing the
`
`user interface as well as the software architecture for the web site since its inception
`
`in 2002.
`
`
`
`Beginning in the mid-1990s, I have been responsible for the design
`
`and implementation of numerous other web sites in addition to the ICDL. For
`
`example, I designed and built my own professional web site when I was an
`
`Assistant Professor of Computer Science at the University of New Mexico in 1995
`
`and have continued to design, write the code for, and update both that site (which
`
`I moved
`
`to
`
`the University of Maryland
`
`in 1998,
`
`currently
`
`at
`
`http://www.cs.umd.edu/~bederson/) as well as numerous project web sites, such as
`
`Pad++, http://www.cs.umd.edu/hcil/pad++/. I received the Janet Fabri
`
`Memorial Award for Outstanding Doctoral Dissertation for my Ph.D. work in
`
`robotics and computer vision. I have combined my hardware and software skills
`
`throughout my career in Human-Computer Interaction research, building various
`
`interactive electrical and mechanical systems that couple with software to provide
`
`an innovative user experience.
`
` My work has been published extensively in more than 140 technical
`
`publications, and I have given about 100 invited talks, including 9 keynote lectures.
`
`I have won a number of awards including the Brian Shackel Award for
`
`7
`
`
`Apple Inc.
`Exhibit 1002
`Page 008
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`“outstanding contribution with international impact in the field of HCI” in 2007,
`
`and the Social Impact Award in 2010 from Association for Computing
`
`Machinery’s (“ACM”) Special Interest Group on Computer Human Interaction
`
`(“SIGCHI”). ACM is the primary international professional community of
`
`computer scientists, and SIGCHI is the primary international professional HCI
`
`community. I have been honored by both professional organizations. I am an
`
`“ACM Distinguished Scientist,” which “recognizes those ACM members with at
`
`least 15 years of professional experience and 5 years of continuous Professional
`
`Membership who have achieved significant accomplishments or have made a
`
`significant impact on the computing field.” I am a member of the “CHI Academy,”
`
`which is described as follows: “The CHI Academy is an honorary group of
`
`individuals who have made substantial contributions to the field of human-
`
`computer interaction. These are the principal leaders of the field, whose efforts
`
`have shaped the disciplines and/or industry, and led the research and/or innovation
`
`in human-computer interaction.” The criteria for election to the CHI Academy are:
`
`(1) cumulative contributions to the field; (2) impact on the field through
`
`development of new research directions and/or innovations; and (3) influence on
`
`the work of others.
`
`
`
`I have appeared on radio shows numerous times to discuss issues
`
`relating to user interface design and people’s use and frustration with common
`
`8
`
`
`Apple Inc.
`Exhibit 1002
`Page 009
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`technologies, web sites, and mobile devices. My work has been discussed and I
`
`have been quoted by mainstream media around the world over 120 times, including
`
`by the New York Times, the Wall Street Journal, the Washington Post, Newsweek,
`
`the Seattle Post-Intelligencer, the Independent, Le Monde, NPR’s All Things
`
`Considered, New Scientist Magazine, and MIT’s Technology Review.
`
`
`
`I have designed, programmed, and publicly deployed dozens of user-
`
`facing software products that have cumulatively had millions of users. My work is
`
`cited in several patents, including U.S. Patent Nos. 6,307,562; 6,608,549;
`
`7,576,756; and 7,834,849.
`
`
`
`I am the co-inventor of 10 U.S. patents and 15 U.S. patent
`
`applications. The patents are generally directed to user interfaces/experience with
`
`some directed to mobile devices, including U.S. Patent No. 9,778,810 (2017),
`
`entitled Techniques to modify content and view content on mobile devices.
`
` My curriculum vitae, which includes a more detailed summary of
`
`my background, experience, and publications, is attached to the accompanying
`
`Petition as Ex. 1003.
`
`
`
`I am being compensated at my standard consulting rate of $600 per
`
`hour for my services. My compensation does not depend on the outcome of this
`
`review proceeding or of any pending litigation.
`
`9
`
`
`Apple Inc.
`Exhibit 1002
`Page 010
`
`

`

`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
` As I have already touched on above, I was heavily involved in the
`
`research and development of mobile device applications during the time period that
`
`encompassed (before and after) the September 21, 2004 claimed priority date of
`
`the ’251 patent, and also through to the actual October 31, 2014 filing date of the
`
`parent ’838 patent. For example, I envisioned, designed and built DateLens in
`
`2002-2004 to create a richer and more usable calendar for the Microsoft PocketPC
`
`platform as well as desktops. One of its innovations was its design that enabled it
`
`to scale between small (mobile) and large (desktop) computers. I started DateLens
`
`as a research project and eventually sold it commercially. DateLens displayed
`
`calendar data that came straight from the Microsoft PocketOutlook database and
`
`allowed modification and viewing of appointments. The research website from that
`
`time
`
`period
`
`that
`
`describes DateLens
`
`is
`
`still
`
`available
`
`at
`
`http://www.cs.umd.edu/hcil/datelens/ which shows screenshots and a picture of me
`
`demonstrating DateLens to Bill Gates. The commercial website from that time
`
`period is also available at http://www.windsorinterfaces.com/datelens.shtml which
`
`shows a number of news articles reviewing DateLens at that time.
`
`
`
`In 2007 and 2008, I wrote an app for Apple iOS called “ICDL on
`
`iPhone” which provided a zoomable interface for reading children’s picture books
`
`from my International Children’s Digital Library on iPhones. At the time, this was
`
`10
`
`
`Apple Inc.
`Exhibit 1002
`Page 011
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`the only effort of its kind to provide the ability read children’s picture books on
`
`phones.
`
` That led to my design and development of StoryKit starting in 2009
`
`and still in use today. StoryKit is an iOS app that supports children to write their
`
`own stories, including text, images, drawings and sound. It has become quite
`
`popular among school teachers and gets about 10,000 hours a month of use of
`
`children writing stories.
`
`
`
`I have been asked my technical opinions regarding the understanding
`
`of a person of ordinary skill in the art as it relates to the ’251 patent and other
`
`reference documents.
`
`
`
`I have also been asked to provide my technical opinions on concepts
`
`discussed in the ’251 patent and other reference documents, as well as my technical
`
`opinions on how these concepts relate to several claim limitations of the ’251
`
`patent in the context of the specification.
`
`
`
`In reaching the opinions, I have considered the ’251 patent, its
`
`prosecution history, and the references below, and have also drawn as appropriate
`
`upon my own education, training, research, knowledge, and personal and
`
`professional experience.
`
`11
`
`
`Apple Inc.
`Exhibit 1002
`Page 012
`
`

`

`
`II.
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`SUMMARY OF MATERIALS REVIEWED AND CONSIDERED
` All of the opinions contained in this Declaration are based on the
`
`documents I reviewed and my knowledge and professional judgment. In forming
`
`the opinions expressed in this Declaration, while drawing on my experience in the
`
`field of software applications for mobile devices, I reviewed the following
`
`documents:
`
`Ex. No.
`
`Description
`
`1001
`1003
`1004
`
`1005
`
`1006
`
`1007
`
`1008
`
`1009
`
`1010
`
`1011
`
`1012
`
`1017
`
`U.S. Patent No. 9,445,251 (the “’251 patent”)
`Curriculum Vitae of Dr. Benjamin Bederson
`File History for U.S. Patent No. 9,467,838 (the “’838 FH”)
`
`File History for U.S. Patent No. 9,445, 251 (the “’251 FH”)
`
`U.S. Patent Application No. 14/027,410 (the “’410 application”)
`
`U.S. Patent No. 7,031,728 (the “’728 patent”)
`
`U.S. Patent No. 7,630,724 (the “’724 patent”)
`
`U.S. Patent No. 8,126,441 (the “’441 patent”)
`
`Computer-generated document comparison showing differences in
`U.S. Patent Application No. 10/711,490 and U.S. Patent Application
`No. 11/308,648 (“’724 to ’728 Comparison”)
`Computer-generated document comparison showing differences in
`U.S. Patent Application No. 11/308,648 and U.S. Patent Application
`No. 11/615,472 (“’441 to ’724 Comparison”)
`Computer-generated document comparison showing differences in
`U.S. Patent Application No. 11/615,472 and U.S. Patent Application
`No. 12/761,533 (“’129 to ’441 Comparison”)
`Computer-generated document comparison showing differences in
`U.S. Patent Application No. 14/027,410 and U.S. Patent Application
`No. 11/308,648 (“’410 to 724 Comparison”)
`
`12
`
`
`Apple Inc.
`Exhibit 1002
`Page 013
`
`

`

`
`
`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`Ex. No.
`
`Description
`
`1018
`
`1019
`
`1020
`
`1021
`
`1022
`
`1023
`
`1024
`
`GeoTIFF Format Specification, GeoTIFF Rev. 1.0, Specification
`version 1.8.1, October 31, 1995 (GeoTIFF Specification”)
`Hornbaek and Bederson, “Navigation Patterns and Usability of
`Zoomable User Interfaces with and without and Overview,” ACM
`Transaction on Computer-Human Interaction, Vol. 9, No. 4,
`December 2002, pages 362-389.
`MapInfo, “Spatially Enhancing Business Data with Geocoding
`Solutions, A MapInfo White Paper (1997) (“MapInfo White Paper”)
`MapInfo Professional User’s Guide Version 7.0 (“MapInfo User
`Guide”)
`Python Documentation 2.0 Homepage (Oct. 16, 2000), available at
`https://docs.python.org/release/2.0/
`Python Documentation 2.0, Section 7.2 Socket, available at
`https://docs.python.org/release/2.0/lib/module-socket.html
`Internet Engineering Task Force RFC 1034, Domain Names –
`Concepts and Facilities (November 1987), available at
`https://tools.ietf.org/pdf/rfc1034.
`
` My opinions are additionally guided by my appreciation of how a
`
`person of ordinary skill in the art would have understood the claims of the ’251
`
`patent at the time of the alleged invention, which I have been asked to assume is
`
`the 2013-14 timeframe. For the purposes of determining whether there is written
`
`description support in the ’410 application, I understand that the relevant date is its
`
`actual filing date, September 16, 2013. For purposes of whether the teachings of
`
`prior art render obvious the ’251 patent’s claims, I understand that the relevant date
`
`is that patent’s actual filing date, October 31, 2014, once the “priority chain” is
`
`broken. Although my analysis takes into account these two separate dates
`
`depending on the context, I did not notice any relevant differences. Accordingly, I
`
`13
`
`
`Apple Inc.
`Exhibit 1002
`Page 014
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`will use the term “date of invention” or “relevant timeframe” interchangeably with
`
`this understanding.
`
`III. LEVEL OF SKILL IN THE ART AND PERSPECTIVE APPLIED IN
`THIS DECLARATION
`
`I understand that certain issues relating to validity must be judged
`
`from the perspective of a person of ordinary skill in the relevant art, as I discuss
`
`below.
`
` The invention of the ’251 patent relates to a method and system that
`
`provides individuals the ability to set up and utilize an ad hoc digital and voice
`
`network for location sharing and communications among mobile devices, such as
`
`smartphones and PDAs.
`
`
`
`In determining the level of ordinary skill, I have been asked to
`
`consider, for example, the types of problems encountered in the field, prior
`
`solutions to those problems, the rapidity with which innovations are made, the
`
`sophistication of the technology, and the educational level of active workers in the
`
`field. Taking those factors into consideration, a person of ordinary skill in the art
`
`at the relevant time would have had at least a bachelor’s degree in a suitable field
`
`(e.g., computer science, or electrical or computer engineering), or equivalent
`
`proficiency. In addition, it would have taken 2-3 years of work on software
`
`applications for mobile devices, such as smartphones and PDAs, for a person to
`
`become familiar with the problems encountered in the field and the prior and
`14
`
`
`Apple Inc.
`Exhibit 1002
`Page 015
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`current solutions to those problems. Through education and relevant work
`
`experience, the person of ordinary skill would have an understanding and
`
`familiarity with the function and development of software applications for mobile
`
`devices, such as smartphones and PDAs.
`
`
`
`I would have qualified as a person of at least ordinary skill in the art
`
`as of the relevant timeframe, and also as of the earliest claimed September 21, 2004
`
`priority date on the face of the ’251 patent. I have a sufficient level of knowledge,
`
`experience, and education to provide an expert opinion in the field of the ’251
`
`patent.
`
` While my own level of skill level exceeded that of the ordinary level
`
`of skill in the art at the relevant timeframe, as I had earned my Ph.D. in computer
`
`science and had been working on the research and development of human-
`
`computer interaction for over two decades by the 2013 timeframe, I am well-
`
`acquainted with the actual performance and capabilities of a person of ordinary
`
`skill in the art as defined above. This is because, during the relevant timeframe, I
`
`was
`
`teaching and working with graduate students, and professional
`
`programmers—so I was quite familiar with the skills that people with a range of
`
`experience had at the time.
`
` My opinions in this Declaration are based on the perspective of a
`
`person of ordinary skill in the art as of the relevant timeframes discussed above, or
`
`15
`
`
`Apple Inc.
`Exhibit 1002
`Page 016
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`somewhat before (approximately 2013-14). This is true even if the testimony is
`
`stated in the present tense. Each of the statements below reflects my opinion based
`
`on my review of the disclosures of the ’251, ’724, and ’728 patents, the file history
`
`of the ’251 patent, the ’410 application, and the petitioned claims.
`
`IV. THE ’251 PATENT
` An Explanation of Georeferencing
` Before I discuss the patent, I will discuss a key concept called
`
`georeferencing.
`
` Using maps to describe the world around us is centuries old, and
`
`plotting data on those maps drew attention in 1855 when John Snow used such a
`
`map to identify the location of a cholera outbreak in London. So as computer
`
`capabilities advanced to include graphical displays and databases, it is not
`
`surprising that computer-based mapping systems were developed – coming to be
`
`called Geographical Information Systems (GIS), starting in the 1960s. By the mid
`
`1980’s, broadly available commercial GIS software was available with one early
`
`system called MapInfo founded in 1986 (by some friends of mine in college).
`
` The essential element of GIS systems that gave them their power was
`
`the fact that the maps themselves were associated with spatial locations on planet
`
`earth – typically using longitude and latitude coordinates to identify those
`
`locations. Furthermore, databases of objects (such as buildings) and regions (such
`
`16
`
`
`Apple Inc.
`Exhibit 1002
`Page 017
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`as city or state boundaries) could be created that were also associated with spatial
`
`locations on earth, and displayed directly on those maps. A 1997 white paper1 from
`
`MapInfo, titled “Spatially Enhancing Business Data with Geocoding Solutions”
`
`(Ex. 1020) included a figure showing such a table of data points along with their
`
`display on a map:
`
`
`
`
`1 1997 MapInfo white paper on “Spatially Enhancing Business Data with Geocoding
`Solutions”.
`https://web.archive.org/web/19970112202655/http://www.mapinfo.com:80/software/library/ge
`ocoding_wp.pdf
`
`17
`
`
`Apple Inc.
`Exhibit 1002
`Page 018
`
`

`

`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
` As the MapInfo white paper explains, geocoding is “the process of
`
`attaching longitude and latitude coordinates to data; spatial enhancement.”
`
`MapInfo at 14 (glossary).
`
` Without geocoding (i.e., georeferencing, see my discussion below),
`
`there is no way to associate locations on the map to actual locations in the real
`
`world.
`
` That is, the mere display of a map or even symbols at specific
`
`locations on a map does not imply that the map or symbols are associated with
`
`spatial locations on planet earth. For example, it is and would have been trivial in
`
`2004 to create an image of a map with symbols on it with no such correspondence.
`
`I did so myself when I performed a study examining the effectiveness of overview
`
`windows for navigating large visual displays such as maps. I described this study
`
`in a 2002 paper2 (Ex. 1019) that displayed symbols in a map with a small
`
`navigation window that did not have any correspondence to spatial locations in the
`
`real world. That is, while the map did represent an actual part of the world, the map
`
`in my software did not include latitude and longitude or any other means to
`
`associate the visual elements of the map with planet earth:
`
`
`2 Hornbæk, K., Bederson, B. B., & Plaisant, C. (2002) Navigation Patterns and Usability of
`Zoomable User Interfaces With and Without an Overview, ACM Transactions on Computer-
`Human Interaction, 9 (4), 362-389. https://dl.acm.org/citation.cfm?doid=586081.586086
`18
`
`
`Apple Inc.
`Exhibit 1002
`Page 019
`
`

`

`
`
`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
` While the process for generating and storing maps and data that
`
`correspond to the real world was well understood since GIS systems were first
`
`created, the nomenclature describing these technologies varied. For example,
`
`MapInfo used the term “georeferenced raster images” to describe raster image data
`
`“containing control points and a projection (e.g., GeoTIFF file)” in their 2002
`
`product user guide (MapInfo User Guide, Ex. 1021, at 386). MapInfo used the term
`
`“geocoding” to “assign X and Y coordinates to each record” (MapInfo user guide
`
`at 134), which were in turn associated with spatial coordinates on the earth. The
`19
`
`
`Apple Inc.
`Exhibit 1002
`Page 020
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`guide explains, “Before you can view your data on a map, you must first assign
`
`geographic coordinates to each record so MI Pro knows where to spot the record
`
`on the map.” (MapInfo User Guide at 133). This document went on to describe
`
`several ways that their software could geocode data including looking up street
`
`addresses for pinpoint accuracy or zip code for general locations.
`
` The referenced GeoTIFF file format (See Ex. 1018, the GeoTIFF
`
`specification) uses these terms slightly differently – where “georeferencing” an
`
`image is defined as one where “… the location of its pixels in some model space
`
`is defined, but the transformation tying model space to the earth is not known.”
`
`(GeoTIFF specification at 100) and “geocoding” an image is defined as one where
`
`“… a precise algorithm for determining the earth-location of each point in the
`
`image is defined.” (GeoTIFF specification at 99).
`
` While the GeoTIFF definition of georeferencing is slightly broader
`
`than MapInfo’s, it would be clear to a POSITA in 2004 that “georeferencing” refers
`
`to the general process of associating objects or maps to real spatial locations on
`
`planet earth.
`
` There were multiple ways to associate maps and objects with spatial
`
`locations on earth. The above-referenced approach described looking up addresses
`
`and zip codes from databases. However, in order to georeference a map, having a
`
`single point is not sufficient as that would not specify the scale or orientation of
`
`20
`
`
`Apple Inc.
`Exhibit 1002
`Page 021
`
`

`

`Bederson Declaration for Inter Partes Review of USP 9,445,251
`
`
`the map. As described at length in the GeoTIFF specification (e.g., pp. 26-27),
`
`associating three points from an image with a spatial location such as longitude
`
`and latitude is necessary. In practice, additional information may be needed as well
`
`for more precision mapping.
`
` An example of applying this technique to an aerial photo such as a
`
`satellite image is described in the GeoTIFF specification (p. 50). A key point of
`
`this example is to note that in the “ModelTiepointTag” description, three points on
`
`the image are specified and each is associated with a latitude and longitude.
`
` Finally, the GeoTIFF speci