IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`Christopher J. Desmond, et al.
`In re Patent of:
`U.S. Patent No.:
`10,621,228
` Attorney Docket No.: 39843-0117IP1
`April 14, 2020
`Issue Date:
`Appl. Serial No.: 16/578,238
`Filing Date:
`September 20, 2019
`Title:
`METHOD AND APPARATUS FOR MANAGING DIGITAL
`FILES
`
`DECLARATION OF DR. PHILIP GREENSPUN
`
`Declaration
`
`I declare that all statements made herein on my own knowledge are true and
`
`that all statements made on information and belief are believed to be true, and
`
`further, that these statements were made with the knowledge that willful false
`
`statements and the like so made are punishable under Section 1001 of Title 18 of
`
`the United States Code.
`
`By:
`
`Philip Greenspun, Ph.D.
`
`December 3, 2021 in Jupiter, Florida
`
`1
`
`SAMSUNG 1003
`
`

`

`
`
`Contents
`
`I. QUALIFICATIONS AND BACKGROUND INFORMATION ........................ 4
`II. OVERVIEW OF CONCLUSIONS FORMED ................................................... 9
`III. LEVEL OF ORDINARY SKILL IN THE ART ............................................... 12
`IV. LEGAL STANDARDS ..................................................................................... 13
`A. Terminology ................................................................................................... 13
`B. Legal Standards for Anticipation.................................................................... 13
`C. Legal Standards for Obviousness ................................................................... 14
`V. The ’228 Patent .................................................................................................. 19
`A. Overview of the ’228 Patent ........................................................................... 19
`B. File History of the ’228 Patent ....................................................................... 23
`VI. TECHNOLOGY BACKGROUND ................................................................... 24
`VII. SUMMARY OF THE PRIOR ART ................................................................. 36
`A. Overview of Okamura .................................................................................... 36
`B. Overview of Belitz.......................................................................................... 38
`C. Combination of Okamura and Belitz .............................................................. 40
`VIII. GROUND 1 – Claims 1-19 are rendered obvious by Okamura in view of
`Belitz ........................................................................................................................ 54
`A. Claim 1 ........................................................................................................... 54
`B. Claim 2 ........................................................................................................... 81
`C. Claim 3 ........................................................................................................... 84
`D. Claim 4 ........................................................................................................... 84
`E. Claim 5 ........................................................................................................... 85
`F. Claim 6 ........................................................................................................... 86
`G. Claim 7 ........................................................................................................... 86
`H. Claim 8 ........................................................................................................... 86
`I. Claim 9 ........................................................................................................... 88
`J. Claim 10 ......................................................................................................... 89
`K. Claim 11 ......................................................................................................... 92
`
`2
`
`

`

`L. Claim 12 ......................................................................................................... 94
`M. Claim 13 ......................................................................................................... 94
`N. Claim 14 ......................................................................................................... 96
`O. Claim 15 ......................................................................................................... 97
`P. Claim 16 ......................................................................................................... 99
`Q. Claim 17 .......................................................................................................106
`R. Claim 18 .......................................................................................................108
`S. Claim 19 .......................................................................................................109
`IX. CONCLUSION ................................................................................................110
`
`
`3
`
`

`

`I.
`
`1.
`
`QUALIFICATIONS AND BACKGROUND INFORMATION
`
`I have been retained on behalf of Samsung Electronics Co., Ltd.
`
`(“Petitioner”) and asked to review and provide my opinion on the patentability of
`
`claims 1-19 of U.S. Patent No. 10,621,228 (“the ’228 patent”).
`
`2.
`
`I am a salaried employee of Fifth Chance Media LLC, which I understand is
`
`being compensated for my work in this matter. I am not an owner of Fifth Chance
`
`Media LLC and my compensation is not contingent on the outcome of this matter
`
`or the specifics of my testimony.
`
`3.
`
`A detailed description of my professional qualifications, including a listing
`
`of my specialties/expertise and professional activities, is contained in my
`
`curriculum vitae, a copy of which is provided as Exhibit 1004. In what follows, I
`
`provide a short summary of my professional qualifications.
`
`4.
`
`In terms of my background and experiences that qualify me as an expert in
`
`this case, I earned a Ph.D. in Electrical Engineering and Computer Science from
`
`Massachusetts Institute of Technology in 1999. I also obtained a Bachelor of
`
`Science Degree in Mathematics from Massachusetts Institute of Technology in
`
`1982 and a Master of Science Degree in Electrical Engineering and Computer
`
`Science from Massachusetts Institute of Technology in 1993.
`
`4
`
`

`

`5. My Ph.D. thesis concerned the engineering of large online Internet
`
`communities with a Web browser front-end and a relational database management
`
`system (RDBMS) containing site content and user data.
`
`6.
`
`I have authored five computer science textbooks in total, including Database
`
`Backed Web Sites (Macmillan), Software Engineering for Internet Applications
`
`(MIT Press), and a SQL language tutorial.
`
`7.
`
`I have served as an independent member of various advisory and corporate
`
`boards, mostly for technology companies. For example, I joined the corporate
`
`board of an MIT materials science spin-off in late 2005 during a $550,000 seed
`
`capital phase. I stepped down when the company secured $10 million in venture
`
`capital in mid-2007.
`
`8.
`
`I have previously served as an expert witness for Amazon.com, IBM,
`
`Microsoft, Xerox, and Google, among others, in patent cases.
`
`9.
`
`I began working full-time as a computer programmer in 1978, developing a
`
`database management system for the Pioneer Venus Orbiter at the National
`
`Aeronautics and Space Administration’s Goddard Space Flight Center.
`
`10.
`
`I developed my first computer-supported mapping applications in the mid-
`
`1980s, starting with an anti-tank warfare simulator for the U.S. Department of
`
`Defense (as a subtractor to Textron). I also developed a computer system for
`
`supporting civil engineering, especially earthmoving, that included a digital three-
`
`5
`
`

`

`dimensional map. The latter system was the topic of my Master’s thesis at MIT and
`
`also U.S. Patents 5,150,310 and 5,964,298 (“Integrated civil engineering and
`
`earthmoving system”).
`
`11.
`
`I developed my first program using a relational database management
`
`system in 1994. It was a Web interface to the Children’s Hospital Oracle RDBMS
`
`version 6. This enabled doctors at the hospital to view patient clinical data using
`
`any computer equipped with a Web browser.
`
`12.
`
`In 1995, I led an effort by Hearst Corporation to set up an infrastructure for
`
`Internet applications across all their newspaper, magazine, radio, and television
`
`properties. This infrastructure included software for managing users, shopping
`
`carts, electronic commerce, advertising, and user tracking.
`
`13. Between 1995 and 1997, I significantly expanded the photo.net online
`
`community that I had started, in 1993, in order to help people teach each other to
`
`become better photographers. I began distributing the source code behind
`
`photo.net to other programmers as a free open-source toolkit, called “ArsDigita
`
`Community System.”
`
`14. The photo.net site enabled users to upload photos as attachments to
`
`discussion forum postings and also, beginning in 1999, included a complete photo-
`
`sharing system along the lines of Flickr.
`
`6
`
`

`

`15.
`
`In May 1997, Macmillan published my first textbook on Internet Application
`
`development, “Database Backed Web Sites.” This book advises readers that
`
`server-based programs should, to the extent possible, summarize information from
`
`the database and make previews of content available so that users don’t have to
`
`navigate as much. See Chapter 131, for example: “One-line summaries of the
`
`newest [classified] ads should be displayed on the very first page of the system.
`
`Users shouldn't have to click to see ads.”
`
`16.
`
`In 1997, I started a company, ArsDigita, to provide support and service for
`
`the free open-source toolkit. Between 1997 and the middle of 2000, I managed the
`
`growth of ArsDigita to 80 people, almost all programmers, and $20 million per
`
`year in annual revenue. This involved supervising dozens of software
`
`development projects, nearly all of which were Internet Applications with a Web
`
`front-end and an Oracle RDBMS back-end. As the founder, CEO, and chief
`
`technical employee of the company, I personally developed functional
`
`specifications, SQL data models (Structured Query Language, or “SQL,” is the
`
`standard programming language for relational database management systems), and
`
`Web page flows that determined the user experience.
`
`
`
`1 One-line summaries of the newest ads should be displayed on the very first
`
`page of the system. Users shouldn't have to click to see ads.
`
`7
`
`

`

`17. Between 2000 and the present, I have done software development projects
`
`for philip.greenspun.com and photo.net, two online services that are implemented
`
`as relational database management applications. In addition, I developed
`
`postclipper.com, a database-backed Web application that works in conjunction
`
`with Facebook to allow parents to produce electronic baby books based on
`
`photographs previously included in Facebook posts.
`
`18. Separately from this commercial and public work, I have been involved, as a
`
`part-time teacher within the Department of Electrical Engineering and Computer
`
`Science, educating students at MIT in how to develop Internet Applications with
`
`an RDBMS back-end. In the Spring of 1999, I taught 6.916, Software Engineering
`
`of Innovative Web Services, with Professors Hal Abelson and Michael Dertouzos.
`
`In the Spring of 2002, this course was adopted into the standard MIT curriculum as
`
`6.171. I wrote 15 chapters of a new textbook for this class, “Software Engineering
`
`for Internet Applications.” This book was published on the Web at
`
`http://philip.greenspun.com/seia/ starting in 2002 and 2003 and also in hardcopy
`
`from MIT Press in 2006. I am the sole author of a supplementary textbook for the
`
`class, “SQL for Web Nerds,” a succinct SQL programming language tutorial
`
`available only on the Web at http://philip.greenspun.com/sql/. I am also one of the
`
`creators and teachers of a three-day intensive course in developing database
`
`applications. We teach this class periodically at MIT.
`
`8
`
`

`

`19.
`
`I periodically teach a database programming class at Harvard Medical
`
`School. Students have access to a relational database of more than 5 billion
`
`insurance claims and write SQL programs to try to identify correlations and trends.
`
`I taught this course most recently in March 2021. I am currently teaching an
`
`Information Security class at Florida Atlantic University.
`
`II. OVERVIEW OF CONCLUSIONS FORMED
`20. As part of my analysis, I have reviewed the ’228 patent, relevant excerpts of
`
`the prosecution history of the ’228 patent. I have also reviewed at least the
`
`following prior art references:
`
`Prior Art Reference
`
`U.S. Patent No. 10,621,228 to Desmond et al. (“Desmond”) (Ex. 1001)
`
`U.S. Patent App. Pub. No. 2011/0122153 A1 to Okamura et al. (“Okamura”)
`(Ex. 1005)
`
`U.S. Patent App. Pub. No. 2010/0058212 A1 to Belitz et al. (“Belitz”) (Ex. 1006)
`
`Tim Grey, Adobe Photoshop Lightroom Workflow: The Digital Photographer's
`Guide (2007) (Ex. 1020)
`
`U.S. Patent App. Pub. No. 2011/0074811 A1 to Hanson et al. ("Hanson") (Ex.
`1021)
`
`Stephen Shankland, “What’s the best Web site for geotagged photos?,” CNET
`(Mar. 18, 2009), available at https://www.cnet.com/tech/computing/whats-the-
`best-web-site-for-geotagged-photos/ (Ex. 1022)
`
`Panoramio, “Embedding a Panoramio map into your web page” (Archive.org:
`Mar. 28, 2010), available at
`https://web.archive.org/web/20100328215828/http://www.panoramio.com:80/hel
`p/embedding (Ex. 1023)
`
`9
`
`

`

`Shu-Wai Chow, PHP Web 2.0 Mashup Projects, Packt Publishing (2007) (Ex.
`1024)
`
`U.S. Patent App. Pub. No. 2009/0113350 A1 to Hibino et al. (“Hibino”) (Ex.
`1026)
`
`U.S. Patent App. Pub. No. 2006/0165380 A1 to Tanaka et al. (“Tanaka”) (Ex.
`1027)
`
`U.S. Patent No. 6,215,523 (“Anderson”) (Ex. 1031)
`
`U.S. Patent App. Pub. No. 2008/0133526 (“Haitani”) (Ex. 1032)
`
`U.S. Patent App. Pub. No. 2008/0306921 (“Rothmuller”) (Ex. 1033)
`
`U.S. Patent App. Pub. No. 2013/0198602 (“Kokemohr”) (Ex. 1034)
`
`U.S. Patent App. Pub. No. 2013/0326338 (“Secord”) (Ex. 1035)
`
`U.S. Patent App. Pub. No. 2007/0016575 (“Hurst-Hiller”) (Ex. 1036)
`
`
`
`21.
`
`I have also reviewed various supporting references and other documentation
`
`in forming my opinions below.
`
`22. Counsel has informed me that I should consider these materials through the
`
`lens of one of ordinary skill in the art related to the ’228 patent at the time of the
`
`earliest possible priority date of the ’228 patent, and I have done so during my
`
`review of these materials. The application leading to the ’228 patent was filed on
`
`September 20, 2019, and claims the benefit of priority to U.S. patent application
`
`No. 13/157,214 filed June 9, 2011 (“the Critical Date”). Counsel has informed me
`
`that the Critical Date represents the earliest possible priority date to which the
`
`10
`
`

`

`challenged claims of ’228 patent are entitled, and I have therefore used that Critical
`
`Date in my analysis below.
`
`23.
`
`I have no financial interest in the party or in the outcome of this proceeding.
`
`I am being compensated for my work as an expert on an hourly basis. My
`
`compensation is not dependent on the outcome of these proceedings or the content
`
`of my opinions.
`
`24.
`
`In writing this declaration, I have considered the following: my own
`
`knowledge and experience, including my work experience in the fields of
`
`computer science and electrical engineering; my experience in teaching those
`
`subjects; and my experience in working with others involved in those fields.
`
`25. My opinions, as explained below, are based on my education, experience,
`
`and expertise in the fields relating to the ’228 patent. Unless otherwise stated, my
`
`testimony below refers to the knowledge of one of ordinary skill in the fields as of
`
`the Critical Date, or before. Any figures that appear within this document have
`
`been prepared with the assistance of Counsel and reflect my understanding of the
`
`’228 patent and the prior art discussed below.
`
`26. This declaration explains the conclusions that I have formed based on my
`
`analysis. To summarize those conclusions, based upon my knowledge and
`
`experience and my review of the prior art publications listed above, I believe that:
`
` Claims 1-19 are rendered obvious by Okamura in view of Belitz (Ground 1)
`
`11
`
`

`

`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`27.
`In my opinion, one of ordinary skill in the art relating to, and at the time of,
`
`the invention of the ’228 patent would have been someone with at least a
`
`Bachelor’s degree in computer science, computer engineering, electrical
`
`engineering, or a related field, and at least one year of experience designing
`
`graphical user interfaces for applications such as photo organization systems.
`
`Additional graduate education could substitute for professional experience, or
`
`significant experience in the field could substitute for formal education.
`
`28. Based on my experiences, I have a good understanding of the capabilities of
`
`one of ordinary skill. Indeed, I have taught, participated in organizations, and
`
`worked closely with many such persons over the course of my career. Based on
`
`my knowledge, skill, and experience, I have an understanding of the capabilities of
`
`one of ordinary skill. For example, from my industry experience, I am familiar
`
`with what an engineer would have known and found predictable in the art. From
`
`teaching and supervising my post-graduate students, I also have an understanding
`
`of the knowledge that a person with this academic experience possesses.
`
`Furthermore, I possess those capabilities myself.
`
`
`
`12
`
`

`

`IV. LEGAL STANDARDS
`A.
`Terminology
`I have been informed by Counsel and understand that the best indicator of
`
`29.
`
`claim meaning is its usage in the context of the patent specification as understood
`
`by one of ordinary skill. I further understand that the words of the claims should
`
`be given their plain meaning unless that meaning is inconsistent with the patent
`
`specification or the patent’s history of examination before the Patent Office.
`
`Counsel has also informed me, and I understand that, the words of the claims
`
`should be interpreted as they would have been interpreted by one of ordinary skill
`
`at the time of the invention was made (not today). Because I do not know at what
`
`date the invention as claimed was made, I have used the earliest priority date of the
`
`’228 patent as the point in time for claim interpretation purposes. That date was
`
`June 9, 2011.
`
`Legal Standards for Anticipation
`B.
`I have been informed by Counsel and understand that documents and
`
`30.
`
`materials that qualify as prior art can render a patent claim unpatentable as
`
`anticipated. I am informed by Counsel and understand that all prior art references
`
`are to be looked at from the viewpoint of a person of ordinary skill in the art.
`
`31.
`
`I am informed by Counsel and understand that a challenged claim is
`
`unpatentable as “anticipated” under 35 U.S.C. § 102 if it is determined that all the
`
`limitations of the claim are described in a single prior art reference. I am informed
`
`13
`
`

`

`by Counsel and understand that, to anticipate a claim, a prior art reference must
`
`disclose, either expressly or inherently, each and every limitation of that claim and
`
`enable one of ordinary skill in the art to make and use the invention.
`
`32.
`
`I have been informed by Counsel and understand that in an inter partes
`
`review, “the petitioner shall have the burden of proving a proposition of
`
`unpatentability,” including a proposition of anticipation, “by a preponderance of
`
`the evidence.” 35 U.S.C. §316(e).
`
`Legal Standards for Obviousness
`C.
`I have been informed by Counsel and understand that documents and
`
`33.
`
`materials that qualify as prior art can render a patent claim unpatentable as
`
`obvious. I am informed by Counsel and understand that all prior art references are
`
`to be looked at from the viewpoint of a person of ordinary skill in the art at the
`
`time of the invention, and that this viewpoint prevents one from using his or her
`
`own insight or hindsight in deciding whether a claim is obvious.
`
`34.
`
`I have been informed by Counsel and understand that a claim is unpatentable
`
`for obviousness under 35 U.S.C. § 103 “if the differences between the subject
`
`matter sought to be patented and the prior art are such that the subject matter as a
`
`whole would have been obvious at the time the invention was made to a person
`
`having ordinary skill in the art to which said subject matter pertains.” I am
`
`informed by Counsel and understand that obviousness may be based upon a
`
`14
`
`

`

`combination of references. I am informed by Counsel and understand that the
`
`combination of familiar elements according to known methods is likely to be
`
`obvious when it does no more than yield predictable results. However, I am
`
`informed by Counsel and understand that a patent claim composed of several
`
`elements is not proved obvious merely by demonstrating that each of its elements
`
`was, independently, known in the prior art.
`
`35.
`
`I am informed by Counsel and understand that when a patented invention is
`
`a combination of known elements, a court must determine whether there was an
`
`apparent reason to combine the known elements in the fashion claimed by the
`
`patent at issue by considering the teachings of prior art references, the effects of
`
`demands known to people working in the field or present in the marketplace, and
`
`the background knowledge possessed by a person having ordinary skill in the art.
`
`36.
`
`I am informed by Counsel and understand that a patent claim composed of
`
`several limitations is not proved obvious merely by demonstrating that each of its
`
`limitations was independently known in the prior art. I am informed by counsel for
`
`the Patent Owner and understand that identifying a reason those elements would be
`
`combined can be important because inventions in many instances rely upon
`
`building blocks long since uncovered, and claimed discoveries almost of necessity
`
`will be combinations of what, in some sense, is already known. I am informed by
`
`15
`
`

`

`Counsel and understand that it is improper to use hindsight in an obviousness
`
`analysis, and that a patent’s claims should not be used as a “roadmap.”
`
`37.
`
`I am informed by Counsel and understand that an obviousness inquiry
`
`requires consideration of the following factors: (1) the scope and content of the
`
`prior art; (2) the differences between the claims and the prior art; (3) the level of
`
`ordinary skill in the pertinent art; and (4) any objective indicia of non-obviousness,
`
`such as commercial success, long-felt but unresolved need, failure of others,
`
`industry recognition, copying, and unexpected results. I understand that the
`
`foregoing factors are sometimes referred to as the “Graham factors.”
`
`38.
`
`I have been informed by Counsel and understand that an obviousness
`
`evaluation can be based on a combination of multiple prior art references. I
`
`understand that the prior art references themselves may provide a suggestion,
`
`motivation, or reason to combine, but that the nexus linking two or more prior art
`
`references is sometimes simple common sense. I have been informed by Counsel
`
`and understand that obviousness analysis recognizes that market demand, rather
`
`than scientific literature, often drives innovation, and that a motivation to combine
`
`references may be supplied by the direction of the marketplace.
`
`39.
`
`I have been informed by Counsel and understand that if a technique has been
`
`used to improve one device, and a person of ordinary skill at the time of invention
`
`would have recognized that it would improve similar devices in the same way,
`
`16
`
`

`

`using the technique is obvious unless its actual application is beyond his or her
`
`skill.
`
`40.
`
`I have been informed by Counsel and understand that practical and common
`
`sense considerations should guide a proper obviousness analysis, because familiar
`
`items may have obvious uses beyond their primary purposes. I have been
`
`informed by Counsel and understand that a person of ordinary skill looking to
`
`overcome a problem will often be able to fit together the teachings of multiple
`
`prior art references. I have been informed by Counsel and understand that
`
`obviousness analysis therefore takes into account the inferences and creative steps
`
`that a person of ordinary skill would have employed at the time of invention.
`
`41.
`
`I have been informed by Counsel and understand that a proper obviousness
`
`analysis focuses on what was known or obvious to a person of ordinary skill at the
`
`time of invention, not just the patentee. Accordingly, I understand that any need or
`
`problem known in the field of endeavor at the time of invention and addressed by
`
`the patent can provide a reason for combining the elements in the manner claimed.
`
`42.
`
`I have been informed by Counsel and understand that a claim can be obvious
`
`in light of a single reference, without the need to combine references, if the
`
`elements of the claim that are not found explicitly or inherently in the reference
`
`can be supplied by the common sense of one of skill in the art.
`
`17
`
`

`

`43.
`
`I have been informed by Counsel and understand that secondary indicia of
`
`non-obviousness may include (1) a long felt but unmet need in the prior art that
`
`was satisfied by the invention of the patent; (2) commercial success of processes
`
`covered by the patent; (3) unexpected results achieved by the invention; (4) praise
`
`of the invention by others skilled in the art; (5) taking of licenses under the patent
`
`by others; (6) deliberate copying of the invention; (7) failure of others to find a
`
`solution to the long felt need; and (8) skepticism by experts. I understand that
`
`evidence of secondary indicia of non-obviousness, if available, should be
`
`considered as part of the obviousness analysis.
`
`44.
`
`I have been informed by Counsel and understand that there must be a
`
`relationship between any such secondary considerations and the invention, and that
`
`contemporaneous and independent invention by others is a secondary consideration
`
`supporting an obviousness determination.
`
`45.
`
`In sum, my understanding is that prior art teachings are properly combined
`
`where one of ordinary skill having the understanding and knowledge reflected in
`
`the prior art and motivated by the general problem facing the inventor, would have
`
`been led to make the combination of elements recited in the claims. Under this
`
`analysis, the prior art references themselves, or any need or problem known in the
`
`field of endeavor at the time of the invention, can provide a reason for combining
`
`the elements of multiple prior art references in the claimed manner.
`
`18
`
`

`

`46.
`
`I have been informed by Counsel and understand that in an inter partes
`
`review, “the petitioner shall have the burden of proving a proposition of
`
`unpatentability,” including a proposition of obviousness, “by a preponderance of
`
`the evidence.” 35 U.S.C. §316(e).
`
`
`
`V.
`
`The ’228 Patent
`
`A. Overview of the ’228 Patent
`
`47. The ’228 patent and all of the prior art references discussed herein are from
`
`the field of managing digital image files. This is an old and well-established area
`
`of technology and displaying digital image files on the basis of location and/or
`
`people depicted, in particular, is not at all new. In fact, as indicated above and on
`
`my CV, I began working with computer-generated maps in the 1980s and with
`
`web-based systems for managing digital photographs in 1994, most notably for the
`
`photo.net service.
`
`48. According to the inventors, the ’228 patent is generally directed to managing
`
`digital files and displaying a representation of the digital files using a “map view”
`
`or a “people view.” Ex. 1001, 22:59-23:11, 34:16-54, 29:41-64, FIGS. 32, 34, 41.
`
`Despite the “Field of the Invention” (Ex. 1001, 1:20-22) implying a relation to the
`
`management of any digital file, the ’228 patent is related to managing digital image
`
`files (including videos) rather than describing a general-purpose file-management
`
`19
`
`

`

`tool, such as the Unix file system or Microsoft’s NTFS, included with the
`
`Windows operating system.
`
`49. The ’228 patent describes a map view (as illustrated in FIG. 41 below) that
`
`displays an interactive map and location selectable thumbnail images at different
`
`locations on the map. The location selectable thumbnail images (e.g., indicators
`
`0874 and 0875) allow the user “to select the thumbnail to see all the Digital Files
`
`with the same location.” Ex. 1001, 29:51-52. Each pinned location on the
`
`interactive map includes “a thumbnail of the Digital File” and “the number of
`
`Digital Files for that location.” Ex. 1001, 29:55-57. As shown in FIG. 41, the
`
`location associated with the indicator 0875 has four digital files, as represented by
`
`“4” in the right corner of the thumbnail image.
`
`50. The map view of the ’228 patent is shown below.
`
`20
`
`

`

`Ex. 1001, FIG. 41
`
`
`
`51. When the user selects one of the location selectable thumbnail images (e.g.,
`
`indicator 0875), the user interface displays a location view (as illustrated in FIG.
`
`34 below). The location view displays a first location name (e.g., “Wrigley Field”)
`
`associated with the selected location and a representation of at least a portion of
`
`one digital file (e.g., photo 1633) among digital files associated with the selected
`
`location. The digital files that “were taken or originated” at the selected location
`
`are associated with said location. Ex. 1001, 6:20.
`
`52. The location view of the ’228 patent is shown below.
`
`21
`
`

`

`Ex. 1001, FIG. 34
`
`
`
`53. The ’228 patent also describes providing a people view (as illustrated in
`
`FIG. 32 below as a “Multiple People Application View”) that displays a “person
`
`selectable” thumbnail image and a name associated with the person. The person
`
`selectable thumbnail images (e.g., indicator 1403) include a representation of each
`
`person’s face and each person’s name (e.g., “Jon Smith”). Ex. 1001, 23:1-4.
`
`54. Responsive to selection of one of the person selectable thumbnail images
`
`(e.g., indicator 1403), the user interface displays a single person view (as
`
`illustrated in FIG. 32 below as a “Single People Profile Application View”). The
`
`single person view displays thumbnails (e.g., 1446) of digital files associated with
`
`the selected person. Ex. 1001, 23:12-24.
`
`22
`
`

`

`55. The people view of the ’228 patent is shown below.
`
`Ex. 1001, FIG. 32
`
`
`
`B.
`
`File History of the ’228 Patent
`
`56. The underlying application to the ’228 patent, U.S. Application No.
`
`16/578,238, was allowed without a rejection on the merits. The Examiner
`
`provided only a cursory discussion of how the allowed claims were patentable in
`
`that the Examiner summarized the interview that led to allowance as “Allowable
`
`Subject Matter was discussed. … Examiner’s amendment was authorized to
`
`advance the case” and only stated that “people view” features in claim 1 were
`
`missing from two considered references: Hibino (Ex. 1026) and Tanaka (Ex.
`
`1027). See Ex. 1002, 42-44. Okamura and Belitz were not considered by the
`
`23
`
`

`

`Examiner in granting the ’228 patent. Prior art references cited below demonstrate
`
`that the “people view” features of claim 1 were well-known at the time of the ’228
`
`patent, and, had the Examiner been aware of this prior art, the application would
`
`not have been allowed.
`
`VI. TECHNOLOGY BACKGROUND
`
`57. The earliest widely used computer-managed databases for digital images
`
`were 1980s medical systems, e.g., for radiology. A general-purpose system was
`
`described in an October 21, 1986 New York Times “Company Briefs” article2:
`
`Eastman Kodak Co., Rochester, said its Edicon subsidiary had introduced its first product
`line, a photo image management system that integrates photographic images with text
`and graphics in a computerized database.
`
`58. The seeds of modern location-based photo management tools for consumers
`
`may be seen in U.S. Patent 5,262,867 (1990), describing a Sony system in which
`
`location data are recorded with each image capture (the then-new Global
`
`Positioning System is referenced Col 4, ll. 4-4-13). A similar idea may be seen in
`
`U.S. Patent 5,666,578 (1993), describing a Nikon system in which GPS location