IPR2021-01413
`U.S. Patent 10,621,228
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________________________________________
`
`UNIFIED PATENTS, LLC
`Petitioner
`v.
`MEMORYWEB, LLC
`Patent Owner
`IPR2021-01413
`U.S. PATENT 10,621,228
`
`
`DECLARATION OF BENJAMIN B. BEDERSON, PH.D.
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 1 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`TABLE OF CONTENTS
`I.
`INTRODUCTION ............................................................................................. 1
`II. BACKGROUND AND QUALIFICATIONS ................................................... 2
`III. Summary of My Opinions ............................................................................... 10
`A. Overview ................................................................................................... 10
`B.
`Bases of My Opinions and Materials Considered .................................... 11
`C.
`Level of Skill in the Art ............................................................................ 11
`D.
`Instructions and Legal Framework ........................................................... 12
`E.
`Effective Filing Dates and Prior Art Patents and Printed Publications .... 17
`IV. BACKGROUND ............................................................................................. 18
`V. U.S. PATENT 10,621,228 ............................................................................... 19
`VI. CLAIM CONSTRUCTION ............................................................................. 27
`VII. THE CHALLENGED CLAIMS ARE INVALID ........................................... 27
`A. Ground 1: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over Okamura
`and Flora. ............................................................................................................ 28
`1. Okamura .................................................................................................... 28
`2. Flora .......................................................................................................... 33
`3. Technical Concepts ................................................................................... 35
`Ground 2: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over
`B.
`Okamura, Flora, and Wagner. ............................................................................. 79
`1. Wagner ...................................................................................................... 79
`2. Technical Concepts - Motivations to Combine Okamura, Flora, and
`Wagner ............................................................................................................. 80
`3. Other Technical Concepts ......................................................................... 91
`i
`
`
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 2 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`Ground 3: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over
`C.
`Okamura, Flora, and Gilley. ................................................................................ 91
`1. Gilley ........................................................................................................ 92
`2. Technical Concepts - Motivations to Combine Okamura, Flora, and Gilley
`
`93
`3. Other Technical Concepts ....................................................................... 100
`D. Ground 4: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over
`Okamura, Flora, Wagner, and Gilley. ............................................................... 100
`1. Technical Concepts - Motivations to Combine Okamura, Flora, Wagner,
`and Gilley ....................................................................................................... 100
`2. Other Technical Concepts ....................................................................... 101
`VIII.
`SECONDARY CONSIDERATIONS .................................................... 101
`IX. MAPPINGS OF THE PETITION ................................................................. 101
`X. CONCLUSION .............................................................................................. 102
`
`
`
`
`
`ii
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 3 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`I, Benjamin B. Bederson, declare as follows:
`
`I.
`
`INTRODUCTION
`
`1. My name is Benjamin B. Bederson, and I am over 21 years and
`
`otherwise competent to make this Declaration. I make this Declaration based on facts
`
`and matters within my own knowledge and on information provided to me by others,
`
`and, if called as a witness, I could and would competently testify to the matters set
`
`forth herein.
`
`2.
`
`I have been asked by Unified Patents, LLC (“Petitioner”) to provide
`
`technical assistance in connection with the Inter Partes Review of U.S. Patent
`
`10,621,228 (which I refer to as the “’228 Patent”). I have been informed that,
`
`according to public records, MemoryWeb, LLC (“Patent Owner”) is the owner of
`
`the ’228 Patent.
`
`3.
`
`I have reviewed, had input into, and endorse the technological
`
`discussions in the Petitioner’s Petition for Inter Partes Review of U.S. Patent
`
`10,621,228 challenging claims 1-7, including the statements in the Petition regarding
`
`the ’228 Patent, the scope of the claims, the prior art’s disclosure of the claims, and
`
`the statements throughout the Petition regarding a person of ordinary skill in the art’s
`
`(POSITA’s) knowledge and understanding.
`
`
`
`1
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 4 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`In addition, this declaration is a statement of my opinions on issues
`
`4.
`
`related to the patentability of claim 1-7 of the ’228 Patent (collectively, the
`
`“Challenged Claims”).
`
`5. My compensation is not based on the content of my opinions or the
`
`resolution of this matter. I have no financial interest in Petitioner or Patent Owner,
`
`and I have no other interest in the outcome of this matter.
`
`II. BACKGROUND AND QUALIFICATIONS
`
`6.
`
`In formulating my opinions, I have relied on my knowledge, training,
`
`and experience in the relevant field, which I have summarized below. My curriculum
`
`vitae is included in Attachment A.
`
`7.
`
`I hold Ph.D. (1992), M.S. (1989), and B.S. (1986) degrees in Computer
`
`Science. I also earned an undergraduate minor in electrical engineering. I received
`
`the Janet Fabri Memorial Award for Outstanding Doctoral Dissertation for my Ph.D.
`
`work in robotics and computer vision. I have combined my hardware and software
`
`skills throughout my career in Human-Computer Interaction research, building
`
`various interactive electrical and mechanical systems that couple with software to
`
`provide an innovative user experience.
`
`8.
`
`Since 1998, I have been a Professor of Computer Science at the
`
`University of Maryland (“UMD”), where I have joint appointments at the Institute
`
`
`
`2
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 5 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`for Advanced Computer Studies and the College of Information Studies (Maryland’s
`
`“iSchool”). I was also Associate Provost of Learning Initiatives and Executive
`
`Director of the Teaching and Learning Transformation Center until June 2018. I am
`
`currently Professor Emeritus.
`
`9.
`
`At UMD, my research is in the area of Human-Computer Interaction
`
`(“HCI”), a field that relates to the development and understanding of computing
`
`systems to serve users’ needs. Researchers in this field are focused on making
`
`universally usable, useful, efficient, and appealing systems to support people in their
`
`wide range of activities. My approach is to balance the development of innovative
`
`technology that serves people’s practical needs.
`
`10.
`
`I am a member and previous director (2000-2006) of the Human-
`
`Computer Interaction Lab (“HCIL”), the oldest and one of the best-known Human-
`
`Computer Interaction research groups in the country.
`
`11.
`
`I am also co-founder and co-director of the International Children’s
`
`Digital Library (“ICDL”), a web site providing the world’s largest collection of
`
`freely available online children’s books from around the world with an interface
`
`aimed to make it easy for children and adults to search and read children’s books
`
`online.
`
`
`
`3
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 6 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`I co-founded Zumobi, Inc. in 2006, where I was responsible for
`
`12.
`
`investigating new software platforms and developing new user interface designs that
`
`provide efficient and engaging interfaces to permit end users to access a wide range
`
`of content on mobile platforms (including the iPhone and Android-based devices).
`
`For example, I designed and implemented software called “Ziibii,” a “river” of news
`
`for iPhone that used a capacitive sensor for controlling linear movement through
`
`news; software called “ZoomCanvas,” a zoomable user interface for several iPhone
`
`apps; and iPhone apps including “Inside Xbox” for Microsoft and Snow Report for
`
`REI. At the ICDL, I have since 2002 been the technical director responsible for the
`
`design and implementation of the web site, www.childrenslibrary.org (originally at
`
`www.icdlbooks.org). In particular, I have been closely involved in designing the
`
`user interface as well as the software architecture for the web site since its inception
`
`in 2002.
`
`13. Starting in 2001 and continuing for several years, I worked on photo
`
`management systems. I wrote a paper1 in 2001 describing PhotoMesa, a zoomable
`
`
`1 See EX1014, Benjamin B. Bederson, PhotoMesa: A Zoomable Image Browser
`
`Using Quantum Treemaps and Bubblemaps, Proceedings of the 14th Annual ACM
`
`Symposium on User Interface Software and Technology (UIST ’01), 71–80,
`
`
`
`4
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 7 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`image browser using a novel 2D layout algorithm I called “quantum treemaps” to
`
`position the images on the screen in a way that attempted to fill the screen while
`
`keeping groups of related images together. As indicated in the figure below from
`
`that paper, PhotoMesa could display many hundreds of images at a time by showing
`
`small thumbnails, by showing larger thumbnails when the mouse hovered over a
`
`small thumbnail, and by allowing a user to zoom in to see a high resolution version
`
`of the images.
`
`EX1014, Fig. 1
`
`
`
`
`Association for Computing Machinery, New York, NY, USA, (2001)
`
`(DOI:https://doi.org/10.1145/502348.502359).
`
`
`
`5
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 8 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`In 2003, I wrote a paper2 that performed image processing to identify
`
`14.
`
`salient portions of images and then automatically crop those images to show those
`
`portions. In this way, image thumbnails could focus attention on faces or other
`
`objects of a photo without showing all of the background. In 20063, I wrote a paper
`
`that explained some of the trends in photo management software including rich
`
`support for annotation, browsing and sharing of photos. In that same year, I
`
`
`2 EX1015, Bongwon Suh, Haibin Ling, Benjamin B. Bederson, and David W.
`
`Jacobs, Automatic Thumbnail Cropping and its Effectiveness, Proceedings of the
`
`16th Annual ACM symposium on User Interface Software and Technology (UIST
`
`'03), 95–104, Association for Computing Machinery, New York, NY, USA, (2003)
`
`(DOI:https://doi.org/10.1145/964696.964707).
`
`3 EX1016, Ben Shneiderman, Benjamin B. Bederson, and Steven M. Drucker, Find
`
`that photo! interface strategies to annotate, browse, and share. Commun. ACM
`
`49, 4 (April 2006), 69–71. (DOI:https://doi.org/10.1145/1121949.1121985).
`
`
`
`
`
`6
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 9 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`published another paper4 that provided a review of innovative personal photo
`
`management work going on at that time. That paper describes a version of my
`
`PhotoMesa software that by 2007 as shown in the following figure, included the
`
`ability to annotate photos with the location and name of people in photos.
`
`
`4 EX1009, Hyunmo Kang, Benjamin B. Bederson, and Bongwon Suh, Capture,
`
`Annotate, Browse, Find, Share: Novel Interfaces for Personal Photo Management,
`
`International Journal of Human-Computer Interaction, 23(3), 315-337 (2007).
`
`
`
`7
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 10 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`
`
`
`
`
`
`EX1009, Fig. 1
`
`
`
`15. That work on photo management is exemplary of my 30 years of
`
`teaching, research and entrepreneurial experience in Human-Computer Interaction
`
`and the software and technology underlying today’s interactive computing systems.
`
`Example systems following this approach that I have built include PhotoMesa
`
`(software for end users to browse personal photos), DateLens (2002 software for end
`8
`
`
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 11 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`users to use their mobile devices to efficiently access their calendar information),
`
`LaunchTile (2005 “home screen” software for mobile devices to allow users to
`
`navigate apps in a zoomable environment), SpaceTree (2001 software for end users
`
`to efficiently browse very large hierarchies), ICDL (as described above), and
`
`StoryKit (a 2009 iPhone app for children to create stories).
`
`16. My work has been published extensively in more than 160 technical
`
`publications, and I have given about 100 invited talks, including 9 keynote lectures.
`
`I have won a number of awards including the Brian Shackel Award for “outstanding
`
`contribution with international impact in the field of HCI” in 2007, and the Social
`
`Impact Award in 2010 from Association for Computing Machinery’s (“ACM”)
`
`Special Interest Group on Computer Human Interaction (“SIGCHI”). ACM is the
`
`primary international professional community of computer scientists, and SIGCHI
`
`is the primary international professional HCI community. I have been honored by
`
`both professional organizations. I am an “ACM Distinguished Scientist,” which
`
`“recognizes those ACM members with at least 15 years of professional experience
`
`and 5 years of continuous Professional Membership who have achieved significant
`
`accomplishments or have made a significant impact on the computing field.”
`
`17. Since 1999, I’ve advised about a hundred students, including 11
`
`doctoral candidates, on their research and dissertations in the field of computer
`
`
`
`9
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 12 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`science, including on topics related to data collection, visualization, and analysis,
`
`interface design, personal information management, and many more listed in my
`
`CV.
`
`18.
`
`I have appeared on radio shows numerous times to discuss issues
`
`relating to user interface design and people’s use and frustration with common
`
`technologies, web sites, and mobile devices. My work has been discussed and I have
`
`been quoted by mainstream media around the world over 120 times, including by
`
`the New York Times, the Wall Street Journal, the Washington Post, Newsweek, the
`
`Seattle Post Intelligencer, the Independent, Le Monde, NPR’s All Things
`
`Considered, New Scientist Magazine, and MIT’s Technology Review.
`
`19.
`
`I have designed, programmed, and publicly deployed dozens of
`
`userfacing software products that have cumulatively had millions of users. My work
`
`is cited by several major companies, including Amazon, Apple, Facebook, Google,
`
`and Microsoft. I am a named inventor on 12 U.S. patents and 18 U.S. patent
`
`applications. The patents are generally directed to user interfaces/experience.
`
`III. SUMMARY OF MY OPINIONS
`
`A. Overview
`20.
`In my opinion, the Challenged Claims are unpatentable. My opinions
`
`are based on my expertise in the technology of the ’228 Patent as of its earliest
`
`
`
`10
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 13 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`possible priority date, as well as my review of the ’228 Patent, its file history, and
`
`the prior art discussed in the Petition.
`
`B.
`21.
`
`Bases of My Opinions and Materials Considered
`I have reviewed the ’228 patent and the prior art and other documents
`
`and materials cited herein. For ease of reference, a list of documents that I have
`
`considered is included in Attachment B.
`
`22. My opinions in this Declaration, as well as those reflected in the
`
`Petition, are based on my review of these documents, as well as upon my education,
`
`training, research, knowledge, and experience.
`
`Level of Skill in the Art
`C.
`23. My analysis assumes that a “person of ordinary skill in the art,”
`
`(“POSITA”) at the time of the alleged invention would have had at least a bachelor’s
`
`degree in computer science, electrical engineering, or a related field, and at least
`
`two years of academic or industry experience in software development related to
`
`content management systems and user interfaces. More education can
`
`supplement practical experience and vice-versa.
`
`24.
`
`I qualified as a POSITA for the ’228 Patent as of its earliest possible
`
`priority date (June 9, 2011) because by that date I had obtained my Ph.D. in computer
`
`science and had approximately 19 years of experience in software development
`
`related to content management systems and user interfaces.
`11
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 14 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`Instructions and Legal Framework
`I am not an attorney. My analysis and opinions are based on my
`
`D.
`25.
`
`expertise in this technical field, as well as the instructions I have been given by
`
`counsel for the legal standards relating to patentability. As stated above, I have
`
`reviewed and endorse the substance of the Petition regarding the ’228 Patent, the
`
`scope of the claims, the prior art’s disclosure of the claims, and the statements
`
`throughout the Petition regarding a POSITA’s knowledge and understanding. But,
`
`for the avoidance of any doubt, as I am not an attorney, I provide no opinions as to
`
`the legal conclusions in the Petition, including any characterizations of laws or legal
`
`cases.
`
`26.
`
`I understand that in inter partes review proceedings, invalidity must be
`
`shown under a preponderance of the evidence standard, and this is the standard I
`
`have used throughout my declaration. Further, I understand that each patent claim is
`
`considered separately for purposes of unpatentability.
`
`27.
`
`I understand that, for a patent claim to be “anticipated” by the prior art,
`
`each and every requirement of the claim must be found, expressly or inherently, in
`
`a single prior art reference as recited in the claim. I understand that claim limitations
`
`that are not expressly described in a prior art reference may still be there if they are
`
`“inherent” to the thing or process being described in the prior art. For example, an
`
`
`
`12
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 15 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`indication in a prior art reference that a particular process complies with a published
`
`standard would indicate that the process must inherently perform certain steps or use
`
`certain data structures that are necessary to comply with the published standard.
`
`28.
`
`I understand that it is acceptable to consider evidence other than the
`
`information in a particular prior art document to determine if a feature is necessarily
`
`present in or inherently described by that reference.
`
`29.
`
`I understand that a patent claim is unpatentable as “obvious” if, in view
`
`of a prior art reference or a combination of prior art references, it would have been
`
`obvious to a POSITA at the time of the alleged invention, taking into account:
`
`a. the scope and content of the prior art;
`
`b. the differences between the prior art and the claim under
`
`construction; and
`
`c. the level of ordinary skill in the art.
`
`30. Also, I understand that obviousness does not require physical
`
`combination/bodily incorporation, but rather consideration of what the combined
`
`teachings would have suggested to persons of ordinary skill in the art at the time of
`
`the alleged invention. I understand that the combination of familiar elements
`
`according to known methods is likely to be obvious when it does no more than yield
`
`predictable results.
`
`
`
`13
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 16 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`I further understand that a claim would have been obvious if it unites
`
`31.
`
`old elements with no change to their respective functions, or alters prior art by mere
`
`substitution of one element for another known in the field and that combination
`
`yields predictable results. While it may be helpful to identify a reason for this
`
`combination, I understand that there is no rigid requirement of finding an express
`
`teaching, suggestion, or motivation to combine within the references. When a
`
`product is available, design incentives and other market forces can prompt variations
`
`of it, either in the same field or different one. If a POSITA can implement a
`
`predictable variation, obviousness likely bars its patentability. For the same reason,
`
`if a technique has been used to improve one device and a POSITA would recognize
`
`that it would improve similar devices in the same way, using the technique would
`
`have been obvious. I understand that a claim would have been obvious if common
`
`sense directs one to combine multiple prior art references or add missing features to
`
`reproduce the alleged invention recited in the claims.
`
`32.
`
`I further understand that certain factors may support or rebut the
`
`obviousness of a claim. I understand that such secondary considerations include,
`
`among other things, commercial success of the patented invention, skepticism of
`
`those having ordinary skill in the art at the time of the alleged invention, unexpected
`
`results of the alleged invention, any long-felt but unsolved need in the art that was
`
`
`
`14
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 17 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`satisfied by the alleged invention, the failure of others to make the alleged invention,
`
`praise of the alleged invention by those having ordinary skill in the art, and copying
`
`of the alleged invention by others in the field. I understand that there must be a
`
`nexus—a connection—between any such secondary considerations and the alleged
`
`invention. I also understand that contemporaneous and independent invention by
`
`others is a secondary consideration tending to show obviousness.
`
`33.
`
`I am informed that legal principles regarding unpatentability of a claim
`
`due to obviousness have been addressed by the U.S. Supreme Court. I am informed
`
`that, while not absolute, the principles relating to a “motivation,” “suggestion,” or
`
`“teaching” in the prior art to combine references are useful in analyzing whether an
`
`invention is obvious. I am informed that the suggestion or motivation may be either
`
`explicit or implicit and may come from knowledge generally available to a POSITA,
`
`from the nature of the problem to be solved, or from a combination of these factors.
`
`The test for an implicit motivation, suggestion, or teaching is what the combined
`
`teachings, knowledge of a POSITA, and the nature of the problem to be solved as a
`
`whole would have suggested to those of ordinary skill in the art. The problem
`
`examined is not the specific problem solved by the invention, but the general
`
`problem that confronted the inventor before the invention was made.
`
`
`
`15
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 18 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`I am further informed that the U.S. Supreme Court has clarified that
`
`34.
`
`additional principles may also be applied in such an analysis. Some of those
`
`principles are set forth below.
`
`35. As I understand it, it is no longer always required to present evidence
`
`of an explicit teaching, suggestion, or motivation to combine prior art references for
`
`purposes of determining whether an invention is obvious. Prior art can be combined
`
`based on an express teaching, suggestion, or motivation from the prior art itself, or
`
`from a reasoned explanation of an expert or other witness.
`
`36. A patent claim composed of several elements, however, is not proved
`
`obvious merely by demonstrating that each of its elements was, independently,
`
`known in the prior art. In order to prove obviousness, it must be shown that the
`
`improvement is not more than the predictable use of prior-art elements according to
`
`their established functions. To determine whether there was an apparent reason to
`
`combine the known elements in the way a patent claims, it will often be necessary
`
`to look to interrelated teachings of multiple pieces of prior art, to the effects of
`
`demands known to the design community or present in the marketplace, and to the
`
`background knowledge possessed by a POSITA. Also, in determining obviousness,
`
`one must be aware of the distortion caused by hindsight bias and be cautious of
`
`arguments relying upon hindsight reasoning. An obviousness argument cannot be
`
`
`
`16
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 19 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`sustained by mere conclusory statements. Instead, it must be some articulated
`
`reasoning with some rational underpinnings to support the legal conclusion of
`
`obviousness.
`
`37.
`
`In an obviousness analysis, it is my understanding that there are
`
`“secondary considerations” that should be analyzed if they apply. I am told that these
`
`considerations include (a) whether the prior art teaches away from the claimed
`
`invention, (b) whether there was a long felt but unresolved need for the claimed
`
`invention, (c) whether others tried but failed to make the claimed invention, (d)
`
`skepticism of experts, (e) whether the claimed invention was commercially
`
`successful, (f) whether the claimed invention was praised by others, and (g) whether
`
`the claimed invention was copied by others.
`
`38.
`
`I understand that the claims are construed according to their ordinary
`
`and customary meaning as understood by one of ordinary skill in the art and the
`
`prosecution history pertaining to the patent.
`
`E.
`
`39.
`
`Effective Filing Dates and Prior Art Patents and Printed
`Publications
`I am informed that I am to consider June 9, 2011 to be the priority date
`
`for the ’228 Patent.
`
`40.
`
`I rely upon the following references, all of which I understand are prior
`
`art to all claims of the ’228 Patent:
`
`
`
`17
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 20 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`
`• U.S. Patent Application Publication 2011/0122153, filed October 20, 2010
`
`(“Okamura”) (EX1004).
`
`• U.S. Patent 6,714,215, filed May 19, 2000, published March 30, 2004 (“Flora”)
`
`(EX1005).
`
`• U.S. Patent Application Publication 2011/0163971, filed May 27, 2010
`
`(“Wagner”) (EX1006).
`
`• U.S. Patent Application Publication 2010/0172551, filed September 25, 2009,
`
`published July 8, 2010 (“Gilley”) (EX1007).
`
`
`IV. BACKGROUND
`
`41. As noted above, I understand that the ’228 patent’s priority date is June
`
`9, 2011. File management systems, including those that organize content such as
`
`digital images and video, were notoriously well-known before this date and have
`
`been since at least the 2000s. For example, I co-authored and published the paper
`
`“Capture, Annotate, Browse, Find, Share: Novel Interfaces for Personal Photo
`
`Management” (EX1009) in 2007, which discussed various known tools for
`
`photomanagement and a vision for improving upon them. EX1009, p. 1. Some of
`
`those known photomanagement tools, such as PhotoFinder and PhotoMesa, were
`
`present in the early 2000s— between 2000 and 2002—while others such as Flickr,
`
`
`
`18
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 21 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`Picasa, Snapfish, Shutterfly, PhotoBucket, Adobe Photoshop Album, and ACDSee
`
`gained prevelance shortly after. EX1009, p. 2.
`
`42. Content management systems that organize content according to
`
`location and people were also well-known prior to 2011. As EX1009 explains,
`
`“[e]ven in a simple personal photo library, images can be organized by time lines,
`
`locations, events, people, or other attributes.” EX1009, p. 12.5 EX1010, which
`
`discusses the software iPhoto ’09 and was publicly available at least by March of
`
`2010, further explains organizing photos by locations and people, stating that a
`
`“Places” feature “allows you to display photos on a map using location information”
`
`and a “Faces” feature “uses face recognition to help you organize your photos based
`
`on the people who appear in them.” EX1010, pp. 1, 3, 4.
`
`V. U.S. PATENT 10,621,228
`
`43.
`
`I agree with the Summary of the ’228 Patent and description of the ’228
`
`Patent’s file history provided in the Petition. I provide a listing of the Challenged
`
`Claims, as well as further analysis below.
`
`Claim 1
`
`[1a-preamble] “A method comprising:”
`
`
`5 Bolding emphasis added.
`
`
`
`19
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 22 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`[1b] “responsive to a first input, causing a map view to be displayed on an
`
`interface,”
`
`[1c] “the map view including: (i) an interactive map;”
`
`[1d] “[the map view including:] (ii) a first location selectable thumbnail image
`
`at a first location on the interactive map; and”
`
`[1e] “[the map view including:] (iii) a second location selectable thumbnail
`
`image at a second location on the interactive map;”
`
`[1f] “responsive to an input that is indicative of a selection of the first location
`
`selectable thumbnail image, causing a first location view to be displayed on
`
`the interface,
`
`[1g] the first location view including (i) a first location name associated with
`
`the first location and (ii) a representation of at least a portion of one digital
`
`file in a first set of digital files,
`
`[1h] each of the digital files in the first set of digital files being produced from
`
`outputs of one or more digital imaging devices, the first set of digital files
`
`including digital files associated with the first location;”
`
`[1i] “responsive to an input that is indicative of a selection of the second
`
`location selectable thumbnail image, causing a second location view to be
`
`displayed on the interface,
`
`
`
`20
`
`UNIFIED PATENTS EXHIBIT 1002
`Page 23 of 251
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Declaration of Benjamin B. Bederson, Ph.D.
`[1j] “the second location view including (i) a second location name associated
`
`with the second location and (ii) a representation of at least a portion of one
`
`digital file in a second set of digital files,
`
`[1k] “each of the digital files in the second set of digital files being produced
`
`from outputs of the one or more digital imaging devices, the second set of
`
`digital files including digital files associated with the second location; and”
`
`[1l] “responsive to a second input that is subsequent to the first input, causing
`
`a people view to be displayed on the interface,”
`
`[1m] “the people view including: (i) a first person selectable thumbnail image
`
`including a representation of a face of a first person, the first person being
`
`associated with a third set of digital files including digital photographs and
`
`videos;”
`
`[1n] “[the people view including:] (ii) a first name associated with the first
`
`person, the first name being displayed adjacent to the first person selectable
`
`thumbnail image;”
`
`[1o] “[the people view including:] (iii) a second person selectable thumbnail
`
`image including a representation of a face of a second person, the second
`
`person being assoc