IPR2021-01413
`U.S. Patent 10,621,228
`
`DOCKET NO.: MEM658
`Filed on behalf of Unified Patents, LLC
`By: Ellyar Y. Barazesh, Reg. No. 74,096
`
`Ashraf Fawzy, Reg. No. 67,914
`Unified Patents, LLC
`P.O. Box 53345
`Washington, DC 20009
`Tel: (925) 434-8754
`Email: ellyar@unifiedpatents.com
`Email: afawzy@unifiedpatents.com
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________________________________
`
`UNIFIED PATENTS, LLC
`Petitioner
`v.
`MEMORYWEB, LLC
`Patent Owner
`IPR2021-01413
`
`PETITION FOR INTER PARTES REVIEW OF
`U.S. PATENT 10,621,228
`
`
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`TABLE OF CONTENTS
`
`I.
`
`Page
`MANDATORY NOTICES ............................................................................. 1
`A.
`Real Party-in-Interest ............................................................................ 1
`B.
`Related Matters ..................................................................................... 1
`C.
`Counsel .................................................................................................. 2
`D.
`Service Information ............................................................................... 2
`CERTIFICATION OF GROUNDS FOR STANDING .................................. 2
`II.
`III. OVERVIEW OF CHALLENGE AND RELIEF REQUESTED .................... 2
`A.
`Prior Art Patents and Printed Publications ............................................ 3
`B.
`Grounds for Challenge .......................................................................... 4
`IV. U.S. PATENT 10,621,228 .............................................................................. 4
`A.
`Summary ............................................................................................... 4
`B.
`File History ........................................................................................... 8
`C.
`Level of Ordinary Skill in the Art ......................................................... 8
`CLAIM CONSTRUCTION ............................................................................ 8
`V.
`VI. SPECIFIC GROUNDS FOR PETITION ........................................................ 8
`A. Ground 1: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over
`Okamura and Flora ................................................................................ 9
`1.
`Okamura ..................................................................................... 9
`2.
`Flora .......................................................................................... 11
`3.
`Claim 1 ...................................................................................... 13
`4.
`Claim 2 ...................................................................................... 62
`5.
`Claim 3 ...................................................................................... 66
`6.
`Claim 4 ...................................................................................... 67
`7.
`Claim 5 ...................................................................................... 67
`8.
`Claim 6 ...................................................................................... 68
`9.
`Claim 7 ...................................................................................... 69
`
`
`
`
`i
`
`

`

`C.
`
`B.
`
`IPR2021-01413
`U.S. Patent 10,621,228
`Ground 2: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over
`Okamura, Flora, and Wagner .............................................................. 69
`1. Wagner ...................................................................................... 69
`2.
`Claim 1 ...................................................................................... 70
`3.
`Claims 2-7 ................................................................................. 85
`Ground 3: Claim 1-7 are obvious under 35 U.S.C. § 103(a) over
`Okamura, Flora, and Gilley ................................................................. 85
`1.
`Gilley ........................................................................................ 85
`2.
`Claim 1 ...................................................................................... 86
`3.
`Claims 2-7 ................................................................................. 94
`D. Ground 4: Claim 1-7 are obvious under 35 U.S.C. § 103(a) over
`Okamura, Flora, Wagner, and Gilley .................................................. 95
`1.
`Claim 1 ...................................................................................... 95
`2.
`Claims 2-7 ................................................................................. 96
`VII. THE BOARD SHOULD NOT EXERCISE DISCRETION TO DENY THE
`PETITION ..................................................................................................... 96
`VIII. CONCLUSION ............................................................................................. 97
`
`
`
`
`
`ii
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`I. MANDATORY NOTICES
`A. Real Party-in-Interest
`Pursuant to 37 C.F.R. § 42.8(b)(1), Unified Patents, LLC (“Unified” or
`
`“Petitioner”) certifies that Unified is the real party-in-interest and certifies that no
`
`other party exercised control or could exercise control over Unified’s participation
`
`in this proceeding, filing this petition, or conduct in any ensuing trial. In view of
`
`Worlds Inc. v. Bungie, Inc., 903 F.3d 1237, 1242-44 (Fed. Cir. 2018), Unified has
`
`submitted voluntary discovery to support its certification. See EX1017 (Declaration
`
`of Keven Jakel).
`
`B. Related Matters
`According to public records, U.S. Patent 10,621,228 (“the ’228 patent”
`
`(EX1001)) is owned by MemoryWeb, LLC (“Patent Owner”).
`
`As of the filing date of this petition, to the best of Petitioner’s knowledge, the
`
`’228 patent is or has been involved in the litigations listed below, of which Petitioner
`
`is not involved. See EX1018. To the best of Petitioner’s knowledge, the ’228 patent
`
`has not been involved in any post-grant proceedings.
`
`
`
`Case Caption
`
`1. MemoryWeb, LLC v. Samsung Electronics Co., Ltd. et al., 6:21-cv-00411
`
`(W.D.Tex)
`
`2. MemoryWeb, LLC v. Apple Inc., 6:21-cv-00531 (W.D.Tex)
`
`
`
`
`1
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`3. MyHeritage (USA), Inc. et. al. v. MemoryWeb, LLC, 1:21-cv-02666
`
`(N.D.Ill)
`
`
`
`C. Counsel
`Ellyar Y. Barazesh (Reg. No. 74,096) is lead counsel; Ashraf Fawzy (Reg.
`
`No. 67,914) is back-up counsel.
`
`D.
`Service Information
`Unified consents to electronic service at ellyar@unifiedpatents.com and
`
`afawzy@unifiedpatents.com. Petitioner can be reached at Unified Patents, LLC,
`
`P.O. Box 53345, Washington, DC 20009, Tel: (925) 434-8754.
`
`II. CERTIFICATION OF GROUNDS FOR STANDING
`Petitioner certifies pursuant to Rule 42.104(a) that the patent for which review
`
`is sought is available for inter partes review and Petitioner is not barred or estopped
`
`from requesting inter partes review challenging the patent claims on the grounds
`
`identified herein.
`
`III. OVERVIEW OF CHALLENGE AND RELIEF REQUESTED
`Pursuant to Rules 42.22(a)(1) and 42.104(b)(1)–(2), Petitioner challenges
`
`claims 1-7 (the “Challenged Claims”) of the ’228 patent based on the specific
`
`grounds identified below.
`
`
`
`
`2
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`A.
`Prior Art Patents and Printed Publications
`Each reference below was at least filed before June 9, 20111 and is prior art
`
`under at least 35 U.S.C. §§ 102(a), 102(b), and/or 102(e).
`
`1.
`
`U.S. Patent Application Publication 2011/0122153, filed October 20,
`
`2010 (“Okamura”) (EX1004).
`
`2.
`
`U.S. Patent 6,714,215, filed May 19, 2000, published March 30, 2004
`
`(“Flora”) (EX1005).
`
`3.
`
`U.S. Patent Application Publication 2011/0163971, filed May 27, 2010
`
`(“Wagner”) (EX1006).
`
`4.
`
`U.S. Patent Application Publication 2010/0172551, filed September 25,
`
`2009, published July 8, 2010 (“Gilley”) (EX1007).
`
`None of these references were cited during the ’228 patent’s prosecution.
`
`
`
`
`
`
`
`
`
`
`
`
`1 The ’228 patent claims priority to a continuation-in-part application filed June 9,
`
`2011. Petitioner assumes this priority date and applies pre-AIA statutory framework
`
`but makes no representations the Challenged Claims are entitled to this priority.
`
`
`
`
`3
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`B. Grounds for Challenge
`Ground 1: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over Okamura
`and Flora.
`Ground 2: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over Okamura,
`Flora, and Wagner.
`Ground 3: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over Okamura,
`Flora, and Gilley.
`Ground 4: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over Okamura,
`Flora, Wagner, and Gilley.
`
`This Petition, supported by Dr. Bederson’s declaration (EX1002),
`
`demonstrates a reasonable likelihood Petitioner will prevail with respect to
`
`cancellation of at least one challenged claim. 35 U.S.C. § 314(a). Petitioner
`
`respectfully requests institution on all challenged grounds. SAS Institute Inc. v.
`
`Iancu, 138 S. Ct. 1348 (2018).
`
`IV. U.S. PATENT 10,621,228
`A.
`Summary
`The ’228 patent describes managing digital files according to associated
`
`location using a “map view” and people present in files using a “people view.”
`
`EX1001, 22:59-23:11, 34:16-54, 29:41-64, Figs. 32, 34, 41.
`
`
`
`
`4
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`Figure 41 shows an exemplary map view including an “interactive map” that
`
`allows users to zoom-in and out. EX1001, 29:52-55; EX1011, ¶¶21, 40, 43; EX1012,
`
`¶¶22, 37, 40.
`
`EX1001, Fig. 41 (annotated)
`
`
`
`The map view includes thumbnails 0874 and 0875, where a “user can select the
`
`thumbnail to see all the Digital Files with the same location.” EX1001, 29:48-52,
`
`Fig. 41. By selecting a thumbnail, a “location view” (shown in Figure 34 below)
`
`
`
`
`5
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`associated with the selected thumbnail’s location is displayed. Id., 24:16-54, 29:48-
`
`52, Figs. 34, 41.
`
`EX1001, Fig. 34 (in part)
`
`The location view displays a location name 1632 and thumbnail(s) 1634 of file(s)
`
`captured at the same location. Id., 24:37-54.
`
`
`
`Figure 32 below shows an exemplary people view. EX1001, 22:59-64, Figs.
`
`32, 6.
`
`
`
`
`6
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`EX1001, Fig. 32 (annotated)
`
`
`
`Figure 32’s “multiple people” 1400 view displays thumbnail(s) of each person’s face
`
`featured in digital files and the person’s name. Id., 23:1-4. Selecting a thumbnail
`
`(e.g., thumbnail 1404 of “JC Jon Smith”) displays a “single people” view 1430 for
`
`the person of the selected thumbnail. Id., 23:12-49, Fig. 32. View 1430 includes
`
`thumbnails of each photo in set of photos 1446 associated with the person. Id.
`
`
`
`
`
`
`
`
`7
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`B.
`File History
`The examiner issued a first-action notice of allowance due to an examiner’s
`
`amendment amending claim 40, which issued as claim 1, to include limitations [1l]
`
`though [1p] (shown in Sections VI.A.3.l-VI.A.3.p below) relating to the “people
`
`view” discussed above. EX1003, pp. 355-364.
`
`C. Level of Ordinary Skill in the Art
`A POSITA as of the earliest claimed priority date of the ’228 patent would
`
`have had at least a bachelor’s degree in computer science, electrical engineering, or
`
`a related field, and at least two years of academic or industry experience in software
`
`development related to content management systems and user interfaces. More
`
`education can supplement practical experience and vice-versa. EX1002, ¶23.
`
`V. CLAIM CONSTRUCTION
`Petitioner submits no terms of the ’228 patent warrant construction beyond
`
`their ordinary and customary meaning. If Patent Owner, seeking to avoid the prior
`
`art, offers a specific construction or interpretation for terms, Petitioner reserves the
`
`right to respond.
`
`VI. SPECIFIC GROUNDS FOR PETITION
`Pursuant to Rule 42.104(b)(4)–(5), the following sections detail the grounds
`
`of unpatentability, limitations of the Challenged Claims, and how these claims were
`
`obvious in view of the prior art.
`
`
`
`
`8
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`A. Ground 1: Claims 1-7 are obvious under 35 U.S.C. § 103(a) over
`Okamura and Flora
`1. Okamura
`Okamura describes “managing contents such as image files” according to
`
`associated location using cluster map display area 414 and people present in content
`
`using face cluster image display area 431. EX1004, ¶¶0091, 0232-0247, Figs. 18-
`
`19, 21.
`
`Selecting PLACE tab 413 with cursor 419 causes a content playback
`
`application to display cluster map display area 414, shown by Figure 18 below,
`
`including cluster maps 417 each representing a set of content captured at different
`
`locations. EX1004, ¶¶0018, 0110, 0123, 0130, 0135-0143, 0213-0220, 0225, 0232-
`
`0254, 0261, 0265, 0272, Figs. 18-19, 22-23, 26.
`
`
`
`
`9
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`EX1004, Fig. 18 (annotated)
`
`Selecting FACE tab 412 with cursor 419 causes the content playback
`
`application to display face cluster image display area 431, shown by Figure 21
`
`below, including thumbnail images 432 of each person’s face (including a first
`
`person and a second person) present in content, where each thumbnail 432 represents
`
`a set of content featuring the displayed person. EX1004, ¶¶0099, 0110, 0139, 0246-
`
`0248, Fig. 21.
`
`
`
`
`10
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`
`
`EX1004, Fig. 21 (annotated)
`
`Okamura is analogous art to the ’228 patent; it is within the same field of
`
`endeavor as the ’228 patent (managing digital files) and reasonably pertinent to at
`
`least one problem concerning the ’228 patent’s inventors (how to organize and
`
`provide the viewing of digital files to enhance user experience). EX1001, 1:40-47,
`
`1:54-67, 2:55-59; EX1004, ¶¶0091, 0232-0247; EX1002, ¶65.
`
`2.
`Flora
`Flora organizes content according to location, describing “an interactive map
`
`that allows users to display different items of visual and/or audio media
`11
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`corresponding to a location” on the map. EX1005, 1:7-11, 11:15-17. Figure 3 shows
`
`graphical user interface (GUI) 40 including “scalable geographic map” 46. Id., 5:65-
`
`6:8, 6:66-67, 7:9-11, Fig. 3.
`
`EX1005, Fig. 3
`
`12
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`When a user moves cursor 56 to a position on map 46, icons 58 and 59
`
`(including a first icon and second icon) “representing images [] associated with the
`
`locations proximate to the cursor 56” are displayed on the map. EX1005, 7:1-13.
`
`The icons are “reduced-pixel” or “‘thumbnail’ versions of the available media items”
`
`at each location. Id., 6:29-32, 7:1-13, Figs. 2, 3. When a user selects an icon, a media
`
`viewer 64 “is opened and displays a full-size image,” shown as media item 62, which
`
`is the full-size image of the media “represented by the icon.” Id., 7:23-34. Media
`
`viewer 64 also shows a set of “all other media items associated with the map
`
`location” represented by icons 66 in scrolling list 68. Id., 7:34-42.
`
`Flora is analogous art to the ’228 patent; it is within the same field of endeavor
`
`as the ’228 patent (mediums that allow users to view digital content) and reasonably
`
`pertinent to at least one problem concerning the ’228 patent’s inventors (how to
`
`provide access to and viewing of digital content to enhance user experience).
`
`EX1001, Abstract, 1:40-47, 1:54-67, 2:55-59; EX1005, 1:7-11, 2:2-37, 3:22-46,
`
`11:13-32; EX1002, ¶68.
`
`3.
`
`Claim 1
`a)
`[1a-preamble] “A method comprising:”
`To the extent limiting, Okamura discloses or at least renders obvious the
`
`preamble because it describes a method of using a “content playback application” to
`
`display media content. EX1004, ¶¶0232-0248, Figs. 17-21.
`
`
`
`
`13
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`b)
`
`[1b] “responsive to a first input, causing a map view to
`be displayed on an interface,”
`Okamura discloses or at least renders obvious limitation [1b]. The content
`
`playback application displays a screen having FACE tab 412 and PLACE tab 413,
`
`where each tab is selectable by depressing cursor 419 on a desired tab. EX1004,
`
`¶¶0091, 0232-0247, Figs. 17-21.
`
`EX1004, Fig. 18 (annotated)
`
` “When the ‘PLACE’ tab 413 is depressed using the cursor 419 by a user operation”
`
`(responsive to a first input), cluster map display area 414 is displayed (causing a
`
`
`
`
`14
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`map view to be displayed) on a display interface of the content playback application
`
`(on an interface). Id., ¶¶0232-0241, Figs. 17-19.2 Cluster map display area 414 is a
`
`map view because it has cluster maps 417, which are geographic areas of a map
`
`showing where media content (e.g., digital images) have been captured. Id.; see also
`
`id., ¶¶0018, 0110, 0130-0135 (describing a displayed cluster map as “a map image
`
`formed by a map included in a circle corresponding to each cluster”), 0139, 0213,
`
`0275-0281, Figs. 27A-27B (discussing a modification to cluster map display area
`
`414 where map display area 282 is displayed); EX1002, ¶¶69-70.
`
`A POSITA would have understood Okamura’s content playback application
`
`discloses or at least renders obvious the claimed interface because the application
`
`provides a display medium for displaying cluster map display area 414, event cluster
`
`image display area 421, and face cluster image display area 431, shown in Figures
`
`18 through 21. Id., ¶¶0232-0248, Figs. 17-21. This understanding is corroborated by
`
`Wagner (EX1006), which describes an application’s graphical user interface
`
`provides a display medium that is updated to display new user interface objects on
`
`
`2 If interface is interpreted to include display hardware, Okamura discloses or at least
`
`renders obvious this interpretation because cluster map display area 414 is displayed
`
`on a physical display of apparatus 100 in response to selection of PLACE tab 413.
`
`EX1004, ¶¶0091, 0232-0247, Figs. 1, 17-19.
`
`
`
`
`15
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`its interface and change the position of displayed objects on its interface. EX1006,
`
`¶¶0004, 0115, 0120, 0130, 0282; EX1002, ¶¶71-73.
`
`A POSITA would have further understood Okamura at least renders obvious
`
`having a single interface window on which the claimed map view and the claimed
`
`people view (see Section VI.A.3.l) are displayed. This is because Okamura’s tabs
`
`411 through 413 for changing the display of how content is organized (i.e., by event,
`
`face, or place), as well as left and right control buttons 415 and 416, are displayed
`
`regardless of the currently displayed screen showing how content is organized, and
`
`the only item changing between screens 410 through 430 is cluster map display area
`
`414, event cluster image display area 421, and face cluster image display area 431,
`
`identified below in brown. EX1004, ¶¶0232-0248, Figs. 17-21.
`
`
`
`
`16
`
`
`
`

`

`
`
`IPR2021-01413
`U.S. Patent 10,621,228
`
`
`
`
`
`EX1004, Figs. 18, 19, 21 (annotated)
`
`Thus, since most displayed components (411-413, 415-416) do not change between
`
`screens 410 through 430, a POSITA would have understood Okamura renders
`17
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`obvious having a single interface window screen displaying tabs 411 through 413
`
`and left and right control buttons 415 and 416, where cluster map display area 414,
`
`event cluster image display area 421, or face cluster image display area 431 is
`
`displayed on the interface according to selection of a tab 411 through 413. Id. In
`
`such a case, selection of PLACE tab 413 causes the single interface to display cluster
`
`map display area 414 (map view) on the interface, and selection of FACE tab 412
`
`causes the single interface to display face cluster image display area 431 (people
`
`view) on the interface. Id.; Section VI.A.3.l; EX1002, ¶¶74-75.
`
`c)
`[1c] “the map view including: (i) an interactive map;”
`Okamura alone, or Okamura and Flora, discloses or at least renders obvious
`
`limitation [1c].
`
`Cluster map display area 414 (the map view) includes cluster maps 417
`
`arranged in a 3x5 matrix (an interactive map). EX1004, ¶¶0234-0241, Figs. 18-19.
`
`
`
`
`18
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`EX1004, Fig. 18
`
`The arranged cluster maps 417 form an interactive map because they show
`
`geographic map areas where content has been captured, and when mouse cursor 419
`
`is “placed over a cluster map 417 by a user operation on the index screen 410,” the
`
`“color of the cluster map 417 is changed” and “pieces of information 418 related to
`
`the cluster map 417” are displayed, making the map arrangement interactive.
`
`EX1004, ¶¶0018, 0110, 0130-0135, 0139, 0213, 0232-0248, 0275-0281, Figs. 18-
`
`19; EX1002, ¶76. Okamura therefore discloses or at least renders obvious limitation
`
`[1c].
`
`
`
`
`19
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`If Okamura is not found to disclose or render obvious an interactive map,
`
`Flora discloses or at least renders obvious this element.
`
`Flora describes a GUI having scalable geographic map 46 (an interactive
`
`map) with “icons [] (or thumbnail versions)” of “media items,” such as icons 58 and
`
`59, positioned at various map locations of the map. EX1005, 5:65-6:11, 6:66-7:42,
`
`Figs. 2, 3 (parenthetical of quote in original).
`
`
`
`
`20
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`EX1005, Fig. 3
`
`
`
`Flora’s scalable geographic map 46 with icons 58 and 59 is an interactive map
`
`because (1) it is “scalable and can show fine levels of geography, such as individual
`
`cities and towns,” where the map can be scaled upwards (e.g., zoomed out) or
`
`downwards (e.g., zoomed in) and categories of media displayed on the map via icons
`21
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`can be filtered; (2) a user can “click” an icon (e.g., using a mouse ), interacting with
`
`the map to “obtain direct access to the content of the associated media item;” and (3)
`
`if a user’s mouse cursor is moved to a new map location, “icons [] displayed
`
`proximate to the old location will eventually disappear or fade away” and, if present,
`
`“new icons will appear proximate to the new cursor position.” Id., 6:11-7:42, Figs.
`
`2, 3; id., 2:4-37 (describing Flora’s GUI displaying the map is “interactive” and
`
`allows a user to “interact with a geographic map”). EX1002, ¶¶77-80.
`
`Motivation to Combine Okamura and Flora
`
`A POSITA would have been motivated to combine Okamura and Flora such
`
`that when organizing content according to location, Okamura’s cluster map display
`
`area 414 displays content as taught by Flora’s geographic map 46 and media viewer
`
`64, where Okamura’s content is indicated at various locations on the map by Flora’s
`
`icons 58 and 59 and Flora’s media viewer 64 provides a window for viewing
`
`Okamura’s content associated with the locations, shown by icons 66. EX1002, ¶81.
`
`
`
`
`22
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`EX1004, Fig. 18
`
`
`
`
`23
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`EX1005, Fig. 3
`
`
`
`A POSITA would have been motivated to combine Okamura and Flora in this
`
`manner because the combination enhances how Okamura displays content
`
`associated with various locations, using Flora’s scalable geographic map with icons
`
`and media viewer, improving user experience. EX1002, ¶82. Combining the
`24
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`teachings provides an interface displaying locations where Okamura’s content has
`
`been captured on a scalable map and allows a user to view the content captured at a
`
`specific location using a media viewer, which provides a user with improved
`
`awareness regarding locations associated with content. Id. Consistent with the
`
`teaching, suggestion, or motivation (TSM) test, Okamura explains its displayed
`
`cluster maps help a user “easily grasp[]” areas where content has been captured and
`
`allow a user to “easily grasp the distribution of the location of generation of
`
`contents” included in the cluster (EX1004, ¶¶0213-0215, 0222, 0272; see also
`
`¶¶0018, 0110, 0123, 0130, 0139); enhancing Okamura with Flora’s discussed
`
`teachings furthers these goals because Flora’s system improves how a user views
`
`content organized by location, “allow[ing] a user to interface with” a map displaying
`
`content and “facilitate[][ing] access to content associated with locations of the
`
`electronic map.” EX1005, 1:56-55, 2:2-9, 3:22-46; EX1002, ¶¶82.
`
`A POSITA would have combined Okamura and Flora in the above manner
`
`using known programming techniques, adjusting the software of Okamura’s content
`
`playback application such that cluster map display area 414 includes Flora’s
`
`teachings. EX1004, ¶¶0232-0241. In particular, the software of Okamura’s cluster
`
`display control section 180 would have been adjusted using such techniques so
`
`cluster map display area 414 is modified to display content as taught by Flora’s
`
`scalable map 46 with icons 58 and 59 and media viewer 64, where icons 58 and 59
`25
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`represent locations on the scalable map 46 where Okamura’s content was captured
`
`and the media viewer 64, displaying Okamura’s content captured at specific
`
`locations via icons 66, is accessed by selecting respective icons 58 and 59. EX1004,
`
`¶¶0018, 0091-0093, 0103-0106, 0110, 0123, 0130, 0135-0143, 0213-0220, 0225,
`
`0232-0241, 0267; EX1005, 3:40-45, 7:3-52, Fig. 3; EX1002, ¶83. Okamura and
`
`Flora’s teachings would have performed the same function of displaying content
`
`according to location whether separate or combined. Id.
`
`A POSITA would have recognized the combination’s results would have been
`
`predictable: using Flora’s geographic map 46 with icons 58 and 59 and media viewer
`
`64 to organize and display Okamura’s content on cluster map display area 414
`
`according to location associated with the content. EX1004, ¶¶0018, 0091-0093,
`
`0103-0106, 0110, 0123, 0130, 0135-0143, 0213-0220, 0225, 0232-0241, 0267;
`
`EX1005, 3:40-45, 6:66-7:52, Fig. 3; EX1002, ¶84. This is because Okamura
`
`organizes and displays content based on the location at which content was captured,
`
`and Flora organizes and displays content according to its associated location using
`
`an interactive map. Id. A POSITA would have had a reasonable expectation of
`
`success in combining Okamura and Flora’s teachings because the combination’s
`
`results would have been predictable as discussed above, and because both references
`
`relate to organizing content according to location associated with content. Id.
`
`Combining their teachings would have been routine to a POSITA due to this overlap
`26
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`and the simple software modifications to adjust cluster map display area 414 to
`
`include Flora’s teachings. Id.
`
`This analysis demonstrates at least the following motivations to combine:
`
`• Combining prior art elements (Okamura’s cluster map display area 414;
`
`Flora’s geographic map 46 with icons 58 and 59 and media viewer 64)
`
`according to known methods (known programming techniques to adjust
`
`the software of Okamura’s content playback application) to yield
`
`predictable results (using Flora’s geographic map 46 with icons 58 and
`
`59 and media viewer 64 to organize and display Okamura’s content on
`
`cluster map display area 414 according to location associated with the
`
`content).
`
`• Simple substitution of one known element (Flora’s geographic map 46
`
`with icons 58 and 59 and media viewer 64) for another (Okamura’s
`
`cluster map display area 414) to obtain predictable results (using
`
`Flora’s geographic map 46 with icons 58 and 59 and media viewer 64
`
`to organize and display Okamura’s content according to location
`
`associated with the content).
`
`MPEP 2143; KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 415-421 (2007); EX1002,
`
`¶¶85-86.
`
`
`
`
`27
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`d)
`
`[1d] “[the map view including:] (ii) a first location
`selectable thumbnail image at a first location on the
`interactive map; and”
`Okamura and Flora disclose or at least render obvious limitation [1d]. In the
`
`combination, Okamura’s cluster map display area 414 (map view) includes the
`
`display of content as taught by Flora’s scalable geographic map 46 having icons 58
`
`and 59 representing the locations where Okamura’s content was captured. Section
`
`VI.A.3.c. Flora describes a first of the multiple selectable icons 58 and 59 (a first
`
`location selectable thumbnail image) displayed at a first location (at a first location)
`
`on scalable geographic map 46 (on the interactive map), where each icon 58 and 59
`
`(including the first) is a selectable thumbnail image because they are each an “icon[]
`
`[] (or thumbnail version[])” of a “media item[].” EX1005, 5:65-6:11, 7:4-13, Figs.
`
`2, 3 (parenthetical in original); Section VI.A.3.c. Flora states icons 58 and 59 can be
`
`presented for each specific location on map 46 in a “reduced-pixel or ‘thumbnail’
`
`format.” Id. Each icon 58 and 59 is selectable by a “user input, such as a mouse
`
`click,” which opens media viewer window 64. Id., 7:4-42, Fig. 3.
`
`
`
`
`28
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`
`
`EX1005, Fig. 3 (annotated)
`
`e)
`
`[1e] “[the map view including:] (iii) a second location
`selectable thumbnail image at a second location on the
`interactive map;”
`Okamura and Flora disclose or at least render obvious limitation [1e]. In the
`
`combination, Okamura’s cluster map display area 414 (map view) includes the
`
`
`
`
`29
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`display of content as taught by Flora, which describes a second of the multiple
`
`selectable icons 58 and 59 (a second location selectable thumbnail image) displayed
`
`at a second location (at a second location) different from the first location on
`
`scalable map 46 (on the interactive map). EX1005, 5:65-6:11, 7:4-13, 7:23-42, Figs.
`
`2, 3; Sections VI.A.3.c-VI.A.3.d. Each icon 58 and 59 (including the second) is a
`
`selectable thumbnail image as discussed in Section VI.A.3.d.
`
`EX1005, Fig. 3 (annotated)
`30
`
`
`
`
`
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`f)
`
`[1f] “responsive to an input that is indicative of a
`selection of the first location selectable thumbnail image,
`causing a first location view to be displayed on the
`interface,
`Okamura and Flora disclose or at least render obvious limitation [1f]. In the
`
`combination, Flora describes responsive to a “user input, such as a mouse click”
`
`selecting the first of multiple selectable icons 58 and 59 (responsive to an input that
`
`is indicative of a selection of the first location selectable thumbnail image), a media
`
`viewer 64 for the selected first icon opens and provides access to a media item 62
`
`represented by the first icon and “all other media items associated with the map
`
`location” of media item 62 (causing a first location view to be displayed on the
`
`interface). EX1005, 5:65-6:11, 7:4-42, Figs. 2, 3; Sections VI.A.3.c-VI.A.3.d.
`
`Media viewer 64 for the first icon is a first location view because it provides a display
`
`window for viewing each media item for a map location associated with the first
`
`icon; this is just like the ’228 patent’s description of a location view showing content
`
`associated with a selected location. Id.; EX1001, 24:37-51, Fig. 34. Media viewer
`
`64 is displayed on the interface because it is displayed on GUI 40 and, combined
`
`with Okamura, is displayed using cluster map display area 414 that is itself displayed
`
`on the interface provided by Okamura’s content playback application; content
`
`playback application is further displayed on a physical display of apparatus 100.
`
`EX1005, 7:8-42, Fig. 3; Sections VI.A.3.b-VI.A.3.d; EX1002, ¶¶87-89.
`
`
`
`
`31
`
`

`

`noe
`
`EX1005, Fig. 3 (annotated)
`
`as
`
`‘Spasnics} “hpisb
`
`
`
`
`IPR2021-01413
`U.S. Patent 10,621,228
`
`
`
`
`
`
`
`
`
`
`
`32
`
`

`

`IPR2021-01413
`U.S. Patent 10,621,228
`
`g)
`
`[1g] the first location view including (i) a first location
`name associated with the first location and (ii) a
`representation of at least a portion of one digital file in a
`first set of digital files,
`Okamura and Flora disclose or at least render obvious limitation [1g]. Flora
`
`describes media viewer 64 for the selected first icon (the first location view) includes
`
`selectable icons 66 representing “all other media items associated with the map
`
`location” of the first icon (i.e., Okamura’s content captured at the location of the
`
`icon). EX1005, 7:23-42, Fig. 3; Sections VI.A.3.c, VI.A.3.f. In view of the combined
`
`teaching, Flora’s selectable icons 66 are a representation of each of Okamura’s