`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`
`FACEBOOK, INC., INSTAGRAM, LLC, and WHATSAPP INC.,
`Petitioners
`
`v.
`
`BLACKBERRY LIMITED
`Patent Owner
`
`
`
`Case IPR2019-00528
`U.S. Patent No. 8,279,173 B2
`
`
`PETITIONER RESPONSE TO PATENT OWNER’S
`CONDITIONAL MOTION TO AMEND UNDER 37 C.F.R. § 42.1211
`
`
`
`
`
`
`
`
`
`1 An identical response is filed in IPR2019-00516.
`
`
`
`
`
`
`
`
`Table of Contents
`
`
`Page
`
`
`INTRODUCTION .......................................................................................... 1
`I.
`II. ARGUMENT .................................................................................................. 1
`A.
`Zuckerberg discloses “displaying a photograph comprising at
`least one subject or object; receiving a user selection of a
`location in the photograph corresponding to the at least one
`subject or object” and “associating at least one of the tags in the
`tag list with the at least one subject or object.” .................................... 2
`Rothmuller and Plotkin disclose and render obvious “wherein
`the tags in the tag list are displayed in a vertical list, and
`wherein: a first tag from a first tag source is displayed above a
`second tag from a second tag source; and the second tag from
`the second tag source is displayed above a third tag from the
`first tag source” .................................................................................... 5
`The Additional Teachings of Zuckerberg, Rothmuller and
`Plotkin Render the Substitute Claims Obvious When Applied to
`the Existing Instituted Grounds .......................................................... 14
`III. CONCLUSION ............................................................................................. 22
`
`B.
`
`C.
`
`
`
`
`
`-i-
`
`
`
`
`
`
`
`I.
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`INTRODUCTION
`Patent Owner filed identical contingent motions to amend in IPR2019-00516
`
`and IPR2019-00528, urging the Board to allow substitute claims 21-32 in the event
`
`the challenged claims are found unpatentable based on the instituted grounds. But
`
`the new features recited in substitute claims 21-32 do not provide any meaningful
`
`distinction over the prior art, and are obvious for the same reasons as the original
`
`challenged claims. The new limitations added by substitute claims 21-32 are so
`
`insubstantial, in fact, that they are disclosed and rendered obvious by the prior art
`
`already of record in these IPRs. Patent Owner’s motion should be denied.
`
`II. ARGUMENT
`Substitute claims 21-32 would replace claims 1, 2, 4, 6-7, 10, 12-14, 16, and
`
`18, the challenged claims. These substitute claims introduce new limitations only
`
`with respect to independent claims 21, 25, and 29, which would replace original
`
`independent claims 1, 7, and 13, respectively. The remaining substitute dependent
`
`claims are unchanged from their corresponding original claims other than modifying
`
`the claim dependency. (Motion at 2.)
`
`Generally speaking, the new limitations would add three additional features
`
`into the independent claims: (1) display of a photograph and user selection of a
`
`subject or object in the photograph; (2) display of a “vertical” tag list showing at
`
`least three tags from two tag sources in a particular arrangement; and (3) associating
`
`
`
`
`
`1
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`at least one of the tags in the list with the selected subject or object. But as explained
`
`below, each of these features is disclosed by the prior art.
`
`A. Zuckerberg discloses “displaying a photograph comprising at least
`one subject or object; receiving a user selection of a location in the
`photograph corresponding to the at least one subject or object” and
`“associating at least one of the tags in the tag list with the at least
`one subject or object.”
`These claim limitations, which correspond to features (1) and (3) in the brief
`
`summary above, are readily disclosed by Zuckerberg as shown in Figure 5:
`
`
`
`
`
`
`
`2
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`(Zuckerberg, Fig. 5 (partial figure; annotations added); Ex. 1027, ¶¶6-8.) As shown
`
`highlighted in yellow in Figure 5 above, the system instructs the user to “[c]lick on
`
`people in the photo to tag them.” (Zuckerberg, Fig. 5.) As shown in the red box,
`
`the user has selected a region 520 corresponding to an individual’s face. (Id.)2
`
`Zuckerberg discloses “displaying a photograph comprising at least one
`
`subject or object,” as shown by the display of digital image 362 in Figure 5 above.
`
`The photograph has “at least one subject or object,” in this case a human being.
`
`Zuckerberg also discloses “receiving a user selection of a location in the
`
`photograph corresponding to the at least one subject or object.” As disclosed in
`
`Figure 5 above and further discussed in Zuckerberg’s textual description, “[t]he
`
`region selection component 410 is configured to receive input from a user 101,” and
`
`“select a region (e.g., a selected region 520) within a digital image 362 according to
`
`the input.” (Zuckerberg, 8:15-22.) The “location in the photograph” in
`
`Zuckerberg thus corresponds to a point in the digital image 362 selected by the user
`
`(such as the cross-hatch shown in Figure 5), which can correspond to a subject or
`
`object in the photo. Zuckerberg explains that in some embodiments, “the user 101
`
`
`2 Zuckerberg is Exhibit 1003 in IPR2019-00516 but was not included in IPR2019-
`
`00528. Accordingly, Zuckerberg will be added as previously-unused Exhibit 1003
`
`in IPR2019-00528 upon the filing of this response.
`
`
`
`
`
`3
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`moves a cursor 530 on the user device 110 to a point in the digital image 362,” and
`
`then “clicks on the point and the region selection component 410 places a border
`
`525 around the selected region 520.” (Zuckerberg, 8:22-27.) The particular photo
`
`shown in Figure 5 shows that the point corresponds to a human being (“the at least
`
`one subject or object”), which is consistent with the instruction in Figure 5: “Click
`
`on people in the photo to tag them.” (Zuckerberg, Fig. 5.) It would have been
`
`obvious to a person of ordinary skill in the art the user in Zuckerberg could have
`
`selected a point in any photo corresponding to a subject or object. (Ex. 1027, ¶7.)
`
`Finally, Zuckerberg discloses “associating at least one of the tags in the tag
`
`list with the at least one subject or object,” recited in the substitute claims.
`
`Zuckerberg discloses the tag component 340 discussed above can, after receiving a
`
`selection of a tag in the tag list, associate that tag with the selected region in the
`
`photo. (Zuckerberg, e.g., 7:54-56 (“The tag component 340 is configured to select
`
`a region in the image and associate text with the region.”) (underlining added).)
`
`“Clicking on any of the previously used tags may associate the tag with the selected
`
`region 520.” (Zuckerberg, 8:66-9:1 (underlining added).) As explained previously,
`
`the selection region 520 in Zuckerberg can correspond to a subject or object in the
`
`photo. Zuckerberg therefore discloses this limitation. (Ex. 1027, ¶8.)
`
`
`
`
`
`4
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`
`U.S. Patent No. 8,279,173
`B. Rothmuller and Plotkin disclose and render obvious “wherein the
`tags in the tag list are displayed in a vertical list, and wherein: a
`first tag from a first tag source is displayed above a second tag from
`a second tag source; and the second tag from the second tag source
`is displayed above a third tag from the first tag source”
`Rothmuller and Plotkin disclose and render obvious these claim limitations as
`
`discussed in detail below. But as a more threshold issue, it is questionable whether
`
`the claimed visual and vertical arrangement of the first, second, and third tags is even
`
`a patentable feature. This feature is discussed nowhere in the textual description of
`
`the ’173 patent. Patent Owner’s sole support for these claim limitations under 35
`
`U.S.C. § 112 is the fortuity that Figure 4B of the ’173 patent shows a vertical tag list
`
`that happens to include three tags in this particular arrangement, i.e. a first tag from
`
`a first source, a second tag from a second source, and a third tag from the first source.
`
`(Motion at 4 (quoting and annotating ’173, Fig. 4B).) The particular sequence of
`
`tags in Figure 4B appears to be little more than the happenstance of the particular
`
`screenshot used to create that figure. Nothing in the ’173 patent suggests that this
`
`particular sequence or ordering of tags was in any way a point of novelty over the
`
`prior art or significant in any way to the invention. (Ex. 1027, ¶25.)
`
`Rothmuller and Plotkin specifically disclose this same sequence as discussed
`
`below, but even without those disclosures, the claimed arrangement would have
`
`been plainly obvious to a person of ordinary skill in the art. A skilled artisan would
`
`have appreciated that different tagging datasets will generate different lists, and it
`
`
`
`
`
`5
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`only takes the right combination and sequence of tag data to generate a list with the
`
`display arrangement recited in these new limitations. (Ex. 1027, ¶¶24-25.) It would
`
`have been obvious that Rothmuller and Plotkin could display tags in this sequence
`
`if it was consistent with the underlying tag data. (Id., ¶24.)
`
`1.
`Rothmuller (Exs. 1004 and 1005)
`Figure 9A of the Rothmuller Provisional discloses a “tag list” in the form of
`
`a list of recently used tags, with each tag shown with a respective tag type indicator
`
`icon. (Ex. 1002 in IPR2019-00516, ¶¶88, 89, 171-172, 197-198; Ex. 1002 in
`
`IPR2019-00528, ¶¶87-89.) The annotated used tag list from Rothmuller Figure 9A,
`
`below, shows this list and how it applies to the substitute claims:
`
`(Rothmuller Provisional, Fig. 9A (partial figure; highlighting and annotations
`
`added); Ex. 1027, ¶9.) Figure 9A above discloses or suggests displaying tags in a
`
`tag list “wherein the tags in the tag list are displayed in a vertical list,” as recited
`
`
`
`in the substitute claims.
`
`
`
`
`6
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`The three tags shown in highlighting above, i.e. Landscapes, Lori, and
`
`Animals, are shown listed one after another in a vertical arrangement. As shown,
`
`the “first tag” (“Landscapes”) and the “third tag” (“Animals”) both come from “a
`
`first tag source,” as indicated by the common tag type indicator icon (
`
` ) next to
`
`them. The “second tag” (“Lori”) comes from a second and different tag source as
`
`reflected by the different indicator icon (
`
` ).
`
`More specifically, an example of the claimed “a first tag from a first tag
`
`source” is shown in “Landscapes” tag shown in yellow, which is shown alongside a
`
`tag type indicator icon (
`
` ). For all of the grounds identified in IPR2019-00516
`
`and IPR2019-00528, the “first tag source” corresponds to the collection of tags
`
`associated with the displayed tag type indicator icon (e.g.,
`
` ). More information
`
`about how the teachings of Rothmuller apply in combination with the Zuckerberg
`
`and MacLaurin “tag sources” is provided in Part II.C below.
`
`With respect to the “vertical list” limitation, Figure 9A of Rothmuller shows
`
`tags in the recently used tag list arranged in two adjacent vertical columns, rather
`
`than a single vertical list. But this does not impact whether Rothmuller discloses the
`
`claimed “vertical” tag list. This is because the claim only requires “a list including
`
`tags… wherein the tags in the tag list are displayed in a vertical list,” which on
`
`its face merely requires a single vertical list with more than one tag. The claim does
`
`not require that the claimed “list” include every tag that might be presented on the
`
`
`
`
`
`7
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`display. The listing on the right side of the recently used tags in Figure 9A, therefore,
`
`satisfies the claim requirement and can independently be applied to the “tag list”
`
`limitation. Because the substitute claims are “comprising” claims, moreover, they
`
`do not preclude the existence of additional elements not recited in the claims.
`
`Accordingly, the presence of other tags on the left side of the recently used tag area
`
`in Figure 9A does not change this result. (Ex. 1027, ¶13.)
`
`But even assuming the claim required a single vertical list containing all
`
`displayed tags (which it does not), this would have been obvious to a person of
`
`ordinary skill in the art based on the disclosures of Rothmuller, for two reasons.
`
`First, it would also have been obvious to a person of ordinary skill in the art
`
`that the recently used tag area in Figure 9A could have included fewer tags such that
`
`all of the tags would be listed in a single vertical list. (Ex. 1027, ¶14.) For example,
`
`suppose the user had only three recently used tags instead of the six shown in Figure
`
`9A; in this scenario, all of the recently used tags would have appeared in a single
`
`vertical list on the left side of the recently used tag area of Figure 9A. It would have
`
`been obvious to a person of ordinary skill in the art that, at some earlier point during
`
`the use of the Rothmuller system – when fewer tags had been used – the recently
`
`used tag area in Figure 9A could have included fewer tags such that all of them
`
`would fit in a single vertical list. (Id.) In other words, it would have been obvious
`
`to a person of ordinary skill in the art that Rothmuller could display a single column
`
`
`
`
`
`8
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`list of tags, even if the particular scenario captured by Figure 9A happens to show
`
`two columns of tags.
`
`More fundamentally, arranging those tags in the form of a single vertical list
`
`(as opposed to two lists) would have been obvious to a person of ordinary skill in
`
`the art. The ability to display icons or other objects in a single column, or multiple
`
`columns, was basic knowledge of persons of ordinary skill in the art and even casual
`
`users. (Ex. 1027, ¶15.) For example, the popular Microsoft Windows operating
`
`system since at least the 1990s had the ability (through the ubiquitous Windows
`
`“View” menu) to list items in a window as a single vertical list or in a multi-column
`
`arrangement similar to Figure 9A. (Id.) This is illustrated in Fred Davis, The
`
`Windows 95 Bible (1996) [Ex. 1028]; for example:
`
`
`
`
`
`9
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`
`
`(Ex. 1028, at 034-036 (also quoted in Ex. 1027, ¶15).)
`
`As these screenshots from the ubiquitous 1990s Microsoft Windows operating
`
`system demonstrate, a person of ordinary skill in the art would have understood that
`
`displaying the tags in Rothmuller in the form of a single vertical list (similar to the
`
`“Details” view of Figure 3.17 in Davis shown above) was one of a finite number of
`
`techniques for arranging or listing items on the screen. (Ex. 1027, ¶16.) The use of
`
`a single vertical list, or adjacent vertical columns as shown in Figure 9A (and in the
`
`“List” view in Figure 3.18 of Davis shown above), present obvious alternatives with
`
`predictable tradeoffs; a single vertical list, for example, may show fewer items at
`
`one time (without scrolling) but it can show more information about each item and
`
`accommodate longer item names. (Id.) A person of ordinary skill in the art would
`
`have been motivated to arrange the tag list in Rothmuller as a vertical list, for
`
`example, to accommodate longer tag names that might be truncated or cut off under
`
`the multiple column arrangement shown in Figure 9A. (Id.) In any case, the ability
`
`to arrange items (such as the recently used tags of Figure 9A) in a single vertical list
`
`
`
`
`
`10
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`would have been regarded by a skilled artisan as so basic and well-understood that
`
`would not even remotely have been perceived as a patentable feature. (Id.)
`
`Continuing to the next part of this claim limitation, Figure 9A of the
`
`Rothmuller Provisional discloses or renders obvious the claimed “first tag… above
`
`a second tag from a second tag source.” An example of the claimed “second tag”
`
`is the tag “Lori” shown in blue, which sits below the “Landscapes” tag and has a
`
`different tag type indicator icon (
`
` ). The “second tag source” corresponds to the
`
`group or collection of tags associated with the displayed tag type indicator icon (
`
` ).
`
`Finally, Figure 9A of the Rothmuller Provisional discloses or renders obvious
`
`that “the second tag from the second tag source is displayed above a third tag
`
`from the first tag source.” An example is the tag “Animals” above shown in
`
`yellow, which is shown alongside the same tag type indicator icon as the “first tag”
`
`(
`
` ), indicating that it also comes from “the first tag source.”
`
`2.
`Plotkin (Ex. 1008)
`Plotkin discloses these limitations for many of the same reasons as Rothmuller
`
`discussed extensively above. Plotkin discloses a “tag list” in the form of a list of
`
`tags used to import tags, with each tag shown with a respective tag type indicator
`
`icon. (Ex. 1002 in IPR2019-00516, ¶¶54, 110; Ex. 1002 in IPR2019-00528, ¶¶55,
`
`107.) The annotated tag list of Plotkin, below, shows how it applies to the claims:
`
`
`
`
`
`11
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`
`
`(Plotkin, p.328 (red annotations added; partial figure).)
`
`The figure above from Plotkin discloses or suggests displaying tags in a tag
`
`list “wherein the tags in the tag list are displayed in a vertical list,” as recited.
`
`The three tags shown alongside the red annotations above, i.e. “Birds,” “David P,”
`
`and “Seals/Sealions,” are shown listed in a vertical arrangement. As shown, the
`
`“first tag” (“Birds”) and the “third tag” (“Seals/Sealions”) both come from the “first
`
`tag source,” as indicated by the common tag type indicator icon (in this case a bird).
`
`The “second tag” (“David P”), as shown, comes from a second and different tag
`
`source, as indicated by a different tag type indicator icon (in this case an icon
`
`showing two people).
`
`The arrangement shown above discloses and renders obvious that the first tag
`
`from the first tag source “is displayed above a second tag from a second tag
`
`source,” which in turn “is displayed above a third tag from the first tag source.”
`
`Petitioner notes that in the exemplary tag list in Plotkin, the “second tag” (e.g.
`
`12
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`“David P”) is above the “third tag” (e.g. “Seals/Sealions”) in the list, but not
`
`immediately above it as there is an intermediate tag (e.g. “Russ H”) between them.
`
`But the claim does not require that the three recited tags be displayed one-after-
`
`another with nothing in between them; it merely requires “a first tag” displayed
`
`“above a second tag,” which “is displayed above a third tag,” which the tag list from
`
`Plotkin above discloses. Because the substitute claims are “comprising” claims, they
`
`do not preclude the existence of other displayed tags.
`
`But even if the claim were interpreted to require that the first, second, and
`
`third tags be displayed vertically in sequence, with no intermediate tags between
`
`them, this would have been obvious to a person of ordinary skill in the art. (Ex.
`
`1027, ¶16.) The tag list in Plotkin is alphabetically organized, and it would have
`
`been obvious to a person of ordinary skill in the art that the sequence of tag type
`
`indicator icons in the list (and thus tag sources) was simply the happenstance of the
`
`particular tags that were being imported as captured in that screenshot. (Id., ¶23.) It
`
`would have been obvious to a person of ordinary skill in the art that the Plotkin user
`
`interface could have generated different arrangements meeting even this narrower
`
`view; if for example the “Russ H” tag was not present in the tag import data, the
`
`software would not display any intermediary tag between the “second” and “third”
`
`tags. (Id.) As noted, the claimed visual and vertical arrangement of the first, second,
`
`
`
`
`
`13
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`and third tags is not a significant or patentable feature; different sets of input data
`
`would have predictably generated different sequences of displayed tags. (Id., ¶24.)
`
`C. The Additional Teachings of Zuckerberg, Rothmuller and Plotkin
`Render the Substitute Claims Obvious When Applied to the
`Existing Instituted Grounds
`As demonstrated above, all of the new limitations introduced in the substitute
`
`claims are disclosed or rendered obvious by Zuckerberg, Rothmuller and Plotkin.
`
`The substitute claims are thus unpatentable based on the prior art already of record
`
`in these IPRs. But how the above described teachings of Zuckerberg, Rothmuller,
`
`and Plotkin apply to the instituted grounds differs from one set of grounds to another.
`
`For the convenience of the Board, therefore, Petitioner will walk through how the
`
`existing instituted grounds render the substitute claims obvious when the additional
`
`disclosures of Zuckerberg, Rothmuller, and Plotkin are taken into account.
`
`1.
`
`Instituted Grounds in IPR2019-00516
`(a) Application to Grounds 2-5 of IPR2019-00516
`Grounds 2-5 in IPR2019-00516 were based on Zuckerberg in combination
`
`with either Rothmuller (Grounds 2-3) or Plotkin (Grounds 4-5). These grounds thus
`
`already include all the prior art needed with respect to both the original claims and
`
`the additional limitations introduced by corresponding substitute claims 21-32.
`
`For example, the Petition explained how Zuckerberg itself discloses a “tag
`
`list,” and how Zuckerberg in view of Rothmuller or Plotkin renders obvious a “tag
`
`type indicator for each tag appearing in the tag list, said tag type indicator being
`
`14
`
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`indicative of a tag source associated with the tag.” (Petition in IPR2019-00516 at
`
`24-27, 43-45, 52-53; Ex. 1002 in IPR2019-00516, ¶¶74-79, 80-83, 98-104, 111-
`
`114.) The explanation previously provided in connection with the combination of
`
`Zuckerberg and Rothmuller, and Zuckerberg and Plotkin, also fully addresses the
`
`new claim limitations. For example, Zuckerberg plainly discloses a “tag list” where
`
`“the tags in the tag list are displayed in a vertical list.” (Zuckerberg, e.g., Fig. 5.)
`
`A person of ordinary skill in the art would have been motivated to combine
`
`Rothmuller and Plotkin with Zuckerberg because doing so would have provided the
`
`benefit of a more flexibly-organized tag list, where tags could be displayed in any
`
`order, without the constraints of a separate list for each type. (Petition in IPR2019-
`
`00516 at 44-45; Ex. 1002 in IPR2019-00516, ¶102; see also id., ¶112.)
`
`For purposes of the specific combination of Zuckerberg with Rothmuller or
`
`Plotkin, the “first tag source” corresponds to either the collection of tags used to
`
`populate the friends list 546, or the collection of tags used to populate the text list
`
`544. (Petition in IPR2019-00516 at 24-26; Ex. 1002 in IPR2019-00516, ¶¶75-77.)
`
`For example, the combination of Zuckerberg with Rothmuller or Plotkin would have
`
`predictably resulted in a vertical tag list displaying three tags, e.g., a “first tag” from
`
`the friends list, a “second tag” from the text list and below the first tag, and a “third
`
`tag” from the friends list and sitting below the second tag, each tag displayed
`
`alongside an appropriate tag type indicator icon as taught in Rothmuller and Plotkin.
`
`
`
`
`
`15
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`It does not matter whether the “first tag source” is applied to the “friends list” tag
`
`collection or the “text list” tag collection of Zuckerberg; if the “friends list”
`
`collection is chosen as the “first tag source,” then the “text list” collection becomes
`
`the “second tag source,” and vice versa. (Ex. 1027, ¶11.)
`
`As noted, although both Rothmuller and Plotkin disclose the claimed vertical
`
`tag list and arrangement of first, second, and third tags, their disclosures also render
`
`these features obvious. A skilled artisan would have appreciated that the system
`
`created based on the combination of Zuckerberg with Rothmuller or Plotkin could
`
`have displayed three tags in the claimed vertical arrangement if the underlying tags
`
`used to populate the tag list happened to include a sequence of three tags from a first
`
`tag source, second tag source, and the first tag source, respectively. (Ex. 1027, ¶24.)
`
`For all of these reasons, the combination of Zuckerberg with Rothmuller, and
`
`Zuckerberg with Plotkin, thus renders obvious “wherein the tags in the tag list are
`
`displayed in a vertical list, and wherein: a first tag from a first tag source is displayed
`
`above a second tag from a second tag source; and the second tag from the second tag
`
`source is displayed above a third tag from the first tag source,” along with all of the
`
`other limitations in the substitute claims. (Id., ¶¶11, 28.)
`
`(b) Application to Grounds 6-7 of IPR2019-00516
`These grounds cited Rothmuller as the primary reference and did not include
`
`Zuckerberg, which was cited above with respect to new limitations relating to
`
`
`
`
`
`16
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`display of a photograph, user selection of a location, and tag association based on
`
`the user selection. Accordingly, in order to account for the new limitations of the
`
`substitute claims, Zuckerberg would be added to Grounds 6-7. The motivation to
`
`combine Rothmuller with Zuckerberg would have been straightforward, as shown
`
`below. As explained in the Petition, Rothmuller and Zuckerberg are analogous
`
`references in the field of computer-based systems for tagging content. (Petition in
`
`IPR2019-00516, at 43; Ex. 1002 in IPR2019-00516, ¶99.)
`
`It would have been obvious to a person of ordinary skill in the art to adapt
`
`Zuckerberg’s techniques to Rothmuller, predictably resulting in the system of
`
`Rothmuller in which the system displays a photograph comprising at least one
`
`subject or object, receives a user selection of a location in the photograph
`
`corresponding to the at least one subject or object, and in response, associates a tag
`
`with the at least one subject or object. (Ex. 1027, ¶30.) A person of ordinary skill
`
`in the art would have been motivated to combine Zuckerberg and Rothmuller in this
`
`fashion to achieve at least two clear benefits.
`
`First, the combination would have enhanced the system of Rothmuller by
`
`allowing the selection and association of tags with a particular object or subject in
`
`the photo rather than the photo as a whole. (Ex. 1027, ¶31.) A person of ordinary
`
`skill in the art would have appreciated that photographs commonly contain many
`
`different objects. For example, a photo could include multiple people, natural or
`
`
`
`
`
`17
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`architectural landmarks (such as trees or mountains), animals, and other things.
`
`Allowing a tag in Rothmuller to be associated with a particular subject or object in
`
`a photo makes the tag much more specific and useful than assignment to the photo
`
`as a whole, because it records where in the photo tagged item appears. (Id.)
`
`A second but related motivation to combine flows from the ability in both
`
`Zuckerberg and Rothmuller to assign multiple tags to a single photo. (Ex. 1027,
`
`¶32.) As explained in IPR2019-00516, Zuckerberg and Rothmuller both allow users
`
`to assign multiple tags to a single photo. (Petition in IPR2019-00516 at 22-23, 54-
`
`55; Ex. 1002 in IPR2019-00516, ¶¶72-73, 168-169.) Zuckerberg, for example,
`
`shows an example in which two people (i.e. “erin” and “betty jo”) were tagged in a
`
`single photo. (Zuckerberg, Figs. 6 & 7 (showing a “betty jo” tag being added to a
`
`photo that already has “erin” tag); see also id., 1:65 (“The media owner may select
`
`and tag multiple regions.”), 9:67-10:1.) Rothmuller similarly discloses an example
`
`in which a photo (“Lori on the road at Legoland”) was tagged with “Lori R” (a
`
`people tag) and “San Diego” (a places tag). (Rothmuller, 4:42-45, Fig. 2.) The
`
`common ability to assign multiple tags to a photo strengthens the motivation a
`
`skilled artisan would have had to add Zuckerberg’s region selection features to
`
`Rothmuller. For example, without the ability to granularly associate a tag with a
`
`particular subject or object in the photograph, the tag is much less useful because it
`
`cannot be used to identify where the tagged subject or object appears in the photo,
`
`
`
`
`
`18
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`or distinguish more significant objects from in the photo. (Ex. 1027, ¶32.) To take
`
`a tangible example, if the photograph “Lori on the road at Legoland” in Rothmuller
`
`included several other people in addition to Lori, the “Lori” people tag could not be
`
`used to determine which of the people in the photo was Lori. (Id.)
`
`A person of ordinary skill in the art would have perceived no technical
`
`obstacle in making this combination. (Ex. 1027, ¶33.) The addition to Rothmuller
`
`of the Zuckerberg user interface for displaying a photograph and selecting a region
`
`would have involved nothing more than routine and conventional programming
`
`techniques. The ability to display photographs and receive location input from the
`
`user (e.g. through a mouse, trackball, etc.) involved basic and well-known
`
`techniques. (Id.) A person of ordinary skill would have found the benefits of
`
`Zuckerberg’s image display and region selection techniques sufficiently compelling
`
`to adapt them to the user interface and tagging system in Rothmuller. (Id.)
`
`2.
`
`Instituted Grounds in IPR2019-00528
`(a) Application to Grounds 3-6 in IPR2019-00528
`These four instituted grounds cited MacLaurin as the primary reference and
`
`combined it with either Rothmuller (Grounds 3-4) or Plotkin (Grounds 5-6), but
`
`these grounds did not cite Zuckerberg. Accordingly, in order to account for the new
`
`limitations of the substitute claims, Zuckerberg would be added to these existing
`
`grounds, and added as Exhibit 1003 to IPR2019-00528. The motivation to combine
`
`
`
`
`
`19
`
`
`
`
`
`
`
`
`
`
`
`IPR2019-00516 & IPR2019-00528
`U.S. Patent No. 8,279,173
`
`MacLaurin with Zuckerberg would have been straightforward, as shown below.
`
`MacLaurin and Zuckerberg are analogous references in the field of computer-based
`
`systems for tagging content. (Ex. 1027, ¶34.)
`
`It would have been obvious to a person of ordinary skill in the art to adapt
`
`Zuckerberg’s techniques to MacLaurin, predictably resulting in the tagging system
`
`of MacLaurin further adapted to display a photograph comprising at least one subject
`
`or object, receives a user selection of a location in the photograph corresponding to
`
`the at least one subject or object, and in response, associates a tag with the at least
`
`one subject or object. (Id., ¶35.)
`
`The same clear benefits discussed above for the combination of Rothmuller
`
`and Zuckerberg would have applied equally to the combination of MacLaurin and
`
`Zuckerberg. (Ex. 1027, ¶¶36, 31-33.) The combination would have enhanced the
`
`system of MacLaurin by allowing the selection and association of tags with a
`
`particular object or subject in the photo rather than the photo as a whole, resulting in
`
`the benefits discussed above. MacLaurin allows – and in fact expressly
`
`encourages – assignment of multiple tags to a single photo. (Ex. 1002 in IPR2019-
`
`00528, ¶¶65-67.) MacLaurin explains that its tagging technique “excels in allowing
`
`items to have multiple tags.” (MacLaurin, 6:53-54.) By allowing an item to “belong
`
`to multiple groups and associations without requiring the item to be moved or copied
`
`into many different locations,” MacLaurin exp
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
