`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`________________
`
`FACEBOOK, INC., INSTAGRAM, LLC, and WHATSAPP INC.,
`Petitioners,
`
`v.
`
`BLACKBERRY LIMITED,
`Patent Owner
`________________
`IPR2019-00516
`U.S. Patent No. 8,279,173
`________________
`
`DECLARATION OF DR. RAJEEV SURATI
`IN SUPPORT OF PATENT OWNER’S RESPONSE
`
`
`
`
`
`05710-00016B/11180713.5
`
`
`
`Blackberry's Exhibit No. 2001
`Page 1 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`TABLE OF CONTENTS
`
`I.
`II.
`
`Page
`INTRODUCTION ........................................................................................... 1
`BASIS FOR OPINION ................................................................................... 2
`A. Qualifications ........................................................................................ 2
`B. Materials Considered ............................................................................ 5
`III. LEVEL OF ORDINARY SKILL IN THE ART ............................................. 6
`
`IV. LEGAL STANDARDS FOR PATENTABILITY .......................................... 7
`V.
`BACKGROUND OF THE RELEVANT TECHNOLOGY ........................... 8
`VI. OVERVIEW OF THE ’173 PATENT .......................................................... 10
`VII. OVERVIEW OF THE CITED REFERENCES ............................................ 15
`A.
`Zuckerberg .......................................................................................... 15
`B.
`Rothmuller .......................................................................................... 16
`C.
`Plotkin ................................................................................................. 18
`D. MacLaurin ........................................................................................... 19
`E.
`Ortega .................................................................................................. 25
`F. Matthews ............................................................................................. 25
`VIII. CLAIM CONSTRUCTION .......................................................................... 27
`A.
`“Tag Sources” ..................................................................................... 27
`B.
`“A Tag Type Indicator . . . Said Tag Type Being Indicative of a
`Tag Source” ........................................................................................ 34
`IX. RESPONSE TO ARGUMENT ..................................................................... 35
`A. Grounds 1-5 Fail Because The Cited Art Does Not Disclose
`Any Indicator “Indicative of a Tag Source” ....................................... 35
`
`05710-00016B/11180713.5
`
`i
`
`Blackberry's Exhibit No. 2001
`Page 2 of 76
`
`

`

`B.
`
` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`Grounds 1-5 Also Fail Because Petitioners’ Proposed
`Combination Does Not Render Obvious “a Tag Type Indicator
`for Each Tag Appearing in the Tag List” ............................................ 48
`Grounds 6-7 Fail Because Rothmuller Does Not Disclose
`“Displaying a Tag Type Indicator . . . Indicative of a Tag
`Source” ................................................................................................ 65
`
`C.
`
`05710-00016B/11180713.5
`
`ii
`
`Blackberry's Exhibit No. 2001
`Page 3 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`I, Rajeev Surati, declare as follows:
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`I.
`
`INTRODUCTION
`1. My name is Rajeev Surati. I am President at Computation and
`
`Imaging, a consultancy providing strategic and implementation services for
`
`intellectual property and business issues both related to computation and imaging
`
`with cameras and display devices and storage of such imagery. I am also an
`
`Engineer at Hydrow.com working on AR/VR and machine vision. I have been
`
`retained to serve as an expert on behalf of Patent Owner BlackBerry Limited
`
`(“BlackBerry” or “Patent Owner”) in connection with the above-captioned inter
`
`partes review (“IPR”) proceeding, to provide my analyses and opinions in certain
`
`technical aspects of this proceeding. I understand that this Declaration is used to
`
`support BlackBerry’s Patent Owner Response regarding challenged claims of U.S.
`
`Patent No. 8,279,173 (“the ’173 Patent”) (Ex. 1001).
`
`2.
`
`I am being compensated for my work on this case at my standard
`
`consulting rate of $540 per hour. I am also being reimbursed for all incurred
`
`expenses. No part of my compensation is contingent upon the outcome of this
`
`proceeding. I have no other interests in this proceeding or with any of the parties.
`
`3.
`
`I am competent to testify to the matters stated in this Declaration,
`
`have personal knowledge of the facts and statements herein, and each of the
`
`statements is true and correct.
`
`05710-00016B/11180713.5
`
`
`
`Blackberry's Exhibit No. 2001
`Page 4 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`II. BASIS FOR OPINION
`A. Qualifications
`4.
`I have more than 25 years of experience in the fields of electrical
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`engineering and computer science. Additionally, I have at least 19 years of
`
`experience in owning and running software companies that utilize database-backed
`
`systems, including Internet photo databases, as well as extensive experience in
`
`designing and building user interfaces.
`
`5.
`
`I received a bachelor’s degree (1992), master’s degree (1995), and a
`
`Ph.D. (1999) in Electrical Engineering and Computer Science from Massachusetts
`
`Institute of Technology (“MIT”).
`
`6.
`
`I am an inventor of U.S. Patent No. 5,943,478, entitled “System for
`
`Immediate Popup Messaging Across the Internet,” which describes a system and
`
`user interface for sending and displaying instant messages in popup windows
`
`between Internet users.
`
`7.
`
`In 1996, I founded Flash Communications, a company focused on
`
`technology related to U.S. Patent No. 5,943,478 and associated technology that I
`
`had developed related to pop-up instant messaging over the Internet. Microsoft
`
`Corporation purchased Flash Communications in 1998 and incorporated its
`
`messaging technology into Microsoft’s Messenger service and Exchange 2000
`
`Server Instant Messaging Service.
`
`05710-00016B/11180713.5
`
`2
`
`Blackberry's Exhibit No. 2001
`Page 5 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`I joined Microsoft in 1999 after completing my Ph.D. Between 1999
`
`8.
`
`and 2000, I worked in the Microsoft Exchange Server group, which was
`
`responsible for all of Microsoft’s messaging products, including email.
`
`9.
`
`Between 2000 and 2004, I worked as a consultant and investor at
`
`Nexaweb Corporation, where I helped implement several two-way messaging
`
`systems.
`
`10.
`
`In 2000, I founded Photo.net, a large online photography community
`
`and one of the first social networking and photo sharing web sites. At the core of
`
`Photo.net was a photo database that implemented an early form of photo tagging.
`
`From 2000-2007, before it was purchased by NameMedia, I designed,
`
`implemented, and administered Photo.net’s underlying database system for almost
`
`seven years.
`
`11. Throughout my career, I have regularly designed and built user
`
`interfaces (“UIs”) for the software applications and websites I developed. During
`
`my tenure at Photo.net, I created several novel UIs with an emphasis on ease of
`
`use.
`
`12.
`
`In 2002, our user interface received high praise from renowned
`
`entrepreneur and venture capitalist Joichi Ito. Additionally, Photo.net’s photo
`
`sharing system was one of the first on the Internet and received praise as one of the
`
`best systems released in 2003.
`
`05710-00016B/11180713.5
`
`3
`
`Blackberry's Exhibit No. 2001
`Page 6 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`In 2004, I founded another company, Scalable Display Technologies
`
`13.
`
`(“SDT”). I have been the President and Chairman of SDT since its founding. SDT
`
`operates in the audio-video domain and has licensed software and firmware to
`
`various companies, including Sony, Hitachi, Pixar, Disney, and Universal. The
`
`company focuses on displaying multimedia imagery on projected displays using
`
`multiple projectors and camera feedback.
`
`14.
`
`I am or have served on the advisory boards of several technology
`
`companies, including UnifySquare, which offers collaboration management
`
`software and consulting services; Nexaweb, which develops real-time web
`
`application frameworks using HTTPS; Permabit, which develops content
`
`addressable storage; and Evoque, an e-commerce enabling platform publisher.
`
`15.
`
`I have received several awards for my contributions as an inventor
`
`and entrepreneur. I received the Global Indus Technovator Award in 2008 and
`
`was named a Computerworld Honors Laureate in 2009.
`
`16.
`
`I have provided a copy of my curriculum vitae as an attachment to this
`
`report. See Appendix A.
`
`17. A list of cases in which I have testified as an expert over the past four
`
`years is attached to this report as Appendix B.
`
`05710-00016B/11180713.5
`
`4
`
`Blackberry's Exhibit No. 2001
`Page 7 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`B. Materials Considered
`18. As part of my preparation for writing this Declaration, I reviewed the
`
`following materials: the ’173 Patent and its prosecution history, U.S. Patent No.
`
`7,945,653 (“Zuckerberg”), U.S. Patent No. 7,415,662 (“Rothmuller”), David
`
`Plotkin, How to Do Everything with Photoshop Elements 4.0 (“Plotkin”), U.S.
`
`Patent No. 7,831,913 (“MacLaurin”), U.S. Patent No. 6,564,213 (“Ortega”), U.S.
`
`Patent App. Pub. No. 2006/0218503 (“Matthews”), and Petitioner’s Petition for
`
`inter partes review and all cited exhibits, including the Declaration of Dr. Sandeep
`
`Chatterjee and any exhibits cited therein.
`
`19. Additionally, I reviewed the transcript of Dr. Chatterjee’s deposition.
`
`20.
`
`I also reviewed Patent Owner’s Response and all exhibits thereto.
`
`21. My opinions are based on my years of education, research, and
`
`industry experience, as well as my investigation and study of relevant materials.
`
`22.
`
`I may rely upon these materials and/or additional materials to respond
`
`to arguments raised by Petitioner. I may also consider additional documents and
`
`information informing any necessary opinions, including documents that may not
`
`yet have been provided to me.
`
`23. My analysis of the materials produced in this investigation is ongoing
`
`and I will continue to review any new material as it is provided. This report
`
`represents only those opinions I have formed to date. I reserve the right to revise,
`
`05710-00016B/11180713.5
`
`5
`
`Blackberry's Exhibit No. 2001
`Page 8 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`supplement, and/or amend my opinions stated herein based on new information
`
`and on my continuing analysis of the materials already provided.
`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`24. The ’173 patent is directed to a user interface for selecting a photo tag.
`
`Ex. 1001 at 1:52-55. In response to a user-entered string, the interface returns a list
`
`of matching tags the user may apply to a selected photograph. Id. at Abstract. The
`
`inventors of the ’173 patent developed a novel way to allow the user to retrieve
`
`tags in a convenient and organized fashion on the limited space of a device with a
`
`small display. Id. at 1:53-55.
`
`25. Therefore, a person of ordinary skill in the art relevant to the ’173
`
`Patent at the time of the filing of the invention would have (a) a bachelor’s degree
`
`in software engineering, computer science, or a closely related field, and at least
`
`two years of work or research experience in the field of software application
`
`development, including graphical user interface development.
`
`26. At the time of filing of the ’173 Patent invention, I had my Ph.D. with
`
`extensive software development and graphical user interface experience and was
`
`President, Chairman, and Co-Founder of Scalable Display Technologies and had
`
`previously served as President, co-Founder, and Chairman of Photo.net for seven
`
`years. I meet these criteria and consider myself a person with at least ordinary skill
`
`05710-00016B/11180713.5
`
`6
`
`Blackberry's Exhibit No. 2001
`Page 9 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`in the art pertaining to the ’173 Patent. I was such a person at the time of the filing
`
`of the invention of the ’173 Patent.
`
`IV. LEGAL STANDARDS FOR PATENTABILITY
`27.
`I understand that “prior art” includes patents and printed publications
`
`that existed before the earliest applicable filing date of the ’173 Patent.
`
`28.
`
`I understand that in order for a claim to be anticipated, each and every
`
`requirement of the claim must be found, expressly or inherently, in a single prior
`
`art reference as recited in the claim.
`
`29.
`
`I understand that a claimed invention is not patentable if the claimed
`
`invention would have been obvious to a person of ordinary skill in the field of the
`
`invention at the time the invention was made.
`
`30.
`
`I understand that in order to show obviousness based on a
`
`combination of references, a particular motivation to combine the teachings in the
`
`references must be shown.
`
`31.
`
`I understand that claim terms are generally given their ordinary and
`
`customary meaning, which is the meaning that the term would have to a person of
`
`ordinary skill in the art at the time of the invention. I further understand that a
`
`person of ordinary skill in the art must read the claim term not only in the context
`
`of the particular claim in which the term appears but in the context of the entire
`
`patent, including the specification.
`
`05710-00016B/11180713.5
`
`7
`
`Blackberry's Exhibit No. 2001
`Page 10 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`I understand that the obviousness inquiry should not be done in
`
`32.
`
`hindsight, and depends on the scope and content of the prior art, the differences
`
`between the prior art and the claims at issue, the knowledge of a person of ordinary
`
`skill in the pertinent art at the time of invention, and any other objective factors
`
`indicating obviousness or non-obviousness.
`
`33.
`
`I understand that in order to rely on a reference for obviousness, the
`
`reference must be analogous art. I also understand that to be analogous art, the
`
`reference must be either (1) from the same field of endeavor as the claimed subject
`
`matter, regardless of the problem addressed, or (2) if not in the same field of
`
`endeavor, reasonably pertinent to the particular problem with which the inventor is
`
`involved. I am also familiar with the premise that for a reference to be reasonably
`
`pertinent, it must have logically commended itself to an inventor’s attention at the
`
`time of invention.
`
`V. BACKGROUND OF THE RELEVANT TECHNOLOGY
`34. The ’173 Patent relates to computer software for tagging photographs.
`
`For example, a user who wants to share a photograph on a social network might
`
`like to identify people or objects in the photograph by selecting a “tag” to associate
`
`with an identified point in the photograph. Ex. 1001 at 1:21-25.
`
`35. As of the priority date of the ’173 Patent—May 9, 2007—the
`
`technology for sharing photographs on the Internet was still in its infancy.
`
`05710-00016B/11180713.5
`
`8
`
`Blackberry's Exhibit No. 2001
`Page 11 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`36. Photo.net, a website I founded in 2000 and ran until 2007, is a prime
`
`example of the state of photo tagging user interfaces at that time. The early
`
`version of Photo.net simply allowed users to post photographs and to browse
`
`photographs posted by others on the site. Later versions allowed users to enter
`
`keywords describing certain properties or characteristics pertaining to their
`
`photographs, such as “Location,” “Equipment,” or other “Technical Details,” and
`
`to add other “Custom Fields,” as seen in the screenshot below:
`
`
`
`37. These keywords could be considered tags, but they could only be
`
`manually entered by the user. The focus of the site was on allowing users to
`
`display and describe their own, individual photographs. The keyword tags were
`
`05710-00016B/11180713.5
`
`9
`
`Blackberry's Exhibit No. 2001
`Page 12 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`attached to the entire photograph, not to a location in the photograph like in
`
`the ’173 Patent. The manual entry method suffered many of the user interface
`
`issues discussed as the state of the prior art in the ’173 Patent as well. Each type of
`
`tag had its own field, so even if individual fields could be populated using a search
`
`function (which was not the case), there was no motivation for a search across
`
`multiple fields to help in tagging an image.
`
`VI. OVERVIEW OF THE ’173 PATENT
`38. The ’173 Patent—unlike Photo.net and other prior art photography
`
`sharing software—recognized that, given the explosion in the quantity and variety
`
`of online media, selecting a single “tag” representing an identified person or object
`
`has become an increasingly complicated task. Ex. 1001 at 1:23-25. This was only
`
`more difficult on wireless mobile communication devices, where display size and
`
`input modes might be constrained. Id. at 1:25-29.
`
`39. The ’173 Patent solves these and other problems by allowing a user to
`
`find and select tags from multiple sources simultaneously using a universal search
`
`functionality, and then presenting the search results in a single display along with a
`
`visual indication of the source of each tag.
`
`40. One example of this solution appears in Figure 3B of the ’173 Patent,
`
`which shows a user interface for viewing a shared photograph:
`
`05710-00016B/11180713.5
`
`10
`
`Blackberry's Exhibit No. 2001
`Page 13 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`
`
`41.
`
`In this example, the user can click the “Add” button to enter into the
`
`photo tagging mode. Id. at 4:10-18. In this mode, the user can then move the
`
`cross-hair pointer 308 on the photo to identify the particular area of the photo that
`
`will be the subject of one or more “Tags.” Id. at 4:10-37.
`
`42. After the user selects the “Add” button, the ’173 Patent presents the
`
`user interface shown in Figure 4B that allows the user to search for tags from
`
`multiple sources. Id. at 4:44-60, 5:39-47.
`
`05710-00016B/11180713.5
`
`11
`
`
`
`Blackberry's Exhibit No. 2001
`Page 14 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`43. When the user starts entering text into tag entry field 406, the
`
`interface displays a list of tags 412 matching the input text. Id. at 5:32-55.
`
`44. The tags in the search results can be from multiple sources, such as a
`
`list of contacts from the user’s address book, a list of the user’s browser
`
`bookmarks, a list of friends from a social network like Facebook, recently sent and
`
`received text messages stored on the mobile communication device, and the
`
`geographic coordinates of places that the user has recently visited with the mobile
`
`communication device. Id. at 5:39-47, 6:5-16.
`
`45.
`
`In this example, most if not all of these tag sources correspond to
`
`different applications or components on a user’s mobile device. See Fig. 1
`
`(annotated):
`
`05710-00016B/11180713.5
`
`12
`
`Blackberry's Exhibit No. 2001
`Page 15 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`46. For each tag source, the ’173 Patent provides a visual indicator, giving
`
`the user a quick way to identify the source of the tag so that he or she can make a
`
`more informed decision about which tag to apply to a photograph. For example,
`
`Figure 4B (annotated) discloses using an icon or other visual identifier (e.g., 412a-
`
`
`
`05710-00016B/11180713.5
`
`13
`
`Blackberry's Exhibit No. 2001
`Page 16 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`1 and 412b-1) for each tag in the intermingled list of matching tags from different
`
`sources (e.g., 412a-2 and 412b-2) indicating the source of each tag. Id. at 5:52-55.
`
`
`
`47.
`
`In the figure above, a friends icon (visual indicator 412a-1) indicates
`
`that the tag “Tara Chmiel” 412a-2 is from a list of friends on Facebook, and a text
`
`icon (visual indicator 412b-1) indicates that the tag “text i typed before” 412b-2 is
`
`from a collection of recently sent and received text messages. Id. at 5:39-55.
`
`48. Using the visual icons, a user can scroll through the list of tags 412
`
`and quickly identify the source of each tag to whether to associate a tag with the
`
`photograph 302. Id. at 5:62-65.
`
`05710-00016B/11180713.5
`
`14
`
`Blackberry's Exhibit No. 2001
`Page 17 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`VII. OVERVIEW OF THE CITED REFERENCES
`A. Zuckerberg
`49. U.S. Patent No. 7,945,653 (“Zuckerberg”) (Ex. 1003) describes
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`desktop-based systems and methods for tagging digital media using tags from a
`
`single list of previously entered tags. Ex. 1003 at 1:54-56.
`
`50. A social network user may upload a digital image to an album on his
`
`or her web page, select a region of the image by clicking on it, and “typ[e]
`
`appropriate text to tag the region.” Id. at 1:59-65.
`
`51. As shown in Figure 5, once a “region 520” is selected, a “tag list 540”
`
`pops up, which “may include a text entry window 542” and “a list of previously
`
`used tags” (id. at 8:49-54):
`
`05710-00016B/11180713.5
`
`15
`
`Blackberry's Exhibit No. 2001
`Page 18 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`
`
`52. The “list of previously used tags” in Figure 5 is, in turn, visually
`
`subdivided into “a text list 544 and a friends list 546” using a line. Id. at 8:56-58.
`
`53. Clicking on any of the entries from the tag list 540 will “associate the
`
`tag with the selected region 520.” Clicking on an entry “in the friends list 546, for
`
`example “may associate the friend’s email address with the selected region 520.”
`
`Id. at 8:66-9:3.
`
`B. Rothmuller
`54. U.S. Patent No. 7,415,662 (“Rothmuller”) (Ex. 1004) is directed to
`
`organizing a database of images and photos using tags. Ex. 1004 at Abstract. All
`
`05710-00016B/11180713.5
`
`16
`
`Blackberry's Exhibit No. 2001
`Page 19 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`of those tags are stored in the same “tag drawer,” also called a “tag keeper.” See
`
`Ex. 1005 (Rothmuller Prov.) at 68 (“The Tag Drawer holds all the tags currently
`
`defined in the system”); see also Ex. 1004 at Fig. 1 (annotated to identify TAG
`
`KEEPER with red box):
`
`55. Users can create and modify tags in a tag editor and assign tags to
`
`categories that describe photos associated with a particular tag. Id. at 3:51-58
`
`(“For example, in one embodiment tags are divided into people, events, places and
`
`miscellaneous tag categories.”).
`
`
`
`05710-00016B/11180713.5
`
`17
`
`Blackberry's Exhibit No. 2001
`Page 20 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`56. Rothmuller discloses only four predefined categories—people, events,
`
`places, and miscellaneous—and furthermore teaches that the user cannot create
`
`new categories. Ex. 1005 at 62 (“There are four categories of tags: People[;]
`
`Events[;] Places[;] Miscellaneous[.] Users cannot create their own categories.”
`
`(emphasis in original)). Each tag category can, in turn, be further sub-divided into
`
`tag types that more narrowly describe the tagged photo. Ex. 1004 at 4:1-39 (“The
`
`events tag category includes default tag types for parties and vacations, and can be
`
`customized to include tag types for particular types of events such as concerts,
`
`plays, shows and sporting events, and for particular events such as the 2002 Boston
`
`Marathon.”).
`
`57. Rothmuller’s system displays each tag in the “tag keeper”/“tag
`
`drawer” alongside a small icon representing the tag category associated with that
`
`tag. Id. at Fig. 1; see also Ex. 1005 at 34 (“Small icons are used to distinguish
`
`between tag categories (people, places, events, and miscellaneous), [and] whether
`
`the image is a favorite or not (the heart icon) . . . .”).
`
`C.
`Plotkin
`58. David Plotkin, How to Do Everything with Photoshop Elements 4.0
`
`(“Plotkin”) (Ex. 1008) describes a feature that allows a user to edit tags assigned to
`
`imported pictures without explaining how those tags are sourced.
`
`05710-00016B/11180713.5
`
`18
`
`Blackberry's Exhibit No. 2001
`Page 21 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`59. Plotkin, like Rothmuller, describes a hierarchy of tag keywords or
`
`phrases organized by subject matter. Ex. 1008 at 323. Plotkin shows a user
`
`interface for importing pictures with existing tags that have a category icon
`
`displayed next to them, but provides no context as to how that user interface was
`
`generated. Id. at 327-29.
`
`60. For example, the user interface described in Plotkin has an icon with
`
`an image of two people next to “David P.,” but it does not explain how the “David
`
`P.” tag was sourced or selected. See id. at 328.
`
`D. MacLaurin
`61. U.S. Patent No. 7,831,913 (“MacLaurin”) (Ex. 1006) describes
`
`methods for tagging and finding items in a file management system, such as word
`
`processing documents and other files. Ex. 1006 at 7:32-35; id. at Fig. 4.
`
`
`
`05710-00016B/11180713.5
`
`19
`
`Blackberry's Exhibit No. 2001
`Page 22 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`62. MacLaurin’s primary stated goal is to “provide users with automated
`
`item tagging with minimal impact to the user.” Id. at 2:39-44; see also id. at 4:15-
`
`19 (“The systems and methods herein provide an improved user interface for
`
`applying tags automatically when the user has made a selection of items to be
`
`tagged and/or provides an input such as, for example, typing any character on a
`
`keyboard.”).
`
`63. Beyond automatic tagging, MacLaurin discloses two other, distinct
`
`modes: (1) a tagging mode for tagging items; and (2) a recall mode for selecting
`
`from a list of previously applied tags to locate tagged items. Ex. 1006 at 7:48-
`
`8:57.
`
`64. Tagging Mode.
`
`65. MacLaurin discloses a “tagging mode” allowing a user to tag a set of
`
`files. Id. at 7:66-8:3.
`
`66. Another goal of MacLaurin’s tagging mode is to enforce the use of
`
`consistent tags. See id. at 7:21-28 (emphasis added):
`
`[I]f a user is looking for a house, they may tag items with
`
`“house” during the day. On the next day, the same user
`
`may have forgotten the previous day’s tag and start to tag
`
`items with “home.” Thus, at the moment the user is
`
`applying tags, they can be reminded that they previously
`
`05710-00016B/11180713.5
`
`20
`
`Blackberry's Exhibit No. 2001
`Page 23 of 76
`
`

`

` Case No. IPR2019-00516
`DECLARATION OF DR. RAJEEV SURATI
`U.S. Patent No. 8,279,173
`
`
`used “house” instead of “home,” saving them from
`
`utilizing multiple tags when they did not intend to do so.
`
`67. MacLaurin’s “tagging mode” serves this goal of consistent tags
`
`through a “best guess” tag search interface. MacLaurin discloses a text-entry
`
`interface that “accumulate[s] each key a user types into a ‘tag buffer,’” “use[s] this
`
`tag buffer to guess at likely tags,” and then “display[s] the current ‘best guess’ tag
`
`in a textual readout associated with the window.” Id. 7:66-8:10. If the user wishes
`
`to view other “likely tags,” the user may “choose between ‘tag guesses’ using
`
`cursor arrows.” Id. at 8:11-12. Once the user has chosen a tag, “the user hits the
`
`enter/return key (or similar), [to] apply the items to the tag.” Id. at 8:17-18.
`
`68. Figure 8 in MacLaurin illustrates its tagging mode. For example, the
`
`“possible tags that begin with the letter ‘g’” include “‘graphics,’ ‘group A,’ ‘group
`
`B,’ ‘green,’ and/or ‘garage’ and the like.” Id. at 5:31-34. But when a user in
`
`tagging mode (as indicated by “tagging icon 808” in Figure 8, below) enters “gr”
`
`(see “input” 804), only a single, “best guess” suggested tag, “graphics” (see
`
`“suggested tag” 802), is displayed:
`
`05710-00016B/11180713.5
`
`21
`
`Blackberry's Exhibit No. 2001
`Page 24 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`
`69. Tag “Recall” Mode. MacLaurin’s recall mode is “allows easy recall
`
`of the tagged items at another time.” Id. at 2:41-44; see also id. at 6:6-8 (“The
`
`utilized tags are then relayed to the user via the user interface 308 at appropriate
`
`times to facilitate the user 304 in recalling items based on tag information.”); id. at
`
`6:40-44 (“Simple text-based strings or tags . . . allow a variety of items to be easily
`
`recalled later utilizing only a single tag.”).
`
`70. MacLaurin’s recall mode has a window that “shows tags already
`
`created.” Id. at 8:33-35. For example, Figure 6 shows a “tag list 604” of
`
`previously applied tags that “allows the user to quickly find items associated with
`
`the tags in the list.” Id. at 8:46-47; id. at Figure 6:
`
`05710-00016B/11180713.5
`
`22
`
`Blackberry's Exhibit No. 2001
`Page 25 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`
`71. Thus, as shown in Figure 7, when a user selects tag 702 (“graphics”)
`
`from the list, MacLaurin displays only files 704 that were previously associated
`
`with the “graphics” tag. Id. at 8:47-51; id. at Figure 7:
`
`05710-00016B/11180713.5
`
`23
`
`Blackberry's Exhibit No. 2001
`Page 26 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`
`72. Recall mode also allows users to differentiate between (1) automated
`
`tags generated by the tagging system and (2) explicit tags entered by the user. Id.
`
`at 7:48-65.
`
`73. MacLaurin explains that, because “a user may have high confidence
`
`in their explicit tags and lesser confidence in system generated tags,” “a user can
`
`be alerted to their confidence level with regard to the tags” if the user can easily
`
`distinguish between these two types of tags. Id. at 7:49-53. For example,
`
`MacLaurin distinguishes between these two types of tags “utilizing different sizes,
`
`fonts, colors, and/or symbols and the like.” Id. at 8:19-22.
`
`05710-00016B/11180713.5
`
`24
`
`Blackberry's Exhibit No. 2001
`Page 27 of 76
`
`

`

`DECLARATION OF DR. RAJEEV SURATI
`
`
`
` Case No. IPR2019-00516
`U.S. Patent No. 8,279,173
`
`E. Ortega
`74. U.S. Patent No. 6,564,213 (“Ortega”) (Ex.1007) relates to searching a
`
`database, but does so without the use of tags. Specifically, Ortega discusses
`
`“searching a particular catalog or database, such as the products database of [a]n
`
`online merchant.” Ex. 1007 at 1:55-60.
`
`75. Ortega then describes a search function where a user “enters a search
`
`query into a search field . . . of the Amazon.com web site,” resulting in a display of
`
`a drop-down box consisting of “suggested autocompletion terms and phrases.” Id.
`
`at 5:25-29.
`
`76.
`
`If the user selects one of the suggested terms or phrases, “the string is
`
`automatically added to the search field” and potentially even “automatically
`
`submitted as the search query.” Id. at 5:42-46.
`
`F. Matthews
`77. Like Ortega, U.S. Patent App. Pub. No. 2006/0218503 (“Matthews”)
`
`(Ex. 1009) does not disclose any tagging.
`
`78.
`
`Instead, Matthews describes “an operating system program launch
`
`menu” (for example, a Start menu) that provides various mechanisms (for example,
`
`a search box within the program launch menu) for a user to “locate and launch
`
`desired data items su