UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________
`
`WHATSAPP INC. and FACEBOOK, INC.
`
`Petitioner
`
`v.
`
`TRIPLAY, INC.
`
`Patent Owner
`
`____________
`
`IPR2016-01659
`
`Patent 9,049,574
`
`DECLARATION OF RAJEEV SURATI, Ph.D.
`
`PATENT OWNER'S EXHIBIT 2101
`
`
`
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________
`
`WHATSAPP INC. and FACEBOOK, INC.
`
`Petitioner
`
`v.
`
`TRIPLAY, INC.
`
`Patent Owner
`
`____________
`
`IPR2016-01659
`
`Patent 9,049,574
`
`DECLARATION OF RAJEEV SURATI, Ph.D.
`
`PATENT OWNER'S EXHIBIT 2101
`
`
`
`I, Rajeev Surati, Ph.D., declare as follows:
`
`I.
`
`QUALIFICATIONS
`
`1.
`
`I have more than twenty (20) years of experience in electrical
`
`engineering, computer science, and electronic messaging.
`
`2.
`
`I attended the Massachusetts Institute of Technology (MIT) from
`
`1988 to 1999, during which time I earned Bachelor of Science (1992), Master of
`
`Science (1995), and Doctor of Philosophy (1999) degrees in electrical engineering
`
`and computer science.
`
`3.
`
`I am the inventor of US Patent No. 5,943,478, entitled “System for
`
`Popup Messaging over the Internet,” which describes a two-way messaging system
`
`like AOL Instant Messenger and MIT’s Zephyr service built at Internet scale.
`
`4.
`
`In 1996, I founded a company called Flash Communications, which
`
`focused on technology related to US Patent No. 5,943,478 and associated
`
`technology that I had developed related to pop-up two-way messaging over the
`
`Internet. Flash Communications was sold to Microsoft Corporation in 1998, and
`
`Flash Communications’ messaging technology was incorporated into Microsoft’s
`
`Messenger service and Microsoft Exchange 2000 Instant Messaging Server.
`
`5. While working at Microsoft between 1999 and 2000, I implemented
`
`an XML-based protocol that formed a basis for the Extensible Messaging and
`
`Presence Protocol (XMPP), which is now an IETF standard for the Exchange
`
`- 1 -
`
`
`
`
`Instant Messaging Server. I participated internally with the program management
`
`team on helping specify this protocol for the IETF standardization process.
`
`6.
`
`Between 2000 and 2004, I worked as a consultant and investor at
`
`Nexaweb Corporation, where I helped implement several two-way messaging
`
`features over HTTP.
`
`7.
`
`Also in 2000, I started a company known as photo.net, which was a
`
`large online photography community where I worked with many consumer
`
`electronics manufacturers in the digital camera business. I also implemented a
`
`number of multimedia transformation systems in implementing some of the first
`
`photo sharing systems for the internet on photo.net. Notably, the website in
`
`outputting HTML and WML formatted documents allowed me to experience and
`
`understand many of the issues related to layout and format and style sheets
`
`discussed in this declaration.
`
`8.
`
`In 2004, I founded another company, Scalable Display Technologies
`
`(SDT). I have been the President and Chairman of SDT since its founding. SDT
`
`operates in the audio-video domain and has licensed software and firmware to
`
`various companies including Sony, Hitachi and NEC. I also implemented a
`
`distributed multimedia content playback system and spend a great deal of time
`
`dealing with multimedia transcoding and rendering systems.
`
`9.
`
`I am on the advisory boards of several technology companies,
`
`- 2 -
`
`
`
`
`including: UnifySquare, which is a unified communications/realtime collaboration
`
`consultancy that focuses on telephony and instant messaging systems that
`
`Microsoft sells (Lync, an outgrowth of the company I sold Microsoft); Paneve,
`
`which develops general purpose ASIC coupled with compiler technology;
`
`Nexaweb, which develops realtime web application frameworks using HTTPS;
`
`Antix Labs, which develops compiler technology for universal gaming platform;
`
`Permabit, which develops content addressable storage; and Evoque, which is an
`
`ecommerce enabling platform publisher.
`
`10.
`
`I have received several awards for my contributions as an inventor
`
`and entrepreneur, including the Global Indus Technovator Award 2009 and
`
`Laureate of 2009 Computer World Honors Program.
`
`11. Additional information regarding my qualifications is set forth in my
`
`current curriculum vita, which is attached hereto as Exhibit A.
`
`12.
`
`I have no financial interest in the Petitioner, the Patent Owner, or the
`
`outcome of this proceeding. I am being compensated for my work as an expert on
`
`an hourly basis at the rate of $350 per hour. My compensation is not dependent on
`
`the outcome of these proceedings or the content of my opinions.
`
`II. MATERIAL CONSIDERED
`
`13. The analysis provided in this Declaration is based on my education as
`
`well as my experience in the field of computer systems, generally, and electronic
`
`- 3 -
`
`
`
`
`messaging systems, in particular. In addition to relying upon my knowledge based
`
`on written materials and other information that was known in 2005, I have
`
`considered the Petition for Inter Partes Review of U.S. Patent No. 9,055,574, No.
`
`IPR2016-01659 (the “Petition”). I have also considered the exhibits to the Petition
`
`(Exs. 1101-1124), which include a Declaration of David Klausner.
`
`III. OVERVIEW AND LEGAL STANDARDS
`
`14.
`
`I have been asked to consider the Petition and offer my opinion on
`
`whether claims 12-15 and 18-20 are obvious over Bellordre [Ex. 1103] in view of
`
`Han [Ex. 1104], Coulombe [Ex. 1105], Rupper [Ex. 1106], and Huynh [Ex. 1107].
`
`In particular, for purposes of Patent Owner’s preliminary response, I have been
`
`asked to consider whether or not either Bellordre or Han disclose the following
`
`limitation from claim 12:
`
`the media block is configured to:
`
`i)
`
`before delivery to the destination communication device associated
`
`with the at least one receiver, generate a notification of the initial
`
`message that includes the audio file, the notification being an adapted
`
`version of the initial message,
`
`ii)
`
`provide at least an image icon based on an avatar of a sender of the
`
`initial message, and
`
`iii)
`
`determine an adapted layout for the notification, the adapted layout
`
`- 4 -
`
`
`
`
`comprising the image icon based on the avatar of the sender of the
`
`initial message…
`
`In addition, I have been asked to offer my opinion as to whether or not the
`
`Petition’s articulated reasons for the combining Bellordre and Han would have
`
`motivated a person of ordinary skill in the art (“POSITA”) to make the proposed
`
`combination.
`
`15.
`
`I have been informed by counsel that claim terms are given their
`
`broadest reasonable construction that is consistent with the specification of the
`
`patent in which it appears and the understanding of a person of ordinary skill in the
`
`art at the appropriate time.
`
`16.
`
`I have been informed that a patent is obvious if the differences
`
`between the claimed subject matter and the prior art are such that the subject
`
`matter as a whole would have been obvious at the time the invention was made to a
`
`person having ordinary skill in the art.
`
`17.
`
`I understand that a patent claim composed of several elements is not
`
`deemed obvious simply because each of its elements was, independently, known in
`
`the prior art. Instead, it is my understanding that a party contending that a patent
`
`claim is obvious in view of a combination of references must show that a person of
`
`ordinary skill in the relevant field would have had a reason to combine the
`
`elements disclosed in the references in the manner claimed. I also understand that
`
`- 5 -
`
`
`
`
`the mere fact that it is technically possible to combine references is not enough of a
`
`reason to combine them.
`
`18.
`
`I further understand that a patent challenger’s reason for combining
`
`references must include some rational underpinning to support a legal conclusion
`
`of obviousness. Although the obviousness analysis is not confined by a formalistic
`
`conception of the words teaching, suggestion, and motivation, I understand that it
`
`does require identifying a reason or motivation that would have prompted a person
`
`of ordinary skill in the relevant field to combine the elements in the way the
`
`claimed invention does.
`
`19.
`
`I further understand that a prior art reference must be considered in its
`
`entirety, i.e., as a whole, including portions that would lead away from the claimed
`
`invention. Accordingly, I understand that it is improper to rely on isolated
`
`teachings of the prior art without considering the overall-context within which the
`
`teachings are presented. I have also been informed that the failure to consider the
`
`prior art references as a whole in assembling the elements of a claimed invention is
`
`indicative of impermissible hindsight.
`
`IV. LEVEL OF ORDINARY SKILL IN THE ART
`
`20.
`
`It is my opinion that a person of ordinary skill in the art at the time of
`
`the invention of the ’574 patent (i.e., in 2005) is a person with a bachelor’s degree
`
`in either electrical engineering or computer science, at least two years of experience
`
`- 6 -
`
`
`
`
`designing and implementing messaging systems between user devices, and at least
`
`one year of experience working with format encoding and layout of images or
`
`video.
`
`21. At the time of the invention of the ’574 patent, I was a person of more
`
`than ordinary skill in the art defined above based on my qualifications and
`
`experience. However, my opinions have been rendered based on the perspective of
`
`a person of ordinary skill in the art as of the time of the invention.
`
`22.
`
`I have reviewed the Klausner Declaration [Ex. 1102] and note that his
`
`proposed level of ordinary skill in the art is similar to my own. Applying his
`
`proposed level of ordinary skill in the art would not alter my opinions as expressed
`
`herein.
`
`V. ANALYSIS
`
`A. Claim Construction
`
`23. The “Generation” limitation of the Challenged Claims requires the
`
`following:
`
`the media block is configured to:
`
`iv)
`
`before delivery to the destination communication device associated
`
`with the at least one receiver, generate a notification of the initial
`
`message that includes the audio file, the notification being an adapted
`
`version of the initial message,
`
`- 7 -
`
`
`
`
`v)
`
`provide at least an image icon based on an avatar of a sender of the
`
`initial message, and
`
`vi)
`
`determine an adapted layout for the notification, the adapted layout
`
`comprising the image icon based on the avatar of the sender of the
`
`initial message… (Ex. 1101 at Claim 12.)
`
`24. A person of ordinary skill in the art (POSITA) would understand that
`
`the “the notification” is “an adapted version of the initial message” – in other
`
`words, the notification is a message. Based on the underlined text above, that
`
`“notification” message is further defined by “an adapted layout,” and that adapted
`
`layout must include “an image icon based on an avatar of the sender” that is
`
`provided by the messaging system.
`
`25. Thus, the plain language of the “Generation” limitation requires
`
`inserting “an image icon based on an avatar of the sender” into a notification, and
`
`that notification is a message. That the notification is a message is clear from the
`
`claim language that states “the notification being an adapted version of the initial
`
`message.”
`
`B.
`
`Bellordre does not disclose a messaging system that generates a
`message that includes an avatar image of the sender
`
`26. As described in Figs. 1 & 2 and in the accompanying text, Bellordre
`
`discloses a messaging system for the delivery of multi-media messages that include
`
`video and audio objects. (See also Ex. 1103 [Bellordre] at ¶¶ 46-47.) The
`
`- 8 -
`
`
`
`
`messaging system extracts the audio/video objects, stores the objects, and
`
`generates a substitute message that is delivered to a recipient. (Ex. 1103
`
`[Bellordre] at ¶¶ 84-106.) Visually, this is illustrated by Figs. 1, 3, and 4 –
`
`components within “application server 6” (Fig. 1, highlighted in yellow) receives a
`
`“multimedia message 21” (Fig. 3, highlighted in blue) and creates a “substitute
`
`message 26” (Fig. 4, highlighted in green):
`
`27. Then, after receipt of the substitute message, the receiving terminal
`
`can make a streaming request to the streaming server in Bellordre to download the
`
`audio or video object. (Ex. 1003 [Bellordre] at ¶ 109.)
`
`
`
`- 9 -
`
`
`
`
`28. The Petition points to Bellordre’s substitute message (i.e., the one that
`
`is generated after extraction of the audio and video objects) as the generation, by
`
`the messaging system, of the claimed message of notification. (Petition at 34-35.)
`
`The Petition also alleges that Bellordre describes a number of different objects that
`
`are included in the substitute message including, for example, sequence 27, which
`
`is described as a sequence that is “representative” of the audio or video object.
`
`(Petition at 38; see also Bellordre at ¶63.) Bellordre does not describe any image
`
`objects relating to the identity of the sender that are added to or included in the
`
`substitute message. (Ex. 1103 [Bellordre] at ¶¶ 46-49.)
`
`29. Bellordre does not disclose including into the substitute message an
`
`“image icon based on an avatar of the sender of the initial message.” The Petition
`
`admits this fact. (Petition at 34.) Instead, the Petition relies on the combination of
`
`Bellordre and Han to disclose the limitation. (Petition at 39.)
`
`C. Han does not disclose a messaging system that generates a
`message that includes an avatar image of the sender.
`
`30. Han describes its invention as a “multimedia communication method
`
`using a virtual world interface and an instant messenger program.” (Ex. 1104
`
`[Han] 1:10-12.) Han also states that an object of the invention is “a mobile
`
`personal computer [that] is capable of readily transmitting or retrieving multimedia
`
`data using a virtual world interface and an instant message program.” (Id. at 2:3-
`
`5.) As discussed below, Han’s avatar disclosures relate to creating the “virtual
`
`- 10 -
`
`
`
`
`world interface” that is displayed while the exchange of instant messages occurs.
`
`Han has no disclosure of including avatars as part of the exchange of instant
`
`messages.
`
`31. The Han System, as depicted in Fig. 1 below, “comprises a messaging
`
`server 30 connected to a plurality of clients 20a-20c via a communication line 10
`
`… and a database (DB) server 40 connected to the messaging server 30.
`
`32. After user authentication, a user is presented with an initial menu
`
`picture of the “Virtual World Interface” as shown in Fig. 3:
`
`
`
`- 11 -
`
`
`
`
`
`
`(Ex. 1104 [Han] at 3:64-66.) If the, “avatar chatting” function is selected from the
`
`above “Virtual World Interface,” the avatars shown in Fig. 5 (reproduced below)
`
`are displayed on the screen. (Ex. 1104 [Han] at 4:24-25.) The “avatars represent
`
`clients connected to server 30, respectively, and information thereof must be pre-
`
`registered in DB server 40.” (Id. 4:25-28).
`
`- 12 -
`
`
`
`
`
`
`33.
`
` After creating the user-interface association between a user and an
`
`avatar, the avatars can be used to select the recipient of a chat message or the
`
`recipient of a message including a file. (Ex. 1104 [Han] at 4:65-5:6.) However,
`
`Han contains no disclosure of including the avatars in the delivered message, and
`
`the Petition does not argue that Han contains such a disclosure. Rather, the avatars
`
`are displayed as part of the user interface associated with the chatting system.
`
`This is best illustrated in Fig. 6 (depicted below), which shows the display of
`
`avatars as part of the “virtual world interface” on left while the ongoing instant
`
`messaging occurs on the right. (Ex. 1104 [Han] at Fig. 6.) In the chat, the parties’
`
`identities are simply the names of the senders (e.g. “Blue,” “Cute,” etc.). (Ex.
`
`1104 [Han] at Fig. 6.) The sender’s avatars are not shown as part of the chat
`
`message.
`
`- 13 -
`
`
`
`
`
`
`34. The Petition discusses the avatars disclosed in Han but that disclosure
`
`in Han relates only to the “virtual world interface” in which the avatars in Han
`
`identify users in the system. However, as noted above, the Petitioner does not even
`
`argue that Han discloses including an avatar in a delivered message, nor could it.
`
`35. As I discussed above, the plain language of the “Generation”
`
`limitation requires inserting “an image icon based on an avatar of the sender” into
`
`a notification, and that notification is a message. Han’s disclosure of “providing”
`
`of an “avatar of a sender” as part of a “virtual world interface” does not disclose
`
`the claim limitation.
`
`D. Even If The Board Finds That The Combination Of Bellordre
`And Han Discloses The Claimed Limitation, The Petition Has Not
`
`- 14 -
`
`
`
`
`Offered Any Rational Motivation For Why A POSITA Would
`Have Been Motivated To Combine The Two.
`
`36. The Petition offers the following six rationale and motivations for for
`
`combining Bellordre and Han:
`
`
`
`
`
`
`
`
`
`1. “It would have been obvious to … to combine … with no change in
`their respective functions.” (Petition at 41.)
`
`2. “The combination would have predictably resulted in the messaging
`system of Bellordre with the additional capability to provide an image
`icon of the avatar of the sender of the initial message.” (Id. at 41-42.)
`
`3. “Bellordre and Han are analogous reference in the same field of
`providing messaging
`services between
`computing devices.
`(Bellordre, ¶ 0004; Han, 1:8-12)” (Petition at 42.)
`
`4. A POSITA “would have found Han to be a natural combination with
`Bellordre.” (Id.)
`
`5. “As explained by Mr. Klausner, the use of an image icon to represent
`an object in a computing system was known since the earliest days of
`graphical user interfaces, by at least the release of the Apple
`Macintosh line of computers in the 1980s. It was well-known … that
`using icons to represent objects … can make a user interface seem
`more friendly and intuitive to the user. It is therefore not surprising
`that Han would state that using avatars can improve user experience
`by facilitating user expression beyond the traditional channels of “text
`
`- 15 -
`
`
`
`
`and voice,” such that users “can more realistically and intimately chat
`with one another.” (Han, 6:51-54.) [A POSITA] would have
`recognized that inserting an image icon of the sender’s avatar into the
`notification could have improved the system of Bellordre by
`providing
`the advantages of more
`realistic and
`intuitive
`communication.” (Id.)
`
`
`
`6. “[A POSITA] would have perceived no technical obstacle to making
`this combination. In fact, Bellordre specifically notes that the
`substitute multimedia message 26 can include “picture objects.” [A
`POSITA] would have understood that this picture object capability
`could have been used to provide an image icon based on the user’s
`avatar with the substitute message 26.” (Petition at 42-43.)
`
`
`37.
`
`It is my opinion that all the above motivations are conclusory and do
`
`not set forth basis for why a POSITA would have been motivated to make the
`
`proposed combination of Bellordre and Han.
`
`38. As to motivation 2, the Petition offers no rationale explaining why a
`
`POSITA would have been motivated to add the capability of providing an image
`
`icon of a user’s avatar to Bellordre’s substitute message.
`
`39. Han and Bellordre are disparate references, and Petitioner provides no
`
`support for its conclusion that the two references could be combined “with no
`
`change in their respective functions,” “with predictable results,” or as a “natural
`
`combination.” Han describes a “virtual world interface in mobile personal
`
`- 16 -
`
`
`
`
`computers, wherein information can rapidly and accurately be exchanged at any
`
`time or place.” (Ex. 1104 [Han] at 1:64-67). Bellordre describes a messaging
`
`system for sending a multimedia message that include a video or audio object
`
`between communication terminals. The method described includes: (i) removing,
`
`processing and storing the video or audio objects, (ii) generating a substitute
`
`message to be delivered to the destination terminal, and (iii) streaming the video or
`
`audio object upon request from the destination terminal.
`
`40. The alleged support for Petitioner’s contention that Bellordre and Han
`
`are “analogous” are unrelated, high-level descriptions from the “Background”
`
`section of Bellordre (Ex. 1103 at ¶ 4) and the “Field of the Invention” section of
`
`Han (Ex. 1104 at 1:8-12)
`
`41.
`
`I agree with Mr. Klausner that a POSITA would know that “using
`
`icons to represent objects … can make a user interface seem more friendly and
`
`intuitive to the user.” But that statement doesn’t explain why a POSITA would
`
`have been motivated to insert avatars into the substitute message in Bellordre.
`
`Indeed, Bellordre has no discussion relating to user interfaces.
`
`42. The Petition’s motivations cite to Han for the statement that “using
`
`avatars can improve user experience by facilitating expression … such that user
`
`‘can more realistically and intimately chat with one another” (quoting Han at 6:51-
`
`54). However, a POSITA would not understand how that statement would relate to
`
`- 17 -
`
`
`
`
`Bellordre and the Petition provides no explanation for connecting that statement in
`
`Han to Bellordre in support of a combination.
`
`43. The Petition also states that a POSITA “would have recognized that
`
`inserting an image icon of the sender’s avatar into the notification could have
`
`improved the system of Bellordre by providing the advantages of more realistic
`
`and intuitive communication.” But the Petition does not offer an explanation
`
`regarding how inserting avatars into the message of Bellordre “could have
`
`improved the system of Bellordre by providing the advantages of more realistic
`
`and intuitive communication.” Based on the information provided in the Petition,
`
`a POSITA would not have understood how inserting an avatar into the notification
`
`would have result in an improved system of Bellordre with advantages of a more
`
`realistic and intuitive communication.
`
`44. The Petition’s reference to the fact that the picture object capability in
`
`Bellordre “could have been used to provide an image icon based on the user’s
`
`avatar” is not a reason why a POSITA would have been motivated to do it.
`
`Bellordre discusses a number of different objects that are included in the substitute
`
`message, but it does not discuss inserting image icons in the form of avatars related
`
`to the sender of a message. (Bellordre [Ex. 1103] at ¶ 47.) As I discussed above,
`
`Petitioner alleges that avatars “could” have improved the Bellordre system, but
`
`provides no explanation of how doing so would yield a more realistic or intuitive
`
`- 18 -
`
`
`
`
`communication. (See Petition at 42.) Yet the Petition contains no discussion of
`
`how doing so would yield a more realistic and intuitive communication. Further,
`
`that justification would make little sense in view of Bellordre, which does not
`
`provide any details regarding a user interface.
`
`
`
`Dated: December 22, 2016
`
`Respectfully submitted,
`
`
`
`
`
`
`
`Rajeev Surati, Ph.D.
`
`- 19 -
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
