UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________
`
`WHATSAPP INC.
`
`Petitioner
`
`v.
`
`TRIPLAY, INC.
`
`Patent Owner
`
`____________
`
`IPR2016-00717
`
`Patent 8,874,677 B2
`
`DECLARATION OF RAJEEV SURATI, Ph.D.
`
`

`
`TABLE OF CONTENTS
`
`I. QUALIFICATIONS................................................................................ - 1 -
`II. MATERIAL CONSIDERED ............................................................... - 3 -
`III. OVERVIEW AND LEGAL STANDARDS......................................... - 4 -
`IV. SUMMARY OF U.S. PATENT NO. 8,874,677 ................................... - 7 -
`A. Cross-platform delivery of messages including a video ..................... - 7 -
`B. Cross-platform messaging based on pre-defined templates.............. - 10 -
`V. LEVEL OF ORDINARY SKILL IN THE ART................................. - 11 -
`VI. THE SCOPE AND CONTENT OF THE PRIOR ART ...................... - 12 -
`A. Coulombe [Ex. 1003]....................................................................... - 12 -
`B.
`Friedman [Ex. 1005]........................................................................ - 15 -
`C. Bellordre [Ex. 1004] ........................................................................ - 18 -
`D. Meyer [Ex. 1008]............................................................................. - 19 -
`E.
`Ito [Ex. 1007] .................................................................................. - 20 -
`F.
`Surana [Ex. 1006]............................................................................ - 21 -
`VII. CLAIM CONSTRUCTION............................................................. - 22 -
`A.
`“wherein the conversion [of the initial message into an adapted
`message] comprises: a) providing[, by the messaging system,] a
`clickable icon” ......................................................................................... - 22 -
`VIII. VALIDITY ANALYSIS.................................................................. - 28 -
`A. Ground 1: Claims 1, 2, 11, 13, 14, 16, 17, 20, and 21 in relation to
`the combination of Coulombe in view of Bellordre and Friedman ........... - 29 -
`1.
`Friedman does not disclose the “Providing” limitation.................. - 30 -
`2.
`The 717 Petition’s purported motivations for piecing together
`various portions of Coulombe, Friedman and Bellordre to arrive at the
`method of claim 1 are lacking or absent ................................................ - 32 -
`a)
`Claim Limitation 1d: the 717 Petition’s invalidity analysis is
`inadequate because Mr. Klausner failed to consider Coulombe’s
`passages teaching an existing method of video delivery ..................... - 36 -
`b) Claim limitation 1f-1: Motivation offered does not support
`combining Friedman’s attachment methods with Coulombe............... - 40 -
`
`i
`
`

`
`Claim Limitation 1f-2: the 717 Petition offers no motivation to
`c)
`combine Friedman’s clickable icon with Bellordre’s adaptation
`techniques to arrive at an icon that is clickable into an adapted video. - 42 -
`d) Claim Limitation 1g: the 717 Petition offers no motivation to
`combine Coulombe, Friedman and Bellordre
`to arrive at
`determining, by the messaging system, an adapted message layout
`comprising the clickable icon ............................................................ - 43 -
`e)
`Claim Limitations 1a-1h: the 717 Petition offers no motivation
`to combine all the various aspects of Coulombe, Friedman and
`Bellordre to arrive at the claimed invention........................................ - 44 -
`B. Ground 2: Claims 3, 5, 12, and 19 in relation to the combination of
`Coulombe in view of Bellordre and Friedman, and further in view of
`Meyer and Ito........................................................................................... - 45 -
`1.
`Claim 3 ......................................................................................... - 45 -
`a)
`“Template” as used in the ’677 patent ........................................ - 46 -
`b)
`The accused “template”.............................................................. - 48 -
`c)
`Style sheets (CSS files) are not templates................................... - 50 -
`d) A person of ordinary skill in the art would not have been
`motivated to combine Coulombe with Bellordre, Friedman, Meyer,
`and Ito in the way claim 3 requires..................................................... - 52 -
`e)
`Claim Limitations 1a-1h & 3a-3b: the 717 Petition offers no
`motivation to combine all the various aspects of Coulombe,
`Friedman, Bellordre, Meyer, and Ito to arrive at the claimed
`invention. ........................................................................................... - 55 -
`2.
`Claim 5 ......................................................................................... - 56 -
`C. Ground 3: Claims 4 and 18 in relation to the combination of
`Coulombe in view of Bellordre, Friedman, Meyer, and Ito, and further
`in view of Surana ..................................................................................... - 56 -
`1.
`Claim 4 ......................................................................................... - 57 -
`a)
`The Claim 4 combination does not disclose the limitations of
`Claim 4............................................................................................... - 57 -
`b) No motivation to combine Surana with other prior art
`combinations to render claim 4 obvious ............................................. - 60 -
`
`ii
`
`

`
`Claim Limitations 1a-1h, 3a-3b, and 4a-4b: the 717 Petition
`c)
`offers no motivation to combine all the various aspects of
`Coulombe, Friedman, Bellordre, Meyer, Ito, and Surana to arrive at
`the claimed invention ......................................................................... - 63 -
`
`iii
`
`

`
`I, Rajeev Surati, Ph.D., declare as follows:
`
`I.
`
`QUALIFICATIONS
`
`1.
`
`I have more than twenty (20) years of experience in electrical
`
`engineering, computer science, and electronic messaging.
`
`2.
`
`I attended the Massachusetts Institute of Technology (MIT) from
`
`1988 to 1999, during which time I earned Bachelor of Science (1992), Master of
`
`Science (1995), and Doctor of Philosophy (1999) degrees in electrical
`
`engineering and computer science.
`
`3.
`
`I am the inventor of US Patent No. 5,943,478, entitled “System for
`
`Popup Messaging over the Internet,” which describes a two-way messaging
`
`system like AOL Instant Messenger and MIT’s Zephyr service built at Internet
`
`scale.
`
`4.
`
`In 1996, I founded a company called Flash Communications,
`
`which focused on technology related to US Patent No. 5,943,478 and associated
`
`technology that I had developed related to pop-up two-way messaging over the
`
`Internet. Flash Communications was sold to Microsoft Corporation in 1998,
`
`and Flash Communications’ messaging technology was incorporated into
`
`Microsoft’s Messenger service and Microsoft Exchange 2000 Instant Messaging
`
`Server.
`
`5. While working at Microsoft between 1999 and 2000, I
`
`implemented an XML-based protocol that formed a basis for the Extensible
`
`Messaging and Presence Protocol (XMPP), which is now an IETF standard for
`
`- 1 -
`
`

`
`the Exchange Instant Messaging Server. I participated internally with the
`
`program management team on helping specify this protocol for the IETF
`
`standardization process.
`
`6.
`
`Between 2000 and 2004, I worked as a consultant and investor at
`
`Nexaweb Corporation, where I helped implement several two-way messaging
`
`features over HTTP.
`
`7.
`
`Also in 2000, I started a company known as photo.net, which was a
`
`large online photography community where I worked with many consumer
`
`electronics manufacturers in the digital camera business. I also implemented a
`
`number of multimedia transformation systems in implementing some of the first
`
`photo sharing systems for the internet on photo.net. Notably, the website in
`
`outputting HTML and WML formatted documents allowed me to experience
`
`and understand many of the issues related to layout and format and style sheets
`
`discussed in this declaration.
`
`8.
`
`In 2004,
`
`I
`
`founded another company, Scalable Display
`
`Technologies (SDT). I have been the President and Chairman of SDT since its
`
`founding. SDT operates in the audio-video domain and has licensed software
`
`and firmware to various companies including Sony, Hitachi and NEC. I also
`
`implemented a distributed multimedia content playback system and spend a
`
`great deal of time dealing with multimedia transcoding and rendering systems.
`
`9.
`
`I am on the advisory boards of several technology companies,
`
`including:
`
` UnifySquare, which
`
`is a unified communications/realtime
`
`- 2 -
`
`

`
`collaboration consultancy that focuses on telephony and instant messaging
`
`systems that Microsoft sells (Lync, an outgrowth of the company I sold
`
`Microsoft); Paneve, which develops general purpose ASIC coupled with
`
`compiler technology; Nexaweb, which develops realtime web application
`
`frameworks using HTTPS; Antix Labs, which develops compiler technology for
`
`universal gaming platform; Permabit, which develops content addressable
`
`storage; and Evoque, which is an ecommerce enabling platform publisher.
`
`10.
`
`I have received several awards for my contributions as an inventor
`
`and entrepreneur, including the Global Indus Technovator Award 2009 and
`
`Laureate of 2009 Computer World Honors Program.
`
`11. Additional information regarding my qualifications is set forth in
`
`my current curriculum vita, which is attached hereto as Exhibit A.
`
`12.
`
`I have no financial interest in the Petitioner, the Patent Owner, or
`
`the outcome of this proceeding. I am being compensated for my work as an
`
`expert on an hourly basis at the rate of $350 per hour. My compensation is not
`
`dependent on the outcome of these proceedings or the content of my opinions.
`
`II. MATERIAL CONSIDERED
`
`13.
`
`The analysis provided in this Declaration is based on my education
`
`as well as my experience in the field of computer systems, generally, and
`
`electronic messaging systems, in particular. In addition to relying upon my
`
`knowledge based on written materials and other information that was known in
`
`2005, I have considered the 717 Petition for Inter Partes Review of U.S. Patent
`
`- 3 -
`
`

`
`No. 8,874,677, No. IPR2016-00717 (“717 Petition” or “717 Pet.”). I have also
`
`considered the exhibits to the 717 Petition (Exs. 1001-1021), which includes a
`
`Declaration of David Klausner (Ex. 1002).
`
`III. OVERVIEW AND LEGAL STANDARDS
`
`14.
`
`I have been asked to consider the 717 Petition and offer my
`
`opinion on the following questions:
`
`a. What is the broadest reasonable construction of certain claim
`
`terms in the ’677 patent.
`
`b. Whether claims 1, 2, 11, 13, 14, 16, 17, 20, and 21 are obvious
`
`over Coulombe [Ex. 1003] in view of Bellordre [Ex. 1004] and
`
`Friedman [Ex. 1005].
`
`c. Whether claims 3, 5, 12, and 19 are obvious over Coulombe in
`
`view of Bellordre and Friedman, and further in view of Meyer
`
`[Ex. 1008] and Ito [Ex. 1007].
`
`d. Whether claims 4 and 18 are obvious over Coulombe in view of
`
`Bellordre, Friedman, Meyer, and Ito, and further in view of
`
`Surana [Ex. 1006].
`
`15.
`
`I have also been asked to accept, for purposes of this declaration,
`
`the following assertions in the 717 Petition:
`
`a. Claims 1, 2, and 14 are materially identical to claims 11, 16, and
`
`17, respectively (717 Pet. at 32-34);
`
`b. Claims 1, 2, and 14 are materially identical to claims 13, 20, and
`
`- 4 -
`
`

`
`21, respectively (id. at 34-38);
`
`c. Claims 3 and 5 are materially identical to claims 12 and 19,
`
`respectively (id. at 51-52);
`
`d. Claim 4 is materially identical to claim 18 (id. at 59-60).
`
`In view of my acceptance of these assertions, I have not provided a separate
`
`invalidity analysis for claims 11, 16, and 17; claims 13, 20, and 21; claims 12
`
`and 19; or claim 18, because claims that are materially identical to the claims I
`
`analyzed are not obvious in view of the asserted prior art combinations for the
`
`same reasons as the respective representative claims analyzed herein.
`
`16.
`
`I have been informed by counsel that a claim term is given the
`
`broadest reasonable construction that is consistent with the specification of the
`
`patent in which it appears and the understanding of a person of ordinary skill in
`
`the art at the appropriate time.
`
`17.
`
`I have been informed that a patent is obvious if the differences
`
`between the claimed subject matter and the prior art are such that the subject
`
`matter as a whole would have been obvious at the time the invention was made
`
`to a person having ordinary skill in the art.
`
`18.
`
`I understand that a patent claim composed of several elements is
`
`not deemed obvious simply because each of its elements was, independently,
`
`known in the prior art. Instead, it is my understanding that a party contending
`
`that a patent claim is obvious in view of a combination of references must show
`
`that a person of ordinary skill in the relevant field would have had a reason to
`
`- 5 -
`
`

`
`combine the elements disclosed in the references in the manner claimed. I also
`
`understand that the mere fact that it is technically possible to combine
`
`references is not enough of a reason to combine them.
`
`19.
`
`I further understand that a patent challenger’s reason for combining
`
`references must include some rational underpinning to support a legal
`
`conclusion of obviousness. Although the obviousness analysis is not confined
`
`by a formalistic conception of the words teaching, suggestion, and motivation, I
`
`understand that it does require identifying a reason or motivation that would
`
`have prompted a person of ordinary skill in the relevant field to combine the
`
`elements in the way the claimed invention does.
`
`20.
`
`I further understand that a prior art reference must be considered in
`
`its entirety, i.e., as a whole, including portions that would lead away from the
`
`claimed invention. Accordingly, I understand that it is improper to rely on
`
`isolated teachings of the prior art without considering the overall-context within
`
`which the teachings are presented. I have also been informed that the failure to
`
`consider the prior art references as a whole in assembling the elements of a
`
`claimed invention is indicative of impermissible hindsight.
`
`21.
`
`Finally, in order to render opinions on the three obviousness issues
`
`set out above, I understand that I must first address the following factual issues:
`
`(1) resolving the level of ordinary skill in the prior art; (2) determining the
`
`scope and content of the prior art; and (3) ascertaining the differences between
`
`the claimed invention and the prior art.
`
`- 6 -
`
`

`
`IV.
`
`SUMMARY OF U.S. PATENT NO. 8,874,677
`
`A.
`
`Cross-platform delivery of messages including a video
`
`22.
`
`The ’677 patent “relates to [the] field of electronic messaging and
`
`to cross-platform messaging.” (Ex. 1001 [’677 patent] at 1:5-6). In particular,
`
`the ’677 patent recognizes the existence of cross-platform compatibility issues
`
`associated with electronic message “layout” and “format.” With respect to
`
`layout, “[t]he originating and destination devices may have different
`
`communication and displaying capabilities and may use different
`
`communication protocols.”
`
`(Id.
`
`at 11:53-56).
`
` Formats also present
`
`compatibility issues because the digital encoding of a media item at the
`
`originating device (e.g. a video encoded in an MPEG format) may not be
`
`supported at the destination device. (Id. at 12:16-21).
`
`23.
`
`Fig. 2 (reproduced below), depicts certain elements of an
`
`exemplary “messaging system” that includes an “access block” 21 and “media
`
`block” 23. The “access block” 21 includes “a users’ gateway 211 and 3rd party
`
`applications’ gateway 214 supporting communication with communication
`
`devices and 3rd party application(s) via corresponding networks(s) . . . .” (Id. at
`
`13:4-8). The “media block” 23 “comprises transcoder 232 operatively coupled
`
`with a message manager 231 further optionally comprising a template module
`
`51 operatively coupled with the database 26.” (Id. at 16:15-18). The “media
`
`block” is further “configured to select the format and message layout fitting to
`
`the destination device and to convert the message accordingly before facilitating
`
`- 7 -
`
`

`
`its delivery to the destination device.” (Id. at 16:18-20).
`
`24. But in addition to addressing device compatibility issues, the ’677
`
`patent recognized that the delivery of electronic messages containing a video
`
`may be limited by available bandwidth and/or the inability of the receiving
`
`device to download the complete message (because it is too large). (Id. at 14:7-
`
`10, 16:30-32, and 16:67-17:3). Accordingly, the “media block” 23 is also
`
`structured to make delivery decisions that can include a “repackage” of the
`
`message in view of the limitations of bandwidth and destination device
`
`capabilities associated with certain messages. (Id. at 16:27-17:4). The
`
`“repackaging” decisions are controlled by the “message manager” 231 of the
`
`“media block” 23, which may choose to send the converted messages “as one
`
`entity” or “as multiple entities to be re-assembled when received,” and/or to
`
`replace “one or more media items” in the message with “corresponding links.”
`
`- 8 -
`
`

`
`(Id. at 10:55-58).
`
` Thus, media items whose delivery is limited by
`
`“communication media [i.e., bandwidth] and/or destination device” such as
`
`video, can be replaced for delivery with corresponding links. (Id. at 16:64-
`
`17:3).
`
`25.
`
`Further, when the media items are replaced with links, “media
`
`items contain[ed] in the originated message and/or media items converted” are
`
`stored at the intermediary “messaging system” until the link/icon is clicked. (Id.
`
`at 14:1-4). Subsequently, when requested, an adapted version of the media item
`
`is “downloaded to the destination devices via relevant protocols, including but
`
`not limited to HTTP, SMTP, MMS, etc.” or, if the media item is “too large for
`
`successful downloading,” “transmitted to the destination device with the help of
`
`streaming protocols, e.g., RTSP, RTP, etc.” (Id. at 14:4-10).
`
`26.
`
`The links to the media items that are replaced as part of the
`
`repackaging decisions are further described in the ’677 patent as “clickable
`
`icons.” A clickable icon is a visual representation of a link and would have
`
`been understood as such by a person of skill in the art at the time. (See, e.g., Ex.
`
`2005 [Computer Desktop Encyclopedia, Version 18.3 (3rd Quarter 2005)]
`
`defining a “hyperlink” as a “link [which] is displayed either as text or as an
`
`icon”). In Friedman, a “hypergraphic” is the counterpart of hypertext on the
`
`web in which an icon or image is clicked instead of text, and in practice a
`
`hypergraphic is called a “clickable image.” For example, the layout for a cell-
`
`phone in Table 2 describes “a list of clickable icons into reduced media” for
`
`- 9 -
`
`

`
`audio and video items and a “list of clickable media” for images and the layout
`
`for PC describes a “List of clickable media thumbnails.” (Ex. 1001 [’677
`
`patent] at Col. 21, Table 2). The clickable links/icons replacing the media are
`
`further shown in Figures 11 and 12, which show exemplary layouts for a cell-
`
`phone and PC respectively.
`
`27.
`
`The ’677 patent contains no disclosure of the use of links for
`
`delivery of media other than in the context of a repackaging decision in which
`
`links are inserted in place of the media in the message.
`
`B.
`
`28.
`
`Cross-platform messaging based on pre-defined templates
`
`The template-based messaging aspects of the ’677 patent build
`
`upon other aspects of the described messaging system by “facilitat[ing]
`
`composing messages using pre-defined templates.” (Id. at 19:40-42). For
`
`example, the messaging system and/or the client can be configured to provide
`
`the subscriber with a template for composing such messages. (Id. at 20:33-35).
`
`29.
`
`The templates can be of varying types (General, Greeting-like,
`
`- 10 -
`
`

`
`Interactive reply, etc.) and have different “Content Structure.” (Id. at Col. 19,
`
`Table 1). Further, “[e]ach type of template and/or each template is provided
`
`with [a] unique identifier capable to be recognized by the messaging system
`
`and/or client and stored in the message metadata.” (Id. at 19, 45:48).
`
`30. Accordingly, for
`
`the
`
`template-based messaging system,
`
`the
`
`template “serves as an ‘envelope’ for the message during the communication.”
`
`(Id. at 20:52-55). This envelope concept enables the delivery decisions to be
`
`made based upon the unique identifier of a template (which identifies
`
`information about the content structure of the message based on the template
`
`type) and the capabilities of the destination device. (Id. at 20:55-60).
`
`31.
`
`The ’677 patent recognized that an advantage of the template
`
`approach to messaging “is a reduction in need for content analysis.” (Id. at
`
`20:63-65). This reduction in need for content analysis arises from the fact that
`
`the unique identifier provides information to the messaging system about the
`
`content structure of the incoming message. Thus, the system does not have to
`
`expend the resources it otherwise would to determine the content structure of
`
`the incoming message.
`
`V.
`
`LEVEL OF ORDINARY SKILL IN THE ART
`
`32.
`
`It is my opinion that a person of ordinary skill in the art at the time
`
`of the invention of the ’677 patent (i.e., in 2005) is a person with a bachelor’s
`
`degree in either electrical engineering or computer science, at least two years of
`
`experience designing and implementing messaging systems between user
`
`- 11 -
`
`

`
`devices, and at least one year of experience working with format encoding and
`
`layout of images or video.
`
`33. At the time of the invention of the ’677 patent, I was a person of
`
`more than ordinary skill in the art defined above based on my qualifications and
`
`experience. However, my opinions have been rendered based on the perspective
`
`of a person of ordinary skill in the art as of the time of the invention.
`
`34.
`
`I have reviewed the Klausner Declaration [Ex. 1002] and note that
`
`his proposed level of ordinary skill in the art is similar to my own. Applying his
`
`proposed level of ordinary skill in the art would not alter my opinions as
`
`expressed herein.
`
`VI. THE SCOPE AND CONTENT OF THE PRIOR ART
`
`A.
`
`Coulombe [Ex. 1003]
`
`35. Coulombe describes a system for addressing interoperability of
`
`sending messages between terminal devices using the session initiation protocol
`
`(SIP) (Ex. 1003 [Coulombe] at ¶ 1) and describes a “framework” for “the
`
`adaptation of messages between sender and recipient.” (Id. at ¶ 53).
`
`36. As of the May 2002 filing date of Coulombe, SIP supported
`
`session-based communications between participants as well as
`
`instant
`
`messaging applications (which were not session-based). Coulombe recognized
`
`the existence of both modes of SIP communication and sought to build upon the
`
`existing SIP infrastructure to provide a messaging system that supported both
`
`session-based and non-session based communications. (Id. at ¶¶ 65-69).
`
`- 12 -
`
`

`
`37.
`
`Some brief background on the two modes of SIP communication is
`
`useful before I address the specific points of novelty claimed in Coulombe with
`
`respect to messaging.
`
`38.
`
`The protocol for Session-based SIP communications is described
`
`in RFC 2543 (Ex. 2003 [“SIP: Session Initiation Protocol,” IETF, March 1999])
`
`(which is referenced at ¶ 56 of Coulombe). Session-based SIP is described as:
`
`an application-layer control (signaling) protocol for creating,
`modifying and terminating sessions with one or more participants.
`These sessions include Internet multimedia conferences, Internet
`telephone calls and multimedia distribution….SIP invitations used to
`create sessions carry session descriptors which allow participants to
`agree on a set of compatible media types.
`
`(Id. [RFC 2543] at abstract). Media types include video. (See, e.g., id. at § B.1
`
`(configuring media streams)). As the name implies, communications over SIP,
`
`as described in RFC 2543, are session based communications in which
`
`participants negotiate capabilities in advance of the communication. (See, e.g.,
`
`id. at § 1.1).
`
`39. A protocol governing SIP instant messaging is known as SIMPLE
`
`and the version of protocol titled draft-ietf-simple-im-01 [Ex. 2004] is referenced
`
`at ¶ 69 of Coulombe. The protocol defines SIP “instant messaging” as “the
`
`exchange of content between a set of participants in real time. Generally, the
`
`content is short textual messages, although that need not be the case.” (Id. at 3).
`
`SIP instant messages “do not create any implied session” or have “any concept
`
`of call state.” (Id. at 5).
`
`- 13 -
`
`

`
`40.
`
`Turning to Coulombe, the patent application notes that “capability
`
`negotiation occurs between two clients during session establishment (using SDP
`
`(Session Description Protocol)”
`
`in
`
`the context of session-based SIP
`
`communications. (Ex. 1003 [Coulombe] at ¶ 67). But “[w]ithout a session,
`
`which is the case for SIP instant messaging, there [is] no means of knowing the
`
`capabilities of user preferences of the destination terminal.” (Id.). Accordingly,
`
`Coulombe addresses this limitation in SIP instant messaging by “provid[ing] a
`
`method for capability negotiation regardless if the application is session-based or
`
`not.” (Id.).
`
`41.
`
`By addressing the capability negotiation in the case of a non-
`
`session based communication, Coulombe is also able to address the then-existing
`
`limitation of message adaptation in the context of instant messaging. As
`
`Coulombe correctly observed, it was known that SIP “proxies may transcode
`
`content.” (Id. at ¶ 69). Coulombe notes that transcoding could be done for
`
`“multimedia sessions (audio or video calls) where codecs or the bandwidths
`
`between users do not match” based on the capability negotiation described
`
`above, which can be used to “fill the gaps” between the two terminals.
`
`Similarly, the SIP protocol describes the use of session-based negotiation to
`
`exchange capabilities to deliver video streams. (See, e.g., Ex. 2003 [RFC 2543]
`
`at § B.1 (configuring media streams), 135-139). But with respect to SIP instant
`
`messaging, no such negotiation takes place and thus Coulombe observes “that
`
`there is no mention [of] … adaptation ….” (Ex. 1003 [Coulombe] at ¶ 69). In
`
`- 14 -
`
`

`
`support of that point, Coulombe notes that the SIP instant messaging protocol
`
`states that “it should return an error message” if it is determined that “a recipient
`
`doesn’t support a certain format.” (Id. at ¶ 69). Accordingly, by providing the
`
`capability information for non-session based communications, Coulombe sets
`
`forth a proposed SIP architecture that covers communications whether or not
`
`they are session based.
`
`42. Coulombe’s framework for solving the limitations of SIP instant
`
`messaging framework “comprises
`
`three elements
`
`in combination: SIP
`
`proxy/registrar 12, Capability Negotiation Manager 16 and Message Adaptation
`
`Engine 20.” (Id. at ¶ 54). The SIP proxy/registrar 12 is the SIP proxy specified
`
`in RFC 2543 (the SIP session protocol described above). (Id. at ¶ 56).
`
`B.
`
`43.
`
`Friedman [Ex. 1005]
`
`Friedman is entitled “Systems and Methods for Processing
`
`Attachments Associated with Electronic Messages.” It is directed to an
`
`improved electronic message client for managing, displaying and accessing
`
`email attachments. In particular, Friedman notes that “to access an attachment
`
`in a received email message, a recipient typically has to know that a button,
`
`icon, or other element, needs to be selected followed by numerous other steps
`
`that may be involved in opening the attachment.”
`
`(Ex. 1005 [Friedman] at
`
`2:27-30). Per Friedman, while such “traditional methods may be adequate for
`
`knowledgeable or experienced users of electronic mail clients or programs, a
`
`more straightforward and less arcane management structure is desired.” (Id. at
`
`- 15 -
`
`

`
`2:30-34).
`
`44.
`
`The basic operation of the improved email client is described in
`
`Figures 5A and 5B and the accompanying text. In Block 305 of Fig. 5A, an
`
`electronic message including an attachment is received. (Id. at 7:60-8:2). The
`
`attachments are referred to as “objects” and can include images, audio files and
`
`video files, as well as text files. (Id. at 8:2-12). In Block 305, the processing
`
`system “automatically detaches and saves the contents of the object, partially or
`
`wholly, without any manual intervention on the part of the message recipient.”
`
`(Id. at 8:30-35). This “automatic access is carried out irrespective of the type of
`
`electronic message transport or the type of the attachment.” (Id. at 8:36-38).
`
`“In Block 315, a thumbnail graphic is generated for a portion of the object, for
`
`example, the first page of a text document.” (Id. at 8:59-60). Elsewhere,
`
`Friedman also depicts in Fig. 4 a thumbnail graphic 525 of a video file
`
`Riddick.mov. (Id. at Fig.4 and 7:19-21).
`
`45. At Block 305, the various thumbnail graphics generated from the
`
`various received messages are displayed on the graphical user interface
`
`associated with the electronic messaging client. (Id. at 9:7-9). Interface 500 of
`
`Fig. 4 (depicted below) is an example of such an interface.
`
`- 16 -
`
`

`
`46.
`
`“Once the thumbnail graphic is displayed, the attachment is more
`
`easily operated on by the user ….” (Id. at 9:9-11). For example, in Block 335,
`
`the user can “open the attachment [by] … double-clicking … the thumbnail
`
`graphic.” (Id. at 9:23-25). In addition, in block 330, “the user can more easily
`
`carry out a move or a copy operation by acting upon the thumbnail graphic.”
`
`(Id. at 9:12-14).
`
`47.
`
`Friedman notes that the claimed client messaging client application
`
`for detaching and displaying thumbnail graphics can be “located in a client
`
`device, a server device or a combination thereof.” (Id. at 4:33-35). However,
`
`Friedman provides no guidance as to the implementation of the various features
`
`of the messaging client application embodiment that would contain pieces of the
`
`application on both the client device and server device. Absent further guidance
`
`in the specification, a person of skill in the art would not have any
`
`understanding of what features of the system would be located on the server and
`
`- 17 -
`
`

`
`what pieces of the system would be on the client in a combined device.
`
`C.
`
`Bellordre [Ex. 1004]
`
`48. Bellordre is entitled “Method of Processing a Multimedia Message,
`
`a Storage Medium, and an Associated Processing System” and describes a
`
`method of processing multimedia messages between telecommunication
`
`terminals. (Ex. 1004 [Bellordre] at Abstract).
`
`49.
`
`“The
`
`messaging
`
`processing
`
`system
`
`comprises
`
`a
`
`telecommunications terminal 2 of a party sending a multimedia message, a
`
`multimedia server 4, an application server 6, a streaming server 8, and a
`
`telecommunications terminal 10 of the recipient of the multimedia message sent
`
`by terminal 2.” (Id. at ¶ 42). These various elements of the disclosed system
`
`are detailed at Fig. 1, which is reproduced below.
`
`50.
`
`Fig. 2 details the method of the system. The described process
`
`begins with the multimedia message server 4 sending a multimedia message
`
`from terminal 2 to application server 6. (Id. at ¶ 82). Then, if the recipient 10 is
`
`- 18 -
`
`

`
`authorized to use the processing method of the invention, application server 6
`
`will extract audio or video objects to be sent in streaming mode. (Id. at ¶¶ 89-
`
`90). The application server 6 will also adapt the audio or video objects, if
`
`necessary, for delivery to terminal 10 (Id. at ¶ 93) and then send the adapted
`
`object to streaming server 8.
`
`51. At the streaming server 8, an SDP definition file containing the
`
`address of the storage location for the audio or video object is created and sent
`
`to application server 6. (Id. at ¶¶ 98-99). After receipt, the application server 6
`
`substitutes the SDP definition file for the video or audio object that was
`
`previously extracted. (Id. at ¶ 101). Then, after additional processing at the
`
`application server 6, the substitute message is sent to multimedia server 4,
`
`which then communicates the substituted message to the terminal 10 receiving
`
`the message. (Id. at ¶¶ 105-106). Finally, when the terminal receives the
`
`substitute message, the terminal makes a request for the audio or video object
`
`and server 8 sends the audio or video object to the terminal 10 using the
`
`downloading mode (progressive downloading or streaming) appropriate to the
`
`sampling of the audio visual content. (Id. at ¶¶ 107-109).
`
`D. Meyer [Ex. 1008]
`
`52.
`
`The Meyer reference (Ex. 1008) consists of excerpts from a book
`
`on developing Cascading Style Sheets. CSS files (including external style
`
`sheets) were used to separate the visual effects of a web page from the HTM