`
`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________________
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`Google LLC,
`
`Petitioner,
`
`v.
`
`Touchstream Technologies, Inc.,
`
`Patent Owner.
`
`U.S. Patent No. 8,356,251
`Filing Date: September 26, 2011
`Issue Date: January 15, 2013
`
`____________________
`
`Case No. IPR2022-00795
`
`
`
`DECLARATION OF BENJAMIN B. BEDERSON, PH.D.
`
`1
`
`Page 1 of 350
`
`GOOGLE EXHIBIT 1005
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`TABLE OF CONTENTS
`
`
`INTRODUCTION ........................................................................................... 1
`I.
`II. QUALIFICATIONS ........................................................................................ 2
`III. MATERIALS CONSIDERED ...................................................................... 11
`IV. LEGAL PRINCIPLES ................................................................................... 12
`V.
`THE ’251 PATENT ....................................................................................... 14
`VI. OVERVIEW OF PRIOR ART ...................................................................... 18
`A. Muthukumarasamy .............................................................................. 18
`B. Hayward .............................................................................................. 23
`VII. LEVEL OF ORDINARY SKILL IN THE ART ........................................... 25
`VIII. CLAIM CONSTRUCTION .......................................................................... 27
`IX. SUMMARY OF OPINIONS ......................................................................... 27
`X. GROUNDS I AND II: MUTHUKUMARASAMY ALONE OR IN
`VIEW OF HAYWARD RENDERS OBVIOUS THE CHALLENGED
`CLAIMS ........................................................................................................ 30
`A.
`Summary of Grounds I and II ............................................................. 30
`1.
`Ground I: Muthukumarasamy Would Have Rendered
`Obvious Claims 1-2 and 5-9 ..................................................... 30
`Ground II: Muthukumarasamy and Hayward Would
`Have Rendered Obvious Claims 1-2 and 5-9 ........................... 31
`Independent Claim 1 ........................................................................... 35
`1.
`Element 1(pre) - a machine-implemented method: of
`controlling presentation of video content on a display
`device that loads any one of a plurality of different media
`players ....................................................................................... 35
`
`2.
`
`B.
`
`
`
`i
`
`Page 2 of 350
`
`

`

`
`
`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`Element 1(a) - assigning, by a server system, a
`synchronization code to the display device .............................. 42
`Element 1(b) - receiving, in the server system, a message
`from a personal computing device that is separate from
`the server system and separate from the display device,
`wherein the message includes the synchronization code .......... 46
`Element 1(c) - storing, by the server system, a record
`establishing an association between the personal
`computing device and the display device based on the
`synchronization code ................................................................ 48
`Element 1(d)(i) - receiving, in the server system, one or
`more signals from the personal computing device ................... 50
`Element 1(d)(ii) - the one or more signals specifying a
`video file to be acted upon and identifying a particular
`media player for playing the video content .............................. 50
`Element 1(d)(iii) - the one or more signals further
`including a universal playback control command for
`controlling playing of the video content on the display
`device by the particular media player ....................................... 58
`Element 1(e)(i) - converting, by the server system, the
`universal playback control command into corresponding
`programming code to control playing of the video
`content on the display device by the particular media
`player ......................................................................................... 65
`Element 1(e)(ii) - wherein converting the universal
`playback control command includes selecting from
`among a plurality of specific commands each of which
`represents a corresponding playback control command
`for a respective media player .................................................... 70
`10. Element 1(f) - storing, in a database associated with the
`server system, information for transmission to or retrieval
`by the display device, wherein the information specifies
`
`7.
`
`8.
`
`9.
`
`ii
`
`Page 3 of 350
`
`

`

`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`the video file to be acted upon, identifies the particular
`media player for playing the video content, and includes
`the corresponding programming code to control playing
`of the video content on the display device by the
`particular media player in accordance with the universal
`playback control command ....................................................... 76
`C. Dependent Claims ............................................................................... 81
`1.
`Claim 2 ...................................................................................... 81
`a.
`Element 2(a) - The method of claim 1 including:
`checking, in the server system, the identity of the
`media player identified in the one or more signals
`from the personal computing device .............................. 81
`Element 2(b) - loading an appropriate set of
`protocols or application programming interfaces
`from a library based on the identity of the media
`player .............................................................................. 85
`Element 2(c) - converting the command from the
`personal computing device into corresponding
`code to control the media player..................................... 88
`Claim 5 - The method of claim 1 wherein the universal
`command represents an instruction to play the video
`content, to stop playing the video content or to pause
`playing the video content .......................................................... 89
`Claim 6 - The method of claim 1 wherein the video
`content is an interactive video game ......................................... 90
`Claim 7 - The method of claim 1 wherein the video
`content is streaming media ........................................................ 90
`Claim 8 - The method of claim 1 wherein the
`synchronization code is uniquely associated with the
`display device on which the video content is to be played ....... 90
`
`2.
`
`3.
`
`4.
`
`5.
`
`b.
`
`c.
`
`iii
`
`
`
`
`
`
`
`
`Page 4 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`6.
`
`Claim 9 - The method of claim 8 wherein the
`synchronization code is different from an IP address
`associated with the display device and is different from a
`MAC address associated with the display device ..................... 91
`XI. CONCLUSION .............................................................................................. 92
`
`
`
`iv
`
`Page 5 of 350
`
`

`

`
`
`
`
`I.
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`INTRODUCTION
`1. My name is Benjamin B. Bederson, Ph.D. I am currently Professor
`
`Emeritus of Computer Science at the University of Maryland. I have been retained
`
`as an independent expert by Google LLC (“Google”) in this proceeding.
`
`2.
`
`I have been asked to provide my opinion as to whether claims 1, 2 and
`
`5-9 of U.S. Patent No. 8,356,251 are patentable. This report sets forth my opinions
`
`and the bases of my opinions.
`
`3.
`
`All of the opinions stated herein are based on my own personal
`
`knowledge and professional judgment and my analysis of the materials and
`
`information I considered in preparing this report. The materials I reviewed in
`
`connection with preparing this report are listed below. I make the following
`
`statements based on my own personal knowledge and, if called as a witness, I could
`
`and would testify to the following. I am being paid for my work in this litigation at
`
`the rate of $600.00 per hour, plus reimbursement of reasonable expenses. My
`
`compensation does not depend on the outcome of this proceeding, and I have no
`
`financial interest in that outcome.
`
`
`
`1
`
`Page 6 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`II. QUALIFICATIONS
`4. My qualifications for forming the opinions set forth in this report are
`
`summarized here and are addressed more fully in my résumé, which is attached
`
`hereto as Exhibit 1006.
`
`5.
`
`Based on my education, research, and work experience, I am
`
`knowledgeable about the subject matter of the ’251 patent and the related prior art.
`
`I am currently Professor Emeritus of Computer Science at the University of
`
`Maryland ("UMD"). From 2014 to 2018, I was the Associate Provost of Learning
`
`Initiatives and Executive Director of the Teaching and Learning Transformation
`
`Center at the UMD. I am a member and previous director of the Human-Computer
`
`Interaction Lab ("HCIL"), the oldest and one of the best known Human-Computer
`
`Interaction ("HCI") research groups in the country. I was also co-founder and Chief
`
`Scientist of Zumobi, Inc. from 2006 to 2014, a Seattle-based startup that is a
`
`publisher of content applications and advertising platforms for smartphones. I am
`
`also co-founder and co-director of the International Children's Digital Library
`
`("ICDL"), a web site launched in 2002 that provides the world's largest collection of
`
`freely available online children's books from around the world with an interface
`
`aimed to make it easy for children and adults to search and read children's books
`
`online. I am also co-founder and prior Chief Technology Officer of Hazel Analytics,
`
`
`
`2
`
`Page 7 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`a data analytics company to improve food safety and better public health whose
`
`product sends alerts in warranted circumstances. In addition, I have for more than
`
`25 years consulted for numerous companies in the area of user interfaces, including
`
`Hillcrest Labs, Lockheed Martin, Logitech, Microsoft, NASA Goddard Space Flight
`
`Center, the Palo Alto Research Center, and Sony.
`
`6.
`
`For more than 30 years, I have studied, designed, and worked in the
`
`field of computer science and HCI. That
`
`includes
`
`teaching, research,
`
`entrepreneurship, and administration in HCI and the software and technology
`
`underlying interactive computing systems. I have worked on the design and
`
`implementation of hardware and software systems including the use of capacitive,
`
`resistive, optical and IR among other sensors, and interactive applications on a range
`
`of devices, including embedded systems, controllers, smart phones, and PDAs for
`
`2D and 3D interfaces.
`
`7.
`
`In 1992, I completed my Ph.D. dissertation at New York University
`
`titled "A Miniature Space-Variant Active Vision System: Cortex-I" in which I built
`
`a custom small motor that aimed a custom camera along with a custom DSP-based
`
`computing system. Collectively, these worked together to create a miniature
`
`computer vision system that one could carry in one hand - dramatically smaller than
`
`
`
`3
`
`Page 8 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`other computer vision systems of the day. This system included the ability to
`
`generate a real-time video signal with the result of the image processing. Appx. 1.
`
`8.
`
`Starting in the mid-1990s and going through the 2000s, I worked on a
`
`range of "zoomable user interfaces" (ZUIs) that supported users in navigating large
`
`information spaces by zooming in and out of the information on the screen. I built
`
`several different ZUI systems over the years, including Pad++, Jazz, and Piccolo.
`
`Starting in the early 2000s, I adapted that work to work on mobile devices including
`
`PDAs, tablets, and mobile phones. Appx. 2.
`
`9.
`
`Starting in 2001 and continuing for several years, I worked on photo
`
`management systems. I wrote a paper in 2001 describing PhotoMesa, a zoomable
`
`image browser using a novel 2D layout algorithm I called "quantum treemaps" to
`
`position the images on the screen in a way that attempted to fill the screen while
`
`keeping groups of related images together. Appx. 3. As indicated in the figure below
`
`from that paper, PhotoMesa could display many hundreds of images at a time by
`
`showing small thumbnails, by showing larger thumbnails when the mouse hovered
`
`over a small thumbnail, and by allowing a user to zoom in to see a high resolution
`
`version of the images.
`
`
`
`4
`
`Page 9 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`In 2003, I wrote a paper that performed image processing to identify
`
`10.
`
`salient portions of images and then automatically crop those images to show those
`
`portions. Appx. 4. In this way, image thumbnails could focus attention on faces or
`
`other objects of a photo without showing all of the background. In 2006, I wrote a
`
`paper that explained some of the trends in photo management software including
`
`rich support for annotation, browsing and sharing of photos. Appx. 7. In 2007, I
`
`published another paper that provided a review of innovative personal photo
`
`management work going on at that time. Appx. 8. That paper describes a version of
`
`my PhotoMesa software that by 2007 as shown in the following figure, included the
`
`ability to annotate photos with the location and name of people in photos.
`
`
`
`5
`
`Page 10 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`I was heavily involved in the research and development of mobile
`
`11.
`
`device applications during the early 2000's. For example, I envisioned, designed and
`
`built DateLens in 2002-2004 to create a richer calendar for the Microsoft PocketPC
`
`platform as well as desktops as described in a 2004 paper. Appx. 5. One of its
`
`innovations was its design that enabled it to scale between small (mobile) and large
`
`(desktop) computers. It also had a special double-headed scroll bar that allowed a
`
`
`
`6
`
`Page 11 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`user to touch scroll and also control how much information was presented on the
`
`screen at a time. I started DateLens as a research project and eventually sold it
`
`commercially. DateLens displayed calendar data that came from the Microsoft
`
`PocketOutlook database and allowed modification and viewing of appointments.
`
`The research website from that time period that describes DateLens is still available
`
`at http://www.cs.umd.edu/hcil/datelens/ which shows screenshots and a picture of
`
`me demonstrating DateLens to Bill Gates. The commercial website from that time
`
`period is also available at http://www.windsorinterfaces.com/datelens.shtml, which
`
`shows a number of news articles reviewing DateLens at that time.
`
`12.
`
`In 2004-2005, with graduate student Amy Karlson and John
`
`SanGiovanni of Microsoft, I worked on a prototype software called LaunchTile.
`
`LaunchTile allowed zooming and scrolling of multiple applications on a handheld
`
`touchscreen PDA. The interface offered a grid of applications that could be zoomed
`
`and scrolled. It included several different methods of scrolling, including gestures
`
`where the scrolling continued after a finger was lifted from the touch screen, and
`
`then slowed to a stop. Appx. 6.
`
`13. That work led to my co-founding of Zumobi, where I was responsible
`
`for investigating new software platforms and developing new user interface designs
`
`that provide efficient and engaging interfaces for mobile touchscreen platforms
`
`
`
`7
`
`Page 12 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`including the iPhone and Android-based devices. For example, I designed and
`
`implemented software called "Ziibii," a "river" of news for iPhone, software called
`
`"ZoomCanvas," a zoomable user interface for several iPhone apps, and iPhone apps
`
`including "Inside Xbox" for Microsoft and Snow Report for REI. All of these
`
`applications included touch based zooming and scrolling.
`
`14. At UMD, I am focused primarily on the area of HCI, a field that relates
`
`to the development and understanding of computing systems to serve users' needs.
`
`Researchers and practitioners in this field are focused on making universally usable,
`
`useful, efficient, and appealing systems to support people in their wide range of
`
`activities. My approach is to balance the development of innovative technology that
`
`serves people's practical needs. Example systems following this approach that I have
`
`built include Cortex-I (1992 embedded computer vision system that sensed licensed
`
`plates with custom motor, camera, and controller), Audio Augmented Reality (1995
`
`embedded system for sensing a user's location and playing audio suited to that
`
`location), Fisheye Menus (2000 software for sensing movement within and selection
`
`of linear list of items in a menu), PhotoMesa (2001 software for end users to browse
`
`personal photos), DateLens (2002 software for end users to use their mobile devices
`
`to efficiently access their calendar information), SlideBar (2005 linear sensor to
`
`control scrolling), LaunchTile (2005 "home screen" software for mobile devices to
`
`
`
`8
`
`Page 13 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`allow users to navigate apps in a zoomable environment), SpaceTree (2001 software
`
`for end users to efficiently browse very large hierarchies), ICDL (as described
`
`above), and StoryKit (a 2009 iPhone app for children to create stories).
`
`15. My work has been published extensively in more than 160 technical
`
`publications, and I have given about 100 invited talks, including 9 keynote lectures.
`
`I have won a number of awards including the Brian Shackel Award for "outstanding
`
`contribution with international impact in the field of HCI" in 2007, and the Social
`
`Impact Award in 2010 from the Association for Computing Machinery's ("ACM")
`
`Special Interest Group on Computer Human Interaction ("SIGCHI"). ACM is the
`
`primary international professional community of computer scientists, and SIGCHI
`
`is the primary international professional HCI community. I have been honored by
`
`both professional organizations. I am an "ACM Distinguished Scientist," which
`
`"recognizes those ACM members with at least 15 years of professional experience
`
`and 5 years of continuous Professional Membership who have achieved significant
`
`accomplishments or have made a significant impact on the computing field." I am a
`
`member of the "CHI Academy," which is described as follows: "The CHI Academy
`
`is an honorary group of individuals who have made substantial contributions to the
`
`field of HCI. These are the principal leaders of the field, whose efforts have shaped
`
`the disciplines and/or industry, and led the research and/or innovation in human-
`
`
`
`9
`
`Page 14 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`computer interaction." The criteria for election to the CHI Academy are: (1)
`
`cumulative contributions to the field; (2) impact on the field through development
`
`of new research directions and/or innovations; and (3) influence on the work of
`
`others.
`
`16.
`
`I have appeared on radio shows numerous times to discuss issues
`
`relating to user interface design and people's use and frustration with common
`
`technologies, web sites, and mobile devices. My work has been discussed and I have
`
`been quoted by mainstream media around the world over 120 times, including by
`
`the New York Times, the Wall Street Journal, the Washington Post, Newsweek, the
`
`Seattle Post Intelligencer, the Independent, Le Monde, NPR's All Things
`
`Considered, New Scientist Magazine, and MIT's Technology Review.
`
`17.
`
`I have designed, programmed, and publicly deployed dozens of user-
`
`facing software products that have cumulatively had millions of users. My work is
`
`cited by several major companies, including Amazon, Apple, Facebook, Google, and
`
`Microsoft. I am a named inventor on 14 U.S. patents and 19 U.S. patent applications.
`
`The patents are generally directed to user interfaces and experience.
`
`18.
`
`I received a B.S. degree in Computer Science with a minor in Electrical
`
`Engineering in 1986 from the Rensselaer Polytechnic Institute. I received M.S. and
`
`
`
`10
`
`Page 15 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`Ph.D. degrees in Computer Science in 1989 and 1992, both from New York
`
`University.
`
`III. MATERIALS CONSIDERED
`19.
`In forming my opinions, I have reviewed the following documents, and
`
`any other document cited in this declaration:
`
`Exhibit
`Ex-1001
`Ex-1002
`Ex-1003
`Ex-1008
`Ex-1009
`Ex-1010
`
`Ex-1011
`Ex-1014
`
`Description
`U.S. Patent No. 8,356,251 to Strober (“the ’251 Patent”)
`RESERVED
`’251 Patent Prosecution History (U.S. App. No. 13/245,001)
`U.S. Patent App. Pub. No. 2010/0241699 (“Muthukumarasamy”)
`U.S. Patent No. 8,918,812 (“Hayward”)
`Touchstream Technologies, Inc.’s Opening Claim Construction
`Brief, Touchstream Techs., Inc. v. Vizbee, Inc., No. 1:17-cv-6247-
`PGG-KNF (S.D.N.Y., Sept. 4, 2018)
`U.S. Patent App. Pub. No. 2011/00600998 (“Schwartz”)
`Touchstream Techs., Inc. v. Google LLC, No. 6:21-cv-00569-ADA,
`Joint Claim Construction Statement (W.D. Tex. Feb. 8, 2022)
`
`20.
`
`I have also relied on my education, experience, research, training, and
`
`knowledge in the relevant art, and my understanding of any applicable legal
`
`principles described in this declaration.
`
`21. All of the opinions contained in this declaration are based on the
`
`documents I reviewed and my knowledge and professional judgment. My opinions
`
`have also been guided by my understanding of how a person of ordinary skill in the
`
`art would have understood the claims of the ’251 Patent at the time of the alleged
`
`
`
`11
`
`Page 16 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`invention. For purposes of this declaration, I have been asked to assume that the date
`
`of the alleged invention is the earliest claimed priority date: April 21, 2011.
`
`22.
`
`I reserve the right to supplement and amend any of my opinions in this
`
`declaration based on documents, testimony, and other information that becomes
`
`available to me after the date of this declaration.
`
`IV. LEGAL PRINCIPLES
`23.
`It is my understanding that there are two ways that prior art references
`
`can render a patent claim unpatentable: anticipation and obviousness. Counsel has
`
`informed me that the petitioner has the burden in an inter partes review to show
`
`unpatentability by a preponderance of the evidence.
`
`24.
`
`I also understand that there is a set process as follows: (a) the claims of
`
`a patent are properly construed, (b) then, you must compare the claim language to
`
`the prior art on a limitation-by-limitation basis. If the prior art reference contains all
`
`the elements of the claim language (explicitly or inherently), arranged as in the
`
`claims, then that is considered anticipation.
`
`25.
`
`I understand that an invention is obvious when the differences between
`
`the subject matter sought to be patented and the prior art are such that the subject
`
`matter as a whole would have been obvious at the time of the invention to a person
`
`having ordinary skill in the art. For this reason, I have been asked to consider the
`
`
`
`12
`
`Page 17 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`level of ordinary skill in the field that someone would have had at the time of the
`
`claimed invention.
`
`26. Counsel has also instructed me that in an obviousness determination,
`
`the factors to consider are (1) the scope and content of the prior art, (2) the
`
`differences between the prior art and the asserted claims, (3) the level of ordinary
`
`skill in the pertinent art, and (4) the existence of secondary considerations of
`
`nonobviousness. Secondary considerations include a long-felt need, commercial
`
`success, unexpected results, praise of the invention, licensing, copying, failure of
`
`others, and skepticism by experts.
`
`27. Counsel has also instructed me that an obviousness inquiry may involve
`
`assessing the motivation of a person of ordinary skill to combine references. The
`
`prior art references themselves may provide a suggestion, motivation, or reason to
`
`combine, but other sources may also support a rationale for combining two or more
`
`prior art references.
`
`28.
`
`It is also my understanding through counsel that the combination of
`
`familiar elements according to known methods is likely to be obvious when it does
`
`no more than yield predictable results. It is further my understanding that a proper
`
`obviousness analysis focuses on what was known or obvious to a person of ordinary
`
`skill in the art, not just the patentee.
`
`
`
`13
`
`Page 18 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`29.
`
`I have been informed that, in this proceeding, claim terms in the ’251
`
`Patent must be given their broadest reasonable interpretation in light of the patent
`
`specification.
`
`30.
`
`I have been asked to assume that April 21, 2011, is the date of the
`
`purported invention, as that is the earliest date to which the ’251 Patent claims
`
`priority. I have not conducted a separate analysis to determine whether the
`
`challenged claims are entitled to this priority date, as I understand that the prior art
`
`references discussed below qualify as prior art even if all claims are deemed to be
`
`entitled to the April 21, 2011, date.
`
`V. THE ’251 PATENT
`31. The ’251 Patent purports to allow a user to use a personal computing
`
`device (e.g., smartphone) to control the presentation of content from the Internet on
`
`a display device. ’251 Patent, Abstract, 1:22-39, 3:56-62. Because “different media
`
`players are required to present different content” from the Internet, the ’251 Patent
`
`recognizes it is necessary for the computing device’s control of the content at the
`
`display device to be mediated by a server system. Id., 2:16-34.
`
`32. The computing device, display device, and server system are illustrated
`
`in Figure 1 below.
`
`
`
`14
`
`Page 19 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`
`Id., FIG. 1 (annotated).
`
`33. The computing device’s server-system-mediated control of content
`
`presented at the display device may be understood with reference to Figure 2. The
`
`computing device displays content from various Internet sources (Step 106), and a
`
`user selects content to be presented on the display device. Id., 3:7-15, 3:64-67, 4:23-
`
`27. As shown, the computing device then sends signals through the server system to
`
`cause presentation of the content at the display device (Steps 110, 122, 124, 126,
`
`
`
`15
`
`Page 20 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`128, 130). Id., 3:10-18. The signals sent by the computing device specify, in addition
`
`to the content itself, a particular media player for playing the content. Id., 4:29-35.
`
`In this manner, the computing device can cause different content to be played on the
`
`display device, even when different media players are required for the different
`
`content. Id., 3:33-36.
`
`
`
`Id., FIG. 2 (annotated).
`
`34. The computing device can also be used to “control how the content is
`
`displayed (e.g., play, pause, stop, rewind, fast forward, etc.)” at the display device.
`
`Id., 3:36-39. To this end, signals from the computing device are passed through the
`
`
`
`16
`
`Page 21 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`server system to the display device. Id., 3:39-41. This is because “[v]arious types of
`
`[media] players may use different … commands to control their respective
`
`playback.” Id., 5:57-58. Accordingly, the computing device sends to the server
`
`system a universal playback control command that is not specific to any media
`
`player. Id., 5:58-62. The server system translates the universal playback control
`
`command into the appropriate playback control command for the media player
`
`playing the content (Step 120) and provides this media-player-specific playback
`
`control command to the display device. Id., 5:62-6:3. The media-player-specific
`
`playback control command is programming code, such as JavaScript code, that the
`
`media player will recognize. Id. For example, one media-player-specific playback
`
`control command may be used to control a media player playing content from
`
`YouTube®, while another may be used to control a media player playing content
`
`from Ted.com. Id., 6:9-17.
`
`35. Claims 2 and 5-9 of the ’251 Patent add only incremental limitations
`
`such as (i) loading resources associated with the media player (id., Claim 2, 11:61-
`
`12:3) and (ii) specifics of the universal playback control command (id., Claim 5,
`
`12:11-14), video content (id., Claims 6-7, 12:15-18), and synchronization code (id.,
`
`Claims 8-9, 12:19-25).
`
`
`
`17
`
`Page 22 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`VI. OVERVIEW OF PRIOR ART
`A. Muthukumarasamy
`36. Muthukumarasamy was filed on March 22, 2010, and published on
`
`September 23, 2010. I understand it is prior art.
`
`37. Like the ’251 Patent, Muthukumarasamy aims to allow a user to use an
`
`Internet-enabled device (“IED”), such as a smartphone, to control the presentation
`
`of content,
`
`including content from
`
`the Internet, on a display device.
`
`Muthukumarasamy, Abstract, [0005], [0026], [0031], [0041], [0047]. Just as the
`
`’251 Patent allows the computing device to control, through the server system, the
`
`display device’s presentation of content from different Internet sources,
`
`Muthukumarasamy provides a “device-agnostic and source-agnostic entertainment
`
`experience” in which the IED controls, through a Device-Based Control System
`
`(“DBCS”), the display device’s presentation of content from different Internet
`
`sources. ’251 Patent, 5:57-58; Muthukumarasamy, [0005], [0027], [0047], [0134].
`
`The IED, display device, and DBCS are illustrated in Figure 1 below.
`
`
`
`18
`
`Page 23 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`
`Muthukumarasamy, FIG 1 (annotated).
`
`38. The IED’s DBCS-mediated control of content presented at the display
`
`device may be understood with reference again to Figure 1. The IED displays
`
`content from various Internet sources (see, e.g., FIGs. 2 and 3), and a user selects
`
`content to be presented on the display device. Id., [0048]. As in the ’251 Patent,
`
`signals to control presentation of the content are sent from the IED to the display
`
`device through the DBCS, so that the DBCS translates universal control commands
`
`into those specific to the current content presentation. For example, in some
`
`embodiments, the IED’s signal is sent to the display device through a server system
`
`
`
`19
`
`Page 24 of 350
`
`

`

`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`of the DBCS, as shown in Figure 1. Id., [0057]-[0058]. As Muthukumarasamy
`
`explains, the IED sends to the zNode a command having a “universal control format”
`
`(also called “UES Format”), and the zNode “resolves the command into a specific
`
`instruction directed at a specific zNode” associated with the display device. Id.,
`
`[0057], [0066]-[0067]. The zNode then “translate[s] the command[]” into a “media
`
`device control protocol” interpretable by the display device. Id., [0058].
`
`39. Muthukumarasamy also describes DBCS-mediated control of Internet
`
`content presented at a display device. For example, as shown in Figure 18,
`
`Muthukumarasamy explains that the DBCS includes remote-controlled Internet
`
`browser software (“RCIBS”) that is “optimized for internet-media consumption.”
`
`Id., [0083]. Examples of Internet-media platforms in Muthukumarasamy are Netflix
`
`and Amazon Video On Demand. Id., [0134].
`
`
`
`20
`
`Page 25 of 350
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00795 Petition
`U.S. Patent 8,356,251
`
`
`
`
`Muthukumarasamy, FIG. 18 (annotated).
`
`40. Muthukumarasamy recognizes that different media players are needed
`
`to present different Internet content. For example, as shown in Figure 19 below,
`
`Muthukumarasamy envisions using the RCIBS to “[p]lay [a] video at [a] given
`
`URL,” and recognizes that, to play a video “using 3rd party APIs [application
`
`programming interfaces] and Video Players,” the RCIBS mus