`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________________
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`Google LLC,
`
`Petitioner,
`
`v.
`
`Touchstream Technologies, Inc.,
`
`Patent Owner.
`
`U.S. Patent No. 8,904,289
`Filing Date: September 26, 2011
`Issue Date: December 2, 2014
`
`____________________
`
`Case No. IPR2022-00794
`
`
`
`DECLARATION OF BENJAMIN B. BEDERSON, PH.D.
`
`
`
`
`
`
`Page 1 of 346
`
`GOOGLE EXHIBIT 1005
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`TABLE OF CONTENTS
`
`
`INTRODUCTION ........................................................................................... 1 
`I. 
`QUALIFICATIONS ........................................................................................ 2 
`II. 
`III.  MATERIALS CONSIDERED ...................................................................... 11 
`IV.  LEGAL PRINCIPLES ................................................................................... 12 
`V. 
`THE ’289 PATENT ....................................................................................... 14 
`VI.  OVERVIEW OF PRIOR ART ...................................................................... 17 
`A.  Muthukumarasamy .............................................................................. 17 
`B. 
`Hayward .............................................................................................. 23 
`VII.  LEVEL OF ORDINARY SKILL IN THE ART ........................................... 25 
`VIII.  CLAIM CONSTRUCTION .......................................................................... 27 
`IX.  SUMMARY OF OPINIONS ......................................................................... 27 
`X.  GROUNDS I AND II: MUTHUKUMARASAMY ALONE OR IN
`VIEW OF HAYWARD RENDERS OBVIOUS THE CHALLENGED
`CLAIMS ........................................................................................................ 29 
`A. 
`Summary of Grounds I and II ............................................................. 30 
`1. 
`Ground I: Muthukumarasamy Would Have Rendered
`Obvious Claims 1-2 and 6-8 ..................................................... 30 
`Ground II: Muthukumarasamy and Hayward Would
`Have Rendered Obvious Claims 1-2 and 6-8 ........................... 31 
`Independent Claim 1 ........................................................................... 34 
`1. 
`Element 1(pre) - A method of controlling presentation of
`content on a content presentation device that loads any
`one of a plurality of different media players, the method
`comprising: ................................................................................ 34 
`
`2. 
`
`B. 
`
`
`
`i
`
`Page 2 of 346
`
`

`

`2. 
`
`3. 
`
`4. 
`
`5. 
`
`6. 
`
`7. 
`
`8. 
`
`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`Element 1(a)(i) - receiving, in a server system, one or
`more messages from a personal computing device that is
`separate from the server system and separate from the
`content presentation device, ...................................................... 42 
`Element 1(a)(ii) - wherein the one or more messages,
`taken together, include information associated with a
`unique identification code assigned to the content
`presentation device, ................................................................... 45 
`Element 1(a)(iii) - [wherein the one or more messages,
`taken together,] specify a file to be acted upon, identify a
`particular media player for playing content from the
`specified file wherein the media player is a computer
`application operable to present content and control
`presentation of the content, identify a location of the
`particular media player, ............................................................ 50 
`Element 1(a)(iv) - [wherein the one or more messages,
`taken together, …] include an action control command
`for presentation of the content on the content presentation
`device by the particular media player, the action control
`command being independent of the particular media
`player; ........................................................................................ 60 
`Element 1(b) - using the information associated with the
`unique identification code to store a record establishing
`an association between the personal computing device
`and the content presentation device; and .................................. 68 
`Element 1(c)(i) - identifying, by the server system,
`programming code corresponding to the action control
`command, wherein the programming code is for
`controlling presentation of the content by the content
`presentation device using the particular media player, ............. 69 
`Element 1(c)(ii) - wherein, based on information received
`or retrieved from the server system, the content
`presentation device uses the particular media player to
`execute the programming code with respect to the file. ........... 74 
`
`ii
`
`Page 3 of 346
`
`

`

`
`
`
`
`
`
`C. 
`
`D. 
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`Dependent Claim 2 - The method of claim 1 further including
`loading the particular media player in the content presentation
`device prior to executing the programming code with respect to
`the file. ................................................................................................. 77 
`Independent Claim 6 ........................................................................... 78 
`1. 
`Elements 6(pre) through 6(b) and Element 6(d)(ii) .................. 78 
`2. 
`Element 6(c) - loading, by the server system, a set of
`protocols or application programming interfaces from a
`library based on the identity of the particular media
`player; and ................................................................................. 80 
`Element 6(d)(i) - identifying, based on the set of
`protocols or application programming interfaces,
`programming code corresponding to the action control
`command, wherein the programming code is for
`controlling presentation of the content by the content
`presentation device using the particular media player; ............. 83 
`Dependent Claims 7 and 8 .................................................................. 88 
`E. 
`XI.  CONCLUSION .............................................................................................. 89 
`
`
`
`3. 
`
`
`
`iii
`
`Page 4 of 346
`
`

`

`
`
`
`
`I.
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`INTRODUCTION
`1. My name is Benjamin B. Bederson, Ph.D. I am currently Professor
`
`Emeritus of Computer Science at the University of Maryland. I have been retained
`
`as an independent expert by Google LLC (“Google”) in this proceeding.
`
`2.
`
`I have been asked to provide my opinion as to whether claims 1, 2, 6,
`
`7, and 8 of U.S. Patent No. 8,904,289 (“the ’289 Patent”) are patentable. This report
`
`sets forth my opinions and the bases of my opinions.
`
`3.
`
`All of the opinions stated herein are based on my own personal
`
`knowledge and professional judgment and my analysis of the materials and
`
`information I considered in preparing this report. The materials I reviewed in
`
`connection with preparing this report are listed below. I make the following
`
`statements based on my own personal knowledge and, if called as a witness, I could
`
`and would testify to the following. I am being paid for my work in this litigation at
`
`the rate of $600.00 per hour, plus reimbursement of reasonable expenses. My
`
`compensation does not depend on the outcome of this proceeding, and I have no
`
`financial interest in that outcome.
`
`
`
`1
`
`Page 5 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`II. QUALIFICATIONS
`4. My qualifications for forming the opinions set forth in this report are
`
`summarized here and are addressed more fully in my résumé, which is attached
`
`hereto as Exhibit 1006.
`
`5.
`
`Based on my education, research, and work experience, I am
`
`knowledgeable about the subject matter of the ’289 patent and the related prior art.
`
`I am currently Professor Emeritus of Computer Science at the University of
`
`Maryland ("UMD"). From 2014 to 2018, I was the Associate Provost of Learning
`
`Initiatives and Executive Director of the Teaching and Learning Transformation
`
`Center at the UMD. I am a member and previous director of the Human-Computer
`
`Interaction Lab ("HCIL"), the oldest and one of the best known Human-Computer
`
`Interaction ("HCI") research groups in the country. I was also co-founder and Chief
`
`Scientist of Zumobi, Inc. from 2006 to 2014, a Seattle-based startup that is a
`
`publisher of content applications and advertising platforms for smartphones. I am
`
`also co-founder and co-director of the International Children's Digital Library
`
`("ICDL"), a web site launched in 2002 that provides the world's largest collection of
`
`freely available online children's books from around the world with an interface
`
`aimed to make it easy for children and adults to search and read children's books
`
`online. I am also co-founder and prior Chief Technology Officer of Hazel Analytics,
`
`
`
`2
`
`Page 6 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`a data analytics company to improve food safety and better public health whose
`
`product sends alerts in warranted circumstances. In addition, I have for more than
`
`25 years consulted for numerous companies in the area of user interfaces, including
`
`Hillcrest Labs, Lockheed Martin, Logitech, Microsoft, NASA Goddard Space Flight
`
`Center, the Palo Alto Research Center, and Sony.
`
`6.
`
`For more than 30 years, I have studied, designed, and worked in the
`
`field of computer science and HCI. That
`
`includes
`
`teaching, research,
`
`entrepreneurship, and administration in HCI and the software and technology
`
`underlying interactive computing systems. I have worked on the design and
`
`implementation of hardware and software systems including the use of capacitive,
`
`resistive, optical and IR among other sensors, and interactive applications on a range
`
`of devices, including embedded systems, controllers, smart phones, and PDAs for
`
`2D and 3D interfaces.
`
`7.
`
`In 1992, I completed my Ph.D. dissertation at New York University
`
`titled "A Miniature Space-Variant Active Vision System: Cortex-I" in which I built
`
`a custom small motor that aimed a custom camera along with a custom DSP-based
`
`computing system. Collectively, these worked together to create a miniature
`
`computer vision system that one could carry in one hand - dramatically smaller than
`
`
`
`3
`
`Page 7 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`other computer vision systems of the day. This system included the ability to
`
`generate a real-time video signal with the result of the image processing. Appx. 1.
`
`8.
`
`Starting in the mid-1990s and going through the 2000s, I worked on a
`
`range of "zoomable user interfaces" (ZUIs) that supported users in navigating large
`
`information spaces by zooming in and out of the information on the screen. I built
`
`several different ZUI systems over the years, including Pad++, Jazz, and Piccolo.
`
`Starting in the early 2000s, I adapted that work to work on mobile devices including
`
`PDAs, tablets, and mobile phones. Appx. 2.
`
`9.
`
`Starting in 2001 and continuing for several years, I worked on photo
`
`management systems. I wrote a paper in 2001 describing PhotoMesa, a zoomable
`
`image browser using a novel 2D layout algorithm I called "quantum treemaps" to
`
`position the images on the screen in a way that attempted to fill the screen while
`
`keeping groups of related images together. Appx. 3. As indicated in the figure below
`
`from that paper, PhotoMesa could display many hundreds of images at a time by
`
`showing small thumbnails, by showing larger thumbnails when the mouse hovered
`
`over a small thumbnail, and by allowing a user to zoom in to see a high resolution
`
`version of the images.
`
`
`
`4
`
`Page 8 of 346
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
` 
`In 2003, I wrote a paper that performed image processing to identify
`
`10.
`
`salient portions of images and then automatically crop those images to show those
`
`portions. Appx. 4. In this way, image thumbnails could focus attention on faces or
`
`other objects of a photo without showing all of the background. In 2006, I wrote a
`
`paper that explained some of the trends in photo management software including
`
`rich support for annotation, browsing and sharing of photos. Appx. 7. In 2007, I
`
`published another paper that provided a review of innovative personal photo
`
`management work going on at that time. Appx. 8. That paper describes a version of
`
`my PhotoMesa software that by 2007 as shown in the following figure, included the
`
`ability to annotate photos with the location and name of people in photos.
`
`
`
`5
`
`Page 9 of 346
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
` 
`I was heavily involved in the research and development of mobile
`
`11.
`
`device applications during the early 2000's. For example, I envisioned, designed and
`
`built DateLens in 2002-2004 to create a richer calendar for the Microsoft PocketPC
`
`platform as well as desktops as described in a 2004 paper. Appx. 5. One of its
`
`innovations was its design that enabled it to scale between small (mobile) and large
`
`(desktop) computers. It also had a special double-headed scroll bar that allowed a
`
`
`
`6
`
`Page 10 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`user to touch scroll and also control how much information was presented on the
`
`screen at a time. I started DateLens as a research project and eventually sold it
`
`commercially. DateLens displayed calendar data that came from the Microsoft
`
`PocketOutlook database and allowed modification and viewing of appointments.
`
`The research website from that time period that describes DateLens is still available
`
`at http://www.cs.umd.edu/hcil/datelens/ which shows screenshots and a picture of
`
`me demonstrating DateLens to Bill Gates. The commercial website from that time
`
`period is also available at http://www.windsorinterfaces.com/datelens.shtml, which
`
`shows a number of news articles reviewing DateLens at that time.
`
`12.
`
`In 2004-2005, with graduate student Amy Karlson and John
`
`SanGiovanni of Microsoft, I worked on a prototype software called LaunchTile.
`
`LaunchTile allowed zooming and scrolling of multiple applications on a handheld
`
`touchscreen PDA. The interface offered a grid of applications that could be zoomed
`
`and scrolled. It included several different methods of scrolling, including gestures
`
`where the scrolling continued after a finger was lifted from the touch screen, and
`
`then slowed to a stop. Appx. 6.
`
`13. That work led to my co-founding of Zumobi, where I was responsible
`
`for investigating new software platforms and developing new user interface designs
`
`that provide efficient and engaging interfaces for mobile touchscreen platforms
`
`
`
`7
`
`Page 11 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`including the iPhone and Android-based devices. For example, I designed and
`
`implemented software called "Ziibii," a "river" of news for iPhone, software called
`
`"ZoomCanvas," a zoomable user interface for several iPhone apps, and iPhone apps
`
`including "Inside Xbox" for Microsoft and Snow Report for REI. All of these
`
`applications included touch based zooming and scrolling.
`
`14. At UMD, I am focused primarily on the area of HCI, a field that relates
`
`to the development and understanding of computing systems to serve users' needs.
`
`Researchers and practitioners in this field are focused on making universally usable,
`
`useful, efficient, and appealing systems to support people in their wide range of
`
`activities. My approach is to balance the development of innovative technology that
`
`serves people's practical needs. Example systems following this approach that I have
`
`built include Cortex-I (1992 embedded computer vision system that sensed licensed
`
`plates with custom motor, camera, and controller), Audio Augmented Reality (1995
`
`embedded system for sensing a user's location and playing audio suited to that
`
`location), Fisheye Menus (2000 software for sensing movement within and selection
`
`of linear list of items in a menu), PhotoMesa (2001 software for end users to browse
`
`personal photos), DateLens (2002 software for end users to use their mobile devices
`
`to efficiently access their calendar information), SlideBar (2005 linear sensor to
`
`control scrolling), LaunchTile (2005 "home screen" software for mobile devices to
`
`
`
`8
`
`Page 12 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`allow users to navigate apps in a zoomable environment), SpaceTree (2001 software
`
`for end users to efficiently browse very large hierarchies), ICDL (as described
`
`above), and StoryKit (a 2009 iPhone app for children to create stories).
`
`15. My work has been published extensively in more than 160 technical
`
`publications, and I have given about 100 invited talks, including 9 keynote lectures.
`
`I have won a number of awards including the Brian Shackel Award for "outstanding
`
`contribution with international impact in the field of HCI" in 2007, and the Social
`
`Impact Award in 2010 from the Association for Computing Machinery's ("ACM")
`
`Special Interest Group on Computer Human Interaction ("SIGCHI"). ACM is the
`
`primary international professional community of computer scientists, and SIGCHI
`
`is the primary international professional HCI community. I have been honored by
`
`both professional organizations. I am an "ACM Distinguished Scientist," which
`
`"recognizes those ACM members with at least 15 years of professional experience
`
`and 5 years of continuous Professional Membership who have achieved significant
`
`accomplishments or have made a significant impact on the computing field." I am a
`
`member of the "CHI Academy," which is described as follows: "The CHI Academy
`
`is an honorary group of individuals who have made substantial contributions to the
`
`field of HCI. These are the principal leaders of the field, whose efforts have shaped
`
`the disciplines and/or industry, and led the research and/or innovation in human-
`
`
`
`9
`
`Page 13 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`computer interaction." The criteria for election to the CHI Academy are: (1)
`
`cumulative contributions to the field; (2) impact on the field through development
`
`of new research directions and/or innovations; and (3) influence on the work of
`
`others.
`
`16.
`
`I have appeared on radio shows numerous times to discuss issues
`
`relating to user interface design and people's use and frustration with common
`
`technologies, web sites, and mobile devices. My work has been discussed and I have
`
`been quoted by mainstream media around the world over 120 times, including by
`
`the New York Times, the Wall Street Journal, the Washington Post, Newsweek, the
`
`Seattle Post Intelligencer, the Independent, Le Monde, NPR's All Things
`
`Considered, New Scientist Magazine, and MIT's Technology Review.
`
`17.
`
`I have designed, programmed, and publicly deployed dozens of user-
`
`facing software products that have cumulatively had millions of users. My work is
`
`cited by several major companies, including Amazon, Apple, Facebook, Google, and
`
`Microsoft. I am a named inventor on 14 U.S. patents and 19 U.S. patent applications.
`
`The patents are generally directed to user interfaces and experience.
`
`18.
`
`I received a B.S. degree in Computer Science with a minor in Electrical
`
`Engineering in 1986 from the Rensselaer Polytechnic Institute. I received M.S. and
`
`
`
`10
`
`Page 14 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`Ph.D. degrees in Computer Science in 1989 and 1992, both from New York
`
`University.
`
`III. MATERIALS CONSIDERED
`19.
`In forming my opinions, I have reviewed the following documents, and
`
`any other document cited in this declaration:
`
`Exhibit
`Ex-1001
`Ex-1002
`Ex-1004
`
`Ex-1008
`Ex-1009
`Ex-1010
`
`Ex-1011
`Ex-1012
`Ex-1014
`
`Description
`U.S. Patent No. 8,904,289 to Strober (“the ’289 Patent”)
`The ’289 Patent Prosecution History (U.S. App. No. 13/157,821)
`U.S. Patent No. 8,782,528 to Strober (“the ’528 Patent”) Prosecution
`History (U.S. App. No. 13/736,590)
`U.S. Patent Pub. No. 2010/0241699 (“Muthukumarasamy”)
`U.S. Patent No. 8,918,812 (“Hayward”)
`Touchstream Technologies, Inc.’s Opening Claim Construction
`Brief, Touchstream Techs., Inc. v. Vizbee, Inc., No. 1:17-cv-6247-
`PGG-KNF (S.D.N.Y. Sept. 4, 2018)
`U.S. Patent App. Pub. No. 2008/0301737 (“Almas”)
`U.S. Patent App. Pub. No. 2012/0110074 (“Getchius”)
`Touchstream Techs., Inc. v. Google LLC, No. 6:21-cv-00569-ADA,
`Joint Claim Construction Statement (W.D. Tex. Feb. 8, 2022)
`
`20.
`
`I have also relied on my education, experience, research, training, and
`
`knowledge in the relevant art, and my understanding of any applicable legal
`
`principles described in this declaration.
`
`21. All of the opinions contained in this declaration are based on the
`
`documents I reviewed and my knowledge and professional judgment. My opinions
`
`have also been guided by my understanding of how a person of ordinary skill in the
`
`art (“POSA”) would have understood the claims of the ’289 Patent at the time of the
`
`
`
`11
`
`Page 15 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`alleged invention. For the purposes of this declaration, I have been asked to assume
`
`that the date of the alleged invention is the earliest claimed priority date: April 21,
`
`2011.
`
`22.
`
`I reserve the right to supplement and amend any of my opinions in this
`
`declaration based on documents, testimony, and other information that becomes
`
`available to me after the date of this declaration.
`
`IV. LEGAL PRINCIPLES
`23.
`It is my understanding that there are two ways that prior art references
`
`can render a patent claim unpatentable: anticipation and obviousness. Counsel has
`
`informed me that the petitioner has the burden in an inter partes review to show
`
`unpatentability by a preponderance of the evidence.
`
`24.
`
`I also understand that there is a set process as follows: (a) the claims of
`
`a patent are properly construed, (b) then, you must compare the claim language to
`
`the prior art on a limitation-by-limitation basis. If the prior art reference contains all
`
`the elements of the claim language (explicitly or inherently), arranged as in the
`
`claims, then that is considered anticipation.
`
`25.
`
`I understand that an invention is obvious when the differences between
`
`the subject matter sought to be patented and the prior art are such that the subject
`
`matter as a whole would have been obvious at the time of the invention to a person
`
`
`
`12
`
`Page 16 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`having ordinary skill in the art. For this reason, I have been asked to consider the
`
`level of ordinary skill in the field that someone would have had at the time of the
`
`claimed invention.
`
`26. Counsel has also instructed me that in an obviousness determination,
`
`the factors to consider are (1) the scope and content of the prior art, (2) the
`
`differences between the prior art and the asserted claims, (3) the level of ordinary
`
`skill in the pertinent art, and (4) the existence of secondary considerations of
`
`nonobviousness. Secondary considerations include a long-felt need, commercial
`
`success, unexpected results, praise of the invention, licensing, copying, failure of
`
`others, and skepticism by experts.
`
`27. Counsel has also instructed me that an obviousness inquiry may involve
`
`assessing the motivation of a POSA to combine references. The prior art references
`
`themselves may provide a suggestion, motivation, or reason to combine, but other
`
`sources may also support a rationale for combining two or more prior art references.
`
`28.
`
`It is also my understanding through counsel that the combination of
`
`familiar elements according to known methods is likely to be obvious when it does
`
`no more than yield predictable results. It is further my understanding that a proper
`
`obviousness analysis focuses on what was known or obvious to a POSA, not just the
`
`patentee.
`
`
`
`13
`
`Page 17 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`V. THE ’289 PATENT
`29. The ’289 Patent purports to allow a user to use a personal computing
`
`device (e.g., smartphone) to control the presentation of content from the Internet on
`
`a display device. ’289 Patent, Abstract, 1:21-38, 3:23-41. Because “different media
`
`players are required to present different content” from the Internet, the computing
`
`device’s control of content at the display device needs to be mediated by a server
`
`system. Id., 2:26-34; 3:56-62.
`
`30. The computing device, content presentation device, and server system
`
`are illustrated in Figure 1 below.
`
`
`
`14
`
`Page 18 of 346
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`
`Id., FIG. 1 (annotated).
`
`31. The computing device’s server-system-mediated control of content
`
`presented at the content presentation device may be understood with reference to
`
`FIG. 2. The computing device displays content from various Internet sources
`
`(Step 106), and a user selects content to be presented on the content presentation
`
`device. Id., 3:7-15, 3:64-67, 4:23-27. As shown, the computing device then sends
`
`signals through the server system to cause presentation of the content at the content
`
`
`
`15
`
`Page 19 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`presentation device (Steps 110, 122, 124, 126, 128, 130). Id., 3:10-18. The signals
`
`sent by the computing device specify, in addition to the content itself, a particular
`
`media player for playing the content. Id., 4:29-35. In this manner, the computing
`
`device can cause different content to be played on the content presentation device,
`
`even when different media players are required for the different content. Id., 3:33-
`
`36.
`
`Id., FIG. 2 (annotated).
`
`32. The computing device can also be used to “control how the content is
`
`displayed (e.g., play, pause, stop, rewind, fast forward, etc.)” at the display device.
`
`
`
`
`
`16
`
`Page 20 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`Id., 3:36-39. To this end, signals from the computing device are passed through the
`
`server system to the display device. Id., 3:39-41. This is because “[v]arious types of
`
`[media] players may use different … commands to control their respective
`
`playback.” Id., 5:57-58. Accordingly, the computing device sends to the server
`
`system a universal playback control command that is not specific to any media
`
`player. Id., 5:58-62. The server system translates the universal playback control
`
`command into the appropriate playback control command for the media player
`
`playing the content (Step 120) and provides this media-player-specific playback
`
`control command to the content presentation device. Id., 5:62-6:3. The media-
`
`player-specific playback control command is programming code, such as JavaScript
`
`code, that the media player will recognize. Id. For example, one media-player-
`
`specific playback control command may be used to control a media player playing
`
`content from YouTube®, while another may be used to control a media player
`
`playing content from Ted.com. Id., 6:9-17.
`
`VI. OVERVIEW OF PRIOR ART
`A. Muthukumarasamy
`33. Muthukumarasamy was filed on March 22, 2010, and published on
`
`September 23, 2010. I understand it is prior art.
`
`
`
`17
`
`Page 21 of 346
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`34. Like the ’289 Patent, Muthukumarasamy allows a user to use an
`
`Internet-enabled device (“IED”), such as a smartphone, to control the presentation
`
`of content,
`
`including content from
`
`the Internet, on a display device.
`
`Muthukumarasamy, Abstract, [0005], [0026], [0031], [0041], [0047]. Just as the
`
`’289 Patent allows the computing device to control, through the server system, the
`
`content presentation device’s presentation of content from different Internet sources
`
`(’289 Patent, 5:57-58), Muthukumarasamy provides a “device-agnostic and source-
`
`agnostic entertainment experience” in which the IED controls, through a Device-
`
`Based Control System (“DBCS”), the display device’s presentation of content from
`
`different Internet sources (Muthukumarasamy, Abstract, [0005], [0027], [0047],
`
`[0134]). The IED, content presentation device, and DBCS are illustrated in Figure 1
`
`below.
`
`
`
`18
`
`Page 22 of 346
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`
`Id., FIG. 1 (annotated).
`
`35. The IED’s DBCS-mediated control of content presented at the content
`
`presentation device may be understood with reference to Figure 1. The IED displays
`
`content from various Internet sources (see, e.g., id., FIGs. 2 and 3), and a user selects
`
`content to be presented on the display device. Id., [0048]. As in the ’289 Patent,
`
`signals to control presentation of the content are sent from the IED to the display
`
`device through the DBCS, so that the DBCS translates standard-format control
`
`commands into those specific to the current content presentation. For example, in
`
`some embodiments, the IED’s signal is sent to the content presentation device
`
`
`
`19
`
`Page 23 of 346
`
`

`

`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`through a zNode and zHub of the DBCS, as shown in Figure 1. Id., [0057]-[0058].
`
`As Muthukumarasamy explains, the IED sends to the zNode a command having a
`
`“universal control format” (also called “UES Format”), and the zNode “resolves the
`
`command into a specific instruction directed at a specific zNode” associated with
`
`the display device. Id., [0057], [0066]-[0067]. The zNode then “translate[s] the
`
`command[]” into a “media device control protocol” interpretable by the display
`
`device. Id., [0058].
`
`36. Muthukumarasamy also specifically describes DBCS-mediated control
`
`of Internet content presented at a content presentation device. For example, as shown
`
`in Figure 18, Muthukumarasamy explains that the DBCS includes remote-controlled
`
`Internet browser software (“RCIBS”) that is “optimized for internet-media
`
`consumption.”
`
`Id.,
`
`[0083]. Examples of
`
`Internet-media platforms
`
`in
`
`Muthukumarasamy are Netflix and Amazon Video On Demand. Id., [0134].
`
`
`
`20
`
`Page 24 of 346
`
`

`

`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`
`
`Id., FIG. 18 (annotated).
`
`37. Muthukumarasamy recognizes that different media players are needed
`
`to present different Internet content. For example, as shown in Figure 19 below,
`
`Muthukumarasamy envisions using the RCIBS to “[p]lay [a] video at [a] given
`
`URL,” and recognizes that, to play a video “using 3rd party APIs [application
`
`programming interfaces] and Video Players,” the RCIBS must “load[] the 3rd Party
`
`Video Player specified” for the video. Id., FIG. 19.
`
`
`
`21
`
`Page 25 of 346
`
`

`

`
`
`
`
`
`
`
`
`
`
`IPR2022-00794 Petition
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`USS. Patent 8,904,289
`
`
`Go to webpage with given URL
`
`ZRCIBSloads and displays the webpage that is
`present at the given intermet URL
`
`While on webpage, Navigate: Up/down/right/left|ZRCIBSjumps hyperiink to next hyperiink.
`
`(This is done via button presses or
`gestures on the Zelfy application)
`
`Play video at given URL
`
`ZRCIBSplays the video present at the given URL
`
`party using 3 party APIs and audio Players|and plays the audio with given audioid)
`
`Execute Transport Command (Play, Pausa|ZRCIBS executes the transport command
`Fastforward, rewind) on currently playing video|n the current playing video.
`
`Play audio at given URL
`
`ZRCIBSplays the audio present at the given URL
`
`Execute Transport Command (Play, Pause|ZRCIBS executes the transport command
`Fastforward, rewind) on currently playing audio|on the current playing audio.
`
`Play flash swf game or other game at given URL|ZRCIBS loads and presents the game (adobe flash
`wi of other) present at the given URL
`
`While on game: Pass game command.
`
`ZIMMS executes the command in the game.
`
`(This is done via button presses or
`gestures on the Zelfy application)
`
`party using 3° party APIsand Video Players|and plays the video with given videoid)
`
`esteel
`
`Play audio with given audio-id (as specified by 3%|ZRCIBS loadsthe 3° PartyAudio Player specified
`
`FIG.19
`22
`
`
`Page 26 of 346
`
`Page 26 of 346
`
`

`

`
`
`
`
`Id.
`
`
`
`
`IPR2022-00794 Petition
`U.S. Patent 8,904,289
`
`
`B. Hayward
`38. Hayward on October 24, 2001, published on June 20, 2002 and claims
`
`priority to a provisional applicatio