IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`
`BOSHERNITZAN et al.
`In re Patent of:
`Attorney Docket No. 50095-0190IP1
`9,996,738
`
`
`U.S. Patent No.:
`
`June 12, 2018
`Issue Date:
`
`Appl. Serial No.: 15/043,283
`
`Filing Date:
`February 12, 2016
`Title:
`SYSTEM AND METHOD FOR CONTROLLING A TERMINAL
`DEVICE
`
`
`
`
`DECLARATION OF DR. BENJAMIN B. BEDERSON
`
`I declare that all statements made of my own knowledge are true and that all
`
`statements made on information and belief are believed to be true. I further declare
`
`that these statements were made with the knowledge that willful false statements
`
`and the like so made are punishable by fine or imprisonment, or both, under
`
`Section 1001 of the Title 18 of the United States Code and that such willful false
`
`statements may jeopardize the validity of the application or any patents issued
`
`thereon.
`
`
`
`Dated: September 29, 2024
`
`
`
`By:
`Dr. Benjamin B. Bederson
`
`
`
`
`
`
`i
`
`
`
`1
`
`APPLE 1003
`
`

`

`Table of Contents
`
`I.
`
`II.
`
`INTRODUCTION AND SCOPE OF WORK ................................................ 1
`
`QUALIFICATIONS ........................................................................................ 2
`
`III. MATERIALS CONSIDERED ........................................................................ 9
`
`IV.
`
`V.
`
`VI.
`
`SUMMARY OF CONCLUSIONS ...............................................................19
`
`LEGAL PRINCIPLES ...................................................................................20
`A. Person of Ordinary Skill in the Art .....................................................20
`B. Obviousness .........................................................................................20
`C. Claim Construction Standard ..............................................................22
`
`PERSON OF ORDINARY SKILL IN THE ART ........................................23
`A. A POSITA’s Ordinary Level of Skill ..................................................23
`B. A POSITA’s Backgrounds Knowledge...............................................24
`Background of the Technology .....................................................24
`1.
`Devices for Performing Actions Based on Sensed Data ...............26
`2.
`Accelerometer Basics ....................................................................29
`3.
`
`VII. OVERVIEW OF THE ’738 PATENT ..........................................................31
`A. Overview of the ’738 Patent ................................................................31
`B. Summary of Prosecution History of the ’738 Patent ..........................36
`
`VIII. INTERPRETATIONS OF THE ‘738 PATENT CLAIMS AT ISSUE.........37
`A. “engagement means…” .......................................................................43
`B. “said contact interaction being comprised of an impact on said
`mounting surface” ...............................................................................48
`C. “routing module … said routing module receiving said data signals
`from said sensor” .................................................................................51
`D. “processing module … said processing module determining a data
`pattern corresponding to said data signals of said contact interaction
`and matching said data pattern with a gesture profile” .......................53
`E. “output module … said output module transmitting said command to
`said receiving module” ........................................................................56
`F. “terminal device” .................................................................................57
`G. “receiving module” ..............................................................................60
`H. “means for initiating activity of said terminal device corresponding to
`said command” ....................................................................................61
`
`i
`
`2
`
`

`

`IX. APPLICATION OF PRIOR ART TO THE CHALLENGED CLAIMS ......63
`A. GROUND 1A: THE MURAKOSHI-STEWART COMBINATION
`RENDERS CLAIMS 1-2, 4-5, AND 8-10 OBVIOUS .......................68
`1. Overview of Murakoshi (APPLE-1004) .......................................68
`2. Overview of Stewart (APPLE-1006) ............................................71
`3.
`The Murakoshi-Stewart Combination ...........................................73
`a) The Combination ............................................................................73
`
`b) Reasons for the Combination .........................................................76
`
`Element-by-Element Analysis .......................................................84
`4.
`c) Claim 1 ...........................................................................................84
`
`[1pre] A control system comprising: ...............................................84
`
`[1a] a housing having an engagement means for a mounting surface;
` ....................................................................................................................86
`
`[1b-i] a sensor contained within said housing, ................................89
`
`[1b-ii] said sensor forming an interactive zone defined by a range of
`said sensor, .................................................................................................90
`
`[1b-iii] said sensor being comprised of an accelerometer, ..............91
`
`[1b-iv] said interactive zone being aligned with said mounting
`surface and overlaying said mounting surface outside a perimeter of said
`housing, ......................................................................................................92
`
`[1b-v] said sensor being in a fixed position relative to said
`engagement means, ....................................................................................94
`
`[1b-vi] wherein a contact interaction associated with said mounting
`surface within said interactive zone is detected by said sensor as data
`signals, ........................................................................................................95
`
`[1b-vii] said contact interaction being comprised of an impact on
`said mounting surface, ...............................................................................96
`
`[1b-viii] said data signals being comprised of vibration data of said
`contact interaction; .....................................................................................97
`
`[1c-i] a server in communication with said sensor, .........................98
`
`[1c-ii] said server being comprised of a routing module, ..............100
`
`[1c-iii] a processing module being connected to said routing
`module, and ..............................................................................................100
`
`[1c-iv] an output module connected to said processing module, ..101
`
`ii
`
`3
`
`

`

`[1c-v] said routing module receiving said data signals from said
`sensor, .......................................................................................................102
`
`[1c-vi] said processing module determining a data pattern
`corresponding to said data signals of said contact interaction and matching
`said data pattern with a gesture profile, said gesture profile being
`associated with a command; and ..............................................................103
`
`[1d-i] a terminal device ..................................................................106
`
`[1d-ii] being comprised of a receiving module .............................107
`
`[1d-iii] and means for initiating activity of said terminal device
`corresponding to said command,..............................................................108
`
`[1d-iv] said terminal device being in communication with said
`server, .......................................................................................................110
`
`[1d-v] said output module transmitting said command to said
`receiving module, .....................................................................................111
`
`[1e] wherein said engagement means of said housing comprises: an
`attachment means between said housing to said mounting surface; and a
`transmission portion connecting said sensor to said attachment means of
`said housing and being comprised of a material with flexibility different
`than said mounting surface so as to set a rigid position of said sensor
`relative to said mounting surface, said contact interaction generating said
`data signals of said sensor through said transmission portion. ................112
`
`d) Claim 2 .........................................................................................120
`
`[2] The control system, according to claim 1, wherein said
`interactive zone of said sensor aligns with said mounting surface, said
`interactive zone being coplanar with said mounting surface. ..................120
`
`e) Claim 8 .........................................................................................122
`
`[8] The control system, according to claim 1, wherein said terminal
`device is comprised of one device selected from a group consisting of: a
`television, a thermostat, a computer, a software system, a game console, a
`fan, a mattress adjustor, an alarm clock, and a lighting fixture. ..............122
`
`f) Claim 9 .........................................................................................123
`
`[9] The control system, according to claim 1, further comprising: an
`additional terminal device being comprised of an additional receiving
`module and additional means for initiating activity of said additional
`terminal device corresponding to an additional command, said additional
`terminal device being in communication with said server, said output
`
`iii
`
`4
`
`

`

`module transmitting said additional command to said additional receiving
`module. .....................................................................................................123
`
`g) Claim 10 .......................................................................................125
`
`[10pre]-[10k-i] ...............................................................................129
`
`[10k-ii] said command corresponding to activity of said terminal
`device; and ................................................................................................131
`
`[10l] performing said activity with said terminal device. ..............131
`
`B. GROUND 1B: THE MURAKOSHI-STEWART-SACHS
`COMBINATION RENDERS CLAIMS 1-2, 4-5, AND 8-11
`OBVIOUS .........................................................................................132
`1.
`Sachs ............................................................................................132
`2. Murakoshi-Stewart-Sachs Combination .....................................137
`a) The Combination ..........................................................................137
`
`b) Reasons for the Combination .......................................................139
`
`Element-by-Element Analysis .....................................................148
`3.
`a) Claim 1 .........................................................................................148
`
`[1pre]-[1c-v], [1d-i]-[1e] ................................................................148
`
`[1c-vi] said processing module determining a data pattern
`corresponding to said data signals of said contact interaction and matching
`said data pattern with a gesture profile, said gesture profile being
`associated with a command; and ..............................................................149
`
`b) Claims 2, 8-9 ................................................................................151
`
`c) Claim 4 .........................................................................................151
`
`[4a] The control system, according to claim 1, wherein said contact
`interaction is comprised of an impact on said mounting surface, ............151
`
`[4b] said data signals having a respective defined peak
`corresponding to each impact, a defined time period after a last defined
`peak, [4c] said data pattern being comprised of each defined peak and said
`defined time period after said last defined peak. .....................................153
`
`d) Claim 5 .........................................................................................155
`
`[5a] The control system, according to claim 1, wherein said contact
`interaction is comprised of a plurality of impacts on said mounting
`surface, .....................................................................................................155
`
`[5b] said data signals having a respective defined peak
`corresponding to each impact, a measured time period between each
`
`iv
`
`5
`
`

`

`defined peak, and a defined time period after a last defined peak, said data
`pattern being comprised of each defined peak, each measured time period,
`and said defined time period after said last defined peak. .......................157
`
`e) Claim 10 .......................................................................................158
`
`[10pre]-[10l] ...................................................................................158
`
`f) Claim 11 .......................................................................................159
`
`[11] The method of controlling the terminal device, according
`to claim 10, wherein the step of making a physical impact on said
`mounting surface further comprises making a plurality of physical impacts
`on said mounting surface, said contact interaction being associated with
`more than one physical impact. ................................................................159
`
`C. GROUND 1C: THE MURAKOSHI-STEWART-SACHS-ORR
`COMBINATION RENDERS CLAIMS 1-2, 4-5, AND 8-11
`OBVIOUS .........................................................................................160
`1. Orr ................................................................................................160
`2.
`The Combination of Murakoshi-Stewart-Sachs and Orr ............162
`a) The Combination ..........................................................................162
`
`b) Reasons for the Combination .......................................................165
`
`Element-by-Element Analysis .....................................................173
`3.
`a) Claim 1 .........................................................................................173
`
`[1pre] A control system comprising: .............................................173
`
`[1a] a housing having an engagement means for a mounting surface;
` ..................................................................................................................173
`
`[1b-i] a sensor contained within said housing, ..............................178
`
`[1b-ii] said sensor forming an interactive zone defined by a range of
`said sensor, ...............................................................................................179
`
`[1b-iii] said sensor being comprised of an accelerometer, ............180
`
`[1b-iv] said interactive zone being aligned with said mounting
`surface and overlaying said mounting surface outside a perimeter of said
`housing, ....................................................................................................180
`
`[1b-v] said sensor being in a fixed position relative to said
`engagement means, ..................................................................................181
`
`[1b-vi] wherein a contact interaction associated with said mounting
`surface within said interactive zone is detected by said sensor as data
`signals, ......................................................................................................182
`
`v
`
`6
`
`

`

`[1b-vii] said contact interaction being comprised of an impact on
`said mounting surface, .............................................................................183
`
`[1b-viii] said data signals being comprised of vibration data of said
`contact interaction; ...................................................................................184
`
`[1c-i] a server in communication with said sensor, .......................185
`
`[1c-ii] said server being comprised of a routing module, ..............187
`
`[1c-iii] a processing module being connected to said routing
`module, and ..............................................................................................188
`
`[1c-iv] an output module connected to said processing module, ..189
`
`[1c-v] said routing module receiving said data signals from said
`sensor, .......................................................................................................190
`
`[1c-vi] said processing module determining a data pattern
`corresponding to said data signals of said contact interaction and matching
`said data pattern with a gesture profile, said gesture profile being
`associated with a command; and ..............................................................194
`
`[1d-i] a terminal device ..................................................................198
`
`[1d-ii] being comprised of a receiving module .............................199
`
`[1d-iii] and means for initiating activity of said terminal device
`corresponding to said command,..............................................................200
`
`[1d-iv] said terminal device being in communication with said
`server, .......................................................................................................201
`
`[1d-v] said output module transmitting said command to said
`receiving module, .....................................................................................203
`
`[1e] wherein said engagement means of said housing comprises: an
`attachment means between said housing to said mounting surface; and a
`transmission portion connecting said sensor to said attachment means of
`said housing and being comprised of a material with flexibility different
`than said mounting surface so as to set a rigid position of said sensor
`relative to said mounting surface, said contact interaction generating said
`data signals of said sensor through said transmission portion. ................206
`
`b) Claim 10 .......................................................................................214
`
`[10pre]-[10c], [10e-ii]-[10g], [10k-ii]-[10l] ..................................214
`
`[10k-ii]-[10l] ..................................................................................214
`
`c) Claims 2, 8-9 ................................................................................215
`
`vi
`
`7
`
`

`

`d) Claims 4, 5, 11 .............................................................................215
`
`D. GROUND 2A: THE LI-IFIXIT COMBINATION RENDERS
`CLAIMS 1-2 AND 9-10 OBVIOUS .................................................216
`1.
`Li ..................................................................................................216
`2.
`iFixit ............................................................................................220
`a) iFixit Report is a prior art printed publication .............................220
`
`(i)
`
`The iFixit Report published September 2014......................221
`
`(ii) A POSITA would have been familiar with iFixit and the
`iFixit Report .........................................................................222
`
`(iii) ....... A POSITA would have been able to locate the iFixit Report
`because it was indexed .............................................................................224
`
`b) iFixit Teardown Report ................................................................227
`
`Li-iFixit Combination .................................................................228
`3.
`a) The Combination ..........................................................................228
`
`b) Reasons for the Combination .......................................................234
`
`Element-by-Element Analysis .....................................................241
`4.
`a) Claim 1 .........................................................................................241
`
`[1pre] A control system comprising: .............................................241
`
`[1a] a housing having an engagement means for a mounting surface;
` ..................................................................................................................242
`
`[1b-i] a sensor contained within said housing, ..............................248
`
`[1b-ii] said sensor forming an interactive zone defined by a range of
`said sensor, ...............................................................................................249
`
`[1b-iii] said sensor being comprised of an accelerometer, ............251
`
`[1b-iv] said interactive zone being aligned with said mounting
`surface and overlaying said mounting surface outside a perimeter of said
`housing, ....................................................................................................252
`
`[1b-v] said sensor being in a fixed position relative to said
`engagement means, ..................................................................................254
`
`[1b-vi] wherein a contact interaction associated with said mounting
`surface within said interactive zone is detected by said sensor as data
`signals, ......................................................................................................256
`
`[1b-vii] said contact interaction being comprised of an impact on
`said mounting surface, .............................................................................258
`
`vii
`
`8
`
`

`

`[1b-viii] said data signals being comprised of vibration data of said
`contact interaction; ...................................................................................260
`
`[1c-i] a server in communication with said sensor, .......................260
`
`[1c-ii] said server being comprised of a routing module, ..............261
`
`[1c-iii] a processing module being connected to said routing
`module, and ..............................................................................................262
`
`[1c-iv] an output module connected to said processing module, ..263
`
`[1c-v] said routing module receiving said data signals from said
`sensor, .......................................................................................................265
`
`[1c-vi] said processing module determining a data pattern
`corresponding to said data signals of said contact interaction and matching
`said data pattern with a gesture profile, said gesture profile being
`associated with a command; and ..............................................................266
`
`[1d-i] a terminal device ..................................................................267
`
`[1d-ii] being comprised of a receiving module .............................269
`
`[1d-iii] and means for initiating activity of said terminal device
`corresponding to said command,..............................................................271
`
`[1d-iv] said terminal device being in communication with said
`server, .......................................................................................................272
`
`[1d-v] said output module transmitting said command to said
`receiving module, .....................................................................................273
`
`[1e] wherein said engagement means of said housing comprises: an
`attachment means between said housing to said mounting surface; and a
`transmission portion connecting said sensor to said attachment means of
`said housing and being comprised of a material with flexibility different
`than said mounting surface so as to set a rigid position of said sensor
`relative to said mounting surface, said contact interaction generating said
`data signals of said sensor through said transmission portion. ................274
`
`b) Claim 2 .........................................................................................283
`
`[2] The control system, according to claim 1, wherein said
`interactive zone of said sensor aligns with said mounting surface, said
`interactive zone being coplanar with said mounting surface. ..................283
`
`
`
`c) Claim 8…………………………………………………………284
`
`viii
`
`9
`
`

`

` [8] The control system, according to claim 1, wherein said terminal
`device is comprised of one device selected from a group consisting of: a
`television, a thermostat, a computer, a software system, a game console, a
`fan, a mattress adjustor, an alarm clock, and a lighting fixture.
`d) Claim 9 ..........................................................................................284
`
`[9] The control system, according to claim 1, further comprising: an
`additional terminal device being comprised of an additional receiving
`module and additional means for initiating activity of said additional
`terminal device corresponding to an additional command, said additional
`terminal device being in communication with said server, said output
`module transmitting said additional command to said additional receiving
`module. .....................................................................................................286
`
`d) Claim 10 ........................................................................................288
`
`[10pre]-[10k-i] ...............................................................................288
`
`[10k-ii] said command corresponding to activity of said terminal
`device; and ................................................................................................288
`
`[10l] performing said activity with said terminal device. ..............291
`
`E. GROUND 2B: THE LI-iFIXIT-SACHS COMBINATION RENDERS
`CLAIMS 4-5 AND 11 OBVIOUS ....................................................291
`1.
`Li-iFixit-Sachs Combination .......................................................291
`a) The Combination ..........................................................................291
`
`b) Reasons for the Combination .......................................................296
`
`c) Claim 4 .........................................................................................304
`
`[4a] The control system, according to claim 1, wherein said contact
`interaction is comprised of an impact on said mounting surface, ............304
`
`[4b] said data signals having a respective defined peak
`corresponding to each impact, a defined time period after a last defined
`peak, said data pattern being comprised of each defined peak and said
`defined time period after said last defined peak. .....................................305
`
`d) Claim 5 .........................................................................................307
`
`[5a] The control system, according to claim 1, wherein said contact
`interaction is comprised of a plurality of impacts on said mounting
`surface, .....................................................................................................307
`
`[5b] said data signals having a respective defined peak
`corresponding to each impact, a measured time period between each
`defined peak, and a defined time period after a last defined peak, said data
`
`ix
`
`10
`
`

`

`pattern being comprised of each defined peak, each measured time period,
`and said defined time period after said last defined peak. .......................308
`
`e) Claim 11 .......................................................................................310
`
`[11] The method of controlling the terminal device, according
`to claim 10, wherein the step of making a physical impact on said
`mounting surface further comprises making a plurality of physical impacts
`on said mounting surface, said contact interaction being associated with
`more than one physical impact. ................................................................310
`
`
`
`x
`
`11
`
`

`

`I, Dr. Benjamin B. Bederson Ph.D., of College Park, Maryland, declare that:
`
`I.
`
`1.
`
`INTRODUCTION AND SCOPE OF WORK
`
`I have been retained by Fish & Richardson P.C., on behalf of Apple
`
`(“Petitioner”), as an independent expert consultant in this inter partes review
`
`(“IPR”) proceeding before the United States Patent and Trademark Office.
`
`2.
`
`I have been asked by Petitioner’s counsel (“Counsel”) to consider
`
`whether certain references teach or suggest the features recited in Claims 1, 2, 4, 5,
`
`and 8-11 of U.S. Patent No. 9,996,738 (“the 738 patent”) (APPLE-1001). My
`
`opinions and the bases for my opinions are set forth below. My opinions are based
`
`on my education and experience.
`
`3.
`
` In writing this Declaration, I have considered the following: the prior
`
`art publications cited below and my own knowledge and experience, including my
`
`teaching and work experience in the below-described fields, and my experience of
`
`working with others in those fields during the relevant period.
`
`4.
`
`I have no financial interest in either party or in the outcome of this
`
`proceeding. I am being compensated for my work on an hourly basis for all tasks
`
`involved and am being compensated at my standard consulting rate of $600 per
`
`hour for my services. My compensation is not dependent on the outcome of these
`
`proceedings or on any pending litigation, and my compensation is also not
`
`1
`
`
`
`12
`
`

`

`dependent on the content of my opinions. I have no direct financial interest in any
`
`of the parties associated with this matter.
`
`II. QUALIFICATIONS
`
`5. My curriculum vitae, which includes a more detailed summary of
`
`my background, experience, and publications, is attached as APPENDIX-A.
`
`6.
`
`I received a Bachelor of Science degree in Computer Science with a
`
`minor in Electrical Engineering from Rensselaer Polytechnic Institute (“RPI”) in
`
`1986. I received a Master of Science degree and a Ph.D. in Computer Science
`
`from New York University (“NYU”) in 1989 and 1992, respectively.
`
`7.
`
`Since 1998, I have been a Professor of Computer Science at the
`
`University of Maryland (“UMD”), where I have had joint appointments at the
`
`Institute for Advanced Computer Studies and the College of Information Studies
`
`(Maryland’s “iSchool”) and am currently Professor Emeritus. I was also Associate
`
`Provost of Learning Initiatives and Executive Director of the Teaching and
`
`Learning Transformation Center from 2014 to 2018. I am a member and previous
`
`director of the Human-Computer Interaction Lab (“HCIL”), the oldest and one of
`
`the best known Human-Computer Interaction research groups in the country.
`
`8.
`
`I am also co-founder and co-director of the International Children’s
`
`Digital Library (“ICDL”), a web site launched in 2002 that provides the world’s
`
`largest collection of freely available online children’s books from around the world
`
`2
`
`13
`
`

`

`with an interface aimed to make it easy for children and adults to search and read
`
`children’s books online. I was also co-founder and Chief Scientist of Zumobi, Inc.
`
`from 2006 to 2014, a Seattle-based startup that is a publisher of content
`
`applications and advertising platforms for smartphones. I was also co-founder and
`
`prior Chief Technology Officer of Hazel Analytics, a data analytics company
`
`whose product sends alerts in warranted circumstances. In addition, I have for
`
`more than 25 years consulted for numerous companies in the area of user
`
`interfaces, including Logitech, Microsoft, the Palo Alto Research Center, Sony,
`
`Lockheed Martin, and NASA Goddard Space Flight Center.
`
`9.
`
`For more than 30 years, I have studied, designed, and worked in the
`
`field of computer science and human-computer interaction. My experience
`
`includes 30 years of teaching and research, with research interests in human-
`
`computer interaction and the software and technology underlying today’s
`
`interactive computing systems. This includes the design and implementation of
`
`user interfaces and software applications using a wide range of technologies
`
`including web servers and websites, databases, embedded systems, and mobile
`
`devices, including smart phones and PDAs. I have combined my hardware and
`
`software skills throughout my career in Human-Computer Interface research,
`
`building various interactive electrical and mechanical systems that couple with
`
`software to provide an innovative user experience.
`
`3
`
`14
`
`

`

`10. At UMD, my research has been in the area of Human-Computer
`
`Interaction (“HCI”), a field that relates to the development and understanding of
`
`computing systems to serve users’ needs. Researchers in this field are focused on
`
`making universally usable, useful, efficient, and appealing systems to support
`
`people in their wide range of activities. My approach is to balance the
`
`development of innovative technology that serves people’s practical needs.