UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`ALIGN TECHNOLOGY, INC.
`Petitioner
`
`v.
`
`3SHAPE A/S
`Patent Owner
`
`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`DECLARATION OF DR. CHANDRAJIT L. BAJAJ, PH.D.
`IN SUPPORT OF INTER PARTES REVIEW OF U.S. RE48,221
`
`Align EX1003
`Align v. 3Shape
`IPR2022-00145
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`TABLE OF CONTENTS
`Introduction ........................................................................................1
`I.
`II. Qualifications and Expertise ..................................................................6
`III. Legal Understanding .......................................................................... 11
`A. My Understanding of Claim Construction ....................................... 11
`B. A Person of Ordinary Skill in the Art .............................................. 11
`C. My Understanding of Obviousness ................................................. 13
`IV. Background of the Technologies Disclosed in the ’221 Patent ................. 14
`A. 3D handheld digital scanning was well known. ................................ 15
`1. Dental application of three-dimensional handheld digital scanning
`was well known..................................................................... 16
`Incorporating human-computer interfaces into three-dimensional
`handheld digital scanning and interactive computer graphics display
`was well known..................................................................... 19
`B. Three-dimensional human-computer interfaces with motion sensors
`were well known. ......................................................................... 21
`1. Medical application of three-dimensional human-computer
`interfaces were well known. .................................................... 23
`Incorporation of motion sensors into medical applications and
`three-dimensional human-computer interfaces were well known. . 25
`The ’221 Patent ................................................................................. 29
`A. Overview .................................................................................... 29
`B. Summary of the Prosecution History............................................... 30
`VI. Claim Construction ............................................................................ 32
`VII. Overview of Grounds Asserted in the Petition and Prior Art .................... 32
`A. Overview of Boerjes ..................................................................... 33
`B. Overview of Marvit ...................................................................... 35
`C. Overview of Gandyra ................................................................... 36
`D. Overview of Quadling................................................................... 38
`VIII. Ground 1: Claims [1, 19] 20-44 are unpatentable as obvious under 35 U.S.C.
`§ 103 based on Boerjes and Marvit. ...................................................... 39
`
`2.
`
`2.
`
`V.
`
`
`
`- i -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`
`2.
`
`3.
`
`4.
`
`A. A POSITA Would Have Known How and Why To Combine the
`Teachings of Boerjes and Marvit with a Reasonable Expectation of
`Success. ...................................................................................... 39
`B. Claim 33 ..................................................................................... 43
`1.
`[33.P]: A scanning system for scanning a 3D environment, the
`scanning system comprising:................................................... 44
`[33.1]: a handheld device including an optical scanner, wherein the
`3D environment to be scanned is selected by pointing the optical
`scanner at the 3D environment; and ......................................... 46
`[33.2]: at least one display remotely connected to the handheld
`device .................................................................................. 47
`[33.3.1]: wherein the handheld device is adapted for performing at
`least one scanning action in a physical 3D environment and ........ 49
`[33.3.2]: the at least one display is adapted for visually representing
`the physical 3D environment; and ............................................ 50
`[33.4]: wherein the handheld device includes at least one motion
`sensor for remotely controlling the display to adjust the view with
`which the 3D environment is represented on the display; and ...... 52
`a) Boerjes ....................................................................................... 52
`b) Marvit ........................................................................................ 54
`c) Motivation to Combine ................................................................. 56
`7.
`[33.5]: wherein the at least one motion sensor is an accelerometer,
`gyro, or magnetometer. .......................................................... 59
`a) Boerjes ....................................................................................... 59
`b) Marvit ........................................................................................ 61
`c) Motivation to Combine ................................................................. 62
`C. Claim 34 ..................................................................................... 65
`1.
`[34.0] The scanning system according to claim 33, wherein the
`handheld device further comprises at least two user interface
`elements. .............................................................................. 65
`D. Claim 35 ..................................................................................... 67
`1.
`[35.0] The scanning system according to claim 34, [a] wherein the
`at least two user interface elements comprises at least one button
`
`5.
`
`6.
`
`
`
`- ii -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`
`and a touch-sensitive element, and [b] wherein the display is on a
`cart. ..................................................................................... 67
`a) wherein the at least two user interface elements comprises at least one
`button and a touch-sensitive element, and ........................................ 67
`b) wherein the display is on a cart....................................................... 68
`E. Claim 36 ..................................................................................... 70
`1.
`[36.0] The scanning system according to claim 35, wherein the at
`least one button and the touch-sensitive element provides more than
`one user input. ...................................................................... 70
`F. Claim 37 ..................................................................................... 72
`1.
`[37.0] The scanning system according to claim 36, wherein at least
`one of the user input provides for manually switching between
`performing the at least one scanning action and remotely
`controlling the view. .............................................................. 72
`a) Boerjes ....................................................................................... 72
`b) Marvit ........................................................................................ 74
`c) Motivation to Combine ................................................................. 75
`G. Claim 38 ..................................................................................... 75
`1.
`[38.0] The scanning system according to claim 37, [a] wherein
`switching to remotely controlling the view puts the handheld device
`into a controller mode, [b] wherein holding at least one user
`interface element on the handheld device in conjunction with
`moving the handheld device determines the view of the 3D
`environment on the display in accordance with signals from the
`motion sensor........................................................................ 75
`a) Boerjes ....................................................................................... 76
`b) Marvit ........................................................................................ 78
`c) Motivation to Combine ................................................................. 79
`H. Claim 39 ..................................................................................... 80
`1.
`[39.0] The scanning system according to claim 37, [a] wherein
`switching to remotely controlling the view [p]uts the handheld
`device into a controller mode and [b] wherein when in [the]
`controller mode, moving the handheld device down results in
`
`
`
`- iii -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`showing the view of the 3D environment from a downward viewing
`angle on the display. .............................................................. 80
`a) Boerjes ....................................................................................... 80
`b) Marvit ........................................................................................ 83
`c) Motivation to Combine ................................................................. 84
`I. Claim 44 ..................................................................................... 85
`1.
`[44.0] The scanning system according to claim 33, [a] wherein the
`at least one display is a 3D display, [b] whereby a 3D representation
`of the 3D environment is displayed on the 3D display; [c] wherein
`the handheld device is an intra-oral 3D scanner or an in-ear 3D
`scanner; and [d] wherein the 3D display is configured to project
`stereoscopic image pairs of the 3D representation. ..................... 85
`Invalidated/Cancelled Claim 1 ....................................................... 87
`J.
`K. Claim 20 ..................................................................................... 91
`1.
`[20.0] The scanning system according to claim 1, wherein the at
`least one motion sensor is an accelerometer, gyro, or magnetometer.
` ........................................................................................... 91
`a) Boerjes ....................................................................................... 91
`b) Marvit ........................................................................................ 92
`c) Motivation to Combine ................................................................. 93
`L. Claim 21 ..................................................................................... 93
`1.
`[21.0] The scanning system according to claim 1, [a] wherein the at
`least one motion sensor is adapted for taking the movement of the
`scanner into account [b] while performing the scanning. ............. 93
`a) Boerjes ....................................................................................... 94
`b) Marvit ........................................................................................ 95
`c) Motivation to Combine ................................................................. 96
`M. Claim 22 ..................................................................................... 96
`1.
`[22.0] The scanning system according to claim 1, [a] wherein the
`system comprises at least two motion sensors and [b] wherein the at
`least two motion sensors provide sensor fusion. ......................... 96
`a) Boerjes ....................................................................................... 97
`b) Marvit ........................................................................................ 98
`
`
`
`- iv -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`c) Motivation to Combine ............................................................... 100
`N. Claim 23 ................................................................................... 100
`1.
`[23.0] The scanning system according to claim 1, [a] wherein the at
`least one motion sensor is the 3D user interface for remotely
`controlling the display, [b] wherein the view on the display is
`determined by moving the handheld scanner. .......................... 100
`a) Boerjes ..................................................................................... 101
`b) Marvit ...................................................................................... 102
`c) Motivation to Combine ............................................................... 104
`O. Claim 24 ................................................................................... 104
`1.
`[24.0] The scanning system according to claim 23, wherein moving
`the handheld scanner to point down causes the view on the display
`to be a downward viewing angle. ........................................... 105
`a) Boerjes ..................................................................................... 105
`b) Marvit ...................................................................................... 106
`c) Motivation to Combine ............................................................... 107
`P. Claim 25 ................................................................................... 108
`1.
`[25.0] The scanning system according to claim 1, wherein the
`handheld device further comprises a user-interface element....... 108
`Q. Claim 26 ................................................................................... 109
`1.
`[26.0] The scanning system according to claim 25, wherein the
`user-interface element comprises a touch-sensitive element, a
`button, a scroll-wheel, or a proximity sensor. .......................... 109
`R. Claim 27 ................................................................................... 111
`1.
`[27.0] The scanning system according to claim 25, wherein the
`user-interface element provides more than one user input.......... 112
`S. Claim 28 ................................................................................... 113
`1.
`[28.0] The scanning system according to claim 25, wherein the at
`least one motion sensor and/or the user-interface element are
`utilized in a workflow. ......................................................... 114
`T. Claim 30 ................................................................................... 116
`1.
`[30.0] The scanning system according to claim 1, wherein the at
`least one display is arranged on a cart. .................................... 116
`
`
`
`- v -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`U. Claim 31 ................................................................................... 118
`1.
`[31.0] The scanning system according to claim 1, wherein the at
`least one display is divided into multiple regions. .................... 118
`V. Claim 40 ................................................................................... 120
`1.
`[40.0] The scanning system according to claim 1, [a] wherein the at
`least one display is a 3D display, [b] whereby a 3D representation
`of the 3D environment is displayed on the 3D display; and [c]
`wherein the handheld device is an intra-oral 3D scanner or an in-ear
`3D scanner. ........................................................................ 120
`W. Claim 41 ................................................................................... 122
`1.
`[41.0] The scanning system according to claim 40, wherein the 3D
`display is configured to project stereoscopic image pairs of the 3D
`representation...................................................................... 122
`X. Claim 42 ................................................................................... 124
`1.
`[42.0] The scanning system according to claim 40, wherein the 3D
`display is [a] volumetric display configured to create a 3D effect of
`the 3D representation. .......................................................... 124
`Y. Invalidated/Cancelled Claim 19 ................................................... 126
`Z. Claim 29 ................................................................................... 131
`1.
`[29.0] The system according to claim 19, [a] wherein the handheld
`device is adapted to change a magnification factor of the view
`represented on the at least one display [b] which is determined by
`the motion of the operator’s hand holding the handheld device, [c]
`through the use of the at least one motion sensor...................... 131
`a) Boerjes ..................................................................................... 131
`b) Marvit ...................................................................................... 132
`c) Motivation to Combine ............................................................... 134
`AA. Claim 32 ................................................................................... 134
`1.
`[32.0] The system according to claim 19, [a] wherein the handheld
`device further comprises a user interface element, [b] wherein
`switching to remotely controlling the view puts the handheld device
`into a controller mode, [c] wherein holding the user interface
`element and/or the actuator on the handheld device in conjunction
`with moving the handheld device determines the view of the 3D
`
`
`
`- vi -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`
`2.
`
`3.
`
`4.
`
`environment on the display in accordance with signals from the
`motion sensor...................................................................... 134
`a) Boerjes ..................................................................................... 135
`b) Marvit ...................................................................................... 137
`c) Motivation to Combine ............................................................... 138
`BB. Claim 43 ................................................................................... 139
`1.
`[43.0] The system according to claim 19, [a] wherein the at least
`one display is a 3D display, [b] whereby a 3D representation of the
`3D environment is displayed on the 3D display; and [c] wherein the
`3D display is configured to project stereoscopic image pairs of the
`3D representation. ............................................................... 139
`IX. Ground 2: Claims [1, 19] 20-44 are unpatentable as obvious under 35 U.S.C.
`§ 103 based on Boerjes, Gandyra, and Marvit. ..................................... 141
`A. A POSITA Would Have Known How and Why to Combine the
`Teachings of Boerjes, Gandyra, and Marvit with a Reasonable
`Expectation of Success................................................................ 142
`1. Boerjes-Gandyra Combination .............................................. 142
`2. Boerjes-Gandyra-Marvit Combination ................................... 147
`B. Claim 33 ................................................................................... 151
`1.
`[33.P]: A scanning system for scanning a 3D environment, the
`scanning system comprising:................................................. 151
`[33.1]: a handheld device including an optical scanner, wherein the
`3D environment to be scanned is selected by pointing the optical
`scanner at the 3D environment .............................................. 151
`[33.2]: and at least one display remotely connected to the handheld
`device, ............................................................................... 152
`[33.3.1]: wherein the handheld device is adapted for performing at
`least one scanning action in a physical 3D environment, ........... 152
`[33.3.2]: and the at least one display is adapted for visually
`representing the physical 3D environment;.............................. 153
`[33.4]: and wherein the handheld device includes at least one
`motion sensor for remotely controlling the display to adjust the
`view with which the 3D environment is represented on the display;
` ......................................................................................... 153
`
`5.
`
`6.
`
`
`
`- vii -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`a) Boerjes ..................................................................................... 154
`b) Gandyra .................................................................................... 155
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 157
`d) Marvit ...................................................................................... 159
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 161
`7.
`[33.5]: and wherein the at least one motion sensor is an
`accelerometer, gyro, or magnetometer. ................................... 163
`a) Boerjes ..................................................................................... 163
`b) Gandyra .................................................................................... 165
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 167
`d) Marvit ...................................................................................... 169
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 170
`C. Claim 34 ................................................................................... 173
`1.
`[34.0] The scanning system according to claim 33, wherein the
`handheld device further comprises at least two user interface
`elements. ............................................................................ 173
`D. Claim 35 ................................................................................... 173
`1.
`[35.0] The scanning system according to claim 34, [a] wherein the
`at least two user interface elements comprises at least one button
`and a touch-sensitive element, and [b] wherein the display is on a
`cart. ................................................................................... 173
`E. Claim 36 ................................................................................... 174
`1.
`[36.0] The scanning system according to claim 35, wherein the at
`least one button and the touch-sensitive element provides more than
`one user input. .................................................................... 174
`F. Claim 37 ................................................................................... 174
`1.
`[37.0] The scanning system according to claim 36, wherein at least
`one of the user input provides for manually switching between
`performing the at least one scanning action and remotely
`controlling the view. ............................................................ 174
`a) Boerjes ..................................................................................... 175
`b) Marvit ...................................................................................... 176
`c) Motivation to Combine ............................................................... 177
`
`
`
`- viii -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`G. Claim 38 ................................................................................... 177
`1.
`[38.0] The scanning system according to claim 37, [a] wherein
`switching to remotely controlling the view puts the handheld device
`into a controller mode, [b] wherein holding at least one user
`interface element on the handheld device in conjunction with
`moving the handheld device determines the view of the 3D
`environment on the display in accordance with signals from the
`motion sensor...................................................................... 178
`a) Boerjes ..................................................................................... 178
`b) Gandyra .................................................................................... 180
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 181
`d) Marvit ...................................................................................... 183
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 185
`H. Claim 39 ................................................................................... 186
`1.
`[39.0] The scanning system according to claim 37, [a] wherein
`switching to remotely controlling the view [p]uts the handheld
`device into a controller mode and [b] wherein when in [the]
`controller mode, moving the handheld device down results in
`showing the view of the 3D environment from a downward viewing
`angle on the display. ............................................................ 186
`a) Boerjes ..................................................................................... 186
`b) Gandyra .................................................................................... 188
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 189
`d) Marvit ...................................................................................... 191
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 193
`I. Claim 44 ................................................................................... 193
`1.
`[44.0] The scanning system according to claim 33, [a] wherein the
`at least one display is a 3D display, [b] whereby a 3D representation
`of the 3D environment is displayed on the 3D display; [c] wherein
`the handheld device is an intra-oral 3D scanner or an in-ear 3D
`scanner; and [d] wherein the 3D display is configured to project
`stereoscopic image pairs of the 3D representation. ................... 194
`Invalidated/Cancelled Claim 1 ..................................................... 194
`J.
`K. Claim 20 ................................................................................... 198
`
`
`
`- ix -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`
`1.
`
`[20.0] The scanning system according to claim 1, wherein the at
`least one motion sensor is an accelerometer, gyro, or magnetometer.
` ......................................................................................... 198
`a) Boerjes ..................................................................................... 198
`b) Gandyra .................................................................................... 199
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 201
`L. Claim 21 ................................................................................... 202
`1.
`[21.0] The scanning system according to claim 1, [a] wherein the at
`least one motion sensor is adapted for taking the movement of the
`scanner into account [b] while performing the scanning. ........... 203
`a) Boerjes ..................................................................................... 203
`b) Gandyra .................................................................................... 205
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 206
`M. Claim 22 ................................................................................... 208
`1.
`[22.0] The scanning system according to claim 1, [a] wherein the
`system comprises at least two motion sensors and [b] wherein the at
`least two motion sensors provide sensor fusion. ....................... 208
`a) Boerjes ..................................................................................... 209
`b) Gandyra .................................................................................... 210
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 211
`d) Marvit ...................................................................................... 212
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 214
`N. Claim 23 ................................................................................... 214
`1.
`[23.0] The scanning system according to claim 1, [a] wherein the at
`least one motion sensor is the 3D user interface for remotely
`controlling the display, [b] wherein the view on the display is
`determined by moving the handheld scanner. .......................... 214
`a) Boerjes ..................................................................................... 215
`b) Gandyra .................................................................................... 217
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 218
`d) Marvit ...................................................................................... 220
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 222
`
`
`
`- x -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`O. Claim 24 ................................................................................... 222
`1.
`[24.0] The scanning system according to claim 23, wherein moving
`the handheld scanner to point down causes the view on the display
`to be a downward viewing angle. ........................................... 223
`a) Boerjes ..................................................................................... 223
`b) Gandyra .................................................................................... 224
`c) Motivation to Combine (Boerjes-Gandyra) .................................... 226
`d) Marvit ...................................................................................... 228
`e) Motivation to Combine (Boerjes-Gandyra-Marvit) ......................... 229
`P. Claim 25 ................................................................................... 230
`1.
`[25.0] The scanning system according to claim 1, wherein the
`handheld device further comprises a user-interface element....... 230
`Q. Claim 26 ................................................................................... 230
`1.
`[26.0] The scanning system according to claim 25, wherein the
`user-interface element comprises a touch-sensitive element, a
`button, a scroll-wheel, or a proximity sensor. .......................... 231
`R. Claim 27 ................................................................................... 231
`1.
`[27.0] The scanning system according to claim 25, wherein the
`user-interface element provides more than one user input.......... 231
`S. Claim 28 ................................................................................... 232
`1.
`[28.0] The scanning system according to claim 25, wherein the at
`least one motion sensor and/or the user-interface element are
`utilized in a workflow. ......................................................... 232
`T. Claim 30 ................................................................................... 233
`1.
`[30.0] The scanning system according to claim 1, wherein the at
`least one display is arranged on a cart. .................................... 233
`U. Claim 31 ................................................................................... 233
`1.
`[31.0] The scanning system according to claim 1, wherein the at
`least one display is divided into multiple regions. .................... 233
`V. Claim 40 ................................................................................... 234
`1.
`[40.0] The scanning system according to claim 1, [a] wherein the at
`least one display is a 3D display, [b] whereby a 3D representation
`of the 3D environment is displayed on the 3D display; and [c]
`
`
`
`- xi -
`
`

`

`Case IPR2022-00145
`U.S. Patent No. RE48,221
`
`wherein the handheld device is an intra-oral 3D scanner or an in-ear
`3D scanner. ........................................................................ 234
`W. Claim 41 ................................................................................... 235
`1.
`[41.0] The scanning system according to claim 40, wherein the 3D
`display is configured to project stereoscopic image pairs of the 3D
`representation...................................................................... 235
`X. Claim 42 ................................................................................... 236
`1.
`[42.0] The scanning system according to claim 40, wherein the 3D
`display is [a] volumetric display configured to create a 3D effect of
`the 3D representation. .......................................................... 236
`Y. Invalidated/Cancelled Claim 19 ................................................... 236
`Z. Claim 29 ................................................................................... 241
`1.
`[29.0] The system according to claim 19, [a] wherein the handheld
`device is adapted to change a magnification factor of the view
`represented on the at least one display [b] which is determined by
`the motion of the operator’s hand holding the handheld device,
`through the use of the at least one motion sensor...................... 241
`a) Boerjes ..................................................................................... 241
`b) Gandyra .................................................................................... 242
`c) Motivation to Combine (Boerjes-Gandyra) ......................