`
`
`Paper 52
`Entered: September 14, 2021
`
`Trials@uspto.gov
`571-272-7822
`
`
`
`
`
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`___________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`___________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`___________
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`___________
`
`Record of Oral Hearing
`Held: August 12, 2021
`_____________
`
`
`
`
`Before BRYAN F. MOORE, JOHN F. HORVATH, and
`MONICA S. ULLAGADDI, Administrative Patent Judges.
`
`
`
`
`
`
`
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`
`
`APPEARANCES:
`
`ON BEHALF OF THE PETITIONER:
`
`
`
`MICHAEL PARSONS, ESQUIRE
`
`Haynes & Boone, LLP
`
`6000 Headquarters Drive
`
`Suite 200
`
`Plano, TX 75024
`
`
`ON BEHALF OF PATENT OWNER:
`
`
`
`
`
`
`
`
`
`
`
`MARC FENSTER, ESQUIRE
`Russ, August & Kabat
`12424 Wilshire Blvd.
`12th Floor
`Los Angeles, CA 90025
`
`
`
`
`The above-entitled matter came on for hearing on Thursday, August
`12, 2021, commencing at 12:59 p.m., EDT, at the U.S. Patent and
`Trademark Office, by video/by telephone, before Chris Hofer, Notary
`Public.
`
`
`
`
`
` 2
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`
`
`
`
`
`
`
`
`P R O C E E D I N G S
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
` - - - - -
`
`
`
`JUDGE HORVATH: Good afternoon everyone. This is
`
`Judge Horvath at the Patent Trial & Appeal Board. With me
`
`today are Judges Ullagaddi and Moore and we here for oral
`
`argument in Apple, Inc. v. Corephotonics, Ltd , case No. IPR
`
`2020-00905 in which Apple challenges the patentability of
`
`certain claims in U.S. patent No. 10,225,479 B2 which is
`
`assigned to Corephotonics. Can I ask counsel for Apple to
`
`please introduce yourself.
`
`10
`
`
`
`MR. PARSONS: Judge Horvath, this is Michael Parsons
`
`11
`
`with Haynes & Boone representing Petitioner Apple.
`
`12
`
`
`
`JUDGE HORVATH: Okay. Thank you, Mr. Parsons. And
`
`13
`
`will you be presenting the argument on behalf of Apple today?
`
`14
`
`15
`
`
`
`
`
`MR. PARSONS: Yes, I will be. Thank y ou.
`
`JUDGE HORVATH: Thank you, sir. And counsel for
`
`16
`
`Corephotonics, please introduce yourself.
`
`17
`
`
`
`MR. FENSTER: Yes. Good morning, Your Honor. This is
`
`18
`
`Marc Fenster with Russ, August & Kabat and I'll be presenting
`
`19
`
`argument on behalf of the Patent Owner Cor ephotonics. With me
`
`20
`
`on the line are my colleagues Neil Rubin and James Tsuei.
`
`21
`
`
`
`JUDGE HORVATH: Okay. Thank you, Mr. Fenster. So
`
`22
`
`before we begin I'd like to address some administrative issues
`
`23
`
`and provide you with some guidance. Before I do that though I 'd
`
`24
`
`like to thank you for participating in this hearing remotely and
`
`
`
`
`
` 3
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`for also agreeing to allow this hearing to be recorded in order to
`
`be presented at the InSide PTAB conference or event to be held
`
`later this month.
`
`
`
`Now this is the first time, for me anyw ay, that we've done
`
`one of these hearings via Webex. I think this is a new
`
`technology platform we're trying. I don't expect there to be any
`
`technical issues. We've already had a few with audio but those
`
`were mostly, I think at least on my part -- just -- I didn't unmute
`
`myself and I think that's -- so there's a little bit of a learning
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`curve here. Hopefully we won't have any technical issues that
`
`11
`
`we have to resolve, but in the event that we do, please try to let
`
`12
`
`us know and we will of course suspend the proceeding until we
`
`13
`
`can resolve that issue and try and keep track of the time so that
`
`14
`
`no one's time is lost.
`
`15
`
`
`
`Let's see. I think, as I said, a portion at least of this
`
`16
`
`hearing is going to be recorded to be presented at the InSide
`
`17
`
`PTAB event. I will note that Patent Owner Corephotonics has
`
`18
`
`produced evidence of secondary considerations of
`
`19
`
`nonobviousness. That evidence is currently under seal and so it's
`
`20
`
`possible that to the extent that evidence will be discussed at
`
`21
`
`today's hearing, a portion of this heari ng will have to be
`
`22
`
`conducted under seal. That portion of course will not be
`
`23
`
`recorded and will not be shown at the InSide PTAB event.
`
`24
`
`
`
`Per the terms of our Order on July 6, each party will have a
`
`25
`
`total of 45 minutes argument time. Because Apple bears the
`
`
`
`
`
` 4
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`burden of demonstrating the unpatentability of the claims in this
`
`challenge, Apple will proceed first and may reserve a portion of
`
`its total argument time for rebuttal argument. Next,
`
`Corephotonics will present its response to Apple's arguments and
`
`may introduce argument on secondary considerations of
`
`nonobviousness. Corephotonics may also then reserve a portion
`
`of its total time for sur -rebuttal. As I said, because that
`
`secondary consideration evidence is under seal, we'd ask that
`
`Corephotonics present any argument on that issue at the end of
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`its response time and we'd also ask that Corephotonics let the
`
`11
`
`Board know prior to discussing any secondary considerations
`
`12
`
`evidence so that we could discontinue the recording of this
`
`13
`
`hearing and seal the remaining portion of the hearing.
`
`14
`
`
`
`After Corephotonics presents its response, Apple may
`
`15
`
`present rebuttal argument with the time it had reserved including
`
`16
`
`argument on secondary considerations of nonobvi ousness and
`
`17
`
`then Corephotonics may have the last word and present its sur-
`
`18
`
`rebuttal argument in whatever time Corephotonics has reserved
`
`19
`
`for rebuttal. Does everyone understand those instructions?
`
`20
`
`21
`
`22
`
`
`
`
`
`
`
`MR. PARSONS: Yes, Your Honor.
`
`MR. FENSTER: Yes, Your Honor.
`
`JUDGE HORVATH: Okay. Thank you. So, Mr. Parsons,
`
`23
`
`would Apple like to reserve time for rebuttal?
`
`24
`
`
`
`MR. PARSONS: Yes, Your Honor. We'd like to reserve 15
`
`25
`
`minutes for rebuttal.
`
`
`
`
`
` 5
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`
`
`JUDGE HORVATH: I'm sorry, I didn't catch that. How
`
`many?
`
`
`
`
`
`MR. PARSONS: Fifteen minutes.
`
`JUDGE HORVATH: Fifteen minutes. Okay. So I am the
`
`official timekeeper and I am just using a -- so you have 45
`
`minutes, you're reserving 15 minutes, that would give you 30
`
`minutes for your principal argument and I am just going to set
`
`my digital phone here which has a timer on it. When you begin
`
`your argument I will start the timer and I will give you a three
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`minute warning when your time is about to expire. So whenever
`
`11
`
`you're ready, you may begin.
`
`12
`
`
`
`MR. PARSONS: Thank you, Your Honor. Well, I'd like
`
`13
`
`the slide deck for IPR 2020 -905, the Petitioner's slide deck
`
`14
`
`brought up and I'll just give slide numbers to have the person
`
`15
`
`controlling flip through them.
`
`16
`
`17
`
`
`
`
`
`STAFF: Which one do you need?
`
`MR. PARSONS: IPR -- the 905 IPR, the Petitioner's slide
`
`18
`
`deck.
`
`19
`
`20
`
`21
`
`22
`
`23
`
`24
`
`
`
`
`
`
`
`
`
`
`
`
`
`STAFF: Okay.
`
`MR. PARSONS: All right. Can every body see it?
`
`JUDGE HORVATH: No.
`
`MR. PARSONS: I can't see it either.
`
`JUDGE HORVATH: There we go. I can see it now.
`
`MR. PARSONS: Okay. I'm not able to see it but I'll follow
`
`25
`
`a paper copy on my end. Great. And I'm ready to start, Judge
`
`
`
`
`
` 6
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`Horvath.
`
`
`
`JUDGE HORVATH: Okay. Let me begin then. Okay.
`
`Begin.
`
`
`
`MR. PARSONS: Thank you. Your Honors, I'm Michael
`
`Parsons with Haynes & Boone representing Petitioner Apple
`
`today. For IPR 2020-905 that addresses the '479 patent the main
`
`issues before -- that we'd like to discuss with the Board today
`
`mostly address claim 1 of the arguments that Patent Owner has
`
`made with respect to claim 1 and particularly the issues that
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`Patent Owner has raised in its surreply.
`
`11
`
`
`
`Flipping to slide 2. The first topic that we would like to
`
`12
`
`discuss today deals with the claim construction that Patent
`
`13
`
`Owner has presented with the term in claim 1 and it's the only
`
`14
`
`claim construction dispute before the Board, and specifically
`
`15
`
`flipping to slide 2, specifically the problem with Patent Owne r's
`
`16
`
`claim construction is that it requires this concept of wide
`
`17
`
`perspective POV and that that be maintained in a fused image. If
`
`18
`
`you look at the limitation of 1(e) in slide 3 the highlighted
`
`19
`
`portion captures what's in dispute -- that limitation that's in
`
`20
`
`dispute between the parties and it says Wherein the camera
`
`21
`
`controller is further operative to output the fused image point of
`
`22
`
`view of the wide -- of the wide camera. And so what does this
`
`23
`
`mean? What this means is is that in claim 1 you have a wide
`
`24
`
`camera and a telephoto camera and each of these cameras have a
`
`25
`
`specific perspective on a scene that is overlapping between the
`
`
`
`
`
` 7
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`two of them. And so when you capture an image between these
`
`two cameras, you end up having one image on the telephoto side
`
`and one image on the wide camera side and those images will
`
`capture objects in a particular position and they will have a
`
`particular shape. And so what the parties are disputing in this is
`
`that the wide POV of the wide camera is whether or not just the
`
`object positions must be maintained or whether the object shapes
`
`must be maintained. Petitioner's construction in this regard is
`
`that limitation 1(e) should be read to output a fused image in
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`which the positions or shapes of objects reflect those of the wide
`
`11
`
`camera and Patent Owner's construction replaces the "or" with an
`
`12
`
`"and" and they think that this term means that both the shape and
`
`13
`
`the position of the objects should be maintained in the fused
`
`14
`
`image.
`
`15
`
`
`
`Now flipping to slide 4. We're referring to Patent Owner's
`
`16
`
`response here on slide 4 and what they've described POV -- point
`
`17
`
`of view -- is that it refers to how objects are seen by each sub -
`
`18
`
`camera or how the objects will be shifted and have different
`
`19
`
`perspective or shape for the two cameras . And so what this
`
`20
`
`means is is that if you're using the camera with different point of
`
`21
`
`views, like a wide and a telephoto camera with overlapping
`
`22
`
`fields of view, you can shift an object or in other words change
`
`23
`
`its position in an image or change the perspective of an object
`
`24
`
`which means to change its shape. And in reference to this
`
`25
`
`specific thing, if you look at the two images at the bottom of
`
`
`
`
`
` 8
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`slide 4 that are borrowed from the Szeliski reference , it shows
`
`two scenes that are captured with cameras that have slightly
`
`different points of view. And if you look closely you can see
`
`that there's some objects that are shifted in different positions
`
`and then there is some objects that have slightly different shapes
`
`between the two images.
`
`
`
`Now in the specification, flipping to slide 5, what w e're
`
`pointing at here is column 5 of the '479 patent. This is where it
`
`describes the concept of point of view. The first sentence is
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`what Patent Owner addressed in its response , which we just
`
`11
`
`talked about, which is that in a system where you have dual
`
`12
`
`aperture cameras of a wide and a telephoto image with an
`
`13
`
`overlapping field of view, that both that the positions in between
`
`14
`
`the two cameras can have different shapes or positions of the
`
`15
`
`objects. And if the objects have the shape of a particular
`
`16
`
`camera, then that is maintaining wide perspective POV. If the
`
`17
`
`objects maintain the positions of a particular camera, then that
`
`18
`
`ends up being position POV and so when you look --
`
`19
`
`JUDGE HORVATH: So Mr. Parsons, can I ask you a
`
`20
`
`question?
`
`21
`
`22
`
`MR. PARSONS: Yes.
`
`JUDGE HORVATH: So would you agree that if I have two
`
`23
`
`images and let's say there's just an object in each image, would
`
`24
`
`you agree that there are really four possible combinations of
`
`25
`
`those two images. You could have the shape and the perspective
`
`
`
`
`
` 9
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`of the object from the wide camera, or I could have the shape and
`
`perspective of the object from the tele camera, or I could have
`
`the shape of the object from the wide camera but in the position
`
`where it would be in the tele camera, or I could have the shape of
`
`the object from t he tele camera but in the position it would be in
`
`the wide camera. Do you agree that those are the only four
`
`possibilities?
`
`MR. PARSONS: Yes, Your Honor, I do . And in fact if you
`
`look at slide 5 and the portion that we've highlighted in yellow
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`from the patent it says that exact thing. It says that the system
`
`11
`
`output image, which in this case would be the fused image, can
`
`12
`
`have the shape and position of either sub -camera or the shape or
`
`13
`
`position of a combination thereof . And so based on this
`
`14
`
`statement, I do agree with you that there are only four possible
`
`15
`
`combinations of this.
`
`16
`
`JUDGE HORVATH: Okay. So given that , if I say that an
`
`17
`
`object has the -- let's say the wide position point of view -- so it
`
`18
`
`has the position the object would have if it was taken fro m the
`
`19
`
`point of view of the wide camera , would you then agree that it
`
`20
`
`can either have the wide shape or the tele camera shape? Those
`
`21
`
`are the only two possibilities; right?
`
`22
`
`23
`
`MR. PARSONS: Right.
`
`JUDGE HORVATH: So why would I not distinguish
`
`24
`
`between those two possibilities? If I just say it has the wide
`
`25
`
`position point of view, then I don't know necessarily whether it
`
`
`
`
`
` 10
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`also has the wide shape point of view or the teleposition point of
`
`view, I'm sorry, the teleshape point of view? But if I say -- and
`
`there is a way to distinguish that isn't there ? Because I could say
`
`if it has the wide point of view then it has both the wide position
`
`and the wide shape. But if I just say it has the wide position
`
`point of view, then I automatically -- I would know that well, i t
`
`has the wide position point of view, not the wide point of view ,
`
`so therefore it must have the tele shape point of view. Why isn't
`
`it as simple as that?
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`MR. PARSONS: Well, Your Honor, if you look at the
`
`11
`
`second portion that is highlighted in green in sl ide 5, this
`
`12
`
`specifically refers to the fact the only thing that's necessary to
`
`13
`
`maintain a wide POV is doing a registration mapping process
`
`14
`
`that matches the pixels between the telephoto image with the
`
`15
`
`wide image and so if you just want to maintain a POV of the
`
`16
`
`wide image, then all you have to do is this registration process.
`
`17
`
`If you look at the second portion highlighted in pink at the
`
`18
`
`bottom of the paragraph, it calls out a different process that's
`
`19
`
`used to therefore maintain the perspective point of view . And
`
`20
`
`this specifically says that it's also possible to perform the
`
`21
`
`registration after each sub -camera is shifted and in that case the
`
`22
`
`output of the image will retain the respective wide or tele
`
`23
`
`perspective point of view. So if you're just doing a registrati on
`
`24
`
`process which is what the claim says, if we look at the claim it
`
`25
`
`specifically says that it maintains the POV of the wide camera by
`
`
`
`
`
` 11
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`doing the registration mapping process of matching the wide
`
`pixels and the tele pixels. So based on this description here , if
`
`you're just doing that and you're not also shifting before
`
`registration, then you're just maintaining wide position POV
`
`because in order to maintain wide perspective POV with the
`
`portions of the tele image that get fused in, you have to shift
`
`those tele pixels in some way. And based on the fact that a
`
`shifting process is required more than just happens before
`
`registration mapping, that's the distinction here is that in order to
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`maintain wide perspective you have to do something in addition
`
`11
`
`to just register.
`
`12
`
`Now if we flip to slide 6 . Slide 6 shows the figure 5
`
`13
`
`embodiment from the '479 patent and this is the embodiment that
`
`14
`
`describes image fusion . And if you look at the portion on the
`
`15
`
`right this is the figure from the patent and you'll note that in st ep
`
`16
`
`506 that's where it performs the registration process. There are
`
`17
`
`only two steps that occur before that. The first one in 502 is it
`
`18
`
`just contains the images and the second step is that it rectifies
`
`19
`
`the images. Now, the rectification process here is re cited in
`
`20
`
`claim 2 so based on claim differentiation that can't be the
`
`21
`
`shifting process that is required in order to maintain the wide
`
`22
`
`perspective POV. So based on this embodiment in the
`
`23
`
`specification, the proper claim construction of this term is that if
`
`24
`
`you're just doing pixel matching to do the registration process ,
`
`25
`
`which is what the claim says, then all you're doing is maintaining
`
`
`
`
`
` 12
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`wide position POV. If you want to maintain the wide
`
`perspective, you have to do the additional step before
`
`registration and there's no mention in the claim of doing an
`
`additional shifting step in order to shift those pixels over.
`
`JUDGE HORVATH: So I do have a question about this. It
`
`seems to me, and let me know if you disagree, but it seems to me
`
`that figure 5 is really the o nly figure in this patent that talks
`
`about how you create a fused image. Would you agree with that?
`
`MR. PARSONS: Correct. I agree with that.
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`JUDGE HORVATH: Okay. So there are other steps in
`
`11
`
`figure 5. So if you look at step 508, it says the tele image is
`
`12
`
`resampled based on the registration map. That's step 508. Now
`
`13
`
`doesn't that mean -- doesn't resampling mean basically shifting
`
`14
`
`the tele image so that the, you know, you use this registration
`
`15
`
`map to resample the tele image, like you get something that
`
`16
`
`looks more akin to what the wide image is. Isn't that what that's
`
`17
`
`saying?
`
`18
`
`MR. PARSONS: Well, no, Your Honor. I disagree with
`
`19
`
`that because if you look back at slide 5 in the bottom of the
`
`20
`
`paragraph that we've pulled out that's in pink it requires that th e
`
`21
`
`shifting process be performed before registration. What you're
`
`22
`
`referring to in step 508 is happening after registration so that's
`
`23
`
`not shifting. What we believe that's actually doing is that it's
`
`24
`
`upscaling the image so that the objects in the two images are the
`
`25
`
`same size and in order to fuse the tele portions with the wide
`
`
`
`
`
` 13
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`portion because they have different fields of view but they're
`
`overlapping. Objects in the tele image are going to appear larger
`
`than they are in the wide image and so by resampling t he tele
`
`image it means you're actually going to scale it to the same -- so
`
`the objects are the same size as what they are in the wide image .
`
`And therefore when you pull those portions out, that they can go
`
`into the wide image at the same, you know, at the same size so
`
`you're not obscuring other portions of the image.
`
`JUDGE HORVATH: Okay. So let me ask you this. There
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`are other steps then in figure 5 and what is intriguing to me is
`
`11
`
`this last step which is in 510, this decision step where they
`
`12
`
`compare the pixels in the tele image to the pixels in the wide
`
`13
`
`image and determine if there's an error in that registration map .
`
`14
`
`And then if there is an error, they pick the wide image pixel.
`
`15
`
`Now, the way I look at that , and I'm taking that and I'm
`
`16
`
`considering that in combination with what they said about the
`
`17
`
`point of view shifting -- and I sort of had an image in my mind
`
`18
`
`that, you know, I could have a camera that's looking let's say
`
`19
`
`straight on at an I-beam so that all I really see is the “I” of the I-
`
`20
`
`beam; right? And let's say that's painted red. I don't see any of
`
`21
`
`the sides. But then I have another camera which is to the right
`
`22
`
`of that camera and so it has a different point of view. So now
`
`23
`
`that camera is seeing two things. It sees the red of the I-beam,
`
`24
`
`the face of the I-beam, but it also sees the side of the I-beam
`
`25
`
`which let's say is colored blue; right? So if I try to register
`
`
`
`
`
` 14
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`pixels and map pixels, I'm going to map some of the red from the
`
`tele camera, from the camera on the right, and it sees both the
`
`front of the I-beam and the side of the I-beam and so it's going to
`
`have those red pixels from the front face of the I-beam and
`
`they're going to map to the pixels that I see from the wide
`
`camera. But there's going to be pixels from the camera to the
`
`right, the tele camera that sees the side of the I-beam that's blue.
`
`Those aren't going to match anything that I see from the wide
`
`camera. So isn't that what they mean by registration error ? And
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`in that case they say choose the information from the wide
`
`11
`
`camera, so I'm throwing away the tele -perspective.
`
`12
`
`MR. PARSONS: Well, Your Honor, that concept is
`
`13
`
`actually claimed in claim 5 which also -- based on claim
`
`14
`
`differentiation -- does not determine whether or not you're
`
`15
`
`maintaining wide POV because that error process is actually
`
`16
`
`happening in a different claim. So that's not what's doing the
`
`17
`
`maintaining of the wide POV. Also that’s happening again after
`
`18
`
`the registration process and the specification specifically says
`
`19
`
`that you have to shift before registration because ot herwise you
`
`20
`
`end up with exactly this error that you described where you will
`
`21
`
`have errors between the two of them because you didn't shift the
`
`22
`
`tele pixels in to have the perspective of the wide camera before
`
`23
`
`you did the registration mapping process.
`
`24
`
`25
`
`
`
`
`
`JUDGE HORVATH: Uh-huh. Okay.
`
`MR. PARSONS: Well, thank you, Your Honor. In light of
`
` 15
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`that discussion I'd like to skip to our second topic. So if we can
`
`go ahead and skip to slide 9. Our second topic that we'd like to
`
`discuss addresses specifically Parulski's figure 14 that we
`
`pointed to in the petition as being an image enhancement
`
`technique that broadens the depth of field of the wide image .
`
`And so what Parulski teaches is -- Parulski teaches a dual
`
`aperture camera with wide and tele lenses that overlap and what
`
`happens in figure 14 is that it's an image enhancement method
`
`and it's described as having a specific purpose.
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`On slide 9, this is the figure that we provided in the
`
`11
`
`petition that was annotated by Dr. Durand and what it says in
`
`12
`
`502 is that it looks at the zoom position and if the zoom position
`
`13
`
`is such that the primary capture device is the wide camera and
`
`14
`
`the secondary capture device is the tele camera, then in 510 it
`
`15
`
`captures the wide image, in step 512 it captures the telephoto
`
`16
`
`image and in step 514 it uses the telephoto image to enhance the
`
`17
`
`depth of field of the wide image.
`
`18
`
`Now, if you flip to slide 10, this is referring to column 28
`
`19
`
`of Parulski and this further describes figure 14. It says that
`
`20
`
`figure 14 is a first type of augmentation method that' s depicted
`
`21
`
`in Parulski and specific to figure 14 . If you look at the second
`
`22
`
`portion that's highlighted in yellow there, it says that the two
`
`23
`
`images are combined into a modified image with a broadened
`
`24
`
`depth of field. So in other words, what figure 14 is doing in that
`
`25
`
`last step, it is taking portions of the secondary stage, which is
`
`
`
`
`
` 16
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`the telephoto, and then taking portions of that and combining it
`
`with the wide image to broaden the depth of the wide image
`
`which means to bring more portions of the wide image into
`
`focus.
`
`Now if we flip back to slide 8 , and specifically look at the
`
`last step in figure 14 , which is block 514 , it says that the
`
`secondary still image is used to enhance the depth of field of the
`
`primary image, for instance, where the secondary still image is
`
`used to provide an enhancement (indiscernible) used to sharpen
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`portions of the primary still image that are positioned near the
`
`11
`
`secondary focused images. So in other words, what step 514 is
`
`12
`
`doing is it's taking portions of the telephoto image and it's
`
`13
`
`identifying them based on the focus distance from the tele
`
`14
`
`camera and then it takes those and it combines them with the
`
`15
`
`wide image to produce a modified image with a broader depth of
`
`16
`
`field to bring more of the wide image into focus.
`
`17
`
`Now, Patent Owner has argued that there is no teaching in
`
`18
`
`Parulski to combine figure 14 to use the range map discussed in
`
`19
`
`figure 11. Now, but Parulski actually says it right here. It says
`
`20
`
`that it's using portions of the secondary image which in this case
`
`21
`
`would be the telephoto, to sharpen the same portions of the wide
`
`22
`
`image. And it's focusing on the portions of the telephoto image
`
`23
`
`that are at the focus distance of the telephoto image which , in
`
`24
`
`other words, means that you have to have some way to identify
`
`25
`
`what the portions of the image are that are at the telephoto focus
`
`
`
`
`
` 17
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`distance in order to identify them and extract them in order to
`
`combine them with the wide image.
`
`Now, if we go ahead and skip to slide 13, this discuses
`
`Parulski's figure 11 embodiment . And the first portion that
`
`we've called out here -- it discusses the method that Parulski
`
`uses in order to build a registration map . And what it's doing is
`
`it's correlating the pixels and portions of the two images to
`
`identify portions of the objects that are similar . And there's no
`
`dispute from Patent Owner that this is a registration map process
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`that satisfies the last portion of claim 1 that does pixel matching
`
`11
`
`between the wide and tele cameras.
`
`12
`
`Once you have a range map, Parulski teaches in the second
`
`13
`
`portion that we've called out here on slide 13 -- is that the range
`
`14
`
`map can then be used to modify an image for a variety of
`
`15
`
`purposes. And two important things that would be relevant to
`
`16
`
`figure 14 for figuring out what portions of the telephoto image or
`
`17
`
`the telephoto focus distance is that the range map can be used to
`
`18
`
`improve object identification within the image by identifying the
`
`19
`
`continuous boundaries of the object so the shape of the object
`
`20
`
`can be defined. And then it also enables object extraction from
`
`21
`
`the errors by using that same identification of the continuous
`
`22
`
`boundaries of the object so it can be segmented within the image.
`
`23
`
`So in other words, figure 14 says that you use portions of the
`
`24
`
`telephoto image at the telephoto focus distance and then you take
`
`25
`
`those and you take them out of the telephoto and then you merge
`
`
`
`
`
` 18
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`them with the wide image to create a combined image . And what
`
`this does is it enables the in -focus portions of the telephoto
`
`image to then enhance the wide image and increase the number
`
`of -- the portions of the wide image that are in focus , which is
`
`what broadening the depth of field means.
`
`JUDGE HORVATH: So, Mr. Parsons, if I can ask you a
`
`question. So when you do this, you know, extracting an image
`
`from the telephoto lens for example, or from the tel ephoto image,
`
`extracting an object and then copying that and placing it in the
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`position of that object from the wide image , would you agree
`
`11
`
`that in doing that you are only preserving the position, the wide
`
`12
`
`position point of view and not both the wide posit ion and the
`
`13
`
`wide perspective point of view?
`
`14
`
`MR. PARSONS: Well, Your Honor, our position in the
`
`15
`
`petition was exactly that . That if you just perform the
`
`16
`
`registration process to match the tele pixels with the wide pixels
`
`17
`
`to identify where the objects are and then you just extract the
`
`18
`
`objects from the tele and use it with the wide, that that does in
`
`19
`
`fact maintain the wide position POV because the wide image is
`
`20
`
`the one that's maintained . And all of the objects that have been
`
`21
`
`moved over are in the same position and then the objects that are
`
`22
`
`moved over from the tele image would be in the same position
`
`23
`
`because you used the registration mapping process. But Parulski
`
`24
`
`isn't clear on whether or not there's a shifting process that
`
`25
`
`happens before registration. So that was our position in the
`
`
`
`
`
` 19
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`petition.
`
`But let me supplement that a little bit. On page 3 of the
`
`Patent Owner's surreply , in the second paragraph, the last
`
`sentence, they specifically say that our position of just
`
`maintaining position POV is incorrect because , and this is what
`
`they say, they say this is incorrect as it i gnores that the
`
`registering pixels to matching pixels will necessarily address
`
`both position and perspective. So in the surreply Patent Owner
`
`has taken the position it doesn't matter if you us e the registration
`
`1
`
`2
`
`3
`
`4
`
`5
`
`6
`
`7
`
`8
`
`9
`
`10
`
`map, it does both position and perspective and there's no dispute
`
`11
`
`at the end that Parulski's registration mapping process satisfies
`
`12
`
`the limitation in the claim. So even based on Patent Owner's
`
`13
`
`position in the surreply it seems to be tha t if you just do the
`
`14
`
`registration process , based on their argument, that it would
`
`15
`
`maintain both.
`
`16
`
`JUDGE HORVATH: I'm sorry. What page number was
`
`17
`
`that from the surreply?
`
`18
`
`MR. PARSONS: Page 3 of the surreply. It's in the second
`
`19
`
`paragraph.
`
`20
`
`21
`
`JUDGE HORVATH: P age 3.
`
`MR. PARSONS: It's the last sentence of the first full
`
`22
`
`paragraph.
`
`23
`
`24
`
`JUDGE HORVATH: Okay. Okay. Thank you.
`
`MR. PARSONS: Right. Thank you. So I'll continue on.
`
`25
`
`In slide 14 Parulski provides a specific example for using the
`
`
`
`
`
` 20
`
`
`
`IPR2020-00905 (Patent 10,225,479 B2)
`IPR2020-00906 (Patent 10,225,479 B2)
`
`range map. For the por tion that we called out in slide 14
`
`Parulski presented an example scene where it talks about having
`
`mountains in the distance, flowers in the middle and a dog that's
`
`out of focus at five feet. Now, when you then apply the
`
`registration map data , which in this case I'll refer to as the range
`
`map, you know, several features of this scene can be modified .
`
`And if we move to slide 15 one of these methods that this
`
`example talks about is -- specifically -- is the depth of field can
`
`be