I 1111111111111111 11111 111111111111111 IIIII IIIII IIIII IIIII 111111111111111111
`US008363228B2
`
`c12) United States Patent
`Babayoff
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,363,228 B2
`Jan.29,2013
`
`(54) METHOD AND APPARATUS FOR COLOUR
`IMAGING A THREE-DIMENSIONAL
`STRUCTURE
`
`(75)
`
`Inventor: Noam Babayoff, Rishon le Zion (IL)
`
`(73) Assignee: Cadent Ltd., OrYehuda (IL)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`4,575,805 A
`3/1986 Moermann et al.
`4,836,674 A
`6/1989 Lequime et al.
`4,837,732 A
`6/1989 Brandestini et al.
`6/1990 O'harra
`4,935,635 A
`(Continued)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`
`DE
`EP
`
`(21) Appl. No.: 13/333,351
`
`(22)
`
`Filed:
`
`Dec. 21, 2011
`
`(65)
`
`Prior Publication Data
`
`US 2012/0092678 Al
`
`Apr. 19, 2012
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 12/770,379, filed on
`Apr. 29, 2010, now Pat. No. 8,102,538, which is a
`continuation of application No. 12/379,343, filed on
`Feb. 19, 2009, now Pat. No. 7,724,378, which is a
`continuation of application No. 11/889,112, filed on
`Aug. 9, 2007, now Pat. No. 7,511,829, which is a
`continuation of application No. 11/154,520, filed on
`Jun. 17, 2005, now Pat. No. 7,319,529.
`
`(60) Provisional application No. 60/580,109, filed on Jun.
`17, 2004, provisional application No. 60/580,108,
`filed on Jun. 17, 2004.
`
`(51)
`
`Int. Cl.
`(2006.01)
`GOJB 11124
`(52) U.S. Cl. ................ 356/601; 250/559.4; 250/559.49;
`356/479; 356/609
`(58) Field of Classification Search .. 250/559.4-559.49;
`356/479, 601, 609
`See application file for complete search history.
`
`FOREIGN PATENT DOCUMENTS
`196 36 354 Al
`3/1998
`5/1990
`0 367 647 Al
`(Continued)
`
`OTHER PUBLICATIONS
`
`Jerval, J. and L. Seliinne "Laser confocal microscopy and geographic
`information systems
`in
`the study of dental morphology,"
`Palaeontologica electronica 2( 1) [ online] Retrieved from the
`Internet: <URL :http:/ /www. bi ocenter.hel sinki. fi/bi/ evodevo/pdf/
`pe99 .pdf> ( 1999).
`
`(Continued)
`
`Primary Examiner - Gregory J Toatley
`Assistant Examiner -
`Iyabo S Alli
`(7 4) Attorney, Agent, or Firm - Wilson Sonsini Goodrich &
`Rosati
`
`(57)
`
`ABSTRACT
`
`Provided is a device for determining the surface topology and
`associated color of a structure, such as a teeth segment,
`including a scanner for providing depth data for points along
`a two-dimensional array substantially orthogonal to the depth
`direction, and an image acquisition means for providing color
`data for each of the points of the array, while the spatial
`disposition of the device with respect to the structure is main(cid:173)
`tained substantially unchanged. A processor combines the
`color data and depth data for each point in the array, thereby
`providing a three-dimensional color virtual model of the sur(cid:173)
`face of the structure. A corresponding method for determin(cid:173)
`ing the surface topology and associated color of a structure is
`also provided.
`
`26 Claims, 11 Drawing Sheets
`
`,-------------7
`I
`I
`~
`-~ - - I
`22~
`.
`I~~
`I
`I
`L"--------;--.-----' __________ _J
`3D B&W
`2D color
`71
`data
`data
`~C~o-lor~
`illumination
`
`3D color image
`3D B&Wimage
`Object topography data
`
`J-z
`
`y
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`US 8,363,228 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`5,177,556 A
`l/ 1993 Rioux
`5,363,159 A
`1111994 Melvin
`5,372,502 A
`12/1994 Massenet al.
`5,440,393 A
`8/1995 Wenz
`5,690,486 A
`1111997 Zigelbaum
`5,749,830 A
`5/ 1998 Kaneko et al.
`5,766,006 A
`6/ 1998 Murljacic
`5,864,640 A
`1/1999 Miramonti et al.
`6,205,243 Bl
`3/2001 Migdal et al.
`6,208,788 Bl
`3/2001 Nosov
`6,525,819 Bl
`2/2003 Delawter et al.
`6,594,539 Bl
`7/2003 Geng
`6,697,164 Bl
`2/2004 Babayoff et al.
`7,098,435 B2 *
`8/2006 Mueller et al. ............. 250/559.4
`7,319,529 B2 *
`1/2008 Babayoff ...................... 356/601
`7,511,829 B2
`3/2009 Babayoff
`7,724,378 B2
`5/2010 Babayoff
`2002/0091402 Al
`7/2002 Feinsod
`2002/0100884 Al
`8/2002 Maddock
`2004/0029068 Al
`2/2004 Sachdeva et al.
`
`EP
`JP
`WO
`WO
`WO
`
`FOREIGN PATENT DOCUMENTS
`4/1998
`0 837 659 Al
`2001-82935 A
`3/2001
`89/11260 Al
`11/1989
`00/08415 Al
`2/2000
`03/105289 A2
`12/2003
`
`OTHER PUBLICATIONS
`
`Okuda et al., "Evaluation of in vitro secondary caries using confocal
`laser scanning microscope and X-ray analytical microscope" Journal
`of Dentistry 16(3), Summary pp. 191-196 (2003).
`Watson, T.F., "Applications of confocal scanning optical microscopy
`to dentistry," British Dental Journal 171 (9), Summary pp. 287-291
`(1991).
`Watson, T.F. and A. Boyde, "Confocal light microscopic techniques
`for examining dental operative procedures and dental materials. A
`status report for the American Journal of Dentistry," American Jour(cid:173)
`nal ofDentistry4(4), Surmnarypp. 193-200 (1991).
`
`* cited by examiner
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 1 of 11
`
`US 8,363,228 B2
`
`92
`
`
` ["iJoyeouabUO!EUILUNIII
`90usnbesJojo9eyepejep
`
`
`
`JOJOO-l|WiJO|O9QZMW8dGE
`
`
`
`Josseoojdabew|
`
`v¢
`
`
`
`
`
`LSlsByepAydeibodo}joalqo-
`
`
`
`obeuwlJOIONGE.
`
`ebew!Made°
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`
`
`
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 2 of 11
`
`US 8,363,228 B2
`
`N96
`
`FIG.2A
`
`;
`
`FIG.2B
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`|
`N [X',Y',C]
`
`E [X,Y,Z]
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 3 of 11
`
`US 8,363,228 B2
`
` N [X', ¥', C]
`
`> [iavza]
`
`FIG. 2C
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`aie’X,Y] Yox,Y']
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`/
`
`SN
`
`NY,
`
`/
`
`J
`f
`oN
`
`. EE
`NOwS.
`
`os
`Y
`/
`IeN‘
`
`y
`
`ON.
`
`—
`
`SN
`
`/
`/
`
`f
`/
`
`SO
`
`Sy
`
`FIG. 3
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 4 of 11
`
`US 8,363,228 B2
`
`o
`IdOOSOaGN3SultdOeoS9ILdO|WOO4ANOONIV§
`
`oFAV134OIMLNFOS1S
`rrYSwe0¢/\7cS
` AVYEYV
`YOLONGNOODINSS
`veuaCNvdXd||O1LdO
`ceY3Ziuv10d
`
`
`
`82
`
`Yssvl
`
`vr
`
`YOONILVYSD
`
`SNATIONOIN
`
`3SHAPE 1016
`
`3Shape v Align
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 5 of 11
`
`US 8,363,228 B2
`
`
`
`IMAGE CAPTURE
`DEVICE
`
`TO
`TELEPHONE
`LINE
`
`
`
`FIG. 4B
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 6 of 11
`
`US 8,363,228 B2
`
`
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 7 of 11
`
`US 8,363,228 B2
`
`
`
`FIG. 7A
`
`Transmission
`
`Dichroic coating
`spectral behaviour
`
`
`
`Reflection
`
`FIG. 7B
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 8 of 11
`
`US 8,363,228 B2
`
`FIG.8
`
`L
`
`o>
`
`© L
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 9 of 11
`
`US 8,363,228 B2
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`<©
`
`\O\ =
`jas ©
`
`LL
`
`oS
`LO
`wt
`
`[7]
`~~
`wt
`
`©
`-
`-
`Oo
`LL
`
`410
`
`
`So
`v
`
`Nw
`
`S&S
`oO
`—+
`
`
`
`/\
`
`S&S
`oS
`wT
`
`oO
`co
`i
`
`S&S
`Lo
`wt
`
`CSO
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 10 of 11
`
`US 8,363,228 B2
`
`510
`
`‘oh
`
`a
`
`4
`
`“
`
`\
`
`an
`
`\
`
`of confocal
`
`Rest
`system
`150
`
`
`
`530
`
`PP 4
`
`500
`
`26
`
`/
`
`/
`
`N
`
`|
`I
`~ WV—
`
`166
`
`FIG. 12
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 11 of 11
`
`US 8,363,228 B2
`
`system
`
`of confocal
`
`620
`
`FIG. 13
`
`3SHAPE 1016
`
`3ShapevAlign
`
`IPR2021-01383
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`US 8,363,228 B2
`
`1
`METHOD AND APPARATUS FOR COLOUR
`IMAGING A THREE-DIMENSIONAL
`STRUCTURE
`
`This application is a continuation of U.S. application Ser.
`No. 12/770,379 filed on Apr. 29, 2010, which is a continua(cid:173)
`tion of U.S. Ser. No. 12/379,343, filed on Feb. 19, 2009, now
`U.S. Pat. No. 7,724,378, which is a continuation of U.S.
`application Ser. No. 11/889,112, filed on Aug. 9, 2007, now
`U.S. Pat. No. 7,511,829, which is a continuation of U.S.
`application Ser. No. 11/154,520, filed on Jun. 17, 2005, now
`U.S. Pat. No. 7,319,529, an application claiming the benefit
`under 35 U.S.C. § 119( e) of U.S. Provisional Application No.
`60/580,109, filed on Jun. 17, 2004, and claiming the benefit
`under 35 U.S.C. § 119( e) of U.S. Provisional Application No.
`60/580,108, filed on Jun. 17, 2004, the contents of each of
`which are hereby incorporated by reference in their entirety.
`
`FIELD OF THE INVENTION
`
`The present invention relates to optical scanners, particu(cid:173)
`larly for providing a digital representation of three-dimen(cid:173)
`sional objects including color. The invention finds particular
`application in the surveying of the intraoral cavity.
`
`BACKGROUND OF THE INVENTION
`
`Many methods have been developed for obtaining the three
`dimensional location of surface points of an object, for a host
`of applications including, inter alia, the intraoral cavity. Tech(cid:173)
`niques for direct non-contact optical measurement, in par(cid:173)
`ticular for direct optical measurement of teeth and the subse(cid:173)
`quent automatic manufacture of dentures, are known. The
`term "direct optical measurement" signifies surveying of
`teeth in the oral cavity of a patient. This facilitates the obtain(cid:173)
`ment of digital constructional data necessary for the com(cid:173)
`puter-assisted design (CAD) or computer-assisted manufac(cid:173)
`ture (CAM) of tooth replacements without having to make
`any cast impressions of the teeth. Such systems typically
`include an optical probe coupled to an optical pick-up or 40
`receiver such as charge coupled device (CCD) and a proces(cid:173)
`sor implementing a suitable image processing technique to
`design and fabricate virtually the desired product. Such meth(cid:173)
`ods include, for example, confocal imaging techniques as
`described in WO 00/08415 assigned to the present assignee. 45
`These methods provide a digital three-dimensional surface
`model that is inherently monochromatic, i.e., no color infor(cid:173)
`mation is obtained in the imaging process.
`Associating color information with three-dimensional
`objects is not straightforward, particularly when the position 50
`information is obtained by using a three dimensional scan(cid:173)
`ning method, while the color information is obtained by using
`a two dimensional scanning method. The problem of confor(cid:173)
`mally mapping the two dimensional color information onto
`the three dimensional surface model is difficult and it is com- 55
`mon for mismatching of the color with three-dimensional
`points to occur. Essentially, where two-dimensional color
`detectors are used for obtaining the color information, it is
`difficult to accurately associate color information from the
`detectors with the correct points on the three dimensional
`surface model, particularly where relative movement
`between the object and the device occurs between the acqui(cid:173)
`sition of the three-dimensional topological data and acquisi(cid:173)
`tion of the two-dimensional image data.
`EP 837 659 describes a process and device for obtaining a
`three dimensional image of teeth. Three-dimensional surface
`data is obtained by first covering the surface with an opaque,
`
`2
`diffusely reflecting material, and the object is illuminated
`with monochromatic light. The image of the object under the
`layer is obtained by the process described in U.S. Pat. No.
`4,575,805 using intensity pattern techniques. In order to
`5 obtain a two-dimensional color image of the object, the
`reflecting layer has to be removed. The method thus requires
`the camera to be manually re-aligned so that the two-dimen(cid:173)
`sional color image should more or less correspond to the same
`part of the object as the three dimensional image. Then, the
`10 three dimensional image may be viewed on a screen as a
`two-dimensional image, and it is possible to superimpose on
`this two-dimensional image the two-dimensional color image
`of the teeth taken by the camera.
`U.S. Pat. No. 6,594,539 provides an intraoral imaging sys-
`15 tern that produces images of a dental surface, including three
`dimensional surface images and also two dimensional color
`images, with the same camera.
`In U.S. Pat. No. 5,440,393, the shape and dimensions of a
`dental patients mouth cavity including upper and lower tooth
`20 areas and the jaw structure, are measured by an optical scan(cid:173)
`ner using an external radiation source, whose reflected signals
`are received externally and converted into electronic signals
`for analysis by a computer. Both surface radiation and reflec(cid:173)
`tion from translucent internal surfaces are scanned, and pro-
`25 cessing of reflections may involve a triangulation system or
`holograms.
`In U.S. Pat. No. 5,864,640, a scanner is described having a
`multiple view detector responsive to a broad spectrum of
`visible light. The detector is operative to develop several
`30 images of a three dimensional object to be scanned. The
`images are taken from several relative angles with respect to
`the object. The images depict several surface portions of the
`object to be scanned. A digital processor, coupled to the
`detector, is responsive to the images and is operative to
`35 develop with a computational unit 3-D coordinate positions
`and related image information of the surface portions of the
`object, and provides 3-D surface information that is linked to
`color information without need to conformally map 2-D color
`data onto 3-D surface.
`Of general background interest, U.S. Pat. No. 4,836,674,
`U.S. Pat. No. 5,690,486, U.S. Pat. No. 6,525,819, EP
`0367647 and U.S. Pat. No. 5,766,006 describe devices for
`measuring the color of teeth.
`
`SUMMARY OF THE INVENTION
`
`In accordance with the present invention, a device and
`method for determining the surface topology and color of at
`least a portion of a three dimensional structure is provided.
`Preferred non-limiting embodiments of the invention are con(cid:173)
`cerned with the imaging of a three-dimensional topology of a
`teeth segment, optionally including such where one or more
`teeth are missing. This may allow the generation of data for
`subsequent use in design and manufacture of, for example,
`prosthesis of one or more teeth for incorporation into said
`teeth segment. Particular examples are the manufacture of
`crowns, bridges dental restorations or dental filings. The color
`and surface data is provided in a form that is highly manipu(cid:173)
`lable and useful in many applications including prosthesis
`60 color matching and orthodontics, among others.
`The determination of the 3D surface topology of a portion
`of a three-dimensional structure is preferably carried out
`using a confocal focusing method, comprising:
`(a) providing an array of incident light beams propagating in
`65 an optical path leading through a focusing optics and a prob(cid:173)
`ing face; the focusing optics defining one or more focal planes
`forward said probing face in a position changeable by said
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`US 8,363,228 B2
`
`3
`optics, each light beam having its focus on one of said one or
`more focal plane; the beams generating a plurality of illumi(cid:173)
`nated spots on the structure;
`(b) detecting intensity of returned light beams propagating
`from each of these spots along an optical path opposite to that
`of the incident light;
`(c) repeating steps (a) and (b) a plurality of times, each time
`changing position of the focal plane relative to the structure;
`and
`(d) for each of the illuminated spots, determining a spot(cid:173)
`specific position, being the position of the respective focal
`plane, yielding a maximum measured intensity of a respective
`returned light beam; and
`based on the determined spot-specific positions, generating
`data representative of the topology of said portion.
`The determination of the spot-specific positions in fact
`amounts to determination of the in-focus distance. The deter(cid:173)
`mination of the spot-specific position may be by measuring
`the intensity per se, or typically is performed by measuring
`the displacement (S) derivative of the intensity (I) curve (dI/
`dS) and determining the relative position in which this deriva(cid:173)
`tive function indicates a maximum intensity. The term "spot(cid:173)
`specific position (SSP)" will be used to denote the relative
`in-focus position regardless of the manner in which it is
`determined. It should be understood that the SSP is always a
`relative position as the absolute position depends on the posi(cid:173)
`tion of the sensing face. However the generationofthe surface
`topology does not require knowledge of the absolute position,
`as all dimensions in the cubic field of view are absolute.
`The SSP for each illuminated spot will be different for 30
`different spots. The position of each spot in an X-Y frame of
`reference is known and by knowing the relative positions of
`the focal plane needed in order to obtain maximum intensity
`(namely by determining the SSP), the Z or depth coordinate
`can be associated with each spot and thus by knowing the 35
`X-Y-Z coordinates of each spot the surface topology can be
`generated.
`In order to determine the Z coordinate (namely the SSP) of
`each illuminated spot the position of the focal plane may be
`scanned over the entire range of depth or Z component pos- 40
`sible for the measured surface portion. Alternatively the
`beams may have components, each of which has a different
`focal plane. Thus, by independent determination of SSP for
`the different light components, e.g. 2 or 3 with respective
`corresponding 2 or 3 focal planes, the position of the focal 45
`planes may be changed by the focusing optics to scan only
`part of the possible depth range, with all focal planes together
`covering the expected depth range. Alternatively, the deter(cid:173)
`mination of the SSP may involve a focal plane scan of only
`part of the potential depth range and for illuminated spots 50
`where a maximum illuminated intensity was not reached, the
`SSP is determined by extrapolation from the measured values
`or other mathematical signal processing methods. Thus, in
`each case, a Z-value is obtained for each point along an X-Y
`grid representing a plurality oflight beams. In this manner, a 55
`three-dimensional (3D) numerical entity E may be crated,
`comprising a plurality of coordinates (X, Y, Z) representative
`of the surface topology of the object being scanned.
`Alternatively, any other suitable method may be employed
`to obtain the 3D entity E.
`According to the present invention, a two dimensional (2D)
`color image of the 3D structure that is being scanned is also
`obtained, but typically within a short time interval with
`respect to the 3D scan. Further, the 2D color image is taken at
`substantially the same angle and orientation with respect to 65
`the structure as was the case when the 3D scan was taken.
`Accordingly, there is very little or no substantial distortion
`
`4
`between the X-Y plane of3D scan, and the plane of the image,
`i.e., both planes are substantially parallel, and moreover sub(cid:173)
`stantially the same portion of the structure should be com(cid:173)
`prised in both the 3D scan and the 2D image. This means that
`5 each X-Y point on the 2D image substantially corresponds to
`a similar point on the 3D scan having the same relative X-Y
`values. Accordingly, the same point of the structure being
`scanned has substantially the same X-Y coordinates in both
`the 2D image and the 3D scan, and thus the color value at each
`10 X, Y coordinate of the 2D color scan may be mapped directly
`to the spatial coordinates in the 3 D scan having the same X, Y
`coordinates, wherein to create a numerical entity I represent(cid:173)
`ing the color and surface topology of the structure being
`scanned.
`15 Where the X,Y coordinates of the color image do not
`precisely correspond to those of the 3D scan, for example as
`may arise where one CCD is for the 3D scanning, while
`another CCD is used for the 2D color image, suitable inter(cid:173)
`polation methods may be employed to map the color data to
`20 the 3D spartial data.
`To provide a more accurate mapping, it is possible to con(cid:173)
`struct a 2D image along the X-Y plane of the 3D model, and
`using procedures such as optical recognition, manipulate the
`color 2D image to best fit over this 3D image. This procedure
`25 may be used to correct for any slight misalignment between
`the 2D color scan and the 3D scan. Once the color 2D image
`has been suitably manipulated, the color values of the color
`2D image are mapped onto the adjusted X-Y coordinates of
`the 3D scan.
`Thus the present invention provides a relatively simple and
`effective way for mapping 2D color information onto a 3D
`surface model.
`The present invention thus provides a device and method
`for obtaining a numerical entity that represents the color and
`surface topology of an object. When applied particularly to
`the intraoral cavity, the device of the invention provides
`advantages over monochrome 3D scaners, including such
`scanners that are based on confocal focusing techniques. For
`example, the 2D color image capability on its own enables the
`dental practitioner to identify the area of interest within the
`oral cavity with a great degree of confidence in order to better
`aim the device for the 3D scanning. In other words, an
`improved viewfinder is automatically provided. Further, ren(cid:173)
`dition of a full color 3D image of the target area can help the
`practitioner to decide on the spot whether the scan is suffi(cid:173)
`ciently good, or whether there are still parts of the teeth or soft
`tissues that should have been included, and thus help the
`practitioner to deciode whether or not to acquire another 3D
`color entity.
`Creation of a color 3D entity that is manipulable by a
`computer is extremely useful in enabling the practictioner to
`obtain data from such an entity that is useful for procedures
`carried out in the dental cavity.
`Thus, according to the present invention, a device is pro(cid:173)
`vided for determining the surface topology and associated
`color of at least a portion of a three dimensional structure,
`comprising:
`scanning means adapted for providing depth data of said
`portion corresponding to a two-dimensional reference array
`60 substantially orthogonal to a depth direction;
`imaging means adapted for providing two-dimensional
`color image data of said portion associated with said refer(cid:173)
`ence array;
`wherein the device is adapted for maintaining a spatial
`disposition with respect to said portion that is substantially
`fixed during operation of said scanning means and said imag(cid:173)
`ing means. In other words, operation of the scanning means
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`US 8,363,228 B2
`
`15
`
`5
`and the imaging means is substantially or effectively simul(cid:173)
`taneous in practical terms, and thus the actual time interval
`that may exist between operation of the two means is so short
`that the amplitude of any mechanical vibration of the device
`or movement of the oral cavity will be so small as can be 5
`ignored.
`The device is adapted for providing a time interval between
`acquisition of said depth data and acquisition of said color
`image data such that substantially no significant relative
`movement between said device and said portion occurs. The 10
`time interval may be between about 0 seconds to about 100
`milliseconds, for example 5, 10, 20, 30, 40, 50, 60, 70; 80, 90
`or 100 milliseconds, and preferably between about Oto about
`50 milliseconds, and more preferably between about 0 and 20
`milliseconds.
`The device further comprise processing means for associ(cid:173)
`ating said color data with said depth data for corresponding
`data points of said reference array. In described embodi(cid:173)
`ments, the operation of said scanning means is based on
`confocal imaging techniques. Such scanning means may 20
`comprise:
`a probing member with a sensing face;
`first illumination means for providing a first array of inci(cid:173)
`dent light beams transmitted towards the structure along an
`optical path through said probing unit to generate illuminated
`spots on said portion along said depth direction, wherein said
`first array is defined within said reference array;
`a light focusing optics defining one or more focal planes
`forward said probing face at a position changeable by said
`optics, each light beam having its focus on one of said one or
`more focal plane;
`a translation mechanism for displacing said focal plane
`relative to the structure along an axis defined by the propaga(cid:173)
`tion of the incident light beams;
`a first detector having an array of sensing elements for
`measuring intensity of each of a plurality of light beams
`returning from said spots propagating through an optical path
`opposite to that of the incident light beams;
`a processor coupled to said detector for determining for
`each light beam a spot-specific position, being the position of
`the respective focal plane of said one or more focal planes
`yielding maximum measured intensity of the returned light
`beam, and based on the determined spot-specific positions,
`generating data representative of the topology of said portion.
`The first array is arranged to provide depth data at a plu(cid:173)
`rality of predetermined spatial coordinates substantially cor(cid:173)
`responding to the spatial disposition of said incident light
`beams.
`The first illumination means comprises a source emitting a
`parent light beam and a beam splitter for splitting the parent
`beam into said array of incident light beams. The first illumi(cid:173)
`nation means may comprise a grating or microlens array.
`The device may comprise a polarizer for polarizing said
`incident light beams are polarized. Further, the device may
`comprise a polarization filter for filtering out from the
`returned light beams light components having the polariza(cid:173)
`tion of the incident light beams.
`The illumination unit may comprise at least two light
`sources and each of said incident beams is composed of light
`components from the at least two light sources. The at least 60
`two light sources emit each a light component of different
`wavelength. The light directing optics defines a different
`focal plane for each light component and the detector inde(cid:173)
`pendently detects intensity of each light components.
`The at least two light sources may be located so as to define 65
`optical paths of different lengths for the incident light beams
`emitted by each of the at least two light sources.
`
`6
`Typically, the focusing optics operates in a telecentric con(cid:173)
`focal mode.
`Optionally, the light directing optics comprises optical
`fibers.
`Typically, the sensing elements are an array of charge
`coupled devices (CCD). The detector unit may comprise a
`pinhole array, each pinhole corresponding to one of the CCDs
`in the CCD array.
`The operation of said imaging means may be based on:
`illuminating said portion with three differently-colored
`illumination radiations, the said illuminations being combin(cid:173)
`able to provide white light,
`capturing a monochromatic image of said portion corre(cid:173)
`sponding to each said illuminating radiation, and
`combining the monochromatic images to create a full color
`image,
`wherein each said illuminating radiation is provided in the
`form of a second array of incident light beams transmitted
`towards the portion along an optical path through said prob(cid:173)
`ing unit to generate illuminated spots on said portion along
`said depth direction, wherein said second array is defined
`within said reference frame.
`The second array is arranged to provide color data at a
`plurality of spatial coordinates substantially corresponding to
`25 the spatial coordinates of said first array. The device may
`comprise color illumination means adapted for providing
`three second illuminating radiations, each of a different color.
`The color illumination means comprises second illumination
`means for providing said three second illuminating radia-
`30 tions, each of a different color. Alternatively, the color illu(cid:173)
`mination means comprises second illumination means for
`providing two said second illuminating radiations, and
`wherein said first illumination means provides another said
`second illuminating radiation each said second illuminating
`35 radiation being of a different color. Optionally, each one of
`said second illumination radiations is a different one of red,
`green or blue light. The second illumination means may com(cid:173)
`prise radiation transmission elements that are configured to
`be located out of the path of said light beams or said returned
`40 light beam at least within said light focusing optics. The
`probing member may be made from a light transmissive
`material having an upstream optical interface with said light
`focusing optics and a reflective face for reflecting light
`between said optical interface and said sensing face. The
`45 second illumination means may be optically coupled to said
`optical interface for selectively transmitting illuminating
`radiations in at least two colors to said portion via said sensing
`face. The color illumination means may comprise second
`illumination means for providing two said second illuminat-
`50 ing radiations, and wherein said first illumination means pro(cid:173)
`vides another said second illuminating radiation each said
`second illuminating radiation being of a different color. The
`probing member may comprise a removable sheath having an
`inner surface substantially complementary to an outer surface
`55 of said probing member, and having a window in registry with
`said sensing face, wherein said sheath is made from a
`waveguiding material and is adapted to transmit said light
`from said second illuminating means from an upstream face
`thereof to a downstream face associated with said window.
`The second illumination means may be optically coupled to
`said upstream face for selectively transmitting said second
`illuminating radiations in at least two colors to said portion
`via said downstream face. Preferably, the sheath is disposable
`after use with a patient.
`In another embodiment, the reflective face comprises a
`dichroic coating, having relatively high reflectivity and low
`optical transmission properties for a said second illuminating
`
`3SHAPE 1016 3Shape v Align IPR2021-01383
`
`

`

`US 8,363,228 B2
`
`5
`
`7
`radiation provided by said first illumination means, and rela(cid:173)
`tively low reflectivity and high optical transmission proper(cid:173)
`ties for the two said second illuminating radiations provided
`by said second illumination means.
`The second illumination means may be adapted for pro-
`viding second illuminating radiations within said light focus(cid:173)
`ing optics. In particular, the second illumination means may
`be adapted for providing second illuminating radiations at an
`aperture stop plane of said light focusing optics. The second
`illumination means may be provided on a bracket having an
`aperture configured to allow said light beams and said return(cid:173)
`ing light beams to pass therethrough without being optically
`affected by said bracket.
`Optionally, the device further comprises:
`a first polarizing element located just downstream of said
`illumination means so as to polarize the light emitted
`therefrom;
`a second polarizing element located just upstream of said
`first detector, wherein said second polarizing element is
`crossed with respect to the first polarizing element; and
`a quarter waveplate at the downstream end of said device.
`Further optionally the second illumination means are
`adapted for selective movement in the depth direction.
`The device may comprise a mirror inclined to the optical
`axis of said light focusing optics and having, an aperture
`configured to allow said light beams and said returning light
`beams to pass therethrough without being optically affected
`by said mirror, and wherein said second illumination means
`comprises at least one white illumination source optically
`coupled with suitable color filters, said filters selectively pro(cid:173)
`viding illumination radiation in each color in cooperation
`with said white