1111111111111111 IIIIII IIIII 11111 1111111111 11111 1111111111 11111 1111111111 1111111111 11111111
`US 20090087050Al
`
`c19) United States
`c12) Patent Application Publication
`Gandyra
`
`c10) Pub. No.: US 2009/0087050 Al
`Apr. 2, 2009
`(43) Pub. Date:
`
`(54) DEVICE FOR DETERMINING THE 3D
`COORDINATES OF AN OBJECT, IN
`PARTICULAR OF A TOOTH
`
`(76)
`
`Inventor:
`
`Michael Gandyra, Rosenheim
`(DE)
`
`Correspondence Address:
`DILWORTH & BARRESE, LLP
`333 EARLE OVINGTON BLVD., SUITE 702
`UNIONDALE, NY 11553 (US)
`
`(21) Appl. No.:
`
`12/228,917
`
`(22) Filed:
`
`Aug.18,2008
`
`(30)
`
`Foreign Application Priority Data
`
`Aug. 16, 2007
`(DE) ......................... 102007038721.2
`Dec. 14, 2007
`(DE) ......................... 102007060263.6
`Publication Classification
`
`(51)
`
`Int. Cl.
`G06K 9/00
`(2006.01)
`G06K 9/20
`(2006.01)
`(52) U.S. Cl. ......................................... 382/128; 382/312
`ABSTRACT
`(57)
`
`A scanner is used for scanning an object (3, 4, 5), in particular
`a tooth or a plurality of teeth or a dental cast. The scanner (1)
`comprises a projector (2) for projecting a pattern (7) onto the
`object (3, 4, 5) and a camera which comprises a recording
`optics and an image sensor (18). To improve such scanner, the
`recording optics comprises a first imaging optics (9) and a
`second imaging optics (10).
`
`1
`r/ "f t
`
`' 8 2
`
`3
`
`s
`
`Align EX1007
`Align v. 3Shape
`IPR2022-00145
`
`

`

`Patent Application Publication
`
`Apr. 2, 2009 Sheet 1 of 2
`
`US 2009/0087050 Al
`
`-
`
`··Al
`
`1
`f 1/C t
`
`3
`
`s
`
`

`

`Patent Application Publication
`
`Apr. 2, 2009 Sheet 2 of 2
`
`US 2009/0087050 Al
`
`~ · lr.:ir
`
`20
`
`26
`I
`
`32
`
`

`

`US 2009/0087050 Al
`
`Apr. 2, 2009
`
`1
`
`DEVICE FOR DETERMINING THE 3D
`COORDINATES OF AN OBJECT, IN
`PARTICULAR OF A TOOTH
`
`[0001] This invention relates to a scanner for scanning an
`object, in particular a tooth or a plurality of teeth or a dental
`cast, and a device for determining the 3D coordinates of an
`object, in particularofa tooth or a plurality of teeth or a dental
`cast.
`[0002] Devices and methods for determining the 3D coor(cid:173)
`dinates of an object are known already. EP 299 490 B2
`describes a method for making a dental prosthesis, in which
`contour lines on the ground tooth and its surroundings are
`produced, the lines are detected with an optoelectronic
`means, in particular a video camera, the detected values are
`entered in a computer, and the three-dimensional structure of
`the tooth and the dental prosthesis is calculated. By means of
`the structure thus calculated, the dental prosthesis can be
`fabricated.
`[0003] However, the detection of the 3D coordinates of
`objects located in regions which are hard to reach, i.e. in
`particular of teeth in the oral cavity of a patient, involves
`certain difficulties.
`[0004] Proceeding therefrom, it is the object underlying the
`invention to propose an improved scanner for scanning an
`object, in particular a tooth, and an improved device for
`determining the 3D coordinates of an object, in particular of
`a tooth.
`[0005]
`In accordance with the invention, this object is
`solved by a scanner with the features of claim 1. The scanner
`is used for scanning an object, in particular one or more teeth,
`wherein one or more or all teeth can be prepared. The term of
`preparing on the one hand comprises the dental preparation,
`i.e. for instance grinding the stump of a tooth, and on the other
`hand the preparation which is necessary for scanning a tooth
`with an optical measurement technique, for instance spraying
`white spray onto the region to be scanned.
`[0006] The scanner comprises a projector for projecting a
`pattern onto the object and a camera which comprises a
`recording optics and an image sensor, in particular a CCD
`sensor or a CMOS sensor. In accordance with the invention,
`the recording optics comprises a first imaging optics and a
`second imaging optics. In this way, a stereoscopic pair is
`produced by the scanner. The imaging optics can be located at
`a distance from each other. The optical axes of the imaging
`optics can be arranged at an angle with respect to each other.
`Preferably, they are arranged at such an angle with respect to
`each other that they are directed towards a corresponding
`region of the object or tooth. As a result, the region of the
`object or tooth thus is observed with two cameras.
`[0007] The scanner of the invention in particular can con(cid:173)
`stitute a miniaturized scanner. It is particularly useful for
`scanning teeth in the mouth of a patient. It is, however, also
`particularly useful for other applications in which the objects
`to be scanned are hard to reach. In particular, the scanner of
`the invention can be used for intra-ear scanning, for endo(cid:173)
`scopic digitization and/or in hard-to-reach cavities and/or
`channels of machines and/or apparatuses.
`[0008] Advantageous developments are described in the
`sub-claims.
`[0009] Preferably, the recording optics comprises a beam
`splitter. The images from the imaging optics can be supplied
`to the beam splitter. The beam splitter can project these
`
`images onto the image sensor. In doing so, the images from
`the imaging optics can each be projected onto different
`regions of the image sensor.
`[0010]
`It is possible to provide further imaging optics. In
`particular, two more imaging optics can be provided, so that
`a total of four imaging optics are present. Each image can be
`projected onto a separate quarter of the image sensor.
`[0011]
`It is furthermore possible to provide further image
`sensors. In particular, one image sensor can be present for
`each imaging optics.
`[0012] By taking a picture, a particular region of the object
`or tooth can be covered. To be able to cover the entire object
`or the entire tooth or several or all teeth and possibly also the
`surroundings thereof, a plurality of pictures can be taken
`sequentially. The individual shots can be combined to a total
`object representation.
`[0013] For this purpose, the scanner is moved around the
`object, in order to cover several or all regions of the object.
`Since the scanner makes each individual shot in its coordinate
`system, the movement of the scanner advantageously should
`be detected, so as to be able to combine the individual shots as
`detailed and accurate as possible. This procedure is referred
`to as "registering" or "matching".
`[0014]
`In accordance with an advantageous development,
`the scanner includes one or more sensors for detecting the
`location and orientation of the scanner. In this way, a tracking
`system can be formed for the scanner.
`[0015] For detecting the location and orientation of the
`scanner, 6D data preferably are supplied by the sensor( s ). The
`6D data consist of three translational data and three rotational
`data. As a result, both the translation and the rotation of the
`scanner are detected completely. In this way, a tracking sys(cid:173)
`tem is formed for the scanner.
`[0016]
`In accordance with another advantageous develop(cid:173)
`ment, the scanner includes one or more acceleration sensors.
`The scanner movement can be detected by the acceleration
`sensors, and the location of the scanner can be determined
`thereby. It is possible to detect the location of the scanner by
`the acceleration sensors.
`[0017]
`In accordance with another advantageous develop(cid:173)
`ment, the scanner includes one or more gyrometers. The
`rotations, i.e. the orientation, of the scanner can be detected
`by the gyrometers.
`[0018] The 6D information corresponds to the six degrees
`of freedom, which must be defined to unambiguously define
`a body, namely the scanner, in space in terms of its location
`(position) and orientation (rotation). The 6D information of
`the scanner is defined by three translatory components and
`three rotary components.
`[0019] The temporal integration of an acceleration sensor
`supplies a velocity. The further temporal integration thereof
`supplies a path component. Proceeding therefrom, three
`acceleration sensors can be used. Sensors which determine all
`three translatory components at the same time are, however,
`also available.
`[0020] The same is true for gyrometers, which detect the
`rotary accelerations, whose temporal integration two times
`over supplies an angle of rotation. It is possible to use three
`gyrometers, in order to obtain the three spatial angles of
`rotation of the scanner. However, there are also gyrometers
`which supply all three spatial angles of rotation.
`[0021] Furthermore, there are sensors which supply all
`three translations and all three rotations.
`
`

`

`US 2009/0087050 Al
`
`Apr. 2, 2009
`
`2
`
`[0022]
`Instead or in addition, markers for a tracking system
`can be provided on the scanner. The markers for the tracking
`system can be one or more active markers. In particular,
`infrared markers can be used, which are triggered time-se(cid:173)
`quentially. However, the markers for the tracking system can
`also be one or more passive markers. In particular, the passive
`markers can be coded and/or non-coded, reflecting and/or
`non-reflecting markers. The markers can have different pat(cid:173)
`terns.
`[0023] The markers present on the scanner initially are
`surveyed. Hence, the position of the markers on the scanner is
`known. The markers then are tracked by a tracking camera.
`By means of the tracking system, the location and rotary
`position of the scanner can be covered in the different shots.
`In this way, possibilities for combining the different indi(cid:173)
`vidual shots of the scanner are improved or created.
`[0024]
`In a device for determining the 3D coordinates of an
`object, in particular of a tooth, the object underlying the
`invention is solved by the features of claim 12. The device
`comprises an inventive scanner for scanning the object or
`tooth and an evaluation means, in particular a computer, espe(cid:173)
`cially a PC including the associated software, for determining
`the 3D coordinates of the object from the pictures taken by the
`scanner.
`[0025] Preferably, the device in accordance with the inven(cid:173)
`tion comprises a tracking system for determining the location
`and orientation of the scanner. The tracking system can be
`formed in that the scanner includes one or more sensors for
`detecting the location and orientation of the scanner. In par(cid:173)
`ticular, the tracking system can be formed in that the scanner
`includes acceleration sensors and/or gyrometers. It is pos(cid:173)
`sible that the data of the sensor(s) for detecting the location
`and orientation of the scanner and/or of the acceleration sen(cid:173)
`sors and/or of the gyrometer(s), in particular the 6D data
`thereof, are calculated back onto the location and rotary posi(cid:173)
`tion of the scanner by temporal integration. Instead or in
`addition, however, another tracking system can also be used,
`in particular an infrared tracking system.
`[0026] Embodiments of the invention will be explained in
`detail below with reference to the attached drawing, in which:
`[0027] FIG.1 shows a scanner for scanning a group of teeth
`in a schematic view,
`[0028] FIG. 2 shows the scanner of FIG. 1, which addition(cid:173)
`ally includes acceleration sensors, and
`[0029] FIG. 3 shows the scanner of FIG. 1 with infrared
`markers for an infrared tracking system.
`[0030] FIG. 1 shows a scanner 1 which includes a projector
`2 for projecting a pattern onto the teeth 3, 4, 5. The tooth 4 is
`ground, the adjacent teeth 3, 5 are not ground and form the
`surroundings of the ground tooth 4. Theprojector2 comprises
`a light source 6, a pattern transparency 7 and a projection
`optics 8. The light source 6 can be a light bulb or an LED.
`Instead of the pattern transparency 7 a transmitted-light LCD
`can also be used, which can be activated for forming a pattern.
`The pattern can, however, also be projected by means of a
`DMD or LCOS method.
`In the scanner 1, a first imaging optics 9 and a
`[0031]
`second imaging optics 10 furthermore are provided, which
`are spaced from each other and whose optical axes 11, 12
`form an angle with respect to each other. The distances of the
`imaging optics 9, 10 and the directions of the optical axes 11,
`12 are chosen such that they are directed towards a common
`region of the ground tooth 4.
`
`[0032] The scanner 1 furthermore comprises a beam split(cid:173)
`ter 13. In the optical path from the imaging optics 9 to the
`beam splitter 13 a first mirror 14 is provided, which reflects
`the light which comes from the first tooth 4 and is projected by
`the first imaging optics 9 onto the first mirror 15 of the beam
`splitter 13. Correspondingly, a second mirror 16 is provided
`in the optical path of the second imaging optics 10, which
`reflects the light coming from the ground tooth 4 and pro(cid:173)
`jected by the second imaging optics 10 to the second mirror
`surface 17 of the beam splitter 13. From the mirror surfaces
`15, 17 of the beam splitter 13, the light is reflected to a CCD
`sensor 18. The image from the first imaging optics 9 is pro(cid:173)
`jected onto the left half of the CCD sensor 18, the image from
`the second imaging optics 10 is projected onto the right half of
`the CCD sensor 18. The images from the imaging optics 9, 10
`are supplied to the beam splitter 13, which projects the same
`onto different regions of the CCD sensor 18. In this way, a
`stereoscopic pair is produced on the CCD sensor 18, which
`can be evaluated by a software present on a PC. The CCD
`sensor 18 can constitute a divided CCD chip.
`[0033] The scanner shown in FIG. 2 corresponds to the
`scanner shown in FIG. 1, so that the components explained
`above will not be described again. The scanner as shown in
`FIG. 2 additionally includes sensors for detecting the location
`and orientation of the scanner 3. These sensors are three
`acceleration sensors 19, 20, 21, which each supply data in x-,
`y- and z-direction, from which 6D data can be derived. These
`acceleration data can be integrated temporally, so that the
`position in space and the rotary position of the scanner can be
`determined therefrom. The acceleration sensors 19, 20 are
`disposed in the handle 27 of the scanner 1.
`[0034] FIG. 3 shows the scanner 1 of FIG. 1 or 2 with
`cantilevers 22, 23, 24, at whose ends infrared markers 27, 28,
`29; 30, 31, 32; 33, in particular infrared diodes, are provided,
`which supply data for an infrared tracking camera 25. The
`infrared markers 27-33 emit infrared beams, which are
`received by the infrared tracking camera. The tracking system
`26, which consists of the markers 27-33 and the camera 25,
`can determine therefrom the location and orientation of the
`hand-guided scanner 1.
`[0035] The method can be performed such that the infrared
`diodes 27-33 light up one after the other. This is time-sequen(cid:173)
`tially detected by the tracking camera 25 (which can be a
`3-line camera), from which the distance can be determined.
`The interconnection of the infrared markers 27-33 is unam(cid:173)
`biguously defined by its calibration. As a result, location and
`orientation in space can be calculated. The more markers can
`be evaluated, the more precisely can location and orientation
`be determined (some markers for instance are concealed dur(cid:173)
`ing the measurement).
`[0036] By means of the invention, a meteorological method
`can be performed, which is based on the evaluation of stereo(cid:173)
`scopic pairs. In the method, an object is observed with two
`cameras. With suitable algorithms, the same features can be
`found and correlated in both images. In calibrated systems,
`i.e. when the location and orientation of both cameras are
`known, a distance value and hence a 3D coordinate-in the
`coordinate system of the scanner-can be calculated for each
`object point, which can be observed in both cameras. The
`recording optics is equipped with a beam splitter. In this way,
`the scanner can be miniaturized. The object to be scanned, in
`particular a dental preparation or an object in a hard-to-reach
`region of a machine or apparatus, can be observed from two
`directions. The same scene is projected from two directions
`
`

`

`US 2009/0087050 Al
`
`Apr. 2, 2009
`
`3
`
`onto only one image sensor or CCD chip. An extension to
`more than two directions, in particular to four directions, is
`possible. All images can be projected onto one CCD chip. It
`is, however, also possible to use a plurality of image sensors.
`[0037] To find the same features of the object in the differ(cid:173)
`ent images, a pattern is projected onto the object. By means of
`the projected pattern, the associated image regions can be
`found in both images.
`[0038] The pattern to be projected can be a stochastic or
`ordered pattern. It can consist oflines or cross gratings. It can
`be a time-constant or time-variable or time-sequential pat(cid:173)
`tern. The pattern can be any graphical pattern ( dots, lines,
`grating, etc.). The pattern can be a grey-scale pattern or a
`color pattern. The pattern can be projected with transmitted
`light, for instance as a chrome mask (transparency) or as an
`LED projection. The pattern can, however, also be projected
`by reflection, for instance as an LCOS or DLP projection.
`[0039] One image recording each can produce a data
`recording in the form ofa 3D aggregate of points. However,
`the same can represent only part of the entire representation of
`the real object. For this reason, a plurality of individual shots
`of the object can be made successively, and these individual
`shots can be combined to form one total object representation.
`To achieve this, the scanner can be moved around the object,
`in order to cover all regions of the object. Instead or in addi(cid:173)
`tion, the scanner can be tracked, e.g. by an externally mounted
`tracking system.
`[0040] The present invention creates a method and a device
`for dynamically covering the surfaces of objects, in particular
`for the dynamic intraoral coverage of the surfaces of dental
`preparations. A pattern is projected onto the surface, which is
`observed from two or more directions for generating the
`digital 3D data and is included in a corresponding number of
`2D images. There can be effected a photogrammetric evalu(cid:173)
`ation of the 2D images, and the time-sequential individual 3D
`images calculated therewith can be combined by means of a
`tracking system by determining the location and orientation
`of the scanner in space.
`[0041] The photogrammetric evaluation can be an evalua(cid:173)
`tion of stereoscopic pairs, which represent the object to be
`surveyed from two different directions. It can, however, also
`be an evaluation of several images from several views, in
`particular an evaluation of four images from four views. The
`stereoscopic pair can be an individual image, which consists
`of two image halves which were taken from two viewing
`directions. Four individual shots can be combined in four
`"image quarters".
`[0042] The tracking system can be an optical or an inter(cid:173)
`ferometric tracking system. It is, however, also possible to
`realize the tracking system by sensors for detecting the loca(cid:173)
`tion and orientation of the scanner, which are attached to the
`scanner, and/or by acceleration sensors and/or gyrometers, by
`means of which the velocity and position of the scanner can
`be calculated back via the temporal integration. The accelera(cid:173)
`tion sensors can supply three translational data. The gyrom(cid:173)
`eters can supply three rotational data, i.e. data on the orien(cid:173)
`tation of the scanner. It is particularly advantageous to
`provide both acceleration sensors and gyrometers, so as to
`obtain 6D data.
`[0043] By means of the invention it is possible to determine
`the 3D coordinates of an object. This can be accomplished in
`that the 3D coordinates are calculated by correlating identical
`features in several, in particular two, images. By this method,
`associated identical image points can be found in both images
`
`of a stereoscopic pair, which were taken from different direc(cid:173)
`tions. By calibrating the imaging optics and by triangulation,
`the 3D coordinates of the object points thus can be calculated.
`To be able to provide an unambiguous allocation of the same
`features in the images, a pattern is projected. The projected
`pattern is used for this process referred to as feature recogni(cid:173)
`tion. In this method, the projector is not part of the calibration;
`it is independent of the stereo camera system.
`[0044]
`It is, however, also possible to perform other meth(cid:173)
`ods for determining the 3D coordinates of the object. In
`particular, methods of the "single-image measurement tech(cid:173)
`nique" can be performed, i.e. methods with which the 3D
`coordinates of the object can be calculated from a single
`image recording. To be able to perform this method and to
`avoid a plurality of sequential image recordings of the same
`region, the picture taken must contain all data necessary for
`calculating the 3D coordinates. To ensure this, a pattern usu(cid:173)
`ally is projected onto the object. To be able to project this
`pattern, different properties of the light can be utilized. The
`pattern can, for instance, be a cross grating with different
`colors, as it is described in DE 102 12 364Al. In this method,
`the projector must be calibrated together with the camera. The
`method can be performed such that with the image from an
`imaging optics and with the projected pattern the 3D coordi(cid:173)
`nates are calculated and that these 3D coordinates are corre(cid:173)
`lated and optimized with the coordinates calculated from an
`image of the other imaging optics and from the projected
`pattern.
`
`1. A scanner for scanning an object (3,4,5), in particular a
`tooth or a plurality of teeth or a dental cast, with a projector (2)
`for projecting a pattern (7) onto the object (3, 4,5) and with a
`camera which comprises a recording optics and an image
`sensor (18), wherein the recording optics comprises a first
`imaging optics (9) and a second imaging optics (10).
`2. The scanner according to claim 1, wherein the recording
`optics comprises a beam splitter (13).
`3. The scanner according to claim 1, comprising further
`imaging optics.
`4. The scanner according to claim 1, comprising further
`image sensors.
`5. The scanner according to claim 1, wherein the scanner
`(1) includes one or more sensors for detecting the location and
`orientation of the scanner (1).
`6. The scanner according to claim 5, wherein the one or
`more sensors supply 6D data.
`7. The scanner according to claim 1, wherein the scanner
`(1) includes one or more acceleration sensors (19, 20, 21).
`8. The scanner according to claim 1, wherein the scanner
`(1) includes one or more gyrometers.
`9. The scanner according to claim 1, wherein markers (27,
`28, 29; 30, 31, 32; 33) for a tracking system (26) are provided
`on the scanner (1).
`10. The scanner according to claim 9, wherein the markers
`are active markers.
`11. The scanner according to claim 9, wherein the markers
`are passive markers.
`12. A device for determining the 3D coordinates of an
`object (3, 4, 5), in particular of a tooth or a plurality of teeth or
`a dental cast, with a scanner (1) for scanning the object (3, 4,
`5) and evaluation means for determining the 3D coordinates
`of the object (3, 4, 5) from the images taken by the scanner(l),
`comprising a scanner (1) according to claim 1.
`
`

`

`US 2009/0087050 Al
`
`Apr. 2, 2009
`
`4
`
`13. The device according to claim 12, comprising a track(cid:173)
`ing system (26) for determining the location and orientation
`of the scanner (1).
`14. The scanner according to claim 2, comprising further
`imaging optics.
`15. The scanner according to claim 10, wherein the mark(cid:173)
`ers are passive markers.
`16. The scanner according to claim 2, comprising further
`image sensors.
`17. The scanner according to claim 3, comprising further
`image sensors.
`
`18. The scanner according to claim 14, comprising further
`image sensors.
`19. The scanner according to claim 18, wherein the scanner
`(1) includes one or more sensors for detecting the location and
`orientation of the scanner (1).
`20. The scanner according to claim 17, wherein the scanner
`(1) includes one or more sensors for detecting the location and
`orientation of the scanner (1).
`20. (canceled)
`
`* * * * *
`
`