I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US009336336B2
`
`(12) United States Patent
`Deichmann et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 9,336,336 B2
`May 10, 2016
`
`(54) 2DIMAGEARRANGEMENT
`
`(75)
`
`Inventors: Nikolaj Deichmann, Copenhagen 0
`(DK); Tais Clausen, Klagshamn (SE);
`Rune Fisker, Virum (DK); Ilenrik
`()jelund, Lyngby (DK)
`
`(73) Assignee: 3SHAPE AJS, Kobenhavn K (DK)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 722 days.
`
`(21) Appl. No.:
`
`13/807,443
`
`(22) PCT Filed:
`
`Jun. 29, 2011
`
`(86) PCT No.:
`
`PCT/DK2011/050246
`
`§ 371 (c)(l),
`(2), ( 4) Date: Mar. 1, 2013
`
`(87) PCT Pub. No.: W02012/000511
`
`PCT Pub. Date: Jan. 5, 2012
`
`(65)
`
`Prior Publication Data
`US 2013/0218530 A 1
`
`Aug. 22, 2013
`
`Related U.S. Application Data
`
`(60) Provisional application No. 61/359,454, filed on Jun.
`29, 2010, provisional application No. 611454,200,
`filed on Mar. 18, 2011.
`
`(30)
`
`Foreign Application Priority Data
`
`Jun. 29, 2010
`Mar. 18, 2011
`
`(DK) ................................. 2010 00568
`(DK) ................................. 2011 00191
`
`(51)
`
`Int. Cl.
`G06F 17150
`A61C 13100
`
`(2006.01)
`(2006.01)
`
`(Continued)
`
`(52) U.S. Cl.
`CPC .............. <;06F 17150 (2013.01 ); A61C 910046
`(2013.01); A6JC 1310004 (2013.01); A61B
`110005 (2013.01)
`
`(58) Field of Classification Search
`CPC ..................................................... A6 l B 1/0005
`USPC ............................................................ 703/13
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`6,068,482 A
`6,261,248 Bl
`
`512000 Snow
`712001 Takaishi et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`
`8/2001
`1124487 Al
`5/2007
`l 124 487 Bl
`(Continued)
`OTHER PUBLICATIONS
`An English Translation of the Office Action (Notice of Reasons for
`Rejection) issued on May 19, 2015, by the Japanese Patent Office in
`corresponding Japanese Patent Application No. 2013-517019. (2
`pages).
`
`(Continued)
`
`Primary Axaminer - Hugh Jones
`(74) Attorney, Agent, or Firm -Buchanan Ingersoll &
`Rooney PC
`ABSTRACT
`(57)
`Disclosed is a method of designing a dental restoration for a
`patient, wherein the method includes providing one or more
`2D images, where at least one 2D image includes at least one
`facial foature; providing a 3D virtual model of at least part of
`the patient's oral cavity; arranging at least one of the one or
`more 2D images relative lo the 31) virtual model in a virtual
`3D space such that the 2D image and the 3D virtual model are
`aligned when viewed from a viewpoint, whereby the 3D
`virtual model and the 2D image are both visualized in the 3D
`space; and modeling a restoration on the 3D virtual model,
`where the restoration is designed to fit the facial feanire of the
`al least one 2D image.
`30 Claims, 27 Drawing Sheets
`
`301
`
`x _..
`
`0001
`
`exocad GmbH, et. al.
`Exhibit 1001
`
`

`

`US 9,336,336 B2
`Page2
`
`Int. Cl.
`A61C 9100
`A61B1100
`
`(51)
`
`(56)
`
`(2006.01)
`(2006.01)
`
`References Cited
`
`WO
`WO
`WO
`WO
`WO
`WO
`
`00/25677 Al
`2008/128700 Al
`WO 2008/128700 Al
`20101008435 Al
`WO 20101008435 Al
`WO 2010/031404
`
`5/2000
`10/2008
`10/2008
`1/2010
`1/2010
`3/2010
`
`U.S. PATENT DOCUMENTS
`
`OTHER PUBLICATIONS
`
`2003/0163291 Al
`2003/0169913 Al
`2006/0127836 Al
`2010/0049351 Al
`2010/0076581 Al
`2010/0145898 Al
`
`812003 Jordan et al.
`9/2003__Kopehnan et~
`6/2006 Wen
`2/2010 Monloneyer
`3/2010 Violante et al.
`6/2010 Malfliet et al.
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`JP
`
`2002-528215 A
`2010-503437 A
`2010-524529 A
`
`9/2002
`2/2010
`7/2010
`
`International Search Report (PCT/ISA/210) issued on Sep. 7, 2011,
`by the Denmark Patent Office as the International Searching Author(cid:173)
`ity for International Application No. PCT/DK201 l /050246, 3 pp.
`Search Report issued by the Danish Paten! and Trademark Office on
`Jan. 28, 20 II, in the corresponding Danish Patent Application No. PA
`2010 00568. (4 pages).
`Notification ofTransmittal of the International Search Report (Forms
`PCTllSA/220 and PCT/ISA/210) and the Written Opinion of the
`International Searching Authority (Form PCTIISA/237) dated Jul. 9,
`2011 , issued in corresponding International Application No. PCT/
`DK201 l/050246. (I I pages).
`
`0002
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 1 of27
`
`US 9,336,336 B2
`
`101
`
`102
`
`103
`
`104
`
`Fig. 1
`
`0003
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 2 of27
`
`US 9,336,336 B2
`
`Fig. 2A
`
`0004
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 3of27
`
`US 9,336,336 B2
`
`202~
`
`Fig. 2B
`
`0005
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 4of27
`
`US 9,336,336 B2
`
`Fig. 3A
`
`0006
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 5 of27
`
`US 9,336,336 B2
`
`Fig. 3B
`
`'
`
`0007
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 6 of27
`
`US 9,336,336 B2
`
`x
`
`301
`
`302
`
`Fig. 3C
`
`0008
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 7of27
`
`US 9,336,336 B2
`
`Fig.4A
`
`Fig.48
`
`.iift"'rc\-'IE~
`
`, ,)!,;,. I" ~· ...,q'
`
`~r-_,, ~ -·:o~ --··
`:!I
`"" 1'-·
`
`~
`
`0009
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 8 of27
`
`US 9,336,336 B2
`
`I
`
`"),i
`
`:·
`.
`f' 1
`i
`~1
`~'i
`i)
`~i
`k l
`wi
`"\?i
`
`'V''' ;1
`
`P i
`~ ·
`l
`&:
`
`~ ·
`
`Fig.4C
`
`0010
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 9 of27
`
`US 9,336,336 B2
`
`509
`
`510
`
`512
`
`506
`
`505
`
`Fig. 5A
`
`0011
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 10of27
`
`US 9,336,336 B2
`
`Fig. 58
`
`509
`
`510
`
`511
`
`512
`
`506
`
`513
`
`509
`
`\__514
`
`Fig. SC
`
`0012
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 11 of 27
`
`US 9,336,336 B2
`
`Fig. 50
`
`Fig. 5E
`
`0013
`
`

`

`U.S. Patent
`
`May to, 2016
`
`Sheet 12 of 27
`
`US 9,336,336 B2
`
`Fig. 5F
`
`Fig. 5G
`
`0014
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet 13 of 27
`Sheet 13 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`602
`
`V
`
`‘
`
`1‘:
`
`:3"
`
`: far. for. y‘x-vvu y..-
`i
`A W‘
`"
`i
`1%"!33%
`3"
`«Awwax-.uuwtw-A.v
`‘7
`3193'.
` 2»
`fie»(GWEN/v.~.-
`
`%
`
`figflmfififinfiafldéfififé‘
`
`
`
`-,4.“-2-a:ww.-:i.».o,wrmsu.:.-
`
`WW.Mm.“
`
`.m.
`
`
`
`4ii
`
`,.g
`
`.‘w -
`-; 4:!NI-‘m‘é
`Jfigfi
`; Mmimv‘
`
`Fig. 6A
`F@.6A
`
`0015
`0015
`
`

`

`U.S. Patent
`US Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet 14 of 27
`Sheet 14 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`4620
`
`620
`
`I
`
`612
`
`
`
`ffiflgfififigfiggfi
`wwfifififi
`
`
`'---v‘MWm
`
`
`aux-m.
`
`
`'(‘Wmhmufl,
`
`g4;
`
`“Wu-(m: Wm“ammmm
`
`54-:- EM
`WI..M\ scum-m‘
`2-vmmy
`.3
`
`w- 4.!»
`-
`$3 “N".QW‘ -
`Mumm-
`M r w?
`’Wfin-Em 1“.
`”mum:
`
`601
`
`Fig. 6B
`FMSB
`
`0016
`0016
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 15 of 27
`
`US 9,336,336 B2
`
`Fig. 7
`
`Fig. 8
`
`0017
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 16 of 27
`
`US 9,336,336 B2
`
`Fig. 9
`
`Fig. 10
`
`0018
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet17of27
`Sheet 17 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`
`.3::
`S..,§.e.;
`
`mt_
`
`<3
`.21
`LL
`
`513...
`
`3&me
`
`0019
`0019
`
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 18 of 27
`
`US 9,336,336 B2
`
`~ @ 1.$ ¥!t!D t!A fll 19 1!.1:;~ ~-"~,
`'Fu:: Pr J~ , ~»?
`
`~·
`
`N
`0 ..(cid:173)
`..-
`
`.E>
`u.
`
`0020
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheetl9of27
`Sheet 19 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`vvrr 74¢...
`
`wké»mg“
`
`Zzgfi
`
`
`
` (illzfilliulilimv...fiuurhu{In1
`
`
`
`4fl/3.A.02me“flow«a...»
`
`Afialdnfial.
`
`1..iv..
`
`
`
`awwnawxéfiwg.
`
`0:.5
`
`en
`LL
`
`0021
`0021
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet 20 of 27
`Sheet 20 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`..a.,231.168.
`
`3:.
`
`w
`
`.lnfifiRm-‘wfikv‘hufi-DMM
`
`shiurkvigiflwifinusnfl.
`
`
`...ammo—vméaumni
`
`uJanna.
`
`.I.:3.a:
`
`D:.9“—
`
`
`
`
`
`A..............L.i...IL...”—
`
`0022
`0022
`
`
`
`
`
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`M y10, 2016
`
`Sheet 21 of 27
`Sheet210f27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`.....-
`.....-
`
`v v
`
`......
`..--
`
`.......
`.....-
`
`i~
`" ~~ ,
`
`• h t rt
`
`i~-
`1
`lf
`
`,!>
`?:
`
`
`
`?f
`
`0 v ....,..
`..--
`
`C) ·-u.
`
`fiflwm
`3.223;»
`
`0023
`0023
`
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet220f27
`Sheet 22 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`I·
`
`!
`
`~'.
`
`J
`l
`j.
`t i
`' r r
`I , . •
`" ·~
`
`'"""'
`
`
`
`11
`
`ii > ·I
`J
`
`}!
`
`~ j
`
`!
`
`0024
`0024
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet230f27
`Sheet 23 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`3“..._
`
`94L.s‘
`
`$43.53.x
`
`worv
`
`55%F6»...
`
`3..“isan
`
`0025
`0025
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet 24 of 27
`Sheet 24 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`
`
`0026
`0026
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet250f27
`Sheet 25 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`
`
`0027
`0027
`
`ti)
`u..
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet26of27
`Sheet 26 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`.5.9".
`
`0028
`0028
`
`

`

`U.S. Patent
`US. Patent
`
`May 10, 2016
`May 10, 2016
`
`Sheet270f27
`Sheet 27 of 27
`
`US 9,336,336 B2
`US 9,336,336 B2
`
`weer
`
`er
`.9".
`
`5...:3.
`
`
`
`
`
`mmwmmv
`mfiaumzd‘
`
`0029
`0029
`
`
`
`
`

`

`US 9,336,336 B2
`
`1
`i D IMAGE ARRANGEMENT
`
`FIELD OF THE INVENTION
`
`TI1is invention generally relates to a method of visualizing 5
`and modeling a set of teeth for a patient. More particularly, the
`invention relates to providing a 30 virtual model of the
`patient's set of teeth. TI1c method is at least partly computcr(cid:173)
`implemented.
`
`BACKGROUND OF THE INVENTION
`
`2
`Tirns orthodontics relates to movement of teeth, so the
`desired position of a tooth or teeth is detem1ined, and based
`on the present position of that tooth or teeth, the movement
`from the present position to the desired position is deter(cid:173)
`mined. Tirns within orthodontics the desired or resulting posi(cid:173)
`tion of a tooth or teeth is/are is known before planning the
`steps of the movement.
`It remains a problem to provide an improved method and
`system for providing esthetically beautiful and/orphysiologi-
`10 cally suitable results of modeling teeth, both within the field
`ofresloratinns, implants, ortlmdontics etc.
`
`SUMMARY
`
`Visualization and modeling or design of teeth are known in
`the field of dental restorations.
`When a patient requires a dental restoration, such as 15
`crowns, bridges, abutments, or implants, the dentist will pre(cid:173)
`pare the teeth e.g. a damaged tooth is grinded down to make
`a preparation where a crown is glued onto. An alternative
`treatment is to insert implants, such <is titanium screws, into
`the jaw of the patient and mount crowns or bridges on the 20
`implants. After preparing the teeth or inserting an implant the
`dentist can make an impression of the npper jaw, the lower
`jaw and a bite registration or a single impression in a double(cid:173)
`sided tray, also known as triple trays. ·111e impressions are sent
`to the dental technicians who manufacture the restorations 25
`e.g. the bridge. TI1e first step to manufacture the restoration is
`traditionally to cast the upper and lower dental models from
`impressions of the upper and the lower jaw, respectively. The
`models are usually made of gypsnm and often aligned in a
`dental articulator using the bite registration to simulate the JO
`real bite and chewing motion. The dental technician builds up
`the dental restoration inside the articulator to ensure a nice
`visual appearance and bite functionality.
`CAD technology for manufacturing dental restoration is
`rapidly expanding improving quality, reducing cost and 35
`facilitating the possibility to manufacture in attractive mate(cid:173)
`rials otherwise not available. The first step in the CAD manu(cid:173)
`facturing process is to create a 3-dimensional model of the
`patient's teeth. This is traditionally done by 3D scanning one
`or both of the dental gypsum models. TI1c 3-dimcnsional 40
`replicas of the teeth are imported into a CAD program, where
`the entire dental restoration, such as a bridge substructure, is
`designed. ·111e final restoration 30 design is then manufac(cid:173)
`t11red e.g. using a milling machine, 3D printer, rapid proto(cid:173)
`typing manufacturing or other manufacturing equipment. 45
`Accuracy requirements for the dental restorations are very
`high otherwise the dental restoration will not be visual
`appealing, fit onto the Leeth, could cause pain or cause infec(cid:173)
`tions.
`W010031404A relates to tools in a system for the design 50
`of customized three-dimensional models of dental restora(cid:173)
`tions for subsequent manufacturing, where the dental resto(cid:173)
`rations are such as implant abutments, copings, crowns, wax(cid:173)
`ups, and bridge frameworks. Moreover, the invention relates
`to a computer-readable medium for implementing such a 55
`system on a computer.
`Visualizing and modeling teeth for a patient based are also
`known from the field of orthodontics.
`US2006127836A discloses orthodontic systems and meth(cid:173)
`ods for determining movement of a tooth model from a first 60
`position to a second position by identifying one or more
`common features on the tooth model; detecting the position
`of the common features on the tooth model at the fin>L posi(cid:173)
`tion; detecting the position of the common features on the
`tooth model at the second position; and determining a differ- 65
`ence between the position of each common feature at the first
`and second positions.
`
`Disclosed is a method of designing a dental restoration for
`a patient, wherein the method comprises;
`providing one or more 2D images, where at least one 2D
`image comprises at least one facial feature;
`providing a 30 virtual model of at least part of the patient's
`oral cavity;
`arranging at least one of the one or more 2D images relative
`to the 3D virtual model in a virtual 3D space such that the
`2D image and the 3D virtual model are aligned when
`viewed from a viewpoint, whereby the 30 virtual model
`and the 2D image are both visualized in the 3D space;
`and
`modeling a restoration on the 3D virtual model, where the
`restoration is designed to fit the facial feature of the at
`least one 2D image.
`TI1e tenns designing and modeling arc used interchange(cid:173)
`ably in this document to describe what is done to the restora(cid:173)
`tion to make it fit to the patient. The user, e.g. a dental tech(cid:173)
`nician, may be digitally designing or modeling a restoration
`on the 3D virtual model.
`It is an advantage that the 3D CAD modeling of the 3D
`virtual model is based on a 2D digital image, since the 2D
`image determines or indicates what kind of modeling is suit(cid:173)
`able, where the expression suitable may comprise physiologi(cid:173)
`cally suitable or esthetic<Jlly suitable or llppealing. Thus the
`2D image is used to perfonn a correct modeling of the 3D
`model, since the 2D image functions as a benchmark or rule
`for what kind of modeling is possible or how the modeling
`can be with the limits provided by the 2]) image. ·nrns the
`modeling of the 3D virtual model is decided and performed
`based on the one or more 2D image, i.e. such as that the
`modeling of the 3D virtual model is based on the visualization
`of the 2D image.
`'I11epatient' s oral cavity may comprise at least the patient's
`present set of teeth, such as prepared teeth or unprepared
`teeth, if the patient is not toothless, and maybe part of the
`gun1s. If the patient is toothless, then the oral cavity may
`comprise the gums of the patient.
`The 20 image(s) may typically be a digital image, and the
`tenn 2D digital image may be used interchangeably with the
`tenn 2D image in the specification.
`It is an advantage that there may be one or more 2D images.
`If there are more 2D images, one 2D image may be used for
`alignment relative to the 3D virtual mode, and another 2D
`image may be used for designing the restoration. However,
`even if there are more 2D images, the same 2D image may be
`used both for alignment and for designing the restoration. The
`other 2D images may then just be used for visualization and
`presentation etc. If there is only one 20 image, !hat 2D image
`is used both for aligrunent with the 3D virtual model and for
`designing the restoration.
`Thus the 2D image comprising the facial features may be
`denoted the first 2D image, and the 2D image which is used
`0030
`
`

`

`US 9,336,336 B2
`
`lO
`
`4
`3
`for alignment relative to the 3D virtual model may be denoted
`formed, whereby the contour of the teeth on the 2D image
`the second 2D image. If there is only one 2D image, then the
`and/or on the 30 model is automatically derived. Edge detec(cid:173)
`first 2D image and the second 2D image is the same 2D image.
`tion can be perfonned by means of a software algorithm.
`If there are more 2D images, then the first 2D image and Lhe
`Edges are points where there is a boundary or edge between lo
`second 2d image may be the same 2D image, but they may
`image regions, and edges can thus be defined as sets of points
`also be two different 2D images.
`in the image which have a strong gradient magnitude. The
`TI1e restoration is configured to be manufactured, such as
`contour of the teeth may thus be detected by detecting the
`by rapid manufacturing, such as by milling, printing etc. The
`edge between image portions showing the teeth and the gin-
`restorntion may be veneered, such as by adding porcelain to
`gival.
`the surface of it after machine manufacturing. When the res-
`One or more 2D images may be provided in the method,
`!oration is finished, it may be inserted in the patient's mouth.
`and the 2D images may e.g. show the patient' s face from
`It is an advantage that the 2D digital image and the 3D
`different directions, show different parts of the patient's face,
`virtual model are aligned when viewed from one viewpoint,
`such as facial features in the form of the lips and the eyes or
`since hereby the useroroperatornfthe system performing the
`nose for example for detem1ining facial lines, show different
`method, can view the 2D image and the 3D model from a 15
`examples of new teeth which the teeth of the 3D model can be
`viewpoint where they are aligned, since this enables and
`modeled to look like, show the patient's teeth before prepar-
`facilitates that modeling of the 3D model is based on the 20
`ing the teeth for restorations and after preparing the teeth, etc.
`image. That the 2D image and 3D model are aligned when
`In some embodiments the restoration is designed on at least
`seen from a viewpoint means that at least some structures of
`one prepared toolh in the 31) virtual model.
`the 2D image and the 3D model arc coinciding when seen 20
`In some embodiments the 20 image and the 3D model arc
`from a viewpoint. Thus the 2D image and 3D model may not
`aligned based on one or more unprepared teeth.
`be aligned when seen from any viewpoint, thus there may be
`In some embodiments the prepared tooth in the 3D virtual
`only one viewpoint from which the 2D image and the 3D
`model is a physical preparation of the patient's teeth.
`model are aligned.
`In some embodiments the prepared tooth in the 30 virtual
`Furthermore, it is an advantage that the 20 image and the 25
`model is a virtual preparation modeled on the 30 virtual
`3D model are arranged and remain as separate data represen(cid:173)
`model.
`tations which are not merged or fused together into one rep(cid:173)
`In some embodiments the 3D virtual model comprises at
`resentation. By keeping the data representations as separate
`least one prepared tooth.
`representations, time is saved and data processing time and
`In some embodiments the 3D virtual model comprises no
`capacity is reduced. Thus the 2D image is not superimposed 30
`prepared teeth, and where the 3D virtual model is of the
`or overlaid onto the 3D virtual model for creating one repre(cid:173)
`patient's oral cavity before at least one tooth is prepared.
`sentation with all data included. Prior art documents describe
`In some embodiments the method comprises providing
`that the data from e.g. a color image-is added to the 3D model,
`two 3D virtual models. where the first 3D virtual model
`such that the color content from the image is transforred to the
`comprises at least one prepared tooth and the second 3D
`3D model, whereby the result is one representation, i.e. the 35
`virnial model comprises no prepared teeth, and where the first
`3D model including color. Creating such models requires
`and the second JD virtual models are aligned.
`more time and exhaustive data processing.
`In some embodiments the 2D image and the second 3D
`Tims, it is an advantage that the present method may be
`virtual model comprising no prepared teeth are aligned.
`In some embodiments the 2D image and the first 30 virtual
`performed faster than prior art methods .
`TI1c method is for use when modeling teeth, but can of 40
`model comprising at least one prepared tooth arc aligned
`course also with advantage be used by students within the
`based on the alignment between the first and the second 3D
`dental field when learning how to model teeth and what to
`virtual model and based on the alig1unent between the 2D
`take into consideration when modeling teeth.
`image and the second 3D model.
`Modeling of teeth is defined as comprising modeling of
`When aligning the 2D image and the 3D model, the 2D
`one or more dental restorations, modeling of one or more 45
`image may be of the patient's unprepared teeth, since it may
`implants, modeling orthodontic movement of one or more
`be easier to align the 20 image and the 3D model, when the
`teeth, modeling one or more teeth in a denture, e.g. a fixed or
`teeth on the 2D image are unprepared. When modeling the
`removable denture, to provide a visually pleasing appearance
`restomtion e.g. new teeth oft he 3]) model, the 20 image may
`of the set of teeth etc.
`then be of the patient's prepared teeth, since e.g. restorations
`Tims the modeling may comprise modeling of restorations,
`50 normally are modeled after having prepared the teeth by
`orthodontic plam1ing and/or treatment, modeling of implants,
`cutting part of the teeth such that crowns etc. can be attached
`modeling of dentures etc. When the CAD modeling com(cid:173)
`to the prepared part of the teeth.
`prises for example restorations, the virtually modeled resto(cid:173)
`·111e 20 image of the prepared teeth may be aligned to the
`rations, such as crowns and bridges, can be manufactured by
`2D image of the unprepared teeth before the restoration on the
`means of CAM, and the manufactured restorations can then
`55 3D model is designed/modeled based on the 2D image with
`eventually be inserted onto the patient's teeth by a dentist.
`the prepared teeth, since it may be easier to align the 2D
`Arranging, placing, or positioning the 2D digital image on
`images of the prepared and unprepared teeth, e.g. using the
`the 3D virtual model is performed digitally on a computer and
`lips and other features of the face or teeth, than to align the 2D
`shown on a user interface such as a screen, such that the user
`image of the prepared teeth with the 3D model, since here it
`or operator obtains a visual representation of the 2D image
`60 may be difficult to find corresponding features on these.
`and the 3D model together in the san1e field of view, whereby
`However, the method may also be used before the dentist
`the operator can perform the modeling based on the simulta(cid:173)
`prepares any tooth or teeth, e.g. for presenting and showing
`neous view of the 2D image and the 30 model instead of
`the palienl how his set of teeth may look if a reslornlion is
`based on either one combined representation or separate
`made on one or more of the teeth.
`views of the 2D image and/or the 3D model.
`TI1e method may be used for designing a diagnostic wax-up
`For facilitating the arrangement of the 20 image and the
`used to visualize the results of a restoration prior to the treat(cid:173)
`3D model relative to each other, edge detection may be per-
`ment being executed.
`0031
`
`65
`
`

`

`US 9,336,336 B2
`
`5
`When designing a diagnostic wax-up, a virtual margin line
`and a virtual preparation may be made for designing the
`diagnostic wax-up, even though no real preparation is made.
`TI1e method may be used for designing a temporary, which
`the patient can wear after the dentist has prepared a tooth and
`before the final restoration is manufactured and placed on the
`prepared tooth.
`TI1e restoration may be designed, e.g. automatically, by
`selecting a tooth in the 2D image, e.g. the tooth in the position
`where the restoration should be placed or a different aesthetic 10
`tooth. In the 2D image the selected tooth is only seen from one
`viewpoint, so only the front side, the width and the height of
`the loolh may be seen in the 20 image. Thus the backside of
`the tooth cannot be seen. A standard model tooth may be
`selected from a library, and this model tooth may be shaped as
`the selected tooth in the 2D image. The model tooth or resto(cid:173)
`ration can only be shaped as the selected tooth in the surfaces
`which arc seen in the 2D image. The rest of the model tooth or
`restoration may be shaped according to some standard for a 20
`tooth in that respective location in a mouth. E.g. the backside
`or the distal surface of a central tooth may typically be flat,
`whereas the distal surface of a canine may typically be trian(cid:173)
`gularly shaped, and lhe distal surface of a molar may typically
`resemble the mesial surface of the tooth. Or the distal surface
`of the neighbor teeth or the corresponding tooth on the other
`side of the midline in the mouth may be used to shape the
`surfaces of the restoration which cannot be derived from the
`2D image. The restoration can be designed on the 3D virtual
`model, and the part of the restoration which is in contact with
`e.g. the preparation may be automatically designed to
`resemble the shape of the restoration.
`TI1e restoration can be a crown, a bridge, an abutment, an
`implant, a denture, such as a fixed orremovable denture, a foll
`denture or partial denture, a diagnostic wax-up, a temporary 35
`etc .
`Designing a restoration may comprise designing at least
`part of a preparation, designing at least a part of the gingival
`surrounding the restoration in the patient's mouth etc.
`It is an advantage that the restoration is designed to fit or
`match the facial feature of the at least one 2D image, since this
`will provide a restoration which looks natural relative to the
`patient's face and/or this will provide a reslnralinn which is
`aesthetic, such as symmetrical. TI1c dental technical mies for
`designing teeth, mathematical or algorithmic rules and/or
`mies for aesthetics may be programmed into the software or
`used in the software or method for designing the restoration to
`fil !he facial features, and based on these rules the restoration
`may be designed, e.g. partly automatically. The dental tech(cid:173)
`nician or the dentist may use his/her experience and knowl(cid:173)
`edge about dental aesthetics and rules to design and deter(cid:173)
`mine when the restoration fits the facial feature in the
`patient's image or in a template or standard image of a face.
`Designing the restoration to fit the facial foatures of the 2D
`image may be based on purely objective rules for restoration
`design. However designing the restoration to fit the facial
`features of the 2D image may alternatively and/or addition(cid:173)
`ally be based on more subjective opinions and choices of the
`dental technician or dentist.
`In some embodiments facial features are present in an
`image of the patient and/or in a generic image of a person.
`In some embodiments the facial feature is one or two lips,
`one or more leeth, and/or the shape and/or size of the face.
`In some embodiments the facial features comprise one or
`more imaginary lines of a face adapted to be detected in the
`2D image, such as the midline, the horizontal line, and/or the
`bi-pupillar line.
`
`6
`If the 2D image is an image of at least part of the patient's
`face, then the facial features used for designing the restoration
`may be the Ii ps of the patient, the smile line of the patient's
`mouth, the symmetry lines in thepatienl's face, the midline of
`the patient's face, the horizontal line of the patient's face, the
`patient's anterior teeth etc. Thus the restoration may be
`designed by fitting the restoration to the lips of the patient, by
`fitting the restoration to the smile line of the patient's mouth,
`by fitting the restoration to the palienl's anterior leelh elc.
`If the 2D image is an image, such as a drawing, of a generic
`template face, then the facial features used for designing the
`restoration may be symmetry lines of the template face,
`shapes and sizes of the teeth on the template face etc.
`When designing the restoration lo fil lhe facial features, lhe
`15 restoration may be designed such that there is a certain dis(cid:173)
`tance from the edge of the upper lip to the incisal edge of the
`anterior teeth, e.g. the centrals, when the patient smiles a
`natural smile; and/or such that a certain percentage or amount
`of the centmls are visible when the patient smiles.
`Furthermore, when designing the restoration to fit the
`facial features, the restoration may be designed by consider(cid:173)
`ing the shape of the patient's face, the gender of the patient,
`the phenotypic characteristics of the patient, i.e. whether the
`patient is Asian, African, Caucasoid elc. for example Asians
`25 typically has smaller teeth, men typically have bigger teeth
`than women, oval teeth typically suit an oval face shape etc.
`Furthermore, if the patient has a small dental arch or jaw,
`then the distance between the canines will typically be
`smaller, and the anterior teeth should then typically be more
`30 narrow, than the teeth in a patient with a large arch and a larger
`distance between the canines.
`In some embodiments the restoration is a crown, a bridge,
`an abutment, an implant, a denture, a diagnostic wax-up,
`and/or a temporary.
`In some embodiments the designing of the restoration is
`performed to automatically fit the facial features of the at least
`one 2D digital image.
`In some embodiments the restoration is designed by select(cid:173)
`ing a tooth in the 2D image, and modeling the restoration to
`40 have the same shape as the selected tooth.
`In some embodiments the 3D virtual model is generated by
`scanning a physical model of the patient's teeth, by scanning
`an impression of the patient's teeth, and/or by perfonning a
`direct scanning of the patient's teeth. If the patient is tooth-
`45 less, then the gums, a model or an impression of the gums may
`be scanned for creating a 3D model of the oral cavity.
`In 3D scanning the object is analyzed to collect data on its
`shape. 'l11e collecled data can then he used lo construe! digital,
`three dimensional models. In 3D scmming usually a point
`50 cloud of geometric samples on the surface of the subject is
`created. Tiiese points can then be used to extrapolate the
`shape of the subject.
`In some embodiments the one or more 2D digital image
`comprises a patient-specific image of at least part of the
`55 patient's face.
`An advantage of this embodiment is that the modeling can
`be based on an image of the patient, such that the modeling is
`perfonned with respect to the facial foatures fonning the look
`or appearance of the patient, or with respect to some, a few or
`60 a single, specific visual facial features of the patient, such as
`the lips.
`In some embodiments the one or more 2D digital image
`comprises a generic image nf al least part of a human face.
`An advantage of this embodiment is that the modeling can
`65 be based on a generic image, whereby it is not patient-specific
`facial features which determine the modeling, but instead it is
`a general image, e.g. the facial features may be some visually
`0032
`
`

`

`US 9,336,336 B2
`
`7
`pleasing teeth from another person, or the facial feature may
`be a drawing of some ideal teeth etc .
`In some embodiments the one or more 2D digital image is
`retrieved from a library comprising a number of images of
`teeth.
`An advantage of this embodiment is that the 2D image,
`such as a generic image, can be selected from a library which
`contains for example several images of teeth, so that the
`palient e.g. can choose his/her desired new sel of teeth from
`the library. The library may be a so called smile guide library
`comprising images of teeth and/or mouths which are shown
`while smiling, since visually pleasing teeth may be most
`important when smiling, since this may be when most teeth
`are shown lo the surrow1dings.
`The images of teeth in the library may be photos of teeth, 15
`may be drawings of teeth, etc. and thus the facial features are
`then teeth.
`In some embodiments the 2D image comprises a cross for
`pmviding a visual symmetry