(12) United States Patent
`Kopelman et al.
`
`111111111111111111111111111 lllll lllll llHI 111111111111111111111111111111111
`US006845175B2
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 6,845,175 B2
`Jan.18,2005
`
`(54) DENTAL IMAGE PROCESSING METHOD
`AND SYSTEM
`
`(75)
`
`Inventors: Avi Kopelman, Tel Aviv (IL); Eld:1d
`Taub, Reul (IL)
`
`(73) Assignee: Cadent Ltd., Or Yehuda (IL)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 479 days.
`
`(21)
`
`Appl. No.: 09/830,264
`
`(22)
`
`Filed:
`
`Apr. 23, 2001
`
`(65)
`
`Prior Publication Data
`
`US 2003/0169913 Al Sep. 11, 2003
`
`Related U.S. Application Data
`
`( 63) Continuation of application No. PCT/IL99/00577, filed on
`Nov. 1, 1999.
`
`(30)
`
`Foreign Application Priority Data
`
`Nov. 1, 1998
`
`(IL)
`
`........... ... ...................... ........•... 126838
`
`Int. Cl.7
`. .... .. . ... . . _ ••• •. ...... . . •..• .. •.... . . . .. .. . . . . G06K 9/00
`(51)
`(52) U.S. Cl • ......... ... ............ 382/154; 382/294; 128/922
`(58) Field of Search ..... ............................ 382/128, 131,
`382/154, 285, 294; 600/587, 589, 590;
`128/922; 433/24, 29, 68, 69, 215
`
`(56)
`
`References Cifod
`
`U.S. PATENT DOCUMENTS
`
`5,151,856 A
`
`9/1992 llalmann el al.
`
`INPUT A .10 VIRi1JAL Monr
`
`MA<11( OOIC: LANDMARKS
`
`IHi'UT i' CEltw.OMETRIC X- RAY
`Of THE SAJ.<E PATIENT
`
`5,278,756 A
`5,318,441 A *
`6,068,482 A *
`6,081,739 A *
`
`1/1994 Lemchen et al.
`6/1994 Keller .............. ........ ... 433/68
`5/2000 Snow ............ ...... ....... 433/223
`6/2000 Lemchen .................... 600/407
`
`FOREIGN PATENT DOCUMENTS
`
`DE
`EP
`EP
`JP
`WO
`WO
`
`4141311
`0488987
`0741994
`4-336048
`WO 97/03622
`01/80761 A2
`
`8/1993
`6/1992
`11/1996
`11/1992
`2/1997
`11/2001
`
`OTHER PUBLIC.AJlONS
`
`International Search Report for International Application
`No. PCT/ IL 99/00577, mailed on Feb. 17, 2000.
`An English language title and abstract is provided for DE
`4141311.
`* cited by examiner
`
`Primary Examiner-Todd E. Manahan
`(74) Allorne)\ Agenr, or Firm-Nath & Associates PLLC;
`Gary M. Nath; Lee C. Heiman
`
`(57)
`
`ABSTRACT
`
`An image processing method for use in dentistry or orth(cid:173)
`odontic is provided. Two images of teeth, one being a
`two-dimensional image and one a three-dimensional image
`are combined in a manner to allow the use of information
`obtained from one to the other. In order lo combine the two
`images a set of basic landmarks is defined in one, identified
`in the other and then the two images are registered.
`
`23 Claims, 11 Drawing Sheets
`
`-
`C':"':?-
`
`( 131
`
`\40 .....---~-----.
`MOVE lEETH AND JAWS ON
`30 \lllmJAL MODEL 10
`RECEIVE OESIRl':D RE:SULTS
`
`' '
`
`Y MOVE s~oc .. l:l.11 ..
`i.l.Hl.I OIJIJ RIC X-
`ON
`[[[IJ~NTS ACWRDING TO MOVEMENTS lHE
`LS...'1l i'OU'Olll.lll> ON oD MODE\.
`
`J.IO'<t:. ~Ill
`0'1 :nJ'H'lCL!o C X-AA
`. AttCRDtttG ~O 11.CUEMEITTS TiiE USER
`PERFORM<D ON oD t.ICO(l.
`
`lltl CEL•l\l.l.O ;<iRJC X·FIA" UPfl.\'f[ T,>.C
`RESVL TS Cf OOIRED PP.Of':JR'TTlOWJ.
`~·EASuRDJP<JS
`
`0001
`
`exocad GmbH, et.. al.
`Exhibit 1008
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 1of11
`
`US 6,845,175 B2
`
`FIG. 18
`
`FIG. 1A
`
`0002
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 2of11
`
`US 6,845,175 B2
`
`-
`
`~
`
`FIG. 2
`
`0003
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 3of11
`
`US 6,845,175 B2
`
`FIG. 3
`
`0004
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 4of11
`
`US 6,845,175 B2
`
`FIG.3A
`
`FIG.3B
`
`0005
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 5of11
`
`US 6,845,175 B2
`
`FIG. 4
`
`0006
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 6of11
`
`US 6,845,175 B2
`
`FIG.4A
`
`·
`
`FIG.4B
`
`0007
`
`

`

`U.S. Patent
`
`Jan. 18,2005
`
`Sheet 7of11
`
`US 6,845,175 B2
`
`FIG. 5
`
`0008
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 8of11
`
`US 6,845,175 B2
`
`40
`
`/20
`
`28
`
`p
`
`M
`
`30
`
`26
`
`u1
`
`u2
`
`u3
`
`32
`
`FIG.6
`
`0009
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 9of11
`
`US 6,845,175 B2
`
`10 0 ......_
`
`11 0
`\._
`
`INPUT A 30 VlRTUAL MODE
`t
`MARK BASIC LANDMARKS
`t
`1 20
`\..._ INPUT A CELHALOMETRIC X-RAY
`OF THE SAME PATIENT
`t
`MARK THE SAME
`BASIC LANDMARKS
`t
`40
`1
`~ MOVE TEETH AND JAWS ON
`3D VIRTUAL MODEL TO
`RECEIVE DESIRED RES UL TS
`
`30
`1
`\....
`
`FIG.7A .
`
`I
`I
`I
`f
`
`I '
`
`15
`ON CELHALOMETRIC X-RAY MOVE SKELETAL
`0
`'-._ ELEMENTS ACCORDING TO MOVEMENTS THE
`USER PERFORMED ON 30 MODEL
`~
`ON C£LPHALOMETRIC X-RAY MOVE TEETH
`ACCORDING TO MOVEMENTS THE USER
`PERFORMED ON 3D MODEL
`t
`ON CELPHALOMETRIC X-RAY UPDATE THE
`RESULTS OF DESIRED PROPORTIIONAL
`MEASUREMENTS
`
`16
`0
`\..___
`
`70
`1
`\...._
`
`FIG.7B
`
`0010
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 10 of 11
`
`US 6,845,175 B2
`
`200
`~
`INPUT A- CEPHALOMITRIC X-RAY
`
`211
`
`210~
`
`DRAW THE OUTLINE: OF UPPER
`AND LOWER CENTARLS ANO Of
`UPPER AND LOWER FIRST
`MOlARST MARK LANO MARKS
`ON CROWNS AND ROOTS OF
`THESE TEETH
`
`220~
`
`INPUT A 30 VfRTUAL MODEL
`OF THE SAME PATIENT
`
`230~
`
`MARK THE SAME LAND MARKS
`ON CROWNS AND ROOTS AS
`WAS MARKED ON CEPHALOMCTRlC
`X-RAY
`
`24-0~
`MOVE TEETH AND SKELETAL
`ELEMENTS ON CEPHALOMETRIC
`X-RAY TO RECEIVE DESIRED
`PROPORTIONAL MEASUREMENTS
`
`FIG.BA
`
`0011
`
`

`

`U.S. Patent
`
`Jan.18,2005
`
`Sheet 11 of 11
`
`US 6,845,175 B2
`
`
`2 50
`ON 3D MODEL CHANGE THE POSITION
`RELATIONS BETWEEN THE JAWS L/
`ACCORDING TO TRANSFORMATIONS
`ON THE CEPHALOMETRIC X-RAY
`
`ON 30 MODEL MOVE THE TEETH
`2 60
`ACCORDING TO MOVEMENTS _ /
`PERFORMED BY THE USER ON THE
`CEPHALOMETRIC X-RAY
`
`ON 30 MODEL ARRANGE ALL TEETH
`RELATIVE TO THE POSITION OF THE J
`TEErH THAT WAS DEFINED BY THE
`USER ON CEPHALOMETRIC X-RAY
`
`2 70
`
`FIG.BB
`
`0012
`
`

`

`US 6,845,175 B2
`
`1
`DENTAL IMAGE PROCESSING METHOD
`AND SYSTEM
`
`This is a continuation of prior International application
`number PCT/IL99/00577, filed Nov. 1, 1999 which is hereby
`incorporated herein by reference in ils entirety.
`
`FIELD OF THE INVENTION
`The present invention is generally in the field of dentistry
`and provides an image processing method and system useful 10
`as a tool by the dentist or orthodont.
`
`2
`(a) applying at least a first imaging technique and a second
`imagining technique to acquire a first, two-dimensional
`image of at least a first portion of teeth and a second,
`three-dimensional virtual image of al least a second
`portion of the teeth, respective! y, there being at least a
`partial overlap between said first and second portions;
`and
`(b) defining a set of basic landmarks in either one of the
`two images, locating said set in the other of the two
`images and registering said set in the two images.
`Dy another of its aspects the present invention provides an
`image processing system comprising:
`(i) a first utility for receipt of first data representative of
`a first two-dimensional image of at least a first teeth
`portion;
`(ii) a second utility for receipt of second data represen(cid:173)
`tative of a second three-dimensional virtual image of
`leelh of al leasl a second leelh porlion;
`(iii) a module for defining basic landmarks in both images
`and for generating data representative thereof; and
`(iv) a processor associated with said first and said second
`utility and with said module, for receiving said first and
`said second data and for mapping elemenL~ in one of lhe
`two images to the other of the two images according lo
`the data representative of said basic landmarks.
`In accordance with one embodiment of the invention, the
`imaging method and system is used to obtain orthodontic(cid:173)
`relevant information, namely information to be used by an
`orthodont within the framework of an orthodontic treatment
`or for the design of such a treatment. This embodiment
`involves a registration of at least two images, one being a
`three-dimeasional virtual image of a teeth model and the
`other being a two-dimensional image, e.g. a cephalometric
`image. Occasionally, other images may also be brought into
`registration consisting, for example of one or more of a
`lateral videographic image, a frontal videographic image
`and a frontal cephalometric image.
`In accordance with another embodiment of the invention,
`40 the method and system are used for proper design of a dental
`implant or of a crown. For proper placement of an implant,
`the bone has lo be carefully studied beforehand and exam(cid:173)
`ined whether it can receive the dental implant In addition,
`the exact position and orientation of the dental implant has
`to be properly pre-designed. Typically, for a proper design of
`an implant, a three-dimensional virtual image of a teeth
`model is brought into registration with both a lateral cepha(cid:173)
`lomelric image and al times also with a frontal cephalom(cid:173)
`etric image. This will allow to properly predict the manaer
`of receipt of the implant within the bones of the jaw.
`In the following, the invention will be described with
`particular reference to imaging for the purpose of design of
`Lhe orthodontic lrealmenl. It will however be appreciated,
`that the invention applies, mutatis mutandis also lo its
`application for he purpose of proper design of tooth
`implants.
`The first image is preferably an x-ray image, typically a
`cephalometric image obtained by radiographic cephalomet(cid:173)
`ric technique. The x-ray image is preferably a lateral image
`60 although at times the image may be from aaother
`orientation, e.g. a frontal image. In a cephalometric image,
`some facial profile aspects may at times be seea. However,
`typically, before an orthodontic treatment also a third, side
`elevational view of the face is taken from the same direction
`65 in which the radiographic cephalometric image was
`obtained. In accordance with an embodiment of the
`invention, such a third image, comprising at least a profile of
`0013
`
`25
`
`BACKGROUND OF THE INVENTION
`There are a wide variety of imaging techniques used
`routinely in orthrdontics. One important imaging technique 15
`is the so-called radiographic cephalomelric technique. A
`radiographic cephalometric image is then used for a cepha(cid:173)
`lometric analysis. Such an analysis is essentially a measure(cid:173)
`ment system designed to describe relationships between the
`various parts of the skeletal, dental and soft tissue elements 20
`of the cranofacial complex. The two cephalometric images
`typically used are a lateral cephalometric image, which is of
`prime use in orthodontic and a front cephalometric image
`which is of somewhat less importance.
`Cephalometric methods enable to define certain norms of
`a skeletal, dental and soft tissue of the cranofacial complex.
`A cephalometric measurement of individuals can then be
`compared with norms for age, sex and population group. A
`cephalogram is in eITecl a two-dimensional represenlalion of 30
`a three-dimensional cranofacial structure. It is thus difficult
`in an analysis performed on such an image to distinguish
`between bilateral structures to trace them independently.
`Additionally, facial aspects are not entirely asymmetrical,
`this may add a further inaccuracy to an analysis of Lhis kind. 35
`Other sources of errors in a cephalometric image include
`different magnification of different aspects depending on the
`distance from the film and imperfect positioning of the
`patient in the cephalostat. These all add up lo considerable
`errors in cephalomelry.
`An orlhodonl, prior lo heginning lhe orlhodonlic lreat(cid:173)
`menl typically takes a teeth impression on the basis of which
`a plaster model may be prepared There are known also a
`number of imaging techniques which allow to obtain, within
`a computer environment, a virtual three-dimensional image 45
`of the teeth. Such techniques are described for example in
`WO 97/03622 and DE-C-414311. A three-dimensional teeth
`image provides a different information than that obtained by
`a cephalometric analysis. Particularly, a virtual teeth image
`allows better appreciation of the three-dimensional structure 50
`of the teeth and Lht: relative position of different teelh.
`Fm the purpose of proper design of orlhodonlic trealmenl
`it would have been high advantageous to have a method and
`system whereby information which can be acquired from
`one type of image can be transferred or superpositioned to 55
`information available from another type of image.
`
`GENERAL DESCRIPTION OF THE INVENTION
`In accordance with the invention a novel method and
`system is provided in which information and dala available
`from one type of teeth imaging technique is transferred and
`used in an image obtained by another kind of teeth imaging
`technique. This transfer of information provides the dentist
`or the orthodont with a powerful tool for designing of
`orthodontic treatment.
`In accordance with the invention there is provided an
`image processing method comprising:
`
`

`

`US 6,845,175 B2
`
`4
`3
`facial aspects, is also obtained and used in the imaging
`until there is an essential complete overlap of the basic
`landmarks in the two images. It should however be noted
`technique of the invention. The side elevational image may
`that registering of different images may not necessarily
`be obtained, for example, by video cephalometry.
`imply superpositioning, but rather at times the two regis(cid:173)
`The term "virtual three-dimensional teeth image" refers lo
`tered images may be represented separately, e.g. side-by(cid:173)
`an image, represented within the computer environment
`side. '!be important result of the act of registering is that
`which consists primarily of the teeth of one or both jaws. For
`manipulation made on one of the images will eJTect the other
`example, a virtual three-dimensional teeth image may be
`as well.
`represented in a manner resembling an image of a plaster
`In the following, the invention will be described with
`model. A virtual three-dimensional image may be obtained
`particular reference to an embodiment in which the first
`by a variety of techniques, e.g. those described in the 10
`image is a cepbalometric image and the second image is a
`references mentioned above . Particularly, the three(cid:173)
`virtual three-dimensional image. This specific reference
`dimensional virtual image may be obtained by the method
`should not however be construed as meaning that the
`described in WO 97/03622, which is incorporated herein by
`invention is limited thereto. On the contrary, by applying the
`reference as an example of the manner of obtaining a
`general teaching of the invention, information may be trans(cid:173)
`three-dimensional virtual image for use in the method and 15
`ferred between images obtained by other imaging tech(cid:173)
`system of the invention. It should be understood that the
`niques.
`invention is not limited to a specific type of image obtained
`In accordance with one embodiment of the invention,
`by one imaging technique or another. For example, the
`after landmarks have been defined in the three-dimensional
`two-dimensional image may be obtained by a variety of
`virtual images and in the cepbalometric image, the correct
`orientation of the virtual three-dimensional teeth model bas
`different imaging techniques including magnetic resonance 20
`imaging (MRI), computerized tomography (CT) various
`to be determined so as to allow it to be brought into
`conformity with the cephalometric image. This may at times
`radio-imaging techniques, etc. Similarly, the three(cid:173)
`require extensive computational time. It has however been
`dimensional teeth image may be obtained by any one of a
`found th al the process of registration of the lwo images can
`number of imaging techniques available including those
`disclosed in the aforementioned references as well as others 25
`be considerably accelerated by defining the cephalometric
`image to overlap the mid palatal plane of the virtual tbree(cid:173)
`such as those making use of a scanning probe, various
`photographic techniques, techniques in which teeth are
`dimensional teeth image. In other words, the cephalometric
`image is defined to lie on the mid palatal plane and the
`scanned by a probing light beam, etc.
`The term "image" as used herein should not be under(cid:173)
`cephalographic image is then adjusted until the basic land(cid:173)
`marks overlap with the projection of the corresponding basic
`stood only as referring to the image as acquired in the 30
`imaging technique but rather may be also a result of initial
`landmarks of the virtual three-dimensional image onto the
`mid palatal plane.
`image processing, e.g. an image processing intended to
`The invention permits also an analysis of the effect of
`define boundaries of various objects in the image. Thus, the
`teeth displacement on various aspects of the cranofacial
`term "image" encompasses also a representation, prepared
`on the basis of an acquired image, of boundaries of objects, 35
`complex. For example teeth may be displaced on the virtual
`three-dimensional image of teeth model in a manner they are
`e.g. teeth, bones, a profile of facial aspects, etc.
`expected to be shifted during the course of the orthodontic
`Often, the imaging technique and analysis in accordance
`with the invention will make use of a third image, which
`treatment. Thus, for example, by marking various landmarks
`may be the elevational image mentioned above, or any other
`on a displaced teeth and marking and then displacing the
`same landmarks in the cephalometric model, it may be
`image useful in improving the orthodontic analysis. Thus, by 40
`way of example, where said first image is a lateral two(cid:173)
`possible to check on both images whether the orthodontic
`treatment achieves a result which matches a certain accept(cid:173)
`dimensional image, said third image may be one or both of
`the afore-mentioned lateral elevalional image or a frontal
`able norm or how changes should be made lo achieve such
`a norm. If, for example, a desired result as viewed in an
`x-ray or videographic image.
`The basic landmarks which are used for registering the 45
`amended cephalometric image (namely a cepbalometric
`two sets of images, are typically defined points at either the
`image after a tooth has been displaced) does not match the
`desired results, it is possible to go back to the virtual
`base or the apex of certain selected teeth e.g. the incisors and
`three-dimensional teeth model and proceed with a simula(cid:173)
`the first molars. Such basic landmarks may be selected by
`tion and then map the results onto the cepbalometric image,
`the user or may be automatically selected by the system's
`and so forth.
`processor, e.g. based on established norms. After selecting 50
`By way of example, in order to achieve the same degree
`the basic landmarks and marking them in one of the images,
`of displacement in one image, the shifting of a certain
`then the landmarks may be marked in the other images to
`landmark which is associated with a displaced object is then
`allow to register both images. The term "registering" should
`compared to some basic landmarks and the same relation of
`not necessarily be understood as meaning a physical regis(cid:173)
`displacements is then related to the other image.
`tration of the two images but rather as meaning the mapping 55
`One particular example of analysis which can be made by
`of each feature in one image to a corresponding feature in
`such simulation is to determine the effect of such displace(cid:173)
`another. The outcome of such registration is that any
`ment on soft facial tissue, particularly outer facial tissue.
`manipulation made on one image will yield a corresponding
`manipulation in the other image. For example, if one image
`This will allow an estimation of the effect of the orthodontic
`is manipulated by displacing one tooth, this should result in 60 treatment on the estbetic appearance of the individual.
`a corresponding displacement of the same tooth in the other
`A simulation of the treatment and then translation of the
`results to a cephalometric image allows also to determine
`image
`At limes it may be desired to view both images on a screen
`whether shifts in various elements such as the jaw, are within
`permitted physiological or aestbetical limits. An uncon(cid:173)
`superimposed one on the other. As two or more images have
`trolled shifting of a tooth or a jaw in an orthodontic
`to be superimposed may be presented initially at a different 65
`scale, an initial step which is necessary to be taken by the
`treatment may give rise to various physiological and func(cid:173)
`system is to either enlarge or reduce the scale of one image
`tional problems.
`0014
`
`

`

`20
`
`35
`
`DETAILED DESCRIPTION OF SPECIFIC
`EMBODIMENTS
`
`40
`
`US 6,845,175 B2
`
`5
`The invention will now be illustrated below with refer(cid:173)
`ence to some specific, non-limiting embodiments, with
`occasional reference to the accompanying drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. lA shows an example of a radiographic cephalom(cid:173)
`etric image.
`FIG. lB shows a virtual three-dimensional image of teeth,
`presented in the manner resembling a plaster teeth model. 10
`FIG. 2 shows a super-position of a three-dimensional
`teeth model and a cephalometric image.
`FIGS. 3A and 3B show two examples of super-position of
`a three-dimensional model and a video cephalometric
`image.
`FIG. 4A shows a cephalomelric image with some basic
`landmarks marked thereon.
`FIG. 4B shows a three-dimensional virtual image of the
`same teeth as those shown in the cephalometric image of
`FIG. 4A, with the same basic landmarks marked thereon.
`FIG. 5 shows a super-position of the two images.
`FIG. 6 is a block diagram representation of a system in
`accordance with the invention.
`FIGS. 7A and 7B are flowcharts showing the manner of
`mapping elemenL<; from a three-dimensional virlual teeth
`model to a cephalometric image. FIG. 7A shows the user's
`interaction modules whereas FIG. 7B shows the software
`functionality underlying the manner of performing of dis(cid:173)
`placement and mapping the displacement from the three(cid:173)
`dimensional virtual teeth model to the cephalometric image.
`FIGS. SA and SB are flowcharts showing the manner of
`mapping elements from a ceplhalometric image to a three(cid:173)
`dimensional virtual teeth model. FIG. SA shows the user's
`interaction modules whereas FIGS. SB shows the software
`functionality underlying the manner of performing of dis(cid:173)
`placement and mapping the displacement from the cepha(cid:173)
`lometric image to the virtual three-dimensional teeth model.
`
`6
`for example, to test the effect of the treatment on some
`physiological or functional parameters as well as the aes(cid:173)
`thetic parameters. There is, however, a significant deficiency
`in lhal it is impossible lo fully traoslale this information lo
`tbe three-dimensional real-life environment. The present
`invention permits a proper analysis of the effect of displace(cid:173)
`ment of elements and or better understanding of how
`changes will effect the real-life situation.
`Reference is first being made to FIGS. lA and lB,
`showing respectively, a cephalometric radiograph and a
`three-dimensional virtual teeth image. The virtual teeth
`image which is shown in FIG. lB, is represented in a manner
`resembling a plaster teeth model. As will no doubt be
`appreciated by the artisan, this is but an example, and the
`15 two-dimensional or the virtual three dimensional teeth
`image may be represented in a different way.
`Prior to the present invention, each of these different
`images, was represented separately. The three-dimensional
`virtual image was repr cnted either as a plaster model or a
`three-dimensional virtual representation in a computer envi-
`rooemenl. Io accordance with the invention, two different
`images, one being a two-dimensional image. e.g. a cepha(cid:173)
`lometric radiograph, is combined with a three-dimensional
`teeth image. A super-position of two such images is repre-
`25 seated in exemplary FIG. 2. As can be seen, the ccphalom(cid:173)
`etric image is combined with the three-dimensional virtual
`teeth image such that it lies on the mid palatal plane of the
`three-dimensional virtual teeth image. The relative position
`of the two images is fixed such that basic landmarks defined
`30 in the two images concur, as will be described farther below.
`Another implementation of the invention can be seen in
`exemplary PIGS. 3A and 3Il. In these figures, a three(cid:173)
`dimensiooal virtual teeth image is superpositioned with a
`lateral or profile picture of an individual. The profile pictures
`in FIGS. 3A and 3B are each from a slightly differently
`orientation and accordingly the virtual three-dimensional
`teeth model in FIG. 3B is rotated with respect to the
`orientation it of the model in FIG. 3A.
`Io order to combine a cephalomelric image and a three-
`dimensional virtual model, hasic landmarks have lo be
`defined and marked in both images. These basic landmarks
`may be entered manually by the user, although alternatively,
`In accordance with the present invention images arc
`they may be automatically generated by a computer, based
`acquired including at least one two-dimensional teeth image
`45 on standard image analysis method, or based on an earlier
`and at least one three-dimensional teeth image and both are
`user input. Generally, such basic landmarks may be arbitrary
`combined for lhe purpose of improving the orlhodonl's
`landmarks or may be orthodontic relevant landmarks which
`ability to predict the effect of orthodontic treatment on
`may be used later in a ccphalometric analysis, in accordance
`various parameters. This combination allows the orthodont
`with one of the acceptable norms therefor. (For review on a
`to considerably increase the depth of his understanding on
`50 cephalometric analysis see Radiographic Cepbalometry,
`the outcome of the orthodontic treatment. Hitherto, analysis
`From Basics lo Videoimaging, Jacobson A, el al., Quintes(cid:173)
`which was made on a cephalomelric images could not have
`sence Publishing Co., Inc., Chicago, Berlin, 1995).
`been readily translated to the other tools available to him(cid:173)
`this being the three-dimensional teeth model, typically a
`A cephalometric radiograph and the three-dimensional
`plaster model. In the reverse, information gained by him
`teeth model from the same individual, arc shown in FIGS.
`from studying a three-dimensional teeth model, could not
`55 4A and 4B. Io these figures, two basic landmarks have been
`have been readily translated lo a cephalometric image. As is
`marked-Ll and L2. After these landmarks have been
`well known to the artisan, each one of the images allows a
`marked, the two images are brought into registration wbich
`limited range of analysis which can be made and a true
`results in super-positioning as can be seen in FIG. 5 (the
`analysis can only be gained from thorough analysis based on
`same two landmarks Ll and L2 can also be seen here). The
`the two types of images.
`60 registration in the manner shown in FIGS. 4A, 4B and 5 is
`performed using two defined landmarks. Obviously, it is
`It is only with the present invention that a proper analysis
`possible at times to use more landmarks for this purpose to
`becomes feasible.
`increase accuracy of registration.
`An image, once acquired and converted lo a representa(cid:173)
`Io order to reduce computational time, the cephalometric
`tion within a computer environment can be manipulated, e.g.
`65 radiograph is combined with the three-dimensional virtual
`by displacing certain elements, such as one or more teeth or
`teeth image by placing (in a virtual sense) the cephalometric
`even an entire jaw. The cepthalometric image allows to view
`image on the mid palatal plane. For proper registration, the
`the, interrelation between some elements and may be used,
`0015
`
`

`

`US 6,845,175 B2
`
`7
`scale of the two images has to be adjusted and then one
`image has to be shifted versus the other until the projection
`of the basic landmarks of the three-dimensional virtual
`image of teeth model onto its mid palatal plane are in
`register with the corresponding landmarks in the cephalo(cid:173)
`melric image.
`The cephalometric radiograph and the cephalometric
`videograph as shown herein, arc images as acquired by the
`utilized imaging technique. IL should however be noted that
`to produce initia!Iy a represen(cid:173)
`at times it i advantagcou
`tation of the irnngc, e.g. a graphic representation of bound(cid:173)
`aries of objects of interest within the image. For example,
`rather than a full cephalographic image a re_prcsentation
`comprising boundaries of some major bones and several
`teeth, e.g. the first and second molar teeth and the incisocs.
`These aforementioned Leeth are usually lhi.: important lt:clh
`for the cephalomelric analysis, as their posilion is relatively
`sensitive to displacement of teeth and jaws. ·uriberrnore, the
`position of these teeth is an important marker for studying or
`gauging the effect of teeth position on various functional as
`well as aesthetical facial aspects.
`Producing a graphical represemation of some a pect of
`an image pa rt icularly of a ccphalometric image, is very
`useful for tbe purpose of a virtual displacement or the teeth
`in the image so as lo study the e!Iecl of the lrealmen l on
`functional or facial aesthetic parameters, as generally known
`per se.
`A system in accordance with the embodiment of the
`invention can be seen in FIG. 6. It comprises a central
`computing unit 20 with three input utilities 22, 24 and 26,
`which may be integral within module 28. These utilities may 30
`comprise, ns known per se, a data entry pori aod the
`ncce nry data transfer ofl.warc. Furthermore, ralber than
`importing of data through a data entry port, tbc data to Lbcsc
`utilities may be imported from a
`loragc media or from an
`information carrier. e.g. a magnetic or an optical disk . As 35
`will no doubt be further under tood, module 28 may also
`comprise a scanner for scanning images, may comprise a
`camera for direct image acquisition, etc.
`The system still furLher comprises a module 30, connected
`to a user input interface 32 e.g. a keypad, a cursor driver etc. 40
`Dy means of interface 32 the user may define the landmarks
`or may induce the system to enter into various operational
`modes, some of which will be explained below.
`Module 30 and utility 28 are connected to a processor 40
`for image processing so as lo combine the two images as 45
`described, for example further below. Processor 40 may be
`connected to monitor 50 and may be also connected to other
`display means, e.g. a printer.
`A tlowchart of an embodiment of the manner of linking
`between a three-dimensional virtual teeth model and a 50
`cephalometric image can be seen in FIGS. 7A and 7B. FIG.
`7 A is a flowchart of the user interaction steps whereas FIG.
`7B is a software functionality !lowcharl on the manner of
`combining the two images. Al a first step 100, the system
`receives an input of data representative of a three- 55
`dimensional virtual teeth model. Then at 110, basic land(cid:173)
`marks are marked on discernable objects in the three(cid:173)
`dimensional virtual teeth model as represented in image 111.
`Such basic landmarks may, for example, be points on crowns
`and roots of upper and lower first molars (landmarks 1-4 of 60
`image 111) as well as on crowns and roots of upper and
`lower centrals (landmarks 5-8 in image 111). Landsmarks 1
`and 4 as well as landmarks 5 and 8 mark the approximate
`position of the roots of the teeth. The real root position
`cannot be seen in such a model but the orthodont, based on 65
`his experience, can relatively accurately mark their roots'
`position.
`
`8
`At a next step 120, a cephalometric image of the same
`patient is input and on this image, the same key points are
`then marked (see 131). Then, the two images may be
`matched, which may be by way of super-position as shown
`5 above, which can be represented on a screen, or by any other
`way of mapping of each location in one image lo that of the
`olher image.
`At a next step 140 teeth and jaws in the three-dimensional
`model may be displaced on the three-dimensional model lo
`10 receive a desired result. Then, a represented in the flowchart
`of FIG. 78, the software at next step 150 and 160 moves
`skeletal elements and teeth, respectively, according to move(cid:173)
`mem J>erformcd by the user on the three-dimensional virtual
`teeth model. Then, al 170, a cephalometric analysis can be
`l $ made on the amended (after displacement) ceptbalometric
`image to sec whether desired proportional measurements
`have been reacbcd in such teeth displacement or whether
`any medication should be made.
`'The reverse sequence of operation, namely the mapping
`20 of each point from a cephalomelric imag