as) United States
`a2) Patent Application Publication co) Pub. No.: US 2013/0110469 Al
` Kopelman (43) Pub. Date: May2, 2013
`
`
`
`US 20130110469A1
`
`(54) METHODS AND SYSTEMS FOR CREATING
`AND INTERACTING WITH THREE
`DIMENSIONAL VIRTUAL MODELS
`
`Inventor: Avi Kopelman, Tenalfy, NJ (US)
`(76)
`(21) Appl. No.:
`13/574,723
`
`(22)
`
`PCTFiled:
`
`Jul. 19, 2011
`
`(86) PCT No::
`
`PCT/IL11/00574
`
`§ 371 (©),
`(2), (4) Date:
`
`Jan. 4, 2013
`
`Related U.S. Application Data
`
`(60) Provisional application No. 61/365,556, filed on Jul.
`19, 2010.
`
`Publication Classification
`
`(51)
`
`Int. CL.
`G06T 19/00
`(52) U.S.C
`CPC viccccsccsescrstenssseectenenenees G06T 19/00 (2013.01)
`USPC iececescsseecesessssescnesensssescenssanscneeesensensees 703/1
`
`(2006.01)
`
`(57)
`
`ABSTRACT
`
`Systems and methods are provided for modifying a virtual
`model of a physical structure with additional 3D data
`obtained from the physical structure to provide a modified
`virtual model.
`
`
`
`
`23
`
`
`Align EX1026
`Align v. 3Shape
`IPR2022-00145
`
`Align EX1026
`Align v. 3Shape
`IPR2022-00145
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 1 of 8
`
`US 2013/0110469 Al
`
`Lo
`
`Providing a first three dimensional (3D)virtual model representative
`of a first physical part of a physical structure to a computer system
`
`410
`
`Identifying on said first image at least a first image portion thereof by
`
`420
`
`430
`
`
`
`Displayingafirst image on a display operatively connected to said
`computer system, said first image corresponding to said first 3D
`
`virtual model
`interacting with said display
`
`
`
`
`
`Causing the computer system to modify thefirst 3D virtual model,
`to delete or removeor replacea first virtual model part of thefirst
`3D virtual model correspondingto the first image portion by
`applying a corresponding function(i.e., a delete function ora
`remove function or a replace function) to thefirst image portion via
`
`interaction with said first image on said display, to provide a
`
`modified first 3D virtual model, wherein said first virtual model part
`
`is representativeof a first physical portion of the first physical part
`
`440
`
`Providing a second 3Dvirtual model representative of a second part
`of said physical structure, wherein said second part at least partially
`
`overlaps said first physical portion of said first physical part of said
`physical structure
`
`
`
`
`
`
`
`
`
`
`Registering said second 3D virtual model with respectto said
`modified first 3D virtual model to provide a composite third 3D
`
`
`virtual model wherein said first virtual model part is at least partially
`replaced with a corresponding part of said second 3D virtual model
`
`
`460
`
`450
`
`Outputting said third 3D virtual model from said computer system.
`Fig. 1
`
`470
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 2 of 8
`
`US 2013/0110469 Al
`
`
`
`
`
`
`
`
`Fig. 2
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 3 of 8
`
`US 2013/0110469 Al
`
`Fig.3
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 4 of 8
`
`US 2013/0110469 Al
`
`
`
`31
`
`P2
`
`32
`
`DRM1
`
`RM2
`
`Fig. 5
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 5 of 8
`
`US 2013/0110469 Al
`
`
`
`Fig. 6
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 6 of 8
`
`US 2013/0110469 Al
`
`
`
`Fig. 7
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 7 of 8
`
`US 2013/0110469 Al
`
`
`
`Fig. 8
`
`

`

`Patent Application Publication
`
`May 2, 2013 Sheet 8 of 8
`
`US 2013/0110469 Al
`
`
`
`Providing a first three dimensional (3D) virtual model representative
`of a first part of a physical structure to a computer system
`
`
`
`Displaying a first image on a display operatively connected to said
`computer system, said first image correspondingto said first 3D
`
`virtual model
`
`
`Providing a second 3D virtual model representative of a second part
`of said physical structure, wherein said second part at least partially
`
`overlaps said first part of said physical structure 430'
`
`
`Displaying said second image on the display, said second image
`
`corresponding to the second 3D virtual model, and registering said
`
`
`second 3D virtual model with respect to said modified first 3D
`
`virtual model
` Marking/identifying on said first image at least a first image portion
`
`
`
`thereof by interacting with said display
`
`
`
`
`Causing the computer system to modify the first 3D virtual model, to
`delete or remove orreplacea first virtual model part of the first 3D
`virtual model correspondingto thefirst image portion by applying a
`corresponding function(i.e., a delete function or a remove function or
`a replace function) to the first image portion via interaction with said
`
`first image on said display, to provide a modified first 3D virtual
`model, and concurrently or subsequently replacing saidfirst virtual
`modelpart with a corresponding part of said second 3D virtual
`
`
`model to provide a composite third 3D virtual model
`
`
`
`
`
`
`Outputting said third 3D virtual model from said computer system.
`
`Fig. 9
`
`470!
`
`
`
`
`
`
`
`
`
`
`

`

`US 2013/0110469 Al
`
`May2, 2013
`
`METHODS AND SYSTEMS FOR CREATING
`AND INTERACTING WITH THREE
`DIMENSIONAL VIRTUAL MODELS
`
`FIELD OF THE INVENTION
`
`[0001] This invention relates to computer based methods
`and systems, particularly to computer based methods and
`systemsthat are useful in dentistry and otherfields, and that
`are particularly useful in prosthodontics and/or orthodontics
`in particular.
`
`BACKGROUND OF THE INVENTION
`
`[0002] There are many procedures associated withtheintra
`oral cavity in which a precise three-dimensionalvirtual rep-
`resentation ofthe intra oral cavity can be useful to the dental
`practitioner.
`[0003]
`Such virtual representations (also referred to inter-
`changeably herein as “virtual models”, “computer models”,
`“3D numerical entities”, and so on) enable the practitioner to
`study the intra oral cavity of individual patients via a com-
`puter system, in a similar mannerto the study ofthetradi-
`tional physical plaster model. Furthermore,
`three-dimen-
`sional numericalentities ofthe intra oral cavity also allow the
`practitioner to study methods or approaches when dealing
`with particular dental problems of any given patient, and for
`the design ofphysicalentities, for example prostheses, dental
`brackets, aligners and so on,
`in relation therewith. For
`example, in prosthodontics, a computer model ofa patient’s
`teeth may be manipulated to provide machining data to manu-
`facture a physical model of the intra oral cavity, and/or to
`design and manufacture a coping and/or a prosthesis, while in
`orthodontics a computer model of a patient’s teeth may be
`manipulated to enable a dental appliance,
`including for
`example orthodontic brackets and/oraligners, to be designed
`and manufactured, and/or for designing a treatment.
`[0004] A parameter used in the design and manufacture of
`a dental prosthesis, such as a crown or bridge, is the finish
`line, or transition boundary between the prosthesis and the
`dental preparation, and this needs to be precisely defined in
`three-dimensions. Obtaining finish line coordinates from a
`computer virtual model is more efficient and often more
`accurate than from a plaster cast, and moreover facilitates the
`production of such a prosthesis,
`for example via CNC
`machining, rapid prototyping, or other computerised tech-
`nologies, if desired.
`[0005] However, it is often the case that when scanning the
`intra oral cavity to obtain 3D data ofthe preparation andfinish
`line, on which the virtual model is based, part of the finish
`line, and possibly also the shoulder and other parts of the
`preparation, may be obscuredbysoft tissues such as the gum
`that, no longer being supported by the dental surfaces that
`have been removed, deform to coveratleasta part ofthe finish
`line on the prepared dentalsite.
`[0006] Additionally or alternatively, part or all of the finish
`line may be obscured by other agents or materials, including,
`for example, accumulation of one or moreofsaliva, blood,
`lubricant used with a dentaldrill, debris resulting from work-
`ing the dental site, and so on.
`[0007]
`Similar issues may arise when scanning the intra
`oral cavity to obtain 3D data ofthe position and orientation of
`a dental implantin relation to the surrounding portionsof the
`intra-oral cavity, and in addition the corresponding impres-
`
`sion abutment (also referred to herein as a scanning body)
`maypartially obscure part of the intra-oral cavity.
`[0008]
`Similarly, there are other situations in whicha vir-
`tual model of a physical item, obtained from scanning the
`physical item, is partially obscured or incomplete,or in which
`part of the physical item needs to be subsequently modified
`after obtaining the virtual model. Such situations would con-
`ventionally require a rescanning ofthe entire physical item,
`and this may involve significant additional time, inconve-
`nience and, where the physical item is a part of the body ofa
`patient (such as for example the intra oral cavity), this may
`also involve significant patient discomfort.
`
`SUMMARY OF THE INVENTION
`
`[0009] Herein “intra oral cavity” (also referred to inter-
`changeably herein as dental cavity) is taken to include, but not
`be limited to, one or more real teeth and/or one or more
`prosthetic teeth and/or part of one or more real teeth, of one
`jaw orofbothjawsofa patient, and/or can also includeall the
`real teeth and/or prosthetic teeth in one or both jaws, and/or
`adjacent gingiva and other adjacent objects of the patient,
`and/or can include a physical model or other physical repre-
`sentation of one or more orall the real teeth, and/or one or
`moreor all of the prosthetic teeth, and/or part of one or more
`or all the real teeth, of one jaw or of both jaws, and/or of
`adjacent gingiva and/orother adjacent objects,ofthe patient.
`[0010] Herein, “dental material” refers to any material
`associated with dental structures of the intra oral cavity,
`including but not limited to natural dental materials such as
`for example enamel, dentine, pulp, dental roots, and also
`including non-natural dental materials from which items such
`as for example metallic and non-metallic fillings, restora-
`tions, crowns, bridges, copings, preparations, and so on, are
`made from.
`
`[0011] Herein, “dental clinic” refers to the interface
`between a dental practitioner and a patent, and thus includes
`any physical entity, in particular a clinic, in which there is
`interaction between a dental patient and a dental practitioner.
`While “dental practitioner”typically refers to a dentist, doc-
`tor or dental technician,it also includes herein all other car-
`egivers, including for example dental surgeons, orthodontists,
`prosthodontists, dental assistants or any other caregiver or
`professional that may interact with a dental patient during the
`course of a dental treatment, or that may be involved in
`determining, preparing or providing dental treatment to a
`patient, particularly prosthodontic treatment and/or orth-
`odontic treatment. While “dental patient” (also referred to
`interchangeably herein as “patient”) typically refersto a per-
`son requiring the dental services of a dental practitioner, it
`also includes herein any person regarding whomitis desired
`to create a 3D numerical modeloftheintra oral cavity thereof,
`for example for the purpose of practicing the same or for
`carrying out research.
`[0012] The term “prosthesis” is herein taken to include any
`restoration and any onlays, such as crowns andbridges, for
`example, and inlays, such as caps, for example, or veneering,
`or any otherartificial partial or complete denture.
`[0013] The term “virtual”, applied herein with respect to
`models, manipulation of models, and so on,in particular 3D
`virtual models, for example, refers to being created, simu-
`lated, manipulated, carried out, and so on by means of a
`CAD/CAMsystem, a computer, a computer network, or the
`like, i.e., in a computer environment, particularly with refer-
`ence to digital dentistry.
`
`

`

`US 2013/0110469 Al
`
`May2, 2013
`
`(B) identifying at least a portion of said virtual
`[0022]
`[0014] While the term “preparation”typically refers to the
`modelthat is desired to be modified by interacting with
`stump and includingthefinish line and shoulderthatis left of
`the displayed image,
`the tooth that is to be replaced by the prosthesis—typically a
`crown—and on which the crownor other prosthesis is to be
`[0023]
`(C) modifying said virtual model by replacing in
`mounted or seated,
`the term “preparation” herein also
`the computer system at least said portion ofsaid virtual
`includesartificial stumps, pivots, cores and posts, or other
`model with additional 3D data obtained from the physi-
`devices that may be implantedin the intraoral cavity in such
`cal structure to provide a modified virtual model.
`a position or in a position that is optimal for implanting the
`[0024]
`For example, the physical structure can comprise
`crownorother prosthesis.
`any one of an intra-oral cavity of a patient and a physical
`[0015] The term “prosthodontic procedure” refers, inter
`dental model representative of said intra-oral cavity. Addi-
`alia,
`to any procedure involving the intraoral cavity and
`tionally or alternatively, said virtual model includesafirst
`directed to the design, manufactureorinstallation of a dental
`three dimensional (3D) virtual model representative of a first
`prosthesis at a dental site within the intraoral cavity, or a real
`physical part ofthe physical structure, and step (A) comprises
`or virtual model thereof, or directed to the design and prepa-
`providing to the computer system saidfirst 3D virtual model
`ration of the dental site to receive such a prosthesis.
`and displaying on said display a first display image corre-
`[0016] The term “3D virtual model”is used herein synony-
`sponding to said first 3D virtual model.
`mously with digital model, virtual model, 3D virtual model,
`[0025]
`For example, step (B) comprises identifying on said
`3D model, three dimensional model, 3D numerical entity,
`first display imageat leasta first display image portion thereof
`numerical entity, computer model, 3D computer model,
`by interacting with said display, saidfirst display image por-
`dimensional data, 3D digitized data, 3D representation, and
`tion correspondingto said portion of said virtual model, and
`other such terms, and relates to a virtual representation in a
`said portion of said virtual model being representative of a
`computer environmentofa real object, such as for example a
`first physical portionof saidfirst physical part, and optionally
`dentition or at least a part of the intraoral cavity, or of a real
`step (C) comprises, subsequentto step (B):
`(physical) modelthereof, for example of a plaster model or a
`[0026]
`causing the computer system to at least one of
`stone model of the dentition or any dental structure. In par-
`delete, remove and replace said portion of said virtual
`ticular a virtual dental model is one example of such a 3D
`modelby applying a corresponding function,1.e., a cor-
`virtual model of a dental structure.
`responding computer-implemented function (ie.
`a
`delete function, a remove function or a replace function,
`[0017] The term “physical part” is used herein, synony-
`respectively)to said first display image portionvia inter-
`mously with “real part” to refer to a physical object, in par-
`action with said first display image on said display, to
`ticular a physical dental object having a real (physical) sur-
`provide a modified first 3D virtual model;
`face and included butnot limitedto part or all ofthe dentition
`of the intraoral cavity including dies, a coping, a prosthesis,
`[0027]
`providing said additional 3D data in the form ofa
`and so on, or to a physical dental model ofpart or all of the
`second 3D virtual model representative of a second
`dentition of the intraoral cavity including dies, a coping, a
`physicalpart of said physical structure, whereinaspatial
`prosthesis, and so on.
`disposition of said second physical part with respect to
`said first physical part is known or determinable;
`[0018] The term “scanning”andits analoguesrefer to any
`procedure directed at obtaining 3D topographic data of a
`[0028]
`virtually registering said second 3D virtual model
`surface, particularly of a dental surface, wherein to provide a
`with respect to said modified first 3D virtual model to
`3D virtual model, and thus includes mechanical-based or
`provide said modified virtual model wherein said por-
`other contact systems and methods, typically based on 3D
`tion of said virtual model is replaced with a correspond-
`probes for example, and/or any other non-contact systems
`ing part of said second 3D virtual modelrepresentative
`and methods included but not limited to optical-based sys-
`ofa second physicalportion of said second physicalpart;
`tems and methods and/orradiation-based systems and meth-
`[0029]
`outputting said modified virtual model from said
`computer system.
`ods, including for example confocal-based systems and meth-
`ods, for example as disclosed in WO 00/08415, the contents
`[0030] Alternatively, steps (B) and (C) comprise:
`of which are incorporated herein in their entirety by refer-
`[0031]
`providing said additional 3D data in the form ofa
`ence, x-ray systems and methodsincluding CT systems and
`second 3D virtual model representative of a second
`methods, laser scanners, ultrasound scanners, and/or indeed
`physicalpart of said physical structure, whereinaspatial
`any other suitable system and method for providing 3D vir-
`disposition of said second physical part with respect to
`tual model.
`said first physical part is known or determinable;
`[0032]
`virtually registering said second 3D virtual model
`with respect to said first 3D virtual model and further
`displaying in said display a second display image corre-
`sponding to said second 3D virtual model in registry
`with said first 3D virtual model;
`[0033]
`identifying on said first display imageat least a
`first display image portion thereof by interacting with
`said display, said first display image portion correspond-
`ing to said portion of said virtual model, and said portion
`[0020] According toafirst aspect of the invention there is
`of said virtual model being representative of a first
`provided a computer based method for modifying a virtual
`physical portion of said first physical part;
`model of a physical structure, comprising:
`[0034]
`causing the computer system to at least one of
`[0021]
`(A) displaying an imageofsaid virtual model on
`delete, remove and replace said portion of said virtual
`a display operatively connected to a computer system;
`modelby applying a corresponding function,1.e., a cor-
`
`[0019] The terms “tool” and “machining tool” are taken
`herein interchangeably to include any toolthatis adapted for
`material removal, and may includeinter alia mechanical tools
`such as drills for example, laser tools such as for example
`laser drills or cutters, ultrasonic tools such as for example
`ultrasonic cutters, and so on. Preferably, the machining paths
`and material removal characteristics of such tools can be
`
`finely controlled, typically by computer systems or other
`automated means.
`
`

`

`US 2013/0110469 Al
`
`May2, 2013
`
`responding computer-implementedfunction,i.e., a cor-
`responding computer-implemented function (ie.
`a
`delete function, a remove function or a replace function,
`respectively) to said first display image portionvia inter-
`action with said first display image on said display, to
`provide said modified virtual model, wherein said por-
`tion of said virtual model is replaced with a correspond-
`ing part of said second 3D virtual model representative
`ofa second physical portion of said second physicalpart;
`[0035]
`outputting said modified virtual model from said
`computer system.
`[0036]
`For example, said first display image is visually
`encodedin a different manner to said second display image to
`facilitate identifying said first display image portion
`[0037] Additionally or alternatively, said second physical
`part at least partially overlaps said first physical portion of
`said first physical part of said physical structure to provide
`data on said spatial disposition of said second physical part
`with respect to saidfirst physical part.
`[0038] Additionally or alternatively, said second physical
`portion of said second physical part spatially corresponds to
`but is topographically different from said first physical por-
`tion ofsaid first physical part.
`[0039] Additionally or alternatively, said corresponding
`part of said second 3D virtual model spatially corresponds to
`but is topographically different from said portion of said
`virtual model.
`
`[0040] Additionally or alternatively,saidfirst virtual model
`part represents a corresponding saidfirst physical part of the
`physical structure, wherein said first virtual model part is
`considered to fail to comply with a predetermined require-
`menttherefor. For example, said predetermined requirement
`comprises providing high surface definition of a surface of
`interest in saidfirst physical part ofthe physical structure. For
`example, at least a part of said surface of interest in saidfirst
`portion of said first physical part was obscured whenthe said
`first 3D virtual model was created, and wherein said second
`portion of said second physical part correspondsto saidfirst
`portion ofsaidfirst physical part wherein said at least a part of
`said surface of interest is now unobscured. For example, said
`first portion of said first physical part was obscured with a
`material including one or moreofsaliva, debris, blood, and
`wherein said second portion of said second physical part
`corresponds to said first portion of said first physical part
`wherein said material has been removed from said surface of
`interest. For example, said first portion of said first physical
`part was obscured with an artifact, and wherein said second
`portion of said second physical part correspondsto saidfirst
`portion of said first physical part wherein said artifact has
`been removed. For example, said artifact comprises an
`impression abutment or any other structure mounted onto a
`dental implantthat is embeddedin said physical structure.
`[0041] Additionally or alternatively, said predetermined
`requirement comprises providing a surface ofinterest in said
`first physical part of the physical structure that is configured
`for enabling a prosthesis to be mounted with respect thereto.
`For example, at least a part of said surface of interest in said
`first portion of said first physical part is inadequate for
`enabling a prosthesis to be mounted with respect thereto, and
`wherein said second portion of said second physical part
`corresponds to said first portion of said first physical part
`whereinsaid at least a part of said surface of interest has been
`physically altered. For example, where said at least a part of
`said surface of interest has been physically altered via a
`
`material removal operation wherein to modify the topology of
`said at least a part of said surface of interest. An example of
`this is where said at least a part of said surface of interest
`comprises a finish line of a tooth onto whichit is desired to
`mount the prosthesis, and said finish line is inadequately
`defined in said first portion of said first physical part, but
`where said finish line is adequately defined in said second
`portion of said second physical part.
`[0042]
`For example, said at least part of said surface of
`interest comprises a preparation surface of a preparation of a
`tooth onto whichit is desired to mountthe prosthesis, wherein
`in saidfirst portion of saidfirst physical part said preparation
`surface is inadequate for mounting the prosthesis thereto, and
`wherein in said second portion of said second physical part
`said preparation surface is inadequate for mounting the pros-
`thesis thereto.
`
`[0043] Additionallyor alternatively, saidfirst virtual model
`part is distorted or deficient and fails to properly represent a
`corresponding said first physical part of the physical struc-
`ture.
`
`[0044] Additionally or alternatively, the method further
`comprises manufacturing a physical dental modeloftheintra
`oral cavity based on said modified virtual model.
`[0045] Additionally or alternatively, the method further
`comprises designing an orthodontic treatment plan based on
`said modified virtual model.
`
`[0046] Additionally or alternatively, the method further
`comprises manufacturing a dental aligner based on said
`modified virtual model.
`
`[0047] Additionally or alternatively, the method further
`comprises manufacturing orthodontic appliances based on
`said modified virtual model.
`
`[0048] Additionally or alternatively, the method further
`comprises manufacturing a dental prosthesis based on said
`modified virtual model.
`
`[0049] Accordingto thefirst aspect ofthe invention thereis
`also provided a computer system configured for modifying a
`virtual model of a physical structure, comprising:
`[0050]
`a display configured for displaying a display
`image ofsaid virtual modelandfor identifyingat least a
`portion of said virtual modelthat is desired to be modi-
`fied responsive to interaction of a user with the displayed
`display image, and further configured for modifying
`said virtual modelby replacing in the computer system
`at least said identified portion ofsaid virtual model with
`additional 3D data obtained from the physical structure
`to provide a modified virtual model.
`[0051]
`For example, the computer system can configured
`for applying a computer implemented method as defined
`aboveforthe first aspect of the invention, mutatis mutandis.
`[0052] Accordingto thefirst aspect ofthe invention thereis
`also provided a computer readable medium that embodies in
`a tangible manner a program executable for:
`[0053]
`(A) enabling displaying a display image of a vir-
`tual modelof a physical structure;
`[0054]
`(B) enabling interaction with the displayed dis-
`play imageto identify at least a portion of said virtual
`model that is desired to be modified
`
`(C) enabling modifying said virtual model by
`[0055]
`replacing at least said identified portion of said virtual
`model with additional 3D data obtained from the physi-
`cal structure to provide a modified virtual model.
`[0056]
`For example, the program can be configured for
`applying a computer implemented method as defined above
`
`

`

`US 2013/0110469 Al
`
`May2, 2013
`
`provide said modified virtual model wherein said por-
`for the first aspect of the invention, mutatis mutandis. Addi-
`tion of said virtual model is replaced with a correspond-
`tionally or alternatively, the computer readable medium com-
`ing part of said second 3D virtual modelrepresentative
`prises any one of optical discs, magnetic discs, magnetic
`ofa second physicalportion of said second physicalpart;
`tapes, or solid state memory storage.
`[0057] According to a second aspectof the invention there
`[0070]
`outputting said modified virtual model from said
`is provided a computer based method for modifying a virtual
`computer system.
`model of a physical structure, comprising:
`[0071]
`For example, steps (B) and (C) comprise:
`[0058]
`(A) providing to the computer system said virtual
`[0072]
`providing said additional 3D data in the form of a
`modelobtained from the physical structure;
`second 3D virtual model representative of a second
`[0059]
`(B) providing to the computer system additional
`physicalpart of said physical structure, whereinaspatial
`3D data obtained from at least a part of the physical
`disposition of said second physical part with respect to
`structure;
`said first physical part is known or determinable;
`[0060]
`(C) identifying at least one portion ofthe virtual
`[0073]
`virtually registering said second 3D virtual model
`modelthatis desired to be modified withat least a part of
`with respect to said first 3D virtual model and further
`said additional 3D data; and
`displaying in said display a second display image corre-
`[0061]
`(D) modifying the virtual model in the computer
`sponding to said second 3D virtual model in registry
`with said first 3D virtual model;
`system at least by replacing said at least one identified
`portion of said virtual model with said at least part of
`[0074]
`identifying on said first display imageat least a
`said 3D data to provide a modified virtual model.
`first display image portion thereof by interacting with
`[0062]
`For example, step (A) comprises displaying a dis-
`said display, said first display image portion correspond-
`play image of said virtual model on a display operatively
`ing to said portion of said virtual model, and said portion
`connected to a computer system. For example, step (B) com-
`of said virtual model being representative of a first
`prises displaying a display image of said modified virtual
`physical portion of said first physical part;
`modelon said display. For example, step (C) comprises iden-
`[0075]
`causing the computer system to at least one of
`tifying said at least one portion of the virtual model on said
`delete, remove and replace said portion of said virtual
`display. For example, step (C) comprises interacting with said
`model by applying a corresponding function (ie. a
`display images on said display to thereby replace said corre-
`delete function, a remove function or a replace function,
`sponding portion of said virtual model with said portion of
`respectively)to said first display image portionvia inter-
`said additional 3D data.
`action with said first display image on said display, to
`provide said modified virtual model, wherein said por-
`tion of said virtual model is replaced with a correspond-
`ing part of said second 3D virtual modelrepresentative
`ofa second physicalportion of said second physicalpart;
`[0076]
`outputting said modified virtual model from said
`computer system.
`[0077]
`For example, said first display image is visually
`encodedin a different manner to said second display image to
`facilitate identifying said first display image portion.
`[0078] Additionally or alternatively, said second physical
`part at least partially overlaps said first physical portion of
`said first physical part of said physical structure to provide
`data on said spatial disposition of said second physical part
`with respect to saidfirst physical part.
`[0079] Additionally or alternatively, said second physical
`portion of said second physical part spatially corresponds to
`but is topographically different from said first physical por-
`tion ofsaid first physical part.
`[0080] Additionally or alternatively, corresponding part of
`said second 3D virtual modelspatially corresponds to but is
`topographically different from said portion of said virtual
`model.
`
`For example, said physical structure comprises any
`[0064]
`one of an intra-oral cavity of a patient and a physical dental
`modelrepresentative of said intra-oral cavity.
`[0065] Additionally or alternatively, said virtual model
`includesa first three dimensional (3D) virtual model repre-
`sentative of a first physical part of the physical structure, and
`step (A) comprises providing to the computer system saidfirst
`3D virtual model and displaying on said display a first display
`image corresponding to said first 3D virtual model. For
`example, the method can comprise identifying on saidfirst
`display imageatleasta first display image portion thereof by
`interacting with said display, said first display image portion
`corresponding to said portion of said virtual model, and said
`portion of said virtual model being representative of a first
`physical portion of saidfirst physical part.
`[0066]
`For example, step (C) comprises:
`[0067]
`causing the computer system to at least one of
`delete, remove andreplace said portion of said virtual
`modelby applying a corresponding function,1.e., a cor-
`[0081] Additionallyor alternatively, saidfirst virtual model
`responding computer-implemented function (ie.
`a
`part represents a corresponding said first physical part of the
`delete function, a remove function or a replace function,
`physical structure, wherein said first virtual model part is
`respectively) to said first display image portionvia inter-
`considered to fail to comply with a predetermined require-
`action with said first display image on said display, to
`menttherefor. For example, said predetermined requirement
`provide a modified first 3D virtual model;
`comprises providing high surface definition of a surface of
`[0068]
`providing said additional 3D data in the form ofa
`interest in saidfirst physical part ofthe physical structure. For
`second 3D virtual model representative of a second
`example, at least a part of said surface of interest in saidfirst
`physicalpart of said physical structure, whereinaspatial
`portion of said first physical part was obscured whenthe said
`first 3D virtual model was created, and wherein said second
`disposition of said second physical part with respect to
`said first physical part is known or determinable;
`portion of said second physical part correspondsto saidfirst
`[0069]
`virtually registering said second 3D virtua