US009299192B2
`
`(12) United States Patent
`US 9,299,192 B2
`(10) Patent N0.:
`
`(45) Date of Patent: Mar. 29, 2016
`Kopelman
`
`(54)
`
`METHODS AND SYSTEMS FOR CREATING
`AND INTERACTING WITH THREE
`DIMENSIONAL VIRTUAL MODELS
`
`19/20 (2013.01); G06T2210/41 (2013.01);
`G06T2219/2021 (2013.01)
`(58) Field of Classification Search
`None
`
`(75)
`
`Inventor: Avi Kopelman, Tenalfy, NJ (US)
`
`See application file for complete search history.
`
`(73)
`
`Assignee: ALIGN TECHNOLOGY, INC., San
`Jose, CA (US)
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 108 days.
`
`(21)
`
`Appl. N0.:
`
`13/574,723
`
`(22)
`
`PCT Filed:
`
`Jul. 19, 2011
`
`PCT No.:
`
`PCT/IL2011/000574
`
`§ 371 (C)(1)s
`(2), (4) Date:
`
`Jan. 4, 2013
`
`PCT Pub. No.: WO2012/011101
`
`PCT Pub. Date: Jan. 26, 2012
`
`Prior Publication Data
`
`US 2013/0110469 A1
`
`May 2,2013
`
`Related US. Application Data
`
`Provisional application No. 61/365,556, filed on Jul.
`19, 2010.
`
`Int. Cl.
`
`G06T 19/00
`G06T 19/20
`A61C 13/00
`A61C 7/00
`US. Cl.
`
`(2011.01)
`(2011.01)
`(2006.01)
`(2006.01)
`
`(86)
`
`(87)
`
`(65)
`
`(60)
`
`(51)
`
`(52)
`
`6,099,314 A
`6,334,853 B1
`6,739,869 B1
`7,347,686 B2 *
`7,476,100 B2 *
`7,536,234 B2
`2010/0114351 A1
`
`8/2000 Kopelman et a1.
`1/2002 Kopelman et a1.
`5/2004 Taub et a1.
`........................... 433/3
`3/2008 Marshall
`1/2009 Kuo .................................. 433/6
`5/2009 Kopelman et a1.
`5/2010 Kopelman et a1.
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`W0
`W0
`
`1901235 A1
`WO 00/08415 A1
`WO 2004/087000 A1
`
`3/2008
`2/2000
`10/2004
`
`(Continued)
`OTHER PUBLICATIONS
`
`Chapuis, et al. A new system for computer-aided preoperative plan-
`ning and intraoperative navigation during corrective jaw surgery.
`IEEE Trans Inf Technol Biomed. May 2007;11(3):274-87.
`(Continued)
`
`Primary Examiner 7 Said Broome
`(74) Attorney, Agent, or Firm 7 Wilson, Sonsini, Goodrich
`& Rosati
`
`(57)
`
`ABSTRACT
`
`Systems and methods are provided for modifying a virtual
`model of a physical structure with additional 3D data
`obtained from the physical structure to provide a modified
`virtual model.
`
`CPC ................. G06T19/00 (2013.01);A61C 7/002
`(2013.01); A61C13/0004 (2013.01); G06T
`
`33
`35
`
`32 Claims, 8 Drawing Sheets
`
`M
`
`
`
`1:1
`
`
`\\/r22
`
`l"”////10
`
`a‘RM‘l
`23
`20
`
`
`
`
`
`
` E—“
`
`
`
`16
`
`
`267
`
`
`011
`
`
`
`
`
`3SHAPE EXHIBIT 1001
`SSHAPE EXHIBIT 1001
`3Shape v. Align
`3Shape V. Align
`IPR2019-00134
`IPR2019—00134
`
`

`

`US 9,299,192 B2
`
`Page 2
`
`(56)
`
`W0
`W0
`
`References Cited
`
`OTHER PUBLICATIONS
`
`FOREIGN PATENT DOCUMENTS
`WO 2009/146164 A1
`12/2009
`WO 2010/001401 A1
`1/2010
`
`International search report and written opinion dated Apr. 11, 2014
`for PCT/IL20 1 1/000574.
`.
`.
`* Clted by exammer
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 1 of8
`
`US 9,299,192 B2
`
`Providing a first three dimensional (3D) virtual model representative
`of a first physical part of a physical structure to a computer system
`
`
`
`i
`
`/400
` 410
`
`
`
`Displaying a first image on a display operatively connected to said
`computer system, said first image corresponding to said first 3D
`virtual model
`
`Identifying on said first image at least a first image portion thereof by
`interacting with said display
`
`420
`
`Causing the computer system to modify the first 3D virtual model,
`to delete or remove or replace a first virtual model part of the first
`3D virtual model corresponding to the first image portion by
`applying a corresponding function (Le, a delete function or a
`remove function or a replace function) to the first image portion via
`interaction with said first image on said display, to provide a
`modified first 3D virtual model, wherein said first virtual model part
`is representative of a first physical portion of the first physical part
`
`
`430
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`440
`
`
`
`Providing a second 3D virtual model representative of a second part
`of said physical structure, wherein said second part at least partially
`
`
`overlaps said first physical portion of said first physical part of said
`
`
`physical structure
`
`
`450
`
`
`Registering said second 3D virtual model with respect to said
`modified first 3D virtual model to provide a composite third 3D
`
`
`virtual model wherein said first virtual model part is at least partially
`
`
`replaced with a corresponding part of said second 3D virtual model
`
`
`
`Outputting said third 3D virtual model from said computer system.
` 470
`Fig. 1
`
`460
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 2 of8
`
`US 9,299,192 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`Fig. 2
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 3 of8
`
`US 9,299,192 B2
`
`Fig.3
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 4 of8
`
`US 9,299,192 B2
`
`VMT
`
`DVM1
`
`Fig. 4
`
`31
`
`P2
`
`DRM1
`
`RMZ
`
`Fig. 5
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 5 of8
`
`US 9,299,192 B2
`
`
`
`Fig. 6
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 6 of8
`
`US 9,299,192 B2
`
`
`
`Fig. 7
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 7 of8
`
`US 9,299,192 B2
`
`
`
`Fig. 8
`
`

`

`US. Patent
`
`Mar. 29, 2016
`
`Sheet 8 of8
`
`US 9,299,192 B2
`
`/400'
` Providing a first three dimensional (3D) virtual model representative
`
`of a first part of a physical structure to a computer system
` 410'
`
`Displaying a first image on a display operatively connected to said
`computer system, said first image corresponding to said first 3D
`
`virtual model Providing a second 3D virtual model representative of a second part
`
`
`
`of said physical structure, wherein said second part at least partially
`
`
`
`overlaps said first part of said physical structure
`
`
`
`
`corresponding to the second 3D virtual model, and registering said
`
`second 3D virtual model with respect to said modified first 3D
`
`
` 430'
`
`Displaying said second image on the display, said second image
`
`
`
` 440'
`
`
`
`
`virtual model
`
`
`
`
`Marking/identifying on said first image at least a first image portion
`thereof by interacting with said display
`
` Causing the computer system to modify the first 3D virtual model, to
`
`
`
`
`
`delete or remove or replace a first virtual model part of the first 3D
`virtual model corresponding to the first image portion by applying a
`corresponding function (i.e., a delete function or a remove function or
`a replace function) to the first image portion via interaction with said
`
`
`first image on said display, to provide a modified first 3D virtual
`
`
`model, and concurrently or subsequently replacing said first virtual
`
`
`model part with a corresponding part of said second 3D virtual
`
`
`model to provide a composite third 3D virtual model
`
`
` 460'
`
`
`
`Outputting said third 3D virtual model from said computer system.
`
`

`

`US 9,299,192 B2
`
`1
`METHODS AND SYSTEMS FOR CREATING
`AND INTERACTING WITH THREE
`DIMENSIONAL VIRTUAL MODELS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`The present application is a US. National Phase Applica-
`tion under 35 U.S.C. §371 of International Application No.
`PCT/1L1 1/00574, filed Jul. 19, 2011, entitled “METHODS
`AND SYSTEMS FOR CREATING AND INTERACTING
`WITH THREE DIMENSIONAL VIRTUAL MODELS,”
`which claims priority to US. Provisional Patent Application
`No. 61/365,556, filed Jul. 19, 2010, entitled “METHOD FOR
`CREATING AND INTERACTING WITH THREE DIMEN-
`SIONAL VIRTUAL MODELS USEFUL FOR DENTAL
`
`PROCEDURES,” the entire contents of which are incorpo-
`rated herein by reference.
`
`FIELD OF THE INVENTION
`
`This invention relates to computer based methods and sys-
`tems, particularly to computer based methods and systems
`that are useful in dentistry and other fields, and that are
`particularly useful in prosthodontics and/or orthodontics in
`particular.
`
`BACKGROUND OF THE INVENTION
`
`There are many procedures associated with the intra oral
`caVity in which a precise three-dimensional Virtual represen-
`tation of the intra oral caVity can be useful to the dental
`practitioner.
`Such Virtual representations (also referred to interchange-
`ably herein as “Virtual models”, “computer models”, “3D
`numerical entities”, and so on) enable the practitioner to study
`the intra oral caVity of indiVidual patients Via a computer
`system, in a similar manner to the study of the traditional
`physical plaster model. Furthermore,
`three-dimensional
`numerical entities of the intra oral caVity also allow the prac-
`titioner to study methods or approaches when dealing with
`particular dental problems of any giVen patient, and for the
`design of physical entities, for example prostheses, dental
`brackets, aligners and so on,
`in relation therewith. For
`example, in prosthodontics, a computer model of a patient’s
`teeth may be manipulated to pr0Vide machining data to manu-
`facture a physical model of the intra oral caVity, and/or to
`design and manufacture a coping and/or a prosthesis, while in
`orthodontics a computer model of a patient’s teeth may be
`manipulated to enable a dental appliance,
`including for
`example orthodontic brackets and/or aligners, to be designed
`and manufactured, and/or for designing a treatment.
`A parameter used in the design and manufacture of a dental
`prosthesis, such as a crown or bridge, is the finish line, or
`transition boundary between the prosthesis and the dental
`preparation, and this needs to be precisely defined in three-
`dimensions. Obtaining finish line coordinates from a com-
`puter Virtual model is more efficient and often more accurate
`than from a plaster cast, and more0Ver facilitates the produc-
`tion of such a prosthesis, for example Via CNC machining,
`rapid prototyping, or other computerised technologies, if
`desired.
`
`HoweVer, it is often the case that when scanning the intra
`oral caVity to obtain 3D data ofthe preparation and finish line,
`on which the Virtual model is based, part ofthe finish line, and
`possibly also the shoulder and other parts of the preparation,
`may be obscured by soft tissues such as the gum that, no
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`longer being supported by the dental surfaces that haVe been
`rem0Ved, deform to c0Ver at least a part of the finish line on
`the prepared dental site.
`Additionally or alternatiVely, part or all of the finish line
`may be obscured by other agents or materials, including, for
`example, accumulation of one or more of saliVa, blood, lubri-
`cant used with a dental drill, debris resulting from working
`the dental site, and so on.
`Similar issues may arise when scanning the intra oral caV-
`ity to obtain 3D data ofthe position and orientation of a dental
`implant in relation to the surrounding portions of the intra-
`oral caVity, and in addition the corresponding impression
`abutment (also referred to herein as a scanning body) may
`partially obscure part of the intra-oral caVity.
`Similarly, there are other situations in which a Virtual
`model ofa physical item, obtained from scanning the physical
`item, is partially obscured or incomplete, or in which part of
`the physical item needs to be subsequently modified after
`obtaining the Virtual model. Such situations would conVen-
`tionally require a rescanning of the entire physical item, and
`this may inVolVe significant additional time, inconVenience
`and, where the physical item is a part of the body of a patient
`(such as for example the intra oral caVity), this may also
`inVolVe significant patient discomfort.
`
`SUMMARY OF THE INVENTION
`
`Herein “intra oral caVity” (also referred to interchangeably
`herein as dental caVity) is taken to include, but not be limited
`to, one or more real teeth and/or one or more prosthetic teeth
`and/or part of one or more real teeth, of one jaw or of both
`jaws of a patient, and/or can also include all the real teeth
`and/or prosthetic teeth in one or both jaws, and/or adjacent
`gingiVa and other adjacent objects of the patient, and/or can
`include a physical model or other physical representation of
`one or more or all the real teeth, and/or one or more or all of
`the prosthetic teeth, and/or part of one or more or all the real
`teeth, of one jaw or of both jaws, and/or of adjacent gingiVa
`and/or other adjacent objects, of the patient.
`Herein, “dental material” refers to any material associated
`with dental structures ofthe intra oral caVity, including but not
`limited to natural dental materials such as for example
`enamel, dentine, pulp, dental roots, and also including non-
`natural dental materials from which items such as for example
`metallic and non-metallic fillings,
`restorations, crowns,
`bridges, copings, preparations, and so on, are made from.
`Herein, “dental clinic” refers to the interface between a
`dental practitioner and a patent, and thus includes any physi-
`cal entity, in particular a clinic, in which there is interaction
`between a dental patient and a dental practitioner. While
`“dental practitioner” typically refers to a dentist, doctor or
`dental technician, it also includes herein all other caregiVers,
`including for example dental surgeons, orthodontists, prosth-
`odontists, dental assistants or any other caregiVer or profes-
`sional that may interact with a dental patient during the course
`of a dental treatment, or that may be inVolVed in determining,
`preparing or pr0Viding dental treatment to a patient, particu-
`larly prosthodontic treatment and/or orthodontic treatment.
`While “dental patient” (also referred to interchangeably
`herein as “patient”) typically refers to a person requiring the
`dental serVices of a dental practitioner, it also includes herein
`any person regarding whom it is desired to create a 3D
`numerical model of the intra oral caVity thereof, for example
`for the purpose of practicing the same or for carrying out
`research.
`
`The term “prosthesis” is herein taken to include any resto-
`ration and any onlays, such as crowns and bridges, for
`
`

`

`US 9,299,192 B2
`
`3
`example, and inlays, such as caps, for example, or veneering,
`or any other artificial partial or complete denture.
`The term “Virtual”, applied herein with respect to models,
`manipulation of models, and so on, in particular 3D Virtual
`models, for example, refers to being created, simulated,
`manipulated, carried out, and so on by means of a CAD/CAM
`system, a computer, a computer network, or the like, i.e., in a
`computer environment, particularly with reference to digital
`dentistry.
`While the term “preparation” typically refers to the stump
`and including the finish line and shoulder that is left of the
`tooth that is to be replaced by the prosthesisitypically a
`crowniand on which the crown or other prosthesis is to be
`mounted or seated,
`the term “preparation” herein also
`includes artificial stumps, pivots, cores and posts, or other
`devices that may be implanted in the intraoral cavity in such
`a position or in a position that is optimal for implanting the
`crown or other prosthesis.
`The term “prosthodontic procedure” refers, inter alia, to
`any procedure involving the intraoral cavity and directed to
`the design, manufacture or installation of a dental prosthesis
`at a dental site within the intraoral cavity, or a real or virtual
`model thereof, or directed to the design and preparation ofthe
`dental site to receive such a prosthesis.
`The term “3D virtual model” is used herein synonymously
`with digital model, virtual model, 3D virtual model, 3D
`model,
`three dimensional model, 3D numerical entity,
`numerical entity, computer model, 3D computer model,
`dimensional data, 3D digitized data, 3D representation, and
`other such terms, and relates to a virtual representation in a
`computer environment of a real object, such as for example a
`dentition or at least a part of the intraoral cavity, or of a real
`(physical) model thereof, for example of a plaster model or a
`stone model of the dentition or any dental structure. In par-
`ticular a virtual dental model is one example of such a 3D
`virtual model of a dental structure.
`
`The term “physical part” is used herein, synonymously
`with “real part” to refer to a physical object, in particular a
`physical dental object having a real (physical) surface and
`included but not limited to part or all of the dentition of the
`intraoral cavity including dies, a coping, a prosthesis, and so
`on, or to a physical dental model of part or all of the dentition
`of the intraoral cavity including dies, a coping, a prosthesis,
`and so on.
`
`The term “scanning” and its analogues refer to any proce-
`dure directed at obtaining 3D topographic data of a surface,
`particularly of a dental surface, wherein to provide a 3D
`virtual model, and thus includes mechanical-based or other
`contact systems and methods, typically based on 3D probes
`for example, and/or any other non-contact systems and meth-
`ods included but not limited to optical-based systems and
`methods and/or
`radiation-based systems and methods,
`including for example confocal-based systems and methods,
`for example as disclosed in WO 00/08415, the contents of
`which are incorporated herein in their entirety by reference,
`x-ray systems and methods including CT systems and meth-
`ods, laser scanners, ultrasound scanners, and/or indeed any
`other suitable system and method for providing 3D virtual
`model.
`
`The terms “tool” and “machining tool” are taken herein
`interchangeably to include any tool that is adapted for mate-
`rial removal, and may include inter alia mechanical tools such
`as drills for example, laser tools such as for example laser
`drills or cutters, ultrasonic tools such as for example ultra-
`sonic cutters, and so on. Preferably, the machining paths and
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`material removal characteristics of such tools can be finely
`controlled, typically by computer systems or other automated
`means.
`
`According to a first aspect ofthe invention there is provided
`a computer based method for modifying a virtual model of a
`physical structure, comprising:
`(A) displaying an image of said virtual model on a display
`operatively connected to a computer system;
`(B) identifying at least a portion of said virtual model that
`is desired to be modified by interacting with the dis-
`played image,
`(C) modifying said virtual model by replacing in the com-
`puter system at least said portion of said virtual model
`with additional 3D data obtained from the physical
`structure to provide a modified virtual model.
`For example, the physical structure can comprise any one
`of an intra-oral cavity of a patient and a physical dental model
`representative of said intra-oral cavity. Additionally or alter-
`natively, said virtual model includes a first three dimensional
`(3D) virtual model representative of a first physical part ofthe
`physical structure, and step (A) comprises providing to the
`computer system said first 3D virtual model and displaying
`on said display a first display image corresponding to said first
`3D virtual model.
`
`For example, step (B) comprises identifying on said first
`display image at least a first display image portion thereof by
`interacting with said display, said first display image portion
`corresponding to said portion of said virtual model, and said
`portion of said virtual model being representative of a first
`physical portion of said first physical part, and optionally step
`(C) comprises, subsequent to step (B):
`causing the computer system to at least one of delete,
`remove and replace said portion of said virtual model by
`applying a corresponding function, i.e., a corresponding
`computer-implemented function (i.e. a delete function, a
`remove function or a replace function, respectively) to
`said first display image portion via interaction with said
`first display image on said display, to provide a modified
`first 3D virtual model;
`providing said additional 3D data in the form of a second
`3D virtual model representative of a second physical
`part of said physical structure, wherein a spatial dispo-
`sition of said second physical part with respect to said
`first physical part is known or determinable;
`virtually registering said second 3D virtual model with
`respect to said modified first 3D virtual model to provide
`said modified virtual model wherein said portion of said
`virtual model is replaced with a corresponding part of
`said second 3D virtual model representative of a second
`physical portion of said second physical part;
`outputting said modified virtual model from said computer
`system.
`Alternatively, steps (B) and (C) comprise:
`providing said additional 3D data in the form of a second
`3D virtual model representative of a second physical
`part of said physical structure, wherein a spatial dispo-
`sition of said second physical part with respect to said
`first physical part is known or determinable;
`virtually registering said second 3D virtual model with
`respect to said first 3D virtual model and further display-
`ing in said display a second display image correspond-
`ing to said second 3D virtual model in registry with said
`first 3D virtual model;
`identifying on said first display image at least a first display
`image portion thereof by interacting with said display,
`said first display image portion corresponding to said
`portion of said virtual model, and said portion of said
`
`

`

`US 9,299,192 B2
`
`5
`virtual model being representative of a first physical
`portion of said first physical part;
`causing the computer system to at least one of delete,
`remove and replace said portion of said Virtual model by
`applying a corresponding function, i.e., a corresponding
`computer-implemented function, i.e., a corresponding
`computer-implemented function (i.e. a delete function, a
`remove function or a replace function, respectively) to
`said first display image portion via interaction with said
`first display image on said display, to provide said modi-
`fied virtual model, wherein said portion of said virtual
`model is replaced with a corresponding part of said
`second 3D virtual model representative of a second
`physical portion of said second physical part;
`outputting said modified virtual model from said computer
`system.
`For example, said first display image is visually encoded in
`a different manner to said second display image to facilitate
`identifying said first display image portion
`Additionally or alternatively, said second physical part at
`least partially overlaps said first physical portion of said first
`physical part of said physical structure to provide data on said
`spatial disposition of said second physical part with respect to
`said first physical part.
`Additionally or alternatively, said second physical portion
`of said second physical part spatially corresponds to but is
`topographically different from said first physical portion of
`said first physical part.
`Additionally or alternatively, said corresponding part of
`said second 3D virtual model spatially corresponds to but is
`topographically different from said portion of said virtual
`model.
`
`Additionally or alternatively, said first virtual model part
`represents a corresponding said first physical part of the
`physical structure, wherein said first virtual model part is
`considered to fail to comply with a predetermined require-
`ment therefor. For example, said predetermined requirement
`comprises providing high surface definition of a surface of
`interest in said first physical part ofthe physical structure. For
`example, at least a part of said surface of interest in said first
`portion of said first physical part was obscured when the said
`first 3D virtual model was created, and wherein said second
`portion of said second physical part corresponds to said first
`portion of said first physical part wherein said at least a part of
`said surface of interest is now unobscured. For example, said
`first portion of said first physical part was obscured with a
`material including one or more of saliva, debris, blood, and
`wherein said second portion of said second physical part
`corresponds to said first portion of said first physical part
`wherein said material has been removed from said surface of
`
`interest. For example, said first portion of said first physical
`part was obscured with an artifact, and wherein said second
`portion of said second physical part corresponds to said first
`portion of said first physical part wherein said artifact has
`been removed. For example, said artifact comprises an
`impression abutment or any other structure mounted onto a
`dental implant that is embedded in said physical structure.
`Additionally or alternatively, said predetermined require-
`ment comprises providing a surface of interest in said first
`physical part of the physical structure that is configured for
`enabling a prosthesis to be mounted with respect thereto. For
`example, at least a part of said surface of interest in said first
`portion of said first physical part is inadequate for enabling a
`prosthesis to be mounted with respect thereto, and wherein
`said second portion of said second physical part corresponds
`to said first portion of said first physical part wherein said at
`least a part of said surface of interest has been physically
`
`6
`altered. For example, where said at least a part of said surface
`of interest has been physically altered via a material removal
`operation wherein to modify the topology of said at least a
`part of said surface of interest. An example of this is where
`said at least a part of said surface of interest comprises a finish
`line of a tooth onto which it is desired to mount the prosthesis,
`and said finish line is inadequately defined in said first portion
`of said first physical part, but where said finish line is
`adequately defined in said second portion of said second
`physical part.
`For example, said at least part of said surface of interest
`comprises a preparation surface of a preparation of a tooth
`onto which it is desired to mount the prosthesis, wherein in
`said first portion of said first physical part said preparation
`surface is inadequate for mounting the prosthesis thereto, and
`wherein in said second portion of said second physical part
`said preparation surface is inadequate for mounting the pros-
`thesis thereto.
`
`Additionally or alternatively, said first virtual model part is
`distorted or deficient and fails to properly represent a corre-
`sponding said first physical part of the physical structure.
`Additionally or alternatively, the method further comprises
`manufacturing a physical dental model ofthe intra oral cavity
`based on said modified virtual model.
`
`Additionally or alternatively, the method further comprises
`designing an orthodontic treatment plan based on said modi-
`fied virtual model.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`Additionally or alternatively, the method further comprises
`manufacturing a dental aligner based on said modified virtual
`model.
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Additionally or alternatively, the method further comprises
`manufacturing orthodontic appliances based on said modified
`virtual model.
`
`Additionally or alternatively, the method further comprises
`manufacturing a dental prosthesis based on said modified
`virtual model.
`
`According to the first aspect of the invention there is also
`provided a computer system configured for modifying a vir-
`tual model of a physical structure, comprising:
`a display configured for displaying a display image of said
`virtual model and for identifying at least a portion of said
`virtual model that is desired to be modified responsive to
`interaction of a user with the displayed display image,
`and further configured for modifying said virtual model
`by replacing in the computer system at least said iden-
`tified portion of said virtual model with additional 3D
`data obtained from the physical structure to provide a
`modified virtual model.
`
`the computer system can configured for
`For example,
`applying a computer implemented method as defined above
`for the first aspect of the invention, mutatis mutandis.
`According to the first aspect of the invention there is also
`provided a computer readable medium that embodies in a
`tangible manner a program executable for:
`(A) enabling displaying a display image of a virtual model
`of a physical structure;
`(B) enabling interaction with the displayed display image
`to identify at least a portion of said virtual model that is
`desired to be modified
`
`(C) enabling modifying said virtual model by replacing at
`least said identified portion of said virtual model with
`additional 3D data obtained from the physical structure
`to provide a modified virtual model.
`For example, the program can be configured for applying a
`computer implemented method as defined above for the first
`aspect of the invention, mutatis mutandis. Additionally or
`
`

`

`US 9,299,192 B2
`
`7
`alternatively, the computer readable medium comprises any
`one of optical discs, magnetic discs, magnetic tapes, or solid
`state memory storage.
`According to a second aspect of the invention there is
`provided a computer based method for modifying a virtual
`model of a physical structure, comprising:
`(A) providing to the computer system said virtual model
`obtained from the physical structure;
`(B) providing to the computer system additional 3D data
`obtained from at least a part of the physical structure;
`(C) identifying at least one portion ofthe virtual model that
`is desired to be modified with at least a part of said
`additional 3D data; and
`(D) modifying the virtual model in the computer system at
`least by replacing said at least one identified portion of
`said virtual model with said at least part of said 3D data
`to provide a modified virtual model.
`For example, step (A) comprises displaying a display
`image of said virtual model on a display operatively con-
`nected to a computer system. For example, step (B) comprises
`displaying a display image of said modified virtual model on
`said display. For example, step (C) comprises identifying said
`at least one portion of the virtual model on said display. For
`example, step (C) comprises interacting with said display
`images on said display to thereby replace said corresponding
`portion of said virtual model with said portion of said addi-
`tional 3D data.
`
`Additionally or alternatively, said additional 3D data
`includes at least one portion thereof that corresponds to but is
`different from a corresponding portion of said virtual model.
`For example, said physical structure comprises any one of
`an intra-oral cavity of a patient and a physical dental model
`representative of said intra-oral cavity.
`Additionally or alternatively, said virtual model includes a
`first three dimensional (3D) virtual model representative of a
`first physical part of the physical structure, and step (A)
`comprises providing to the computer system said first 3D
`virtual model and displaying on said display a first display
`image corresponding to said first 3D virtual model. For
`example, the method can comprise identifying on said first
`display image at least a first display image portion thereof by
`interacting with said display, said first display image portion
`corresponding to said portion of said virtual model, and said
`portion of said virtual model being representative of a first
`physical portion of said first physical part.
`For example, step (C) comprises:
`causing the computer system to at least one of delete,
`remove and replace said portion of said virtual model by
`applying a corresponding function, i.e., a corresponding
`computer-implemented function (i .e. a delete function, a
`remove function or a replace function, respectively) to
`said first display image portion via interaction with said
`first display image on said display, to provide a modified
`first 3D virtual model;
`providing said additional 3D data in the form of a second
`3D virtual model representative of a second physical
`part of said physical structure, wherein a spatial dispo-
`sition of said second physical part with respect to said
`first physical part is known or determinable;
`virtually registering said second 3D virtual model with
`respect to said modified first 3D virtual model to provide
`said modified virtual model wherein said portion of said
`virtual model is replaced with a corresponding part of
`said second 3D virtual model representative of a second
`physical portion of said second physical part;
`outputting said modified virtual model from said computer
`system.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`For example, steps (B) and (C) comprise:
`providing said additional 3D data in the form of a second
`3D virtual model representative of a second physical
`part of said physical structure, wherein a spatial dispo-
`sition of said second physical part with respect to said
`first physical part is known or determinable;
`virtually registering said second 3D virtual model with
`respect to said first 3D virtual model and further display-
`ing in said display a second display image correspond-
`ing to said second 3D virtual model in registry with said
`first 3D virtual model;
`identifying on said first display image at least a first display
`image portion thereof by interacting with said display,
`said first display image portion corresponding to said
`portion of said virtual model, and said portion of said
`virtual model being representative of a first physical
`portion of said first p