( 58 )
`
`( 56 )
`
`CN
`CN
`
`( 12 ) United States Patent
`Esbech et al .
`
`( 54 ) FOCUS SCANNING APPARATUS
`RECORDING COLOR
`( 71 ) Applicant : 3SHAPE A / S , Copenhagen K ( DK )
`( 72 )
`Inventors : Bo Esbech , Gentofte ( DK ) ; Christian
`Romer Rosberg , Bronshoj ( DK ) ; Mike
`Van Der Poel , Rodovre ( DK ) ; Rasmus
`Kjaer , Copenhagen ( DK ) ; Michael
`Vinther , Copenhagen ( DK ) ; Karl - Josef
`Hollenbeck , Copenhagen ( DK )
`( 73 ) Assignee : 3SHAPE A / S , Copenhagen K ( DK )
`( * ) Notice :
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U . S . C . 154 ( b ) by 223 days .
`( 21 ) Appl . No . :
`14 / 764 , 087
`Feb . 13 , 2014
`( 22 ) PCT Filed :
`PCT / EP2014 / 052842
`( 86 ) PCT No . :
`$ 371 ( c ) ( 1 ) ,
`Jul . 28 , 2015
`( 2 ) Date :
`( 87 ) PCT Pub . No . : WO2014 / 125037
`PCT Pub . Date : Aug . 21 , 2014
`Prior Publication Data
`US 2016 / 0022389 A1
`Jan . 28 , 2016
`Related U . S . Application Data
`( 60 ) Provisional application No . 61 / 764 , 178 , filed on Feb .
`13 , 2013
`Foreign Application Priority Data
`( 30 )
`Feb . 13 , 2013 ( DK ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2013 70077
`( 51 )
`Int . CI .
`HO1J 40 / 14
`A61C 9 / 00
`
`( 65 )
`
`( 2006 . 01 )
`( 2006 . 01 )
`( Continued )
`
`THIRUTTUNUTMUTTU
`
`US009962244B2
`
`US 9 , 962 , 244 B2
`May 8 , 2018
`
`( 10 ) Patent No . :
`( 45 ) Date of Patent :
`( 52 ) U . S . CI .
`CPC . . . . . . . . . . . . A61C 9 / 0073 ( 2013 . 01 ) ; A61C 9 / 006
`( 2013 . 01 ) ; A61C 9 / 0066 ( 2013 . 01 ) ; GOIB
`11 / 24 ( 2013 . 01 ) ;
`
`( Continued )
`Field of Classification Search
`CPC . . GO1B 11 / 24 ; GOIB 11 / 2509 ; GOIB 11 / 2518
`( Continued )
`References Cited
`U . S . PATENT DOCUMENTS
`4 / 2010 Babayoff
`7 , 698 , 068 B2
`1 / 2012 Babayoff
`8 , 102 , 538 B2
`( Continued )
`FOREIGN PATENT DOCUMENTS
`102008282 A
`4 / 2011
`102112845 A
`6 / 2011
`( Continued )
`OTHER PUBLICATIONS
`The First Office Action dated Aug . 2 , 2016 , by the State Intellectual
`Property Office of People ' s Republic of China in corresponding
`Chinese Patent Application No . 201480020976 . 3 , and an English
`Translation of the Office Action . ( 18 pages ) .
`( Continued )
`Primary Examiner — Kevin Pyo
`( 74 ) Attorney , Agent , or Firm — Buchanan Ingersoll &
`Rooney P . C .
`ABSTRACT
`( 57 )
`Disclosed are a scanner system and a method for recording
`surface geometry and surface color of an object where both
`surface geometry information and surface color information
`for a block of said image sensor pixels at least partly from
`one 2D image recorded by said color image sensor .
`35 Claims , 4 Drawing Sheets
`
`wwwwwwwwwwwwww
`
`180 . . . commons
`1000 mm
`
`101
`
`50
`
`we
`
`130
`
`140
`
`*
`
`*
`
`*
`
`100
`
`151
`
`w
`
`mwa w ma w swmv
`
`wwww
`
`+
`
`+
`
`r . 200
`
`Align Ex. 1001
`U.S. Patent No. 9,962,244
`
`0001
`
`

`

`US 9 , 962 , 244 B2
`Page 2
`
`2013 / 0236850 A1 9 / 2013 Wu et al .
`2014 / 0022356 A11 / 2014 Fisker et al .
`2014 / 0146142 Al
`5 / 2014 Duret et al .
`FOREIGN PATENT DOCUMENTS
`102402799 A
`4 / 2012
`102802520 A
`11 / 2012
`2 241 248 A2
`10 / 2010
`1 / 2004
`2004 - 029373 A
`2007117152 A
`5 / 2007
`2009109263 A
`5 / 2009
`WO 2010 / 145669 Al
`12 / 2010
`1 / 2012
`2012 / 007003 Al
`WO 2012 / 083967 A1
`6 / 2012
`WO 2013 / 008097 Al
`1 / 2013
`
`CN
`CN
`EP
`JP
`JP
`JP
`wo
`WO
`WO
`WO
`
`OTHER PUBLICATIONS
`The First Chinese Search dated Jul . 25 , 2016 , by the State Intel
`lectual Property Office of People ' s Republic of China in corre
`sponding Chinese Patent Application No . 201480020976 . 3 . ( 2
`pages ) .
`International Search Report ( PCT / ISA / 210 ) dated Jul . 7 , 2014 , by
`the European Patent Office as the International Searching Authority
`for International Application No . PCT / EP2014 / 052842 .
`Office Action ( Notice of Reasons for Rejection ) dated Jan . 9 , 2018 ,
`by the Japanese Patent Office in Japanese Patent Application No .
`2015 - 557430 , and an English Translation of the Office Action . ( 8
`pages ) .
`* cited by examiner
`
`( 58 )
`
`( 56 )
`
`( 51 )
`
`( 52 )
`
`Int . CI .
`GOIB 11 / 25
`( 2006 . 01 )
`( 2006 . 01 )
`GO1J 3 / 51
`GO1J 3 / 02
`( 2006 . 01 )
`GO1J 3 / 50
`( 2006 . 01 )
`( 2006 . 01 )
`GOIB 11 / 24
`U . S . CI .
`CPC . . . GOIB 11 / 2509 ( 2013 . 01 ) ; GOIB 11 / 2513
`( 2013 . 01 ) ; GOIB 11 / 2518 ( 2013 . 01 ) ; G01J
`3 / 0208 ( 2013 . 01 ) ; G01J 3 / 0224 ( 2013 . 01 ) ;
`GO1J 3 / 0237 ( 2013 . 01 ) ; G01J 3 / 0278
`( 2013 . 01 ) ; GO1J 3 / 508 ( 2013 . 01 ) ; G01J 3 / 51
`( 2013 . 01 ) ; G01J 3 / 513 ( 2013 . 01 )
`Field of Classification Search
`USPC . . . . . . . . . . . . . . . . .
`. . . . . . . . . . . . . . 250 / 226 , 234 ; 348 / 47
`USPC
`See application file for complete search history .
`References Cited
`U . S . PATENT DOCUMENTS
`9 , 212 , 898 B2 * 12 / 2015 Banyay . . . . . . . . . . . . . . GO2B 21 / 0072
`9 , 456 , 754 B2 * 10 / 2016 Kocherscheidt . . . . . A61B 5 / 0088
`2005 / 0285027 Al 12 / 2005 Favalora et al .
`2010 / 0145898 A1 6 / 2010 Malfliet et al .
`2011 / 0134225 Al
`6 / 2011 Saint - Pierre et al .
`2011 / 0221880 A1 9 / 2011 Liang et al .
`2012 / 0062716 Al 3 / 2012 Dillon et al .
`2012 / 0075425 A1 3 / 2012 Thiel
`2012 / 0092461 A1
`4 / 2012 Fisker et al .
`2012 / 0140243 Al 6 / 2012 Colonna de Lega
`
`0002
`
`

`

`U . S . Patent
`
`May 8 , 2018
`
`Sheet 1 of 4
`
`US 9 , 962 , 244 B2
`
`go
`
`X
`
`o
`
`X
`
`2000 :
`
`170
`160
`
`150
`
`200
`
`*
`
`* W WW
`
`* * * * * * * * * *
`
`WWWWWW
`
`*
`
`* * * *
`
`* XXVN
`
`Y
`
`*
`
`n
`
`Saman wartawan www . me
`RYTYY
`
`KWAMWAMY " W9
`
`1902
`
`*
`
`* * * *
`
`* *
`
`. *
`
`w
`
`ww
`
`w
`
`*
`
`* * * i
`
`
`
`00L win
`
`140
`130
`
`
`
`WWWWWWWWWWMWMN WWWMWWWW
`
`San
`
`~ 000T
`
`*
`
`* * * *
`*
`
`*
`
`101
`
`0811
`
`Anak
`
`wanne wa ainan
`
`man
`
`who
`
`was
`
`inten
`
`s
`
`e wat die
`
`man
`
`Fig . 1
`
`0003
`
`

`

`U . S . Patent
`
`May 8 , 2018
`
`Sheet 2 of 4
`
`US 9 , 962 , 244 B2
`
`. .
`
`Fig . 2A
`
`11111
`11
`
`Fig . 2B
`
`????
`
`??
`
`. .
`
`. . . .
`
`19
`
`1 - 10 1919
`
`. >
`
`6 : 10
`Q .
`Fig . 3A
`
`CCCCCCCC . CCCCCCCCCCCCCCCCC
`
`1
`
`099
`
`DIA
`
`Fig . 3B
`
`GGGGGG
`GLUCORIC
`QOGGIO
`GBOGBG
`???
`Fig . 4
`
`0 0 CO999
`
`0004
`
`

`

`atent
`
`May 8 , 2018
`
`Sheet 3 of 4
`
`US 9 , 962 , 244 B2
`
`541
`
`Obtain scanner system obtained
`
`542
`
`Illuminating obejct surface w .
`multichromatic probe light
`
`543
`
`Capturing a series of 2D images of
`said object
`
`544
`
`Derived geometry and color
`information
`
`545
`
`Generate a sub - scan of the object
`
`546
`
`Generate a colored digital 3D
`representation of the object from
`several sub - scans
`
`547
`
`Fig . 5
`
`0005
`
`

`

`U . S . Patent
`
`May 8 , 2018
`
`Sheet 4 of 4
`
`US 9 , 962 , 244 B2
`
`662
`
`663
`
`661
`
`Fig . 6A
`
`mondo . . . . . . .
`
`. . . . . . .
`
`Correlation
`measure
`
`665
`
`N N
`
`664
`
`664
`
`Fig . 6B
`
`R - value
`
`666 -
`
`667
`
`mmmmmmmmmmmmmmmmmmmmmmm
`
`Fig . 6C
`
`- 666
`
`2
`
`0006
`
`

`

`US 9 , 962 , 244 B2
`
`5
`
`FOCUS SCANNING APPARATUS
`RECORDING COLOR
`FIELD OF THE APPLICATION
`The application relates to three dimensional ( 3D ) scan
`ning of the surface geometry and surface color of objects . A
`particular application is within dentistry , particularly for
`intraoral scanning .
`
`One aspect of this application is to provide a scanner
`system for recording surface geometry and surface color of
`an object , and wherein all 2D images are captured using the
`same color image sensor .
`One aspect of this application is to provide a scanner
`system and a method for recording surface geometry and
`surface color of an object , in which the information relating
`to the surface geometry and to the surface color are acquired
`simultaneously such that an alignment of data relating to the
`10 recorded surface geometry and data relating to the recorded
`surface color is not required in order to generate a digital 3D
`BACKGROUND
`representation of the object expressing both color and geom
`etry of the object .
`3D scanners are widely known from the art , and so are
`Disclosed is a scanner system for recording surface geom
`intraoral dental 3D scanners ( e . g . , Sirona Cerec , Cadent 15
`etry and surface color of an object , the scanner system
`Itero , 3Shape TRIOS ) .
`comprising :
`The ability to record surface color is useful in many
`a multichromatic light source configured for providing a
`applications . For example in dentistry , the user can differ
`multichromatic probe light for illumination of the
`entiate types of tissue or detect existing restorations . For
`object ,
`a color image sensor comprising an array of image sensor
`example in materials inspection , the user can detect surface 20
`abnormalities such as crystallization defects or discoloring .
`pixels for capturing one or more 2D images of light
`None of the above is generally possible from surface geom
`received from said object , and
`etry information alone .
`a data processing system configured for deriving both
`WO2010145669 mentions the possibility of recording
`surface geometry information and surface color infor
`mation for a block of said image sensor pixels at least
`color . In particular , several sequential images , each taken for 25
`partly from one 2D image recorded by said color image
`an illumination in a different color - typically blue , green ,
`and red — are combined to form a synthetic color image . This
`sensor .
`Disclosed is a method of recording surface geometry and
`approach hence requires means to change light source color ,
`surface color of an object , the method comprising :
`such as color filters . Furthermore , in handheld use , the
`obtaining a scanner system comprising a multichromatic
`scanner will move relative to the scanned object during the 30
`light source and a color image sensor comprising an
`illumination sequence , reducing the quality of the synthetic
`array of image sensor pixels ;
`color image .
`illuminating the surface of said object with multichro
`Also U . S . Pat . No . 7 , 698 , 068 and U . S . Pat . No . 8 , 102 , 538
`matic probe light from said multichromatic light
`( Cadent Inc . ) describe an intraoral scanner that records both
`source ;
`geometry data and texture data with one or more image
`capturing a series of 2D images of said object using said
`sensor ( s ) . However , there is a slight delay between the color
`color image sensor ; and
`and the geometry recording , respectively . U . S . Pat . No .
`deriving both surface geometry information and surface
`7 , 698 , 068 requires sequential illumination in different colors
`color information for a block of said image sensor
`to form a synthetic image , while U . S . Pat . No . 8 , 102 , 538 40
`pixels at least partly from one captured 2D image .
`mentions white light as a possibility , however from a second
`In the context of the present application , the phrase
`" surface color ” may refer to the apparent color of an object
`illumination source or recorded by a second image sensor ,
`the first set being used for recording the geometry .
`surface and thus in some cases , such as for semi - transparent
`or semi - translucent objects such as teeth , be caused by light
`WO2012083967 discloses a scanner for recording geom -
`etry data and texture data with two separate cameras . While 45 from the object surface and / or the material below the object
`the first camera has a relatively shallow depth of field as to
`surface , such as material immediately below the object
`provide focus scanning based on multiple images , the sec
`surface .
`ond camera has a relatively large depth of field as to provide
`in the context of the present application , the phrase
`color texture information from a single image .
`" derived at least partly from one 2D image ” refers to the
`Color - recording scanning confocal microscopes are also 50 situation where the surface geometry information for a given
`known from the prior art ( e . g . , Keyence VK9700 ; see also
`block of image sensor pixels at least in part is derived from
`JP2004029373 ) . A white light illumination system along
`one 2D image and where the corresponding surface color
`with a color image sensor is used for recording 2D texture ,
`information at least in part is derived from the same 2D
`while a laser beam forms a dot that is scanned , i . e . , moved
`image . The phase also covers cases where the surface
`over the surface and recorded by a photomultiplier , provid - 55 geometry information for a given block of image sensor
`ing the geometry data from many depth measurements , one
`pixels at least in part is derived from a plurality of 2D images
`for each position of the dot . The principle of a moving dot
`of a series of captured 2D images and where the correspond
`requires the measured object not to move relative to the
`ing surface color information at least in part is derived from
`microscope during measurement , and hence is not suitable
`the same 2D images of that series of captured 2D images .
`60 An advantage of deriving both surface geometry infor
`for handheld use .
`mation and surface color information for a block of said
`image sensor pixels at least partly from one 2D image is that
`a scanner system having only one image sensor can be
`realized .
`It is an advantage that the surface geometry information
`and the surface color information are derived at least partly
`from one 2D image , since this inherently provides that the
`
`SUMMARY
`One aspect of this application is to provide a scanner
`system and a method for recording surface geometry and 65
`surface color of an object , and where surface geometry and
`surface color are derived from the same captured 2D images
`
`0007
`
`

`

`US 9 , 962 , 244 B2
`two types of information are acquired simultaneously . There
`For a given block of image sensor pixels the correspond
`is hence no requirement for an exact timing of the operation
`ing portions of the captured 2D images in the stack may be
`of two color image sensors , which may the case when one
`analyzed to derive the surface geometry information and
`image sensor is used for the geometry recording and another
`surface color information for that block .
`for color recording . Equally there is no need for an elaborate 5
`In some embodiments , the surface geometry information
`calculation accounting for significant differences in the
`relates to where the object surface is located relative to the
`scanner system coordinate system for that particular block of
`timing of capturing of 2D images from which the surface
`image sensor pixels .
`geometry information is derived and the timing of the
`One advantage of the scanner system and the method of
`capturing of 2D images from which the surface color
`10 the current application is that the informations used for
`information is derived .
`generating the sub - scan expressing both geometry and color
`The present application discloses is a significant improve
`of the object ( as seen from one view ) are obtained concur
`ment over the state of the art in that only a single image
`rently .
`sensor and a single multichromatic light source is required ,
`Sub - scans can be generated for a number of different
`and that surface color and surface geometry for at least a part 15 views of the object such that they together cover the part of
`of the object can be derived from the same 2D image or 2D
`the surface .
`images , which also means that alignment of color and
`in some embodiments , the data processing system is
`configured for combining a number of sub - scans to generate
`surface geometry is inherently perfect . In the scanner system
`according to the present application , there is no need for
`a digital 3D representation of the object . The digital 3D
`taking into account or compensating for relative motion of 20 representation of the object then preferably expresses both
`the object and scanner system between obtaining surface
`the recorded geometry and color of the object .
`geometry and surface color . Since the surface geometry and
`The digital 3D representation of the object can be in the
`the surface color are obtained at precisely the same time , the
`form of a data file . When the object is a patient ' s set of teeth
`scanner system automatically maintains its spatial disposi
`the digital 3D representation of this set of teeth can e . g . be
`tion with respect to the object surface while obtaining the 25 used for CAD / CAM manufacture of a physical model of the
`surface geometry and the surface color . This makes the
`patient ' s set teeth .
`scanner system of the present application suitable for hand -
`The surface geometry and the surface color are both
`held use , for example as an intraoral scanner , or for scanning
`determined from light recorded by the color image sensor .
`moving objects .
`In some embodiments , the light received from the object
`In some embodiments , the data processing system is 30 originates from the multichromatic light source , i . e . it is
`configured for deriving surface geometry information and
`probe light reflected or scattered from the surface of the
`surface color information for said block of image sensor
`object .
`pixels from a series of 2D images , such as from a plurality
`In some embodiments , the light received form the object
`of the 2D images in a series of captured 2D images . I . e . the
`comprises fluorescence excited by the probe light from the
`data processing system is capable of analyzing a plurality of 35 multichromatic light source , i . e . fluorescence emitted by
`the 2D images in a series of captured 2D images in order to
`fluorescent materials in the object surface .
`derive the surface geometry information for a block of
`In some embodiments , a second light source is used for
`image sensor pixels and to also derive surface color infor -
`the excitation of fluorescence while the multichromatic light
`mation from at least one of the 2D images from which the
`source provides the light for obtaining the geometry and
`surface geometry information is derived .
`40 color of the object .
`In some embodiments , the data processing system is
`The scanner system preferably comprises an optical sys
`configured for deriving surface color information from a
`tem configured for guiding light emitted by the multichro
`plurality of 2D images of a series of captured 2D images and
`matic light source towards the object to be scanned and for
`for deriving surface geometry information from at least one
`guiding light received from the object to the color image
`of the 2D images from which the surface color information 45 sensor such that the 2D images of said object can be
`captured by said color image sensor .
`is derived .
`In some embodiments , the data processing system is
`In some embodiments , the scanner system comprises a
`configured for deriving surface geometry information from
`first optical system , such as an arrangement of lenses , for
`a plurality of 2D images of a series of captured 2D images
`transmitting the probe light from the multichromatic light
`and for deriving surface color information from at least one 50 source towards an object and a second optical system for
`of the 2D images from which the surface geometry infor -
`imaging light received from the object at the color image
`mation is derived .
`sensor .
`In some embodiments , the set of 2D images from which
`In some embodiments , single optical system images the
`surface color information is derived from is identical to the
`probe light onto the object and images the object , or at least
`set of 2D images from which surface geometry information 55 a part of the object , onto the color image sensor , preferably
`along the same optical axis , however in opposite directions
`is derived from .
`In some embodiments , the data processing system is
`along optical axis . The scanner may comprise at least one
`configured for generating a sub - scan of a part of the object
`beam splitter located in the optical path , where the beam
`surface based on surface geometry information and surface
`splitter is arranged such that it directs the probe light from
`color information derived from a plurality of blocks of 60 the multichromatic light source towards the object while it
`image sensor pixels . The sub - scan expresses at least the
`directs light received from the object towards the color
`geometry of the part of the object and typically one sub - scan
`image sensor .
`Several scanning principles are suitable , such as triangu
`is derived from one stack of captured 2D images .
`In some embodiments , all 2D images of a captured series
`lation and focus scanning .
`of images are analyzed to derive the surface geometry 65
`In some embodiments , the scanner system is a focus
`information for each block of image sensor pixels on the
`scanner system operating by translating a focus plane along
`an optical axis of the scanner system and capturing the 2D
`color image sensor .
`
`0008
`
`

`

`US 9 , 962 , 244 B2
`
`A
`
`=
`
`f : / =
`
`A = f . 1 = ? file
`
`One way to define the correlation measure mathematically
`with a discrete set of measurements is as a dot product
`computed from a signal vector , I = ( 1 , . . . , In ) , with n > 1
`elements representing sensor signals and a reference vector ,
`f = ( fl , . . . , fn ) , of reference weights . The correlation
`measure A is then given by
`
`images at different focus plane positions such that each
`series of captured 2D images forms a stack of 2D images
`The focus plane position is preferably shifted along an
`optical axis of the scanner system , such that 2D images
`captured at a number of focus plane positions along the 5
`optical axis forms said stack of 2D images for a given view
`of the object , i . e . for a given arrangement of the scanner
`system relative to the object . After changing the arrange
`ment of the scanner system relative to the object a new stack
`of 2D images for that view can be captured . The focus plane 10
`position may be varied by means of at least one focus
`element , e . g . , a moving focus lens .
`The indices on the elements in the signal vector represent
`In some focus scanner embodiments , the scanner system
`sensor signals that are recorded at different pixels , typically
`comprises a pattern generating element configured for incor
`or incor - 15 in a block of pixels . The reference vector f can be obtained
`porating a spatial pattern in said probe light .
`in a calibration step .
`In some embodiments , the pattern generating element is
`By using knowledge of the optical system used in the
`scanner , it is possible to transform the location of an
`configured to provide that the probe light projected by
`scanner system onto the object comprises a pattern consist -
`extremum of the correlation measure , i . e . , the focus plane
`ing of dark sections and sections with light having the a 20 into depth data information , on a pixel block basis . All pixel
`wavelength distribution according to the wavelength distri -
`blocks combined thus provide an array of depth data . In
`other words , depth is along an optical path that is known
`bution of the multichromatic light source .
`In some embodiments , the multichromatic light source
`from the optical design and / or found from calibration , and
`comprises a broadband light source , such as a white light
`each block of pixels on the image sensor represents the end
`25 point of an optical path . Therefore , depth along an optical
`source
`In some embodiments , the pixels of the color image
`path , for a bundle of paths , yields a surface geometry within
`sensor and the pattern generating element are configured to
`the field of view of the scanner , i . e . a sub - scan for the present
`provide that each pixel corresponds to a single bright or dark
`view .
`region of the spatial pattern incorporated in said probe light .
`It can be advantageous to smooth and interpolate the
`For a focus scanner system the surface geometry infor - 30 series of correlation measure values , such as to obtain a
`mation for a given block of image sensor pixels is derived
`more robust and accurate determination of the location of
`by identifying at which distance from the scanner system the
`the maximum .
`object surface is in focus for that block of image sensor
`In some embodiments , the generating a sub - scan com
`pixels .
`prises determining a correlation measure function describing
`In some embodiments , deriving the surface geometry 35 the variation of the correlation measure along the optical
`information and surface color information comprises calcu -
`axis for each block of image sensor pixels and identifying
`lating for several 2D images , such as for several 2D images
`for the position along the optical axis at which the correla
`in
`a captured stack of 2D images , a correlation measure
`tion measure functions have their maximum value for the
`between the portion of the 2D image captured by said block
`block .
`of image sensor pixels and a weight function . Here the 40
`In some embodiments , the maximum correlation measure
`weight function is preferably determined based on informa -
`value is the highest calculated correlation measure value for
`tion of the configuration of the spatial pattern . The correla -
`the block of image sensor pixels and / or the highest maxi
`tion measure may be calculated for each 2D image of the
`mum value of the correlation measure function for the block
`of image sensor pixels .
`stack .
`The scanner system may comprise means for evaluating a 45
`For example , a polynomial can be fitted to the values of
`A for a pixel block over several images on both sides of the
`correlation measure at each focus plane position between at
`least one image pixel and a weight function , where the
`recorded maximum , and a location of a deducted maximum
`weight function is determined based on information of the
`can be found from the maximum of the fitted polynomial ,
`configuration of the spatial pattern .
`which can be in between two images . The deducted maxi
`In some embodiments , deriving the surface geometry 50 mum is subsequently used as depth data information when
`information and the surface color information for a block of
`deriving the surface geometry from the present view , i . e .
`image sensor pixels comprises identifying the position along
`when deriving a sub - scan for the view .
`the optical axis at which the corresponding correlation
`In some embodiments , the data processing system is
`measure has a maximum value . The position along the
`configured for determining a color for a point on a generated
`optical axis at which the corresponding correlation measure 55 sub - scan based on the surface color information of the 2D
`has a maximum value may coincide with the position where
`image of the series in which the correlation measure has its
`a 2D image has been captured but it may even more likely
`maximum value for the corresponding block of image sensor
`be in between two neighboring 2D images of the stack of 2D
`pixels . The color may e . g . be read as the RGB values for
`pixels in said block of image sensor pixels .
`images .
`Determining the surface geometry information may then 60
`In some embodiments , the data processing system is
`configured for deriving the color for a point on a generated
`relate to calculating a correlation measure of the spatially
`structured light signal provided by the pattern with the
`sub - scan based on the surface color informations of the 2D
`variation of the pattern itself ( which we term reference ) for
`images in the series in which the correlation measure has its
`every location of the focus plane and finding the location of
`maximum value for the corresponding block of image sensor
`an extremum of this stack of 2D images . In some embodi - 65 pixels and on at least one additional 2D image , such as a
`ments , the pattern is static . Such a static pattern can for
`neighboring 2D image from the series of captured 2D
`example be realized as a chrome - on - glass pattern .
`images . The surface color information is still derived from
`
`0009
`
`

`

`US 9 , 962 , 244 B2
`
`In the embodiments where the resolution of the derived
`at least one of the 2D images from which the surface
`geometry information is derived .
`color is higher than the resolution of the surface geometry
`for the generated digital 3D representation of the object , a
`In some embodiments , the data processing system is
`pattern will be visible when at least approximately in focus ,
`configured for interpolating surface color information of at
`least two 2D images in
`a series when determining the 5 which preferably is the case when color is derived . The
`sub - scan color , such as an interpolation of surface color
`image can be filtered such as to visually remove the pattern ,
`information of neighboring 2D images in a series .
`however at a loss of resolution . In fact , it can be advanta
`In some embodiments , the data processing system is
`geous to be able to see the pattern for the user . For example
`configured for computing a smoothed color for a number of
`in intraoral scanning , it may be important to detect the
`points of the sub - scan , where the computing comprises an 10 position of a margin line , the rim or edge of a preparation .
`averaging of sub - scan colors of different points , such as a
`The image of the pattern overlaid on the geometry of this
`weighted averaging of the colors of the surrounding points
`edge is sharper on a side that is seen approximately perpen
`on the sub - scan .
`dicular , and more blurred on the side that is seen at an acute
`Surface color information for a block of image sensor
`angle . Thus , a user , who in this example typically is a dentist
`pixels is at least partially derived from the same image from
`15 or dental technician , can use the difference in sharpness to
`which surface geometry information is derived . In case the
`more precisely locate the position of the margin line than
`location of the maximum of A is represented by a 2D image ,
`may be possible from examining the surface geometry
`then also color is derived from that same image . In case the
`alone .
`location of the maximum of A is found by interpolation to be
`High spatial contrast of an in - focus pattern image on the
`between two images , then at least one of those two images 20 object is desirable to obtain a good signal to noise ratio of
`should be used to derive color , or both images using inter -
`the correlation measure on the color image sensor . Improved
`polation for color also . It is also possible to average color
`spatial contrast can be achieved by preferential imaging of
`data from more than two images used in the determination
`the specular surface reflection from the object on the color
`of the location of the maximum of the correlation measure ,
`image sensor . Thus , some embodiments comprise means for
`or to average color from a subset or superset of multiple 25 preferential / selective imaging of specularly reflected light .
`images used to derive surface geometry . In any case , some
`This may be provided if the scanner further comprises
`image sensor pixels readings are used to derive both surface
`means for polarizing the probe light , for example by means
`color and surface geometry for at least a part of the scanned
`of at least one polarizing beam splitter .
`object .
`In some embodiments , the polarizing optics is coated such
`Typically , there are three color filters , so the overall color 30 as to optimize preservation of the circular polarization of a
`is composed of three contributions , such as red , green , and
`part of the spectrum of the multichromatic light source that
`blue , or cyan , magenta , and yellow . Note that color filters
`is used for recording the surface geometry .
`typically allow a range of wavelengths to pass , and there is
`The scanner system may further comprise means for
`typically cross - talk between filters , such that , for example ,
`changing the polarization state of the probe light and / or the
`some green light will contribute to the intensity measured in 35 light received from the object . This can be provided by
`pixels with red filters .
`means of a retardation plate , preferably located in the optical
`For an image sensor with a color filter array , a color
`path . In some embodiments , the retardation plate is a quarter
`component c ; within a pixel block can be obtained as
`wave retardation plate .
`Especially for intraoral applications where the scanned
`40 object e . g . is the patient ' s set or teeth , the scanner can have
`an elongated tip , with means for directing the probe light
`and / or imaging an object . This may be provided by means of
`at least one folding element . The folding element could be
`a light reflecting element such as