Medivis Exhibit 1023
`
`356
`
`Medivis Exhibit 1023
`
`356
`
`

`

`Part VI
`
`Microscopy Option User’s
`Guide
`
`339
`
`Medivis Exhibit 1023
`
`357
`
`

`

`Medivis Exhibit 1023
`
`358
`
`Medivis Exhibit 1023
`
`358
`
`

`

`10 Deconvolution
`
`The deconvolution modules of the Microscopy Option provide powerful algorithms
`for improving the quality of microscopic images recorded by 3D widefield and
`confocal microscopes. Two different methods are supported, namely a so-called
`non-blind and a blind deconvolution method, both based on iterative maximum-
`likelihood image restoration. In the first case a measured or computed point spread
`function (PSF) is required. In the second case the PSF is estimated along with the
`data itself.
`The deconvolution documentation is organized as follows:
`• General remarks about image deconvolution
`• Data acquisition and sampling rates
`• Standard deconvolution tutorial
`• Blind deconvolution tutorial
`• Bead extraction tutorial
`• Performance issues and multi-processing
`The following modules are provided:
`• BeadExtract - obtain a PSF from a bead measurement
`• Convolution - convolve two 3D images
`• CorrectZDrop - corrects attenuation in z-direction
`• DataPreprocess - background and flatfield correction
`• Deconvolution - the actual deconvolution front-end
`• FourierTransform - computes FFT and power spectrum
`• PSFGenerate - calculates a theoretical PSF
`Examples:
`• Confocal data set
`• Widefield data set
`
`341
`
`Medivis Exhibit 1023
`
`359
`
`

`

`Chapter 10: Deconvolution
`
`10.1 General remarks about image deconvolution
`
`Deconvolution is a technique for removing out-of-focus light in a series of images
`recorded via optical sectioning microscopy. Intended to investigate 3D biological
`objects, optical sectioning microscopy works by creating multiple images (opti-
`cal sections) of a fluorescing object, each with a different focus plane. However,
`besides the in-focus structures, the images usually also contain out-of-focus light
`from other parts of the object, causing haze and severe axial blur. This is even the
`case for a confocal laser scanning microscope, where most of the out-of-focus light
`is removed from the image by a pinhole system. Mathematically, the image pro-
`duced by any microscopic system can be described as the convolution of the ideal
`unblurred image of the specimen and the microscope’s so-called point spread func-
`tion (PSF), i.e., the image of an ideal point light source. With the inverse of this
`process, called deconvolution, a deblurred image of the specimen can be obtained,
`provided the point spread function is known, or at least can be estimated.
`The Amira deconvolution modules mainly provide two variants of a powerful it-
`erative maximum-likelihood image restoration algorithm, namely a non-blind one
`and a blind one. The difference between them is that in the first case a measured
`or computed point spread function is used, while in the second case the PSF is
`estimated along with the data itself. Maximum-likelihood image restoration can
`be considered as the de-facto standard for deconvolution of 3D optical sections.
`Although computationally quite expensive, the method is able to significantly en-
`hance image quality. At the same time it is very robust and insensitive with respect
`to noise artifacts. However, it should be noted that, although rejecting most of the
`out-of-focus light, by no means all of it is rejected. Therefore, some noticeable
`haze remains in the images. Also, the images retain a substantial axial smearing in
`z-direction, which cannot be removed by any deconvolution algorithm.
`At first sight, one may wonder why both a non-blind and a blind deconvolution
`algorithm are provided; blind deconvolution seems to be more general because the
`PSF is calculated automatically. One answer is that blind deconvolution is compu-
`tationally even more expensive than non-blind iterative maximum-likelihood im-
`age restoration. The other answer is that in a blind deconvolution algorithm a
`meaningful estimate of the PSF can only be computed if severe constraints are im-
`posed. For example, a trivial solution of the blind deconvolution problem would
`be an image which is identical to the input image and a PSF with the shape of an
`ideal delta peak. Obviously, this solution isn’t useful at all. Therefore, if for ex-
`ample confocal data is to be deconvolved, the algorithm fits the actual PSF in such
`a way that it looks like a possible measured PSF of a confocal microscope. More
`
`342
`
`Medivis Exhibit 1023
`
`360
`
`

`

`Data acquisition and sampling rates
`
`precisely, the fit is constrained to be in agreement with the experimental parame-
`ters (the refractive index of the medium, the numerical aperture of the objective,
`and the voxel sizes). Sometimes this can lead to wrong results, for example when
`the confocal pinhole aperture of the microscope wasn’t stopped down sufficiently
`during confocal image acquisition, in which case the microscope actually didn’t
`behave like a true confocal microscope. As a matter of fact you should try which
`approach provides the best results for your own image data, blind deconvolution
`or non-blind deconvolution with either a measured or an automatically computed
`PSF.
`
`10.2 Data acquisition and sampling rates
`
`In order to obtain best quality when deconvolving microscopic images some fun-
`damental guidelines should be obeyed during image aquisition. Good results may
`be obtained even if some of these guidelines are not followed exactly, but in gen-
`eral the chances to get satisfactory results improve if they are. Below we discuss
`the most important recommendations.
`
`Adjusting the Scanned Image Volume
`
`The region of interest should be centered in the middle of the image volume, as the
`optics of the microscope has usually the least aberrations in this region and it helps
`to avoid possible boundary artifacts, which can arise during the deconvolution
`procedure. Especially for widefield data it is important to record a sufficiently
`large (preferably empty) region below and above the actual sample. Ideally, this
`region should be as large as the sample itself. For example, if the sample covers
`100 micrometers in the z-direction, the scanned image volume should range from
`50 micrometers below the sample to 50 micrometers above it.
`
`Choosing the Right Sampling Rate
`
`The sampling rate is determined by the pixel sizes in the x and y directions as
`well as the distance between two subsequent optical sections, both measured in
`micrometers. Generally speaking, image deconvolution works best if the data is
`apparently oversampled, i.e., if the pixel or optical section spacing is smaller than
`required. The maximal required sampling distance (Nyquist sampling) to avoid
`
`343
`
`Medivis Exhibit 1023
`
`361
`
`

`

`Chapter 10: Deconvolution
`
`ambiguities in the data can be obtained from considerations in Fourier-space yield-
`ing
`
`dxy =
`
`λ
`4NA ,
`where λ denotes the wavelength and NA is the numerical aperture of the micro-
`scope. Similar considerations yield for the maximal distance between adjacent
`image planes:
`
`dz =
`
`λ
`2n(1− cos(α)) ,
`where n denotes the refractive index of the object medium and al pha the aperture
`half angle as determined by NA = n sin(α).
`For a confocal microscope, both the in-plane sampling distance and the axial sam-
`pling distance need to be in theory approximately 2 times smaller. However, this
`requirement is far too strict for most practical cases and even in the widefield case,
`approximately fullfilling the above requirements is often sufficient.
`The total number of optical sections is obtained by dividing the height of the image
`volume by the sampling distance dz. It should be mentioned that deconvolution
`also works if the sampling distances are not matched rigorously, but matching them
`improves the chances to get good results. In general, oversampling the object is
`less harmful than undersampling it, with one exception: In the case of confocal
`data, the sampling distance dxy should not be much smaller than indicated, if the
`blind deconvolution algorithm or the non-blind deconvolution algorithm together
`with a theoretically computed confocal PSF are used. Otherwise the unconstrained
`Maximum Likelihood algorithm and the predominant noise in the data might lead
`to unsatisfactory results.
`
`Black Level and Saturation
`
`Before grabbing images from the microscope’s camera, the light level should be
`adjusted in such a way that saturated pixels, either black or white ones, are avoided.
`Saturated pixels are pixels which are clamped to either black or white because
`their actual intensity values are outside the range of representable intensities. In
`any case, saturation means a loss of information and thus prevents proper post-
`processing or deconvolution. At the same time, a high background level should
`be avoided because it decreases the dynamic range of the imaging system and
`the deconvolution works worse. This means that empty regions not showing any
`fluorescence should appear almost black. A background level close to zero is
`
`344
`
`Medivis Exhibit 1023
`
`362
`
`

`

`Standard Deconvolution Tutorial
`
`especially important when bead measurements are performed in order to extract
`an experimental point spread function. Details are discussed is a separate tutorial
`about bead extraction.
`
`10.3 Standard Deconvolution Tutorial
`
`This tutorial explains how 3D image data sets can be deconvolved in Amira. It is
`asumed that the reader is already familiar with the basic concepts of Amira itself.
`If this is not the case, it is strongly recommended to work through the standard
`Amira tutorials first. In this section the following topics are covered:
`
`1. Prerequisites for deconvolution
`2. Resampling a measured PSF
`3. Deconvolving an image data set
`4. Calculating a theoretical PSF
`
`As an example we are going to use a confocal test data set (polytrichum.am) pro-
`vided with the Amira deconvolution modules. The data file is located in the direc-
`tory Amira-5/data/deconv.
`• Load the data set polytrichum.am.
`• Visualize it, for example, using a ProjectionView module.
`The data set shows four chloroplasts in a spore of the moss polytrichum commune.
`
`Prerequisites for Deconvolution
`
`Besides the image data itself, for the standard non-blind deconvolution algorithm
`a so-called point spread function (PSF) is also required. The PSF is the image of a
`single point source, or as a close approximation, the image of a single fluorescing
`sub-resolution sphere. PSF images can either be computed from theory (see be-
`low) or they can be obtained from measurements. In the latter case tiny so-called
`beads are recorded under the same conditions as the actual object. This means that
`the same objective lens, the same dye and wavelength, and the same immersion
`medium are used. Typically, the images of multiple beads are averaged to obtain
`an estimate of a single PSF. Amira provides a module called BeadExtract facilitat-
`ing this process. The use of this module is discussed in a separate tutorial about
`bead extraction. At this point let us simply load a measured PSF from a file.
`
`345
`
`Medivis Exhibit 1023
`
`363
`
`

`

`Chapter 10: Deconvolution
`
`Figure 10.1: Maximum intensity projection of polytrichum-psf.am
`
`• Load the data set polytrichum-psf.am.
`• Use the ProjectionView module to visualize it.
`The PSF appears as a bright spot located in the middle of the image volume (Figure
`10.1). It is important that the PSF is exactly centered. Otherwise, the deconvolved
`data set will be shifted with respect to the original image. Also, it is important that
`the PSF fades out to black at the boundaries. If this is not the case, the black level
`of the PSF image needs to be adjusted using the Arithmetic module. Finally, nei-
`ther the PSF nor the image to be deconvolved should exhibit intensity attenuation
`artifacts, i.e., image slices with decreased average intensity due to excessive light
`absorption in other slices. If such artifacts are present, they can be removed using
`the CorrectZDrop module.
`
`Resampling a Measured PSF
`
`Next, select both, the PSF and the image data. You’ll notice that the voxel sizes
`of both objects are not the same. It is recommended to adjust different voxel sizes
`of PSF and image data prior to deconvolution using the Resample module. The
`deconvolution module itself also accounts for different voxel sizes, but is does so
`by using point sampling with trilinear interpolation. This is OK as long as the
`voxel size of the PSF is larger than that of the image data. However, in our case
`the voxel size of the PSF is smaller than that of the image data, i.e., the resolution
`of the PSF higher. Using the Resample module provides slightly more accurate
`
`346
`
`Medivis Exhibit 1023
`
`364
`
`

`

`Standard Deconvolution Tutorial
`
`Figure 10.2: Resampling a PSF using the Resample module.
`
`results here, since all samples will be filtered correctly using a Lanczos kernel.
`• Connect a Resample module to polytrichum-psf.am.
`• Connect the Reference port of the Resample module to the image data set
`polytrichum.am
`• In the Mode port of the Resample module, choose voxel size (see Figure
`10.2).
`• Resample the PSF by pressing the Apply button.
`The voxel size option means that the PSF will be resampled on a grid with exactly
`the same voxel size as the image data set, which is connected to the Reference port.
`While the original PSF had a resolution of 12 x 12 x 30 voxels, the resampled one
`only has 12 x 12 x 16 voxels. However, the extent of a single voxel in z-direction
`is bigger now.
`
`Deconvolving an Image Data Set
`After a suitable PSF has been obtained we are ready for deconvolving the image
`data set. This can be done by attaching a Deconvolution module to the image data.
`
`347
`
`Medivis Exhibit 1023
`
`365
`
`

`

`Chapter 10: Deconvolution
`
`• Connect a Deconvolution module to polytrichum.am.
`• Connect the Kernel port of the Deconvolution module to the resampled PSF
`polytrichum-psf.Resampled.
`
`Once the deconvolution module is connected to its two input objects, some addi-
`tional parameters need to be adjusted (for a detailed discussion of these parameters
`see also the reference documentation of the Deconvolution module itself). Figure
`10.3 shows these settings:
`Border width: For deconvolution the image data has to be enlarged by a guardband
`region. Otherwise boundary artifacts can occur, i.e., information from one side of
`the data can be passed to the other. There is no need to make the border bigger than
`the size of the PSF. However, if the data set is dark at the boundaries, a smaller
`border width is sufficient. In our case, let us choose the border values 0, 0, and 8
`in the x, y, and z direction.
`Iterations: The number of iterations of the deconvolution algorithm. Let us choose
`a value of 20 here.
`Initial estimate: Specifies the initial estimate of the deconvolution algorithm. If
`const is chosen a constant image is used initially. This is the most robust choice,
`yielding good results even if the input data is very noisy. We keep this option here.
`
`Overrelaxation: Overrelaxation is a technique to speed up the convergence of the
`iterative deconvolution process. In most cases the best compromise between speed
`and quality is fixed overrelaxation. Therefore we keep this choice also.
`Method: Selects between standard (non-blind) and blind deconvolution. Let us
`specify the standard option here.
`The actual deconvolution process is started by pressing the Apply button. Please
`press this button now. The deconvolution should take about 10 seconds on a mod-
`ern computer. During the deconvolution the progress bar informs you about the
`status of the operation. Also, after every iteration a message is printed in the Amira
`console window indicating the amount of change of the data. If the change seems
`to be small enough, you can terminate the deconvolution procedure by pressing
`the Stop button. However, note that the stop button is evaluated only once between
`two consecutive iterations.
`When deconvolution is finished, a new data set called polytrichum.deconv appears
`in the Pool. You might take a look at the deconvolved data by moving the Projec-
`tionView connection line from polytrichum.am to polytrichum.deconv.
`
`348
`
`Medivis Exhibit 1023
`
`366
`
`

`

`Standard Deconvolution Tutorial
`
`Figure 10.3: Deconvolution module attached to polytrichum.am.
`
`Calculating a Theoretical PSF
`
`Sometimes bead measurements are difficult to perform, so that an experimental
`PSF cannot easily be obtained. In such cases a theoretical PSF can be used instead.
`Amira provides the module PSFGen, allowing you to calculate theoretical PSFs.
`The module can be created by selecting PSFGen from the Create Others menu of
`the Amira main window.
`Once the module is created again some parameters have to be entered. The reso-
`lution and the voxel size can be most easily specified by connecting the Data port
`of the PSGGen module to the image data set to be convolved. In our case, please
`connect this port to polytrichum.am.
`In order to generate a PSF, you also need to know the numerical aperture of the mi-
`croscope objective, the wavelength of the emitted light (to be entered in microm-
`eters!), and the refractive index of the immersion medium. In our test example
`these values are NA=1.4, lambda=0.58, and n=1.516 (oil medium). Also, change
`the microscopic mode from widefield to confocal.
`After you press the Apply button, the computed PSF appears as an icon labelled
`PSF in the Pool. You can compare the theoretical PSF with the measured one
`using the OrthoSlice module. You’ll notice that the measured PSF appears to be
`
`349
`
`Medivis Exhibit 1023
`
`367
`
`

`

`Chapter 10: Deconvolution
`
`Figure 10.4: The PSFGen module calculates theoretical PSFs.
`
`slightly wider. This is a common observation in many experiments.
`Once you have computed a theoretical PSF, you can perform non-blind deconvo-
`lution as described above. However, for convenience the Deconvolution module
`is also able to compute a theoretical PSF by itself. You can check this by discon-
`necting the Kernel port of the Deconvolution module. If no input is present at this
`port, additional input fields are shown, allowing you to enter the same parame-
`ters (numerical aperture, wavelength, refractive index, and microscopic mode) as
`in PSFGen. After these parameters have been entered, the deconvolution process
`again can be started by pressing the Apply button.
`Note that any previous result connected to the Deconvolution module will be over-
`written when starting the deconvolution process again. Therefore, be sure to dis-
`connect a previous result if you want to compare deconvolution with different input
`PSFs.
`
`10.4 Blind Deconvolution Tutorial
`
`This tutorial explains how blind deconvolution can be perfomed in Amira. At the
`same time it describes how deconvolution jobs can be processed using the Amira
`job queue. Like in the previous tutorial, it is assumed that the reader is already
`familiar with the basic concepts of Amira itself. If not, we recommend to work
`through the standard Amira tutorials first.
`
`350
`
`Medivis Exhibit 1023
`
`368
`
`

`

`Blind Deconvolution Tutorial
`
`Figure 10.5: Parameters for blind deconvolution.
`
`A Blind Deconvolution Example
`
`Let us start by loading a raw image data set first.
`• Load the file alphalobe.am from the directory Amira-5/data/deconv.
`• Visualize the data set by attaching a ProjectionView module to it.
`
`The data set has been recorded using a standard fluorescence microscope under
`so-called widefield conditions. It shows a neuron from the alpha-lobe of the hon-
`eybee brain. Compared to the confocal data set used in the standard deconvolution
`tutorial, alphalobe.am is much bigger. It has a resolution of 248 x 248 x 256 vox-
`els with a uniform voxel size of 1 micrometer. In the xy-plane of the projection
`view the structure of the neuron can be clearly identified. However, the contrast of
`the image is quite poor because there is a significant amount of out-of-focus light
`or haze present. With Amira’s blind deconvolution algorithm we can enhance the
`image data without needing to know an explicit PSF in advance.
`• Attach a deconvolution module to alphalobe.am.
`• Adjust the parameters like shown in Figure 10.5.
`
`351
`
`Medivis Exhibit 1023
`
`369
`
`

`

`Chapter 10: Deconvolution
`
`The individual parameters have the following meaning:
`Border width: As for standard non-blind deconvolution, the image data has to
`be enlarged by a guardband region. Otherwise boundary artifacts can occur, i.e.,
`information from one side of the data can be passed to the other. In our case we
`only provide a small guardband region of 8 voxels in x- and y-direction. In z-
`direction we do not provide any border because there are sufficiently many empty
`slices below and above the actual neuron. The resulting size of the data arrays
`on which the computations are performed then is 256 x 256 x 256. Because 256
`is a power of two (2ˆ8), the Fast Fourier Transforms, the computationally most
`expensive part of the deconvolution algorithm, can be executed somewhat faster.
`Iterations: We choose a value of 25 here. Depending on the data, usually at least
`10 iterations are required. With overrelaxation being enabled (see below), results
`usally don’t improve much after 40 iterations.
`Initial estimate: Specifies the initial estimate of the deconvolution algorithm.
`Since there is not much noise present in the original alphalobe images it is safe
`to chose input data here. This causes the algorithm to converge even faster.
`Overrelaxation: Overrelaxation is a technique to speed up the convergence of the
`iterative deconvolution process. We enable overrelaxation by chosing the fixed
`toggle.
`Regularization: We chose none here in order to do no regularization.
`Method: We chose blind here in order to select the blind deconvolution algorithm.
`PSF Parameters: For alphalobe.am the numerical aperture is 0.5, the wavelength
`is 0.58 micrometers, and the refractive index is 1.33 (water). These parameters
`are required in order to apply certain constraints to the estimated point spread
`function. They are also used in order to compute an initial PSF. If a data set would
`be connected to the Kernel port of the deconvolution module, this data set would
`be used as the inital PSF with the given PSF parameters still acting as constraints.
`For example, you could provide a measured PSF and let it be fitted to the actual
`data by the deconvolution algorithm.
`Microscopic mode: alphalobe.am is a widefield data set, so select this option here.
`
`Submitting a Deconvolution Job
`
`After all parameters have been entered, the deconvolution process can be started.
`On a modern computer, blind deconvolution of our test data set roughly takes about
`20 minutes. Especially, if you want to deconvolve multiple data sets at once it is
`inconvenient to do this in an interactive session. Therefore multiple deconvolution
`
`352
`
`Medivis Exhibit 1023
`
`370
`
`

`

`Blind Deconvolution Tutorial
`
`Figure 10.6: Dialog for submitting a deconvolution job.
`
`jobs can be submitted to the Amira job queue and then, for example, processed
`overnight. This works as follows:
`• Press the Batch Job button of the Action port. A dialog as shown in Figure
`10.6 pops up.
`• In the dialog choose a file name under which you want to save the decon-
`volved data set, e.g. C:/Temp/alphalobe-deconv.am.
`• Modify the text field, so that check point files are written after every 5 iter-
`ations.
`
`Check point files are used to store intermediate results. With the above
`settings the deconvolved data is written into a file after every 5 itera-
`tions.
`Check point files are named like the final result, but a consec-
`utive number
`is inserted just before the file name suffix.
`For exam-
`ple,
`if the result file name is C:/Temp/alphalobe-deconv.am,
`the
`check point files are named C:/Temp/alphalobe-deconv-0005.am,
`C:/Temp/alphalobe-deconv-0010.am and so on. Now we are ready to
`actually submit the batch job.
`• Press the Submit button of the deconvolution dialog. After a few seconds
`the Amira batch job dialog appears, compare Figure 10.7.
`• Select the deconvolution job and press the Start button.
`You now have to wait about 20 minutes until the deconvolution job is finished.
`Once the job queue has been started, you can quit Amira. The batch jobs will be
`
`353
`
`Medivis Exhibit 1023
`
`371
`
`

`

`Chapter 10: Deconvolution
`
`Figure 10.7: The Amira job dialog showing a pending deconvolution job.
`
`continued automatically. If Amira is still running when the deconvolution job exits
`then the result will be loaded automatically in Amira. Otherwise you have to restart
`Amira and load the deconvolved data set manually.
`
`10.5 Bead Extraction Tutorial
`
`Non-blind deconvolution is a powerful and robust method for enhancing the qual-
`ity of 3D microscopic images. However, the method requires that the image of the
`point spread function (PSF) responsible for image blurring is provided. As stated
`in the standard deconvolution tutorial, the PSF can either be calculated theoreti-
`cally or it can be obtained from a bead measurement. Amira provides a special-
`purpose module called BeadExtract which facilitates the extraction of PSF images
`from one or multiple bead mesurements. In this tutorial the use of the module shall
`be explained. The following topics are covered:
`
`1. Bead measurements
`2. Projection View and Projection View Cursor
`3. Resampling and averaging the beads
`
`354
`
`Medivis Exhibit 1023
`
`372
`
`

`

`Bead Extraction Tutorial
`
`Bead Measurements
`The PSF is the image of a single point source recorded under the same conditions
`as the actual specimen. It can be approximated by the image of a fluorescing sub-
`resolution microsphere, a so-called bead. Performing good bead measurements
`requires some practice and expertise. In order to obtain good results the following
`hints should be obeyed:
`
`It is important that the bead size be smaller
`1. Use appropriate beads.
`than approximately 1/2 full width at half maximum (FWHM) of the
`PSF. Good sources for obtaining beads suitable for PSF measurements
`are Molecular Probes (http://www.probes.com/) or Polysciences
`(http://www.polysciences.com/).
`2. The beads must be solid. Besides solid beads, there are also beads with the
`shape of a spherical shell, allowing to check the focus plane of a mircoscope.
`Such beads cannot be used as a source for PSF generation in the current
`version of Amira.
`3. Don’t record clusters of multiple beads. Sometimes multiple beads may
`glue together, appearing as a single big bright spot. Computing a PSF from
`such a spot obviously leads to wrong results.
`4. Note that beads are not resistant to a variety of embedding media. In par-
`ticular beads will be destroyed in xylene-based embedding media such as
`Permount (Fisher Scientific) and methyl salicylate (frequently used to clear
`up the tissue). As a substitute you might use immersion oil instead, which
`has a refractive index similar to methyl salicylate, for example.
`5. Sample and beads should always be imaged as close to the coverslip as
`possible. When it is not possible to attach the sample to the coverslip, the
`beads should also be imaged in a comparable depth, embedded in the same
`mounting medium. Imaging the beads with better quality than the sample
`will yield a slightly blurred deconvolution result. However, when the PSF
`used for deconvolution is too wide, artifacts can arise during deconvolution.
`
`The objective lense should always be selected according to the mounting
`medium, i.e. if the sample is attached to the slide and embedded in a buffer
`of refractive index close to water, a severe loss of image quality can be ex-
`pected when using an oil-immersion objective without a correction collar.
`Deconvolution of properly imaged data will allways be supperior to decon-
`volution of data suffering from aberrations.
`
`355
`
`Medivis Exhibit 1023
`
`373
`
`

`

`Chapter 10: Deconvolution
`
`6. Problems occur if the mounting medium remains liquid. In that case the
`sample distribution may not be permanent. If your specimen is to be em-
`bedded in water, you can try to immerse the beads in an agarose gel of
`moderate concentration instead. Attaching the small beads to the coverslip
`(for example by letting them dry) is often also sufficient for immobilization.
`
`An example of an image data set containing multiple beads is provided in the file
`beads.am in the directory Amira-5/data/deconv.
`• Load the data set beads.am.
`• Visualize the data using a ProjectionView module.
`
`Projection View and Projection View Cursor
`The bead data set contains five different beads which can be clearly seen in the
`three orthogonal planes of the ProjectionView module. In order to obtain a single
`PSF we first want to select several ”good” beads. These beads are then resampled
`and averaged, thus yielding the final PSF. A bead can be considered as ”good”
`if it is clearly visible and if it is not superimposed by other beads (even when
`defocused),
`Selecting ”good” beads is an interactive process. It is most easily accomplished
`using the ProjectionView’s Cursor module. This module allows you to select a
`point in 3D space by clicking on one of the three planes of the ProjectionView
`module. The third coordinate is automatically set by looking for the voxel with
`the highest intensity. Points selected with the Cursor module can be stored in a
`LandmarkSet data object.
`• Attach a Cursor module to the ProjectionView module.
`• Click on any bead on one of the three planes.
`• Store the current cursor position in a LandmarkSet object by pressing the
`Add button.
`• Select and add some other beads too.
`The landmarks need not to be located exactly at the center of a bead. The exact
`center positions can be fitted automatically later on.
`You can remove incorrect bead positions from the landmark set by invoking the
`landmark set editor. In order to activate the editor, select the landmark set object
`and press Landmark Editor button. If you want to add additional bead positions to
`
`356
`
`Medivis Exhibit 1023
`
`374
`
`

`

`Bead Extraction Tutorial
`
`Figure 10.8: Individual beads can be interactively identified using a Cursor
`module.
`
`an existing landmark set object, make sure that the master port of the landmark set
`object is connected to the Cursor module. Otherwise, a new landmark set object
`will be created.
`
`Resampling and Averaging the Beads
`Now we are ready to extract and average the individual beads. This is done by
`means of the BeadExtract module.
`• Connect a BeadExtract module to the Landmarks object.
`• Make sure that the Data port of BeadExtract is connected to the bead data
`set beads.am. If the landmarks are still connected to the beads via the Cursor
`and ProjectionView modules, the connection is established automatically.
`
`The BeadExtract module provides two buttons called Adjust centers and Estimate
`size, which should be invoked in a preprocessing step before the beads are actually
`extracted.
`The first button (Adjust centers) modifies the landmark positions so that they are
`precisely located in the center of gravity of the individual beads.
`The second button (Estimate size) computes an estimate for the number of voxels
`of the PSF image to be generated. This button is only active if no PSF image is
`connected as a result to BeadExtract. If there is a result object, the resolution and
`voxel sizes of the result are used and the port becomes insensitive.
`
`357
`
`Medivis Exhibit 1023
`
`375
`
`

`

`Chapter 10: Deconvolution
`
`Figure 10.9: The BeadExtract module resamples and averages multiple beads.
`
`Any of the actions of the two preprocessing buttons can be undone using the Undo
`button. This can be necessary for example if two beads are too close so that no cor-
`rect center position could be computed. In general, overlaps between neighboring
`beads should be avoided. Small overlaps might be tolerated because during resam-
`pling intensities are weighted according to the influence of surrounding beads.
`• Perform the preprocessing steps Adjust centers and Estimate size.
`• Compute a resampled and averaged PSF by pressing the Apply button.
`The data type of the resulting PSF will be float, irrespective of the data type of the
`input image. The individual beads will be weighted on a per-voxel basis and added
`to the result. No normalization will be performed afterwards. You may investigate
`the resulting PSF image using any of the standard visualization modules. In Figure
`10.10 a volume rendering of the resulting PSF is shown.
`In some cases you may want to average multiple beads recorded in different input
`data sets. This can be easily achieved by creating a Landmarks object for each
`input data set. For the first input data set extract the beads as described above.
`For the other input data set also use the BeadExtract module. However, make sure
`that the PSF obtained from the first input data set is connected as a result object
`to BeadExtract before pressing the Apply button. In order to use an existing PSF
`as a result object connect the