Algorithms for accurate 3D registration of neuronal images acquired by confocal scanning laser microscopy

This paper presents automated and accurate algorithms based on high‐order transformation models for registering three‐dimensional (3D) confocal images of dye‐injected neurons. The algorithms improve upon prior methods in several ways, and meet the more stringent image registration needs of applications such as two‐view attenuation correction recently developed by us. First, they achieve high accuracy (≈ 1.2 voxels, equivalent to 0.4 µm) by using landmarks, rather than intensity correlations, and by using a high‐dimensional affine and quadratic transformation model that accounts for 3D translation, rotation, non‐isotropic scaling, modest curvature of field, distortions and mechanical inconsistencies introduced by the imaging system. Second, they use a hierarchy of models and iterative algorithms to eliminate potential instabilities. Third, they incorporate robust statistical methods to achieve accurate registration in the face of inaccurate and missing landmarks. Fourth, they are fully automated, even estimating the initial registration from the extracted landmarks. Finally, they are computationally efficient, taking less than a minute on a 900‐MHz Pentium III computer for registering two images roughly 70 MB in size. The registration errors represent a combination of modelling, estimation, discretization and neuron tracing errors. Accurate 3D montaging is described; the algorithms have broader applicability to images of vasculature, and other structures with distinctive point, line and surface landmarks.     

Intensity correction of fluorescent confocal laser scanning microscope images by mean-weight filtering

This paper addresses the problem of intensity correction of fluorescent confocal laser scanning microscope images. Confocal laser scanning microscope images are frequently used in medicine for obtaining 3D information about specimen structures by imaging a set of 2D cross sections and performing 3D volume reconstruction afterwards. However, 2D images acquired from fluorescent confocal laser scanning microscope images demonstrate significant intensity heterogeneity, for example, due to photo‐bleaching and fluorescent attenuation in depth. We developed an intensity heterogeneity correction technique that (a) adjusts the intensity heterogeneity of 2D images, (b) preserves fine structural details and (c) enhances image contrast, by performing spatially adaptive mean‐weight filtering. Our solution is obtained by formulating an optimization problem, followed by filter design and automated selection of filtering parameters. The proposed filtering method is experimentally compared with several existing techniques by using four quality metrics: contrast, intensity heterogeneity (entropy) in a low frequency domain, intensity distortion in a high frequency domain and saturation. Based on our experiments and the four quality metrics, the developed mean‐weight filtering outperforms other intensity correction methods by at least a factor of 1.5 when applied to fluorescent confocal laser scanning microscope images.     

Alignment with sub‐pixel accuracy for images of multi‐modality microscopes using automatic calibration

A biological specimen is often imaged with various imaging modalities, and it is crucial that such images are well aligned to best reveal physiological structures and functions of the specimen for in‐depth analyses. In this paper, we present a methodology for automatic calibration of multiple optical imaging modalities within the x–y detector plane using a custom chrome‐on‐glass target and an automatic and accurate registration algorithm. The target contains lines crossing at random angles, and our method of registration is based on the alignment of salient features extracted from the lines within the individual images. Once spatial relationships are found between the various detectors and applied to the resultant images, no further registration is required for all static samples, and the registered images serve as the starting point for registration of dynamic samples, where the remaining misalignment is caused by sample movement. We have validated our algorithm with 40 inter‐modal and 30 intra‐modal image pairs, and the success rates are 95 and 100%, respectively, with sub‐pixel accuracy. This methodology is widely applicable to any multi‐modal microscope that combines a number of imaging modalities on a common platform assuming images of the target can be obtained.     

Three‐dimensional imaging of whole mouse models: comparing nondestructive X‐ray phase‐contrast micro‐CT with cryotome‐based planar epi‐illumination imaging

In this study, we compare two evolving techniques for obtaining high‐resolution 3D anatomical data of a mouse specimen. On the one hand, we investigate cryotome‐based planar epi‐illumination imaging (cryo‐imaging). On the other hand, we examine X‐ray phase‐contrast micro‐computed tomography (micro‐CT) using synchrotron radiation. Cryo‐imaging is a technique in which an electron multiplying charge coupled camera takes images of a cryo‐frozen specimen during the sectioning process. Subsequent image alignment and virtual stacking result in volumetric data. X‐ray phase‐contrast imaging is based on the minute refraction of X‐rays inside the specimen and features higher soft‐tissue contrast than conventional, attenuation‐based micro‐CT. To explore the potential of both techniques for studying whole mouse disease models, one mouse specimen was imaged using both techniques. Obtained data are compared visually and quantitatively, specifically with regard to the visibility of fine anatomical details. Internal structure of the mouse specimen is visible in great detail with both techniques and the study shows in particular that soft‐tissue contrast is strongly enhanced in the X‐ray phase images compared to the attenuation‐based images. This identifies phase‐contrast micro‐CT as a powerful tool for the study of small animal disease models.     

Image reconstruction from sparse data in synchrotron-radiation-based microtomography

Synchrotron-radiation-based microcomputed-tomography (SR-μCT) is a powerful tool for yielding 3D structural information of high spatial and contrast resolution about a specimen preserved in its natural state. A large number of projection views are required currently for yielding SR-μCT images by use of existing algorithms without significant artifacts. When a wet biological specimen is imaged, synchrotron x-ray radiation from a large number of projection views can result in significant structural deformation within the specimen. A possible approach to reducing imaging time and specimen deformation is to decrease the number of projection views. In the work, using reconstruction algorithms developed recently for medical computed tomography (CT), we investigate and demonstrate image reconstruction from sparse-view data acquired in SR-μCT. Numerical results of our study suggest that images of practical value can be obtained from data acquired at a number of projection views significantly lower than those used currently in a typical SR-μCT imaging experiment.     

Automated in‐chamber specimen coating for serial block‐face electron microscopy

When imaging insulating specimens in a scanning electron microscope, negative charge accumulates locally (‘sample charging’). The resulting electric fields distort signal amplitude, focus and image geometry, which can be avoided by coating the specimen with a conductive film prior to introducing it into the microscope chamber. This, however, is incompatible with serial block‐face electron microscopy (SBEM), where imaging and surface removal cycles (by diamond knife or focused ion beam) alternate, with the sample remaining in place. Here we show that coating the sample after each cutting cycle with a 1–2 nm metallic film, using an electron beam evaporator that is integrated into the microscope chamber, eliminates charging effects for both backscattered (BSE) and secondary electron (SE) imaging. The reduction in signal‐to‐noise ratio (SNR) caused by the film is smaller than that caused by the widely used low‐vacuum method. Sample surfaces as large as 12 mm across were coated and imaged without charging effects at beam currents as high as 25 nA. The coatings also enabled the use of beam deceleration for non‐conducting samples, leading to substantial SNR gains for BSE contrast. We modified and automated the evaporator to enable the acquisition of SBEM stacks, and demonstrated the acquisition of stacks of over 1000 successive cut/coat/image cycles and of stacks using beam deceleration or SE contrast.     

Comparison of three‐dimensional analysis and stereological techniques for quantifying lithium‐ion battery electrode microstructures

Lithium‐ion battery performance is intrinsically linked to electrode microstructure. Quantitative measurement of key structural parameters of lithium‐ion battery electrode microstructures will enable optimization as well as motivate systematic numerical studies for the improvement of battery performance. With the rapid development of 3‐D imaging techniques, quantitative assessment of 3‐D microstructures from 2‐D image sections by stereological methods appears outmoded; however, in spite of the proliferation of tomographic imaging techniques, it remains significantly easier to obtain two‐dimensional (2‐D) data sets. In this study, stereological prediction and three‐dimensional (3‐D) analysis techniques for quantitative assessment of key geometric parameters for characterizing battery electrode microstructures are examined and compared. Lithium‐ion battery electrodes were imaged using synchrotron‐based X‐ray tomographic microscopy. For each electrode sample investigated, stereological analysis was performed on reconstructed 2‐D image sections generated from tomographic imaging, whereas direct 3‐D analysis was performed on reconstructed image volumes. The analysis showed that geometric parameter estimation using 2‐D image sections is bound to be associated with ambiguity and that volume‐based 3‐D characterization of nonconvex, irregular and interconnected particles can be used to more accurately quantify spatially‐dependent parameters, such as tortuosity and pore‐phase connectivity.     

Shack-Hartmann wave front measurements in cortical tissue for deconvolution of large three-dimensional mosaic transmitted light brightfield micrographs

We present a novel approach for deconvolution of 3D image stacks of cortical tissue taken by mosaic/optical‐sectioning technology, using a transmitted light brightfield microscope. Mosaic/optical‐sectioning offers the possibility of imaging large volumes (e.g. from cortical sections) on a millimetre scale at sub‐micrometre resolution. However, a blurred contribution from out‐of‐focus light results in an image quality that usually prohibits 3D quantitative analysis. Such quantitative analysis is only possible after deblurring by deconvolution. The resulting image quality is strongly dependent on how accurate the point spread function used for deconvolution resembles the properties of the imaging system. Since direct measurement of the true point spread function is laborious and modelled point spread functions usually deviate from measured ones, we present a method of optimizing the microscope until it meets almost ideal imaging conditions. These conditions are validated by measuring the aberration function of the microscope and tissue using a Shack‐Hartmann sensor. The analysis shows that cortical tissue from rat brains embedded in Mowiol and imaged by an oil‐immersion objective can be regarded as having a homogeneous index of refraction. In addition, the amount of spherical aberration that is caused by the optics or the specimen is relatively low. Consequently the image formation is simplified to refraction between the embedding and immersion medium and to 3D diffraction at the finite entrance pupil of the objective. The resulting model point spread function is applied to the image stacks by linear or iterative deconvolution algorithms. For the presented dataset of large 3D images the linear approach proves to be superior. The linear deconvolution yields a significant improvement in signal‐to‐noise ratio and resolution. This novel approach allows a quantitative analysis of the cortical image stacks such as the reconstruction of biocytin‐stained neuronal dendrites and axons.     

Digital correction of motion artefacts in microscopy image sequences collected from living animals using rigid and nonrigid registration

Digital image analysis is a fundamental component of quantitative microscopy. However, intravital microscopy presents many challenges for digital image analysis. In general, microscopy volumes are inherently anisotropic, suffer from decreasing contrast with tissue depth, lack object edge detail and characteristically have low signal levels. Intravital microscopy introduces the additional problem of motion artefacts, resulting from respiratory motion and heartbeat from specimens imaged in vivo. This paper describes an image registration technique for use with sequences of intravital microscopy images collected in time-series or in 3D volumes. Our registration method involves both rigid and nonrigid components. The rigid registration component corrects global image translations, whereas the nonrigid component manipulates a uniform grid of control points defined by B-splines. Each control point is optimized by minimizing a cost function consisting of two parts: a term to define image similarity, and a term to ensure deformation grid smoothness. Experimental results indicate that this approach is promising based on the analysis of several image volumes collected from the kidney, lung and salivary gland of living rodents.     

Three-dimensional volume reconstruction of extracellular matrix proteins in uveal melanoma from fluorescent confocal laser scanning microscope images

The distribution of looping patterns of laminin in uveal melanomas and other tumours has been associated with adverse outcome. Moreover, these patterns are generated by highly invasive tumour cells through the process of vasculogenic mimicry and are not therefore blood vessels. Nevertheless, these extravascular matrix patterns conduct plasma. The three‐dimensional (3D) configuration of these laminin‐rich patterns compared with blood vessels has been the subject of speculation and intensive investigation. We have developed a method for the 3D reconstruction of volume for these extravascular matrix proteins from serial paraffin sections cut at 4 µm thicknesses and stained with a fluorescently labelled antibody to laminin ( Maniotis et al., 2002 ). Each section was examined via confocal laser‐scanning focal microscopy (CLSM) and 13 images were recorded in the Z‐dimension for each slide. The input CLSM imagery is composed of a set of 3D subvolumes (stacks of 2D images) acquired at multiple confocal depths, from a sequence of consecutive slides. Steps for automated reconstruction included (1) unsupervised methods for selecting an image frame from a subvolume based on entropy and contrast criteria, (2) a fully automated registration technique for image alignment and (3) an improved histogram equalization method that compensates for spatially varying image intensities in CLSM imagery due to photo‐bleaching. We compared image alignment accuracy of a fully automated method with registration accuracy achieved by human subjects using a manual method. Automated 3D volume reconstruction was found to provide significant improvement in accuracy, consistency of results and performance time for CLSM images acquired from serial paraffin sections.     

Three-dimensional surface texture visualization of bone tissue through epifluorescence-based serial block face imaging

Serial block face imaging is a microscopy technique in which the top of a specimen is cut or ground away and a mosaic of images is collected of the newly revealed cross-section. Images collected from each slice are then digitally stacked to achieve 3D images. The development of fully automated image acquisition devices has made serial block face imaging more attractive by greatly reducing labour requirements. The technique is particularly attractive for studies of biological activity within cancellous bone as it has the capability of achieving direct, automated measures of biological and morphological traits and their associations with one another. When used with fluorescence microscopy, serial block face imaging has the potential to achieve 3D images of tissue as well as fluorescent markers of biological activity. Epifluorescence-based serial block face imaging presents a number of unique challenges for visualizing bone specimens due to noise generated by sub-surface signal and local variations in tissue autofluorescence. Here we present techniques for processing serial block face images of trabecular bone using a combination of non-uniform illumination correction, precise tiling of the mosaic in each cross-section, cross-section alignment for vertical stacking, removal of sub-surface signal and segmentation. The resulting techniques allow examination of bone surface texture that will enable 3D quantitative measures of biological processes in cancellous bone biopsies.     

Automated high-resolution three-dimensional fluorescence imaging of large biological specimens

We describe a novel automated technique for visualizing the three‐dimensional distribution of fluorochrome‐labelled components, in which image resolution is uncoupled from specimen size. This method is based on computer numerically controlled milling technology and combines an arrayed imaging technique with fluorescence capabilities. Fluorescent signals are segmented by emission spectra such that multiple fluorochromes present within a single specimen may be reconstructed and visualized individually or as a group. The automated nature of the system minimizes the workload and time involved in image capture and volume reconstruction. As an application, the system was used to image zones of fluorochrome‐labelled microdamage within an 8‐mm diameter cylinder of trabecular bone at a voxel size of 3 × 3 × 8 μm³. Our reconstruction of this specimen provides a visual map and quantitative measures of the volume of damage present throughout the cylinder, clearly demonstrating the interpretive power afforded by three‐dimensional visualization. The three‐dimensional nature of this highly automated and adaptable system has the potential to facilitate new diagnostic tools and techniques with application to a wide range of biological and medical research fields.     

Volume reconstruction of large tissue specimens from serial physical sections using confocal microscopy and correction of cutting deformations by elastic registration

A set of methods leading to volume reconstruction of biological specimens larger than the field of view of a confocal laser scanning microscope (CLSM) is presented. Large tissue specimens are cut into thin physical slices and volume data sets are captured from all studied physical slices by CLSM. Overlapping spatial tiles of the same physical slice are stitched in horizontal direction. Image volumes of successive physical slices are linked in axial direction by applying an elastic registration algorithm to compensate for deformations because of cutting the specimen. We present a method enabling us to keep true object morphology using a priori information about the shape and size of the specimen, available from images of the cutting planes captured by a USB light microscope immediately before cutting the specimen by a microtome. The errors introduced by elastic registration are evaluated using a stereological point counting method and the Procrustes distance. Finally, the images are enhanced to compensate for the effect of the light attenuation with depth and visualized by a hardware accelerated volume rendering. Algorithmic steps of the reconstruction, namely elastic registration, object morphology preservation, image enhancement, and volume visualization, are implemented in a new Rapid3D software package. Because confocal microscopes get more and more frequently used in scientific laboratories, the described volume reconstruction may become an easy‐to‐apply tool to study large biological objects, tissues, and organs in histology, embryology, evolution biology, and developmental biology. In this work, we demonstrate the reconstruction using a postcranial part of a 17‐day‐old laboratory Wistar rat embryo. Microsc. Res. Tech., 2009. © 2008 Wiley‐Liss, Inc.     

Robust, globally consistent and fully automatic multi-image registration and montage synthesis for 3-D multi-channel images

The need to map regions of brain tissue that are much wider than the field of view of the microscope arises frequently. One common approach is to collect a series of overlapping partial views, and align them to synthesize a montage covering the entire region of interest. We present a method that advances this approach in multiple ways. Our method (1) produces a globally consistent joint registration of an unorganized collection of three-dimensional (3-D) multi-channel images with or without stage micrometer data; (2) produces accurate registrations withstanding changes in scale, rotation, translation and shear by using a 3-D affine transformation model; (3) achieves complete automation, and does not require any parameter settings; (4) handles low and variable overlaps (5-15%) between adjacent images, minimizing the number of images required to cover a tissue region; (5) has the self-diagnostic ability to recognize registration failures instead of delivering incorrect results; (6) can handle a broad range of biological images by exploiting generic alignment cues from multiple fluorescence channels without requiring segmentation and (7) is computationally efficient enough to run on desktop computers regardless of the number of images. The algorithm was tested with several tissue samples of at least 50 image tiles, involving over 5000 image pairs. It correctly registered all image pairs with an overlap greater than 7%, correctly recognized all failures, and successfully joint-registered all images for all tissue samples studied. This algorithm is disseminated freely to the community as included with the Fluorescence Association Rules for Multi-Dimensional Insight toolkit for microscopy (http://www.farsight-toolkit.org).     

Fully automated intensity compensation for confocal microscopic images

One well-recognized problem in three-dimensional (3D) confocal microscopic images is that the intensities in deeper slices are generally weaker than those in shallower slices. The loss of intensity with depth hampers both qualitative observation and quantitative measurement of specimens. Two major types of methods exist to compensate for this intensity loss: the first is based on the geometrical optics inside the specimen, and the second applies an empirical parametric intensity decay function (IDF) of depth. A common feature shared by both methods is that they are parameter-dependent. However, for the optics-based method there are as yet no fully automated parameter-setting approaches; and for the IDF method the traditional profile-fitting approach cannot provide proper parameters if the presumed IDF model does not match the experimental intensity–depth profile of the 3D image. In this paper, we propose a novel maximum-entropy (ME) approach to fully automated parameter-setting. In principle the ME approach is suitable for any compensation method as long as it is parameter-dependent. The basic assumption is that without intensity loss an ideal 3D image should be generally homogeneous with respect to depth and this axial homogeneity can be represented by the entropy of a normalized intensity–depth profile. Experiments on real confocal images showed that such a profile was consistent with visual evaluation of axial intensity homogeneity and that the ME approach could provide proper parameters for both compensation methods mentioned above. Moreover, for the IDF method, experiments on both real and simulated data showed that the ME approach could provide more precise parameters than with traditional profile-fitting. The Appendix provides a proof that under certain conditions the global maximization of the profile-entropy is guaranteed.     

Two dimensional photoacoustic imaging based on an acoustic lens and the peak-hold technology

A new method of photoacoustic (PA) imaging based on an acoustic lens and the peak-hold technology is presented in this article. A fast PA imaging system, which consists of an acoustic lens, a 64-element linear transducer array, and a peak detection-and-hold circuit, is developed to obtain the two dimensional (2D) PA images of the experimental samples. By utilizing an acoustic lens, the PA signals generated from the sample are directly imaged on the imaging plane and collected by the 64-element linear transducer array which changes the PA signals into the corresponding electronic signals. Then the electronic signals are converted into a one dimensional image using the peak detection-and-hold circuit. After vertical scanning with a step motor on the imaging plane, the 2D PA image of the sample is achieved successfully. The results show that the images reconstructed in this experiment agree well with the original samples. Compared to other methods, this PA imaging system can acquire the PA images more rapidly without any complex algorithms, and it may provide a more convenient method for future in vivo noninvasive imaging of tissues and clinic diagnosis.     

Automated microscopy system for mosaic acquisition and processing

An automatic mosaic acquisition and processing system for a multiphoton microscope is described for imaging large expanses of biological specimens at or near the resolution limit of light microscopy. In a mosaic, a larger image is created from a series of smaller images individually acquired systematically across a specimen. Mosaics allow wide‐field views of biological specimens to be acquired without sacrificing resolution, providing detailed views of biological specimens within context. The system is composed of a fast‐scanning, multiphoton, confocal microscope fitted with a motorized, high‐precision stage and custom‐developed software programs for automatic image acquisition, image normalization, image alignment and stitching. Our current capabilities allow us to acquire data sets comprised of thousands to tens of thousands of individual images per mosaic. The large number of individual images involved in creating a single mosaic necessitated software development to automate both the mosaic acquisition and processing steps. In this report, we describe the methods and challenges involved in the routine creation of very large scale mosaics from brain tissue labelled with multiple fluorescent probes.     

Three dimensional image restoration in fluorescence lifetime imaging microscopy

A microscope set-up and numerical methods are described which enable the measurement and reconstruction of three-dimensional nanosecond fluorescence lifetime images in every voxel. The frequency domain fluorescence lifetime imaging microscope (FLIM) utilizes phase detection of high-frequency modulated light by homodyne mixing on a microchannel plate image intensifier. The output signal at the image intensifier's phosphor screen is integrated on a charge coupled device camera. A scanning stage is employed to obtain a series of phase-dependent intensity images at equally separated depths in a specimen. The Fourier transform of phase-dependent data gives three-dimensional (3D) images of the Fourier coefficients. These images are deblurred using an Iterative Constrained Tikhonov–Miller (ICTM) algorithm in conjunction with a measured point spread function. The 3D reconstruction of fluorescence lifetimes are calculated from the deblurred images of the Fourier coefficients. An improved spatial and temporal resolution of fluorescence lifetimes was obtained using this approach to the reconstruction of simulated 3D FLIM data. The technique was applied to restore 3D FLIM data of a live cell specimen expressing two green fluorescent protein fusion constructs having distinct fluorescence lifetimes which localized to separate cellular compartments.     

Blind deconvolution of 3D fluorescence microscopy using depth‐variant asymmetric PSF

The 3D wide‐field fluorescence microscopy suffers from depth‐variant asymmetric blur. The depth‐variance and axial asymmetry are due to refractive index mismatch between the immersion and the specimen layer. The radial asymmetry is due to lens imperfections and local refractive index inhomogeneities in the specimen. To obtain the PSF that has these characteristics, there were PSF premeasurement trials. However, they are useless since imaging conditions such as camera position and refractive index of the specimen are changed between the premeasurement and actual imaging. In this article, we focus on removing unknown depth‐variant asymmetric blur in such an optical system under the assumption of refractive index homogeneities in the specimen. We propose finding few parameters in the mathematical PSF model from observed images in which the PSF model has a depth‐variant asymmetric shape. After generating an initial PSF from the analysis of intensities in the observed image, the parameters are estimated based on a maximum likelihood estimator. Using the estimated PSF, we implement an accelerated GEM algorithm for image deconvolution. Deconvolution result shows the superiority of our algorithm in terms of accuracy, which quantitatively evaluated by FWHM, relative contrast, standard deviation values of intensity peaks and FWHM. Microsc. Res. Tech. 79:480–494, 2016. © 2016 Wiley Periodicals, Inc.     

Surface imaging microscopy using an ultramiller for large volume 3D reconstruction of wax- and resin-embedded tissues

Three-dimensional reconstruction of large tissue volumes using histological thin sections poses difficulties because of registration of sections, section distortion, and the possibility of incomplete data set collection due to section loss. We have constructed an integrated surface imaging system that successfully addresses these problems. Embedded tissue is mounted on a high precision XYZ stage and the upper surface is iteratively: (i) stained to provide an effective optical section, (ii) imaged using a digital camera, and (iii) removed with an ultramiller. This approach provides for the reconstruction of high-quality 3D images by inherently preserving image registration, eliminates section distortion, thus removing the need for complex realignment and correction, and also ensures full capture of all image planes. The system has the capacity to acquire images of tissue structure with voxel sizes from 0.5 to 50 mum over dimensions ranging from micrometers to tens of millimeters. The ultramiller enables large samples to be imaged by reliably removing tissue over their full extent. The ability to visualize key features of 3D tissue structure across such a range of scale and resolution will facilitate the development of a greater understanding of the relationship between structure and function. This understanding is essential for better analyses of the structural changes associated with different disease states, and the development of structure-based computer models of biological function.