Correction of image drift and distortion in a scanning electron microscopy

Continuous research on small‐scale mechanical structures and systems has attracted strong demand for ultrafine deformation and strain measurements. Conventional optical microscope cannot meet such requirements owing to its lower spatial resolution. Therefore, high‐resolution scanning electron microscope has become the preferred system for high spatial resolution imaging and measurements. However, scanning electron microscope usually is contaminated by distortion and drift aberrations which cause serious errors to precise imaging and measurements of tiny structures. This paper develops a new method to correct drift and distortion aberrations of scanning electron microscope images, and evaluates the effect of correction by comparing corrected images with scanning electron microscope image of a standard sample. The drift correction is based on the interpolation scheme, where a series of images are captured at one location of the sample and perform image correlation between the first image and the consequent images to interpolate the drift–time relationship of scanning electron microscope images. The distortion correction employs the axial symmetry model of charged particle imaging theory to two images sharing with the same location of one object under different imaging fields of view. The difference apart from rigid displacement between the mentioned two images will give distortion parameters. Three‐order precision is considered in the model and experiment shows that one pixel maximum correction is obtained for the employed high‐resolution electron microscopic system.     

Shack-Hartmann wave front measurements in cortical tissue for deconvolution of large three-dimensional mosaic transmitted light brightfield micrographs

We present a novel approach for deconvolution of 3D image stacks of cortical tissue taken by mosaic/optical‐sectioning technology, using a transmitted light brightfield microscope. Mosaic/optical‐sectioning offers the possibility of imaging large volumes (e.g. from cortical sections) on a millimetre scale at sub‐micrometre resolution. However, a blurred contribution from out‐of‐focus light results in an image quality that usually prohibits 3D quantitative analysis. Such quantitative analysis is only possible after deblurring by deconvolution. The resulting image quality is strongly dependent on how accurate the point spread function used for deconvolution resembles the properties of the imaging system. Since direct measurement of the true point spread function is laborious and modelled point spread functions usually deviate from measured ones, we present a method of optimizing the microscope until it meets almost ideal imaging conditions. These conditions are validated by measuring the aberration function of the microscope and tissue using a Shack‐Hartmann sensor. The analysis shows that cortical tissue from rat brains embedded in Mowiol and imaged by an oil‐immersion objective can be regarded as having a homogeneous index of refraction. In addition, the amount of spherical aberration that is caused by the optics or the specimen is relatively low. Consequently the image formation is simplified to refraction between the embedding and immersion medium and to 3D diffraction at the finite entrance pupil of the objective. The resulting model point spread function is applied to the image stacks by linear or iterative deconvolution algorithms. For the presented dataset of large 3D images the linear approach proves to be superior. The linear deconvolution yields a significant improvement in signal‐to‐noise ratio and resolution. This novel approach allows a quantitative analysis of the cortical image stacks such as the reconstruction of biocytin‐stained neuronal dendrites and axons.     

Correlative light‐electron fractography for fatigue striations characterization in metallic alloys

The correlative light‐electron fractography technique combines correlative microscopy concepts to the extended depth‐from‐focus reconstruction method, associating the reliable topographic information of 3‐D maps from light microscopy ordered Z‐stacks to the finest lateral resolution and large focus depth from scanning electron microscopy. Fatigue striations spacing analysis can be precisely measured, by correcting the mean surface tilting with the knowledge of local elevation data from elevation maps. This new technique aims to improve the accuracy of quantitative fractography in fatigue fracture investigations. Microsc. Res. Tech. 76:909–913, 2013. © 2013 Wiley Periodicals, Inc.     

Optical far-field microscopy of single molecules with 3.4 nm lateral resolution

Optical far‐field imaging of single molecules in a frozen solution at 1.2 K with a lateral resolution of 3.4 nm is reported. The mechanical stability of the fluorescence microscope, especially of the low‐temperature insert, allows for the localization of fluorescing molecules with a reproducibility of better than 5 nm within observation times up to 10 min. For observation times of 9 h the reproducibility of the lateral position is limited to about 20 nm due to mechanical drift. Lateral position and orientation of 314 single molecules, present within the confocal detection volume of ~10 µm³, are obtained. The possibility to correct for mechanical drift by monitoring the position of a spatial reference in the sample is demonstrated.     

Calibration of the automated z-axis of a microscope using focus functions

Applications in automated microscopy and three-dimensional microscopy require careful calibration of the microscope system. This paper presents methods for calibration of the motorized z -axis (focus or optical axis) of an automated microscope. Apart from the automated microscope the procedures require a CCD camera and a test slide containing a simple bar pattern. The calibration embraces the following characteristics of the z -axis: (a) measuring the motor step size in nanometres; (b) measuring the mechani-cal backlash in the focus mechanism of the microscope and (c) measuring the reproducibility and the stability of the focus position over time. The measurements employ focus functions to determine the z -position of the microscope stage.     

Extended depth of focus in optical microscopy: Assessment of existing methods and a new proposal

Due to depth of focus constraints, the acquisition of a single 2‐D completely in‐focus image of 3‐D objects characterized by a relevant depth dimension is not possible with a standard light microscope. Since the Seventies numerous methods have been proposed to overcome this problem, mainly through different fusion processing techniques to extend the microscope's depth of focus. However, given a specific application, it is very difficult to know which method yields the best results because there are no validated approaches or tested metrics that are suitable for real world cases typically lacking in a reference ground truth. Although the Universal Quality Index (UQI) is widely used to evaluate output quality in image processing, it requires a reference ground truth. Some UQI extensions have been proposed to evaluate the output of fusion methods without a ground truth, but sufficient analyses have not been carried out to confirm their equivalence to the standard UQI in terms of (evaluation) performance. We propose a new method to extend the microscope's depth of focus and, using synthetic stacks of images with ground truth attached, show that it is superior to state‐of‐the‐art methods. We also demonstrate that the output of metrics proposed as UQI extensions is different from that of the UQI. Finally, we validate a new approach to evaluate extended depth of focus methods using real world stacks of slices, as per the UQI, but without the need for a reference ground truth. Microsc. Res. Tech. © 2012 Wiley Periodicals, Inc.     

A self‐adaptive and nonmechanical motion autofocusing system for optical microscopes

For the design of a passive autofocusing (AF) system for optical microscopes, many time‐consuming and tedious experiments have been performed to determine and design a better focus criterion function, owing to the sample‐dependence of this function. To accelerate the development of the AF systems in optical microscopes and to increase AF speed as well as maintain the AF accuracy, this study proposes a self‐adaptive and nonmechanical motion AF system. The presented AF system does not require the selection and design of a focus criterion function when it is developed. Instead, the system can automatically determine a better focus criterion function for an observed sample by analyzing the texture features of the sample and subsequently perform an AF procedure to bring the sample into focus in the objective of an optical microscope. In addition, to increase the AF speed, the Z axis scanning of the mechanical motion of the sample or the objective is replaced by focusing scanning performed by a liquid lens, which is driven by an electrical current and does not involve mechanical motion. Experiments show that the reproducibility of the results obtained with the proposed self‐adaptive and nonmechanical motion AF system is better than that provided by that of traditional AF systems, and that the AF speed is 10 times faster than that of traditional AF systems. Also, the self‐adaptive function increased the speed of AF process by an average of 10.5% than Laplacian and Tenegrad functions.     

Contour extraction of a laser stripe located on a microscope image from a stereo light microscope

A stereo light microscope has a special optical structure that consists of two optical paths. With two cameras fitted on its imaging planes, a microscopic binocular vision system can be constructed, and this system can be applied in three‐dimensional (3D) shape measurements of a microscopic object. A novel‐shape reconstruction system is developed in this article composed of laser fringe scanning and a stereo light microscope. A laser projector emits a thin light sheet and projects it onto the microscopic object's surface. The microscopic object is placed on a micro‐displacement mechanism and moved in a given direction. The entire surface of the object is scanned by the light sheet. This system captures a series of microscopic images of the laser stripe, which are used to restore the 3D shape of the object. In this article, we mainly focus on the study of the laser stripe detection method, which is derived based on the Canny rule. First, the Canny rule outputs pixels at the left and right edges of the laser stripe. Then, a subpixel edge extraction method is proposed based on polynomial fitting that outputs the center curve of the laser stripe. Finally, an edge filter is used to smooth the edge burrs, and the Hermite interpolation method is used to link the broken edges and construct continuous, smoothing edge contours. The results show that this method can effectively find the subpixel position of the laser stripe and output high‐quality edge contours. The method is suitable for extracting the contours of a laser stripe located in a microscope image.     

Five-colour in vivo imaging of neurons in Caenorhabditis elegans

In the last few years variants of the ‘green fluorescent protein’ (GFP) with different spectral properties have been generated. This has greatly increased the number of possible applications for these fluorochromes in cell biology. The significant overlap of the excitation and emission spectra of the different GFP variants imposes constraints on the number of variants that can be used simultaneously in a single sample. In particular, the two brightest variants, GFP and YFP, are difficult to separate spectrally. This study shows that GFP and YFP can be readily separated with little spectral overlap (cross‐talk) with the use of a confocal microscope equipped with an acusto‐optical beam splitter and freely adjustable emission windows. Under optimal recording conditions cross‐talk is less than 10%. Together with two other fluorescent proteins and the lipophilic dye DiD a total of five different colours can now be used simultaneously to label in vivo distinct anatomical structures such as neurons and their processes. Spatial resolution of the confocal microscope is sufficient to resolve the relative position of labelled axons within a single axon bundle. The use of five distinct marker dyes allows the in vivo analysis of the Caenorhabditis elegans nervous system at unprecedented resolution and richness in detail at the light microscopic level.     

Automated in‐chamber specimen coating for serial block‐face electron microscopy

When imaging insulating specimens in a scanning electron microscope, negative charge accumulates locally (‘sample charging’). The resulting electric fields distort signal amplitude, focus and image geometry, which can be avoided by coating the specimen with a conductive film prior to introducing it into the microscope chamber. This, however, is incompatible with serial block‐face electron microscopy (SBEM), where imaging and surface removal cycles (by diamond knife or focused ion beam) alternate, with the sample remaining in place. Here we show that coating the sample after each cutting cycle with a 1–2 nm metallic film, using an electron beam evaporator that is integrated into the microscope chamber, eliminates charging effects for both backscattered (BSE) and secondary electron (SE) imaging. The reduction in signal‐to‐noise ratio (SNR) caused by the film is smaller than that caused by the widely used low‐vacuum method. Sample surfaces as large as 12 mm across were coated and imaged without charging effects at beam currents as high as 25 nA. The coatings also enabled the use of beam deceleration for non‐conducting samples, leading to substantial SNR gains for BSE contrast. We modified and automated the evaporator to enable the acquisition of SBEM stacks, and demonstrated the acquisition of stacks of over 1000 successive cut/coat/image cycles and of stacks using beam deceleration or SE contrast.     

Accurate shape from focus based on focus adjustment in optical microscopy

Optical microscopy allows a magnified view of the sample while decreasing the depth of focus. Although the acquired images from limited depth of field have both blurred and focused regions, they can provide depth information. The technique to estimate the depth and 3D shape of an object from the images of the same sample obtained at different focus settings is called shape from focus (SFF). In SFF, the measure of focus–sharpness–is the crucial part for final 3D shape estimation. The conventional methods compute sharpness by applying focus measure operator on each 2D image frame of the image sequence. However, such methods do not reflect the accurate focus levels in an image because the focus levels for curved objects require information from neighboring pixels in the adjacent frames too. To address this issue, we propose a new method based on focus adjustment which takes the values of the neighboring pixels from the adjacent image frames that have approximately the same initial depth as of the center pixel and then it re-adjusts the center value accordingly. Experiments were conducted on synthetic and microscopic objects, and the results show that the proposed technique generates better shape and takes less computation time in comparison with previous SFF methods based on focused image surface (FIS) and dynamic programming. Microsc. Res. Tech., 2009. © 2008 Wiley-Liss, Inc.     

Correcting the axial shrinkage of skeletal muscle thick sections visualized by confocal microscopy

Confocal microscopy is a suitable method for measurements and visualization of skeletal muscle fibres and the neighbouring capillaries. When using 3D images of thick sections the tissue deformation effects should be avoided. We studied the deformation in thick sections of the rat skeletal muscle from complete stacks of images captured with confocal microscope. We measured the apparent thickness of the stacks and compared it to the slice thickness deduced from calibrated microtome settings. The ratio of both values yielded the axial scaling factor for every image stack. Careful sample preparation and treatment of the tissue cryosections with cold Ringer solution minimize the tissue deformation. We conclude that rescaling by the inverse of the axial scaling factor of the stack of optical slices in the direction of the microscope optical axis satisfactorily corrects the axial deformation of skeletal muscle samples.     

Thermally induced volumetric error modeling based on thermal drift and its compensation in Z-axis

Traditional thermally induced volumetric error modeling requires 21 geometric error components at different temperatures. Taking thermal drift errors into account, 30 geometric errors are described to model volumetric error in this paper. The main sources of thermally induced volumetric error are positioning errors of each axis and thermal drift errors. An experiment on milling and boring machine is carried out, which shows that volumetric error in Z-axis is affected significantly. To compensate volumetric error of Z-axis, a model of positioning error is proposed based on nut temperature. A finite elements analysis of headstock thermal characteristic is carried out, and error chain is established, which shows the main source of thermal drift of Z-axis is ram expansion. Thermal drift compensation system of Z-axis is developed based on Invar metal and thermal error compensation module of Siemens CNC system. Therefore, the positioning error and thermal drift of Z-axis are compensated. The thermally induced volumetric error in Z-axis is reduced by 80 % after compensation.     

Depth discrimination in scanned heterodyne microscope systems

The optical transfer function of several scanning microscope systems is derived, using a physically intuitive approach. The technique allows a wide range of systems to be modelled with only minor modifications to the basic formulation. The results are then used to determine the response of various scanning microscopes for objects both in and out of the focal plane. The possibility of performing extended-focus phase imaging in heterodyne microscopes by scanning the sample along the optical axis is also examined. This mode of operation should allow measurements of minute topographical and phase variations on tilted or warped samples with the same lateral resolution as would be obtained when the sample is in focus throughout the entire scan.     

Improved depth of field in the scanning electron microscope derived from through-focus image stacks

The depth of field limit in the scanning electron microscope (SEM) can be overcome by recording stacks of through-focus images (as in conventional and confocal optical microscopy) which are postprocessed to generate an all-in-focus image. Images are recorded under constant electron optical conditions by mechanical Z-axis movement of the sample. This gives rise to a change in magnification through the stack due to the perspective projection of the SEM image. Calculation of the necessary scaling as well as the derivation of best focus information at every patch in the image--and a contour map function derived from the selected patch depths--are incorporated in a new software package (Auto-Montage Pro). The utility of these procedures is demonstrated with examples from the study of human osteoporotic bone, where results show uncoupling of resorption and formation. The procedure can be combined with pseudo-colour coding for the direction of apparent illumination when using backscattered electron (BSE) detectors in contrasting positions.     

Constant tip-surface distance with atomic force microscopy via quality factor feedback

The atomic force microscope (AFM) is a powerful and widely used instrument to image topography and measure forces at the micrometer and nanometer length scale. Because of the high degree of operating accuracy required of the instrument, small thermal and mechanical drifts of the cantilever and piezoactuator systems hamper measurements as the AFM tip drifts spatially relative to the sample surface. To compensate for the drift, we control the tip-surface distance by monitoring the cantilever quality factor (Q) in a closed loop. Brownian thermal fluctuations provide sufficient actuation to accurately determine cantilever Q by fitting the thermal noise spectrum to a Lorentzian function. We show that the cantilever damping is sufficiently affected by the tip-surface distance so that the tip position of soft cantilevers can be maintained within 40 nm of a setpoint in air and within 3 nm in water with 95% reliability. Utilizing this method to hover the tip above a sample surface, we have the capability to study sensitive interactions at the nanometer length scale over long periods of time.     

Digital holographic microscopy and focusing methods based on image sharpness

Digital holographic microscope allows imaging of opaque and transparent specimens without staining. A digitally recorded hologram must be reconstructed numerically at the actual depth of the object to obtain a focused image. We have developed a high‐resolution digital holographic microscope for imaging amplitude and phase objects with autofocusing capability. If the actual depth of an object is not known a priori, it is estimated by comparing the sharpness of several reconstructions at different distances, which is very demanding in means of computational power when the recorded hologram is large. In this paper, we present 11 different sharpness metrics for estimating the actual focus depths of objects. The speed performance of focusing is discussed, and a scaling technique is introduced where the speed of autofocusing increases on the order of square of the scale ratio. We measured the performance of scaling on computer‐generated holograms and on recorded holograms of a biological sample. We show that simulations are in good agreement with the experimental results.     

Low axial drift stage and temperature controlled liquid cell for z-scan fluorescence correlation spectroscopy in an inverted confocal geometry

A recent iteration of fluorescence correlation spectroscopy (FCS), z-scan FCS, has drawn attention for its elegant solution to the problem of quantitative sample positioning when investigating two-dimensional systems while simultaneously providing an excellent method for extracting calibration-free diffusion coefficients. Unfortunately, the measurement of planar systems using (FCS and) z-scan FCS still requires extremely mechanically stable sample positioning, relative to a microscope objective. As axial sample position serves as the inherent length calibration, instabilities in sample position will affect measured diffusion coefficients. Here, we detail the design and function of a highly stable and mechanically simple inverted microscope stage that includes a temperature controlled liquid cell. The stage and sample cell are ideally suited to planar membrane investigations, but generally amenable to any quantitative microscopy that requires low drift and excellent axial and lateral stability. In the present work we evaluate the performance of our custom stage system and compare it with the stock microscope stage and typical sample sealing and holding methods.     

Extended depth from focus reconstruction using NIH ImageJ plugins: Quality and resolution of elevation maps

In this work, NIH ImageJ plugins for extended depth‐from‐focus reconstructions (EDFR) based on spatial domain operations were compared and tested for usage optimization. Also, some preprocessing solutions for light microscopy image stacks were evaluated, suggesting a general routine for the ImageJ user to get reliable elevation maps from grayscale image stacks. Two reflected light microscope image stacks were used to test the EDFR plugins: one bright‐field image stack for the fracture of carbon‐epoxy composite and its darkfield corresponding stack at same (x,y,z) spatial coordinates. Image quality analysis consisted of the comparison of signal‐to‐noise ratio and resolution parameters with the consistence of elevation maps, based on roughness and fractal measurements. Darkfield illumination contributed to enhance the homogeneity of images in stack and resulting height maps, reducing the influence of digital image processing choices on the dispersion of topographic measurements. The subtract background filter, as a preprocessing tool, contributed to produce sharper focused images. In general, the increasing of kernel size for EDFR spatial domain‐based solutions will produce smooth height maps. Finally, this work has the main objective to establish suitable guidelines to generate elevation maps by light microscopy. Microsc. Res. Tech. 2012. © 2012 Wiley Periodicals, Inc.     

A prototype system of three-dimensional non-contact measurement

A prototype 3D measurement system is proposed in this paper which consists of a 1D laser displacement sensor, a 2D image system and a servo controlX,Y-table. The laser sensor and CCD camera are installed on theZ-axis perpendicular to theX,Y-table. The image-processing system employs the adaptive resonant theory (ART) neural networks to classify the outer shape of the measured object. The edge values of the object are then obtained by using the image processing procedures of sliding, stretching, edge enhancement and binary disposal. The 2D dimensions of the object are searched by employing the Hough theory based upon the edge values. The 3D dimensions of the object are measured and assembled by combining theX,Y-coordinates of the table and the scanning results of the 1D laser for the height of theZ-axis. A 3D plaster model is chosen as the specimen for non-contact measurement to verify the feasibility of this approach. The limitations and resolution of the 3D measurement of this system are discussed.