Automated cell nuclei segmentation for breast fine needle aspiration cytology

Breast cancer detection and segmentation of cytological images is the standard clinical practice for the diagnosis and prognosis of breast cancer. This paper presents a fully automated method for cell nuclei detection and segmentation in breast cytological images. The images are enhanced with histogram stretching and contrast-limited adaptive histogram equalization (CLAHE). The locations of the cell nuclei in the image are detected with circular Hough transform (CHT) and local maximum filtering. The elimination of false positive findings (noisy circles and blood cells) is achieved using Otsu’s thresholding method and fuzzy C-means clustering technique. The segmentation of the nuclei boundaries is accomplished with the application of the marker controlled watershed transform in the gradient image, using the nuclei markers extracted in the detection step. The proposed method is evaluated using 92 breast cytological images containing 11,502 cell nuclei. Experimental evidence shows that the proposed method has very effective results even in the case of images with high degree of blood cells, noisy circles.