A Novel Probabilistic Nuclei Segmentation Algorithm for H&E Stained Histopathological Tissue Images

In this study, we propose a novel, fast and accurate segmentation algorithm to segment nuclei in H&E stained histopathological tissue images. The proposed algorithm does not require pre-processing, post-processing, and any manual parameter or threshold. The algorithm utilizes probabilistic and statistical properties of the pixels' color value in the images with RGB color space, and determines whether pixels are a part of any nuclei or not by using an automatically calculated threshold value. The algorithm provides time efficiency and reduced overall cost in the segmentation. Two more algorithms are also proposed to distinguish nuclei cluster from the other clusters obtained by K-means, and eliminate false positives in nuclei cluster, which are not nuclei. In order to compare and evaluate the performance of the proposed segmentation algorithm in terms of time and cost efficiency, K -Means is preferred because of its common usage. Expert evaluation is declared as ground truth for determining the accuracy of the results. The experiments are performed on 60 healthy and 60 damaged kidney, and 60 healthy and 60 damaged liver tissue images. The evaluations show that the proposed algorithm can effectively segment nuclei. The comparison results also demonstrate that the deviation between proposed algorithm and the expert is 2%, while the deviation between K -Means and expert is 5%.