Segmentation and Classification of Breast Cancer Histopathological Image Utilizing U-Net and Transfer Learning ResNet50
Keywords:
Breast, Cancer, Classification, Segmentation, Transfer Learning, U-Net
Abstract
Breast cancer is the most common type of cancer among various types of cancer. Approximately 1 in 8 women in the United States die from breast cancer. Early screening and accurate diagnosis are essential for prevention and accelerated treatment intervention. Several artificial intelligence methods have emerged to develop effective segmentation, detection, and classification to determine cancer types. Although there has been progress in automated algorithms for breast cancer histopathology image analysis, many of these approaches still face several challenges. This study aims to address the challenges in breast cancer image analysis. This research method uses the development of the U-Net architecture combined with Transfer Learning using ResNet50. The encoder path aims to improve the model’s sensitivity in the segmentation and classification of cancer areas by utilizing deep hierarchical features extracted by ResNet50. In addition, data augmentation techniques are used to create a diverse and comprehensive training dataset, which improves the model’s ability to distinguish between different tissue types and cancer areas. The results of this study are U-Net and ResNet50, which show an average IoU of 0.482 and a Dice coefficient of 0.916. This study concludes that integrating UNet with Transfer Learning ResNet50 improves the segmentation and classification accuracy in breast cancer histopathology images and overcomes the problem of high computational requirements. This approach shows significant potential for improvement in early breast cancer detection and diagnosis.
Downloads
Download data is not yet available.
References
[1] K. B. Soulami, N. Kaabouch, M. N. Saidi, and A. Tamtaoui, “Breast cancer: One-stage automated detection, segmentation,
and classification of digital mammograms using UNet model based-semantic segmentation,” Biomedical Signal Processing
and Control, vol. 66, pp. 1–12, Apr. 2021, https://doi.org/10.1016/j.bspc.2021.102481.
[2] N. M. U. Din, R. A. Dar, M. Rasool, and A. Assad, “Breast cancer detection using deep learning: Datasets,
methods, and challenges ahead,” Computers in Biology and Medicine, vol. 149, pp. 1–23, Oct. 2022, https:
//doi.org/10.1016/j.compbiomed.2022.106073.
[3] Y. Zhao, J. Zhang, D. Hu, H. Qu, Y. Tian, and X. Cui, “Application of Deep Learning in Histopathology Images of Breast
Cancer: A Review,” Micromachines, vol. 13, no. 12, pp. 1–30, Dec. 2022, https://doi.org/10.3390/mi13122197.
[4] K. Rautela, D. Kumar, and V. Kumar, “A Systematic Review on Breast Cancer Detection Using Deep Learning
Techniques,” Archives of Computational Methods in Engineering, vol. 29, no. 7, pp. 4599–4629, Nov. 2022,
https://doi.org/10.1007/s11831-022-09744-5.
[5] R. Krithiga and P. Geetha, “Breast Cancer Detection, Segmentation and Classification on Histopathology Images Analysis:
A Systematic Review,” Archives of Computational Methods in Engineering, vol. 28, no. 4, pp. 2607–2619, Jun. 2021,
https://doi.org/10.1007/s11831-020-09470-w.
[6] Z. Rezaei, “A review on image-based approaches for breast cancer detection, segmentation, and classification,” Expert Systems
with Applications, vol. 182, pp. 1–12, Nov. 2021, https://doi.org/10.1016/j.eswa.2021.115204.
[7] R. Ranjbarzadeh, S. Dorosti, S. Jafarzadeh Ghoushchi, A. Caputo, E. B. Tirkolaee, S. S. Ali, Z. Arshadi, and M. Bendechache,
“Breast tumor localization and segmentation using machine learning techniques: Overview of datasets, findings, and methods,”
Computers in Biology and Medicine, vol. 152, pp. 1–20, Jan. 2023, https://doi.org/10.1016/j.compbiomed.2022.106443.
[8] J. De Matos, S. Ataky, A. De Souza Britto, L. Soares De Oliveira, and A. Lameiras Koerich, “Machine Learning
Methods for Histopathological Image Analysis: A Review,” Electronics, vol. 10, no. 5, pp. 1–42, Feb. 2021,
https://doi.org/10.3390/electronics10050562.
[9] V. Mohanakurup, S. M. Parambil Gangadharan, P. Goel, D. Verma, S. Alshehri, R. Kashyap, and B. Malakhil, “Breast
Cancer Detection on Histopathological Images Using a Composite Dilated Backbone Network,” Computational Intelligence
and Neuroscience, vol. 2022, pp. 1–10, Jul. 2022, https://doi.org/10.1155/2022/8517706.
[10] E. Michael, H. Ma, H. Li, F. Kulwa, and J. Li, “Breast Cancer Segmentation Methods: Current Status and Future Potentials,”
BioMed Research International, vol. 2021, pp. 1–29, Jul. 2021, https://doi.org/10.1155/2021/9962109.
[11] X. Lu and X. Zhu, “Automatic segmentation of breast cancer histological images based on dual-path feature extraction network,”
Mathematical Biosciences and Engineering, vol. 19, no. 11, pp. 11 137–11 153, 2022, https://doi.org/10.3934/mbe.2022519.
[12] A. Juhong, B. Li, C.-Y. Yao, C.-W. Yang, D. W. Agnew, Y. L. Lei, X. Huang, W. Piyawattanametha, and Z. Qiu,
“Super-resolution and segmentation deep learning for breast cancer histopathology image analysis,” Biomedical Optics
Express, vol. 14, no. 1, pp. 18–36, Jan. 2023, https://doi.org/10.1364/BOE.463839.
[13] T. Ilyas, Z. I. Mannan, A. Khan, S. Azam, H. Kim, and F. De Boer, “TSFD-Net: Tissue specific feature
distillation network for nuclei segmentation and classification,” Neural Networks, vol. 151, pp. 1–15, Jul. 2022,
https://doi.org/10.1016/j.neunet.2022.02.020.
[14] A. Pedersen, E. Smistad, T. V. Rise, V. G. Dale, H. S. Pettersen, T.-A. S. Nordmo, D. Bouget, I. Reinertsen, and M. Valla,
“H2G-Net: A multi-resolution refinement approach for segmentation of breast cancer region in gigapixel histopathological
images,” Frontiers in Medicine, vol. 9, pp. 1–13, Sep. 2022, https://doi.org/10.3389/fmed.2022.971873.
[15] M.-A. Khalil, Y.-C. Lee, H.-C. Lien, Y.-M. Jeng, and C.-W. Wang, “Fast Segmentation of Metastatic Foci in
H&E Whole-Slide Images for Breast Cancer Diagnosis,” Diagnostics, vol. 12, no. 4, pp. 1–16, Apr. 2022,
https://doi.org/10.3390/diagnostics12040990.
[16] X. Zhang, X. Zhu, K. Tang, Y. Zhao, Z. Lu, and Q. Feng, “DDTNet: A dense dual-task network for tumor-infiltrating
lymphocyte detection and segmentation in histopathological images of breast cancer,” Medical Image Analysis, vol. 78, pp.
1–17, May 2022, https://doi.org/10.1016/j.media.2022.102415.
[17] B. M. Priego-Torres, D. Sanchez-Morillo, M. A. Fernandez-Granero, and M. Garcia-Rojo, “Automatic segmentation of
whole-slide H&E stained breast histopathology images using a deep convolutional neural network architecture,” Expert
Systems with Applications, vol. 151, pp. 1–14, Aug. 2020, https://doi.org/10.1016/j.eswa.2020.113387.
[18] I. Ahmad, Y. Xia, H. Cui, and Z. U. Islam, “DAN-NucNet: A dual attention based framework for nuclei segmentation in
cancer histology images under wild clinical conditions,” Expert Systems with Applications, vol. 213, pp. 1–8, Mar. 2023,
https://doi.org/10.1016/j.eswa.2022.118945.
[19] W. R. Drioua, N. Benamrane, and L. Sais, “Breast Cancer Histopathological Images Segmentation Using Deep Learning,”
Sensors, vol. 23, no. 17, pp. 1–14, Aug. 2023, https://doi.org/10.3390/s23177318.
[20] S. Graham, Q. D. Vu, S. E. A. Raza, A. Azam, Y. W. Tsang, J. T. Kwak, and N. Rajpoot, “Hover-Net: Simultaneous
segmentation and classification of nuclei in multi-tissue histology images,” Medical Image Analysis, vol. 58, pp. 1–18, Dec.
2019, https://doi.org/10.1016/j.media.2019.101563.
[21] W. Zhang and J. Zhang, “AugHover-Net: Augmenting Hover-net for Nucleus Segmentation and Classification,” arXiv Preprint,
pp. 1–4, Apr. 2022, https://doi.org/abs/2203.03415.
[22] F. H¨orst, M. Rempe, L. Heine, C. Seibold, J. Keyl, G. Baldini, S. Ugurel, J. Siveke, B. Gr¨unwald, J. Egger, and J. Kleesiek,
“CellViT: Vision Transformers for precise cell segmentation and classification,” Medical Image Analysis, vol. 94, pp. 1–16,
May 2024, https://doi.org/10.1016/j.media.2024.103143.
[23] C. Tommasino, C. Russo, A. M. Rinaldi, and F. Ciompi, “”HoVer-UNet”: Accelerating Hovernet with Unet-Based Multi-Class
Nuclei Segmentation Via Knowledge Distillation,” in 2024 IEEE International Symposium on Biomedical Imaging (ISBI).
Athens, Greece: IEEE, May 2024, pp. 1–4, https://doi.org/10.1109/ISBI56570.2024.10635755.
[24] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Medical
Image Computing and Computer-Assisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F.
Frangi, Eds. Cham: Springer International Publishing, 2015, vol. 9351, pp. 234–241.
[25] N. Wahab, A. Khan, and Y. S. Lee, “Transfer learning based deep CNN for segmentation and detection of mitoses in breast
cancer histopathological images,” Microscopy, vol. 68, no. 3, pp. 216–233, Jun. 2019, https://doi.org/10.1093/jmicro/dfz002.
[26] J. Gamper, N. A. Koohbanani, K. Benes, S. Graham, M. Jahanifar, S. A. Khurram, A. Azam, K. Hewitt, and N. Rajpoot,
“PanNuke Dataset Extension, Insights and Baselines,” arXiv Preprint, pp. 1–12, Apr. 2020, https://doi.org/abs/2003.10778.
[27] B. Abhisheka, S. K. Biswas, and B. Purkayastha, “A Comprehensive Review on Breast Cancer Detection, Classification and
Segmentation Using Deep Learning,” Archives of Computational Methods in Engineering, vol. 30, no. 8, pp. 5023–5052, Nov.
2023, https://doi.org/10.1007/s11831-023-09968-z.
[28] T. Alam, W.-C. Shia, F.-R. Hsu, and T. Hassan, “Improving Breast Cancer Detection and Diagnosis through Semantic
Segmentation Using the Unet3+ Deep Learning Framework,” Biomedicines, vol. 11, no. 6, pp. 1–12, May 2023,
https://doi.org/10.3390/biomedicines11061536.
and classification of digital mammograms using UNet model based-semantic segmentation,” Biomedical Signal Processing
and Control, vol. 66, pp. 1–12, Apr. 2021, https://doi.org/10.1016/j.bspc.2021.102481.
[2] N. M. U. Din, R. A. Dar, M. Rasool, and A. Assad, “Breast cancer detection using deep learning: Datasets,
methods, and challenges ahead,” Computers in Biology and Medicine, vol. 149, pp. 1–23, Oct. 2022, https:
//doi.org/10.1016/j.compbiomed.2022.106073.
[3] Y. Zhao, J. Zhang, D. Hu, H. Qu, Y. Tian, and X. Cui, “Application of Deep Learning in Histopathology Images of Breast
Cancer: A Review,” Micromachines, vol. 13, no. 12, pp. 1–30, Dec. 2022, https://doi.org/10.3390/mi13122197.
[4] K. Rautela, D. Kumar, and V. Kumar, “A Systematic Review on Breast Cancer Detection Using Deep Learning
Techniques,” Archives of Computational Methods in Engineering, vol. 29, no. 7, pp. 4599–4629, Nov. 2022,
https://doi.org/10.1007/s11831-022-09744-5.
[5] R. Krithiga and P. Geetha, “Breast Cancer Detection, Segmentation and Classification on Histopathology Images Analysis:
A Systematic Review,” Archives of Computational Methods in Engineering, vol. 28, no. 4, pp. 2607–2619, Jun. 2021,
https://doi.org/10.1007/s11831-020-09470-w.
[6] Z. Rezaei, “A review on image-based approaches for breast cancer detection, segmentation, and classification,” Expert Systems
with Applications, vol. 182, pp. 1–12, Nov. 2021, https://doi.org/10.1016/j.eswa.2021.115204.
[7] R. Ranjbarzadeh, S. Dorosti, S. Jafarzadeh Ghoushchi, A. Caputo, E. B. Tirkolaee, S. S. Ali, Z. Arshadi, and M. Bendechache,
“Breast tumor localization and segmentation using machine learning techniques: Overview of datasets, findings, and methods,”
Computers in Biology and Medicine, vol. 152, pp. 1–20, Jan. 2023, https://doi.org/10.1016/j.compbiomed.2022.106443.
[8] J. De Matos, S. Ataky, A. De Souza Britto, L. Soares De Oliveira, and A. Lameiras Koerich, “Machine Learning
Methods for Histopathological Image Analysis: A Review,” Electronics, vol. 10, no. 5, pp. 1–42, Feb. 2021,
https://doi.org/10.3390/electronics10050562.
[9] V. Mohanakurup, S. M. Parambil Gangadharan, P. Goel, D. Verma, S. Alshehri, R. Kashyap, and B. Malakhil, “Breast
Cancer Detection on Histopathological Images Using a Composite Dilated Backbone Network,” Computational Intelligence
and Neuroscience, vol. 2022, pp. 1–10, Jul. 2022, https://doi.org/10.1155/2022/8517706.
[10] E. Michael, H. Ma, H. Li, F. Kulwa, and J. Li, “Breast Cancer Segmentation Methods: Current Status and Future Potentials,”
BioMed Research International, vol. 2021, pp. 1–29, Jul. 2021, https://doi.org/10.1155/2021/9962109.
[11] X. Lu and X. Zhu, “Automatic segmentation of breast cancer histological images based on dual-path feature extraction network,”
Mathematical Biosciences and Engineering, vol. 19, no. 11, pp. 11 137–11 153, 2022, https://doi.org/10.3934/mbe.2022519.
[12] A. Juhong, B. Li, C.-Y. Yao, C.-W. Yang, D. W. Agnew, Y. L. Lei, X. Huang, W. Piyawattanametha, and Z. Qiu,
“Super-resolution and segmentation deep learning for breast cancer histopathology image analysis,” Biomedical Optics
Express, vol. 14, no. 1, pp. 18–36, Jan. 2023, https://doi.org/10.1364/BOE.463839.
[13] T. Ilyas, Z. I. Mannan, A. Khan, S. Azam, H. Kim, and F. De Boer, “TSFD-Net: Tissue specific feature
distillation network for nuclei segmentation and classification,” Neural Networks, vol. 151, pp. 1–15, Jul. 2022,
https://doi.org/10.1016/j.neunet.2022.02.020.
[14] A. Pedersen, E. Smistad, T. V. Rise, V. G. Dale, H. S. Pettersen, T.-A. S. Nordmo, D. Bouget, I. Reinertsen, and M. Valla,
“H2G-Net: A multi-resolution refinement approach for segmentation of breast cancer region in gigapixel histopathological
images,” Frontiers in Medicine, vol. 9, pp. 1–13, Sep. 2022, https://doi.org/10.3389/fmed.2022.971873.
[15] M.-A. Khalil, Y.-C. Lee, H.-C. Lien, Y.-M. Jeng, and C.-W. Wang, “Fast Segmentation of Metastatic Foci in
H&E Whole-Slide Images for Breast Cancer Diagnosis,” Diagnostics, vol. 12, no. 4, pp. 1–16, Apr. 2022,
https://doi.org/10.3390/diagnostics12040990.
[16] X. Zhang, X. Zhu, K. Tang, Y. Zhao, Z. Lu, and Q. Feng, “DDTNet: A dense dual-task network for tumor-infiltrating
lymphocyte detection and segmentation in histopathological images of breast cancer,” Medical Image Analysis, vol. 78, pp.
1–17, May 2022, https://doi.org/10.1016/j.media.2022.102415.
[17] B. M. Priego-Torres, D. Sanchez-Morillo, M. A. Fernandez-Granero, and M. Garcia-Rojo, “Automatic segmentation of
whole-slide H&E stained breast histopathology images using a deep convolutional neural network architecture,” Expert
Systems with Applications, vol. 151, pp. 1–14, Aug. 2020, https://doi.org/10.1016/j.eswa.2020.113387.
[18] I. Ahmad, Y. Xia, H. Cui, and Z. U. Islam, “DAN-NucNet: A dual attention based framework for nuclei segmentation in
cancer histology images under wild clinical conditions,” Expert Systems with Applications, vol. 213, pp. 1–8, Mar. 2023,
https://doi.org/10.1016/j.eswa.2022.118945.
[19] W. R. Drioua, N. Benamrane, and L. Sais, “Breast Cancer Histopathological Images Segmentation Using Deep Learning,”
Sensors, vol. 23, no. 17, pp. 1–14, Aug. 2023, https://doi.org/10.3390/s23177318.
[20] S. Graham, Q. D. Vu, S. E. A. Raza, A. Azam, Y. W. Tsang, J. T. Kwak, and N. Rajpoot, “Hover-Net: Simultaneous
segmentation and classification of nuclei in multi-tissue histology images,” Medical Image Analysis, vol. 58, pp. 1–18, Dec.
2019, https://doi.org/10.1016/j.media.2019.101563.
[21] W. Zhang and J. Zhang, “AugHover-Net: Augmenting Hover-net for Nucleus Segmentation and Classification,” arXiv Preprint,
pp. 1–4, Apr. 2022, https://doi.org/abs/2203.03415.
[22] F. H¨orst, M. Rempe, L. Heine, C. Seibold, J. Keyl, G. Baldini, S. Ugurel, J. Siveke, B. Gr¨unwald, J. Egger, and J. Kleesiek,
“CellViT: Vision Transformers for precise cell segmentation and classification,” Medical Image Analysis, vol. 94, pp. 1–16,
May 2024, https://doi.org/10.1016/j.media.2024.103143.
[23] C. Tommasino, C. Russo, A. M. Rinaldi, and F. Ciompi, “”HoVer-UNet”: Accelerating Hovernet with Unet-Based Multi-Class
Nuclei Segmentation Via Knowledge Distillation,” in 2024 IEEE International Symposium on Biomedical Imaging (ISBI).
Athens, Greece: IEEE, May 2024, pp. 1–4, https://doi.org/10.1109/ISBI56570.2024.10635755.
[24] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Medical
Image Computing and Computer-Assisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F.
Frangi, Eds. Cham: Springer International Publishing, 2015, vol. 9351, pp. 234–241.
[25] N. Wahab, A. Khan, and Y. S. Lee, “Transfer learning based deep CNN for segmentation and detection of mitoses in breast
cancer histopathological images,” Microscopy, vol. 68, no. 3, pp. 216–233, Jun. 2019, https://doi.org/10.1093/jmicro/dfz002.
[26] J. Gamper, N. A. Koohbanani, K. Benes, S. Graham, M. Jahanifar, S. A. Khurram, A. Azam, K. Hewitt, and N. Rajpoot,
“PanNuke Dataset Extension, Insights and Baselines,” arXiv Preprint, pp. 1–12, Apr. 2020, https://doi.org/abs/2003.10778.
[27] B. Abhisheka, S. K. Biswas, and B. Purkayastha, “A Comprehensive Review on Breast Cancer Detection, Classification and
Segmentation Using Deep Learning,” Archives of Computational Methods in Engineering, vol. 30, no. 8, pp. 5023–5052, Nov.
2023, https://doi.org/10.1007/s11831-023-09968-z.
[28] T. Alam, W.-C. Shia, F.-R. Hsu, and T. Hassan, “Improving Breast Cancer Detection and Diagnosis through Semantic
Segmentation Using the Unet3+ Deep Learning Framework,” Biomedicines, vol. 11, no. 6, pp. 1–12, May 2023,
https://doi.org/10.3390/biomedicines11061536.
Published
2024-11-15
How to Cite
Sudianjaya, N., & Fatichah, C. (2024). Segmentation and Classification of Breast Cancer Histopathological Image Utilizing U-Net and Transfer Learning ResNet50. MATRIK : Jurnal Manajemen, Teknik Informatika Dan Rekayasa Komputer, 24(1), 155-166. https://doi.org/https://doi.org/10.30812/matrik.v24i1.4186
Issue
Section
Articles
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
.png)
