Comparative evaluation of Detectron2 Mask R-CNN and YOLOV8-Seg for microstructural image analysis in high-temperature superalloys

I.Iwediba*, V.A. Vorontsov

Department of Design Manufacturing and Engineering Management, University of Strathclyde, Glasgow, United Kingdom, G1 1XJ

*isaac.iwediba@strath.ac.uk

A dataset of microstructural images of a Ni-based superalloy labeled for the gamma-prime intermetallic phase was split into “train” and “test” sets. Both datasets were employed to evaluate two deep learning models for segmentation quality. The same preprocessing, image augmentations and data splits were used for both models, to compare them under fair conditions.

The first model used was the Detectron2 Mask R-CNN. It is a two-staged framework which first generates potential regions and subsequently refines them as object masks[1, 2]. It achieved a 41.89% in terms of mean average precision (mAP@[0.50:0.95]) on the “test” data. There was a particularly high performance for images with large gamma-prime precipitates, with an average precision of 99.0%. This demonstrates the model’s capability for picking up large, high-contrast features. It had a lower performance when used on images with small features, with an average precision of 32.3%. These sources of weakness are prevalent in approaches using region proposals. The second method was YOLOv8-Seg. It is one-stage model that directly predicts and segments objects in a single shot. It was more memory efficient, faster to train and could be run in real time [3]. This makes it suitable for high-throughput pipelines, or use under the constraints of a slow hardware configuration. Analysis of the two datasets further proved that YOLOv8-Seg was well suited to segmentation of images with more varied precipitate distributions. The comparative analysis highlights an inherent trade-off. On the large gamma-prime regions, Detectron2 presented more accuracy. YOLOv8-Seg achieved faster training and inference, higher flexibility for precipitate-dense microstructures. These observations are also consistent with what is known from computer vision systems in general: two-stage models produce more precise, but less computationally efficient outputs, whereas one-stage models focus on speed instead [4, 5]. This work will assist in the determination of segmentation tools towards materials informatics in the context ofe metallurgical manufacture. It shows that precise segmentation is critical to the automation of phase, particle/precipitate size, and morphology quantification in high-temperature superalloys. Through analyzing the strengths and limitations of each approach, this work demonstrates case-specific rationale for choosing one segmentation model over another based on either accuracy or efficiency demand. This can be used to expedite metallurgical research and design of advanced manufacturing processes.

Keywords: instance segmentation; Detectron2; YOLOv8-Seg; microstructure; materials informatics

Acknowledgements

This research was supported by the EPSRC REA studentship, with additional resources provided through the doctoral training grant (DTG) to support research expenses, travel, and stipend top-up.

References

[1] K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN," in 2017 IEEE International Conference on Computer Vision (ICCV), 22-29 Oct. 2017 2017, pp. 2980-2988, doi: 10.1109/ICCV.2017.322.

[2] D. B, M. Afthab, and S. C, Advancing Object Detection: A Comprehensive Study Utilizing Detectron2 Framework. 2024.

[3] V. Afifah and S. Erniwati, "YOLOv8 for Object Detection: A Comprehensive Review of Advances,Techniques, and Applications," IJACI : International Journal of Advanced Computing and Informatics, vol. 2, pp. 53-61, 07/28 2025, doi: 10.71129/ijaci.v2i1.pp53-61.

[4] J. Muñoz Rodenas, F. Garcia-Sevilla, V. Miguel, J. Coello, and A. Martínez-Martínez, "A Deep Learning Approach to Semantic Segmentation of Steel Microstructures," Applied Sciences, vol. 14, p. 2297, 03/08 2024, doi: 10.3390/app14062297.

[5] L. Zhao, J. Locke, F. Xu, T. Yao, and X. Guo, Accurate Segmentation of Localized Corrosion in Structural Alloys via Deep Learning. 2025.