In the field of transmission electron microscopy (TEM), accurate  analysis of acquired data often lags behind the advancements in  acquisition methods. Manual tailoring of image processing techniques to  individual datasets hinders progress in data interpretation. To address  this challenge, we propose a study focused on improving the performance  of deep learning models for semantic segmentation in high-resolution TEM  images by leveraging pretraining on unlabeled images.

Drawing insights from previous works, our study encompasses two key  stages: denoising and semantic segmentation. For denoising, we employ  two distinct approaches. Firstly, an encoder-decoder model is trained to  reconstruct clean TEM images from noisy inputs generated by applying  Poisson noise, as described in [1]. Secondly, we utilize the  conditional Generative Adversarial Network (GAN) model, Pix2Pix [2], to  train a generator capable of producing clean TEM images from noisy  inputs, while a discriminator learns to distinguish between the  generated and original clean images.

The pretrained denoising models, both the encoder-decoder and the  generator, are then fine-tuned on annotated TEM images for semantic  segmentation. Our study reveals several notable findings. Firstly, finetuning semantic segmentation models using pretrained denoising  models accelerates convergence, reducing the time and computational  resources required for training. Secondly, this approach proves to be  effective with smaller datasets, highlighting the potential for  leveraging limited annotated data in TEM image analysis. Additionally,  our approach facilitates comparable results across TEM datasets with  varying magnifications and resolutions, obviating the need for  customizing architectures to accommodate different receptive field  sizes which is extensively studied in [3].

By combining denoising and semantic segmentation through pretraining on  unlabeled data, our study offers a practical solution for enhancing the  analysis of high-resolution TEM images. These findings have significant  implications for the TEM community, as they enable faster and more  accurate interpretation of TEM data, while also reducing the resource  requirements for model training.



References
[1] L., Gambini; T., Mullarkey, L., Jones; S., Sanvito. Machine-learning approach for quantified resolvability enhancement of low-dose STEM data. Machine Learning: Science and Technology, 2023, 4(1), p.015025.
[2] P., Isola; J.Y., Zhu; T., Zhu; A.A., Efros. Image-to-image translation with conditional adversarial networks. CVPR, 2017, pp. 1125-1134
[3] K., Sytwu; C., Groschner; M.C., Scott. Understanding the influence of receptive field and network complexity in neural network-guided TEM image analysis JMicroscopy and Microanalysis, 2022, 28(6), pp. 1896-1904.