The production of 3D parts from metal by powder bed fusion (PBF) is a new but in some application areas established technique. However, the development of the PBF process is still ongoing to fully understand the complex interaction of the different parameters or to avoid process instabilities with resulting occurrence of defects or quality problems. Consequently, process control and active parameter regulation are more and more in focus of actual developments. A necessary requirement towards achieving the aim of regulated process is reliable defect detection as early as possible in a build job. In this publication, a new approach will be discussed: An equipment independent image acquisition system in combination with deep learning image processing, computational cheap Convolutional-Neural-Network (CNN), is used. An image is acquired of each layer after the building platform is recoated. The surface of the entire powder bed is evaluated by the CNN and it reports the defect classification result within seconds. The CNN is trained with a dataset of images taken from various real build jobs. Evaluating the entire powder bed provides the ability to detect and categorize defects with large extents. Typical results as classification of specific failures and defects, deduced by this setup and the CNN, show the potential of this new approach. Perspectively it is aimed at real-time process monitoring and control. Depending on the component defect that occurs, correction strategies are to be derived that interrupt the build process at the required point and intervene in a targeted corrective manner.