Directed Energy Deposition (DED)-Arc is suitable for the additive manufacturing, modification and repair of large metal components with high deposition rates. Residual stresses and distortion are of central importance when characterising the manufactured components. Residual stresses caused by the thermal cycles during the manufacturing process can impair the mechanical properties of the manufactured parts and lead to cracking. Unacceptable distortion during manufacturing makes precise material deposition difficult and compromises the geometric and structural integrity of the final component. Therefore, understanding and controlling residual stresses and distortion, especially when combining different base and filler materials, is critical to improving the quality and efficiency of the DED-Arc process. This article deals with the influence of heat conduction and build-up height on the residual stress distribution and distortion of additively manufactured components with selected base and filler material combinations. Using a robot-assisted welding system and a controlled short arc, systematic step cancellation tests were carried out at selected interpass temperatures (100, 200 and 300 °C), line energies (200, 400 and 600 kJ/m) and increasing build-up height. The height of the component geometry showed a significant influence on the level of residual stress in the welding tests carried out. Lower heights were generally associated with increased residual stress. The stiffness of the substrate plate prevented the structure from shrinking during the cooling phase in the bonding area. As the component height increased, the underlying component layers were tempered, which resulted in a reduction in residual stresses. Furthermore, tensile stresses were measured in the area of the top layer and compressive stresses in the area of the underlying layers, which can be explained by the shrinkage of the non-heat-treated top layer. A boundary layer number was determined for all combinations of base and filler material tested. Once this number was exceeded, the distribution of residual stresses no longer changed, but merely shifted with the increasing height of the component in the direction of build-up. The correlations presented make a significant contribution to the understanding of residual stress development and distortion with increasing structure height. Considering the observations presented and optimising the process parameters, components with high geometric accuracy and low residual stresses can be produced more economically and ecologically in future using DED-Arc.