Large computational databases with millions of entries and high-resolution experiments such as transmission electron microscopy contain large (and growing) amount of valuable information. To leverage this under-utilized - yet very valuable - data towards discovering hidden patterns and eventually novel physics, automatic analytical methods need to be developed. Here, we discuss how Bayesian deep learning can be employed to classify a diverse set of single- and polycrystalline structures, in robust and threshold-independent fashion. Besides supervised machine-learning techniques, we discuss the application of unsupervised machine learning (clustering, manifold learning). This allows to unravel and explain the internal neural-network representations. In particular, we access the structural similarity information embedded in the learned representations and use it for the characterization of crystal systems, from both synthetic and experimental resources.