Finding efficient means of quantitatively describing material microstructure is a critical step towards harnessing data-centric machine learning approaches to understanding and predicting processing-microstructure-property relationships. Common quantitative descriptors of microstructure tend to consider only specific, narrow features such as grain size or phase fractions, but these metrics discard vast amounts of information. Since the gain in traction of machine learning and computer vision, more abstract methods for describing image data in a concise and quantitative manner have become available but have yet to be fully exploited within materials science. Here, we utilise variational autoencoders (VAEs) and vision transformers (ViTs) to construct an encoded space of microstructural image data. The encoded space is explored with dimensionality reduction and methods of traversal to uncover how various morphological features are distributed across the space. Morphological features are predicted directly from the encoded representations alongside classification of the microstructure. Several datasets are explored, with a focus on greyscale optical micrographs and extensions to EBSD data.