Bayesian optimization (BO) is already widely used to solve complex optimization problems, particularly when the problem space to be explored is expensive to query. In essence, BO methods have to major ingredients: the notion of a metric that can be used to establish a distance-dependent correlation between points in the design space, as well as a utility function that can be used to select an optimal sequence of experiments that arrives at a solution with the minimum (expected) effort/expense. BO is now widely used in materials discovery/design problems as long as they can be mapped to a black box optimization problem. While effective, most applications of BO to materials problems employ off-the shelf methods. Yet, a principle of materials science is the existence of process-structure-property (PSP) relationships. While predictive PSP relationships are espoused as essential to materials design, they are seldom used as part of the decision-making process. In this talk, I will present recent work by my group and collaborators where we have explicitly explored the impact of injecting PSP relationships into BO approaches to materials design. Using computational experiments, we explore the performance of microstructure-agnostic vs microstructure-aware Bayesian materials design frameworks. Advantages and limitations of the present approach as well as its applicability in 'real' materials discovery tasks are discussed.