Modern applications demand lightweight, high-performance magnesium alloys with tailored stress–strain responses, achievable through meso-scale inverse design of microstructure and texture: identifying optimal, physically realistic microstructures and textures to achieve specific target mechanical properties. While previous works demonstrated accurate prediction of properties from microstructure-texture and identified key influential features using interpretable AI, translating desired properties back into synthesizable microstructures remains complex. Direct optimization using computationally expensive physics-based models like Crystal Plasticity Finite Element Method (CPFEM) is often computationally prohibitive. This work introduces a closed-loop framework that integrates generative AI, surrogate modeling, and physics-based simulation for accelerated and practical inverse design. Leveraging a database of 115 extruded Mg alloy specimens with experimental microstructure, texture, and tensile data, we first employ automated image processing and data multiplication techniques (Latin Hypercube/Monte Carlo sampling on grain statistics and generalized spherical harmonics coefficients for orientation distributions) to enrich the design space. We then generate realistic 2D synthetic microstructural RVEs from this design space using Dream3D. Advanced generative models (Stable Diffusion, Autoregressive architectures) are fine-tuned on these RVEs to learn a compressed latent space representation capturing complex features and reconstruct it back from the latent space. A Gaussian Process surrogate model is trained to rapidly map this latent space to mechanical properties (yield stress, hardening exponent) predicted by a pre-calibrated CPFEM model (fitted to the stress-strain curves from the database). This surrogate guides the acquisition function for a Bayesian Optimization loop where target properties of yield stress, strain hardeing exp. and ultimate stress define the objective function. In each iteration, the optimizer proposes promising latent space points, the corresponding RVE is reconstructed, its properties are validated via the CPFEM model of the alloy type, and the surrogate is updated. This physics-informed, generative approach enables efficient exploration and optimization, yielding realistic microstructure and texture designs tailored for specific performance goals in unseen alloy conditions, significantly advancing the practical application of AI in materials discovery.