Training diffusion models from scratch requires large datasets and high computational costs, limiting accessibility and sustainability. This research aims to improve training efficiency by integrating Bayesian active learning and sampling techniques to select the most informative samples during training, reducing redundancy and optimizing convergence. We also address limited expressiveness and bias in diffusion models trained on small datasets, which can lead to mode collapse and reduced diversity. By incorporating uncertainty-aware diversity preservation, we intend broader representation in generated images. Our approach develops an adaptive training pipeline that balances efficiency, diversity, and fairness. We apply this framework to anomaly detection for data-scarce or privacy-critical domains such as engineering (e.g., manufacturing), and healthcare (e.g., medical image synthesis), improving data efficiency while mitigating bias. This research contributes with advances of sustainable AI by reducing computational costs and promoting resource-conscious generative modeling.