Cardiovascular disease (CVD) is a major cause of mortality in women, and women are more likely to develop heart failure with preserved ejection fraction (HFpEF) than men. While vascular aging is suspected to play a role in this prevalence, the precise mechanism behind it still remains unclear. Current diagnostic criteria and treatment protocols for heart disease are predominantly biased towards male patients, and sex-specific differences in cardiovascular physiology and pathophysiology are poorly understood. To address this, we propose a novel multimodal, multi-scale computational framework that integrates medical images, hemodynamics, patient history, and vascular function to develop sex-specific cardiovascular models. Our framework integrates static and dynamic statistical shape analysis, hemodynamic reduced order modeling, multiscale biomechanical modeling, and machine learning techniques to investigate sex differences in cardiovascular anatomy and function in both healthy and HFpEF patients. Our research aims to elucidate the mechanistic implications of cardiovascular differences between men and women, with a focus on the effect of vascular aging on the long-term remodeling of HFpEF. Ultimately, we seek to close the knowledge gap on cardiovascular sex-differences, and uncover sex-specific HFpEF pathways that can inform the development of treatment and diagnostic protocols that ensure women receive timely and effective cardiac care.