Language models (LMs) lack complex human reasoning capabilities, particularly social reasoning, for truly understanding intended meanings. The ability to critically analyze the identities and relations of social actors within and beyond a text for drawing nuanced, informed conclusions are vital for creating empathetic, socially sensitive systems in an increasingly AI-driven world. This project advances LMs by addressing their limitations in social reasoning through three key innovations. First, we design methods to predict social graphs from text that map real-world relationships among authors, readers, and people mentioned in a text. Second, we devise advanced reasoning strategies inspired by human cognition that steer the social graph building process and force models to actively use social graphs for their main task predictions. Third, we develop context-adaptive natural language processing models that adapt their predictions to subtle shifts in the social graphs, improving the fairness and accuracy of their judgments. We validate the transferability of our approaches in multiple natural language understanding tasks, with hate speech detection as the main use case. This projects represents a major step toward socially intelligent LMs that can better understand and navigate the intricacies of human interactions with empathy and contextual awareness.