Probabilistic graphical models, and especially causal Bayesian networks, allow combining data and causal knowledge in a principled way. Based on this framework, we can reason probabilistically about complex systems, even when faced with a significant degree of uncertainty. Despite these benefits, the practical adoption of these models remains limited, due to inability to process data with challenging, but common data properties such as (1) a mixture of discrete and continuous variables, (2) temporal data and (3) unstructured data such as text. The downsides of Bayesian networks are exactly where neural networks excel. They are ideally suited to model such data and have seen great success over recent years due to their strong pattern recognition abilities. Nonetheless, neural networks cannot exploit causal knowledge, lack interpretability and are hardly suited for probabilistic reasoning. In this research proposal, I plan to combine the best of both worlds in a single neuro-probabilistic framework. The overall goal is to build neural networks that inherit the probabilistic capabilities of causal Bayesian networks and can handle challenging real-world data properties.