In this project, we will teach a robotic arm to manipulate objects of various challenging materials. To this end, we will develop differentiable physical models of the robots, the objects and the interaction between them. These differentiable models can be improved by learning form data. They integrate seamlessly with deep learning techniques and can be jointly optimized through gradient descent. This approach offers an interesting possibility of combining built-in prior knowledge and data-driven learning. Learning will help overcome model inaccuracies while the built-in prior knowledge can dramatically improve data-efficiency over fully black-box techniques, which have seen limited success in real-world applications for this reason.