With the rise of the internet of things there exists a range of electronic sensors that are small enough to be integrated in embedded devices, such as cars and robots. These sensors also require modern deep learning algorithms to perform detection, tracking, etc. The goal of my proposal is to develop new neural network design methodologies for deep learning that are optimized for embedded devices. As there is a wide range of use-cases for small devices with onboard AI capabilities, deep learning algorithms need to be able to scale to the available computing resources. I will therefore design algorithms that adapt to the available power budget, latency, etc. The main novelty of my work is to focus on scaling the computational complexity of the algorithm by adapting the neural network execution to the input. Some inputs are easier to process than others, as humans we also perceive this, sometimes we recognize things immediately and other times we need to focus longer. In deep learning the trend of the recent years has been that bigger networks achieve better accuracy. However for many inputs, a smaller and faster network is sufficient. By making neural networks scalable on a per-input basis, the required computations to analyse an input sample can scale with the perceived difficulty of that sample. This will allow faster execution for easy to analyse inputs while maintaining the full power of a slower but more accurate network for inputs that are harder to interpret.