Edge computing and distributed AI offer promising solutions to counteract the centralization of AI. By leveraging edge devices such as smartphones, IoT sensors, and local servers, AI algorithms can be distributed across a network of devices rather than relying solely on centralized data centers. This reduces latency and improves the efficiency and privacy of applications such as autonomous vehicles, remote sensing, and smart cities. While Edge AI is a popular research topic, most current research focuses on optimizing the inference stage of the model, assuming that the model is completely trained offline in a cloud data center. We propose to move away from the separate train and deploy stages and instead also equip the edge devices with the capability to update the model. It is unlikely that the model will be trained completely from scratch on the edge device but at least, the model should be able to adapt itself to the location specific dynamics, handling domain shifts over time. This approach not only enhances the adaptability and efficiency of AI systems but also addresses privacy concerns by reducing the need for constant data transmission to centralized servers.

To realize this vision of efficient, on-device representation learning, we will need to change how we think about training a model and move part of this workload to the edge device. To enable this, we will focus on:

• Unsupervised learning paradigms: It is unlikely that labelled information will be available on the edge device. We will therefore need to develop novel algorithms that are able to extract useful feature representations from rich input data streams without labels.
• Novel compute platforms: Several companies offer custom hardware accelerators that promise energy-efficient inference of deep neural networks. More novel approaches rely on technologies such as analog or approximate computing. Regardless of the underlying technology, they all introduce constraints on the precision of the weights or the size of the model. We will develop novel model architectures that take these limitations into account from the start. 
• Online domain adaptation: Most likely, we do not need to train a model from scratch on the edge device. It is more realistic to train a model offline and then finetune it further on the edge device in order to specialize it for the environment where it is being deployed. Many techniques have been proposed to achieve this but recently, a more biologically inspired approach is gaining popularity which would be worthwhile to explore further.