The great challenge of the agricultural sector is to meet the increasing demand without further affecting the environment or the climate. Precision agriculture can play an important role in this regard, and recent new technologies have tremendously increased our capacity to map fields spatially. However, the interpretation and automated analysis of these data remains a major bottleneck. Deep learning, a rapidly developing artificial intelligence technology, has a huge potential to tackle this, but so far, most deep learning applications rely on supervised learning, which is less suited in an agricultural context. The goal of this proposal is to develop an operational and user-friendly application for the detection of anomalies – diseases, infections, or growth reductions due to abiotic or biotic stress factors – in UAV imagery of potato cultivation. We explore an alternative deep learning method that uses weakly and unsupervised learning methods. An unsupervised learning method to reconstruct images forms the base of the method, and is coupled to a weakly supervised anomaly detection framework, that benefits from the farmer’s tacit knowledge, yet without requiring excessive labelling input. The method will be fine-tuned with a range of datasets containing the most important stress conditions and diseases of potato cultivation. The end result is a system that detects and visualises these anomalies for the end user, functioning as future decision support tool.