Relational responding, the ability to respond to one event in terms of its relationship to another, is a cornerstone of human higher cognition. Humans show an apparently unparalleled capacity for arbitrarily applicable relational responding (AARR), responding to relations that are not defined by formal stimulus properties, but controlled by contextual features (e.g., we can respond as if a small coin, a dime, is more valuable than a larger coin, a nickel). This capacity has been implicated in language, reasoning, perspective-taking and psychopathology. Despite recent spectacular advances in computational models of human learning, efforts to model the arbitrary, flexible and generative nature of AARR have been limited. A computational model of AARR would produce valuable insights for computational and learning psychologists alike. The current project aims to contribute to the development of a computational model of AARR. In WP1, we will investigate how simple reinforcement learning (RL) algorithms can be extended to allow for rudimentary demonstrations of AARR. In WP2, we will broaden our approach, exploring a family of models with various relational mechanisms and background knowledge, and exposing them to more complex environments to study the dynamics of AARR. Finally, in WP3, we will investigate whether deep RL approaches can be used to model the development of AARR through learning contextually controlled structured representations.