Our ability to engage in goal-directed behavior, inhibit prepotent responses, and switch back and forth between different tasks, is often referred to as cognitive control or "executive functions". As part of their definition, cognitive control functions are often distinguished from arguably lower-level forms of learning, such as reinforcement learning, associative learning, or habit formation. This traditional view, which considers cognitive control functions as independent, isolated, domain-general processes, has also inspired large research endeavors and a multi-billion dollar industry into cognitive training. However, several meta-analyses have now shown that while these studies and programs show clear benefits on the specific cognitive task or game that people were trained, they rarely show transfer (i.e., benefits outside these tasks). Here, I start from a different perspective which considers control processes deeply embedded in the same associative network that regulates other forms of learning. I propose to use different paradigms and computational models coming from two different research traditions, to find a converging answer on how cognitive control is learned in a multi-learning environment (i.e., where also "simpler" contingencies can be learned), and show how a focus on the learning of cues for control, rather than control processes as "cognitive muscles", can be a promising framework to think about the transfer of learned control strategies.