A computationally rational model of human reinforcment learning

Author(s): Fang, Zeming; Sims, Chris R | Abstract: Human learning efficiency in reinforcement learning tasks decreases when the number of the presented stimuli increases, a finding known as the "set size effect". From the computational rationality perspective, this effect can be interpreted as the brain’s balancing task performance against rising cognitive costs. Still, it remains unclear how best to quantify cognitive cost in learning tasks. One candidate is policy complexity, defined in terms of information theory as the mutual information between the sensory input and behavioral response. However, using a published data set (Collins a Frank, 2012), we show that policy complexity alone cannot explain the set size effect because the optimal policy complexity does not necessarily increase with the set size. We therefore propose a computational model and conduct a model-based analysis to show the minimal constituents of cognitive cost are policy complexity and representation complexity---the information quantity conveyed from sensory inputs to internal representations.