T&C Chen Center for Social and Decision Neuroscience Seminar

Abstract: Cognitive science has long sought to develop theories to explain natural behavior in naturalistic settings. However, theory-driven computational cognitive science faces two key challenges: (1) naturalistic settings can require giving up experimental control of dependent variables specified by a computational theory, and (2) naturalistic settings can preclude development of task-performing models that operate within them. In this talk, I'll first describe a framework for pursuing "naturalistic" computational cognitive science that aims to ameliorate these challenges to develop both human-level and human-like AI models. To facilitate this research, I will introduce NiceWebRL, a new research tool that enables using identical code for developing deep reinforcement learning models, collecting human data, and for increasing the naturalism of our experiments.

To demonstrate this research strategy, I will present "Multitask Preplay", a research project that develops a novel reinforcement learning theory for human behavior and tests it across increasingly naturalistic experimental conditions. We observe that human tasks of interest are commonly co-located—for example, stove and fridge tasks commonly appear in kitchens, and coffee shops and restaurants are commonly co-located in city centers. We hypothesize that humans exploit this structure with Multitask Preplay to leverage experience with one task to preemptively learn solutions to other tasks that were accessible but not achieved. I will first present behavioral predictions that scale from small artificial grid-worlds to a 2D minecraft environment. Afterwards, I will show that Multitask Preplay enables AI agents to learn behaviors that better transfer to novel open-world environments that share task co-occurrence structure.