Synergizing Anatomy and Function: A Goal-driven Model of Frontoparietal Dexterous Object Manipulation

Goal-driven deep learning produced significant advances in perception modelling. Models, however, often implement a single sensory domain and thus isolate a specific function. In this work, we go a step beyond and close the perception-action loop with a model of the frontoparietal network. The model implements biologically plausible macro-level structure by connecting cell count-fitted sensorimotor regions by pathways extracted from structural connectivity data. The model interfaces an anthropomorphic robotic hand and is trained to manipulate objects. We show that the biologically-inspired architecture significantly outperforms an architecture used in state-of-the-art robotics while converging substantially faster and relying only on raw sensory data. Moreover, preliminary in silico decoding analyses show promise in aligning with in vivo expectations.