Workshop @ SfN¶
Simulating embodied sensorimotor control with NeuroMechFly
Satellite event at the Neuroscience 2024 meeting, Society for Neuroscience (SfN)Important
When: Monday October 7 (SfN Day 3), 6:30 pm – 8:30 pm Where: Hyatt Regency McCormick Place, 2233 S. Martin Luther King Drive, Chicago Organized by: Pavan Ramdya, Sibo Wang-Chen, Victor Stimpfling, Thomas Lam Practical information: ‣ To attend, please register using the button below. ‣ Please bring your own laptop for the coding sessions.
Background
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Until now, such models have primarily been used to investigate motor control. Far less studied with realistic body models is how the brain and motor systems work together to perform hierarchical sensorimotor control.
To bridge this gap, we developed NeuroMechFly (Lobato-Rios et al., Nature Methods, 2022; Wang-Chen et al., Nature Methods, 2024). With NeuroMechFly, one can test models of the following embodied in an anatomically realistic body model:
- Motor control and mechanosensation
- Example: Building walking controllers that integrate Central Pattern Generators with sensory feedback.
- Visual sensing and processing
- Example: Interfacing NeuroMechFly with a connectome-constrained neural network vision model (Lappalainen et al., Nature, 2024) to obtain simulated neural responses in closed loop.
- Olfactory sensing and procession
- Example: Interfacing NeuroMechFly with fluid dynamics simulation and implementing a Drosophila plume tracking algorithm (Demir et al., eLife, 2020).
- Social behavior
- Example: Following another fly as during courtship behavior.
- Reinforcement learning for integrative tasks
- Example: Training a controller to search for an attractive odor source and using vision to avoid an obstacle while walking over rugged terrain.
In this workshop, we will demonstrate the capacities of NeuroMechFly, followed by practical coding sessions to help you get started with the FlyGym Python library.
Agenda
Time |
Activity |
Speaker |
|---|---|---|
6:30–6:50 |
Opening talk: Why NeuroMechFly? |
|
6:50–7:10 |
Practical: Getting started with FlyGym |
|
7:10–7:20 |
Break and ad hoc assistance with installation, etc. |
|
7:20–7:50 |
Practical: Kinematic replay and walking control |
|
7:50–8:20 |
Practical: Building a controller to follow another fly |
|
8:20–8:30 |
Q&A |