Schedule for: 19w5235 - Optimal Neuroethology of Movement and Motor Control

A buffet dinner is served daily between 5:30pm and 7:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

Gather for conversation and snacks (not hosted), or to work/read on your own amongst others. There are also quieter options, including BIRS Lounge in Corbett 5210 and Reading Room in Corbett 5310.

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

A brief introduction to BIRS with important logistical information, technology instruction, and opportunity for participants to ask questions.

Path integration is the neural computational process that enables animals to estimate their location in an environment by integrating sensory cues that provide information about derivatives of position (velocity, acceleration etc.). This computation requires an initial estimation of position using known landmarks, measurements of self-motion, and knowledge of a path integration gain, the multiplicative factor that relates integrated self-motion cues to movement in the physical world. We built an augmented reality apparatus (the dome) to test and quantify elements of the computation of path integration in rodents. In the dome, rats run on a circular track while visual cues are projected around them. In the first set of experiments, we recorded place cells from hippocampal CA1 of individual rats while visual landmarks were moved as a function of a rat’s translation velocity. When these landmarks were subsequently removed, the spatial frequency of place fields established that the path integration gain is a plastic variable that is constantly adapted using a set of reliable landmarks. (Jayakumar*, Madhav*, et al., Nature, 2019). Here, we examine the role of optic flow on path integration and how it interacts with other self-motion cues. The overwhelming influence of landmarks was replaced by a set of 80 uniformly spaced vertical stripes. These stripes provided an optic flow cue but no polarizing information. In contrast to landmark manipulation, the place cell ensemble did not stay locked to the absolute position of the moving optic flow cues. In N=2 rats, the integrated position revealed by the location of place fields drifted in both the cue and laboratory frames. We used system identification to fit a parsimonious, second-order model of the stochastic hippocampal dynamics (output) in response to optic flow manipulation (input). Using this model, we implemented a control scheme whereby the real-time estimate of the hippocampal gain was regulated to a desired value via feedback manipulation of the gain to the optic-flow pattern. Once stabilized via this cognitive clamp, we drove the hippocampal gain to a desired steady-state value. After removing the stripes, we found that optic flow had induced a recalibration of non-visual path integration gain, demonstrating that optic flow serves as a cue to regulate the non-vusal path integration gain. NOTE: Portions from 2019 Society for Neuroscience abstract with the same authors.

Natural environments can have dynamics spanning multiple timescales, and the evidence animals use to make decisions is often relevant to future decisions. To understand decision-making under these conditions we analyze how a model ideal observer accumulates evidence to freely make choices across a sequence of correlated trials. We use principles of probabilistic inference to show that an ideal observer incorporates information obtained on one trial as an initial bias on the next. This bias decreases the time, but not the accuracy of the next decision. Furthermore, in finite sequences of trials the rate of reward is maximized when the observer deliberates longer for early decisions, but responds more quickly towards the end of the sequence. Our model also explains experimentally observed patterns in decision times and choices, providing a mathematically principled foundation for evidence-accumulation models of sequential decisions.

Since the seminal findings from patient H.M., motor skill learning is often relegated to the “procedural” branch of memory taxonomies. However, it has become increasingly clear that cognitive strategies play a central role in many human learning tasks, even though, the computations underlying these strategies remain poorly understood. Here, I will discuss our approach to pin down the computational and neural underpinnings of these strategies. We find that human motor learning strategies appear to assume two broad forms, reflecting high-level cognitive representations for flexible action selection, as well as simpler associative mappings from specific states to actions. Several lines of additional evidence suggest that working memory plays a central role in human motor learning, and in particular, may be a key player in setting the stage for motor skill acquisition.

Lunch is scheduled for 12:00 - 13:00 for this workshop. However, the dining room is open 11:30 - 13:30.

An outdoor walk on trails neighboring Banff Centre for Arts. Those not joining may continue lunch or relax on their own.

Meet in foyer of TCPL to participate in the BIRS group photo. The photograph will be taken outdoors, so dress appropriately for the weather. Please don't be late, or you might not be in the official group photo!

"Reinforcement learning (RL) provide a potentially powerful framework for designing control strategies that enable robots and simulated digital creatures to learn to move with skill and grace. However, there are significant drawbacks from a design perspective: reward functions can be unintuitive, solutions are prone to local minima and hyperparameter choices, there is no direct support for iterative design, and the transfer of motions from simulation to the real world is uncertain. We present a number of insights and refinements in support of learning realistic, controllable movements. These include motion mimicry, multi-step iterative design, sample-based transfer learning, and hybrid learning that mixes supervised learning with policy gradients. We demonstrate simulated human and animal skills that can reproduce a large variety of highly dynamic motions. We further show successful sim2real transfer of dynamic locomotion to Cassie, a large bipedal robot produced by Agility Robotics. Lastly, we highlight recent work by others that builds on key aspects of these ideas, including learning skills from video and the control of full-body muscle-driven motions."

In both a classic children's song and actual human anatomy, there is a path connecting the leg bone to the arm bone. The same is not true for optimization approaches to human movement, in which the leg bone (locomotion) has no connections to the arm bone (reaching). The objective functions proposed to govern these respective movements are thus far incompatible with each other, even though biologically, they share considerable neural machinery. Optimization is thought to be an important tool for predicting and understanding neural control of movement. But that approach should yield some commonality between arms and legs, or failing that, at least reveal fundamental reasons for a gap. I will discuss basic optimization approaches to date and their shortfalls, and propose a path towards greater commonality. The buzzwords to be repeated are "task performance and effort" and "common currency." Application of these buzzwords to both types of movements may eventually yield a connection between leg bone and arm bone (more universal principles) not easily gained from either movement alone.

Predictive simulations hold the potential to greatly expedite advances in understanding healthy and pathological movement. We developed a computationally efficient framework to predict human movement based on optimization of a performance criterion. The framework generates three-dimensional muscle-driven simulations, without relying on experimental data, in about 36 minutes on a standard laptop—more than 20 times faster than existing simulations—by using direct collocation, implicit differential equations, and algorithmic differentiation. The simulations produce physiologically realistic gaits with varied gait speed, and changes in gait caused by muscle strength deficits or prosthesis use. We extended this framework to generate simulations that are robust to uncertainty (e.g., sensorimotor noise) to further increase the realism of our simulations. We expect these predictions to enable optimal design of treatments aiming to restore gait function.

A buffet dinner is served daily between 5:30pm and 7:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

Gather for conversation and snacks (not hosted), or to work/read on your own amongst others. There are also quieter options, including BIRS Lounge in Corbett 5210 and Reading Room in Corbett 5310.

Continual perceptual processing, such as required for decisions that necessitate the accumulation of evidence over time, is essential for efficient behavior. The behavior emerging from a wide range of such decisions turns out to be well-described by a surprisingly simple model family - the diffusion decision model. Taking a decision-theoretical perspective, I will discuss why diffusion models might have this property across perceptual and value-based decisions. I furthermore will highlight scenarios in which such simple models will become insufficient.

The language of linear control has one word: K. As in u = Kx. As in dx/dt = Ax + Bu. A state based nonlinear control has, instead of Kx, F(x). But that is too broad. Is there a natural language of nonlinear control in which simple non-linear controls are simple to express and more complicated, but useful, control laws are efficiently expressed. Perhaps, also, this controller operates on sensors rather than states. In my guestimation there must be some better languages than, say, a) C++, or b) Taylor series, or c) Neural nets. A better language would be less general than those, but would A) not so easily express useless controllers, and B) more efficiently express pretty good controllers. I don't really have any ideas about what this language could be. But I feel like it must be out there. Why?Because animals learn to control things. And animals don't have access to the big data needed, for, say deep neural nets. So, animals must be working with a simpler, learnable or evolvable, language of control.What are candidates for that? This seems like a good crowd to ask this to.

Standing balance relies on the integration of multiple sensory inputs to generate the motor commands required to stand. Among these, the vestibular system in the inner ear detects head motion in space, providing the brain with vital information for maintaining upright balance. Imposed head motion while standing, however, do not evoke an isolated vestibular signal of self-motion. Transmastoid electrical stimuli activate the vestibular system in isolation but their head-motion equivalent remains a subject of debate. I will present neural data providing the physiological basis of percutaneous electrical activation of the vestibular system as well as the initial development and testing of a mechanical-equivalent model of electrical vestibular stimuli in humans. I will describe how a model of vestibular processing can explain human behaviors evoked by electrical vestibular stimuli. Finally, I will present experiments using electrical vestibular stimuli to reveal the adaptability and sensory recalibration underpinning the control of standing balance in humans.

Although lunch is scheduled at 11:50-13:00 for this workshop, the dining room is open 11:30 - 13:30.

An outdoor walk on trails neighboring Banff Centre for Arts. Those not joining may continue lunch or relax on their own.

Sea stars use hundreds of specialized “tube feet" lining their ventral surfaces to crawl on various terrains from rocky to smooth surfaces. A tube foot consists of soft, water-filled, muscular membranes that extend and contract through changing the hydraulic pressure, forming a perfect example of a soft actuator. Individual tube feet are equipped with integrated sensing and actuation, and the activity of arrays of tube feet is orchestrated by a nerve net that is distributed throughout the body; there is no central brain. Here, we analyze the biomechanics of individual tube feet, and construct low-order mathematical models of these soft actuators consisting of passive and active force elements. We then formulate hierarchical motor control laws, where the direction of motion of all tube feet is centrally controlled, while individual tube feet have local autonomy over their power and recovery cycle, via peripheral sensori-motor feedback loops. We use these models to rigorously examine the crawling and bouncing gaits in sea stars. We find that the mechanical interaction of multiple tube feet lead to stable and robust locomotion with minimal cost to the nervous system in terms of sensory integration. To conclude, we comment on the utility of this system as a new paradigm for robotic movement using distributed arrays of soft actuators.

Animals are motivated to act so as to maximize their subjective reward rate, which depends on factors such as the subjective value and probability of a favorable outcome, the metabolic cost of action, as well as temporal variables such as deliberation and handling time. I’ll show mathematically that in a wide range of conditions, maximizing reward rates can be accomplished by having a criterion of decision accuracy that decreases over time as one deliberates. I’ll then present data from neural recordings in monkeys, suggesting that this is implemented in the brain through an “urgency signal” from the basal ganglia, which causes neural activity in sensorimotor decision circuits to build-up over time, pushing the system to decide while also influencing the vigor of the chosen action. Finally, I’ll review evidence suggesting how individual settings of urgency may explain traits such as impulsivity as well as motivation-related symptoms of clinical disorders such as depression and Parkinson’s disease.

Spring-like tissues attached to the swinging legs of animals are thought to improve running economy by simply reducing the effort of leg swing. Here we show that a spring, or ‘exotendon,’ connecting the legs of a human runner improves economy instead through a more complex mechanism that produces savings during both swing and stance. The spring increases the energy optimal stride frequency; when runners adopt this new gait pattern, savings occur in both phases of gait. Remarkably, the simple device improves running economy by 6.4 ± 2.8%, comparable to savings achieved by motorized assistive robotics that directly target the costlier stance phase of gait. Our results highlight the importance of considering both the dynamics of the body and the adaptive strategies of the user when designing systems that couple human and machine. Authors: Cole S. Simpson, Cara G. Welker, Scott D. Uhlrich, Sean M. Sketch, Rachel W. Jackson, Scott L. Delp, Steve H. Collins, Jessica C. Selinger, and Elliot W. Hawkes

Most research in human locomotion is limited to steady-state and constant-speed conditions. However, moving about in everyday life requires us to constantly adapt our locomotion strategies in response to intrinsic noise-like transients, uncertainties and external environmental irregularities. In this talk, I will discuss our work on understanding and predicting such adaptive locomotion behaviors in three separate studies: (1) The energetics of walking while changing speeds predicts overground walking behavior over short distances, (2) control strategies for running stably inferred from running variability stabilize running in simulation and (3) metabolic energy optimality predicts the behavior people converge to at steady-state on a split-belt treadmill. Part 2 (if there is time): Computer vision-based pose estimation for movement science Traditional sensor-based experimental measures used in movement science limit the scope and scale of the science. Recent developments in computer vision promise to transform movement science by enabling tracking of in-the-wild movement behaviors. In this talk, I will discuss how we are using such computer vision-based tools in three movement science applications: (1) providing automatic exercise feedback for individuals performing deadlifts, (2) automated infant neuromotor risk assessment using a big data approach and (3) tracking infant emotion and understanding its role in predicting neuromotor risk.

Gather for conversation and snacks (not hosted), or to work/read on your own amongst others. There are also quieter options, including BIRS Lounge in Corbett 5210 and Reading Room in Corbett 5310.

Insects are the first evolved to fly, and to fly is not to fall. How does an insect fly, why does it fly so well, and how can we infer its ‘thoughts’ from its flight dynamics? We have been seeking mechanistic explanations of the complex movement of insect flight. Starting from the Navier-Stokes equations governing the unsteady aerodynamics of flapping flight, we worked to build a theoretical framework for computing flight. This has led to new interpretations and predictions of the functions of an insect’s internal machinery that orchestrate its flight. I will discuss our recent computational and experimental studies of the balancing act of dragonflies and fruit flies: how a dragonfly recovers from falling upside-down and how a fly balances in air. In each case, the physics of flight informs us about the neural feedback circuitries underlying their fast reflexes.

A simple, abstract model for rhythmic locomotion is a stable periodic orbit of a vector field. Stability often requires feedback and control. This talk will discuss how an organism might estimate parameters in a model controller. Noise in the system is inevitable. The resulting fluctuations give information useful for estimating the parameters. However, the time required may be implausibly slow in systems with Gaussian noise, prompting the search for alternative models. What are experiments that might give further insight into these issues?

In this two-part talk, I will first describe our human locomotion experiments and optimization calculations, demonstrating that energy optimality can predict many aspects of humans locomotion behavior. Energy optimality not only (roughly) explains steady locomotion behavior in a straight line, but also unsteady locomotion with changing speeds, non-straight-line locomotion in complex curves, and other less conventional tasks. Next, I will describe our attempts to characterize the controller humans use to walk and run stably. To infer the controller, we use human responses to natural intrinsic noise and externally applied perturbations during walking and running. We will show that some human responses to such perturbations can also be explained (roughly) by energy-optimal perturbation recovery.

Gather for conversation and snacks (not hosted), or to work/read on your own amongst others. There are also quieter options, including BIRS Lounge in Corbett 5210 and Reading Room in Corbett 5310.

Incentives act to improve the speed and accuracy of our behaviour. Velocity measurements in human saccades demonstrate that reward increases movement vigour. Remarkably, this is achieved without a cost to movement accuracy. I will present a recent study that suggests that control precision may itself carry a cost. The cost of control can be framed at a cognitive level, and mathematically described by the magnitude of control signals. This formulation captures the accuracy requires effort.

Rat rhythmically brush and tap their whiskers against objects to tactually explore the environment. I will describe our laboratory's work on understanding the "fastest" sensorimotor loop that connects sensors to actuators: from the whisker, to mechanoreceptors, to primary sensory neurons, to secondary sensory neurons in the brainstem, and then out to motor neurons and the whisker muscles. I will specifically focus on how the animal might use mechanical signals at the whisker base to infer the 3D location at which a whisker makes contact with an object.

In recent years, advances in reinforcement learning have made it increasingly possible to learn to solve complex tasks in a variety of domains. Our recent work mostly focuses on motor control for simulated bodies, with an aim towards building systems capable of generating flexible, adaptive behavior in rich environments. A motivating premise of this work is that it does not always make sense to learn how to perform a task from scratch, end-to-end when certain basic motor skills should, in principle, be reusable. The core research questions we seek to address have to do with how human-like motor skills can be learned and represented, as well as what architectures support transfer and reusability in new settings. Reuse may require learning how to sequence and coordinate available skills as well as integrating additional sensory information to determine what motor behaviors are relevant in a given setting. This talk will survey our progress on these topics.

Recent work shows that motor skill acquisition is driven not only by movement error signals but is also influenced by reinforcement feedback—binary information signalling successes and failures associated with particular movements. Here we describe two experiments designed to test how characteristics of the reinforcement landscape influence how motor learning unfolds. In one experiment we found that a steeper reinforcement gradient leads to faster learning. In a second experiment participants were more likely to direct their behaviour in the direction of the steeper portion of a complex reinforcement landscape. We propose a model of sensorimotor reinforcement learning that does not depend on a full representation of the entire reinforcement landscape (storing the expected reward for all possible actions). Rather, many empirical characteristics of learning can be predicted using only temporally recent and spatially local information about actions and rewards.

Perception of motion involves the integration of information from visual, vestibular, and proprioceptive sensory organs. Early models representing how the brain combines sensory cues use state-space dynamical models to reproduce perceptual phenomena such as velocity storage, wherein participants perceive a constant angular velocity as a decaying exponential angular velocity. While sensory signal states are the focus of these early models, our recent observations suggest that the variance of both sensory signals and the internally represented motion state are as important in representing how the brain optimally integrates sensory information to generate perceptions of motion.

An outdoor walk on trails neighboring Banff Centre for Arts. Those not joining may continue lunch or relax on their own.

Complex ecological behaviors are often driven by an internal model, which integrates sensory information over time and facilitates long-term planning. Inferring the internal model is a crucial ingredient for interpreting neural activities of agents and is beneficial for imitation learning. We introduce methods to infer an agent's internal model and dynamic beliefs for a dynamic foraging and game-like navigation tasks. We model agents as rational according to their (possibly defective) understanding of the task and the relevant causal variables that cannot be fully observed. Using a novel gradient-based constrained EM algorithm, we show that it's possible to invert Partially Observable Markov Decision Process (POMDP) from behavior with unknown transition dynamics, partially unknown observation functions and parametrically unknown rewards. We allow that the agent may have wrong assumptions about the task, and our method learns these assumptions from the agent's actions. We validate our method on simulated agents performing suboptimally on a foraging task, and successfully recover the agent's actual model. We show how to extend this approach to a larger range of ecological tasks. The result is a powerful method for eliciting trajectories of latent belief states from behavior that can serve as a powerful tool for interpreting neural activity.

Perhaps the most general principle underlying human movement is that people prefer to move in ways that minimize their energetic cost. Although aspects of this preference are likely established over evolutionary and developmental timescales, we recently discovered that the nervous system can continuously optimize cost in real-time. Here I will present our new research focused on uncovering the mechanisms underlying the initiation of this optimization, as well as its process. Our collective findings indicate that energetic cost is not just an outcome of movement, but also plays a central role in continuously shaping it.

Most research in human locomotion is limited to steady-state and constant-speed conditions. However, moving about in everyday life requires us to constantly adapt our locomotion strategies in response to intrinsic noise-like transients, uncertainties and external environmental irregularities. In this talk, I will discuss our work on understanding and predicting such adaptive locomotion behaviors in three separate studies: (1) The energetics of walking while changing speeds predicts overground walking behavior over short distances, (2) control strategies for running stably inferred from running variability stabilize running in simulation and (3) metabolic energy optimality predicts the behavior people converge to at steady-state on a split-belt treadmill. Part 2 (if there is time): Computer vision-based pose estimation for movement science Traditional sensor-based experimental measures used in movement science limit the scope and scale of the science. Recent developments in computer vision promise to transform movement science by enabling tracking of in-the-wild movement behaviors. In this talk, I will discuss how we are using such computer vision-based tools in three movement science applications: (1) providing automatic exercise feedback for individuals performing deadlifts, (2) automated infant neuromotor risk assessment using a big data approach and (3) tracking infant emotion and understanding its role in predicting neuromotor risk.

Gather for conversation and snacks (not hosted), or to work/read on your own amongst others. There are also quieter options, including BIRS Lounge in Corbett 5210 and Reading Room in Corbett 5310.

Each participant should submit 1 slide to be presented in 3 minutes, followed by 3 minutes discussion.

Each participant should submit 1 slide to be presented in 3 minutes, followed by 3 minutes discussion.