Inverse Inference of Subjective Cost and Internal Models from Clinical Sensorimotor Behavior

Question: How have inverse reinforcement learning and inverse optimal control methods been used to infer altered subjective cost functions and internal models from human sensorimotor behavior in psychiatric or neurodevelopmental conditions?

Answer

Taken together, the retrieved papers show that this literature exists but is thin and lopsided: exactly one study inverts a full sensorimotor control model in a psychiatric population, using Inverse Rational Control on closed-loop virtual-reality navigation to attribute autistic motor differences to a reduced subjective action cost alongside altered sensory weighting [8]. Everything else in the retrieved set reaches the same class of conclusion by adjacent routes — inverse estimation of precision and cost parameters from effort psychophysics, effort-based choice, and response-inhibition models [5] [14] [17] — while inverse optimal control in its mature form is applied to musculoskeletal and clinical-decision data rather than to psychiatric movement [20] [2]. Across these papers the collective position is that clinical differences are better characterized as shifts in separable latent parameters (effort cost, action cost, sensory precision, reward sensitivity) than as failures of optimality, and that recovering those parameters requires data-rich trajectories with joint inference of dynamics, horizon, and cost. The unresolved problem is identifiability: the same trajectory can be produced by altered cost, altered internal model, altered dynamics, or a shortened planning horizon, and no retrieved clinical study tests all four against each other.

Data-rich closed-loop trajectories, not summary performance, carry the inverse-inference signal

Summary measures of task success were insufficient to separate autistic from neurotypical participants in a virtual-reality reach-and-stop task, whereas the full movement trajectories did differentiate the groups, indicating distinct navigational strategies and/or cost functions [8]. The same lesson appears in social planning: aggregate performance showed only context-specific group differences, and Bayesian model-based estimates of latent strategy use carried limited individual-level group information, while model-free choice-history kernels preserved a robust temporal signature that outperformed model-based representations in classification [10]. Both results imply that the inferential payoff of inverse methods lies in the trajectory-level statistics that a generative model is fitted to, not in the endpoint accuracy that clinical batteries usually report.

Inverse rational control decomposes autism-related sensorimotor differences into sensory weighting and subjective action cost

Inverse Rational Control, which assumes agents are rational with respect to an internal model rather than optimal with respect to the true task, recovered both sensory and motor differences between autistic and neurotypical participants [8]. The sensory component was consistent with Bayesian inference under altered weighting of sensory evidence, while the motor component — more frequent and abrupt actions — was explained by a reduced subjective action cost [8]. This framing was explicitly motivated as a correction to computational psychiatry's prior practice of assuming that neurotypical and divergent populations optimize the same cost function [8]. Independent evidence is consistent with a precision-side account in the same population: autistic participants showed lower overall belief precision yet intact probabilistic rule generalization and, like neurotypical participants, rationally judged advantageous decisions as optimal [9].

Effort and inhibition tasks invert the same parameter family and converge on shifted costs rather than gross suboptimality

In major depressive disorder, hierarchical Bayesian modelling of effort ratings showed that experienced effort was compressed across effort levels, that this compression arose from reduced precision of effort signals with increased reliance on prior expectations, and that compression rather than elevated ratings tracked symptom burden and effort-reward choice [5]. In a large effort-based choice study, four computational mechanisms replicated across samples — acceptance bias, reward sensitivity, and linear and quadratic effort sensitivity — and the reduced willingness to exert effort in current and remitted depression was driven by lower effort acceptance bias rather than altered effort or reward sensitivity [14]. In Parkinson's disease, depression was associated with markedly lower reward sensitivity that did not respond to dopaminergic medication [13]. In multiple sclerosis, increased exertion variability was associated with less accurate effort assessment and inflated effort reports, a candidate mechanism for why effort feels costlier in that population [16].

Response-inhibition models recover cost and precision parameters dimensionally in large developmental cohorts

Formalizing the Stop Signal Task as a partially observable Markov decision process yielded two components — noisy perceptual inference and optimal control balanced against costs — and amortized simulation-based inference produced individual-level posteriors in 3,567 children from the ABCD cohort [17]. Higher ADHD scores were associated with reduced go-cue directional precision and blunted sensitivity to stop-error, go-error, and time costs, and the learned embedding space placed these children on a continuous manifold rather than in distinct disorder clusters, so similar clinical characteristics arose from diverse combinations of mechanisms [17]. Prediction-side parameters are also recoverable from purely behavioural signatures in autism: autistic adults benefited less from target predictability in reaction time and showed reduced anticipatory motor ramping potentials, pointing to reduced spontaneous predictive motor preparation [21].

Amortized and Bayesian inverse methods exist precisely to make individual-level clinical inference feasible

Amortized inference via transformer-encoded sequence embeddings made individual posterior estimation tractable at cohort scale, with validation confirming reliable and identifiable parameters [17]. Kernel density Bayesian inverse reinforcement learning was developed for the regime typical of clinical and biological data — few demonstrations per task but known reward functions for related training tasks — and provides posterior concentration guarantees [15]. A free-energy-based model combined with a machine-learning framework inversely estimated latent cognitive updating and optimism-pessimism bias from behaviour [7]. Generalized inverse optimal control extends this agenda to multi-criteria objectives, nested objectives across scales, active constraints, and switches in the optimality principle over the observation window, while flagging the well-posedness of the inference problem as the central difficulty [19].

Outside psychiatry, inverse optimal control is already mature on human movement — which sharpens the gap

Bi-level inverse optimal control identified individualized and group-level cost functions that reproduced postoperative squat kinematics after total knee replacement with mean apex differences of 1.53° and 5.70° respectively, with squat depth emerging from the optimization rather than being prescribed [20]. Inverse reinforcement learning has likewise been applied to observational clinical data, identifying clinician priorities and deviant actions in ICU trajectories [2]. What is missing is the transfer of these mature inverse pipelines to the movement of patients with psychiatric or neurodevelopmental conditions.

Conflicts and gaps

The retrieved papers disagree on which parameter is altered in depression. Reduced effort acceptance bias with intact effort and reward sensitivity [14] conflicts with effort-signal precision loss and prior overreliance as the operative mechanism [5], and with reward sensitivity as the locus in Parkinson's disease depression [13]. For autism, a reduced subjective action cost account [8] sits alongside precision accounts in which choices remain normatively optimal [9] and alongside accounts emphasizing a stability-over-flexibility trade-off in task control [12]; no retrieved paper pits these against one another within a single inverse fit. Identifiability is a further open problem: estimated rewards are biased when dynamics are not estimated jointly [3], and an effective horizon shorter than the ground-truth horizon often fits better, which argues for learning reward and horizon together rather than reward alone [1] — in clinical data, altered motor noise or a foreshortened horizon could therefore be misread as altered cost. Task-irrelevant sensorimotor demands demonstrably shift the measured values used for clinical inference, with stationary versus ambulatory responding changing stay rates in an order-dependent way without disrupting individual-difference structure, a bias that could move patients across diagnostic thresholds [4]. Two gaps are conspicuous: no retrieved study applies inverse reinforcement learning or inverse optimal control to movement trajectories in schizophrenia, OCD, or ADHD, and the broader field has been criticized for testing only lower-level processes rather than participant-defined goals, which is exactly what inverse methods are suited to recover [11]. Forward models fitted to clinical motor phenotypes — reactive robotic control predicting planning failures in Parkinson's disease, mild cognitive impairment, and stroke [6], and reinforcement learning with clamped temporal-difference error reproducing freezing of gait [18] — remain the dominant alternative to inverse inference.

What would change the answer

A decisive study would fit Inverse Rational Control to closed-loop virtual-reality or robotic reaching in schizophrenia, ADHD, and OCD while jointly estimating transition dynamics [3] and the effective planning horizon [1], with preregistered model comparison against altered-dynamics and reduced-horizon alternatives and parameter-recovery checks. Pairing trajectory-derived action-cost estimates with independent effort psychophysics [5] [16] would test whether the two routes recover the same latent quantity, and adding sensorimotor-condition manipulations of the kind that shift measured values without changing individual-difference structure [4] would bound the contamination. Scaling to cohort-level individual posteriors via amortized simulation-based inference [17] and relaxing single-objective assumptions using generalized inverse optimal control [19] would make the dimensional, heterogeneous structure of these conditions testable rather than assumed.

References

[1] Yiqing Xu, Finale Doshi-Velez, David Hsu. "On the Effective Horizon of Inverse Reinforcement Learning". arxiv. 2025-02-21T00:00:00.000Z. URL: https://arxiv.org/abs/2307.06541

[2] Inko Bovenzi, Adi Carmel, Michael Hu, Rebecca M. Hurwitz, Fiona McBride, Leo Benac, José Roberto Tello Ayala, Finale Doshi-Velez. "Pruning the Path to Optimal Care: Identifying Systematically Suboptimal Medical Decision-Making with Inverse Reinforcement Learning". arxiv. 2024-11-11T00:00:00.000Z. URL: https://arxiv.org/abs/2411.05237

[3] Michael Herman, Tobias Gindele, Jörg Wagner, Felix Schmitt, Wolfram Burgard. "Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics". arxiv. 2026-06-04T00:00:00.000Z. URL: https://arxiv.org/abs/1604.03912

[4] Elliot Huang, William Xu, Robert C. Wilson. "Sensorimotor features of a reversal learning task bias decision behavior without disrupting individual difference structure". arxiv. 2026-08-11T00:00:00.000Z. URL: https://arxiv.org/abs/2608.08206

[5] Harlev, D., Wolpe, N.. "Not just harder: Compression of perceived effort in major depressive disorder is associated with symptom burden and altered decision-making". medRxiv. 2026-10-14T00:00:00.000Z. URL: https://doi.org/10.64898/2026.10.13.26365398

[6] Michael Migacev, Vito Mengers, Antonia Köngeter, Oliver Brock. "No Plan, Yet Human: A Reactive Robotics Model Predicts Human Planning Failures on a Clinical Task". arxiv. 2026-08-28T00:00:00.000Z. URL: https://arxiv.org/abs/2605.16514

[7] Higashino, I., Ito, R., Okochi, Y., Inutsuka, K., Yokoyama, H., Kato, R., Yada, Y., Amemori, K.-i., Naoki, H.. "Dynamic optimism and pessimism states shape risk-taking behavior in monkeys". bioRxiv. 2026-07-07T00:00:00.000Z. URL: https://doi.org/10.64898/2026.05.01.722186

[8] Chen, Y., Noel, J.-P., Schrater, P., Angelaki, D., Pitkow, X.. "Rationalizing sensorimotor deficits in autism spectrum disorder". bioRxiv. 2026-10-19T00:00:00.000Z. URL: https://doi.org/10.64898/2026.10.12.758517

[9] Pultsina, K., Kozunova, G., Chernyshev, B., Prokofyev, A., Tretyakova, V., Novikov, A., Rytikova, A., Stroganova, T.. "Neural Adaptation to Expected Uncertainty in Neurotypical Adults and High-Functioning Adults with Autism Spectrum Disorder". bioRxiv. 2025-06-06T00:00:00.000Z. URL: https://doi.org/10.1101/2025.03.30.646160

[10] Tahmineh Godazgar, Rana Ghamari, Abolfazl Nadi, Atiye Sarabi-Jamab. "Depth Before Strategy: Temporal Choice-History Signatures Reveal Reduced Recursive Social Planning in Autism". psyarxiv. 2026-07-11T19:28:47.151Z. URL: https://osf.io/mu9cr_v2

[11] Marta Migó, Michael Treadway. "What's the goal? Reconsidering the importance of higher-order cognition in internalizing psychopathology". psyarxiv. 2025-12-15T22:16:16.579Z. URL: https://osf.io/rqtah_v1

[12] Leslie Held, Judith Goris, Senne Braem. "Balancing Cognitive Flexibility and Stability: The Role of Reward, Autism, and Transdiagnostic Traits". psyarxiv. 2025-09-25T14:21:07.271Z. URL: https://osf.io/r97dy_v2

[13] Costello, H., Yamamori, Y., Kieslich, K., Murphy, M., Bobyreva, K., Schrag, A., Howard, R., Roiser, J.. "Depression in Parkinson's disease is associated with dopamine unresponsive reduced reward sensitivity during effort-based decision making". bioRxiv. 2025-01-29T00:00:00.000Z. URL: https://doi.org/10.1101/2024.05.09.592897

[14] Valton, V., Mkrtchian, A., Moses-Payne, M., Gray, A., Kieslich, K., VanUrk, S., Samborska, V., Halahakoon, D. C., Manohar, S. G., Dayan, P., Husain, M., Roiser, J. P.. "A computational approach to understanding effort-based decision-making in depression". bioRxiv. 2025-08-26T00:00:00.000Z. URL: https://doi.org/10.1101/2024.06.17.599286

[15] Aishwarya Mandyam, Didong Li, Jiayu Yao, Diana Cai, Andrew Jones, Barbara E. Engelhardt. "Kernel Density Bayesian Inverse Reinforcement Learning". arxiv. 2025-07-04T00:00:00.000Z. URL: https://arxiv.org/abs/2303.06827

[16] Dryzer, M. H., Nourbakhsh, B., Keller, J., Chib, V. S.. "Increased Exertion Variability is Linked to Disruptions in Effort Assessment in Multiple Sclerosis". bioRxiv. 2025-07-21T00:00:00.000Z. URL: https://doi.org/10.1101/2025.07.16.665234

[17] Wang, W., Kaufmann, T., Dayan, P.. "Decomposing response inhibition: a POMDP model". bioRxiv. 2026-09-25T00:00:00.000Z. URL: https://doi.org/10.64898/2026.02.26.708416

[18] Bhattacharya, N., Nair, S. S., Chakravarthy, V. S., Muralidharan, V.. "Impaired value estimation over space explains turn-induced freezing of gait in Parkinson's disease: A cortico-basal ganglia-inspired reinforcement learning model". bioRxiv. 2026-10-08T00:00:00.000Z. URL: https://doi.org/10.64898/2026.10.01.755546

[19] Julio R. Banga, Sebastian Sager. "Generalized Inverse Optimal Control and its Application in Biology". arxiv. 2025-10-17T00:00:00.000Z. URL: https://doi.org/10.1016/j.arcontrol.2025.101029

[20] Song, H.. "Bi-level inverse optimal control for preoperative prediction of postoperative squat kinematics after total knee replacement". bioRxiv. 2026-06-15T00:00:00.000Z. URL: https://doi.org/10.64898/2026.06.11.731549

[21] Soroush Bagheri, Annie Cardinaux, Lindsay Bungert, Cindy Li, Amanda O'Brien, Pawan Sinha, Jonathan Cannon. "Reduced anticipatory motor preparation and altered EEG signatures of prediction in autistic adults". psyarxiv. 2025-05-12T19:57:11.138Z. URL: https://osf.io/297tq_v1

How this survey was made

This survey answers one question: “How have inverse reinforcement learning and inverse optimal control methods been used to infer altered subjective cost functions and internal models from human sensorimotor behavior in psychiatric or neurodevelopmental conditions?”

We keep our own collection of recent preprints and refresh it every day from bioRxiv, medRxiv, arXiv, PsyArXiv, SocArXiv and ChemRxiv. For each survey, we search that collection for recent work and keep the strongest matches: papers that use the same terms as the question, and papers on the same topic in different words.

The text was written from the titles and abstracts of those papers; the full texts were not read. Statements that draw on a particular paper are marked with a number in brackets. The reference list just above is the 21 papers whose work this text draws on, chosen from the 180 papers those searches returned; the rest of the shortlist is not printed. It is not a complete review of everything published on the question.

Treat this as a starting point and a reading list, not a peer-reviewed review. The searches behind this survey ran on October 21, 2026 and covered work from the previous two years.