Trans-dimensional Hamiltonian model selection and parameter estimation from sparse, noisy data
1Committee on Computational and Applied Mathematics, University of Chicago, Chicago, IL 60637, USA
2Materials Science Division, Argonne National Laboratory, Lemont, IL 60439, USA
3QuTech, Delft University of Technology, PO Box 5046, 2600 GA Delft, The Netherlands
4Kavli Institute of Nanoscience, Delft University of Technology, PO Box 5046, 2600 GA Delft, The Netherlands
5Department of Chemistry, University of Chicago, Chicago, IL 60637, USA
6Pritzker School of Molecular Engineering, University of Chicago, Chicago, IL 60637, USA
| Published: | 2026-04-08, volume 10, page 2055 |
| Editor: | Marco Cerezo |
| Eprint: | arXiv:2506.18802v2 |
| Doi: | https://doi.org/10.22331/q-2026-04-08-2055 |
| Citation: | Quantum 10, 2055 (2026). |
Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.
Abstract
High-throughput characterization often requires estimating parameters and model dimension from experimental data of limited quantity and quality. Such data may result in an ill-posed inverse problem, where multiple sets of parameters and model dimensions are consistent with available data. This ill-posed regime may render traditional machine learning and deterministic methods unreliable or intractable, particularly in high-dimensional, nonlinear, and mixed continuous and discrete parameter spaces. To address these challenges, we present a Bayesian framework that hybridizes several Markov chain Monte Carlo (MCMC) sampling techniques to estimate both parameters and model dimension from sparse, noisy data. By integrating sampling for mixed continuous and discrete parameter spaces, reversible-jump MCMC to estimate model dimension, and parallel tempering to accelerate exploration of complex posteriors, our approach enables principled parameter estimation and model selection in data-limited regimes. We apply our framework to a specific ill-posed problem in quantum information science: recovering the locations and hyperfine couplings of nuclear spins surrounding a spin-defect in a semiconductor from sparse, noisy coherence data. We show that a hybridized MCMC method can recover meaningful posterior distributions over physical parameters using an order of magnitude less data than existing approaches, and we validate our results on experimental measurements. More generally, our work provides a flexible, extensible strategy for solving a broad class of ill-posed inverse problems under realistic experimental constraints.

Featured image: Schematic of inputs and outputs to a hybrid MCMC algorithm
Popular summary
This work introduces a computational approach to tackle that ambiguity head-on. Rather than trying to find one best solution, the method uses Bayesian statistics to map out all plausible explanations consistent with the data—and how likely each one is. The key innovation is combining several advanced sampling techniques into a single framework that can handle both continuous parameters (like interaction strengths) and discrete parameters (like how many particles are present).
We demonstrate the utility of this approach on a problem from quantum information science: identifying the positions and interactions of nuclear spins surrounding a quantum defect in a solid. These nuclear spins' interactions subtly influence the defect’s behavior, but the available measurements are sparse and noisy, making the reconstruction especially difficult. The new method is able to recover meaningful information about the spin environment using far less data than previous techniques, and its predictions agree with experimental results.
Beyond this specific application, the framework offers a powerful and flexible way to solve a wide range of inverse problems in physics—particularly in situations where data are scarce, models are complex, and uncertainty cannot be ignored.
► BibTeX data
► References
[1] Stefania Castelletto and Alberto Boretti. Silicon carbide color centers for quantum applications. Journal of Physics: Photonics, 2 (2): 022001, 2020. 10.1088/2515-7647/ab77a2.
https://doi.org/10.1088/2515-7647/ab77a2
[2] Lila VH Rodgers, Lillian B Hughes, Mouzhe Xie, Peter C Maurer, Shimon Kolkowitz, Ania C Bleszynski Jayich, and Nathalie P de Leon. Materials challenges for quantum technologies based on color centers in diamond. MRS Bulletin, 46 (7): 623–633, 2021. 10.1557/s43577-021-00137-w.
https://doi.org/10.1557/s43577-021-00137-w
[3] Christopher P Anderson, Elena O Glen, Cyrus Zeledon, Alexandre Bourassa, Yu Jin, Yizhi Zhu, Christian Vorwerk, Alexander L Crook, Hiroshi Abe, Jawad Ul-Hassan, Takeshi Ohshima, T. Son Nguyen, Giulia Galli, and David D Awschalom. Five-second coherence of a single spin with single-shot readout in silicon carbide. Science Advances, 8 (5): eabm5912, 2022. 10.1126/sciadv.abm5912.
https://doi.org/10.1126/sciadv.abm5912
[4] Shun Kanai, F Joseph Heremans, Hosung Seo, Gary Wolfowicz, Christopher P Anderson, Sean E Sullivan, Mykyta Onizhuk, Giulia Galli, David D Awschalom, and Hideo Ohno. Generalized scaling of spin qubit coherence in over 12,000 host materials. Proceedings of the National Academy of Sciences, 119 (15): e2121808119, 2022. 10.1073/pnas.2121808119.
https://doi.org/10.1073/pnas.2121808119
[5] Alexandre Bourassa, Christopher P Anderson, Kevin C Miao, Mykyta Onizhuk, He Ma, Alexander L Crook, Hiroshi Abe, Jawad Ul-Hassan, Takeshi Ohshima, Nguyen T Son, Giulia Galli, and David D Awschalom. Entanglement and control of single nuclear spins in isotopically engineered silicon carbide. Nature Materials, 19 (12): 1319–1325, 2020. 10.1038/s41563-020-00802-6.
https://doi.org/10.1038/s41563-020-00802-6
[6] CE Bradley, SW de Bone, PFW Möller, S Baier, MJ Degen, SJH Loenen, HP Bartling, M Markham, DJ Twitchen, R Hanson, D Elkouss, and TH Taminiau. Robust quantum-network memory based on spin qubits in isotopically engineered diamond. npj Quantum Information, 8 (1): 122, 2022. 10.1038/s41534-022-00637-w.
https://doi.org/10.1038/s41534-022-00637-w
[7] AM Waeber, G Gillard, G Ragunathan, M Hopkinson, P Spencer, DA Ritchie, MS Skolnick, and EA Chekhovich. Pulse control protocols for preserving coherence in dipolar-coupled nuclear spin baths. Nature Communications, 10 (1): 3157, 2019. 10.1038/s41467-019-11160-6.
https://doi.org/10.1038/s41467-019-11160-6
[8] Wenzheng Dong, FA Calderon-Vargas, and Sophia E Economou. Precise high-fidelity electron–nuclear spin entangling gates in nv centers via hybrid dynamical decoupling sequences. New Journal of Physics, 22 (7): 073059, 2020. 10.1088/1367-2630/ab9bc0.
https://doi.org/10.1088/1367-2630/ab9bc0
[9] Julia Cramer, Norbert Kalb, M Adriaan Rol, Bas Hensen, Machiel S Blok, Matthew Markham, Daniel J Twitchen, Ronald Hanson, and Tim H Taminiau. Repeated quantum error correction on a continuously encoded qubit by real-time feedback. Nature Communications, 7 (1): 11526, 2016. 10.1038/ncomms11526.
https://doi.org/10.1038/ncomms11526
[10] TH Taminiau, JJT Wagenaar, T Van der Sar, Fedor Jelezko, Viatcheslav V Dobrovitski, and R Hanson. Detection and control of individual nuclear spins using a weakly coupled electron spin. Physical Review Letters, 109 (13): 137602, 2012. 10.1103/PhysRevLett.109.137602.
https://doi.org/10.1103/PhysRevLett.109.137602
[11] Tim H Taminiau, Julia Cramer, Toeno van der Sar, Viatcheslav V Dobrovitski, and Ronald Hanson. Universal control and error correction in multi-qubit spin registers in diamond. Nature Nanotechnology, 9 (3): 171–176, 2014. 10.1038/nnano.2014.2.
https://doi.org/10.1038/nnano.2014.2
[12] Jonathan C Marcks, Mykyta Onizhuk, Nazar Delegan, Yu-Xin Wang, Masaya Fukami, Maya Watts, Aashish A Clerk, F Joseph Heremans, Giulia Galli, and David D Awschalom. Guiding diamond spin qubit growth with computational methods. Physical Review Materials, 8 (2): 026204, 2024. 10.1103/PhysRevMaterials.8.026204.
https://doi.org/10.1103/PhysRevMaterials.8.026204
[13] Connor P Horn, Christina Wicker, Antoni Wellisz, Cyrus Zeledon, Pavani Vamsi Krishna Nittala, F Joseph Heremans, David D Awschalom, and Supratik Guha. Controlled spalling of 4h silicon carbide with investigated spin coherence for quantum engineering integration. ACS Nano, 18 (45): 31381–31389, 2024. 10.1021/acsnano.4c10978.
https://doi.org/10.1021/acsnano.4c10978
[14] Raffi Budakian, Amit Finkler, Alexander Eichler, Martino Poggio, Christian L Degen, Sahand Tabatabaei, Inhee Lee, P Chris Hammel, S Polzik Eugene, Tim H Taminiau, et al. Roadmap on nanoscale magnetic resonance imaging. Nanotechnology, 35 (41): 412001, 2024. 10.1088/1361-6528/ad4b23.
https://doi.org/10.1088/1361-6528/ad4b23
[15] Abdelghani Laraoui, Florian Dolde, Christian Burk, Friedemann Reinhard, Jörg Wrachtrup, and Carlos A Meriles. High-resolution correlation spectroscopy of 13c spins near a nitrogen-vacancy centre in diamond. Nature Communications, 4 (1): 1651, 2013. 10.1038/ncomms2685.
https://doi.org/10.1038/ncomms2685
[16] MH Abobeih, J Randall, CE Bradley, HP Bartling, MA Bakker, MJ Degen, M Markham, DJ Twitchen, and TH Taminiau. Atomic-scale imaging of a 27-nuclear-spin cluster using a quantum sensor. Nature, 576 (7787): 411–415, 2019. 10.1038/s41586-019-1834-7.
https://doi.org/10.1038/s41586-019-1834-7
[17] Kyunghoon Jung, MH Abobeih, Jiwon Yun, Gyeonghun Kim, Hyunseok Oh, Ang Henry, TH Taminiau, and Dohun Kim. Deep learning enhanced individual nuclear-spin detection. npj Quantum Information, 7 (1): 41, 2021. 10.1038/s41534-021-00377-3.
https://doi.org/10.1038/s41534-021-00377-3
[18] B Varona-Uriarte, C Munuera-Javaloy, E Terradillos, Y Ban, A Alvarez-Gila, E Garrote, and J Casanova. Automatic detection of nuclear spins at arbitrary magnetic fields via signal-to-image ai model. Physical Review Letters, 132 (15): 150801, 2024. 10.1103/PhysRevLett.132.150801.
https://doi.org/10.1103/PhysRevLett.132.150801
[19] Naoto Kura and Masahito Ueda. Finite-error metrological bounds on multiparameter hamiltonian estimation. Physical Review A, 97 (1): 012101, 2018. 10.1103/PhysRevA.97.012101.
https://doi.org/10.1103/PhysRevA.97.012101
[20] Wenjun Yu, Jinzhao Sun, Zeyao Han, and Xiao Yuan. Robust and efficient hamiltonian learning. Quantum, 7: 1045, 2023. 10.22331/q-2023-06-29-1045.
https://doi.org/10.22331/q-2023-06-29-1045
[21] Hsin-Yuan Huang, Yu Tong, Di Fang, and Yuan Su. Learning many-body hamiltonians with heisenberg-limited scaling. Physical Review Letters, 130 (20): 200403, 2023. 10.1103/PhysRevLett.130.200403.
https://doi.org/10.1103/PhysRevLett.130.200403
[22] Anurag Anshu, Srinivasan Arunachalam, Tomotaka Kuwahara, and Mehdi Soleimanifar. Sample-efficient learning of interacting quantum systems. Nature Physics, 17 (8): 931–935, 2021. 10.1038/s41567-021-01232-0.
https://doi.org/10.1038/s41567-021-01232-0
[23] Peter J Green. Reversible jump markov chain monte carlo computation and bayesian model determination. Biometrika, 82 (4): 711–732, 1995. 10.1093/biomet/82.4.711.
https://doi.org/10.1093/biomet/82.4.711
[24] WD Vousden, Will M Farr, and Ilya Mandel. Dynamic temperature selection for parallel tempering in markov chain monte carlo simulations. Monthly Notices of the Royal Astronomical Society, 455 (2): 1919–1937, 2016. 10.1093/mnras/stv2422.
https://doi.org/10.1093/mnras/stv2422
[25] David GT Denison, Christopher C Holmes, Bani K Mallick, and Adrian FM Smith. Bayesian methods for nonlinear classification and regression, volume 386. John Wiley & Sons, 2002.
[26] Daniel Sanz-Alonso and Omar Al-Ghattas. A first course in monte carlo methods. arXiv preprint arXiv:2405.16359, 2024. 10.48550/arXiv.2405.16359.
https://doi.org/10.48550/arXiv.2405.16359
arXiv:2405.16359
[27] Peter J Green and David I Hastie. Reversible jump mcmc. Technical report, University of Bristol, 2009.
[28] Alexandre Toubiana, Michael L Katz, and Jonathan R Gair. Is there an excess of black holes around 20 M⊙? optimizing the complexity of population models with the use of reversible jump mcmc. Monthly Notices of the Royal Astronomical Society, 524 (4): 5844–5853, 2023. 10.1093/mnras/stad2215.
https://doi.org/10.1093/mnras/stad2215
[29] Michael Zevin, Chris Pankow, Carl L Rodriguez, Laura Sampson, Eve Chase, Vassiliki Kalogera, and Frederic A Rasio. Constraining formation models of binary black holes with gravitational-wave observations. The Astrophysical Journal, 846 (1): 82, 2017. 10.3847/1538-4357/aa8408.
https://doi.org/10.3847/1538-4357/aa8408
[30] Dehan Zhu and Richard Gibson. Seismic inversion and uncertainty quantification using transdimensional markov chain monte carlo method. Geophysics, 83 (4): R321–R334, 2018. 10.1190/geo2016-0594.1.
https://doi.org/10.1190/geo2016-0594.1
[31] Yongchae Cho, Richard L Gibson Jr, and Dehan Zhu. Quasi 3d transdimensional markov-chain monte carlo for seismic impedance inversion and uncertainty analysis. Interpretation, 6 (3): T613–T624, 2018. 10.1190/INT-2017-0136.1.
https://doi.org/10.1190/INT-2017-0136.1
[32] Jamie R Oaks, Perry L Wood Jr, Cameron D Siler, and Rafe M Brown. Generalizing bayesian phylogenetics to infer shared evolutionary events. Proceedings of the National Academy of Sciences, 119 (29): e2121036119, 2022. 10.1073/pnas.2121036119.
https://doi.org/10.1073/pnas.2121036119
[33] Mark Pagel and Andrew Meade. Modelling heterotachy in phylogenetic inference by reversible-jump markov chain monte carlo. Philosophical Transactions of the Royal Society B: Biological Sciences, 363 (1512): 3955–3964, 2008. 10.1098/rstb.2008.0178.
https://doi.org/10.1098/rstb.2008.0178
[34] István Takács and Viktor Ivády. Accurate hyperfine tensors for solid state quantum applications: case of the nv center in diamond. Communications Physics, 7 (1): 178, 2024. 10.1038/s42005-024-01668-9.
https://doi.org/10.1038/s42005-024-01668-9
[35] GL Van de Stolpe, DP Kwiatkowski, CE Bradley, J Randall, MH Abobeih, SA Breitweiser, LC Bassett, M Markham, DJ Twitchen, and TH Taminiau. Mapping a 50-spin-qubit network through correlated sensing. Nature Communications, 15 (1): 2006, 2024. 10.1038/s41467-024-46075-4.
https://doi.org/10.1038/s41467-024-46075-4
[36] Bjorn Engquist, Brittany D Froese, and Yunan Yang. Optimal transport for seismic full waveform inversion. arXiv preprint arXiv:1602.01540, 2016. 10.48550/arXiv.1602.01540.
https://doi.org/10.48550/arXiv.1602.01540
arXiv:1602.01540
[37] Hyunseok Oh, Jiwon Yun, MH Abobeih, Kyung-Hoon Jung, Kiho Kim, TH Taminiau, and Dohun Kim. Algorithmic decomposition for efficient multiple nuclear spin detection in diamond. Scientific Reports, 10 (1): 14884, 2020. 10.1038/s41598-020-71339-6.
https://doi.org/10.1038/s41598-020-71339-6
[38] David M Walker, F Javier Pérez-Barbería, and Glenn Marion. Stochastic modelling of ecological processes using hybrid gibbs samplers. Ecological Modelling, 198 (1-2): 40–52, 2006. 10.1016/j.ecolmodel.2006.04.008.
https://doi.org/10.1016/j.ecolmodel.2006.04.008
[39] Boby Mathew, AM Bauer, Petri Koistinen, TC Reetz, Jens Léon, and MJ Sillanpää. Bayesian adaptive markov chain monte carlo estimation of genetic parameters. Heredity, 109 (4): 235–245, 2012. 10.1038/hdy.2012.35.
https://doi.org/10.1038/hdy.2012.35
[40] Jian Zhang, Jingye Li, Xiaohong Chen, and Yuanqiang Li. Geological structure-guided hybrid mcmc and bayesian linearized inversion methodology. Journal of Petroleum Science and Engineering, 199: 108296, 2021. 10.1016/j.petrol.2020.108296.
https://doi.org/10.1016/j.petrol.2020.108296
[41] Sebastian Reuschen, Fabian Jobst, and Wolfgang Nowak. Efficient discretization-independent bayesian inversion of high-dimensional multi-gaussian priors using a hybrid mcmc. Water Resources Research, 57 (8): e2021WR030051, 2021. 10.1029/2021WR030051.
https://doi.org/10.1029/2021WR030051
[42] Binh Duong Nguyen, Pavlo Potapenko, Aytekin Demirci, Kishan Govind, Sébastien Bompas, and Stefan Sandfeld. Efficient surrogate models for materials science simulations: Machine learning-based prediction of microstructure properties. Machine Learning with Applications, 16: 100544, 2024. 10.1016/j.mlwa.2024.100544.
https://doi.org/10.1016/j.mlwa.2024.100544
[43] Agrim Babbar, Sriram Ragunathan, Debirupa Mitra, Arnab Dutta, and Tarak K Patra. Explainability and extrapolation of machine learning models for predicting the glass transition temperature of polymers. Journal of Polymer Science, 62 (6): 1175–1186, 2024. 10.1002/pol.20230714.
https://doi.org/10.1002/pol.20230714
[44] Chandramouli Nyshadham, Matthias Rupp, Brayden Bekker, Alexander V Shapeev, Tim Mueller, Conrad W Rosenbrock, Gábor Csányi, David W Wingate, and Gus LW Hart. Machine-learned multi-system surrogate models for materials prediction. npj Computational Materials, 5 (1): 51, 2019. 10.1038/s41524-019-0189-9.
https://doi.org/10.1038/s41524-019-0189-9
[45] Marc Verriere, Nicolas Schunck, Irene Kim, Petar Marević, Kevin Quinlan, Michelle N Ngo, David Regnier, and Raphael David Lasseri. Building surrogate models of nuclear density functional theory with gaussian processes and autoencoders. Frontiers in Physics, 10: 1028370, 2022. 10.3389/fphy.2022.1028370.
https://doi.org/10.3389/fphy.2022.1028370
[46] Thantip Roongcharoen, Giorgio Conter, Luca Sementa, Giacomo Melani, and Alessandro Fortunelli. Machine-learning-accelerated dft conformal sampling of catalytic processes. Journal of Chemical Theory and Computation, 20 (21): 9580–9591, 2024. 10.1021/acs.jctc.4c00643.
https://doi.org/10.1021/acs.jctc.4c00643
[47] Anand Chandrasekaran, Deepak Kamal, Rohit Batra, Chiho Kim, Lihua Chen, and Rampi Ramprasad. Solving the electronic structure problem with machine learning. npj Computational Materials, 5 (1): 22, 2019. 10.1038/s41524-019-0162-7.
https://doi.org/10.1038/s41524-019-0162-7
[48] Merlise A Clyde, Joyee Ghosh, and Michael L Littman. Bayesian adaptive sampling for variable selection and model averaging. Journal of Computational and Graphical Statistics, 20 (1): 80–101, 2011. 10.1198/jcgs.2010.09049.
https://doi.org/10.1198/jcgs.2010.09049
[49] Giovanni Seni and John Elder. Ensemble methods in data mining: improving accuracy through combining predictions. Morgan & Claypool Publishers, 2010. 10.2200/S00240ED1V01Y200912DMK002.
https://doi.org/10.2200/S00240ED1V01Y200912DMK002
[50] Peter Bühlmann. Bagging, boosting and ensemble methods. In Handbook of computational statistics: Concepts and methods, pages 985–1022. Springer, 2011. 10.1007/978-3-642-21551-3_33.
https://doi.org/10.1007/978-3-642-21551-3_33
[51] Melissa Adrian, Jake A Soloff, and Rebecca Willett. Stabilizing black-box model selection with the inflated argmax. arXiv preprint arXiv:2410.18268, 2024. 10.48550/arXiv.2410.18268.
https://doi.org/10.48550/arXiv.2410.18268
arXiv:2410.18268
[52] Yongchao Li, Yanyan Wang, and Liang Yan. Surrogate modeling for bayesian inverse problems based on physics-informed neural networks. Journal of Computational Physics, 475: 111841, 2023. 10.1016/j.jcp.2022.111841.
https://doi.org/10.1016/j.jcp.2022.111841
[53] Chad Lieberman, Karen Willcox, and Omar Ghattas. Parameter and state model reduction for large-scale statistical inverse problems. SIAM Journal on Scientific Computing, 32 (5): 2523–2542, 2010. 10.1137/090775622.
https://doi.org/10.1137/090775622
[54] Matthew D Hoffman and Andrew Gelman. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. Journal of Machine Learning Research, 15 (1): 1593–1623, 2014. 10.48550/arXiv.1111.4246.
https://doi.org/10.48550/arXiv.1111.4246
[55] Abigail N. Poteshman, Mykyta Onizhuk, Christopher Egerstrom, Daniel P. Mark, David D. Awschalom, F. Joseph Heremans, and Giulia Galli. High-throughput spin-bath characterization of spin defects in semiconductors. Phys. Rev. Appl., 24: 054048, 2025. 10.1103/p57x-8kk7.
https://doi.org/10.1103/p57x-8kk7
[56] A Dréau, J-R Maze, M Lesik, J-F Roch, and V Jacques. High-resolution spectroscopy of single nv defects coupled with nearby 13 c nuclear spins in diamond. Physical Review B, 85 (13): 134107, 2012. 10.1103/PhysRevB.85.134107.
https://doi.org/10.1103/PhysRevB.85.134107
Cited by
[1] Giulia Galli, Alfonso Castillo, Swarnabha Chattaraj, Siyuan Chen, Marco Govoni, Yu Jin, Jonah Nagura, Abigail N. Poteshman, Vrindaa Somjit, Michael Y. Toriyama, Victor Wen-zhe Yu, and Cunzhi Zhang, "Strategies to Predict and Design Spin Defects for Quantum Technologies", Journal of Chemical Theory and Computation 22 15, 7481 (2026).
[2] Federico Belliardo, Erik M. Gauger, Mohamed H. Abobeih, Tim H. Taminiau, Yoann Altmann, and Cristian Bonato, "Multidimensional Quantum Estimation and Model Learning Framework Based on Variational Bayesian Inference", PRX Quantum 7 2, 020360 (2026).
[3] Abigail N. Poteshman, Mykyta Onizhuk, Christopher Egerstrom, Daniel P. Mark, David D. Awschalom, F. Joseph Heremans, and Giulia Galli, "High-throughput spin-bath characterization of spin defects in semiconductors", Physical Review Applied 24 5, 054048 (2025).
[4] Pierre Kuna, Erik Hesselmeier-Hüttmann, Phillip Schillinger, Felix Gloistein, István Takács, Viktor Ivády, Wolfgang Knolle, Jawad Ul-Hassan, Jörg Wrachtrup, and Vadim Vorobyov, "Localization and coherent control of 25 nuclear spins in Silicon Carbide", arXiv:2512.20281, (2025).
The above citations are from Crossref's cited-by service (last updated successfully 2026-08-13 23:43:40) and SAO/NASA ADS (last updated successfully 2026-08-13 23:43:40). The list may be incomplete as not all publishers provide suitable and complete citation data.
This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.