Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems

Hannah Lange; Guillaume Bornet; Gabriel Emperauger; Cheng Chen; Thierry Lahaye; Stefan Kienle; Antoine Browaeys; Annabelle Bohrdt

doi:10.22331/q-2025-03-26-1675

Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems

Hannah Lange^1,2,3, Guillaume Bornet⁴, Gabriel Emperauger⁴, Cheng Chen⁴, Thierry Lahaye⁴, Stefan Kienle⁵, Antoine Browaeys⁴, and Annabelle Bohrdt^3,6

¹Ludwig-Maximilians-University Munich, Theresienstr. 37, Munich D-80333, Germany
²Max-Planck-Institute for Quantum Optics, Hans-Kopfermann-Str.1, Garching D-85748, Germany
³Munich Center for Quantum Science and Technology, Schellingstr. 4, Munich D-80799, Germany
⁴Université Paris-Saclay, Institut d’Optique Graduate School, CNRS, Laboratoire Charles Fabry, 91127 Palaiseau Cedex, France
⁵FORRS Partners GmbH, Happelstr. 11, 69120 Heidelberg, Germany
⁶University of Regensburg, Universitätsstr. 31, Regensburg D-93053, Germany

Published:	2025-03-26, volume 9, page 1675
Editor:	Philipp Schindler
Eprint:	arXiv:2406.00091v2
Doi:	https://doi.org/10.22331/q-2025-03-26-1675
Citation:	Quantum 9, 1675 (2025).

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.

Abstract

Owing to their great expressivity and versatility, neural networks have gained attention for simulating large two-dimensional quantum many-body systems. However, their expressivity comes with the cost of a challenging optimization due to the in general rugged and complicated loss landscape. Here, we present a hybrid optimization scheme for neural quantum states (NQS), involving a data-driven pretraining with numerical or experimental data and a second, Hamiltonian-driven optimization stage. By using both projective measurements from the computational basis as well as expectation values from other measurement configurations such as spin-spin correlations, our pretraining gives access to the sign structure of the state, yielding improved and faster convergence that is robust w.r.t. experimental imperfections and limited datasets. We apply the hybrid scheme to the ground state search for the 2D transverse field Ising model and dipolar XY model on $6\times 6$ and $10\times 10$ square lattices with a patched transformer wave function, using numerical data as well as experimental data from a programmable Rydberg quantum simulator [Chen et al., Nature 616 (2023)], and show that the information from a second measurement basis highly improves the performance. Our work paves the way for a reliable and efficient optimization of neural quantum states.

Featured image: The hybrid training: a. We use a data-driven pretraining (part 1) based on the Wasserstein distance or the Kullback Leibler divergence (KL) for snapshots from the Z basis and on the scaled mean-square error (MSE) for expectation values from the X basis, before the Hamiltonian-driven variational Monte Carlo training (part 2). We apply this hybrid training scheme to the dipolar XY model (b.) and the transverse field Ising model (c.).

Popular summary

Neural networks are powerful tools for capturing complex dependencies, as seen in many applications we use every day. In quantum physics, they have shown great promise for simulating quantum systems, but their expressiveness comes with the challenge of difficult optimization. In this work, we introduce a hybrid optimization approach for neural quantum states (NQS) that combines data-driven and physics-driven training. Our method leverages both standard quantum measurements and additional data, such as spin-spin correlations, to achieve a well-informed initialization of the NQS, improving accuracy and accelerating overall convergence. By testing our approach on key quantum spin models using both simulated and experimental data from a quantum simulator, we demonstrate that incorporating measurements from different configurations significantly enhances performance.

► BibTeX data

@article{Lange2025transformerneural,
  doi = {10.22331/q-2025-03-26-1675},
  url = {https://doi.org/10.22331/q-2025-03-26-1675},
  title = {Transformer neural networks and quantum simulators: a hybrid approach for simulating strongly correlated systems},
  author = {Lange, Hannah and Bornet, Guillaume and Emperauger, Gabriel and Chen, Cheng and Lahaye, Thierry and Kienle, Stefan and Browaeys, Antoine and Bohrdt, Annabelle},
  journal = {{Quantum}},
  issn = {2521-327X},
  publisher = {{Verein zur F{\"{o}}rderung des Open Access Publizierens in den Quantenwissenschaften}},
  volume = {9},
  pages = {1675},
  month = mar,
  year = {2025}
}

► References

[1] Román Orús. ``A practical introduction to tensor networks: Matrix product states and projected entangled pair states''. Annals of Physics 349, 117–158 (2014).
https://doi.org/10.1016/j.aop.2014.06.013

[2] Ulrich Schollwöck. ``The density-matrix renormalization group in the age of matrix product states''. Annals of Physics 326, 96–192 (2011).
https://doi.org/10.1016/j.aop.2010.09.012

[3] J. Eisert, M. Cramer, and M. B. Plenio. ``Colloquium: Area laws for the entanglement entropy''. Rev. Mod. Phys. 82, 277–306 (2010).
https://doi.org/10.1103/RevModPhys.82.277

[4] M B Hastings. ``An area law for one-dimensional quantum systems''. Journal of Statistical Mechanics: Theory and Experiment 2007, P08024 (2007).
https://doi.org/10.1088/1742-5468/2007/08/P08024

[5] G. Cybenko. ``Approximation by superpositions of a sigmoidal function''. Mathematics of Control, Signals and Systems 2, 303–314 (1989).
https://doi.org/10.1007/BF02551274

[6] Kurt Hornik. ``Approximation capabilities of multilayer feedforward networks''. Neural Networks 4, 251–257 (1991).
https://doi.org/10.1016/0893-6080(91)90009-T

[7] Giuseppe Carleo and Matthias Troyer. ``Solving the quantum many-body problem with artificial neural networks''. Science 355, 602–606 (2017).
https://doi.org/10.1126/science.aag2302

[8] Hannah Lange, Anka Van de Walle, Atiye Abedinnia, and Annabelle Bohrdt. ``From architectures to applications: a review of neural quantum states'' (2024).

[9] Matija Medvidović and Javier Robledo Moreno. ``Neural-network quantum states for many-body physics'' (2024).
https://doi.org/10.1140/epjp/s13360-024-05311-y

[10] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. ``Attention is all you need''. Advances in neural information processing systems 30 (2017). url: https://arxiv.org/pdf/1706.03762.
https://arxiv.org/pdf/1706.03762

[11] Yuan-Hang Zhang and Massimiliano Di Ventra. ``Transformer Quantum State: A Multi-Purpose Model for Quantum Many-Body Problems''. Physical Review B 107, 075147 (2023).
https://doi.org/10.1103/PhysRevB.107.075147

[12] Kyle Sprague and Stefanie Czischek. ``Variational monte carlo with large patched transformers'' (2024).
https://doi.org/10.1038/s42005-024-01584-y

[13] Luciano Loris Viteritti, Riccardo Rende, and Federico Becca. ``Transformer variational wave functions for frustrated quantum spin systems''. Physical Review Letters 130, 236401 (2023).
https://doi.org/10.1103/PhysRevLett.130.236401

[14] Riccardo Rende, Federica Gerace, Alessandro Laio, and Sebastian Goldt. ``Mapping of attention mechanisms to a generalized potts model'' (2024).
https://doi.org/10.1103/PhysRevResearch.6.023057

[15] Riccardo Rende, Luciano Loris Viteritti, Lorenzo Bardone, Federico Becca, and Sebastian Goldt. ``A simple linear algebra identity to optimize large-scale neural network quantum states''. Communications Physics 7, 260 (2024).
https://doi.org/10.1038/s42005-024-01732-4

[16] Di Luo, Zhuo Chen, Juan Carrasquilla, and Bryan K. Clark. ``Autoregressive neural network for simulating open quantum systems via a probabilistic formulation''. Phys. Rev. Lett. 128, 090501 (2022).
https://doi.org/10.1103/PhysRevLett.128.090501

[17] Di Luo, Zhuo Chen, Kaiwen Hu, Zhizhen Zhao, Vera Mikyoung Hur, and Bryan K. Clark. ``Gauge-invariant and anyonic-symmetric autoregressive neural network for quantum lattice models''. Phys. Rev. Res. 5, 013216 (2023).
https://doi.org/10.1103/PhysRevResearch.5.013216

[18] Marin Bukov, Markus Schmitt, and Maxime Dupont. ``Learning the ground state of a non-stoquastic quantum Hamiltonian in a rugged neural network landscape''. SciPost Phys. 10, 147 (2021).
https://doi.org/10.21468/SciPostPhys.10.6.147

[19] Chae-Yeun Park and Michael J. Kastoryano. ``Geometry of learning neural quantum states''. Phys. Rev. Res. 2, 023232 (2020).
https://doi.org/10.1103/PhysRevResearch.2.023232

[20] Estelle M. Inack, Stewart Morawetz, and Roger G. Melko. ``Neural annealing and visualization of autoregressive neural networks in the newman-moore model''. Condensed Matter 7 (2022).
https://doi.org/10.3390/condmat7020038

[21] G. Torlai, G. Mazzola, Ju. Carrasquilla, M. Troyer, R. Melko, and G. Carleo. ``Neural-network quantum state tomography''. Nature Phys 14, 447–450 (2018).
https://doi.org/10.1038/s41567-018-0048-5

[22] Sanjaya Lohani, Brian T Kirby, Michael Brodsky, Onur Danaci, and Ryan T Glasser. ``Machine learning assisted quantum state estimation''. Machine Learning: Science and Technology 1, 035007 (2020).
https://doi.org/10.1088/2632-2153/ab9a21

[23] Dominik Koutný, Libor Motka, Zdeněk Hradil, Jaroslav Řeháček, and Luis L. Sánchez-Soto. ``Neural-network quantum state tomography''. Phys. Rev. A 106, 012409 (2022).
https://doi.org/10.1103/PhysRevA.106.012409

[24] Stefanie Czischek, M. Schuyler Moss, Matthew Radzihovsky, Ejaaz Merali, and Roger G. Melko. ``Data-enhanced variational Monte Carlo simulations for Rydberg atom arrays''. Phys. Rev. B 105, 205108 (2022).
https://doi.org/10.1103/PhysRevB.105.205108

[25] M. Schuyler Moss, Sepehr Ebadi, Tout T. Wang, Giulia Semeghini, Annabelle Bohrdt, Mikhail D. Lukin, and Roger G. Melko. ``Enhancing variational Monte Carlo using a programmable quantum simulator'' (2023). arXiv:2308.02647.
arXiv:2308.02647

[26] Elizabeth R. Bennewitz, Florian Hopfmueller, Bohdan Kulchytskyy, Juan Carrasquilla, and Pooya Ronagh. ``Neural error mitigation of near-term quantum simulations''. Nature Machine Intelligence 4, 618–624 (2022).
https://doi.org/10.1038/s42256-022-00509-0

[27] Mohamed Hibat-Allah, Martin Ganahl, Lauren E. Hayward, Roger G. Melko, and Juan Carrasquilla. ``Recurrent neural network wave functions''. Phys. Rev. Res. 2, 23358 (2020).
https://doi.org/10.1103/PhysRevResearch.2.023358

[28] Cédric Villani. ``The wasserstein distances''. Pages 93–111. Springer Berlin Heidelberg. Berlin, Heidelberg (2009).
https://doi.org/10.1007/978-3-540-71050-9_6

[29] Cheng Chen, Guillaume Bornet, Marcus Bintz, Gabriel Emperauger, Lucas Leclerc, Vincent S. Liu, Pascal Scholl, Daniel Barredo, Johannes Hauschild, Shubhayu Chatterjee, Michael Schuler, Andreas M. Läuchli, Michael P. Zaletel, Thierry Lahaye, Norman Y. Yao, and Antoine Browaeys. ``Continuous symmetry breaking in a two-dimensional rydberg array''. Nature 616, 691–695 (2023).
https://doi.org/10.1038/s41586-023-05859-2

[30] David Fitzek, Yi Hong Teoh, Hin Pok Fung, Gebremedhin A. Dagnew, Ejaaz Merali, M. Schuyler Moss, Benjamin MacLellan, and Roger G. Melko. ``Rydberggpt'' (2024). arXiv:2405.21052.
arXiv:2405.21052

[31] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. ``An image is worth 16x16 words: Transformers for image recognition at scale'' (2021). arXiv:2010.11929.
arXiv:2010.11929

[32] W. L. McMillan. ``Ground state of liquid ${\mathrm{he}}^{4}$''. Phys. Rev. 138, A442–A451 (1965).
https://doi.org/10.1103/PhysRev.138.A442

[33] Li Huang and Lei Wang. ``Accelerated monte carlo simulations with restricted boltzmann machines''. Phys. Rev. B 95, 035105 (2017).
https://doi.org/10.1103/PhysRevB.95.035105

[34] Jong Chul Ye. ``Geometry of deep learning - a signal processing perspective''. Springer Nature. Singapore (2022).
https://doi.org/10.1007/978-981-16-6046-7

[35] Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, Ievgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. ``Pot: Python optimal transport''. Journal of Machine Learning Research 22, 1–8 (2021). url: http://jmlr.org/papers/v22/20-451.html.
http://jmlr.org/papers/v22/20-451.html

[36] Hsin-Yuan Huang, Richard Kueng, and John Preskill. ``Predicting many properties of a quantum system from very few measurements''. Nature Physics 16, 1050–1057 (2020).
https://doi.org/10.1038/s41567-020-0932-7

[37] Yihui Quek, Stanislav Fort, and Hui Khoon Ng. ``Adaptive quantum state tomography with neural networks'' (2018). arXiv:1812.06693.
arXiv:1812.06693

[38] Alistair W. R. Smith, Johnnie Gray, and M. S. Kim. ``Efficient quantum state sample tomography with basis-dependent neural networks''. PRX Quantum 2, 020348 (2021).
https://doi.org/10.1103/PRXQuantum.2.020348

[39] Hannah Lange, Matjaž Kebrič, Maximilian Buser, Ulrich Schollwöck, Fabian Grusdt, and Annabelle Bohrdt. ``Adaptive quantum state tomography with active learning''. Quantum 7, 1129 (2023).
https://doi.org/10.22331/q-2023-10-09-1129

[40] Federico Becca and Sandro Sorella. ``Quantum monte carlo approaches for correlated systems''. Cambridge University Press. (2017).
https://doi.org/10.1017/9781316417041

[41] Cole Miles, Annabelle Bohrdt, Ruihan Wu, Christie Chiu, Muqing Xu, Geoffrey Ji, Markus Greiner, Kilian Q. Weinberger, Eugene Demler, and Eun-Ah Kim. ``Correlator convolutional neural networks as an interpretable architecture for image-like quantum matter data''. Nature Communications 12, 3905 (2021).
https://doi.org/10.1038/s41467-021-23952-w

[42] R. Verdel, V. Vitale, R. K. Panda, E. D. Donkor, A. Rodriguez, S. Lannig, Y. Deller, H. Strobel, M. K. Oberthaler, and M. Dalmonte. ``Data-driven discovery of statistically relevant information in quantum simulators''. Phys. Rev. B 109, 075152 (2024).
https://doi.org/10.1103/PhysRevB.109.075152

[43] Moritz Reh, Markus Schmitt, and Martin Gärttner. ``Optimizing design choices for neural quantum states''. Phys. Rev. B 107, 195115 (2023).
https://doi.org/10.1103/PhysRevB.107.195115

[44] Diederik P. Kingma and Jimmy Ba. ``Adam: A method for stochastic optimization'' (2017). arXiv:1412.6980.
arXiv:1412.6980

[45] Sandro Sorella. ``Green function monte carlo with stochastic reconfiguration''. Phys. Rev. Lett. 80, 4558–4561 (1998).
https://doi.org/10.1103/PhysRevLett.80.4558

[46] Sandro Sorella. ``Generalized lanczos algorithm for variational quantum monte carlo''. Phys. Rev. B 64, 024512 (2001).
https://doi.org/10.1103/PhysRevB.64.024512

[47] Ao Chen and Markus Heyl. ``Empowering deep neural quantum states through efficient optimization'' (2024).
https://doi.org/10.1038/s41567-024-02566-1

[48] Kaelan Donatella, Zakari Denis, Alexandre Le Boité, and Cristiano Ciuti. ``Dynamics with autoregressive neural quantum states: Application to critical quench dynamics''. Phys. Rev. A 108, 022210 (2023).
https://doi.org/10.1103/PhysRevA.108.022210

[49] Hannah Lange, Fabian Döschl, Juan Carrasquilla, and Annabelle Bohrdt. ``Neural network approach to quasiparticle dispersions in doped antiferromagnets'' (2023). arXiv:2310.08578.
arXiv:2310.08578

[50] Pascal Scholl, Michael Schuler, Hannah J. Williams, Alexander A. Eberharter, Daniel Barredo, Kai-Niklas Schymik, Vincent Lienhard, Louis-Paul Henry, Thomas C. Lang, Thierry Lahaye, Andreas M. Läuchli, and Antoine Browaeys. ``Quantum simulation of 2d antiferromagnets with hundreds of rydberg atoms''. Nature 595, 233–238 (2021).
https://doi.org/10.1038/s41586-021-03585-1

[51] Daniel Barredo, Sylvain de Léséleuc, Vincent Lienhard, Thierry Lahaye, and Antoine Browaeys. ``An atom-by-atom assembler of defect-free arbitrary two-dimensional atomic arrays''. Science 354, 1021–1023 (2016).
https://doi.org/10.1126/science.aah3778

[52] Sylvain de Léséleuc, Daniel Barredo, Vincent Lienhard, Antoine Browaeys, and Thierry Lahaye. ``Optical control of the resonant dipole-dipole interaction between rydberg atoms''. Phys. Rev. Lett. 119, 053202 (2017).
https://doi.org/10.1103/PhysRevLett.119.053202

[53] Cheng Chen, Gabriel Emperauger, Guillaume Bornet, Filippo Caleca, Bastien Gély, Marcus Bintz, Shubhayu Chatterjee, Vincent Liu, Daniel Barredo, Norman Y. Yao, Thierry Lahaye, Fabio Mezzacapo, Tommaso Roscilde, and Antoine Browaeys. ``Spectroscopy of elementary excitations from quench dynamics in a dipolar xy rydberg simulator'' (2023). arXiv:2311.11726.
arXiv:2311.11726

[54] C. Hubig, F. Lachenmaier, N.-O. Linden, T. Reinhard, L. Stenzel, A. Swoboda, and M. Grundner. ``The SyTen toolkit''. https://syten.eu.
https://syten.eu

[55] C. Hubig. ``Symmetry-protected tensor networks''. PhD thesis. LMU München. (2017). url: https://edoc.ub.uni-muenchen.de/21348/.
https://edoc.ub.uni-muenchen.de/21348/

Cited by

[1] Andrew Jreissaty, Hang Zhang, Jairo C. Quijano, Juan Carrasquilla, and Roeland Wiersema, "Entanglement and optimization within autoregressive neural quantum states", Physical Review Research 8 1, 013147 (2026).

[2] Yuxuan Du, Yan Zhu, Yuan-Hang Zhang, Min-Hsiu Hsieh, Patrick Rebentrost, Weibo Gao, Yi-Zhuang You, Jens Eisert, Giulio Chiribella, Dacheng Tao, Barry C. Sanders, and Ya-Dong Wu, "Artificial intelligence for representing and characterizing quantum systems", Nature Reviews Physics (2026).

[3] Frederik Møller, Gabriel Fernández-Fernández, Thomas Schweigler, Paulin de Schoulepnikoff, Jörg Schmiedmayer, and Gorka Muñoz-Gil, "Learning minimal representations of many-body physics from snapshots of a quantum simulator", Physical Review Research 8 2, 023094 (2026).

[4] Harriet L. Dao, "Hyperbolic recurrent neural network as the first type of non-Euclidean neural quantum state ansatz", The European Physical Journal Plus 141 2, 199 (2026).

[5] Alessandro Sinibaldi, Douglas Hendry, Filippo Vicentini, and Giuseppe Carleo, "Time-Dependent Neural Galerkin Method for Quantum Dynamics", Physical Review Letters 136 12, 120402 (2026).

[6] Ao Chen, Vighnesh Dattatraya Naik, and Markus Heyl, "Convolutional transformer wave functions", Physical Review Research 8 2, L022040 (2026).

[7] Zongkang Zhang, Ying Li, and Xiaosi Xu, "Quantum-Enhanced Neural Networks for Quantum Many-Body Simulations", PRX Intelligence 1 1, 013002 (2026).

[8] Christian Reinmoser, Muqing Xu, Lev Haldar Kendrick, Anant Kale, Youqi Gang, Martin Lebrat, Markus Greiner, Fabian Grusdt, and Annabelle Bohrdt, "Optimized Gutzwiller projected states for doped antiferromagnets in Fermi-Hubbard simulators", Physical Review B 114 6, 065131 (2026).

[9] Marco Julian Solanki, Lexin Ding, and Markus Reiher, "Neural Quantum States Based on Selected Configurations", The Journal of Physical Chemistry Letters 17 18, 5180 (2026).

[10] Mohamed Hibat-Allah, Ejaaz Merali, Giacomo Torlai, Roger G. Melko, and Juan Carrasquilla, "Recurrent neural network wave functions for Rydberg atom arrays on kagome lattice", Communications Physics 8 1, 308 (2025).

[11] João Augusto Sobral, Michael Perle, and Mathias S. Scheurer, "Physics-informed transformers for electronic quantum states", Nature Communications 16 1, 10811 (2025).

[12] Xavi F. Aragones and Miguel A. González Ballester, Communications in Computer and Information Science 2948, 307 (2027) ISBN:978-981-92-1545-4.

[13] Tatiana Vovk, Anka Van de Walle, Hannes Pichler, and Annabelle Bohrdt, "Neural quantum states for emitter dynamics in waveguide QED", Physical Review A 113 5, 053708 (2026).

[14] Hannah Lange, Anka Van de Walle, Atiye Abedinnia, and Annabelle Bohrdt, "From architectures to applications: a review of neural quantum states", Quantum Science and Technology 9 4, 040501 (2024).

[15] Jian Yao and Yi-Zhuang You, "ShadowGPT: Learning to Solve Quantum Many-Body Problems from Randomized Measurements", arXiv:2411.03285, (2024).

[16] Yangrui Hu, Yi Hong Teoh, William Witczak-Krempa, and Roger G. Melko, "Neural network enhanced cross entropy benchmark for monitored circuits", Machine Learning: Science and Technology 6 4, 045030 (2025).

[17] Eduardo Ibarra-García-Padilla, Hannah Lange, Roger G. Melko, Richard T. Scalettar, Juan Carrasquilla, Annabelle Bohrdt, and Ehsan Khatami, "Autoregressive neural quantum states of Fermi Hubbard models", Physical Review Research 7 1, 013122 (2025).

[18] Tatiana Vovk, Anka Van de Walle, Hannes Pichler, and Annabelle Bohrdt, "Neural quantum states for emitter dynamics in waveguide QED", arXiv:2508.08964, (2025).

[19] Tao Xu, Zhixin Hu, Li Luo, and Momiao Xiong, "Physical Transformer", arXiv:2601.02433, (2026).

[20] H. L. Dao, "New non-Euclidean neural quantum states from additional types of hyperbolic recurrent neural networks", arXiv:2604.24337, (2026).

The above citations are from Crossref's cited-by service (last updated successfully 2026-08-07 23:30:06) and SAO/NASA ADS (last updated successfully 2026-08-07 23:30:13). The list may be incomplete as not all publishers provide suitable and complete citation data.

This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.