Quantum Lazy Training

Erfan Abedi; Salman Beigi; Leila Taghavi

doi:10.22331/q-2023-04-27-989

Quantum Lazy Training

Erfan Abedi, Salman Beigi, and Leila Taghavi

QuOne Lab, Phanous Research & Innovation Centre, Tehran, Iran

Published:	2023-04-27, volume 7, page 989
Eprint:	arXiv:2202.08232v7
Doi:	https://doi.org/10.22331/q-2023-04-27-989
Citation:	Quantum 7, 989 (2023).

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.

Abstract

In the training of over-parameterized model functions via gradient descent, sometimes the parameters do not change significantly and remain close to their initial values. This phenomenon is called $\textit{lazy training}$ and motivates consideration of the linear approximation of the model function around the initial parameters. In the lazy regime, this linear approximation imitates the behavior of the parameterized function whose associated kernel, called the $\textit{tangent kernel}$, specifies the training performance of the model. Lazy training is known to occur in the case of (classical) neural networks with large widths. In this paper, we show that the training of $\textit{geometrically local}$ parameterized quantum circuits enters the lazy regime for large numbers of qubits. More precisely, we prove bounds on the rate of changes of the parameters of such a geometrically local parameterized quantum circuit in the training process, and on the precision of the linear approximation of the associated quantum model function; both of these bounds tend to zero as the number of qubits grows. We support our analytic results with numerical simulations.

► BibTeX data

@article{Abedi2023quantumlazytraining,
  doi = {10.22331/q-2023-04-27-989},
  url = {https://doi.org/10.22331/q-2023-04-27-989},
  title = {Quantum {L}azy {T}raining},
  author = {Abedi, Erfan and Beigi, Salman and Taghavi, Leila},
  journal = {{Quantum}},
  issn = {2521-327X},
  publisher = {{Verein zur F{\"{o}}rderung des Open Access Publizierens in den Quantenwissenschaften}},
  volume = {7},
  pages = {989},
  month = apr,
  year = {2023}
}

► References

[1] John Preskill. Quantum Computing in the NISQ era and beyond. Quantum, 2:79, 2018. doi:10.22331/q-2018-08-06-79.
https://doi.org/10.22331/q-2018-08-06-79

[2] Marco Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, et al. Variational quantum algorithms. Nature Reviews Physics, 3(9):625–644, 2021. doi:10.1038/s42254-021-00348-9.
https://doi.org/10.1038/s42254-021-00348-9

[3] Jarrod R McClean, Sergio Boixo, Vadim N Smelyanskiy, Ryan Babbush, and Hartmut Neven. Barren plateaus in quantum neural network training landscapes. Nature communications, 9(1):1–6, 2018. doi:10.1038/s41467-018-07090-4.
https://doi.org/10.1038/s41467-018-07090-4

[4] Samson Wang, Enrico Fontana, Marco Cerezo, Kunal Sharma, Akira Sone, Lukasz Cincio, and Patrick J Coles. Noise-induced barren plateaus in variational quantum algorithms. Nature communications, 12(1):1–11, 2021. doi:10.1038/s41467-021-27045-6.
https://doi.org/10.1038/s41467-021-27045-6

[5] Francis Bach. Effortless optimization through gradient flows. https://francisbach.com/gradient-flows.
https://francisbach.com/gradient-flows

[6] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems (NeurIPS 2018), 31:8571–8580, 2018. doi:10.1145/3406325.3465355.
https://doi.org/10.1145/3406325.3465355

[7] Lenaic Chizat, Edouard Oyallon, and Francis Bach. On lazy training in differentiable programming. Advances in Neural Information Processing Systems, 32, 2019.

[8] Kouhei Nakaji, Hiroyuki Tezuka, and Naoki Yamamoto. Quantum-enhanced neural networks in the neural tangent kernel framework. 2021. arXiv:2109.03786.
arXiv:2109.03786

[9] Norihito Shirai, Kenji Kubo, Kosuke Mitarai, and Keisuke Fujii. Quantum tangent kernel. 2021. arXiv:2111.02951.
arXiv:2111.02951

[10] Maria Schuld and Nathan Killoran. Quantum machine learning in feature hilbert spaces. Phys. Rev. Lett., 122:040504, Feb 2019. doi:10.1103/PhysRevLett.122.040504.
https://doi.org/10.1103/PhysRevLett.122.040504

[11] Vojtěch Havlíček, Antonio D. Córcoles, Kristan Temme, Aram W. Harrow, Abhinav Kandala, Jerry M. Chow, and Jay M. Gambetta. Supervised learning with quantum-enhanced feature spaces. Nature, 567(7747):209–212, Mar 2019. doi:10.1038/s41586-019-0980-2.
https://doi.org/10.1038/s41586-019-0980-2

[12] Junyu Liu, Francesco Tacchino, Jennifer R. Glick, Liang Jiang, and Antonio Mezzacapo. Representation learning via quantum neural tangent kernels. PRX Quantum, 3:030323, 2022. doi:10.1103/PRXQuantum.3.030323.
https://doi.org/10.1103/PRXQuantum.3.030323

[13] Di Luo and James Halverson. Infinite neural network quantum states. 2021. arXiv:2112.00723.
arXiv:2112.00723

[14] Junyu Liu, Khadijeh Najafi, Kunal Sharma, Francesco Tacchino, Liang Jiang, and Antonio Mezzacapo. An analytic theory for the dynamics of wide quantum neural networks. Phys. Rev. Lett., 130(15):150601, 2023. doi:10.1103/PhysRevLett.130.150601.
https://doi.org/10.1103/PhysRevLett.130.150601

[15] Junyu Liu, Zexi Lin, and Liang Jiang. Laziness, barren plateau, and noise in machine learning, 2022. doi:10.48550/arXiv.2206.09313.
https://doi.org/10.48550/arXiv.2206.09313

[16] Edward Farhi and Hartmut Neven. Classification with quantum neural networks on near term processors. 2018. arXiv:1802.06002.
arXiv:1802.06002

[17] M. Cerezo, Akira Sone, Tyler Volkoff, Lukasz Cincio, and Patrick J. Coles. Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nature Communications, 12(1):1791, 2021. doi:10.1038/s41467-021-21728-w.
https://doi.org/10.1038/s41467-021-21728-w

[18] Adrián Pérez-Salinas, Alba Cervera-Lierta, Elies Gil-Fuster, and José I. Latorre. Data re-uploading for a universal quantum classifier. Quantum, 4:226, 2020. doi:10.22331/q-2020-02-06-226.
https://doi.org/10.22331/q-2020-02-06-226

[19] Maria Schuld, Ryan Sweke, and Johannes Jakob Meyer. Effect of data encoding on the expressive power of variational quantum-machine-learning models. Phys. Rev. A, 103:032430, Mar 2021. doi:10.1103/PhysRevA.103.032430.
https://doi.org/10.1103/PhysRevA.103.032430

[20] Colin McDiarmid. On the method of bounded differences. In Surveys in combinatorics, 1989 (Norwich, 1989), volume 141 of London Math. Soc. Lecture Note Ser., pages 148–188. Cambridge Univ. Press, Cambridge, 1989. doi:10.1017/cbo9781107359949.008.
https://doi.org/10.1017/cbo9781107359949.008

[21] Ville Bergholm, Josh Izaac, Maria Schuld, Christian Gogolin, M Sohaib Alam, Shahnawaz Ahmed, Juan Miguel Arrazola, Carsten Blank, Alain Delgado, Soran Jahangiri, et al. Pennylane: Automatic differentiation of hybrid quantum-classical computations. 2018. arXiv:1811.04968.
arXiv:1811.04968

[22] Kerstin Beer, Dmytro Bondarenko, Terry Farrelly, Tobias J Osborne, Robert Salzmann, Daniel Scheiermann, and Ramona Wolf. Training deep quantum neural networks. Nature communications, 11(1):1–6, 2020. doi:10.1038/s41467-020-14454-2.
https://doi.org/10.1038/s41467-020-14454-2

Cited by

[1] Anderson Melchor Hernandez, Filippo Girardi, Davide Pastorello, and Giacomo De Palma, "Quantitative Convergence of Trained Quantum Neural Networks to a Gaussian Process", Annales Henri Poincaré (2025).

[2] Choy Boy, Edoardo Altamura, Dilhan Manawadu, Ivano Tavernelli, Stefano Mensa, and David J Wales, "Encoding molecular structures in quantum machine learning", Machine Learning: Science and Technology 6 4, 045076 (2025).

[3] Filippo Girardi and Giacomo De Palma, "Trained Quantum Neural Networks are Gaussian Processes", Communications in Mathematical Physics 406 4, 92 (2025).

[4] Yanbin Chen, Christian B. Mendl, and Helmut Seidl, "Dead gate elimination in hybrid quantum–classical programs", Journal of Computational Science 95, 102789 (2026).

[5] Yanbin Chen, Christian B. Mendl, and Helmut Seidl, Lecture Notes in Computer Science 15905, 135 (2025) ISBN:978-3-031-97631-5.

[6] Li-Wei Yu, Weikang Li, Qi Ye, Zhide Lu, Zizhao Han, and Dong-Ling Deng, "Expressibility-induced concentration of quantum neural tangent kernels", Reports on Progress in Physics 87 11, 110501 (2024).

[7] Anderson Melchor Hernandez, Davide Pastorello, and Giacomo De Palma, "Efficient classical computation of the neural tangent kernel of quantum neural networks", Quantum 10, 2118 (2026).

[8] Seungeun Oh, Jinhyuk Kim, Jihong Park, Hankyul Baek, Hyunsoo Lee, Joongheon Kim, and Seong-Lyun Kim, "Quantum infidelity codistillation for fast and accurate distributed quantum machine learning", The Journal of Supercomputing 81 10, 1151 (2025).

[9] Massimiliano Incudini, Michele Grossi, Antonio Mandarino, Sofia Vallecorsa, Alessandra Di Pierro, and David Windridge, "The Quantum Path Kernel: A Generalized Neural Tangent Kernel for Deep Quantum Machine Learning", IEEE Transactions on Quantum Engineering 4, 1 (2023).

[10] Sabri Meyer, Francesco Scala, Francesco Tacchino, and Aurelien Lucchi, "Gradient scalability and Taylor surrogation of quantum cost landscapes", Physical Review Research 8 2, 023325 (2026).

[11] Yanbin Chen, Christian B. Mendl, and Helmut Seidl, "Dead Gate Elimination in Hybrid Quantum-Classical Programs", (2025).

[12] Junyu Liu, Zexi Lin, and Liang Jiang, "Laziness, barren plateau, and noises in machine learning", Machine Learning: Science and Technology 5 1, 015058 (2024).

[13] Michelle Gelman, "A Survey of Methods for Mitigating Barren Plateaus for Parameterized Quantum Circuits", arXiv:2406.14285, (2024).

[14] Yanbin Chen, Christian B. Mendl, and Helmut Seidl, "Dead Gate Elimination", arXiv:2504.12729, (2025).

[15] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, and Dacheng Tao, "Erratum: Learnability of Quantum Neural Networks [PRX QUANTUM 2, 040337 (2021)]", PRX Quantum 3 3, 030901 (2022).

The above citations are from Crossref's cited-by service (last updated successfully 2026-08-11 22:56:12) and SAO/NASA ADS (last updated successfully 2026-08-11 22:56:13). The list may be incomplete as not all publishers provide suitable and complete citation data.

This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.