Entanglement entropy production in Quantum Neural Networks

Marco Ballarin1,2,3, Stefano Mangini1,4,5, Simone Montangero2,3,6, Chiara Macchiavello4,5,7, and Riccardo Mengoni8

1These authors contributed equally to this work
2Dipartimento di Fisica e Astronomia "G. Galilei", via Marzolo 8, I-35131, Padova, Italy
3INFN, Sezione di Padova, via Marzolo 8, I-35131, Padova, Italy
4Dipartimento di Fisica, Università di Pavia, Via Bassi 6, I-27100, Pavia, Italy
5INFN Sezione di Pavia, Via Bassi 6, I-27100, Pavia, Italy
6Padua Quantum Technologies Research Center, Università degli Studi di Padova
7CNR-INO - Largo E. Fermi 6, I-50125, Firenze, Italy
8CINECA Quantum Computing Lab,Via Magnanelli, 6/3, 40033 Casalecchio di Reno, Bologna, Italy

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.


Quantum Neural Networks (QNN) are considered a candidate for achieving quantum advantage in the Noisy Intermediate Scale Quantum computer (NISQ) era. Several QNN architectures have been proposed and successfully tested on benchmark datasets for machine learning. However, quantitative studies of the QNN-generated entanglement have been investigated only for up to few qubits. Tensor network methods allow to emulate quantum circuits with a large number of qubits in a wide variety of scenarios. Here, we employ matrix product states to characterize recently studied QNN architectures with random parameters up to fifty qubits showing that their entanglement, measured in terms of entanglement entropy between qubits, tends to that of Haar distributed random states as the depth of the QNN is increased. We certify the randomness of the quantum states also by measuring the expressibility of the circuits, as well as using tools from random matrix theory. We show a universal behavior for the rate at which entanglement is created in any given QNN architecture, and consequently introduce a new measure to characterize the entanglement production in QNNs: the entangling speed. Our results characterise the entanglement properties of quantum neural networks, and provides new evidence of the rate at which these approximate random unitaries.

► BibTeX data

► References

[1] Michael A. Nielsen and Isaac L. Chuang. ``Quantum computation and quantum information''. Cambridge University Press. Cambridge, UK (2010). 10th anniversary ed. (2010) edition.

[2] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. ``Deep learning''. MIT Press. (2016). url: http:/​/​www.deeplearningbook.org.

[3] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. ``Deep learning''. Nature 521, 436–444 (2015).

[4] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ``Imagenet classification with deep convolutional neural networks''. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1. Page 1097–1105. NIPS'12Red Hook, NY, USA (2012). Curran Associates Inc.

[5] David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. ``Mastering the game of Go with deep neural networks and tree search''. Nature 529, 484–489 (2016).

[6] Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de las Casas, Craig Donner, Leslie Fritz, Cristian Galperti, Andrea Huber, James Keeling, Maria Tsimpoukelli, Jackie Kay, Antoine Merle, Jean-Marc Moret, Seb Noury, Federico Pesamosca, David Pfau, Olivier Sauter, Cristian Sommariva, Stefano Coda, Basil Duval, Ambrogio Fasoli, Pushmeet Kohli, Koray Kavukcuoglu, Demis Hassabis, and Martin Riedmiller. ``Magnetic control of tokamak plasmas through deep reinforcement learning''. Nature 602, 414–419 (2022).

[7] Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd. ``Quantum machine learning''. Nature 549, 195–202 (2017).

[8] Vedran Dunjko and Peter Wittek. ``A non-review of quantum machine learning: trends and explorations''. Quantum 4, 32 (2020).

[9] M. Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R. McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, and et al. ``Variational quantum algorithms''. Nature Reviews Physics 3, 625–644 (2021).

[10] S. Mangini, F. Tacchino, D. Gerace, D. Bajoni, and C. Macchiavello. ``Quantum computing models for artificial neural networks''. Europhysics Letters 134, 10002 (2021).

[11] Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S. Kottmann, Tim Menke, Wai-Keong Mok, Sukin Sim, Leong-Chuan Kwek, and Alán Aspuru-Guzik. ``Noisy intermediate-scale quantum algorithms''. Rev. Mod. Phys. 94, 015004 (2022).

[12] John Preskill. ``Quantum Computing in the NISQ era and beyond''. Quantum 2, 79 (2018).

[13] Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Alán Aspuru-Guzik, and Jeremy L. O'Brien. ``A variational eigenvalue solver on a photonic quantum processor''. Nat. Commun. 5 (2014).

[14] Amira Abbas, David Sutter, Christa Zoufal, Aurelien Lucchi, Alessio Figalli, and Stefan Woerner. ``The power of quantum neural networks''. Nature Computational Science 1, 403–409 (2021).

[15] Hsin-Yuan Huang, Richard Kueng, and John Preskill. ``Information-theoretic bounds on quantum advantage in machine learning''. Phys. Rev. Lett. 126, 190505 (2021).

[16] Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R. McClean. ``Power of data in quantum machine learning''. Nature Communications 12, 2631 (2021).

[17] Franz J. Schreiber, Jens Eisert, and Johannes Jakob Meyer. ``Classical surrogates for quantum learning models'' (2022) arXiv:2206.11740.

[18] Thomas Hubregtsen, Josef Pichlmeier, Patrick Stecher, and Koen Bertels. ``Evaluation of parameterized quantum circuits: On the relation between classification accuracy, expressibility, and entangling capability''. Quantum Machine Intelligence 3, 9 (2021).

[19] M. Cerezo, Akira Sone, Tyler Volkoff, Lukasz Cincio, and Patrick J. Coles. ``Cost function dependent barren plateaus in shallow parametrized quantum circuits''. Nat. Commun. 12 (2021).

[20] Iris Cong, Soonwon Choi, and Mikhail D. Lukin. ``Quantum convolutional neural networks''. Nature Physics 15, 1273–1278 (2019).

[21] Johannes Jakob Meyer, Marian Mularski, Elies Gil-Fuster, Antonio Anna Mele, Francesco Arzani, Alissa Wilms, and Jens Eisert. ``Exploiting symmetry in variational quantum machine learning''. PRX Quantum 4, 010328 (2023).

[22] Andrea Skolik, Michele Cattelan, Sheir Yarkoni, Thomas Bäck, and Vedran Dunjko. ``Equivariant quantum circuits for learning on weighted graphs''. npj Quantum Information 9, 47 (2023).

[23] Sukin Sim, Peter D. Johnson, and Alán Aspuru-Guzik. ``Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms''. Adv. Quantum Technol. 2, 1900070 (2019).

[24] Adrián Pérez-Salinas, Alba Cervera-Lierta, Elies Gil-Fuster, and José I. Latorre. ``Data re-uploading for a universal quantum classifier''. Quantum 4, 226 (2020).

[25] Maria Schuld, Ryan Sweke, and Johannes Jakob Meyer. ``Effect of data encoding on the expressive power of variational quantum-machine-learning models''. Phys. Rev. A 103, 032430 (2021).

[26] Francesco Tacchino, Stefano Mangini, Panagiotis Kl. Barkoutsos, Chiara Macchiavello, Dario Gerace, Ivano Tavernelli, and Daniele Bajoni. ``Variational learning for quantum artificial neural networks''. IEEE Transactions on Quantum Engineering 2, 1–10 (2021).

[27] B Jaderberg, L W Anderson, W Xie, S Albanie, M Kiffner, and D Jaksch. ``Quantum self-supervised learning''. Quantum Science and Technology 7, 035005 (2022).

[28] David A. Meyer and Nolan R. Wallach. ``Global entanglement in multiparticle systems''. Journal of Mathematical Physics 43, 4273–4278 (2002).

[29] Pietro Silvi, Ferdinand Tschirsich, Matthias Gerster, Johannes Jünemann, Daniel Jaschke, Matteo Rizzi, and Simone Montangero. ``The tensor networks anthology: Simulation techniques for many-body quantum lattice systems''. SciPost Physics Lecture Notes (2019).

[30] S. Montangero. ``Introduction to tensor network methods''. Springer International Publishing. Cham, CH (2018).

[31] J. Eisert. ``Entanglement and tensor network states'' (2013). arXiv:1308.3318.

[32] Sebastian Paeckel, Thomas Köhler, Andreas Swoboda, Salvatore R. Manmana, Ulrich Schollwöck, and Claudius Hubig. ``Time-evolution methods for matrix-product states''. Annals of Physics 411, 167998 (2019).

[33] Patrick Hayden, Debbie W. Leung, and Andreas Winter. ``Aspects of Generic Entanglement''. Communications in Mathematical Physics 265, 95–117 (2006).

[34] Elizabeth S. Meckes. ``The Random Matrix Theory of the Classical Compact Groups''. Cambridge Tracts in Mathematics. Cambridge University Press. Cambridge (2019).

[35] Alan Edelman and N. Raj Rao. ``Random matrix theory''. Acta Numerica 14, 233–297 (2005).

[36] Don N. Page. ``Average entropy of a subsystem''. Phys. Rev. Lett. 71, 1291–1294 (1993).

[37] Jarrod R McClean, Jonathan Romero, Ryan Babbush, and Alán Aspuru-Guzik. ``The theory of variational hybrid quantum-classical algorithms''. New J. Phys. 18, 023023 (2016).

[38] Francisco Javier Gil Vidal and Dirk Oliver Theis. ``Input redundancy for parameterized quantum circuits''. Front. Phys. 8, 297 (2020).

[39] E. Torrontegui and J. J. Garcia-Ripoll. ``Unitary quantum perceptron as efficient universal approximator''. EPL 125, 30004 (2019).

[40] Jarrod R. McClean, Sergio Boixo, Vadim N. Smelyanskiy, Ryan Babbush, and Hartmut Neven. ``Barren plateaus in quantum neural network training landscapes''. Nat. Commun. 9, 4812 (2018).

[41] Maria Schuld, Ville Bergholm, Christian Gogolin, Josh Izaac, and Nathan Killoran. ``Evaluating analytic gradients on quantum hardware''. Phys. Rev. A 99, 032331 (2019).

[42] Andrew Arrasmith, M. Cerezo, Piotr Czarnik, Lukasz Cincio, and Patrick J. Coles. ``Effect of barren plateaus on gradient-free optimization''. Quantum 5, 558 (2021).

[43] Zoë Holmes, Kunal Sharma, M. Cerezo, and Patrick J. Coles. ``Connecting ansatz expressibility to gradient magnitudes and barren plateaus''. PRX Quantum 3, 010313 (2022).

[44] Carlos Ortiz Marrero, Mária Kieferová, and Nathan Wiebe. ``Entanglement-induced barren plateaus''. PRX Quantum 2, 040316 (2021).

[45] Samson Wang, Enrico Fontana, M. Cerezo, Kunal Sharma, Akira Sone, Lukasz Cincio, and Patrick J. Coles. ``Noise-induced barren plateaus in variational quantum algorithms''. Nature Communications 12, 6961 (2021).

[46] Christoph Dankert, Richard Cleve, Joseph Emerson, and Etera Livine. ``Exact and approximate unitary 2-designs and their application to fidelity estimation''. Physical Review A 80 (2009).

[47] Andrew Arrasmith, Zoë Holmes, Marco Cerezo, and Patrick J Coles. ``Equivalence of quantum barren plateaus to cost concentration and narrow gorges''. Quantum Science and Technology 7, 045015 (2022).

[48] Stefan H. Sack, Raimel A. Medina, Alexios A. Michailidis, Richard Kueng, and Maksym Serbyn. ``Avoiding barren plateaus using classical shadows''. PRX Quantum 3, 020365 (2022).

[49] Taylor L. Patti, Khadijeh Najafi, Xun Gao, and Susanne F. Yelin. ``Entanglement devised barren plateau mitigation''. Phys. Rev. Research 3, 033090 (2021).

[50] Zi-Wen Liu, Seth Lloyd, Elton Zhu, and Huangjun Zhu. ``Entanglement, quantum randomness, and complexity beyond scrambling''. Journal of High Energy Physics 2018, 41 (2018).

[51] Edward Grant, Leonard Wossnig, Mateusz Ostaszewski, and Marcello Benedetti. ``An initialization strategy for addressing barren plateaus in parametrized quantum circuits''. Quantum 3, 214 (2019).

[52] Tyler Volkoff and Patrick J Coles. ``Large gradients via correlation in random parameterized quantum circuits''. Quantum Science and Technology 6, 025008 (2021).

[53] Andrea Skolik, Jarrod R McClean, Masoud Mohseni, Patrick van der Smagt, and Martin Leib. ``Layerwise learning for quantum neural networks''. Quantum Machine Intelligence 3, 1–11 (2021).

[54] Joonho Kim and Yaron Oz. ``Entanglement diagnostics for efficient vqa optimization''. Journal of Statistical Mechanics: Theory and Experiment 2022, 073101 (2022).

[55] Vojtěch Havlíček, Antonio D. Córcoles, Kristan Temme, Aram W. Harrow, Abhinav Kandala, Jerry M. Chow, and Jay M. Gambetta. ``Supervised learning with quantum-enhanced feature spaces''. Nature 567, 209–212 (2019).

[56] Aram W. Harrow and Richard A. Low. ``Random Quantum Circuits are Approximate 2-designs''. Communications in Mathematical Physics 291, 257–302 (2009).

[57] Jonas Haferkamp and Nicholas Hunter-Jones. ``Improved spectral gaps for random quantum circuits: Large local dimensions and all-to-all interactions''. Phys. Rev. A 104, 022417 (2021).

[58] Maria Schuld. ``Supervised quantum machine learning models are kernel methods'' (2021) arXiv:2101.11020.

[59] Sofiene Jerbi, Lukas J Fiderer, Hendrik Poulsen Nautrup, Jonas M Kübler, Hans J Briegel, and Vedran Dunjko. ``Quantum machine learning beyond kernel methods''. Nature Communications 14, 517 (2023).

[60] Seth Lloyd. ``Quantum approximate optimization is computationally universal'' (2018) arXiv:1812.11075.

[61] M. E. S. Morales, J. D. Biamonte, and Z. Zimborás. ``On the universality of the quantum approximate optimization algorithm''. Quantum Information Processing 19, 291 (2020).

[62] Fernando G. S. L. Brandão, Aram W. Harrow, and Michał Horodecki. ``Local Random Quantum Circuits are Approximate Polynomial-Designs''. Communications in Mathematical Physics 346, 397–434 (2016).

[63] Aram W Harrow and Saeed Mehraban. ``Approximate unitary t-designs by short random quantum circuits using nearest-neighbor and long-range gates''. Communications in Mathematical PhysicsPages 1–96 (2023).

[64] Pasquale Calabrese and John Cardy. ``Evolution of entanglement entropy in one-dimensional systems''. Journal of Statistical Mechanics: Theory and Experiment 2005, P04010 (2005).

[65] Tianci Zhou and Adam Nahum. ``Emergent statistical mechanics of entanglement in random unitary circuits''. Phys. Rev. B 99, 174205 (2019).

[66] Adam Nahum, Jonathan Ruhman, Sagar Vijay, and Jeongwan Haah. ``Quantum entanglement growth under random unitary dynamics''. Phys. Rev. X 7, 031016 (2017).

[67] M. Aeberhard, Stefan & Forina. ``Wine''. UCI Machine Learning Repository (1991). DOI: https:/​/​doi.org/​10.24432/​C5PC7J.

[68] Milan Zwitter, Matjaz & Soklic. ``Breast Cancer''. UCI Machine Learning Repository (1988). DOI: https:/​/​doi.org/​10.24432/​C51P4M.

[69] Marko Žnidarič. ``Entanglement of random vectors''. Journal of Physics A: Mathematical and Theoretical 40, F105 (2006).

[70] Daniel Jaschke and Simone Montangero. ``Is quantum computing green? an estimate for an energy-efficiency quantum advantage''. Quantum Science and Technology (2022).

[71] V A Marčenko and L A Pastur. ``Distribution of eigenvalues for some sets of random matrices''. Mathematics of the USSR-Sbornik 1, 457 (1967).

[72] Zbigniew Puchała, Łukasz Pawela, and Karol Życzkowski. ``Distinguishability of generic quantum states''. Physical Review A 93, 062112 (2016).

[73] Maxime Dupont, Nicolas Didier, Mark J. Hodson, Joel E. Moore, and Matthew J. Reagor. ``Entanglement perspective on the quantum approximate optimization algorithm''. Phys. Rev. A 106, 022423 (2022).

[74] Andreas J. C. Woitzik, Panagiotis Kl. Barkoutsos, Filip Wudarski, Andreas Buchleitner, and Ivano Tavernelli. ``Entanglement production and convergence properties of the variational quantum eigensolver''. Phys. Rev. A 102, 042402 (2020).

[75] Michael Ragone, Paolo Braccia, Quynh T. Nguyen, Louis Schatzki, Patrick J. Coles, Frederic Sauvage, Martin Larocca, and M. Cerezo. ``Representation theory for geometric quantum machine learning'' (2022) arXiv:2210.07980.

[76] Kunal Sharma, M. Cerezo, Zoë Holmes, Lukasz Cincio, Andrew Sornborger, and Patrick J. Coles. ``Reformulation of the no-free-lunch theorem for entangled datasets''. Phys. Rev. Lett. 128, 070501 (2022).

[77] Martin Larocca, Nathan Ju, Diego García-Martín, Patrick J. Coles, and M. Cerezo. ``Theory of overparametrization in quantum neural networks'' (2021) arXiv:2109.11676.

[78] Bobak Toussi Kiani, Seth Lloyd, and Reevu Maity. ``Learning unitaries by gradient descent'' (2020) arXiv:2001.11897.

[79] Eric R. Anschuetz and Bobak T. Kiani. ``Quantum variational algorithms are swamped with traps''. Nature Communications 13 (2022).

[80] Md Sajid Anis et al. ``Qiskit: An open-source framework for quantum computing''. Zenodo (2021).

[81] Marco Ballarin. ``Quantum computer simulation via tensor networks''. Università degli Studi di Padova, Master Thesis (2021). url: https:/​/​hdl.handle.net/​20.500.12608/​21799.

[82] Ville Bergholm, Josh Izaac, Maria Schuld, Christian Gogolin, M Sohaib Alam, Shahnawaz Ahmed, Juan Miguel Arrazola, Carsten Blank, Alain Delgado, Soran Jahangiri, et al. ``Pennylane: Automatic differentiation of hybrid quantum-classical computations'' (2018). arXiv:1811.04968.

[83] Julian Havil. ``Gamma: exploring euler's constant''. The Australian Mathematical SocietyPage 250 (2003). url: https:/​/​ieeexplore.ieee.org/​document/​9452347.

[84] Juan Carlos Garcia-Escartin and Pedro Chamorro-Posada. ``Equivalent quantum circuits'' (2011). arXiv:1110.2998.

[85] Karol Życzkowski and Hans-Jürgen Sommers. ``Average fidelity between random quantum states''. Phys. Rev. A 71, 032313 (2005).

Cited by

[1] Seongwook Shin, Yong Siah Teo, and Hyunseok Jeong, "Dequantizing quantum machine learning models using tensor networks", Physical Review Research 6 2, 023218 (2024).

[2] Aastha Sharma, Harsh Kumar Sharma, and Kirti Sharma, 2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) 982 (2023) ISBN:979-8-3503-8247-1.

[3] Paolo Andreetto, Fulvia Costa, Alberto Crescente, Sergio Fantinel, Federica Fanzago, Alberto Garfagnini, Loris Lazzaro, Rita Lenzo, Paolo Emilio Mazzon, Matteo Menguzzato, Gianpietro Sella, Massimo Sgaravatto, Sergio Traldi, Marco Verlato, Lisa Zangrando, R. De Vita, X. Espinal, P. Laycock, and O. Shadura, "The CloudVeneto initiative: 10 years of operations to support interdisciplinary open science", EPJ Web of Conferences 295, 07042 (2024).

[4] Hirotoshi Hirai, "Practical application of quantum neural network to materials informatics", Scientific Reports 14 1, 8583 (2024).

[5] Yuchen Guo and Shuo Yang, "Noise effects on purity and quantum entanglement in terms of physical implementability", npj Quantum Information 9 1, 11 (2023).

[6] Francesco Scala, Andrea Ceschini, Massimo Panella, and Dario Gerace, "A General Approach to Dropout in Quantum Neural Networks", Advanced Quantum Technologies 2300220 (2023).

[7] Azar C. Nakhl, Thomas Quella, and Muhammad Usman, "Calibrating the role of entanglement in variational quantum circuits", Physical Review A 109 3, 032413 (2024).

[8] Han Qi, Sihui Xiao, Zhuo Liu, Changqing Gong, and Abdullah Gani, "Variational quantum algorithms: fundamental concepts, applications and challenges", Quantum Information Processing 23 6, 224 (2024).

[9] Matteo Capone, Marco Romanelli, Davide Castaldo, Giovanni Parolin, Alessandro Bello, Gabriel Gil, and Mirko Vanzan, "A Vision for the Future of Multiscale Modeling", ACS Physical Chemistry Au 4 3, 202 (2024).

[10] Dirk Heimann, Gunnar Schönhoff, Elie Mounzer, Hans Hohenfeld, and Frank Kirchner, "Learning capability of parametrized quantum circuits", arXiv:2209.10345, (2022).

The above citations are from Crossref's cited-by service (last updated successfully 2024-06-21 20:04:18) and SAO/NASA ADS (last updated successfully 2024-06-21 20:04:19). The list may be incomplete as not all publishers provide suitable and complete citation data.