Reinforcement Learning Based Quantum Circuit Optimization via ZX-Calculus
1Qilimanjaro Quantum Tech, Carrer de Veneçuela, 74, Sant Martí, 08019, Barcelona, Spain
2Universitat Politècnica de Catalunya, Carrer de Jordi Girona, 3, 08034 Barcelona, Spain
3Barcelona Supercomputing Center, Plaça Eusebi Güell, 1-3, 08034 Barcelona, Spain
| Published: | 2025-05-28, volume 9, page 1758 |
| Editor: | John van de Wetering |
| Eprint: | arXiv:2312.11597v4 |
| Doi: | https://doi.org/10.22331/q-2025-05-28-1758 |
| Citation: | Quantum 9, 1758 (2025). |
Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.
Abstract
We propose a novel Reinforcement Learning (RL) method for optimizing quantum circuits using graph-theoretic simplification rules of ZX-diagrams. The agent, trained using the Proximal Policy Optimization (PPO) algorithm, employs Graph Neural Networks to approximate the policy and value functions. We demonstrate the capacity of our approach by comparing it against the best performing ZX-Calculus-based algorithm for the problem in hand. After training on small Clifford+T circuits of 5-qubits and few tenths of gates, the agent consistently improves the state-of-the-art for this type of circuits, for at least up to 80-qubit and 2100 gates, whilst remaining competitive in terms of computational performance. Additionally, we illustrate the versatility of the agent by incorporating additional optimization routines on the workflow during training, improving the two-qubit gate count state-of-the-art on multiple structured quantum circuits for relevant applications of much larger dimension and different gate distributions than the circuits the agent trains on. This conveys the potential of tailoring the reward function to the specific characteristics of each application and hardware backend. Our approach is a valuable tool for the implementation of quantum algorithms in the near-term intermediate-scale range (NISQ).
Featured image: Overview of the RL-ZX approach. From a random quantum circuit, the equivalent ZX-diagram in graph-like form is obtained. The agent uses its learned policy to choose between several graph transformations to be applied to the ZX-diagram. After the action is applied, the environment returns the resulting diagram to the agent as an observation, as well as its corresponding reward that is obtained after extracting the new circuit. The process is repeated until the agent decides to terminate the episode or no actions are available.
Popular summary
This paper teaches a reinforcement-learning agent to perform those diagrammatic simplifications automatically. The agent:
1. Sees a circuit as a graph (nodes and edges) and uses a graph neural network to understand its structure.
2. Learns via a popular RL algorithm (PPO) which rewrites actually lead to the biggest savings.
3. Practices on small 5-qubit circuits, then scales up its moves to much larger ones—up to 80 qubits and a couple of thousand gates—outperforming the best hand-coded ZX-calculus methods for the training rule sets.
4. Adapts its “rewards” to focus on different goals (e.g., minimizing two-qubit gates for a particular hardware), showing it can be fine-tuned for varied use cases.
The RL agent not only generalizes to circuits far larger than its training set but also delivers concrete gate-count reductions (up to ~10 %) on key quantum subroutines when used in combination with other powerful ZX-based approaches, making it a useful hardware-aware optimization tool for NISQ devices.
► BibTeX data
► References
[1] John Preskill. ``Quantum computing in the NISQ era and beyond''. Quantum 2, 79 (2018).
https://doi.org/10.22331/q-2018-08-06-79
[2] Beatrice Nash, Vlad Gheorghiu, and Michele Mosca. ``Quantum circuit optimizations for nisq architectures''. Quantum Science and Technology 5, 025010 (2020).
https://doi.org/10.1088/2058-9565/ab79b1
[3] Scott Aaronson and Daniel Gottesman. ``Improved simulation of stabilizer circuits''. Phys. Rev. A 70, 052328 (2004).
https://doi.org/10.1103/PhysRevA.70.052328
[4] Vadym Kliuchnikov and Dmitri Maslov. ``Optimization of clifford circuits''. Phys. Rev. A 88, 052307 (2013).
https://doi.org/10.1103/PhysRevA.88.052307
[5] Richard S. Sutton and Andrew G. Barto. ``Reinforcement learning: An introduction''. The MIT Press. (2018). Second edition. url: http://incompleteideas.net/book/the-book-2nd.html.
http://incompleteideas.net/book/the-book-2nd.html
[6] Thomas Fösel, Murphy Yuezhen Niu, Florian Marquardt, and Li Li. ``Quantum circuit optimization with deep reinforcement learning'' (2021). arXiv:2103.07585.
arXiv:2103.07585
[7] Zikun Li, Jinjun Peng, Yixuan Mei, Sina Lin, Yi Wu, Oded Padon, and Zhihao Jia. ``Quarl: A learning-based quantum circuit optimizer'' (2023). arXiv:2307.10120.
arXiv:2307.10120
[8] Bob Coecke and Ross Duncan. ``Interacting quantum observables: categorical algebra and diagrammatics''. New Journal of Physics 13, 043016 (2011).
https://doi.org/10.1088/1367-2630/13/4/043016
[9] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. ``Proximal policy optimization algorithms'' (2017). arXiv:1707.06347.
arXiv:1707.06347
[10] Jan Nogué. ``Reinforcement Learning based Circuit Compilation via ZX-calculus''. Master's thesis. Universitat de Barcelona. (2023). url: https://diposit.ub.edu/dspace/bitstream/2445/202911/1/Memoria_TFM-JanNogue.pdf.
https://diposit.ub.edu/dspace/bitstream/2445/202911/1/Memoria_TFM-JanNogue.pdf
[11] Daniel Gottesman. ``Theory of fault-tolerant quantum computation''. Phys. Rev. A 57, 127–137 (1998).
https://doi.org/10.1103/PhysRevA.57.127
[12] Daniel Gottesman. ``The heisenberg representation of quantum computers'' (1998). arXiv:quant-ph/9807006.
arXiv:quant-ph/9807006
[13] Aleks Kissinger and John van de Wetering. ``Pyzx: Large scale automated diagrammatic reasoning''. Electronic Proceedings in Theoretical Computer Science 318, 229–241 (2020).
https://doi.org/10.4204/eptcs.318.14
[14] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. ``A comprehensive survey on graph neural networks''. IEEE Transactions on Neural Networks and Learning Systems 32, 4–24 (2021).
https://doi.org/10.1109/tnnls.2020.2978386
[15] Bob Coecke and Aleks Kissinger. ``Picturing quantum processes: A first course in quantum theory and diagrammatic reasoning''. Cambridge University Press. (2017).
https://doi.org/10.1017/9781316219317
[16] John van de Wetering. ``Zx-calculus for the working quantum computer scientist'' (2020). arXiv:2012.13966.
arXiv:2012.13966
[17] Ross Duncan, Aleks Kissinger, Simon Perdrix, and John van de Wetering. ``Graph-theoretic simplification of quantum circuits with the ZX-calculus''. Quantum 4, 279 (2020).
https://doi.org/10.22331/q-2020-06-04-279
[18] Niel de Beaudrap, Aleks Kissinger, and John van de Wetering. ``Circuit extraction for zx-diagrams can be p-hard''. In Schloss Dagstuhl – Leibniz-Zentrum für Informatik. (2022). arXiv:2202.09194.
https://doi.org/10.4230/LIPIcs.ICALP.2022.119
arXiv:2202.09194
[19] Calum Holker. ``Causal flow preserving optimisation of quantum circuits in the zx-calculus'' (2024). arXiv:2312.02793.
arXiv:2312.02793
[20] Korbinian Staudacher, Tobias Guggemos, Wolfgang Gehrke, and Sophia Grundner-Culemann. ``Reducing 2-qubit gate count for zx-calculus based quantum circuit optimization''. In Quantum Processing and Languages (QPL22). Pages 1–17. (2022). url: https://elib.dlr.de/188470/.
https://elib.dlr.de/188470/
[21] Anton Kotzig. ``Eulerian lines in finite 4-valent graphs and their transformations''. In Colloqium on Graph Theory Tihany 1966Pages pages 219 – 230 (Academic Press, 1968).
[22] André Bouchet. ``Graphic presentations of isotropic systems''. J. Comb. Theory Ser. A 45, 58–76 (1987).
[23] Daniel E Browne, Elham Kashefi, Mehdi Mhalla, and Simon Perdrix. ``Generalized flow and determinism in measurement-based quantum computation''. New Journal of Physics 9, 250 (2007).
https://doi.org/10.1088/1367-2630/9/8/250
[24] Robert Raussendorf, Daniel Browne, and Hans Briegel. ``The one-way quantum computer–a non-network model of quantum computation''. Journal of Modern Optics 49, 1299–1306 (2002).
https://doi.org/10.1080/09500340110107487
[25] Miriam Backens, Hector Miller-Bakewell, Giovanni de Felice, Leo Lobski, and John van de Wetering. ``There and back again: A circuit extraction tale''. Quantum 5, 421 (2021).
https://doi.org/10.22331/q-2021-03-25-421
[26] Aleks Kissinger and John van de Wetering. ``Reducing the number of non-clifford gates in quantum circuits''. Phys. Rev. A 102, 022406 (2020).
https://doi.org/10.1103/PhysRevA.102.022406
[27] Vincent Danos and Elham Kashefi. ``Determinism in the one-way model''. Phys. Rev. A 74, 052310 (2006).
https://doi.org/10.1103/PhysRevA.74.052310
[28] Mehdi Mhalla and Simon Perdrix. ``Finding optimal flows efficiently''. Page 857–868. Springer Berlin Heidelberg. (2008).
https://doi.org/10.1007/978-3-540-70575-8_70
[29] John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. ``High-dimensional continuous control using generalized advantage estimation'' (2018). arXiv:1506.02438.
arXiv:1506.02438
[30] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. ``Graph attention networks'' (2018). arXiv:1710.10903.
arXiv:1710.10903
[31] Shaked Brody, Uri Alon, and Eran Yahav. ``How attentive are graph attention networks?'' (2022). arXiv:2105.14491.
arXiv:2105.14491
[32] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. ``Pytorch: An imperative style, high-performance deep learning library'' (2019). arXiv:1912.01703.
arXiv:1912.01703
[33] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. ``Openai gym'' (2016) arXiv:1606.01540.
arXiv:1606.01540
[34] Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. ``Graph matching networks for learning the similarity of graph structured objects''. In International conference on machine learning. Pages 3835–3845. PMLR (2019).
https://doi.org/10.48550/arXiv.1904.12787
[35] Neil J Ross. code: njross/optimizer.
https://github.com/njross/optimizer
[36] Yunseong Nam, Neil J. Ross, Yuan Su, Andrew M. Childs, and Dmitri Maslov. ``Automated optimization of large quantum circuits with continuous parameters''. npj Quantum Information 4 (2018).
https://doi.org/10.1038/s41534-018-0072-4
[37] Palak Chawla, Shweta, K. R. Swain, Tushti Patel, Renu Bala, Disha Shetty, Kenji Sugisaki, Sudhindu Bikash Mandal, Jordi Riu, Jan Nogué, V. S. Prasannaa, and B. P. Das. ``Relativistic variational-quantum-eigensolver calculations of molecular electric dipole moments on quantum hardware''. Phys. Rev. A 111, 022817 (2025).
https://doi.org/10.1103/PhysRevA.111.022817
[38] Palak Chawla, Disha Shetty, Peniel Bertrand Tsemo, Kenji Sugisaki, Jordi Riu, Jan Nogue, Debashis Mukherjee, and V. S. Prasannaa. ``Vqe calculations on a nisq era trapped ion quantum computer using a multireference unitary coupled cluster ansatz: application to the beh$_2$ insertion problem'' (2025). arXiv:2504.07037.
arXiv:2504.07037
[39] Aashna Anil Zade, Kenji Sugisaki, Matthias Werner, Ana Palacios, Jordi Riu, Jan Nogue, Artur Garcia-Saez, Arnau Riera, and V. S. Prasannaa. ``Capturing strong correlation effects on a quantum annealer: calculation of avoided crossing in the h$_4$ molecule using the quantum annealer eigensolver'' (2025). arXiv:2412.20464.
arXiv:2412.20464
[40] Maximilian Nägele and Florian Marquardt. ``Optimizing zx-diagrams with deep reinforcement learning'' (2023). arXiv:2311.18588.
https://doi.org/10.1088/2632-2153/ad76f7
arXiv:2311.18588
[41] Francois Charton, Alexandre Krajenbrink, Konstantinos Meichanetzidis, and Richie Yeung. ``Teaching small transformers to rewrite ZX diagrams''. In The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS'23. (2023). url: https://openreview.net/forum?id=btQ7Bt1NLF.
https://openreview.net/forum?id=btQ7Bt1NLF
[42] Jordi Riu and Jan Nogué. ``Code for quantum circuit optimization with rl via zx-calculus''. Github Repository (2023). url: https://github.com/qilimanjaro-tech/Circopt-RL-ZXCalc.
https://github.com/qilimanjaro-tech/Circopt-RL-ZXCalc
Cited by
[1] Keyu Xiong, Tao Shang, Chenyi Zhang, and Yuchen Liu, "ZX-Diagram T-Count Reduction Based on Proximal Policy Optimization", Intelligent Computing 5, 0418 (2026).
[2] Einat Buznach Ahituv, Debanjan Chowdhury, and Jonathan Ruhman, "Unveiling a Hidden Percolation Transition in Monitored Clifford Circuits: Inroads from ZX Calculus", Physical Review Letters 135 5, 050402 (2025).
[3] Remmy Zen, Maximilian Nägele, and Florian Marquardt, "Reusability report: Optimizing T count in general quantum circuits with AlphaTensor-Quantum", Nature Machine Intelligence 8 1, 113 (2025).
[4] Elham Torabian and Roman V Krems, "Molecular representations of quantum circuits for quantum machine learning", Machine Learning: Science and Technology 7 2, 025009 (2026).
[5] Disha Shetty, Supriyo Dutta, Palak Chawla, Akshaya Jayashankar, Jordi Riu, Jan Nogué, K. Sugisaki, and V. S. Prasannaa, "Assessing advantage from quantum linear solvers for graph-based problems", Physical Review A 113 5, 052402 (2026).
[6] P Ashok, Ravi Gorli, S Lakshmi Sridevi, and Harishchander Anandaram, Reinforcement Learning 295 (2026) ISBN:9783111633596.
[7] Aashna Anil Zade, Kenji Sugisaki, Matthias Werner, Ana Palacios, Jordi Riu, Jan Nogué, Artur Garcia-Saez, Arnau Riera, and V. S. Prasannaa, "Quantum annealing eigensolver as a NISQ era tool for probing strong correlation effects in quantum chemistry", The European Physical Journal Plus 140 9, 930 (2025).
[8] Kriti Sharma and Adarsh Kumar, 2025 2nd International Conference On Multidisciplinary Research and Innovations in Engineering (MRIE) 128 (2025) ISBN:979-8-3315-8673-7.
[9] Laura Baird and Armin Moin, 2025 IEEE International Conference on Quantum Computing and Engineering (QCE) 426 (2025) ISBN:979-8-3315-5736-2.
[10] Yuxin Huang, Benjamin Grossman-Ponemon, and David Hyde, "Automated Synthesis of Quantum Algorithms via Classical Numerical Techniques", ACM Transactions on Quantum Computing 6 4, 1 (2025).
[11] Palak Chawla, Disha Shetty, Peniel Bertrand Tsemo, Kenji Sugisaki, Jordi Riu, Jan Nogué, Debashis Mukherjee, and V. S. Prasannaa, "Trapped ion quantum hardware demonstration of energy calculations using a multireference unitary coupled cluster ansatz: application to the BeH$$_2$$ insertion problem", The European Physical Journal Plus 140 9, 924 (2025).
[12] Junxia YANG, Qizhi CAI, Jinrong GUO, and Guangwei DENG, "Quantum artificial intelligence: a review of the bidirectional empowerment mechanisms and frontier progress in AI and quantum computing", Acta Physica Sinica 75 10(2026).
[13] Ge Yan, Wenjie Wu, Yuheng Chen, Kaisen Pan, Xudong Lu, Zixiang Zhou, Yuhan Wang, Ruocheng Wang, and Junchi Yan, "Quantum Circuit Synthesis and Compilation Optimization: Overview and Prospects", arXiv:2407.00736, (2024).
[14] Daniele Lizzio Bosco, Lukasz Cincio, Giuseppe Serra, and M. Cerezo, "Quantum Circuit Pre-Synthesis: Learning Local Edits to Reduce $T$-count", arXiv:2601.19738, (2026).
[15] Palak Chawla, Shweta, K. R. Swain, Tushti Patel, Renu Bala, Disha Shetty, Kenji Sugisaki, Sudhindu Bikash Mandal, Jordi Riu, Jan Nogué, V. S. Prasannaa, and B. P. Das, "Relativistic variational-quantum-eigensolver calculations of molecular electric dipole moments on quantum hardware", Physical Review A 111 2, 022817 (2025).
[16] Maximilian Nägele, Jan Olle, Thomas Fösel, Remmy Zen, and Florian Marquardt, "Tackling Decision Processes with Non-Cumulative Objectives using Reinforcement Learning", arXiv:2405.13609, (2024).
[17] Benjamin Rodatz, Boldizsár Poór, and Aleks Kissinger, "Fault Tolerance by Construction", arXiv:2506.17181, (2025).
[18] Marin Bukov and Florian Marquardt, "Reinforcement Learning for Quantum Technology", arXiv:2601.18953, (2026).
[19] Krishnageetha Karuppasamy, Varun Puram, Stevens Johnson, and Johnson P Thomas, "A Comprehensive Review of Quantum Circuit Optimization: Current Trends and Future Directions", arXiv:2408.08941, (2024).
[20] Alexander Mattick, Maniraman Periyasamy, Christian Ufrecht, Abhishek Y. Dubey, Christopher Mutschler, Axel Plinge, and Daniel D. Scherer, "Optimizing Quantum Circuits via ZX Diagrams using Reinforcement Learning and Graph Neural Networks", arXiv:2504.03429, (2025).
[21] Laxmisha Ashok Attisara and Sathish Kumar, "Reinforcement Learning for Optimizing Large Qubit Array based Quantum Sensor Circuits", arXiv:2508.21253, (2025).
[22] Remmy Zen, Maximilian Nägele, and Florian Marquardt, "Reusability Report: Optimizing T-count in General Quantum Circuits with AlphaTensor-Quantum", arXiv:2511.09951, (2025).
[23] Laxmisha Ashok Attisara and Sathish Kumar, "Quantum Machine Learning for Optimizing Entanglement Distribution in Quantum Sensor Circuits", arXiv:2508.21252, (2025).
[24] Ethan Decker, Christopher Watson, Junyu Zhou, Yuhao Liu, Chenxu Liu, Ang Li, Gushu Li, and Samuel Stein, "F2: Offline Reinforcement Learning for Hamiltonian Simulation via Free-Fermionic Subroutine Compilation", arXiv:2512.08023, (2025).
[25] Daniel Mills, Ifan Williams, Jacob Swain, Gabriel Matos, Enrico Rinaldi, and Alexander Koziell-Pipe, "Reinforcement Learning for Adaptive Composition of Quantum Circuit Optimisation Passes", arXiv:2601.21629, (2026).
[26] Richie Yeung, Aleks Kissinger, and Rob Cornish, "Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis", arXiv:2605.10910, (2026).
The above citations are from Crossref's cited-by service (last updated successfully 2026-08-09 04:49:37) and SAO/NASA ADS (last updated successfully 2026-08-08 16:23:28). The list may be incomplete as not all publishers provide suitable and complete citation data.
Could not fetch ADS cited-by data during last attempt 2026-08-09 04:49:37: Cannot retrieve data from ADS due to rate limitations.
This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.