Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning

Francesco Preti1,2, Michael Schilling1,2, Sofiene Jerbi3,4, Lea M. Trenkwalder3, Hendrik Poulsen Nautrup3, Felix Motzoi1,2, and Hans J. Briegel3

1Forschungszentrum Jülich, Institute of Quantum Control (PGI-8), D-52425 Jülich, Germany
2University of Cologne, Institute of Theoretical Physics, , D-50937 Köln, Germany
3University of Innsbruck, Institute for Theoretical Physics, A-6020 Innsbruck, Austria
4Freie Universität Berlin, Dahlem Center for Complex Quantum Systems, D-14195 Berlin, Germany.

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.


Shortening quantum circuits is crucial to reducing the destructive effect of environmental decoherence and enabling useful algorithms. Here, we demonstrate an improvement in such compilation tasks via a combination of using hybrid discrete-continuous optimization across a continuous gate set, and architecture-tailored implementation. The continuous parameters are discovered with a gradient-based optimization algorithm, while in tandem the optimal gate orderings are learned via a deep reinforcement learning algorithm, based on projective simulation. To test this approach, we introduce a framework to simulate collective gates in trapped-ion systems efficiently on a classical device. The algorithm proves able to significantly reduce the size of relevant quantum circuits for trapped-ion computing. Furthermore, we show that our framework can also be applied to an experimental setup whose goal is to reproduce an unknown unitary process.

Whenever we run an algorithm on a classical processor, the abstract programming language that describes our program needs to be compiled in machine language, i.e., instructions that are simple enought to be executed on the logical circuits of the processor. The same is true in quantum computers, where the algorithm, represented by an arbitrary unitary operation, needs to be decomposed in a series of physical operations (laser pulses, microwave pulses, etc.). These simpler unitary operations should be powerful enough together to represent any unitary, in order for the quantum computer to be able to implement any potential quantum algorithm. In this work, we construct a compiler for potentially black-box unitaries based on a bi-level optimization scheme that combines gradient-based optimization for the continuous parameters of the gates and a reinforcement learning scheme for the discrete optimization of the different gate elements. We focus in particular on one of the gate sets usually implemented in trapped-ion quantum computing and develop an anaytic representation of such gate set that allows us not only to calculate analytical gradients with respect to the gate parameters, but also to speeds up our simulations considerably. This enables us to train the reinforcement learning agents for a longer time. We test our method on different compilation tasks that are relevant for trapped-ion computing and observe that it can significantly reduce the size of the circuits required to compile such unitaries.

► BibTeX data

► References

[1] F. Arute, K. Arya, R. Babbush, et al., Quantum supremacy using a programmable superconducting processor, Nature 574, 505 (2019).

[2] J. Preskill, Quantum Computing in the NISQ era and beyond, Quantum 2, 79 (2018).

[3] C. Bravo-Prieto, R. LaRose, M. Cerezo, et al., Variational Quantum Linear Solver, Quantum 7, 1188 (2023).

[4] A. Peruzzo, J. McClean, P. Shadbolt, et al., A variational eigenvalue solver on a photonic quantum processor, Nature Communications 5, 4213 (2014).

[5] M. Cerezo, A. Arrasmith, R. Babbush, et al., Variational quantum algorithms, Nature Reviews Physics 3, 625 (2021a).

[6] F. Kreppel, C. Melzer, D. Olvera Millán, et al., Quantum Circuit Compiler for a Shuttling-Based Trapped-Ion Quantum Computer, Quantum 7, 1176 (2023).

[7] E. Younis, C. C. Iancu, W. Lavrijsen, et al., Berkeley quantum synthesis toolkit (bqskit) v1 (2021).

[8] Qiskit contributors, Qiskit: An open-source framework for quantum computing (2023).

[9] D. Venturelli, M. Do, B. O'Gorman, et al., Quantum circuit compilation: An emerging application for automated reasoning, in Scheduling and Planning Applications Workshop (2019).

[10] R. Wille, S. Hillmich, and L. Burgholzer, Efficient and correct compilation of quantum circuits, in 2020 IEEE International Symposium on Circuits and Systems (ISCAS) (2020) pp. 1–5.

[11] M. Maronese, L. Moro, L. Rocutto, and E. Prati, Quantum compiling, preprint at arXiv:2112.00187 (2021).

[12] A. Sørensen and K. Mølmer, Quantum computation with ions in thermal motion, Physical Review Letters 82, 1971–1974 (1999).

[13] K. Mølmer and A. Sørensen, Multiparticle entanglement of hot trapped ions, Phys. Rev. Lett. 82, 1835 (1999).

[14] M. Malekakhlagh, How the Cross Resonance Gate Works, https:/​/​thequantumaviary.blogspot.com/​2021/​07/​how-cross-resonance-gate-works.html (2021), [Online; accessed 06-January-2022].

[15] P.-L. Dallaire-Demers and F. K. Wilhelm, Quantum gates and architecture for the quantum simulation of the fermi-hubbard model, Phys. Rev. A 94, 062304 (2016).

[16] F. Preti, T. Calarco, and F. Motzoi, Continuous quantum gate sets and pulse-class meta-optimization, PRX Quantum 3, 040311 (2022).

[17] M. S. Daoud, M. Shehab, H. M. Al-Mimi, et al., Gradient-based optimizer (gbo): A review, theory, variants, and applications, Archives of Computational Methods in Engineering (2022).

[18] D. Simon, Evolutionary optimization algorithms (Wiley-Blackwell, Hoboken, NJ, 2013).

[19] J. A. Nelder and R. Mead, A Simplex Method for Function Minimization, The Computer Journal 7, 308 (1965).

[20] C. M. Dawson and M. A. Nielsen, The solovay-kitaev algorithm, Quantum Info. Comput. 6, 81–95 (2006).

[21] F. Vatan and C. Williams, Optimal quantum circuits for general two-qubit gates, Phys. Rev. A 69, 032315 (2004).

[22] R. Wille, O. Keszocze, M. Walter, et al., Look-ahead schemes for nearest neighbor optimization of 1d and 2d quantum circuits, in 2016 21st Asia and South Pacific Design Automation Conference (ASP-DAC) (2016) pp. 292–297.

[23] M. G. Davis, E. Smith, A. Tudor, et al., Towards optimal topology aware quantum circuit synthesis, in 2020 IEEE International Conference on Quantum Computing and Engineering (QCE) (2020) pp. 223–234.

[24] L. Moro, M. G. A. Paris, M. Restelli, and E. Prati, Quantum compiling by deep reinforcement learning, Communications Physics 4, 178 (2021).

[25] T. Fösel, M. Y. Niu, F. Marquardt, and L. Li, Quantum circuit optimization with deep reinforcement learning, preprint at arXiv:2103.07585 (2021).

[26] M. Ostaszewski, L. M. Trenkwalder, W. Masarczyk, et al., Reinforcement learning for optimization of variational quantum circuit architectures, in Advances in Neural Information Processing Systems, Vol. 34, edited by M. Ranzato, A. Beygelzimer, Y. Dauphin, et al. (Curran Associates, Inc., 2021) pp. 18182–18194.

[27] V. Sivak, A. Eickbusch, H. Liu, et al., Model-free quantum control with reinforcement learning, 12 (2022).

[28] S. Borah, B. Sarma, M. Kewming, et al., Measurement-based feedback quantum control with deep reinforcement learning for a double-well nonlinear potential, Phys. Rev. Lett. 127, 190403 (2021).

[29] Y.-H. Zhang, P.-L. Zheng, Y. Zhang, and D.-L. Deng, Topological quantum compiling with reinforcement learning, Phys. Rev. Lett. 125, 170501 (2020).

[30] J. Yao, L. Lin, and M. Bukov, Reinforcement learning for many-body ground-state preparation inspired by counterdiabatic driving, Phys. Rev. X 11, 031070 (2021).

[31] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. (The MIT Press, 2018).

[32] D. Silver, J. Schrittwieser, K. Simonyan, et al., Mastering the game of go without human knowledge, Nature 550, 354 (2017).

[33] J. Jumper, R. Evans, A. Pritzel, et al., Highly accurate protein structure prediction with alphafold, Nature 596, 583 (2021).

[34] H. J. Briegel and G. De las Cuevas, Projective simulation for artificial intelligence, Scientific Reports 2, 400 (2012).

[35] A. A. Melnikov, A. Makmal, V. Dunjko, and H. J. Briegel, Projective simulation with generalization, Scientific Reports 7, 14430 (2017).

[36] A. A. Melnikov, H. Poulsen Nautrup, M. Krenn, et al., Active learning machine learns to create new quantum experiments, Proceedings of the National Academy of Sciences 115, 1221 (2018).

[37] J. Wallnöfer, A. A. Melnikov, W. Dür, and H. J. Briegel, Machine learning for long-distance quantum communication, PRX Quantum 1, 010301 (2020).

[38] M. Dalgaard, F. Motzoi, J. J. Sørensen, and J. Sherson, Global optimization of quantum dynamics with alphazero deep exploration, npj Quantum Information 6, 6 (2020a).

[39] H. P. Nautrup, N. Delfosse, V. Dunjko, et al., Optimizing Quantum Error Correction Codes with Reinforcement Learning, Quantum 3, 215 (2019).

[40] K. Ried, T. Müller, and H. J. Briegel, Modelling collective motion based on the principle of agency: General framework and the case of marching locusts, PLOS ONE 14, 1 (2019).

[41] S. Jerbi, L. M. Trenkwalder, H. P. Nautrup, et al., Quantum enhancements for deep reinforcement learning in large spaces, PRX Quantum 2 (2021).

[42] F. Preti, Deep projective simulation and state preparation, master thesis (2020).

[43] H. P. Nautrup, T. Metger, R. Iten, et al., Operationally meaningful representations of physical systems in neural networks, Machine Learning: Science and Technology 3, 045025 (2022).

[44] B. Eva, K. Ried, T. Müller, and H. J. Briegel, How a minimal learning agent can infer the existence of unobserved variables in a complex environment, Minds and Machines (2022).

[45] S. Kwon, A. Tomonaga, G. Lakshmi Bhai, et al., Gate-based superconducting quantum computing, Journal of Applied Physics 129, 041102 (2021).

[46] M. Saffman, Quantum computing with atomic qubits and rydberg interactions: progress and challenges, Journal of Physics B: Atomic, Molecular and Optical Physics 49, 202001 (2016).

[47] C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, Trapped-ion quantum computing: Progress and challenges, Applied Physics Reviews 6, 021314 (2019).

[48] T. P. Harty, D. T. C. Allcock, C. J. Ballance, et al., High-fidelity preparation, gates, memory, and readout of a trapped-ion quantum bit, Phys. Rev. Lett. 113, 220501 (2014).

[49] C. J. Ballance, T. P. Harty, N. M. Linke, et al., High-fidelity quantum logic gates using trapped-ion hyperfine qubits, Phys. Rev. Lett. 117, 060504 (2016).

[50] A. Erhard, J. J. Wallman, L. Postler, et al., Characterizing large-scale quantum computers via cycle benchmarking, Nature Communications 10 (2019).

[51] W. Paul, Electromagnetic traps for charged and neutral particles, Rev. Mod. Phys. 62, 531 (1990).

[52] E. A. Martinez, T. Monz, D. Nigg, et al., Compiling quantum algorithms for architectures with multi-qubit gates, New Journal of Physics 18, 063029 (2016).

[53] J. P. Gaebler, A. M. Meier, T. R. Tan, et al., Randomized benchmarking of multiqubit gates, Phys. Rev. Lett. 108, 260503 (2012).

[54] C. Monroe, R. Raussendorf, A. Ruthven, et al., Large-scale modular quantum-computer architecture with atomic memory and photonic interconnects, Phys. Rev. A 89, 022317 (2014).

[55] M. Ringbauer, M. Meth, L. Postler, et al., A universal qudit quantum processor with trapped ions, Nature Physics 18, 1053 (2022).

[56] M. A. Nielsen, A simple formula for the average gate fidelity of a quantum dynamical operation, Physics Letters A 303, 249 (2002).

[57] M. Dalgaard, F. Motzoi, and J. Sherson, Predicting quantum dynamical cost landscapes with deep learning, Phys. Rev. A 105, 012402 (2022).

[58] V. Strassen, Gaussian elimination is not optimal, Numerische Mathematik 13, 354 (1969).

[59] N. Khaneja, T. Reiss, C. Kehlet, et al., Optimal control of coupled spin dynamics: design of nmr pulse sequences by gradient ascent algorithms, Journal of Magnetic Resonance 172, 296 (2005).

[60] M. Dalgaard, F. Motzoi, J. H. M. Jensen, and J. Sherson, Hessian-based optimization of constrained quantum control, Phys. Rev. A 102, 042612 (2020b).

[61] A. Morningstar, M. Hauru, J. Beall, et al., Simulation of quantum many-body dynamics with tensor processing units: Floquet prethermalization, PRX Quantum 3 (2022).

[62] I. L. Chuang and M. A. Nielsen, Prescription for experimental determination of the dynamics of a quantum black box, Journal of Modern Optics 44, 2455 (1997).

[63] V. Bužek and M. Hillery, Quantum copying: Beyond the no-cloning theorem, Phys. Rev. A 54, 1844 (1996).

[64] S. Khatri, R. LaRose, A. Poremba, et al., Quantum-assisted quantum compiling, Quantum 3, 140 (2019).

[65] J. R. McClean, S. Boixo, V. N. Smelyanskiy, et al., Barren plateaus in quantum neural network training landscapes, Nature Communications 9, 4812 (2018).

[66] M. Cerezo, A. Sone, T. Volkoff, et al., Cost function dependent barren plateaus in shallow parametrized quantum circuits, Nature Communications 12, 1791 (2021b).

[67] S. Jerbi, J. Gibbs, M. S. Rudolph, et al., The power and limitations of learning quantum dynamics incoherently, preprint at arXiv:2303.12834 (2023).

[68] O. Kyriienko and V. E. Elfving, Generalized quantum circuit differentiation rules, Phys. Rev. A 104, 052417 (2021).

[69] D. Wierichs, J. Izaac, C. Wang, and C. Y.-Y. Lin, General parameter-shift rules for quantum gradients, Quantum 6, 677 (2022).

[70] L. Bittel, J. Watty, and M. Kliesch, Fast gradient estimation for variational quantum algorithms, preprint at arXiv:2210.06484 (2022).

[71] M. Ostaszewski, E. Grant, and M. Benedetti, Structure optimization for parameterized quantum circuits, Quantum 5, 391 (2021b).

[72] N. Mazyavkina, S. Sviridov, S. Ivanov, and E. Burnaev, Reinforcement learning for combinatorial optimization: A survey, Computers & Operations Research 134, 105400 (2021).

[73] Y. Li, Deep reinforcement learning: An overview, preprint at arXiv:1701.07274 (2017).

[74] W. L. Boyajian, J. Clausen, L. M. Trenkwalder, et al., On the convergence of projective-simulation–based reinforcement learning in markov decision processes, Quantum Machine Intelligence 2, 13 (2020).

[75] J. Mautner, A. Makmal, D. Manzano, et al., Projective simulation for classical learning agents: A comprehensive investigation, New Generation Computing 33, 69 (2015).

[76] V. Mnih, K. Kavukcuoglu, D. Silver, et al., Human-level control through deep reinforcement learning, Nature 518, 529 (2015).

[77] S. Hochreiter, Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis (1991).

[78] S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Computation 9, 1735 (1997).

[79] D. P. DiVincenzo, Two-bit gates are universal for quantum computation, Phys. Rev. A 51, 1015 (1995).

[80] M. Bukov, A. G. Day, D. Sels, et al., Reinforcement learning in different phases of quantum control, Physical Review X 8 (2018).

[81] D. C. Liu and J. Nocedal, On the limited memory bfgs method for large scale optimization, Math. Program. 45, 503 (1989).

[82] T. Toffoli, Reversible computing, in Automata, Languages and Programming, edited by J. de Bakker and J. van Leeuwen (Springer Berlin Heidelberg, Berlin, Heidelberg, 1980) pp. 632–644.

[83] T. Monz, K. Kim, W. Hänsel, et al., Realization of the quantum toffoli gate with trapped ions, Phys. Rev. Lett. 102, 040501 (2009).

[84] A. G. Taube and R. J. Bartlett, New perspectives on unitary coupled-cluster theory, International Journal of Quantum Chemistry 106, 3393 (2006).

[85] R. J. Bartlett and M. Musiał, Coupled-cluster theory in quantum chemistry, Rev. Mod. Phys. 79, 291 (2007).

[86] F. Motzoi, M. P. Kaicher, and F. K. Wilhelm, Linear and logarithmic time compositions of quantum many-body operators, Phys. Rev. Lett. 119, 160503 (2017).

[87] M. Takahashi, Thermodynamics of One-Dimensional Solvable Models (Cambridge University Press, 1999).

[88] S. K. Lam, A. Pitrou, and S. Seibert, Numba: A llvm-based python jit compiler, in Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC (2015) pp. 1–6.

[89] J. Bradbury, R. Frostig, P. Hawkins, et al., JAX: composable transformations of Python+NumPy programs (2018).

[90] A. Paszke, S. Gross, F. Massa, et al., Pytorch: An imperative style, high-performance deep learning library, in Advances in Neural Information Processing Systems 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, et al. (Curran Associates, Inc., 2019) pp. 8024–8035.

[91] P. Virtanen, R. Gommers, T. E. Oliphant, et al., SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Methods 17, 261 (2020).

[92] J. Li, X. Yang, X. Peng, and C.-P. Sun, Hybrid quantum-classical approach to quantum optimal control, Phys. Rev. Lett. 118, 150503 (2017).

[93] N. Barhate, Minimal pytorch implementation of proximal policy optimization, code: https:/​/​github.com/​nikhilbarhate99/​PPO-PyTorch (2021).

[94] P. E. Hart, N. J. Nilsson, and B. Raphael, A formal basis for the heuristic determination of minimum cost paths, IEEE Transactions on Systems Science and Cybernetics 4, 100 (1968).

[95] A. Hodler and M. Needham, Graph algorithms (O'Reilly Media, Sebastopol, CA, 2019).

Cited by

[1] Florian Fürrutter, Gorka Muñoz-Gil, and Hans J. Briegel, "Quantum circuit synthesis with diffusion models", Nature Machine Intelligence 6 5, 515 (2024).

[2] Remmy Zen, Jan Olle, Luis Colmenarez, Matteo Puviani, Markus Müller, and Florian Marquardt, "Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning", arXiv:2402.17761, (2024).

[3] Florian Fürrutter, Gorka Muñoz-Gil, and Hans J. Briegel, "Quantum circuit synthesis with diffusion models", arXiv:2311.02041, (2023).

[4] Sebastian Rietsch, Abhishek Y. Dubey, Christian Ufrecht, Maniraman Periyasamy, Axel Plinge, Christopher Mutschler, and Daniel D. Scherer, "Unitary Synthesis of Clifford+T Circuits with Reinforcement Learning", arXiv:2404.14865, (2024).

[5] Francesco Preti and József Zsolt Bernád, "Statistical evaluation and optimization of entanglement purification protocols", arXiv:2402.12287, (2024).

The above citations are from Crossref's cited-by service (last updated successfully 2024-05-26 11:21:28) and SAO/NASA ADS (last updated successfully 2024-05-26 11:21:28). The list may be incomplete as not all publishers provide suitable and complete citation data.