Optimizing Quantum Error Correction Codes with Reinforcement Learning
1Institute for Theoretical Physics, University of Innsbruck, Technikerstr. 21a, A-6020 Innsbruck, Austria
2Station Q Quantum Architectures and Computation Group, Microsoft Research, Redmond, WA 98052, USA
3LIACS, Leiden University, Niels Bohrweg 1, 2333 CA Leiden, The Netherlands
4Department of Philosophy, University of Konstanz, Konstanz 78457, Germany
5Institute for Quantum Optics and Quantum Information, Austrian Academy of Sciences, Boltzmanngasse 3, 1090 Vienna, Austria
| Published: | 2019-12-16, volume 3, page 215 |
| Eprint: | arXiv:1812.08451v5 |
| Doi: | https://doi.org/10.22331/q-2019-12-16-215 |
| Citation: | Quantum 3, 215 (2019). |
Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.
Abstract
Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached. Using efficient simulations with about 70 data qubits with arbitrary connectivity, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of data qubits, for various error models of interest. Moreover, we show that agents trained on one setting are able to successfully transfer their experience to different settings. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization from off-line simulations to on-line laboratory settings.
Popular summary
We develop an approach to quantum error correction where a machine learning algorithm (or learning agent) learns to design good error correction tools (called codes) that use as few basic building elements (qubits) as possible. We provide extensive computer simulations of this method for various realistic situations with qubit numbers soon available in state-of-the art laboratories. Our results suggest that a learning agent can not only find near-optimal solutions for a variety of problems, but is also able to transfer its experience from one situation to another. This feature is particularly valuable because it facilitates pre-training learning agents on cheap simulations before deployment to the actual, expensive device. Our work thus provides a stepping-stone for connecting quantum technologies and artificial intelligence that can be vital for future quantum devices.
► BibTeX data
► References
[1] Michael A. Nielsen and Isaac L. Chuang, Quantum Computation and Quantum Information (Cambridge University Press, Cambridge, U.K., 2000).
[2] Vedran Dunjko, Yimin Ge, and J. Ignacio Cirac, Computational Speedups Using Small Quantum Devices, Phys. Rev. Lett. 121, 250501 (2018), arXiv:1807.08970.
https://doi.org/10.1103/PhysRevLett.121.250501
arXiv:arXiv:1807.08970
[3] Earl Campbell, Ankur Khurana, and Ashley Montanaro, Applying quantum algorithms to constraint satisfaction problems, Quantum 3, 167 (2019), arXiv:1810.05582.
https://doi.org/10.22331/q-2019-07-18-167
arXiv:arXiv:1810.05582
[4] John Preskill, Fault-tolerant quantum computation, in Introduction to Quantum Computation, edited by H.-K. Lo, S. Popescu, and T. P. Spiller (World-Scientific, 1997) Chap. 8, pp. 213–269, arXiv:quant-ph/9712048.
https://doi.org/10.1142/9789812385253_0008
arXiv:arXiv:quant-ph/9712048
[5] Daniel Gottesmann, Stabilizer Codes and Quantum Error Correction, Ph.D. thesis, Caltech (1997), arXiv:quant-ph/9705052.
arXiv:quant-ph/9705052
[6] Barbara M. Terhal, Quantum error correction for quantum memories, Rev. Mod. Phys. 87, 307 (2015), arXiv:1302.3428.
https://doi.org/10.1103/RevModPhys.87.307
arXiv:arXiv:1302.3428
[7] David K. Tuckett, Stephen D. Bartlett, and Steven T. Flammia, Ultrahigh Error Threshold for Surface Codes with Biased Noise, Phys. Rev. Lett. 120, 050505 (2018), arXiv:1708.08474.
https://doi.org/10.1103/PhysRevLett.120.050505
arXiv:arXiv:1708.08474
[8] Keisuke Fujii and Yuuki Tokunaga, Error and loss tolerances of surface codes with general lattice structures, Phys. Rev. A 86, 020303(R) (2012), arXiv:1202.2743.
https://doi.org/10.1103/PhysRevA.86.020303
arXiv:arXiv:1202.2743
[9] Thomas Monz, Philipp Schindler, Julio T. Barreiro, Michael Chwalla, Daniel Nigg, William A. Coish, Maximilian Harlander, Wolfgang Hänsel, Markus Hennrich, and Rainer Blatt, 14-Qubit Entanglement: Creation and Coherence, Phys. Rev. Lett. 106, 130506 (2011), arXiv:1009.6126.
https://doi.org/10.1103/PhysRevLett.106.130506
arXiv:arXiv:1009.6126
[10] Philipp Schindler, Daniel Nigg, Thomas Monz, J. T. Barreiro, Esteban Martinez, S. X. Wang, Stephan Quint, M. F. Brandl, Volckmar Nebendahl, Christian F. Roos, Michael Chwalla, M. Hennrich, and Rainer Blatt, A quantum information processor with trapped ions, New J. Phys. 15, 123012 (2013), arXiv:1308.3096.
https://doi.org/10.1088/1367-2630/15/12/123012
arXiv:arXiv:1308.3096
[11] Vedran Dunjko and Hans J. Briegel, Machine learning & artificial intelligence in the quantum domain: a review of recent progress, Rep. Prog. Phys. 81, 074001 (2018), arXiv:1709.02779.
https://doi.org/10.1088/1361-6633/aab406
arXiv:arXiv:1709.02779
[12] Giacomo Torlai and Roger G. Melko, Neural Decoder for Topological Codes, Phys. Rev. Lett. 119, 030501 (2017), arXiv:1610.04238.
https://doi.org/10.1103/PhysRevLett.119.030501
arXiv:arXiv:1610.04238
[13] Stefan Krastanov and Liang Jiang, Deep Neural Network Probabilistic Decoder for Stabilizer Codes, Sci. Rep. 7, 11003 (2017), arXiv:1705.09334.
https://doi.org/10.1038/s41598-017-11266-1
arXiv:arXiv:1705.09334
[14] Savvas Varsamopoulos, Ben Criger, and Koen Bertels, Decoding small surface codes with feedforward neural networks, Quant. Sci. Techn. 3, 015004 (2017), arXiv:1705.00857.
https://doi.org/10.1088/2058-9565/aa955a
arXiv:arXiv:1705.00857
[15] Paul Baireuther, Thomas E. O'Brien, Brian Tarasinski, and Carlo W. J. Beenakker, Machine-learning-assisted correction of correlated qubit errors in a topological code, Quantum 2, 48 (2018), arXiv:1705.07855.
https://doi.org/10.22331/q-2018-01-29-48
arXiv:arXiv:1705.07855
[16] Nikolas P. Breuckmann and Xiaotong Ni, Scalable Neural Network Decoders for Higher Dimensional Quantum Codes, Quantum 2, 68 (2018), arXiv:1710.09489.
https://doi.org/10.22331/q-2018-05-24-68
arXiv:arXiv:1710.09489
[17] Christopher Chamberland and Pooya Ronagh, Deep neural decoders for near term fault-tolerant experiments, Quant. Sci. Techn. 3, 044002 (2018), arXiv:1802.06441.
https://doi.org/10.1088/2058-9565/aad1f7
arXiv:arXiv:1802.06441
[18] Ryan Sweke, Markus S. Kesselring, Evert P. L. van Nieuwenburg, and Jens Eisert, Reinforcement learning decoders for fault-tolerant quantum computation, (2018), arXiv:1810.07207.
arXiv:arXiv:1810.07207
[19] Paul Baireuther, M. D. Caio, B. Criger, Carlo W. J. Beenakker, and Thomas E. O'Brien, Neural network decoder for topological color codes with circuit level noise, New J. Phys. 21, 013003 (2019), arXiv:1804.02926.
https://doi.org/10.1088/1367-2630/aaf29e
arXiv:arXiv:1804.02926
[20] Xiaotong Ni, Neural network decoders for large-distance 2d toric codes, (2018), arXiv:1809.06640.
arXiv:arXiv:1809.06640
[21] Nishad Maskara, Aleksander Kubica, and Tomas Jochym-O'Connor, Advantages of versatile neural-network decoding for topological codes, Phys. Rev. A 99, 052351 (2019), arXiv:1802.08680.
https://doi.org/10.1103/PhysRevA.99.052351
arXiv:arXiv:1802.08680
[22] Ye-Hua Liu and David Poulin, Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes, Phys. Rev. Lett. 122, 200501 (2019), arXiv:1811.07835.
https://doi.org/10.1103/PhysRevLett.122.200501
arXiv:arXiv:1811.07835
[23] Amarsanaa Davaasuren, Yasunari Suzuki, Keisuke Fujii, and Masato Koashi, General framework for constructing fast and near-optimal machine-learning-based decoder of the topological stabilizer codes, (2018), arXiv:1801.04377.
arXiv:arXiv:1801.04377
[24] Philip Andreasson, Joel Johansson, Simon Liljestrand, and Mats Granath, Quantum error correction for the toric code using deep reinforcement learning, Quantum 3, 183 (2019), arXiv:1811.12338.
https://doi.org/10.22331/q-2019-09-02-183
arXiv:arXiv:1811.12338
[25] Savvas Varsamopoulos, Koen Bertels, and Carmen G. Almudever, Comparing neural network based decoders for the surface code, IEEE T. Comput. (2019a), 10.1109/TC.2019.2948612, arXiv:1811.12456.
https://doi.org/10.1109/TC.2019.2948612
arXiv:arXiv:1811.12456
[26] Savvas Varsamopoulos, Koen Bertels, and Carmen G. Almudever, Decoding surface code with a distributed neural network based decoder, (2019b), arXiv:1901.10847.
arXiv:arXiv:1901.10847
[27] Laia Domingo Colomer, Michalis Skotiniotis, and Ramon Muñoz-Tapia, Reinforcement learning for optimal error correction of toric codes, (2019), arXiv:1911.02308.
arXiv:arXiv:1911.02308
[28] Thomas Wagner, Hermann Kampermann, and Dagmar Bruß, Symmetries for a High Level Neural Decoder on the Toric Code, (2019), arXiv:1910.01662.
arXiv:arXiv:1910.01662
[29] Chaitanya Chinni, Abhishek Kulkarni, Dheeraj M. Pai, Kaushik Mitra, and Pradeep Kiran Sarvepalli, Neural Decoder for Topological Codes using Pseudo-Inverse of Parity Check Matrix, (2019), arXiv:1901.07535.
arXiv:arXiv:1901.07535
[30] Milap Sheth, Sara Zafar Jafarzadeh, and Vlad Gheorghiu, Neural ensemble decoding for topological quantum error-correcting codes, (2019), arXiv:1905.02345.
arXiv:arXiv:1905.02345
[31] Nicolas Delfosse, Pavithran Iyer, and David Poulin, A linear-time benchmarking tool for generalized surface codes, (2016), arXiv:1611.04256.
arXiv:arXiv:1611.04256
[32] Nicolas Delfosse and Pavithran Iyer, Squab – a fast benchmarking software for surface quantum computing architectures, (2016), [Online; accessed 13-December-2019].
http://quantum-squab.com/
[33] Nicolas Delfosse and Naomi H. Nickerson, Almost-linear time decoding algorithm for topological codes, (2017), arXiv:1709.06218.
arXiv:arXiv:1709.06218
[34] Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction (MIT press, Cambridge, 1998).
[35] Nicolai Friis, Oliver Marty, Christine Maier, Cornelius Hempel, Milan Holzäpfel, Petar Jurcevic, Martin B. Plenio, Marcus Huber, Christian Roos, Rainer Blatt, and Ben Lanyon, Observation of Entangled States of a Fully Controlled 20-Qubit System, Phys. Rev. X 8, 021012 (2018), arXiv:1711.11092.
https://doi.org/10.1103/PhysRevX.8.021012
arXiv:arXiv:1711.11092
[36] Jiehang Zhang, Guido Pagano, Paul W. Hess, Antonis Kyprianidis, Patrick Becker, Harvey Kaplan, Alexey V. Gorshkov, Zhexuan Gong, and Christopher Monroe, Observation of a many-body dynamical phase transition with a 53-qubit quantum simulator, Nature 551, 601 (2017), arXiv:1708.01044.
https://doi.org/10.1038/nature24654
arXiv:arXiv:1708.01044
[37] Hannes Bernien, Sylvain Schwartz, Alexander Keesling, Harry Levine, Ahmed Omran, Hannes Pichler, Soonwon Choi, Alexander S. Zibrov, Manuel Endres, Markus Greiner, Vladan Vuletić, and Mikhail D. Lukin, Probing many-body dynamics on a 51-atom quantum simulator, Nature 551, 579 (2017), arXiv:1707.04344.
https://doi.org/10.1038/nature24622
arXiv:arXiv:1707.04344
[38] Héctor Bombín and Miguel Angel Martin-Delgado, Quantum measurements and gates by code deformation, J. Phys. A: Math. Theor. 42, 095302 (2009), arXiv:0704.2540.
https://doi.org/10.1088/1751-8113/42/9/095302
arXiv:arXiv:0704.2540
[39] Sergey Bravyi and Alexei Kitaev, Quantum codes on a lattice with boundary, (1998), arXiv:quant-ph/9811052.
arXiv:arXiv:quant-ph/9811052
[40] Eric Dennis, Alexei Kitaev, Andrew Landahl, and John Preskill, Topological quantum memory, J. Math. Phys. 43, 4452 (2002), arXiv:quant-ph/0110143.
https://doi.org/10.1063/1.1499754
arXiv:arXiv:quant-ph/0110143
[41] Austin G. Fowler, Matteo Mariantoni, John M. Martinis, and Andrew N. Cleland, Surface codes: Towards practical large-scale quantum computation, Phys. Rev. A 86, 032324 (2012), arXiv:1208.0928.
https://doi.org/10.1103/PhysRevA.86.032324
arXiv:arXiv:1208.0928
[42] Hans J. Briegel and Gemma De las Cuevas, Projective simulation for artificial intelligence, Sci. Rep. 7, 400 (2012), arXiv:1104.3787.
https://doi.org/10.1038/srep00400
arXiv:arXiv:1104.3787
[43] Julian Mautner, Adi Makmal, Daniel Manzano, Markus Tiersch, and Hans J. Briegel, Projective Simulation for Classical Learning Agents: A Comprehensive Investigation, New Gener. Comput. 33, 69 (2015), arXiv:1305.1578.
https://doi.org/10.1007/s00354-015-0102-0
arXiv:arXiv:1305.1578
[44] Alexey A. Melnikov, Adi Makmal, Vedran Dunjko, and Hans J. Briegel, Projective simulation with generalization, Sci. Rep. 7, 14430 (2017), arXiv:1504.02247.
https://doi.org/10.1038/s41598-017-14740-y
arXiv:arXiv:1504.02247
[45] Alexey A. Melnikov, Adi Makmal, and Hans J. Briegel, Benchmarking projective simulation in navigation problems, IEEE Access 6, 64639 (2018a), arXiv:1804.08607.
https://doi.org/10.1109/ACCESS.2018.2876494
arXiv:arXiv:1804.08607
[46] Simon Hangl, Emre Ugur, Sandor Szedmak, and Justus Piater, Robotic playing for hierarchical complex skill learning, in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016) pp. 2799–2804, arXiv:1603.00794.
https://doi.org/10.1109/IROS.2016.7759434
arXiv:arXiv:1603.00794
[47] Alexey A. Melnikov, Hendrik Poulsen Nautrup, Mario Krenn, Vedran Dunjko, Markus Tiersch, Anton Zeilinger, and Hans J. Briegel, Active learning machine learns to create new quantum experiments, Proc. Natl. Acad. Sci. U.S.A. 115, 1221 (2018b), arXiv:1706.00868.
https://doi.org/10.1073/pnas.1714936115
arXiv:arXiv:1706.00868
[48] Sebastian Thrun, Is learning the n-th thing any easier than learning the first? in Advances in Neural Information Processing Systems 8, edited by D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo (MIT Press, 1996) pp. 640–646.
http://papers.nips.cc/paper/1034-is-learning-the-n-th-thing-any-easier-than-learning-the-first.pdf
[49] Karl Weiss, Taghi M. Khoshgoftaar, and DingDing Wang, A survey of transfer learning, Journal of Big Data 3, 9 (2016).
https://doi.org/10.1186/s40537-016-0043-6
[50] Nicolas Delfosse and Gilles Zémor, Linear-Time Maximum Likelihood Decoding of Surface Codes over the Quantum Erasure Channel, (2017), arXiv:1703.01517.
arXiv:arXiv:1703.01517
[51] Rami Barends, Julian Kelly, Anthony Megrant, Andrzej Veitia, Daniel Sank, Evan Jeffrey, Ted C. White, Josh Mutus, Austin G. Fowler, B. Campbell, Yu Chen, Zijun Chen, Ben Chiaro, Andrew Dunsworth, Charles Neill, Peter O'Malley, Pedram Roushan, Amit Vainsencher, Jim Wenner, Alexander N. Korotkov, Andrew N. Cleland, and John M. Martinis, Superconducting quantum circuits at the surface code threshold for fault tolerance, Nature 508, 500 (2014), arXiv:1402.4848.
https://doi.org/10.1038/nature13171
arXiv:arXiv:1402.4848
[52] Torsten Karzig, Christina Knapp, Roman M. Lutchyn, Parsa Bonderson, Matthew B. Hastings, Chetan Nayak, Jason Alicea, Karsten Flensberg, Stephan Plugge, Yuval Oreg, Charles M. Marcus, and Michael H. Freedman, Scalable designs for quasiparticle-poisoning-protected topological quantum computation with Majorana zero modes, Phys. Rev. B 95, 235305 (2017), arXiv:1610.05289.
https://doi.org/10.1103/PhysRevB.95.235305
arXiv:arXiv:1610.05289
[53] Jason M. Amini, Hermann Uys, Janus H. Wesenberg, Signe Seidelin, Joseph Britton, John J. Bollinger, Dietrich Leibfried, Christian Ospelkaus, Aaron P. VanDevender, and David J. Wineland, Toward scalable ion traps for quantum information processing, New J. Phys. 12, 033031 (2010), arXiv:0909.2464.
https://doi.org/10.1088/1367-2630/12/3/033031
arXiv:arXiv:0909.2464
[54] Ryan Bowler, John Gaebler, Y. Lin, T. R. Tan, D. Hanneke, J. D. Jost, J. P. Home, Dietrich Leibfried, and David J. Wineland, Coherent Diabatic Ion Transport and Separation in a Multizone Trap Array, Phys. Rev. Lett. 109, 080502 (2012), arXiv:1206.0780.
https://doi.org/10.1103/PhysRevLett.109.080502
arXiv:arXiv:1206.0780
[55] Sergey Bravyi and Robert König, Classification of Topologically Protected Gates for Local Stabilizer Codes, Phys. Rev. Lett. 110, 170503 (2013), arXiv:1206.1609.
https://doi.org/10.1103/PhysRevLett.110.170503
arXiv:arXiv:1206.1609
[56] Fernando Pastawski and Beni Yoshida, Fault-tolerant logical gates in quantum error-correcting codes, Phys. Rev. A 91, 012305 (2015), arXiv:1408.1720.
https://doi.org/10.1103/PhysRevA.91.012305
arXiv:arXiv:1408.1720
[57] Danna Rosenberg, David Kim, Rabi Das, Donna Yost, Simon Gustavsson, David Hover, Philip Krantz, Alexander Melville, Livia Racz, Gabriel O. Samach, Steven J. Weber, Fei Yan, Jonilyn L. Yoder, Andrew J. Kerman, and William D. Oliver, 3d integrated superconducting qubits, npj Quantum Information 3, 42 (2017), arXiv:1706.04116.
https://doi.org/10.1038/s41534-017-0044-0
arXiv:arXiv:1706.04116
[58] Charles H. Bennett, David P. DiVincenzo, and John A. Smolin, Capacities of Quantum Erasure Channels, Phys. Rev. Lett. 78, 3217 (1997), arXiv:quant-ph/9701015.
https://doi.org/10.1103/PhysRevLett.78.3217
arXiv:arXiv:quant-ph/9701015
[59] Markus Grassl, Thomas Beth, and Thomas Pellizzari, Codes for the quantum erasure channel, Phys. Rev. A 56, 33 (1997), arXiv:quant-ph/9610042.
https://doi.org/10.1103/PhysRevA.56.33
arXiv:arXiv:quant-ph/9610042
[60] Scott Kirkpatrick, C. Daniel Gelatt, and Mario P. Vecchi, Optimization by Simulated Annealing, Science 220, 671 (1983).
https://doi.org/10.1126/science.220.4598.671
[61] Michael Reimpell and Reinhard F. Werner, Iterative Optimization of Quantum Error Correcting Codes, Phys. Rev. Lett. 94, 080501 (2005), arXiv:quant-ph/0307138.
https://doi.org/10.1103/PhysRevLett.94.080501
arXiv:arXiv:quant-ph/0307138
[62] Robert L. Kosut and Daniel A. Lidar, Quantum error correction via convex optimization, Quant. Inf. Proc. 8, 443 (2009), arXiv:quant-ph/0606078.
https://doi.org/10.1007/s11128-009-0120-2
arXiv:arXiv:quant-ph/0606078
[63] Peter D. Johnson, Jonathan Romero, Jonathan Olson, Yudong Cao, and Alán Aspuru-Guzik, QVECTOR: an algorithm for device-tailored quantum error correction, (2017), arXiv:1711.02249.
arXiv:arXiv:1711.02249
[64] Anonymous, Improving Exploration of Deep Reinforcement Learning using Planning for Policy Search, in Submitted to International Conference on Learning Representations (2020) under double-blind review [Online at https://openreview.net/forum?id=rJe7CkrFvS; accessed 13-December-2019].
https://openreview.net/forum?id=rJe7CkrFvS
[65] Sergey Levine and Vladlen Koltun, Guided Policy Search, in Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ICML'13 (JMLR.org, 2013) pp. III–1–III–9.
http://dl.acm.org/citation.cfm?id=3042817.3042937
[66] Richard Cleve and Daniel Gottesman, Efficient computations of encodings for quantum error correction, Phys. Rev. A 56, 76 (1997), arXiv:quant-ph/9607030.
https://doi.org/10.1103/PhysRevA.56.76
arXiv:arXiv:quant-ph/9607030
[67] Scott Aaronson and Daniel Gottesman, Improved simulation of stabilizer circuits, Phys. Rev. A 70, 052328 (2004), arXiv:quant-ph/0406196.
https://doi.org/10.1103/PhysRevA.70.052328
arXiv:arXiv:quant-ph/0406196
[68] David P. DiVincenzo and Peter W. Shor, Fault-Tolerant Error Correction with Efficient Quantum Codes, Phys. Rev. Lett. 77, 3260 (1996), arXiv:quant-ph/9605031.
https://doi.org/10.1103/PhysRevLett.77.3260
arXiv:arXiv:quant-ph/9605031
[69] Simon Anders and Hans J. Briegel, Fast simulation of stabilizer circuits using a graph-state representation, Phys. Rev. A 73, 022334 (2006), arXiv:quant-ph/0504117.
https://doi.org/10.1103/PhysRevA.73.022334
arXiv:arXiv:quant-ph/0504117
[70] Lorenza Saitta and Jean-Daniel Zucker, Abstraction in Artificial Intelligence and Complex Systems (Springer, New York, USA, 2013).
[71] Novi Patricia and Barbara Caputo, Learning to Learn, from Transfer Learning to Domain Adaptation: A Unifying Perspective, in 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014) pp. 1442–1449.
https://doi.org/10.1109/CVPR.2014.187
[72] Tatiana Tommasi and Barbara Caputo, The more you know, the less you learn: from knowledge transfer to one-shot learning of object categories, in Proceedings of the British Machine Vision Conference, edited by A. Cavallaro, S. Prince, and D. Alexander (BMVA Press, 2009) pp. 80.1–80.11.
https://doi.org/10.5244/C.23.80
[73] Tatiana Tommasi, Francesco Orabona, and Barbara Caputo, Safety in numbers: Learning categories from few examples with multi model knowledge transfer, in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2010) pp. 3081–3088.
https://doi.org/10.1109/CVPR.2010.5540064
[74] Yusuf Aytar and Andrew Zisserman, Tabula rasa: Model transfer for object category detection, in 2011 International Conference on Computer Vision (2011) pp. 2252–2259.
https://doi.org/10.1109/ICCV.2011.6126504
[75] Panos Aliferis, Frederico Brito, David P. DiVincenzo, John Preskill, Matthias Steffen, and Barbara M. Terhal, Fault-tolerant computing with biased-noise superconducting qubits: a case study, New J. Phys. 11, 013061 (2009), arXiv:0806.0383.
https://doi.org/10.1088/1367-2630/11/1/013061
arXiv:arXiv:0806.0383
[76] Michael D. Shulman, Oliver E. Dial, Shannon P. Harvey, Hendrik Bluhm, Vladimir Umansky, and Amir Yacoby, Demonstration of Entanglement of Electrostatically Coupled Singlet-Triplet Qubits, Science 336, 202 (2012), arXiv:1202.1828.
https://doi.org/10.1126/science.1217692
arXiv:arXiv:1202.1828
[77] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis, Human-level control through deep reinforcement learning, Nature 518, 529 (2015).
https://doi.org/10.1038/nature14236
[78] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis, Mastering the game of Go without human knowledge, Nature 550, 354 (2017).
https://doi.org/10.1038/nature24270
[79] Will Knight, Reinforcement learning – by experimenting, computers are figuring out how to do things that no programmer could teach them, (2017), [Online; accessed 13-December-2019].
https://www.technologyreview.com/s/603501/10-breakthrough-technologies-2017-reinforcement-learning/
[80] Lenka Zdeborová, New tool in the box, Nat. Phys. 13, 420 (2017).
https://doi.org/10.1038/nphys4053
[81] Raban Iten, Tony Metger, Henrik Wilming, Lidia del Rio, and Renato Renner, Discovering physical concepts with neural networks, Phys. Rev. Lett. (accepted, 2019), arXiv:1807.10300.
arXiv:arXiv:1807.10300
https://journals.aps.org/prl/accepted/9e07eY09T2e1fd7f88ae46166090ef41fa6ad4c34
[82] Thomas Fösel, Petru Tighineanu, Talitha Weiss, and Florian Marquardt, Reinforcement Learning with Neural Networks for Quantum Feedback, Phys. Rev. X 8, 031084 (2018), arXiv:1802.05267.
https://doi.org/10.1103/PhysRevX.8.031084
arXiv:arXiv:1802.05267
[83] Moritz August and José Miguel Hernández-Lobato, Taking Gradients Through Experiments: LSTMs and Memory Proximal Policy Optimization for Black-Box Quantum Control, in High Performance Computing, edited by Rio Yokota, Michèle Weiland, John Shalf, and Sadaf Alam (Springer International Publishing, Cham, 2018) arXiv:1802.04063.
https://doi.org/10.1007/978-3-030-02465-9_43
arXiv:arXiv:1802.04063
[84] Matthew R. Kretchmar, Parallel reinforcement learning, in The 6th World Conference on Systematics, Cybernetics, and Informatics (2002) pp. 165–170.
[85] Enda Barrett, Jim Duggan, and Enda Howley, A parallel framework for bayesian reinforcement learning, Connect. Sci. 26, 7 (2014).
https://doi.org/10.1080/09540091.2014.885268
[86] Sepp Hochreiter and Jürgen Schmidhuber, Long Short-Term Memory, Neural Comput. 9, 1735 (1997).
https://doi.org/10.1162/neco.1997.9.8.1735
[87] Hendrik Poulsen Nautrup, Nicolai Friis, and Hans J. Briegel, Fault-tolerant interface between quantum memories and quantum processors, Nat. Commun. 8, 1321 (2017), arXiv:1609.08062.
https://doi.org/10.1038/s41467-017-01418-2
arXiv:arXiv:1609.08062
[88] Dorit Aharonov, Alexei Kitaev, and John Preskill, Fault-Tolerant Quantum Computation with Long-Range Correlated Noise, Phys. Rev. Lett. 96, 050504 (2006), arXiv:quant-ph/0510231.
https://doi.org/10.1103/PhysRevLett.96.050504
arXiv:arXiv:quant-ph/0510231
[89] Hui Khoon Ng and John Preskill, Fault-tolerant quantum computation versus Gaussian noise, Phys. Rev. A 79, 032318 (2009), arXiv:0810.4953.
https://doi.org/10.1103/PhysRevA.79.032318
arXiv:arXiv:0810.4953
[90] Austin G. Fowler and John M. Martinis, Quantifying the effects of local many-qubit errors and nonlocal two-qubit errors on the surface code, Phys. Rev. A 89, 032316 (2014), arXiv:1401.2466.
https://doi.org/10.1103/PhysRevA.89.032316
arXiv:arXiv:1401.2466
[91] Naomi H. Nickerson and Benjamin J. Brown, Analysing correlated noise on the surface code using adaptive decoding algorithms, Quantum 3, 131 (2019), arXiv:1712.00502.
https://doi.org/10.22331/q-2019-04-08-131
arXiv:arXiv:1712.00502
[92] Adi Makmal, Alexey A. Melnikov, Vedran Dunjko, and Hans J. Briegel, Meta-learning within Projective Simulation, IEEE Access 4, 2110 (2016), arXiv:1602.08017.
https://doi.org/10.1109/ACCESS.2016.2556579
arXiv:arXiv:1602.08017
Cited by
[1] Friederike Metz and Marin Bukov, "Self-correcting quantum many-body control using reinforcement learning with tensor networks", Nature Machine Intelligence 5 7, 780 (2023).
[2] Jelena Mackeprang, Durga B. Rao Dasari, and Jörg Wrachtrup, "A reinforcement learning approach for quantum state engineering", Quantum Machine Intelligence 2 1, 5 (2020).
[3] Sofiene Jerbi, Lea M. Trenkwalder, Hendrik Poulsen Nautrup, Hans J. Briegel, and Vedran Dunjko, "Quantum Enhancements for Deep Reinforcement Learning in Large Spaces", PRX Quantum 2 1, 010328 (2021).
[4] Yun-Jia Xue, Hao-Wen Wang, Yan-Bing Tian, Yi-Nuo Wang, Yu-Xuan Wang, Shu-Mei Wang, and Liuguo Yin, "Quantum Information Protection Scheme Based on Reinforcement Learning for Periodic Surface Codes", Quantum Engineering 2022, 1 (2022).
[5] V. Saggio, B. E. Asenbeck, A. Hamann, T. Strömberg, P. Schiansky, V. Dunjko, N. Friis, N. C. Harris, M. Hochberg, D. Englund, S. Wölk, H. J. Briegel, and P. Walther, "Experimental quantum speed-up in reinforcement learning agents", Nature 591 7849, 229 (2021).
[6] Borja Requena, Gorka Muñoz-Gil, Maciej Lewenstein, Vedran Dunjko, and Jordi Tura, "Certificates of quantum many-body properties assisted by machine learning", Physical Review Research 5 1, 013097 (2023).
[7] Sebastiano Corli and Enrico Prati, 2022 IEEE International Conference on Rebooting Computing (ICRC) 1 (2022) ISBN:979-8-3503-4709-8.
[8] Terry Farrelly, David K Tuckett, and Thomas M Stace, "Local tensor-network codes", New Journal of Physics 24 4, 043015 (2022).
[9] Onur Danaci, Sanjaya Lohani, Brian T Kirby, and Ryan T Glasser, "Machine learning pipeline for quantum state estimation with incomplete measurements", Machine Learning: Science and Technology 2 3, 035014 (2021).
[10] Artem Kryukov, Roman Abramov, Leonid E. Fedichkin, Alexander Alodjants, and Alexey A. Melnikov, "Supervised graph classification for chiral quantum walks", Physical Review A 105 2, 022208 (2022).
[11] Ran-Yi-Liu Chen, Ben-Chi Zhao, Zhi-Xin Song, Xuan-Qiang Zhao, Kun Wang, and Xin Wang, "Hybrid quantum-classical algorithms: Foundation, design and applications", Acta Physica Sinica 70 21, 210302 (2021).
[12] Alexey A. Melnikov, Pavel Sekatski, and Nicolas Sangouard, "Setting Up Experimental Bell Tests with Reinforcement Learning", Physical Review Letters 125 16, 160401 (2020).
[13] Robin Yunfei Wen and Achim Kempf, "The transfer of entanglement negativity at the onset of interactions", Journal of Physics A: Mathematical and Theoretical 55 49, 495304 (2022).
[14] S. Varona and M. A. Martin-Delgado, "Determination of the semion code threshold using neural decoders", Physical Review A 102 3, 032411 (2020).
[15] Yu-Qin Chen, Yu Chen, Chee-Kong Lee, Shengyu Zhang, and Chang-Yu Hsieh, "Optimizing quantum annealing schedules with Monte Carlo tree search enhanced with neural networks", Nature Machine Intelligence 4 3, 269 (2022).
[16] Mario Krenn, Jonas Landgraf, Thomas Foesel, and Florian Marquardt, "Artificial intelligence and machine learning for quantum technologies", Physical Review A 107 1, 010101 (2023).
[17] Simon D. Reiß and Peter van Loock, "Deep reinforcement learning for key distribution based on quantum repeaters", Physical Review A 108 1, 012406 (2023).
[18] Karl Hammar, Alexei Orekhov, Patrik Wallin Hybelius, Anna Katariina Wisakanto, Basudha Srivastava, Anton Frisk Kockum, and Mats Granath, "Error-rate-agnostic decoding of topological stabilizer codes", Physical Review A 105 4, 042616 (2022).
[19] Jian Lin, Meng Ye, Jia-Wei Zhu, and Xiao-Peng Li, "Machine learning assisted quantum adiabatic algorithm design", Acta Physica Sinica 70 14, 140306 (2021).
[20] Andrey Zhukov and Walter Pogosov, "Quantum error reduction with deep neural network applied at the post-processing stage", Quantum Information Processing 21 3, 93 (2022).
[21] Antonio A. Gentile, Brian Flynn, Sebastian Knauer, Nathan Wiebe, Stefano Paesani, Christopher E. Granade, John G. Rarity, Raffaele Santagati, and Anthony Laing, "Learning models of quantum systems from experiments", Nature Physics 17 7, 837 (2021).
[22] Sangkha Borah, Bijita Sarma, Michael Kewming, Gerard J. Milburn, and Jason Twamley, "Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential", Physical Review Letters 127 19, 190403 (2021).
[23] Xianchao Zhu and Xiaokai Hou, "Quantum architecture search via truly proximal policy optimization", Scientific Reports 13 1, 5157 (2023).
[24] Valentin Gebhart, Martin Bohmann, Karsten Weiher, Nicola Biagi, Alessandro Zavatta, Marco Bellini, and Elizabeth Agudelo, "Identifying nonclassicality from experimental data using artificial neural networks", Physical Review Research 3 2, 023229 (2021).
[25] Samuel Yen-Chi Chen, Chao-Han Huck Yang, Jun Qi, Pin-Yu Chen, Xiaoli Ma, and Hsi-Sheng Goan, "Variational Quantum Circuits for Deep Reinforcement Learning", IEEE Access 8, 141007 (2020).
[26] Lorenzo Moro, Matteo G. A. Paris, Marcello Restelli, and Enrico Prati, "Quantum compiling by deep reinforcement learning", Communications Physics 4 1, 178 (2021).
[27] Chenfeng Cao, Chao Zhang, Zipeng Wu, Markus Grassl, and Bei Zeng, "Quantum variational learning for quantum error-correcting codes", Quantum 6, 828 (2022).
[28] Juan Carrasquilla, "Machine learning for quantum matter", Advances in Physics: X 5 1, 1797528 (2020).
[29] Oleksandr Balabanov and Mats Granath, "Unsupervised interpretable learning of topological indices invariant under permutations of atomic bands", Machine Learning: Science and Technology 2 2, 025008 (2021).
[30] Hendrik Poulsen Nautrup, Tony Metger, Raban Iten, Sofiene Jerbi, Lea M Trenkwalder, Henrik Wilming, Hans J Briegel, and Renato Renner, "Operationally meaningful representations of physical systems in neural networks", Machine Learning: Science and Technology 3 4, 045025 (2022).
[31] Saikat Basu, Amit Saha, Amlan Chakrabarti, and Susmita Sur-Kolay, " i -QER: An Intelligent Approach Towards Quantum Error Reduction ", ACM Transactions on Quantum Computing 3 4, 1 (2022).
[32] Emanuele Polino, Mauro Valeri, Nicolò Spagnolo, and Fabio Sciarrino, "Photonic quantum metrology", AVS Quantum Science 2 2, 024703 (2020).
[33] Kentaro Murakami and Jianjun Zhao, 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) 694 (2022) ISBN:978-1-6654-7704-8.
[34] Xiaosi Xu, Simon C. Benjamin, and Xiao Yuan, "Variational Circuit Compiler for Quantum Error Correction", Physical Review Applied 15 3, 034068 (2021).
[35] Julius Wallnöfer, Alexey A. Melnikov, Wolfgang Dür, and Hans J. Briegel, "Machine Learning for Long-Distance Quantum Communication", PRX Quantum 1 1, 010301 (2020).
[36] Juan Carrasquilla and Giacomo Torlai, "How To Use Neural Networks To Investigate Quantum Many-Body Physics", PRX Quantum 2 4, 040201 (2021).
[37] Pavithran Iyer, Aditya Jain, Stephen D. Bartlett, and Joseph Emerson, "Efficient diagnostics for quantum error correction", Physical Review Research 4 4, 043218 (2022).
[38] Jiahao Yao, Lin Lin, and Marin Bukov, "Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving", Physical Review X 11 3, 031070 (2021).
[39] Bobak Toussi Kiani, Giacomo De Palma, Milad Marvian, Zi-Wen Liu, and Seth Lloyd, "Learning quantum data with the quantum earth mover’s distance", Quantum Science and Technology 7 4, 045002 (2022).
[40] Hao-Wen Wang , Qian Cao , Yun-Jia Xue , Li Ding , Han-Yang Liu , Yu-Min Dong , and Hong-Yang Ma , "Determining quantum topological semion code decoder performance and error correction effectiveness with reinforcement learning", Frontiers in Physics 10, 981225 (2022).
[41] Fulvio Flamini, Arne Hamann, Sofiène Jerbi, Lea M Trenkwalder, Hendrik Poulsen Nautrup, and Hans J Briegel, "Photonic architecture for reinforcement learning", New Journal of Physics 22 4, 045002 (2020).
[42] Riccardo Porotti, Dario Tamascelli, Marcello Restelli, and Enrico Prati, "Coherent transport of quantum states by deep reinforcement learning", Communications Physics 2 1, 61 (2019).
[43] Lucas Lamata, "Quantum Reinforcement Learning with Quantum Photonics", Photonics 8 2, 33 (2021).
[44] Valentin Gebhart and Martin Bohmann, "Neural-network approach for identifying nonclassicality from click-counting data", Physical Review Research 2 2, 023150 (2020).
[45] Alexander Erhard, Hendrik Poulsen Nautrup, Michael Meth, Lukas Postler, Roman Stricker, Martin Stadler, Vlad Negnevitsky, Martin Ringbauer, Philipp Schindler, Hans J. Briegel, Rainer Blatt, Nicolai Friis, and Thomas Monz, "Entangling logical qubits with lattice surgery", Nature 589 7841, 220 (2021).
[46] Alexey Melnikov, Mohammad Kordzanganeh, Alexander Alodjants, and Ray-Kuang Lee, "Quantum machine learning: from physics to software engineering", Advances in Physics: X 8 1, 2165452 (2023).
[47] V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsioutsios, and M. H. Devoret, "Model-Free Quantum Control with Reinforcement Learning", Physical Review X 12 1, 011059 (2022).
[48] Lirandë Pira and Chris Ferrie, "An invitation to distributed quantum neural networks", Quantum Machine Intelligence 5 2, 23 (2023).
[49] David Fitzek, Mattias Eliasson, Anton Frisk Kockum, and Mats Granath, "Deep Q-learning decoder for depolarizing noise on the toric code", Physical Review Research 2 2, 023230 (2020).
[50] W. L. Boyajian, J. Clausen, L. M. Trenkwalder, V. Dunjko, and H. J. Briegel, "On the convergence of projective-simulation–based reinforcement learning in Markov decision processes", Quantum Machine Intelligence 2 2, 13 (2020).
[51] Liang-Ying Chih and Murray Holland, "Reinforcement-learning-based matter-wave interferometer in a shaken optical lattice", Physical Review Research 3 3, 033279 (2021).
[52] Xiao-Ming Zhang, Zezhu Wei, Raza Asad, Xu-Chen Yang, and Xin Wang, "When does reinforcement learning stand out in quantum control? A comparative study on state preparation", npj Quantum Information 5 1, 85 (2019).
[53] Benjamin Eva, Katja Ried, Thomas Müller, and Hans J. Briegel, "How a Minimal Learning Agent can Infer the Existence of Unobserved Variables in a Complex Environment", Minds and Machines 33 1, 185 (2023).
[54] Akshaya Jayashankar and Prabha Mandayam, "Quantum Error Correction: Noise-Adapted Techniques and Applications", Journal of the Indian Institute of Science 103 2, 497 (2023).
[55] David A. Herrera-Martí, "Policy Gradient Approach to Compilation of Variational Quantum Circuits", Quantum 6, 797 (2022).
[56] Sanjaya Lohani, Brian T Kirby, Michael Brodsky, Onur Danaci, and Ryan T Glasser, "Machine learning assisted quantum state estimation", Machine Learning: Science and Technology 1 3, 035007 (2020).
[57] Hamza Jaffali and Luke Oeding, "Learning algebraic models of quantum entanglement", Quantum Information Processing 19 9, 279 (2020).
[58] F Battistel, C Chamberland, K Johar, R W J Overwater, F Sebastiano, L Skoric, Y Ueno, and M Usman, "Real-time decoding for fault-tolerant quantum computing: progress, challenges and outlook", Nano Futures 7 3, 032003 (2023).
[59] Zhikang T. Wang, Yuto Ashida, and Masahito Ueda, "Deep Reinforcement Learning Control of Quantum Cartpoles", Physical Review Letters 125 10, 100401 (2020).
[60] Jia-Hao Cao, Feng Chen, Qi Liu, Tian-Wei Mao, Wen-Xin Xu, Ling-Na Wu, and Li You, "Detection of Entangled States Supported by Reinforcement Learning", Physical Review Letters 131 7, 073201 (2023).
[61] Laia Domingo Colomer, Michalis Skotiniotis, and Ramon Muñoz-Tapia, "Reinforcement learning for optimal error correction of toric codes", Physics Letters A 384 17, 126353 (2020).
[62] Omar Shindi, Qi Yu, Parth Girdhar, and Daoyi Dong, 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC) 2116 (2022) ISBN:978-1-6654-5258-8.
[63] Hossein Dehghani, Ali Lavasani, Mohammad Hafezi, and Michael J. Gullans, "Neural-network decoders for measurement induced phase transitions", Nature Communications 14 1, 2918 (2023).
[64] Oleg M Sotnikov and Vladimir V Mazurenko, "Neural network agent playing spin Hamiltonian games on a quantum computer", Journal of Physics A: Mathematical and Theoretical 53 13, 135303 (2020).
[65] Zebo Yang, Maede Zolanvari, and Raj Jain, "A Survey of Important Issues in Quantum Computing and Communications", IEEE Communications Surveys & Tutorials 25 2, 1059 (2023).
[66] Pai Peng, Xiaoyang Huang, Chao Yin, Linta Joseph, Chandrasekhar Ramanathan, and Paola Cappellaro, "Deep Reinforcement Learning for Quantum Hamiltonian Engineering", Physical Review Applied 18 2, 024033 (2022).
[67] Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, and Lenka Zdeborová, "Machine learning and the physical sciences*", Reviews of Modern Physics 91 4, 045002 (2019).
[68] Samuel Yen-Chi Chen, Chih-Min Huang, Chia-Wei Hsing, Hsi-Sheng Goan, and Ying-Jer Kao, "Variational quantum reinforcement learning via evolutionary optimization", Machine Learning: Science and Technology 3 1, 015025 (2022).
[69] Samuel Yen-Chi Chen, Shinjae Yoo, and Yao-Lung L. Fang, "Quantum Long Short-Term Memory", arXiv:2009.01783, (2020).
[70] J. Darulová, S. J. Pauka, N. Wiebe, K. W. Chan, G. C. Gardener, M. J. Manfra, M. C. Cassidy, and M. Troyer, "Autonomous Tuning and Charge-State Detection of Gate-Defined Quantum Dots", Physical Review Applied 13 5, 054005 (2020).
[71] Vincent Paul Su, ChunJun Cao, Hong-Ye Hu, Yariv Yanay, Charles Tahan, and Brian Swingle, "Discovery of Optimal Quantum Error Correcting Codes via Reinforcement Learning", arXiv:2305.06378, (2023).
[72] Justin Reyes and Miles Stoudenmire, "A Multi-Scale Tensor Network Architecture for Classification and Regression", arXiv:2001.08286, (2020).
[73] Natalie C. Brown and Kenneth R. Brown, "Leakage mitigation for quantum error correction using a mixed qubit scheme", Physical Review A 100 3, 032325 (2019).
[74] Jun-Jie Chen and Ming Xue, "Manipulation of Spin Dynamics by Deep Reinforcement Learning Agent", arXiv:1901.08748, (2019).
[75] Kai-Wen Zhao, Wen-Han Kao, Kai-Hsin Wu, and Ying-Jer Kao, "Generation of ice states through deep reinforcement learning", Physical Review E 99 6, 062106 (2019).
[76] Katja Ried, Benjamin Eva, Thomas Müller, and Hans J. Briegel, "How a minimal learning agent can infer the existence of unobserved variables in a complex environment", arXiv:1910.06985, (2019).
[77] Chaitanya Chinni, Abhishek Kulkarni, Dheeraj M. Pai, Kaushik Mitra, and Pradeep Kiran Sarvepalli, "Neural Decoder for Topological Codes using Pseudo-Inverse of Parity Check Matrix", arXiv:1901.07535, (2019).
[78] Alexey A. Melnikov, Leonid E. Fedichkin, and Alexander Alodjants, "Predicting quantum advantage by quantum walk with convolutional neural networks", arXiv:1901.10632, (2019).
[79] Sathwik Chadaga, Mridul Agarwal, and Vaneet Aggarwal, "Encoders and Decoders for Quantum Expander Codes Using Machine Learning", arXiv:1909.02945, (2019).
The above citations are from Crossref's cited-by service (last updated successfully 2023-09-27 21:49:06) and SAO/NASA ADS (last updated successfully 2023-09-27 21:49:07). The list may be incomplete as not all publishers provide suitable and complete citation data.
This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.