A volumetric framework for quantum computer benchmarks

Robin Blume-Kohout; Kevin C. Young

doi:10.22331/q-2020-11-15-362

A volumetric framework for quantum computer benchmarks

Robin Blume-Kohout and Kevin C. Young

Quantum Performance Laboratory, Sandia National Laboratories\vspace-.1cm \ Albuquerque, NM 87185 and Livermore, California 94550

Published:	2020-11-15, volume 4, page 362
Eprint:	arXiv:1904.05546v4
Doi:	https://doi.org/10.22331/q-2020-11-15-362
Citation:	Quantum 4, 362 (2020).

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.

Updated after initial publication: This publication was updated to version v4 after the initial publication. The authors left the following comment on the arXiv:

Latest version published in Quantum. 16 pages, 6 figures

Abstract

We propose a very large family of benchmarks for probing the performance of quantum computers. We call them {volumetric benchmarks} (VBs) because they generalize IBM's benchmark for measuring quantum volume [1]. The quantum volume benchmark defines a family of {square} circuits whose depth $d$ and width $w$ are the same. A volumetric benchmark defines a family of {rectangular} quantum circuits, for which $d$ and $w$ are uncoupled to allow the study of time/space performance trade-offs. Each VB defines a mapping from circuit shapes -- $(w,d)$ pairs -- to test suites ${\mathcal{C}}(w,d)$. A test suite is an ensemble of test circuits that share a common structure. The test suite ${\mathcal{C}}$ for a given circuit shape may be a single circuit $C$, a specific list of circuits $\{C_1\ldots C_N\}$ that must all be run, or a large set of possible circuits equipped with a distribution $Pr(C)$. The circuits in a given VB share a structure, which is limited only by designers' creativity. We list some known benchmarks, and other circuit families, that fit into the VB framework: several families of random circuits, periodic circuits, and algorithm-inspired circuits. The last ingredient defining a benchmark is a success criterion that defines when a processor is judged to have ``passed'' a given test circuit. We discuss several options. Benchmark data can be analyzed in many ways to extract many properties, but we propose a simple, universal graphical summary of results that illustrates the Pareto frontier of the $d$ vs $w$ trade-off for the processor being benchmarked.

► BibTeX data

@article{BlumeKohout2020volumetricframework,
  doi = {10.22331/q-2020-11-15-362},
  url = {https://doi.org/10.22331/q-2020-11-15-362},
  title = {A volumetric framework for quantum computer benchmarks},
  author = {Blume-Kohout, Robin and Young, Kevin C.},
  journal = {{Quantum}},
  issn = {2521-327X},
  publisher = {{Verein zur F{\"{o}}rderung des Open Access Publizierens in den Quantenwissenschaften}},
  volume = {4},
  pages = {362},
  month = nov,
  year = {2020}
}

► References

[1] Andrew W Cross, Lev S Bishop, Sarah Sheldon, Paul D Nation, and Jay M Gambetta. Validating quantum computers using randomized model circuits. Phys. Rev. A, 100 (3): 032328, September 2019. ISSN 1050-2947. 10.1103/PhysRevA.100.032328.
https://doi.org/10.1103/PhysRevA.100.032328

[2] Rami Barends, Julian Kelly, et al. Superconducting quantum circuits at the surface code threshold for fault tolerance. Nature, 508 (7497): 500, 2014. 10.1038/nature13171.
https://doi.org/10.1038/nature13171

[3] Simon J Devitt. Performing quantum computing experiments in the cloud. Phys. Rev. A, 94 (3): 032329, September 2016. ISSN 1050-2947. 10.1103/PhysRevA.94.032329.
https://doi.org/10.1103/PhysRevA.94.032329

[4] S Debnath, N M Linke, C Figgatt, K A Landsman, K Wright, and C Monroe. Demonstration of a small programmable quantum computer with atomic qubits. Nature, 536 (7614): 63–66, August 2016. ISSN 0028-0836, 1476-4687. 10.1038/nature18648.
https://doi.org/10.1038/nature18648

[5] Norbert M Linke, Dmitri Maslov, Martin Roetteler, Shantanu Debnath, Caroline Figgatt, Kevin A Landsman, Kenneth Wright, and Christopher Monroe. Experimental comparison of two quantum computing architectures. Proceedings of the National Academy of Sciences, 114 (13): 3305–3310, 2017. 10.1073/pnas.1618020114.
https://doi.org/10.1073/pnas.1618020114

[6] James R Wootton. Benchmarking of quantum processors with random circuits. arXiv:1806.02736, 2018.
arXiv:1806.02736

[7] Julian Kelly, Rami Barends, et al. State preservation by repetitive error detection in a superconducting quantum circuit. Nature, 519 (7541): 66, 2015. 10.1038/nature14270.
https://doi.org/10.1038/nature14270

[8] IonQ press release. [Online], 2018. https://ionq.co/news/december-11-2018 [2019-03-18].
https://ionq.co/news/december-11-2018

[9] IBM Q ``Tokyo'' Specifications. [Online], 2019a. https://www.research.ibm.com/ibm-q/technology/devices/#ibmq-20-tokyo [2019-03-18].
https://www.research.ibm.com/ibm-q/technology/devices/#ibmq-20-tokyo

[10] Frank Arute, Kunal Arya, et al. Quantum supremacy using a programmable superconducting processor. Nature, 574 (7779): 505–510, October 2019. ISSN 0028-0836, 1476-4687. 10.1038/s41586-019-1666-5.
https://doi.org/10.1038/s41586-019-1666-5

[11] IBM news room. [Online], 2019b. https://newsroom.ibm.com/2019-09-18-IBM-Opens-Quantum-Computation-Center-in-New-York-Brings-Worlds-Largest-Fleet-of-Quantum-Computing-Systems-Online-Unveils-New-53-Qubit-Quantum-System-for-Broad-Use [2019-09-18].
https://newsroom.ibm.com/2019-09-18-IBM-Opens-Quantum-Computation-Center-in-New-York-Brings-Worlds-Largest-Fleet-of-Quantum-Computing-Systems-Online-Unveils-New-53-Qubit-Quantum-System-for-Broad-Use

[12] Fernando G S L Brandão, Aram W Harrow, and Michał Horodecki. Local random quantum circuits are approximate Polynomial-Designs. Commun. Math. Phys., 346 (2): 397–434, September 2016. ISSN 1432-0916. 10.1007/s00220-016-2706-8.
https://doi.org/10.1007/s00220-016-2706-8

[13] Joseph Emerson, Robert Alicki, and Karol Życzkowski. Scalable noise estimation with random unitary operators. Journal of Optics B: Quantum and Semiclassical Optics, 7 (10): S347, 2005. 10.1088/1464-4266/7/10/021.
https://doi.org/10.1088/1464-4266/7/10/021

[14] Joseph Emerson, Marcus Silva, Osama Moussa, Colm Ryan, Martin Laforest, Jonathan Baugh, David G Cory, and Raymond Laflamme. Symmetrized characterization of noisy quantum processes. Science, 317 (5846): 1893–1896, 2007. 10.1126/science.1145699.
https://doi.org/10.1126/science.1145699

[15] Emanuel Knill, Dietrich Leibfried, et al. Randomized benchmarking of quantum gates. Physical Review A, 77 (1): 012307, 2008. 10.1103/PhysRevA.77.012307.
https://doi.org/10.1103/PhysRevA.77.012307

[16] Easwar Magesan, Jay M Gambetta, and Joseph Emerson. Scalable and robust randomized benchmarking of quantum processes. Physical Review Letters, 106 (18): 180504, 2011. 10.1103/PhysRevLett.106.180504.
https://doi.org/10.1103/PhysRevLett.106.180504

[17] Easwar Magesan, Jay M Gambetta, and Joseph Emerson. Characterizing quantum gates via randomized benchmarking. Physical Review A, 85 (4): 042311, 2012. 10.1103/PhysRevA.85.042311.
https://doi.org/10.1103/PhysRevA.85.042311

[18] Robin Blume-Kohout, John King Gamble, Erik Nielsen, Kenneth Rudinger, Jonathan Mizrahi, Kevin Fortier, and Peter Maunz. Demonstration of qubit operations below a rigorous fault tolerance threshold with gate set tomography. Nature Communications, 8: 14485, 2017. 10.1038/ncomms16226.
https://doi.org/10.1038/ncomms16226

[19] Daniel Greenbaum. Introduction to quantum gate set tomography. arXiv:1509.02921, 2015.
arXiv:1509.02921

[20] Juan P Dehollain, Juha T Muhonen, Robin Blume-Kohout, Kenneth M Rudinger, John King Gamble, Erik Nielsen, Arne Laucht, Stephanie Simmons, Rachpon Kalra, Andrew S Dzurak, and Andrea Morello. Optimization of a solid-state electron spin qubit using gate set tomography. New J. Phys., 18 (10): 103018, October 2016. ISSN 1367-2630. 10.1088/1367-2630/18/10/103018.
https://doi.org/10.1088/1367-2630/18/10/103018

[21] Jack J Dongarra, Piotr Luszczek, and Antoine Petitet. The LINPACK benchmark: past, present and future. Concurrency and Computation: Practice and Experience, 15 (9): 803–820, 2003. 10.1002/cpe.728.
https://doi.org/10.1002/cpe.728

[22] Joseph Emerson. Benchmarking into the future. IARPA LogiQ Principal Investigators Meeting, 6 September, 2017.

[23] Joseph Emerson. (private communication, 2019).

[24] Alexander Erhard, Joel J Wallman, Lukas Postler, Michael Meth, Roman Stricker, Esteban A Martinez, Philipp Schindler, Thomas Monz, Joseph Emerson, and Rainer Blatt. Characterizing large-scale quantum computers via cycle benchmarking. Nat. Commun., 10 (1): 5347, November 2019. ISSN 2041-1723. 10.1038/s41467-019-13068-7.
https://doi.org/10.1038/s41467-019-13068-7

[25] Lev Bishop. Is the `Quantum Volume' a fair metric for future, elaborate, high value quantum computations? Quantum Computing Stack Exchange, 2018. https://quantumcomputing.stackexchange.com/a/4001 [2019-03-21].
https://quantumcomputing.stackexchange.com/a/4001

[26] Peter W Shor. Algorithms for quantum computation: Discrete logarithms and factoring. In Proceedings 35th Annual Symposium on Foundations of Computer Science, pages 124–134. IEEE, 1994. 10.1109/SFCS.1994.365700.
https://doi.org/10.1109/SFCS.1994.365700

[27] Lov K Grover. Quantum mechanics helps in searching for a needle in a haystack. Physical Review Letters, 79 (2): 325, 1997. 10.1103/PhysRevLett.79.325.
https://doi.org/10.1103/PhysRevLett.79.325

[28] Richard Cleve and John Watrous. Fast parallel circuits for the quantum Fourier transform. In Proceedings 41st Annual Symposium on Foundations of Computer Science, pages 526–536. IEEE, 2000. 10.1109/SFCS.2000.892140.
https://doi.org/10.1109/SFCS.2000.892140

[29] Yasuhiro Sekino and Leonard Susskind. Fast scramblers. Journal of High Energy Physics, 2008 (10): 065, 2008. 10.1088/1126-6708/2008/10/065.
https://doi.org/10.1088/1126-6708/2008/10/065

[30] Scott Aaronson and Lijie Chen. Complexity-Theoretic foundations of quantum supremacy experiments. December 2016.

[31] Steven J van Enk and Robin Blume-Kohout. When quantum tomography goes wrong: drift of quantum sources and other errors. New Journal of Physics, 15 (2): 025024, 2013. 10.1088/1367-2630/15/2/025024.
https://doi.org/10.1088/1367-2630/15/2/025024

[32] Kenneth Rudinger, Timothy Proctor, Dylan Langharst, Mohan Sarovar, Kevin Young, and Robin Blume-Kohout. Probing Context-Dependent errors in quantum processors. Phys. Rev. X, 9 (2): 021045, June 2019. 10.1103/PhysRevX.9.021045.
https://doi.org/10.1103/PhysRevX.9.021045

[33] Kristine Boone, Arnaud Carignan-Dugas, Joel J Wallman, and Joseph Emerson. Randomized benchmarking under different gate sets. Phys. Rev. A, 99 (3): 032329, March 2019. ISSN 1050-2947. 10.1103/PhysRevA.99.032329.
https://doi.org/10.1103/PhysRevA.99.032329

[34] Timothy J Proctor, Arnaud Carignan-Dugas, Kenneth Rudinger, Erik Nielsen, Robin Blume-Kohout, and Kevin Young. Direct randomized benchmarking for multiqubit devices. Phys. Rev. Lett., 123 (3): 030503, July 2019. ISSN 0031-9007, 1079-7114. 10.1103/PhysRevLett.123.030503.
https://doi.org/10.1103/PhysRevLett.123.030503

[35] Jay M Gambetta, Antonio D Córcoles, et al. Characterization of addressability by simultaneous randomized benchmarking. Physical Review Letters, 109 (24): 240504, 2012. 10.1103/PhysRevLett.109.240504.
https://doi.org/10.1103/PhysRevLett.109.240504

[36] Sergio Boixo, Sergei V Isakov, Vadim N Smelyanskiy, Ryan Babbush, Nan Ding, Zhang Jiang, Michael J Bremner, John M Martinis, and Hartmut Neven. Characterizing quantum supremacy in near-term devices. Nature Physics, 14 (6): 595, 2018. 10.1038/s41567-018-0124-x.
https://doi.org/10.1038/s41567-018-0124-x

[37] Charles Neill, Pedran Roushan, et al. A blueprint for demonstrating quantum supremacy with superconducting qubits. Science, 360 (6385): 195–199, 2018. 10.1126/science.aao4309.
https://doi.org/10.1126/science.aao4309

[38] Shelby Kimmel, Guang Hao Low, and Theodore J Yoder. Robust calibration of a universal single-qubit gate set via robust phase estimation. Physical Review A, 92 (6): 062315, 2015. 10.1103/PhysRevA.92.062315.
https://doi.org/10.1103/PhysRevA.92.062315

[39] Richard Cleve, Artur Ekert, Chiara Macchiavello, and Michele Mosca. Quantum algorithms revisited. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 454 (1969): 339–354, 1998. 10.1098/rspa.1998.0164.
https://doi.org/10.1098/rspa.1998.0164

[40] Alberto Peruzzo, Jarrod McClean, et al. A variational eigenvalue solver on a photonic quantum processor. Nature Communications, 5: 4213, 2014. 10.1038/ncomms5213.
https://doi.org/10.1038/ncomms5213

[41] David C McKay, Christopher J Wood, Sarah Sheldon, Jerry M Chow, and Jay M Gambetta. Efficient $Z$ gates for quantum computing. Phys. Rev. A, 96 (2): 022330, August 2017. ISSN 1050-2947. 10.1103/PhysRevA.96.022330.
https://doi.org/10.1103/PhysRevA.96.022330

[42] M A Rol, L Ciorciaro, F K Malinowski, B M Tarasinski, R E Sagastizabal, C C Bultink, Y Salathe, N Haandbaek, J Sedivy, and L DiCarlo. Time-domain characterization and correction of on-chip distortion of control pulses in a quantum processor. Appl. Phys. Lett., 116 (5): 054001, February 2020. ISSN 0003-6951. 10.1063/1.5133894.
https://doi.org/10.1063/1.5133894

[43] Adam Bouland, Bill Fefferman, Chinmay Nirkhe, and Umesh Vazirani. On the complexity and verification of quantum random circuit sampling. Nat. Phys., 15 (2): 159–163, February 2019. ISSN 1745-2473, 1745-2481. 10.1038/s41567-018-0318-2.
https://doi.org/10.1038/s41567-018-0318-2

[44] Robin Blume-Kohout et al. Idle Tomography. In preparation, 2020.

[45] Seth Lloyd. Universal quantum simulators. Science, 273: 1073–1078, 1996. 10.1126/science.273.5278.1073.
https://doi.org/10.1126/science.273.5278.1073

Cited by

[1] René Zander and Colin Kai‐Uwe Becker, "Benchmarking Multipartite Entanglement Generation with Graph States", Advanced Quantum Technologies 8 1, 2400239 (2025).

[2] N. Paraskevopoulos, M. Steinberg, B. Undseth, A. Sarkar, L.M.K. Vandersypen, X. Xue, and S. Feld, "Near-Term Spin-Qubit Architecture Design via Multipartite Maximally Entangled States", PRX Quantum 6 2, 020307 (2025).

[3] Xingxin Liu, Timothy Proctor, Alaina M. Green, and Norbert M. Linke, Optica Quantum 2.0 Conference and Exhibition QW2A.2 (2025) ISBN:978-1-957171-48-7.

[4] Anthony Ransford, M. S. Allman, Jake Arkinstall, J. P. Campora, Samuel F. Cooper, Robert D. Delaney, Joan M. Dreiling, Brian Estey, Caroline Figgatt, Alex Hall, Ali A. Husain, Akhil Isanaka, Colin J. Kennedy, Nikhil Kotibhaskar, Ivaylo S. Madjarov, Karl Mayer, Alistair R. Milne, Annie J. Park, Adam P. Reed, Riley Ancona, Molly P. Andersen, Pablo Andres-Martinez, Will Angenent, Liz Argueta, Benjamin Arkin, Leonardo Ascarrunz, William Baker, Corey Barnes, John Bartolotta, Jordan Berg, Ryan Besand, Bryce Bjork, Matt Blain, Paul Blanchard, Robin Blume-Kohout, Matt Bohn, Agustín Borgna, Daniel Y. Botamanenko, Robert Boutelle, Natalie Brown, Grant T. Buckingham, Nathaniel Q. Burdick, William Cody Burton, Varis Carey, Christopher J. Carron, Joe Chambers, Jia Wen Chan, John Children, Victor E. Colussi, Steven Crepinsek, Andrew Cureton, Joe Davies, Daniel Davis, Matthew DeCross, David Deen, Conor Delaney, Davide DelVento, B. J. DeSalvo, Jason Dominy, Sydney Drotar, Ross Duncan, Vanya Eccles, Alec Edgington, Neal Erickson, Stephen Erickson, Christopher T. Ertsgaard, Jay Esposito, Bruce Evans, Tyler Evans, Maya I. Fabrikant, Andrew Fischer, Cameron Foltz, Michael Foss-Feig, David Francois, Brad Freyberg, Charles Gao, Robert Garay, Jane Garvin, David M. Gaudiosi, Christopher N. Gilbreth, Josh Giles, Erin Glynn, Jeff Graves, Azure Hansen, David Hayes, Lukas Heidemann, Bob Higashi, Tyler Hilbun, Jordan Hines, Ariana Hlavaty, Kyle Hoffman, Ian M. Hoffman, Craig Holliman, Isobel Hooper, Bob Horning, James Hostetter, Daniel Hothem, Jack Houlton, Jared Hout, Ross Hutson, Ryan T. Jacobs, Trent Jacobs, Melf Johannsen, Jacob Johansen, Loren Jones, Sydney Julian, Ryan Jung, Aidan Keay, Todd Klein, Mark Koch, Ryo Kondo, Chang Kong, Asa Kosto, Alan Lawrence, David Liefer, Michelle Lollie, Dominic Lucchetti, Nathan K. Lysne, Christian Lytle, Callum MacPherson, Andrew Malm, Spencer Mather, Brian Mathewson, Daniel Maxwell, Lauren McCaffrey, Hannah McDougall, Robin Mendoza, David B. Miller, Michael Mills, Richard Morrison, Louis Narmour, Nhung Nguyen, Lora Nugent, Scott Olson, Daniel Ouellette, Jeremy Parks, Zach Peters, Timothy A. Peterson, Jessie Petricka, Juan M. Pino, Frank Polito, Andrew C. Potter, Matthias Preidl, Gabriel Price, Timothy Proctor, McKinley Pugh, Noah Ratcliff, Daisy Raymondson, Peter Rhodes, Conrad Roman, Craig Roy, Ciaran Ryan-Anderson, Fernando Betanzo Sanchez, George Sangiolo, Tatiana Sawadski, Andrew Schaffer, Peter Schow, Jon Sedlacek, Henry Semenenko, Peter Shevchuk, Susan Shore, Peter Siegfried, Kartik Singhal, Seyon Sivarajah, Thomas Skripka, Lucas Sletten, Ben Spaun, R. Tucker Sprenkle, Paul Stoufer, Mariel Tader, Stephen F. Taylor, Travis H. Thompson, Raanan Tobey, Anh Tran, Tam Tran, Grahame Vittorini, Curtis Volin, Jim Walker, Sam White, Garrett R. Williams, Douglas Wilson, Quinn Wolf, Chester Wringe, Kevin Young, Jian Zheng, Kristen Zuraski, Charles H. Baldwin, Alex Chernoguzov, John P. Gaebler, Steven J. Sanders, Brian Neyenhuis, Russell Stutz, and Justin G. Bohnet, "A 98-qubit trapped-ion quantum computer with all-to-all connectivity", Nature 655 8121, 81 (2026).

[5] A. Barış Özgüler and Davide Venturelli, "Numerical gate synthesis for quantum heuristics on bosonic quantum processors", Frontiers in Physics 10, 900612 (2022).

[6] Paul D. Nation and Matthew Treinish, "Suppressing Quantum Circuit Errors Due to System Variability", PRX Quantum 4 1, 010327 (2023).

[7] Florian J. Kiwit, Maximilian A. Wolf, Marwa Marso, Philipp Ross, Jeanette M. Lorenz, Carlos A. Riofrío, and Andre Luckow, "Benchmarking Quantum Generative Learning: A Study on Scalability and Noise Resilience using QUARK", KI - Künstliche Intelligenz 38 4, 379 (2024).

[8] Valentin Gilbert, Stéphane Louise, and Renaud Sirdey, Lecture Notes in Computer Science 14077, 168 (2023) ISBN:978-3-031-36029-9.

[9] Timothy Proctor, Kevin Young, Andrew D. Baczewski, and Robin Blume-Kohout, "Benchmarking quantum computers", Nature Reviews Physics 7 2, 105 (2025).

[10] Tim Weaving, Alexis Ralli, Peter J. Love, Sauro Succi, and Peter V. Coveney, "Accurately Simulating the Time Evolution of an Ising Model with Echo Verified Clifford Data Regression on a Superconducting Quantum Computer", Quantum 9, 1732 (2025).

[11] Eduard Alarcón, Sergi Abadal, Fabio Sebastiano, Masoud Babaie, Edoardo Charbon, Peter Haring Bolívar, Maurizio Palesi, Elena Blokhina, Dirk Leipold, Bogdan Staszewski, Artur Garcia-Sáez, and Carmen G. Almudever, 2023 IEEE International Symposium on Circuits and Systems (ISCAS) 1 (2023) ISBN:978-1-6654-5109-3.

[12] Junchao Wang, Guoping Guo, and Zheng Shan, "SoK: Benchmarking the Performance of a Quantum Computer", Entropy 24 10, 1467 (2022).

[13] Konrad Wojciechowski, Krzysztof Kurowski, Mateusz Slysz, and Tomasz Pecyna, "Open QBench: A benchmarking framework for evaluating quantum computing platforms", Quantum Information Processing 25 3, 98 (2026).

[14] Siyuan Niu, Efekan Kökcü, Sonika Johri, Anurag Ramesh, Avimita Chatterjee, David E. Bernal Neira, Daan Camps, and Thomas Lubinski, 2025 IEEE International Conference on Quantum Computing and Engineering (QCE) 375 (2025) ISBN:979-8-3315-5736-2.

[15] Nishant Saurabh, Pradeep Mantha, Florian J. Kiwit, Shantenu Jha, and Andre Luckow, Proceedings of the 2024 Workshop on High Performance and Quantum Computing Integration 11 (2024) ISBN:9798400706431.

[16] Akel Hashim, Long B. Nguyen, Noah Goss, Brian Marinelli, Ravi K. Naik, Trevor Chistolini, Jordan Hines, J.P. Marceaux, Yosep Kim, Pranav Gokhale, Teague Tomesh, Senrui Chen, Liang Jiang, Samuele Ferracin, Kenneth Rudinger, Timothy Proctor, Kevin C. Young, Irfan Siddiqi, and Robin Blume-Kohout, "Practical Introduction to Benchmarking and Characterization of Quantum Computers", PRX Quantum 6 3, 030202 (2025).

[17] Jwo-Sy Chen, Erik Nielsen, Matthew Ebert, Volkan Inlek, Kenneth Wright, Vandiver Chaplin, Andrii Maksymov, Eduardo Páez, Amrit Poudel, Peter Maunz, and John Gamble, "Benchmarking a trapped-ion quantum computer with 30 qubits", Quantum 8, 1516 (2024).

[18] "Performance evaluation and control improvements for solving optimization problems on NISQ platforms", université Paris-Saclay (2024).

[19] Alex Krasnok, "What quantum computer to buy?", AVS Quantum Science 8 3, 034101 (2026).

[20] Rafał Bistroń, Marcin Rudziński, Ryszard Kukulski, and Karol Życzkowski, "Benchmarking quantum devices beyond classical capabilities", Physical Review A 113 6, 062604 (2026).

[21] Vedika Saravanan and Samah Mohamed Saeed, 2022 IEEE 40th International Conference on Computer Design (ICCD) 1 (2022) ISBN:978-1-6654-6186-3.

[22] Davide Ferrari and Michele Amoretti, Proceedings of the 19th ACM International Conference on Computing Frontiers 237 (2022) ISBN:9781450393386.

[23] E. Wes Bethel, Roel Van Beeumen, and Talita Perciano, "Quantum Computing and Visualization Research Challenges and Opportunities", IEEE Computer Graphics and Applications 46 2, 120 (2026).

[24] Pranit Malhotra, Ajay Kumar, and Sunita Garhwal, "A Systematic Review of Quantum BenchMarking", International Journal of Theoretical Physics 63 11, 278 (2024).

[25] Kübra Yeter‐Aydeniz, Bryan T. Gard, Jacek Jakowski, Swarnadeep Majumder, George S. Barron, George Siopsis, Travis S. Humble, and Raphael C. Pooser, "Benchmarking Quantum Chemistry Computations with Variational, Imaginary Time Evolution, and Krylov Space Solver Algorithms", Advanced Quantum Technologies 4 7, 2100012 (2021).

[26] Hyunho Cha, Sangwoo Hong, and Jungwoo Lee, "Operator-aware shadow importance sampling for accurate fidelity estimation", Physical Review A 113 4, 042602 (2026).

[27] Timothy Proctor, Stefan Seritan, Kenneth Rudinger, Erik Nielsen, Robin Blume-Kohout, and Kevin Young, "Scalable Randomized Benchmarking of Quantum Computers Using Mirror Circuits", Physical Review Letters 129 15, 150502 (2022).

[28] Arturo Acuaviva, David Aguirre, Rubén Peña, and Mikel Sanz, "Benchmarking quantum computers: towards a standard performance evaluation approach", Quantum Science and Technology 11 2, 025004 (2026).

[29] Ibrahim H. Osman and Soha Maad, Lecture Notes in Operations Research 503 (2026) ISBN:978-3-032-23492-6.

[30] "Circuit compilation and error mitigation for near-term quantum computing", Université de Montpellier (2022-....) (2022).

[31] Tirthak Patel and Devesh Tiwari, Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems 443 (2021) ISBN:9781450383172.

[32] R Au-Yeung, B Camino, O Rathore, and V Kendon, "Quantum algorithms for scientific computing", Reports on Progress in Physics 87 11, 116001 (2024).

[33] Timothy Proctor, Kenneth Rudinger, Kevin Young, Erik Nielsen, and Robin Blume-Kohout, "Measuring the capabilities of quantum computers", Nature Physics 18 1, 75 (2022).

[34] Y.F. Zolotarev, I.A. Luchnikov, J.A. López-Saldívar, A.K. Fedorov, and E.O. Kiktenko, "Continuous Monitoring for Noisy Intermediate-Scale Quantum Processors", Physical Review Applied 19 1, 014027 (2023).

[35] Jeonghoon Lee, Hyeongjun Jeon, and Taehyun Kim, "Ion-trap chip architecture optimized for the implementation of quantum error-correcting codes", Physical Review A 113 3, 032432 (2026).

[36] Elijah Pelofske, Andreas Bärtschi, and Stephan Eidenbenz, "Quantum Volume in Practice: What Users Can Expect From NISQ Devices", IEEE Transactions on Quantum Engineering 3, 1 (2022).

[37] Garrelt J. N. Alberts, M. Adriaan Rol, Thorsten Last, Benno W. Broer, Cornelis C. Bultink, Matthijs S. C. Rijlaarsdam, and Amber E. Van Hauwermeiren, "Accelerating quantum computer developments", EPJ Quantum Technology 8 1, 18 (2021).

[38] Andreas J. C. Woitzik, Lukas Hoffmann, Andreas Buchleitner, and Edoardo G. Carnio, "An Energy Estimation Benchmark for Quantum Computing Hardware", Open Systems & Information Dynamics 31 01, 2450006 (2024).

[39] Colin Kai-Uwe Becker, Nikolay Tcholtchev, Ilie-Daniel Gheorghe-Pop, Sebastian Bock, Raphael Seidel, and Manfred Hauswirth, 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C) 160 (2022) ISBN:978-1-6654-9493-9.

[40] Georgios Ioannou, Gopika Kizhuvettil, Mohammad Walid Charrwi, and Samah Mohamed Saeed, 2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) 607 (2024) ISBN:979-8-3503-5411-9.

[41] Medina Bandic, Carmen G. Almudever, and Sebastian Feld, "Interaction graph-based characterization of quantum benchmarks for improving quantum circuit mapping techniques", Quantum Machine Intelligence 5 2, 40 (2023).

[42] Tom Weber, Kerstin Borras, Karl Jansen, Dirk Krücker, and Matthias Riebisch, "Construction and volumetric benchmarking of quantum computing noise models", Physica Scripta 99 6, 065106 (2024).

[43] Jernej Rudi Finzgar, Philipp Ross, Leonhard Holscher, Johannes Klepsch, and Andre Luckow, 2022 IEEE International Conference on Quantum Computing and Engineering (QCE) 226 (2022) ISBN:978-1-6654-9113-6.

[44] Kathleen Hamilton, Titus Morris, Raphael Pooser, Kübra Yeter-Aydeniz, Luning Zhao, Nouamane Laanait, Harrison Cooley, Muhun Kang, George Barron, Sophia Economou, Akhil Francis, and Alexander F. Kemper, 2023 IEEE International Conference on Quantum Computing and Engineering (QCE) 703 (2023) ISBN:979-8-3503-4323-6.

[45] Priyabrata Senapati, Waylon Luo, Bo Peng, and Qiang Guan, Proceedings of the Great Lakes Symposium on VLSI 2026 830 (2026) ISBN:9798400724312.

[46] Nicholas Bornman, Tanay Roy, Joshua A Job, Namit Anand, Gabriel N Perdue, Silvia Zorzetti, and M Sohaib Alam, "Benchmarking the performance of a high-Q cavity qudit using random unitaries", Quantum Science and Technology 10 2, 025062 (2025).

[47] Chisomo Daka and Somnath Bhattacharyya, "NISQ in practice: navigating noise, scale, and hardware constraints of near-term devices in quantum machine learning workflows", New Journal of Physics 28 8, 081201 (2026).

[48] Linghang Kong, Fang Zhang, and Jianxin Chen, "Benchmarking fault-tolerant quantum computing hardware via QLOPS", ACM Transactions on Quantum Computing 7 2, 1 (2026).

[49] Daniel Mills, Seyon Sivarajah, Travis L. Scholten, and Ross Duncan, "Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack", Quantum 5, 415 (2021).

[50] Xiaonan Liu, Ming He, Junchao Wang, Haoshan Xie, and Chenyan Zhao, "Automated Quantum Volume Test", Journal of Physics: Conference Series 2221 1, 012029 (2022).

[51] Jordan Hines and Timothy Proctor, "Scalable Full-Stack Benchmarks for Quantum Computers", IEEE Transactions on Quantum Engineering 5, 1 (2024).

[52] Benjamin A. Cordier, Nicolas P. D. Sawaya, Gian Giacomo Guerreschi, and Shannon K. McWeeney, "Biology and medicine in the landscape of quantum advantages", Journal of The Royal Society Interface 19 196, 20220541 (2022).

[53] Daniel Hothem, Kevin Young, Tommie Catanach, and Timothy Proctor, "Learning a Quantum Computer's Capability", IEEE Transactions on Quantum Engineering 5, 1 (2024).

[54] Andrea Mari, "Counting collisions in random circuit sampling for benchmarking quantum computers", APL Quantum 1 4, 046113 (2024).

[55] Bryan T. Gard and Adam M. Meier, "Classically efficient quantum scalable Fermi-Hubbard benchmark", Physical Review A 105 4, 042602 (2022).

[56] Sahar Ben Rached, Isaac Lopez Agudo, Santiago Rodrigo, Medina Bandic, Artur Garcia-Saez, Sebastian Feld, Hans van Someren, Eduard Alarcón, Carmen G. Almudéver, and Sergi Abadal, "Characterizing the Inter-Core Qubit Traffic in Large-Scale Quantum Modular Architectures", IEEE Access 13, 113236 (2025).

[57] Kazunobu Maruyoshi, Takuya Okuda, Juan W Pedersen, Ryo Suzuki, Masahito Yamazaki, and Yutaka Yoshida, "Conserved charges in the quantum simulation of integrable spin chains", Journal of Physics A: Mathematical and Theoretical 56 16, 165301 (2023).

[58] S. P. Kulik, "Quantum Computing: Predictions and Challenges", Bulletin of the Lebedev Physics Institute 50 S12, S1330 (2023).

[59] Oleksa Hryniv, Vladyslav Los, Yuriy Pryyma, Dmitri Iouchtchenko, Maciej Koch-Janusz, and Mykola Maksymenko, 2024 IEEE International Conference on Quantum Computing and Engineering (QCE) 16 (2024) ISBN:979-8-3315-4137-8.

[60] Tiffany Khou and Don Roosan, Quantum Computing in Medicine 343 (2026) ISBN:9780443341939.

[61] Junjie Zhu, Filippo Orazi, Sara Marzella, Daniele Ottaviani, and Stefano Lodi, 2025 IEEE International Conference on Quantum Artificial Intelligence (QAI) 241 (2025) ISBN:979-8-3315-6986-0.

[62] Sebastian Feld, Communications in Computer and Information Science 2513, 3 (2025) ISBN:978-3-031-94262-4.

[63] Daniel Hothem, Jordan Hines, Karthik Nataraj, Robin Blume-Kohout, and Timothy Proctor, 2023 IEEE International Conference on Quantum Computing and Engineering (QCE) 709 (2023) ISBN:979-8-3503-4323-6.

[64] M. Morgado and S. Whitlock, "Quantum simulation and computing with Rydberg-interacting qubits", AVS Quantum Science 3 2, 023501 (2021).

[65] Nathan D. Jansen, Matthew Loucks, Scott Gilbert, Corbin Fleming-Dittenber, Julia Egbert, and Katharine L. C. Hunt, "Shannon and von Neumann entropies of multi-qubit Schrödinger's cat states", Physical Chemistry Chemical Physics 24 13, 7666 (2022).

[66] Avimita Chatterjee, Sonny Rappaport, Anish Giri, Sonika Johri, Timothy Proctor, David E. Bernal Neira, Pratik Sathe, and Thomas Lubinski, "A Comprehensive Cross-Model Framework for Benchmarking the Performance of Quantum Hamiltonian Simulations", IEEE Transactions on Quantum Engineering 6, 1 (2025).

[67] Thomas Lubinski, Sonika Johri, Paul Varosy, Jeremiah Coleman, Luning Zhao, Jason Necaise, Charles H. Baldwin, Karl Mayer, and Timothy Proctor, "Application-Oriented Performance Benchmarks for Quantum Computing", IEEE Transactions on Quantum Engineering 4, 1 (2023).

[68] Michele Amoretti, "An Effective Framework for Full-Stack Benchmarking of Quantum Computers", Quantum Views 5, 52 (2021).

[69] Tuomas Laakkonen, Enrico Rinaldi, Chris N. Self, Eli Chertkov, Matthew DeCross, David Hayes, Brian Neyenhuis, Marcello Benedetti, and Konstantinos Meichanetzidis, "End-to-End Quantum Algorithms for the Jones Polynomial", PRX Quantum 7 2, 020355 (2026).

[70] Tiffany Khou and Christopher Stuetzle, Quantum Computing in Medicine 71 (2026) ISBN:9780443341939.

[71] Nils Herrmann, Daanish Arya, Marcus W. Doherty, Angus Mingare, Jason C. Pillay, Florian Preis, and Stefan Prestel, 2023 IEEE International Conference on Quantum Software (QSW) 162 (2023) ISBN:979-8-3503-0479-4.

[72] Daniel Stilck França, Sergii Strelchuk, and Michał Studziński, "Efficient Classical Simulation and Benchmarking of Quantum Processes in the Weyl Basis", Physical Review Letters 126 21, 210502 (2021).

[73] Medina Bandic, Luise Prielinger, Jonas Nüßlein, Anabel Ovide, Santiago Rodrigo, Sergi Abadal, Hans van Someren, Gayane Vardoyan, Eduard Alarcon, Carmen G. Almudever, and Sebastian Feld, 2023 IEEE International Conference on Quantum Computing and Engineering (QCE) 790 (2023) ISBN:979-8-3503-4323-6.

[74] Medina Bandic, Pablo le Henaff, Anabel Ovide, Pau Escofet, Sahar Ben Rached, Santiago Rodrigo, Hans van Someren, Sergi Abadal, Eduard Alarcón, Carmen G Almudever, and Sebastian Feld, "Profiling quantum circuits for their efficient execution on single- and multi-core architectures", Quantum Science and Technology 10 1, 015060 (2025).

[75] W. Michael Brown, Anurag Ramesh, Thomas Lubinski, Thien Nguyen, and David E. Bernal Neira, "Multi-GPU quantum circuit simulation and the impact of network performance", Computer Physics Communications 324, 110126 (2026).

[76] Thomas Lubinski, Carleton Coffrin, Catherine McGeoch, Pratik Sathe, Joshua Apanavicius, David Bernal Neira, and Quantum Economic Development Consortium(QED-C) Collaboration, "Optimization Applications as Quantum Performance Benchmarks", ACM Transactions on Quantum Computing 5 3, 1 (2024).

[77] Elise Jennings, Martin Rüfenacht, and Stefan Kister, Lecture Notes in Computer Science 15385, 16 (2025) ISBN:978-3-031-90199-7.

[78] Gilles Buchs, Thomas L. Beck, Ryan S. Bennink, Daniel Claudino, Andrea Delgado, Nur Aiman Fadel, Peter Groszkowski, Kathleen E. Hamilton, Travis S. Humble, Neeraj Kumar, Ang Li, Phillip C. Lotshaw, Olli Mukkula, Ryousei Takano, Amit Saxena, In-Saeng Suh, Miwako Tsuji, Roel Van Beeumen, Ugo Varetto, Yan Wang, Kazuya Yamazaki, and Mikael P. Johansson, "The role of quantum computing in advancing scientific high-performance computing: A perspective from the ADAC institute", Future Generation Computer Systems 182, 108487 (2026).

[79] Carlos A. Riofrío, Johannes Klepsch, Jernej Rudi Finžgar, Florian Kiwit, Leonhard Hölscher, Marvin Erdmann, Lukas Müller, Chandan Kumar, Youssef Achari Berrada, and Andre Luckow, Quantum Computational AI 181 (2026) ISBN:9780443302596.

[80] Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S. Kottmann, Tim Menke, Wai-Keong Mok, Sukin Sim, Leong-Chuan Kwek, and Alán Aspuru-Guzik, "Noisy intermediate-scale quantum algorithms", Reviews of Modern Physics 94 1, 015004 (2022).

[81] Erik Nielsen, John King Gamble, Kenneth Rudinger, Travis Scholten, Kevin Young, and Robin Blume-Kohout, "Gate Set Tomography", Quantum 5, 557 (2021).

[82] Simon Martiel, Thomas Ayral, and Cyril Allouche, "Benchmarking Quantum Coprocessors in an Application-Centric, Hardware-Agnostic, and Scalable Way", IEEE Transactions on Quantum Engineering 2, 1 (2021).

[83] Cristina Cirstoiu, Silas Dilkes, Daniel Mills, Seyon Sivarajah, and Ross Duncan, "Volumetric Benchmarking of Error Mitigation with Qermit", Quantum 7, 1059 (2023).

[84] Seyon Sivarajah, Silas Dilkes, Alexander Cowtan, Will Simmons, Alec Edgington, and Ross Duncan, "t|ket⟩: a retargetable compiler for NISQ devices", Quantum Science and Technology 6 1, 014003 (2021).

[85] Antonio D. Corcoles, Abhinav Kandala, Ali Javadi-Abhari, Douglas T. McClure, Andrew W. Cross, Kristan Temme, Paul D. Nation, Matthias Steffen, and Jay M. Gambetta, "Challenges and Opportunities of Near-Term Quantum Computing Systems", IEEE Proceedings 108 8, 1338 (2020).

[86] Arnaud Carignan-Dugas, Dar Dahlen, Ian Hincks, Egor Ospadov, Stefanie J. Beale, Samuele Ferracin, Joshua Skanes-Norman, Joseph Emerson, and Joel J. Wallman, "The Error Reconstruction and Compiled Calibration of Quantum Computing Cycles", arXiv:2303.17714, (2023).

[87] Peter J. Karalekas, Nikolas A. Tezak, Eric C. Peterson, Colm A. Ryan, Marcus P. da Silva, and Robert S. Smith, "A quantum-classical cloud platform optimized for variational hybrid algorithms", Quantum Science and Technology 5 2, 024003 (2020).

[88] Erik Nielsen, Kenneth Rudinger, Timothy Proctor, Antonio Russo, Kevin Young, and Robin Blume-Kohout, "Probing quantum processor performance with pyGSTi", Quantum Science and Technology 5 4, 044002 (2020).

[89] Carlos A. Riofr\'io, Johannes Klepsch, Jernej Rudi Fin\v{z}gar, Florian Kiwit, Leonhard H\"olscher, Marvin Erdmann, Lukas M\"uller, Chandan Kumar, Youssef Achari Berrada, and Andre Luckow, "Quantum Computing for Automotive Applications", arXiv:2409.14183, (2024).

[90] Erik Gustafson, Patrick Dreher, Zheyue Hang, and Yannick Meurice, "Indexed improvements for real-time trotter evolution of a (1 + 1) field theory using NISQ quantum computers", Quantum Science and Technology 6 4, 045020 (2021).

[91] Alba Cervera-Lierta, José Ignacio Latorre, and Dardo Goyeneche, "Quantum circuits for maximally entangled states", Physical Review A 100 2, 022342 (2019).

[92] Karl Mayer, Alex Hall, Thomas Gatterman, Si Khadir Halit, Kenny Lee, Justin Bohnet, Dan Gresh, Aaron Hankin, Kevin Gilmore, Justin Gerber, and John Gaebler, "Theory of mirror benchmarking and demonstration on a quantum computer", arXiv:2108.10431, (2021).

[93] Benjamin A. Cordier, Nicolas P. D. Sawaya, Gian G. Guerreschi, and Shannon K. McWeeney, "Biology and medicine in the landscape of quantum advantages", arXiv:2112.00760, (2021).

[94] Timothy Proctor, Kenneth Rudinger, Kevin Young, Erik Nielsen, and Robin Blume-Kohout, "Measuring the Capabilities of Quantum Computers", arXiv:2008.11294, (2020).

[95] Henry L. Nourse, Vanessa C. Olaya-Agudelo, and Ivan Kassal, "Using bosons to improve resource efficiency of quantum simulation of vibronic molecular dynamics", arXiv:2512.20828, (2025).

[96] Salonik Resch and Ulya R. Karpuzcu, "Benchmarking Quantum Computers and the Impact of Quantum Noise", arXiv:1912.00546, (2019).

[97] Tobias Rohe, Federico Harjes Ruiloba, Sabrina Egger, Sebastian von Beck, Jonas Stein, and Claudia Linnhoff-Popien, "Quantum Computer Benchmarking: An Explorative Systematic Literature Review", arXiv:2509.03078, (2025).

[98] Julien Gacon, "Scalable Quantum Algorithms for Noisy Quantum Computers", arXiv:2403.00940, (2024).

[99] Brandon Rodenburg, "Estimating the Power of a Quantum Computer", arXiv:2502.00113, (2025).

[100] Florian J. Kiwit, Maximilian A. Wolf, Marwa Marso, Philipp Ross, Jeanette M. Lorenz, Carlos A. Riofrío, and Andre Luckow, "Benchmarking Quantum Generative Learning: A Study on Scalability and Noise Resilience using QUARK", arXiv:2403.18662, (2024).

[101] Attila Portik, Orsolya Kálmán, Thomas Monz, and Zoltán Zimborás, "Clifford Volume and Free Fermion Volume: Complementary Scalable Benchmarks for Quantum Computers", arXiv:2512.19413, (2025).

[102] Vincenzo Sammartino, "Feature Encoding in Quantum Machine Learning: A Survey and Practical Guidelines", arXiv:2606.05387, (2026).

[103] Travis L. Scholten, Yi-Kai Liu, Kevin Young, and Robin Blume-Kohout, "Classifying single-qubit noise using machine learning", arXiv:1908.11762, (2019).

[104] E. Wes Bethel, Roel Van Beeumen, and Talita Perciano, "Quantum Computing and Visualization Research Challenges and Opportunities", arXiv:2601.07872, (2026).

[105] Priyabrata Senapati, Vibin Abraham, Qiang Guan, and Bo Peng, "Unified Uncertainty Quantification Framework Bridging Noisy Quantum Backends Across Variational Quantum Algorithms and Quantum Signal Processing", arXiv:2607.14392, (2026).

[106] Akash Kundu, Amit Kumar Jaiswal, Sebastian Feld, and Prayag Tiwari, "Generative Replay Mitigates Sample Starvation in Quantum Architecture Search", arXiv:2609.11248, (2026).

[107] Jiachen Shen and Hui Zhong, "Capability-Gated Conformance Testing of Quantum Error-Correction Decoder Libraries", arXiv:2609.07035, (2026).

[108] Noé Olivier and Michel Nowak, "Quantum WalkScore: Benchmarking Quantum Computers on the Graph Nodefinding Problem", arXiv:2609.19931, (2026).

[109] Yi-Ting Chen, Aniket S. Dalvi, and Nils Quetschlich, "Resource-Optimal Screening for Two-Tier Quantum System Characterization", arXiv:2610.34003, (2026).

[110] Badhon Rahman, Majid Haghparast, and Tommi Mikkonen, "On the Figures of Merit for Quantum Software Security: Toward a Benchmarking Rubric", arXiv:2608.05831, (2026).

[111] Hsiang-Wei Huang, Kuo-Feng Chiu, Yi-Te Huang, Jhen-Dong Lin, Tse-Ming Chen, and Yueh-Nan Chen, "Detecting basis-dependent hardware errors through spatio-temporal quantum steering", arXiv:2606.16451, (2026).

[112] Isaac Barouch Essayag and Aryeh Lev Zabokritskiy, "From Round-Trip State Echo to Error Recovery: Snapshot-Resolved Quantum-Hardware Diagnostics", arXiv:2608.26010, (2026).

The above citations are from Crossref's cited-by service (last updated successfully 2026-11-06 14:59:52) and SAO/NASA ADS (last updated successfully 2026-11-06 14:59:54). The list may be incomplete as not all publishers provide suitable and complete citation data.

This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.