Operational meanings of a generalized conditional expectation in quantum metrology

Mankei Tsang

Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583
Department of Physics, National University of Singapore, 2 Science Drive 3, Singapore 117551

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.


A unifying formalism of generalized conditional expectations (GCEs) for quantum mechanics has recently emerged, but its physical implications regarding the retrodiction of a quantum observable remain controversial. To address the controversy, here I offer operational meanings for a version of the GCEs in the context of quantum parameter estimation. When a quantum sensor is corrupted by decoherence, the GCE is found to relate the operator-valued optimal estimators before and after the decoherence. Furthermore, the error increase, or regret, caused by the decoherence is shown to be equal to a divergence between the two estimators. The real weak value as a special case of the GCE plays the same role in suboptimal estimation – its divergence from the optimal estimator is precisely the regret for not using the optimal measurement. For an application of the GCE, I show that it enables the use of dynamic programming for designing a controller that minimizes the estimation error. For the frequentist setting, I show that the GCE leads to a quantum Rao-Blackwell theorem, which offers significant implications for quantum metrology and thermal-light sensing in particular. These results give the GCE and the associated divergence a natural, useful, and incontrovertible role in quantum decision and control theory.

The conditional expectation is an essential concept in probability theory and a basic tool in data analysis, as it allows one to infer hidden variables from observations in an optimal sense. Many have tried to generalize the concept for quantum mechanics, but it remains questionable whether such quantum conditional expectations are meaningful or useful. This work shows that a certain generalized conditional expectation can indeed be useful for quantum sensor design, thanks to its convenient mathematical properties.

► BibTeX data

► References

[1] K. R. Parthasarathy. Introduction to Probability and Measure. Hindustan Book Agency, New Delhi, 2005. ISBN 9789386279279. 10.1007/​978-93-86279-27-9. URL https:/​/​www.springer.com/​gp/​book/​9789386279279.

[2] James O. Berger. Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, 1985. 10.1007/​978-1-4757-4286-2.

[3] C. Radhakrishna Rao. Information and accuracy attainable in the estimation of statistical parameters. Bulletin of the Calcutta Mathematical Society, 37: 81–91, 1945.

[4] David Blackwell. Conditional expectation and unbiased sequential estimation. Annals of Mathematical Statistics, 18 (1): 105–110, March 1947. ISSN 0003-4851. 10.1214/​aoms/​1177730497.

[5] E. L. Lehmann and George Casella. Theory of Point Estimation. Springer, New York, second edition, 1998. ISBN 978-0-387-22728-3. 10.1007/​b98854.

[6] Steven K. Thompson. Sampling. John Wiley & Sons, Ltd., Chichester, England, UK, February 2012. ISBN 978-0-47040231-3. 10.1002/​9781118162934.

[7] Christian P. Robert and George Casella. Monte Carlo Statistical Methods. Springer, New York, NY, USA, 2004. ISBN 978-1-4757-4145-2. 10.1007/​978-1-4757-4145-2.

[8] Hisaharu Umegaki. Conditional expectation in an operator algebra. Tohoku Mathematical Journal, Second Series, 6 (2-3): 177–181, 1954. ISSN 0040-8735. 10.2748/​tmj/​1178245177.

[9] S. Personick. Application of quantum estimation theory to analog communication over quantum channels. IEEE Transactions on Information Theory, 17 (3): 240–246, May 1971. ISSN 0018-9448. 10.1109/​TIT.1971.1054643.

[10] V. P. Belavkin and B. A. Grishanin. Optimum estimation in quantum channels by the generalized Heisenberg inequality method. Probl. Peredachi Inf., 9: 44–52, 1973. URL http:/​/​mi.mathnet.ru/​eng/​ppi907.

[11] Luigi Accardi and Carlo Cecchini. Conditional expectations in von Neumann algebras and a theorem of Takesaki. Journal of Functional Analysis, 45 (2): 245–273, February 1982. ISSN 0022-1236. 10.1016/​0022-1236(82)90022-2. URL https:/​/​www.sciencedirect.com/​science/​article/​pii/​0022123682900222.

[12] Dénes Petz. Quantum Information Theory and Quantum Statistics. Springer, Berlin, Germany, 2010. ISBN 978-3-540-74636-2. 10.1007/​978-3-540-74636-2. URL https:/​/​link.springer.com/​book/​10.1007/​978-3-540-74636-2.

[13] Yakir Aharonov, David Z. Albert, and Lev Vaidman. How the result of a measurement of a component of the spin of a spin-1/​2 particle can turn out to be 100. Physical Review Letters, 60: 1351–1354, Apr 1988. 10.1103/​PhysRevLett.60.1351. URL http:/​/​doi.org/​10.1103/​PhysRevLett.60.1351.

[14] Justin Dressel. Weak values as interference phenomena. Physical Review A, 91 (3): 032116, March 2015. 10.1103/​PhysRevA.91.032116. URL https:/​/​doi.org/​10.1103/​PhysRevA.91.032116.

[15] Masahito Hayashi. Quantum Information Theory: Mathematical Foundation. Springer, Berlin, second edition, 2017. 10.1007/​978-3-662-49725-8.

[16] Kentaro Ohki. A smoothing theory for open quantum systems: The least mean square approach. In 2015 54th IEEE Conference on Decision and Control (CDC), page 4350–4355, Osaka, Japan, December 2015. IEEE. 10.1109/​CDC.2015.7402898.

[17] Kentaro Ohki. An invitation to quantum filtering and smoothing theory based on two inner products. In 2018 Building Foundations for Quantum Statistical Modeling, volume 2018, page 18–44. Kyoto University, Kyoto, 2018. URL http:/​/​hdl.handle.net/​2433/​231710.

[18] Mankei Tsang. Generalized conditional expectations for quantum retrodiction and smoothing. Physical Review A, 105 (4): 042213, April 2022. ISSN 2469-9934. 10.1103/​PhysRevA.105.042213.

[19] Alexander S. Holevo. Statistical Structure of Quantum Theory. Springer-Verlag, Berlin, 2001. 10.1007/​3-540-44998-1.

[20] Viacheslav P. Belavkin. Nondemolition principle of quantum measurement theory. Foundations of Physics, 24 (5): 685–714, 1994. ISSN 0015-9018. 10.1007/​BF02054669. URL http:/​/​dx.doi.org/​10.1007/​BF02054669.

[21] L. Bouten, R. Van Handel, and M. James. An introduction to quantum filtering. SIAM Journal on Control and Optimization, 46 (6): 2199–2241, 2007. 10.1137/​060651239. URL http:/​/​epubs.siam.org/​doi/​abs/​10.1137/​060651239.

[22] Howard M. Wiseman and Gerard J. Milburn. Quantum Measurement and Control. Cambridge University Press, Cambridge, 2010. 10.1017/​CBO9780511813948.

[23] Stephen M. Barnett, David T. Pegg, and John Jeffers. Bayes' theorem and quantum retrodiction. Journal of Modern Optics, 47 (11): 1779–1789, September 2000. ISSN 0950-0340. 10.1080/​09500340008232431. URL https:/​/​www.tandfonline.com/​doi/​abs/​10.1080/​09500340008232431.

[24] Stephen M. Barnett, John Jeffers, and David T. Pegg. Quantum Retrodiction: Foundations and Controversies. Symmetry, 13 (4): 586, April 2021. 10.3390/​sym13040586. URL https:/​/​www.mdpi.com/​2073-8994/​13/​4/​586.

[25] Masahiro Yanagisawa. Quantum smoothing. arXiv:0711.3885 [quant-ph], December 2007. 10.48550/​arXiv.0711.3885. URL http:/​/​arxiv.org/​abs/​0711.3885.

[26] Mankei Tsang. Time-symmetric quantum theory of smoothing. Physical Review Letters, 102: 250403, Jun 2009a. 10.1103/​PhysRevLett.102.250403. URL http:/​/​doi.org/​10.1103/​PhysRevLett.102.250403.

[27] Mankei Tsang. Optimal waveform estimation for classical and quantum systems via time-symmetric smoothing. Physical Review A, 80: 033840, Sep 2009b. 10.1103/​PhysRevA.80.033840. URL http:/​/​doi.org/​10.1103/​PhysRevA.80.033840.

[28] Mankei Tsang. Optimal waveform estimation for classical and quantum systems via time-symmetric smoothing. II. Applications to atomic magnetometry and Hardy's paradox. Physical Review A, 81: 013824, Jan 2010. 10.1103/​PhysRevA.81.013824. URL http:/​/​doi.org/​10.1103/​PhysRevA.81.013824.

[29] Ivonne Guevara and Howard Wiseman. Quantum state smoothing. Physical Review Letters, 115: 180407, Oct 2015. 10.1103/​PhysRevLett.115.180407. URL http:/​/​doi.org/​10.1103/​PhysRevLett.115.180407.

[30] Areeya Chantasri, Ivonne Guevara, Kiarn T. Laverick, and Howard M. Wiseman. Unifying theory of quantum state estimation using past and future information. Physics Reports, 930: 1–40, October 2021. ISSN 0370-1573. 10.1016/​j.physrep.2021.07.003. URL https:/​/​www.sciencedirect.com/​science/​article/​pii/​S0370157321002787.

[31] M. S. Leifer and Robert W. Spekkens. Towards a formulation of quantum theory as a causally neutral theory of Bayesian inference. Physical Review A, 88 (5): 052130, November 2013. 10.1103/​PhysRevA.88.052130. URL https:/​/​doi.org/​10.1103/​PhysRevA.88.052130.

[32] Dominic Horsman, Chris Heunen, Matthew F. Pusey, Jonathan Barrett, and Robert W. Spekkens. Can a quantum state over time resemble a quantum state at a single time? Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473 (2205): 20170395, 2017. 10.1098/​rspa.2017.0395. URL https:/​/​royalsocietypublishing.org/​doi/​abs/​10.1098/​rspa.2017.0395.

[33] Arthur J. Parzygnat and James Fullwood. From time-reversal symmetry to quantum bayes' rules. PRX Quantum, 4 (2): 020334, June 2023. ISSN 2691-3399. 10.1103/​PRXQuantum.4.020334.

[34] Christopher Ferrie and Joshua Combes. Weak value amplification is suboptimal for estimation and detection. Physical Review Letters, 112: 040406, Jan 2014a. 10.1103/​PhysRevLett.112.040406. URL http:/​/​doi.org/​10.1103/​PhysRevLett.112.040406.

[35] Christopher Ferrie and Joshua Combes. How the result of a single coin toss can turn out to be 100 heads. Physical Review Letters, 113: 120404, Sep 2014b. 10.1103/​PhysRevLett.113.120404. URL http:/​/​doi.org/​10.1103/​PhysRevLett.113.120404.

[36] Lev Vaidman. Comment on "how the result of a single coin toss can turn out to be 100 heads". arXiv:1409.5396, September 2014. 10.48550/​arXiv.1409.5386.

[37] Andrew N. Jordan, Julián Martínez-Rincón, and John C. Howell. Technical advantages for weak-value amplification: When less is more. Physical Review X, 4 (1): 011031, March 2014. ISSN 2160-3308. 10.1103/​PhysRevX.4.011031.

[38] John E. Gough. How to estimate past measurement interventions on a quantum system undergoing continuous monitoring. arXiv:1904.06364v1 [quant-ph], April 2019. 10.48550/​arXiv.1904.06364. URL http:/​/​arxiv.org/​abs/​1904.06364v1.

[39] John E. Gough. How to Estimate Past Quantum Measurement Interventions After Continuous Monitoring. Russian Journal of Mathematical Physics, 27 (2): 218–227, April 2020. ISSN 1555-6638. 10.1134/​S1061920820020089. URL https:/​/​doi.org/​10.1134/​S1061920820020089.

[40] M. R. James. Optimal quantum control theory. Annual Review of Control, Robotics, and Autonomous Systems, 4 (1): 343–367, May 2021. ISSN 2573-5144. 10.1146/​annurev-control-061520-010444.

[41] Dimitri P. Bertsekas. Dynamic Programming and Optimal Control, volume I. Athena Scientific, Belmont, 3rd edition, 2005.

[42] Mankei Tsang, Howard M. Wiseman, and Carlton M. Caves. Fundamental Quantum Limit to Waveform Estimation. Physical Review Letters, 106 (9): 090401, March 2011. 10.1103/​PhysRevLett.106.090401. URL https:/​/​doi.org/​10.1103/​PhysRevLett.106.090401.

[43] Katarzyna Macieszczak, Martin Fraas, and Rafał Demkowicz-Dobrzański. Bayesian quantum frequency estimation in presence of collective dephasing. New Journal of Physics, 16 (11): 113002, October 2014. ISSN 1367-2630. 10.1088/​1367-2630/​16/​11/​113002. URL https:/​/​doi.org/​10.1088/​1367-2630/​16/​11/​113002.

[44] Jesús Rubio and Jacob Dunningham. Quantum metrology in the presence of limited data. New Journal of Physics, 21 (4): 043037, April 2019. ISSN 1367-2630. 10.1088/​1367-2630/​ab098b. URL https:/​/​doi.org/​10.1088.

[45] Mădălin Guţăand Cristina Butucea. Quantum u-statistics. Journal of Mathematical Physics, 51 (10), October 2010. ISSN 0022-2488. 10.1063/​1.3476776.

[46] Alexander S. Holevo. Probabilistic and Statistical Aspects of Quantum Theory. Scuola Normale Superiore Pisa, Pisa, Italy, 2011. 10.1007/​978-88-7642-378-9.

[47] Lokenath Debnath and Piotr Mikusiński. Introduction to Hilbert Spaces with Applications. Elsevier, Amsterdam, 2005.

[48] Daniel Gottesman. The Heisenberg representation of quantum computers. In S. P. Corney, R. Delbourgo, and P. D. Jarvis, editors, Group22: Proceedings of the XXII International Colloquium on Group Theoretical Methods in Physics, page 32–43, Cambridge, MA, 1999. International Press. URL https:/​/​arxiv.org/​quant-ph/​9807006.

[49] Mark M. Wilde. Quantum Information Theory. Cambridge University Press, Cambridge, England, UK, February 2017. ISBN 978-1-10717616-4. 10.1017/​9781316809976.

[50] Alexander S. Holevo. Quantum Systems, Channels, Information. De Gruyter, Berlin, 2nd edition, July 2019. ISBN 978-3-11064249-0. 10.1515/​9783110642490.

[51] Brian D. O. Anderson and John B. Moore. Optimal Filtering. Prentice-Hall, Englewood Cliffs, 1979.

[52] Arthur J. Parzygnat and Francesco Buscemi. Axioms for retrodiction: achieving time-reversal symmetry with a prior. Quantum, 7: 1013, May 2023. 10.22331/​q-2023-05-23-1013.

[53] Carl W. Helstrom. Quantum Detection and Estimation Theory. Academic Press, New York, 1976. URL http:/​/​www.sciencedirect.com/​science/​bookseries/​00765392/​123.

[54] Tsachy Weissman. The relationship between causal and noncausal mismatched estimation in continuous-time AWGN channels. IEEE Transactions on Information Theory, 56 (9): 4256–4273, August 2010. ISSN 1557-9654. 10.1109/​TIT.2010.2054430.

[55] Yihong Wu and Sergio Verdu. Functional properties of minimum mean-square error and mutual information. IEEE Transactions on Information Theory, 58 (3): 1289–1301, November 2011. ISSN 1557-9654. 10.1109/​TIT.2011.2174959.

[56] Mankei Tsang. Mismatched quantum filtering and entropic information. In 2014 IEEE International Symposium on Information Theory (ISIT), page 321–325, June 2014. 10.1109/​ISIT.2014.6874847.

[57] Aolin Xu and Maxim Raginsky. Minimum excess risk in Bayesian learning. IEEE Transactions on Information Theory, 68 (12): 7935–7955, May 2022. ISSN 1557-9654. 10.1109/​TIT.2022.3176056.

[58] Wim Bogaerts, Daniel Pérez, José Capmany, David A. B. Miller, Joyce Poon, Dirk Englund, Francesco Morichetti, and Andrea Melloni. Programmable photonic circuits. Nature, 586 (7828): 207–216, October 2020. ISSN 1476-4687. 10.1038/​s41586-020-2764-0. URL https:/​/​www.nature.com/​articles/​s41586-020-2764-0.

[59] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms. MIT Press, Cambridge, 2009.

[60] Dimitri P. Bertsekas. Reinforcement Learning and Optimal Control. Athena Scientific, Belmont, 2019.

[61] H. M. Wiseman. Weak values, quantum trajectories, and the cavity-QED experiment on wave-particle correlation. Physical Review A, 65 (3): 032111, February 2002. 10.1103/​PhysRevA.65.032111. URL https:/​/​doi.org/​10.1103/​PhysRevA.65.032111.

[62] Michael J. W. Hall. Prior information: How to circumvent the standard joint-measurement uncertainty relation. Physical Review A, 69: 052113, May 2004. 10.1103/​PhysRevA.69.052113. URL http:/​/​doi.org/​10.1103/​PhysRevA.69.052113.

[63] Alberto Barchielli and Giancarlo Lupieri. Instruments and channels in quantum information. Banach Center Publications, 73: 65–80, 2006. 10.4064/​bc73-0-4. https:/​/​arxiv.org/​abs/​quant-ph/​0412116.

[64] Kalyan B. Sinha. Sufficient statistic and Rao–Blackwell theorem in quantum probability. Infinite Dimensional Analysis, Quantum Probability and Related Topics, 25: 2240005, October 2022. ISSN 0219-0257. 10.1142/​S0219025722400057.

[65] Andrzej Łuczak. On a general concept of sufficiency in von Neumann algebras. Probability and Mathematical Statistics, 35 (2): 313–324, 2015. URL https:/​/​www.math.uni.wroc.pl/​ pms/​files/​35.2/​Article/​35.2.9.pdf.

[66] Aglaé N. Kellerer and Erez N. Ribak. Beyond the diffraction limit via optical amplification. Optics Letters, 41 (14): 3181–3184, July 2016. ISSN 1539-4794. 10.1364/​OL.41.003181. URL http:/​/​www.osapublishing.org/​ol/​abstract.cfm?uri=ol-41-14-3181.

[67] Hermann A. Haus. Electromagnetic Noise and Quantum Optical Measurements. Springer-Verlag, Berlin, 2000. 10.1007/​978-3-662-04190-1.

[68] Carlton M. Caves. Quantum limits on noise in linear amplifiers. Physical Review D, 26 (8): 1817–1839, October 1982. 10.1103/​PhysRevD.26.1817. URL https:/​/​doi.org/​10.1103/​PhysRevD.26.1817.

[69] W. M. Itano, J. C. Bergquist, J. J. Bollinger, J. M. Gilligan, D. J. Heinzen, F. L. Moore, M. G. Raizen, and D. J. Wineland. Quantum projection noise: Population fluctuations in two-level systems. Physical Review A, 47 (5): 3554–3570, May 1993. ISSN 2469-9934. 10.1103/​PhysRevA.47.3554.

[70] Vladimir B. Braginsky and Farid Ya. Khalili. Quantum Measurement. Cambridge University Press, Cambridge, 1992.

[71] John Watrous. The Theory of Quantum Information. Cambridge University Press, Cambridge, 2018. ISBN 9781107180567. 10.1017/​9781316848142. URL https:/​/​www.cambridge.org/​core/​books/​theory-of-quantum-information/​AE4AA5638F808D2CFEB070C55431D897.

[72] A. W. van der Vaart. Asymptotic Statistics. Cambridge University Press, Cambridge, UK, 1998. 10.1017/​CBO9780511802256.

[73] Leonard Mandel and Emil Wolf. Optical Coherence and Quantum Optics. Cambridge University Press, Cambridge, 1995. 10.1017/​CBO9781139644105.

[74] Mankei Tsang. Quantum nonlocality in weak-thermal-light interferometry. Physical Review Letters, 107: 270402, Dec 2011. 10.1103/​PhysRevLett.107.270402. URL http:/​/​doi.org/​10.1103/​PhysRevLett.107.270402.

[75] Ranjith Nair and Mankei Tsang. Quantum optimality of photon counting for temperature measurement of thermal astronomical sources. The Astrophysical Journal, 808 (2): 125, 2015. 10.1088/​0004-637X/​808/​2/​125. URL http:/​/​stacks.iop.org/​0004-637X/​808/​i=2/​a=125.

[76] Mankei Tsang, Ranjith Nair, and Xiao-Ming Lu. Quantum theory of superresolution for two incoherent optical point sources. Physical Review X, 6: 031033, Aug 2016. 10.1103/​PhysRevX.6.031033. URL http:/​/​doi.org/​10.1103/​PhysRevX.6.031033.

[77] Mankei Tsang. Resolving starlight: a quantum perspective. Contemporary Physics, 60 (4): 279–298, October 2019. ISSN 0010-7514. 10.1080/​00107514.2020.1736375. URL https:/​/​doi.org/​10.1080/​00107514.2020.1736375.

[78] Ranjith Nair and Mankei Tsang. Far-Field Superresolution of Thermal Electromagnetic Sources at the Quantum Limit. Physical Review Letters, 117 (19): 190801, November 2016. 10.1103/​PhysRevLett.117.190801. URL http:/​/​doi.org/​10.1103/​PhysRevLett.117.190801.

[79] Cosmo Lupo and Stefano Pirandola. Ultimate Precision Bound of Quantum and Subwavelength Imaging. Physical Review Letters, 117 (19): 190802, November 2016. 10.1103/​PhysRevLett.117.190802. URL http:/​/​doi.org/​10.1103/​PhysRevLett.117.190802.

[80] Shilin Ng, Shan Zheng Ang, Trevor A. Wheatley, Hidehiro Yonezawa, Akira Furusawa, Elanor H. Huntington, and Mankei Tsang. Spectrum analysis with quantum dynamical systems. Physical Review A, 93 (4): 042121, April 2016. ISSN 2469-9934. 10.1103/​PhysRevA.93.042121.

[81] Wojciech Górecki, Alberto Riccardi, and Lorenzo Maccone. Quantum metrology of noisy spreading channels. Physical Review Letters, 129 (24): 240503, December 2022. ISSN 1079-7114. 10.1103/​PhysRevLett.129.240503.

[82] Haowei Shi and Quntao Zhuang. Ultimate precision limit of noise sensing and dark matter search. npj Quantum Information, 9 (27): 1–10, March 2023. ISSN 2056-6387. 10.1038/​s41534-023-00693-w.

[83] Mankei Tsang. Quantum noise spectroscopy as an incoherent imaging problem. Physical Review A, 107 (1): 012611, January 2023. ISSN 2469-9934. 10.1103/​PhysRevA.107.012611.

[84] John M. Lee. Introduction to Smooth Manifolds. Springer-Verlag, New York, 2003. ISBN 9780387217529. 10.1007/​978-0-387-21752-9. URL https:/​/​www.springer.com/​gp/​book/​9780387217529.

[85] Michael Reed and Barry Simon. Methods of Modern Mathematical Physics. I: Functional Analysis. Academic Press, San Diego, 1980.

[86] Eran Shmaya. Comparison of information structures and completely positive maps. Journal of Physics A: Mathematical and General, 38 (44): 9717, October 2005. ISSN 0305-4470. 10.1088/​0305-4470/​38/​44/​008.

[87] Anthony Chefles. The quantum Blackwell theorem and minimum error state discrimination. arXiv:0907.0866, July 2009. 10.48550/​arXiv.0907.0866.

[88] Kôsaku Yosida. Functional Analysis. Springer, Berlin, 6th edition, 1995. ISBN 978-3-642-61859-8. 10.1007/​978-3-642-61859-8.

Cited by

[1] Xiao-Jie Tan and Mankei Tsang, "Quantum limit to subdiffraction incoherent optical imaging. III. Numerical analysis", Physical Review A 108 5, 052416 (2023).

[2] Arthur J. Parzygnat and James Fullwood, "From Time-Reversal Symmetry to Quantum Bayes' Rules", PRX Quantum 4 2, 020334 (2023).

[3] James Fullwood and Arthur J. Parzygnat, "On dynamical measures of quantum information", arXiv:2306.01831, (2023).

[4] Arthur J. Parzygnat, James Fullwood, Francesco Buscemi, and Giulio Chiribella, "Virtual quantum broadcasting", arXiv:2310.13049, (2023).

The above citations are from Crossref's cited-by service (last updated successfully 2023-12-07 04:11:22) and SAO/NASA ADS (last updated successfully 2023-12-07 04:11:24). The list may be incomplete as not all publishers provide suitable and complete citation data.