To test this hypothesis, Quantum started a pilot outreach project, Quantum Leaps, in collaboration with Cherrybrook Technology High School in Sydney, Australia. We encouraged quantum information and computation researchers around the world to write a popular science article about their work.
We received thirteen ambitious submissions, and took six to a science class from the Year 8 cohort of the school. These are 13–14 year-old students that are especially keen to hear about cutting edge science and the process of peer review. Over the course of three sessions, the students learned about peer review, discussed the articles, and wrote their first report, which we have now sent back to the authors. Here are our conclusions from the first round of reviews.
We worked with a class of about thirty students, organised in groups of four to five people. This meant that we had to pick six out of the thirteen submissions received to bring to the students. We picked at random; the remaining articles may be used in the next edition of this project.
- 50 minutes of discussion on peer review in science. Homework: read papers.
- 50 minutes of in-class discussion on topics of papers. Homework: begin report.
- 50 minutes of discussion on final verdict. Homework: complete report.
We received six reports, which we have now sent back to the authors. Once they implement the reports’ suggestions, the original group of referees will write up a final assessment, after which the articles will be published in Quantum Views. The tentative timeline for publication is early June.
The students were very engaged and excited to be a part of this! They enjoyed reading about quantum research, and most said that they want to give a talk explaining the articles to their peers. To encourage the maximum possible engagement, we suggested that the students not hold back any punches in their reviews. Warning: some of the comments are not for the faint of heart!
We ran the articles through an online readability test and found that their language level was too high – and this is just referring to the grammatical structure! A randomly chosen article in the New York Times had a reading grade level of 8, which is the average for a US newspaper. The average grade level of the submissions was 10. Not bad, but that also means some of the submissions could only be read by college graduates!
The paper was quite lengthy resulting in lots of information to grasp, causing us to re-read certain parts multiple times, slowly losing a bit of interest.
On the other hand, our referees were gentle on typos and grammar mistakes:
Our group agrees that the paper was explained well, but there were a few faults in terms of English skills and grammar. The few grammar issues that there existed can easily be fixed by editing.
The larger problem (which readability tests do not capture) was technical language. The most common criticism raised by our referees was that the articles were rife with technical terms unknown to them. Expressions like superconductor, quantum state, spin, superposition, and entanglement were often not defined and even used in explaining other concepts.
The results were extensive and the scientific explanations were precise however the scientific terms could have been replaced with simpler terms so that the reader could understand the article better. The article was also too long and should be more straightforward when explaining.
The paper was interesting and had a clear concept but sometimes the explanations/other points were not very clear and did not belong at the current point of the report. These areas could have been ordered in a better and satisfying way. That is why we recommend revising it slightly and just tweaking the order of arguments in order for it to flow better. We also would like to recommend to define scientific terms more clearly and precisely due to the fact that we were not able to understand some terms that were defined.
The paper was not interesting to read because it described many terms in a technical and complicated way; hard to understand.
Some attempts at explaining basic concepts were not deemed satisfactory:
Author: In a superposition, it’s like one atom is in two places at once; it feels effects from both of those places at the same time.
Referees: We were not fully clear on the concept of an atom being ‘in two places at once’. Also, you could explain why the particles are not in one place but in two places…
Although some of the terms were explained, the phrasing of those explanations were confusing, and hard to understand to the point where it had to be re-read multiple times to grasp the basic concept.
Visual aids, please!
The students applauded the use of diagrams and pictures, and requested them when absent. However, not all visuals aided…
Although there were visual aids to assist understanding of a concept, it was often unclear what the concept being talked about was, or what it meant. For example, Figure 2 is effective in explaining that the “quantum states” flipped abruptly and not gradually, however that could only be understood if the meaning of “quantum states” or “electronic spin flip” was defined clearly – which it unfortunately was not.
Referees also suggested improved structure for the articles, including subheadings and a glossary section:
Things like mind maps, highlighted words, explanations of words within the paragraphs etc. would help to further this paper.
Impact of the research
Some students were thrilled by purely foundational research:
The research is relevant to society because by proving their idea, they change the way that we view scientific principles, such as Einstein’s Relativity Principle. It makes important findings and that is good because it can change how we view gravity and do cool stuff! 😁😁😁
Others felt that the authors had not done enough to motivate their work:
What is a quantum computer and why are we trying to build it? What would we use his findings for and how would it benefit society? We don’t know… Why are we trying to figure out how to build internet quantum computers? How costly is this process?
And for some, the opacity of explanations rendered them unable to assess the relevance of the research:
It probably was important to those who could understand it.
Chris and Lídia found that the articles were well-written, but more suitable for an older, more knowledgeable audience, like physics undergrads. Indeed the articles we found the most interesting were the ones struck the hardest by our referees, for having too many technical terms and too opaque explanations for the target audience. Articles that we thought were too simple were lauded by the students as clear and accessible.
We believe that our student referees model the best of a general audience: readers with no technical knowledge of the physics involved, but intelligent and motivated to understand the articles.
The articles succeeded at intriguing the readers and spiking their curiosity, motivating them to learn more about these topics. However, in most cases the students did not obtain a good understanding of the concepts in the articles. This main culprit was an abundance of poorly explained technical terms. Scientists have had difficulty in explaining the concepts of quantum physics since its inception. But we seem to have settled on a few popular metaphorical explanations. For example, superposition is often explained with the simile of being in “two places at once”. We were quite surprised—and, quite frankly, impressed—when we found that the students recognised this as being incomplete and demanded more. Other concrete suggestions included improved structure and visual aids: a popular science article is not an academic paper, and our referees were put off by walls of text.
The challenge is now on the authors to try to implement all the referees’ suggestions! We will keep you posted on updates.
Lídia del Rio and Chris Ferrie