Peer Review – a Flawed Process for Science

Back from a Christmas break, and I hope all my readers had a happy and satisfying break. 2020 has arrived, more with a bang than a whimper, but while languishing in what has started off as a dreadful summer here, thanks to Australia (the heat over Central Australia has stirred up the Southern Ocean to give us cold air, while their bush fires have given us smoky air, even though we are about 2/3 the width of the Atlantic away) I have been thinking of how science progresses, or doesn’t. One of the thoughts that crossed my mind was the assertion that we must believe climate change is real because the data are published in peer-reviewed journals. Climate change is certainly real, Australia is certainly on fire, but what do you make of the reference to peer-reviewed journals? Does such publication mean it is right, and that peer review is some sort of gold standard?

Unfortunately, that is not necessarily so, and while the process filters out some abysmal rubbish it also lets through some fairly mediocre stuff, although we can live with that. If the work reports experimental observations we should have more faith in it, right? After all, it will have been looked at by experts in the field who use the same techniques and they will filter out errors. There are two reasons why that is not so. The first is that the modern scientific paper, written to save space, usually gives insufficient evidence to tell. The second is illustrated by climate change; there are a few outlets that are populated solely by deniers, in which another denier reviews the work, in other words, prejudice rules. 

Chemistry World reported a study carried out by the Royal Society of Chemistry that reviewed the performance of peer review, and came to the conclusion that peer review is sexist. Females as corresponding authors made up 23.9% of submissions, and 25.6% of the rejections without peer review. Only 22.9% of the papers accepted after peer review came from female corresponding authors. Female corresponding authors are less likely to receive an immediate “accept”, or “accept with minor revisions”, but interestingly, if the reviewer is female, the males are less likely to receive that. These figures come from 700,000 submissions, so although the percentages are not very big, the question remains: are they meaningful, and if so, what do they mean?

There is a danger in drawing conclusions from statistics because correlations do not mean cause. It may be nothing more than women are more likely to be younger, and hence being early in their careers are more likely to need help, or they are more likely to have sent the paper to a less than appropriate journal, since journals tend to publish only in very narrow fields. It could also indicate that style is more important than substance, because the only conceivable difference with a gender bias is the style used in presentation. It would be of greater interest to check out how status affects the decision. Is a paper from Harvard, say, more likely to be accepted than a paper from a minor college, or something non-academic, such as a patent office?

One of my Post-doc supervisors once advised me that a new idea will not get published, but publication is straightforward if you report the melting point of a new compound. Maybe he was a little bitter, but it raises the question, does peer review filter out critical material because it does not conform to the required degree of comfort and compliance with standard thinking? Is important material rejected simply because of the prejudices or incompetence of the reviewer? What happens if the reviewer is not a true peer? Irrespective of what the editor tells the author, is a paper that criticizes the current paradigm rejected on that ground? I have had some rather pathetic experiences, and I expect a number of others have too, but the counter to that is, maybe the papers had insufficient merit. That is the simple out, after all, who am I?

Accordingly, I shall end by citing someone else. This related to a paper about spacetime, which at its minimum is a useful trick for solving the equations of General Relativity. However, for some people, spacetime is actually a “thing”; you hear about the “fabric of spacetime” and in an attempt to quantize it, scientists have postulated that it exists in tiny lumps. In 1952 an essay was written that was against the prevailing view that spacetime is filled with fields that froth with “virtual particles”. I don’t know whether this was right or not because nobody would publish it, so it is not to be discussed in polite scientific society. It was probably rejected because it went totally against the prevailing view, and we must not challenge that. And no, it was no written by an ignorant fool, although it should have been judged on content and not the status of the person. The author was Albert Einstein, who could legitimately argue that he knew a thing or two about General Relativity, so nobody is immune to such rejection. If you want to see such prejudice in action, try arguing that quantum field theory is flawed in front of an advocate. You would be sent to the corner wearing a conical hat. The advocate will argue that the theory has calculated the magnetic moment of the electron and this is the most accurate calculation in physics. The counter is yes, but only through some rather questionable mathematics (like cancelling out infinities), while another calculation based on Einstein’s relativity gives an error in the cosmological constant of about 120 orders of magnitude (ten followed by 120 zeros), the worst error in all of physics. Oops!