BBC Sounds, music, radio, podcasts. Hello and thanks for downloading the More or Less podcast. We're the programme that injects a shot of clarity into the sluggish arteries of statistical uncertainty and I'm Tim Harford. Loyal listener Lin wrote into More or Less at BBC.co.uk to ask us to look into a claim made by Caleb Betts. He's a right-leaning content creator who focuses on mental health and well-being. Lin was sent his claim by her son and wanted us to throw some light on it, as I think Caleb did too. Oh buddy, he's talking about this study and it absolutely blows my mind. There was a study that was published on September 26th, 2025 in Baumarker Research. It's a peer-reviewed scientific journal with a strong reputation that analysed 8.4 million people. Go on. The report was indeed published in a journal that uses peer-review. However, the paper was published as a correspondence rather than a full-blown report. This is reviewed at the editor's discretion, so although the journal that the paper was published in does use peer-review for full reports, correspondence articles might not. But it's to the data that we must really turn our attention. We spoke to Justin Fendos to help us sort through the stats. I'm a professor at Xi'an Jiao Tong Liverpool University in the city of Suzhou in China. While coming to China, Justin spent 11 years as a professor at Dongxia University in Busan, South Korea, specialising in cancer and biophysics, and these days his work focuses on big data health informatics. So using statistical methods to try to understand predictors for different kinds of health outcomes in large data sets. Ideal. Justin has been pouring over the report to see whether we can conclusively say that receiving the COVID vaccine causes cancer. And the results are? No, I do not believe that this paper decisively, definitively, or even very convincingly shows that COVID vaccine uptake causes or is strongly related to cancer incidents. Well that's that. Although it's all very well us saying this, but you'd just be taking our word for it. So let's look more closely. This kind of very large database contains a wealth of information. Virtually anything that would be in doctors' files about patients would have been contained in this database. But for whatever reason, the researchers decided to essentially focus on seven different types of variables only. The first three were demographic variables, so age, gender, income. The fourth was something called the Charles Co-Morbidity Index, which represents your previous disease history. The fifth type of variable that they looked at was the COVID infection history. The sixth type of variable was the vaccination history. And then the seventh type of variable, this was their output, the outcome that they wanted to study, which was the cancer incidence. Importantly, they did not look at some vital variables that we know raise the risk of developing cancer, for example, genetics or smoking. And we also don't know if they had a previous cancer diagnosis. They didn't account for people's health seeking behaviors either. And this is important. I think it's pretty universally true that people who really care about their health and engage in one or more health activities are also more likely to engage in other types of health activities when presented the opportunity to do this. So people who are more likely to get the vaccine are also more likely to go to GP checkups, to sign up for cancer screening programs and so on. And South Korea is proactive in trying to diagnose cancers early. South Korea does have pretty well organized screening programs, especially for the elderly or people who have been identified as high risk groups. If you look for something, you are more likely to find it. There is another important question mark over this paper, and that's the timeframe. For the vaccinated people, they only look for new cancer diagnoses within a year of the first vaccination event. And then for the unvaccinated people, they look for new cancer diagnoses within a year of an arbitrary date that they set, which was January 1st, 2022. The issue here is that the timeframe is very short. And if you understand molecularly and similarly how cancers tend to develop, they tend to develop over many years, sometimes even a decade or two. Any new diagnosis of cancer within a year of receiving the jab is very unlikely to have been caused by the vaccine. At least so far as we know, there are scare stories about turbo cancers, but so far there is no medical or scientific evidence that backs those stories up, no matter what people say at political party conferences and rallies. There is another problem with the findings. Almost all the population of South Korea were vaccinated. The first wave of vaccinations, the uptake was extraordinarily high, I believe is above 96%. The second wave was also very high above 90%. So if there really is a 27% increase in short-term cancer risk for vaccinated people, South Korea should be experiencing a sharp rise in cases of cancer. It isn't. The final problem with this paper is a big one, and also the name of a craft-fake-inspired musical I'm writing, the Hazard Ratio. In statistics, the way we compare a rate of an event, in this case cancer, that occurs in one group against another. When the number is above one, it means that the probability for the vaccinated group is higher than the unvaccinated group. And then if the value of the hazard ratio is less than one, that means the opposite that the probability of cancer diagnosis for the vaccinated group is less than the unvaccinated group. Now, for this particular research, instead of just looking at two or three specific types of cancers, they actually looked at a whopping 29 different types of cancers. And statistically, however, when you are looking at so many different outcomes, because each of these rates of cancer needs to be looked at as a unique potential outcome, you have to do something called a statistical correction. And so you have to calculate statistical significance, which essentially is a number that describes how confident you can be that the shadows you are observing accurately represent the true object. And so the hazard ratio calculation is always accompanied by a measure of statistical significance. And so in this research, we were able to see that there were six specific types of cancers that exhibited this kind of statistical significance, that we could say that there is a potential real relationship between whether you were vaccinated or not, and each of these different types of cancer diagnoses. So initially, it looks good. However, the authors missed out one very important step called a correction. And the way the correction works is that you have to adjust the threshold for statistical significance. Essentially, the more outcomes you're looking at, the more you have to reduce your confidence in the statistical significance, because it's like fishing. The more times you go fishing, the more likely you are just by random chance to catch something. And so you have to make an adjustment a correction for this possibility. It looks like they did not do this kind of a correction. If you do correct for the fact that so many outcomes are being studied, the relationship between the vaccines and cancer disappears. It looks to me just by eye that probably most of their statistically significant results would actually disappear. So the six statistical significant cancer outcomes that they have here probably will be reduced to one or maybe even to zero. So no, we do not have evidence that the vaccine causes cancer risks to rise. It's easy to see why people look at the summary of these types of reports and take their findings at face value. This type of paper is a statistical analysis. It doesn't tell you the underlying cause. And that's the problem with this sort of paper being reported by people who don't know how to read the data. Thanks to Justin Fendos and that's all we have time for this week. If you have any questions or comments, please do write in to more or less at bbc.co.uk. Till next time, goodbye.
