BBC Sounds Music Radio Podcasts Hello and thanks for downloading the More or Less podcast with a programme that looks at the numbers in the news, in life and in systematic reviews. I'm Charlotte MacDonald. The Winter Olympics finished a couple of weeks ago in Italy and apart from a few mixed team events, for the most part, men competed against men and women competed against women. Generally in sport, that is what we consider fair, because as the game showed, men are faster, more powerful and have greater endurance. But increasingly in sport, there's a controversy about transgender women, people who were born male and now identify as women. Is it fair for them to compete against women or do they have an advantage? A recent study in the British Journal of Sports Medicine has weighed into this debate. Here's the headline from their press release. Physical fitness of transgender and cisgender women is comparable, current evidence suggests. The study uses the term cisgender to differentiate women who were born female from transgender women who were born male. The study has been written up in online news stories with headlines like this. Trans women in sport have no advantage over cis women, study finds. But is this what the study in question really found? Tom Coles has been looking into this one. Hi Tom. Hello. Let's start with the big picture here. The reason men and women don't compete against each other in sport is because that wouldn't be fair. On account of physical differences between the sexes, right? Yeah, that's right. So let me just introduce Professor Alan Williams, a sports scientist from Manchester Metropolitan University. And let's start with that thing you mentioned, the male advantage in sport. There's a mountain of existing evidence before this paper appeared that says that there are differences between men and women in terms of their height and other skeletal dimensions like limb length, in terms of their muscle mass and muscle strength, where again men are bigger muscles and stronger muscles. Now transgender women are born male and then as part of their transition often undergo hormone therapy. Among other things, this lowers their testosterone levels and leads to changes in their bodies, including other things being equal, a loss of muscle mass and strength. The big question is whether the declines in muscle mass and strength are big enough to make it fair for transgender women to compete in the women's sport category. Okay. And so the scientific paper we're talking about today says that two groups are comparable. Some people have taken that as meaning that the changes in transgender women's bodies are big enough to mean the two groups are similar. Did the researchers involved go out and run tests on people to measure their strength and speed, for instance? No, no they didn't. This paper is what's known as a systematic review. The researchers searched out all the scientific papers they could find comparing transgender men and women with other men and women in terms of their height and the amount of muscle and their fitness and strength levels. They then analyzed that in a bunch of different ways to try and draw out overall conclusions from all these different studies. In their summary of this process, they say that while transgender women had more lean mass, that basically means muscle, and women, their physical fitness was comparable. Fitness here is a general term covering upper and lower body strength and endurance. Alan says that that's counterintuitive as you'd expect people with more muscle to be stronger. I mean, that does sound quite strange. Are we sure that the transgender women who are being tested are a good comparison with the women who are being tested? No, we're not. In some of the papers they use in this analysis, you can quite clearly see that they are definitely not comparing like with like. An extreme example is a paper from Brazil which compared the size and strength of transgender women volleyball players and women volleyball players. There were only seven or eight people in each comparison group, so it was quite a small study. Also, the amount of training done by both groups was quite striking. The transgender women did four hours training a week. The women group did 14 hours per week, over three times as much training as the transgender women. Yeah, that doesn't sound like the comparison is fair. No, and Professor Alan Williams says training is really important. There's a mountain of evidence that physical training affects these things. So yes, if one group is training more than three times more than the other group, we can safely assume that that is affecting the results. The results in question found that the group of transgender women volleyball players were not as strong as the group of women volleyball players, which is not surprising at all when you look at the two groups. The transgender women were shorter on average, weighed less, and were older. They also played volleyball at a lower level compared to the women in the research who all played in national, not just regional competitions. It doesn't sound to me like you should use a study like that to draw out wider conclusions on this question. It does seem odd, but that's the point of these systematic reviews. You look at a lot of studies and assess them in various ways to see what you can say. The volleyball study, for example, was used with three others in a comparison of upper body strength. The other three studies found that transgender women were stronger, but the analysis found the difference was not statistically significant. The researchers told me that was still the case when they removed the volleyball paper. At the same time, they also analyzed the confidence you should have in this conclusion and found it to be very low. Right, but saying they haven't found a statistically significant difference isn't the same as saying there's no difference, especially when the controls aren't in place to ensure you're doing a fair comparison. No. The trouble with lots of these studies is that they're what's known as cross sectional. They're just comparing two groups at a single point in time. Without carefully controlling all the variables, you're likely to get some fairly meaningless results. What you really want are longitudinal studies, where you take a group of transgender women who are starting hormone treatment and measuring their muscle and strength over time to see if they change by enough to make them comparable to women. Yes, that sounds like the kind of studies you'd want. Did they have them? There were a handful of longitudinal studies included in the analysis, but they didn't show the scale of change in terms of loss of muscle and strength you'd need to see. Either there were small changes, such as a small increase in body fat and a small reduction in muscle mass and a small reduction in muscle strength, or in several respects, there were no changes. Looking at the effect of three years of hormone treatment, they did not have enough information to analyse absolute muscle mass, upper or lower body strength or endurance. So how after all that did they conclude that the strength and fitness of the two groups were comparable? That is the question. My opinion is there are reasons why people haven't done a systematic review like this before, and that is because there wasn't a need for a systematic review of such poor evidence to proceed with a systematic review and conclude then what they have, while partly acknowledging how poor the evidence is, but then be so clear in apparent conclusions is very strange. What do the authors of the paper say about this criticism? Well, I've been talking over email with one of the authors, Bruno Gualano, an associate professor at the School of Medicine of the University of Sao Paulo. He says that this description of the groups being comparable should not be taken as meaning the two groups are the same. That interpretation is wrong. It's rather describing the fact that the analysis did not find statistically significant differences between the groups. He believes that despite all the problems with the studies they looked at, you would still expect large and robust intrinsic advantages to come through if they were there, although he does accept the possibility that the weakness of the studies could have interfered with that. It's clear, he says, that after hormone treatment transgender women do not have the physiological profiles of typical men. And he also told me, well controlled longer term studies, especially in trained athletic populations would be definitive. Our paper calls for exactly that. I think we can all agree on that one. Thanks Tom and thanks to professors Alan Williams and Bruno Gualano. That's all we've got time for this week. If you've seen a number in the news, you think we should look at email more or less at bbc.co.uk. Goodbye.
