The Natural Order of Things? Part III: The Song Remains The Same

It’s the same old story, same old song and dance, my friend…

Same Old Song and Dance, Aerosmith. Track 1, Side 1, “Get Your Wings” (1974). Perry/Tyler. Lyrics © BMG Rights Management


Zombie arguments are the lifeblood of the internet. Resistant to all counter-evidence, it doesn’t matter how often they’re shot down — they will arise (under a pale gray sky[i]) to live and breed again. The reason for their immortality is laid out in a classic post by David McRaney entitled The Backfire Effect[ii]:

The Misconception: When your beliefs are challenged with facts, you alter your opinions and incorporate the new information into your thinking.

The Truth: When your deepest convictions are challenged by contradictory evidence, your beliefs get stronger.

Or, as McRaney puts it further down that post:

The last time you got into, or sat on the sidelines of, an argument online with someone who thought they knew all there was to know about health care reform, gun control, gay marriage, climate change, sex education, the drug war, Joss Whedon or whether or not 0.9999 repeated to infinity was equal to one – how did it go?

Did you teach the other party a valuable lesson? Did they thank you for edifying them on the intricacies of the issue after cursing their heretofore ignorance, doffing their virtual hat as they parted from the keyboard a better person?

No, probably not.

The Backfire Effect in turn underpins Brandolini’s Law:

The amount of energy needed to refute bullshit is an order of magnitude bigger than to produce it.

aka The Bullshit Asymmetry Principle.

The wholly biased presentation on gender bias in science given by Alessandro Strumia at the end of last month featured a veritable army of zombie arguments. These were, of course, unblinkingly accepted by those who share Strumia’s ideological bias:

Without wanting to get too ‘meta’ here, Damore’s “I’ve not seen anyone address the points this put-upon man has raised” claim is itself a zombie argument.  Each time an aggrieved and over-emotional gentleman decides to put across personal opinion, bias, and methodologically-unsound analysis of gender differences as “The Truth” (unfailingly covered up by The Big Bad Establishment[iii]), their ‘arguments’ are addressed and rebutted. As just one example, Strumia’s pseudoscience was dissected and demolished very quickly by his fellow particle physicist Jon Butterworth in a masterful blend of snark, satire and sharp insight.

You might, therefore, quite reasonably ask just why I’m returning to this theme. Hasn’t Strumia had his 15 minutes of fame and shouldn’t we just ignore him now, given that, for one, his = <Ncitations> pseudoscientific nonsense has been thoroughly rebutted? I have quite some sympathy for that view, I must admit, but I’m exhuming the corpse of Strumia’s pseudostats, and returning to the zombie fray, in order to provide a direct response to David Allen, who left a series of comments and questions under my “The Worm That (re)Turned” post on Strumia. David’s questions and comments were made in a very polite and genuine manner. He deserves a considered response. I’ll address David directly from here on in.

DA: So what explains the tendency for nations that have traditionally less gender equality to have more women in science and technology than their gender-progressive counterparts do? That question is posed here: https://www.theatlantic.com/science/archive/2018/02/the-more-gender-equality-the-fewer-women-in-stem/553592/

Unless there is a consensus of credible proof that the reason for low women numbers n STEM is that they are being discriminated against in some way, I personally will elect to believe it is down to their free choice and agency rather than down to victimhood.

[David’s full comment is here.]

First, David, this is the third in a trilogy of posts entitled “The Natural Order of Things?” that I’ve written[iv]. (Parts I and II are here and here, respectively.) I’m therefore going to be repeating myself to some extent. Again. But such is the nature of zombie arguments. Let’s go through the points in your comments one by one.

Before dealing with the gender equality article you cite, and subsequently getting into the weeds of Strumia’s beliefs, let’s deal with that “proof” word you use repeatedly in your comments. Science is not about proof; any credible scientist knows this. I’ve lost count of the number of times I’ve quoted Carlo Rovelli on this point (but then his statement about scientific ‘proof’ bears repeating ad infinitum):

The very expression “scientifically proven” is a contradiction in terms. There’s nothing that is scientifically proven. The core of science is the deep awareness that we have wrong ideas, we have prejudices. We have ingrained prejudices. In our conceptual structure for grasping reality, there might be something not appropriate, something we may have to revise to understand better. So at any moment we have a vision of reality that is effective, it’s good, it’s the best we have found so far. It’s the most credible we have found so far; it’s mostly correct.
“Mostly correct.” In other words, we look for evidence to support a particular model but we should always have the humility and insight to realise that science is not about certainty; that our understanding is provisional. (Religion, on the other hand, is all about certainty: this is the natural order of things as ordained by God (or gods.))

 

Being aware of the nature of scientific evidence is especially important when it comes to arguments about gender balance where, even if we leave aside the issue of ideological bias (in both directions), there is a wealth of conflicting data out there. In “The Worm That (re-)Turned At CERN” I mentioned the Heterodox Academy analysis of Damore’s cherry-picking of data for his “Google manifesto”. (Strumia took that cherry-picking approach and added quite a few more punnets on top. More on that later). I would suggest that you carefully read that HA analysis, David. Note that the evidence is not incontrovertibly pointing one way or another: there is a great deal of controversy and debate in the literature regarding many aspects of gender differences.

 

Any good scientist — or, indeed, anyone who, like yourself, would claim to be an open-minded “fence-sitter”, only interested in where the data leads them[v] — must take into account this conflict in the literature. Any credible analysis must start from a position of recognising the lack of consensus in the literature. This is not what Strumia did. He instead made definitive statements on the basis of both cherry-picked arguments and shockingly weak suppositions (of a type I would not expect high school students, let alone a scientist of Strumia’s position, to make. We’ll get to those.)

 

So, let’s leave aside the naive and simplistic idea that there’s a definitive deductive “proof” one way or the other. The key issue is the extent to which the scientific evidence supports a given claim. If there is insufficient evidence and/or a lack of consensus in the studies the very best we can be is agnostic. Anything else would be unscientific. I hope we can at least agree on this.

 

That out of the way, let’s turn to the article in The Atlantic you cited…

 

Issues with the Global Gender Gap Index

 

One key problem with so much of the online debate on gender differences — and just about any other subject under the sun — is that there’s a very strong tendency to rely on secondary sources, eg. news articles about a particular study, rather than consult the original source. This is often understandable because the source can be trapped behind a paywall, as in this case. (I’ve, however, released the paper cited by The Atlantic article into the wild via this link.) Nonetheless, it’s essential not to credulously swallow headlines hook, line, and sinker (especially when a significant fraction of the internet population do nothing more than read the headline. [vi]) Even going beyond the headline to read the article itself is, of course, no guarantee that the study has been accurately reported or that its nuances (and/or deficiencies) have been highlighted.

 

So let’s take some time to explore exactly what the graph of “Global Gender Gap Index vs % women among STEM graduates” discussed in the article actually means. First, what is the Global Gender Gap Index (GGGI)? How is it calculated, David? Do you know? I certainly didn’t until recently.

 

GGGI.png

 

I have a deep mistrust of all rankings and league tables because the methodology used to produce these is so very often pseudoquantitative and pseudostatistical at best, and complete nonsense at worst. And, lo and behold, what do we find when we take even the most cursory look at the origin of the GGGI figures? “In short, the country rankings in the Global Gender Gap Report are misleading at best and completely meaningless at worst.” I urge you to to take the time to read the analysis at that link for yourself, David. The author considers the life expectancy metric as just one example:

 

As the Index is rewarding a greater ratio, lower development values are rewarded (i.e. lower healthy life expectancy). Consequently, since the gender gap was the same in Denmark and Rwanda in 2016, but Rwanda had a lower life expectancy, they performed better on the Index (13 places better than Denmark). This problem becomes more and more serious when the overall level of development decreases and the gender gap increases.
Always look beyond the headlines, David. Question the methodology. Be sceptical — or, if you must, be skeptical — and think critically, rather than blindly accept an analysis because it’s got lots of numbers, and graphs, and looks “sciencey”. (More on this soon.)

 

Taken in isolation, there’s a distinct absence of error bars on that graph above. How are we supposed to know whether any of those relative country placements are statistically significant? It’s clear from the distribution of ‘data’ on the graph that a ‘resolution’ down to the third decimal place has been used to place the points. How is that justified? Moreover, it’s exceptionally weak to just do a naive linear regression without showing (or knowing) the effective uncertainty in each of the points. (But how would we accurately determine an uncertainty for each point?) Even then, because the methodology in generating the index is flawed, the numerical analysis is always going to be suspect.

 

To quote Pauli just slightly out of context, if the methodology is incorrect and the uncertainties aren’t accounted for, then any assertions made are not even wrong. And, as Einstein is (most likely apocryphally) reported to have said: not everything that counts can be counted, and not everything that can be counted, counts. For example, here’s another set of reasons why you shouldn’t take the GGGI data at face value: Is gender inequality really so low in the Philippines? Or, if you want a broader discussion of why you shouldn’t place your faith in rankings in general, try this. (Or, in a higher education context, try this, this, or this).

 

Even, however, if we faithfully and naively take the GGGI data purely at face value (a la Strumia), then the Psychological Science paper to which you refer via that article in The Atlantic –you’ve read the original paper, right? — is hardly supportive of Strumia’s stance. To be clear, Strumia is arguing that there is an innate and immutable biological/genetic difference that underpins the difference in performance in physics for males and females. (He ludicrously use citation rates as an indication of mean IQ level and, thus, physics performance. Again, we’ll get back to that.)

 

You are making a slightly different, though related, argument to that Strumia attempts to put forward. Your focus in your comment above is on the question of discrimination vs “free agency”. It’s worth noting that Janet Shibley Hyde, to whom I refer at length in The Natural Order Of Things, and someone who is hardly ideologically aligned with Strumia’s beliefs, is quoted as follows in that article:

 

“Some would say that the gender stem gap occurs not because girls can’t do science, but because they have other alternatives, based on their strengths in verbal skills,” she said. “In wealthy nations, they believe that they have the freedom to pursue those alternatives and not worry so much that they pay less.”

Instead, this line of research, if it’s replicated, might hold useful takeaways for people who do want to see more Western women entering stem fields. In this study, the percentage of girls who did excel in science or math was still larger than the number of women who were graduating with stem degrees. That means there’s something in even the most liberal societies that’s nudging women away from math and science, even when those are their best subjects.

 

Like Shibley Hyde (and, indeed, the vast majority of those of us with interests in improving gender balance in STEM subjects), I am more than willing to accept that direct discrimination is not always necessary in order for women and girls to choose other career options or make non-STEM subject choices at school, respectively. (Or, to use your rather overwrought description instead, no “victimhood” is required.) However, just as it’s best to leave behind the naive idea that science provides us with definitive, deductive proofs, it’s also a good idea not to assume yes/no, black/white answers to everything. Uncomfortable, I know, but as Rovelli explains, science is not about certainty. (And before you assume that this is some new-fangled, social justice-enabled liberal definition of science, here’s the physicist’s physicist, Richard Feynman, on the same subject back in the seventies:

 

It is necessary and true that all of the things we say in science, all of the conclusions, are uncertain, because they are only conclusions. They are guesses as to what is going to happen, and you cannot know what will happen, because you have not made the most complete experiments.
So it’s not a binary state; there’s a spectrum of possibilities. And there is good evidence that discrimination occurs — as described here, here, and here, for example. (You could in turn, after a modicum of research, point me to the Ceci and Williams article that argued precisely the opposite. And I, in turn, could point you to the large number of deficiencies in Ceci and Williams’ work. Again, this is how scientific debate and discussion work.)

 

DA: So far, sitting on the fence in this matter, I’ve seen no point by point uniquivicable rebuttal of each of Strumia’s slides. Only supperfical attempts where someon interprets 1-3 slides differntly to Strunio and uses that interpretation to claim everything he said is fallacious.

 

Instead of a genine attempt at proving him wrong, what we see iare widespread claims of ‘poor arguments’, ‘shameful’ type comments followed by ad hominem attacks and the cherrypicking and vague assertions in the open letter itself. And of course suspension.

 

Strumia’s core premise is so flawed as to be laughable. His argument rests on the idea that citations scale directly not only with the quality of science but, remarkably, with intelligence. Let’s deal with the quality issue first. Citations are a measure of the popularity and “impact” of a paper; nothing more, nothing less. (How would you or Strumia “prove” otherwise, David? What’s your (normalised) metric for quality as distinct from impact?) A paper can attract a large number of citations for reasons other than the quality of the science (including the prestige of the group that produced the work). More worryingly, sometimes papers that are fundamentally methodologically flawed attract a large number of citations. (And before anyone suggests otherwise, let me state categorically that I am not suggesting that any of Strumia’s work is flawed, although he does seem to have spent quite some time fruitlessly developing explanations for what was a mundane noise blip (at 750 GeV), gaining many citations in the process.) Here are just two examples of which I’m especially familiar: stripy nanoparticles and nanoflares.

 

As Jon Butterworth alludes to, citations also need to be normalised to a particular (sub-)sub-field for them to be of any value at all. (And just how do we normalise to a particular field?) Citation patterns, and the scale of collaboration, vary dramatically across even just one discipline: solid state physicists tend to have much smaller numbers of co-authors on a paper as compared to the experimental particle physics community. Citation rates within the particle physics community (or, indeed, any community) alone will also depend on the “visibility” of a researcher in terms of their networking and collaborative activity. (Butterworth points to his involvement with the ATLAS and CMS collaborations). Moreover, the journal in which a paper is published makes a significant difference to the citation rate.

 

Attempts to balance quality and quantity of science for a given researcher via something called the h-index are similarly problematic. (See here for Philip Ball’s insightful critique of h-indices). I have referred in the past to this Popperian analysis of the h-index:

 

HIndexBollox.png

 

So, to cut a long story short (and I haven’t even begun to scratch the surface of the issues with citation analyses), drawing a direct line from number of citations to scientific quality is clearly not a particularly scientific strategy to adopt. (If you’re going to argue otherwise, David, I’ll ask again: which metric do you use to disentangle impact from scientific quality in an entirely numerically and statistically robust manner? And what sort of measurement uncertainty would you place on that value?) As Jon Butterworth points out (see his footnote #2), Strumia’s “asymmetry parameter” re. citation rates also looks distinctly odd and far from evidence-based. (The definition Strumia uses does, however, helpfully align with his argument and ideology. Who’d have thunk it?)

 

That’s bad enough. What’s worse is that Strumia made an additional wild pseudoscientific leap to claim that there’s a direct link between IQ and the number of citations an individual scientist attracts. Even if we blindly accepted that citations and scientific quality go hand in hand — which we can’t; see above — Strumia drags out the classic zombie argument about the tails of the IQ distribution. This has already been critiqued by a number of authors — see my first The Natural Order Of Things post for just a few examples. More topically, however, shortly after Strumia’s mish-mash of stats, pseudostats, and groundless inferences made the headlines, the Institute of Physics published this. Here are the key points from the article of relevance to our exchange (although you should, of course, go to the primary source as well…):

 

The authors say that their study disproves the “variability hypothesis”, which suggests that male over-representation in STEM careers comes from a greater variability in grades among boys than girls (Nature Communications9 3777).

 

By analysing the grade distributions, the researchers found that the top 10% of grades in STEM subjects had an equal gender ratio, while non-STEM subjects were female-heavy. “Our results support greater male variability in academic performance, but they don’t support gender differences in variability as an explanation for gender differences in workforce participation because we find the smallest gender differences in variability in maths and science,” O’Dea told Physics World.
(Note that I dislike the use of the term “disproves” in the preceding quote for all of the reasons discussed above.)

 

Strumia not only runs with the “variability hypothesis”, he decides he’s going to arbitrarily cherry-pick the cut-off point he needs in order to “fit” his data. This is not, to put it mildly, credible analysis.

 

DA: If those in science are in the business of rebutting numbers and graphs (even ‘bad’ numbers and graphs) with insults, anger, and tears of hurt feelings, then it doesn’t engender confidence in the scientific commumity working dispassionatlely. It imples political bias rather than an honest quest for the truth – because what if that truth is ‘uncomfortable’? Scionce should not be concerned with feelings. What next – trigger warnings outside conferences?

 

Like you, I see hurt feelings, anger, and an overwrought, over-emotional, and unscientific analysis. We differ on the source, however. Strumia’s presentation was heavy on petulance, arrogance, and whining: “I’ve got more citations. Why wasn’t I given the job? Why? It’s just not fair. Physics was invented and built by men. Men, I tell you.” It is beyond unprofessional to use a conference presentation to make personal attacks and whine about failing to get a job. Strumia not only insulted a colleague, he insulted an entire gender. He’s clearly not on an “honest quest” for the truth. If he were, he’d have presented a much less biased and cherry-picked analysis, and spent rather more time thinking carefully about the (lack of) validity of nonsensical assertions like IQ scales with <Ncitations>.

 

On the other hand, I see in the community response a professional and sober rebuttal of Strumia’s claims. (But a bit of a naff URL, admittedly…)

 

It is rather naive, David, to claim that the scientific process isn’t concerned with “feelings”. I know that’s the myth but Strumia’s impassioned/overwrought (delete to taste) presentation was driven, at least in part, by his emotional reaction to being passed over for a job. Similarly, science is a social enterprise. We are not emotionless, wholly objective automatons free of all external and internal biases. Peer review, for one, is a messy, all-too-human process. To argue otherwise betrays a deep lack of understanding of just how science progresses. (See “The truth, the whole truth, and nothing but…” for more on this.)

 

DA: Thank you for the link to David Smith. David is doing what we all do. He starts with an opinion, and sorts the millions of related facts into a hierarchy, with those at the top placed there to support his view. Then challenging any contrary opinions while accepting supporting ones.

…which is exactly what Strumia did. But somehow Strumia’s analysis is robust while Smith’s rebuttal isn’t? (Are you quite sure that you haven’t fallen off that fence, David…?) Strumia made assertions. Smith went through those assertions and provided counter-evidence. That’s how scientific debate works.

DA: For example point 6. “If you are interested in whether there are innate gender differences between male and female brains, then you must read neuropsychologist Lise Eliot, who refutes the dominance of biology concluding socialization is vital.”

So, Lisa Elliot is irrefutable? Are there no eminent biologists out there claiming the opposite?

Um, who said that Elliot is irrefutable, David? Point me to where David S has said that? I can’t speak for David Smith but I, for one, have been at pains before to highlight both sides of the argument. I’ve spent quite a bit of time reading Baron-Cohen and Pinker (among others). I would, however, ask you in turn to do me the courtesy of reading Angela Saini’s “Inferior” and Cordelia Fine’s “Testosterone Rex” and “Delusions of Gender” before responding. I have done my homework for this post.

DA: I want you to know that I don’t agree with a lot of what Strumia says. But just because his reasoning on, say, IQ and number of citations is probably way off, it doesn’t mean that everything he says is way off.

Thanks for this, David. I’m a little confused, however. You argued above that Strumia produced a robust, quantitative analysis and that he should be listened to on that basis. Yet the entire premise of that ‘quantitative’ analysis was based on his faulty reasoning between IQ and citation numbers. If you agree that his reasoning was faulty, what is it that you think isn’t “way off”? And on what basis do you think that, given you feel his quantitative reasoning is “probably way off”?

DA: 1) I believe that someone’s race, sex, religion etc should not be a factor in what someone is allowed to say, or as a consideration in whether they get a job or not.

2) Therefore I do not believe in positive discrimation since it involves negative discrimination against others. If person A is using sex in their decision to hire someone, they are also using it not to hire somone else.

3) I do not believe in retaliation in the form of suspension or job loss, or verbal/written lynch mobs of 150 people against 1 person because that person (male or femail) challanges allowed norms. Apart from in extreme cases.

4) I do not believe in group A deciding what is moral and then conveniently claiming they are morally superior to group B. There are some exceptions of course.

5) I believe that femails have the same intelligence, (and in the West) the same potential and agency as males, and that in general if they are not in STEM it’s because they don’t want to be. There are more women than men in universities and unless someone can demonstrate the opposite, my presumption is that they are choosing the qualifications they are taking.

6) I don’t believe in witch hunts, trial by twitter, or labelling a person’s indentity as x or y because they said a few things. In the rational world there is big difference beween saying ‘that was a misogynist remark’ and ‘you are a misogynist’.

 1) Hmmm. So a fully committed, evanegelical creationist should be employed to teach cosmology and/or evolution? They disclose this at interview and say that they will teach science according to their belief system or not at all. Should they (a) be employed, and (b) have free rein over what they cover in that course? Or let’s say that, as admissions tutor, I am asked to give a talk to A-level students at a Catholic school and I decide to turn up there in a Slayer “God Hates Us All” T-shirt, or, worse, something emblazoned with a Cannibal Corpse album cover/title (or anything from this delightful list). Or I give a talk along the lines suggested here. I should be free to say whatever I want under whatever circumstances? Really?

2) Positive discrimination of the type you describe is unlawful in the UK.

3) I agree. See my original post. Dismissal helps foster that victimhood/martydom mentality for those like Strumia and Damore who will claim they were silenced for speaking “The Truth” despite their version of the truth being rather ideologically-skewed and easily rebutted. My suggestions for alternative strategies are outlined in this response to my colleague Anne Green.

4) This happens with any in-group vs out-group dynamic. “The right” is just as guilty of this as “the left”. See, for example, the moral outrage re. lampooning Trump, “taking the knee“, or the general patriotic correctness of the right.

5) But there are social biases everywhere. I, for one, would much rather see greater numbers of men involved in primary school teaching and in other so-called “nurturing” professions. On what basis are those decisions being made? If you say they’re genetically/biologically hard-wired to the extent that the sexually dimorphic signal outweighs the environmental (i.e. societal) influence, I will ask you — as I have asked so many others — to provide me with conclusive evidence that this is the case. Again, I would suggest you read Saini’s and Fine’s books, to which I refer above.

6) I agree.

I’m glad we could finish on a point of agreement, David. I already alluded to the point you make in the final paragraphs of “The Worm That (re-)Turned...“. There is often a rush to judgement and it is too easy to damn someone for a few hasty or misinterpreted comments. (I enjoyed Jon Ronson’s Shamed, which examines this social dynamic in a number of situations.) In Strumia’s case (and, before him, Damore), however, his were not a few hasty, off-the-cuff remarks. He designed an entire talk around an ideologically-biased and unscientific premise. We all make mistakes. To err is human. But it’s how we change our behaviour in the light of those mistakes that’s key.

Errare humanum est, sed in errare perseverare diabolicum…


[i] And today’s metal reference is*…
(*Aerosmith aren’t metal.)

[ii] I may have occasionally referred to this exceptionally important piece in previous posts.

[iii] See also #34 on this list. Oh, and #5. And #28. And #33. And #36.

[iv] Currently a trilogy of three. (This, in Adams-esque fashion, may well change…)

[v] Loaded terms like “victimhood” do not, however, lend credence to your claim to be totally unbiased.

[vi] …or do they?

Author: Philip Moriarty

Physicist. Rush fan. Father of three. (Not Rush fans. Yet.) Rants not restricted to the key of E minor...

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s