Media goes gaga over study showing women code better than men. The truth’s more nuanced.

A lot of pioneering computer scientists were women. What happened since then? Image credit: Wikipedia

The media ran amok last week with a research study which looked at gender bias on Github, a site where programmers can submit open-source code vetted by a community.

Vice went for the balls with this simplistic headline: “Women Are Better at Coding than Men.”

The article itself devoted only two paragraphs to the study, which included this line: “According to their findings, code written by women is viewed more favorably when the coder’s gender is kept secret.”

The writer then hit a homerun with this:

If feminism has taught us anything, it’s that almost all men are sexist. As this GitHub data shows, whether or not bros think that they view women as equals, women’s work is not being judged impartially. On the web, a vile male hive mind is running an assault mission against women in tech.

Somebody give her a Pulitzer Prize.

Business Insider too kept its hype dial at maximum volume, with this opening to its report: “A new research report shows just how ridiculously tough it can be to be a woman programmer, especially in the very male-dominated world of open-source software.”

BBC did better by writing up the study in a more neutral and buttoned-up fashion. But it still wasn’t nuanced enough. It wrote: “computer code written by women has a higher approval rating than that written by men – but only if their gender is not identifiable.”

Now, here’s what you really need to know about the study:

1. The study hasn’t been peer-reviewed.

Peer review is a process in the science world in which a research paper is evaluated by other researchers before being submitted to academic journals. It’s an important step to verify the credibility of the findings.

The paper in question was submitted online for public feedback, but has not been peer-reviewed. There’s no way to miss this: if you click on to the web page, the words “not peer-reviewed” are marked out in bold red and uppercase. Even a blind bat can see it.

Of the three news reports I mentioned, only BBC highlighted this fact. So either the other two writers had no clue what a peer review is, or chose to omit the fact for the sake of clicks.

2. The research is much more nuanced about whether women have a higher chance of getting code approved.

3. The conclusion that women are better coders than men appears shaky.