When the numbers tell different stories

It’s commonly assumed that men publish much more than women, but what happens when you take a closer look? Lynn P. Nygaard has studied what is actually being measured. Are gender differences in productivity affected by staff category, form of publication, co-authorship or academic background?

Since the 1970s, researchers have been studying gender differences in academic productivity level. In the past 40 years, the results have been consistent both nationally and internationally: Men publish about 50 percent more than their female research colleagues. The reasons given have been that women have different priorities and preferences, that they are perfectionistic in their research, or that they simply take less risk when it comes to publishing.

In 2015, Kristoffer Rørstad and Dag W. Aksnes of the NIFU research institute challenged this conclusion about the gender gap, noting that staff category is much more significant for productivity than gender. There are more women at the bottom of the academic hierarchy and more men at the top, where production is greatest. Now Lynn Nygaard of PRIO and her colleague Karim Bahgat have given us a more nuanced view of the situation.

Only journal articles studied

This autumn they published the article What’s in a number? How (and why) measuring research productivity in different ways changes the gender gap, in which they point out that how productivity is measured changes the results regarding the differences between women and men.

“In connection with my doctoral work, I became curious as to why men apparently produce 50 percent more publications than women. What and who are being measured? It was often not specified. Was it the entire university or just the senior staff? It was very unclear. As it turned out, it was mainly scientific articles in international journals that were being measured,” she says.

Book chapters not included

Nygaard and Bahgat studied a Norwegian social science institute to find out if the gender gap is really as it’s presented. They compiled statistics for a five-year period, and the data set included 91 authors and 879 publications. Their finding is that different measurements produce different results about the gender differences: In other words, the situation is much more complex than the claim that men produce 50 percent more.

One of the findings is that women have a more varied publication practice than men. They publish more often in book chapters, a fact which does not come to light if only journal articles are counted.

“Many results differ substantially – in most studies men have higher productivity, but in some studies the women had the highest, so we wanted to get to the bottom of the actual measurements. We found huge variations in the gender differences depending on how productivity is measured,” Nygaard explains.

Since they focused on the social sciences in their study, Nygaard and Bahgat could not directly compare different academic disciplines. But they compared social scientists who primarily take a quantitative approach with those who work mostly with quantitative data. In this way, they could emulate the differences between, for example, the social sciences and STEM subjects. They found clear evidence of gender segregation between the two groups.

“In the social sciences and humanities, it’s more common to publish books and book chapters than it is in the natural sciences. Also, as we know, there are gender differences between these disciplines. There are many more women in the social sciences and humanities than in the natural sciences, also known as the STEM subjects. The typical measurements don’t take this into account,” says Nygaard.

In the STEM subjects, which is an acronym for science, technology, engineering and mathematics, co-publication of scientific articles is more common than in the social sciences.

“Feminine” and “masculine” fields

Co-publication is an important point for Nygaard and Bahgat – while men are more likely to publish in teams when they hold junior or professor positions, women co-publish equally across all position levels and academic fields. In general, men co-publish more than women, especially internationally.

In addition, men use quantitative methodology much more than women, and this also affects the measurements. Nygaard believes that fields which are regarded as “masculine” may look down on qualitative methods.

The choice of methodology is reflected in the fact that women are in the majority in the social sciences and humanities, where qualitative methods are more widely used.

“It’s not the case that women don’t like quantitative research, but that men are in the majority in fields where quantitative methodology is most common and that co-publication is used most often in quantitative research,” says Nygaard.

Men more often at the top

The Norwegian bibliometric system increases the gender gap in productivity: specifically, whether the publication is scored at the highest level, known as “tier 2”. In this system, publications are assigned points based on a combination of quality level and form of publication. Men publish more than women in high-ranking, prestigious journals, or “tier 2 journals”.

“Studies have shown that women underestimate their own work, but also that they are evaluated more harshly than men. We also know from previous research that fields which are regarded as ‘feminine’ are viewed as less excellent. When women write about gender research or sexualized violence, it’s not always seen as ‘hard research’.”

Professors publish the most

The two researchers at PRIO studied what would happen to the gender gap if they included prestigious publications, fractionalized authorship, credit for international cooperation, genre, leaves of absence and research methodology in their analysis. Nygaard explains that their article is the first to point out whether or not the publication is credited to multiple authors.

“Fractionalized means that you divide up and give fractions of points to each author instead of giving all the points to one author. Do you get 50 articles in your name even though you have published together with others or are the points divided up?”

In line with the results from NIFU, the staffing category is crucial for how much a researcher has published. The majority of doctoral research fellows in Norway are women, while men still dominate the highest levels of academia.

“What happens if we compare production within the same rank, for example, junior researchers, senior researchers and professors? In many studies, the reader doesn’t know if they included junior positions such as doctoral research fellows when they count publications. It takes time before research fellows publish anything, while senior researchers must publish in order to become a professor.”

Leaves of absence shrink the gap

When the researchers adjusted for leave of absence, the gender gap shrunk. The women’s productivity increased by 12 percent and the men’s rose by 4 percent when the researchers included leaves of absence as a factor.

“If you measure productivity over a certain period without giving consideration to whether the researcher has worked 100 percent or 80 percent, of course the results will be skewed. Women take more leaves of absence than men, so it’s not right that women score lower on productivity because they have spent less time at work.”

“Is there evidence for the explanation that woman have other preferences or priorities?”

“It’s not the case that the genders have equal opportunity to choose and that they happen to choose differently. They have different choices from the very beginning. The world looks different for women than for men because we live in a gendered society. You can’t expect academia to be any different.”

“It’s assumed that everyone has an equal starting point for writing articles and that if they don’t do it, it’s their own fault – instead of the explanation being that they have written reports or book chapters,” says Nygaard.

Other explanations in Norway

“In Norway, we use a different argument. The explanations are more structural in nature, for example, that female academics have more children than in other countries, where they must choose more often between children and career.”

Research productivity is measured in terms of the success of academic publications. The Norwegian database Cristin compiles and makes Norwegian research available, and is used to distribute research funding.

“Cristin is the most advanced database in the world. It collects and quality assures information, not only about articles from journals, but also from books and book chapters. Cristin does this in both English and Norwegian. The Norwegian Scientific Index (NVI), which is used to measure productivity in Norway, is more gender neutral than in other countries because it fractionalizes authorship and includes books as well.”

Got totally different results

In their study, Nygaard and Bahgat conclude that the gender differences are less than assumed when the method of measurement and who is being compared are taken into account.

Women are more concentrated in the social sciences and humanities, and therefore have a broader publication profile. This came to light in the finding that they publish other types of works than just articles and that they have a different pattern of co-publication. When looking at the lowest and highest staff categories, there is almost no difference: Male and female junior researchers and male and female professors publish an equal amount.

So instead of men publishing 50 percent more when only journal articles are counted, the gender gap almost disappears when factors such as staff category, academic background, book chapters and fractionalized authorship are included in the analysis.

Karim Bahgat, who is currently living in the US, wrote the following comment about the findings in an email:

“When you’re going to measure differences between the genders, it’s important to understand that women and men have different distributions in academia with regard to the type of position, age, research field and methodology, and this affects which forms of publication are more common. To ensure that what you’re observing is actually driven by gender differences, it’s important to think carefully about what kind of measurement you’re using and whether you’re comparing gender from the same academic environment.”

Can give a false impression

“Nygaard, what can managers or the system do about the gender gap?”

“Norway has come far with regard to leaves of absence and pre-schools, and this makes it possible to combine academia with family life. Perhaps the challenge lies in outdated expectations of what an academic should be. There’s an expectation that everyone should be equally productive from day one, without a break. Maybe we should think instead that an academic life is built up over a long period of time and can be strengthened with slightly different life experiences.”

“Why is it important to study gender and research productivity?”

“It’s important because we like having simple numbers that we can easily toss around. But those numbers don’t show an understanding for how scientific production is actually generated. For instance, if you don’t include staff category and academic discipline in the analysis, you will get a false impression of gender and productivity. It’s critical for society that we don’t give people the idea that women are less productive researchers. In the worst case, this can give the impression that women don’t fit into academia.”

Translated by Connie Stultz.