Paper Review: ‘Writing Matters’

A study on how professional editing might help economists publish papers

Is it useful to care about the quality of the writing in an academic paper?

The answer seems obvious, but remember: academic papers aren’t written primarily to communicate ideas. They’re written for the purpose of being accepted at a conference or journal so that you can add them to your academic CV. So, does the quality of your writing increases the odds of being accepted by referees, editors, and conference committees? This is the question studied by Feld et al. in their March 2022 paper “Writing matters”.

Jan Feld is an economist in New Zealand. The two other coauthors, Corinna Lines and Libby Ross, are professional “plain language” consultants. They devised an experiment that consisted of getting unpublished papers from economics PhD students, creating a professionally edited version of each, and asking a bunch of economists and writing experts to rate the quality of a random sample of unedited and edited papers.

The results: Writing experts thought that the professionally edited papers were better written, and the economists thought that the professionally edited papers were more likely to be reviewed positively and accepted at conferences and journals. There you go: writing matters!

But by how much? The Flesch-Kincaid grade level score estimates the school grade that a reader should have to be able to understand a text. The original, unedited papers had an average score of 15.3 for their introductions (meaning you should be above “grade 15,” so something like senior undergraduate level); the edited introductions had an average of 14.7. The chances of accepting an edited paper for a conference or good journal, according to the economists, were improved by 8.4% and 4.1% respectively over the unedited versions. Not too surprisingly, these numbers were higher when the original paper was rated as particularly poorly written.

How convincing are these results? An improvement in the Flesch-Kincaid score of 0.6 year of education doesn’t sound that amazing, and the one-digit percentage improvements in acceptance rate aren’t mind blowing either. But the authors do conclude that writing matters. They even suggest that it would be a good philanthropy cause to offer free professional editing to researchers, especially non-native speakers of English. My impression is that this comes from a logic of “every little bit counts”: publishing papers in the right places is so important to a scientist’s career, yet it is a process that lies outside their control for the most part, so they should make sure that each part that they do control is done well.

An obvious limitation is that this study looked at the effect of professional editing. Professional editing is done by people who are not economists or academics, so they can’t improve on the actual information being presented, only on the style. Furthermore, they can’t use much creativity in their work — any creativity should stem from the intentions of the original authors.

I suspect that such editing leads to avoiding the worst (tedious, awkward writing that is full of jargon) but cannot generate the best (original, fresh papers that express new ideas in a pleasant way). The paper “Writing matters” itself, in my opinion, is a piece of evidence for this. Although it is fairly easy to read, it’s also rather drab. I wouldn’t personally qualify it as “good” writing, but it is better than most academic papers I’ve read.

On the bright side, this approach to editing may be scalable, since it is standardized. It also takes into account the scarcity of professional work time: the editors were instructed to spend a reasonable amount of time on any paper, and the total ended up around 6 hours per paper.

The editors also worked from a set of guidelines created by Feld et al. These guidelines (published in the appendix) emphasize that most of the editing time should be devoted to the title, abstract, and introduction, which are the parts with the highest impact. Any editing done on the rest of the paper should aim at optimizing skimming, which I like. Few people do more than skimming the methods and results of an academic paper, especially when they need to quickly decide on acceptance or rejection.

Here is the general outline of the editing process from the appendix:

The goal is to edit the paper so that an expert who has 10 minutes to evaluate the paper will understand it more easily. We’ll focus on improving the title, abstract, and introduction using these guidelines. For the rest of the paper, we’ll focus on making the paper easier to skim read. We’ll prioritise what we edit based on what (in our experience) will best help the reader easily understand the paper while skim reading. We’ll edit for readers who can comfortably read academic documents, but who aren’t familiar with the discipline of economics.

If the writing is very poor, we’ll rewrite some parts instead of editing.

There are many good reasons we’ll deviate from these guidelines: if it ain’t broke, don’t fix it. If we’re unsure, we’ll ask the writer.

To save time, we may outsource some tasks to the writers. For example, we might ask them to round all numbers in a table to 2 decimal points.

This is a good set of guidelines. But of course, like any guidelines, following them closely will not result in great writing. They are more of a stopgap measure to avoid the excruciatingly bad writing that could lead to a quick rejection.

I’d be curious to read a study that compared great, original academic writing with ordinarily edited writing. But would it be possible to even design such a study?