The Velocity of Knowledge

Optimising Research Utility through Rapid Peer Review

As much as I gripe, the peer review system plays a crucial role in both the supply and circulation of knowledge. It acts as a gatekeeper, ensuring that only high-quality, credible research is added to the supply of knowledge. But as the review process can be time-consuming,¹ it impedes the dissemination and thus usefulness of new findings.

The question is: At what point does the marginal increase in quality from an additional review get outweighed by the slower velocity?

To model this system, we introduce three key functions:

1. Quality of Research (Q(t)) : An increasing function of time, reflecting the cumulative effect of peer review over time, but with diminishing returns, so lets assume a natural log function.

\(Q(t) = \ln(1 + c t) \)

Here, c is a constant representing the rate at which peer review improves research quality.

2. Timeliness of Research (T(t)): A decreasing exponential function of time, capturing the diminishing relevance or impact of research as it becomes older. We assume a negative exponential, in line with economic discounting.

\(T(t) = e^{-i t}\)

where i is the discount rate, representing how rapidly the timeliness of the research decreases over time. We see for instance that older papers are cited less than recent papers, but whether that is because they are less valuable is not obvious.

3. Utility (U(t)): A function of Quality and Timeliness, representing the overall utility of the research. We assume it is the product of those two terms.

\(U(t) = Q(t) \cdot T(t) = \ln(1 + c t) \cdot e^{-i t}\)

Quality, timeliness, and utility of research vs. time.

Above is the resulting plot, simplified by removing the numbers on the axes to focus on the relationships between the functions.

We observe that the quality improves over time with peer review, but with diminishing returns. The relevance of the research diminishes over time. Therefore, there is a peak in utility, indicating the optimal time to allow for peer review to balance the quality of peer review and the timeliness of dissemination. While the value depends on the modeled parameters, and is hard to know in retrospect and is probably impossible to know looking forward, I am fairly confident it is a lot shorter than most journals in the transport field assume, and should be measured in weeks rather than years.

This question has obvious relationship to the Open Access journal Findings, which I edit. We are battling a reverse salient against sluggishness. (See our most recent Newsletter.)

This issue is not unique to one journal but is a systemic problem across academia. Journals are often too slow to meet societal needs. Editors frequently delegate decision-making to reviewers and hesitate to apply their own judgment. Reviewers face skewed incentives: if they favor a paper, particularly if it cites their work, they are inclined to approve it; if they oppose it due to ideological or professional disagreements, they are likely to reject it. However, their motivation is generally weak since they are volunteers. Reviewers are unpaid and largely unrecognized, acknowledged only through annual lists, minor CV entries, and occasional support letters from editors during promotion processes. Consequently, reviewing often becomes a low priority, particularly for senior academics, for whom an additional citation has diminishing returns.

If asked to review for a good, open access, non-profit journal, please do so quickly (or decline immediately). The worst outcome for an author is a slow rejection.

If asked to review for a bad, for-profit journal,² prioritise based on what they pay you.

The Velocity of Knowledge, Envisioned by Dall-E.

1. My longest reviewed paper, almost 10 years in the system, thanks Transportation Research part A.

2. All for-profit journals are bad, some more bad than others.