On the State of Science

"Science is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe." -- (Wikipedia).

To be clear, science is a process, not a set of facts or findings. It produces facts and findings, and refutes them from time to time. Much of the scientific enterprise now occurs at research universities, which are strange, chimerical beasts charged with both advancing human knowledge and entertaining 18-22 year olds.

To be clear, I think science has been extraordinarily successful as an enterprise, moving from a fringe program pursued by a few mostly wealthy individuals with excess leisure to one with millions of practitioners, responsible for most of the world around us.

However, the enterprise is also far from perfect.  As with any social system, status games matter.

The first problem is Pseudo-Science vs Science. While science generally wins these battles, that it keeps fighting them is a drain on more productive activities. Carl Sagan spent a surprisingly large time in the original Cosmos series complaining about astrology. Stephen Jay Gould famously devoted much ink against creationism. They were famous, media-savvy scientists, not like most practitioners, who could proceed day-to-day conducting normal science. But they also captured attention fighting, since the media and the public like nothing better than conflict.  Today, the anti-vaxxers may get millions killed from preventable diseases, and simultaneously chill discussion of real risks associated with vaccines, as any risk is exaggerated and inflated. More attention is given to these ideas than they are worth.

Examples of trending pseudo-science:

1. Biblical Literalism / Creationism / Intelligent Design / Young Earth

2. Climate Deniers

3. Anti-Vaxxers

4. Astrology

5. Flat Earth

6. Alchemy

But like perpetual motion and phlogiston before them, wrong, refutable ideas are pushed to the fringe. As long as we hold fast to the process of conjectures and refutations, and ensure testable hypotheses are presented, science triumphs.

The second problem is Publish or Perish, and its reaction. Publication is essential to science, it is how knowledge is communicated. The US Promotion and Tenure (P&T) system requires academics to go up for tenure, usually after  6 years, at which point they get promoted or are relieved of duty. Other countries have less existential systems. Promotion at good schools requires evidence of journal publication, and sometimes requires that some number of these publications are in journals of a certain tier or have a certain impact factor (which is completely backward, and frankly embarrassing that the logic skills of those checking on impact factor is so poor). (Notably, it is rare that promotion committees check the actual citations of the works themselves or other measures of impact, the excuse given that the publications are too new to have garnered many citations).

While peer review predates journals, modern peer review is a relatively recent invention that acts as a random throttle on the release of knowledge.

As we learn from statistics there are Type I and Type II errors: False positives and false negatives. False positives are results that are published, but are wrong, meaningless, or trivial. False negatives are results that are rejected, but were not wrong, meaningless, or trivial. Peer review aims to reduce false positives, at the cost of false negatives.

Some evaluating agencies even complain that there is too much science being produced (they frame it as too many publications, but publications just document science). They are whining the knowledge is too hard to process, that authors salami-slice results to maximize the number of publications. But what is the right size of a result? The more words to convey the same information, the fewer the readers I suspect, all else equal.

There is bad science being produced, as the incentive is to produce publications, not science, and peer review is a highly random filter.

These problems include Lack of Reproducibility and Transparency and P-hacking, among others.  I have seen a few academics engage in Dual Publication (publishing the same paper, or essentially the same paper) in more than one outlet in parallel, and try to take credit for both. To be charitable, this is especially a risk with multiple authors who are not communicating well, or with conferences that publish, or semi-publish, proceedings. It is also done intentionally.

But there is also good science being produced, and authors spend too much time satisfying reviewers to jump through the peer review hoop to publish in "legitimate" journals so that their research will be accepted by the community of cited authors and itself accumulate citations. These hoops include reviewers and editors requiring additional citations and long introductions, literature reviews, and policy discussions that are largely irrelevant to the knowledge actually produced. Transport Findings aims to remedy this problem of too much verbose bullshit contaminating science.

Good science rejected often winds up in desk drawers, and is needlessly replicated by those who don't know the work was already done, since it was never published. Good science that is not in the excessively standardized form of a 6000-word paper is hard to publish.

A photograph, because articles with photos are more likely to be read, and take up more screen space on Twitter. Photo by Flickr on Pexels.com

I think we worry too much about false positives and not enough about false negatives, Science is self-correcting, and a wrong result will eventually be discovered, and the paper either retracted or come-to-be-recognized as wrong. Academics with a reputation for many errors will eventually be discovered. Mistakes happen, and not all are due to evil intent.

Publish or perish is a function of the current state, and to its credit helps ensure academics overcome their fear of the imperfect and lack of performance for anything but externally imposed deadlines and ensures papers get out. Failing to make it through this system is not a comment on you personally, and there are many ex-academics leading perfectly productive lives.

Third, the Publications Crisis sees the cost of reading science (legally) getting higher and higher. Scientists are good at routing around problems, and most papers can be acquired conveniently from sci-hub for no charge (and it is easier to use than your local University library, what does that tell you?)

These for-profit, closed-access journals have their own incentives to jack their "impact factor" so that more authors will send their papers there, and more subscribers will read the journal at higher and higher prices. Journals play games like "online first" to garner citations before the date of publication to increase the impact factor.

These journals also engage in `Positive results bias', being more likely to publish articles that have positive findings (A significantly affects B) instead of null findings (A and B are unrelated). While in a complex world, most things are unrelated, if someone hypothesized that they were, and we find they are not, that is still a valuable contribution to knowledge.

The science journalism community is complicity with this hype cycle, since 'news' requires everything to be framed as a breakthrough rather than normal science, as if (a) most 'breakthroughs' are real, and (b) progress comes through 'breakthroughs' rather than plodding. I'm all in favor of working smart rather than working hard, but actually collecting data and promulgating theory go hand in hand.  Most of us cannot be Newton, Darwin, or Einstein, and we cannot be continuously overthrowing Kuhnian paradigms. There is a bias toward Ravenclaw over Hufflepuff, but most work is Hufflepuff.

There are open access journals that fight this, like Transport Findings and Journal of Transport and Land Use, among others. Unfortunately, there are also fake and low-quality journals that are simply pay-to-play, taking advantage of naive and noob researchers, and those stuck in a Publish or Perish situation with weak oversight based simply on numerics rather than quality, which have become easier and easier to start in the age of the modern internet.  The community can sort this over time.

Fourth, the Academic Ponzi Scheme occurs because each academic at a top school is expected to produce Ph.D. students who themselves become academics at top schools, repeat. Many more PhDs are awarded than faculty positions exist. Most go to lower-ranked schools or industry, and lead happy, healthy lives (alternatively, maybe they retain anger and resentment forever, which is the impression I get from reading the Chronicle.) Some hang around and become adjuncts, working on temporary contracts, teaching more (and researching less), with a job that could disappear at any moment when the economics of the university program change.

Fifth, there is a lot of discussion of self-censorship due to Political Correctness. I don't see it much in my own world, but that doesn't mean it doesn't exist. Of course, everyone self-censors, one cannot, for instance, expect anyone to say bad things publicly about colleagues they have to work with in the future, or funding organizations that pay them now or are expected to in the future. But that is common sense.  Most of us with tenure or the equivalent have academic freedom to say what we want as long as we do so politely, without slandering or libeling people, etc. There are exceptions, the case of Paul Mees comes to mind, but that is more an issue of academic freedom than political correctness.

Finally, I want to discuss Problem Invention,  bringing distant dangers near. All the low-hanging fruit has been picked, so too much attention is paid to minor issues. I don't believe it is my place to pass judgment on the importance of work as a reviewer or editor, history can judge. But I sure wish y'all would work on better problems. Much of transport research is about mathematical and statistical games with little practical application. Transport is only important because it is a practical problem.

The problems within science, unlike so many other sets of problems, are largely self-correcting.