"It Shouldn't Happen to Dogs," Says Dogs
Grant funded research comes under Trump's scrutiny: Is there reason to be concerned?
Trump 2.0 has caused considerable consternation in the academic community, particularly related to his targeting of research funding. In classic Trumpian fashion, his administration’s approach, the latest of which involves capping “indirect” grant costs to 15%, has been chaotic. Some bits, such as freezing grant awards already awarded have been rolled back by the courts, but others may stick. Is important research in the United States facing a Trumpacalypse? Or is this a well-needed correction given considerable inefficiency, grift, and unreliability in the research grant system? As is often the case, the truth is complicated.
I’d gotten in a brief online conversation with psychologist Todd Kashdan about this and he encouraged me to do a longer post. So here it goes…thank (or blame!) him for this! The gist though is that Trump’s attack on academia is likely to be bad. But this is a hole academia dug for itself.
I’ll note too…I’ve been an academic in some form for nearly 30 years (21 in a tenure-track position). I’ve served on numerous grant review panels, as well as on the APA Council of Representatives. On the other hand, I’ve never gotten any big grants…it’s not something super crucial at a liberal arts university. I’d say my perspective is reasonably well-informed but without the vested interest but interpret my background how you like.
Academics howling about being defunded is not a surprise. Any group losing its gravy train is going to be pissed. If you told hot dog venders you were going to outlaw pork, the reaction wouldn’t be much different1. Calamity! That’s human nature, neither a good nor bad thing. It doesn’t mean that this makes academics wrong, but we’ve set up a lot of academia, particularly at big research universities, to make grant-getting a matter of prestige and even job security. So, pulling the rug out is going to create a lot of consternation.
But I (and others) had been trying to warn academics this was coming for some time. That’s because a lot of research is really, really bad, some of it is an obvious waste of the taxpayer’s dime, and academia has allowed itself to slide into outright unapologetic politicization and snobbery when it comes to the general public. Universities have lost the public trust, and a lot of that is well-deserved. Academics could have been alert to that and engaged in modest data-based reforms but, nope, there was to be none of that. And now we’re where we’re at. With a wrecking ball staring at us.
Yes, a lot of this has to do with the dumb politicization of science. That so much of academia can’t just admit that DEI (Diversity, Equity and Inclusion) has been an abject failure, whatever the good intentions, is depressing. If even the New York Times, hardly the bastion of deep-red thought, has big essays pointing out what a sinkhole of tax dollars DEI is, it’s time to pull the plug. Instead, some federal granting agencies included DEI requirements explicitly in proposals and it’s probably fair to say that DEI discussions were likely part of broader (and under the radar) conversations about grants and which should get funded.
I can say that, even for grant proposals that didn’t require DEI components, applicants really seemed pressured to make some kind of DEI argument. I’m guessing they picked up on the likely mood among reviewers and I can’t blame them. Though not related to grants, I recently sat in on a focus group about a new psychology textbook. Just about everyone else in the focus group of maybe 15 gushed over the new book’s diversity “Look at all the pictures of black people in the book!”, whereas almost none commented on whether the book covered scientific information accurately. I actually raised a comment about being unable to judge the book’s scientific accuracy after reading only one chapter on an area I didn’t know well. It was politely received before the conversation quicky returned to diversity. An anecdote to be sure and, again, not a grant review, but I think this is the current culture in far too much of academia. Diversity is good, but the obsession with it…to the ignorance of pretty much all else, including telling the truth has become cult-like and the general populace has noticed.
When you have the scientific community telling people during covid19 to shelter in place and avoid crowds…unless those crowds happen to be protests and riots celebrating progressive causes…losing the trust of the public is the inevitable result. This was a mistake…an arrogant and stupid mistake no less…that played no small part in the current moment.
The bigger issue here is that so much funded research is crap, whether it’s politicized or not. To be fair, this is going to be at least somewhat discipline specific. I’m willing to believe that astrophysics is more rigorous than sociology. Perhaps some of it also is important enough we tolerate a kind of “needle in the haystack” effect. I don’t believe medical research is much more rigorous than psychology (which is to say both are permeated with junk), but maybe we’re willing to take more risks with medical science given its importance in promoting health on those comparatively rare occasions when it is pretty good.
When I was talking to Todd, he raised the question of whether we wanted to risk defunding research on teen suicide or neurodegenerative disorders. But I’d say these are both examples of why so much research funding is a money pit.
Jesse Singal just happened to do a post on Alzheimer’s research which has been riddled with outright fraud and ideological rigidity for decades. So much so that decades were wasted on bad theories as to Alzheimer’s cause, and advancements in this area have been remarkably modest given the research investment2. This line of research is a great example of why it may be time to pause taxpayer funding until researchers can get their act together and increase the rigor of what they are doing.
I’m more familiar with teen suicide research given it overlaps with much of the media stuff I do (particularly now with social media research). Psychology, as a research discipline, is a complete and utter mess. It’s very close to an outright pseudoscience…unable to acknowledge that most results are likely noise, a complete ignorance of effect sizes, a continued obsession with bivariate correlations and p-values, few grounds for theory falsification leading to a plethora of undead theories, shot through with progressive politics, etc., etc.
Take the current moment. People are so beholden to yet another moral panic attempting to link teen suicide to social media they’ve remained ignorant to data that’s right in front of them (in the CDC’s Youth Risk Behavior Survey) that it’s mostly neglectful and abusive parents promoting teen suicide3. And that teen suicide, far from being isolated, is deeply correlated with high levels of suicide among their parents’ generation.
Even this moral panic aside, do we really know a lot more about teen suicide than we did 30 years ago? No, I don’t think so. Bullying and parental neglect and other adverse childhood events are bad. We knew that then, we still know that now. But the psychological science community (and medical community) have been without value in addressing the last decade’s rise (and recent decline which barely anyone seems to know about) in youth suicide…again, particularly that it occurred concurrent with a far more massive rise in their parents’ demographic suicide rate This is something that should have raised curiosity but mostly didn’t as we got sidetracked by cellphones and social media panics.
Don’t get me wrong, I do think psychology has some bright stars. Off the top of my head, I’d list the Big-5 personality stuff and behavioral genetics as gold medal social science stuff. But aside from that and maybe a few other areas, is social science in a better place than 50 years ago? No, I don’t think so. Not that we should dump millions or billions of dollars into, certainly. The good spots are good despite social science’s complete lack of rigor. Even if we were to ignore psychology’s replication crisis, the truth is that even most “open science” studies are of poor quality and produce results that are unreliable, likely noise, and of trivial practical importance.
Or put a different way, social science remains in a mainly defensive posture rather than one interested in improving rigor. Fewer studies, with advanced criteria for hypothesis support (including substantial effect sizes) would be a good direction. Alas, that’s not where the perverse incentives remain, even with the open science movement4.
In this sense, defunding most social science might actually be good. The government refusing to fund any more trivial garbage could prompt the field to do some real soul-searching about how to advance rigor and theory falsification/support.
As of this writing, 22 states have sued to block Trump’s attempt to cap the indirect costs of research grants to 15%5 (a federal judge temporary granted the order). Lee Jussim has what I think is a fair rundown of the indirect costs issue. It’s a pretty good overview for the uninitiated and he’s a bit generous toward universities if anything. It will be interesting to see how this all plays out in the courts.
It's interesting in some of the news coverage of this how the arguments bounce between the “lifesaving” nature of the research (which is likely true for a microfraction of research being done, even if we make the argument that research builds on itself which, of course it does), but also the economic costs of layoffs. This latter is a compelling humanitarian argument (people losing jobs sucks). But it’s not a compelling scientific argument…should we keep paying for junk science because it keeps people employed? Ironically, I remember similar arguments made about keeping useless military bases open, or building unnecessary weapons platforms so long as it kept people employed6. This is precisely the kind of government waste Republicans are banging on about.
Ultimately it probably would be good for the research grant process to be overhauled. It would be best if it came from within the academic community rather than the bull in the China shop Trump administration. Here are some broad thoughts of mine:
1) Any review of the grants award process would need to be done by uninvested individuals capable of critically reviewing scientific rigor. This mostly likely means “hard” scientists, although scientists should not be reviewing their own field. As we see from psychology, they just get too defensive and internal standards can be too low.
2) It’s possible we may need to prioritize which research is both more reliable and more needed. Here, again, I think most social science and the humanities fails this test. Fields such as psychology or sociology may actually benefit from having their feet held to the fire and cut off from the sweet, sweet research loot.
3) Medicine is probably the toughest field to deal with given it combines high value with low reliability (and apparently, as alleged by Singal’s article on Alzheimer’s, significant fraud).
4) Grant proposals should be reviewed strictly on scientific merit. Not DEI, nor any conservative fever-dream. It’s impossible to take values entirely out of science, probably, but there’s no need to make it blatant.
5) Many grants are written in such a way as to promise a particular result. This is probably one pressure that leads to junk findings. Granting agencies should be clearer that they don’t expect the results to fall one way or another and that null results won’t be used to penalize applicants in future applications.
6) Preregistration should be required for all empirical grant applications.
7) For empirical studies in the social and medical sciences a smallest effect sizes of interest (SESOI) of r = .10 should be universally applied for hypothesis support. This is actually pretty modest. I’d really suggest r = .20 for practical significance. No, no exceptions. Exceptions are the devil’s work.
8) Trendy sociopolitical theories should be deemphasized for grant support.
9) Peer reviewers for grant applications should be reasonably paid. They should be educated in the methods used in the topic area, but not directly invested in the hypothesis being proposed. Those who push trendy sociopolitical theories in their reviews should not be asked to review again.
10) Grant recipients should promise in writing to publish (or make reasonable attempts to publish) their preregistered designs, whatever they find. There should be financial penalties for those who do not (meaning return of some or all of the grant funding). Recently, the New York Times alleged a grant funded research project on transgender medicine for youth failed to publish some research findings because they were unfavorable to sociopolitical causes the lead investigator favored7. This should not be allowed to happen.
There are probably other reforms that would be positive that I’m not thinking of. Again, it would be great if some kind of reform movement started from within academia or funding agencies rather than foisted on us by political entities with their own agendas. Admitting our problems and taking concrete steps to fix them can only make us stronger.
“But our stuff is life-critical!” the scholars will say. I’d say a life without chilidogs is not worth living.
This isn’t merely academic for me…there’s family history of Alzheimer’s which means I might have my own bullseye on me! As often as I search desperately for people’s names and words, I do wonder if it’s coming sooner rather than later. So it would be nice of people got on the ball here.
Mike Males has done a lot of important work with the CDC’s Youth Risk Behavioral Survey that, for some reason, the CDC itself seems entirely disinterested in. Of course, the Trump administration taking this information off the internet is going to make further data-gathering difficult.
I’m not trying to pick on the Open Science Movement here. Many of their solutions such as preregistration were definitely necessary. But they’re not sufficient to improve social science’s rigor. Maybe now we can just reliably replicate the same garbage over and over.
Which, I read somewhere, would actually be consistent with caps in Europe. When the US is more progressive than Europe that should often serve as a red flag.
Progressives and conservatives are never really that different in the end.
I’m generally sympathetic to individual human frailties. People make mistakes. There but for the grace of god and all that…in this case I’d rather she be given a chance to swiftly publish the real results rather than ruin her career.