Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?
This is your opportunity to ask questions. No question too simple or too silly.
Culture war topics are accepted, and proposals for a better intro post are appreciated.
Jump in the discussion.
No email address required.
Notes -
Does anyone have sources that go over the replication crisis? I am collecting information for a podcast I want to do.
So far I've got:
https://www.palladiummag.com/2024/08/02/the-academic-culture-of-fraud/
https://today.ucsd.edu/story/a-new-replication-crisis-research-that-is-less-likely-be-true-is-cited-more
https://www.honest-broker.com/p/the-ten-warning-signs
Well crap, I could have sworn I saved some links but apparently not. I do have these from a blog that has a whole section on replication more generally as well as a "file-drawer" section too. Some good reads there.
I would strongly recommend reading the original Ioannidis paper at a minimum to get an idea for not just what started it all, but some of the most prominent arguments from the beginning. Also, there are a plethora of response articles and counter-responses that can also be read.
It's important to also realize that "replication crisis" has a few different meanings. One the one hand, historically and sociologically, "science" reached a broad point where people "realized" that they had to take replication failures seriously. Ever since that "crisis", we've seen a much higher awareness of the issues, as well as a bit of institutional action.
But then statistically and more precisely, the crisis can refer to a few interrelated but still distinct phenomena. You've got the "file drawer problem" and "publication bias", you've got outright fraud and faking data, you've got "p-hacking" and fishing expeditions, you've also got lackluster published methodological info that makes faithful replications impossible, you've got generalization across culture issues along with sample problems (more men than women, too many college students), lack of money for properly-powered large-sample tests, etc.
There's also a distinction between exact replication, and conceptual replication. Famously in the psychological sciences, they got an extra-big black eye because an astonishing number of famous psych studies, where the author would go on to write books and achieve wide fame and give speeches, showed to conceptually be completely bunk in real life. Power poses, shopping while hungry, finite willpower, marshmellow test, certain types of priming, Standford Prison, Mozart, implicit racial bias tests, type A and type B people, all of these have reached cult pop status and nearly all of them were misrepresented or failed to actually have the real-world implications people were told to expect.
Ty for sharing! Yeah it's pretty bad stuff. Sigh.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link