A Timeline of psychology's replication crisis
2011: A precognition paper is published in JPSP.
2012: p-hacking is everywhere.
2013-2017: Nothing replicates.
2018: The Twitter account of a cookbook lectures one of the most prolific social scientists on research methods.
(THREAD) Inspired by
@stpehaniemlee
’s new piece, we have decided to share this. We have the dubious honor of being a victim of
@BrianWansink
and Collin R. Payne’s early work.
Absolutely delighted to have received this award for the third year running, which would not have been possible without Elson (2021ab), Elson & Elson (2021), and Elson et al. (2021)
Very excited that I will join
@unibern
next year as professor for Psychology of Digitalisation. If you're looking for a PhD or Postdoc position in meta science, privacy and IT security, or technology effects - or a combination thereof - please get in touch!
New paper with
@ianhussey
,
@TaymAlsalti
and
@rubenarslan
. We show that most measures are only used once or twice, with no tendency to agree on gold standards. This is a serious barrier to cumulative science, and leads to lots of papers without meaningful progress.
If psychological measures are used only once or twice it threatens the credibility of research
@maltoesermalte
,
@ianhussey
@taymalsalti
@rubenarslan
propose guidelines to ensure that psychological measures are standardized and unlike toothbrushes, reused.
10 years ago we conducted a large CATI study on media use in Germany. Participants were asked to name their favourite game. The interviewers themselves were not necessarily familiar with games at all, so instead they wrote down hilarious stuff like Butterworld and Call of Beauty:
@nicebread303
@chrisdc77
To me the frustration was never with being rejected, but trying to exist in a system that is overtly a meritocracy and covertly a casino.
Reviewer
#2
(round 1): Please change your hypotheses so they better fit the overall narrative.
We: No. You never reformulate your Hs after seeing the data.
R
#2
(round 2): Oh so you're biased against exploratory research. Reject.
We have come a long way. But not as far as lot D.
I tell my graduate students and RAs that if they're working 60 hours per week they they can take off 20 hours of the next because that's how overtime works.
I tell my graduate students and post-docs that if they’re working 60 hours per week, they’re working less than the full professors, and less than their peers.
Social norms are probably the most powerful predictor of expressions of prejudice.
An analysis of 105 social groups found that the normative appropriateness of prejudice was highly correlated with expressions of prejudice (r = .96!) via
@ChrisCrandall19
Suppose we have data set (Xᵢ,Yᵢ) with n points. We sort the Xᵢ and Yᵢ values independently. In my mind this completely destroys our data. But my RA says he gets "better regressions most of the time" when he does this. I have a feeling he is deceiving himself.
Ich sehe unter
#ichbinhanna
viele hoffnungsvolle Stimmen zur Bedeutung des Urteils des Bundesarbeitsgerichts zur Zeiterfassung für (Jung-)Wissenschafter:innen. Ich befürchte, verpflichtende Dokumentation ohne weitere Maßnahmen wird die prekäre Situation eher verschlimmern. /1
I am beyond excited to run the seminar "Replications and Reform Movements in Psychological Science" together with
@ianhussey
this fall. We cover the antecedents, consequences, and debate around the replication crisis in psychology.
This investigation RE: plagiarism in a dissertation takes a bizarre turn, in which the supposedly plagiarised source, a 400-page book (including a preamble by Elena Ceaușescu!), turns out to be a forgery, purposely created to damage the researcher.
Auch
#IchbinHanna
's Chef. In meiner Arbeitsgruppe sind 3 WiMis ohne Aussicht auf Entfristung, denn: ich bin selbst auch Hanna mit einer Juniorprofessur ohne Tenure Track. 1/5
In case you ever feel the job market dynamics in academia are completely arbitrary and opaque, I'll have you know that a friend of mine was asked in a private sector job interview which traffic sign she would choose if she had to be one.
Sent a data request to authors of a 2013 multi-study paper. Turns out the studies were from from 2005, one author has retired and the other left academia years ago.
They sent data, analysis scripts, stimulus materials, and lab notes within 48 hours.
Honestly don't understand how we ended up here again: Underpowered study that doesn't follow its own preregistration protocol gets published in JPSP, conclusions based on non-preregged one-tailed p = .051, paper is reported on widely and mostly uncritically.
I like to imagine that whoever maintained the "Timeline of psychology" on Wikipedia, after
@BrianNosek
et al published RP:P, went outside to have a smoke, stared at the sunset, and just never returned to their desk.
#behavioralscience
Twitter. NBC has a show scheduled for fall “The Irrational” based on Prof Dan Ariely studies & life (IMDB lists him as a writer)
NBC says “The NBC drama is based on Dan Ariely’s novel Predictably Irrational”
Novel?
Our older daughter is teaching herself to "write" by randomly combining letters she knows.
My wife: You have to show this to your father.
D (anxious): Why, is it a bad word?
I'm all in favor of data sharing, even mandatory where possible, but I've now repeatedly seen datasets on
@OSFramework
that expose research participants to unknown risk and substantial opportunities for abuse by not removing identifiers.
New paper from the lab, together with
@darahallinan
and Franziska Boehm from
@FIZKarlsruhe
! We provide general practical guidance to psychological researchers for creating GDPR-compliant consent forms.
Very grateful to
@dfg_public
to approving the grant proposal "Standardization of Behavior Research Methods" led by
@rubenarslan
and myself as part of the priority program
@meta_rep
. What are our plans for the next three years, besides finding better soccer metaphors? 1/10
Was mir in der Diskussion um das WissZeitVG bislang gefehlt hat: obskure 2000er-Popkultur-Referenzen
#IchbinHanna
, wie ich mich beweg in mei'm Büro, Hanna!
Eine von vielen. Überarbeitet. Hanna.
Ich weiß, dass ich übertrieben Hanna bin.
Warum bin ich nicht geblieben?
1/8
Short answer: No.
Long answer: Noooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.
Es ist wahr. Ab dem 01.04. ist die
#Fischst
äbchenpizza (links im Bild), belegt mit Original Käpt’n
@igloDeutschland
Fischstäbchen, als limitierte Edition im Handel erhältlich. Kein Scherz, versprochen. Teilt die frohe Kunde.
For what it's worth, I do not consider the revocation of the junior author's PhD a "payoff". It is one of the rather unfortunate events in this long and tiring story.
Although their efforts ultimately paid off — the paper was retracted, and the first author lost her PhD —
@maltoesermalte
and
@patmarkey
also paid a price.
"Dude was so clueless that he copied an entire article he’d written, then edited that article, never remembering that he he already published it two years ago."
High school students around the world got lonelier between 2012 and 2019.
@jean_twenge
and I found this in PISA data, and looked at what global trends correlated. Only smartphone & social media adoption. With links to larger reviews:
To me the main story here is not whether this study provides credible evidence for the ego depletion hypothesis, but that only preregistration allows us to even discuss the discrepancy between protocol and report. With the conventional mode, we'd be completely in the dark!
Der Verweis darauf, man könne ja Überstunden abfeiern, ist wertlos bis zynisch, je nach Lesart, wenn durch Befristung, etwa im WissZeitVG, eigentlich gar keine Zeit dafür ist, frei zu haben. Insofern: ich befürchte für die Wissenschaft null bis negative Effekte /6
Ob das für die Qualifikation so ideal ist?
Ich möchte meine eigene Situation nicht mit den Dauerkurzverträgen anderer gleichsetzen. Verantwortung für ECRs zu haben ohne eigene Dauerperspektive ist eben nur ein weiterer absurder Auswuchs der Stellenplanung in der Wissenschaft 5/5
I'm man enough to admit when I was wrong about something, so I'll go on record saying that behavioral science can indeed be very useful to fight COVID.
For today's session, we had students read
@BrianNosek
et al. (Scientific Utopia II) and Shu et al. (Signing at the top), without telling them it had been retracted. We asked them for applications of the techniques from the latter to achieve the goals of the former. /1
I am beyond excited to run the seminar "Replications and Reform Movements in Psychological Science" together with
@ianhussey
this fall. We cover the antecedents, consequences, and debate around the replication crisis in psychology.
Very happy to announce this support & promotion scheme for
@RegReports
by JMP and the Media Psych Division:
1. Submit a Registered Report to JMP until May 31, 2019
2. Present your work in an RR Panel at
#MediaPsych19
in September 19
3. Compete for financial support for your study
New preprint out now! We discuss assumptions underlying mediation, moderation & their combination. Come for the stupid subheadings, stay for the causal inference mess! With
@PHuenermund
@rubenarslan
@maltoesermalte
We've seen lots of arguments about the causes of psychology's replication crisis: incentive structures, mindlessness, statistical inaptitude. But what if the key to such meta-scientific questions was inside of us all along: THE BRAIN!
(1/12)
Momentan machen alle Überstunden und alle wissen es. Aber wenn erstmal rechtsverbindlich dokumentiert ist, dass es (scheinbar!) gar kein Problem gibt mit Arbeitsverhältnissen in der Wissenschaft, wird der Reformwille sinken. /5