Gustav Nilsonne Profile Banner
Gustav Nilsonne Profile
Gustav Nilsonne

@GustavNilsonne

2,672
Followers
2,963
Following
129
Media
5,313
Statuses

Associate professor at Karolinska Institutet, researcher in metascience. Affiliate at Stanford University.

San Francisco
Joined August 2013
Don't wanna be here? Send us removal request.
@GustavNilsonne
Gustav Nilsonne
5 years
In the last years, I have tried to always publish data and code openly, as far as possible. Just received this comment from a reviewer, who did not appreciate the value of doing so: (1/n)
Tweet media one
77
432
2K
@GustavNilsonne
Gustav Nilsonne
4 years
A paper last week claimed to show that an EEG-based model could predict response to antidepressant treatment. Important, if true. So I took a closer look and reanalysed some of their data. The model doesn't work. Here's the story: (1/x)
20
241
583
@GustavNilsonne
Gustav Nilsonne
5 years
I, for one, would be happy to get rid of the peer review theater, which creates an illusion of quality certification, but which is slow, capricious, and hinders progress, in favour of letting authors stand for their own work and doing systematic post-publication review. (4/4)
19
55
493
@GustavNilsonne
Gustav Nilsonne
5 years
Does this not suggest that peer review is less of a quality control mechanism and more of a barrier to presenting claims that contradict prevailing theories? (3/n)
6
43
493
@GustavNilsonne
Gustav Nilsonne
5 years
This makes me reflect: I have never had data or code peer-reviewed, except in data articles, even though the data and the code are supposedly what give credence to my claims. Instead, what is reviewed is the manuscript containing my narrative interpretation of the results (2/n)
13
26
328
@GustavNilsonne
Gustav Nilsonne
6 years
Hooray! Sweden cancels agreement with Elsevier, as part of the move towards open access. Govt paid 12 million euros last year to buy back access to texts that researchers gave the publisher for free or against a fee.
0
143
279
@GustavNilsonne
Gustav Nilsonne
3 months
The BMJ is looking for a freelance statistical reviewer, offering to pay £115/review. Yes we can professionalise peer review and pay academics for peer review as a service.
12
76
221
@GustavNilsonne
Gustav Nilsonne
4 years
@wendy_bickmore I appreciate your situation. But the system is built on free work by reviewers to support incredible profit margins and a lock-in of results. Another option would be to take this opportunity to start paying reviewers for your journal. Or, better yet, to dismantle the system.
4
4
217
@GustavNilsonne
Gustav Nilsonne
4 years
Testing a new approach to peer review requests:
Tweet media one
9
21
199
@GustavNilsonne
Gustav Nilsonne
2 years
In October I will again teach the PhD course in Open Science and Reproducible Research @karolinskainst (3 ECTS credits). The course is fully online and applicants are welcome from universities worldwide. Applications now open:
0
74
173
@GustavNilsonne
Gustav Nilsonne
2 years
On gatekeeping in scientific publishing: a defense of "predatory journals" 🧵 1/
4
49
157
@GustavNilsonne
Gustav Nilsonne
4 years
Very happy to announce that next semester I will once again give the PhD-level course in Open Science and Reproducible Research at @karolinskainst , fully online. Application open to PhD students and postdocs from any university in Sweden.
7
36
110
@GustavNilsonne
Gustav Nilsonne
2 years
Peer review and publication in a scientific journal should never be treated as canonisation of a claim as accepted. Words do not mysteriously become science by getting bundled with a publisher's logo on a pdf. 8/
3
11
93
@GustavNilsonne
Gustav Nilsonne
3 years
The #EEGManyPipelines project is now open for teams to register! The teams will independently analyse the same EEG data and hypotheses. The project will then map the analysis pipelines and their effects on results.
0
53
89
@GustavNilsonne
Gustav Nilsonne
4 years
The paper has been covered by several news outlets. I wrote a small commentary about this and submitted to the journal this morning. We'll see what they say... (8/8)
@NatureBiotech
Nature Biotechnology
4 years
An electroencephalographic signature predicts antidepressant response in major depression
Tweet media one
7
48
111
9
7
81
@GustavNilsonne
Gustav Nilsonne
1 year
My course in open science and reproducible research @karolinskainst will be given again September 19-28, fully online and open to applications for PhD students and postdocs from anywhere. Welcome to apply! 1/2
1
54
77
@GustavNilsonne
Gustav Nilsonne
2 years
I see it almost completely in reverse: the quality of the scientific literature is threatened mainly by publication bias and "file-drawering" of results. We have too strong and much too biased gatekeeping. 6/
2
8
73
@GustavNilsonne
Gustav Nilsonne
2 years
Do you want to do a postdoc in metascience? I am seeking 1-2 candidates whom I can host for a Marie Curie fellowship. Strong support available for grant writing. Other funding also possible. Please DM if interested.
4
59
73
@GustavNilsonne
Gustav Nilsonne
6 years
I often advise colleagues about open data. But for interview-based research, I have only been able to give superficial recommendations. So when my colleague @Cormac_McGrath proposed to collaborate on this issue, I was delighted. Here is our preprint:
6
36
69
@GustavNilsonne
Gustav Nilsonne
5 months
Now hiring: a postdoc in metascience to work with me on testing interventions for improved reproducibility in the @iRISE_EU project at @karolinskainst . Please RT.
Tweet media one
0
54
65
@GustavNilsonne
Gustav Nilsonne
2 years
It's high time to replace the academic journals with sholarly-led open platforms, as argued eg here by @brembs , myself, and others: 12/
2
3
65
@GustavNilsonne
Gustav Nilsonne
1 year
Happy to be back at Stanford to visit with John Ioannidis for a few weeks. If anyone is in the Bay Area and wants to have a chat, please let me know!
Tweet media one
4
3
64
@GustavNilsonne
Gustav Nilsonne
6 years
Calling on Swedish colleagues! It's time for us to make a stand for open science and support the Bibsam consortium after they canceled national Elsevier subscriptions. DM or e-mail me if you want to help think about what we can do! Should get moving pretty soon.
12
37
63
@GustavNilsonne
Gustav Nilsonne
1 year
Excellent feature in @TheEconomist on research fraud; includes this diagram which I think might be an instant classic:
Tweet media one
4
8
61
@GustavNilsonne
Gustav Nilsonne
3 years
Happy to see the @eegmanylabs project featured on the cover of @CORTEXjournal . Position paper:
Tweet media one
2
10
59
@GustavNilsonne
Gustav Nilsonne
3 years
Now I have read the provocative paper by @KAuspurg and Josef Brüderl on interpreting multi-analyst studies. A thread: (1/x)
Tweet media one
1
17
60
@GustavNilsonne
Gustav Nilsonne
6 years
Now published in Royal Society Open Science @royalsociety : Michael Ingre and I estimate that a lot of negative findings are suppressed in exp psych - 55–98 observations for each published. If correct, means almost any theory can look like proven by data.
1
36
59
@GustavNilsonne
Gustav Nilsonne
3 years
Excellent guidance here by @lakens on how to justify sample sizes: (1/n)
Tweet media one
@lakens
Daniël Lakens
4 years
New preprint: Sample Size Justification. It tries to help you to write an honest sample size justification for your next study. This means thinking about how to deal with resource limitations and uncertainty.
14
333
979
1
6
56
@GustavNilsonne
Gustav Nilsonne
4 years
Put your scientific data in a repository and not in a supplementary file to an article! Here is another case of supplementary files being lost as time passes. From:
Tweet media one
3
16
55
@GustavNilsonne
Gustav Nilsonne
5 years
Addendum: So who should review scientific code? Well, here is a promising new initiative to match volunteer code reviewers to colleagues in need for review before publication: . Please check it out if you might want to use it in either role!
4
10
52
@GustavNilsonne
Gustav Nilsonne
5 years
@dremalt Yes! This is one of the few examples I know of where a scientific journal really adds value to the publishing process.
1
0
51
@GustavNilsonne
Gustav Nilsonne
6 years
New preprint on open data policy at Cognition. We tried to reproduce results in 35 articles; it was hard. Open code would have helped greatly. ANOVA a particular culprit. Thanks to co-authors @Tom_Hardwicke @mcxfrank @_MMathur @kidwellmc and others!
0
31
52
@GustavNilsonne
Gustav Nilsonne
2 years
If only all the research were published, I could read it and appraise the quality myself for a certain question. Instead, we have a literature where >90% of papers confirm the authors' hypotheses, making it impossible to tell what, if anything, is "true". 7/
4
6
52
@GustavNilsonne
Gustav Nilsonne
6 years
Here we go! The Swedish scientists' call for open access starts in this collaborative google doc: . Please join the discussion and invite colleagues that may be interested!
5
38
50
@GustavNilsonne
Gustav Nilsonne
1 year
The excellent science illustrator Meike Latz () helped me make this figure to illustrate how publication bias makes effects look bigger than they really are. The figure is available at @ResearchEquals with a CC-BY licence:
Tweet media one
0
11
52
@GustavNilsonne
Gustav Nilsonne
5 years
Very happy to have been offered a fellowship at @questbih starting Jan 2020! I look forward to exploring new collaborations with @ulf_toelch , @dirnagl , @Tom_Hardwicke and others. Will keep my job in Stockholm and visit Berlin from time to time during the year.
Tweet media one
4
2
51
@GustavNilsonne
Gustav Nilsonne
7 years
New preprint: we estimate that for each published pos result in exp psychology, 36-82 neg results were suppressed
5
34
49
@GustavNilsonne
Gustav Nilsonne
1 year
Very excited to talk about open science and reproducibility with these stars next year:
@thorneby
Andreas Thörneby
1 year
Tweet media one
4
1
10
2
6
50
@GustavNilsonne
Gustav Nilsonne
4 years
This case illustrates that validation of predictive models should check if the model predicts the thing that is the actual real-world target. The analysis reported in the paper committed a "fallacy of transposition" by switching around the variables. (6/x)
3
7
45
@GustavNilsonne
Gustav Nilsonne
1 year
Do you want to apply for a Marie Curie postdoc in metascience? I would be happy to supervise and support an application this year. Please forward if you know anyone who may be interested. 1/
2
24
46
@GustavNilsonne
Gustav Nilsonne
5 years
Happy to announce the next round of my PhD course in Open Science and Reproducible Research @karolinskainst next semester. Applications welcome!
1
20
42
@GustavNilsonne
Gustav Nilsonne
6 years
Happy to announce that we will give our graduate course in open science and reproducible research at @karolinskainst again next semester. Applications now open:
0
23
42
@GustavNilsonne
Gustav Nilsonne
1 year
Swedish Medical Journal now publishing special edition on metascience (in Swedish ofc). I am happy to be one of the guest editors. Lots of interesting work highlithing transparency, reproducibility, responsible reporting of clinical trials, misconduct, etc:
Tweet media one
@lakartidn
Läkartidningen
1 year
Utmaningar för den forskande läkaren
0
1
2
1
7
40
@GustavNilsonne
Gustav Nilsonne
2 years
Yesterday I gave a talk on data standards. Made use of the delightful "pasta model" by @RemiGau , which illustrates, among other things, how the messy steps tend to be glossed over in reporting data:
Tweet media one
2
11
40
@GustavNilsonne
Gustav Nilsonne
2 years
@RetractionWatch reports that @PLOSONE is to retract >100 papers for manipulated peer review. I can't get angry - some thoughts:🧵 1/
1
21
38
@GustavNilsonne
Gustav Nilsonne
3 years
Applications now open for my doctoral course in Open and Reproducible Research at Karolinska Institutet next semester. The course is fully online; applicants are welcome from all Swedish universities:
2
23
39
@GustavNilsonne
Gustav Nilsonne
4 years
Conclusion: we don't yet have evidence for claims made by the authors that EEG can identify treatment responders nor that they have found biologically different subtypes. (7/x)
2
4
36
@GustavNilsonne
Gustav Nilsonne
5 years
Looks like the Swedish government is increasing ambitions for open access publishing: roadmap published yesterday gives 2020 as target year for 100% OA, instead of 2026. Good news!
Tweet media one
0
17
36
@GustavNilsonne
Gustav Nilsonne
3 years
Replacing scientific journals! With what? Today, @brembs , @MostlyPhysics , and I argue in @ResearchEurope that we need to shift to infrastructures built on common open standards, services for which could be procured with open tenders in real competition: 1/
2
21
34
@GustavNilsonne
Gustav Nilsonne
2 years
Joint annual meeting of @OscSweden and the Swedish Reproducibility Network starts off with a keynote by the excellent @MicrobiomDigest on research misconduct:
Tweet media one
1
6
33
@GustavNilsonne
Gustav Nilsonne
4 years
The #EEGManyLabs preprint is now out, describing the design of this large-scale collaborative replication experiment which will test 27 influential EEG findings. Very happy to be a part of this, together with @ugpavlov and many others.
2
4
34
@GustavNilsonne
Gustav Nilsonne
2 years
New preprint! Another instalment in our ongoing efforts to tidy up claims based on statistical artifacts. This time it's about prospective associations between self-esteem and quality of social relations. 1/
1
5
33
@GustavNilsonne
Gustav Nilsonne
3 years
Why do we call it "preprints"? The term seems to imply that work is preliminary or unfinished. As far as I can tell, the term introduced by @arxiv , the first online repository for scientific manuscripts, is "e-print". Is "preprint" a marketing device invented by publishers?
5
10
32
@GustavNilsonne
Gustav Nilsonne
5 years
Today I write in @TidningenCurie (in Swedish) that the new "transformative deal" with Elsevier should be the last. We need to break out of the prestige-based journal publishing system:
4
21
32
@GustavNilsonne
Gustav Nilsonne
3 years
Finally! The European Data Protection Board offers guidance on processing on personal information for health research, with more to come later this year.
1
14
32
@GustavNilsonne
Gustav Nilsonne
1 year
More metascience! Last week I submitted two grant proposals to follow up research and check whether results are published. 1/
2
4
32
@GustavNilsonne
Gustav Nilsonne
2 years
Meanwhile, legacy publishers operate peer review mostly as a black box. When I read a paper in a trad journal, I typically know nothing. Was it reviewed by only an editor or also by "peers"? Who were they? What expertise did they have? Did they recommend publication or not? 11/
1
1
32
@GustavNilsonne
Gustav Nilsonne
2 years
Until then, the "predatory" journals are probably a less malign influence of knowledge than the traditional publishers. - end 12/12
6
0
31
@GustavNilsonne
Gustav Nilsonne
1 year
@Aella_Girl It's a horror show. Masses of papers published only for the sake of publication and not to be read or built upon. Peer review mostly opaque and unsystematic. Actual truth claims rarely tested. Strong need for reform towards assessing actual quality of outputs.
1
4
31
@GustavNilsonne
Gustav Nilsonne
3 years
Finally I am back at the @questbih to spend some time as a visiting fellow, working on open and reproducible research. View from the office over the Spree and Museumsinsel:
Tweet media one
1
0
30
@GustavNilsonne
Gustav Nilsonne
5 years
@westbynoreaster @avehtari I am almost certain the journal offers no compensation. And I agree with you - in fact I do sympathize with the reviewer to an extent. They are acting rationally in a system that lacks proper quality control mechanisms and incentives.
5
0
30
@GustavNilsonne
Gustav Nilsonne
6 years
@mrgunn Hello! I am a Swedish researcher in need of access. Can you please give it? To all your articles would be best, but we can start with the ones I have written myself and given to you.
1
0
29
@GustavNilsonne
Gustav Nilsonne
3 months
Excited about this event at Karolinska Institutet in Stockholm on April 26 (in person): Research Assessment for Transparency and Reproducibility - event with John Ioannidis
1
11
30
@GustavNilsonne
Gustav Nilsonne
3 years
Today @krstoffr and I taught a seminar on sample size justification using the recent paper by @lakens : (1/n)
2
3
30
@GustavNilsonne
Gustav Nilsonne
1 year
Many people think AI will soon cause large and unexpected breakthroughs in science. Some fear it. Here's why I think such breakthroughs are unlikely to happen soon, except in some specialised scenarios (thread): 1/8
4
7
29
@GustavNilsonne
Gustav Nilsonne
4 years
Very happy to announce that the #EEGManyPipelines project website is now up. This will be a crowdsourced analysis of reproducibility of EEG results given the same dataset. Join the mailing list for further information including how to sign up, eventually:
1
19
29
@GustavNilsonne
Gustav Nilsonne
2 years
Monday May 16th is last day to apply to my PhD course in Open Science and Reproducible Research @karolinskainst , fully online and open to doctoral candidates anywhere in the world: Here is how the students evaluated it last time:
Tweet media one
1
21
29
@GustavNilsonne
Gustav Nilsonne
2 years
@EegManyPipes has a lovely new logo designed by @MeikeLatz :
Tweet media one
0
7
28
@GustavNilsonne
Gustav Nilsonne
4 years
@jd_wilko I see that the participants you gave higher numbers were more likely to become female. This could turn into a very high impact publication!
0
1
28
@GustavNilsonne
Gustav Nilsonne
4 years
A publisher just e-mailed me that the "Version of Record" of our latest paper is out. I disagree: because of the paywall, our preprint is the version that people will likely read and cite. Instead of "Version of Record", it should be called "pdf with publisher's logo".
Tweet media one
2
0
28
@GustavNilsonne
Gustav Nilsonne
4 years
Hooray! For the first time I managed to fit a Bayesian ordinal mixed-effects model. Here are estimated effects of sleep deprivation on ratings on the Karolinska Sleepiness Scale. Thanks @paulbuerkner and @vuorre for the excellent brms package and tutorial:
Tweet media one
3
0
28
@GustavNilsonne
Gustav Nilsonne
4 years
So I grabbed their data and tested how well the model predicted patient outcome, compared to 1000 bootstraps with labels switched randomly. Result: no better than chance. (5/x)
Tweet media one
4
4
26
@GustavNilsonne
Gustav Nilsonne
2 years
An ideal system of scientific communication would publish everything, without biases and while letting researchers be responsible for their own work. 9/
4
0
27
@GustavNilsonne
Gustav Nilsonne
2 years
Applications are open for @karolinskainst PhD courses next semester, including my course in open science and reproducible research, which will be in the first 2 weeks of May. PhD students from all universities are welcome to apply:
0
15
25
@GustavNilsonne
Gustav Nilsonne
4 years
The OpenBrainConsent preprint is now out, describing a valuable set of recommendations for obtaining consent for data sharing from research participants. Great work by @fmrwhy , @CyrilRPernet , @peerherholz , @yarikoptic , and many others:
1
8
25
@GustavNilsonne
Gustav Nilsonne
5 years
This afternoon our research group had sparkling wine to celebrate a paper by my colleagues which was accepted after 17 submissions and 2 referrals, over the course of a year and a half. Another example of the journal publishing system as a barrier to research dissemination?
2
1
25
@GustavNilsonne
Gustav Nilsonne
1 year
The Coalition for Advancing Research Assessment (CoARA) has opened a call for working groups. Who else is thinking about developing community-based new ways to assess quality? Eg by testing computational reproducibility and such things.
1
16
25
@GustavNilsonne
Gustav Nilsonne
4 years
Hello @450Movement ! Today I am requesting payment as agreed for reviewing grant applications. As it happens, the European Commission pays in units of 450€. This is normal. I suspect anyone suggesting academics should review these grants for free would be laughed out the door.
Tweet media one
1
3
23
@GustavNilsonne
Gustav Nilsonne
1 year
Hello @DanishRepro and welcome to the family of European reproducibility networks!
@DanishRepro
Danish Reproducibility Network
1 year
We are very excited to announce the launch of the Danish Reproducibility Network! 🎉 Check out our website and sign up for our newsletter to stay up to date: @MartaTopor @vibeke_dam @DrVeronikaCH
5
20
56
0
2
23
@GustavNilsonne
Gustav Nilsonne
4 months
Reviewing grant proposals is a part of my job that I quite enjoy. The main difficulty is that I never get to know what happens with the rejected proposals, so I have no way of calibrating whether my judgement is any good. 1/
1
0
23
@GustavNilsonne
Gustav Nilsonne
3 years
Another day, another peer review request turned down. I still refuse to work for free for journals that do not have good policies for transparency and openness. Even when their institutionalized gratitude is on offer. Ping @450Movement
Tweet media one
3
4
23
@GustavNilsonne
Gustav Nilsonne
4 years
@olivier_klein Yes, I think I will continue to do it for free for journals that are free and open :-)
1
0
23
@GustavNilsonne
Gustav Nilsonne
6 years
Our paper on open qualitative research data now published @MedEdPublish : . Journal operates open post-publication peer review; all comments are welcome. Thanks to @Cormac_McGrath for collaboration on this!
0
5
22
@GustavNilsonne
Gustav Nilsonne
5 years
@lakens Very nice post. Would add: work on existing data when possible instead of running new experiment.
1
1
22
@GustavNilsonne
Gustav Nilsonne
11 months
"Monitoring open science practices - Examples of new indicators and metrics" On September 19 at 15:00 CEST I will moderate this online webinar with contributions by Vladislav Nachev from @questbih and Fredrik Sjögårde from @kibiblioteket . All are welcome:
1
16
22
@GustavNilsonne
Gustav Nilsonne
2 years
Exciting opportunities to participate in the SCORE project to assess credibility of findings in the social-behavioral sciences! Researchers can contribute to do reproduction, replication, or robustness tests: 1/
1
11
22
@GustavNilsonne
Gustav Nilsonne
2 years
Our paper in @BioMedCentral Pediatrics, led by Kimmo Sorjonen, was awarded the Nordic Mensa Fund article of the year: 1/
2
0
22
@GustavNilsonne
Gustav Nilsonne
7 years
Thanks @Tom_Hardwicke and @mcxfrank for this morning's reproducibility hack at @Stanford - exctiting open data policy work coming out
Tweet media one
2
2
21
@GustavNilsonne
Gustav Nilsonne
4 years
But an alternative explanation is that the model predictions did not generalize well due to overfitting on the training dataset. Luckily, they also tested the “antidepressant-predictive signature” in a second independent sample. But wait! The variables are switched around. (3/x)
1
2
18
@GustavNilsonne
Gustav Nilsonne
1 year
Posting again: Do you want to apply for a Marie Curie postdoc in metascience? I would be happy to supervise and support an application this year. Please forward if you know anyone who may be interested. My contact information is at
0
12
21
@GustavNilsonne
Gustav Nilsonne
4 years
~500 million SEK (~50 million Euro). That is the lower-bound estimate of what Swedish higher education paid in 2018 for scientific publications, including subscriptions and APC:s, according to a new report from @kungbib . Enough to fund a small university.
0
9
20
@GustavNilsonne
Gustav Nilsonne
6 years
Have just given a talk at #SFN18 on open data in neuroscience. Am grateful to @INCForg for organising the workshop. Here are my slides:
1
5
21
@GustavNilsonne
Gustav Nilsonne
3 years
"Nepotistic journals" - a new concept coined by Alexandre Scanff, @IoanaA_Cristea , @NaudetFlorian , @deevybee , @dmoher & @Clara_Locher to describe journals where many papers are by the same author, often an editor, and apparently less stringently reviewed:
Tweet media one
0
8
20