The most successful people I’ve met:
1) are jerks.
2) have sacrificed family for success
3) do whatever benefits them
The people I want to be like:
1) are nice
2) have sacrificed a small part of career for family
3) do the right thing, not the thing that benefits them the most
The most successful people I've met:
1. Read constantly
2. Workout daily
3. Are innately curious
4. Have laser focus
5. Believe in themselves
6. Build incredible teams
7. Admit they know very little
8. Constantly work to improve
9. Demand excellence in everything they do
Statistician's tip of the day: if someone tells you that they "just have a few Excel sheets" that they want help with, run the other way. Also, you may want to give them a fake phone number, possibly a fake name. It may be worth faking your own death, in extreme circumstances.
Update: our effort to compile crowdsourced list of references on some "Common Statistical Myths" is now a wiki on DataMethods - please feel free to go there and add your own topics and/or references!
Ok: best statistics-themed band names. I’ll go first.
Proportional Hazards
(Now that I think about this, a *ton* of statistics terms work as band names...hey did you get the new Proportional Hazards album?)
The NEJM compassionate-use remdesivir data was always sort of a "passing curiosity while we await trial data" but it's disappointing that the analysis was done wrong.
But, it offers an opportunity for a teachable moment about "time-to-event" analysis of non-mortality outcomes.
Occasionally, I see papers with >20 authors (all MD) & an "acknowledgement" for the person that did the statistical analyses.
Why? Would've been too much trouble for *one more* author's name to appear on the byline? It's not like you're adhering to an author limit anyway...
After 10 years of doing >90 percent of my analysis in SAS, today I decided to muck around some with R (which I’ve only dipped into sporadically since undergrad).
WHY DIDN’T ANYONE TELL ME HOW MUCH COOL STUFF HAS BEEN ADDED TO R IN THOSE 10 YEARS?!
I've said/done this before, but again: the BMJ Statistics Notes series from the mid-late 1990's really give some nice, neat little examples to help explain tricky concepts with statistics in medicine, and I try to read through one or two every week...
Hi! I’m a biostatistician. You might know me from my greatest hits including “No, we can’t really calculate 10 year survival for something that’s only been used for seven years” and “No, you shouldn’t say that p=0.08 is ‘trending towards’ significance”
Hi! I'm a general internist. You might know me from my greatest hits including "So, tell me how you are? you heard me, I really care about everything!" or "Yup, I can (try to) coordinate all of that!" or "Not only will I figure out what you have, but I'll fill out your forms"
Hello, I'm a professor in a movie, I only reach the main point of my lecture right as class is ending. Then I yell at students about the reading / homework as they leave.
JAMA and JAMA Network journals join journals worldwide to call on health professionals to warn the public about major danger to health and essential life support systems posed by the threat of nuclear war and urge action to prevent use of nuclear weapons.
Here's your daily reminder that "do univariable analysis & stick all variables with p<0.05 into a multivariable model" is not an optimal model-building strategy for pretty much anything.
I wonder if anyone that lost their mind about this on Twitter over the last two days has bothered to read this, or to consider any of the following things:
Today, I received a resubmission of a paper that I had previously reviewed that used stepwise regression. In my comments, I advised that they use a DAG, as it seemed appropriate for their problem...
AND THEY ACTUALLY DID IT!
Just about fell out of my chair.
Hey stats twitter, what’s the preferred method for handling “missing outcome data due to global pandemic that canceled outcome assessments for a few months?”
Asking for a friend. Ok, for me.
Small academic wins:
Today, we received comments back on a grant with the following note in the first paragraph: "It is clear from the write up of the sample size estimation and analysis plan that there is a good statistician involved in the project."
As promised last week, here is a thread to explore and explain some beliefs about interim analyses and efficacy stopping in randomized controlled trials.
Personal news: after 4+ wonderful years in
@PittGIM
and 14+ years affiliated with the University of Pittsburgh as a graduate student and collaborative statistician, I've joined
@Medtronic
in the Structural Heart & Aortic unit.
My ten-day N-of-1 trial strongly suggests that fatherhood has a benefit on quality of life.
Possible confounding variable: not going to work during those ten days.
Yesterday I got blocked by a sportswriter for explaining that the ability to call a coin toss is not an advantage, and that both teams have 50% chance to win the toss regardless of what is called.
“Ugh, this paper has another logistic regression with way too many variables for such a small dataset. You’d think by now people would know, but NO! This is the 12th time this year I’ve written the same reviewer comment, and it’s only March!”
“Sir, this is an Arby’s.”
Trial finds no significant difference in 60-day mortality between early ECMO and conventional treatment with ECMO backup in patients with severe ARDS. Full trial results:
Intro: why did you do this?
Methods: what did you do?
Results: what did you find?
Discussion: what did we learn, and how does this fit with what else we currently think / know / do?
Every statistician / analyst, here’s the rationalization you’ve been searching for to explain why after 10+ years of working with your took of choice (SAS, Stata, R, whatever) you still google “reshape data long to wide”
I dislike how the association of the word "significant" with "is the p-value<0.05?" makes me afraid to ever use the word "significant" in any phrase describing data.
When I arrive in town for a conference at a city I’ve never been to, I like to go to the hotel bar alone and imagine that I’m actually some sort of CIA action hero coolly scoping the joint and waiting for my contact to reveal themselves.
Noninferiority trials: a musing thread that I may regret.
NI trials are fickle & unsatisfying. Sometimes there's a legitimately good reason to do them (discussed below); the stats are maddening (also discussed below).
Here is a little intro thread on how to do simulations of randomized controlled trials.
This thread will take awhile to get all the way through & posted, so please be patient. Maybe wait a few minutes and then come back to it.
Hey everyone, I'm not *trying* to brag here, but I just wrote an R function that actually does the thing that I wanted it to, and it only took me two help pages and an hour of trial and error.
PROGRESS.
To whom it may concern: a power calculation that is used to justify a trial's sample size should be performed using the same statistical approach that will be used to analyze the data.
Someone told me the graphs in my draft chapter were ugly. So I learned R* and made all the graphs with ggplot just to spite this person, and if that doesn't summarize me...
*By "learned R", I mean I copied pieces of code from around the internet.
#rstats
#phdlife
A look ahead:
The year is 2050...
COVID19 is still endemic.
HCQ has repeatedly failed to improve outcomes for COVID19 prevention & treatment in RCT’s.
A nonrandomized single center study shows better outcomes in patients that got HCQ.
Media headline: NEW STUDY SHOWS...
Hear me out: an app for toddler parents that shows the location of all construction vehicles and other heavy machinery in your area. Does this exist? If not, can someone make this happen?
I've been a mere spectator to the Wansink scandal, but I think the cautionary tale is worth amplifying across the fields of statistics, medicine & the entire research community. Thus far, the discussion *seems* mostly confined to psychology researchers & some statisticians.
Reminder: a treatment effect is shown in an RCT by showing a difference between the treatment group and control group, not by showing a significant change from baseline in one group only.
From a friend:
“My main concern is whether the journal wants to publish a study that does not include a significant result as the main outcome.”
This is not a thing that should be written by reviewers in 2019.
OK. The culmination of a year-plus, um, argument-like thing is finally here, and it's clearly going to get discussed on Twitter, so I'll post a thread on the affair for posterity & future links about my stance on this entire thing.
CLINICAL COLLEAGUES ONLY (
#statstwitter
, please sit this one out): if you were reading the primary outcomes paper from a randomized controlled trial, do you expect to see p-values comparing baseline characteristics of the treatment arms in Table 1?
If you have <100 patients and <20 incidents of your primary outcome, please, don’t create a multivariable logistic regression and say that you have identified “independent risk factors” for that outcome in that population.
Hi
@scottlemaire
, I would like to issue a public expression of concern about a paper published recently in the Journal of Surgical Research, on which you are the editor in chief.
Yeah, it’s a no from me.
Making authors reformat a paper every time they want to submit to a new journal just to learn if the EIC will send for a review is not a win for anyone except the journal feeling like authors “respect the process”
Amazing that “responders” survive longer than “non-responders” when part of being defined as a responder is *surviving long enough* to be classified as a responder
Reviewer Comment just received on primary RCT report: "Table 1 should include significance values for the values reported between (treatment name redacted) and placebo groups"
Seems that I picked up some new followers overnight. I’m just about out of useful ideas after 3 years on Twitter but next week I do plan to put together a thread on interim analyses in clinical trials and some myths-vs-truths for interpreting trials that were stopped early.
@DrMuzz57
@EndChildCrisis
@ZackBergerMDPhD
I’m an editor, too, and see no reason that all formatting req’s need be met for the EIC to take a first look and decide whether the paper merits being sent for review or not. How many thousand of man-hours that could be spent more productively are wasted on this?
The *most amazing* thing happened today: a statistical reviewer told us to take the p-values OUT of table 1! I will be showing this to all current and future collaborators that insist on putting them in.
Andrew: “hey Editor, this paper used an unpaired t-test on what’s clearly paired data, suggest looking into this.”
Editor: “Thank you for your comment. This paper passed a rigorous peer review process. You are welcome to submit your concerns as a Letter.”
Hilariously, minutes after I posted this thread, one of my own collaborative papers was rejected with the following comment: "It seems that the authors do not have formal statistical degree training. With the help of a statistician..."
Ouchtown, population me.
MEMO: really, it's okay to NOT do "multivariable analysis" if your dataset is too small. I would rather read a dozen descriptive studies than another 1 attempting to cram "multivariable" logistic regression to identify "independent predictors" of something with <100 patients.
As more stuff continues to break on the
@NEJM
and
@TheLancet
papers using the Surgisphere 'data' there's another possibility which has occurred to me that I want to play out.
Inspired by this & some other recent discussions of the win ratio approach, here comes a little tutorial on the "win ratio" approach to analysis of RCT data.
At work (I work on clinical trials), I just learned how to perform a win ratio analysis using
#rstats
(using the BuyseTest package)!! Thanks for showing me how
@ADAlthousePhD
I just want to highlight that for anyone interested in statistical analyses in the practice of epidemiology or medicine, there is an exchange between two field giants going on here that is very thought-provoking:
It really happened! The push-up challenge to end our breakfast feud! 💪🏻💪🏻 Tune in Wednesday to see my full workout with
@markwahlberg
at
@f45training
. We also discuss intermittent fasting and the food hack that helped Mark lose weight. Set your DVR now!
This poor student, in Korea, a country about which I know precious little but others have said is not a bastion of gender equality, wrote a paper trying to provide data to help *combat* gender stereotypes, and medtwitter came out in full force to beat her skull in for it.
(THREAD)
A few months ago, the Annals of Medicine published a controversial piece titled “Why all randomized controlled trials produce biased results.” Topic: not a bad idea - we should examine trials carefully. Execution: left something to be desired.
Today I fulfilled a lifelong dream: I was actually the “random man at community event interviewed by the local news” (in this case, dropping off glass bottles for recycling...)
one of my favorite things about working with extremely smart and senior academics is seeing how many of them still hit Enter 15 times to get onto a new page of a Word document instead of inserting page breaks
The frustrating thing about the EXCEL “controversy” (for me, on Twitter) is watching people without full understanding of “how a trial happens” sling the word “misconduct” around & call for NEJM to retract the paper because...there were changes to the protocol
Differences between men and women, volume 9372: there are currently 12 bottles of soap / shampoo / conditioner in our shower. One of these belongs to me. The remaining 11 belong to my wife.
Harms from uninformed reviewer comments:
Today, my co-authors received a comment on our paper that questioned our sample size calculation because “higher power increases the chance for a Type II error” (not a typo: they said this in two different places in their review).
That time when a female medical student and a kind male mentor from a country with a severe gender bias problen decided to write a paper that they hoped would *combat* negative gender stereotypes in medicine, and twitter ate them alive for it.
@DrMuzz57
@EndChildCrisis
@ZackBergerMDPhD
A few journals (
@JClinEpi
,
@AnnalsATS
) have adopted a very reasonable “we’ll review your first submission in any reasonable format, then worry about meeting final formatting requirements on the second round” which saves wasted time of reformatting papers headed for desk rejection
The longer I’ve stayed in academia, the more I’m convinced we all have an academic superpower. If you had to bet your career on it, what would yours be?
I'm probably about the 1,723rd person toc complain about this, but what is the point of Editorial Manager if you still have to create a new account every time you're a co-author on a paper for another journal?
Today in bad Reviewer Comments - we bring you the story of Reviewer Two stating that you cannot use a generalized linear model to analyze a binary outcome variable.
⚠️EXCITING BLOOD THINNER study: “For patients on ventilators, the difference was more sig. About 63% of patients who did not receive the medications died compared with 29% who received [blood thinners]. Not a trial, but good cohort study... 🧵
#covid19
To people who are questioning the vaccine’s “true” efficacy because they note that people are wearing masks, distancing & say they don’t know if the low incidence of COVID in vaccine arm was due to vaccine or those measures...um, what do you think the control group is for?
The more that I think about it, the more I feel like simulations for power / sample size calcs are the way to go. Maybe it’s because I’m not a good enough math-stat person to power my way through more complex designs, but it just feels more natural. Anyone else feel the same?
"Reviewing scientific manuscripts: A comprehensive guide for peer reviewers" - applause for this effort from Katie Nason to provide some guidance. Not all researchers have been adequately trained to do this...
Reading the ASA discussion board every once in a while is a good reminder that anyone who thinks the solution to fixing medical research is "just make sure statisticians review every paper" has probably not met enough statisticians.
there's a special circle in hell reserved for people that send Doodle invites asking for people's availability in every 30-minute window from 8AM-7PM, all 5 days of the working week, for 2 straight months.
Here's a mushy thought that's been rolling around my head over the last few days as the reality of
#COVID19
starts to sink in to more people:
In the coming months, *everyone* is gonna have to give *everyone else* a little extra slack.
@f2harrell
@arthur_alb1
@TheSGEM
@GidMK
@dnunan79
@EpiEllie
@f_g_zampieri
I feel like one of the hardest things to get people's mind around when reading any sort of power and sample size description is that the effect size used in the power calc is not (or should not be) what you think the effect **is** but rather the smallest effect of interest.
If I had a nickel for every time a reviewer says we have to redo the analysis because the data aren’t normally distributed, I’d have a whole bunch of nickels.
As many folks have explained, calculating post-hoc power using the observed effect size is just a transformed version of the p-value. OF COURSE STUDIES THAT FOUND NO SIGNIFICANT DIFFERENCE SEEM TO HAVE LOW POWER when you use the "observed" effect size to calculate power.
Amazing to see "rise up against statistical significance" backfiring (or, to be fair, being misunderstood) so badly. And it's just a paper about the coronavirus, so it's not like it MATTERS or anything...