🧵 1/ What’s at stake with race-based equations for lung function? Clinical, occupational, and financial reclassifications for millions of patients.
I’m delighted to share this labor of love, published today in
@NEJM
and unveiled at
#ATS2024
.
Special Article: Implications of Race Adjustment in Lung-Function Equations
Editorial: Beyond Diagnostics — Removing Race from Lung-Function Test Interpretation
#CriticalCare
🧵 1/ Statins and BP meds can be lifesaving for patients at risk of heart attack or stroke. But what happens when we redefine who is “at risk”?
In today’s
@JAMA_current
, we explore the potential impact of new
@American_Heart
PREVENT risk equations.
Link:
@emollick
The authors explicitly deny parity with clinicians, and there are a lot of reasons why.
E.g., MedPaLM's responses contained incorrect/inappropriate info 18.7% of the time vs. 1.4% for clinicians.
Very impressive/promising results ofc, but still some work to go!
1/ Can a simple thumb trick save thousands from sudden cardiac death?
Now out in
#AmJCardiology
,
@arjunmanrai
,
@GloriaJWu
, and I investigate the questionable clinical utility of a newly proposed “digital”😉 biomarker.
My brother is coming to
@MIT_CSAIL
for his PhD!!
Could not be prouder of
@CD1AO
and all the hard work that brought him here.
Excited to finally live in the same city again after 9 years!
Honored to join
@ForbesUnder30
in Healthcare among some familiar (and much-admired) faces!! 🥳
As always, a hearty shoutout to my incredible mentor and friend
@arjunmanrai
, without whom none of this would've been possible.
Meet the
#ForbesUnder30
Class of 2022: defying the odds amidst a global pandemic and supply chain breakdowns, these honorees are resilient and destined to change the world.
Collecting expert annotations at scale can be a huge cost sink for medical ML. Check out our (non-technical) editorial in
@Nature_NPJ
on HALS: a human+AI workflow that increases labeling efficiency by up to 10-fold! With
@richardjchen
and
@jkvedar
:
Grateful and delighted to join the
@PDSoros
Fellows community!! A huge thank you to the Fellowship for supporting my education—I can’t wait to meet everyone 😀
We're thrilled to announce the 2021 Paul & Daisy Soros Fellows: 30 New Americans—immigrants and children of immigrants—pursuing graduate school, who exemplify the extraordinary contributions of immigrants to the United States. 🗽🇺🇸🎓
🧵1/ I’m beyond proud of my student
@JamesADiao
who today became the 22nd student in the >200-year history of Harvard Medical School to receive his MD Summa Cum Laude.
No better way to celebrate this by noting the uncanny similarities between James Diao and … Taylor Swift.
Now out in
@NatureComms
: AI-derived image features for interpretable molecular prediction! 🔬🧬
Proud to have led this work at
@Path_AI
alongside my amazing classmates
@jkuwang
and
@wanfungchui
.
Tweetorial + press release linked below:
✨ Best weekend ever ✨
@GloriaJWu
and I won intermediate waltz, foxtrot, and quickstep in the UK nationals at Blackpool! 🏆
Best moment: solo in the Empress Ballroom for the waltz honor dance 💃🏻🕺🏻
The
@NEJM
Image Challenge has accumulated >40 million responses across 15 years. What can these data tell us about the relationship between skin color and diagnostic accuracy?
Check out my latest piece with
@AdeAdamson
in
@JAADjournals
!
Reduced representation and decreased diagnostic accuracy in dermatologic disorders in dark skin – a retrospective analysis of NEJM Image Challenge responses
@AdeAdamson
Gave this sleek new clinical tool a spin.
The ddx explanations are articulate but hilariously wrong - classic LLM smoke and mirrors.
If a med student read this A/P on rounds, they would never live it down 😄
We're proud to announce the launch of Glass AI 🩺
Glass AI can generate a differential diagnosis or clinical plan based on your problem representation.
Anyone can try Glass AI for free at .
#MedTwitter
15/ That's all, thanks for reading!
For more on sketchy probabilistic reasoning and the base rate fallacy, check out
@arjunmanrai
's aptly-titled study:
BREAKING: Today, the
@NKF
and
@ASNKidney
Task Force on Reassessing the Inclusion of Race in Diagnosing Kidney Diseases has released its final report, which outlines a new race-free way to diagnose this life-threatening illness. Read more.
Excited to announce our research study on an interpretable pipeline for predicting molecular phenotypes from cancer pathology images!
@amarotaylorw
@jamesadiao
@jkuwang
@wanfungchu
Tweetorial to follow ⬇️
I don't buy it.
If "no answer/not in choices/not enough info" are considered wrong answers, ChatGPT failed every test under every query condition (assuming passing is >60%).
Not to mention that the study was severely underpowered: N≈100 and p≈0.5, so CIs ≈ +/- 10%.
Scaling laws strike again!!
Med-PaLM = Google's PaLM (540B parameters trained on >6k TPUs) + prompting/tuning strategies
The accelerating progress is wild to watch. For better or for worse, it's hard to imagine LLMs not infiltrating clinical practice/consumer use sometime soon.
💡New paper - Large Language Models Encode Clinical Knowledge💡 Our work
@GoogleHealth
@GoogleAI
@DeepMind
advances state-of-art in 7 medical question-answering tasks - including achieving 67% on MedQA (USMLE qs) improving prior work by >17%
1/n
Sending in 70ish letters of recommendation for undergrads applying to grad school this weekend. It is such an honor to write on their behalf. The world truly is in good hands.
Sobering thought for physician-(data) scientists in training.
A typical MD/PhD cardiologist🫀 doesn't finish clinical training until *checks notes* 8-10 YEARS after defending their thesis.
Imagine writing a thesis on AI before AlexNet (2012) and coming back to GPT-3 (2020) ☠️
We've seen GPT-4 excel on MC medical challenges, but how do things change now that it can "see"?
tl;dr - sometimes better, sometimes worse! It depends on how much informative text is available.
Great work led by star
@HarvardDBMI
summer student
@tabuckley_
!
In our new preprint, we evaluated GPT-4V on 934 challenging
@NEJM
medical image cases and 69 clinicopathological conferences. GPT-4V outperformed human respondents overall and across difficulty levels, skin tones, and image types except radiology, where it matched humans. GPT-4V
No patient should ever be denied appropriate care due to their race. In the case of eGFR, race-free equations can deliver both performance and equity in estimates of kidney function. Check out our latest in
@NEJM
:
@timothykeyes
I've always thought the 2-4-2 MD/PhD model was a bit odd. PhD after/late in clinical training seems like a better setup for continuous skill-building.
Ex: Atul Butte, Andy Beck, Mia Levy, etc.
2/ Tl;dr: using or removing race reclassifies lung disease for millions of patients and reallocates billions in disability compensation.
These shifts result from trade-offs in sensitivity vs. specificity with no overall effect on discriminative accuracy.
@Reuters
Finally, a huge thank you to all of our coauthors and collaborators, and especially to my longtime mentor and senior author
@arjunmanrai
(picture below introducing our talk at
#ATS2024
).
I learned so much and had an incredible time working with all of you!
1/ Last year, NKF-ASN recommended a new race-free eGFR equation for immediate nationwide use. The improved equation is already reshaping recommended care for millions. In what ways?
Check out our latest in
@JASN
! With
@ArjunManrai
and many others ⬇️
1/ Most chronic health conditions have huge disparities in access and outcomes. Mobile tech promises to help bridge the gap, but what do the data show?
@jkvedar
and I discuss a new meta-analysis in
@Nature_NPJ
led by
@elainekhoong
+
@urmimalasarkar
6/ The use of race in spirometry is also controversial because of its sordid history. Early studies on race and lung function were used to defend White supremacy and justify slavery.
Dr.
@JamesADiao
, resident physician
@BrighamWomens
, discusses the impact of changing lung function equations on Veterans Affairs disability compensation. Learn more on the latest episode of Intention to Treat:
An inspiring startup journey! I was such a hater when Glass 1.0 came out, not seeing the tide that would rise with GPT-4, RAG, etc. or the mission-driven resolve of the team.
Wishing
@GlassHealthHQ
the very best in 2024 as you work toward launching Glass 3.0 🌟
Happy New Year
@GlassHealthHQ
🎉🎇🎊
We have had a remarkable year together. And since Walter Isaacson and Ashlee Vance have yet to take on telling our story, for now, you'll have to settle for me.
Glass Health 2023 Year in Review 🖤🎆
16/ Our study found that using race did NOT improve discriminative accuracy in predicting respiratory outcomes. Instead, it traded-off sensitivity for specificity, with diverging effects by race.
@emollick
When evaluating free text responses (e.g., "why take simvastatin"), there's no binary right or wrong.
92.6%/92.9% refers to overall agreement with scientific consensus, not that everything in that response was true!
5/ This means that three otherwise identical people could experience different diagnoses, work eligibility, and disability payments, depending on their assumed or reported race.
14/ To me, this exchange indicates the challenge of tackling not just existing instances of low value care, but also the continuing influx.
As others have shown, especially difficult to overturn when more screening = more $$$.
1/ How would switching to cystatin-based race-free eGFR affect patient care? We answer this excellent question posed by van der Burgh et al. in today's issue of
@JAMA_current
.
Growing up (and now), I spent quite a few weekends putting off homework to interpret and fix grammar for my dad's business (docs/forms/emails).
Even now, the company voicemail is my voice from high school 😄
Being a daughter of immigrants means spending a few hours on a Saturday filing their taxes because English is their second language.... meanwhile I still need to catch up on 8 lectures from this week 😅
Love
@GoogleAI
's transparency here. Thailand 🇹🇭 user studies show: low-quality images + poor internet ➡️ low throughput ➡️ subpar UX. Needs more user studies, multisite RCTs, and postmarket surveillance like any other practice-transforming intervention.
8/ These shifts often cut both ways.
Consider a 50-year-old Black man seeking firefighting work. Using race-neutral equations may disqualify him from the job.
But if he later seeks disability benefits for COPD, the race-neutral equation would justify greater compensation.
Together with
@Apple
and
@American_Heart
, we're launching a multi-year longitudinal study of how signals from your heart and mobility signals relate to hospitalizations, falls, heart health & quality of life in order to promote healthy movement and improved cardiovascular health.
19/ We also hope that lung function assessment can incorporate personalized baselines and more comprehensive measures that continue to move away from crude heuristics like race and toward the shared ideal of precision medicine.
ChatGPT scored ~40-60% across the USMLEs. N is low (~100), so CIs are large (+/- 10%).
While promising, an LLM that underperforms 99% of med students under ideal conditions (answer choices given, complete info, and text only) has much room to grow.
Evidence of stronger effect size & dose-response for the most commonly-prescribed and cost-effective intervention in medicine 🏃🏻♂️, analyzed from nearly 100,000 subjects wearing wrist-based sensors ⌚️
#ExerciseIsMedicine
@jflier
As a
@harvardmed
student, I am saddened that you have chosen to use your platform this way. It is a mistake and a distraction to focus on verbal generalizations ("all cops.../white people...") over ongoing and life-threatening anti-Blackness in our communities.
7/ Our data find that, for every true positive, you’d find over 1000 false positives, the vast majority of whom will never develop ascending aortic aneurysm anytime in the future.
This is because (surprise!) ascending aortic aneurysms are quite rare.
11/ Some of my favorites: (1) that we are not doctors and therefore don’t understand physical exams, and (2) that many common exam maneuvers are similarly low value (we quite agree with this!)
As the cold of December sets in, a reminder that another sublime summer will come to Boston in 2022... along with the 18th year of the Summer Institute in Biomedical Informatics
#SIBMI
for undergrad quants! Applications close January 31.
My first two research experiences were supervised by brilliant female mentors and PIs
@Goodell_Lab
@SteitzLab
. Similar for my work
@Apple
. I hope no one shies from female mentorship due to the
@NatureComms
paper 😣
11/ Occupational Implications: 2.3 million with eligibility changes for firefighting work.
Similar eligibility changes are expected for jobs in construction, mining, and textile industries.
7/ In 2022, US and European respiratory societies recommended a new race-neutral equation, but we still knew little about the expected downstream effects on patients (or equivalently, the effects of adding race in the first place).
3/ Spirometry is a test of lung function that measures the volume of air someone can breathe out in one second or in one full breath.
It’s used to diagnose common diseases like asthma and COPD, and to measure respiratory health.
NKF-ASN task force interim report on
#race
and
#eGFR
forthcoming!!
"1) race modifiers should not be included in equations to estimate kidney function and 2) current race-based equations should be replaced by a suitable approach that is accurate, inclusive, and standardized"
13/ Transplant Implications: rearrangement of 90% of patients on the 2020 US lung transplant waitlist, with Asian and Black patients moving forward 21 positions on average and Hispanic and White patients moving back 4 positions on average.
Cool article by Jorge et al—accurate HR/RR monitoring via video camera. Works while sitting, laying in bed, at night, tucked in, on a ventilator, etc.
What else could video tech measure? Where else can it add value? Check out our editorial for more😀
w/
@jayson_marwaha
@jkvedar
14/ Amid all these trade-offs, we asked the question: does using race make the equation more accurate?
Accuracy is less meaningful for lung function than for kidney function because there is no “gold-standard” like measured GFR.
10/ Clinical Implications: 12.5 million Americans with reclassified ventilatory impairment and 2 million with reclassified COPD severity.
This has major implications for diagnoses and follow-up testing, treatment, insurance coverage, and trial enrollment.
17/ Also grateful for an incisive editorial by
@JelaniGrantMD
@ChiadiNdumele
@SethShayMartin
:
“the key message is that the establishment of optimal PREVENT-ASCVD risk thresholds for guiding therapy is critical in the development of future guidelines”
The work of undoing implicit racial bias in
#meded
begins with precise language and appropriate context. Racism, not race. Race-conscious, not race-based.
Check out this must-read
@NEJM
article from an all-star cast 👇
It really does matter that medicine starts getting its definitions and uses of race correct. What an exciting honor to work with this team, and to see my name in
@nejm
next to
@dorothyeroberts
.
12/ Financial Implications: $2B reallocated in disability compensation among 413,000 U.S. veterans, including $1B more among Black veterans and $0.5B less among White veterans.
Motion health unveils new metrics for the Apple watch⌚️!!
In the age of COVID, remote monitoring of mobility stats like walking/stair speed, asymmetry, and VO2 max has never been more important (esp. in post-op and aging patients!)
Movement 👣= medicine 🩺
❗IT’S ANNOUNCEMENT DAY❗
We're proud to announce the 2023
#PDSoros
Fellows!
From nearly 2,000 applicants, these 30 graduate school students are immigrants or children of immigrants who will each receive up to $90,000 in funding. Meet the Fellows here ➡️
8/ Let's do some math.
If you screened 300M in the US, you’d get ~10M positives (sensitivity=3.3%). 10M transesophageal echos = $10-20 billion.
That’s more than the CDC’s annual budget. It also ignores all clinic fees, surgical/procedural costs, and follow-up imaging.
Prompt injection shows a fairly simple set of instructions, perhaps OpenAI API + a wrapper?
The barrier to deploying LLMs (ready or not) has truly never been lower.
"Symphony can compress a 500k-cell reference from 8.9 GB to 1.3 MB, and map a new 10k-cell query in <1 second" 🤯
Congrats
@joycebkang
and
@aparnanathan
!!
I’m very excited to share Symphony, our new tool for efficient single-cell reference mapping: . Enables accurate mapping of single-cell datasets in seconds! With
@soumya_boston
,
@ilyakorsunsky
,
@aparnanathan
, and other awesome folks. Tweetorial 👇
⚡️📣Today we are tremendously excited to announce
@modella_ai
the first startup from Mahmood Lab, based on an array of foundation models and generative AI tools including our recent PathChat article in
@Nature
()
@modella_ai
will actualize these exciting
17/ For over a century, race adjustment was supported by the assumption that racial differences in lung function are natural and benign.
The new GLI-Global equations, though imperfect, offer an important opportunity to move beyond this.
18/ We hope that quantifying the implications of race-based spirometry can inform interventions to redress its harms, preparations for expected changes to care, and improvements to future equations.
Congrats to **Dr.**
@luke_sagers
who brilliantly defended his PhD dissertation! Luke rigorously evaluated whether synthetic data can improve medical image classifiers across populations. It's been a privilege to work w/ Luke from when he was a summer undergrad at
@HarvardDBMI
I don't buy it.
If "no answer/not in choices/not enough info" are considered wrong answers, ChatGPT failed every test under every query condition (assuming passing is >60%).
Not to mention that the study was severely underpowered: N≈100 and p≈0.5, so CIs ≈ +/- 10%.
TI BASIC was my baby steps into computing!
My middle school teachers would snatch up my calculator thinking I was playing games, but my dork ass was just writing code 🤓
Today in 1964 BASIC made its debut, in an effort to take programming out of the world of machine language.
Share if you're old enough to have gotten your start with it.
How do environmental exposures and lifestyle factors compare to genetics and clinical factors for predicting risk of type 2 diabetes? Our paper just out in Diabetes Care introduces the polyexposure risk score (
#PXS
) to investigate this question:
1/n
9/ Our study used data from 370,000 participants in the U.S. and U.K. to compare race-based and race-neutral equations for interpreting lung function.
We looked at clinical, occupational, financial, and statistical implications.
15/ We therefore evaluated discriminative accuracy, which measures how well a lung function index can rank order individuals with respect to a clinical endpoint like new-onset disease or mortality.
Does
@US_FDA
have oversight over clinical LLMs?
For many cases, likely not.
Requirements for exemption:
✅ Provides basis for recs
⛔️ No probabilities (ranked ddx is grey area)
Most LLMs can comply and would be classified as non-device CDS. It's the wild west out here y'all 🤠
Just tried a simpler example without any medical history.
The AI gave her new medical conditions to justify its differential:
- "History of abdominal surgery"
- "History of diverticulosis"
Could easily be missed in a more complex patient & contribute to chart lore.
Gave this sleek new clinical tool a spin.
The ddx explanations are articulate but hilariously wrong - classic LLM smoke and mirrors.
If a med student read this A/P on rounds, they would never live it down 😄
"First, Do No Harm" - must-read editorial in
@JAMA_current
that provides essential context for our study on
#race
and
#eGFR
.
See thread from the one and only
@AmakaEMD
here 👇
@AlecStapp
Maybe the difference lies in expected reward?
The chance to test into college and out of a lifetime of manual labor is one hell of an incentive.