Tokenization (aka the root of suffering, iykyk) has gotten a terrible rap this past week😅 but in nascent fields this rap is wack🌶️
I am ecstatic to share➡️TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis
TOTEM learns discrete time tokens (not patches!!)
🔥 Hot take 🔥
Domain specific knowledge in machine learning is seriously underrated.
In the coming years it's going to become increasingly important as more people master the tools but not necessarily the domain.
Deeply saddened to hear of Krishna's passing.
His warmth, insight, and encouragement shaped me into the scientist and person I am today. You will be missed.
I said I'd release a
@CosyneMeeting
blog... but there was so much info (🤓&😭) so...
I decided to release two! 🤩 🧠
For
#cosyne2022
resources:
For
#COSYNE22
trends:
Please retweet and like so this can reach the most people! 🙏
A ~super~ quick thread on how I approach EDA (exploratory data analysis) when handling terabytes of data 🧠👩💻
This 🧵 is motivated by my own struggles over the past N years, but in particular over the past 2 weeks!
I hope you learn from my pain!
With that, let's go! 🚀
This week starts off the academic year
@Caltech
making it the perfect time to announce that I've decided to join
@yisongyue
's lab for my PhD!!!
I'll be continuing my work in decoding invasive neural signals w/ deep neural nets👩🔬🧠👩💻
I'm looking forward to the next few years 🤓
I generally only discuss science, but this is an important aside... the Loving Supreme Court decision permitting interracial marriage in the US was handed down only 53 years ago today!
I'm thankful because neither I nor my sister would exist without it
#LovingDay2020
💕
Pumped to
#tweetprint
our Deep Neural Imputation Framework!!👩🔬👩💻
DNI recovers missing 🧠 recordings in a day-generalizable manner across multiple participants, even with divergent electrode coverage😱
Intrigued? You should be!
Let's dive in 🤓🧵:
#MachineLearning
#Neuroscience
Thrilled to announce that I received
@Caltech
's Merkin Institute for Translational Research Spark Grant!!
The Merkin support will greatly advance our research on day-generalizable & participant-generalizable brain machine interfaces 🥳🧠🤓
#Neuroengineering
#MachineLearning
Last year for
@CosyneMeeting
I did 2 posts, this year I did 3 🤓🥲
🧠Workshop Recordings:
🧠Resources:
🧠Trends:
Your retweets & likes are appreciated♥️ Let me know what you think🤗
#cosyne23
#cosyne2023
I'm a big believer that concepts generalize across domains!
With this in mind, I read Peter Thiel &
@bgmasters
's Zero to One through the lens of doing PhD instead of building a startup 😎
It was a great thought experiment & although not every insight transfers...
Here is one of
@yisongyue
's insights that I keep thinking about.
Papers in ML have to generally be strong in 2 of the following:
1⃣ Strong Benchmark
2⃣ New Algorithm
3⃣ New Theory
4⃣ New Perspective
5⃣ New Application
Do you think that covers it?
Live from Lisbon it’s your host for
@CosyneMeeting
, me!
Humbled and overjoyed to be the official voice of
#Cosyne24
. We’ve got lots of fun surprises planned so if you see me come up and say hi 👋 who knows what may happen 🙃
My office is pretty when set up 😍
I also put up an excerpt from Hamming's "You and Your Research" to persuade people to come say hi!
Here's a snippet:
"there is a pretty good correlation between those who work with the doors open and those who ultimately do important things"
Impressive results of 93% accuracy on distinguishing between schizophrenics and those w/o major psychiatric illness. Incredible to see how
#NLP
#NLU
are helping diagnose mental disorders!
@dan_iter
@jurafsky
The
@AllenInstitute
🧠 atlases are a phenomenal resource! They really allow you to get a simultaneously feel for anatomy and structure.
I highly recommend perusing their data portal! Lots of well thought out and artfully complied content!
Books I'm diving into to learn more about high dimensional data analysis 🤓:
1⃣ Statistical Shape Analysis
@ian_dryden
Mardia
2⃣ High Dimensional Data Analysis
@YiMaTweets
Wright
3⃣ Data Driven Science & Engineering
@eigensteve
Kutz
Do you have other recs in the same vein? 🙏
Was talking with my parents about the search for great research ideas, dissatisfaction & uncertainty
My mom mapped this to a quandary in "Art & Fear" so I had her find the excerpt bc it strongly resonated
Tl;dr: implementing, not thinking, leads to quality - perfection v pounds
NeurIPS officially ended yesterday 😭 and it was 🔥 both in person and virtually! I enjoyed seeing old faces and meeting so many new ones ♥️
I put together some topics + papers that piqued my interest and maybe they'll pique yours too🤗
Thrilled to announce I'm officially coadvised by the stellar
@georgiagkioxari
🤗
Ive learnt so much working in tandem with Georgia &
@yisongyue
and I'm lucky to have not 1 but 2 inspiring advisers who are both amazing people🫶
I can't wait to share what we've been cooking up😉
Before
@Caltech
a professor said "You're lucky to be here, and we're lucky to have you"
This is an indispensable decision making framework
I now ask myself:
✦ Do I feel lucky engaging in X?
✦ Do those in X feel lucky engaging with me?
If the answer is no to either, I move on
Reading papers is a critical skill in academia. Here are some tips I've collected to help you do it effectively 🤓:
(1) Download the pdf & keep it electronic!
I ♥️ physical copies, but after 100s of papers it's too unwieldy. Cant find the paper? Try
@scihub
. It's magic!
I swore I'd never take another class... but
@yisongyue
@JenJSun
@yeh_im_excited
& Megan are proving me wrong!
"Representation Learning for Science" is 🔥
This content is indispensable for anyone working in AI + Science!
Luckily, you can follow along 🤗:
Last day of
@CosyneMeeting
😭 don’t know if I can wait until Lisbon 🇵🇹🧠
@yisongyue
@GeelingC
started our day learning from
@BTolooshams
& Demba on using neural networks to generate testable hypotheses in biological neural networks 👀
Machine Learning at
#cosyne2023
represent!!
PIs be like:
🔺 Do you write code?
🔹 Barely.
🔺 Do you know how to work a git repo?
🔹 No, I have no technical ability.
🔺 So, what are you being paid for?
🔹 The confidence that I have in my taste, and my ability to express what I feel has proven helpful for students.
@CosyneMeeting
there is (for the first time I think - but don’t quote me) a research beyond academia panel! This reflects the changing tides in neuroscience and neuroengineering! Looking forward to see what
@SussilloDavid
@CLWarriner
and Dawer Jamshad have to say!!
#cosyne2022
Had a lovely time at the
@CosyneMeeting
Diversity, Equity, and Inclusion luncheon organized by
@TPVogels
Here were some of the take always from the incredibly interactive and collaborative meeting. A 🧵:
#cosyne2022
#COSYNE22
GUYS the
@CosyneMeeting
workshop program just dropped 🥳
AND
@EricMTrautmann
's + my workshop "Taming Complexity: Discovering Interpretable Latent Spaces in Human Brains & Behaviors" will be on your Mont Tremblant menu🧑🍳
Too early to say
#Cosyne2023
? 🙊🤫
Check out my review of
#NeurIPS2020
Biology Edition! 🦠🩺🧠
I dive into my thoughts on 1 talk and then link to work I ♥️ in Covid19, Healthcare, Biology, and Neuroscience & Neuroengineering!
I hope you enjoy it 😊 Likes and retweets more than welcome!
Ecstatic to announce I've joined the
#Cosyne2024
Organizing Committee!
Huge🙏to GCs
@jess_cardin
@tyrell_turing
PCs Chand
@bingbrunton
& the rest of the committee for welcoming me!
My♥️shots from the last🇵🇹Cosyne! Join us for more crowds, camaraderie, coffee & Cascais corridors
Get you a PI that gets you your own card!
This seems much more common at Caltech than other schools, but reimbursements should be a thing of the past (frees up your students to do more research 😉)
As always thank you to the 🐐
@yisongyue
!
Making scientific writing concise and beginner friendly should be the gold standard for academic publications.
Unfortunately, it doesn't seem like our current publication system prioritizes clarity.
For those who don't know me I'm a certified night owl, Annenberg at 1am is my happy place
So I ♥️ my new + now most prized possession!!
Annenberg Night Watch🌛🦉👩💻
Ecstatic my 1st last author paper was accepted to
#Cosyne2024
In 🇵🇹 keep an 👀 out for: Generalizability Under Sensor Failure: Tokenization + Transformers Enable More Robust Latent Spaces
Lead by
@GeelingC
, Yujin An, Raffey Iqbal with
@SoonJoChung
&
@yisongyue
Deets soon😉😘
Went to my first in person seminar since covid
@Caltech
today 🤩🤓
@ArchanaVen
gave a wonderful talk on deep learning for both neural imaging and neural time series data 🧠
Thanks to
@klbouman
for organizing!!
I just finished The Creative Destruction of Medicine by
@EricTopol
🧬 🧑🔬
I throughly enjoyed many of his messages so I’ve distilled them down for you in a tl;dr form 🤓
A 🧵 on my key points, outstanding questions, and main takeaway:
I am incredibly excited to announce that I will be moving to
@Caltech
&
@ChenInstitute
to continue my
#NeuroEngineering
research 👩💻🧠🤓
I could not have done it without my crew and am overwhelmingly grateful! LA here we come 👋
Day 4
@CosyneMeeting
, Further into the Great White North we go 🏔🚌🎒
Today:
🔹Network Fxn
🔹Synaptic Underpinning
🔹Special Lec
🔹Credit Assignment
On this, the last day (minus workshops of course 😉), I want to preemptively give HUGE thanks to
@jess_cardin
@tyrell_turing
👏👏
What an amazing resource from
@AllenInstitute
using
#Neuropixels
in 🐭
-58 experiments
-Up to 6 Neuropixels
-Recording from cortex, hippocampus, and thalamus
Overview:
Data + Allen SDK:
White Paper:
👋
@iclr_conf
day 3!!
Started with
@deviparikh
's wonderful talk on 5 stories through her 20 years in the AI field!
In brief you should:
🔹 follow threads that excite you
🔹 optimize the objective functions that make you happy
🔹 understand multiple lens of your life
#ICLR2024
The (sometimes) divergent goals between science and engineering drive me crazy
How does it work? vs. Get it to work.
If you want to do both, it seems like you have to pick one discipline to stand in, and then occasionally dip your toe in the other 🧐
Stopping by the Amsterdam airport tulip shops on the way to
@iclr_conf
🌷😱
Catch me chairing session 2A on the 7th, or presenting on the 11th 🤓♥️
Reach out if you want to chat about time series foundation models, multimodal models, data synthesis, ++ ✌️
#ICLR2024
#ICLR24
On this glorious
@CosyneMeeting
Day 3 the sun is out 🤗☀️ and I gotta say Montréal hits differently when it’s shining!
We got a 💯 lineup from our neuro prophets
@tyrell_turing
@jess_cardin
:
🔹Decision Making🧐
🔹Machine Learning💻
🔹Circuits in Context👩🔬
🔹Behavior & Circuits🥼
Venturing into the exciting unknown of
@CosyneMeeting
Day 2 🤗
Amazing topic lineup today:
🔹Movement in the 🧠
🔹Neural sig of cognition
🔹Integration in the 🧠
🔹🧠s in time & space
P.S. I’m also presenting my poster tonight II-135 so come through 🙃🤓
#cosyne2023
#cosyne23
Your
@CaltechN
summer of learning🏝️🧠🤓
July 5-8:
✦comp. basics
✦dynamical time series
✦high dimensional data
✦autoencoders & ML
July 11-15:
✦single cell seq
✦BYO data
✦deep learning & LFADS
✦generative modeling & MYOW
✦RNNs & dynamical systems
I used to think 10,000-dimensional data was a lot and then I started working with neural datasets...
⠀ ⠀ 🤠
🧠 🧠 🧠
🧠 🧠 🧠
👇🏽 🧠 🧠 👇🏽
🧠 🧠
🧠 🧠
👢 👢
Howdy. I'm the sheriff of 1,000,000-dimensional data.
I recently found out that I got my first citation! 😊 It's a surreal feeling to know that someone found what I did useful. Almost as good as publishing the paper!
#WomenInTech
#WomenInSTEM
Last day of
@SfNtweets
#sfn22
#SfN2022
😭🧠🤓
Had an superb start to the day with
@UeliRutishauser
@courellis
the whole lab has posters up in the WW section go check them out!!
P.S. It’s your last day to come say hello ➡️ white shirt + black blazer 😏
One of my favorite science teachers just retired. Day 1 of freshman year he said "I am not your teacher. I am your guide. You have to teach yourself"
This sentiment rang true throughout Stanford & Caltech. I am grateful that early in life my knowledge was made my responsibility.
Good morning
@CosyneMeeting
workshops Day 1! It is a beautiful day for lively discussions of interpretable latent spaces 😎
Join us in the Joseph room!! If you see the sign I put up it means you’re in the right space 😘
🧠
#cosyne2023
🧠
#cosyne23
🧠
More people resonated with this tweet than I anticipated! So...
Here are some additional thoughts, since I've been building at the intersection of engineering & biology for almost a decade now 😅 (man I feel old)
A 🧵 on this divide & bridging the gap:
🔥 Hot take 🔥
Domain specific knowledge in machine learning is seriously underrated.
In the coming years it's going to become increasingly important as more people master the tools but not necessarily the domain.
Next up we have
@arna_ghosh
@KordingLab
@tyrell_turing
on bias, variance, neural networks and the brain 🧠 Had a fun chat with Arna about how genetic algorithms could be incorporated for more plausible neural learning!
#cosyne2022
Good morning
#SfN2022
Day 1 🧠
@SfNtweets
@SfN_Events
I’ll be dutifully keeping you informed on the hottest 🔥 neuroscience and computation happening this week so stay tuned 😉
I’m also presenting today from 1-5 at 088.09 at the Comp Tools for Ephys Session so come say hello 😘
Biology has steadily become an engineering discipline, and imo requires an engineer's problem solving mentality! 👩💻👨🔬
Pleased to see
@a16z
's growing presence in the space, and
@JorgeCondeBio
@vijaypande
@julesyoo
's belief that biology is eating the 🌍!
Wanted to start off the last day 😰 of
#NeurIPS2022
with pictures of all the CRAZY setups we saw over the past week!!
💽A poster with 8 iPads 😱
💽Projecting figures onto your poster 😲
💽Expensive cameras stabilized on 🗑️😄
💽A DIY VR demo 🤯
The poster session game is wild!
If I was 10x faster at context switching I would be ridiculously more productive.
There's a huge mental cost incurred by jumping between modes (coding to learning to conferencing etc.)
🔥 Hot take 🔥
ALWAYS start with a toy problem when trying out a new project.
Starting off with a toy problem:
✦ gives you a feel for your data
✦ primes you to ask the ~right~ questions
✦ gives you perspective on the most suitable tools
✦ boosts your long term productivity
In addition to recording the lectures and releasing all our code, we will also be live tweeting each day 🤓🧠😎 so stay 👀👂 for these awesome instructors' words of wisdom!!!
👋
@iclr_conf
day 5! Workshop day 🥳
You have not 1 but 2 opportunities to check out my poster today!! No excuses to skip TOTEM 👀👀👀:
🔹10:15 - 11:15 in Strauss 1
🔹11:45 - 12:45 in Stolz 2
Get caught up 👇👇👇
Paper:
Code:
This morning I discovered that Tom Clandinin, one of my dearest advisors from Stanford, was elected to the American Academy of Arts & Sciences!
Tom is an amazing scientist & a stellar human being.
@americanacad
couldn't have picked anyone more deserving!