How brain neural nets do computations to process info;
@NIH
. Bearish on AI taking over the world, bullish on neuro advances via understanding AI. Pers. views.
New preprint, on 'sequence filtering'.
Led by
@CianaDeveau
, Z Zhou.
We see this as a key step forw on how cortex works.
All cortical areas have dense exc-exc recurrent connectivity.
What do these connections do, esp in sensory ctx?
Our data say: they do dynamics/time.
1/4
I know scientists are busy and the Supreme Court doesn’t seem like the biggest news in the US even of the last two days.
But I’m begging US scientists:
You must read this article about Chevron and the SCOTUS attack on agencies.
B/c it’s coming for us. 1/
#Science
🧠🚨: Our latest paper is out today in
@NeuroCellPress
! Led by
@JonORawe
, we show salt-and-pepper excitation& suppression in mouse V1 arises from purely exc opto input.
That means the excit-excit recurrent connections in cortex change how neurons respond to input. 1/3
I'm a broken record, but:
The field of neuroscience, if presented with an organism that behaved like LLMs behave, would absolutely conclude that the organism does not have 'understanding'.
“LLMs are just doing next-token prediction without any understanding” is by now so clearly false it’s no longer worth debating.
The next version will be “LLMs are just tools, and lack any intentions or goals”, which we’ll continue hearing until well after it’s clearly false.
Love you,
@nature
, but this is about extracting very coarse signals from brains.
To do “brain-reading” we need to understand the computations brains do, we are very far from that, and we will figure that out via basic science—w/ some philanthropy but mostly w/ public funding.
Neuralink has reportedly successfully implanted a brain chip into a human.
We're filling you in on the rise of brain-reading technology, and all you need to know
Now available on Addgene as virus, can be ordered now: 💉🧠
Our bicistronic construct that expresses GCaMP8s and stChrimsonR, designed for two-photon optogenetics.
1/3
This is why governments are the major funders of scientific research—only govts, societies, have the long time horizon needed.
What private firm would fund the next research on DNA, with treatments coming only decades later?
Science is a public good, and govts fund public goods.
We went from learning DNA is the molecule of heredity (1952) to sequencing a virus, transmitting it across an ocean, and synthesizing a functional vaccine before the virus even made it there in 68 years
There is only tech stagnation if you have a very narrow knowledge of tech
Really excellent interview with Fauci.
Discussion how the mRNA viruses were engineered with a stabilized spike based on VRC, Graham,
@KizzyPhD
work on RSV.
“What’s come home [to me] is you have to focus like a laser on the science that you’re doing.”- Fauci
Yes, neurons in the brain fire irregularly.
We actually know quite a bit about this. Our knowledge comes from 75+ years of largely government-funded research, in which neuroscientists have recorded neurons firing in real-time and built mathematical models.
1/
@Neuro_Skeptic
They don’t fire all at once. Some go several seconds to minutes without firing. Some fire several times per second. Very wide timing distribution.
Sharing new preprint from our lab:
@biorxiv_neursci
.
Led by
@pklafosse
, we show neurons in the awake🧠can filter out inputs: attenuation-by-suppression.
Also: real neurons’ activation function share features w/
#ai
systems (eg ChatGPT).
#neuroscience
Comments welcome!
1/15
This is now published at
@eLife
.
Summary: It's not just V1 needed to do this task, which is about as simple as tasks get. V2 is needed as well.
→V2 (LM) seems to carry sensory representations, so... does the brain simultaneously read out all representations available?
Our work showing cortex is well-described by inhibition-stabilized network (ISN) models, with some diffs from the preprint, is now out in
@elife
. Short thread.
Collaborative effort w/ Nicolas Brunel,
@AlessandroSzeni
, B Akitake,
@hannah_goldbach
/1
This paper is now in early release at eNeuro.
We put both stChrimsonR and GCaMP8s into the same AAV.
This allows us to do two-photon stimulation over weeks or longer. It's very stable, and imaging crosstalk is limited.
Ok, a “basic neuroscience is cool and moving fast” thread.
If you’re a neuroscientist, please post pics or videos from your work in this thread.
Let’s all retweet.
Feel free to note if your work is supported by gov’t/tax/nonprofit funding, too.
@KordingLab
@ArielJLevine
The working definition of “mechanism” most neuroscientists use, in my experience, is “one level of explanation down.”
For ion channel function, protein structure is mechanism.
For low-dimensional neural dynamics, connectivity matrices can be mechanism. &c.
@FoldMani
I don't know. I think the authors understand mechanism better than me. For me mechanism = causation + understandable mediators on at least one level between A and B. Usually people also demand that the mediator must be a visualizable bio thing.
The conclusion of this recent preprint from us is the dense recurrent network of the sensory cortex _filters_ sequences of input.
IOW, the brain learns the temporal and spatial structure of the world in its local connectivity, boosting input seqs it's learned. (cartoon below)
Cool work on effects of electrical stimulation of the cortex.
Stimulation responses are increasingly important for clinical purposes. For building Neurolink-like brain-computer interface devices, and for treatments (see below).
Love this diagram of stim’d neurons! 1/3
Never say never: happy to share work from my post-doc with Michael Stryker, co-first-authored with Jennifer Sun. We investigated how electrical stimulation differentially activates inhibitory and excitatory neurons as a function of ongoing neural activity:
I am not sure who in the government decided that I have to do a 5-hour counterthreat training to attend Cosyne, but I can now say I have learned how to tourniquet 'massive hemorrhage' and how to correctly ram a car used as a barricade.
Bring on the poster sessions
A Republican President in 2025 will be CATASTROPHIC for US science and universities.
They will clean out agencies, strip your grants away and their cronies will send the money to their favored political allies.
New preprint from us showing that secondary visual areas play a role in a simple visual decision – a decision where there is substantial information available in V1.
Implications for how cortical areas interact during simple behaviors.
cliched,but: "/1"
In summary:
the conservative Supreme Court is on a mission to take power from Congress and the executive branch and give it to the President, and to courts. The FedSoc judges were selected to do this.
They are scornful of scientific expertise— of any expertise.
And US scientific research DEPENDS on scientific expertise flowing through agencies like NIH and NSF.
The conservative courts want to strip that away and take grants decisions for themselves, to be made not based on science but for political purposes.
Thread on recent papers about how mouse V1 populations code sensory information.
The paper I had looked at most closely was Kafashan et al. from Harvey and
@drugowitsch
. The data effects I focused on were (1) sublinear scaling of info, and ...
Are neurons noisy?
Let’s come at this a different way:
How can neurons do computations in the presence of variability?
Our recent sequence filtering effect is a computation occurring even when single neurons are noisy.
Each frame of a visual sequence is represented in ctx by 1/
Are Neurons Noisy?
Even in vivo, cortical neurons aren't always noisy
Here is an example of auditory cortex neuron responding with exactly 1 spike to 25 repetitions of the same tone
(DeWeese et al, 2003)
1/
Reminder that whatever Neuralink shows today, it is a small step that builds on decades of mostly government-funded brain research.
Thousands of people worldwide work on brain-machine interfaces, decoding, etc etc,
and also neural stimulation. Ex, from
@pklafosse
et al:
I very much regret that "Requiem for A Spike" was deleted from the title of this paper when it was published.
(I just googled "requiem for a spike" - no dice to this!)
Cool preprint; we just did a fun lab meeting on it. 🧠🧪
The core result is: Changing neural variability in V1 by injecting both E and I current affects perceptual behavioral responses.
1/4
(From the
@AdesnikHillel
lab, w/
@HayleyBounds
.)
For more details, please read and share our new manuscript "Balanced bidirectional optogenetics reveals the causal impact of cortical temporal dynamics in sensory perception" now live:
Just got back from a day in Rome after an amazing Optogenetics Gordon conf.
Excellent science there—lots of talk of understanding recurrent networks and recurrent computations in different brain regions.
And fun to get some time w/ Hugo to see the ancient Roman sites!
@drvox
@brianbeutler
Read this
@brianbeutler
article along with
@drvox
‘s excellent “Epistemic crisis” article. Add
@jayrosen_nyu
‘s “Savviness” series and you’ve got all you need to understand how the modern media’s choices helped get us to this crisis of democracy.
Posted a new ver of our optogenetic learning preprint to bioRxiv.
Highlight of the work is a pattern of input, delivered to the awake cortex, doesn’t change cortical responses—until we train animals to detect that pattern.
Then cortical responses change.
Yes. A lot of discoveries get made this way.
I’ve often recommended this: Collect great data, then print it out and stare at it.
Derived plots and statistics are useful for confirmation but less useful for discovery. Your visual system is good at finding patterns.
There’s no way to look at what’s happening to Fauci in Congress — from all Republicans — or to listen to Trump or Sam Alito or Jim Jordan and think “nah, this will not happen here; it will not happen to me.”
It can and it will.
For a small fee, I’ll attend your enemy’s two-photon in vivo imaging talk and ask “have you considered this might emerge due to voltage-dependent calcium channels?”
For a small fee, I will attend your enemy’s complexity science talk and ask “but is it strong or *weak* emergence?”
For a very small additional fee, I will attend all neuroscience talks that mention "emergence" and ask "but what are the precise mechanisms??"
When you stimulate the brain, animals learn to detect the new activity pattern, they get way better with practice, and the brain changes to accomodate the new processing.
New bioRxiv preprint from us. We appreciate constructive comments and feedback.
/1
NIH is hiring a Data Director, who will advise the NIH Director on all NIH data science questions. Looking for people with deep experience deploying data tech. Spread the word.
#NIH
@PyData
What to do?
Vote in November.
And then we have to unpack the courts.
You can call your Senators now and ask them to launch a major investigation of the courts.
Yes? Feel free to drop questions below.
Nice work here by
@AlessandroSzeni
and with me & Nicolas Brunel.
Alessandro’s impt results show how the balanced state—the fundamental model of cerebral cortical function—works in networks that are conductance-based, as real neurons are.
Some surprises.
Short 🧵:
#neuroscience
I think Geoff Hinton is wrong about this.
Our brains seem to learn the statistical structure of the world (likely in ‘weights’ as well), but we don’t just learn the statistical structure of the world.
We learn about what’s true and what’s false.
The AI legal filing case is… /1
Tough times in the world right now, but will share this:
If you're attending SFN in DC this wknd, we have 2 posters on Sat I'm excited about.
@pklafosse
is presenting work on activation functions in cortical neurons,&
@CianaDeveau
on cortical recurrent nets 'doing sequences.'1/2
Big connected networks — people passing information to each other, or neurons synapsing on each other, for ex — often show unintuitive and nonlinear behaviors.
Linear models can’t tell the whole story, even if that’s all we have data to estimate.
Quantum dots!
Makes an optogenetics prize seem more likely in future.
Also no one needs to hear this, but Moungi Bawendi gave me a (fully deserved) F on a midterm, I worked like crazy to salvage a C in the class, and I’ve definitely had “late for test” dreams abt that class.
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2023
#NobelPrize
in Chemistry to Moungi G. Bawendi, Louis E. Brus and Alexei I. Ekimov “for the discovery and synthesis of quantum dots.”
@mehdirhasan
this is so often the excuse: “waaaah liberal elites don’t respect me. It tickles status threat, just like cancel culture and political correctness and immigrants taking jobs. Our response should be “be better, be tougher.”
Also for non-scientists:
If US fed science grants are politicized, what will happen?
Medical school research will be decimated.
Many university science depts will be gutted.
Cancer research? Decimated; many or most academic labs will close.
The biotech industry will shrivel,
Talk today at NIH about modern, open, distributed tools for analyzing imaging (esp ndarray) data using Python. By
@mrocklin
, author of innovative dask library.
@IRPatNIH
#dask
If you'll be at Cosyne 2024 this week, please come by our posters:
We have 3 presentations, by Paul LaFosse, Remy Yovanno&Bradley Akitake, and Ciana Deveau.
All about what the recurrent network of sensory ctx does.
Looking forward to seeing others' science at the meeting too!
@SamWangPhD
@hildabast
I think the only possible plan after the summer, till a vaccine, is massive test-and-trace. (
@Noahpinion
is talking this up).
US is doing shelter-in-place now bc we screwed up testing in Jan/Feb.
We have a second shot in July/Aug.
Barring unexpected, it's the only way to reopen.
It's an extraordinary thing for a pres candidate to step aside, and
@JoeBiden
deserves credit for what he did today and everything he's accomplished.
Here's my mom abt 20 yrs ago w/ Joe at an event in Scranton. She'd be happy today to see him pass the torch to Kamala.
Fight on!
New preprint from us (Alessandro Sanzeni led proj w/ B Akitake, collaboration w/ N Brunel-sorry no @'s, none on twitter).
A bit of a thread on the findings.
We show evidence for "strong" recurrent coupling in several cortical areas: vis, somato, M1/PM.
Super cool. Improved microendoscopes for brain imaging.
They achieve extended fields of view w/ custom aspheric microlenses to correct aberrations.
check out these images👇
from Tommaso Fellin's group.
-MH
Excellent talk by
@djoshea
, on work with
@leaduncker
, at
@OptoGRC
.
They’re using causal manipulations to understand dynamic computations in motor cortex shaped by the cortical recurrent network.
Lots of interesting discussion in the Q&A on using network models/ANNs to…
This is very cool. "Systematic Perturbation of An Artificial Neural Network."
It's still early, but I think we are going to learn a LOT about rules of how brains work by manipulating neurons inside brains. This kind of simulation in models will guide that work.
I am delighted to share:
1. Our work "Systematic Perturbation of an Artificial Neural Network: A Step Towards Quantifying Causal Contributions in The Brain" is now on
@biorxivpreprint
2. Not only the code but also a Python package is on
@github
Links and summary 👇
This is an excellent paper.
They show that by training a multi-layer model with natural scenes, the _internal states_ of the network reproduce complex non-linear retinal phenomena.
Check out the motion reversal effect below -- left, data, right, internal network states.
1/Our paper
@NeuroCellPress
"Interpreting the retinal code for natural scenes" develops explainable AI (
#XAI
) to derive a SOTA deep network model of the retina and *understand* how this net captures natural scenes plus 8 seminal experiments over >2 decades
This is a good list.
And on the first, hallucination is bad but the reality now is even worse: these systems have no concept of what is true or not. They’re parrots trained on the whole internet.
No, we are not even close. AGI would require systems that
👉essentially never hallucinate
👉reliably reason over abstractions
👉can form long term plans
👉understand causality
👉reliably maintain models of the world
👉reliably handle outliers
We currently have none of that.
Congratulations to Christopher Versteeg, Andrew Sedler, Jonathan McCart, and Chethan Pandarinath for winning our best paper award in the category of neuroscience and interpretability ! 🏆🧠
This award recognizes their work on dynamic models with nonlinear objective readouts 🤩
@roydanroy
not 100% sure, but happy to accelerate that rate!
If people want to work on biological recurrent networks and how they compute - we have some positions. Postdoc, PhD, maybe others.
here are some neurons that show interesting temporal statistics.
Our article just posted at eNeuro, on one purpose of feedforward inhibition (it allows different network input-output functions; adjusting recurrent weights does little in the absence of FF inh). Copy-editing still to come for final ver.
On NIH budget for next year:
The House majority this week released their toplines for next year's budget.
The Labor-HHS-Education subcommittee rec is $184B.
The just-passed budget (FY24) amount was $225B.
This is a **$41 BILLION** dollar cut.
Or twenty percent.‼️
🚨🚨 1/
Terrible news. Sliman was full of joy for science and insights and energy. I will miss the chance to talk to him in future about brain stimulation—it was always a treat.
I was just reading his lab’s recent exciting ppr on bioRxiv about multiplexing stimulation.
Gone too soon.
Controversial opinion:
We have more (cortical) excitatory neurons than inhibitory because information is encoded largely in excitatory synaptic weights. The excitatory-synapse dimensionality is much greater than the I-I or I-E dimensionality.
There are a lot of ideas about why we have more excitatory neurons than inhibitory. I have my own (power for causal inference). But it is such an exciting space. Lots of different proposals.
Congrats to lab grad student
@pklafosse
on earning an
@IRPatNIH
/ NIH Fellows Award for Research Excellence (FARE) for 2024. 🎉
Paul presented his ongoing and awesome work on single-neuron activation functions in vivo. 📷 🧠
#neuroai
#neuroscience
A celebration for Bruce Cumming today. An excellent scientist and mentor, sharp and insightful thinker, who impacted both the people he worked with and the field.
I learned a lot from him about coupling/correlation in the brain, as in Bondy&Cumming 2016.
And what a pic ⬇️
Very important point: good software is expensive to create, in time and salaries and talent. For design, and writing the code, and testing, and infrastructure too.
As science relies more on software we must always take a hard look at whether each step is worth the investment.
I think it is important to acknowledge that grad students and others in the lab did in fact spend a lot of their time debugging in the creation of Spyglass and it was not without cost to them.
Name a bench skill set that you had as a PhD student or postdoc that is now so utterly useless that most people don’t even know wtf it is:
I used to be able to find my insert by visually recognizing Xba1, Kpn1 and BamH1 sites in Sanger seq autorads.
@themetzgerm
This argues cortical areas are discrete with sharp borders, based on animal data.
I don’t want to come down hard, as we don’t really know the final answer.
But based on work in sensory and frontal ctx, I’d be surprised if cortical areas commonly are discrete w/ sharp borders./1
I want to put forward some ideas about *cortical areas*, and why we should care about them when we do neuroimaging.
These ideas can be seen in more extended form in a perspective () led by Steve Petersen, with
@GaganWig
,
@stevenmnelson
, and Ben Seitzman.
Very fun joint science meeting today with our group and
@AfrazArash
and group.
its always fun, and a privilege, to work through ideas with people who think about similar issues w/ different perspectives.
This is early days for this brain cancer treatment; we don’t know how effective this will prove to be yet. But it’s promising.
And the work that led to this was heavily funded by the federal gov’t, especially NIH.
This advance is the direct result of NIH funding. 1/2
“"In a first-ever human clinical trial of four adult patients, an mRNA cancer vaccine developed at the University of Florida quickly reprogrammed the immune system to attack glioblastoma, the most aggressive and lethal brain tumor."
Stunning
Nicolas Brunel giving a stimulating talk after receiving the Valentin Braitenberg award at the
#Bernstein
conference.
This is a nice summary of how balanced state cortical models arose, honoring Carl van Vreeswijk.
After many months we are now starting up in-person lab meetings again. Undirected chats over lunch or coffee are really a key part of science progress.
And another milestone: coffee machine back in action.
🧠🧑🤝🧑☕️💡
Many of the
#BRAIN
Initiative projects have a strong
#NeuroAI
flavor— the research involves using AI methods for understanding brain function.
If we want to bring more AI talent into the US fed gov't, a BRAIN-related Institute (
@NIH
institute?) would be a good idea.
Reading this new report out from the White House today about increasing AI talent across the federal government — over 150 hires have already been made in the past 6 months, and over 500 more are planned in the next 18 months:
“No process in the brain relies on a single area.”
Peter Sterling testified in a 1975 trial against lobotomies — against a lobotomy performed in 1969.
Described here in a Philadelphia paper.
Check out my new review of 80 years of brain "therapies" for mental disturbance:
Causality in Mental Disturbance: A Review of the Neuroscience - Mad In America
A good response to the “no treatments yet” critique is:
The double-helix DNA paper was in 1953, and that led to biologic drugs in the 1980s and on (for MS, cancer, CF, etc).
Basic research by def’n is the research that brings huge, life-changing benefits … decades later.
When critics express concerns like this about brain/mind research, what are your thoughts? What research through-lines are you most optimistic will ultimately lead to treatments for brain and mental disorders?
I just listened to the New Yorker interview of Geoff Hinton.
I love and respect Hinton’s work.
But what he says about AI’s future is— what?
“AI will decide to take over.”
To take the stochastic parrot frame: would a super-trained parrot “decide” to “take over?”
This is excellent advice for grad students from
@arthur_spirling
.
It’s not everything PhD students need to know, but there are some important points here. /1
Re: AI systems and whether they can reason:
Humans train for our whole lives on what is true and what is not true.
(The whole of academia is about working to build new truths.)
LLMs are not trained on truth. They have no representation of truth. We can't say they 'think'.
Packed house today at
@NIH
Take Your Child To Work Day.
This session is about how our brains and eyes work together to allow us to see.
The kids are transfixed. Lots of surprise and enthusiasm on the optical illusions.
@NIMHgov
@IRPatNIH
@NatEyeInstitute
@BlackWidow284
Just sent this to some lab members. I had forgotten some of the great points in here, relevant to the practice of science.
"When you're operating on the maker's schedule, meetings are a disaster. A single meeting can blow a whole afternoon."
Bye bye
#SfN23
!
It was great to have so many friends and colleagues come to us and get to discover vibrant DC.
Hope everyone had a great time here.
When you come back to DC, hmu for recs, or to meet up.
Very nice to have Massimo Scanziani visit
@IRPatNIH
yesterday.
I just marvel at this data every time I see it. On how layer 4 and L2/3 have very different integrative properties across space. Argument here is the cortex knows (predicts) the structure of the world.
Listening to
@KanakaRajanPhD
: “Inputs are the path to mechanism.”
Yes!
This is why I am excited about cellular-resolution stimulation (two-photon, SLM-based, holographic).
Two-photon stimulation allows us to control inputs to networks and ask what outputs are produced.
The Supreme Court “opened new avenues for ideological critics to chip away at government power,”
@narosenblum
writes. “At the same time, it gave itself more power to control the operations of the executive branch.”
@CianaDeveau
Our results suggest the cortical recurrent network learns the temporal and spatial statistics of the visual world.
The work implies the cortical network actively filters _sequences_ of inputs, boosting those sequences corresponding to natural vision.
5/
@eitanhersh
This is very generalizable in the age of corporate social responsibility. Most professional class jobs (in tech, consulting, even finance) are now advertised as "change the world," even for McKinsey when the actual consulting job involved helping dictators track dissidents
Good stuff. Yes.
Scientists should stand up for democracy because science is a core institution of democracy - an indep source of truth.
Science and academia are some of the first things authoritarians go after.
New paper in
@Nature
with all-star team! Ridiculous claims are made about how misinfo isn't a problem, can't be defined, & how fighting it = censorship. Nonsense. We know the playbook. Here's why scientists need to stand up for truth & democracy!