Introducing , a large language model (LLM) to assist neuroscience research. BrainGPT is trained on the vast neuroscience literature and can be used to optimize study design, detect anomalous results, and evaluate models against broad data patterns. (1/3)
"Large language models surpass human experts in predicting neuroscience results" w
@ken_lxl
and . LLMs integrate a noisy yet interrelated scientific literature to forecast outcomes. 1/6
Introducing the embedding space you didn't know you needed: Human similarity judgments for the entire ImageNet (50k images) validation set. Perfect for evaluating representations, including unsupervised models. It's already bearing fruit, w
@BDRoads
(1/3)
This new preprint is a review of the many ways to link models and brain measures. It will appear in the 2nd edition of the "Introduction to model-based cognitive neuroscience" book, but is free here. I could see this working for a course.
New preprint, "A Too-Good-to-be-True Prior to Reduce Shortcut Reliance". If it's too good to be true, it probably is and that holds for deep learning as well. To generalize broadly, models need to learn invariants but instead are fooled by shortcuts. (1/4)
It can be exciting relating aspects of models, including deep networks, to the brain, but what does it mean to say a model layer and brain region correspond? The field has adopted "correlation implies correspondence", but is that valid? w
@nickjsexton
1/3
New work w Daniel Barry, "A neural network account of memory replay and knowledge consolidation", tests some ideas regarding replay's role in learning at ImageNet Scale. Generative replay is akin to dreaming, training the network w novel murky images (1/3)
Interested in a PhD that combines aspects of computational neuroscience, cogsci, real-world behaviour, and deep learning? Please get in touch. I post bc potential students contact me, which is a biased sample. I hope to broaden and diversify, so feel free to get in touch. 1/2
I just listened to a brilliant neuroscientist explain how deep learning is scientifically useless because it's a black box that is too complex to be understood, then in the next beat said the key to understanding the brain is recording from more neurons.
#BiggerGreyBox
New paper w
@BDRoads
, "Learning as the Unsupervised Alignment of Conceptual Systems". Supervised learning tasks can be solved by purely unsupervised means by exploiting correspondences across systems (e.g., text, images, etc.). 1/5
Now out, "Levels of Biological Plausibility", covering how notions of mechanism, reduction, and emergence are tied to levels, and how claims of biological plausibility are empty at best. Instead, specify relevant data and findings for model selection.
"A non-spatial account of place and grid cells..." led by
@Rob_Mok
is out. So called place and grid cells may arise from general learning rules that are not specifically spatial. E.g., Grid response is a consequence of study designs, not core to brain.
Thanks to
@pouget_alex
,
@KordingLab
, and the
@neuromatch
community for the engaging debate and discussion yesterday on the role of Bayesian models in neuroscience. Below is a link to replay the event and my conclusion slide.
"The inevitability and superfluousness of cell types in spatial cognition" w
@ken_lxl
@Rob_Mok
Whether place, border, head direction, Jennifer Aniston, or whatever cells, are we fooling ourselves? Are these intuitive findings scientific discoveries? 1/6
"Estimating the functional dimensionality of neural representations" is published open access in NeuroImage. Easy to use method with Matlab and Python code online. Examples are for fMRI, but suitable to other data types.
Out today, "Ventromedial prefrontal cortex compression during concept learning" w
@mmack
@preston_lab
. vmPFC compresses information during learning in a goal-directed manner. Its timecourse is unique and parallels attention weights from learning models.
New blog, "A neuroscience-inspired approach to transfer learning" w
@ken_lxl
@BDRoads
. We add goal-directed attention to the middle of a deep convolutional network and find it better adapts to new tasks than retraining the top layer as is standard in ML.
New paper with
@adamnhornsby
"How decisions and the desire for coherency shape subjective preferences over time". We find people's preferences align to match their choices, which in turn guides future decisions. 1/n
"Bidirectional influences of information sampling and concept learning" w
@braunlich_k
is out in Psychological Review. What you know affects what you see which affects what you know. We offer a simple account that consists of two model components. (1/6)
New preprint with
@Rob_Mok
, "A Non-spatial Account of Grid and Place cells". So called place and grid cells may arise from general learning rules that are not specifically spatial. E.g., Grid response is a consequence of study designs, not core to brain.
New preprint "The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks" (DCNNs). Continuing the virtuous cycle between neuroscience and ML/AI by incorporating top-down, goal-directed attention into DCNNs. w
@BDRoads
@ken_lxl
New preprint "The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks" (DCNNs). Continuing the virtuous cycle between neuroscience and ML/AI by incorporating top-down, goal-directed attention into DCNNs. w
@BDRoads
@ken_lxl
Rosa Cao's paper on Predictive Processing is BRUTAL. Traditional and predictive processing accounts of the brain are informationally equivalent and only differ in whether one arbitrarily likes to view the "upward" signal as conveying error or the stimulus.
New preprint, "Levels of Biological Plausibility", covering how understanding of mechanism, reduction, and emergence are tied to levels, as well as how biological plausibility is an incoherent concept under a levels of mechanism view.
Our new blog "Model-based fMRI giveth and taketh away" shares our experiences as one of the 70 labs participating in the project on reproducibility in fMRI analyses.
@seb_bobadilla
,
@o_guest
, and I did 2 analyses and found (1/n)
"How a CogSci undergrad invented PageRank three years before Google" is a blog highlighting what cognitive science can contribute to machine learning, touching on my first project and pub way back in 1995.
How can words shape meaning? We (
@ken_lxl
@nickjsexton
) offer "A deep learning account of how language affects thought" that goes from images to meaning to words. Words are a discriminative signal that shapes meaning, like how actions and decisions can.
When does unsupervised training help or hurt human learners? Work with Franziska Bröker and Peter Dayan on getting semi-supervised learning right by aligning a learner's internal representations with unsupervised training.
Our (
@BDRoads
) TiCS spotlight "Similarity as a Window on the Dimensions of Object Representation" discusses exciting work led by
@martin_hebart
on inferring semantic representations from human similarity ratings. (1/2)
New preprint w
@nickjsexton
"Directly interfacing brain and deep networks exposes non-hierarchical visual processing". Instead of correlating model and brain activity, a stricter test of correspondence is driving model behavior with brain activity. (1/6)
Open is good right? In response to
@KriegeskorteLab
's open review of our work on neural similarity, I offer you "An open review of Niko Kriegeskorte", which is a less tedious read that touches on how difficult it is do something novel in this field.
1000th tweet. (1) New profile pic for the pandemic (2) brief review paper on "Model-based fMRI analysis of memory". The paper covers the role cognitive models can play and how they mesh with other approaches.
BrainGPT aims to tame the vast neuroscience literature to assist scientists in their research. Sign-up at to get involved. The video below provides background and motivation for the project. Our benchmarking effort is also covered.
When people do "semantic foraging" from memory (e.g., list all the animals you know) they tend to generate clumps of related items (e.g., pets, then African animals, then farm animals, etc.). ChatGPT shows this same behavior for a bunch of prompts, 1/2
I am struck by neuroscientists often conflating "all our constructs arise from lower-level entities" with naive reductionism, degenerating to name calling at times. How the physical world operates and how to best study and understand minds need not align.
It might be time to go farther and break away from the content heavy training in the traditional psych degree with a new degree track that emphasizes mathematical and computational methods more as these are the skills/knowledge increasingly needed in research and many jobs.
New blog post!
👩🏻💻👨🏿💻👩🏾💻👨🏽💻👩🏼💻👩🏿💻👨🏻💻
Why women in psychology can't program
"About two months ago my brother, who works in a data science on social psychology data, asked me why his colleagues, who are women and have PhDs in psychology, cannot code"
I uploaded my first youtube video; watch your back PewDiePie! The video is my CNS talk from last month. "Category Learning as Compression", which is joint work with
@mmack
,
@preston_lab
,
@braunlich_k
, and others.
System alignment is a different, complementary, way to view learning. Rather than learning from events, entire systems (e.g., visual and linguistic modalities) are aligned based on similarity relations within each system. (1/2)
Here's a recent talk I gave on the project, touching on findings from here, . The video is sectioned for those wishing to skip the bits on explanatory vs. predictive approaches.
New preprint with
@Rob_Mok
, "Abstract neural representations of category membership beyond information coding stimulus or response". For flexibility, the brain represents categories in a format orthogonal to sensory-motor codes, akin to an a amodal symbol
How predictable is neuroscience? Can LLMs outperform humans? Please participate in the survey to help us find out. You choose between two versions of a neuro abstract: the original vs. one with altered results. Which is which? 1/2
New preprint with
@BDRoads
, "Learning as the Unsupervised Alignment of Conceptual Systems". Because items similar in one system (images) are also in other systems (text, audio), supervised tasks can be solved by unsupervised learning.
What makes two brain states similar? New preprint from my lab by
@seb_bobadilla
@mehrotrabhinav
tests competing similarity measures. Same measures do well across brain regions, but change across tasks. Pearson corr, the de facto standard, never does well.
New preprint "The neural link between subjective value and decision entropy" w
@seb_bobadilla
@o_guest
. Subj value and confidence, measured as inverse entropy, are intertwined throughout the brain; tied to the desirability of actions. No pure value areas!
"Adaptive stretching of representations across brain regions and deep learning model layers": Using monkey multi-unit recording data, we find representations across cortex (PFC, FEF, LIP, IT) adaptively stretch along task relevant stimulus dimensions. 1/7
Two postdocs in computational cognitive neuroscience wanted. We are open and supportive of people coming from either the psych/neuro or the compsci side. One push is making deep learning more human-like using brain and behavioral data. Submitted ad,
(1/2)
New work with Daniel Barry, "Human learning follows the dynamics of gradient descent". Artificial neural networks are everywhere in psych/neuro, but few have considered whether humans and networks share common dynamics when learning new tasks. (1/4)
"Bidirectional Influences of Information-Sampling and Concept Learning". New preprint from lab w
@braunlich_k
that casts categorization as an active sampling process in which what you know affects what you see. From eye movements to information bubbles!
Work w
@Rob_Mok
, "A multi-level account of hippocampal function from behaviour to neurons". How can high-level cognitive constructs like symbols, clusters, and so forth be implemented in neurons? That's a gap not addressed by model-based neuroscience. 1/8
In the preprint, , critics worried there were precursors to our BrainBench test items in the scientific literature that helped LLMs predict results. Here's an additional check to rule that out. 1/2
PhD studentship in AI available NOW. Just got word of funding for a project on unsupervised deep learning. Ideal for students interested in human cognition who take some inspiration from the brain. Please get in touch () ASAP!
New work with
@adamnhornsby
on how people form subjective preferences through discriminative learning to promote self-consistency. While standard reinforcement learning (RL) can master Atari games, Go, etc., real life doesn't serve up points to maximise.
New preprint with
@o_guest
, "Levels of Representation in a Deep Learning Model of Categorization" The convolutional network (think ventral stream) provides representations to support novel concept learning (think hippocampus, mPFC). (1/4)
Updated preprint, "Measures of Neural Similarity" For non cogsci folks, we explain similarity≠classification, show sim. generalizes beyond fixed classes, information theory basis for decoding, and how CNNs work with diff. sim. functions.
@seb_bobadilla
One upshot is that there are no value areas per se in the brain because wherever there is value there is also a confidence signal that combines to provide a post decision evaluation of the choice option.
We find a systematic relationship between subjective value (SV) and inverse decision entropy (iDE, similar to decision confidence) in the medial cortex (and other areas). 2/n
"Measures of Neural Similarity" led by
@seb_bobadilla
is out. We used a decoding approach to determine what makes two brain states similar. Findings were striking: the same similarity measures flourished across brain regions, but not across tasks. (1/2)
New preprint with
@adamnhornsby
"Sequential consumer choice as multi-cued retrieval". How do you make a choice when the options are virtually unlimited, like in online grocery shopping? One possibility is that our last choice cues the next in memory. 1/6
New preprint, "A Deep Learning Account of How Language Affects Thought", w
@ken_lxl
@nickjsexton
. Shared labels highlight commonalities between concepts whereas contrasting labels make differences apparent. We offer a straightforward account, (1/2)
Read about an alternative timeline where I don't exist, "To learn how cognition is implemented in the brain, we must build computational models that can perform cognitive tasks, and test such models with brain and behavioral experiments"
@NKriegeskorte
New work, "A controller-peripheral architecture and costly energy principle for learning", by
@ken_lxl
@Rob_Mok
@BDRoads
We offer a framework for how brain regions coordinate to complete a task and apply the approach to concept learning from images. 1/4
@tallinzen
@o_guest
Too much to say on twitter, but just look at the conference. it's a ton of parallel tracks of hit-and-miss quality to suck in sufficient numbers of junior researchers into a pyramid scheme in which they pay tribute to baby boomer club members awarding each other prizes.
SFN, tomorrow afternoon, "The neural link between subjective value and decision entropy" by
@seb_bobadilla
and
@o_guest
. Value and inverse entropy (confidence) are deeply intertwined across the brain, co-occurring with a +/+ and -/- pairing. No pure value areas. (608.03/BB54)
Thread on new paper "Sequential consumer choice as multi-cued retrieval" w
@adamnhornsby
@ScienceAdvances
. We use multiple embedding spaces to explain how people make open-ended sequential decisions. How do you choose from an unlimited set of options? 1/6
Can learning occur offline with asyncrhonous inputs? Yes, in systems alignment entire embeddings (e.g., visual and linguistic) are mapped in an unsupervised manner. Systems alignment appears to support children's word learning. w
@kaarinaho
@BDRoads
(1/3)
For model-based and multivariate fMRI analyses, this can tell what an account is missing, "Estimating the functional dimensionality of neural representations",
Benchmark time! In search of collaborators. To properly develop BrainGPT and other approaches, we need to make and publish a decent neuroscience benchmark. We are looking for 20 or so co-authors to contribute several test cases each (instructions will be provided). (1/2)
Introducing , a large language model (LLM) to assist neuroscience research. BrainGPT is trained on the vast neuroscience literature and can be used to optimize study design, detect anomalous results, and evaluate models against broad data patterns. (1/3)
New blog, "Fast food science is a shit sandwich". Are we expediting science by engaging in real-time, interactive debate or are we instead making deep thought and inquiry that much more difficult?
I've seen related disasters in Psych/Neuro/CogSci. Be weary of complex models when the code is not freely available and does not generate the published results. Pages of magic constants and if-then special cases are red flags. Learning models should be trainable from scratch.
Topic modelling on consumer activity reveals semantic categories organised by goals, not shared features. Big data from supermarket offers alternative to text corpora analysis. For concepts, You are what you eat! Preprint with
@adamnhornsby
@london_rosie
Note: does not summarize nor retrieve articles. In such cases, LLMs often confabulate, which can be harmful. Instead, BrainGPT stitches together existing knowledge too vast for human comprehension to assist humans in expanding scientific frontiers. (3/3)
Bridging levels of analysis, cognitive models reflect individual differences in learned attention for both behavioral and neural measures. Now in press at JEP: General,
Version 1.0 of the Matlab and Python code for "Estimating the functional dimensionality of neural representations" is up thanks to the heroic efforts of
@o_guest
,
@seb_bobadilla
, and
@braunlich_k
in lab. Please give the code a try, available on github,
I had a great discussion with
@pgmid
on his Brain Inspired podcast, touching on concept learning, cognitive models, the hippocampus, deep nets, emergence, levels of analysis, and more. It was super fun. (1/2)
For folks excited by this recent Nature paper on dimensionality and smoothness of neural representations, here's a robust method to estimate () dimensionality with code (). For representational smoothness,
We (
@kaarinaho
@BDRoads
) offer a new viewpoint on learning. Rather than master (x, y) pairs (e.g., stimulus, category), we propose entire systems are mapped back-and-forth. E.g., from X (e.g., images) to Y (e.g., words) and from Y to X. People do this! 1/3
"Model comparison, not model falsification", I think a lot of thorny problems are minimized by model comparison, rather than debate over optimality, etc. I look forward to reading the other commentaries to this enjoyable BBS article.
Good advice for those restricted to bad practices. For students serious about
#openscience
and avoiding errors in 2018, the data analysis should be done by a script (R, Python, whatever) on the raw data files. Even better, properly comment and use version control (e.g., github).
Good morning SFN attendees, start your day with 170.10 / AA18 - "An abstract neural representation of category membership beyond information coding stimulus or response" by
@Rob_Mok
. The title says it all, surprising, like the brain constructed an amodal symbol.
Neuro folks, many are confused: Last week, behavior was key. This week it's philosophy. Oscillate between neural networks are the savior to they are implausible. Build new ontologies from data, but theory is key. Etc. All of this confusion is largely from same circle of people.
Whoa, that was the nicest interview ever; it reminded me that work is a fun intellectual journey. It's probably time to finally join the BNA (
@BritishNeuro
),
For the politically minded, new work with
@adamnhornsby
shows voters readily adopt any position their chosen candidate takes, particularly those from a certain US party. The drive to be coherent in terms of past choices (including votes) is powerful.
The "The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks" is now out in Computational Brain and Behavior. The quote tweet below briefly walks through the preprint. The published version includes an added bonus, (1/2)
People use top-down attention, such as when searching for one's keys. One idea is that prefrontal cortex helps reconfigure our visual system to be more sensitive to features relevant to our goals. This has costs and benefits. 2/n
This is a different way of viewing learning. In addition to trial-by-trial correction, people may align entire conceptual systems to facillitate learning and zero shot generalization. For example, people might align an embedding space for visual objects with one for words. 1/3
New paper w
@BDRoads
&
@ProfData
: "System alignment supports cross-domain learning and zero-shot generalisation" (). We find that alignable systems accelerate learning for cross-system mappings, and facilitate generalisation to completely novel stimuli.
Do you think neuroscientists or large language models (LLMs) will perform better on the benchmark?
(Neuroscientists, please participate in the benchmark. Here's the survey, 11 questions, 15-20 minutes, )
Thanks to everyone for the fruitful discussions on open and closed reviewing. I learned a lot. Open review is clearly not one thing and it's not simply a question of how open, but of how well conceived and executed. E.g., platform used, consent,etc. Not a seat-of-the-pants thing
Cognitive models are "real" at both behavioral and neural levels, linking individual differences in attention across levels.
Preprint with
@braunlich_k
, "Occipitotemporal Representations Reflect Individual Differences in Conceptual Knowledge"
Check out
@Rob_Mok
's poster "A Non-spatial Account of Grid and Place cells". We think place and grid cells are poorly named bc they arise from general learning rules that are not specifically spatial. E.g., Grid response is a consequence of study designs, not core to brain.
The grid cell meeting is starting in a few hours and we will be sharing most of the event with the whole wide world! Link to the livestream on our website from 9am BST:
#gridmeeting18
Great to hear
@SaxeLab
and
@talia_konkle
have critical comments about pushing "biological plausibility" on this podcast. I have the paper for you devoted to this topic and related issues in emergence and reduction, "Levels of biological plausibility",
Once we tackle these 26 ideas and assumptions holding us back (and befriend more crackpots, per
@gershbrain
), we'll sing...
Don't you know I'm still standing better than I ever did
Looking like a true survivor, feeling like a little kid.
anyone...?
And thanks to
@socmathpsych
for the best paper award in
@CompBrainBeh
(a journal full of amazing papers). There's a lot that can be done examining behaviour outside the lab at scale to better understand the human mind.
*Paper published with
@ProfData
and
@dunnhumby
at
@CompBrainBeh
!* . We used data from ~1.3m till receipts to evaluate how people semantically represent objects in-the-wild. Our model revealed that real-world categories are organised around goals👇