We made a planning algorithm faster by reusing action sequences and I'm stoked to share the preprint! This was joint work with my amazing supervisor, Peter Dayan.
1/ The first paper from my PhD work is out in
@PLOSCompBiol
! Deep thanks to my supervisor Peter Dayan for, well, everything; to
@nemethd
and Karolina Janacsek for a rich data set; and to the reviewers for helping make this paper more complete.🧵continues
I'm presenting this work at
#cogsci2023
in this Friday's decision making symposium between 15:20 - 16:40 AEST. I'm in Sydney in person so you can find me for a chat as well 😇
We made a planning algorithm faster by reusing action sequences and I'm stoked to share the preprint! This was joint work with my amazing supervisor, Peter Dayan.
Super excited that our paper on sequence compression and RL got accepted at
#NeurIPS2023
!
Joint work with
@EltetoNoemi
, Peter Dayan,
@marcel_binz
and Eric Schulz
@cpilab
See you in New Orleans in December!
We tested transformers vs compression algorithms for simplifying action sequences in deep RL just recently. Our realization was the same: compression in itself is great! The paper is still fresh and crispy:
Trying to set foot in US academia at the age Katalin did, as a first-generation student from a humble Hungarian community, she's my main source of inspiration now. I am lucky if I share a fraction of her ingenuity but I want to have all of her courage. Thank you for all you did.
BREAKING NEWS
The 2023
#NobelPrize
in Physiology or Medicine has been awarded to Katalin Karikó and Drew Weissman for their discoveries concerning nucleoside base modifications that enabled the development of effective mRNA vaccines against COVID-19.
We distinguish these forms of learning but they are in fact tightly related. Then it's a real puzzle to tell what the brain actually implements. Great debate intro by Sam Gershman at 'RL at Harvard' the other day.
Bengalese finch songs have a lot more interesting patterns than I would have thought! Come to my
#cosyne2023
poster to learn more:
III-059
Variable syllable context depth in Bengalese finch songs: A Bayesian sequence model
Noémi Éltető, Lena Veit, Avani Koparkar, Peter Dayan
I've been melancholic about our sister lab leaving, after so many collaborations, joint meetings, movie nights, table tennis, callisthenics and who even remembers everything. But now I'm just proud. Thank you for the science and the friendship.
🧵1/8 🚀 Exciting News from the Cognitive Science and ML Front! 🧠
After a fantastic journey at
@MPICybernetics
, a new chapter begins. I'm thrilled to announce my new role as the Director of the Institute of Human-Centered AI at
@HelmholtzMunich
based in
@CompHealthMuc
.
This colab was so much fun. We showed that action sequence simplicity is a useful incentive for faster reinforcement learning and finding cheaper strategies. No deepmind lab creatures were harmed.
New preprint: We present a reinforcement learning algorithm that learns to solve tasks with compressible action sequences! Joint work with
@EltetoNoemi
, Peter Dayan, Marcel Binz and Eric Schulz (
@cpilab
).
Applications are invited to participate in the Brains, Minds and Machines summer school [08/04/2024 -- 08/25/2024] in Woods Hole, MA. Deadline = 03/20/2024. More information here: .
Another day of seeing Dune everywhere. We harvest horseshoe crab blood because it's uniquely precious -- it is the only natural source of a type of immune cell that we need for vaccines. Hemocyanin makes things look sci-fi.
I will be at NeurIPS to present our work on simple sequence priors for RL, and why we think that simple is better for policy search! Reach out if you want to have a chat :)
And come hang out with me,
@EltetoNoemi
and
@marcel_binz
at our poster session on Thursday 10:45 am!
Human behavior is hierarchically structured. But what determines *which* hierarchies people use? In a preprint, we run an experiment where people create programs that correspond to hierarchies, finding that people prefer structures with more reuse.
1/7
Your new GP could be GPT. Personally I'm super excited about this because it's always been impossible for human physicians to keep up with the literature and provide precise facts&stats to their patients.
Microsoft Research has released BioGPT, a large language model trained on biomedical research literature. The model achieves better-than-human performance on answering questions from the biomedical literature, as evaluated on PubMedQA. The code for the
Our new preprint is out. We used a principled sequence model that grows with the data to capture how human participants refined a skill over time. Our method can be generally applied to study adaptive sequence complexity in animals and in machines.
We will continue in person on 24 July in Rotterdam. We will task ourselves with getting our amazing speakers from psychology, neuroscience and AI to either agree or agree to disagree. Getting all of them in the same room cannot fail to produce good science 2/n
My colleagues next door are still analysing GPT-3's behaviour tirelessly🧐Turns out that anxiety-inducing prompts elicit more biases, such as racism and sexism, in the responses.
New pre-print: Using computational psychiatry to understand LLMs, we find that GPT-3.5 responds to anxiety-inducing prompts with increased exploration and biases. Joint work with
@juliancodaforno
@WitteKristin
@akjagadish
Marcel Binz and
@zeynepakata
.
Ten months ago, we launched the Vesuvius Challenge to solve the ancient problem of the Herculaneum Papyri, a library of scrolls that were flash-fried by the eruption of Mount Vesuvius in 79 AD.
Today we are overjoyed to announce that our crazy project has succeeded. After 2000
We characterize computer algorithms by their complexity scaling. Susanne took to same approach to see how humans' solution times scale with the input complexity. So cool!
Super thrilled that our paper on the scaling of mental sorting just came out in
@CognitionJourn
!🥳
This is joint work with
@TheCharleyWu
, Ishita Dasgupta, and Eric Schulz
@cpilab
.
You can learn more at
Okayy the ultimate glitch in the matrix is that Barbenheimer is a real movie but it was made before the popcultural phenomenon. It's Asteroid City by Wes Anderson!
It's a universal law of panel discussions that the speakers conclude: 'Everything is ___ [insert topic of session]'. Examples: 'everything is a sequence', 'everything is statistical learning' etc. Leave yours in the comments.
#cosyne2023
You know you arrived in the US when they laugh out loud at you in the coffee shop for ordering one piece of cream puff and not making use of their awesome deal of getting 10 in a bulk.
Me: 'I applied for an AI internship at Tesla'
My boss: 'You still don't drive, do you?'
Me: 'No...'
My boss: 'Well you'll have something in common with the cars.'
I will forever aspire to think and speak like Frans de Waal did. His talk at cosyne last year was just electrifying. It's incredible that it was one of the last ones. Rest in peace!
Football players can tackle, get up, kick and chase a ball in one seamless motion. How could robots master these motor skills? ⚽
We trained AI agents to demonstrate these agile behaviours using end-to-end reinforcement learning.
Find out more:
Love the content and how this is done! Great insights into how
@berkeley_ai
Prof. Mike Jordan thinks about the future of AI and also a look into his research style
And thank you Barbara Rosario for creating!
@fchollet
I agree with the perils of extremism. But I and many psychologists strongly believe that mental health issues come first and extremist beliefs after, not the other way around. AI paranoia/pronoia is not the inducer of mental health issues but the expression of them pre-existing.
My first PhD preprint is out!
"Relating Objective Complexity, Subjective Complexity and Beauty", with a wonderful team -
@aabrielma
, Peter Dayan, Eric Schulz,
@FraBraendle
@cpilab
A thread 🧵
1/11
We are hiring students for paid internships in AI and brain research! This internship is meant for those who face difficult constraints in their careers, coming from underrepresented groups and places.
What doesn't kill you... Well, it just gives you a chance to regress to the mean, based on the figures. It is incredibly harmful to ignore this and instead suggest that rejection motivates good work. We know much better than this.
@PLOSCompBiol
@nemethd
2/ Our skills become much faster with practice because our brain learns the relationships among all the steps and can anticipate an upcoming step. This is called sequence learning and it underlies our motor skills such as making coffee.
Thrilled to announce the first annual Reinforcement Learning Conference
@RL_Conference
, which will be held at UMass Amherst August 9-12! RLC is the first strongly peer-reviewed RL venue with proceedings, and our call for papers is now available: .
@_rdgao
The crappier the economy, the larger role the cheap small luxuries, like freebies, have. They call this the 'lipstick effect'. But this is neurips, so bottles it is 😅
@avt_im
Isn't the main problem that it takes privilege to go to college in the US? For me, HS research was the starting point to become a first-generation academic in a family of carpenters -- in Europe of course. Free education is the key.
5/ To study this, we trained our participants on a motor skill for ten weekly sessions, comprising of thousands of trials, to form larger patterns of actions.
I bought a shirt at Banana Republic and I looked up what the phrase meant: a politically and economically unstable country. Now this comes to mind every time I see homeless on the NYC streets use Banana Republic suit covers as sleeping bags.
8/ Unlike many other seq models, our model is transparent! It not only makes trial-by-trial predictions for participants' speed but also indicates how, and how many, previous elements influenced individual participants’ internal predictions, and how this changed with practice.
New preprint: We present a reinforcement learning algorithm that learns to solve tasks with compressible action sequences! Joint work with
@EltetoNoemi
, Peter Dayan, Marcel Binz and Eric Schulz (
@cpilab
).
3/ A great challenge is that the steps of a skill can depend on a variably deep temporal context of previous steps. For the learner, it is not predefined which steps are related; or, in other words, how large the chunks of related actions are.
This might be a plausible cognitive model of human planning. Simply put: we are biased to think of sequences of steps that we often performed in the past instead of only considering what steps seem useful. We are excited to test this on human behavior. So stay tuned :)
@ChristophMolnar
@nntaleb
Great cautionary tale. But this is not a satisfying answer. You computed this probability *with* the assumption you suggest one should question. The probability of any sequence of 99, including 50 heads/49 tails, is this small.
#Montr
éalFire Day 5 after An's missing
Her parents' Canada Visa was approved yesterday. High living costs in Canada & US + travel will be a big burden on the family’s financial situation. We are raising funds to help cover their travel/living expenses:
4/ E.g. when making coffee, we should place the moka pot on the hob if 1) coffee is ground, 2) the pot is filled and 3) the hob is on; but it doesn't depend on whether we've aired out the kitchen. How do humans carve out such dependence patterns from sequences of actions?
6/ We used a Bayesian sequence model, based on a hierarchy of chunks, to track our participants' skill formation online. The model seamlessly combines predictive information from shorter and longer chunks, weighing them proportionally to their predictive power.
7/ By the way, the model is generally applicable to capture variable-order dependencies in sequence data, for instance, language, bird songs, motor behavior, and more. Feel free to try it out:
In the heart of World War II, as the Nazis took control of Copenhagen, a peculiar situation took place at the Institute of Theoretical Physics, led by physicist Niels Bohr. Two Nobel laureates Max von Laue and James Franck, fearing the confiscation of their gold Nobel Prize
To capture action sequences of arbitrary complexity, we use a Bayesian action chunking mechanism which finds statistically reliable structure at different scales. This gives rise to shorter or longer routines that can be embedded into a Monte-Carlo tree search planner.