We're thrilled to inaugurate our new blog
@FlowersINRIA
with this post:
Language as a Cognitive Tool: Dall-E, Humans and Vygotskian RL Agents
Thanks
@cedcolas
@karch_tristan
for the outstanding work!
1/21
Take Flow Lenia, add random inflow and outflow of matter + ecological niches, what happens ? 🧐🔬
💡Spontaneous self-organization of an evolution process ? Generating exploring "agents"?
Artificial lifeforms are super fascinating to watch.
These self-organizing, self-replicating, “lifeforms” emerged from a continuous time cellular automata system called Flow-Lenia.
Lenia is a family of CAs generalizing Conway’s Game of Life to continuous space, time and states.
New conference on Curiosity, Creativity and Complexity in humans and machines !
With very interdisciplinary panel of speakers in neuroscience, psychology, economics, AI, ...
👉 May 23-25th, NY, information to submit abstracts or just attend:
A great recent news in the team
@FlowersINRIA
was the Best Paper award at Alife conference🏅
It introduced Flow Lenia, a CA wth mass conserv. & local evolving rules, opening possibilities for self-organization of evolution processes
📺
@eplantec
's talk is now available !!! 🎉👇
Video of my presentation at ALIFE 2023 is now available !
I present Flow Lenia, a mass conservative CA in which life reminiscent creatures emerge, interact and potentially evolve !
📄 )(🧵 )
In the beginning, there is only an environment.
No body, no sensors, no agents.
How could creatures self-organize?
Able to move, adapt, interact, generalize?
Use gradient descent + curriculum learning!
👉 New blog post:
1/n
Alan Turing proposed in 1948 the use evolution to “train” a particular type of neural network he called a “B-type unorganised machine”. He evolved binary networks to perform all sorts of tasks back then.
🌍Foundational models are cultural technologies
They (will) mediate and influence human culture in ways that are unprecedented: here are the real opportunities 🚀and challenges🚧of
#AI
today !
📚Here's a paper to make progress understanding them 💡👇
New Paper: "Machine Culture"
AI systems are increasingly integrated into the human cultural fabric. What can we learn from human cultural evolution to anticipate the impact of intelligent machines on culture (socially transmitted information)? [1/9]
🌐Open-ended learning may be the new
#AI
frontier
It's about enabling machines to learn after the've already learned all existing data 🤖 📚 ❓
But what's "open-ended learning"? 🤔
📖 Here are steps towards better understanding this concept:
Automatic Curriculum Learning For Deep RL: A Short Survey
What is ACL? Why use ACL? Which forms of ACL (e.g. goal generation, reward shaping, procedural gen. of envs, self-play)? What open questions?
w
@RPortelas
@cedcolas
@lilianweng
@katjahofmann
Huge congrats
@mayalen_etc
for your amazing work on this
#Neurips2020
paper (oral presentation!):
Meta-diversity search + learning of hierarchies of representations for automated discovery of novel structures in continuous
#cellularautomata
Super happy to share this survey within the field of developmental
#AI
we've helped to grow in the last decade!!!
👉 Intrinsically motivated goal-conditioned reinforcement learning
Special thanks
@cedcolas
@karch_tristan
for the outstanding work!
1/17
Today with
@forestier_seb
we made a fun discovery while setting up a live robot learning experiment for a tv documentary project.
We aimed to compare runs of curiosity-driven goal exploration and a RL scheme (sparsely) rewarded for moving a ball
« algorithmic information theory essentially says that the probability of producing some types of outputs is far greater when randomness operates at the level of the program describing it rather than at the level of the output itself »
Excited to preview our list of invited speakers for
#iclr2019
in New Orleans:
Léon Bottou, Cynthia Dwork, Ian Goodfellow, Noah Goodman, Mirella Lapata,
Pierre-Yves Oudeyer, Emily Shuckburgh, and, Zeynep Tufekci
Tickets go on sale tomorrow @ 4pm PST at
Super happy that the
#popularscience
book for kids I wrote with
@dphroy
is out!
It introduces
#AI
,
#machinelearning
and
#robotics
to >7 years old (in french :))
It was a lot of fun to conceive & now rewarding to see in the hands of children!
1/n
Honored to receive the Atos - Joseph Fourier prize for work on curiosity-driven
#AI
and its applications !
A big thank to the many great colleagues/students I have been working with on this topic over the past decade !
Rendre l'
#iA
autonome dans son apprentissage comme un enfant le ferait, pour rendre cette phase bien moins consommatrice d'énergie, c'est que projet de
@pyoudeyer
récompensé par le prix
@AtosFR
Fourier 2019 Félicitations
@Inria
Bordeaux !
Dans les collèges et lycées tout le monde parle de
#ChatGPT
🦜🧑🎓🗣️
Mais combien savent comment ça fonctionne ?⚙️ Quelles sont ses possibilités et ses limites ? 🤔
Pour les enseignants et les élèves, voici un tour d'horizon en 5 mn 🚀🤖💡
#education
New blog post from
@RPortelas
!
Teacher Algorithms for
#DeepRL
Agents that Generalize in Procedurally Generated Environments
Presents the emerging Deep RL sub-field of automatic curriculum learning with a case-study on ALP-GMM
Many practitioners observe that pretraining LLMs on code may boost natural language "reasoning" abilities
Do you know papers actually showing this through systematic scientific experiments ?
@DrJimFan
@Francis_YAO_
@gneubig
@srush_nlp
@Thom_Wolf
2) wth
@ClementRomac
, development of the TeachMyAgent benchmark for systematic comparison of automatic curriculum learning algorithms in
#DeepRL
(= teacher algorithms)
👉
🧐Curiosity-driven learning has many different forms, functionalities and usages in 🐵animals & 🤖machines
▶️ We review them in this paper led by Alexander Ten and
@Clement_MF_
Amazing: every week I see a
#DeepRL
paper comparing algorithms using mean performance over *3* seeds !
Yes, ****3**** !!!
Please please
#DeepRL
community, your great ideas will be served better using standard scientific methods!
LLMs often encode wrong information about the world.
How to mitigate this?
Not by passive reading: they need to interact wth the env, make causal intervention, get feedback & update.
To understand better this process, systematic studies in simple envs are needed, here's one👇
🚨🚨
Large Language Models encode world knowledge that can benefit RL agents when solving decision-making problems (e.g. in robotics 🤖).
But what happens when their knowledge are wrong? 🎯
Can we align LLMs and ground their knowledge through interactions with online RL? 🕹
In this new blog post, I present my perspective on the future of
#AI
and an approach to scale to human-like intelligence: focus on socio-cultural *environments*
👉
1/n
Today
@BertChakovsky
is giving a talk
@FlowersINRIA
@inria
on evolution and self-organization of intelligent agents from the ground up
(= from envs where there is not already things like « bodies », « senses », « muscles », « brains » or « learning »)
Voici la 5ème vidéo de notre série pédagogique sur
#ChatGPT
pour les collégiens, lycéens, profs, parents 🦜🧑🎓🗣️
#IA
Ici nous expliquons comment il est possible de simuler un "raisonnement" ⚙️🤔avec ChatGPT, et comment celui-ci peut dérailler 🤖😅🤯
We're so happy to start a new collaboration with the amazing
@BertChakovsky
at
@FlowersINRIA
!!!
+
We open a master level internship for 2021 to study the emergence of sensorimotor behavior in
#Lenia
creatures such as the ones below 👾👇
#Alife
#AI
A few years ago, we wrote a paper that ended up being foundational in the team
@FlowersINRIA
: it sparked most of our recent research on autotelic
#AI
and automatic curriculum learning
👉 An extended version just appeared in the
@JmlrOrg
journal !!! 🚀🥳
📰
The future of
#AI
may not be generalist
#LLMs
, i.e.
#AGI
Rather it may be 'societies' of diverse specialized models
In this thread, I'll discuss GPT-42 (yes, 42 :)) and links with cultural evolution, open-source, industry and open-endedness
🧵
Honoured to receive the prize
@inria
@AcadSciences
from French ministry of research
@VidalFrederique
and get an opportunity to talk about how fundamental research on curiosity-driven learning and
#AI
can find societal applications in education
#edtech
J. Gottlieb and I's
@NatRevNeurosci
review is out!
"Towards a neuroscience of active sampling and curiosity"
We cover converging work in
#neuroscience
,
#psychology
and
#AI
, giving rise to a new exciting field.
Lenia: Biology of Artificial Life
Congrats
@BertChakovsky
for this publication in Complex Systems, outstanding work on designing and studying a continuous game of life !
Go play with it:
Unsupervised RL workshop
@iclr2019
Great to see many of the topics and techniques worked on by the developmental
#AI
and
#robotics
community for a decade finally come through within the
#ML
circles !
Comp. models of infant devlopment
@iclr2020
? 😄
📺Les vidéos de notre série pédagogique (
@FlowersINRIA
@Inria
) expliquant les modèles de langage aux collégiens et lycéens sont disponibles en téléchargement 🚀🎉
#Litt
éracie
#IAG
énérative
⬇️
Dans les collèges et lycées tout le monde parle de
#ChatGPT
🦜🧑🎓🗣️
Mais combien savent comment ça fonctionne ?⚙️ Quelles sont ses possibilités et ses limites ? 🤔
Pour les enseignants et les élèves, voici un tour d'horizon en 5 mn 🚀🤖💡
#education
Not a bug !!! The robot discovered that tapping the joystick and scratching the table with hand would make table vibrations that move very slightly the ball ==> non zero reward ==> refining that behaviour !!!
But a clear deceptive local minimum :)
Here is the vibration !
Intrinsically Motivated Exploration of Learned Goal Spaces
Congrats to A. Laversanne-Finot & A. Péré for the journal extension of this CorL paper studying how diversity search algorithms can work from pixel-level perception on clumsy real physical robots
How can an RL agent learn to use language to get help from a knowledge source ?
How can it learn when and what to ask?
Learn how to interpret the meaning of answers?
👉 See our new paper accepted at
#ICML2022
📖
💻
Several open
#PhD
&
#postdoc
positions in my team
@FlowersINRIA
:
- emergent communication, curiosity-driven multi-agent Deep RL
- adaptive training of attention for humans
- Deep RL to optimize intervention in
#Covid
epidemic models
#AI
#MachineLearning
Y. Bengio calling
#machinelearning
to model infant development !
What a great path, looking back 10 years ago when
#developmentalrobotics
papers on intrinsic motivation and curriculum learning where thought to be crazy by the same
#ml
community
#NeurIPS
Hey Twitter, I am searching for online educational material usable to introduce
#AI
and
#machinelearning
concepts to 12-15 years old in school.
It can be full pedagogical activities, or (fun) interactive web simulation that afford educational explanations.
All ideas welcome !
🌻Introducing TransFlower =
#Transformer
+ Normalizing Flows for high-dim contin. control
👉
#Dance
generation🕺💃 cond. on music wth various styles & motion prompting, congrats
@guillefix
!
Paper
Code
Fun
Large language models are essentially cultural transmission technologies 🦜🗣️🦜🗣️🔁
They are culture models, encoding cultural regularities present in texts and in human feedback they were trained on. They amplify them when they are used, by humans or other machines
#LLMs
🧵
Lenia and Expanded Universe
#gameoflife
in higher dimensions, featuring the emergence of "virtual eukaryotes" that reproduce, differentiate, migrate, interact and react with each other.
Outstanding work by
@BertChakovsky
(independent researcher !)
Behavioral and Cognitive Robotics: An Adaptive Perspective
An open-access book by Stefano Nolfi who has been a major actor of evolutionary and behavioral robotics
Autotelic AI:
agents learn by sampling and pursuing their own goals
Vygotskian AI:
agents use language as a cognitive tool (inner speech)
Vygotskian Autotelic AI:
agents use language to imagine their own goals & for planning, reasoning and learning about them
PBCS: Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning
by Matheron, Perrin &
@OlivierSigaud
Extends Go-Explore, builds on Backplay and the idea of decoupling exploration and exploitation
Mind-blowing.
What if the billions invested in
#deeplearning
were used to investigate morphogenesis?
This was the route Alan Turing pointed out to understand the origins of intelligence.
A key dimension of natural intelligence is in choosing what and when to learn (and whether the learning is done through curve fitting or something else is orthogonal).
Beyond statistical learning, AI require advanced motivational/teleological architectures
Whenever I see this kind of headline, I always think "But what if intelligence is mostly about curve-fitting, and we're merely too un-self-aware to notice?"
I just found back my Epirob 2009 presentation discussing theoretical models of how curiosity and language learning are closely linked in children.
Cool to see a posteriori how we implemented this vision with my team
@FlowersINRIA
in the last decade.
💡Curiosity-driven autotelic
#AI
can help scientists uncover behaviors of complex self-organized biological systems🔬
Here we show how they help reveal GRNs surprising (sensorimotor?) abilities👾🚀
With interactive paper, tutorials and open-source code:
New preprint!
@mayalen_etc
@Clement_MF_
@pyoudeyer
- AI-driven Automated Discovery Tools Reveal Diverse Behavioral Competencies of Biological Networks. Next step in our efforts to study molecular networks with the lens of basal cognition.
The mysteries of curiosity-driven learning and extreme specialization in humans
Great new dialog in the latest
@ieeecis
newsletter on Cognitive and Developmental Systems, initiated by
@celestekidd
and a great list of contributors
@cogdevsoc
@DevRobNews
Great that talks of
#Neurips2020
Deep RL Workshop are online + accessible without registration, thanks organizers!
Mine is about machines that invent their own problems, wth a focus on meta-diversity search and language as a tool for creative exploration
So happy to give the inaugural talk of the "Developing Minds" online lectures, with Linda Smith and Josh Tenenbaum following soon!
I will talk about Developmental
#AI
: machines that learn like children and help children learn better
👉 Register here (free attendance):
So many people try to understand
#LLMs
from the lens of human-like intelligence (both people who over hype LLMs and those claiming they're superficial parrots 🦜- all comparing them to humans 🧑🤝🧑🧠)
Maybe we should rather try to study them as alien intelligence ? 👾💡
🧵
A developmental robotics/computational development workshop
@NeurIPSConf
! Amazing and so great !!!
and with a great team of organizers !
"BabyMind: How babies learn and how machines can imitate"
@Chen_Yu_CY
any info available on deadlines?
Large language models could be seen as interactive encyclopedias of human cultural habits (those which can be encoded in text)
If trained on well-chosen corpuses, they could become a useful tool to address scientific questions about human cultures (provided adequate methodology)
Automatic Curriculum Learning For Deep RL: A Short Survey
What is ACL? Why use ACL? Which forms of ACL (e.g. goal generation, reward shaping, procedural gen. of envs, self-play)? What open questions?
w
@RPortelas
@cedcolas
@lilianweng
@katjahofmann
Automatic Curriculum Learning For Deep RL: A Short Survey
What is ACL? Why use ACL? Which forms of ACL (e.g. goal generation, reward shaping, procedural gen. of envs, self-play)? What open questions?
w
@RPortelas
@cedcolas
@lilianweng
@katjahofmann
Instead of asking whether
#AI
systems are (super)intelligent or (half)conscious, maybe we should ask whether they are useful, inspiring, enlightning, pedagogical, beautiful, poetic or simply fun
The IMAGINE approach to using language for creative exploration in
#DeepRL
agents will be presented next week
@NeurIPSConf
Camera ready:
Videos:
Code:
Colab notebook:
#neurips2020
A major challenge in current research in
#LLMs
🦜 is how they can be grounded in interactive environments, aligning their pretrained knowledge to external dynamics 🤖🎮💡
▶️Next week
@icmlconf
@ClementRomac
&
@CartaThomas2
present GLAM, an online RL approach to do this !!! 📖🚀
🚨🚨
Large Language Models encode world knowledge that can benefit RL agents when solving decision-making problems (e.g. in robotics 🤖).
But what happens when their knowledge are wrong? 🎯
Can we align LLMs and ground their knowledge through interactions with online RL? 🕹
Very clear and thoughtful paper on natural curiosity by philosopher Jennifer Nagel, with extended connections among works in philosophy, psychology and
#AI
Thank you everyone for amazing contributions and discussions at this week's Dagstuhl seminar on
#developmentalAI
!!!
This showed great new research opportunities and synergies at the crossroads of
#AI
#devpsychology
#machinelearning
👶🤖💻🚀
In this thread I'd like to share a very elegant idea proposed by Thomas Carta that he'll present this week
@NeurIPSConf
It's about automating task-specific reward shaping in language-conditioned RL without domain-specific expert knowledge
📖
To start the new year with fun, here's a gift from the
@flowersInria
team:
👉An interactive online app to test generalization capabilities of
#DeepRL
agents! 🎁🤖
Chimpanzees 🐒, fishes 🐟, spiders 🕸️and bipedal walkers trained with DeepRL are now exploring the jungle in a
@huggingface
space 😀
👉 Here's the link to launch them in your browser and play ! Thanks a lot
@ClementRomac
!
📢 We just integrated our Deep RL interactive demo in a
@huggingface
space 🤗👉
This version fetches policies uploaded on the hub with the tag 'teach-my-agent-parkour'. You can thus choose policies you want to play with and even upload yours!
1/9 🧵
A great perspective on the origins of play in humans, articulating various theories ranging from predictive processing to the learning progress hypothesis.
It also includes a nice discussion of niche construction mechanisms in this context.
🥳🚨 PUBLICATION ALERT! 🚨 🥳
Have you ever wondered why humans play? How play relates to learning? Why play feels so fun?
In this paper, we propose a cognitive theory of play utilising recent ideas of the brain as a predictive organ.
OA:
🧵 1/5
🌐Building on our work on grounding
#LLMs
🦜 in interactive environments with online RL, we are opening a master level 6 months internship
@FlowersINRIA
to study the impact of such grounding on the knowledge and structure of these LLMS 🚀
👉 To apply:
🚨🚨
Large Language Models encode world knowledge that can benefit RL agents when solving decision-making problems (e.g. in robotics 🤖).
But what happens when their knowledge are wrong? 🎯
Can we align LLMs and ground their knowledge through interactions with online RL? 🕹
Great work showing how intrinsically motivated goal exploration can be used to automate discovery of novel patterns in wet chemical systems !!!
Congratulations
@jgrizou
and team!
Intrinsically motivated
#machinelearning
algorithms open new perspectives in physics/chemistry
We applied intrinsically motivated
#AI
to explore a chemical system in the lab:
@ScienceAdvances
Results:
- 3x more varied observations than screening
- enabled new observation that we study in depth
- data show we improve rate of discoveries
1/8
Tomorrow at 5:05 pm GMT I'll be discussing with
@joelbot3000
on open-endedness at the Never Ending RL workshop
@iclr_conf
#ICLR2021
That will be a lot of fun!
1/N We are excited to introduce our
@iclr_conf
workshop: A Roadmap to Never-Ending RL. We invite you to submit papers (up to 6 pages, excluding references and appendix) in the
@iclr_conf
format. Submission Deadline: February 26, 2021
#NERL2021
#ICLR2021
Large language models can be used as agent policies, and grounded using online RL to solve RL tasks in interactive environments
👉 How their prior knowledge increases sample efficiency?
👉 How can it boost generalization to new tasks?
Study led by
@ClementRomac
@CartaThomas2
!
🚨🚨
Large Language Models encode world knowledge that can benefit RL agents when solving decision-making problems (e.g. in robotics 🤖).
But what happens when their knowledge are wrong? 🎯
Can we align LLMs and ground their knowledge through interactions with online RL? 🕹
Here's the talk I gave
@EcoTheoryRL
workshop
@NeurIPSConf
Sculpting human-like AI by sculpting social environments
TL;DR
1) methods for educating learning machines are key and still mostly unexplored
2) we should put human-like sociality in AI envs
1/n