New paper! Language and large foundation models come together to drive semantically meaningful exploration. This idea helps RL agents learn faster in 3D environments, even when language annotations are unavailable ()
Read on 🔎⬇️
ChatGPT can now browse the internet to provide you with current and authoritative information, complete with direct links to sources. It is no longer limited to data before September 2021.
Very recent work from Jesse Mu, et al () have also shown how language can benefit exploration in MiniHack and MiniGrid environments. We've now shown it can be done even without language annotations, and in 3-D! 8/9
Language structures human-learning in fascinating ways. Why shouldn't it do the same for agents? Vision-language foundation models bridge the two worlds together. 7/9
Agents are instructed to navigate a large-scale simulated City or interact with household objects in a virtual Playroom. Language-shaped explorers significantly outperform baselines. ⚡⚡ Best part: our method doesn't rely on language annotations. 6/9
Language gives agents a different perspective on the state space. It creates high-level abstractions that reflect the semantics of the environment. How does this work in practice? 3/9
Representations from CLIP-style models are particularly powerful. Language pretraining shapes these embeddings with semantics & transfers knowledge from large-scale caption data. So language can improve exploration even in environments that don't provide annotations/captions. 5/9
Humans naturally use language to communicate and highlight the most important parts of our environment. Helpful for novelty-based exploration, since language can help determine what's actually new and what's not. 2/9
We extend existing exploration methods by using language-based state representations, in lieu of controllable states (Never Give Up; ) or random features (Random Network Distillation; ). 4/9
Follow the money.
“However, if they had built a platform to buy and sell images that wasn’t nominally based on crypto, I don’t think it would have taken off. Not because it isn’t distributed… because this is a gold rush.”
We’ve acquired the MuJoCo physics simulator () and are making it free for all, to support research everywhere. MuJoCo is a fast, powerful, easy-to-use, and soon to be open-source simulation tool, designed for robotics research:
People think that the startup journey is straightforward, but ours definitely wasn’t. Here’s how we went from
@ycombinator
demo day in 2019 to pivoting 20 times, and eventually starting
@rutterapi
in 2021 and raising a $27M Series A led by a16z in 2022. A 🧵
@KyleMorgenstein
So far the company hasn’t done anything autonomous. But you have to admit that this technology is really impressive. It’s important to talk about every part of the pipeline!