Prof. of Computer Science at UT Austin with research interests in AI, robotics, machine learning, multiagent systems; Executive Director of Sony AI, America
10 years after DQN, what are deep RL’s impacts on robotics? Which robotic problems have seen the most thrilling real-world successes thanks to DRL? Where do we still need to push the boundaries, and how?
Our latest survey explores these questions! Read on for more details. 👇
I'm honored and proud to have been promoted to Chief Scientist of Sony AI, effective today. Looking forward to working with a fantastic worldwide team on several exciting and very ambitious projects!
Congratulations to (soon to be) Dr. Ishan Durugkar on the successful defense of his dissertation before an illustrious committee. He’s the 24th Ph.D. graduate from my lab and will soon be joining my team at Sony AI.
I can’t express how proud I am of my Sony AI team and all our collaborators on this result of our work over the past 2 years. See the details on today’s issue of Nature!
What is
#GTSophy
? 🏁
GT Sophy is an autonomous
#AI
agent trained utilizing a novel deep reinforcement learning platform. Our breakthrough research "Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning" was published today in
@Nature
.
Congratulations to the whole Sony AI team on what is surely one of the largest ever commercial deployments of an end-to-end reinforcement learning agent! It’s great to see all the positive feedback from gamers!
Introducing
#GTSophy
to
@thegranturismo
players of all levels is a significant milestone for the Sony AI team and the evolution of Gran Turismo.
Watch
@GTPlanet
take on Gran Turismo Sophy 2.0 in
#GY7
.
It's now well-known (and not surprising) that current LLMs have trouble generating correct plans that require real-world knowledge. Here's a possible solution. I'm very excited about this line of work!
Can we train vision-based reinforcement learning agents that achieve super-human performance against human players in racing games?
Check out our latest work to be presented at
@RL_Conference
!
Paper:
Website:
A very fast 🧵(1/7)
WisTex United won the RoboCup 2024 Standard Platform League "Challenge Shield" (lower division) competition. Collaboration with Josiah Hanna's lab at Wisconsin. Most exciting match was a 2nd half comeback (with some luck):
How to design State Space Models (SSM) from principles? We propose to view SSM's recurrence as the per-step closed-form solution to an online learning problem. To this end, we present Longhorn, a novel SSM that achieves 1.8x better sampling efficiency against Mamba.
Congratulations to team
@SonyAI_global
! In just one year, they took their breakthrough AI agent
#GTSophy
from a research project to a commercial product. Learn more below ⬇️
Training sensorimotor Mobile Manipulation policies with RL is hard: many objectives (non-collision, reach…) and high DoF (arm, base, head…)
CausalMoMa (, RSS23) makes it possible by leveraging causal dependencies between action dims and rewards.
Congratulations to the students of the Learning Agents Research Group and colleagues at Sony AI for working with me on 9 papers to be presented at ICRA this week. Here are a few of them: (1/6)
Tomorrow,
@PeterStone_TX
, Executive Director of Sony AI America, will be giving a talk on "Outracing Champion 'Gran Turismo' Drivers with Deep Reinforcement Learning" at
@UTAustin
's Forum for Artificial Intelligence. Learn how to tune in below.
#GTSophy
It's been an honor to serve as chair of the AI 100 Standing Committee for the past 4 years! During my term, we released the 2021 report, recruited some great new members, and are completing an essay competition for your researchers in AI.
Upper Bound presents
@PeterStone_TX
as a Keynote Speaker! He will talk about lessons on practical reinforcement learning from his time as the founder and director of the Learning Agents Research Group (LARG) at The University of Texas in Austin.
#UpperBound2023
#ABTech
#YEGTech
The University of Texas at Austin will start a large-scale, low-cost online Master of Science degree program in artificial intelligence, the institution announced. The new program could help swiftly expand the AI work force in the U.S.
Brad Knox is presenting two papers at AAAI today.
11:15am journal track: Reward (Mis)design for Autonomous Vehicles:
Evening poster: Learning Optimal Advantage from Preferences and Mistaking it for Reward:
@utlarg
@UTCompSci
The Ripon society published my editorial on the state of AI. I didn't choose the title, but otherwise, I like how it turned out. Comments more than welcome.
Get up to speed on AI: what it can and can’t do & its societal implications. Take the new 1 credit-hour course “The Essentials of AI for Life and Society.” Open this fall to all
@UTAustin
students, faculty & staff.
@UTCompSci
@UTGoodSystems
@undergrad_UT
New Article: "Agent-Based Markov Modeling for Improved COVID-19 Mitigation Policies" by Capobianco, Kompella, Ault, Sharon, Jong, Fox, Meyers, Wurman and Stone
Last fall, along with Joydeep Biswas and Don Fussell, I created a 1-credit-hour course on "The Essentials of AI for Life and Society". The lecture videos are now all online:
Value Function Decomposition for Iterative Design of Reinforcement Learning Agents ()
James MacGlashan, Evan Archer, Alisa Devlic, Takuma Seno, Craig Sherstan, Peter Wurman, Peter Stone (
@PeterStone_TX
),
@SonyAI_global
Tu, Nov 29, 16:30 -Poster Session2
Check out our work on intrinsic motivation using adversarial distribution matching at NeurIPS. It is a core part of
@IshanDurugkar
‘s dissertation research.
My favorite conference is a small one: The Multi-disciplinary Conference on Reinforcement Learning and Decision Making. It works best if only those with a genuine interest in crossing disciplines attend.
This sarcastic robot gives a tour of our building while interacting naturally with the user. A year or two ago this wouldn’t have been possible by anyone. Now it’s a (very successful!) undergraduate summer project!
When a robot does task and motion planning (TAMP), the symbolic state space (for task planning) ought to depend on the locations of objects at the motion planning level. This IROS paper addresses how to tailor the symbolic state space to the situation at hand.
Join
@PeterStone_TX
, Executive Director of
#SonyAI
America in a demonstration program at
#AAAI2021
, "Demonstration of the EMPATHIC Framework for Task Learning from Implicit Human Feedback." Details on the research can be found here:
Challenge Shield Round 2
WisTex United 6:0 Naova
WisTex claimed its second victory in two matches with a resounding performance via Naova. Despite the result, the well-organized Naova defense put up a good fight, which should serve them well later in the tournament.
RLDM is a very fun conference to attend - and is planned to be in person. Submit an extended abstract to present a poster. It’s non-archival, so work recently published or under review is fine - as long as it’s accessible to a broad audience.
Honored+awed to take over from the amazing
@PeterStone_TX
as chair of the
#AI100
standing committee. Thanks to
@erichorvitz
&Mary Horvitz for their vision+funding for this 100yr study and the all-star cast of committee members for their time and expertise!
@EugeneVinitsky
Regarding research notebook, based on a research cent suggestion from a colleague, my group has been using a standard format of Goals, Accomplishment, Objectives, and Issues (GAOI). If interested, email me and I'll send the instructions I sent to my students. It's working well!
Navigation has been studied by roboticists for decades. Do you ever wonder why and how learning can help mobile robots navigate better? Read our Autonomous Robots article "Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey" to find out more!
@ylecun
Good seeing you at NeurIPS! But I don't think the field should abandon any of these directions. AI and ML need different people exploring different approaches - for the same reason it was good not to abandon NNs when many people thought they were a dead end.
Fascinating meeting of the Standing Committee meeting of the Hundred Study of AI today.
Our new chair,
@conitzer
and our outgoing chair,
@PeterStone_TX
urged us to take the long-term view of AI’s impact on society.
#AI100
🌟Excited to present our paper () at
#NeurIPS2023
next week! We introduce a novel RL framework, Residual Q-learning, to customize an imitative policy towards additional task requirements. Join us for the poster session on 13 Dec, 10:45 a.m. CST! 🧵
What a touching event. Thanks
@YesItsMeElad
for organizing,
@r_mirsky
for posting, and so many current and former students for being there and for your moving tributes and memories!
@scottniekum
This is an important issue that will be fought by the university and in the legislature. The best chance of winning is by changing the electorate, which requires people coming to Texas, not leaving! TX Robotics and CS hiring too. (Glad you’re happy, but we miss you! :-) ).
Calling for essays from early career AI researchers. The winner will be invited to participate in the next
#AI100
study panel. (Yes, it's permissible to use LLMs as a tool, as long as it's acknowledged and you're prepared to stand behind your entry. See the rules for details.)
How will AI affect work, life, and society in the future? The
#AI100
publishes a report every five years seeking to answer this question. Now add your voice to the conversation in this newly launched essay competition:
Imagine bringing a personal robot home in the future. How fast can you teach it to do new tasks? We introduce ORION — now a robot can imitate manipulation from a single video demonstration!
Inspiring and impressive visit to see what
@Tesla
is doing in Austin. Peter Stone and I are excited to work with
@omead_a
and Tesla to build out what will clearly be an incredible partnership with
@UTAustin
. This is going to be huge, on so many levels! 🤘
Check out
@IshanDurugkar
’s work (with
@mauriciogtec
,
@scottniekum
and me) on learning an intrinsic reward via distribution matching to an idealized target policy state visitation distribution.
Excited to talk about Adversarial Intrinsic Motivation (AIM), accepted to NeurIPS 2021. It learns an intrinsic reward signal that guides the agent's exploration to find the goal in goal-conditioned reinforcement learning. See an example of the system in action below ... 1/🧵
In response to recent news that a Google engineer claimed AI chatbot LaMDA had become sentient,
@PeterStone_TX
spoke with
@fox7austin
about
#AI
capabilities:
@EugeneVinitsky
Also, the number of hours really isn't the point. Work when you're most productive, and be efficient. Don't feel guilty about taking time off during the day when it's nice out. I learned early on that I could often get more done in 4 hours if I spent the first hour exercising.
RoboCup 2021 ended up being a great success! More than 2,000 participants from around the world participated in a set of "remote" robotics competitions and symposium!
@robocup_org
1/2
🏁 COMING SOON 🏁
On 2/9, our team is excited to launch our Gaming Flagship and announce a breakthrough project in
#AI
created through a collab with Polyphony Digital Inc. (PDI) and
@Sony
Interactive Entertainment (SIE). Enjoy this sneak peek.
@EugeneVinitsky
All sage advice - thanks for sharing! The main thing I see missing is self-care. It's a marathon, not a sprint. Maintain hobbies, get exercise, engage with your (non-university) community, make time for your significant other (or invest in finding someone).
In response to a bunch of requests, the reading list for the course "Essentials of AI for Life and Society" is now online (along with the lecture videos):
We catalog the real-world successes of deep RL across various robotic competencies, analyze the underlying trends with a new taxonomy of problem definitions & solution methods, and highlight critical challenges and key avenues of future work for roboticists and RL researchers.
Could Playstations pave the way for new self-driving tech in cars?
Peter Stone
@PeterStone_TX
, executive director of Sony AI America and director of Texas
#Robotics
, joins today's
@TexasStandard
broadcast to talk more about this project!
Listen here:
@ylecun
Regarding MPC vs. RL, it sounds like you're advocating for model-based RL over model-free. I tend to agree, especially in robotics. But model-free RL outperforms MPC (so far) in simulated high-speed racing, which seems perfectly suited for MPC:
@UTCompSci
is introducing "CREATE," a graduate program prioritizing ethical artificial intelligence in robotics. Students will learn how to ethically design and develop service robots that will contribute to the future. Read more here:
@mezaoptimizer
Old time AI people like me have always been working on AGI. We just called it AI. To me AGI is just a marketing term. We old timers think we know how hard the problem is, and we are lolling as the younger generation discovers this
@duke_zzwang
and I will be presenting ELDEN @ 14 Dec (Thurs) 10:45 - 12:45
#1423
at
#NeurIPS2023
. Come by our poster to chat about unsupervised Reinforcement Learning and Causality!
Want the dynamics model and model-based RL to generalize to unseen states? Causality is one of the keys!
Our CDL is a new dynamics model
- discovers causal dependencies from data
- derives a versatile task-independent state abstraction
ICML'22 paper:
1/n
@natolambert
@lvwerra
@lcastricato
@Dahoas1
@carperai
Thanks for a very clear post on RLHF for LLMs. Note that RLHF in settings other than LLMs has been around for much longer than indicated, though. See, e.g.,
@bradknox
's TAMER work from 2008 and James MacGlashan's COACH. And then there was Deep TAMER for RLHF in deep learning..
Bo (
@cranialxix
) and I will be presenting LIBERO at
#NeurIPS2023
. LIBERO is a robot learning benchmark for both lifelong and multitask learning research. We have many exciting follow-ups based on LIBERO. Look forward to chatting about them more at the conference!
@UTAustin
has officially established a new online learning master’s program in artificial intelligence (AI), with the potential to bring thousands of new students into the field. To read more, visit
I recently spoke about the state of AI and how machine learning techniques are closing the gap between autonomous systems and humans in competitions at
@EON_events
– The Applied Artificial Intelligence Event. Check out more about
#EON2022
at .
Fascinating talk by
@PeterStone_TX
at the Workshop on Autonomous Agents for Social Good (AASG 21)
@aamas2021
. Thank you Peter! Looking forward to another exciting talk by Tuomas Sandholm later in the day!
We are thrilled to announce LIBERO, a lifelong robot learning benchmark to study knowledge transfer in decision-making and robotics at scale! 🤖 LIBERO paves the way for prototyping algorithms that allow robots to continually learn! More explanations and links are in the 🧵
This week, we're excited to be a sponsor at the 5th Multi-disciplinary Conference on Reinforcement Learning and Decision Making where
@PeterStone_TX
, Sony AI America Executive Director, is serving as a program chair. Registration is still open.
#RLDM2022
@conitzer
I won’t speak for the whole RoboCup community here, but personally I think their paper connects very well to past work and acknowledges the limitations compared to RoboCup teams including that they use off board global sensing. Overall, I think it’s fun, we’ll-executed work.
I am excited to announce that I started a new position today as a Senior Research Scientist at Sony AI! I will be working to build out their responsible AI practice. (1/4)
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach ()
Bo Liu(
@cranialxix
), Mao Ye, Stephen Wright, Peter Stone(
@PeterStone_TX
), Qiang Liu
We, Nov 30, 16:30 -- Poster Session 4
Tasks at home can involve many objects with complex interdependencies, e.g., the butter needs to be melted on the stove before it can be mixed and fried with meat in a pot.
How can RL efficiently explore such tasks with sparse rewards?
Introducing our
#NeurIPS23
paper ELDEN: 🧵