Yongyuan Liang @cheryyun_l Twitter profile | Pikagi

Pikagi

Yongyuan Liang

@cheryyun_l

881

Followers

175

Following

16

Media

54

Statuses

cs phd @umdcs #RL #TrustworthyAI prev. @Uber AI/ @MSFTResearch

Seattle

https://t.co/VsKZcokzoF

Joined December 2020

Don't wanna be here? Send us removal request.

Pinned Tweet

@cheryyun_l

Yongyuan Liang

10 days

Can generative models synthesize policy networks for agents like text-to-image generation? Yes! Just let one agent's trajectory = a prompt😋 We introduce Make-An-Agent, a policy generator on neural network parameter space. Everything open-sourced👇🏻

2

25

107

Last Seen Profiles

@RhinozzCode

@locopaphoto

@hankjw

@Mr_Humble_Ch

@SolaryTV

@HemlockHouse

@GrampTorino

@CrossfaithJapan

@stw_pdg

@TAEYEONTHAI

@andygazio

@BScotchShenani

@TheDonRebel

@BoDYGuarD_Adana

@tnsfootball1

@aTWZcX7r1nWw3Bk

@Scaramucci

@SpongeBobInAJam

@udacity

@tjleona123

@DierksBentley

@mywelshpool

@namjoonah_ah

@forumplus

@namjoonah_ah

@steve_sleeping

@JohnMac___

@iamgoodmood

@madhal_alteeb

@8291anilaM

@lidocaine___

@MT_WBB

@flutistbichen

@namjoonah_ah

@sukabgtsmboba

@MSV_GG

@cheryyun_l

Yongyuan Liang

3 months

We have 2/2 paper accepted by ICML 2024🤲🏻🥳, including a very effective and efficient RL baseline. #ICML2024 Will introduce more this week. Stay tuned!🐈

1

3

120

@cheryyun_l

Yongyuan Liang

2 months

🐈‍⬛ Ever wonder how much different primitive behaviors impact policy learning? Introducing our #ICML2024 paper, "ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization", — boosting SAC by over 2x with only 41 extra training minutes! 🔗👇🏻:

Tweet media one

3

19

113

@cheryyun_l

Yongyuan Liang

6 months

ICML received 9620 submissions this year😱🤯The ML and AI community is expanding at an unprecedented rate.

6

3

105

@cheryyun_l

Yongyuan Liang

2 months

I am very curious whether #NeurIPS2024 has confirmed the use of conflict filter this year. My labmate and I both saw the collaborator's paper in the bidding system. This seems to be a very serious problem. @NeurIPSConf

9

4

86

@cheryyun_l

Yongyuan Liang

2 months

ACE has been selected as #ICML2024 oral presentation! See you in Vienna! 🤩

@cheryyun_l

Yongyuan Liang

2 months

🐈‍⬛ Ever wonder how much different primitive behaviors impact policy learning? Introducing our #ICML2024 paper, "ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization", — boosting SAC by over 2x with only 41 extra training minutes! 🔗👇🏻:

Tweet media one

3

19

113

1

11

78

@cheryyun_l

Yongyuan Liang

2 months

Only 16 hours left, yet #NeurIPS2024 IDs have already topped 13,000. The odds of surpassing 20,000 IDs are very likely😮As a NeurIPS reviewer for the fourth year, feeling the pressure already!

2

3

70

@cheryyun_l

Yongyuan Liang

6 months

Feeling thrilled as 3/4 of my submissions were accepted by #ICLR2024 , including 2 spotlights (top 5%)! What a miraculous summer! Huge thanks to my advisors for their tremendous guidance. Stay tuned as we will soon release and introduce our works. 🌟 #AI #ML

1

0

62

@cheryyun_l

Yongyuan Liang

3 months

Our spotlight✨ paper DrM will be presented by my awesome collaborator @Kevin_GuoweiXu at #ICLR2024 ! If you're interested in visual RL, please take a look! 👀I'll be available online for discussions💻. Session: Tue 7 May 4:30 - 6:30 p.m. CEST (10:30 a.m. EST), Halle B #202

Tweet media one

1

6

39

@cheryyun_l

Yongyuan Liang

5 months

On my journey in ML research, I've been fortunate to be advised by a female role model @furongh who has greatly influenced me. Many of my collaborators are talented and intelligent women, collectively driving our community toward improvement. Happy #InternationalWomensDay 🌷!!!

4

4

36

@cheryyun_l

Yongyuan Liang

3 months

Our poster "GRAD: Game-Theoretic Robust RL" will be presented at #ICLR2024 ! If you have interests in robust #RL #TrustworthyAI , welcome to check it out! I will be well-prepared for online discussions! 🐣 Session: Fri 10 May 4:30 - 6:30 p.m. CEST (10:30 a.m. EST), Halle B #144

Tweet media one

1

4

36

@cheryyun_l

Yongyuan Liang

3 months

Premier-TACO has been accepted by ICML 2024🎉

@furongh

Furong Huang

5 months

Imagine an AI that not only learns quickly but adapts to your daily tasks autonomously. Meet Premier-TACO 🌮, a multitask feature representation learning designed for few-shot adaptation in sequential decision-making. Project 🔗: #FoundationModel4RL A 🧵

2

12

98

0

2

34

@cheryyun_l

Yongyuan Liang

5 months

Thanks for sharing our work! We will release details soon🤩

@_akhaliq

AK

6 months

Premier-TACO: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss paper page: present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential

Tweet media one

0

18

66

0

1

14

@cheryyun_l

Yongyuan Liang

5 months

Premier-TACO offers a highly efficient representation with excellent generalization capacity for unseen embodiments/views, also suitable for finetuning pretrained models like R3M. Unleash the power of our "easy-to-pick-up" 🌮 by trying it out in various scenarios! ✨ #AI #ML

@furongh

Furong Huang

5 months

Imagine an AI that not only learns quickly but adapts to your daily tasks autonomously. Meet Premier-TACO 🌮, a multitask feature representation learning designed for few-shot adaptation in sequential decision-making. Project 🔗: #FoundationModel4RL A 🧵

2

12

98

0

4

14

@cheryyun_l

Yongyuan Liang

7 days

Our #ICML2024 Oral💐 paper ACE will be presented at Oral 3F Causality, Lehar 1-4, July 24th 11:15 a.m. - 11:30 a.m. CEST. And our poster session is at Hall C 4-9 #1212 , July 24th 11:30 a.m. - 1:00 p.m. CEST. If you have interests in efficient RL and causality, check it out🥳!

Tweet media one

13

10

69

@cheryyun_l

Yongyuan Liang

6 months

#DrM has been accepted at #ICLR2024 as a spotlight!!! We will release our code soon🥳

@ruijie_zheng12

Ruijie Zheng

@ruijie_zheng12

9 months

🚀 The first model-free visual RL algorithm consistently solving the hardest locomotion and robotic manipulation tasks without any expert demonstration or pretrained knowledge! The key is the “dormant ratio” of the agent’s policy network. 🔗:

3

52

209

0

2

11

@cheryyun_l

Yongyuan Liang

2 months

I also want to share a very fun incident occurred with ACE() in ICML review: one reviewer challenged another's critique of our paper, leading to a heated discussion. Grateful for these lively discussions during rebuttal, shaping our community better.

1

0

10

@cheryyun_l

Yongyuan Liang

5 months

After exploring efficient methods to enhance worst-case robustness (arxiv:2210.05927), PROTECTED extends adaptability and focuses beyond worst-case attacks—an intriguing contribution to advancing adversarial robustness for RL and trustworthy AI. #TrustworthyAI #RL #PROTECTED 🙌🏻🥳

@furongh

Furong Huang

5 months

Ever wondered if we can protect AI from attacks without sacrificing its everyday effectiveness? Our #ICLR2024 spotlight paper unveils how. Dive into the future of resilient, uncompromised AI with us! #RobustAI #PerformanceMatters Project page 🔗: A🧵👇

8

40

432

0

1

9

@cheryyun_l

Yongyuan Liang

5 months

Awesome work based on RL framework!!🤯

@xiaolonw

Xiaolong Wang

5 months

Let’s think about humanoid robots outside carrying the box. How about having the humanoid come out the door, interact with humans, and even dance? Introducing Expressive Whole-Body Control for Humanoid Robots: See how our robot performs rich, diverse,

94

210

1K

0

1

9

@cheryyun_l

Yongyuan Liang

10 days

*On real-world robots* Make-An-Agent can generate robust locomotion policies with trajectories from IsaacGym simulator. Real-world deployment shows favorable handling of sharp turns, obstacle avoidance on the mat, and rapid backward movement. 5/

1

2

8

@cheryyun_l

Yongyuan Liang

9 months

Curious about the secret behind stagnant agents in #VisualRL ? It's high time to revamp your baselines, especially for the most challenging unsolved tasks. Our work unveils an effective and efficient solution for model-free VRL. #AI #RL 🎃

@ruijie_zheng12

Ruijie Zheng

@ruijie_zheng12

9 months

🚀 The first model-free visual RL algorithm consistently solving the hardest locomotion and robotic manipulation tasks without any expert demonstration or pretrained knowledge! The key is the “dormant ratio” of the agent’s policy network. 🔗:

3

52

209

0

0

8

@cheryyun_l

Yongyuan Liang

3 months

@Kevin_GuoweiXu Our post: Project Page: OpenReview: Paper:

Tweet card media

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio...

Visual reinforcement learning (RL) has shown promise in continuous control tasks. Despite its progress, current algorithms are still unsatisfactory in virtually every aspect of the performance such...

@cheryyun_l

Yongyuan Liang

9 months

Curious about the secret behind stagnant agents in #VisualRL ? It's high time to revamp your baselines, especially for the most challenging unsolved tasks. Our work unveils an effective and efficient solution for model-free VRL. #AI #RL 🎃

0

0

8

0

2

7

@cheryyun_l

Yongyuan Liang

10 days

How does Make-An-Agent work? Simple but efficient a) A contrastive behavior embedding to process trajectory data. b) An autoencoder to encode and decode policy network parameters. c) A policy network generator using a conditional latent diffusion model as the backbone. 3/

1

1

7

@cheryyun_l

Yongyuan Liang

10 days

Make-An-Agent doesn't just copy existing behaviors. Leveraging generative models, it discovers diverse and superior policies within the parameter space. 6/

1

1

6

@cheryyun_l

Yongyuan Liang

10 days

@hkz222 @LuccaChiang @furongh @HarryXu12 Our paper on Huggingface Daily Paper: Welcome to give comments to us!!

Tweet card media

Paper page - Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted...

0

2

6

@cheryyun_l

Yongyuan Liang

10 days

*Benchmark visualization* in MetaWorld and Robosuite simulators. 4/

1

1

6

@cheryyun_l

Yongyuan Liang

2 months

6/ That is it! ACE is open-sourced now. Try ACE in your tasks! Paper: Code: Website:

Tweet card media

GitHub - jity16/ACE-Off-Policy-Actor-Critic-with-Causality-Aware-Entropy-Regularization: Official...

Official PyTorch implementation of "ACE:Off-Policy Actor-Critic with Causality-Aware Entropy Regularization" - jity16/ACE-Off-Policy-Actor-Critic-with-Causality-Aware-Entropy-Regularization

1

1

5

@cheryyun_l

Yongyuan Liang

10 days

Trajectory data isn't just for training policies or trajectory models. It can be a powerful prompt for generating optimal policy network parameters for diverse, even unfamiliar tasks, unlocking a new paradigm for decision making. 1/

Tweet media one

1

0

5

@cheryyun_l

Yongyuan Liang

10 days

Check out our project page: Paper: Code: Dataset and models: Lead by @cheryyun_l , joint work with Tingqiang Xu, @hkz222 , @LuccaChiang , @furongh , @HarryXu12

Tweet card media

cheryyunl/Make-An-Agent · Hugging Face

1

1

5

@cheryyun_l

Yongyuan Liang

6 months

Also, big congrats and heartfelt thanks to @furongh @HarryXu12 @McaleerStephen @ben_eysenbach for their very helpful and invaluable support!!! 💐🙏

0

0

5

@cheryyun_l

Yongyuan Liang

10 days

*Effectiveness* Make-An-Agent excels at mastering diverse tasks with few-shot trajectories as conditions of policy synthesis. It also shows impressive generalization ability, generating optimal policies with unseen trajectories for unfamiliar tasks or robots. 2/

Tweet media one

1

0

4

@cheryyun_l

Yongyuan Liang

2 months

1/ ACE achieves state-of-the-art 🏆on 29 continuous control tasks across 7 domains, including complex manipulation and locomotion challenges. Even in sparse reward tasks, ACE shines by discerning implicit causality between actions and sparse rewards.

Tweet media one

Tweet media two

1

0

3

@cheryyun_l

Yongyuan Liang

2 months

The area chair also recognized ACE's potential to convince RL practitioners and causal theory community of the value of causal modeling in RL. This inspires us to apply causal theory to a broader range of RL problems, including RLHF, IRL, and beyond🧐

0

0

4

@cheryyun_l

Yongyuan Liang

2 months

3/ Introducing ACE: an off-policy actor-critic that simply prioritizes exploration with the causal-weighted entropy term. We further strengthen ACE with a novel gradient-dormancy-guided reset mechanism.

Tweet media one

Tweet media two

1

1

2

@cheryyun_l

Yongyuan Liang

2 months

Co-led by @jtinyng1 and @cheryyun_l , advised by @HarryXu12 Big thanks to our awesome collaborators and advisors: Yan Zeng, @roypersian @Kevin_GuoweiXu , Jiawei Guo, @ruijie_zheng12 @furongh and Fuchun Sun

0

0

3

@cheryyun_l

Yongyuan Liang

2 months

5/ **Efficiency**: ACE is impressively efficient which unlocks a more powerful, causality-enhanced SAC, requiring just 0.69 GPU hours more training time on average than standard SAC.

Tweet media one

1

0

3

@cheryyun_l

Yongyuan Liang

3 months

GRAD introduces a game-theoretic approach that treats the temporally-coupled robust RL problem as a partially-observable two-player zero-sum game and finds an approximate equilibrium within the game. OpenReview: Paper:

Tweet card media

Game-Theoretic Robust Reinforcement Learning Handles...

Deploying reinforcement learning (RL) systems requires robustness to uncertainty and model misspecification, yet prior robust RL methods typically only study noise introduced independently across...

0

3

3

@cheryyun_l

Yongyuan Liang

7 days

Our paper: Page: Post: Although I can't make it for ICML, @jtinyng1 will be there, welcome to chat with us!!!

@cheryyun_l

Yongyuan Liang

2 months

🐈‍⬛ Ever wonder how much different primitive behaviors impact policy learning? Introducing our #ICML2024 paper, "ACE: Off-Policy Actor-Critic with Causality-Aware Entropy Regularization", — boosting SAC by over 2x with only 41 extra training minutes! 🔗👇🏻:

Tweet media one

3

19

113

0

0

4

@cheryyun_l

Yongyuan Liang

2 months

2/ ✨Our key insight: modeling the causal relationship between actions and rewards under states, which reveals how the significance of primitive behaviors evolves in the learning process, naturally serving as effective guidance for efficient exploration.

Tweet media one

1

0

2

@cheryyun_l

Yongyuan Liang

2 months

@furongh @NSF @ENERGY Congrats Furong!!!🥳🤲🏻

0

0

2

@cheryyun_l

Yongyuan Liang

2 months

4/ Watch it excel:

1

1

2

@cheryyun_l

Yongyuan Liang

2 months

@aneeshers Yes! Our ICLR paper has detailedly discussed the plasticity problem in visual RL and the gradient-dormancy-guided reset mechanism in ACE is an improved version for state-based RL.

0

0

1

@cheryyun_l

Yongyuan Liang

2 months

@XueFz Yea seems oral decision will appear on openreview.

1

0

1

@cheryyun_l

Yongyuan Liang

2 months

@McaleerStephen Congrats Stephen!!!

0

0

1

@cheryyun_l

Yongyuan Liang

5 months

@fangchenliu_ Very interesting great work!! 🤩

0

0

1

@cheryyun_l

Yongyuan Liang

2 months

@WenhuChen @NeurIPSConf Actually @NeurIPSConf has mentioned that paper for which you have a conflict are not shown😅However

1

0

1