Guanya Shi @GuanyaShi Twitter profile | Pikagi

Pikagi

Guanya Shi

@GuanyaShi

3,897

Followers

1,028

Following

43

Media

248

Statuses

Assistant Professor @CMU_Robotics @SCSatCMU . Lead the Learning and Control for Agile Robotics Lab @LeCARLab . Ph.D. from @Caltech . Postdoc @uwcse . He/Him/His.

Pittsburgh

https://t.co/PBlUvmtbqU

Joined December 2018

Don't wanna be here? Send us removal request.

Pinned Tweet

@GuanyaShi

Guanya Shi

19 days

OmniH2O (Omni Human2HumanOid) aims to provide a universal whole-body control interface for full-size humanoids with dexterous hands. By learning a robust whole-body loco-manipulation tracking policy and carefully designing the control interface (sparse kinematic poses), a single

Tweet media one

@TairanHe99

Tairan He

19 days

Introduce OmniH2O, a learning-based system for whole-body humanoid teleop and autonomy: 🦾Robust loco-mani policy 🦸Universal teleop interface: VR, verbal, RGB 🧠Autonomy via @chatgpt4o or imitation 🔗Release the first whole-body humanoid dataset

18

68

375

2

25

140

Last Seen Profiles

@rosa_progress

@thatsean

@DeLaGhettoReal

@cespinov

@crijti

@AdvSalmanShahid

@sax_trade

@luciious_ai

@hunt_bs

@stwmaniax

@BandarStw

@dotcorenft

@Steelistic

@ASTRAL_BDSM

@Kaiden2kKent

@MariaRasheed55

@horaciomdch

@pengen_stw

@amarchadhapatel

@onlymelissaminx

@Cazsually

@LylaWeisha99457

@julianpeterson1

@dchsswimanddive

@Diego_Cg01

@akramalamri1984

@NOWNodes

@Fiscal_Lagarto

@kurumi_nuts

@otokonoko_hcom2

@_Lounette_

@FreudMuseum

@chinabank

@undo_hubris

@JantaKeeAwaaz

@Dhanesh70689066

@GuanyaShi

Guanya Shi

2 years

I'll be joining @CarnegieMellon as an Assistant Professor in @CMU_Robotics and @SCSatCMU in Fall 2023. Deep thanks to my Ph.D. advisors @yisongyue and Soon-Jo Chung, collaborators, and many friends who have supported me all the time. Looking forward to a new journey at CMU!

59

19

795

@GuanyaShi

Guanya Shi

2 years

Our Neural-Fly paper was published by @SciRobotics ! Neural-Fly enables rapid learning for agile flight in time-variant strong winds, with theoretical guarantees! Paper: @Caltech news: Explainer video:

10

112

653

@GuanyaShi

Guanya Shi

11 months

Today, I received a tragic review with the lowest possible rating for my NeurIPS paper. I was all prepared to enter the ring of scholarly debate and start doubting my research taste, but then I found:

Tweet media one

Tweet media two

51

26

589

@GuanyaShi

Guanya Shi

3 years

I wrote a blog post: "Neural-Control Family: What Deep Learning + Control Enables in the Real World." This post discusses some key principles of our work on learning robotic agility in safety-critical systems (e.g., Neural-Swarm below).

5

82

508

@GuanyaShi

Guanya Shi

2 years

Officially Dr. Shi! Many thanks to the committee (Prof. @yisongyue @AdamWierman Soon-Jo Chung and Joel Burdick)! Before joining @CMU_Robotics as an Assistant Professor in the 2023 fall, I will spend one year at @uwcse as a postdoc with Byron Boots, focusing on 🤖️learning.

Tweet media one

11

5

215

@GuanyaShi

Guanya Shi

6 months

Excited to teach Intro to Robot Learning () at CMU this spring (start from the next week)! Can't stop thinking about "how fast this field moves and how interdisciplinary it is" when preparing slides.

Tweet media one

7

22

211

@GuanyaShi

Guanya Shi

29 days

Diffusion models have shown strong capabilities in generating high-fidelity trajectories. However, standard diffusion processes cannot efficiently adapt to new scenarios beyond demonstrations (e.g., new robots with different dynamics). MBD (Model-Based Diffusion) is a

@ChaoyiPan

Chaoyi Pan

29 days

🚀Introducing Model-Based Diffusion (MBD), a diffusion-based traj optimization method that directly computes the score function using model info. MBD doesn't require data, but can be integrated with data of diverse quality. 🌐 🧵1/6

8

53

274

2

29

202

@GuanyaShi

Guanya Shi

2 years

My second blog post: "Learning-theoretic Perspectives on MPC via Competitive Control." This post discusses Model Predictive Control (MPC) from online learning's perspectives: why is MPC a competitive online learner, and what can we learn from it?

Tweet media one

Tweet media two

3

35

201

@GuanyaShi

Guanya Shi

2 months

Such a cool idea! I was trying to use the Kolmogorov-Arnold representation theorem to prove the expressiveness of the heterogeneous deep sets structure in the Neural-Swarm2 paper (, figure below). I think the Deep Sets paper () and

Tweet media one

@ZimingLiu11

Ziming Liu

2 months

MLPs are so foundational, but are there alternatives? MLPs place activation functions on neurons, but can we instead place (learnable) activation functions on weights? Yes, we KAN! We propose Kolmogorov-Arnold Networks (KAN), which are more accurate and interpretable than MLPs.🧵

125

1K

5K

2

35

185

@GuanyaShi

Guanya Shi

3 months

Such a good tutorial about diffusion models! Wanted to read something before sleeping so started reading it. Couldn't stop and now is 3am🤣

@stanley_h_chan

Stanley H. Chan

@stanley_h_chan

3 months

I wrote a tutorial on diffusion models for undergrad and grad students. I tried my best to give intuitive explanations for complicated equations. Your feedback is much appreciated Thanks to those who suggested various reading materials to me

Tweet media one

53

837

4K

1

10

164

@GuanyaShi

Guanya Shi

9 months

Quadrupedal robots are mastering agile skills like running, standing, and parkouring, yet face the peril of damaging falls. Introducing🛡️Guardians as You Fall (GYF), a safe falling and recovery framework that can actively tumble and recover to stable modes to minimize damage in

4

21

150

@GuanyaShi

Guanya Shi

2 years

Grad school applicants (hope it's not too late): my group at @CMU_Robotics is hiring Ph.D. and master's students in Fall 2023! If you are interested in learning & control & robotics, please consider applying to our graduate programs (, deadline is Dec 12).

4

27

149

@GuanyaShi

Guanya Shi

4 months

How to start the data flywheel for human-like embodied intelligence? We think real-time teleoperating a humanoid🤖 in a whole-body manner will be a solution. The embodiment alignment allows for a seamless integration of human cognitive skills with versatile humanoid capabilities.

@TairanHe99

Tairan He

4 months

🤖 Introducing H2O (Human2HumanOid): - 🧠 An RL-based human-to-humanoid real-time whole-body teleoperation framework - 💃 Scalable retargeting and training using large human motion dataset - 🎥 With just an RGB camera, everyone can teleoperate a full-sized humanoid to perform

21

89

432

5

17

113

@GuanyaShi

Guanya Shi

13 days

For humanoid loco-manipulation tasks involving specific sequential contacts (e.g., the clapping & dancing video below), we found that the contact sequence itself is naturally an ideal representation for: - Task decomposition to reduce the exploration burden; - Simple &

@_wenlixiao

Wenli Xiao

13 days

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal

8

40

191

0

20

115

@GuanyaShi

Guanya Shi

10 months

Day 1 of being an Assistant Professor @CMU_Robotics Please consider following our research group @LeCARLab

Tweet media one

Tweet media two

6

2

109

@GuanyaShi

Guanya Shi

5 months

🚀Excited to introduce CoVO-MPC (Covariance-Optimal MPC)! Thanks to its flexibility and parallelizability, sampling-based MPC has been a practical approach in many domains, especially model-based RL. However, there is no convergence analysis or principled way to tune

2

4

103

@GuanyaShi

Guanya Shi

4 years

Two theory papers about online learning and control accepted by #NeurIPS2020 ! In the first paper, we propose a new class of online optimization with memory and connect that with control; Second paper: we study how valuable predictions are in online control and analyze MPC.

Tweet media one

Tweet media two

3

11

87

@GuanyaShi

Guanya Shi

2 months

We open-sourced everything for the Agile But Safe (ABS) project! Including hardware installation, system setup, simulation training, and real-world deployment. 👉 Code: Led by @TairanHe99 @ChongZitaZhang

Tweet card media

GitHub - LeCAR-Lab/ABS

Contribute to LeCAR-Lab/ABS development by creating an account on GitHub.

@TairanHe99

Tairan He

5 months

How to break locomotion agile-safe tradeoffs?Introduce Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion: - Fully onboard - Agile (>3m/s) - Safe (collision-free guarantee) - Robust & versatile How? RL + model-free reach-avoid value! 👉

10

67

325

0

12

84

@GuanyaShi

Guanya Shi

5 months

Legged robots are mastering agile skills like parkouring, but how to unleash their agility in cluttered environments, where collision avoidance is a must? We introduce ABS, a fully onboard and autonomous control framework that enables agile and collision-free locomotion for

@TairanHe99

Tairan He

5 months

How to break locomotion agile-safe tradeoffs?Introduce Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion: - Fully onboard - Agile (>3m/s) - Safe (collision-free guarantee) - Robust & versatile How? RL + model-free reach-avoid value! 👉

10

67

325

3

10

82

@GuanyaShi

Guanya Shi

1 month

Accepted by RSS'24 with 444 scores : ) So proud of @TairanHe99 (first year PhD @CMU_Robotics ) and @ChongZitaZhang (visiting master student from @ETH )! 👉 Fully open source code:

Tweet card media

GitHub - LeCAR-Lab/ABS

Contribute to LeCAR-Lab/ABS development by creating an account on GitHub.

@TairanHe99

Tairan He

5 months

How to break locomotion agile-safe tradeoffs?Introduce Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion: - Fully onboard - Agile (>3m/s) - Safe (collision-free guarantee) - Robust & versatile How? RL + model-free reach-avoid value! 👉

10

67

325

1

9

78

@GuanyaShi

Guanya Shi

8 months

Trajectory tracking is a standard control task, but how does a drone track arbitrary, potentially infeasible trajectories (e.g., the triangle or star shapes below) with large disturbances? Introducing Deep Adaptive Trajectory Tracking (DATT, @corl_conf '23 oral), an RL and control

Tweet media one

Tweet media two

Tweet media three

0

3

67

@GuanyaShi

Guanya Shi

1 month

Unfortunately, I am not at #ICRA2024 , but students from @LeCARLab will present several papers about learning and control in agile robotics. Check them out!

0

7

67

@GuanyaShi

Guanya Shi

10 days

I am going to present "Unifying Semantic and Physical Intelligence for Generalist Humanoids" at #CVPR2024 (The Computer Vision in the Wild Workshop). 11:30am on Jun 17 at Arch 3B. Will cover: - Interface between semantic and physical intelligence: H2O, OmniH2O - Learning

1

7

67

@GuanyaShi

Guanya Shi

8 months

🛫 to ATL for @corl_conf ! Present 4 papers about learning and control for various agile robots ✈️ 🐩 🚗: 1. Safe Deep Policy Adaptation (Deployable Workshop Mon 10:40) jointly tackles the problems of policy adaptation and safe RL under unseen disturbances.

2

10

55

@GuanyaShi

Guanya Shi

3 years

I'll be presenting two works this week at @NeurIPSConf : (Tue, poster session 1, E2) Analyzing MPC from learning-theoretic views in LTV systems - MPC is a competitive online learner (Thu, session 7, E0) Meta-adaptive control - end-to-end guarantees for multi-task nonlinear control

Tweet media one

Tweet media two

2

6

55

@GuanyaShi

Guanya Shi

4 years

How valuable are predictions in online control? How many predictions are needed to achieve performance with O(1) dynamic regret? How well does MPC perform? We answer these in our new paper Joint with Chenkai Yu, @yisongyue , Soon-Jo Chung and Adam Wierman.

Tweet media one

2

7

53

@GuanyaShi

Guanya Shi

9 months

Two papers accepted by #NeurIPS2023 ! @LeCARLab 1. (Spotlight) Which parameters in the dynamics model are most critical for MBRL, and how to quickly learn those parameters? 2. To learn a representation for multi-task robotics, which tasks are most

Tweet media one

0

5

53

@GuanyaShi

Guanya Shi

1 year

How to optimally explore for MBRL in unknown nonlinear systems? We formally quantify which model parameters are most relevant to learning a good policy and provide a statistically optimal alg matching the lower bound! w/ Andrew Wagenmaker and @kgjamieson

Tweet media one

2

5

49

@GuanyaShi

Guanya Shi

3 months

Thank you for visiting us, @YunzhuLiYZ !

@YunzhuLiYZ

Yunzhu Li

3 months

I had the pleasure of visiting @CMU_Robotics over the past two days to give a VASC seminar talk and a guest lecture. Thanks @GuanyaShi for the amazing host! 🙌 The seminar talk was about our recent work on "Foundation Models for Robotic Manipulation": 🤖

Tweet media one

3

30

183

1

6

41

@GuanyaShi

Guanya Shi

3 months

Mind-blowing dexterous manipulation results, at 200Hz! Cannot wait to see what happens when it is combined with some agile & robust lower-body policy. We will make H2O () better to get closer to this goal : )

Tweet card media

Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation

by Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, Guanya Shi

human2humanoid.com

@Figure_robot

Figure

4 months

With OpenAI, Figure 01 can now have full conversations with people -OpenAI models provide high-level visual and language intelligence -Figure neural networks deliver fast, low-level, dexterous robot actions Everything in this video is a neural network:

1K

5K

17K

2

0

42

@GuanyaShi

Guanya Shi

4 months

Thrilled to co-organize the Agile Robotics workshop. We believe robotic agility is far beyond a low-level control problem - it requires unification and reconciliation of reasoning, perception, planning, and control, especially in the era of large foundation models! Consider

@gabe_mrgl

Gabe Margolis

4 months

Traveling to Japan for ICRA? Consider showcasing your recent or upcoming work at the Agile Robotics workshop! The deadline to contribute an extended abstract is 𝗠𝗮𝗿𝗰𝗵 𝟮𝟵. Then, join on Monday, May 13 for an exciting slate of invited speakers:

2

11

53

1

1

43

@GuanyaShi

Guanya Shi

11 months

Here are my thoughts on improving review quality for conferences like @NeurIPSConf : (1) Don't increase review numbers for each paper to 6. This only increases review burdens. (2) Reduce the review burden for junior reviewers. (3) Consider AC's evaluations/feedback for reviewers

3

1

40

@GuanyaShi

Guanya Shi

3 years

Multi-task control is hard (e.g., a drone flying in different winds at Caltech CAST). Excited to share OMAC, an online multi-task nonlinear control algorithm with non-asymptotic end-to-end guarantees! Key idea: integrate meta-learning with adaptive control

Tweet media one

Tweet media two

2

4

39

@GuanyaShi

Guanya Shi

5 years

#ICRA2019 We will present Neural Lander today @ POD 22, 4-5pm One of the first instances of provably robust deep learning based controllers! We show Lyapunov stability while using a deep NN as part of the controller design. Experiments on real robots too!

Tweet media one

0

9

37

@GuanyaShi

Guanya Shi

2 years

We've released an arXiv version: and training code & data:

Tweet card media

GitHub - aerorobotics/neural-fly: Training scripts, training data, and experimental data for Neural...

Training scripts, training data, and experimental data for Neural Fly - aerorobotics/neural-fly

@GuanyaShi

Guanya Shi

2 years

Our Neural-Fly paper was published by @SciRobotics ! Neural-Fly enables rapid learning for agile flight in time-variant strong winds, with theoretical guarantees! Paper: @Caltech news: Explainer video:

10

112

653

0

6

38

@GuanyaShi

Guanya Shi

1 year

Night cherry blossoms @UW

Tweet media one

Tweet media two

0

0

35

@GuanyaShi

Guanya Shi

4 years

Finally we released the method behind this "magic" (the minimum distance between drones is only 24cm): We introduce Heterogeneous Deep Sets to learn the complex interactions with permutation invariance, and use them to design stable controller and planner

Tweet card media

Neural-Swarm2: Planning and Control of Heterogeneous Multirotor...

We present Neural-Swarm2, a learning-based method for motion planning and control that allows heterogeneous multirotors in a swarm to safely fly in close proximity. Such operation for drones is...

@Caltech

Caltech

4 years

This is not a choreographed "dance" - these robots are flying autonomously and not bumping into one another thanks to a system developed by Caltech engineers.

0

33

152

2

5

34

@GuanyaShi

Guanya Shi

3 months

#SolarEclipse in Pittsburgh!

0

0

33

@GuanyaShi

Guanya Shi

2 years

This photo is selected by the @CaltechGSC Arts Committee and will hang at Browne Cafe! Feel as excited as publishing the first paper

Tweet media one

2

0

31

@GuanyaShi

Guanya Shi

2 years

Joint work with Michael O’Connell, @aeropolarbear , @Azizzadenesheli , @AnimaAnandkumar , @yisongyue , and Soon-Jo Chung. We will publish the code & data soon!

Tweet media one

2

2

26

@GuanyaShi

Guanya Shi

1 month

@antoine_leeman I think it really depends on what "RL" means and what solvers we use for TO. If RL here refers to sim2real + PPO-style algorithms and TO uses first-order or second-order methods with simplified models, I cannot really imagine in which cases TO will dominate. Nevertheless, RL +

4

0

25

@GuanyaShi

Guanya Shi

10 months

Incredible achievements! The best agile robotics paper I've read this year. It once again proves the power and necessity of (1) residual learning and real-to-sim for robot learning and (2) perception-control-in-a-loop. I am very much looking forward to seeing champion-level

@davsca1

Davide Scaramuzza

10 months

We are thrilled to share our groundbreaking paper published today in @Nature : "Champion-Level Drone Racing using Deep Reinforcement Learning." We introduce "Swift," the first autonomous vision-based drone that beat human world champions in several fair head-to-head races! PDF

Tweet media one

41

342

1K

0

1

24

@GuanyaShi

Guanya Shi

3 years

Pull-ups on the top of O’ahu Merry Xmas everyone!

Tweet media one

Tweet media two

Tweet media three

3

0

24

@GuanyaShi

Guanya Shi

4 years

Happy Holidays from our robots to you and your robots! With all the best wishes for a wonderful holiday season. From the Caltech ARCL lab, edited by @lupusorina

Tweet card media

Happy Holidays from the LEONARDO robot by Caltech's Aerospace...

Our robots wish you Happy Holidays!Starring world's first robot slackliner (Leonardo)!Music: Somewhere in my memory - John Williams

www.youtube.com

0

1

20

@GuanyaShi

Guanya Shi

2 years

The key is that we meta-learn a DNN from 6 different wind conditions (12-min data in total), and use adaptive control to fine-tune it in real-time. Neural-Fly significantly reduces the control error of the current SOTA and can generalize to unseen stronger winds and unseen drones

Tweet media one

Tweet media two

1

1

19

@GuanyaShi

Guanya Shi

4 years

I will present these online learning & control papers at #NeurIPS2020 in (1) Poster Session 1 Mon, Dec 7 @ 9–11pm PST GatherTown: RL & planning (Town C1-Spot A3) (2) Session 7 Thu, Dec 10 @ 9–11pm PST Optimization (A1-B0) TBH, I doubted this interaction way but now I love it!

@GuanyaShi

Guanya Shi

4 years

Two theory papers about online learning and control accepted by #NeurIPS2020 ! In the first paper, we propose a new class of online optimization with memory and connect that with control; Second paper: we study how valuable predictions are in online control and analyze MPC.

Tweet media one

Tweet media two

3

11

87

1

4

18

@GuanyaShi

Guanya Shi

7 months

In NOLA for @NeurIPSConf ! Present two papers and one workshop paper. Excited to chat about RL, control, and robotics. DM if you want to meet up : ) 1. Optimal Exploration for Model-based RL in Nonlinear Systems (). Thu 10:45-12:45pm #1507 . TL;DR: Not all

Tweet media one

Tweet media two

Tweet media three

1

1

17

@GuanyaShi

Guanya Shi

6 years

#JetsonMeetUp18 So many DL applications in robotics! My favorite demo is @Skydio self-flying camera, where DL is used in objective tracking, depth estimation and even SLAM! I am interested in understanding high-dim dynamics with DL to help robotics control. @NVIDIAEmbedded

Tweet media one

0

1

17

@GuanyaShi

Guanya Shi

3 years

Papers mentioned in this post are with @yisongyue @AnimaAnandkumar @kazizzad @yuqirose @anqi_liu33 @Yashwanth_Nakka Soon-Jo Chung, Michael O'Connell, Xichen Shi, Wolfgang Hoenig

1

0

16

@GuanyaShi

Guanya Shi

1 year

The Carnegie Bosch Institute is inviting the first cohort of postdoc fellows (two-year, fully-funded) in the field of AI and cybersecurity. Come work w/ me and others at @CarnegieMellon @SCSatCMU . Support letters are needed, so reach out if interested!

Tweet card media

CBI Fellowship Program

Announcement Applications are invited for Carnegie Bosch Postdoctoral Fellows for the Fall Semester of 2024 supporting research that seeks to have a positive impact on society (e.g. sustainability)...

carnegiebosch.cmu.edu

0

2

14

@GuanyaShi

Guanya Shi

2 years

Honestly, the abortion right is one of few things I still feel very confused about after spending 5 years in this country: why is it even debatable?

@kirstenappleton

Kirsten Appleton

@kirstenappleton

2 years

The scene at the Supreme Court this afternoon

12K

94K

711K

1

0

13

@GuanyaShi

Guanya Shi

4 years

Joint work with @BenRiviere2 @yisongyue Soon-Jo Chung and Wolfgang Hoenig. Paper links: . To achieve the cool results as shown in the video, the key is to use permutation invariant networks to learn complex interaction between drones

Tweet card media

GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot...

We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, automated distributed policy generation for multi-robot motion planning. Our approach combines the advantage of centralized...

@Caltech

Caltech

4 years

This is not a choreographed "dance" - these robots are flying autonomously and not bumping into one another thanks to a system developed by Caltech engineers.

0

33

152

1

1

14

@GuanyaShi

Guanya Shi

2 years

Just arrived in Atlanta for #ACC2022 . Looking forward to meeting friends and brainstorming new research ideas! I will present our work on online learning and control with inaccurate predictions:

Tweet card media

Competitive Control with Delayed Imperfect Information

This paper studies the impact of imperfect information in online control with adversarial disturbances. In particular, we consider both delayed state feedback and inexact predictions of future...

0

1

13

@GuanyaShi

Guanya Shi

2 years

Thanks for posting our videos! The original press release by @Caltech : Related papers: Neural-Swarm2 (T-RO): GLAS (RA-L):

Tweet card media

GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot...

We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, automated distributed policy generation for multi-robot motion planning. Our approach combines the advantage of centralized...

@IntEngineering

Interesting Engineering

@IntEngineering

2 years

The machine-learning method developed at Caltech aims to solve problems related to multi-robot coordination. 🎥 Caltech

16

232

810

1

2

12

@GuanyaShi

Guanya Shi

4 months

@xiaolonw Great work! Congratulations!

2

0

10

@GuanyaShi

Guanya Shi

3 months

@DrJimFan This is amazing! Congrats Jim!

1

0

7

@GuanyaShi

Guanya Shi

1 year

CAJun shows great potential of hierarchical RL and optimization-based control: RL for a versatile & adaptive centroid policy and control for robust & reactive tracking Benefits: Efficient (20 mins, one GPU); Performance (40% wider jump than SOTA); Robust (critical for continuous)

@yxyang1995

Yuxiang Yang

1 year

We present CAJun, a hierarchical learning-control framework that achieves continuous, long-distance jumps (up to 70cm) on quadrupedal robots! Paper: Video: Website:

2

14

93

0

0

10

@GuanyaShi

Guanya Shi

4 years

A cat good at reinforcement learning, motion planning and low-level control🐱

1

0

9

@GuanyaShi

Guanya Shi

8 months

Kudos to student presenters: @TairanHe99 @RandyXiao_ @therealjsacks3 @yxyang1995 Kevin Huang and Rwik Rana. DM if you want to meet up. Super excited to brainstorm new research ideas : )

0

1

9

@GuanyaShi

Guanya Shi

2 years

Neural-Fly is the third child in the "Neural-Control" family (see my blog post ), a family of deep-learning-based nonlinear control methods. Unlike two earlier systems (Neural-Lander/Swarm), Neural-Fly learns and adapts in real-time!

1

1

8

@GuanyaShi

Guanya Shi

4 years

In both papers, we use metrics (competitive ratio or dynamic regret) directly compared with the globally optimal policy. Paper links: , . Will write a blog soon! With @LinYiheng , Chenkai Yu, @soonjochung , @yisongyue , and @AdamWierman .

Tweet card media

Online Optimization with Memory and Competitive Control

This paper presents competitive algorithms for a novel class of online optimization problems with memory. We consider a setting where the learner seeks to minimize the sum of a hitting cost and a...

0

0

8

@GuanyaShi

Guanya Shi

4 months

Great work!

@xiaolonw

Xiaolong Wang

4 months

Let’s think about humanoid robots outside carrying the box. How about having the humanoid come out the door, interact with humans, and even dance? Introducing Expressive Whole-Body Control for Humanoid Robots: See how our robot performs rich, diverse,

94

211

1K

0

1

7

@GuanyaShi

Guanya Shi

3 years

OMAC is also compatible with deep representation learning (we provide demo PyTorch codes for an inverted pendulum task)! Joint work with @kazizzad , Soon-Jo Chung, and @yisongyue .

0

0

7

@GuanyaShi

Guanya Shi

2 years

Papers mentioned in this post are w/ @AdamWierman @yisongyue @guannanqu @LinYiheng @Yu_Chenkai Soon-Jo Chung, Steven Low, Yang Hu, Haoyuan Sun, Tongxin Li, and Ruixiao Yang

0

1

7

@GuanyaShi

Guanya Shi

3 years

Paper 1 (spotlight): w/ @LinYiheng @guannanqu @AdamWierman Yang Hu, Haoyuan Sun Paper 2: w/ @kazizzad @yisongyue Michael O'Connell, Soon-Jo Chung

0

0

7

@GuanyaShi

Guanya Shi

13 days

Great work by @zipengfu and the team! As long as we can solve the whole-body control problem, the humanoid is such an exciting generalist physical intelligence platform, because of the human-to-humanoid embodiment alignment.

@zipengfu

Zipeng Fu

13 days

Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!

16

162

758

0

1

13

@GuanyaShi

Guanya Shi

2 years

I am interested in: 1. Safe robotic control with learned agility (aerial robot, locomotion, ground vehicle, swarm) 2. Learning & control theory 3. Offline learning + online adaptation 4. Model-based + model-free 5. More safe and structured RL And many other exciting topics!

1

0

7

@GuanyaShi

Guanya Shi

1 year

I am very impressed by this example, which clearly shows that GPT-4 has common sense grounding. But I am also confused by the definition of "zero-shot" in this report: given the unprecedented scale of data, how do we ensure these examples are not included in the training?

Tweet media one

@_akhaliq

AK

1 year

Sparks of Artificial General Intelligence: Early experiments with GPT-4 abs:

Tweet media one

19

190

894

0

0

7

@GuanyaShi

Guanya Shi

8 months

4. CAJun: Continuous Adaptive Jumping using a Learned Centroidal Controller (poster 12:00-12:45 on Wed). CAJun is a hierarchical learning and control framework that enables legged robots to jump continuously with adaptive distances.

1

0

6

@GuanyaShi

Guanya Shi

2 months

Incredible!

@ziwenzhuang_leo

Ziwen Zhuang

@ziwenzhuang_leo

2 months

Introducing 🤖🏃Humanoid Parkour Learning Using vision and proprioception, our humanoid can jump over hurdles, and platforms, leap over gaps, walk up/down stairs, and much more. 🖥️Check our website at 📺Stay tuned for more videos.

14

93

463

0

0

6

@GuanyaShi

Guanya Shi

3 years

Follow-up: time-lapse of O’ahu sunset

0

0

6

@GuanyaShi

Guanya Shi

2 years

See my homepage () and my research statement () for more details. A Chinese version of this post is in Zhihu: . Thanks and best of luck!

GuanyaShi_Research_Statement.pdf

drive.google.com

1

1

6

@GuanyaShi

Guanya Shi

10 months

@davsca1 @Nature Incredible achievements! Congrats!

0

0

5

@GuanyaShi

Guanya Shi

4 years

If you have probabilistic ML models (e.g., GP) in the nonlinear dynamics with safety constraints, you will have to consider uncertainty propagation and safety violations (formulated as chance constraints). We propose a new framework for robust learning, exploration, and planning.

@Yashwanth_Nakka

Yashwanth Nakka

@Yashwanth_Nakka

4 years

A Control Theory perspective on #motionplanning and control for #SafeExploration to learn the interaction with the environment. #Robotics #DeepLearning #controltheory @Caltech

Tweet media one

Tweet media two

2

18

67

0

0

5

@GuanyaShi

Guanya Shi

8 months

2. Deep Model Predictive Optimization (Deployable Workshop Mon 10:40). DMPO learns the inner-loop optimizer of sampling-based MPC directly via experience, outperforming MPC and end-to-end RL baselines.

Tweet media one

1

1

5

@GuanyaShi

Guanya Shi

8 months

3. DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control (oral at 8:30pm on Wed, poster 12:00-12:45 on Wed). DATT can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances.

Tweet media one

2

0

5

@GuanyaShi

Guanya Shi

4 years

A nice blog about optimality/robustness in control. Takeaways: (1) continuous and discrete systems are inherently different in terms of robustness; (2) In LQR, CE controller enjoys large margins but also has natural fragility; (3) (not in this blog)in LQG, CE is much more fragile

@beenwrekt

Ben Recht

4 years

Optimal Control and Its Natural Robustness.

0

19

112

0

0

4

@GuanyaShi

Guanya Shi

5 years

Agree! We can’t say it is “human-like” since they use a powerful sensor to estimate cube state. BTW, the key assumption in DR is the true dynamics lies in the parameter space in sim, which is hard to verify. We need to quantify uncertainties and robustness.

@GaryMarcus

Gary Marcus

5 years

Since @OpenAI still has not changed misleading blog post about "solving the Rubik's cube", I attach detailed analysis, comparing what they say and imply with what they actually did. IMHO most would not be obvious to nonexperts. Please zoom in to read & judge for yourself.

Tweet media one

39

743

2K

1

0

4

@GuanyaShi

Guanya Shi

3 years

#HappyBirthdayTsinghua

@Tsinghua_Uni

Tsinghua University

3 years

April 25 marks #Tsinghua110 anniversary! Join the global online celebrations by sending us your #HappyBirthdayTsinghua wishes and sharing your #MyTsinghuaStory with everyone! Don’t forget to tag us and use #ConnectingTsinghua ! Let’s all show our love and support for Tsinghua💜!

Tweet media one

23

149

5K

0

0

3

@GuanyaShi

Guanya Shi

4 years

Video: . Joint work with Wolfgang Hoenig, @aeropolarbear , @yisongyue , Soon-Jo Chung

Tweet card media

Neural-Swarm2: Planning and Control of Heterogeneous Multirotor...

We present Neural-Swarm2, a learning-based method for motion planning and provably stable control that allows heterogeneous multirotors in a swarm to safely ...

www.youtube.com

0

0

4

@GuanyaShi

Guanya Shi

3 months

@changyi_lin1 Cool ideas! Congrats!

1

0

4

@GuanyaShi

Guanya Shi

5 months

All slides will be posted on the course website:

Tweet card media

Lectures | Introduction to Robot Learning

Course Schedule

16-831-s24.github.io

0

0

4

@GuanyaShi

Guanya Shi

2 years

@AnimaAnandkumar @CaltechGSC It is Tokugawaen in Nagoya, Japan

0

0

3

@GuanyaShi

Guanya Shi

4 years

@beenwrekt Hi Prof. Recht, thanks for this great blog! Really inspiring. In the last blog, you mentioned in LQR, discrete and continuous systems are inherently different (in terms of robustness). I am wondering if this difference holds or not in LQG. Discrete LQG is more fragile?

1

0

3

@GuanyaShi

Guanya Shi

4 years

Looking forward to the first talk of the @ControlMeetsML virtual seminar series!

@ControlMeetsML

Control Meets Learning Virtual Seminars

@ControlMeetsML

4 years

Our first talk: 09/30: Elad Hazan (Princeton) @HazanPrinceton "The Non-Stochastic Control Problem" Please visit for more information (subscribe to the Google group to receive Zoom link and future announcements).

Tweet media one

2

6

28

0

0

3

@GuanyaShi

Guanya Shi

5 years

Beckman auditorium

Tweet media one

0

0

2

@GuanyaShi

Guanya Shi

5 months

@ZhongyuLi4 Great work! Congratulations!

0

0

3

@GuanyaShi

Guanya Shi

3 years

Group F, Match 3 #EURO2020

Tweet media one

0

0

3

@GuanyaShi

Guanya Shi

3 months

@nmboffi @CarnegieMellon @mldcmu Congrats Nick! Welcome to CMU : )

0

0

2

@GuanyaShi

Guanya Shi

2 years

@WeiGaoLab @SciRobotics @Caltech @CaltechEAS @yuyoufree Congratulations!

1

0

2

@GuanyaShi

Guanya Shi

2 years

@yukez @snasiriany @huihan_liu @UTCompSci @texas_robotics Congratulations Yuke!

0

0

2

@GuanyaShi

Guanya Shi

11 months

@aviral_kumar2 @SCSatCMU @CSDatCMU @mldcmu @svlevine Congratulations and welcome!

0

0

2

@GuanyaShi

Guanya Shi

4 years

@SaberaTalukder @Caltech @yisongyue Welcome!

0

0

2

@GuanyaShi

Guanya Shi

4 years

Our 10mins presentation video for #ICRA2020 :

0

0

2

@GuanyaShi

Guanya Shi

4 years

Found a really nice data-driven COVID-19 prediction model!

Tweet media one

0

0

2

@GuanyaShi

Guanya Shi

4 years

@beenwrekt Hi Prof. Recht, thanks for the great blog! I feel our recent work () might be interesting to you. We explain why MPC is near optimal in LQR even with adversarial noise. MPC only needs O(logT) predictions to reach O(1) dynamic regret.

1

0

2

@GuanyaShi

Guanya Shi

8 months

@Wenxuan_Zhou Congratulations Wenxuan!

0

0

2