Ruiyi Wang 王睿仪 @RuiyiWang153 Twitter profile

Pinned Tweet

Ruiyi Wang 王睿仪

2 months

(1/n) Can large language models simulate patients with mental health conditions? We introduce Patient-Ψ🤖, where we integrate cognitive modeling with LLMs to simulate patients for training mental health professionals. 📎Paper link:

5

57

256

Last Seen Profiles

@JoshCornau

@TupaEmondi

@DailyTheo_

@Gladwin_Ngoepe

@lastnumber9nine

@MirrorPoison

@hantini1239

@JessieE9143

@PiKov71808

@bokeplokalmalam

@Mr_Baylor

@BinorRaja

@BrknSokullu

@RecoviraProject

@SadatyAmyr

@SheJobs

@tatsuya_ito3

@Cumulocity

@Toku_Custom

@cables23

@Yaraiaia

@mariaantonia_rs

@rubeneh_5

@MoulinMusicalJP

@thekiller_0303

@chicobonbon_

@bokeplokalmalam

@nomi__owo

@harrisonoladel1

@Blaze_R34

@DrT3T

@bokeplokalmalam

@Bacchus

@omiyameibi860

@Darkgreimon

@neuronexustech

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Happy PI🥧 day! Can language agents🤖 learn social skills through imitation and interaction? We are excited to introduce SOTOPIA-π🥧 , an interactive learning method for training language agents to navigate real-world social scenarios while role-playing!

2

19

67

Ruiyi Wang 王睿仪

@RuiyiWang153

26 days

Visit our poster about training social agents 🤖 at 10:30am today, at Poster Session 4! I'm happy to talk about social intelligence, training language agents, and LLM applications to mental healthcare. #ACL2024

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Happy PI🥧 day! Can language agents🤖 learn social skills through imitation and interaction? We are excited to introduce SOTOPIA-π🥧 , an interactive learning method for training language agents to navigate real-world social scenarios while role-playing!

2

19

67

3

11

56

Ruiyi Wang 王睿仪

@RuiyiWang153

29 days

Arrived in Thailand 🇹🇭 for #ACL2024 . I’ll present SOTOPIA-PI at in-person poster session 4 (August 13, 10:30am)! Stay tuned

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Happy PI🥧 day! Can language agents🤖 learn social skills through imitation and interaction? We are excited to introduce SOTOPIA-π🥧 , an interactive learning method for training language agents to navigate real-world social scenarios while role-playing!

2

19

67

1

6

56

Ruiyi Wang 王睿仪

@RuiyiWang153

21 days

Sharing one of my best memories from #ACL2024 🐘🇹🇭 Spending time with (Baby) Capybaras in a cafe ☕️ ! These little ones are absolutely adorable. Grateful to the organizers and everyone who shared their amazing research! And I’m gonna miss this beautiful city - Bangkok!

3

0

46

Ruiyi Wang 王睿仪

@RuiyiWang153

4 months

Excited to share that SOTOPIA-π was accepted to #ACL2024 main conference 🎉! Check our work on training socially intelligent language agents:

SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents

Humans learn social skills through both imitation and social interaction. This social learning process is largely understudied by existing research on building language agents. Motivated by this...

arxiv.org

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Happy PI🥧 day! Can language agents🤖 learn social skills through imitation and interaction? We are excited to introduce SOTOPIA-π🥧 , an interactive learning method for training language agents to navigate real-world social scenarios while role-playing!

2

19

67

1

11

39

Ruiyi Wang 王睿仪

@RuiyiWang153

5 months

I’ll be presenting SOTOPIA-π🥧 at the poster session at #MSLD2024

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Happy PI🥧 day! Can language agents🤖 learn social skills through imitation and interaction? We are excited to introduce SOTOPIA-π🥧 , an interactive learning method for training language agents to navigate real-world social scenarios while role-playing!

2

19

67

2

15

Ruiyi Wang 王睿仪

@RuiyiWang153

1 year

Having a wonderful time with everyone at CHI @nikola_banovic @im__jane @HossainTahera !!!

Computational HCI Lab

@CompHCILab

1 year

CompHCI lab at #CHI2023 in one place taking a break after @im__jane presented her paper. Left to right: @RuiyiWang153 , @im__jane , @nikola_banovic , and @HossainTahera .

0

4

37

2

13

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

It also uncovers an interesting challenge! 🤔LLM-based eval tends to overestimate the abilities of language agents specifically trained for social interaction. Time to think about more advanced evaluation methods to assess social intelligence! ⚖️

1

9

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(8/n) Huge thanks🙌 to our amazing collaborators on this project! @steph_milani , @imjamiechiu , Jiayin Zhi, Shaun Eack, Travis Labrum, Samuel Murphy, @viscidula , Kate Hardy, @hongshenus , @fangf07 , @ZhiyuChen4

0

7

Ruiyi Wang 王睿仪

@RuiyiWang153

11 months

Ever used an LLM to synthesize data & wondered about its hallucinations? Or thought about giving your AI an authorship credit for co-writing a paper? Join us in uncovering the ethical territories of LLMs & shape a better digital tomorrow! 🤖 ✍️Survey:

0

3

7

Ruiyi Wang 王睿仪

@RuiyiWang153

5 months

I’ll give the talk today at 9:45! #MSLD2024

Hao Zhu 朱昊

@_Hao_Zhu

5 months

If you are at #MSLD2024 /UMich, visit our poster No. 30 today at 11:30 and talk tomorrow morning at 9:45. Both presented by awesome @RuiyiWang153 . She will be introducing our newest method for training socially intelligent agents. Go talk to her about anything related to LLMs,

0

2

20

0

1

7

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(7/n) Auto-eval⚙️ reveals: LLM evaluators (GPT-4, Llama 3 70B) underestimate Patient-Ψ🤖’s fidelity compared to GPT-4 baseline. However, GPT-4's accuracy evaluation aligns closely with expert eval. LLMs excel at understanding cognitive models but fall short in assessing realism

1

0

7

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

SOTOPIA-π🥧 combines behavior cloning (BC) and self-reinforcement (SR) training on filtered social interaction data, all rated by GPT-4. The result? 💥A 7B LLM (Mistral-7B-Instruct) that reaches the social goal completion ability of a GPT-4 agent under automatic evaluation!🎉

1

6

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

We couldn’t have done this without our amazing collaborators🙏! 🌟Huge thanks to @haofeiyu44 , @SharonWXZhang , @qi_zhengyang , @MaartenSap , @gneubig , @ybisk , and @_Hao_Zhu for their contributions to this project!

1

0

5

Ruiyi Wang 王睿仪

@RuiyiWang153

21 days

A huge thanks to @isadorcw and @sashrika_ who spiced up the journey! 🔥

0

1

5

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(4/n) User study shows: Compared with the GPT-4 baseline, Patient-Ψ🤖offers *more realistic* patient simulations in fidelity dimensions of maladaptive cognitions, emotional states, and conversational styles. Patient-Ψ-Trainer🛠️is *more effective* for training practitioners.

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(5/n) User study also shows: Experts rated Patient-Ψ🤖 as *very accurate* on average in reflecting the reference cognitive models used to create the simulated patient.

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Surprising finding! 💥 SOTOPIA-π🥧 not only improves the agent’s social intelligence, but also enhances safety & reduces toxicity🛡️. SOTOPIA-π🥧 has the potential to create more socially aligned language agents😊.

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(3/n) Key components of building Patient-Ψ🤖: 1️⃣Using cognitive conceptualization diagrams in CBT as cognitive models 2️⃣Creating the first dataset of cognitive models grounded in CBT 3️⃣Integrating six conversational styles 4️⃣Real-time feedback for trainees

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

🔍How does SOTOPIA-π🥧 work? 1⃣Generate diverse social tasks by sampling keywords from datasets and prompting GPT-4 2⃣Collect training data through interactions between GPT-4 (BC) and the agent being trained (SR) 3⃣Update the agent’s policy using GPT-4 rated positive examples

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(2/n) We propose Patient-Ψ-Trainer🛠️, an interactive framework for practicing CBT skills on Patient-Ψ🤖. A user study of 33 mental health experts and trainees shows Patient-Ψ🤖 has high fidelity to real patients and Patient-Ψ-Trainer🛠️ is highly effective for training.

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

2 months

(6/n) Additionally, for Patient-Ψ-Trainer🛠️: 1️⃣Experts think it can improve trainees’ confidence over the baseline and traditional methods. 2️⃣Experts find value in the real-time feedback 3️⃣Expert prefer integrated conversational styles.

1

0

4

Ruiyi Wang 王睿仪

@RuiyiWang153

11 months

@hongshenus @shivanikapania In case the tinyurl server is down, please use this link instead:

0

1

3

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Want to learn more about SOTOPIA-π🥧? Check our website and the full paper on arxiv:

SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents

Humans learn social skills through both imitation and social interaction. This social learning process is largely understudied by existing research on building language agents. Motivated by this...

arxiv.org

0

3

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

Moreover, SOTOPIA-π🥧 maintains the agent’s general question-answering ability, as demonstrated by its performance on MMLU tasks. 🧠 This shows that SOTOPIA-π🥧 training doesn’t compromise the model’s overall capabilities. ✅

1

0

3

Ruiyi Wang 王睿仪

@RuiyiWang153

6 months

We evaluate SOTOPIA-π🥧 by simulating interactions between our agent & a GPT-3.5 partner, then assess using GPT-4 & human ratings📊. The evaluation is based on the SOTOPIA framework (explore more on ).

1

0

2

Ruiyi Wang 王睿仪

@RuiyiWang153

26 days

It’s happening now! At section 50

0

1

Ruiyi Wang 王睿仪

@RuiyiWang153

22 days

@isadorcw 🔥🔥🔥

0

1