Wenhao Yu @wyu_nd Twitter profile | Pikagi

Pikagi

Wenhao Yu

@wyu_nd

3,051

Followers

720

Following

38

Media

257

Statuses

Senior Research Scientist at @TencentGlobal AI Lab in Seattle | Bloomberg PhD Fellow | Ex. @MSFTResearch @TechAtBloomberg and @allen_ai

Seattle

https://t.co/AhkABX07ot

Joined December 2021

Don't wanna be here? Send us removal request.

Pinned Tweet

@wyu_nd

Wenhao Yu

10 months

My 2023 summary: 🎓 PhD graduation 🏆 EMNLP Outstanding Paper 💯 Crossed 1000+ citations 🦙 Met a real Alpaca in Peru 🇵🇪 Tried luring it with 🥕 for a fun photo, but the Alpaca had own thought and DOES NOT follow my instruction at all 🤣 Embracing 2024 with fresh enthusiasm 🚀

Tweet media one

Tweet media two

Tweet media three

Tweet media four

2

8

330

Last Seen Profiles

@jkayrm

@stwmaniax

@bokeplokalmalam

@fylw16880186006

@mcduffie023

@Polirio93Andres

@kathalijne

@stwmaniax

@YA_MAN_03

@ToroSinaloense

@JennyMorri33031

@moralvictory

@bokeplokalmalam

@kingmckire

@CelenaKrys23306

@Otb_a511

@Wgrznwcz_lab

@AK3DEMIX

@ryobbu2__

@TorriJ78044

@itsmrdr1ve

@FCourtnay87799

@dorothy_gomez_

@CadieHobie23675

@rogetandil

@bokeplokalmalam

@novaashwoodx

@William31936311

@StacyRicha13457

@stw_pdg

@ChristainN10161

@CasparChar73561

@Remini_77

@lololle93

@AngelitaKy25083

@cukienaknikmati

@wyu_nd

Wenhao Yu

1 month

When I tried OpenAI O1-preview on complex Chinese math problems, the model still thinks in English. This behavior aligns with our findings in our #ACL24 paper on "Leveraging Pivot Language in Cross-Lingual Problems" We found that answering non-English questions while thinking in

Tweet media one

36

51

481

@wyu_nd

Wenhao Yu

9 months

🚢Introduce WebVoyager -> Building an End-to-End Web Agent with Large Multimodal Models 📌A GPT-4V powered web agent, can complete user instructions end-to-end on real-world websites 📌Given [task instruction, trajectory], we show GPT-4V can be a good web agent task evaluator

Tweet media one

6

25

173

@wyu_nd

Wenhao Yu

2 years

𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗥𝗮𝘁𝗵𝗲𝗿 𝗧𝗵𝗮𝗻 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗲 is now 𝗮𝗰𝗰𝗲𝗽𝘁𝗲𝗱 to #𝗜𝗖𝗟𝗥𝟮𝟬𝟮𝟯 🎉🎉 Without using DPR/Google, it achieved SoTA on multiple open-domain QA and knowledge-intensive benchmarks! Work done @ms_knowledgenlp ! Code and paper:

Tweet media one

4

41

230

@wyu_nd

Wenhao Yu

11 months

📢New paper: Chain-of-Note Retrieval-Augmented LMs are often misled by noisy, irrelevant documents. Adding IR could even hurt performance in some scenarios😅 Chain-of-Note improves +7.5 over standard RALM on NQ when all documents are noisy! ArXiv:

Tweet media one

3

34

141

@wyu_nd

Wenhao Yu

1 year

🎉Personal Update: Successfully defend my PhD and now part of @TencentGlobal AI Lab Seattle. Huge thanks to my advisor @Meng_CS for unwavering support. I'll work on frontier NLP research, focusing on novel tech in LLM, IR & Instruction tuning & free feel reach out for internship

Tweet media one

24

2

204

@wyu_nd

Wenhao Yu

2 years

🎉🎉#𝗘𝗠𝗡𝗟𝗣𝟮𝟬𝟮𝟮 𝗔 𝗨𝗻𝗶𝗳𝗶𝗲𝗱 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 𝘄𝗶𝘁𝗵 𝗘𝗻𝘁𝗶𝘁𝘆 𝗠𝗲𝗺𝗼𝗿𝘆: A close-book model with much better performance than 𝗘𝗮𝗘, e.g. 47.2 EM on TriviaQA, and outperform open-book on ELI5! ArXiv:

Tweet media one

1

39

171

@wyu_nd

Wenhao Yu

1 year

📢 I am actively looking for research interns working with me in summer 2024 at @TencentGlobal AI Lab in Seattle. If you have research backgrounds in IR & RAG, Factuality, Reasoning, Agent and interested in the working with me, feel free to DM me! 😊

Tweet media one

3

21

151

@wyu_nd

Wenhao Yu

2 years

#𝐄𝐌𝐍𝐋𝐏𝟐𝟎𝟐𝟐 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐟𝐨𝐫 𝐂𝐨𝐦𝐦𝐨𝐧𝐬𝐞𝐧𝐬𝐞 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠: 𝐀 𝐔𝐧𝐢𝐟𝐢𝐞𝐝 𝐀𝐩𝐩𝐫𝐨𝐚𝐜𝐡. A simple way that retrieves relevant information from commonsense corpora for reasoning tasks. #NLProc

Tweet media one

2

17

141

@wyu_nd

Wenhao Yu

1 month

💡Introducing DSBench: a challenging benchmark to evaluate LLM systems on real-world data science problems. GPT-4o scores only 28% accuracy, while humans achieve 66%. A clear gap, but an exciting challenge for AI advancement! 🧐 Paper: Project lead by our

Tweet media one

11

28

142

@wyu_nd

Wenhao Yu

2 years

🎉 New preprint! Generate rather than Retrieve: Large Language Models are Strong Context Generators. Our proposed method achieved new SoTA on open-domain QA! (1/5) Arxiv link:

Tweet media one

2

20

135

@wyu_nd

Wenhao Yu

19 days

Understanding multiple text-rich images is a challenging task! Today, we are thrilled to introduce a new vision-language model capable of processing multiple visual documents, charts, and snapshots as input, outperforming SoTA models by a large margin. It’s highly useful for

Tweet media one

2

33

131

@wyu_nd

Wenhao Yu

11 months

📢New paper "Sub-sentence Encoder" (led by @soshsihao ), a contrastively-learned contextual embedding model for fine-grained semantic representation of text. 🏆Outperform SimCSE, GTR, ST5 and other sentence embedding methods by large margin! ArXiv:

Tweet media one

@soshsihao

Sihao Chen

11 months

Text embeddings = one embedding for the entire text sequence. But what if the text is long and says many things? Can encoders produce contextual embedding for an individual piece of meaning in one text sequence? ❗Check out: Sub-Sentence Embeddings 1/6

Tweet media one

11

88

577

1

14

105

@wyu_nd

Wenhao Yu

1 year

🎉 #EMNLP paper: LLM is greatly influenced by the quality of instructions, and manually written instructions for each task is laborious and unstable. We (led by @zhihz0535 ) introduce Auto-Instruct, automatically improve the quality of instructions provided to LLMs.

Tweet media one

2

24

104

@wyu_nd

Wenhao Yu

10 months

🎉𝐃𝐞𝐧𝐬𝐞 𝐗 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥: What Retrieval Granularity Should We Use? Both passage and sentence level index are not optimal for dense retrieval. We introduce a novel retrieval unit, proposition, for dense retrieval. See details in this thread ~

Tweet media one

2

18

102

@wyu_nd

Wenhao Yu

2 years

#EMNLP #NLProc Wanted to share some our new research 𝗱𝗶𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 on 𝗢𝗽𝗲𝗻-𝗱𝗼𝗺𝗮𝗶𝗻 𝗤𝗔😁: 1. Generate-then-Read: using GPT-3 to generate contexts 2. Entity Memory: attend knowledge from memory, no retrieval 3. KG for QA: using Wikidata to better retrieve and read

Tweet media one

1

15

99

@wyu_nd

Wenhao Yu

9 months

We (Tencent AI Seattle Lab) still has one summer internship position, focused on RAG, Web Agent, or Multi-modal research. Please DM me if you are interested and have a relevant background. 😊

8

10

94

@wyu_nd

Wenhao Yu

2 years

🎉🎉EMNLP 2022: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering. With the same retriever and the same set of retrieved passages, GRAPE can outperform the state-of-the-art reader FiD by a large margin. ArXiv:

Tweet media one

2

16

91

@wyu_nd

Wenhao Yu

1 month

📢 Introducing Cognitive Kernel: an open-source agent system towards generalist autopilots. The system can interact with real-world environment, handling user-provided files, access websites (e.g., Amazon), and manage long-term chat history. Our system is fully open-sourced and

1

3

90

@wyu_nd

Wenhao Yu

6 months

📢 Excited to share that we will organize the 3rd workshop on Knowledge-Augmented NLP at ACL 2024. We will have six amazing speakers! We welcome your submissions and invite you to discuss with our speakers and organizers at the workshop. Looking forward to seeing you in Thailand!

Tweet media one

1

17

76

@wyu_nd

Wenhao Yu

1 year

📢 Introducing ReFeed: a novel plug-and-play approach to enhance the factuality of large language models via retrieval feedback! Together with @Meng_CS @zhihz0535 @LiangZhenwen @ai2_aristo Read more:

Tweet media one

1

15

74

@wyu_nd

Wenhao Yu

3 months

📌Many LLM systems allow users upload documents, such as GPT-4, Claude, and Kimi. Have you used any of these systems?🤔 𝐇𝐚𝐯𝐞 𝐲𝐨𝐮 𝐞𝐯𝐞𝐫 𝐰𝐨𝐧𝐝𝐞𝐫𝐞𝐝 𝐰𝐡𝐢𝐜𝐡 𝐬𝐲𝐬𝐭𝐞𝐦 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐬 𝐭𝐡𝐞 𝐛𝐞𝐬𝐭 𝐰𝐡𝐞𝐧 𝐲𝐨𝐮 𝐚𝐬𝐤 𝐚 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧 𝐛𝐚𝐬𝐞𝐝 𝐨𝐧

Tweet media one

5

18

72

@wyu_nd

Wenhao Yu

1 year

📢 Introducing IfQA - the first large-scale open-domain question answering (ODQA) dataset centered around counterfactual reasoning. Together with @Meng_CS @ai2_aristo ! Paper link:

Tweet media one

4

16

65

@wyu_nd

Wenhao Yu

2 years

Thanks @TechAtBloomberg ! It is my great honor to receive the fellowship! Thanks also to my advisor @NDengineering @meng_cs for always giving me the best support!

@TechAtBloomberg

Tech At Bloomberg

@TechAtBloomberg

2 years

Congratulations to @NotreDame + @ND_CSE 's @wyu_nd on his being named one of the 2022-2023 @Bloomberg #DataScience Ph.D. Fellows! Learn more about his research focus and the other new Fellows in our fifth cohort: #AI #ML #NLProc

Tweet media one

0

2

14

6

2

59

@wyu_nd

Wenhao Yu

6 months

📢 New paper: Compared to 𝐌𝐮𝐥𝐭𝐢-𝐦𝐨𝐝𝐚𝐥 𝐂𝐨𝐓, We found 𝐃𝐞𝐬𝐜𝐫𝐢𝐛𝐞 (visual description generation)-then-𝐑𝐞𝐚𝐬𝐨𝐧 (generating 𝐌𝐮𝐥𝐭𝐢-𝐦𝐨𝐝𝐚𝐥 𝐂𝐨𝐓 with the assistance of descriptions) could greatly improve math reasoning on MathVista and MathVerse.

Tweet media one

0

10

58

@wyu_nd

Wenhao Yu

8 months

📢 Fall semester internship at @TencentGlobal AI Lab in Seattle: We are actively looking for research interns working on IR & RAG, Complex Reasoning, Multi-modal and Language Agent. If you are interested in the working with us, feel free to DM me! 📷

2

5

51

@wyu_nd

Wenhao Yu

2 months

🤓 Arriving at #ACL2024 with @hongming110 . Excited to meet old and new friends, and to discuss LLM agents, multi-modal learning, and RAG. Our Tencent AI Lab, with locations in Seattle, Shenzhen, and Beijing, has multiple FTE and intern positions available. If you're looking for

Tweet media one

1

2

51

@wyu_nd

Wenhao Yu

9 months

I deeply appreciate of the implementation of WebVoyager and fantastic video that explains how to utilize LangGraph for its construction, as well as the comprehensive discussion surrounding LangGraph. Our team will provide more detailed information and make our source code

@LangChainAI

LangChain

9 months

⛴️ WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models WebVoyager is a new kind of web-browsing agent, developed by Hongliang He, @wyu_nd , et. al. Powered by large multi-modal models, like GPT-4V, it uses browser screenshots to conduct research, analyze

Tweet media one

7

64

414

0

3

42

@wyu_nd

Wenhao Yu

2 years

Successful conclusion of the first Knowledge-Augmented NLP workshop at #AAAI23 ! With over 50 in-person attendees and 20 virtual participants, it was a huge success and one of the most well-attended events at #AAAI . Check out the blog and photos below!

Tweet card media

Highlights of the first workshop on Knowledge-Augmented Methods for NLP at AAAI 2023 | Notion

🤗 Workshop Website: https://knowledge-nlp.github.io/aaai2023 (accepted papers are available now. Workshop videos and keynote slides will be uploaded soon!)

trite-cover-2ec.notion.site

2

7

41

@wyu_nd

Wenhao Yu

2 years

After 3 years, excited to attend my second #AAAI23 with a lot of friends from Notre Dame @ND_CSE !

Tweet media one

0

1

40

@wyu_nd

Wenhao Yu

2 years

Excited to share our #EMNLP2022 #NLProc paper on improving multi-task learning via a very simple but very effective task prefix tuning method!

@zhangzhuosheng

Zhuosheng Zhang

@zhangzhuosheng

2 years

#EMNLP2022 🧭Task Compass: Scaling Multi-task Pre-training with Task Prefix 🤔When multi-task pre-training in scale, how to explore task relationships? 💡We find that task relationships can be probed by simply adding single-token task prefixes!

Tweet media one

2

4

40

0

5

32

@wyu_nd

Wenhao Yu

2 years

My daily routine: star repos -> join waitlist 😶

Tweet media one

0

5

32

@wyu_nd

Wenhao Yu

2 years

𝐍𝐞𝐰 𝐒𝐮𝐫𝐯𝐞𝐲 𝐩𝐚𝐩𝐞𝐫 in #eacl2023 ! New perspectives to summarize multi-task learning in NLP from task relatedness and training methods! Also nice future work discussion. #NLProc

@zhihz0535

Zhihan Zhang

2 years

Our paper "A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods" has been accepted to #eacl2023 main conference! Collaboration with @wyu_nd , @Meng_CS , @Zhichun5 and Mengxia Yu.

Tweet media one

1

4

24

0

3

27

@wyu_nd

Wenhao Yu

9 months

Thanks @_akhaliq for covering our work! WebVoyager🚢 is a GPT-4V powered web agent that can follow human instructions and complete tasks (e.g. ticket booking, shopping) on various real-world websites (e.g. Google flights, Amazon)! The paper also present a new benchmark dataset

@_akhaliq

AK

9 months

Tencent presents WebVoyager Building an End-to-End Web Agent with Large Multimodal Models paper page: The advancement of large language models (LLMs) leads to a new era marked by the development of autonomous applications in the real world, which drives

Tweet media one

4

58

291

2

2

27

@wyu_nd

Wenhao Yu

2 years

Excited to announce four highly esteemed keynote speakers @amit_p , @boydgraber , @scottyih , Chandan at our upcoming @knowledgenlp #AAAI23 workshop on Feb 13th! Dive into the cutting-edge topics of neuro-symbolic AI, code understanding, retrieval-augmented LM, and advanced QA.

Tweet media one

1

8

26

@wyu_nd

Wenhao Yu

8 months

📣Our 3rd workshop of knowledge augmented NLP will happen in ACL 2024 this year! Submission ddl: May 17, 2024! Looking forward to seeing you in Thailand!

@knowledgenlp

KnowledgeNLP Workshop @ACL 2024 🇹🇭

8 months

🎉Excited to announce the 3rd Workshop on Knowledge-Augmented NLP at ACL 2024 in Thailand! Submission deadline: May 17, 2024. Eager to reconnect with old friends and welcome new faces in the Knowledge NLP community! #ACL2024 #NLProc

Tweet media one

0

1

18

0

0

22

@wyu_nd

Wenhao Yu

2 years

🏆 Our work “Empowering Language Models with Knowledge Graph Reasoning for Question Answering” won the best paper award at #SoCalNLP 2022. Paper link:

Tweet card media

Empowering Language Models with Knowledge Graph Reasoning for...

Answering open-domain questions requires world knowledge about in-context entities. As pre-trained Language Models (LMs) lack the power to store all required knowledge, external knowledge sources,...

@ucsbNLP

UC Santa Barbara NLP Group

2 years

Our @MegagonLabs Best Paper Award winner was "Empowering Language Models with Knowledge Graph Reasoning for Question Answering" by Ziniu Hu et al from UCLA! Paper link: Thank you to award sponsor @MegagonLabs for supporting our event! (4/4)

Tweet media one

0

0

12

1

1

22

@wyu_nd

Wenhao Yu

11 months

PLUG is a novel cross-lingual instruction tuning method which could make LLaMa follow Chinese instructions (and other low resource language) very well! Check out our paper at

Tweet card media

PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning

Instruction tuning has remarkably advanced large language models (LLMs) in understanding and responding to diverse human instructions. Despite the success in high-resource languages, its...

@zhihz0535

Zhihan Zhang

11 months

🤨LLMs struggle to follow instructions in low-resource languages? ⚡️Introducing PLUG: leveraging pivot language in cross-lingual instruction tuning 📈Improved LLaMA-2 by 32% on 4 diverse languages! Check out our new preprint at➡️

Tweet media one

2

16

72

0

2

20

@wyu_nd

Wenhao Yu

9 months

Thanks LangChain AI for covering and implementing Chain-Of-Note app as a LangChain template. Chain-Of-Note improves performance when retrieved information contains noise. Check out our paper at

Tweet card media

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

Retrieval-augmented language models (RALMs) represent a substantial advancement in the capabilities of large language models, notably in reducing factual hallucination by leveraging external...

@LangChainAI

LangChain

11 months

🗒️Chain-of-Note Template Chain-of-Note is a new prompting technique by @wyu_nd et al for RAG applications that helps improve performance when the retrieved information might be noisy. We implemented a Chain-of-Note app as a LangChain template. Given a question, query Wikipedia

Tweet media one

3

46

266

0

5

19

@wyu_nd

Wenhao Yu

11 days

Excited to see MLE-Bench out, and a big thanks to @OpenAI for highlighting our DSBench as concurrent effort. Proud to contribute to advancing AI agents for real-world challenges alongside these initiatives! DSBench:

Tweet media one

@OpenAI

OpenAI

12 days

We’re releasing a new benchmark, MLE-bench, to measure how well AI agents perform at machine learning engineering. The benchmark consists of 75 machine learning engineering-related competitions sourced from Kaggle.

217

552

4K

3

11

74

@wyu_nd

Wenhao Yu

1 year

📢Calling all #NLP enthusiasts! The 2nd Knowledge Augmented Methods for NLP workshop at #KDD2023 is now accepting paper submissions 📝👩‍💻! Deadline: May 23rd. Accepted papers will be non-archival. For more info, check out 👉 #AI #MachineLearning #NLProc

Tweet media one

0

4

18

@wyu_nd

Wenhao Yu

10 months

Thanks @LangChainAI for finding our methods useful and have put it in your templates!

@LangChainAI

LangChain

10 months

🔎Proposition-Based Retrieval This new paper by @tomchen0 introduces a new retrieval method by changing 🎯what is indexed🎯 in the first place This can easily use our 🌲multi-vector retriever🌲, and we've added a template to get started with it easily! 💡How does it work? 👇

Tweet media one

7

104

514

0

2

18

@wyu_nd

Wenhao Yu

2 years

Combing Retrieval AND Generation (in step1) can further improve the model performance, as shown in Figure 3. The choice of retrieval or generation is interesting, and their complementarity is worth exploring. Using retriever or generator only where it helps.

@johnjnay

John Nay

2 years

Right now we do: 1. retrieve docs 2. LLM generate output w/ those But this doesn't fully leverage LLM power for step 1. What if we directly generate contextual docs for a question, instead of retrieving external docs?! Paper Code

Tweet media one

7

17

194

0

2

15

@wyu_nd

Wenhao Yu

1 year

📣 Check out this awesome survey on mathematical reasoning at poster session 2 #ACL2023

@lupantech

Pan Lu

1 year

🧲Please stop by our poster on deep learning for math reasoning at Poster Session 2 @aclmeeting #ACL2023NLP . ❤️Thanks to co-authors for their great contributions: @liangqiu_1994 , @wyu_nd , @wellecks , & @kaiwei_chang . abs: github:

Tweet media one

0

5

34

0

3

14

@wyu_nd

Wenhao Yu

3 months

I will be at #ACL2024 , will be hosting our 3rd workshop on knowledge-augmented methods for NLP, on August 16. We invited 6 keynote speakers, with 30 accepted oral and poster papers, covering diverse topics on RAG, KG, Agent … See details at

@knowledgenlp

KnowledgeNLP Workshop @ACL 2024 🇹🇭

3 months

Thrilled to announce our finalized schedule at #ACL2024 ! We're excited to feature 6 keynote speakers and 30 accepted papers. Join us for an inspiring event!

Tweet media one

1

21

47

0

1

15

@wyu_nd

Wenhao Yu

4 months

🧐We introduce a new method: using reflective thoughts to improve the model's reasoning capability, just as we humans often do when we step back to question our assumptions, make analogies, and explore alternative solutions.

@zhihz0535

Zhihan Zhang

4 months

🧐Previous math augmentation focused on improving single-round QA 🎯We introduce a new method that1⃣augments standard math settings2⃣excels in reflective thinking scenarios! 👉Check our latest preprint at

Tweet media one

1

2

10

0

1

14

@wyu_nd

Wenhao Yu

2 years

Pls consider submitting your work to our Knowledge Augmented NLP workshop at #AAAI2023 ! Looking forward to seeing you at Washington DC next February 🎉

@knowledgenlp

KnowledgeNLP Workshop @ACL 2024 🇹🇭

2 years

Hello World! The first workshop on Knowledge Augmented Methods for NLP at #AAAI2023 is welcoming submissions🙌! Papers due by Nov. 8! Accepted paper will be non-archival! Details are available 👉

Tweet media one

0

7

19

0

2

13

@wyu_nd

Wenhao Yu

9 months

The new paper from our Tencent AI lab identifies 8 valuable insights into the current state of machine translation research in the LLM era, and propose potential avenue for future advances! Check the paper below 😊

@wangly0229

Longyue Wang

9 months

💡 How are Large Language Models reshaping the landscape of Machine Translation? 🎈 🚀 Check out our latest paper to find interesting findings. We comprehensively revisited Six Classic Challenges of MT in the context of LLM. 🎉 👉 Dive in here: . And

Tweet media one

0

20

84

0

0

12

@wyu_nd

Wenhao Yu

14 days

If you’re attending #COLM2024 , feel free to chat with my colleagues! We’re jointly hiring interns for exciting projects on LLM agents, self-evolving systems, multimodal (vision language), and RAG models. 🚀

@hongming110

Hongming Zhang

15 days

Arriving at #COLM2024 . Thrilled to meet old/new friends. Come find me discussing llm agents, ai systems, and all the excited things beyond.😆😆😆

Tweet media one

1

1

26

0

0

12

@wyu_nd

Wenhao Yu

2 years

“Retrieves non-parametric memories only when necessary.” This is a very insightful conclusion by asking “how retrieval is complementary to LLM parametric knowledge.” We showed the same observation in paper but did not give detailed analysis. Learned a lot!

Tweet card media

Generate rather than Retrieve: Large Language Models are Strong...

Knowledge-intensive tasks, such as open-domain question answering (QA), require access to a large amount of world or domain knowledge. A common approach for knowledge-intensive tasks is to employ...

@AkariAsai

Akari Asai

2 years

Can we solely rely on LLMs’ memories (eg replace search w ChatGPT)? Probably not. Is retrieval a silver bullet? Probably not either. Our analysis shows how retrieval is complementary to LLMs’ parametric knowledge [1/N] 📝 💻

Tweet media one

15

95

535

0

2

12

@wyu_nd

Wenhao Yu

2 years

Welcome paper submissions to our workshop at #AAAI2023 . Please help to share it! 😁

@knowledgenlp

KnowledgeNLP Workshop @ACL 2024 🇹🇭

2 years

Call for papers! The first workshop on Knowledge Augmented Methods for NLP ( #NLProc ) at #AAAI2023 is welcoming submissions🙌! Papers due on Nov. 4! Papers will be non-archival, so published papers (e.g. #EMMLP2022 ) can also present at our workshop! Details👉

Tweet media one

0

3

12

0

1

10

@wyu_nd

Wenhao Yu

11 months

If you are at #NeurIPS2023 , feel free to talk with my colleagues for internship opportunities next summer!

@KaixinMa9

Kaixin Ma

11 months

Hello friends at #NeurIPS2023 , our @TencentGlobal AI Lab in Seattle is actively looking for research interns for 2024. If you are interested in topics such as RAG, Reasoning, LLM Agent, and user interfaces, feel free to DM me for a chat!😊

1

1

35

0

1

11

@wyu_nd

Wenhao Yu

2 years

Welcome to our presentation at today 11:30-11:45 at Hall B #EMNLP2022 ! Unified entity memory network have much stronger capabilities than EaE (first released by Google’s @professorwcohen ), which is not restricted to only entity outputs.

Tweet card media

Entities as Experts: Sparse Memory Access with Entity Supervision

We focus on the problem of capturing declarative knowledge about entities in the learned parameters of a language model. We introduce a new model - Entities as Experts (EAE) - that can access...

@wyu_nd

Wenhao Yu

2 years

🎉🎉#𝗘𝗠𝗡𝗟𝗣𝟮𝟬𝟮𝟮 𝗔 𝗨𝗻𝗶𝗳𝗶𝗲𝗱 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 𝘄𝗶𝘁𝗵 𝗘𝗻𝘁𝗶𝘁𝘆 𝗠𝗲𝗺𝗼𝗿𝘆: A close-book model with much better performance than 𝗘𝗮𝗘, e.g. 47.2 EM on TriviaQA, and outperform open-book on ELI5! ArXiv:

Tweet media one

1

39

171

1

0

11

@wyu_nd

Wenhao Yu

1 month

Asking LLMs to follow complex instructions to programming with function calls precisely is still a challenging task.

@terryyuezhuo

Terry Yue Zhuo

1 month

o1-preview-2024-09-12 on BigCodeBench-Hard Complete 34.5% (slightly better than Claude-3.5-Sonnet-20240620) Instruct 23.0% (far below other top models) Average 28.8% o1-preview may follow detailed instructions reasonably well, but not the brief ones. Not sure how consistent

Tweet media one

1

4

47

0

3

10

@wyu_nd

Wenhao Yu

11 months

Congratulations! Welcome back to Tencent AI lab for internship again!

@muhao_chen

🌴Muhao Chen🌴

11 months

My awesome student @JamesYHuang36 just received an outstanding paper award at #EMNLP2023 ! He is looking for summer research intern. Please interview him.

Tweet media one

4

7

97

0

0

10

@wyu_nd

Wenhao Yu

2 years

This is a great new benchmark dataset if you work on scientific QA problems!

@lupantech

Pan Lu

2 years

📢📢Excited to have one paper accepted to #NeurIPS2022 ! We present a new dataset, ScienceQA, and develop large language models to learn to generate lectures and explanations as the chain of thought (CoT). Data and code are public now! Please check👇👇

Tweet media one

Tweet media two

Tweet media three

Tweet media four

4

27

145

1

2

10

@wyu_nd

Wenhao Yu

2 years

(2/3) Unified Encoder-Decoder Framework with Entity Memory ( #EMNLP2022 ): The entity knowledge is stored in the memory as latent representations, and the memory is pre-trained on Wikipedia along with encoder-decoder parameters.

2

0

9

@wyu_nd

Wenhao Yu

3 months

In this paper, we introduce DocBench, a new benchmark designed to evaluate LLM-based document reading systems. Our benchmark involves a meticulously crafted process, including the recruitment of human annotators and the generation of synthetic questions. It includes 229 real

Tweet media one

1

4

9

@wyu_nd

Wenhao Yu

2 years

In 2021, we wrote a survey () to hightlight a key LM challenge: augmenting with external knowledge via IR, tools, etc. The introduction of plugins in ChatGPT reaffirms the effectiveness of knowledge augmentation for infusing LLMs with up-to-date information

A Survey of Knowledge-enhanced Text Generation | ACM Computing Surveys

The goal of text-to-text generation is to make machines express like a human in many applications such as conversation, summarization, and translation. It is one of the most important yet challenging...

@OpenAI

OpenAI

2 years

We are adding support for plugins to ChatGPT — extensions which integrate it with third-party services or allow it to access up-to-date information. We’re starting small to study real-world use, impact, and safety and alignment challenges:

894

4K

19K

0

0

9

@wyu_nd

Wenhao Yu

10 months

Thank you, Jerry @jerryjliu0 , for highlighting our proposition retrieval work in the llama-index. The LlamaPack truly demonstrates the practical application and effectiveness of proposition-based retrieval systems!

@jerryjliu0

Jerry Liu

10 months

A big factor for building production RAG is deciding the "chunk" used for retrieval + synthesis: should it be a sentence? Paragraph? In the "Dense X Retrieval" paper ( @tomchen0 et al.), the authors propose a concept that we've advocated for a while: decouple the indexed chunk

4

36

268

0

3

7

@wyu_nd

Wenhao Yu

2 years

(3/3) KG-enhanced DPR/FiD ( #EMNLP2022 ): … Using knowledge graph (Wikidata) to improve the retrieve-then-read pipeline, learn better document representation.

1

0

7

@wyu_nd

Wenhao Yu

2 years

(1/3) Generate-then-read propose a novel pipeline for solving open-domain QA tasks, i.e., replacing the process of retrieving contextual documents from large-scale corpora such as Wikipedia by prompting GPT-3 to generate relevant contextual documents.

Tweet card media

Generate rather than Retrieve: Large Language Models are Strong...

Knowledge-intensive tasks, such as open-domain question answering (QA), require access to a large amount of world or domain knowledge. A common approach for knowledge-intensive tasks is to employ...

1

1

7

@wyu_nd

Wenhao Yu

17 days

Training with reflective thinking is indeed an effective strategy, as shown in both Math and Coding. It may need further refinement, such as scaling the data and domains, improving the critic and verification. Check out our paper for more details:

Tweet card media

Learn Beyond The Answer: Training Language Models with Reflection...

Supervised fine-tuning enhances the problem-solving abilities of language models across various mathematical reasoning tasks. To maximize such benefits, existing research focuses on broadening the...

@_philschmid

Philipp Schmid

1 month

Synthetic data for reflective thinking and Chain of Thought! @OpenAI o1 “produces a long internal chain of thought before responding to the user.” A recent paper introduces RefAug on how we can augment existing data to embed problem reflection and “thinking” into the training

Tweet media one

2

55

213

0

0

6

@wyu_nd

Wenhao Yu

9 months

Tweet card media

WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

The rapid advancement of large language models (LLMs) has led to a new era marked by the development of autonomous applications in real-world scenarios, which drives innovation in creating...

0

1

5

@wyu_nd

Wenhao Yu

5 months

💡𝐍𝐞𝐰 𝐌𝐚𝐭𝐡 𝐁𝐞𝐧𝐜𝐡𝐦𝐚𝐫𝐤: Different from existing single-turn math QA datasets, MathChat is the first benchmark focusing on multi-turn conversations about math. 🔔Existing LLMs exhibit a significant decline in math reasoning ability after multi-turn conversations!

@LiangZhenwen

Zhenwen Liang

5 months

🚀 Excited to share our latest research MathChat! 📊 We explore the new frontiers in interactive math problem-solving. Check it out! 🧵👇 MathChat is a benchmark designed to evaluate LLMs on mathematical multi-turn interaction and open-ended generation.

3

15

76

0

0

5

@wyu_nd

Wenhao Yu

11 months

We improve current RALM on two aspects: (1) Noise Robustness: The ability to discern and disregard noisy information present in irrelevant retrieved documents, (2) Unknown Robustness: The ability to acknowledge its limitations by responding with “unknown” (1/4)

Tweet media one

1

0

5

@wyu_nd

Wenhao Yu

1 month

DSBench requires LLM systems to read user uploaded files, write and execute codes to solve data science problems. This benchmark includes 466 data analysis tasks and 74 data modeling tasks, sourced from Eloquence and Kaggle competitions. The dataset is available at

0

1

5

@wyu_nd

Wenhao Yu

2 years

Huge shoutout to our fantastic organizers for making the #KnowledgeAugmented #NLP workshop a reality! Thank you @MS_KnowledgeNLP , @wyu_nd , @Meng_CS , @ChenguangZhu2 , @shuohangw , @LuWang__ , and @hhsun1 .

0

0

5

@wyu_nd

Wenhao Yu

2 years

If you missed the workshop, you can still find the videos and slides at (AAAI will post the video in around two weeks) and

1

0

4

@wyu_nd

Wenhao Yu

2 years

New paper 🎉: Check out our new work on adaptive pretraining for logical reasoning, lead by @ssanyal8 !

@ssanyal8

Soumya Sanyal

2 years

Want to teach logical reasoning 💭 skills to LMs 🤖? Check out Apollo, our new adaptive pretraining strategy to improve logical reasoning in LMs. It is (a) Simple to implement (b) Generalizable across task formats (c) Needs minimal data processing Paper:

2

11

51

0

1

4

@wyu_nd

Wenhao Yu

2 years

@zhihz0535 just presented the work this morning. If you missed it but interested in related research, DM us and we are happy to chat!

Tweet media one

0

1

4

@wyu_nd

Wenhao Yu

2 years

Code will be at very soon! (4/4)

GitHub - wyu97/RACo: Resources for Retrieval Augmentation for Commonsense Reasoning: A Unified...

Resources for Retrieval Augmentation for Commonsense Reasoning: A Unified Approach. EMNLP 2022. - wyu97/RACo

1

0

4

@wyu_nd

Wenhao Yu

2 years

New paper 🎉: @lupantech Pan’s survey is a good summary and analysis of the recent work of language models in mathematical reasoning. If you are interested in mathematical reasoning, definitely check it out! Feedback welcome!

@lupantech

Pan Lu

2 years

🎉New paper! The survey of deep learning for mathematical reasoning ( #DL4MATH ) is now available. We've seen tremendous growth in this community since 2018, and this review covers the tasks, datasets, and methods from the past decade. Check it out now:

Tweet media one

6

79

336

0

1

4

@wyu_nd

Wenhao Yu

2 years

Code will be at (4/n)

GitHub - DM2-ND/EDMem: Code for EMNLP 2022 paper "A Unified Encoder-Decoder Framework with Entity...

Code for EMNLP 2022 paper "A Unified Encoder-Decoder Framework with Entity Memory" - DM2-ND/EDMem

1

1

4

@wyu_nd

Wenhao Yu

2 years

We also present a novel clustering-based prompting approach to generate diverse contextual documents that increases the likelihood of generating a correct answer with more generations. This approach can significantly improve performance on downstream tasks. (3/5)

1

0

4

@wyu_nd

Wenhao Yu

2 years

Thanks ND research for writing this great article for me 😊

@Meng_CS

Meng Jiang

2 years

Shout for Notre Dame's iSURE program and CSE PhD program. You may get interested in them, if you get a chance to read my student Wenhao's stories. Wenhao Yu is a rising 4th-year PhD with Bloomberg Fellowship, working on NLP / QA.

2

0

23

0

0

4

@wyu_nd

Wenhao Yu

1 month

@nembal I think mainly due to the imbalance in the language distribution in the pre-training corpus. Knowledge embeddings aren't as well connected across different languages. I remember when I studied abroad, it took me longer to learn a new concept compared to taking a similar class

0

0

4

@wyu_nd

Wenhao Yu

4 months

Try BigCodeBench! It is the next generation of HumanEval.

@terryyuezhuo

Terry Yue Zhuo

4 months

In the past few months, we’ve seen SOTA LLMs saturating basic coding benchmarks with short and simplified coding tasks. It's time to enter the next stage of coding challenge under comprehensive and realistic scenarios! -- Here comes BigCodeBench, benchmarking LLMs on solving

Tweet media one

1

39

125

0

0

4

@wyu_nd

Wenhao Yu

11 months

@ZhiruoW This is a great work! We also noticed irrelevant context could hurt model performance in industry applications. We just released a paper yesterday, with similar goal, to improve noise robustness in RAG

Tweet card media

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

Retrieval-augmented language models (RALMs) represent a substantial advancement in the capabilities of large language models, notably in reducing factual hallucination by leveraging external...

0

1

4

@wyu_nd

Wenhao Yu

2 years

👍 An impressive open-domain QA method that generalizes well on both single-hop and multi-hop setting!

@KaixinMa9

Kaixin Ma

2 years

I'm happy to share that our paper "Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge" is now online. We proposed a unified framework for solving single&multi-hop questions that require reasoning over tables and/or text.

2

12

91

0

0

3

@wyu_nd

Wenhao Yu

3 months

Tweet card media

GitHub - Anni-Zou/DocBench: DocBench: A Benchmark for Evaluating LLM-based Document Reading Systems

DocBench: A Benchmark for Evaluating LLM-based Document Reading Systems - Anni-Zou/DocBench

0

0

3

@wyu_nd

Wenhao Yu

11 months

Our experiments across four open-domain QA benchmarks show that RALMs equipped with CoN significantly outperform standard RALMs. Notably, CoN achieves an average improvement of +7.9 in EM score given entirely noisy retrieved documents. (3/4)

1

0

3

@wyu_nd

Wenhao Yu

2 years

[1/n] Follow us on Twitter: @knowledgenlp and get more details at .

1

0

3

@wyu_nd

Wenhao Yu

1 year

[2/n] This plug-and-play pipeline generates initial outputs, retrieves relevant info from document collections, and refines these outputs, thus efficiently addressing LLMs' limitations.

1

0

3

@wyu_nd

Wenhao Yu

9 months

This is an impressive, college entrance-level multi-modal QA that covers a diverse range of subjects. Looking forward to trying it out!

@LiangZhenwen

Zhenwen Liang

9 months

**New Benchmark for Multimodal LLMs❗❗** Introducing SceMQA: A pioneering Scientific College Entrance Level Multimodal Question Answering Benchmark. Learn more at: Paper:

Tweet media one

1

8

18

0

0

3

@wyu_nd

Wenhao Yu

2 years

We propose a novel generate-then-read pipeline for solving open-domain QA tasks, i.e., replacing the process of retrieving contextual documents from large-scale corpora such as Wikipedia by prompting a large language model to generate relevant contextual documents. (2/5)

1

0

3

@wyu_nd

Wenhao Yu

1 year

[3/n] Empirical results: over +6.0% improvement under zero-shot settings and +2.5% under few-shot settings compared to baselines on multiple open-domain QA, dialogue benchmarks.

0

0

3

@wyu_nd

Wenhao Yu

3 months

DocBench construction pipeline. (a) Document Collection: gathering PDF files from five different domains; (b) QA-pair Generation: creating diverse and comprehensive QA pairs through a combination of LLMs and human effort; (c) Quality Check: ensuring data quality through a

Tweet media one

1

0

3

@wyu_nd

Wenhao Yu

2 years

@ZhijingJin @aclmentorship @CausalNLP @NLP4PosImpact Congratulations! Looking forward to seeing you at EMNLP and attending your talk!

0

0

2

@wyu_nd

Wenhao Yu

11 months

Work done with my colleagues at Tencent AI Seattle lab Hongming Zhang ( @hongming110 ) Kaixin Ma ( @KaixinMa9 ), Xiaoman Pan, Hongwei Wang, Dong Yu. (4/4)

0

0

3

@wyu_nd

Wenhao Yu

11 months

Chain-of-note generates a series of reading notes for retrieved documents, enabling a comprehensive assessment of their relevance to the input query. We employed ChatGPT to create training data for CoN, which was subsequently trained on an LLaMa-2 7B model. (2/4)

2

0

3

@wyu_nd

Wenhao Yu

1 year

Arxiv:

Tweet card media

Auto-Instruct: Automatic Instruction Generation and Ranking for...

Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning. Unfortunately, the performance of...

0

0

2

@wyu_nd

Wenhao Yu

2 years

We proposed a unified framework of RetrievalAugmented Commonsense reasoning, including a newly constructed commonsense corpus with over 20 million documents and novel strategies for training a commonsense retriever. (1/4)

1

0

2

@wyu_nd

Wenhao Yu

10 months

We also introduce FACTOIDWIKI, a processed English Wikipedia dump, where each page is segmented into multiple granularities: 100-word passages, sentences and propositions. [3/4]

Tweet card media

GitHub - chentong0/factoid-wiki: Dense X Retrieval: What Retrieval Granularity Should We Use?

Dense X Retrieval: What Retrieval Granularity Should We Use? - chentong0/factoid-wiki

1

0

2

@wyu_nd

Wenhao Yu

1 year

Big congrats!!! 🎉

@Meng_CS

Meng Jiang

1 year

I am promoted to Associate Professor with tenure. Many thanks to my students, collaborators, and colleagues. Thanks for your great ideas, hard work, and continued support. My friends - let me know if you'd like a visit to ND / ND CSE. Go Irish!

25

1

245

0

0

2

@wyu_nd

Wenhao Yu

1 year

[1/n] By automatically incorporating retrieval feedback, ReFeed minimizes the need for resource-intensive manual supervision.

1

0

2

@wyu_nd

Wenhao Yu

1 month

@mediocrelychee We can see summaries are in English, but isn't it a bit weird to train the LLM to think in Chinese and then summarize in English?

1

0

2

@wyu_nd

Wenhao Yu

2 years

Together with @dan_iter @shuohangw @xycking @ssanyal8 @Meng_CS , in collaboration with Microsoft Cognitive Research. (5/5)

0

0

2