Nazneen Rajani @nazneenrajani Twitter profile

Last Seen Profiles

@sara55865486740

@brucewilson

@jccou06

@baderaldamni

@Wimbledon

@stwmaniax

@bokeplokalmalam

@Ashley_Banner

@poziayda

@NathiVolmink

@vocals_exo

@PhChanu

@inubrush

@Oddselection

@Dudamuda12

@teduell

@Zayouii

@BERRI_11

@virani_md

@JULIEN_PASC

@peatanddiesel

@medine243

@AlekseyGutierr1

@OneHoeDelaney

@GaylaBurris

@FrascaFoodWine

@WildfireWhisper

@AnusSmelly

@tamaki1697267

@a6w332

@WikiDokkan

@GaelicMedia

@jecsellyn

@yalanafarroa

@BiggaRank77

@omertost

Nazneen Rajani

@nazneenrajani

2 years

Did you know there are other dialog agents like ChatGPT? 🤔 And what if I told you the secret sauce is IFT, RLHF, CoT, and SFT 🤯 We explain each of these terms and why they are relevant to ChatGPT by comparing with 4 other dialog agents. Check our blog:

12

152

739

Nazneen Rajani

@nazneenrajani

2 years

Here's hoping I don't need to update this slide again before my talk next week @emnlpmeeting If anyone is planning to release anything next week, please lmk soon 😅 Am I missing any text-only LLMs?

25

80

532

Nazneen Rajani

@nazneenrajani

2 years

You can create your own chatbot by fine-tuning pre-trained causal LLM to follow instructions 🤖 Here is a list of datasets on @huggingface hub that you can use for Instruction fine-tuning (IFT) 🧵 /0

Nazneen Rajani

@nazneenrajani

2 years

Did you know there are other dialog agents like ChatGPT? 🤔 And what if I told you the secret sauce is IFT, RLHF, CoT, and SFT 🤯 We explain each of these terms and why they are relevant to ChatGPT by comparing with 4 other dialog agents. Check our blog:

12

152

739

29

111

492

Nazneen Rajani

@nazneenrajani

11 months

Thanks to Open Science, we are releasing Zephyr 🪁, a 7B parameter model that is as good as ChatGPT on AlpacaEval Our model is created using: - @MistralAI Mistral 7B base model - The UltraChat dataset for SFT - The UltraFeedback dataset for DPO Other results and demo link in 🧵

8

79

496

Nazneen Rajani

@nazneenrajani

3 years

Life update: I have joined @huggingface 🤗 and I will be working alongside @douwekiela @Thom_Wolf @mmitchell_ai and all the amazing folks here. I am excited to continue pushing research on model understanding and evaluation.

26

18

473

Nazneen Rajani

@nazneenrajani

4 years

New preprint alert! 🚨 Introducing GeDi (pronounced Jedi): A Powerful New Method for Controlling Language Models. Paper: Code: Blog: This paper has a bunch of really cool results. Here are a few.

5

73

307

Nazneen Rajani

@nazneenrajani

2 years

Just finished teaching my last class on Interpreting ML models and it has been such a rewarding experience🤩 We learned a ton of methods covering feature and instance attributions on three data modalities and evaluated each for plausibility and faithfulness. 4 hands-on projects!

5

27

223

Nazneen Rajani

@nazneenrajani

5 years

This was my first time submitting >1 papers at a conference and I am happy to announce that I have 3 long papers at #acl2020nlp 1. ERASER benchmark for interpretability 2. Causal and commonsense physical reasoning 3. Gender debiasing for word embedding #nlproc #silverlining

6

13

218

Nazneen Rajani

@nazneenrajani

2 years

I am studying ML model lifecycle & had a hypothesis that recent ML models have shorter lifecycles, i.e., their usage peaks and dies out quickly and is replaced by newer more efficient models (Dalle --> Stable diffusion). So I did a systematic analysis of 65K models on HF hub👇

5

34

212

Nazneen Rajani

@nazneenrajani

2 years

I came to terms with the fact that I'd have to update the timeline ever so often, but I must admit that I did not think I'd have to update the model accesses so frequently. PaLM: closed --> limited Claude: closed --> limited

4

47

204

Nazneen Rajani

@nazneenrajani

4 years

🚨New Paper+Toolkit🚨 Excited to introduce "Robustness Gym: Unifying the NLP Evaluation Landscape" (), a collaborative effort of @SFResearch @StanfordAILab @UNCNLP With amazing co-authors: @krandiash @Jesse_vig @CaimingXiong @MohitBan47 @HazyResearch 1/N

2

58

192

Nazneen Rajani

@nazneenrajani

1 year

Stoked to share that our tutorial on Responsible Generative AI got accepted at both @FAccTConference and @icmlconf 🎉 Looking forward to meeting everyone but not looking forward to updating this slide 🫠 I'm open to suggestions on specific topics to cover.

4

33

162

Nazneen Rajani

@nazneenrajani

5 years

#NLProc does not have a standard benchmark for interpretability. I am stoked to announce ERASER: the first-ever effort on unifying and standardizing NLP tasks with the goal of interpretability.

ERASER

www.eraserbenchmark.com

5

57

151

Nazneen Rajani

@nazneenrajani

2 years

If you are interested in learning to interpret ML models using the @huggingface workflow, this is your last chance to sign up for the course that starts in < 2 weeks . It is a hands-on 4 weeks course with exciting projects each week. Sneak peek of wk3 👇

3

31

144

Nazneen Rajani

@nazneenrajani

1 year

Is open-source having its ChatGPT moment? The LLaMA 2 is here (). When LLaMA was released earlier in the year, it was a pivotal moment for the OSS community. The advancement in LLMs has accelerated massively since with research artifacts inspired by or

3

21

139

Nazneen Rajani

@nazneenrajani

11 months

I am stoked to share that I am among the select individuals around the world who would take on the *huge* responsibility of serving on the @UN 's AI Advisory Board along with some prominent individuals including @miramurati @LatifaMKarim @HKitano Sharad Sharma, and many more.

16

12

138

Nazneen Rajani

@nazneenrajani

4 years

Getting both #EMNLP2020 and #NeurIPS2020 reviews on Friday afternoon is not great for work-life balance.

1

2

130

Nazneen Rajani

@nazneenrajani

2 years

Our paper on Systematic Error Analysis and Labeling (SEAL) 🦭 has been accepted at EMNLP demo track 🎉 Problem: How can we help users find systematic bugs in their models? Eg: Image classification model on low light images, sentiment classifier on gym reviews #emnlp2022

2

22

125

Nazneen Rajani

@nazneenrajani

1 year

If I told you the following based on our learnings from working on LLM evaluations using humans and GPT-4, which ones most surprise you? what is your intuition behind them? 1. GPT-4 has a positional bias and is predisposed to generate a rating of “1” in a pairwise preference

2

21

104

Nazneen Rajani

@nazneenrajani

2 years

So glad to be back to in-person @emnlpmeeting and being able to catch up with the amazing @YejinChoinka and Ray Mooney. Congrats again for the MacArthur @YejinChoinka 🚀🚀

0

1

106

Nazneen Rajani

@nazneenrajani

11 months

I have had the honor to work with @miramurati every week as part of our work on the UN’s AI Advisory. I have no doubt she will be able to lead the most powerful AI startup through this turbulence 💪🏽

4

5

99

Nazneen Rajani

@nazneenrajani

2 years

Here's a v0 🤗Datasets explorer: The embeddings use datasets' descriptions & paper abstracts. Here are some interesting things you can do. cc @YJernite @radamar

clem 🤗

@ClementDelangue

2 years

would you be interested in something like but for or ? cc @nazneenrajani @YJernite @srush_nlp

4

7

46

2

27

96

Nazneen Rajani

@nazneenrajani

1 year

I and @hima_lakkaraju really enjoyed presenting our tutorial on Generative AI meets Responsible AI @FAccTConference . I got many requests for our slides, so I added them to my webpage Thanks, #FAccT2023 , for a great conference and fantastic audience 🤗

4

20

97

Nazneen Rajani

@nazneenrajani

4 years

Seeing all the EMNLP reviewers increase their scores after I initiated a discussion based on what does and does not count as a good reason for rejecting a paper is pure joy. Almost feels like it's for my own paper :) #ACduties #emnlp2020

2

4

99

Nazneen Rajani

@nazneenrajani

2 years

Sundar asked Google employees to spend a few hours every day stress-testing their chatbot Bard. Bing's Sydney showed its malevolent alter ego to @kevinroose which led to @OpenAI committing to improving chatbot behavior. What they need is red-teaming

Red-Teaming Large Language Models

huggingface.co

4

12

99

Nazneen Rajani

@nazneenrajani

4 years

Wow, such an honor to be mentioned alongside @timnitGebru @AnimaAnandkumar @mmitchell_ai !

Kathy Baxter

@baxterkb

4 years

Congrats to @SFResearch 's @nazneenrajani for being nominated in the @VentureBeat Women in #AI awards for her research on #XAI ! @salesforce

0

4

24

3

85

Nazneen Rajani

@nazneenrajani

2 years

I will be giving a talk tomorrow morning @NIST 's AI Measurement and Evaluation colloquia series on the topic of evaluating LLMs. I'll be discussing evaluating a chatbot like ChatGPT and how we are thinking about it @huggingface while working on an open-source alternative.

4

13

81

Nazneen Rajani

@nazneenrajani

5 years

I am very proud to announce that our paper on leveraging explanations for Commonsense Question Answering got accepted @ACL2019_Italy Love working with the amazing folks @SFResearch @BMarcusMcCann @CaimingXiong @RichardSocher #ACL2019nlp #NLProc

5

10

74

Nazneen Rajani

@nazneenrajani

1 year

- Interpreting LLMs using LLMs - Redteaming LLMs using LLMs - Evaluating LLMs using LLMs (where the first LLM is smaller than the second) I am seeing a trend. What's next?

9

5

70

Nazneen Rajani

@nazneenrajani

1 year

You can interactively compare the @databricks Dolly instruction-tuned model here Do you agree more with the 3B model or the 7B? RLHF might help - easier to collect but needs a ton. Would sufficient human-written instruction data offset the need for RLHF?

1

19

70

Nazneen Rajani

@nazneenrajani

4 years

Excited to announce our latest work on Explaining Solutions to Physical ReasonIng Tasks (ESPRIT), an interpretable framework for representing the complex physical concepts such as gravity, friction, and collision using natural language accepted at #acl2020nlp !

2

13

65

Nazneen Rajani

@nazneenrajani

2 years

Really proud of this collaboration with @tableau research! We have the interactive demo deployed as @huggingface space You can interactively evaluate and analyze the model on various data slices. By default, it shows perf on US protected groups.(1/4)

Ana Crisan

@amcrisan

2 years

I am delighted to share our work on "Interactive Model Cards". This was a collaboration with Mar Drouhard, @jesse_vig and @nazneenrajani , which we'll be presenting at the @FAccTConference ! 📜 : 🖥️ : (1/2)

2

14

77

1

13

58

Nazneen Rajani

@nazneenrajani

1 year

I am stoked to be featured on the cover of this well-written NYT article I believe *alignment* is the secret sauce behind ChatGPT. Having worked on RLHF, including data collection from external vendors, and finetuning hundreds of open-access models at

The Secret Ingredient of ChatGPT Is Human Advice (Published 2023)

Companies like OpenAI hone their bots using hand-tailored examples from well-educated workers. But is this always for the best?

www.nytimes.com

5

11

57

Nazneen Rajani

@nazneenrajani

4 years

@zacharylipton Hold remote mentorship group sessions. Topics could be: applying to grad school, applying for jobs, help with editing papers and slides, etc.

3

0

56

Nazneen Rajani

@nazneenrajani

5 years

It seems like only yesterday that we moved from ATX to the Bay Area! Grateful to @SFResearch and @RichardSocher for supporting me as I adjusted to my first full time job while being a new mother. Here’s to many more exciting years @SFResearch 🎉

1

2

52

Nazneen Rajani

@nazneenrajani

4 years

Influence functions are great for debugging ML models but they cannot be used in practice because of being prohibitively expensive. FastIF is a more practical and efficient solution for model interpretability and debugging.

Han Guo

@HanGuo97

4 years

Glad to share our latest work "FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging"! Joint work with @nazneenrajani @peterbhase @mohitban47 @caimingxiong ( @uncnlp @sfresearch ). Paper: Code: 1/5

1

24

146

1

4

49

Nazneen Rajani

@nazneenrajani

5 years

I am hiring a Research Scientist to work broadly on Explainable AI (XAI) at @SFResearch with a fun and friendly team of talented researchers with ethical AI practice. Should be available to join in the next few months. JD: Please DM me for any questions.

1

12

47

Nazneen Rajani

@nazneenrajani

11 months

So jealous of Sama rn. He got to read all the eulogies and know who his friends and enemies are. And come back even stronger and more powerful than ever!

2

1

44

Nazneen Rajani

@nazneenrajani

6 years

Excited to announce that I joined @salesforce research and will be working with @RichardSocher @CaimingXiong @nikhil_ai @VictoriaLinML and other really smart folks :D

3

4

43

Nazneen Rajani

@nazneenrajani

2 years

Check out the projects the students attempted for each week here

GitHub - nazneenrajani/interpreting-ml-models-course: Course for Interpreting ML Models

Course for Interpreting ML Models. Contribute to nazneenrajani/interpreting-ml-models-course development by creating an account on GitHub.

github.com

1

3

42

Nazneen Rajani

@nazneenrajani

5 years

I will be giving an invited talk at the Toronto Machine Learning Summit (TMLS) about my work @SFResearch on how we can train language models to generate explanations and use them for performance gain on downstream tasks as well as be transferred to out-of-domain tasks #tmls2019

1

11

43

Nazneen Rajani

@nazneenrajani

1 year

The "honest" part of our RLHF training has gone through the roof 😅

0

2

39

Nazneen Rajani

@nazneenrajani

2 years

@RichardSocher We are doing all of this @huggingface while building an open-source alternative to ChatGPT called H4 Stay tuned for high-quality SFT and RLHF data.

HuggingFaceH4 (Hugging Face H4)

huggingface.co

1

7

39

Nazneen Rajani

@nazneenrajani

11 months

Paper: Demo: Model:

HuggingFaceH4/zephyr-7b-beta · Hugging Face

huggingface.co

1

7

38

Nazneen Rajani

@nazneenrajani

1 year

Are standard NLP benchmarks good enough for evaluating chatty LLMs? 🤔 In my experience, they are good for evaluating pretraining and in-context learning but not for SFT or RLHF models. Here is a straightforward example I tried on both Falcon and RedPajama. Falcon got 1/2, and

Together AI

@togethercompute

1 year

Announcing RedPajama 7B trained on 1T tokens! 🚀 • Instruct, chat, base, and interim checkpoints on @huggingface • The instruct model outperforms all open 7B models on HELM benchmarks • The 5TB dataset has been used to train over 100 models Details👇

9

155

538

2

4

35

Nazneen Rajani

@nazneenrajani

11 months

We found that just doing SFT on both datasets is not as good as our best recipe of SFT + DPO. And just doing DPO directly is the worst.

1

4

35

Nazneen Rajani

@nazneenrajani

4 years

Can GeDi be used to debias this #GPT3 generation? @benwkrause and @AkhileshGotmare used GeDi for filtering #GPT3 generations and the results are fascinating! We have pushed the code so you can try GeDi for #GPT3 while you still have API access

Abubakar Abid

@abidlabs

4 years

I'm shocked how hard it is to generate text about Muslims from GPT-3 that has nothing to do with violence... or being killed...

156

2K

5K

2

3

33

Nazneen Rajani

@nazneenrajani

4 years

Thank you, @aclmeeting organizers, for putting together an amazing virtual conference. Learned a lot and also enjoyed zoom mentoring + paper discussions. PS: I can no longer watch any video at a normal rate, 1.5x is the new normal #acl2020nlp

0

2

33

Nazneen Rajani

@nazneenrajani

2 years

Recently I have been thinking deeply about questions on evaluating LLMs for emerging capabilities. One thing I worry about is overfitting to current capabilities and I'd imagine this becomes even more of a problem in policy where things move even slower.

1

2

32

Nazneen Rajani

@nazneenrajani

2 years

I spend about 5-6 hours each week interacting with @mmitchell_ai and many more working with her and I 100% agree with everything in this thread! So much respect and gratitude for everything she does 🤗

@[email protected] on Mastodon

@timnitGebru

2 years

4-5 years ago @mmitchell_ai was a semifinalist for the MIT Tech Review 35 under 35. I wrote a letter of support. One of the things I mentioned was that her work has been so under appreciated in the field of "AI." 1/n

7

176

985

1

29

Nazneen Rajani

@nazneenrajani

5 years

1/3 I had my O1 (Extraordinary ability) visa interview on Monday morning at @USCGFlorence and they put my case on extra background checking, even though I have an approved O1 petition. I traveled to Florence to present my research on Explainable AI at #acl2019nlp

2

14

27

Nazneen Rajani

@nazneenrajani

5 years

Excited to have @jesse_vig join us @SFResearch ! Jesse has done amazing work on visualizing the inner workings of various #NLProc models. Looking forward to working with him on more cutting edge research in interpretability. Stay tuned!

3

2

27

Nazneen Rajani

@nazneenrajani

5 years

Updated the CoS-E repo with code to reproduce results from our ACL paper on commonsense reasoning using natural language explanations Check it out here: Better late than never 🙂

GitHub - salesforce/cos-e: Commonsense Explanations Dataset and Code

Commonsense Explanations Dataset and Code. Contribute to salesforce/cos-e development by creating an account on GitHub.

github.com

0

3

26

Nazneen Rajani

@nazneenrajani

6 years

Just submitted my first paper to @ACL2019_Italy with @BMarcusMcCann @CaimingXiong and @RichardSocher as a Salesforce Research employee and not a student! #acl2019nlp #NLProc

0

25

Nazneen Rajani

@nazneenrajani

3 years

This is really cool! Thanks @Gradio Check out our controllable summarization interactive demo on @Gradio

AK

@_akhaliq

3 years

CTRLsum: Towards Generic Controllable Text Summarization by @salesforce in @PyTorch on @Gradio paper: github: gradio demo:

2

12

81

1

5

25

Nazneen Rajani

@nazneenrajani

4 years

Our ERASER benchmark leaderboard is live: If you use any of our datasets please consider reporting to the leaderboard.

ERASER

www.eraserbenchmark.com

1

9

25

Nazneen Rajani

@nazneenrajani

5 years

UPDATE: Got an email from the consulate that my visa has been approved and I will get it on my passport on Monday. Thank you all for your support! Very happy that I am part of a very supportive community! #acl2019nlp will forever be etched in my memory 😊

2

1

26

Nazneen Rajani

@nazneenrajani

2 years

I think I found a solution to jet lag -- give an invited talk the very next day so that you keep making last-minute changes and won't have time to nap #EMNLP2022

0

24

Nazneen Rajani

@nazneenrajani

4 years

With both NeurIPS and EMNLP extended, I am just gonna take a break for few days and hopefully not feel guilty.

raia hadsell

@RaiaHadsell

4 years

Due to COVID-19, we have decided to shift the NeurIPS timeline 3 weeks back, giving authors additional time and flexibility. We hope this is helpful to the NeurIPS community! Abstracts now due May 27, paper deadline June 3. Good luck all - stay safe and well. #neurips2020

11

174

710

0

25

Nazneen Rajani

@nazneenrajani

2 years

I have worked and co-authored papers with Drago. He was a very kind soul and went out of his way to help people. I am incredibly shocked to hear this news. We exchanged emails 2 weeks ago 😢 Life is so uncertain. Condolences to his family and friends.

Harlan Krumholz

@hmkyale

2 years

The #AI community, the #computerscience community, the @YaleSEAS community, and humanity have suddenly lost a remarkable person, @dragomir_radev - kind and brilliant, devoted to his family and friends... gone too soon. A sad day @Yale @YINSedge @YaleCompsci #NLP2023

41

87

388

0

1

25

Nazneen Rajani

@nazneenrajani

4 years

Got all the 60/60 EMNLP reviews without having to chase down anyone for last-minute reviews #ACduties Thanks to all the reviewers!

0

24

Nazneen Rajani

@nazneenrajani

2 years

I presented our work on the systematic study of models on HF when we were at 75,000 models just a few weeks ago at EMNLP Slides: Very exciting!

Keynote

EMNLP 2022 Keynote Speakers.

2022.emnlp.org

clem 🤗

@ClementDelangue

2 years

We crossed 100,000 public AI models on the @huggingface hub available for free to all. Thank you to the whole community of contributors. Proud to make ML more open & collaborative!

8

65

472

1

3

24

Nazneen Rajani

@nazneenrajani

1 year

Has anyone benchmarked the instruction fine-tuned models like Dolly, Vicuna, Open Assistant, on HELM or Big Bench?

2

24

Nazneen Rajani

@nazneenrajani

4 years

Honored to also be featured in @QuantaMagazine article on common sense reasoning along with @YejinChoinka @elliepavlick and my former advisor Ray Mooney

Salesforce AI Research

@SFResearch

4 years

Check out the latest work from @nazneenrajani : Explaining Solutions to Physical Reasoning Tasks (ESPRIT), an innovative framework which unifies commonsense physical reasoning and interpretability using natural language explanations.

0

1

3

1

3

22

Nazneen Rajani

@nazneenrajani

5 years

My daughter in the childcare room @NAACLHLT thanks for being accommodating! #naacl2019

0

2

23

Nazneen Rajani

@nazneenrajani

5 years

I will be talking about my work @SFResearch and walk through an application on making DL models more transparent and fair @DeepIndaba #SautiYetu #DLIndaba2019

0

4

23

Nazneen Rajani

@nazneenrajani

5 years

My first paper as non-student got accepted at ACL in first shot! I guess the evil spell is over 😉 #ACL2019nlp #NLProc

0

23

Nazneen Rajani

@nazneenrajani

5 years

Immigration officer at Vancouver airport: What's your reason for coming to Canda? Me: NeurIPS. Officer: You mean NIPS? Me: 😐

0

22

Nazneen Rajani

@nazneenrajani

6 years

We are looking to hire full-time researchers and research interns for Fall'19. Lmk if you are interested.

5

3

22

Nazneen Rajani

@nazneenrajani

11 months

The work is done in collaboration with a lot of amazing folks @huggingface . This would not have been possible without the Mistral model, the Ultrachat and UltraFeedback datasets, and the MTBench, AlpacaEval evaluations.

1

23

Nazneen Rajani

@nazneenrajani

5 years

Super excited to be in ATX for recruiting and speaking @UTCompSci Looking forward share my experience working @SFResearch with old friends and new folks! I will be presenting at FAI on Friday .

1

6

22

Nazneen Rajani

@nazneenrajani

3 years

Both my #ICLR2021 paper poster sessions are today between 5-7pm PST (Session 9). 1. Interpreting protein LMs: in spot A3 2. Counterfactuals to evaluate DST in spot A4 Stop by to learn more about our work+current research directions

1

2

21

Nazneen Rajani

@nazneenrajani

5 years

I am doing my best to flatten two curves right now. One is the #COVID19 curve and the other is my daughter's screen time curve while being in quarantine. We have had some remarkable success in last two days hopefully the same is true for the #COVID19 curve #FlattenTheCuve

1

0

21

Nazneen Rajani

@nazneenrajani

1 year

Check out StackLlama, a research artifact we open-sourced as we build our open-source alternative to ChatGPT/Claude. Let us know what you think and what you would like to see more -- 1. Datasets for SFT and RLHF 2. Instruction fine-tuned/RLHF models 3. Knowledge and findings

Leandro von Werra

@lvwerra

1 year

Excited to introduce: StackLlama🦙 An end-to-end tutorial for training Llama with RLHF on preference data such as the StackExchange questions! Blog: Demo: Code: The resulting model is surprisingly fun!🧵

23

224

889

0

6

20

Nazneen Rajani

@nazneenrajani

1 year

More like a fully disconnected conference with zero communication to invited speakers on their time slot. I declined the invite to speak. It anyway seemed like sprinkling token women in a dude lineup. If you wanted to hear about our work on H4, not happening here.

3

4

21

Nazneen Rajani

@nazneenrajani

2 years

I'm looking forward to discussing the @huggingface ecosystem of NLP models, evaluation, and documentation. Here's one fact from the talk -- 0.2% of the models drive >80% of the usage on @huggingface Join me for more exciting results and findings.

EMNLP 2024

@emnlpmeeting

2 years

We're super excited about our keynote speakers! Mona Diab, Neil Cohn, Gary Marcus, and Nazneen Rajani! @visual_linguist @GaryMarcus @nazneenrajani

0

2

32

0

4

19

Nazneen Rajani

@nazneenrajani

5 years

Next week will be my first ever Dreamforce! I am excited (and nervous) to be talking at the Research Keynote along with @RichardSocher @CaimingXiong @VictoriaLinML @StrongDuality Please join us if you will be at #DF19 Session details:

1

3

21

Nazneen Rajani

@nazneenrajani

2 years

We are ready for the biggest open source meetup ever 🤗

0

21

Nazneen Rajani

@nazneenrajani

5 years

The best swag award @DeepIndaba goes to @SFResearch for these adorable tribal socks! #DLIndaba2019 #sautiyetu

3

21

Nazneen Rajani

@nazneenrajani

1 year

Our LLM leaderboard broke the internet💥 and took the community by storm🌀 We have now expanded our leaderboard to include Human evals in partnership with @scale_AI and GPT4 evals🚀 The most fascinating result to me is how the most human-aligned model, GPT4, is actually not

Nazneen Rajani

@nazneenrajani

1 year

If I told you the following based on our learnings from working on LLM evaluations using humans and GPT-4, which ones most surprise you? what is your intuition behind them? 1. GPT-4 has a positional bias and is predisposed to generate a rating of “1” in a pairwise preference

2

21

104

1

5

21

Nazneen Rajani

@nazneenrajani

4 years

Really wanted to work on this problem from the moment I saw the original paper!

Richard Socher

@RichardSocher

4 years

Interesting work by @nazneenrajani , our team at @SFResearch and @Yale on ESPRIT, a framework for commonsense reasoning about physics in natural language. It generates interpretable descriptions of physical events. Paper: Blog:

5

37

117

0

2

19

Nazneen Rajani

@nazneenrajani

5 years

Really looking forward to this! #DLI2019

Emily Muller

@mathwis_emily

5 years

Meet @DeepIndaba x @UNESCO AI & Fairness Speaker: Nazneen Rajani. @nazneenrajani is a Computer Scientist at Salesforce and will be addressing transparency in AI systems through her work on Explainable Deep Learning for Natural Language Understanding. #Fri30th #SautiYetu #DLI2019

0

6

23

0

4

20

Nazneen Rajani

@nazneenrajani

5 years

I had a lot of fun talking about my work @SFResearch at FAI @UTCompSci during my visit to ATX. I am hiring for the RS position with a focus on XAI. We are a fun and friendly research team. If you are interested, please apply here:

0

4

20

Nazneen Rajani

@nazneenrajani

4 years

Q&A session for this paper today at 10 am PDT and 2 pm PDT. You can watch the talk video here #acl2020nlp

Richard Socher

@RichardSocher

4 years

Interesting work by @nazneenrajani , our team at @SFResearch and @Yale on ESPRIT, a framework for commonsense reasoning about physics in natural language. It generates interpretable descriptions of physical events. Paper: Blog:

5

37

117

0

2

18

Nazneen Rajani

@nazneenrajani

5 years

If you work on explainable AI, multitask learning, AI for good and related areas, consider applying to this.

Salesforce AI Research

@SFResearch

5 years

🤩 📣 Announcing the 2nd Annual @Salesforce Research Deep Learning Grant 🤩 📣 We're looking for diverse individuals with innovative ideas who can join us in shaping the future of AI. Apply today, and earn up to $50,000!

2

58

132

0

8

20

Nazneen Rajani

@nazneenrajani

1 year

Stay tuned! The results might be shocking :)

Andrew Ruiz

@then_there_was

1 year

@ClementDelangue @huggingface Are there any plans to add the HumanEval test to the Hugging Face Leaderboard?

2

0

3

1

2

20

Nazneen Rajani

@nazneenrajani

1 year

Is this what alignment/safety tax looks like in practice? There are tradeoffs between alignment and performance. I can imagine that aligning GPT4 via RLHF after a point leads to a massive degradation in performance.

Matei Zaharia

@matei_zaharia

1 year

Lots of people are wondering whether #GPT4 and #ChatGPT 's performance has been changing over time, so Lingjiao Chen, @james_y_zou and I measured it. We found big changes including some large decreases in some problem-solving tasks:

122

789

3K

1

4

19

Nazneen Rajani

@nazneenrajani

4 years

New preprint on the interpretability of protein language models. The most fascinating result for me was how well attention learns the 3D structure of proteins! We found that attention captures not just structure but also protein functions such as binding sites.

1

3

19

Nazneen Rajani

@nazneenrajani

1 year

My first time @FAccTConference and my first reaction is “it’s sparse”. It’s tiny compared to my first conference which was ACL 2015 in Beijing. This is a keynote and they put tables with chairs and still so sparse 😲

1

18

Nazneen Rajani

@nazneenrajani

1 year

💯ChatGPT makes many factual errors but I still feel it’s impact on learning and education can be disruptive. I can already imagine my 5yo using ChatGPT with strong guardrails to learn and have her understanding about things corrected. Hope RLHF works well for basic concepts.

Arvind Narayanan

@random_walker

1 year

When ChatGPT came out I thought I wouldn't use it for learning because of its tendency to slip in some BS among 10 helpful explanations. Then I tried it, and found that it forces me to think critically about every sentence, which is the most effective mindset for learning.

39

187

2K

1

0

19

Nazneen Rajani

@nazneenrajani

5 years

Why does @ACL2019_Italy camera-ready deadline overlap with the @NAACLHLT main conference? I spent much of my day working on my paper and I also saw a lot of overleaf screens all around :-/ #naacl2019

2

16

Nazneen Rajani

@nazneenrajani

1 year

I couldn't make it to #ACL2023NLP this year, but I am living vicariously through everyone's tweets and have a case of FOMO 😅

0

18

Nazneen Rajani

@nazneenrajani

1 year

The Alpaca Moment of Code is here ✨ We released the instruction-tuned version of @BigCodeProject 's StarCoder called StarChat Alpha🌟 Check it out: More details in the blog: Congrats to the team🤗

Creating a Coding Assistant with StarCoder

huggingface.co

0

6

18

Nazneen Rajani

@nazneenrajani

2 years

I tried the interactive demo and here’s what I found: 1. It is not sure if women should be allowed to vote (img 1). 2. The bot is a woman called jane (img 2) 3. That FB tracks us all over the internet and it might have been involved in rigging the 2016 elections (3-n imgs contd)

AI at Meta

@AIatMeta

2 years

(1/4) Meet BlenderBot 3, the first publicly available 175B-parameter chatbot with model weights, code & datasets. It can chat about nearly any topic & is designed to learn & improve by conversing with people in the real world. Try the interactive demo:

31

173

672

1

2

18

Nazneen Rajani

@nazneenrajani

4 years

Please join me on my virtual presentation of our #acl2020nlp paper on explaining solutions to physical reasoning tasks @SFResearch #ICLR2020 virtual booth tomorrow at 3 pm PST. Details:

Talent Acquisition Software | Yello

Yello's talent acquisition software and recruitment CRM is used by Fortune 500 companies to attract and nurture top talent from hello to hire.

yello.co

1

5

17

Nazneen Rajani

@nazneenrajani

4 years

Our paper on few-shot textual entailment is accepted at #emnlp2020 Preprint and code coming soon!

Caiming Xiong

@CaimingXiong

4 years

Our NLP team got 16 papers (11 long, 2 short, and 3 finds) at #emnlp2020 , which cover dialogue, summarization, question answering, multilingual, few-shot, NLI, semantic parsing, data augmentation, etc. Congrats to team members and coauthors. More info about papers coming soon!

13

67

459

0

18

Nazneen Rajani

@nazneenrajani

11 months

Wow, what a beginning to the weekend! How long before LLMs with browsing capabilities will get the correct answer to "Who is the CEO of OpenAI?"

1

0

18

Nazneen Rajani

@nazneenrajani

2 years

So stoked about this feature — this also means documentation for datasets / models / demos are no longer static objects and keep evolving 👏

Abubakar Abid

@abidlabs

2 years

The @HuggingFace community tab is a game changer for machine learning models! Datasets / models / demos are no longer static objects created once and left to collect dust. Instead, they can be discussed and improved by the open-source community through pull requests 🔥

1

3

28

0

1

16

Nazneen Rajani

@nazneenrajani

5 years

Folks @DeepIndaba I will be there at the Salesforce booth today. Stop by and learn more about the DL research we do @SFResearch and the various open opportunities #DLIn #SautiYetu

0

4

17

Nazneen Rajani

@nazneenrajani

1 year

As we are wrapping up our project on creating the secret sauce of *alignment* for open-access models, we plan to release recipes and artifacts in the coming weeks in this repo Make sure to watch/star so you don't miss out.

0

3

17