Yanai Elazar @yanaiela Twitter profile

Pinned Tweet

Yanai Elazar

@yanaiela

2 months

It's all about the training data.

Yangsibo Huang

@YangsiboHuang

2 months

I recently began exploring how memorization affects model capabilities. E.g., we found that image generation models struggle with prompts that combine more than 3 visual concepts (e.g., "red," "fluffy," "squared," "smartphone") & we attribute this to their training data.

2

8

125

2

0

18

Last Seen Profiles

@Elettra1998

@JohnR23181198

@jitn

@ChiodiJessica

@zack_012345

@muchmagda

@GreyEditing

@TheRevolutionR1

@QJapheth13050

@AFCAmmers

@JulianPatt62379

@JasonAnder59516

@jisunngei

@Nroeje

@BinorRaja

@s_favo_fox

@bokeplokalmalam

@FilomacoBalleto

@KongmengKy24253

@KathleneCo96793

@AllisonRic50676

@BinorRaja

@BrandiiTam44313

@s_fahselt

@stwmaniax

@J_Loviewer

@me461815

@nhfmo

@BlakDymond1

@bima37741069943

@stwmaniax

@scolleNcs

@PTI_Hub

@copy4conversion

@bokeplokalmalam

@tre_hinch

Yanai Elazar

@yanaiela

2 years

New *very exciting* paper We causally estimate the effect of simple data statistics (e.g. co-occurrences) on model predictions. Joint work w/ @KassnerNora , @ravfogel , @amir_feder , @Lasha1608 , @mariusmosbach , @boknilev , Hinrich Schütze, and @yoavgo

Measuring Causal Effects of Data Statistics on Language...

Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. But what exactly in the training data causes a model to make a certain...

arxiv.org

3

52

320

Yanai Elazar

@yanaiela

9 months

As of today I'm officially a doctor ☺️ It's a good time to announce I was the runner-up for the Israeli AI best thesis award. But please don't read my thesis (which is just a concatenation of my papers), read the papers!

46

3

289

Yanai Elazar

@yanaiela

1 year

This past week was terrible. In addition, I faced some academic rejections (3 just this past week). I figured it's a good time to discuss rejections in academia. I created a 502 website describing my recent failures:

19

23

249

Yanai Elazar

@yanaiela

1 year

What's In My Big Data? A question we've been asking ourselves for a while. Here is our attempt to answer it. 🧵 Paper - Demo-

4

69

239

Yanai Elazar

@yanaiela

2 years

I'm... graduating! Last week (the pic): goodbye party from the amazing @biunlp lab and the one and only @yoavgo Now: In the airport, on my way to Seattle for my postdoc at AI2 in the @ai2_allennlp team and UW, where I'll be working with @nlpnoah @HannaHajishirzi and @sameer_

25

4

221

Yanai Elazar

@yanaiela

3 years

Advancements in Commonsense Reasoning and Language Models make the Winograd Schema look solved. After revisiting the experimental setup and evaluation, we show that models perform randomly! Our latest work at EMNLP, w/ @hongming110 , @yoavgo , @DanRothNLP

5

53

216

Yanai Elazar

@yanaiela

1 year

Does arXiving have a casual effect on acceptance? The answer is nuanced, and depends on what assumptions you are willing to make, but arguably more importantly, we observe no difference in acceptance for different groups.

5

41

205

Yanai Elazar

@yanaiela

2 years

Passive aggressive, academia style: "Code and data will be released upon publication"

10

6

174

Yanai Elazar

@yanaiela

2 years

Papers presented at conferences often offer a limited view of a researcher's vision. At EMNLP23, you have the chance to present "The Big Picture" - a workshop dedicated to telling the big picture behind individual papers, often known mainly to a student/PI

Big Picture Workshop

News

www.bigpictureworkshop.com

4

17

149

Yanai Elazar

@yanaiela

3 years

1. Most papers ignore the fact that as opposed to vision, when you change a few pixels - nothing is likely to happen, but in language this is not the case. As a result, most papers don't even test if their perturbed examples, which are supposed to maintain the original label, /

1

2

100

Yanai Elazar

@yanaiela

2 years

I don't really understand this claim. Sure, if you work at one of 5 companies that actually have the compute to run these huge models, you can just go larger. But even then, can you actually use these huge models in production? Are the improved results actually worth the money?

Jason Wei

@_jasonwei

2 years

I struggled a lot with how to do work that "stands the test of time". 99% of methods become obsolete in a few years when we have bigger models trained on more data (The Bitter Lesson). It turns out that scaling plots clearly signal which approaches will stand the test of time.

4

21

272

9

4

92

Yanai Elazar

@yanaiela

6 months

I'm presenting this (tiny) paper today on behalf of @yasaman_razeghi who couldn't attend #ICLR . w/ @hamishivi and @sameer_ 10:45-12:45 at Hall B. Come check it out!

0

5

88

Yanai Elazar

@yanaiela

10 months

Wait, why do we need to fill up 6 pages of "responsible NLP research" now??

7

2

81

Yanai Elazar

@yanaiela

3 years

Our consistency paper got accepted to TACL! Updated paper at: Code: Thanks again to my awesome collaborators @KassnerNora , @ravfogel , @Lasha1608 , Ed Hovy, @HinrichSchuetze and @yoavgo See you at Punta Cana?

GitHub - yanaiela/pararel

Contribute to yanaiela/pararel development by creating an account on GitHub.

github.com

3

16

80

Yanai Elazar

@yanaiela

11 months

The Big Picture workshop @ EMNLP23 is just one week away, and we ( @AllysonEttinger , @KassnerNora , @seb_ruder , @nlpnoah ) have an incredible program awaiting you!

Big Picture Workshop

News

www.bigpictureworkshop.com

3

8

78

Yanai Elazar

@yanaiela

1 year

Now that we've seen that conferences can quickly adapt and make important decisions about changes in the field () - Can we now stop with this anonymity deadline nonsense?

ACL 2023 Policy on AI Writing Assistance

Text generation models have been long available, and they are powering many existing tools assisting with input or the linguistic form of the text, like predictive keyboards or language checkers....

2023.aclweb.org

2

9

78

Yanai Elazar

@yanaiela

3 years

Do you need to explain why your technique improved performance? Just call it regularization

6

2

73

Yanai Elazar

@yanaiela

1 year

Come say hi!

4

0

74

Yanai Elazar

@yanaiela

4 years

"Your method is trivial and easy to use" - Reject

3

69

Yanai Elazar

@yanaiela

10 months

This is a reminder that the advisor, and lab environment are way more important than some random school ranking. (Not that I have anything bad to say about CMU)

Mona Diab

@MonaDiab77

10 months

CMU SCS has done it again! SCS is ranked #1 in AI graduate programs in the USA for 2023 by US News. Kudos and Congratulations to all the departments that make up AI within SCS! Special shoutout to the LTI. Yes Shameless plug here :)

0

9

102

3

61

Yanai Elazar

@yanaiela

8 months

Language Models are nothing without their training data. But the data is large, mysterious, and opaque, which requires selection, filtering, cleaning, and mixing. Checkout our survey paper (led by the incredible @AlbalakAlon ) that describes the best (open) practices in the field.

Alon Albalak

@AlbalakAlon

8 months

{UCSB|AI2|UW|Stanford|MIT|UofT|Vector|Contextual AI} present a survey on🔎Data Selection for LLMs🔍 Training data is a closely guarded secret in industry🤫with this work we narrow the knowledge gap, advocating for open, responsible, collaborative progress

10

77

309

1

8

62

Yanai Elazar

@yanaiela

5 years

After a lot of work, I'm very happy to see this project integrated in the best library for #NLPRoc

spaCy

@spacy_io

5 years

🔢 num_fh: Extension by @yanaiela and @yoavgo for their new paper on identifying and resolving numeric fused heads — crucial for understanding 20-40% of numeric expressions in English!

2

24

91

1

12

61

Yanai Elazar

@yanaiela

8 months

Or some composition of the data? I couldn't find the exact phrase as is, in some of the open-source datasets we have indexed, but parts of it definitely appear on the internet

Alex Albert

@alexalbert__

8 months

Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval. For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of

574

2K

12K

2

57

Yanai Elazar

@yanaiela

3 years

A tweet about accepted and rejected papers from EMNLP

3

0

57

Yanai Elazar

@yanaiela

6 years

using adversarial training for removing sensitive features from text? do not trust its results! new #emnlp2018 paper with @yoavgo now in arxiv

1

20

54

Yanai Elazar

@yanaiela

1 year

These discussions on the anonymity period and what the "community" wants or who it affects are full of hand-waving. We empirically study this question in a paper that (funnily enough) could have been discussed at EMNLP if the anonymity period didn't exist

Estimating the Causal Effect of Early ArXiving on Paper Acceptance

What is the effect of releasing a preprint of a paper before it is submitted for peer review? No randomized controlled trial has been conducted, so we turn to observational data to answer this...

arxiv.org

Yanai Elazar

@yanaiela

1 year

Does arXiving have a casual effect on acceptance? The answer is nuanced, and depends on what assumptions you are willing to make, but arguably more importantly, we observe no difference in acceptance for different groups.

5

41

205

2

5

53

Yanai Elazar

@yanaiela

6 years

my first blog is out! I explain how I work with remote machines, and give some tips on how to make it more convenient.

2

12

48

Yanai Elazar

@yanaiela

3 years

The reasoning behind Calibration is to provide meaningful probabilities. Why is that important? Are there works that make use of these probabilities in meaningful ways?

22

2

43

Yanai Elazar

@yanaiela

2 years

As an AC in EACL, I am responsible for 16 papers, our of which 11 reviewers didn't make it to the review deadline 7 didn't notify us they will be late 3 didn't reply to our emails 2 didn't submit their reviews to this day

5

1

40

Yanai Elazar

@yanaiela

2 years

@annargrs The last thing that in-person attendees want to do is too join a virtual session. For virtual sessions to work (at least with the current technology), they should be in a fully virtual conference, or adjacent (e.g the week before the conference)

5

0

40

Yanai Elazar

@yanaiela

1 year

"raising your availability to 8+ papers"? seriously???

6

0

39

Yanai Elazar

@yanaiela

2 years

MNLI has 392,702 total instances. RoBERTa Tokenizes p+h with the *default* max seq length var in HF yields 997 instances that get truncated. It's only 0.25% of the data, but it can be detrimental, and lead models to rely on heuristics. Be careful of default variables!

2

38

Yanai Elazar

@yanaiela

9 months

Data contamination is one of the biggest challenges for evaluating [small|medium|large|huge] language models. Submit your work that tackles these challenges to our workshop (@ ACL24).

CONDA 2024 | The 1st Workshop on Data Contamination

Evaluation data has been compromised! A workshop on detecting, preventing, and addressing data contamination.

conda-workshop.github.io

Oscar Sainz

@osainz59

9 months

📢 Excited to announce that our Workshop on Data Contamination (CONDA) will be co-located with ACL24 in Bangkok, Thailand on Aug. 16. We are looking forward to seeing you there! Check out the CFP and more information here:

0

9

33

0

4

39

Yanai Elazar

@yanaiela

10 months

No, actually my abstract summarizes another paper

2

0

38

Yanai Elazar

@yanaiela

7 months

Giving a talk at @EdinburghNLP tomorrow! Come to hear about what makes it to text corpora these days and how it's affecting model behavior.

1

0

36

Yanai Elazar

@yanaiela

10 months

We're very close to a new *ACL leadership. Last chance to make a *very needed* policy update to the anonymity period?

1

2

37

Yanai Elazar

@yanaiela

4 years

New paper: Back to Square One: Bias Detection, Training and Commonsense Disentanglement in the Winograd Schema Joint work with @hongming110 , @yoavgo and @dannydanr

2

9

36

Yanai Elazar

@yanaiela

1 year

@OwainEvans_UK @megamor2 Cool! You might be interested in this paper where we studied similar issues:

Measuring and Improving Consistency in Pretrained Language Models

Consistency of a model -- that is, the invariance of its behavior under meaning-preserving alternations in its input -- is a highly desirable property in natural language processing. In this paper...

arxiv.org

1

0

35

Yanai Elazar

@yanaiela

2 years

As an AC, I read reviews that are worried about incremental contributions and about novelty - this simply doesn't mean anything

4

0

36

Yanai Elazar

@yanaiela

7 months

On my way to Munich to give a talk tomorrow at LMU. Come to my talk if you're around! DM if you wanna hang out

3

1

36

Yanai Elazar

@yanaiela

2 years

classic

0

34

Yanai Elazar

@yanaiela

10 months

This is fantastic news!! Somewhat of a coincidence, but our paper that studies the effect of early arxiving on acceptance that suggested this effect is small and that it does not fill its purpose was accepted to CLeaR (Causal Learning and Reasoning) 2024

Graham Neubig

@gneubig

10 months

ACL has removed the anonymity period. This means that ACL submissions can be posted and discussed online at any time, although extensive PR is discouraged.

5

86

353

0

2

34

Yanai Elazar

@yanaiela

4 years

So apparently I can talk about the future of AI now.. Join me tomorrow to listen to me (from the past) talk about commonsense reasoning. 12:55 Israel time (in 13 hours).

1

2

34

Yanai Elazar

@yanaiela

5 years

Unsatisfied with adversarial training? the difficulties of training it, and making it work in practice? We've got your back! INLP: Iterative Nullspace Projection, a data-driven algorithm for removing attributes from representations. the paper -

Shauli Ravfogel

@ravfogel

5 years

Our paper on controlled removal of information from neural representations has been accepted to #acl2020nlp 🙂 A joint work with @yanaiela , @hila_gonen , Michael Twiton and @yoavgo (1\n)

5

25

124

1

7

31

Yanai Elazar

@yanaiela

2 years

We made our workshop open! What does it mean? We publish all our material, including - the proposal - the emails we drafted for reviewers/invited speakers/etc., - tasks (which we update in real time!) - and more Check it out:

Big Picture Workshop - Open Workshop

Open sourcing the workshop

www.bigpictureworkshop.com

Yanai Elazar

@yanaiela

2 years

Papers presented at conferences often offer a limited view of a researcher's vision. At EMNLP23, you have the chance to present "The Big Picture" - a workshop dedicated to telling the big picture behind individual papers, often known mainly to a student/PI

4

17

149

3

6

32

Yanai Elazar

@yanaiela

3 years

preparing a paper for a submission, or "how to exclude information that will make the reviewers ask the wrong questions"

0

1

32

Yanai Elazar

@yanaiela

3 years

Internships are like dropout from your advisor. You get to learn other skills for communicating and doing research

2

0

31

Yanai Elazar

@yanaiela

2 years

Remember BERT? RoBERTa? They were once the best models we had. And while it seemed they weren't great at generalization, we found that simply training them for longer, or using their larger versions, dramatically increases their generalization on OOD (while no diff for in-domain)

Elron Bandel

@ElronBandel

2 years

New paper! ✨ Everyone knows that when increasing size, language models acquire the skills to solve new tasks. But what if I tell you they secretly might also change the way they solve pre-acquired tasks?🤯 #emnlp2022 with @yoavgo & @yanaiela

2

27

103

0

5

31