Ethan Gotlieb Wilcox @weGotlieb Twitter profile

Last Seen Profiles

@AzanAhmedx

@tvQLWCFiEV79654

@oktayokanalpay

@MoldAbdoh

@susanmstapleton

@JShlomo39442

@HeyyJohnn_

@cukienaknikmati

@Sorena375

@EarleAdria80602

@AlanaJoely1816

@DDonicia43086

@Deadfool6969

@abrebaharii

@Vitnhonho

@DrCarolTrim

@IosifThera9040

@BinorRaja

@KlaudiaTer5194

@lakeita41094

@RoxanShant11648

@SebMAS6703

@Sanharan30

@stw_pdg

@jandakembangstw

@LaqueenaRe44438

@vivalachxse

@VenomTech

@arteagahugo

@RJP4UP_

@negapochi

@Kurumi_crewsabu

@sen____erika

@get_you_gang

@Subhash31132700

@13658asmar

Ethan Gotlieb Wilcox

@weGotlieb

11 months

Thank you to #EMNLP2023 chairs for the 😱 two 😱 outstanding paper awards! I am so grateful to have worked on these projects with wonderful colleagues — @tpimentelms (who is the first author on one of the papers!), @clara__meister , @kmahowald and @ryandcotterell

14

335

Ethan Gotlieb Wilcox

@weGotlieb

3 years

What syntactic generalizations can be learned from predicting the next word by domain-general learning algorithms? A paper summarizing the case of filler--gap dependencies (+ theoretical implications!) with @roger_p_levy and @rljfutrell

3

47

207

Ethan Gotlieb Wilcox

@weGotlieb

1 year

🚨🚨New Paper Announcement (to appear in TACL) 📜 from me, @tpimentelms , @clara__meister , @ryandcotterell and @roger_p_levy : Testing the Predictions of Surprisal Theory in 11 Languages 🌎

Testing the Predictions of Surprisal Theory in 11 Languages

A fundamental result in psycholinguistics is that less predictable words take a longer time to process. One theoretical explanation for this finding is Surprisal Theory (Hale, 2001; Levy, 2008),...

arxiv.org

1

17

53

Ethan Gotlieb Wilcox

@weGotlieb

1 year

⚙️ New preprint alert! ⚙️ "Mouse tracking for reading (MoTR): A new incremental processing measurement tool" with @CuiDing_CL , @mrinmayasachan and @LenaAJaeger ! 🐁⚙️🐭

Mouse Tracking for Reading (MoTR): A New Naturalistic Incremental Processing Measurement Tool

We introduce Mouse Tracking for Reading (MoTR) a new incremental processing measurement tool that can be used to collect word-by-word reading times. In a MoTR trial, participants are presented with...

osf.io

3

19

48

Ethan Gotlieb Wilcox

@weGotlieb

11 months

Join me today, at #EMNLP2023 11:00 in PS1, where I will pull off an impossible feat – presenting three posters at the same time! 🎩🌟🎩🌟

1

32

Ethan Gotlieb Wilcox

@weGotlieb

11 months

Is it even possible to present three posters simultaneously? 😱😱 Will he pull it off? 😱😱Only one way to find out – come find me at 11:00 today!

3

1

28

Ethan Gotlieb Wilcox

@weGotlieb

1 year

A project I'm involved in on smaller-scale, cognitively plausible AI was covered in the New York Times today! Thanks @oliverwhang21 for your interest in our work!

The Race to Make A.I. Smaller (and Smarter) (Published 2023)

Teaching fewer words to large language models might help them sound more human.

www.nytimes.com

2

1

24

Ethan Gotlieb Wilcox

@weGotlieb

11 months

Excited for day 1 of #EMNLP2023 ! I’ll be at #CoNLL checking out the #BabyLM posters today and tomorrow. Then presenting work at the main conf, with work on multilingual cognitive modeling, word lengths, and prosody!

1

0

20

Ethan Gotlieb Wilcox

@weGotlieb

1 year

This week at #AMLaP2023 I’ll be presenting two posters about ongoing projects! If you’re around and want to chat, let me know! Excited to connect folks and, of course, eat some Pintxos! 🥘🍢😋 (Teasers for the posters below!)

1

3

21

Ethan Gotlieb Wilcox

@weGotlieb

1 year

Shout out to colleagues @tpimentelms , @clara__meister and @ryandcotterell . Have a look at the poster below, or come to talk me about it at poster PS.3.8 (14:00 - 15:00 on Saturday)

1

2

19

Ethan Gotlieb Wilcox

@weGotlieb

6 years

The goat that the cat that the dog that the stick that the fire that the water that the ox that the butcher that the Angel of Death that the Holy One, Blessed be He smote killed slaughtered drank quenched burned beat bit ate was purchased by my father for two zuzim.

3

8

34

Ethan Gotlieb Wilcox

@weGotlieb

1 year

🔔New Preprint Alert🔔 Why can some presuppositions be used to introduce new information into a discourse while others cannot? Check out“Presupposing Novel Information: A Cross-Trigger Experiment in English” together with @TeaAnd_OrCoffee and @roger_p_levy

1

2

18

Ethan Gotlieb Wilcox

@weGotlieb

5 years

2019 #BlackboxNLP paper is out! Neural LMs trained on English can suppress and recover syntactic expectations, approximating stack-like data structures; but recovery is imperfect! With @roger_p_levy @rljfutrell

0

5

13

Ethan Gotlieb Wilcox

@weGotlieb

11 months

New work quantifying the redundancy between prosody and text to be presented at #EMNLP . So excited to see where this research takes us!

Lukas Wolf

@lukaswolf_

11 months

Ever wondered whether prosody - the melody of speech 🎶 - conveys information beyond the text or whether it's just vocal gymnastics? 🏋️‍♂️ Great work by @MIT and @ETH_en soon to be seen at @emnlpmeeting

2

18

76

0

1

13

Ethan Gotlieb Wilcox

@weGotlieb

1 year

🖱️🐁👀Check out the poster below, or feel free to come talk to me and Cui about it at poster PS.2. 24 (15:00 - 16:30 on Friday)🖱️🐁👀

1

10

Ethan Gotlieb Wilcox

@weGotlieb

3 years

Neural Language Models' incremental predictions under predict human sensitivity to ungrammatical sentences! New #ACL2021 paper with @roger_p_levy and Pranali Vani #NLProc

1

0

7

Ethan Gotlieb Wilcox

@weGotlieb

11 months

Finally, I’ll present work first-authored by @tpimentelms : “Revisiting the Optimality of Word Lengths,” also with @kmahowald , @clara__meister and @ryandcotterell where we ask how lexicons should be optimally structured.

1

0

7

Ethan Gotlieb Wilcox

@weGotlieb

1 year

Congrats @cuier_d !!! Well deserved!

Lena Jäger

@LenaAJaeger

1 year

Congratulations to Cui Ding who was awarded with the Semester Prize by the president of @UZH_ch for her Master's thesis @cl_uzh co-supervises by @weGotlieb on Mouse Tracking for Reading, a low-cost alternative to eye-tracking. Find out more here:

2

30

0

7

Ethan Gotlieb Wilcox

@weGotlieb

6 years

Passover Haggadah provides strong evidence for the non-regularness of natural languages.

1

0

9

Ethan Gotlieb Wilcox

@weGotlieb

3 years

📣 New paper📢 Alexandre Cremers, myself, and @benspect test the influence of prior belief on exhaustivity inferences. This paper combines it all: comprehension / production experiments, lots of RSA modeling, as well as a deep dive into model analysis.

1

0

7

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@thomashikaru I tried more explicit prompting, which worked OK -- "the box moved from the shelf fell on the floor" is technically a gardenpath, but boxes don't really move themselves much, so idk if I would be tricked. Then it just falls apart...

2

0

7

Ethan Gotlieb Wilcox

@weGotlieb

1 year

The piece is all about the BabyLM challenge, which invites folks to train a language model from scratch on the amount of linguistic input available to the typically-developing child. If this sounds interesting, please consider submitting:

1

0

5

Ethan Gotlieb Wilcox

@weGotlieb

11 months

First, we have “Testing the predictions of surprisal theory in 11 languages” with @tpimentelms , @clara__meister , @ryandcotterell and @roger_p_levy . We find remarkable consistency for the predictive ability of surprisal across languages!

1

0

5

Ethan Gotlieb Wilcox

@weGotlieb

4 years

*Alots free hour for ACL review* ... 2 hours later ... "... Aside from the five paragraphs above, could the authors address this list of twelve questions as well as the fifteen missing references I compiled along with their major takeaways..."

1

0

5

Ethan Gotlieb Wilcox

@weGotlieb

1 year

These are really strong results — and and extending these types of experiments to new languages fills a big gap in the literature!

Eyelands Lab/Øy(e)Lab

@oeyelab_ntnu

1 year

New work from Anastasia Kobzeva, @sArehalli , @tallinzen : "Neural networks can learn patterns of island-insensitivity in #Norwegian ", which will be presented at #SCiL2023 Preprint available here: comments welcome!

0

9

13

0

5

Ethan Gotlieb Wilcox

@weGotlieb

3 years

@BlancheMinerva @roger_p_levy @rljfutrell Thanks for your interest! We have a platform for analysis of ANNs, including some transformers, with a larger variety of syntactic structures. Check it out at

1

0

5

Ethan Gotlieb Wilcox

@weGotlieb

1 year

This work dovetails nicely with some current ACL presentations including by @devarda_a , @MarelliMar , as well as recent work on the surprisal/rt relationship by @coryshain and @postylem

Scaling in Cognitive Modelling: a Multilingual Approach to Human Reading Times

Andrea de Varda, Marco Marelli. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2023.

aclanthology.org

1

0

5

Ethan Gotlieb Wilcox

@weGotlieb

5 years

@GettyMuseum I did the #gettychallenge using the most valuable items in my house. Behold! The Mona Beana.

0

4

Ethan Gotlieb Wilcox

@weGotlieb

1 year

My one correction is that “Mr. Mueller” ( @amuuueller ) has leveled up to “Dr. Mueller” 🧑‍🎓 congratulations!

2

0

3

Ethan Gotlieb Wilcox

@weGotlieb

2 years

The most amazing part imo: Models learn different rules for different types of dependencies! When we compare FGD (top) with dependency-like expectations for gendered pronouns (bottom) ... no island effects! See the paper for results also on unboundedness of the dependency!

Roger Levy

@roger_p_levy

2 years

I urge interested readers to consult the @weGotlieb et al. in press: we find that the "GRNN" LSTM of @xsway_ et al. 2018 trained on a childhood's worth of English shows substantial success on filler–gap dependencies and the island constraints on them. 1/3

4

16

74

0

4

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@shota_momma @wavyphd This is right on. As we say in the paper there are two big arguments in favor of nativism: Poverty-of-Stimulus style claims and x-linguistic distribution. Our results provide good evidence against POS for islands.

2

0

4

Ethan Gotlieb Wilcox

@weGotlieb

1 year

Also, shoutout to the previous investigation that HAS looked at these issues from the non-English perspective (in Japanese) @ttk_kuribayashi

Lower Perplexity is Not Always Human-Like

Tatsuki Kuribayashi, Yohei Oseki, Takumi Ito, Ryo Yoshida, Masayuki Asahara, Kentaro Inui. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th...

aclanthology.org

0

4

Ethan Gotlieb Wilcox

@weGotlieb

2 years

Interestingly, #dallemini doesn't seem to have learned much about visual question answering. Images of questions are segments of the question itself.

2

0

4

Ethan Gotlieb Wilcox

@weGotlieb

1 year

We simulate eye tracking in the browser. We blur out a piece of text, except just above the mouse tip. Participants have to move the cursor to reveal and read the text. Then, we analyze their movements similar to eye tracking data. 👀🖱️

1

0

4

Ethan Gotlieb Wilcox

@weGotlieb

4 years

It's a rainy day so I made some math gifs

1

0

3

Ethan Gotlieb Wilcox

@weGotlieb

11 months

In the same theme, we have “Language Model Quality Correlates with Psychometric Predictive Power in Multiple Languages” with @tpimentelms , @clara__meister and @ryandcotterell . We test the relationship between LM quality and cognitive modeling power cross-linguistically.

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

Finally, for 3️⃣ our results are rather striking 😱 When we train generalized additive models to fit the surprisal / reading time relationship they find an essentially linear curve (which is green in this figure). For more in-depth statistical tests, see the paper!

3

0

3

Ethan Gotlieb Wilcox

@weGotlieb

1 year

#2 is an information theoretic treatment of regressions during reading. 📖👀 We ask, how well does pointwise MI between words predict regressions? Also excited for this one because it’s (as far as I know) the first large x-linguistic analysis of regressions out there! 🌎🌍🌏

2

0

3

Ethan Gotlieb Wilcox

@weGotlieb

1 year

We argue that these data support a hybrid theory for some triggers, as elements that both introduce a presupposition and act as anaphors. We also argue that the same mechanism that drives the variation above, also determines when a trigger is obligatory.

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

4 years

0

3

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@thomashikaru Looks like we'll have to keep writing our own test materials for now😭

0

3

Ethan Gotlieb Wilcox

@weGotlieb

1 year

😮Surprisal Theory 😮 posits that a word’s contextual predictability should be related to its processing effort. It’s often argued that the functional form of this relationship should be simple, specifically, if predictability is measured as surprisal, it should be linear ↗️

1

0

3

Ethan Gotlieb Wilcox

@weGotlieb

1 year

#1 is a methods project. We introduce Mouse Tracking for Reading (MoTR) a real-time processing measurement tool that produces reading times similar to eye tracking and runs in the browser. 🖱️🐁👀 Shout out to colleagues @CuiDing_CL , @mrinmayasachan and @LenaAJaeger

1

0

3

Ethan Gotlieb Wilcox

@weGotlieb

2 years

Interestingly, this was the only prompt I tried for which the model was on the right track. But instead of Lenin, these photos look like ... President Taft??

2

0

3

Ethan Gotlieb Wilcox

@weGotlieb

1 year

For 2️⃣ our results are mixed. Using contextual entropy as a predictor alongside surprisal can help predict reading times, but alone, it is not better than surprisal. For a much more detailed exploration, see another recent publication from this team:

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

Recently, this linear relationship has been questioned, with some studies showing: linear ↗️, super-linear ⤴️and even sub-linear relationships ↘️. But these debates have been based on almost exclusively English experimental data 🇺🇸🇬🇧🇦🇺🇳🇿

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

@Alexfan80136457 We believe it can be a good option when (i) there are budget considerations (eye tracking is expensive) and also (ii) geographic considerations (hard to take precision eye trackers into the field). We hope to validate (ii) in future work!

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

🌎🌏 These validation experiments were conducted in English, but I’m hopeful that MoTR can be used to collect eye-tracking style data in lots of languages! It can also run locally on the computer, and we are excited about possible applications in fieldwork. 🌍🌏

2

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

This paper builds on previous work by Alex Göbel and Nadine bade ( @GT1902 ) , and I want to acknowledge their wonderful previous contributions to these topics!

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

For 1️⃣ our results are a resounding “yes” ! We find that in every language tested, surprisal helped us predicting reading times, both when estimated from a large multilingual language model (top row) and from medium or even smaller-sized monolingual models (bottom rows)

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

Ever want to run an eye tracking study but hampered by the cost? 💸 Maybe you wish you could bring your eye tracker somewhere, but it’s just too darn heavy? 🏋🏽 Well, we might have the solution!

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

We use the multilingual 👁️MECO eye tracking dataset 👁️ and run experiments on 11 languages across 5 language families: Indo-European, Turkic, Koreanic, Afroasiatic, and Uralic.

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

@m_heilb @CuiDing_CL @mrinmayasachan @LenaAJaeger Thanks :) We see a few advantages: the first (and biggest) is that you can observe word skipping and regressions, which SPR and maze won’t get you, in the base case.

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@shota_momma @wavyphd I will say, there are a lot of great, non-nativist theories out there that could explain the "origins" of islands, including processing accounts and the discourse-structural accounts of @adelegoldberg1 and @Benambridge

0

2

Ethan Gotlieb Wilcox

@weGotlieb

12 years

At home with no milk

0

1

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

2️⃣ We collected MoTR reading times for more naturalistic reading, specifically for the Provo Corpus. We find that the MoTR RTs correlate well with the eye tracking RTs in this setting. We also find a (roughly) linear relationship between MoTR RTs and surprisal.

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

6 years

Ever wondered what verbs are mostly transitive, mostly ditransitive? I collected all this information and more and posted it on github. It turns out "wondered" is intransitive 99.09595843251683% of the time.

GitHub - wilcoxeg/verb_transitivity: Data on verb transitivity in English and script to extract...

Data on verb transitivity in English and script to extract transitivity information from Google's syntactic ngrams corpus - wilcoxeg/verb_transitivity

github.com

0

1

2

Ethan Gotlieb Wilcox

@weGotlieb

1 year

@postylem Yeah, good question! My understanding from Appendix B of Smith and Levy (2013) is that these modeling choices didn't change the outcome for their dataset. But that might not be true for our dataset!

1

0

2

Ethan Gotlieb Wilcox

@weGotlieb

12 years

Gnome cupcakes! Gnome life '12! http://t.co/u486JfTk

0

1

Ethan Gotlieb Wilcox

@weGotlieb

12 years

There are two books in the window of the Dartmouth Bookstore; the rest of the space is taken up with flip-flops, t-shirts, trail guides...

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

🐁⚙️🐭 Feel free to get in touch with us if you think MoTR could be useful to answer your research questions! Data, scripts and code to implement MoTR in Magpie can be found at 🐁⚙️🐭

GitHub - wilcoxeg/MoTR: Mouse Tracking for Reading

Mouse Tracking for Reading. Contribute to wilcoxeg/MoTR development by creating an account on GitHub.

github.com

0

1

Ethan Gotlieb Wilcox

@weGotlieb

12 years

Forecast says thunder, so I put on Mahler.

0

1

Ethan Gotlieb Wilcox

@weGotlieb

3 years

One interesting thing is that, in certain situations, some RSA models actually predict anti-exhaustivity effects! That is, “I ordered tea” becomes a cost-effective way to communicate that the speaker ordered both tea and a croissant!

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

@oliverwhang21 @amuuueller Hahah well Aaron defended only last week so its a recent development :) So cool that you can update in real time!

0

Ethan Gotlieb Wilcox

@weGotlieb

12 years

Totalitarian State University

0

1

Ethan Gotlieb Wilcox

@weGotlieb

2 years

Should be LHO, or at least someone other than Kennedy.

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

12 years

My dreams!! http://t.co/ys0JTdHg

0

1

Ethan Gotlieb Wilcox

@weGotlieb

12 years

Thombert the capitalist http://t.co/8wC2hWg6

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

@m_heilb @CuiDing_CL @mrinmayasachan @LenaAJaeger Also, compared to SPR, MoTR appears to be more sensitive (although we compare to the SPR data from Boyce et al., 2020 not to data we collect ourselves).

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

3 years

@LChoshen @roger_p_levy @rljfutrell Hi @LChoshen , apologies for the late reply. It should be indexed now on google scholar

Using Computational Models to Test Syntactic Learnability

E Wilcox, R Futrell, R Levy, Linguistic Inquiry, 2021 - Cited by 64

scholar.google.com

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

4 years

@LChoshen Ah, that tweet had nothing to do with the outcome, only my own extraneous commitment. Copious engagement is the hallmark of all good reviews, both reject and accept!

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

A word’s predictability (a.k.a. surprisal) is suspected to influence its reading time. But almost all studies investigating this are in English! Does it hold up cross linguistically? 🌍🌐🌎

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@mathemagic1an Haha Linguistic Relativity is certainly intuitive and it *has* to be true on a very mechanistic level. The problem for me is that to test it rigorously you'd need to control for culture and language.

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

8 years

I just published “Trump Has Been Obsessed with Nukes for Ages”

Trump Has Been Obsessed with Nukes for Ages

Here’s a partial list of what Trump has said about nuclear weapons: He asked why it would be objectionable to nuke ISIS controlled cities…

medium.com

0

1

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@mathemagic1an So you'd need (i) a population that speaks the language but isn't exposed to the culture and (ii) a population that is exposed to the culture but not the language. Basically, impossible to test rigorously. 🤷

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

A whole range of factors may be at play, including the information content of the presupposed material or whether or not interlocutors trust each other. But in this work we focus on semantic properties and information structure.

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

The thing I love about MoTR is that it’s really simple. But does it work? 🤷🏽To test it out, we conduct two types of studies in English. We were pleasantly surprised at what we found.

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@emilymbender For a deeper history of this, I highly recommend “Seeing Like a State” by James Scott

1

0

1

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@nurikolan @roni_katzir Thanks for tagging me—I will, of course, give this a read!

0

1

Ethan Gotlieb Wilcox

@weGotlieb

2 years

@ahnaphor I screen by asking if the participant considers themself a speaker of language x after the experiment (the hope here being that they won't select `yes` just to participate/get paid.)

0

1

Ethan Gotlieb Wilcox

@weGotlieb

1 year

We ask: 1️⃣ Does surprisal help predict reading times? 2️⃣ Does expected surprisal (or contextual entropy) help to predict reading times? 3️⃣ And what is the functional relationship between surprisal and reading?

1

0

1