Exciting personal update!! I am thrilled to share that I defended my PhD
#PHDone
🎉
Next steps: Young Investigator/Postdoc
@allen_ai
. I’m super excited to work with
@YejinChoinka
and the
@ai2_mosaic
team, as well as to join the vibrant NLP community in Seattle 🗻✨🧋☕️
Come intern with us
@ai2_mosaic
next summer!!
I am recruiting interns interested in
📋evaluating generative models
🔥robustness
🔎model interpretability
🥸 privacy
If this interests you, please apply by **November 6th**!
Retweets/shares appreciated 🥰
The Mosaic team at AI2 is looking for interns for Summer 2024. The deadline for applying is November 6th. If building and researching machines with commonsense intelligence while working with awesome mentors sounds exciting to you, apply at
✈️I'm in Seattle for
#NAACL2022
! Would love to meet and chat about NLU evaluation, robustness, trustworthy NLP, interpretability and model analysis, NLP with Friends, grad school, coffee, or anything else☕️🧋. I love to make friends, please say hi if you see me! :)
I was selected as a "Rising Star in Data Science", and was at the workshop at
@DSI_UChicago
last week! The campus was beautiful, and lovely to meet old friends and make some new ones😊 Check out their work here:
I am recruiting a PhD research intern at
@ai2_mosaic
!
Topics of interest include:
📋generative model evaluation
🔥robustness
🔎model interpretability
🥸 privacy
✨Deadline: Nov 6 (today!!)✨
The Mosaic team at AI2 is looking for interns for Summer 2024. The deadline for applying is November 6th. If building and researching machines with commonsense intelligence while working with awesome mentors sounds exciting to you, apply at
What a great
#NAACL2022
!! I couldn't say a proper goodbye to everyone, but it was so meaningful to meet in-person after the long pandemic years.
(Also got
#dalle2
access last week!)
✈️🇸🇬I'm in Singapore for
#EMNLP2023
!
Would love to meet and chat about NLU evaluation, robustness, trustworthy NLP, interpretability and model analysis, privacy, coffee, life, or anything else🍜☕️
I love to make friends, please say hi if you see me! :)
Our NLP systems are getting more accurate—are they also getting useful?
To properly evaluate, we need to consider context(s) of use. New
#EACL2021
paper on QA evaluation w/
@siddalmia05
@maria_ryskina
Florian Metze
@ehovy
Alan Black
All the details: 👇
This peaceful room was one of my favorite places
@SCSatCMU
. This is not ok.
Image description: CMU SCS reading room with "No Women Allowed" scrawled on the glass door
In the coming days, I’ll be joining the
@aclmeeting
microbloggers on the new “Interpretability and Analysis of Models for NLP” track! Looking forward to covering thoughtful work aimed at *understanding* model behavior🤖
#acl2020nlp
#acl2020
It was a delight to host
@johnhewtt
today
@NLPwithFriends
. Thanks for a very *clarifying* talk and interesting discussion. And thanks to all our friends for attending!🤗
We are skipping next week, but join us on Sep 23 for
@davlanade
's talk on NLP for African Languages! 🌍🥳
✈️🇲🇽 I'm in Mexico for
#NAACL2024
!
Would love to meet and chat about NLU evaluation, interpretability and model analysis, data attribution and factuality, privacy, robustness, food, life, or anything else 🌮☕️
I love to make friends, please say hi if you see me! :)
Happy 2022!🎉 Do join us for this talk, where I will discuss:
🔎 My research on how we should analyze and evaluate NLP models
🗣️ My experiences doing a Ph.D. in NLP
You can ask me anything in the QA session!
#NLProc
#Women_in_NLP
I'm excited to announce the first
#women_in_nlp
talk in 2022 with Abhilasha Ravichander (
@Lasha1608
). She is going to tell us about her research and share her experience as a Ph.D. student
@SCSatCMU
. Join us on 21/01/2022 at 18:00 GMT at .
#NLProc
@dair_ai
(Very) late to the Twitter party, but excited to present work w/i
@boknilev
and Ed Hovy
“Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?”
looking at the information probes recover as they relate to task usefulness
#EACL2021
Our work on building EQUATE, a new benchmark evaluation platform for numerical reasoning in NLI is finally out! This is joint work with
@arnaik19
, Carolyn Rose and Ed Hovy, to appear at
@conll2019
. Paper: . Code/data: (1/9)
Check out our new work
@allen_ai
on analyzing web-scale corpora!
Super excited about this large-scale effort to actually understand what is in the data that powers language models🔍🧐
Catch our work on context-driven QA evaluation at
#EACL2021
! We study the effect of ASR, MT and Keyboard interfaces to QA systems.
Link:
Time: Apr 21, 9-11 AM EST (Gather-1A) or 8 AM EST (Zoom-2C)
Website:
Discussion welcome!
I won't be attending
#EMNLP2022
in-person due to visa issues (🥲), but if you are interested in negation, robustness and contrastive data collection, catch
@amarasovic
presenting CondaQA at Poster Session 10, Saturday, Dec 10 at 11 AM.
💡Can LLMs like GPT-4 reason creatively? Excited to share our latest research on AI and creativity! 🚀 Introducing MacGyver: a new playground for everyday innovation and physical reasoning --we collect problems to trigger unconventional usage of objects and innovative solutions.
If you are applying to 🎓🤖grad programs in AI🤖🎓, here are three great resources:
1. Student perspectives on applications:
2. Example SoPs from recent applicants:
3. The CMU application mentorship program:
Looking for potential emergency reviewers for submissions in Interpretability and Model Analysis/NLP Applications! Topics include: LLM Hallucination, Alignment, Privacy.
Please reach out if you have the bandwidth to help!🙏
#NLProc
#ACL2024
New paper "EIGEN" (), a framework that leverages large-scale language models to automatically generate event influences.
An incredible effort led by
@aman_madaan
, starting from a *course project*
(P.S. He's applying to PhD programs soon 😉)
Our new paper "EIGEN: Event Influence GENeration
using Pre-trained Language Models" with
@aman_madaan
@dheerajgopal
Yiming Yang,
@Lasha1608
& Ed Hovy explores the use of language models for generating event influences.
Paper:
Code:
Thanks to
@probablyNitika
for a very interesting conversation at
@NLPwithFriends
last night!
If you missed the talk, you can catch it here:
Don't miss
@sjmielke
next week, telling us about comparing generative LMs and anger-driven development! 💯
How often do SoTA LLMs generate incorrect information on user-queried topics? To find out, we had LLMs produce information on entities extracted from WildChat. We think that accurate knowledge of these entities is necessary for correctly answering related user queries.
Introducing AI2 𝕎𝕚𝕝𝕕𝔹𝕖𝕟𝕔𝕙 ! We aim to benchmark LLMs with challenging tasks from real users in the wild. 🤗 Link:
🤩 What great features does it offer? 🌟x9 ⬇️
🌟1. 𝐂𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐢𝐧𝐠 & 𝐑𝐞𝐚𝐥: We carefully curate a collection of 1024 hard
We are very excited to announce our next speaker!!
🗣Nitika Mathur(
@probablyNitika
), telling us about "Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics"
🗓August 26th, 23:59 UTC
📝Sign up:
Hello
#NLProc
twitter! We are NLP with Friends, a friendly online NLP seminar series where students present ongoing work.
Join us to meet our amazing speakers, learn about cool research, and collectively think about what’s next:
See you soon! ❤️
The "Interpretability and Analysis of Models for NLP" track at
#EMNLP2023
is in need of emergency reviewers!
If you are able to review at least one paper by Aug 10th, please DM me 🙏
#NLProc
New quarantine hobby: since I spend so much time on
@arxiv_cscl
, I decided to make a periodic thread on recent interesting papers (i.e past 24 hours🙃). Largely biased towards natural language understanding (May 7, 2020):
We are very excited to announce our next speaker!!
🗣Tom McCoy(
@RTomMcCoy
), telling us about "Universal Linguistic Inductive Biases via Meta-Learning"
🗓August 12th, 14:00 UTC
📝Sign up:
Keep up to date with talks at
Can we sanity-check our NLI Models? New Paper at
#COLING2018
on "Stress Test Evaluation for Natural Language Inference" : . Joint work with
@LTIatCMU
@isrcmu
researchers, evaluation exercises neural NLI models on large-scale stress tests of their failures:
This list of tips to shorten manuscripts is what I use as well! Another one after applying the others: comment out entire paragraphs and rewrite from scratch more concisely, while trying to preserve content. This can sometimes be more effective than editing a few words at a time.
With the ICLR deadline coming up, a few people asked me for tips on cutting down papers to fit the page limit. Figured it might help others too, so here are my tips, which I apply by the order they are written >>>
#emnlp2020
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
Najoung Kim, Tal Linzen
Paper:
Talk:
@najoungkim
,
@tallinzen
/1
@rctatman
Non-exhaustive list of studies I liked: 1) BERT Rediscovers the Classical NLP Pipeline by
@iftenney
et. al, 2) Still a Pain in the Neck by
@VeredShwartz
&Dagan 3) How Contextual are contextualized word representations by
@ethayarajh
and a nice survey:
@kelina1124
@LTIatCMU
@ETH
I am really sorry to hear this happened, and that proper support for handling the situation institutionally was not available. Thank you for sharing this story. I hope our community can discuss how to do better
#NLProc
Thanks to
@emilymbender
@ojahnn
@zibuyu9
for this initiative, particularly for emphasizing first-time authors. I would like to add that if you're from any underrepresented group in NLP and would like coverage of your paper, please reach out! I would love to amplify your work ❤️
This talk was SO GOOD. Thanks to
@SebastinSanty
and
@Roprajo
for speaking with us, and to all friends who joined in for a fascinating discussion!! The video is now up
We are off next week for EMNLP, but we'll be back on Nov 25th with
@feralvam
#NLProc
We are super excited to announce our next speakers!!
🗣️ Pratik Joshi(
@Roprajo
) and Sebastin Santy(
@SebastinSanty
), telling us about "The State and Fate of Linguistic Diversity and Inclusion in the NLP World."
🗓️ November 11th, 14:00 UTC
📝 Signup here:
We are two days from our first talk with the amazing
@lena_voita
!!
Help us make this the friendliest space in NLP — check out our community guidelines and code of conduct before the talk 🥳
Super excited that we'll be hosting John (
@johnhewtt
) next week! Come join us, friends 🥳
#NLPWithFriends
#NLProc
P.S. We won't be recording this session, so remember to register if you plan to attend! Link:
Very thankful for the chance to give this talk! Students interested in understanding neural representations of language, I’d love if you came and gave your thoughts and perspectives on this ongoing work on the probing methodology.
Nominations open for the next edition of
@NLPwithFriends
! 🎉
Nominate yourself, your friends, your colleagues, your students, or anyone else you would like to hear speak🤗♥️
NLP With Friends is opening nominations for our Fall 2021 series!!
We want to hear from YOU — nominate people you’d like to hear from, self-nominations accepted and encouraged!
🗓️Nominations close July 15
📝Nomination form:
🎙️Talks return on Aug 11!
I am very excited about our initiative to promote the work of marginalized researchers in our community at
#NAACL2022
! If this applies to you, we would *really really really* love to feature your work.
Please fill in your details here by ✨July 3✨:
NAACL D&I is starting a new initiative to promote the work of marginalized researchers in our community. If this is applicable to you and you are presenting at NAACL 2022, we would love to feature your work! Please fill in your details here by July 3:
It is somehow hard to say enough good things about Ana ❤️ Future grad students: I think Ana will be an amazing and kind advisor, and you should definitely consider applying to her group!
I'm recruiting students!
My interests include measuring usefulness of explanations for human-AI collaboration, addressing human factors that confound such measurements, & modeling interactive explainability (multimodality, few/zero-shot learning, dialogs, personalization, etc)
#ACL2023
#ACL2023NLP
track "Interpretability and Analysis of Models for NLP" is missing 20% of assigned reviews 🥲
Please please please, if you can review even 1 paper until March 16, fill out this form: 🙏
Thanks
@pmichelX
for joining us
@NLPwithFriends
, and for a very interesting conversation! In case you missed it, the recording is now up:
Next week we have a double feature,
@Roprajo
and
@SebastinSanty
telling us about linguistic diversity in NLP
#NLProc
Another week, another friend. Next Wednesday we will be joined by Paul Michel!!
🗣Paul Michel (
@pmichelX
), telling us about "Modeling the Second Player in Distributionally Robust Optimization"
🗓November 4th, 14:00 UTC
📝Sign up:
Nominate speakers for the next edition of
@NLPwithFriends
!
Thanks, friends--- the 2020 edition was great, and we hope to have a diverse set of excellent speakers for 2021 as well. Please feel free to DM us if you have any questions about nominations, or feedback on the form❤️🤗
🗣️Who is our next speaker?
We want to hear from YOU — we’re opening nominations for the 2021 edition of NLP with Friends!! We encourage you to nominate people you’d like to hear from, and also yourselves❤️🤗
🗓️Nominations close Sunday, January 10
Aakanksha Naik
@arnaik19
& Abhilasha Ravichander
@Lasha1608
: "Stress Test Evaluation for Natural Language Inference". NLI requires modelling of quantification, compositionality and many more. Paper:
#coling2018
#NLProc
Attending
#EACL2021
? Please come chat if you are interested in probing, or model diagnosis, or probing as a tool for model diagnosis!
🕑Time: 14:00 CET (on Zoom), or 15:00-17:00 CET (on Gather)
📜Paper:
🚀Rocketchat:
(Very) late to the Twitter party, but excited to present work w/i
@boknilev
and Ed Hovy
“Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?”
looking at the information probes recover as they relate to task usefulness
#EACL2021
I've been debating just how candid I should be about my excitement for this upcoming talk. I landed on being forthright.
@sulin_blodgett
's work is IMO some of the most important work that's being done in
#NLProc
these days. I literally cannot wait for this conversation.
🗣️Who is our next speaker?
We want to hear from YOU — we’re opening nominations for the 2021 edition of NLP with Friends!! We encourage you to nominate people you’d like to hear from, and also yourselves❤️🤗
🗓️Nominations close Sunday, January 10
Introducing TLDRs on Semantic Scholar, now available in beta for ~10M Computer Science papers!
These auto-generated extreme summaries help you decide which papers are most relevant to your work.
Learn more:
#TLDR
#sciencetwitter
This work began in a wonderful internship
@allen_ai
. I could *not* ask for more brilliant or empathic mentors than
@nlpmattg
@anmarasovic
.
Many more results and analysis in the paper. We welcome discussion!
📝Paper
🚀Data [8/8]
@NLPWithFriends
is coming back this spring, and will continue to be a friendly student-run space to discuss research. I am so excited about this organizing team!🥳
We’re thrilled to announce NLP with Friends will return this spring with an all-new cast of student organizers!! 🎓🥳
@zhaofengw
@sumanthd17
@BrihiJ
@kayo_yin
Stay tuned for news very soon… 👀
This job search was complicated for multiple reasons. I am deeply grateful to everyone who shared their stories with me, and took the time to give me advice that was helpful for making decisions. I promise to pass it forward.
A nice article on some risks of marginalizing applications research "They(ML researchers) evaluate a model’s performance using metrics that don’t translate to real-world impact, or they choose the wrong target altogether."
AI/ML tweeps, here is a must read.
It’s a wonderful article questioning whether we are spending our valuable brain cycles on the right research questions.
If you disagree, I’d love to hear why. If you agree, what can be done to accelerate this?
Excited for this talk!! Come join us, and check out their
#acl2020nlp
paper if you haven't already-
The State and Fate of Linguistic Diversity and Inclusion in the NLP World
Pratik Joshi*, Sebastin Santy*, Amar Budhiraja*, Kalika Bali, Monojit Choudhury
We are super excited to announce our next speakers!!
🗣️ Pratik Joshi(
@Roprajo
) and Sebastin Santy(
@SebastinSanty
), telling us about "The State and Fate of Linguistic Diversity and Inclusion in the NLP World."
🗓️ November 11th, 14:00 UTC
📝 Signup here:
The deadline for submitting an affinity workshop proposal at NAACL has been extended to ✨Jan 31✨! Do consider submitting. We are excited for this initiative, and its potential to help support and grow marginalized research communities.
Announcing call for affinity workshops for NAACL 2022. The goal here is to establish research communities out of affinity groups and provide opportunities for folks from these communities. Deadline Jan 20. More details here:
Finally, please please DM me if you are based in Seattle and would like to chat about things research-related or otherwise! I recently moved to the city and would love to meet people here!🤗
Our first speaker will be Lena Voita (
@lena_voita
), talking about Information-Theoretic Probing with Minimum Description Length
#NLProc
Paper :
Event details: 👇
We are very excited to host our first speaker in August!!
🗣Lena Voita (
@lena_voita
), telling us about Information-Theoretic Probing with Minimum Description Length
🗓August 5th, 14:00 UTC
📝Sign up:
Keep up to date with talks at
Come catch our oral session at
#EACL2021
, April 23 14:00 CET, or poster at 15:00 CET. This work fits into a broader question of how to understand and diagnose models, so please reach out if you are interested in creating better evaluations and new model analysis techniques!
I think there are lots of cool contributions here:
- Contrastive data
- An evaluation for negation
- Lots of careful “data work” (collecting good data is tons of effort!)
- Crowdsourcing of nuanced linguistic phenomena (“change the scope of the negation”)
- Strong baselines [7/8]
I’m absolutely buzzing after the
@NLPwithFriends
social! It was great to see a very diverse group of people (in terms of geography, academic progression and nationality) talking about research, issues of diversity and the politics of tech and research! 🥰
@TharinduDR
@coling2020
Hi sorry for reaching out unsolicited, but I think this may potentially de-anonymize your paper since reviewers generally have access to paper ID. I just wanted to let you know in case it caused any issues, hope it's not disrespectful!
I thought the first thing I would do with DALLE-2 would be evaluation design. There are a lot of questions here.
Turns out,
#dalle
as a 🖌️stochastic paintbrush🖌️ is really fun😄 I never know what I'm going to get but I'm here for it!
Anyway, here's The Great Wave off New York.
On Friday I had the pleasure to present our work on MCQA artifacts for MASC'24
@jhuclsp
. We also won a best paper award! 🌟
Huge thanks to my amazing advisor
@rachelrudinger
, my collaborator
@lasha_nlp
and everyone
@ClipUmd
! And thanks to
@orionweller
for hosting a great event!!
Thanks also to my wonderful thesis committee Ed Hovy, Norman Sadeh, Alan Black, and
@sameer_
. I learned so much from you and am honored you gave me your time and mentorship!
It is absolutely heartwarming to see names added to this spreadsheet. Makes me feel really happy about our community 🤗❤️ (And now I think we should have this even for non-virtual conferences).
If you are a faculty member in NLP/CL and would be willing to talk to students during ACL2020, please consider adding your information here. Hopefully, this will make faculty more approachable for new members to the field.
(h/t to
@zehavoc
for the priming)
#acl2020nlp