๐ฆ How to avoid a future COVID pandemic? ๐How to detect epidemics early?
Excited to share our latest work SPEED - an Event Detection framework to extract epidemic-related events from Tweets and can provide breakthrough early warnings for the unseen epidemic of Monkeypox! ๐คฏ๐ฑ
Excited to share that Iโll be starting my internship at
@AIatMeta
in the GenAI team from tomorrow! Iโll be based off in Bellevue for the next 3.5 months.
Please say a hi if you are around, would love to catch up! ๐๐ป
๐ข Introducing GENEVA: Generalizability Benchmarking Dataset for Event Argument Extraction, a super diverse and high coverage EAE dataset with 115 event types and 220 argument roles.
Paper:
Github:
Accepted at
#acl2023
main (1/n)
๐๐จHow to improve multilingual performance on structured prediction tasks?
Excited to share our latest work CLaP - a label projection technique utilizing LLMs to do contextualized machine translation, improving two tasks in 47 languages including 10 extremely low-resource ones!
Very excited to share our
@conll_conf
work done with
@emilypahn
, Yulia Tsvetkov, and Alan Black. We study how machines can influence human behavior in code-switched conversations by means of linguistic entrainment.
#conll2020
#emnlp2020
๐๐ Will be honoured to speak at
@USC_ISI
about our societally impactful application of epidemic prediction through event extraction.
๐จJoin in to know how we can prevent any future pandemic!!
๐ข Next Thursday (5/9) is ISI's last NL seminar for this spring semester and we have Tanmay Parekh
@tparekh97
from UCLA giving us a talk on "Event Extraction for Epidemic Prediction"
The talk will be at 11AM PST and is open to the public via Zoom, please email us ahead of time
Good morning NAACL folks! Today, Iโll be presenting my work on a simple and cute way of utilising LLMs for multilingual data augmentation for structured prediction task! Come check out @ 9am in the poster session!
#NAACL2024
๐๐จHow to improve multilingual performance on structured prediction tasks?
Excited to share our latest work CLaP - a label projection technique utilizing LLMs to do contextualized machine translation, improving two tasks in 47 languages including 10 extremely low-resource ones!
I will be at
#NAACL2024
next week. Iโll be talking about two of my event extraction related works: SPEED and CLaP. Looking forward to meeting the community! ๐ฒ๐ฝ
๐ฆ How to avoid a future COVID pandemic? ๐How to detect epidemics early?
Excited to share our latest work SPEED - an Event Detection framework to extract epidemic-related events from Tweets and can provide breakthrough early warnings for the unseen epidemic of Monkeypox! ๐คฏ๐ฑ
๐ฆ How to avoid a future COVID pandemic? ๐How to detect epidemics early?
Excited to share our latest work SPEED - an Event Detection framework to extract epidemic-related events from Tweets and can provide breakthrough early warnings for the unseen epidemic of Monkeypox! ๐คฏ๐ฑ
๐๏ธAnd thatโs a wrap! Thank you everyone for travelling or driving to Los Angeles/
@ucla
and
#SoCalNLP2023
! It was a fun day with great discussions, networking and some gossip strewn in from recent news ๐คญ
See you all next year!!!
Working on event extraction?๐ง Check out our latest event extraction benchmark project! We propose ๐๐๐ญ๐ฉ๐๐, a standardized, fair, and reproducible benchmark for event extraction.
Webpage:
Github:
๐ข This is a great opportunity to publicise your work, meet potential collaborators, and network!
You can simply submit a 2-page in-progress work or an already accepted/under review work as well.
#SoCalNLP
#CallForPapers
๐ข Attention
#NLPoc
community!
Submit and showcase your research at the 4th Southern California Natural Language Symposium (SoCal NLP) ๐
๐๏ธ Submission Deadline: Oct. 21, 2023, 11:59 PM PT
๐ More info:
#SoCalNLP
#CallForPapers
Having worked with
@kuanhaoh_
for 3 years, I can vouch for his experience and long-term vision. I also greatly admire his coding skills.
For all new students, will strongly suggest to consider talking to him once for future PhD prospects!
I'll miss
#NAACL2024
next week due to my visa status. However, feel free to talk to
@tparekh97
about our latest work on event extraction and cross-lingual transfer! I'm also seeking students to join my group at TAMU. Interested? Fill out this Google form!
๐ขCan you generate long multi-scene videos from existing text-to-video (T2V) generative models? Happy to introduce Time-Aligned Captions (TALC), a fundamental framework that allows multi-scene video generation without any further training ๐คฏ
Paper:
๐งต๐
๐จCan existing video generative AI ๐ฅ replace graphic designers in Hollywood ๐ฌ? The answer is NO ๐ซ
๐ข We present VideoPhy, and find that the generated videos lack physical commonsense โ making them poor physical world simulators ๐ค
Paper:
๐งต๐
๐ข We propose a new benchmark called PhonologyBench for testing how well LLMs perform on three tasks that require sound knowledge : phonemic transcription, counting syllables, and listing possible rhymes.
w/ Harshita Khandelwal,
@VioletNPeng
1/3
Very excited to share our
@conll_conf
work done with
@emilypahn
, Yulia Tsvetkov, and Alan Black. We study how machines can influence human behavior in code-switched conversations by means of linguistic entrainment.
#conll2020
#emnlp2020
This course was a great success. Covered a broad range of topics complemented by in-depth class discussions. The guest lecturers were the icing on the cake. Enjoyed working with the team and it was a big learning experience indeed!
We have finished uploading our 23 class videos on Multilingual NLP:
Including two really great guest lectures:
NLP for Indigenous Languages (by Pat Littell, CNRC):
Universal NMT (by Orhan Firat, Google):
How to pick a good number of visual tokens? Too few, you have poor performance; too many, you need quadratically more compute.
In this work, we introduce a model that works with an elastic number of tokens.
arXiv:
๐ข ๐ฝโ๏ธWe introduce VideoCon, a video-text dataset for training SOTA alignment model. It resolves a typical issue in video-text alignment models that struggles with robustness.
w/
@YonatanBitton
, Idan Szpektor,
@kaiwei_chang
,
@adityagrover_
๐งต 1/
6/N โฌ๏ธLarger LLMs: Utilizing larger LLMs for CLaP strongly improves the low-resource languages, while performance for medium-to-high-resource languages remains similar. This highlights how CLaP can continually improve with improved and more advanced LLM capabilities.
5/N ๐Epidemic Prediction: Using events detected by our model, we provide epidemic warnings 4-9 weeks before the number of cases peaked and WHO issued a global health concern warning.
We also show that our model predictions are relevant to the Monkeypox epidemic.
I am honored to be nominated by SIGDAT (the org that oversees EMNLP) to run for VP-elect with other awesome candidates who share the goal of improving our community. Please check your email to vote by 3/24.๐ณ๏ธ See details:
3/N ๐ Event Detection: Our trained models are better than any other data-transfer or zero-shot model. Furthermore, we show significant improvements over the previously utilized keyword-based and the new LLM-based methods.
5/N ๐Better than LLM Inference: Compared to utilizing LLMs directly for inference on the target languages, we observe that LLM usage in CLaP provides significantly stronger improvements up to 66 F1 points (for Bengali).
2/N ๐ Dataset: We simplify existing epidemiological ontologies into seven core events frequently discussed on Twitter. Six expert annotators annotate to create our dataset of 2.2K COVID-based epidemic events. We evaluate in a cross-disease setting on Monkeypox, Zika, and Dengue.
1/N โFormulation: We introduce a new paradigm of contextual machine translation (i.e. translation with a context) to do label projection by emphasizing the label translation to be conditional on the translated sentence to ensure better consistency.
One hub for all Event Extraction needs! Do check out our extensive benchmarking. We also tried five different LLMs and they perform poorly at this task.
Also, it's completely open-source, so feel free to benchmark and add your models to the hub!
Working on event extraction?๐ง Check out our latest event extraction benchmark project! We propose ๐๐๐ญ๐ฉ๐๐, a standardized, fair, and reproducible benchmark for event extraction.
Webpage:
Github:
In our analysis, we find that users find the code-switching agent to be more engaging. Further, making the agent informal by adding some discourse markers makes it more native to the users.
7/N ๐ก Extremely Low Resource Languages: CLaP also has the potential to improve extremely low-resource languages, as evidenced by our experiments on NER for 10 languages from Africa and South America.
4/N โณ Early Detection: In fact, our COVID-only trained model performs better or at par with models trained on limited target epidemic data. This signifies the utility of our framework for early epidemic detection when no or limited target epidemic data is available.
3/N ๐ Evaluation: Intrinsically, CLaP-generated data provides the strongest accuracy while maintaining good faithfulness. This highlights the strong data generated using CLaP.
We develop a generalized bilingual dialogue framework for data collection which can be utilized for other code-switching pairs too. We use AMT to collect the data for Hindi-English dialogues and release the dataset for future research.
2/N โ๏ธ Model: We model contextual machine translation through large language models (LLMs) by incorporating the constraints as natural language instructions into the model and using two in-context examples. We use a small primarily English Llama2-13B model for our work.
๐ขHumans are capable of providing preferences in similar, but non-identical contexts! So why restrict LLMs? Excited to introduce DOVE ๐๏ธ, a new objective for aligning LLMs that optimizes preferences over joint instruction-response pairs.
Paper:
6/N ๐ Disease Profiling: We also show the utility of our framework for disease profiling by mapping the proportions of events detected. These profiles are correlated with various recent findings of these diseases.
1/N โFormulation: We first re-formulate epidemic prediction as an event detection task by aggregating the number of detected events over time. We characterize early warnings by a sharp increase in the number of detected events.
Dear all,
We will extend the submission deadline for
#SocalNLP23
to *Oct 23* 11:59 PM PT. If you have a paper accepted or submitted to recent conferences, you may submit it as is to the symposium
Also, registration is now open
4/N ๐ Evaluation: On downstream tasks, CLaP improves over the best baseline on EAE by 2.4 F1 and NER by 1.4 F1 points averaged over 39 languages.
It's grossly surprising how LLama2-13B primarily focused on English can still understand other languages including low-resource ones!
Utilizing the EAE ontology mappings, we transform FrameNet into GENEVA, a diverse, dense, and high-coverage EAE dataset. We conduct several human validation steps to ensure the high annotation quality and comprehensiveness of our dataset. (5/n)
This work motivates the development of agents to follow similar accommodation when conversing with humans. It also provides more resources for the development of dialogue agents that can cater to code-switched and multilingual communities.
First, we create a large and diverse EAE ontology by exploiting shared properties of SRL and EAE. Through extensive human annotations, we transform FrameNet, an SRL dataset into a large EAE ontology with 179 events and 362 argument roles. (3/n)
Finally, we observe strong trends in accommodation of language choice. Basically, users use English for referring to fruits if the agent uses English for fruits. Also, humans also adapt the style of code-switching to a great extent.
Previous EAE datasets only focused on entity-based arguments. We introduce non-entity argument roles for the first time for EAE, with about 37% of our argument roles being non-entities. (4/n)
Overall, in our work,
โ We build a new comprehensive and diverse EAE ontology
โ We introduce a generalizability benchmarking dataset GENEVA
โ Provide benchmarking results for various models including GPT highlighting the challenges in GENEVA. (2/n)
@Saikallis9012
I remember you telling me this even for planning the day. And it really helps!
Worst case, you finish early and feel good about yourself.
We design four generalizability benchmarking test suites using GENEVA: (1) Low resource, (2) Few-shot, (3) Zero-shot, and (4) Cross-type transfer. They assess models' ability to learn from limited data and generalize to unseen event types and argument roles. (6/n)
Under similar zero-shot settings, the SOTA model DEGREE achieves a mere 39% F1 score on GENEVA while 53% on the ACE dataset. This shows how GENEVA poses additional challenges and can be attributed to the presence of non-entity arguments. (8/n)
We evaluate 8 different models of various model families like traditional classification-based, question-answering-based, generation-based models, and GPT-styled models on the different test suites. We observe that generation-based models provide the best generalization. (7/n)
We also benchmark GPT3.5-turbo on the zero-shot setting where it only achieves an F1 score of 22%. This is significantly lesser than DEGREE's performance of 39% and shows how our dataset is challenging for LLMs in general. (9/n)
DALL-E 3 seems to have reduced gender biases in single-person images.
So, problem solved๐ฅ? NOT YET!๐ค
โ ๏ธWe discovered the UNDERLYING GENDER BIASES in the Text-To-Image Model through a proposed Paired Stereotype Test(PST) evaluation framework.
Full paper:
Delighted to introduce KPEval, a fine-grained semantic-based keyphrase evaluation framework with state-of-the-art reference-based metric and diverse application-oriented reference-free metrics.
Paper:
Toolkit:
๐ New paper alert!
Want to make your LLM's generation more credible and grounded on pieces of evidence?
Our CaLM algorithm, presented at
@ACL
'24, leverages smaller LMs to achieve just that **unsupervisedly**! (1/n)
Hello World! ๐๐ผ
#SoCalNLP2023
is back this year, to be held
@UCLA
! ๐๏ธโ๏ธ
Keep an eye here for more announcements, but for now, our CfP is open! More details โฌ๏ธ
๐
๐Nov. 17, 2023