We have a postdoc and a research engineer position at FAIR to work on LLM+reasoning/alignment with specific focus on social reasoning, metacognition, interaction and others. Please reach out directly to me if you are interested.
Our team in FAIR (at Meta) is hiring researchers (RS & PostDoc) to work on the broad topics of text and multimodal LLMs.
Location: NY, Seattle or Menlo Park for RS, and Seattle for PostDocs.
PostDoc:
Research Scientist, AI (PhD):
Just got some really good news! I am now an Affiliate Associate Professor at the Paul G. Allen School of Computer Science & Engineering at University of Washington
@uwcse
. Thank you
@YejinChoinka
and
@nlpnoah
, i really appreciate your support!
We surveyed works where LMs are augmented with better reasoning and tools: Exploring integration of reasoning and tool-based augmentation in a fully self-supervised manner can lead to more advanced systems for human-machine interaction.
Excited to share our new resource, 'An Introduction to Vision-Language Modeling'! 📘 Dive into the mechanics of VLMs, from mapping images to language to extending to videos. A collaborative effort to boost understanding and innovation in this promising field. 🚀
📝 New from FAIR: An Introduction to Vision-Language Modeling.
Vision-language models (VLMs) are an area of research that holds a lot of potential to change our interactions with technology, however there are many challenges in building these types of models. Together with a set
Natural language is tied to how humans interact with their environment. Can we build intelligent agents that can learn to communicate in different modalities as do humans? Microsoft researchers are using Vision-Language Navigation to find out:
#CVPR2019
Our new work on evaluating step-by-step reasoning of LMs is on arxiv. This was really hard! So we introduce a suite of metrics that can score LM’s reasoning in an unsupervised way (reference free) with
@OlgaNLP
,
@moyapchen
,
@spencerpoff
,
@LukeZettlemoyer
,
@CMacota99
, Maryam F.
ROSCOE is a first-of-its-kind suite of metrics for scoring step-by-step reasoning. By publishing this study we hope to provide a foundation that enables scalable systematic evaluation and benchmarking of new language models.
See the paper on arXiv ⬇️
🚀 Exciting news! We're open sourcing Chameleon, our early fusion multimodal foundation model from last year. It handles multimodal inputs with text generation outputs, though it was trained for both text and image generation.
#OpenSource
#AI
#ChameleonModel
Today is a good day for open science.
As part of our continued commitment to the growth and development of an open ecosystem, today at Meta FAIR we’re announcing four new publicly available AI models and additional research artifacts to inspire innovation in the community and
Why do i feel like there is always a deadline of some sort (e.g., a paper submission, rebuttal for a paper, a workshop proposal, etc.) when i'm at a conference. 🤔of
@NAACLHLT
@
@EMNLP2020
.
Newly published work from FAIR, Chameleon: Mixed-Modal Early-Fusion Foundation Models.
This research presents a family of early-fusion token-based mixed-modal models capable of understanding & generating images & text in any arbitrary sequence.
Paper ➡️
🚨 New paper! 🚨
We introduce Branch-Solve-Merge (BSM) reasoning in LLMs for:
- Improving LLM-as-Evaluator: makes Llama 70B chat+BSM close to GPT4. GPT4+BSM is better than GPT4.
- Constrained Story Generation: improves coherence & constraints satisfied.
A very clearly presented tutorial on Stylized Text Generation. Esp enjoyed a broad overview of methods and evaluation techniques.
@OlgaVechtomova
#Acl2020
🚀 MemWalker: our new approach to long text processing! 📚 Forget traditional methods! With our unique memory tree structure and LLM prompting, we're outperforming long context, retrieval, & recurrent baselines.
Great work by
@__howardchen
!
Long context models are popular, but is it the final solution to long text reading?
We introduce a fundamentally different method, MemWalker:
1. Build a data structure (memory tree)
2. Traverse it via LLM prompting
Outperforms long context, retrieval, & recurrent baselines. (1/n)
*New from MSR* Introducing our work on Transformers for RL! Working Memory Graphs. Great results on BabyAI with factored observations.
work with
@rickyloynd
Roland Fernandez,
@mhauskn
andAdith Swaminathan
@MSFTResearch
Excited to share Crystal, an LM built for introspective reasoning! Operating on two modes, knowledge introspection & grounded reasoning, Crystal provides answers and the reasoning behind them.
Great work by
@liujc1998
to be presented at
#EMNLP2023
.
Introducing 🔮Crystal🔮, an LM that conducts “introspective reasoning” and shows its reasoning process for QA. This improves both QA accuracy and human interpretability => Reasoning made Crystal clear!
Demo:
at
#EMNLP2023
🧵(1/n)
How can we use
#LLM
to answer a complex question over a chart📊? Check out our new work, DOMINO, a dual-system for multi-step visual language reasoning😎!
📜arXiv:
🧑💻code:
Super excited that our workshop on Methods for Optimizing and Evaluating NLG will be hosted at
@NAACLHLT
2019. Reviewers were excited about our invited speakers so make sure to come find out why! See you in Minneapolis!
#nlg
#neuralgen2019
#nlproc
Consider submitting papers to the workshop on Neural Conversational AI: Bridging the gap between research and real world with amazing list of speakers!
The Microsoft Research Ada Lovelace Fellowship aims to increase the pipeline of diverse talent receiving advanced computing-related degrees. Second year PhD students could receive tuition & a $42K stipend for 3 years. Submit nominations by August 14, 2020:
Reach the finish line of your doctoral thesis work with help from the Microsoft Research Dissertation Grant. Show us the impact of your research and receive up to $25,000 in funding:
Do language models really follow the explanations in prompts? To find out more and read about our new findings and approach check out this 🧵and work led by
@xiye_nlp
,
@ramakanth1729
and others.
We should prompt LMs by selecting 🚨diverse🚨 in-context examples to solve our target task, because diverse examples may have complementary explanations‼️
📄Check out our new preprint
where we look into how explanations work in-context.
Excited to be part of the RoboNLP workshop organization taking place together with Spatial Language Understanding (SpLU). Submit papers soon, it will be a great
@NAACLHLT
2019 workshop.
Spatial Language Understanding (SpLU) & Grounded Communication for Robotics (RoboNLP) are joining forces
@NAACLHLT
2019! Exciting topics (see below) and speakers! See CFP for more details -- Deadline March 6!
#robonlp
#nlproc
The *final schedule* and *best papers* of
@NeuralGen
workshop are out 🎉
Congratulation to the best papers authors!
We are really excited about the amazing line-up of speakers & great posters! Full details on the website:
See you soon in Minneapolis 🏙️
Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet.
Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context
Congratulations to our colleague Lin Xiao
@MSFTResearch
for the
#NeurIPS2019
test of time award!!!
Online convex optimization and mirror descent for the win!! (As always? :-).)
1/4 Batchnorm causes grad explosion in random-init MLP! Can’t fix this by changing nonlinearities! Relu+batchnorm explodes grad norm^2 by >=1.47 per layer, but linear activation minimizes the explosion rate at (B-2)/(B-3), B=batchsize. Our ICLR 2019 paper
Congratulations to the 2019 winners of Microsoft Research PhD Fellowship. Very excited that this year we have very stong winner from NLP area, congrats Ram!
@ramakanth1729
We're excited to announce the 2019 winners of the Microsoft Research Ada Lovelace Fellowship and PhD Fellowship! Explore inspiring projects from talented PhD students in computing-related fields:
The SpLU-RoboNLP 2019 workshop is excited to bring many great invited speakers together to discuss work across language interactions with embodied agents, spatial language understanding, and language-driven human-robot interaction!
Full proceedings:
It’s year 2024, and n-gram LMs are making a comeback!!
We develop infini-gram, an engine that efficiently processes n-gram queries with unbounded n and trillion-token corpora. It takes merely 20 milliseconds to count the frequency of an arbitrarily long n-gram in RedPajama (1.4T
HoloLens 2 provides new capabilities that can be used in conjunction with Research Mode. Specifically, articulated hand-tracking and eye-tracking which can be accessed through APIs while using research mode, allowing for a richer set of experiments.
thanks
@real_asli
for the pictures and for helping organizing the workshop. 😊And finally thanks to
@apple
for providing a pair of airpods pro to each of the authors 🎉
#nlphighlights
105: Question Generation with
@raosudha89
. We discussed an overview of the settings in which you would want to generate questions, and focused on Sudha's work of generating clarification questions. Thanks for joining us, Sudha!
The Deep Learning Group at Microsoft Research AI lab seeks applicants for a postdoctoral research position in the areas of deep learning, artificial intelligence, and related fields.
via
@MSFTResearch
"An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling" good to see more papers on the topic - always start with a CNN before reaching for an RNN. You'll be surprised with how far you can get.
My team has open-sourced a pure python implementation of ROUGE (Apache 2 license) that can be used as a replacement for the original perl version (which also had an ambiguous license).
@harvardnlp
@stanfordnlp
It’s an honor to have been named to the
#ForbesUnder30
Europe list in Science & Healthcare. Looking forward to joining
@ICepfl
@EPFL
in a few months and growing research thrusts in
#NLProc
, knowledge representations, and reasoning.
@sleepinyourhat
@jackiecklo
Maybe change the location? Too late ? ICASSP 2018 was originally going to take place in Seoul, Korea, but was later moved to Calgary, Canada, citing political tensions in North Korea back then.
Current status: missing
#NeurIPS2019
workshops, wearing sweater+overcoat in my ❄freezing❄ Vancouver hotel room, fighting LaTeX to finish our ACL submission. Not the best timing,
@aclmeeting
☹
@cpburgess_
@Azhag
@RishabhKabra
@irinavlh
Great work! The VAE component is much simplified version of attend-infer-repeat paper with less latent variables, and introduces the background component (somewhat similar to using global variable in Disentangled Sequential Autoencoder ).
@adawan919
@AIatMeta
ii. Yes, these are indeed statistical methods - they learn from data by optimizing a loss function, which is a statistical process. But i think the performance of such models also contribute to their ability to learn hierarchical representations or perform transfer learning etc.