We are growing our GenAI team in India and will be
#hiring
machine learning research scientists/engineers at multiple levels. As part of our team, you will work on interesting methods and techniques to efficiently train LLMs and image/video generation AI models on large clusters.
Preparing for an NLP Research interview?🫣
Here is a mind map for all things to cover before the interview. Although this might not cover everything, but it touches on all the topics covering anything related to transformer architecture. Feel free to recommend what I have missed!
Unfortunately our long paper work submitted to
#EMNLP2023
was not accepted-3,3,2,4 scores. Although we were not satisfied with the reviews (also meta-review), we plan to improve it and submit it to EACL. Would have been better if we could contest the final decision
@emnlpmeeting
.
In this ~1 year of ML research job search, I have learned more about transformers/model parallelism/MLSys/Scaling/LLMs in inference/SOTA research than from my internships, research projects, and collaborations. Something positive during these rough times. 📈🤯
Only you and the people you started with know how far you have come. Be grateful, humble, driven and never forget where you aim to go. It's all about the vision and the mindset you have developed throughout your struggle. Always aim to do justice to your potential! ✨
Just submitted my 2nd paper for the year. I guess now is a good time to take a break from "Research, coz I enjoy doing it" and start doubling down on my job hunt! 🥲
PS: I am looking for AI/ML opportunities. Feel free to reach out if you are hiring!🫣
#NLProc
#AI
#ML
To all graduates
My job hunt:
Fail
Fail
Fail
Fail
Fail
.
.
.
Fail
Succeed
Just don’t leave it half way. Like my brother said it to me once: “You only need one conversation. Don’t think about all the rejects you get. Only focus on your next potential offer..”. remove the noise 💭
📢 Excited to share that our work on "Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization" has been accepted at SyntheticData4ML workshop
@NeurIPSConf
🥳
🤔 Can we use hallucinations from LLMs to generate preference data for factuality alignment?
When you just completed your workout and go to the kitchen to get the protein but mom's making paratha and say's "Abhi khale, 10 din baad to ja hi raha hai" 😂🥲
You know what I like about being a researcher? The field is always evolving and that too in my field it’s exponential! You always have something to aim, something to chase and something to be proud of. The best part is, this translates to other parts/habits in my life. ✨
📢 Excited to share our new paper "Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization".
Looking forward to attending
#NeurIPS2023
in person!🥳🤗
📢 Excited to share that our work on "Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization" has been accepted at SyntheticData4ML workshop
@NeurIPSConf
🥳
🤔 Can we use hallucinations from LLMs to generate preference data for factuality alignment?
Project
#2
: LLM Visualization
So I created a web-page to visualize a small LLM, of the sort that's behind ChatGPT. Rendered in 3D, it shows all the steps to run a single token inference. (link in bio)
@osanseviero
@ykilcher
has covered a lot of those concepts in his videos. Also reading the paper after that would make it easier to read coz some papers like RoPE are math heavy and require a bit of intuition at start.
📢 Excited to share that my (w/ Rohan Mittal) new independent paper "NeuralNERE: Neural Named Entity Relationship Extraction for End-to-End Climate Change Knowledge Graph Construction" has been accepted for Spotlight talk by the
@ClimateChangeAI
workshop at
@icmlconf
2021.
1/4
I recently moved to Seattle, so please feel free to reach out if you would like to meet up. Would love to have an in person meet up with the research community here!
My dream is to get an apartment in the middle of the city with a full length floor-to-ceiling window over two adjacent walls. Hoping to quote this tweet if I get it one day! 💭
Delighted to share that my latest lead-author independent research paper \w Rohan Mittal was one of the top 13 papers (of 89 accepted papers) accepted for a Spotlight Talk at
@ClimateChangeAI
workshop
@icmlconf
2021 (Insanely low acceptance rate ~14.9% of the accepted papers).
📢 Excited to share that my (w/ Rohan Mittal) new independent paper "NeuralNERE: Neural Named Entity Relationship Extraction for End-to-End Climate Change Knowledge Graph Construction" has been accepted for Spotlight talk by the
@ClimateChangeAI
workshop at
@icmlconf
2021.
1/4
My first time
@icmlconf
and I already know that I will be tuning in every year!😁
Had some interesting chats with some of the pioneer of the field. Would love to talk to you all and get to know about the work you are involved in. Ping me if you find me in the cafe or socials! 🤗
I have always aspired to work as a researcher in a core AI team focusing on fundamental research since I started doing my undergraduate research. I am very excited and looking forward to contributing to the team with impactful and cutting-edge research!
#NLProc
#LLM
#LLMs
#GenAI
I think I will start summarizing the papers I read here. More of a way to force me read these papers without any procrastination. Let’s see how much I stick to this tweet. More like a way to force me to be up to date with the field while doing research in industry. 😄
I am thrilled to share that my latest independent research work on "STEPs-RL: Speech-Text Entanglement for Phonetically Sound Representation Learning" has been accepted as a full paper for presentation
@pakdd_social
.
Stay tuned for the camera-ready version.
The worst thing you can do as an interviewer is to be condescending. Why bring your personal insecurities/trauma/issues and put it on the person you are interviewing. Just be professional about it.
Some valuable stuff here. It was indeed one hell of a ride for you
@RamanDutt4
. Since I was able to follow your journey firsthand I can say that the way you took those rejections (constructively) was very impressive. I never had a doubt that you won't make it. Happy fr you buddy!
I enjoy giving advice (that nobody asked for) on graduate admissions. Wrote an extensive Reddit post to summarize my observations and some key takeaways -
@PhDchatter
@AcademicChatter
NLP Twitter is getting out of my hands with so many new high-quality papers coming up daily. Keeping myself up to date is getting difficult. Lol yesterday only during one of my interviews for an ML role, I was asked to explain StreamLLM. Luckily I had glimpsed the paper before😪
Resources to learn about Varients of Transformers (Linformer/Longformer/Reformer/Roformer), Positional Encodings (Absosult, Relative, RoPE), Parallelism Techniques (Model/Data parallelism/ZeRO optimizer/Megatron LM). Also, do read the original papers afterward for more clarity.
@osanseviero
@ykilcher
has covered a lot of those concepts in his videos. Also reading the paper after that would make it easier to read coz some papers like RoPE are math heavy and require a bit of intuition at start.
@alienelf
When there is no need to fine-tune the embeddings for any other downstream task. At this stage, the embedding would have captured the overall contextual information it requires to perform any other downstream task with high accuracy.
As a researcher, getting my paper accepted at a conference makes me happy, but the true happiness comes when the first time I see my research idea (just from my intuition) actually works and performs better than the existing approaches. Today is one of those days 😌
One of the best advice I had got on how to start research, is to first start working on your intuition skills, then later learn how to transform your intuitions to into defined concepts. Implement after you have crisp concept definition. Iterate till you observe improvements.
💯 We then use sub-domain embeddings to give a score to each sample in the target domain, representing a relevance score of the sample to all the sub-domain topics corresponding each subdomain cluster.
I suggest making "Graduate Applications Process" a benchmark test for testing patience. Why though?
Patience while waiting for:
-Decision on your research papers: 4-6mo 😕
-Admission decisions: 3-5mo 😥
-New COVID guidelines: 2mo 😨
-Visa appointment: 2mo 😖
Want me to cont.?🥴
My brother
@Prakhar_mishra9
asking us to stop clicking pictures while we were clicking 100s, coz we love him a lot and are super proud of him to join
@Columbia
this spring for his masters.
Ps: did I miss how proud and excited I am? 🤭
Isn’t it addicting to try to be the best version of yourself everyday? Not only the rewards compound over time, but it also acts like a self alignment check towards your objective.
And this is what RLHF does while training. Keep improving and you will get there. :)
It is interesting to see how non AI researchers (or the ones not involved with the field) don't understand how much of the components of these larger API based LLMs are derived from open research and how the field is developing so fast because of the open source nature.
@HJCH0
I think if you want to work with LMs I would highly advise you to first understand what they are (also read the paper attention is all you need) and also try implementing the transformer model from scratch. You will learn a lot doing this.
@vipul_1011
I usually plan two weeks before the deadline. Each day each section in the first week. Second week first half is for stitching the sections together and second half is for revisions. So each day I only have to spend 3hrs max on the paper.
As a result, no anonymity period will be required for papers submitted for the Feb. 1, 2024 TACL deadline or the Feb. 15, 2024 ARR deadline (). The submissions themselves must still be fully anonymized. (2/4)
#NLproc