My Transformers Catalog has become one of my most popular posts ever. Some of you told me that you turned into a pdf for easier reading. I thought I should make it into an arXiv preprint. Here you go: 60 Transformers in 36 pages 🤖 🎉
Pretty big update to my Transformer Catalog. I added ChatGPT, Sparrow, and Stable Diffusion among others. I also included a section about RLHF and Diffusion models and a new timeline view. Enjoy!
Today I had coffee with an MIT PhD who, in an effort to build AGI and mathematically prove free will, is coming up with an alternative to back propagation. How was your Monday?
A lot has been speculated about TikTok's recommendations. This is the first paper I've read by the team, and it has many interesting details: expirable embeddings, parameter server, online training... Good
#recsys
stuff
As many of you know, over the past few months I have been sharing Prompt Engineering resources in different forms. I have now compiled them all into a cohesive publication and uploaded to arxiv:
Excited to share I've joined Google as VP of Product for Core ML/AI! Dream job at the perfect time, blending cutting-edge AI with direct user impact across Google's product portfolio.
You have probably heard recently that Direct Preference Optimization (DPO) is taking over RLHF as the preferred method to align LLMs to human preferences (). Well, that is "old news" now. The newest/coolest thing now is Kahneman Tversky optimization (KTO)
Our new survey on LLMs is now available in arXiV. Great team work with awesome collaborators. Our goal is to give a comprehensive overview of LLMs (including forward looking work like post-attention, SLMs and agents) while keeping it very readable.
Google assistant, a product developed by a company with thousands of AI researchers and engineers, cannot auto detect language or describe an image you get on text. That's how hard deploying AI in product is.
Ten years ago
@JustinBasilico
& me published a blog post describing an architectural blueprint for Recommender Systems. I'm now revisiting it by including several alternatives published since, and a new one that in some ways includes all the previous ones:
Thompson Sampling has been one of my favorite algorithms due to its efficiency and simplicity. It turns out that it also works for LLM alignment! Great paper by Deepmind on an extension to DPO
By popular demand, the continuation to my Prompt Engineering 101 where I describe all the more advanced techniques. Starting from Chain of Thought, I include 15 other techniques including e.g. Tree of Thoughts, Rails, ART, constrained prompting and APE.
Trying out the new Bing, and it is seriously revolutionary. Also, for those naysayers, it is different and better than ChatGPT in several ways, among other things, it combines conversation with search, and has access to the internet as you chat with it, which is a huge difference
My Transformer Catalog post has become one of my most popular pieces of content ever, with now almost 50k views only on the post version. Here is the next, and pretty big, update:
- Blog post:
- Arxiv paper:
Big milestone: a fine tuned LLM has surpassed expert human performance on medical QA (as measured on the PubMedQA dataset ). BIOGPT by MSFT () not only beats human performance and is SOTA in medical QA. It is also SOTA on other tasks
@JFPuget
@JajaLiao
You can't be serious. Jazz, Rick, Hip Hop, American 🏈, Baseball, Hippy Culture, Grunge Culture, Halloween, Q Anon, MAGA culture, BLM, Me too... Should I go on? The fact that a culture is newer does not mean it's worse!
Netflix algorithms vs. GPT4. I know I am shooting myself in the foot here a bit, but I am getting much better Netflix recommendations from GPT4 than from Netflix.
After more than 5 years as CTO
@CuraiHQ
, I am transitioning this week to an advisory role (while staying in the board of directors). I am grateful for the opportunity to build something huge, and remain very bullish on the company
RLAIF (Reinforcement Learning with AI Feedback): open source foundation models, instruction tuning, leaked models on 4chan, Google engineers resigning and more AI drama just in the last few weeks
If you are working with LLMs, here are three important things you might want to keep in mind to anticipate how things are going to evolve in the next 6 months and how you should be approaching your strategy (1/5)
Wow, my Transformers Catalog has been viewed over 10k since last week's update. Many good comments too. Quick follow up includes link to github for folks who want to file a PR. Also adding a new timeline view where the Y-axis is model size
I spent the afternoon prompting
#stablediffusion
and I gotta say: either I am a horrible prompter or you people spend A LOT of time doing this until you get something good worth sharing 😄
My new post "Mitigating LLM Hallucinations: a multifaceted approach" is a comprehensive overview of how to tackle hallucinations of large language models. From measuring and detecting, to prompt engineering , to many other aspects that play a role.
@tszzl
The best software engineers don't last more than a year at Tesla, if they ever made the mistake to accept in the first place. Just browse around on LinkedIn.
If you ask "the world" to stop technological advances for a period of time you *might* get the good hearted people to do it. You will definitely *not* get the evil ones to do so. Therefore, you are literally giving evil an unfair advantage.
My friend
@jure
claims that modern
#recsys
at most companies are "based on graphical representations" ( min.8). In my experience that is not the case. Graphs might be *used*, but are never the fundamental construct. Anyone has similar/different experience?
In my latest blog post I compile some of my favorite
#recsys
resources. 50+ links, from the basics to LLMs, with a focus on industry applications. Includes resources from LinkedIn, Netflix, Google, Amazon, Doordash, Bytedance and many more!
Modern LLMs are much more than token predictors. They are also much more than pretrained Transformers. I review the post pretraining processes in my latest blog post:
My introductory course to
#promptengineering
is now live (and free) on LinkedIn! It was very fun to get this course ready as a fast follow-up update to some of my posts over the past few months.
"Can we build conscious AI?" is the wrong question for a simple reason: we still don't understand what consciousness is. What is the right question?: "Will building AI help us understand consciousness?". The answer is a resounding yes.
@jtk
To be honest, I don't even recall interacting with him recently and I can't find anything. It might have been over me not being an Elon fanboy maybe 🤔
A week ago, Meta presented Chain of Verification (), that addresses and mitigates hallucination. It is a specific implementation of Reflexion, similar to DRA. I have added it to my hallucination guide
Yesterday I talked to a pretty "famous" person who was shocked that one of their
@Quora
answers got 4M views. Not surprised. My Quora content has ~32M views, which is far more than what I get on any other medium. I often meet people IRL who got to know me through Quora.
A question I get often given my background: can LLMs be used for recommendations? I included a simple example in my Prompt Engineering 101 post (). Rob goes much further with an interactive feed ranking algo developed in a day!
New blog post where I summarize the main advances in the area of open source LLMs (including Falcon, LlaMa2, and Free Willy). I describe different leaderboards and I enunciate the 3 laws of model evolution, and I ask GPT-4 and Bard about the implications.
"A generation defining AI moment", where I talk about my experience with GPT-4 and my thoughts on what it means. Bonus track: I include some discussion on AGI, consciousness and reasoning
Friends/devs, I need you to welcome/show some love (aka github stars) to a blind coder I have been helping lately (). Keep in mind this is his first coding project in over 20 years, so, be gentle 🤗. A bit more about Ismael... (1/5)
Today I gave a fun talk at
#GTC2024
. I went through the history of AI. I started with old school 2 parameter linear model for personalized recommendations & finished with multi-agent design and AGI. Also included an interactive demo of using Gemini for recommendations.
"Homepage Recommendation with Exploitation and Exploration". Great post by
@DoorDashEng
We will be discussing practical
#recsys
examples such as this one in our upcoming
@get_sphere
course on Recommender Systems
You might know I am an avid runner. I credit running for much of my success in life, including the professional aspects. To celebrate my 50th Bday I ran the NYC marathon. It wasn't pretty, but I got it done. A good time to remember my 10 most epic runs.
Barcelona🚨!!!
I haven't given a talk in my hometown in years. It's about time. September 20th I will be talking about AI-driven innovation and sharing stage with Africa Perianez. All for a great cause! Hope to see many of you there:
All those replies arguing that Google *already* has AI, talent, and deep pockets is precisely why it is going to be fascinating. Heard of the innovator's dilemma? Heard of all the talent and money outside of Google? I wouldn't be surprised the killer app comes from Xooglers.
Today Jensen reunited and talked to the authors of the now mythical "Attention is all you need paper" at hashtag
#GTC2024
()
Here is the longer story. As many things in the current AI revolution, it all started at Google:
Fantastic working session with Jensen Huang and team today. It was followed by a celebration of the long standing Google/NVidia celebration. This included an amazing present: a frame including the GPU boards all the way from Pascal in 2016!
Today
@Quora
announced a new product, Poe, that had been out in private beta for a while. I encourage you to try it out if you are interested in genAI/LLMs, or just for fun
Twitter has open sourced their
#recsys
code. They have also published a summary of the main components. My conclusion: nothing surprising, groundbreaking, or that you wouldn't expect if you work in this area. Nothing really SOTA either
Very excited to announce I am hiring my first direct report at Google! Sr. Director role to lead AI data lifecycle, from acquisition, licensing, and generation through responsible AI governance and model launch.
One of the fascinating things about Elon is how he's creating a strong network of non traditional media support that includes Rogan and Fridman. Very much like Trump did. Is this going to be the new normal for brands and politicians?
The funny thing is that Elon really thinks this will push the weak engineers out when it's exactly the opposite: any good engineer there is probably already interviewing full time and the ones who went over the weekend is because they don't have good/clear alternatives
There are people in my neighborhood still using masks. While running. Outdoors. Alone. I totally respect their personal choice, but I do wonder if they might have missed a CDC update.
For many of us, who have loved this place for years, it is very important that Twitter fails. Why? Mostly because the world needs to understand that Elon's terrible management practices do not work in general.
A few days ago I decided to put AI-driven software development to the test by developing and end-to-end fullstack chatbot. Here is my journey (code and fully functional personalized chatbot included):
Really interesting and detailed post about
@PinterestEng
's refactoring of the Homefeed to include realtime features. New features, encoder only Transformer architecture, constant retraining, and migration to GPUs. Good
#recsys
stuff.
Just landed in Vegas for
#GoogleCloudNext
. Plane was loaded with some of the 30k attendees this year. Happened to randomly sit next to someone I knew, so we ended up talking about everything GenAI for 1.5 hours 😁 . Looking forward to many such conversations in the next 3 days!
Very nice paper. Transformers are just a special case of State-Space Models. The key idea is the notion that the Linear Attention mechanism from Transformers can be generalized by using Structured Matrices.
@balajis
So, that clearly means that the US will not be leading that revolution given how little investment there's here in any kind of "public" infrastructure. Do you agree?
Twitter has made a huge difference on the lives of millions of people. From the Arab Spring to... myself. Everybody laid off today deserves credit and huge respect. Thank you. I'm here for you if I can help 🙏
I had missed this. The video of my invited talk last year at
@kdd_news
was published. Almost two hours that include many good questions from the audience (Slides are missing from the video unfortunately, but are available here )
Many incredible results in the paper describing the medical capabilities of Gemini models (). On the NEJM CPC dataset of complex diagnosis, Med-Gemini not only beats clinicians, but it also beats clinicians when they are allowed to use search.
I don't know how many times I've said this: a time-ordered feed can never create a successful social network experience. You are forcing users to either follow very few people, or read random posts that depend among other things in timezones.
Pros of
@joinmastodon
:
- open-source
- decentralized: run on a federation of servers
- no ads!
- no attention-algorithm: the timeline is chronological!
- 500 instead of 140 characters in each "toot"
- key features that we need conserved: mentions, hashtags, retweets, likes
5/12
Yesterday I was telling someone in Japan that all the hard work put in by new generations to learn English will be made irrelevant by AI automated translation...
(1/3) Until now, AI translation has focused mainly on written languages. Universal Speech Translator (UST) is the 1st AI-powered speech-to-speech translation system for a primarily oral language, translating Hokkien, one of many primarily spoken languages.
Web3 will still happen. At some point, and at some scale. But, it's interesting to see many of the opportunity seekers coming back to AI now realizing that they probably moved too soon. AI is now, and it's huge.
Very interesting work on Multi-agent LLM Debate systems by our team at Google. Sparse topologies facilitate more rounds of debates and result not only in higher accuracy but also much lower cost.
Groundbreaking research from the
@CuraiHQ
team: an ensemble of GPT4 agents playing different roles (researchers and deciders) improves overall decisions, particularly in high stakes domains like healthcare!
It was lots of fun to give a talk in my hometown (Barcelona) this last week after many years. Thanks
@friquifund
for organizing and
@aperianez
for sharing stage with me!
Surprising to realize that even at my age (almost 50) there are many folks who want & need to make new "real" friends. I have many good friends from when I was a kid, but making new strong connections is not something you only do in your youth, it is a life long journey
I'd love to increase my Following count to 1k. Who should I follow that has interesting things to say in AI/ML or tech in general? +100 for minority representatives, -100 if they're radical right (i.e. Trumpers) or left (i.e. woke luminaries).
I know some of you struggled to access my prompt engineering course. As of today it has been opened for free and you can access regardless of whether you have LinkedIn Premium or not.
It was an honor to be invited by
@FCBarcelona
, my (more than a) club, to talk about
#digitalinnovation
and
#ai
. It was a dream come true to watch the 5-0 win from the VIP box.Before entering the box, I was asked to remove my LinkedIn jacket, and borrow a "formal" one.
I will be speaking next week at my Alma Mater Universitat Pompeu Fabra in Barcelona for the 25th anniversary of the School of Engineering. This interview where I talk about the promises and risks of AI is a good preview, minus the cool demos
GPT-4's remarkable abilities raise questions about AGI, consciousness, and reasoning. I dive into those, how we got here, and what it means for our future in my latest blog post:
Event alert for folks in Barcelona! July 3rd 19h - Adevinta Spain Auditori.
"Unpacking Ethics and Risks in ML and Generative AI" with
@PolarBearby
and
@quicola
Organized by
@friquifund
, all proceeds going to eXplorium.
Get your ticket here:
A few improvements and bug fixes to my "Advanced Prompt Engineering" post. Among other things, I added two new methods Dialog-Enabled Resolving Agents (DERA), and Expert Prompting (yeah, the one from the now infamous MIT curriculum preprint)
Really interesting work coming out from MSFT Research. Orca2, a 13B parameter SLM (Small Language Model) that matches (and sometimes outperforms) much larger models including ChatGPT and LlaMa2 at many tasks! How can that be?