@karpathy
Today's demos seem like they introduced a completely different way of programming.
Previously, business logic was in code and AI was allowed to generate specific writing/image outputs.
Today's demos let GPT handle all the biz logic. Absolutely wild.
I've been asked by few first year PhD about how to start LLM research on X, say long context modeling. My number one suggestion -- though it seems a bit of unconventional -- is *not* to read any papers related to long-context, but to talk to the model
- Talk to the model about a
I gave a lecture at
@Stanford
CS 25.
Lecture video:
AI is moving so fast that it's hard to keep up. Instead of spending all our energy catching up with the latest development, we should study the change itself.
First step is to identify and understand
The most interesting piece of the ChatGPT plugin leak was the plugin that
@openai
was using to assess the security of the other plugins. Here's how it works.
The first part of the prompt was the instructions:
Agents capable of executing complex tasks is on the rise: Devin is a programmer, and now we have Weco, a data scientist. 能执行复杂任务的agent正在不断涌现:Devin是程序员,现在又来了Weco,数据科学家
We're excited to announce AIDE has become the first human-level AI agent for data science!
AIDE outperforms half of human data scientists on a wide range of Kaggle competitions, surpassing conventional AutoML, LangChain agents, and ChatGPT with human assistance. 🏆
🚨Announcing The Prompt Report🚨
A 76-page survey of 1,500+ prompting papers, analyzing EVERY prompting technique, Agents, & GenAI
Led by
@learnprompting
, and folks from
@OpenAI
,
@Microsoft
, &
@UofMaryland
Here’s what we found & the 58 prompting techniques you should know👇🧵
I’m excited to share that I’m working on a new book about building applications with foundation models! AI Engineering builds upon Machine Learning Systems Design, but with a focus on large scale, ready made models.
The book covers:
- The new AI stack (e.g. how it differs from
ChatBCG: Generative AI for Slides ✨
This Christmas
@JosephSemrai
and I finally got it working!!
After DALL-E 2 for images and ChatGPT for text, the final step to make all of us redundant:
The world’s first Text-to-PowerPoint AI.
📊 🚀
原来ChatGPT的开发时间比之前媒体报道的还短,只有8天。It turns out that the development time for ChatGPT was even shorter than what was previously reported by the media: only 8 days.
A year ago today, I signed up to be on call for this low key research preview that we were demoing to the world. We built and shipped the product in about 8 days. Nobody, and I mean nobody could have predicted how the world was going to change. Here are some screenshots from a
Explaining 8 Popular Network Protocols in 1 Diagram. The method to download the high-resolution PDF is available at the end.
Network protocols are standard methods of transferring data between two computers in a network.
1. HTTP (HyperText Transfer Protocol)
HTTP is a protocol
Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesis
paper page:
In text-to-speech (TTS) synthesis, diffusion models have achieved promising generation quality. However, because of the pre-defined data-to-noise diffusion process, their
Being willing to ask dumb questions is a superpower. Often by far the fastest way to get oriented in a new domain, and though perhaps counterintuitive, experts tend to love it when people genuinely want to learn about their passion area.
Can we teach LLMs to write long articles from scratch, grounded in trustworthy sources?
Do Wikipedia editors think this can assist them?
📣Announcing STORM, a system that writes Wikipedia-like articles based on Internet search. I now use STORM in my daily research!🧵
@karpathy
@emollick
@VisualCap
I don't think that LLM is suitable for advertising. Imagine you and your friend are having a great conversation, and suddenly he/she interrupts with an advertisement, trying to sell something to you...
New post: Emerging architectures for LLM applications
We compiled a reference stack for developers building apps on top of LLMs, focused especially on in-context learning
With
@rajko_rad
Are you wondering how large language models like ChatGPT and InstructGPT actually work?
One of the secret ingredients is RLHF - Reinforcement Learning from Human Feedback.
Let's dive into how RLHF works in 8 tweets!
📣How does 🔥Alpaca🦙 follow your instructions?
Mechanistic interpretability at scale – our new paper identifies the causal mechanisms the Alpaca 7B model uses to solve simple reasoning tasks (with Atticus Geiger,
@ChrisGPotts
, and
@noahdgoodman
!)
Paper:
With the tremendous success of
#LLMs
(
#OpenAI
#ChatGPT
#GPT4
), it's probably about time to ask THE question: can we discern
#AI
-generated texts from Human-generated ones?
Spoiler 🚨: we can almost always detect with enough observations!
Paper:
A 🧵
Python in Excel is now a thing! Python is now a peer of the Excel formula language and you can mix both languages seamlessly in the Excel grid. Python runs on Azure and is powered by the
@anacondainc
Python distribution.
This was a multi-year collaboration between the Python
This is huge: Llama-v2 is open source, with a license that authorizes commercial use!
This is going to change the landscape of the LLM market.
Llama-v2 is available on Microsoft Azure and will be available on AWS, Hugging Face and other providers
Pretrained and fine-tuned