Announcing LiveBench AI - The WORLD'S FIRST LLM Benchmark That Can't Be Gamed!!
We (Abacus AI) partnered with Yann LeCunn and his team to create LiveBench AI!
LiveBench is a living/breathing benchmark with new challenges that you CAN'T simply memorize. Unlike blind human eval,
there is a revolution taking place in biology before our eyes
I spent the past month reading the papers & interviews of
@drmichaellevin
, and wrote a long-form essay explaining his work and why it's important
What exactly is a prompt/prompt engineering?
We read 13 definitions to find out:
> A prompt is an input to a Generative AI model, that is used to guide its output
> Prompt Engineering Prompt engineering is the iterative process of developing a prompt by modifying or changing
🚨New WP: Can LLMs predict results of social science experiments?🚨
Prior work uses LLMs to simulate survey responses, but can they predict results of social science experiments?
Across 70 studies, we find striking alignment (r = .85) between simulated and observed effects 🧵👇
RAG Or Long Context LLMs - A Hamlet-Level Question!
There has been much debate and argument about RAG vs. long-context LLMs. In this study, they find that the long context outperforms RAG.
Our in-house experiments reveal that long context is ineffective when the LLM has to use
We have gotten a lot of questions about role prompting.
To clarify, we ran ~12 role prompts against 2K MMLU questions on gpt-4-turbo. This is *not yet* in The Prompt Report.
In particular, I created a "Genius" prompt, something like: "You are a Harvard educated scientist..."
If you are a designer/engineer but not having any research experience I’m hiring a resident and this is the ideal program for you! This is really the coolest opportunity for product designers/ product engineers who want to learn research and train lots of models :)
I'm playing around with generative AI tools and stitching them together into visual stories. Here I took the first few sentences of Pride and Prejudice and made it into a video.
The gen stack used for this one:
-
@AnthropicAI
Claude took the first chapter, generated the scenes
Mathematician Terence Tao says AI is already being used to automate mathematical proofs and eventually you will simply dictate a proof to an AI and it will tell you if it can be formally verified
A theory of biological relativity: no privileged level of causation
"Biological relativity can be seen as an extension of the relativity principle by avoiding the assumption that there is a privileged scale at which biological functions are determined."
@AlisaValdesRod1
@CaliforniaCodes
@VirusesImmunity
ACE2 is distributed throughout the entire respiratory tract, from the nasal cavity to the alveoli. Nasal vaccines can be helpful , but will not completely prevent infection, especially those that start in the lower respiratory tract and are more likely to cause severe harm.
Ethan Mollick says studies show that AI is already capable of superhuman persuasion: when humans debate with AI they are 82% more likely to change their view to fit the AI's view than when they debate a human, and AI can reduce belief in conspiracy theories
The correlation between adopted brothers and sisters (genetically unrelated people raised together) shows that the IQ correlation between them basically falls to zero in adulthood — suggesting shared childhood environment has no impact on intelligence in adulthood.
It's well known that lower income people tend to suffer much worse health. Is poverty at the heart of this disparity, and, if so, could a large cash transfer help close this gap? We examine an RCT that provided 1000 low income participants $1000/month for 3 years. We find…
Today, we are making an experimental version (0801) of Gemini 1.5 Pro available for early testing and feedback in Google AI Studio and the Gemini API. Try it out and let us know what you think!
#读后留痕
Google Brain / Deepmind 一位知名 Research Scientist 公布了自己的未来预测
除了『全球人口将开始减少』和『GenAI 将继续改进』这类,还有些较具体预言
他对 AI 在教育应用速度上的悲观我最不能理解
(Murphy 是机器学习领域经典《Machine Learning: a Probabilistic Perspective》作者
M 事业蒸蒸日上,去年开始领导了一个 60 多人写作/编辑团队
公司同年引入 AI ,团队工作量因此得以大幅减轻
蜜月期后,下属多数被上级裁员
——而余者主要『将 AI 所写改得更像人类』
年初,团队只剩 M 一人,做着以前的数十人工作
在 M 的老家纽约地震那天(今年 4 月),上级告知 M ,他也多余
裁
I count 5
#Olympians
collapsing so far through
#Paris2024
An
#OlympicGames
with multiple
#Covid
outbreaks (at least 40 that we know of as per
@WHO
) & over 90 athletes who couldn’t even attend due to illness after recent
#Covid
infections
It’s not a cold
Everyone is “vulnerable”
Stunning display of incompetence.
Misunderstand Technology
> GPT3.5 API was released in March 2022
> it was wrapped into ChatGPT in Nov 2022
> It went viral
> the audacity “We learned about ChatGPT on Twitter”
Misunderstand Board Roles
> “Sam claimed to be an independent