[Paper] Neural diffusion models that...
Invert "noise" applied to syntax trees
Iteratively edit code while preserving syntactic validity, making it easy to combine the model w/search
Learn to convert images into programs that produce those images
[Paper] "Automated Design of Agentic Systems" features a case study on ARC-AGI.
This approach - Meta Agent Search - progressively discovers novel agents that outperform state-of-the-art, hand-designed agents.
Leading AGI research lab DeepMind is implementing nearly identical techniques we're seeing at the top of ARC Prize leaderboards.
1. test time fine-tuning
2. blast inference + search
AlphaProof = closed source
ARC Prize winning solutions = open source (by end of 2024)
[Paper] Current high-scoring team member
@bayesilicon
shares an ARC-AGI training task generator.
More examples "...should enable a wide range of experiments that may be important stepping stones towards making leaps on the benchmark ."
[Paper] ARC-AGI remains unsolved despite its seemingly simple content and evaluation methods.
To overcome the limitations of result-oriented analyses, this study embraces the Language of Thought Hypothesis (LoTH).
The spectrum of key factors that define intelligence.
1. Fluidity: Memorization vs skill synthesis.
2. Operational area: Low vs high abstraction in skill usage.
3. Information efficiency: High vs. low amount of data to acquire skills.
@fchollet
[Paper] ARC-AGI is hard: a vast action space, hard-to-reach goals, a variety of tasks...
ARCLE is an environment for reinforcement learning research on ARC that shows agents can learn via proximal policy optimization.
[Paper] Test-Time Training (TTT) layers show an ability to dynamically adapt & learn during test time, which is essential for the diverse & novel problems in ARC-AGI.
This approach outperforms Transformers & RNNs in long-context tasks.
"Intelligence is the ability to blaze fresh trails and build new roads. It's not the road. Don't confuse the output of the process with the process itself."
@fchollet
at
#AGI24
Full talk:
ARC Prize Co-founder
@mikeknoop
presented a demo of ARC-AGI today to Senator
@Scott_Wiener
, FTC Chair
@linakhanFTC
, and Assistant Attorney General Jonathan Kanter.
@mikeknoop
gave a masterclass at
@ycombinator
today on why we should not be trigger-happy on regulating AI.
He pulled up this graph showing how far we are from actually achieving human-performance AGI according to the
@arcprize
benchmark. (Read: we're NOT close)
"I think our
Introducing the ARC-AGI Public Leaderboard.
A second ARC Prize leaderboard to measure the AGI progress of frontier AI models.
$150k verification fund. No prize money. 1st scores on the board.
LLMs won’t lead to AGI
$1,000,000 Prize to find true solution
Hear the full story from
@fchollet
&
@mikeknoop
talking with
@dwarkesh_sp
* Why LLMs struggle with ARC
* Skill vs intelligence
* Future of AI progress
Full ep:
ARC Prize 2024 has reached 10k entrants on Kaggle!
There's still plenty of time to get involved. In addition to high score prizes, there are also paper awards to reward novel approaches.
10,007 Entrants
726 Participants
677 Teams
4,132 Submissions
"The point of [ARC Prize] is to incentivize people to go beyond large language models to develop AI sytems that truly are intelligent." -
@seancarroll
From a recent
#MindscapePodcast
interview w/
@fchollet
.
[Paper] "Our result [...] unveils the fundamental difference between learning to reason and learning to achieve high performance on NLP benchmarks using statistical features."
"The purpose of ARC is to get researchers to refocus on intelligence (adaptation to novelty, generalization), away from pure memorization, because I believe this is how we'll get to AGI."
To be clear, I've never claimed that solving ARC was equivalent to solving AGI. The first ARC solver is not going to be an AGI (I already stated this all the way back in 2019).
The claim is simply that:
1. Until we solve ARC, we don't have AGI, since the AIs we have cannot
AGI is not a solved problem. Scale is not all we need.
ARC Prize was created to motivate AI researchers to, once again, work on new ideas and openly share progress.
Hear
@mikeknoop
's talk at a recent
@ycombinator
policy event.
[Paper] This is interesting for ARC-AGI because it enhances pattern recognition and reasoning by efficiently encoding and processing relational data, enabling advanced logical inferences and hierarchical representations.
AGI progress has stalled.
2023: $20B LLMs; Maybe ~$100M AGI.
Frontier research is no longer open.
From
@mikeknoop
's presentation at
@arizeai
Observe July 11 in SF.
"This arms race between the big tech companies to build the best possible LLM has led to an intellectual collapse, where everyone is working on the same things."
"The top entrants in ARC Prize have been outsiders coming up interesting, novel ideas."
@fchollet
#AGI24
[edited]
We're just one week away from Singapore's first AI Tinkerers meetup!
We've got community demos discussing the latest insights from testing Llama 3.1 and attempts at tackling the
@arcprize
.
We're calling for more demos!
Registration is now at 50% capacity.
@jheitzeb
"In the words of Jean Piaget, intelligence is what you use when you don’t know what to do. It’s how you learn in the face of new circumstances, how you adapt and improvise, how you pick up new skills."
@fchollet
Behind the scenes of ARC Prize 2024
* What inspired ARC Prize 2024?
* Why require open source solutions?
* Why $1M+ in prize pool?
Hear it all with
@mikeknoop
on the
@NoPriorsPod
Full Episode:
[Paper] "Despite 3 international competitions with [$1M+] in prizes, the best algorithms still fail to solve a majority of ARC tasks and rely on complex hand-crafted rules..."
Explore DSL search, DreamCoder, LLM, and ensemble approaches.
"Discovering and benefiting from artificial general intelligence in our lifetimes depends on open source and open science. We must establish policy and incentives to enable innovation and not let fear of the future set us back for years, or even decades."
On Monday, I shared a letter with Senator Wiener and his team opposing SB-1047 based on my experience leading Zapier AI and ARC Prize.
Zapier is the largest automated AI platform in the world. And ARC Prize is the biggest frontier AGI research competition.
Others have made
🎙️Pod w
@mikeknoop
founder of
@zapier
on new Arc Prize for AGI
We discuss:
-Redefining AGI
-ARC Prize for AGI
-Prizes vs investing - why do each?
-LLM adoption at Zapier
We are pleased to announce that François Chollet
@fchollet
, AI researcher and Senior Staff Software Engineer at
@Google
, will speak at the 17th Annual Conference on Artificial General Intelligence.
Register now to attend
#AGI24
in-person or watch online:
[Paper] "The lifelong learning + flexibility to test-time tack on an arbitrary number of experts makes this research an interesting candidate for ARC-AGI (specifically for a test-time deep learning guided DSL generator)."
@mikeknoop
Inspiration for ARC Prize solutions can come from anywhere.
Here's
@fchollet
on the TensorFlow podcast talking about how children behave like scientists in learning.
What's inspiring your approach?
Source:
Latest ARC Prize community newsletter just sent.
🟩 +13% in ARC-AGI world record
🟦 Chollet's talk @ AGI-24
🟨 Resources (papers, datasets...)
🟥 Community (team up, approaches...)
Not on the list? Sign up now!