Edward Hughes @edwardfhughes Twitter profile | Pikagi

Pikagi

Edward Hughes

@edwardfhughes

1,101

Followers

440

Following

26

Media

469

Statuses

#OpenEndedness . Staff Research Engineer @GoogleDeepMind , Visiting Fellow @LSEnews , Advisor @coop_ai , Choral Director @GodwineChoir . Views my own.

London, UK

https://t.co/niJJ6pTW4M

Joined January 2014

Don't wanna be here? Send us removal request.

Pinned Tweet

@edwardfhughes

Edward Hughes

3 months

📣 New paper! Open-Endedness is Essential for Artificial Superhuman Intelligence 🚀 (co-lead @MichaelD1729 ). 🔎 We propose a simple, formal definition for Open-Endedness to inspire rapid and safe progress towards ASI via Foundation Models. 📖 🧵[1/N]

Tweet media one

9

32

113

Last Seen Profiles

@FTOOMah300

@ChicagoMayor

@invinciblerugby

@BenFrancis1992

@bokeplokalmalam

@Kazookieslama

@IHS_MBasketball

@inthewind875055

@heymikli

@vitsstay

@city_panel

@naxzious

@ominhopt

@sanniesea

@aa2109668

@Mbahmaryono77

@bokeplokalmalam

@underscore_bot

@Keta_Val

@jongjiganjang

@BinorRaja

@LLWT_

@Bos61564528

@AFC_Amsterdam

@konami9x

@AStA_Wuppertal

@Yuuga_Ours

@ritodhi_c

@MatarazzoS66315

@_6oio

@4003I

@ezgipeksen

@par37cmdx

@turkimefkure

@midas_864

@pott95

@edwardfhughes

Edward Hughes

9 months

Absolutely delighted that our paper came out in Nature Communications today. This was an incredible effort from an amazingly talented team at GDM that I had the great privilege to lead.

Tweet card media

Learning few-shot imitation as cultural transmission

Nature Communications - The modelling of human-like behaviours is one of the challenges in the field of Artificial Intelligence. Inspired by experimental studies of cultural evolution, the authors...

5

20

94

@edwardfhughes

Edward Hughes

6 months

Perhaps the most innovative work that I've had the pleasure of being a co-author on. You've heard of generative video (at scale) but what about generative simulation at YouTube scale?!?!

@_akhaliq

AK

6 months

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual

79

531

2K

3

5

69

@edwardfhughes

Edward Hughes

2 years

Such a joy to have worked with this incredible team over the past year, and to have produced these remarkable results!

@FeryalMP

Feryal

2 years

I’m super excited to share our work on AdA: An Adaptive Agent capable of hypothesis-driven exploration which solves challenging unseen tasks with just a handful of experience, at a similar timescale to humans. See the thread for more details 👇 [1/N]

25

266

1K

1

7

50

@edwardfhughes

Edward Hughes

3 months

Enormously excited to co-author this incredible paper. It represents the latest and most exciting step towards generating cultural evolution in AI - a personal mission of mine for the past 7 years! Kudos to @JonnyCoook and @_chris_lu_ who did all the heavy lifting. [1/n]

@JonnyCoook

Jonny Cook

3 months

1/ 🚀 Presenting AGI - Artificial Generational Intelligence 🚀 We apply the concept of cultural accumulation to RL and find that agents can improve across generations, outperforming those trained for a single lifetime of the same experience budget! Co-led w/ @_chris_lu_ . 🧵

Tweet media one

4

29

96

1

6

46

@edwardfhughes

Edward Hughes

25 days

My favourite poster of the week ⁦ @icmlconf ⁩: like Voyager (+ MCTS) but for social deduction games. Really exciting work in the burgeoning field of Open-Ended Cooperative AI by Jonathan Light et al.

Tweet media one

1

5

46

@edwardfhughes

Edward Hughes

2 months

Cool prize: ! The problems remind me of the “production rules” from our AdA paper (). I wonder whether large-scale meta-RL with an LLM-powered curriculum might be a quick(ish) way to win $1M 🌶️?

Tweet media one

2

2

35

@edwardfhughes

Edward Hughes

7 days

We are heading for a paradigm shift in the process of scientific discovery itself. What an incredible privilege it is to work on open-endedness at this moment. Huge congratulations to my friends and colleagues in the field for this leap towards general AI-accelerated science!

@_chris_lu_

Chris Lu

9 days

Excited to share The AI Scientist! We use LLMs to autonomously come up with research ideas, implement them, do literature search, write them up, and review them -- producing full-length papers on AI without human intervention. Co-led with @cong_ml and @RobertTLange

6

38

175

3

7

49

@edwardfhughes

Edward Hughes

4 months

Yet more evidence that data-focussed research is the path to victory! Teaching new in-context abilities is typically bottlenecked by the data distribution. Here the authors prove that point for search.

@gandhikanishk

Kanishk Gandhi

5 months

Language models struggle to search, not due to an architecture problem, but a data one! They rarely see how to search or backtrack. We show how LLMs can be taught to search by representing the process of search in language as a flattened string, a stream of search (SoS)!

7

112

578

0

5

32

@edwardfhughes

Edward Hughes

9 months

Awesome work: enormously exciting to see a reimplementation of XLand which is so efficient and accessible. Here's to a bright future for meta-RL research!

@vladkurenkov

Vladislav Kurenkov

9 months

🔥 Imagine if you could train Meta-RL agents for 1 TRILLION transitions under 40 hours? We present XLand-MiniGrid — JAX-accelerated meta-reinforcement learning environments inspired by XLand ( @FeryalMP ) and MiniGrid ( @Love2Code ). code:

1

34

182

0

3

32

@edwardfhughes

Edward Hughes

2 months

I’m pleased (and unsurprised) that generating synthetic data makes a lot of progress on ARC. I don’t view this as a weakness of the challenge, however - rather as an endorsement of the power of open-ended program-based data generation for model improvement!

@bshlgrs

Buck Shlegeris

2 months

ARC-AGI’s been hyped over the last week as a benchmark that LLMs can’t solve. This claim triggered my dear coworker Ryan Greenblatt so he spent the last week trying to solve it with LLMs. Ryan gets 71% accuracy on a set of examples where humans get 85%; this is SOTA.

Tweet media one

46

181

1K

2

6

32

@edwardfhughes

Edward Hughes

1 month

I don't know whether this five-tier system is genuine. Nevertheless, it is interesting (and unsurprising to me) that the Levels get progressively more Open-Ended and Multi-Agent. Perhaps @sama follows my external talks...

Tweet media one

@rowancheung

Rowan Cheung

1 month

OpenAI reportedly internally introduced a new five-tier system to track its progress toward AGI. The classification system ranges from Level 1 (current conversational AI) to Level 5 (AI capable of running entire organizations).

Tweet media one

15

73

528

0

3

32

@edwardfhughes

Edward Hughes

2 months

The field of cultural evolution in AI is continuing to grow! Inspiring to see this early stage report. Adding a "related work" section would help drive the cultural evolution of the research community.

@francoisfleuret

François Fleuret

@francoisfleuret

2 months

A little report!

Tweet media one

16

52

480

2

3

30

@edwardfhughes

Edward Hughes

27 days

Very proud to have played a small role in this awesome team! It's almost inconceivable to me that 15 years since I was trying (and failing badly) to solve BMO1 problems, we have now made an AI system that is nearly gold-medal standard on this year's IMO 🍾🤯

@GoogleDeepMind

Google DeepMind

@GoogleDeepMind

27 days

We’re presenting the first AI to solve International Mathematical Olympiad problems at a silver medalist level.🥈 It combines AlphaProof, a new breakthrough model for formal reasoning, and AlphaGeometry 2, an improved version of our previous system. 🧵

306

1K

5K

1

1

28

@edwardfhughes

Edward Hughes

30 days

Super chuffed to have been a member of the Genie team which won a best paper award today! Big props to @ashrewards and @jparkerholder for an awesome oral talk describing the work 🧞‍♂️🍾

@ashrewards

Ashley Edwards

30 days

Yayy congrats to the Genie team for receiving best paper award at @icmlconf !! 🎉🧞‍♂️

Tweet media one

11

15

111

0

2

27

@edwardfhughes

Edward Hughes

26 days

Come and see @_chris_lu_ and myself present the fascinating Artificial Generational Intelligence poster right now in Hall A2 @icmlconf . Bravo to @JonnyCoook who led this work!

Tweet card media

Artificial Generational Intelligence: Cultural Accumulation in...

Cultural accumulation drives the open-ended and diverse progress in capabilities spanning human history. It builds an expanding body of knowledge and skills by combining individual exploration...

1

3

27

@edwardfhughes

Edward Hughes

2 months

We are at a moment of maximum leverage for Cooperative AI. The theory and organisational infrastructure is well-developed, and multi-human multi-AI systems are nascent. Now is the best time to develop AI systems and governance protocols towards societal good.

2

6

26

@edwardfhughes

Edward Hughes

6 months

Epicly good work from some good friends who are very talented scientists! Self-improvement in cooperative games (like language) requires self-generation of diversity, unlike for zero-sum games. This paper paves the way for continuous, open-ended discovery (and patching) of safety

@_akhaliq

AK

6 months

Meta presents Rainbow Teaming Open-Ended Generation of Diverse Adversarial Prompts As large language models (LLMs) become increasingly prevalent across many real-world applications, understanding and enhancing their robustness to user inputs is of paramount importance. Existing

Tweet media one

5

37

200

2

5

25

@edwardfhughes

Edward Hughes

1 month

Realising @gregeganSF 's visions is a legitimitely exciting subfield of AI!

Tweet card media

Autoverse: An Evolvable Game Language for Learning Robust Embodied Agents

We introduce Autoverse, an evolvable, domain-specific language for single-player 2D grid-based games, and demonstrate its use as a scalable training ground for Open-Ended Learning (OEL)...

4

6

24

@edwardfhughes

Edward Hughes

1 month

Pre-learning best responses is not enough: safe, efficient autonomous vehicles need to adapt online based on inferences about others' policies. A beautiful real-world illustration of the problem!

@j_foerst

Jakob Foerster

1 month

Waymo car failing to coordinate w/ another Waymo (credits in the comment). Interesting to see a toy example from my grant applications play out in the real world. Two cars playing a best-response to a human driver model are not mutually compatible, multi-agent challenges are real

10

30

368

0

1

24

@edwardfhughes

Edward Hughes

2 months

Very excited to land in Chicago to lecture on Open-Ended Cooperative AI at the @SIOEcon AI Bootcamp tomorrow. There are a wealth of possibilities for interdisciplinary collaboration in this field, and I’ll learn as much as I teach!

0

5

23

@edwardfhughes

Edward Hughes

2 months

What an extraordinary week in Santa Cruz for the @coop_ai Retreat and Summer School. It has been such a privilege to talk with so many deep thinkers and to have the opportunity to inspire the next generation of brilliant students. The future of the field is bright!

@coop_ai

Cooperative AI Foundation

2 months

Thank you to all of the wonderful participants who joined us for the Cooperative AI Retreat in Santa Cruz earlier this week!

Tweet media one

0

3

35

0

3

21

@edwardfhughes

Edward Hughes

5 months

Many congratulations Sir @demishassabis on a hugely well deserved Knighthood. Thanks to Demis, I have spent the last 7 years living my childhood dream of building intelligent machines, amidst the pioneering and deeply kind culture at @GoogleDeepMind , and this is just the start!

0

0

20

@edwardfhughes

Edward Hughes

28 days

Sometimes, when you've been running, you have to walk for a bit before you can carry on running. And that walk gives you perspective to better decide where to run next. Innovation thrives when you can optimally alternate space and pace.

1

1

22

@edwardfhughes

Edward Hughes

3 months

🧠 We build on decades of pioneering thought by @kenneth0stanley , @joelbot3000 , @SchmidhuberAI , Lisa Soros, @togelius , @jeffclune , @OlivierSigaud , @risi1979 to name a few. 👋 We hope our paper provides an ideal compact intro to Open-Endedness for new researchers! 🧵[6/N]

1

1

19

@edwardfhughes

Edward Hughes

2 months

Increasingly I find that I skim many papers rather than reading fewer in detail. I find that this practice annoys me. Therefore for the next week, I am going to read one paper per day thoroughly and share some thoughts.

0

0

19

@edwardfhughes

Edward Hughes

28 days

Is Open-Endedness essential for ASI? Come see our @icmlconf poster in Hall C 4-9 #613 at 1:30pm and our oral at 4:30pm in Hall C 1-3. Can't wait to answer some probing questions and to debate our position with you!

@_rockt

Tim Rocktäschel

28 days

Today, @edwardfhughes and @MichaelD1729 from @GoogleDeepMind 's Open-Endedness Team will be presenting "Open-Endedness is Essential for Artificial Superhuman Intelligence" as an Oral at 4:30pm in Hall C1-3.

1

6

31

0

3

19

@edwardfhughes

Edward Hughes

5 months

A great pleasure to give a lit review lecture on #OpenEndedness @imperialcollege this afternoon. Many thanks to Anastasia Borovykh for the invitation. By chance, I covered many of the same themes as in @DrJimFan 's excellent Nvidia keynote (minus the humanoids!)

Tweet media one

0

1

18

@edwardfhughes

Edward Hughes

2 months

Very excited to arrive in beautiful Santa Cruz for the @coop_ai retreat. Looking forward to a few days of stimulating and imaginative conversations!

0

0

18

@edwardfhughes

Edward Hughes

3 months

Do check out the paper on arXiv and keep an eye out for exciting follow-ups from @j_foerst 's world leading Oxford lab, and our open-endedness community at @GoogleDeepMind ! I'll be at the @coop_ai Summer School in June and at ICML in July if you want to ask questions in person!

Tweet media one

0

2

18

@edwardfhughes

Edward Hughes

2 months

Very nice indeed - learning a good representation for interestingness and novelty!

@risi1979

Sebastian Risi

2 months

We are happy to present "Meta-Learning an Evolvable Developmental Encoding"! 🧬 Generative models can work as learnable representations for blackbox optimization but they are not designed to be easily searchable. We present a system that can meta-learn such representation by

3

44

154

0

3

18

@edwardfhughes

Edward Hughes

2 months

The best way to (re)build your confidence as a scientist is to immerse yourself in the small stuff: engineering a feature, writing a 1 paragraph conjecture, reviewing a paper. Eventually you realise (remind yourself) that the small stuff *is* the science.

0

0

16

@edwardfhughes

Edward Hughes

3 months

❤️ A huge shout out to my wise, patient, imaginative and dedicated co-authors @jparkerholder , @FeryalMP , @aditimavalankar , @YugeTen , Tom Schaul, @_rockt . 🤝We found that this paper helped us align our shared interests and hope it is similarly useful for the community. 🧵[7/N]

2

0

16

@edwardfhughes

Edward Hughes

2 months

Should AI agents write their own sandbox code? Permitting this feels eminently open to backdoors. Yet preventing it may lead to unexpected exploits. After all, humanity evolves its own “sandbox code” in the form of norms and institutions, reducing brittleness.

2

5

16

@edwardfhughes

Edward Hughes

5 months

Honoured to be a speaker at the fantastic @coop_ai Summer School - don't miss it!

@coop_ai

Cooperative AI Foundation

5 months

Applications for the 2024 Cooperative AI Summer School are now open! June 19-23, Santa Cruz, CA. Confirmed speakers include @edwardfhughes , @polynoamial , @fangf07 , @nsrg_shah , and Joe Halpern. Find out more and apply via our website: .

0

9

28

0

1

15

@edwardfhughes

Edward Hughes

2 months

Superb step in the direction of recursive self-improvement!

@_chris_lu_

Chris Lu

2 months

Excited to share my first work from my internship @SakanaAILabs ! We used LLMs to design and implement new preference optimization algorithms for training LLMs, discovering cutting-edge methods! Co-led with @samianholt and Claudio Fanconi. Details in thread 🧵 (1/N)

4

36

157

0

1

15

@edwardfhughes

Edward Hughes

3 months

Awesome to finally meet you in person @jeffclune , after so many years being inspired by your work! There has never been a more exciting time to do Open-Endedness research: it's a privilege to work in such a creative and game-changing field.

@jeffclune

Jeff Clune

3 months

It was magical to return to Oxford to give a talk, seeing old friends and making new ones. That's especially true because Jakob @j_foerst is a great host that really knows how to roll out the red carpet!

Tweet media one

Tweet media two

Tweet media three

Tweet media four

3

4

74

1

1

14

@edwardfhughes

Edward Hughes

2 months

Super excited to see @_rockt ’s book in the pipeline! He is one of the most lucid, knowledgeable and engaging thinkers I have ever worked with, and I’m delighted that a wide audience can now benefit from his insights. The perfect stocking filler!

@_rockt

Tim Rocktäschel

2 months

Excited to contribute to @SevenDialsBooks ' popular science book series "10 Things You Should Know" with a book on AI. AI will be humanity's most transformative technology. In ten short and easy to digest essays written for the general public, I am giving an overview of what AI

Tweet media one

2

47

104

0

1

14

@edwardfhughes

Edward Hughes

5 months

Awesome to see fundamental discoveries about neural networks still being made. Never be afraid to ask a basic question!

@EliSennesh

Eli Sennesh

5 months

Huh. Weird.

6

33

242

0

1

14

@edwardfhughes

Edward Hughes

3 months

🌐 Open-endedness can be supercharged by "standing on the shoulders of giant human data" in the words of @jennyzhangzt & @jeffclune . 🤖 Foundation models provide the knowledge to guide open-endedness towards artifacts that are interesting, novel and useful to humans. 🧵[4/N]

1

1

13

@edwardfhughes

Edward Hughes

1 month

Awesome work on cultural evolution among LLMs! This will only become more relevant as advanced AI becomes embedded in society. We must understand multi-step multi-agent dynamics among LLMs to avoid undesirable attractors and realise the potential for LLM-powered innovation.

@Jeremy__Perez

Jérémy Perez

1 month

What happens when LLMs play the Telephone game? ☎️ In this new preprint, we analyse the evolution of texts as they are transmitted between LLM agents 🤖💬🤖💬🤖💬 Do text properties converge to attractors? 🧲 How is this influenced by the task📝 and model⚙️? 1/13🧵

5

38

132

0

0

13

@edwardfhughes

Edward Hughes

2 months

We are entering the era where evolutionary simulation (with foundation model guidance) can discover new science. This is just the start.

@ylecun

Yann LeCun

2 months

: an AI-for-proteomics startup that just came out of stealth. They are announcing ESM3 a 98B-paramter generative LLM for "programming biology." Using ESM3 and a simulated evolutionary process, they have produced a new type GFP (Green Fluorescent Protein)

49

260

1K

0

4

12

@edwardfhughes

Edward Hughes

11 months

I had the pleasure to be an internal reviewer on this excellent work. Very exciting to see self-referential self-improvement have such a significant impact on a wide range of LLM benchmarks. Another wonderful example of open-endedness "going mainstream"! Bravo to the authors!

@chrisantha_f

Chrisantha Fernando

11 months

🌱 Introducing Promptbreeder: LLMs evolve their own prompts through self-referential self-improvement! Paper: #PromptEngineering #LLM #AI #ML #Promptbreeder

Tweet media one

12

73

364

0

1

13

@edwardfhughes

Edward Hughes

8 months

An introduction to Foundation Models (mainly LLMs) with a hint of @coop_ai ! My 2023 summer school talk is now online here:

Tweet card media

A Foundation Model for Cooperative AI

This lecture was delivered at the 2023 Cooperative AI Summer School. For more information, please visit https://www.cooperativeai.com/summer-school/2023.Edwa...

www.youtube.com

0

1

13

@edwardfhughes

Edward Hughes

3 months

🔮 Open-endedness has the potential to reach superhuman intelligence. This raises vital ethical and safety considerations. ⚠️ AI systems must remain explainable and controllable: by our definition, AI systems which are incomprehensible are not open-ended. 🧵[5/N]

Tweet media one

1

1

12

@edwardfhughes

Edward Hughes

7 months

Super exciting to see the field take off like this!

@_samvelyan

Mikayel Samvelyan

7 months

The surge in #OpenEndedness research on arXiv marks a burgeoning interest in the field! The ascent is largely propelled by the trailblazing contributions of visionaries like @kenneth0stanley , @jeffclune , and @joelbot3000 , whose work continues to pave new pathways.

Tweet media one

3

19

121

0

0

12

@edwardfhughes

Edward Hughes

2 months

Beautiful, visionary work!

@ciaran_regan_

Ciaran

2 months

🥳 We released a new paper! 🥳 LLM-POET: Evolving Complex Environments using Large Language Models A new approach in open-ended evolution using LLMs 🧵

4

29

171

1

2

12

@edwardfhughes

Edward Hughes

6 months

Really pleased to see open-endedness research made more accessible with academic scale compute. A major bottleneck to progress is building a community of great open-endedness researchers across many institutions, and this will go a long way to addressing that challenge. Bravo!

@mitrma

Michael Matthews

6 months

I’m excited to announce Craftax, a new benchmark for open-ended RL! ⚔️ Extends the popular Crafter benchmark with Nethack-like dungeons ⚡Implemented entirely in Jax, achieving speedups of over 100x 1/

10

61

276

0

0

11

@edwardfhughes

Edward Hughes

4 months

It has been a great privilege to think deeply about the Ethics of Advanced AI Assistants with this incredible team of co-authors. For a multi-agent take, jump in at Chapter 14!

@IasonGabriel

Iason Gabriel

4 months

1. What are the ethical and societal implications of advanced AI assistants? What might change in a world with more agentic AI? Our new paper explores these questions: It’s the result of a one year research collaboration involving 50+ researchers… a🧵

Tweet media one

30

199

616

0

0

11

@edwardfhughes

Edward Hughes

3 months

🌍 Models like Gemini are generally capable but don't create new knowledge. 🪄 Advances like AlphaFold have revolutionised their fields but aren't fully general. ♾ How can we define and build AI capable of endless innovation in science and technology? 🧵[2/N]

1

0

8

@edwardfhughes

Edward Hughes

2 years

For those of you who follow the literature, this can be seen as a step forward for @jeffclune 's AI-GA agenda.

0

1

9

@edwardfhughes

Edward Hughes

2 years

For our full results reel, see !

Tweet card media

DeepMind Adaptive Agent: Results Reel

This result reel shows some of the learned behaviours of our Adaptive Agent (AdA). Please visit http://sites.google.com/view/adaptive-agent and read our full...

www.youtube.com

1

3

10

@edwardfhughes

Edward Hughes

2 months

This is just ridiculous. Will @Keir_Starmer do something about this wilful self-sabotage of our research ecosystem, if he is elected? He jolly well should do: innovation is the engine of growth!

@wellcometrust

Wellcome

2 months

New analysis from the @royalsociety shows just how high UK visa costs for researchers are compared to upfront costs for similar visa routes in 13 other countries. Here, we take a look at how the UK compares to countries in the G7 included in the analysis 🔍⤵️ 1/5

Tweet media one

20

646

886

2

1

10

@edwardfhughes

Edward Hughes

3 months

🧑‍🏫 To hear more about the paper please do come along to our @icmlconf oral in Vienna in July, or reach out to us! 🧵[8/N]

1

0

9

@edwardfhughes

Edward Hughes

2 months

There is far too much admin associated with paper publication. Globally we should lobby for changes in legislation to remove the need for unnecessary consent to publish and copyright waivers. Scientists just want their research read and built on!

3

0

9

@edwardfhughes

Edward Hughes

3 months

In this paper, we provide the first proof of concept for cultural evolution among RL agents. Our generational method outperforms "single lifetime" RL both in a few-shot and across training. In other words, culture begets innovation, along the lines of @jeffclune 's AIGAs! [4/n]

Tweet media one

1

1

9

@edwardfhughes

Edward Hughes

2 months

I find NetHack increasingly interesting. Its dynamics are both firmly rooted in human knowledge and require careful experimentation and reasoning. Visually and textually it is in the distribution tail for foundation models, so is a good discriminator for open-ended capabilities.

@_rockt

Tim Rocktäschel

2 months

Happy "AI still can't learn to play NetHack" day for those of you who celebrate. On this day in 2020, we released @NetHack_LE . Despite tremendous progress in AI over the last four years, this challenge is still very far from being solved. From our NeurIPS paper

17

41

171

0

0

9

@edwardfhughes

Edward Hughes

1 month

Excited to have arrived in Vienna for @icmlconf . Looking forward to a week of creative discussions, learning and deep thinking with friends new and old!

0

1

8

@edwardfhughes

Edward Hughes

6 months

Really interesting work - we saw in AdA that conditioning on # shots led to more expressive and adaptive policies. Awesome to see this principle lifted to the scale of an entire RL run!

@JacksonMattT

Matthew Jackson

6 months

Meta-learning can discover RL algorithms with novel modes of learning, but how can we make them adapt to any training horizon? Introducing our #ICLR2024 work on discovering *temporally-aware* RL algorithms! Work co-led with @_chris_lu_ , in @FLAIR_Ox and @whi_rl

1

25

111

0

0

8

@edwardfhughes

Edward Hughes

2 months

@nathanbenaich From a meme’s eye view, aren’t theorems living things that compete for resources and evolve on their own?

2

1

4

@edwardfhughes

Edward Hughes

6 months

Came across the #OpenEndedness related episode of the @lexfridman podcast with @leecronin . I particularly enjoyed this quote, which fits in very well with the philosophy of unsupervised environment design: “I’m excited because I think selection isn’t special at all. I think what

1

0

8

@edwardfhughes

Edward Hughes

5 months

Very cool work - exciting to see a new paradigm for search in imperfect information games, addressing the limitations of public belief state algorithms (which had always somehow jarred with me).

@ssokota

Samuel Sokota

5 months

SOTA AI for games like poker & Hanabi rely on search methods that don’t scale to games w/ large amounts of hidden information. In our ICLR paper, we introduce simple search methods that scale to large games & get SOTA for Hanabi w/ 100x less compute. 1/N

Tweet media one

5

52

335

0

0

8

@edwardfhughes

Edward Hughes

3 months

Cultural evolution is the fastest known intelligence generating mechanism in the universe, and the driving force behind human skills and technology. Recent work by luminaries like @JoHenrich , @CeliaHeyes and @mmuthukrishna have given us significant insight into this process.

1

0

8

@edwardfhughes

Edward Hughes

10 months

Honestly extremely impressive: (1) the underlying AI research, (2) the imaginative product realisations, (3) the organizational and ecosystem design that has enabled this, (4) the engaging, relatable presentation style.

Tweet card media

OpenAI DevDay: Opening Keynote

Join us for the opening keynote from OpenAI DevDay — OpenAI’s first developer conference.We’re gathering developers from around the world for an in-person da...

www.youtube.com

0

3

7

@edwardfhughes

Edward Hughes

6 months

Precisely - this is why open-ended discovery (building on foundation models) is the next frontier for general-purpose AI research.

@fchollet

François Chollet

6 months

There are roughly four levels of generalization: 0. No generalization (e.g. a database) 1. Having memorized *the answers* for a static set of tasks and being able to interpolate between them. Most LLM capabilities are at that level. 2. Having encoded generalizable programs

30

200

1K

0

1

7

@edwardfhughes

Edward Hughes

2 years

Excited to share our new paper!

@GoogleDeepMind

Google DeepMind

@GoogleDeepMind

2 years

Fast, flexible cultural transmission underpins human intelligence. Our team trains an AI capable of real-time cultural transmission in previously unseen navigation tasks. The agent follows expert demos & reproduces them reliably after the expert leaves: 1/

7

122

482

0

0

7

@edwardfhughes

Edward Hughes

3 months

In previous work in @NatureComms () we showed that agents could learn to learn from each other from on-the-fly: just like humans! But can this "social learning" generate an open-ended evolution of ideas in AI, just as it has for humanity? [3/n]

Tweet card media

Learning few-shot imitation as cultural transmission

Nature Communications - The modelling of human-like behaviours is one of the challenges in the field of Artificial Intelligence. Inspired by experimental studies of cultural evolution, the authors...

1

1

7

@edwardfhughes

Edward Hughes

6 months

Very exciting to see another example of autonomous self-improvement. 2024 will be remembered as the year that Open-Endedness went mainstream!

@tesatory

Sainbayar Sukhbaatar

6 months

🎉 New paper 🎉 We teach Transformers to do A* search (I had to relearn how A* works). Then, we're curious to see if it can self-improve, and it did surprisingly well. This direction of search, plan, self-improve is very exciting!

2

18

113

0

0

7

@edwardfhughes

Edward Hughes

3 months

🧑 In our definition, an observer judges artifacts to be open-ended when they are both *novel* and *learnable*. 🧬 Artifacts are novel when they are more surprising further into the future. Artifacts are learnable when they are more predictable given more history. 🧵[3/N]

Tweet media one

1

1

7

@edwardfhughes

Edward Hughes

6 months

In five years time we will think of foundation models like we think of operating systems: the next phase of transformative innovation will be on the level above.

0

0

7

@edwardfhughes

Edward Hughes

7 months

Awesome to see cultural evolution in human-AI systems receiving increasing attention: .

1

2

6

@edwardfhughes

Edward Hughes

2 months

This is extremely exciting work. For a while I have been concerned that humans adapt to AI much faster than AI adapts to humans. This is the first paper that I have seen which gracefully allows us to take into account that process. Critical for safety!

@MicahCarroll

Micah Carroll

3 months

Excited to share a unifying formalism for the main problem I’ve tackled since starting my PhD! 🎉 Current AI Alignment techniques ignore the fact that human preferences/values can change. What would it take to account for this? 🤔 A thread 🧵⬇️

Tweet media one

7

45

262

0

1

6

@edwardfhughes

Edward Hughes

6 months

Point of personal pride ✨: I wrote the gold implementation of the best published model, after several years away from IC coding as as a project lead. #unleading ftw!

0

0

6

@edwardfhughes

Edward Hughes

2 months

@francoisfleuret @ChrSzegedy Hope that's helpful - and excited about the potential for community building here! It would be really cool to think about combining your approach with some of the ones I've mentioned in this thread.

2

0

6

@edwardfhughes

Edward Hughes

7 months

Extraordinary work! #openendedness research is really gathering steam! (1/n)

Tweet card media

AlphaGeometry: An Olympiad-level AI system for geometry

Our AI system surpasses the state-of-the-art approach for geometry problems, advancing AI reasoning in mathematics

deepmind.google

1

0

6

@edwardfhughes

Edward Hughes

5 months

Late to the party, but the DPO paper is a great example of really impactful theoretical work. It's immensely valuable to find mathematical results that take short-cuts to better performance.

Tweet card media

Direct Preference Optimization: Your Language Model is Secretly a...

While large-scale unsupervised language models (LMs) learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult due to the completely...

1

0

6

@edwardfhughes

Edward Hughes

6 months

Tokenisation is an understudied art - it’s exciting to see what new capabilities can be unlocked when it is studied creatively and forensically!

@Aaditya6284

Aaditya Singh

6 months

Ever wondered how your LLM splits numbers into tokens? and how that might affect performance? Check out this cool project I did with @djstrouse : Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs. Read on 🔎⏬

10

33

181

0

1

6

@edwardfhughes

Edward Hughes

3 months

We test our agents on several environments, including a variant of the classic Goal Sequence by @natashajaques and @kandouss , and the Traveling Salesperson Problem. The results are robust, and I fully expect the benefits to scale up with industry compute! [5/n]

Tweet media one

1

0

5

@edwardfhughes

Edward Hughes

2 months

@francoisfleuret @ChrSzegedy A little further afield, the recent Quality Diversity from Human Feedback work by Li Ding et al. has a cultural evolutionary flavor to it for me. Wider point: the open-endedness and cultural evolution communities can learn a lot from each other!

Tweet card media

Quality Diversity through Human Feedback: Towards Open-Ended...

Reinforcement Learning from Human Feedback (RLHF) has shown potential in qualitative tasks where easily defined performance measures are lacking. However, there are drawbacks when RLHF is commonly...

1

0

5

@edwardfhughes

Edward Hughes

27 days

Personal reflection: one thing that I noticed in my time on the team was its incredible tenacity, individually and collectively. Most of what I contributed were negative results, and many ideas didn't work out. Sailing on bravely through these choppy waters was vital to success!

0

1

5

@edwardfhughes

Edward Hughes

3 months

Our methods bear a pleasing analogy to knowledge accumulation and skill accumulation in humans. This continues an increasingly rich tradition of building bridges between multi-agent AI and the social sciences, which @jzl86 and I have collaborated on for many years. [6/n]

1

0

5

@edwardfhughes

Edward Hughes

6 months

Happy International Women’s Day!

0

0

5

@edwardfhughes

Edward Hughes

2 months

True or false: not being able to keep track of all the literature is a good regulariser to keep your brain operating at the level of abstraction that is likely to yield novel research insight.

0

0

5

@edwardfhughes

Edward Hughes

2 months

And even then, some human communication (for physical coordination, with latency / throughput bottlenecks, across language barriers) takes place in few-shot without language in previously unseen situations.

@hardmaru

hardmaru

2 months

Language is primarily a tool for communication rather than thought “Language is a defining characteristic of our species, but the function, or functions, that it serves has been debated for centuries. Here we bring recent evidence from neuroscience and

92

336

2K

1

0

4

@edwardfhughes

Edward Hughes

2 months

@francoisfleuret @ChrSzegedy In the LLM space, there was nice work from INRIA recently on cultural evolution of stories among a population of large language models.

Tweet card media

Cultural evolution in populations of Large Language Models

Research in cultural evolution aims at providing causal explanations for the change of culture over time. Over the past decades, this field has generated an important body of knowledge, using...

1

0

4

@edwardfhughes

Edward Hughes

4 months

Very pleased to see @CBSNews describe with clarity and succinctness some of the societal-scale issues raised by myself and others in our recent GDM publication on the Ethics of AI Assistants:

Tweet card media

Exploring the ethics of advanced AI assistants

Artificial intelligence assistants may soon be able to do much more than play your favorite music or call your mom, but some Google researchers warn about po...

www.youtube.com

0

1

4

@edwardfhughes

Edward Hughes

4 months

@jeffclune @ylecun I have the same intuition as you @jeffclune : math is a (formal) language, and when I reason about this, most of what happens in my head is verbal ("use polar coordinates", "apply distance formulae" etc). If the problem were in 6D, reliance on language would be even more obvious.

3

0

2

@edwardfhughes

Edward Hughes

6 months

@MichaelD1729 @maxjaderberg @_rockt @jparkerholder @ashrewards @YugeTen Echoing Michael, the task creation bottleneck from XLand and XLand 2 was precisely why I chose to join the Genie team - and what a good choice that was! Thanks for the great analysis, Max.

0

0

4

@edwardfhughes

Edward Hughes

1 month

This is interesting, perhaps LLMs do meta-learn some social learning capabilities. I wonder (a) to what extent these can be used beneficially in applications, and (b) whether one can create synthetic data which extends this effect to other desirable meta-cognitive capabilities.

@DavidSKrueger

David Krueger

1 month

Congrats to the whole team! IIRC, our findings in this work blew my mind more than any other result. This may be the first evidence for the existence of a *mechanism* by which sufficiently advanced AI systems would tend to become agentic (and thus have instrumental goals).

2

9

84

0

0

4

@edwardfhughes

Edward Hughes

5 months

@polynoamial Unless you can meta-learn a self-improvement operator from that data? In principle if we had a dataset for the practice of science, one could learn the principles of making new scientific discoveries (though one would still need real-world measurement to validate and iterate).

0

0

1

@edwardfhughes

Edward Hughes

9 years

Have a new paper out today: intriguing structure in quantum corrections to soft theorems.

0

2

2

@edwardfhughes

Edward Hughes

2 months

Brilliant to see further work on many-player zero-sum games. A few years ago my team at GDM also had a foray into this space, showing that 3-player zero-sum games often contain social dilemmas - that’s why multi-party negotiations are hard!

Tweet media one

@chijinML

Chi Jin

2 months

Ever wonder how to play multiplayer games (>2 players, such as Mahjong, Poker) well and what would be the ultimate solution? Check our paper on why classical equilibria and existing self-play systems are not enough, and how to address it:

3

11

49

0

1

3

@edwardfhughes

Edward Hughes

2 months

Great leadership requires at least two things. (1) Humility of the leader to listen to ideas and course-correct when they are mistaken. (2) Generosity of the team to assume good intent and pro-actively provide feedback. Both are broken in contemporary UK politics.

0

0

3

@edwardfhughes

Edward Hughes

6 months

Which makes me wonder, to what extent can Assembly Theory be used as a blueprint for AI open-endedness research?

0

0

3

@edwardfhughes

Edward Hughes

6 months

Moravec's paradox is perhaps not so paradoxical when viewed through the lens of structured data. Mathematics, language, games provide abstractions which naturally compress and accumulate, accounting for the speed of cultural evolution (and the ability of large language models).

@ylecun

Yann LeCun

6 months

@soumithchintala I keep talking about the Moravec paradox in my talks. We have AI systems that can pass the bar exam, but where is the domestic robot that can clear up the dinner table and fill the dishwasher? A task that any 10 year old can learn in one shot. Obviously, we are missing something

41

33

336

0

0

3

@edwardfhughes

Edward Hughes

6 months

This is an interesting, and perhaps not too surprising development. Thus far RLAIF has not benefitted fully from the kinds of ideas that led to open-ended self-improvement in the pre-LLM era (leagues, autocurricula, search). Once it does, the landscape will change.

@natolambert

Nathan Lambert

6 months

A few big papers throwing question on "does RLAIF work" yesterday. The first is a paper by @archit_sharma97 is a pretty timely critique of RLAIF. It shows SFT on GPT 4 outputs > DPO + RLAIF on GPT4 ratings of GPT3.5 completions. A few things aren't surprising: 1. The most

Tweet media one

Tweet media two

Tweet media three

7

42

166

0

0

3

@edwardfhughes

Edward Hughes

9 years

Chapter 1: A Long Expected Paper. #LOTRyourResearch

0

4

3

@edwardfhughes

Edward Hughes

2 months

Both AI scientists and politicans can learn a lot from this. In general, when you encounter multiple options you should first seek out an implement all possible Pareto improvements and *only then* seek trade-offs.

@RyanBoldi

Ryan Boldi

2 months

Excited to share our new paper on Pareto Optimal Preference Learning (POPL)! 🎉 POPL aims to better align AI with diverse human values by building diverse sets of reward functions or policies! Work done with @li_ding_ , Lee Spector and @scottniekum

Tweet media one

1

15

74

0

0

3

@edwardfhughes

Edward Hughes

2 months

Large language model evaluations tend to focus on “correct answers”. Yet human communication is better modelled as providing responses which are “good enough” given the context and compute constraints. What will be the consequences of this mismatch?

0

1

3

@edwardfhughes

Edward Hughes

2 months

@jparkerholder Agreed. I’ve lost track of the time I’ve had to disclaim something. And like 100% of other people, I simply clicked agree and signed without reading it. What a waste of time for us and for the conference organisers.

1

0

3