Enrique Piqueras @epiqueras1 Twitter profile

Last Seen Profiles

@RaymondAmpil

@cos_ChoS

@sui1121885

@privy_io

@Richard_URD

@KudrinOleg

@bokeplokalmalam

@BinorRaja

@runman317

@Dreaxana

@VKM_Criss

@SylpheoB

@stwmaniax

@Obededimu

@puutttttttt

@kentulinu

@showasangyo_co

@Celia__R84

@MRC_LMB

@bokeplokalmalam

@scener

@stw46

@Clement87333202

@haiagarumono

@zakbasch

@KM__arch

@stwmaniax

@pasurtihot

@awadh406

@mbwbbwb185161

@Hijabbacol2883

@BinorRaja

@landofernandog

@stw_pdg

@_odsc

@stw_pdg

Enrique Piqueras

@epiqueras1

11 months

I grew up with Google. When life dropped me in difficult situations, Search gave me a doorway into a world that otherwise seemed so out of reach. I have my entire life to be grateful for. Working tirelessly with this team to get to this milestone has been a privilege I don’t…

Jeff Dean (@🏡)

@JeffDean

11 months

I’m very excited to share our work on Gemini today! Gemini is a family of multimodal models that demonstrate really strong capabilities across the image, audio, video, and text domains. Our most-capable model, Gemini Ultra, advances the state of the art in 30 of 32 benchmarks,

273

3K

13K

8

10

632

Enrique Piqueras

@epiqueras1

2 years

Behold, the power of JAX and XLA!

GitHub - AI-Hypercomputer/maxtext: A simple, performant and scalable Jax LLM!

A simple, performant and scalable Jax LLM! Contribute to AI-Hypercomputer/maxtext development by creating an account on GitHub.

github.com

3

27

196

Enrique Piqueras

@epiqueras1

11 months

They’re onto us!

near

@nearcyan

11 months

gemini paper leaks names of the Chosen Six who are now fated to be merged with gemini when it reaches its final form

14

77

1K

7

3

185

Enrique Piqueras

@epiqueras1

8 months

JAX is the infrastructure of the next scientific revolution.

6

17

145

Enrique Piqueras

@epiqueras1

8 months

I used Gemini 1.5 Pro to write code so that Gemini 1.5 Pro could see all my tabs so that Gemini 1.5 Pro could assist me in making Gemini 1.5 Pro faster so that I could use Gemini 1.5 Pro to write code… I remember in the past when prototyping this I had to spend a painful amount

3

8

118

Enrique Piqueras

@epiqueras1

6 months

GenUI is Web 4.0 @rauchg doesn’t miss.

6

11

116

Enrique Piqueras

@epiqueras1

9 months

@swyx @polynoamial @_sholtodouglas @GoogleDeepMind I’ll do it for him. He has a deep understanding of compute constraints and recent research that he carries in a very Richard Hamming TADSE like way. I’ve benefited tremendously from mirroring this from him and @jekbradbury . With this at least a couple of times he’s burnt the…

3

0

98

Enrique Piqueras

@epiqueras1

8 months

TPUs and the abstractions built on top of them are so good that they are almost ahead of their time. It’s hard to realize this without scaling to hundreds and thousands of devices, but once you do it’s clear parallelism and fusions being declarative and compiler driven are a

4

2

81

Enrique Piqueras

@epiqueras1

8 months

“I know I should be using Jax. A friend just shamed me for using pytorch, but this is a startup and we decided to move fast. I’m sorry we would do better to be cooler next time. I’m not proud of this fact.”

Training great LLMs entirely from ground up in the wilderness as a startup — Yi Tay

Chronicles of training strong LLMs from scratch in the wild

www.yitay.net

2

3

71

Enrique Piqueras

@epiqueras1

11 months

@_sholtodouglas @0interestrates @nearcyan My first real/long interaction with him when he started coming in to help on Gemini was staying at the office until 1am staring at a Colab, fixing a gnarly numerics bug. He grasped my code in minutes. Top tier engineer, scientist, and a pleasure to work with.

0

2

68

Enrique Piqueras

@epiqueras1

8 months

When @sharadvikram started Pallas, his dream was to never have to write a kernel in C++ again. Together with the gembros, we finally achieved that (internally). Pallas now lets you define manual TPU pipelines in Python, and compose them. For example, we’ve been able to

1

5

68

Enrique Piqueras

@epiqueras1

8 months

JAX + Pathways + XLA + TPUs let us optimize new models for serving in days or weeks instead of having to spend months writing kernels every time a researcher changes something.

Emad

@EMostaque

8 months

With the v5e's being such a scalable chip and so low cost being landed in the millions Google can undercut the pricing of anyone else for GPT-4 level models & still make a profit. The large context window attention mechanism is also ideal for the v5e-256 pods vs GPUs..

15

20

249

1

6

59

Enrique Piqueras

@epiqueras1

11 months

Composable. Functional. Transformable. Multi-backend. Just works 😊 at scale. Can drop to manual per device code seamlessly. Juice everything out of the hardware with Pallas. So much to love and so much to be excited for in the future of JAX.

Sharad Vikram

@sharadvikram

11 months

Built with JAX!

5

26

294

1

3

58

Enrique Piqueras

@epiqueras1

4 months

It’s been quiet at the lab, with 2x more CE than the next, since @_sholtodouglas got deported for codingcodingcoding too much instead of working on his visa. But, he’ll be back soon and we’ve been cooking. 📸 @CharlieXYChen chief photo officer, 3d printer in residence, and all

2

1

57

Enrique Piqueras

@epiqueras1

7 months

For those that dare.

3

4

53

Enrique Piqueras

@epiqueras1

8 months

If people knew and understood the details of all the things people make fun of Google for. They would get a 5 second haha and we would move on. Things are not always that deep. Most of the time they’re not. Google has great leadership that loves people and wants the best for

Guillermo Rauch

@rauchg

8 months

Can I say something without people getting mad. I’m bullish on Google. They helped lay down the foundations of the AI revolution. Long history of open source and research contributions. They have amazing people. Lots to be grateful for and optimistic they’ll iron out the kinks.

65

27

646

5

53

Enrique Piqueras

@epiqueras1

7 months

I still don’t know how to make a website at Google, but hey, with only a few hundred lines of Beam I was able to wield thousands of resting TPUs across North America to run some JAX in a multi step pipeline. Set and forget style. Pretty good tech tree tradeoff for the times if

7

2

53

Enrique Piqueras

@epiqueras1

9 months

@swyx @polynoamial @_sholtodouglas @GoogleDeepMind @jekbradbury midnight oil days on end to save the team from making a big mistake or overlooking a big opportunity on compute efficiency. He was also a key part of the design and development of the inference engine and infrastructure. People who can both think like this and change things…

1

0

50

Enrique Piqueras

@epiqueras1

1 month

Alpha

3

1

47

Enrique Piqueras

@epiqueras1

5 months

Michael and team have been working incredibly hard for this moment. It has been awesome to see.

Google DeepMind

@GoogleDeepMind

5 months

We watched #GoogleIO with Project Astra. 👀

70

235

1K

0

7

44

Enrique Piqueras

@epiqueras1

9 months

Sholto, you are doing wonders for my imposter syndrome 🤗 I can’t emphasize enough how important this kind of inclusivity is for everyone and how thankful I am for all the countless hours of mentorship I’ve received from experts in various fields. I hope one day AI can open…

Sholto Douglas

@_sholtodouglas

9 months

He’s 25, and didn’t go to college! That same colleague speculated ‘Maybe he just doesn’t know to be scared of things’. Both of us have benefited immensely from the kindness and time of everyone at @GoogleDeepMind . Brilliant experts will spend hours patiently explaining almost

4

3

84

1

0

41

Enrique Piqueras

@epiqueras1

2 months

Adulthood brings the realization that the natural state of things is decay. By being productive, we’re not just contributing to society, we’re locally clawing back entropy for the good of all. In a way, training a model reflects this, bringing order through entropy minimization,

1

5

41

Enrique Piqueras

@epiqueras1

9 months

@swyx @polynoamial @_sholtodouglas @GoogleDeepMind @jekbradbury and change things with their own hands are rare.

0

40

Enrique Piqueras

@epiqueras1

11 months

have the words to describe. This is the next era of computing. What Search did for knowledge we can now do for expertise. Something still stuck in the manuscript era for the majority of the world. Now back to work! We will continue to organize the world's information and make…

2

1

37

Enrique Piqueras

@epiqueras1

5 months

If you’re at a IO. Go try it out at the AI booths!

Google DeepMind

@GoogleDeepMind

5 months

We’re sharing Project Astra: our new project focused on building a future AI assistant that can be truly helpful in everyday life. 🤝 Watch it in action, with two parts - each was captured in a single take, in real time. ↓ #GoogleIO

221

1K

4K

2

5

35

Enrique Piqueras

@epiqueras1

8 months

JAX, Like Linux in 2004.

theseriousadult

@gallabytes

8 months

Jax on TPU is such a lovely contrast to everyone's complaints about Torch on GPU. Feel like I'm running a Linux webserver in 2004 - this is so much less jank than the market-leading madness, but people haven't yet switched en-masse due to some combination of not knowing that

9

6

110

2

34

Enrique Piqueras

@epiqueras1

7 months

Too much AGI.

4

0

31

Enrique Piqueras

@epiqueras1

7 months

Might be the first podcast I’ll watch from start to finish.

Dwarkesh Patel

@dwarkesh_sp

7 months

Had so much fun chatting with my friends @TrentonBricken and @_sholtodouglas . No way to summarize it, except: This is the best context dump out there on how LLMs are trained, what capabilities they're likely to soon have, and what exactly is going on inside them. You would be

39

125

1K

0

1

29

Enrique Piqueras

@epiqueras1

4 months

I always keep 2M tokens cached at all times.

0

2

26

Enrique Piqueras

@epiqueras1

8 months

I’d much rather code around fixed shapes than have to manually deal with scheduling and memory space assignment. Still, I think partial support of ragged shapes could be the optimal point for JAX. What are some cases where you’ve written off JAX for fixed shape reasons. I bet

8

1

24

Enrique Piqueras

@epiqueras1

5 months

tree map! pjit! shmap! pallas call! You look away for 1 day and the team shaves off another millisecond. They are going to get the model to start predicting the future.

Melvin Johnson

@melvinjohnsonp

5 months

Such an impressive model from every angle: cost, latency and quality. @philhchen and the entire inference crew have been critical.

0

2

27

1

3

22

Enrique Piqueras

@epiqueras1

8 months

The part about being rusty is humbleness. He fixed a bug in Pallas last week as fast as a Principal Engineer. Also when he first joined the team he spent most of his time listening and asking good questions. Now he’s right ahead of us so often that he’s basically a pre-training

Sankeerth Rao Karingula, Ph.D.

@sankeerth1729

8 months

It was so inspiring to listen to Sergey talk about AGI, Gemini other Google initiatives and honestly answer so many questions from everyone!

34

101

567

0

3

21

Enrique Piqueras

@epiqueras1

7 months

Heard this week: “Somehow in the past week the MK turned into grad school. I feel like a TA again. What’s going on here? Is this a pysop to make me and JAX Matt nostalgic?” - JAX Roy “Stack more layers” they said. “Forget about your reproducing kernel Hilbert spaces” they said.

0

3

22

Enrique Piqueras

@epiqueras1

11 months

it universally accessible and useful. It gets better and better!

0

21

Enrique Piqueras

@epiqueras1

9 months

First rough draft/sneak peek of a new Pallas nested kernel API we’ve been working on:

2

0

19

Enrique Piqueras

@epiqueras1

8 months

@rbhar90 @deep_chem @deepforestsci JAX is different. If Google didn’t exist, the core team would simply be working on it elsewhere. It grew organically with full founder authority and runs out of GDM instead of the more traditional way of running things from the infrastructure organization.

4

1

18

Enrique Piqueras

@epiqueras1

8 months

we’ve been here for years more coming 😊

Malte Ubl

@cramforce

8 months

You can call it a comeback. Amazing to have a third really good model in the open-source space

0

20

1

0

18

Enrique Piqueras

@epiqueras1

1 month

Respect your opportunities!

0

14

Enrique Piqueras

@epiqueras1

8 months

Maybe my new favorite LLM demo.

Jeff Dean (@🏡)

@JeffDean

8 months

Analyzing and reasoning about videos Analyzing videos is another great capability brought by the fact that Gemini models are naturally multimodal, and this becomes even more compelling with long contexts. Consider Gemini 1.5 Pro’s ability to analyze movies, like Buster Keaton’s

6

30

260

0

14

Enrique Piqueras

@epiqueras1

8 months

🧠♊️

Google

@Google

8 months

Bard is becoming Gemini, and we’re launching two new experiences: 1️⃣ Gemini Advanced, which gives you access to Ultra 1.0, our most capable AI model 2️⃣ A new mobile app for easier collaboration on the go Learn more ↓

251

680

4K

0

13

Enrique Piqueras

@epiqueras1

7 months

Flops, always has been.

MatX

@MatXComputing

7 months

Introducing MatX: we design hardware tailored for LLMs, to deliver an order of magnitude more computing power so AI labs can make their models an order of magnitude smarter. Our hardware would make it possible to train GPT-4 and run ChatGPT, but on the budget of a small startup.

25

73

454

1

0

13

Enrique Piqueras

@epiqueras1

6 months

Peak hours at an excellent kitchen can be incredibly inspirational. It’s beautiful to see people working so well together to make.

Muddy

@feelmuddy

6 months

T-9 “Intensity”, taken in the kitchen one year before Atelier Crenn is awarded 3 Michelin Stars for the first time. First woman to three stars took an exceptional team who believed that matters

1

4

11

0

11

Enrique Piqueras

@epiqueras1

3 months

Rare footage of pair programming with @sharadvikram on a kernel.

2

1

11

Enrique Piqueras

@epiqueras1

9 months

@duttakapil @_sholtodouglas @GoogleDeepMind I can think of 2, 3 month periods that were very low points where I made it out thanks to old friends and family. Some sort of community is always necessary imo. One CEO I worked for was particularly helpful in helping me deal with stress and social anxiety in particular.

0

10

Enrique Piqueras

@epiqueras1

7 months

The gmm dot py!

Trevor Gale

@Tgale96

7 months

I’m not done with MegaBlocks 😁 @apaszke @epiqueras1 @sharadvikram and I just dropped something we’ve been working on for a bit yesterday. MegaBlocks + JAX + TPU = MegaBlox 🔥

4

11

101

0

10

Enrique Piqueras

@epiqueras1

10 months

Too cheap to meter @_sholtodouglas 😉

rohan anil

@_arohan_

10 months

1

4

50

0

10

Enrique Piqueras

@epiqueras1

6 months

My wife thinks it’s really cute how these resemble toddlers learning to play soccer. They’d be a fun gift sometime in the future.

Sundar Pichai

@sundarpichai

6 months

New research from @GoogleDeepMind brings together soccer and robotics. Using reinforcement learning, robots display agile and reactive movements similar to a soccer player, no shin guards needed:)

117

130

1K

2

0

9

Enrique Piqueras

@epiqueras1

8 months

A big enabler of this is being highly sequential and relying on software pipelining for hiding memory latency instead of the hierarchical many threaded model of GPUs.

Enrique Piqueras

@epiqueras1

8 months

TPUs and the abstractions built on top of them are so good that they are almost ahead of their time. It’s hard to realize this without scaling to hundreds and thousands of devices, but once you do it’s clear parallelism and fusions being declarative and compiler driven are a

4

2

81

0

9

Enrique Piqueras

@epiqueras1

6 months

JAX is

JAX Intro by @idylliclp484 | Suno

Sad girl piano ballad; jazz-trained female singer-songwriter; epic cinematic song. Listen and make your own with Suno.

suno.com

1

0

9

Enrique Piqueras

@epiqueras1

3 months

Flops are to research what calories are to dieting.

1

0

9

Enrique Piqueras

@epiqueras1

4 months

r4j. Should I do it?

1

0

8

Enrique Piqueras

@epiqueras1

7 months

Naaabla 🤙 🤙 🤙

0

5

Enrique Piqueras

@epiqueras1

9 months

One of the best things that ever happened to me was having @aranibatta drag me out to the bay to double down on building things with this team. I am always in awe of their consistency, care, product sense, and timeless aesthetics. I can’t wait to use this. On my birthday too!

Muddy

@feelmuddy

9 months

On April 16, 2024, @sfsailingclub is introducing Muddy.

5

9

65

1

0

8

Enrique Piqueras

@epiqueras1

9 months

🥹 I hope one day soon I can achieve your focus and calm when things go wrong. It’s an incredible asset for any team and for life in general. (And your Pickleball serves.)

Melvin Johnson

@melvinjohnsonp

9 months

@_sholtodouglas and @epiqueras1 definitely belong on this list. They’re the driving force behind the scenes, playing a pivotal role in the success of projects like Gemini and serving as a source of inspiration for all of us. They’re highly productive and a delight to work with.

0

2

16

1

0

8

Enrique Piqueras

@epiqueras1

2 years

A 1000 LOC SOTA single-file arbitrarily scaleable transformer:

1

7

Enrique Piqueras

@epiqueras1

3 months

Sharad Pallas and Adam Mosaic go vrooooom.

Adam Paszke

@apaszke

3 months

Many of you are excited about H100 attention, so it’s a good time to show you Mosaic GPU: a Python DSL for H100s. The attention example matches FA3 performance, while being only ~200 lines of Python: It's easy to install too! Latest JAX packages have it.

14

112

678

0

7

Enrique Piqueras

@epiqueras1

9 months

@duttakapil @_sholtodouglas @GoogleDeepMind Mostly reading deeply trying to understand one complex thing fully and then digesting 1:1 with a mentor until things clicked. I’ve had OCD and bad anxiety since I was 8ish. I went to a bit of therapy and read about CBT to apply it myself to the point where it rarely impedes me.

2

0

7