Omar Kilani @omarkilani Twitter profile

Pinned Tweet

Omar Kilani

@omarkilani

18 years

Hello, world.

2

0

3

Last Seen Profiles

@GrazzelComDoisZ

@superstar132941

@Waepicah123

@AldhybQlb

@bokeplokalmalam

@ngentot_stw

@cassxsae

@asallime

@penyukastw21

@pasifmasor27

@bokeplokalmalam

@4A07C9o7R4w5eWo

@bokeplokalmalam

@ella_silvermfc

@bokeplokalmalam

@Sasha_Etkind

@bosbos8q

@MattGinella

@Xur5cow5dF4Jm

@jandakembangstw

@stew3434

@ibubohay2

@somdelhospi

@ArenAriya

@BinorRaja

@Tante_Binal69

@penyukastw21

@AlexArriola77

@NehaShu29632112

@MaulinaMuzirwan

@Tante_Binal69

@shinji_bostero

@_Sarcasticindia

@CeliaE3453

@Jarczyn95

@jandakembangstw

Omar Kilani

@omarkilani

4 months

400t/s L3 70b has been achieved internally.

11

12

168

Omar Kilani

@omarkilani

2 months

Huge day at @GroqInc ! 🚀 Our world-class engineering team has been relentlessly advancing the field of AI inference. Today, their hard work pays off as we secure $640M in funding. Massive kudos to the team! 🫡

Techmeme

@Techmeme

3 months

AI chip startup Groq raised a $640M Series D led by BlackRock at a $2.8B valuation, up from $1B after raising $300M in 2021, and adds an Intel executive as COO ( @vandermey / Bloomberg) 📫 Subscribe:

8

34

237

0

6

66

Omar Kilani

@omarkilani

5 months

The @GroqInc compiler team are all literal geniuses. We improved Mixtral 8x7 t/s/u by an entire GPT-4o t/s/u (474 to 585 median) with compiler improvements. Just getting started. 🫡

Gavin

@GavinSherry

5 months

We just pushed another optimization to @MistralAI Mixtral 8x7 to @GroqInc . Users will see a ~20% throughput improvement 🙌. These enhancements are driven by compiler software team’s relentless focus on throughput and latency.

7

66

4

11

63

Omar Kilani

@omarkilani

5 months

L3 8B at @GroqInc , a timeline: Last Friday: 946t/s Today at 3:19p: 1157t/s Today at 5:44p: 1270t/s Somehow, though, this is just the start.

Gavin

@GavinSherry

5 months

Good suggestion by pundits to run on fewer chips. Turns out that made @GroqInc faster 🏎️🏎️🏎️

2

7

33

2

5

42

Omar Kilani

@omarkilani

5 months

@yacineMTB We made huge latency improvements across the board this week. More to come. 🫡

2

0

38

Omar Kilani

@omarkilani

5 months

Casually shipping 348t/s of 70B goodness on a Friday afternoon.

Gavin

@GavinSherry

5 months

The @GroqInc team just shipped some optimizations pushing per user tokens per second higher for @AIatMeta Llama 3 70b. Looking forward to seeing what everyone builds this weekend.

3

6

46

2

3

36

Omar Kilani

@omarkilani

3 months

@jiayq @GroqInc We really appreciate the kind words! 🫡

1

2

29

Omar Kilani

@omarkilani

5 months

Tokens go burrr 😍😍😍

Groq Inc

@GroqInc

5 months

We're working hard to deploy more GroqRacks to serve the dev community's growing demand! 🚀

15

21

232

1

23

Omar Kilani

@omarkilani

5 months

LFG. @GroqInc 🤝 @Cloudflare 🎉

Cloudflare Developers

@CloudflareDev

5 months

AI gateway now supports @GroqInc and @Cohere ! Unleash the full potential of your language model, no matter where you are. 👂 We're all ears - let us know which providers or features you'd like to see next!

2

18

109

0

2

22

Omar Kilani

@omarkilani

2 months

We at @GroqInc are thrilled to have the ridiculously talented @AarushSah_ join us full time — Aarush just celebrated his 18th birthday. 🤯

Aarush Sah

@AarushSah_

2 months

Internship got cut a little short. Happy to share that I'm now full-time at @GroqInc - LET'S COOK

50

16

930

1

0

21

Omar Kilani

@omarkilani

5 months

Come make @GroqInc even faster and better. We’re not stopping until we hit 0ms TTFT.

Gavin

@GavinSherry

5 months

We’re expanding the @GroqInc team. If you’re a no nonsense engineer able to do deep systems work with great intensity, DM me.

6

3

38

0

2

19

Omar Kilani

@omarkilani

5 months

L3 8B in production running at 1157t/s/u with the full 8k context window. Only at @GroqInc . 🫡

sunny madra

@sundeep

5 months

The beauty of the @GroqInc design is that it can always be faster... #thefastestinference on, 14nm technology 😉

6

3

45

0

2

19

Omar Kilani

@omarkilani

3 months

. @GroqInc is making agentic workloads a reality— at Groq speed. Huge ship from @RickLamers 🫡

Rick Lamers

@RickLamers

3 months

I’ve been leading a secret project for months … and the word is finally out! 🛠️ I'm proud to announce the Llama 3 Groq Tool Use 8B and 70B models 🔥 An open source Tool Use full finetune of Llama 3 that reaches the #1 position on BFCL beating all other models, including

74

236

1K

1

2

18

Omar Kilani

@omarkilani

5 months

Are you: 1. A world class software engineer. 2. Obsessed with performance optimization. 3. Into helping build the world's fastest inference engine. Nice. 🫡 @GroqInc is hiring distributed systems engineers: Join us on our quest to 0ms TTFT.

Careers - Groq is Fast AI Inference

At Groq, we believe AI will change humanity forever, and that making it affordable and universally accessible is the key to human agency in an AI

groq.com

1

16

Omar Kilani

@omarkilani

4 months

👀

0

1

17

Omar Kilani

@omarkilani

5 months

If you feel like this is still too high — me too. We’re hiring distributed systems engineers. DM for info. 🫡

sunny madra

@sundeep

5 months

Latency 😉😜😘 @GroqInc

4

6

39

1

14

Omar Kilani

@omarkilani

5 months

🫡

Artificial Analysis

@ArtificialAnlys

5 months

Groq extends its lead and is serving Llama 3 8B at almost 1,200 output tokens/s! @GroqInc 's Llama 3 8B speed improvements seen in their chat interface we can now confirm are reflected in performance of their API. This represents the fastest language model inference performance

5

11

62

1

2

13

Omar Kilani

@omarkilani

4 months

We cranked the input speed on this one to 11, thanks to the ingenuity of the @GroqInc compiler team. 🫡

Artificial Analysis

@ArtificialAnlys

4 months

Fast to launch & very fast output speed! Groq has launched their Gemma 2 9B offering and is serving it at ~600 output tokens/s Gemma 2 9B is worthy alternative to Llama 3 8B and other smaller models. It is particularly attractive for generalist and communication-focused

4

22

70

0

3

14

Omar Kilani

@omarkilani

9 months

My guy @ianlandsman laying out the reasons why Section 174 is an extinction level event for a lot of small software companies.

Ian Landsman

@ianlandsman

9 months

I have a few quotes about 174 in Politico's Morning Tech today.

2

4

29

0

2

14

Omar Kilani

@omarkilani

5 months

If you’re the first, @GroqInc is hiring:

Careers - Groq is Fast AI Inference

At Groq, we believe AI will change humanity forever, and that making it affordable and universally accessible is the key to human agency in an AI

groq.com

EvilMog® @mog.evil.af

@Evil_Mog

5 months

35

122

744

1

12

Omar Kilani

@omarkilani

6 months

New @Waymo map + “Ambient Vibes” … awesome.

1

0

12

Omar Kilani

@omarkilani

6 months

@legolasyiu This isn’t even our final form. :)

2

1

10

Omar Kilani

@omarkilani

5 months

Rick shipped a thing: streaming tool calls @GroqInc . 🚢 Always deploy on Fridays (trying to get Rick to YOLO more often).

Rick Lamers

@RickLamers

5 months

I shipped a thing! On Friday, haha yes I’m crazy

7

3

51

1

2

10

Omar Kilani

@omarkilani

5 months

“Invented” real time username availability checks on signup forms.

Malcolm Harris

@BigMeanInternet

5 months

What's something in the world you know you're directly responsible for but if you were to claim credit you'd sound crazy?

995

251

4K

2

1

9

Omar Kilani

@omarkilani

5 months

A lot of very late nights went into this but those TTFT numbers are 😍. More to come.

sunny madra

@sundeep

5 months

The fastest . Ai ⚡️⚡️⚡️

9

71

2

9

Omar Kilani

@omarkilani

8 months

The fate of small software companies hanging in the balance on this vote…

Erik Wasson

@elwasson

8 months

TAX: @SenSchumer tells me he intends to bring business / child tax bill to the floor for vote

2

21

72

0

3

7

Omar Kilani

@omarkilani

9 months

It really sucks that thousands of software companies and their owners, employees, etc are beholden to the dumbest people imaginable, but that’s where we are I guess. H.R. 7024 is the easiest win-win-win to come out of Congress in 2 years. Just pass it.

Burgess Everett

@burgessev

9 months

News: Senate Minority Whip Thune says Senate Republicans will block the House-passes tax deal without an opportunity to amend it on the floor or in committee. Says GOP wants changes to child tax credit work requirements

25

54

110

0

6

Omar Kilani

@omarkilani

4 months

Join us on our quest to make all these numbers better every day:

Careers - Groq is Fast AI Inference

At Groq, we believe AI will change humanity forever, and that making it affordable and universally accessible is the key to human agency in an AI

groq.com

sunny madra

@sundeep

4 months

Our engineering team is cooking. Latency, Throughput, and Quality all keep improving! Across different models.

10

19

95

0

3

9

Omar Kilani

@omarkilani

9 months

Section 174 is one step closer to getting fixed. 🎉

Richard Rubin

@RichardRubinDC

9 months

And it’s done. Tax bill passes 357-70.

6

14

93

0

1

8

Omar Kilani

@omarkilani

4 months

@swyx [insert disclaimer about parameters, variance, non-determinism, macro averages, MoE, etc etc here]

0

1

8

Omar Kilani

@omarkilani

7 months

@ianlandsman For any given question, the answer is always: “just use Postgres”.

1

8

Omar Kilani

@omarkilani

10 months

Have travelled over 1,000 miles in a @Waymo A mind blowing, magical feat of engineering and one of the most important advances in tech in a long time.

Waymo

@Waymo

10 months

Thanks to our Waymo One riders for riding with us more than ever before this year. We can't wait for what's to come in 2024!

5

27

124

2

0

7

Omar Kilani

@omarkilani

5 months

@rudiranck @ArtificialAnlys @ylecun @GroqInc Just circling back on this one. 🫡

0

1

7

Omar Kilani

@omarkilani

7 months

@NWSBayArea

0

7

Omar Kilani

@omarkilani

5 months

@MingXDynasty @GroqInc Awesome. IMHO, competition is great, and it’s nice to see what the H200 can do at scale. OpenAI should simply run GPT on the LPU. :) (The exciting thing about this is that we’re still super early in the LPU performance story.)

2

1

7

Omar Kilani

@omarkilani

6 months

@ilkerndaskin @dr_cintas Groq streams. :)

3

1

7

Omar Kilani

@omarkilani

9 months

@samcraigjohnson 99% of SaaS co’s can get to 10M+ users on a single Scale-A5 from @OVHcloud_US for $663/m (maybe get a couple for redundancy, Postgres, etc).

0

7

Omar Kilani

@omarkilani

5 months

We’re doing an AMA! 🫡 Stop by and ask your Groq Q’s. 🤔

Groq Inc

@GroqInc

5 months

Want to know how Groq can scale to accommodate the growing demand for inference and how the scaling limitations of traditional legacy architectures can be overcome? Tune in on June 5 to find out at our upcoming AMA.

0

3

23

0

1

6

Omar Kilani

@omarkilani

3 months

🫡

AI at Meta

@AIatMeta

3 months

@GroqInc Impressive work from the Groq team! 👏

2

8

61

0

7

Omar Kilani

@omarkilani

3 months

LFG 🫡

AI at Meta

@AIatMeta

3 months

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet. Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context

284

1K

6K

0

6

Omar Kilani

@omarkilani

11 months

@golang Happy birthday! 🎉

0

1

Omar Kilani

@omarkilani

6 months

@koltregaskes @GroqInc Coming very soon! (The playground does support it in the meantime.)

1

6

Omar Kilani

@omarkilani

6 months

@yacineMTB We can make this happen.

0

6

Omar Kilani

@omarkilani

4 months

Went on a walk…

1

0

5

Omar Kilani

@omarkilani

2 months

@AarushSah_ 🫡

0

4

Omar Kilani

@omarkilani

4 months

@brianwilt Common Waymo enthusiast knowledge tbh.

0

4

Omar Kilani

@omarkilani

5 months

🫡

Laura Modiano

@LauraModiano

5 months

The winner of the Paris @cerebral_valley hackathon was definitely @GroqInc Most requested and seen on screen

1

8

43

0

4

Omar Kilani

@omarkilani

9 months

Promising signs for Section 174 (if the Senate does their job).

Vice President Kamala Harris

@VP

9 months

Good news: The Child Tax Credit bill is headed to the Senate. While @POTUS and I continue to fight for the full expanded Child Tax Credit, this bill should be passed quickly. President Biden is ready to sign it into law.

734

2K

6K

0

3

Omar Kilani

@omarkilani

8 years

@ianlandsman just gonna leave this here for you

0

3

4

Omar Kilani

@omarkilani

5 months

That bat man is incredible @geeksplainer (Multi modal is coming to @GroqInc :)

sunny madra

@sundeep

5 months

preview: multimodal model on @GroqInc 👀👀👀

22

21

227

0

3

Omar Kilani

@omarkilani

9 months

@CFDevelop I agree, but the issue (IMHO) is that ~5% of people have the self motivation to work like that.

1

0

3

Omar Kilani

@omarkilani

10 months

Can’t believe this hasn’t been fixed yet. Everyone just keeps hoping Congress fixes this and we’re already at Dec 13. Literal death sentence for most small tech companies. Even worse for LLCs and S-Corps as you personally generate highly inflated phantom income that gets taxed.

1

4

Omar Kilani

@omarkilani

3 months

@freemanjiangg @AarushSah_ LFG @AarushSah_ !!!

1

0

4

Omar Kilani

@omarkilani

9 months

@brianwilt My Waymo drove through the smoke and flames of a car that was on fire yesterday. Took like 2 seconds to think/wait for a lull in oncoming traffic and go around. Was awesome. Wonder if you guys had that in the simulator…

1

0

3

Omar Kilani

@omarkilani

9 months

Section 174 almost fixed (international expenses still broken though, but at least the politics of that make weird sense).

Brendan Pedersen

@BrendanPedersen

9 months

Longer statement just now from the White House on the bipartisan tax deal, via spox Michael Kikukawa:

1

10

13

0

3

Omar Kilani

@omarkilani

4 months

@mycoliza tbf the Google GLB is black magic unrivaled by anything that exists elsewhere.

0

3

Omar Kilani

@omarkilani

9 months

There are two types of engineers…

terminally onλine εngineer 🇺🇦

@tekbog

9 months

it's just sending one json from one service to another how hard can it be?

64

295

4K

0

2

Omar Kilani

@omarkilani

5 months

@jmduke I did this once. It took 5 years and cost $5m. It was successful in the end, but it was also the worst thing I’ve ever done software wise.

0

2

Omar Kilani

@omarkilani

5 months

@eyeofenceladus Honestly… it’s amazing.

0

2

Omar Kilani

@omarkilani

3 months

@RickLamers Simply write the least amount of code possible in the first place. :)

1

0

3

Omar Kilani

@omarkilani

1 year

@brianwilt @Waymo True. I get extremely car sick in (most?) other cars but never in Waymo.

0

3

Omar Kilani

@omarkilani

6 months

@HarleyW_Alt @GroqInc It’s coming soon! (The playground has dark mode already, if you’d rather not stare at the sun.)

0

2

Omar Kilani

@omarkilani

5 months

One more for today…

sunny madra

@sundeep

5 months

Fast Friday continues with speed love for @MistralAI 8x7b 👀 on @GroqInc

1

27

0

3

Omar Kilani

@omarkilani

6 months

@gblazex @GroqInc @ArtificialAnlys It does. :)

1

3

Omar Kilani

@omarkilani

5 months

30,000 t/s in... just the start.

sunny madra

@sundeep

5 months

. @GroqInc engineers working hard on a Friday, improving our stack to get more performance out of LPUs 👀 30,000 tok/s input #youaintseennothingyet

11

9

122

0

3

Omar Kilani

@omarkilani

6 months

@matijagrcic @rafalwilinski You can put a sleep() in between the chunks if you like. :)

0

3

Omar Kilani

@omarkilani

8 years

@stewart some men just want to watch the world burn.

0

2

Omar Kilani

@omarkilani

6 months

@AdrienBrault @sundeep @ilkerndaskin @dr_cintas Soon (tm)

Rick Lamers

@RickLamers

6 months

@yar_vol @GroqInc Streaming is coming! 🔜

1

0

7

0

3

Omar Kilani

@omarkilani

8 months

@ianlandsman “You can just do things”

0

2

3

Omar Kilani

@omarkilani

6 months

@rickykirkendall @GroqInc Thanks for flagging this. Streaming is coming soon. :) We’ll get this limitation added to the docs in the meantime. 🫡

1

0

3

Omar Kilani

@omarkilani

5 months

@KapadiaSoami @JonathanRoss321 @GroqInc Welcome to Groq!

0

2

Omar Kilani

@omarkilani

2 months

@AarushSah_ Happy birthday Aarush!!! 🎉

1

0

3

Omar Kilani

@omarkilani

8 months

Literally how all good software was made…

Jean-Michel Lemieux

@jmwind

8 months

I’ve retired from software… process. No scrum, dds, tdd, stand ups, devops, sre, micro services, retrospectives, pre and post mortems… Instead, we just build and run software together. We do use an issue tracker and a good readme. Everyone posts an eod update to our group

171

248

3K

0

1

3

Omar Kilani

@omarkilani

6 months

@MoonRotator @RickLamers is working on making it the best. :)

0

3

Omar Kilani

@omarkilani

8 years

@andrey_butov assuming there are ever elections again or the world still exists.

0

1

3

Omar Kilani

@omarkilani

8 months

@ptr_to_joel It’s like you were there.

1

0

3

Omar Kilani

@omarkilani

6 months

@BartronPolygon Hey Bart, meet @RickLamers who can help diagnose this. 🫡

2

0

2

Omar Kilani

@omarkilani

8 months

@PatrickFIanagan Trying to explain this to people is ridiculously frustrating.

1

0

2

Omar Kilani

@omarkilani

3 months

@lqiao @sequoia @nvidia @AMD @MongoDB @benchmark Congrats! 🎉

0

2

Omar Kilani

@omarkilani

1 year

@nntaleb Your friend should get a Neo 2T and hook it up to Zwift: Then their cycling shoes can stay on no matter the season.

0

Omar Kilani

@omarkilani

8 years

@andrey_butov Don't watch/read the news man. I stopped years ago. Best decision ever.

1

2

Omar Kilani

@omarkilani

8 years

@andrey_butov @ianlandsman @dhicking 3k for an iMac... should have waited for the update that never comes... ;)

0

2

Omar Kilani

@omarkilani

2 years

@ianlandsman @aarondfrancis Never read the comments, Ian.

0

2

Omar Kilani

@omarkilani

4 months

@AarushSah_ @GroqInc LFG! 🫡

0

2

Omar Kilani

@omarkilani

5 months

@BeardAintWeird_ @sundeep Hey Samee — meet @RickLamers , who’s in charge of tool calls at Groq. Feel free to reach out with more info.

1

0

2

Omar Kilani

@omarkilani

8 months

@tekbog 🥲

0

1

Omar Kilani

@omarkilani

6 months

@hive_echo This hasn’t been fully answered yet because Zuck claims L3 was designed for tool use: We (well, @RickLamers ) implemented it ourselves. Maybe you could work with Rick to see how we can improve our support.

0

2

Omar Kilani

@omarkilani

5 months

@felixchin1 4o is running at 109 t/s/u on a H200 which is pretty impressive, but we don’t know enough about the model to say if that’s “super fast”, IMHO. It’s very unlikely to be faster than the same model running on the LPU. I would be more than happy to spin that up for OAI. :)

1

0

2

Omar Kilani

@omarkilani

8 years

@kazuho that's pretty awesome. Good luck! :)

0

2

Omar Kilani

@omarkilani

2 months