Aakanksha Chowdhery @achowdhery Twitter profile

Pinned Tweet

Aakanksha Chowdhery

@achowdhery

2 years

#GoogleIO2022 showcases reasoning and multilingual capabilities of PaLM model. Blog: arXiv:

14

131

756

Last Seen Profiles

@popinrythym

@Npro1117

@a_stxx

@nuru_oneisan

@PhillyInquirer

@PERUCO_LALAFELL

@Papai0921

@MPHS_FBBoosters

@rjcurran2

@LaurieS18562937

@NikkiLA22

@devigradito

@b66skCWkfbVLF

@ahSHEEK

@issei_ichika_g

@YRhea_

@hiden_rouge

@HumberAlumni

@shashank_r12

@Mitch_Custer77

@LillyKuVT

@Eliesawaya5

@shanhbie

@Aabii1986

@crossemoxHS

@_rubywang

@charfetchd

@dbmk33919777

@TheFlashCentre

@Tomiko_1

@DavidSc2369

@BionicBuzz

@onp_cn

@Belzinsy

@SpringerSurgery

@bokeplokalmalam

Aakanksha Chowdhery

@achowdhery

2 years

Presenting PaLM-E 562B, one-model generalist across robotics, language, and vision-language. It showcases multimodal chain-of-thought reasoning and the ability to reason over multiple images! And positive transfer enables it to work well on robots!!! Check out Danny's thread 👇

Danny Driess

@DannyDriess

2 years

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:

32

521

2K

5

54

399

Aakanksha Chowdhery

@achowdhery

2 years

1/ Reflecting back on 2022: we shared our most advanced language model PaLM - a single 540B-parameter dense language model for multiple domains & tasks, trained over two TPUv4 Pods. Research paper: Blog post:

PaLM: Scaling Language Modeling with Pathways

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific...

arxiv.org

Google AI

@GoogleAI

3 years

Introducing the 540 billion parameter Pathways Language Model. Trained on two Cloud #TPU v4 pods, it achieves state-of-the-art performance on benchmarks and shows exciting capabilities like mathematical reasoning, code writing, and even explaining jokes.

76

1K

4K

9

59

387

Aakanksha Chowdhery

@achowdhery

2 years

Excited to be presenting PaLM and Model Scaling at Stanford’s Advances in Foundation Models Class (CS324) today afternoon!

Avanika Narayan

@Avanika15

2 years

This quarter, Stanford’s Advances in Foundation Models Class (CS 324) will be partnering with the Stanford MLSys Seminar to host a special talk series on foundation models! Our first talk will be @tri_dao . Catch us *TOMORROW* at 3:30 PT:

1

28

122

4

10

343

Aakanksha Chowdhery

@achowdhery

2 years

#palm PaLM presentation at NeurIPS Google booth (incl several works on top of PaLM model this year)!

4

11

339

Aakanksha Chowdhery

@achowdhery

3 years

Really excited to present the first large-scale use of Pathways system! Joint work with so many of colleagues at Google! @sharan0909 @nfiedel @JeffDean @m_isard @ada_rob @bsaeta .

Google AI

@GoogleAI

3 years

Introducing the 540 billion parameter Pathways Language Model. Trained on two Cloud #TPU v4 pods, it achieves state-of-the-art performance on benchmarks and shows exciting capabilities like mathematical reasoning, code writing, and even explaining jokes.

76

1K

4K

10

38

315

Aakanksha Chowdhery

@achowdhery

2 years

Fun to see PaLM paper in top 5 cited ML papers from 2022!

Mahesh Sathiamoorthy

@madiator

2 years

Top 5 cited papers in ML from 2022 Source:

2

13

65

2

5

213

Aakanksha Chowdhery

@achowdhery

2 years

#palm But how long did it need to train? Training PaLM 62B to 1.3 trillion tokens results in significant gains as suggested by Chinchilla data scaling. However it does not bridge the gap to PaLM 540B that 5x training FLOP count. See updated results in:

5

27

188

Aakanksha Chowdhery

@achowdhery

10 months

Incredibly fun and interesting panel discussion with Percy Liang ( @percyliang ) and Angela Fan! Thank you so much to Sasha ( @srush_nlp ) for the amazing work at organizing and moderating this panel!

Sasha Rush

@srush_nlp

10 months

Be sure to make it to Hall F today to check out our exciting panel "Beyond Scaling". Thanks to everyone who provided questions.

5

6

99

3

7

155

Aakanksha Chowdhery

@achowdhery

10 months

Super excited about discussing Gemini and LLM related advances from Google at the Beyond Scaling Panel tomorrow afternoon at NeurIPS, jointly with Sasha Rush ( @srush_nlp ) , Angela Fan, Percy Liang ( @percyliang ), and Jie Tang ( @jietang ).

NeurIPS Conference

@NeurIPSConf

10 months

Two invited talks down, five more to conquer

0

2

30

2

9

130

Aakanksha Chowdhery

@achowdhery

1 year

Med-PaLM goes multimodal. Check it out!

Vivek Natarajan

@vivnat

1 year

Medicine is inherently multimodal. Thrilled to share Med-PaLM M, the first demonstration of a generalist multimodal biomedical AI system with a stellar team @GoogleAI @GoogleDeepMind @GoogleHealth Paper:

12

108

456

1

8

113

Aakanksha Chowdhery

@achowdhery

2 years

PaLM-SayCan combines the understanding of language models with the real-world capabilities of a helper robot. The accuracy improvements in robotic task execution from PaLM combined with SayCan are impressive. Examples of task-planning:

Karol Hausman

@hausman_k

2 years

1) We updated the underlying LLM to PaLM (), resulting in PaLM-SayCan. This resulted in an interesting trend: Improving the underlying LLM resulted in much higher robotics (!) performance (halving the errors)

4

26

161

0

12

84

Aakanksha Chowdhery

@achowdhery

2 years

#NeurIPS2022 It was fun presenting PaLM and answering Q&A in the panel at "Has It Trained Yet" ( #HITY ) workshop ()! Thanks to the organizers for a great program @frankstefansch1 @zacharynado @GeorgeEDahl @naman33k @PhilippHennig5 !

2

7

67

Aakanksha Chowdhery

@achowdhery

2 years

Attending #NeurIPS2022 in person this year (today-Sat)! Looking forward to catching up with many of you! DM me if you would like to meet.

2

60

Aakanksha Chowdhery

@achowdhery

10 months

Enthusiastic to see what the community builds with Gemini Pro models just released through Google AI Studio! Technical whitepaper on the model:

Sundar Pichai

@sundarpichai

10 months

Today developers can start building with our first version of Gemini Pro through Google AI Studio at . Developers have a free quota and access to a full range of features including function calling, embeddings, semantic retrieval, custom knowledge

344

1K

6K

1

3

58

Aakanksha Chowdhery

@achowdhery

2 years

Excited about these improvements on PaLM model: 1) U-PaLM: finetune with UL2 mixture-of-denoisers 2) Flan-PaLM: finetune on 1.8K tasks phrased as instructions You can even stack these two methods! U-PaLM: Flan-PaLM:

Transcending Scaling Laws with 0.1% Extra Compute

Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models and their...

arxiv.org

Yi Tay

@YiTayML

2 years

Introducing U-PaLM 540B! @GoogleAI Training PaLM w UL2's mixture-of-denoisers with only 0.1% more compute unlocks: - Much better scaling 📈 - Emergent abilities on BIGBench 😎 - Saving 2x compute (4.4 million TPU hours!) 🔥 - New prompting ability link:

8

87

509

1

5

55

Aakanksha Chowdhery

@achowdhery

2 years

It was quite a fun event to discuss emerging research and applications for LLMs! Thanks for the invitation to present!

Davis Treybig

@TreybigDavis

2 years

Presentations from a recent event we hosted with @Replit @GoogleAI @huggingface on emerging research and applications of large language models: @amasad @achowdhery @mathemakitten @GoogleAI

4

12

118

0

2

42

Aakanksha Chowdhery

@achowdhery

2 years

Class Recording:

Pathways Language Model and Model Scaling - Aakanksha Chowdhery |...

Episode 69 of the Stanford MLSys Seminar “Foundation Models Limited Series”!Speaker: Aakanksha Chowdhery Abstract:Large language models have been shown to ac...

www.youtube.com

1

4

41

Aakanksha Chowdhery

@achowdhery

2 years

#palm Really fun discussing PaLM pushes the frontier on many BIGbench tasks and enables PaLM-SayCan robots in @wbur @Endless_Thread podcast jointly with @jaschasd @ethansdyer @xf1280 @hausman_k (Thanks to @deanwrussell ). Url:

Good Bot, Bad Bot | Part VI: The quest to build machines like us

Can a machine think like a human? Can it be conscious? For decades the answer was clear: nope. But artificial intelligence today is challenging that notion. Endless Thread visits Google to see just...

www.wbur.org

WBUR

@WBUR

2 years

Can robots think? For our series finale, @deanwrussell reports on AI research stretching the limits of machine learning and studying if robotic sentience REALLY matters for our future.

1

3

5

0

8

40

Aakanksha Chowdhery

@achowdhery

2 years

Combining safety+interpretability via affordance grounding with language model PaLM in robotics is really impressive. PaLM-SayCan results show that the system chooses the correct sequence of skills 84% of the time and executes them successfully 74% of the time.

Google AI

@GoogleAI

2 years

Learn how we combined our latest language model, PaLM, with robot learning algorithms to create PaLM-SayCan, a robotics system that uses natural language to complete complex tasks in a real-world environment →

8

132

407

0

3

37

Aakanksha Chowdhery

@achowdhery

10 months

If you're at #NeurIPS2023 come chat with us, the Gemini team! We're at the Google booths tomorrow from 1:30-3:00 to answer your questions on what it's like to work on Gemini.

Jeff Dean (@🏡)

@JeffDean

10 months

I'm excited to head to @NeurIPSConf #NeurIPS2023 this week. We'll be having a couple of "Chat with the Gemini Team" events in the @GoogleDeepMind / @GoogleResearch booth areas on Tuesday and Wednesday from 1:30 to 3:00 PM (New Orleans time). Quite a few Gemini team members will

12

39

502

0

1

30

Aakanksha Chowdhery

@achowdhery

2 years

0

1

23

Aakanksha Chowdhery

@achowdhery

2 years

11/ There is just a glimpse of the exciting research with PaLM - the list is too long to summarize here, however I am incredibly grateful to all the amazing collaborators and researchers at @GoogleAI for their contributions and innovations. And super excited for what's next!!!

5

1

21

Aakanksha Chowdhery

@achowdhery

3 years

Paper link:

0

2

17

Aakanksha Chowdhery

@achowdhery

2 years

9/ And we can enable low-latency high-throughput efficient inference at PaLM 540B scale.

James Bradbury

@jekbradbury

2 years

Can multi-100B param language models be served efficiently? We think so! Today we’re announcing the PaLM inference paper and releasing code for low-latency, high-throughput inference of 8B–540B models on TPU v4. Paper: Code: 1/5

12

139

904

1

17

Aakanksha Chowdhery

@achowdhery

2 years

4/ Minerva finetunes PaLM on mathematical content and scientific papers to solve mathematical questions using step-by-step natural language reasoning, establishing new SOTA on STEM benchmarks, MATH and MMLU-STEM

alewkowycz

@alewkowycz

2 years

Very excited to present Minerva🦉: a language model capable of solving mathematical questions using step-by-step natural language reasoning. Combining scale, data and others dramatically improves performance on the STEM benchmarks MATH and MMLU-STEM.

104

1K

8K

1

16

Aakanksha Chowdhery

@achowdhery

2 years

@_arohan_ Thanks for highlighting this Rohan! Implementation is also in open source and runs with T5X/Flax:

0

14

Aakanksha Chowdhery

@achowdhery

2 years

Really excited to see this fun and cool application of PaLM to enable robots to execute long abstract instructions from user inputs!

Karol Hausman

@hausman_k

2 years

Have you ever “heard” yourself talk in your head? Turns out it's a useful tool for robots too! Introducing Inner Monologue: feeding continual textual feedback into LLMs allows robots to articulate a grounded “thought process” to execute long, abstract instructions 🧵👇

23

168

895

0

15

Aakanksha Chowdhery

@achowdhery

5 years

Thanks to the support of @MarconiSociety ... our students from Celestini Program showcase their work in TF blog post!

TensorFlow

@TensorFlow

5 years

See how VisionAir is using Federated Learning and the TensorFlow Java API to estimate air quality taken from smartphone photos, while keeping user privacy in mind. 🤳🌏 Read the blog →

0

32

115

2

1

15

Aakanksha Chowdhery

@achowdhery

2 years

8/ LLMs can "hallucinate." RARR automatically researches & revises the output to fix hallucinations with help from PaLM.

Kelvin Guu

@kelvin_guu

2 years

New from Google Research! Language models perform amazing feats, but often still "hallucinate" unsupported content. Our model, RARR🐯, automatically researches & revises the output of any LM to fix hallucinations and provide citations for each sentence. 🧵

12

151

816

1

2

15

Aakanksha Chowdhery

@achowdhery

2 years

10/ We discuss more about PaLM in the #MeetAGoogleResearcher episode:

Introducing language models to robotics

In this first episode of Meet a Google Researcher, Drew Calcagno speaks with researchers Sharan Narang and Aakanksha Chowdhery, folks who theorized and coded...

www.youtube.com

Google AI

@GoogleAI

2 years

Following up on our introduction of the Pathways Language Model (PaLM), watch the first episode of #MeetAGoogleResearcher , where @drewcalcagno speaks with @achowdhery and @sharan0909 , the researchers who are making robots more helpful with language →

6

57

147

1

2

14

Aakanksha Chowdhery

@achowdhery

2 years

2/ PaLM demonstrates breakthrough capabilities on language, reasoning & code tasks, and can even explain jokes.

Jeff Dean (@🏡)

@JeffDean

3 years

Excited about this @GoogleAI work on "PaLM: Scaling Language Modeling with Pathways" with many authors. Be sure to check out the accompanying 83 page PDF!

7

109

566

1

13

Aakanksha Chowdhery

@achowdhery

2 years

3/ #GoogleIO showcased examples of the reasoning and multilingual capabilities of PaLM.

Google

@Google

2 years

Pathways Language Model (PaLM) is a new advanced AI model that uses a technique called chain of thought prompting to do complex tasks like solve math word problems — and even explain its reasoning process step-by-step. #GoogleIO

56

126

713

1

12

Aakanksha Chowdhery

@achowdhery

6 years

⚡️ “Real-time object detection with TensorFlow Lite”

2

4

12

Aakanksha Chowdhery

@achowdhery

2 years

3.2/ Multilingual capabilities of PaLM are surprising and powerful. For example, you can ask novel questions in Bengali, and get surprisingly good answers on both English and Bengali even when it has never seen parallel sentences in both languages.

Jeff Dean (@🏡)

@JeffDean

2 years

Today at #GoogleIO @sundarpichai showed some examples of the capabilities of the PaLM 540B language model. For example, you can prompt the model with: "I will ask a question in Bengali and get English and Bengali answers" And then give it two examples of this behavior. (cont)

13

83

481

1

12

Aakanksha Chowdhery

@achowdhery

2 years

3.1/ Reasoning capabilities from combining chain-of-thought prompting with model scale have several interesting applications.

Google AI

@GoogleAI

2 years

Learn about chain of thought prompting, a method that equips language models to decompose multi-step problems into intermediate steps, enabling models of sufficient scale to solve complex reasoning problems that are not solvable with standard prompting. →

13

134

639

1

10

Aakanksha Chowdhery

@achowdhery

6 years

Congratulations to the Celestini Program India 2018 student team supported by @MarconiSociety ! The Android demo app they built to predict Air Quality in Delhi using #TFLite featured in #TFDevSummit today and in the demos. Link:

TensorFlow Dev Summit 2019 Keynote

Join the TensorFlow team as they kick off #TFDevSummit 2019! Learn about TensorFlow's 2019 roadmap and what's new for Google's open source #MachineLearning p...

www.youtube.com

0

2

10

Aakanksha Chowdhery

@achowdhery

2 years

@russelljkaplan I wonder if there is a new marketplace for token creation and exchange with content creators and data owners…

0

1

9

Aakanksha Chowdhery

@achowdhery

2 years

7/ Med-PaLM aligns the model to the medical domain to generate safe and helpful answers, achieving 67% in MedQA USMLE improving prior work >17%.

Vivek Natarajan

@vivnat

2 years

Delighted to share our new @GoogleHealth @GoogleAI @Deepmind paper at the intersection of LLMs + health. Our LLMs building on Flan-PaLM reach SOTA on multiple medical question answering datasets including 67.6% on MedQA USMLE (+17% over prior work).

37

368

2K

1

9

Aakanksha Chowdhery

@achowdhery

2 years

@JeffDean Favorite so far: "Bald eagle made of chocolate powder, mango, and whipped cream."

1

0

9

Aakanksha Chowdhery

@achowdhery

2 years

5/ Tasks that seem simple to humans are actually incredibly complex for helper robots. PaLM-SayCan showcases how a robotics system uses PaLM to interpret natural language to complete complex tasks in a real-world environment.

Google

@Google

2 years

Tasks that seem simple to humans — like cleaning up a spilled drink — are actually incredibly complex for helper robots. That’s why Google Research and Everyday Robots are using language models to improve robot learning.

41

82

443

1

8

Aakanksha Chowdhery

@achowdhery

2 years

6/ Flan-PaLM instruction tunes 540B PaLM to follow instructions, establishing a new SOTA on MMLU benchmarks and be helpful in the zero-shot setting with high accuracy.

Hyung Won Chung

@hwchung27

2 years

New paper + models! We extend instruction finetuning by 1. scaling to 540B model 2. scaling to 1.8K finetuning tasks 3. finetuning on chain-of-thought (CoT) data With these, our Flan-PaLM model achieves a new SoTA of 75.2% on MMLU.

9

64

350

1

8

Aakanksha Chowdhery

@achowdhery

2 years

Excited to see the possibilities of model scale of PaLM combined with chain-of-thought prompting.

Jeff Dean (@🏡)

@JeffDean

2 years

Chain of thought promoting. Encouraging language models to "show their work" makes them both more interpretable and more accurate at complex reasoning tasks, solving math problems, etc.

5

19

148

0

8

Aakanksha Chowdhery

@achowdhery

4 years

Great discussion on the role of Internet connectivity in the current pandemic: @MarconiSociety

0

1

8

Aakanksha Chowdhery

@achowdhery

4 years

@MarconiSociety Incredibly proud of the work of students in Celestini Program India 2018.

0

3

7

Aakanksha Chowdhery

@achowdhery

5 years

@Mxbonn @petewarden For MobileNet V2, when finegrain_classification_mode is set to False, the model will shrink the last layer small for small multipliers. Please feel free to email me if there are further questions.

0

2

7

Aakanksha Chowdhery

@achowdhery

10 months

@chelseabfinn @rm_rafailov @archit_sharma97 @ericmitchellai @StefanoErmon @chrmanning Congratulations!

0

6

Aakanksha Chowdhery

@achowdhery

1 year

@reinerpope @MikeGunter_ Congratulations! Looking forward to the advances and growth from your company!

0

6

Aakanksha Chowdhery

@achowdhery

6 years

Real-time pollution check in Delhi on Android app with @TensorFlow Lite and ML Kit! Read blog post for details:

The Marconi Society

@MarconiSociety

6 years

Students at @iitdelhi are predicting and tracking #AirPollution with a new app - thanks to Prof. Brejesh Lall and Marconi Society mentor @achowdhery

0

2

0

3

6

Aakanksha Chowdhery

@achowdhery

6 months

@MatXComputing Congratulations!

1

0

5

Aakanksha Chowdhery

@achowdhery

2 years

@character_ai Congratulations!

0

5

Aakanksha Chowdhery

@achowdhery

2 years

@AllennxDD @arankomatsuzaki

1

0

4

Aakanksha Chowdhery

@achowdhery

2 years

@ben_athi Multi query attention is still valuable in batch 1 setting enabling lower memory cost for loading the KV cache

1

0

4

Aakanksha Chowdhery

@achowdhery

2 years

Video snippet:

Google

@Google

2 years

Pathways Language Model (PaLM) is a new advanced AI model that uses a technique called chain of thought prompting to do complex tasks like solve math word problems — and even explain its reasoning process step-by-step. #GoogleIO

56

126

713

0

1

4

Aakanksha Chowdhery

@achowdhery

2 years

@AllennxDD @arankomatsuzaki Allen, PaLM 540B is actually SOTA in STEM category of MMLU. There was a correction from copying GitHub leaderboard. Please see table 6 updated version.

1

0

4

Aakanksha Chowdhery

@achowdhery

2 years

@kaushikpatnaik The downstream results are in line with Chinchilla data scaling laws. The larger models need to be trained longer with more data!

0

4

Aakanksha Chowdhery

@achowdhery

6 years

AI with solar cells:

0

2

3

Aakanksha Chowdhery

@achowdhery

7 years

My research at Princeton covered by Open Fog Consortium @OpenFog blog Part I ((link: ) ) and Part II ((link: ) ) on ‘‘Fog Computing on drone networks

0

3

Aakanksha Chowdhery

@achowdhery

2 years

@shaneguML @MarcusErve @GoogleAI @OpenAI @johnschulman2 Congratulations!

0

3

Aakanksha Chowdhery

@achowdhery

2 years

Video link at specific time:

Google Keynote (Google I/O ‘22)

Tune in to find out about how we're furthering our mission to organize the world’s information and make it universally accessible and useful. To watch this k...

www.youtube.com

0

1

3

Aakanksha Chowdhery

@achowdhery

2 years

@afrozenator @alvin_rajkomar @Mysiak Congratulations!

0

3

Aakanksha Chowdhery

@achowdhery

2 years

Will be at Google Booth in the coffee break at 3:30pm Today.

0

2

Aakanksha Chowdhery

@achowdhery

2 years

@igorcosta @JeffDean Pathways system:

2

0

2

Aakanksha Chowdhery

@achowdhery

6 years

Real-time pet detection on Mobile with @Tensorflow Lite!

TensorFlow

@TensorFlow

6 years

Now you can train your Object Detection models on Cloud TPUs! Learn how in this end-to-end walkthrough. Bonus: we'll run the trained model on a phone using TensorFlow Lite and detect pets. Read the post here →

5

155

433

0

2

Aakanksha Chowdhery

@achowdhery

6 years

@sandeepssrin @random_forests @deliprao @TensorFlow

1

0

2

Aakanksha Chowdhery

@achowdhery

2 years

@RandomlyWalking @Google Amazing impact! Congratulations!

0

1

Aakanksha Chowdhery

@achowdhery

2 years

@JeffDean @icmlconf @quocleix @MarcRanzato @rajatmonga @greg_corrado @AndrewYNg Congratulations!

0

1

Aakanksha Chowdhery

@achowdhery

10 years

“ @killerooo : Gameplay of our kinect v2 24 hour hackathon project: ”- at Seattle Kinect v2 Hackathon 2014

AhKonCha Gameplay

A simple game we made during the Redmond Kinect v2 hackathon, July 26-27, 2014. Available on the windows store: http://tinyurl.com/ahkonchaBackground music:"...

www.youtube.com

0

1

Aakanksha Chowdhery

@achowdhery

2 years

@hardmaru @StabilityAI Congratulations David!

0

1

Aakanksha Chowdhery

@achowdhery

6 years