Rohan Taori @rtaori13 Twitter profile | Pikagi

Pikagi

Rohan Taori

@rtaori13

2,059

Followers

1,229

Following

26

Media

439

Statuses

multimodal @AnthropicAI | phd from @StanfordAILab 🌲| proud @Cal bear 🐻 | taught w @BerkeleyML

San Francisco

https://t.co/HwFUag2MuL

Joined November 2014

Don't wanna be here? Send us removal request.

Pinned Tweet

@rtaori13

Rohan Taori

1 year

2 months ago we brought you Alpaca... today I'm very excited to share the AlpacaFarm!! 🦙🦙🦙 AlpacaFarm is a complete, validated simulator for tuning for instruction-following models: ✅ Cheap (<$200) ✅ FAST (compared to hiring crowdworkers) ✅ Correlates w human data (ρ=0.98)

@tatsu_hashimoto

Tatsunori Hashimoto

@tatsu_hashimoto

1 year

We are releasing AlpacaFarm, a simulator enabling everyone to run and study the full RLHF pipeline at a fraction of the time (<24h) and cost (<$200) w/ LLM-simulated annotators. Starting w/ Alpaca, we show RLHF gives big 10+% winrate gains vs davinci003 ()

Tweet media one

7

134

649

3

20

120

Last Seen Profiles

@h1nacosplay

@LiegeOf

@NuoMiXiaoWanShe

@cn_metropole

@kimcooks4judge

@jadh62147069278

@phantomfx_india

@JMSpiterone

@romov_

@ParveenKumarAh7

@IndyStyle

@JJ_a0127

@Meta_NONE

@CNRSIdFSud

@DisordinanceRBX

@B4byPlut0Jr

@_YooYa_

@stw_pdg

@ryujeenz

@FrankFrank76845

@Shinkigensha

@Muttaqa18

@satayfinance

@DaliaSamir47637

@fluffyunitthing

@NellaChand658

@bokeplokalmalam

@DaliaSamir47637

@bokeplokalmalam

@__mogn

@bokeplokalmalam

@aitana_delaf

@bokeplokalmalam

@Pr102Pratama

@DaliaSamir47637

@bamahoopxposure

@rtaori13

Rohan Taori

2 years

SUPER excited to share our model Alpaca!! ✅ Handles diverse instructions ✅ Simple, reproduceable tuning procedure ✅ Easy to train+host (only 7B params) ✅ Released data pipelines Times are changing. This demo really shows how accessible it is to instruction tune capable LMs.

@tatsu_hashimoto

Tatsunori Hashimoto

@tatsu_hashimoto

2 years

Instruction-following models are now ubiquitous, but API-only access limits research. Today, we’re releasing info on Alpaca (solely for research use), a small but capable 7B model based on LLaMA that often behaves like OpenAI’s text-davinci-003. Demo:

Tweet media one

43

340

1K

15

57

365

@rtaori13

Rohan Taori

4 years

Reliability is a key challenge in ML. There are now dozens of robust training methods and datasets - how do they compare? We ran 200+ ImageNet models on 200+ test sets to find out. TDLR: Distribution shift is *really* hard, but common patterns emerge.

Tweet media one

3

45

325

@rtaori13

Rohan Taori

2 years

This is a pretty bad take. Stanford doesn’t face the insane budget issues that a public univ like Cal does (been to both schools, experienced the difference firsthand). Govt not prioritizing tax $$ for edu while CS popularity exploding. Stanford also charges 4x the tuition.

@chrmanning

Christopher Manning

2 years

Meanwhile at @Stanford , we just encourage all students to take as many CS courses as they would like …

49

29

774

6

9

232

@rtaori13

Rohan Taori

4 years

@Uber @Twitter Lol, looks like no rides in California either way

1

1

153

@rtaori13

Rohan Taori

2 years

🔥🔥 Training code (and data) for Alpaca is now RELEASED! 🔥🔥 Incredibly quick work by @lxuechen @Tianyi_Zh . If you have access to LLaMA, you can now train your own Alpacas!! We also added more capacity to the demo, try it out!

@rtaori13

Rohan Taori

2 years

SUPER excited to share our model Alpaca!! ✅ Handles diverse instructions ✅ Simple, reproduceable tuning procedure ✅ Easy to train+host (only 7B params) ✅ Released data pipelines Times are changing. This demo really shows how accessible it is to instruction tune capable LMs.

15

57

365

3

32

150

@rtaori13

Rohan Taori

3 years

Any ML system that operates in the real world will inevitably face out-of-distribution (OOD) data that differs from the training set. Inconsistency isn't ideal - so how does OOD performance relate to in-domain performance? We tested a range of models + datasets to find out:

Tweet media one

1

28

124

@rtaori13

Rohan Taori

1 year

Here's an idea: release the weights! It's clear these models lag behind in capability, so they don't present a business threat, and the weights would go a long way towards reproducibility of the many experiments the community has run on them 😃 What do you think? @sama @AlecRad

@OpenAI

OpenAI

1 year

GPT-4 API is now available to all paying OpenAI API customers. GPT-3.5 Turbo, DALL·E, and Whisper APIs are also now generally available, and we’re announcing a deprecation plan for some of our older models, which will retire beginning of 2024:

476

1K

5K

2

8

86

@rtaori13

Rohan Taori

7 months

Looking for job opps again after 5 years (prev before my phd) and wow, the typical coding interview feels so.... behind the times? Like, what are we testing for by asking someone to implement a sandboxed, python function in 1 hr that GPT4 could do 80% of anyways?

10

0

82

@rtaori13

Rohan Taori

2 years

🎉 The last few weeks have seen the release of #StableDiffusion , #OPT , and other large models. ⚠️ But should we be concerned about an irreversible influx of AI content on the internet? ⚙️ Will this make it harder to collect clean training data for future AI models? 🧵👇 1/6

1

14

79

@rtaori13

Rohan Taori

8 months

trying to figure out health insurance options while debugging training runs got me like HMO, PPO, DPO, IPO, KTO...

6

4

78

@rtaori13

Rohan Taori

6 months

improving LLM "reasoning" is the new "robustifying" image classifiers

2

3

71

@rtaori13

Rohan Taori

7 months

Things in AI move fast, it’s sometimes hard to step back and appreciate the bigger picture. Gemini and Sora make it clear that video is the next frontier 📷 Here's my take on training for video, what it unlocks, and how we get there:

5

10

57

@rtaori13

Rohan Taori

5 months

@ethanCaballero MoEs don't make sense for running models locally (memory is big bottleneck).... I wouldn't do it at 8B scale, 70B scale maybe? but quantization keeps getting better & better

2

2

55

@rtaori13

Rohan Taori

5 months

it's a good time to be a multimodal boi

2

1

53

@rtaori13

Rohan Taori

7 months

“One needs to learn to love and enjoy the little things in life. One also needs to discover one’s true calling and then should do everything to pursue the selected path,” - wise words @archit_sharma97

Tweet card media

City lad selected for PhD by US varsity - The Tribune

Tribune News Service Amritsar, December 21 Living his dream of working for Google, the city lad, Archit Sharma, has been selected for a PhD programme on artificial intelligence by Stanford Universi...

www.tribuneindia.com

3

4

49

@rtaori13

Rohan Taori

1 year

Looking forward to meeting people at ICML!! 🏖️🏖️Please reach out if you wanna talk 😃 Have been thinking about: - building LLMs (training + finetuning data, efficient inference, etc) - open source - synthetic data loops - multi-modal capabilities very open to chatting, lmk!

4

6

38

@rtaori13

Rohan Taori

1 year

We've had lots of interest in our automatic evals! (AlpacaEval reflects human judgement better than other leaderboards). It's super easy to add models: NEW: Join our discord for community support and discussion -

Tweet card media

Discord - Group Chat That’s All Fun & Games

Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.

@yanndubs

Yann Dubois

1 year

Developing chat LLMs is hard without an automated way to measure improvements 🔥It just became easier with AlpacaEval🔥 An automated evaluation pipeline that’s - easy to use - fast - cheap - validated w/ 20K human annotations 🥇leaderboard: 🧵

Tweet media one

18

134

579

0

4

25

@rtaori13

Rohan Taori

6 months

biggest alpha in AI rn is literally just rewatching Silicon Valley

4

0

25

@rtaori13

Rohan Taori

5 years

Excited to have my work accepted at Info Theory & ML workshop at @NeurIPSConf ! "Autoregressive Models: What Are They Good For?" - joint with Murtaza Dalal and Alex Li #NeurIPS2019 #ITML2019

Tweet card media

Autoregressive Models: What Are They Good For?

Autoregressive (AR) models have become a popular tool for unsupervised learning, achieving state-of-the-art log likelihood estimates. We investigate the use of AR models as density estimators in...

3

5

23

@rtaori13

Rohan Taori

8 months

The best pretraining work to come out in a while. 🔥 Really sets the benchmark for what good science looks like (even when the details can get messy!). Expect to reference this quite heavily. Combing through the interesting takeaways 👇

@XueFz

Fuzhao Xue

8 months

(1/5)🚀 Our OpenMoE Paper is out! 📄 Including: 🔍ALL Checkpoints 📊 In-depth MoE routing analysis 🤯Learning from mistakes & solutions Three important findings: (1) Context-Independent Specialization; (2) Early Routing Learning; (3) Drop-towards-the-End. Paper Link:

Tweet media one

5

105

518

1

3

22

@rtaori13

Rohan Taori

2 years

The future is bright. Frontiers will once again widen as we move into general purpose multimodal models, but it's easy to forget that adoption + accessibility are never far behind. And good, careful science (+ open-source!) can demystify a lot in due time.

1

0

19

@rtaori13

Rohan Taori

4 years

@TIME fauci who else

0

0

18

@rtaori13

Rohan Taori

7 months

happy llamaversary! 🦙

0

2

19

@rtaori13

Rohan Taori

4 years

Very excited to present our new work at #ICML2020 UDL workshop! Join us at 9am PT tomorrow (July 17): Measuring Robustness to Natural Distribution Shifts in Image Classification w/ @Vaishaal , Achal Dave, Nicholas Carlini, @lschmidt3 , @beenwrekt

Tweet media one

1

5

18

@rtaori13

Rohan Taori

3 years

So I got an xray done for my broken toe.... I asked the hospital for scans and turns out the only delivery method is sending a CD through MAIL? Who has a CD player/drive now anyways??? Am I missing something or why can't I view them through my portal or get them via email?

2

0

16

@rtaori13

Rohan Taori

2 years

Thanks for advertising! Now, can you share more details around how much money SBF delivered to Anthropic? What's the value that's left? Is Anthropic going to commit to not touching these funds further, given the scandal surrounding SBF/FTX?

@sleepinyourhat

Sam Bowman

@sleepinyourhat

2 years

🚨 I'm hiring! 🚨 I'm helping the team that I'm on at @AnthropicAI hire more researchers! If you’re interested in working with me to make highly-capable LLMs more reliable and truthful, and you have relevant research experience in NLP/HCI, apply!

12

37

290

3

3

16

@rtaori13

Rohan Taori

3 years

Come learn more at our ICML talk or poster tomorrow! Talk - Wed 21st @ 6:45pm PST: Poster session - Wed 21st, 9-11pm PST: Paper:

Tweet card media

Accuracy on the Line: On the Strong Correlation Between...

For machine learning systems to be reliable, we must understand their performance in unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distribution...

1

2

14

@rtaori13

Rohan Taori

2 years

We hope that the model or our dead simple recipe for creating the model is a useful launching point for other projects. Exploring the use cases and drawbacks of these language models will be even more important as they become more widely deployed.

1

0

16

@rtaori13

Rohan Taori

4 years

Well it took 5 months but all the same

@rtaori13

Rohan Taori

4 years

So, how many days until Trump fires Mark Esper? 1? 4? This is like russian roulette but for our democracy

0

0

2

0

0

15

@rtaori13

Rohan Taori

4 years

Looking at effective robustness paints a consistent picture for the natural distribution shifts in our testbed. Current robustness interventions show little to no gains. The only approach that consistently promotes robustness is training on large, diverse datasets.

Tweet media one

1

2

15

@rtaori13

Rohan Taori

2 years

Wary of a future where a large amount of information on the internet is synthetically generated by models? Come check out my work characterizing this at the Distribution Shift workshop tomorrow! Poster session 1-2:30pm, talk is at 3:15pm. Room 388-390.

@rtaori13

Rohan Taori

2 years

🎉 The last few weeks have seen the release of #StableDiffusion , #OPT , and other large models. ⚠️ But should we be concerned about an irreversible influx of AI content on the internet? ⚙️ Will this make it harder to collect clean training data for future AI models? 🧵👇 1/6

1

14

79

0

3

14

@rtaori13

Rohan Taori

7 months

@ekang426 @sandeepnailwal @ilblackdragon @ai @caseykcaruso @realDanielShorr @avichal @DavideCrapis @chetan_guevara @stephensonhmatt wait, does anyone on the list actually work on AI?

1

0

13

@rtaori13

Rohan Taori

4 years

There is a lot more in our paper, so we built an interactive website to explore all the data we collected: We also made it *very* easy for others to integrate datasets and models into our testbed:

Tweet card media

GitHub - modestyachts/imagenet-testbed: ImageNet Testbed, associated with the paper "Measuring...

ImageNet Testbed, associated with the paper "Measuring Robustness to Natural Distribution Shifts in Image Classification." - modestyachts/imagenet-testbed

2

0

13

@rtaori13

Rohan Taori

1 year

WHAT A MOMENT to be alive. true greatness on display. phenomenal, just phenomenal, by both steph and kevon

0

0

12

@rtaori13

Rohan Taori

1 year

@jposhaughnessy thanks Jim! :)

2

0

12

@rtaori13

Rohan Taori

6 years

Super excited to present our work done at @BerkeleyML during #secml18 #NeurIPS2018 !!

Tweet media one

0

3

10

@rtaori13

Rohan Taori

11 months

poking around the internal tool specs of GPTs

Tweet media one

1

1

11

@rtaori13

Rohan Taori

7 months

Most of my daily coding work is either: - digging through a new codebase to figure out how a specific feature is implemented (& maybe extracting it out) or - fixing a bug / adding a new feature in my existing codebase The key difference in both is *working w/ a large codebase*

1

0

11

@rtaori13

Rohan Taori

4 years

If you go to Cal, consider applying to Accel scholars!! It is a truly great experience (+ you make some amazing friends!)

0

0

10

@rtaori13

Rohan Taori

5 years

Work done with friends all from @BerkeleyML !! Huge shoutout to this org <3

@rtaori13

Rohan Taori

5 years

Excited to have my work accepted at Info Theory & ML workshop at @NeurIPSConf ! "Autoregressive Models: What Are They Good For?" - joint with Murtaza Dalal and Alex Li #NeurIPS2019 #ITML2019

3

5

23

1

3

10

@rtaori13

Rohan Taori

2 years

@ryan_hoque @ericjang11 It’s easy to lose your way when scientific claims aren’t empirically grounded. The problem of engaging with fiat is that it legitimizes the work - EA/Alignment blogs cite each other, operating as an insular sphere. Anything that doesn’t accord with their world model is axxed.

1

0

10

@rtaori13

Rohan Taori

9 months

Upgraded AlpacaEval to be faster & cheaper, but more importantly use a harder baseline and better auto-evaluator! Have heard from many that AE is an invaluable debugging metric to automatically plot perf for each training run. This update should make this workflow smoother

@yanndubs

Yann Dubois

9 months

Major upgrade: AlpacaEval->AlpacaEval2.0 ✅ 7x faster ✅ 3x cheaper ✅ GPT-4 turbo baseline ✅ less length bias ✅ new models (Mixtral, Gemini,...) We've annotated 12 models and hope to add 80. Seeking community support for annotations (~$400 @OpenAI credits). Can anyone help?

Tweet media one

7

28

163

1

1

9

@rtaori13

Rohan Taori

3 years

TLDR: For many distribution shifts, there's a surprising consistency in model performance! OOD performance can often be well predicted from ID performance. This stands in stark contrast to classical domain adaptation theory, which can only provide loose bounds in practice.

Tweet media one

1

0

9

@rtaori13

Rohan Taori

2 years

Lots more discussion & experiments in paper with @tatsu_hashimoto - More on: 1) when sampling-like behavior appears naturally, 2) what this means for bias amplification on the internet, and 3) how to induce stability in otherwise unstable systems. 6/6

Tweet card media

Data Feedback Loops: Model-driven Amplification of Dataset Biases

Datasets scraped from the internet have been critical to the successes of large-scale machine learning. Yet, this very success puts the utility of future internet-derived datasets at potential...

0

2

9

@rtaori13

Rohan Taori

10 months

Had a lot of fun putting this together! Tbh surprised that most GPTs haven't really been taking advantage of calling custom APIs. This package is still a prototype but shows how powerful connections to the proper toolset can be.

@LinzhiQ

Qi Linzhi

10 months

We built a lil toy to give GPTs access to your Mac: Me: complain to my landlord for me and my roommates pls 🥺 GPT: (reads iMessage chat with roomies, summarizes complaints, finds our landlord’s email, pulls up a drafted email)

10

16

117

0

0

9

@rtaori13

Rohan Taori

2 years

chatGPT and world cup go crazy but trevor noah leaving the daily show just makes me feel another way 🥺

1

0

9

@rtaori13

Rohan Taori

4 years

the police just tear gassed and assaulted protestors in front of the white house who were 100% PEACEFUL so trump could make a fucking PHOTOSHOOT?!?! what is this administration

0

1

8

@rtaori13

Rohan Taori

4 years

Learn more at our #NeurIPS Spotlight talk TODAY at 7:50pm PST or at our poster at 9pm PST: Main project page: joint work w/ @achalddave , @Vaishaal , Nicholas Carlini, @beenwrekt , @lschmidt3

0

0

8

@rtaori13

Rohan Taori

2 years

Wow it didn't take long to take our server down....... working on getting it back up 🤞

0

0

8

@rtaori13

Rohan Taori

5 years

@NeurIPSConf We explore failure cases of density models such as PixelCNN. Turns out they can do some funky things 😏 1) CIFAR-trained PixelCNN assigns higher likelihood to all-black images than real CIFAR images! 2) It can perfectly discriminate between GAN-generated and real samples!

0

0

8

@rtaori13

Rohan Taori

3 years

beautiful

@moultano

Ryan Moulton

3 years

Seasons. VQGAN + CLIP

Tweet media one

Tweet media two

Tweet media three

Tweet media four

6

36

211

0

0

8

@rtaori13

Rohan Taori

2 years

Why has the response to all the valid EA criticism over the past week been “oh but they’re nice people at heart!”? Yes I believe they probably are but that doesn’t address any of the damage caused..

@sama

Sam Altman

2 years

but though i think EA is an incredibly flawed movement, i will say: as individuals, EAs are almost always exceptionally nice, well-meaning people. the movement has some very weird emergent behavior, but i'm happy to see the self-reflection and feel confident it'll emerge better.

43

16

592

1

0

8

@rtaori13

Rohan Taori

3 years

Very excited to release this! Reproducibility is super important & we hope this enables others to create something new 😄 Thanks to @Mitchnw @gabriel_ilharco for pushing insanely hard to get this out

@gabriel_ilharco

Gabriel Ilharco

@gabriel_ilharco

3 years

We are releasing an open-source training implementation of OpenAI’s CLIP!📎 CLIP models learn from language supervision, and are capable of strong zero-shot performance at various vision tasks () Our reproduction can be found at

4

77

333

0

1

7

@rtaori13

Rohan Taori

1 year

Got super excited playing around with , then realized remote ssh is completely broken 😢

Tweet card media

The AI Code Editor

1

0

7

@rtaori13

Rohan Taori

4 years

@ylecun @kchonyc Will you hold Facebook to the same standard?

0

0

7

@rtaori13

Rohan Taori

4 years

If you’re like me and hate Facebook’s decision not to censor Trump’s incendiary posts, please consider signing my petition below. Hold Facebook Accountable For Setting a Dangerous Precedent and Risking Civic Safety

1

2

7

@rtaori13

Rohan Taori

7 months

Like, I'd rather give someone Cursor or Replit AI, ask them to screenshare, and see how they tackle a problem. Do they use chat w/ codebase? How quickly can they glean the structure and how pieces interact tgt? Do they search thru GitHub issues if the answer could lie there?

2

0

7

@rtaori13

Rohan Taori

5 months

@jyangballin @_awettig @_carlosejimenez @ShunyuYao12 @KLieret @karthik_r_n @OfirPress where do I wire the money

1

0

7

@rtaori13

Rohan Taori

3 years

Some of the kickers most surprising to me: - We can often model precise linear relationships between OOD & ID (holds on 12/15 datasets we tested). - Linear trends holds for both SOTA deep networks & classical models like SVMs, logistic regression, random features, etc.

Tweet media one

1

1

7

@rtaori13

Rohan Taori

4 years

The results from this are truly fantastic. Large, high-res, and diverse texture generation with just one network forward pass! Super excited that this work from my internship @nvidia with @GuilinL and others is now online 😃

@GuilinL

Guilin Liu

4 years

📢 Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter📢 Video: Paper: We propose a generalizable framework that can perform texture synthesis for unseen texture images in nearly real-time.

2

18

34

0

0

6

@rtaori13

Rohan Taori

5 months

simplicity of DPO but using your reward model 👇 great work!!

@g_k_swamy

Gokul Swamy

5 months

In the words of Billy Idol, give a "rebel yell" for REBEL: a strikingly simple RL algorithm (it's just regression! no clipping / critics!) that scales to generative models (both LLMs and Diffusion Models!) and has deep theoretical interpretations: . [1/8]

7

33

182

0

2

6

@rtaori13

Rohan Taori

5 years

@tanmingxing @quocleix Do you also evaluate the adversarial accuracy of the models? How does it compare to the baseline?

0

0

6

@rtaori13

Rohan Taori

7 months

@bridgitmendler @foundersfund @a16z @CapitalAlso @LongJourneyVC @BoxGroup @humbavc @NorthwoodSpace QUEEN i literally grew up on good luck charlie been following your amazing journey since then!!

0

0

3

@rtaori13

Rohan Taori

2 years

@ccanonne_ Have you tried using latexrun ? It’s the only reliable way I know to debug latex error messages

Tweet card media

GitHub - aclements/latexrun: A 21st century LaTeX wrapper

A 21st century LaTeX wrapper. Contribute to aclements/latexrun development by creating an account on GitHub.

0

0

5

@rtaori13

Rohan Taori

4 years

To organize the 200 distribution shifts, we divide them into two categories: synthetic shifts and natural shifts. Synthetic shifts are derived from existing images by perturbing them with noise, etc. Natural shifts are new, unperturbed images from a different distribution.

Tweet media one

1

0

6

@rtaori13

Rohan Taori

7 months

@maxhbain @Shutterstock damn this is tragic, huge loss for the community

1

0

6

@rtaori13

Rohan Taori

5 months

@Teknium1 do not worry, the long context extensions will come... my guess is there's not much pretraining data to support more than 8k ctx len, except for code

0

0

6

@rtaori13

Rohan Taori

4 years

This is quite possibly the most amazing thing I've watched all month. Rep. Jayapal's questioning this weekend has been simply impressive

Tweet card media

Lawmaker to Barr: I'm starting to lose my temper

Rep. Pramila Jayapal (D-WA) asks Attorney General Bill Barr about federal officers descending on protesters near the White House ahead of a photo-op for Pres...

www.youtube.com

1

0

6

@rtaori13

Rohan Taori

3 years

Very excited to share this meta-survey on machine learning evaluation failures @ neurips this week! Long form version for arxiv w/ additional analysis by external reviewers should be out soon... 😃

@rajiinio

Deb Raji

3 years

We reviewed 100+ ML survey papers & discovered a pattern of evaluation failures distorting performance reporting across various subfields. Often framed as a one-off casual consideration, ML eval is rarely presented as what it is - a chained *process*, rife w/ measurement hazards

Tweet media one

7

133

501

0

0

6

@rtaori13

Rohan Taori

2 years

@JayaGup10 lol but fr india has more people than north america + europe combined. a white person saying their country is the same as an indian saying their state

0

0

6

@rtaori13

Rohan Taori

3 years

@hugo_larochelle @RaiaHadsell @kchonyc I think it would be great to have a “solid code implementation” certification, and something similar for well-documented & accessible datasets. In my experience, dealing with shitty research code is a surprisingly high blocker to quick progress 2/2

1

0

4

@rtaori13

Rohan Taori

7 months

For these 0 to 1 skills, it's sooo much more important to see if people are resourceful in finding out what they want. Everything is figure out-able! Esp w chatgpt, literally anything can be learned. If you're not using an AI tool, you should have a good reason for why not.

1

0

5

@rtaori13

Rohan Taori

4 years

@coachella When will you stop your abusive ticketing practices? The festival has been (understandably) postponed twice, yet you refuse to give a refund only because I'm part of a "group order". We should be more understanding in these times, not less.

2

0

5

@rtaori13

Rohan Taori

3 years

with the amazing team: John Miller, Aditi Raghunathan, @shiorisagawa , @PangWeiKoh , @Vaishaal , @percyliang , Yair Carmon, @lschmidt3

1

0

5

@rtaori13

Rohan Taori

4 years

At a high level, there has been good progress on the synthetic shifts (e.g., ImageNet-C or adversarial examples). Natural distribution shifts (e.g., ImageNetV2 or ObjectNet), on the other hand, are still much harder.

1

0

5

@rtaori13

Rohan Taori

5 years

holy shit everyone stay safe

@FB_FilmAnalysis

Ted Nguyen

@FB_FilmAnalysis

5 years

Don’t know if this is real but it was posted on Reddit. Anyways, stay clear of Westfield San Jose just in case.

Tweet media one

20

156

118

0

1

5

@rtaori13

Rohan Taori

6 years

Excited to be presenting my work in the CAAD Village at #DEFCON26 !

@GeekPwn

GeekPwn

6 years

Rohan Taori @rtaori13 & Amog Kamsetty will introduce a black-box approach to #adversarial generation, combining the approaches of both genetic algorithms and gradient estimation to solve the task in “Targeted Adversarial Examples for Black Box Audio Systems”. #CAAD #DEFCON

0

0

3

1

0

5

@rtaori13

Rohan Taori

1 year

LOONEY AND CURRY 🤯

1

0

5

@rtaori13

Rohan Taori

4 years

To understand if a model is truly more robust (as opposed to being more accurate in-distribution), we introduce “effective robustness” as accuracy beyond the baseline given by standard models. This is best demonstrated graphically:

Tweet media one

1

0

5

@rtaori13

Rohan Taori

9 months

@McaleerStephen AlpacaFarm has some experimental evidence for this. There isn't much winrate benefit increasing SFT from 10k -> 50k samples (even reduces human perf), but incorporating preference data - PPO/DPO - gives a huge lift

Tweet media one

0

0

5

@rtaori13

Rohan Taori

1 year

AlpacaFarm also comes with standardized implementations of 6 methods, including PPO, + baselines. Here's how things stack up on our current leaderboard: We invite you all to try out new methods/models and help us hillclimb!

Tweet media one

0

0

5

@rtaori13

Rohan Taori

4 years

The protests over the past few days have led me to reflect on the racism and privilege I've experienced in my life. It is important to stand in solidarity with our black friends and push back against the forces that cause divisiveness and hatred.

0

2

5

@rtaori13

Rohan Taori

4 months

@ryan_hoque @berkeley_ai @Berkeley_EECS that pic with oski is ICONIC 🧸 congrats bro looking forward to what you build next!!

0

0

4

@rtaori13

Rohan Taori

2 years

@PoojaIsNagpal i understood less than half the words in this tweet

1

0

4

@rtaori13

Rohan Taori

9 months

co-inventor of diffusion models is the new forbes 30 under 30

0

0

5

@rtaori13

Rohan Taori

4 months

@khoomeik Just set your hparams incorrectly and you’ll have a trash 1e26 flops model. I don’t think you need to invoke chinchilla to make this argument

1

0

4

@rtaori13

Rohan Taori

2 years

@jackclarkSF Yup, in fact specific behavioral properties of these models (such as how they sample) are really important for these sorts of feedback loops. My work on it here:

@rtaori13

Rohan Taori

2 years

🎉 The last few weeks have seen the release of #StableDiffusion , #OPT , and other large models. ⚠️ But should we be concerned about an irreversible influx of AI content on the internet? ⚙️ Will this make it harder to collect clean training data for future AI models? 🧵👇 1/6

1

14

79

0

0

4

@rtaori13

Rohan Taori

2 years

@Teknium1 Hi Teknium, we would like to! Currently chatting w Meta about what the license/legal considerations may be, since Alpaca is based off LLaMA. Hopefully we have some updates soon.

1

0

4

@rtaori13

Rohan Taori

4 years

@OpenAI This is amazing! Congrats on the release - excited to play around with the results and models 😀

0

0

4

@rtaori13

Rohan Taori

3 years

@PreetumNakkiran @roydanroy lmao

0

0

4

@rtaori13

Rohan Taori

7 months

... to be able to ask forward looking questions, design questions, probe your intuitions about certain topics/features/technologies and whether they'd be the right bet. But an extremely fun interview/discussion session if done right!

1

0

4

@rtaori13

Rohan Taori

8 months

@minimaxir Different models behave differently. Eg strongly SFT'd/RLHF'd models experience quite significant drop in generation diversity. Could something fishy be going on w Gemini demos? Definitely. But doesn't mean it's a completely unreasonable thing to do :)

0

0

4

@rtaori13

Rohan Taori

2 years

A concrete example - Generating from a language model with beam search is known to be repetitive/disfluent. Under feedback (where a model is re-trained on its outputs), this problem very quickly magnifies by 2-3x! Nucleus sampling, OTOH, is surprisingly stable. 3/6

Tweet media one

2

0

4

@rtaori13

Rohan Taori

7 months

Which coworking spaces in SF offer a dedicated, sit/stand desk?

1

0

4

@rtaori13

Rohan Taori

2 years

Apologize for all the downtime! We deployed a temp patch that should make it a lot faster (w some ugly redirects). Hopefully more to come soon

0

0

4

@rtaori13

Rohan Taori

2 years

@catherineols How is anthropic impacted by the FTX/SBF collapse?

0

0

4

@rtaori13

Rohan Taori

7 months

@PandaAshwinee speculative decoding speeds up autoregressive sampling. both ssms/linear attn and regular attention still need to autoregressively sample. why can't they be stacked?

3

0

4

@rtaori13

Rohan Taori

2 years

@lxuechen @Tianyi_Zh Please note that there is a specific data license to adhere to in the github repo (may be updated/relaxed down the road).

2

0

3

@rtaori13

Rohan Taori

8 months

@amanrsanger yup, Alpaca was finetuned for 3 epochs. Each epoch's validation loss was higher than the previous epoch, but generation quality at epoch 3 was better than at epoch 1

1

0

3