Peter Richtarik @peter_richtarik Twitter profile

Last Seen Profiles

@stwmaniax

@TatumPaxley

@gentechie

@Jattaa_21

@Ibrahim1997I

@ElSuafanson

@divulgogay2

@MediBurbs

@PaddisKath

@lisnagarveyhc

@rikaoki__92

@_ByDessa

@FoodCloud

@ashleydabunny

@Prof_Ritin

@_DJTHED

@SatyamYada77944

@UUdhareja35077

@km_Ochieng

@MR_G000D

@m9fN0C2oagh5f94

@JunliXu

@stwmaniax

@sotwecom

@TatumPaxley

@Jess_MBarker

@HexanyAZX

@Rickie4pf

@1485_22

@winter_NFTs

@JoeIngles7

@SaltireScotland

@vivino

@B3SX02UJUMOu2

@UCDFoodHealth

@LosBordalas

Peter Richtarik

@peter_richtarik

3 years

Machine learning papers trying hard to make it look like it all works well.

37

756

6K

Peter Richtarik

@peter_richtarik

2 years

What??? (IMO 2022, Chinese team)

25

153

1K

Peter Richtarik

@peter_richtarik

2 years

Boris Polyak (1935) passed away today. An immensely gentle person the way I knew him, and a giant of science in general and optimization in particular. His name, results and legacy will live forever.

10

104

603

Peter Richtarik

@peter_richtarik

2 years

1. What would you do if an #iclr2023 reviewer ignored the science of your "Markov Chain Monte Carlo" paper, but would give you the lowest score (1) solely because "chains" may refer to slavery, and because linking slavery to the fine people of Monte Carlo is offensive to them?

39

49

570

Peter Richtarik

@peter_richtarik

4 years

Re #ICML2021 reviews. The system is broken. We are at the stage where mostly non-experts evaluate the work of experts. It is now a very rare occurrence to receive an educated and scholarly review. Can a football player evaluate the technique and performance of a cyclist? No.

15

48

416

Peter Richtarik

@peter_richtarik

4 years

This detail from the CV of an applicant for an internship position to my Optimization and Machine Learning Lab is outright scary.

8

33

371

Peter Richtarik

@peter_richtarik

2 years

NeurIPS.

3

33

352

Peter Richtarik

@peter_richtarik

1 year

A good theory paper does not need experiments. A good empirical paper does not need theory. Why do most #NeurIPS2023 reviewers not understand this?

5

26

342

Peter Richtarik

@peter_richtarik

4 years

Are you unhappy with how top ML conferences (e.g., @icmlconf , @NeurIPSConf ) handle the review process? You are not alone. There are many serious issues here. Let us all brainstorm in this Google Doc on what’s wrong and what can be improved:

Ideas for improvement of the review process at large ML conferences (e.g., ICML, NeurIPS)

For the SVRHM workshop @ NeurIPS we’ve made the acceptance condition a majority voting of score 5.5 or above. There is no hard-coded acceptance rate: Good quality work will be accepted (our accepta...

docs.google.com

12

73

325

Peter Richtarik

@peter_richtarik

4 years

The feeling when you read a @NeurIPSConf review of your paper and find it equaly well-informed as the opinion of a random person you meet in a shopping mall...

5

12

306

Peter Richtarik

@peter_richtarik

1 year

#NeurIPS2023 reviewing if science was sport: any athelete can evaluate any other athlete, irrespective of their specialization, experience, or level. Result? An amateur 100m dash guy criticizes a high jumper for lack of speed during her world-record-breaking jump. Reject.

4

22

280

Peter Richtarik

@peter_richtarik

2 years

I am very very proud of my team at KAUST who have done a wonderful job by authoring several super exciting papers that were just accepted to the #NeurIPS2022 conference. Check them out! Also, I am hiring interns, students, postdocs & research scientists!

9

27

265

Peter Richtarik

@peter_richtarik

4 years

I can now offer remote internships in my Optimization & Machine Learning Lab at KAUST. These are paid internships, up to 6 months in duration. Areas: Optimization for ML, federated learning, theory of ML. To apply, send me an email with CV & transcript.

17

68

263

Peter Richtarik

@peter_richtarik

2 years

All 6 PhD students I graduated since 2019 (1 from @EdinburghUni 4 from @KAUST_News and one from @mipt_eng ) now have 1,000+ citations! I feel incredibly privilleged to have had the chance to work with such fantastically talented people!

2

8

230

Peter Richtarik

@peter_richtarik

4 years

@ilyasut The fact that wheels are much better than other means of terrestrial transport suggests that human legs might be using wheels too.

5

3

224

Peter Richtarik

@peter_richtarik

5 months

As an ICML 2024 Area Chair, I've handled 19 papers. Recommended 7 to be accepted, 10 to be rejected, and 2 withdrew. The avg scores of the accepted papers = 4.25-6.33. The avg scores of the rejected papers = 2.60-6.00. I did not merely threshold the scores.

11

4

220

Peter Richtarik

@peter_richtarik

3 years

Message to all #NeurIPS2021 Area Chairs: Please request reviewers to engage with the authors. Simply reading the rebuttals and updating the original review with a CMT-era one-liner of the type “I’ve read the rebuttal and am keeping my score” should not be allowed anymore.

6

25

206

Peter Richtarik

@peter_richtarik

1 year

Highly recommended if you are interested in the convergence of SGD for smooth nonconvex functions. Prior literature uses various assumptions to carry out the analysis. We show how they are related, and propose a new assumption that is weaker than them all.

Accepted papers at TMLR

@TmlrPub

2 years

Better Theory for SGD in the Nonconvex World Ahmed Khaled, Peter Richtárik. Action editor: Raman Arora. #optimal #convexity #sgd

0

5

38

3

38

169

Peter Richtarik

@peter_richtarik

3 years

@gabrielpeyre A similar method, but one that provably works: Beautiful work by Yura Malitsky and Konstantin Mishchenko ( @konstmish ).

3

23

165

Peter Richtarik

@peter_richtarik

2 years

As in the past several years, I've accepted an invite to serve as an Area Chair for @icmlconf (ICML 2023). However, this time we are asked to write a short 1-2 sentence statement explaining why peer review helps to advance science. This is what I wrote.

9

7

168

Peter Richtarik

@peter_richtarik

4 years

On February 12, one student wrote to me: “To be honest, your course was perfect. The best course I have ever learnt.” He referred to my CS 331 course “Stochastic Gradient Descent Methods”. This made my day! This is why I do teaching.

3

0

160

Peter Richtarik

@peter_richtarik

1 year

My former student Dmitry Kovalev is on the job market. Hire him - he is a genius. I believe he has the same talent as Nesterov. Wrote 30+ highly technical papers in his PhD. Solved several problems no one could. Pic from his thesis defense. @dakovalev1

6

12

158

Peter Richtarik

@peter_richtarik

4 years

I have MS & PhD positions open in my Optimization & Machine Learning Lab at KAUST. We work on fundamental work underpinning stochastic optimization, distributed and federated learning and more. Starting dates: Spring 2021 and Fall 2021.

6

46

150

Peter Richtarik

@peter_richtarik

2 years

@lpachter I knew from day one I heard about him years ago that he was not a prof at MIT, but that he had some sort of (possibly weak) affiliation. He never claimed that he was a prof AFAIK. His podcasting is thougtful, entertaining and successful. Not sure what all this criticism is about.

1

148

Peter Richtarik

@peter_richtarik

3 years

The idea of "discouraging resubmissions without substantial changes" is bad as it relies on the wrong assumption that previously rejected papers necessarily have something wrong with them. Many good, excellent and even breakthrough papers get routinely rejected...

NeurIPS Conference

@NeurIPSConf

3 years

Check out the updates to the reviewing process for NeurIPS 2021!

2

49

152

7

14

157

Peter Richtarik

@peter_richtarik

10 months

Thank you #ICLR2024 for desk-rejecting a paper (with positive scores only) because of the existence of a follow-up paper that builds upon the results of the submitted paper. Great reason. The AC could have reached out to us for clarifications and this could have been avoided.

7

151

Peter Richtarik

@peter_richtarik

3 years

Boris Polyak (Борис Поляк) is celebrating his 86th birthday today. A pioneer of so much in optimization (first order methods, momentum, control, and much much more), with breathtaking achievements over 60 years of active professional life (he is still working!). Happy birthday!!!

1

15

145

Peter Richtarik

@peter_richtarik

4 months

My stellar PhD student Lukang Sun () defended his PhD thesis "Stein Variational Gradient Descent and Consensus-Based Optimization: Towards a Convergence Analysis and Generalization" ()!

2

6

134

Peter Richtarik

@peter_richtarik

2 years

A random photo of KAUST.

3

5

132

Peter Richtarik

@peter_richtarik

3 years

We are indeed hiring in AI at KAUST (research and teaching faculty), and I am hiring at all levels to my own *Optimization and Machine Learning* lab as well: interns, MS/PhD students, PhD students, postdocs and research scientists.

Jürgen Schmidhuber

@SchmidhuberAI

3 years

KAUST (17 full papers at #NeurIPS2021 ) and its environment are now offering huge resources to advance both fundamental and applied AI research. We are hiring outstanding professors, postdocs, and PhD students:

6

84

381

8

27

123

Peter Richtarik

@peter_richtarik

1 year

A 10 million RMB prize for Nemirovski and Nesterov - congrats to these giants of the field! Thanks for their beautiful works over the years, and for insipiring so many, including me.

3

13

121

Peter Richtarik

@peter_richtarik

4 months

Announcing the creation of *KAUST Center of Excellence in Generative AI*; official launch on July 1, 2024. Joint with @BernardSGhanem @SchmidhuberAI @bremen79 @TheSandyCoder and a few more KAUST colleagues. KAUST funding ($11m over 5 years) + industrial funding. We are looking

1

16

119

Peter Richtarik

@peter_richtarik

4 years

Opinion about @NeurIPSConf 2020 papers: In the same way a great applied paper can achieve high scores and be immune to “there is no theory” kind of criticism, a great theoretical paper should be able to achieve high scores without facing “not useful in practice” criticism.

5

8

117

Peter Richtarik

@peter_richtarik

4 years

Some reviewers just won't participate in @NeurIPSConf post author feedback discussion, despite me as the AC prompting them to do so multiple times. This is of course unacceptable and utterly unprofessional. What do you suggest should be done in such cases? Be creative.

29

5

117

Peter Richtarik

@peter_richtarik

6 months

Yet another reviewer saying: reject, because you study the convex setting and deep learning models are nonconvex. In what world are boxing pros allowed to judge chess? What a joke.

10

2

112

Peter Richtarik

@peter_richtarik

3 years

I am looking for a postdoc to join my *Optimization and Machine Learning* Lab at KAUST. We have a new funding mechanism: apply for the KAUST Global Fellowship here

3

36

113

Peter Richtarik

@peter_richtarik

7 months

Just gave an invited talk on *Optimal Parallel SGD* at Apple, held in Cupertino (workshop on Privacy Preserving Machine Learning). Here are my 16 slides. 1/16

1

6

109

Peter Richtarik

@peter_richtarik

3 years

Just arrived at @dagstuhl .

3

1

108

Peter Richtarik

@peter_richtarik

2 years

My brilliant PhD student Samuel Horváth () just defended his PhD thesis entitled *Better Methods and Theory for Federated Learning: Compression, Client Selection, and Heterogeneity*!

5

2

106

Peter Richtarik

@peter_richtarik

6 months

I have never rejected an empirical paper on the basis of it containing no theory. I judge such papers on their own merits. Is it too much to ask reviewers to judge a theory paper based on the theory?

4

6

101

Peter Richtarik

@peter_richtarik

3 years

A key recipe in the empirical success of federated learning methods is performing multiple local gradient-type steps before aggregation. However, no methods of this type have better communication complexity than gradient descent (in the heterogeneous data regime). Not anymore!

9

16

99

Peter Richtarik

@peter_richtarik

8 months

How to run SGD in a distributed setting with n parallel workers who have different computing speeds and can communicate (via a server) with different communication speeds? For the first time, there is a provably optimal method: Shadowheart SGD:

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time...

We consider nonconvex stochastic optimization problems in the asynchronous centralized distributed setup where the communication times from workers to a server can not be ignored, and the...

arxiv.org

2

13

97

Peter Richtarik

@peter_richtarik

3 years

I get it!

2

8

93

Peter Richtarik

@peter_richtarik

2 years

I am attending the "2022 Workshop on Federated Learning and Analytics" organized by Google. This is an invite-only event run virtually via Google Meet with about 60 participants. I just gave a talk. 1/n

3

9

92

Peter Richtarik

@peter_richtarik

3 years

I have openings for students who wish to work on foundational (mathematical and algorithmic) questions related to machine learning and optimization.

Abdullah Hamdi

@Eng_Hemdi

3 years

Applications are open to @KAUST_News MS/PhD fellowship. Deadline: December 16 Apply NOW and share the news

1

24

45

2

43

88

Peter Richtarik

@peter_richtarik

5 months

AI conference reviewing: Sometimes, a single bad (in terms of review quality) but vocal reviewer can sink a paper. This happened to me repeatedly, and at #ICML2024 as well. The AC sank the paper based on a clearly false statement made by this reviewer. That should not happen.

7

5

92

Peter Richtarik

@peter_richtarik

1 year

I've just received an email from the authors of a NeurIPS 2023 spotlight paper pointing me to their paper. They cite one of my prior works on the same topic. However, their key novelty claim is already addressed in that paper, and they do not mention it anywhere. What to do?

8

1

92

Peter Richtarik

@peter_richtarik

18 days

@gabrielpeyre The Malitsky-Mishchenko stepsize is BB done right.

5

7

92

Peter Richtarik

@peter_richtarik

5 months

Snorkelling in the Red Sea with Michael I Jordan. Mike is visiting KAUST this week.

0

88

Peter Richtarik

@peter_richtarik

1 year

This is how you do it.

0

8

85

Peter Richtarik

@peter_richtarik

2 years

A theory paper rejected from @icmlconf because of "concerns with experiments". The experiments are fine (51 plots; experiments designed to support the theory). The theory extends & improves upon previous theoretical SOTA in an important ML subfield.

10

9

85

Peter Richtarik

@peter_richtarik

1 year

My team and I are presenting 10 (conference + workshop) papers at #ICML2023 . I am looking for interns, Ms/PhDs, PhDs, and postdocs at @AI_KAUST to work with me on theoretical & applied optimization for machine learning & federated learning. Apply here:

1

14

83

Peter Richtarik

@peter_richtarik

6 months

Yurii Nesterov is about to give a talk with an intriguing title at NOPTA 2024.

3

84

Peter Richtarik

@peter_richtarik

1 year

In the search for Mountain Gorillas in Rwanda with @kchonyc and @mireillechayer

2

83

Peter Richtarik

@peter_richtarik

2 years

To all journal & conference reviewers:

Graham Hutton

@haskellhutt

2 years

Depresssingly, this quote from 1984 is even more true today than it was then.

27

987

4K

0

13

77

Peter Richtarik

@peter_richtarik

1 year

We are hiring Assistant Professors in Computer Science at KAUST ( @KAUST_News ). Apply here:

1

21

78

Peter Richtarik

@peter_richtarik

4 months

When you get what you didn't know you needed 😂 (courtesy of The Erwin Schroedinger International Institute For Mathematics and Physics, Vienna)

4

7

77

Peter Richtarik

@peter_richtarik

2 years

A random photo of KAUST.

3

1

75

Peter Richtarik

@peter_richtarik

3 months

Congrats to my 3 PhD students, Abdurakhmon Sadiev (computer science), Kaja Gruntkowska (statistics) and Grigory Malinovsky (applied math) for making the Dean's List for their exceptional academic achievements and dedication to their research work. This comes with a $2,500 cash

0

2

75

Peter Richtarik

@peter_richtarik

5 months

More than 21,000 @NeurIPSConf submissions...

6

4

75

Peter Richtarik

@peter_richtarik

3 years

@sama Claims made without evidence can be dismissed without evidence. And this was some claim. That is all that happened.

2

0

76

Peter Richtarik

@peter_richtarik

4 years

I have MS & PhD positions open in my Optimization & Machine Learning Lab at KAUST. We work on fundamental work underpinning stochastic optimization, distributed and federated learning and more. Starting dates: Fall 2020, Spring 2021, Fall 2021.

2

34

74

Peter Richtarik

@peter_richtarik

2 years

Huge congrats to @dakovalev1 who defended his PhD thesis "Optimal Algorithms for Affinely Constrained, Distributed, Decentralized, Minimax, and High-Order Optimization Problems" on Sept 14! Committee: A. Nemirovski, Yu. Nesterov, D. Keyes, M. Parsani, D. Wang and myself. 1/n

4

2

74

Peter Richtarik

@peter_richtarik

1 year

Federated learning -- a subfield of machine learning we started with my former student Jakub Konecny and a Google team led by H. Brendan McMahan -- has become a key part of the US NATIONAL ARTIFICIAL INTELLIGENCE RESEARCH AND DEVELOPMENT STRATEGIC PLAN. Very pleased and honored!

1

8

73

Peter Richtarik

@peter_richtarik

1 month

Hello from Louvain-la-Neuve, Belgium -- the place where I did my postdoc, working with Yurii Nesterov. I was invited to a workshop () celebrating Nesterov's 50 years of career in optimization (counting from the time when he started reading papers in the

1

2

73

Peter Richtarik

@peter_richtarik

1 year

Our "Byzantine robustness" paper got accepted to ICLR 2023 after all (there was a heated debate about our "unethical" use of the word "Byzantine" in the reviews). The poster and the paper: @ed_gorbunov @sam_hrvth @gauthier_gidel

3

7

73

Peter Richtarik

@peter_richtarik

2 years

3. I am not really looking for advice; just venting a bit. We are doing our best to handle this. However, I believe this behavior is highly problematic, and certainly at odds with what a reviewer should really be doing. The reviewer wants us to change established terminology...

1

0

70

Peter Richtarik

@peter_richtarik

2 years

Congrats to my former PhD student Eduard Gorbunov @ed_gorbunov (graduated in Dec 2021 & co-supervised with A. Gasnikov) for reaching 1,000 citations! All his works are deeply theoretical, so this is extremely impressive to me. Eduard is also an amazingly modest & nice person.

3

68

Peter Richtarik

@peter_richtarik

1 year

Looking at my DBLP profile, I realized I've published 25 papers in ICML ( @icmlconf ) and 25 papers in NeurIPS ( @NeurIPSConf ) over the years. My "AI academic age" computed this way is therefore 50. What's yours?

4

1

69

Peter Richtarik

@peter_richtarik

4 years

Celebrating 1 year as a Full Professor. Where is the surprise party and where are the gifts? 😅😂🥳

4

0

66

Peter Richtarik

@peter_richtarik

1 year

We have developed the first minimax optimal SGD method (we call it "Rennala SGD") for optimization with parallel heterogeneous (each worker takes a different amount of time to compute a stochastic gradient) workers. Surprise: Known asynchronous methods are suboptimal! 1/n

2

12

68

Peter Richtarik

@peter_richtarik

2 years

@gabrielpeyre The Malitsky-Mishchenko stepsize is similar, but unlike Barzilai-Borwein, it has beautiful theory: In some sense, MM is BB done right.

Adaptive Gradient Descent without Descent

We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don’t increase the stepsize too fast and 2) don’t overste...

proceedings.mlr.press

2

11

64

Peter Richtarik

@peter_richtarik

4 years

Super happy about this surprise prize; and huge congratulations to my outstanding student and collaborator Samuel Horváth. The paper was recently accepted to #ICLR2021 , check it out!

1

66

Peter Richtarik

@peter_richtarik

5 months

Michael I Jordan giving his Al Kindi lecture at KAUST.

1

0

65

Peter Richtarik

@peter_richtarik

1 year

I am giving a virtual talk at @Apple in about an hour. I'll talk about a recent breakthrough in the understanding of the local training "trick" in the area of federated learning, and the research that followed.

2

6

61

Peter Richtarik

@peter_richtarik

5 months

Hi from #AISTATS2024 , held in Valencia, Spain. Love the size of the conference: all people in a single (large but not humongous) room. Much better than events with tens of parallel sessions.

1

62

Peter Richtarik

@peter_richtarik

2 years

Interesting optimization department...

5

1

62

Peter Richtarik

@peter_richtarik

2 years

Want to visit KAUST ( @KAUST_News : King Abdullah University of Science and Technology, Saudi Arabia) for a week and learn some cool applied mathematics? Here is your chance!

6

22

62

Peter Richtarik

@peter_richtarik

3 years

1

9

61

Peter Richtarik

@peter_richtarik

1 year

Belated celebration: I've supervised 10 (amazing!!!) PhD students! And sometimes they supervised me ;-)

2

60

Peter Richtarik

@peter_richtarik

5 months

Saudi Arabia -> AISTATS @ Spain -> Belgium -> ICLR @ Austria...

4

0

60

Peter Richtarik

@peter_richtarik

3 years

One of the ways "conspiracy theories" get started...

2

5

58

Peter Richtarik

@peter_richtarik

3 years

"Your paper 'Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods' published in COAP was voted by the editorial board as the best paper appearing in the journal in 2020. Congratulations!" @NicLoizou

Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace...

Computational Optimization and Applications - In this paper we study several classes of stochastic optimization algorithms enriched with heavy ball momentum. Among the methods studied are:...

link.springer.com

0

2

58

Peter Richtarik

@peter_richtarik

2 years

Limiting a rebuttal to 1 page (example: @aistats_conf this year) is like being charged with a crime by multiple prosecutors, then asked to defend in a court, but only allowed a 1 minute speech to defend yourself.

4

3

57

Peter Richtarik

@peter_richtarik

2 months

A deep learning optimizer by a former star PhD student ( @konstmish ), a former star intern ( @akhaledv2 ), and their fantastic collaborators ( @aaron_defazio , @AshokCutkosky , Alice Yang, Harsh Mehta) wins in the AlgoPerf competition! Based on this paper: The Road Less Scheduled

Aaron Defazio

@aaron_defazio

2 months

Schedule-Free Wins AlgoPerf Self-Tuning Track 🎉 I'm pleased to announce that Schedule-Free AdamW set a new SOTA for self-tuning training algorithms, besting AdamW and all other submissions by 8% overall. Try it out:

15

28

270

1

3

55

Peter Richtarik

@peter_richtarik

1 year

SGD with biased gradient estimators increasingly shows up in applications. Several papers have been written trying to capture them theoretically. These works use different assumptions, making it hard to understand the field. 1/n

1

10

55

Peter Richtarik

@peter_richtarik

2 years

Random photo of KAUST (not that random actually - one of my favorite photo spots)

0

2

55

Peter Richtarik

@peter_richtarik

4 years

@gabrielpeyre @qberthet If anyone wants to study the theory of (many variants of) SGD in the strongly convex case, I recommend our paper "SGD: General Analysis and Improved Rates" ; joint work with @gowerrobert @NicLoizou and others.

5

11

55

Peter Richtarik

@peter_richtarik

6 months

I thought we've been there for a few years already. Apparently, they've been reviewing papers all this time as well.

Gautam Kamath

@thegautamkamath

6 months

NeurIPS 2024 will have a track for papers from high schoolers.

78

90

598

1

4

53

Peter Richtarik

@peter_richtarik

3 years

And now a genuine photo taken by me a couple minutes ago.

4

0

54

Peter Richtarik

@peter_richtarik

2 months

Hello from @icmlconf in beautiful Vienna!

0

1

54

Peter Richtarik

@peter_richtarik

4 years

I am thinking of establishing a machine learning conference where only “outstanding reviewers” would be allowed to review.

6

4

50

Peter Richtarik

@peter_richtarik

5 months

Celebration of Yurii Nesterov's 50 years long research career in optimization: Tools for AI and Data Science (ALGOPT 2024) Louvain-la-Neuve, Belgium, August 27-30, 2024 PS: I'll be there. I took this photo of Yurii.

2

3

53

Peter Richtarik

@peter_richtarik

4 months

NeurIPS deadline is over, coming back to @KAUST_News @AI_KAUST @SDAIA_KAUST_AI @KaustResearch !

3

0

51

Peter Richtarik

@peter_richtarik

1 year

Random photo of KAUST.

2

3

50

Peter Richtarik

@peter_richtarik

3 years

A random photo of KAUST

1

3

50

Peter Richtarik

@peter_richtarik

4 years

Happy and proud to announce that my former student Nicolas Loizou @NicLoizou has been selected as a runner-up in the 2019 @TheORSociety Doctoral Award (for the best PhD thesis in OR in the UK).