Dimitri Bertsekas @DBertsekas Twitter profile

Lecture 1, 2021. Overview. AlphaZero, DP, policy iteration. ASU

8

252

1K

Dimitri Bertsekas

@DBertsekas

4 years

My Reinforcement Learning course at ASU has started: Class notes, video lectures and slides will be posted weekly at First videolecture can be seen at Dimitri Bertsekas

Slides, class notes, and related textbook material at http://web.mit.edu/dimitrib/www/RLbook.html. An overview of the course. AlphaZero, related game program...

A Course in Reinforcement Learning by Dimitri Bertsekas - Books on Google Play

4

216

867

Dimitri Bertsekas

@DBertsekas

5 months

My 2024 #ReinforcementLearning course at ASU has been completed. Slides, videolectures, supporting material, and PDF of my new 440-page textbook “A Course in Reinforcement Learning”, can be found at

9

177

862

Dimitri Bertsekas

@DBertsekas

3 years

#ReinforcementLearning My Reinforcement Learning course at ASU has just started. First video lecture, slides, class notes have been posted at as well as .pdf of the upcoming book: "Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control"

4

179

827

Dimitri Bertsekas

@DBertsekas

3 years

#ReinforcementLearning An early draft of my forthcoming (2022) research monograph was posted on-line: "Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control,"

4

141

777

Dimitri Bertsekas

@DBertsekas

1 year

My new 420-page textbook “A Course in #ReinforcementLearning " used at my ASU course has just been published as an ebook at Slides, videolectures, and supporting material at

A Course in Reinforcement Learning - Ebook written by Dimitri Bertsekas. Read this book using Google Play Books app on your PC, android, iOS devices. Download for offline reading, highlight, bookmark...

Model Predictive Control and Reinforcement Learning: A Unified...

6

140

742

Dimitri Bertsekas

@DBertsekas

3 months

My new paper “ #ModelPredictiveControl and #ReinforcementLearning : A Unifying Framework Based on #DynamicProgramming " has appeared as The framework aims to bridge the cultural gap between the two fields

In this paper we describe a new conceptual framework that connects approximate Dynamic Programming (DP), Model Predictive Control (MPC), and Reinforcement Learning (RL). This framework centers...

A Course on Reinforcement Learning

9

125

618

Dimitri Bertsekas

@DBertsekas

3 years

The book will support my forthcoming book “Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control" which will be used as textbook for my Reinforcement Learning course at ASU (starts Jan. 12, with video lectures posted on-line). (2/2)

5

67

588

Dimitri Bertsekas

@DBertsekas

9 months

The print version of my new book "Course in #ReinforcementLearning " is now available at Amazon Course material, including video lectures and slides from the class offered at ASU can be found at my web site

This textbook is an outgrowth of a course in Reinforcement Learning (RL) that the author has offered in the years 2019-2023 at Arizona State University. The purpose of the course is to give an...

www.amazon.com

4

88

543

Dimitri Bertsekas

@DBertsekas

3 years

#ReinforcementLearning A new 3rd edition of my “Abstract Dynamic Programming” book has been published as an ebook; check (1/2)

0

80

491

Dimitri Bertsekas

@DBertsekas

8 months

My Spring Semester 2024 #ReinforcementLearning course at ASU has just started. First video lecture, slides, textbook PDF, and other video resources have been posted at Subsequent lectures and slides will be posted on Fridays.

3

91

426

Dimitri Bertsekas

@DBertsekas

3 years

#ReinforcementLearning Video posted: "Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control" from my KTH lecture of Nov. 17, 2021

Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive...

Slides at http://web.mit.edu/dimitrib/www/abstractdp_MIT.htmlAbstract: Some of the most exciting success stories in reinforcement learning have been in the a...

Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control by Dimitri Bertsekas -...

1

72

401

Dimitri Bertsekas

@DBertsekas

2 years

#ReinforcementLearning The book “Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control” has just been published as an ebook; check It is also available as free .pdf from my website

Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control - Ebook written by Dimitri Bertsekas. Read this book using Google Play Books app on your PC, android, iOS devices. Download...

Dimitri Bertsekas, Convex Optimization: A Journey of 60 Years,...

4

68

375

Dimitri Bertsekas

@DBertsekas

2 years

My Spring Semester 2023 #ReinforcementLearning course at ASU has just started. First video lecture, slides, class notes, and other textbook and video resources have been posted at Subsequent lectures and slides will be posted on Fridays

0

78

363

Dimitri Bertsekas

@DBertsekas

3 years

#ReinforcementLearning @DBertsekas @beenwrekt @tdietterich My videolecture "Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control" was posted It provides a summary of my new book (2022):

2

83

353

Dimitri Bertsekas

@DBertsekas

8 months

Posted Youtube video of "A 60-year journey in #convex #optimization ," a lecture on the history and the evolution of the subject. From a 2009 MIT event honoring S Mitter. Includes a review of the MCMC abstract duality framework.

A lecture on the history and evolution of convex optimization theory and algorithms, at an event honoring Prof. Sanjoy Mitter from 2009. The lecture traces t...

The failure of theoretical error bounds in Reinforcement Learning.

0

60

319

Dimitri Bertsekas

@DBertsekas

9 months

The failure of theoretical linear error bounds in #reinforcementlearning . via @YouTube Panel discussion on the relation of theory and practice in reinforcement learning (RL), at the event honoring the career of John Tsitsiklis at MIT, Oct. 7, 2023.

Panel discussion on the relation of theory and practice in reinforcement learning (RL), at the event honoring the career of John Tsitsiklis at MIT, Oct. 7, 2...

New Auction Algorithms for Path Planning, Network Transport, and...

0

20

162

Dimitri Bertsekas

@DBertsekas

2 years

#machinetheory #computerscience #computationallearningtheory My paper New Auction Algorithms for Path Planning, Network Transport, and Reinforcement Learning, proposes a new auction-based algorithmic viewpoint for optimal transport.

We consider some classical optimization problems in path planning and network transport, and we introduce new auction-based algorithms for their optimal and suboptimal solution. The algorithms are...

MIT Lecture, Lessons from AlphaZero for Optimal, Model Predictive,...

3

29

155

Dimitri Bertsekas

@DBertsekas

2 years

#ReinforcementLearning #ArtificialIntelligence Link for the MIT video lecture:

Slides at http://web.mit.edu/dimitrib/www/abstractdp_MIT.htmlAn outline of the main conceptual framework of my new book, which connects the AI/reinforcement ...

Lecture 10, 2024; GPT, HMM, and Markov chains: Rollout variants for...

0

25

155

Dimitri Bertsekas

@DBertsekas

5 months

Fast new #ReinforcementLearning algorithms for most likely sequence generation in #GenerativeAI #MachineLearning #statistics #HMM Videolecture slides and ArXiv paper at Broad applications in #speechrecognition , #LLM , #coding , #bioinformatics , #trading , etc

Slides, class notes, and related textbook material at http://web.mit.edu/dimitrib/www/RLbook.html Lecture delivered by Dimitri Bertsekas and Yuchao Li, bas...

Polyhedral Approximation Algorithms for Convex Optimization, NIPS 2008

2

29

155

Dimitri Bertsekas

@DBertsekas

9 months

Video of my opening remarks at an October 2023 MIT event honoring John Tsitsiklis' academic journey in #optimization #controltheory and #ReinforcementLearning

1

10

131

Dimitri Bertsekas

@DBertsekas

7 months

Just posted video of NIPS 2008 lecture on Polyhedral Approximation Algorithms for #convex #optimization . A unifying framework that subsumes classical methods, such as cutting plane and simplicial decomposition, as well as new methods

Paper by Dimitri Bertsekas and Huizhen Yu at https://web.mit.edu/dimitrib/www/SIAM_Version.pdfRelated Slides at https://web.mit.edu/dimitrib/www/Polyhedral_...

Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control by Dimitri Bertsekas -...

0

21

127

Dimitri Bertsekas

@DBertsekas

3 months

A personal note re Blackwell whom I was honored to know. From my book with Shreve "We are particularly indebted to David Blackwell who inspired us by his pioneering work on dynamic programming in Borel spaces, encouraged us ..."

Who is David Blackwell?

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as […]

www.hpcwire.com

2

20

118

Dimitri Bertsekas

@DBertsekas

7 months

Just posted video of lecture at NorthWestern on Incremental Gradient, Subgradient, and #Proximal Methods for #convex #optimization . An overview of my research, including incremental dual proximal, i.e., augmented Lagrangian methods

0

16

102

Dimitri Bertsekas

@DBertsekas

2 years

An overview lecture based on my “Lessons from AlphaZero for Optimal Model Predictive, and Adaptive Control” book at TAMU: Monday Oct. 3, 2:00-3:00 PM (CT). Can be seen by zoom on-line

1

18

94

Dimitri Bertsekas

@DBertsekas

2 years

#machinetheory #computerscience #computationallearningtheory My new paper Rollout Algorithms and Approximate Dynamic Programming for Bayesian Optimization focuses on #ReinforcementLearning algorithms for estimation and control using #POMDP formulation, arXiv:2212.07998

0

8

94

Dimitri Bertsekas

@DBertsekas

2 years

#ReinforcementLearning #ArtificialIntelligence The printed version of my “Lessons from AlphaZero for Optimal Model Predictive, and Adaptive Control” book has been published; see An ebook version at

Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control - Ebook written by Dimitri Bertsekas. Read this book using Google Play Books app on your PC, android, iOS devices. Download...

Acceptance remarks by Dimitri Bertsekas for 2014 Khachiyan Prize of...

1

20

91

Dimitri Bertsekas

@DBertsekas

3 years

A new paper has been posted: Distributed Asynchronous Policy Iteration for Sequential Zero-Sum Games and Minimax Control, arXiv:2107.10406. It allows a highly parallelizable implementation, and is suitable for the use of reinforcement learning methods based on aggregation.

1

9

90

Dimitri Bertsekas

@DBertsekas

2 years

#ReinforcementLearning #ArtificialIntelligence An MIT videolecture on the main conceptual framework of my new book "Lessons from AlphaZero ...", 2022. The application of the main ideas to decision and control, and the solution of the Wordle/NY Times puzzle is also discussed.

0

11

88

Dimitri Bertsekas

@DBertsekas

11 months

#machinetheory #computerscience Just posted my paper New Auction Algorithms for the Assignment Problem, arXiv:2310.03159, which will be a core part of my forthcoming book Auction Algorithms for Path Planning, Network Transport, and #ReinforcementLearning

0

4

84

Dimitri Bertsekas

@DBertsekas

6 months

Acceptance remarks for the 2014 Khachiyan Prize of the INFORMS Optimization Society Anecdotes, stories, and commentary relating to my intellectual roots and journey in #network #convex #optimization , and #ReinforcementLearning

Anecdotes and discussion relating to the speaker's intellectual roots and journey. For related remarks, see also the speaker's acceptance speech of the Bellm...

TCOptRob Seminar: On the Geometric Foundations of Continuous Control...

0

7

55

Dimitri Bertsekas

@DBertsekas

10 months

A video by Nathan Ratliff Director of systems software @NVIDIA on how my Newton step view of #ReinforcementLearning connects to @robotic and continuous space control. Check 4:15 mins into the video and my Lessons from AlphaZero book

TCOptRob Seminar: On the Geometric Foundations of Continuous Control by Nathan Ratliff of NVIDIA (https://www.nvidia.com)00:00 Intro01:58 The Talk49:40 Q&ATi...

Multiagent Reinforcement Learning: Rollout and Policy Iteration

0

8

48

Dimitri Bertsekas

@DBertsekas

3 years

@jcyranka Yes they will be publicly available at together with the RL course video lectures and other material from the past three years.

1

10

45

Dimitri Bertsekas

@DBertsekas

4 years

#ReinforcementLearning My new book on Rollout, Policy Iteration, and Distributed Reinforcement Learning has just been published, and can be ordered from the publishing company, Athena Scientific, and from Amazon

1

5

29

Dimitri Bertsekas

@DBertsekas

4 years

#ReinforcementLearning A new on-line lecture on Multiagent Reinforcement Learning at

To download the slides in .pdf and the associated research papers, link to the author's web site: http://web.mit.edu/dimitrib/www/RLbook.htmlWe focus on roll...

Reinforcement Learning Methods for Wordle: A POMDP/Adaptive...

0

2

21

Dimitri Bertsekas

@DBertsekas

2 years

In “ #ReinforcementLearning Methods for #Wordle : A POMDP/Adaptive Control Approach” by @sbhambr1 , @Amrita_Bh , and @DBertsekas , we describe the on-line solution for Wordle using rollout, which is within 0.4% of the optimal & solves in seconds! @SCAI_ASU #AI

In this paper we address the solution of the popular Wordle puzzle, using new reinforcement learning methods, which apply more generally to adaptive control of dynamic systems and to classes of...

Feature-Based Aggregation and Deep Reinforcement Learning: A...

0

2

23

Dimitri Bertsekas

@DBertsekas

1 year

@beenwrekt The Newton step view of policy improvement is the central theoretical fact in my books "Rollout and Policy Iteration..." and "Lessons from #alphazero " and upends the traditional view of the 1/(1-alpha) (global) error bounds in #ReinforcementLearning which are totally misleading

0

4

23

Dimitri Bertsekas

@DBertsekas

5 years

#ReinforcementLearning Slides for an extended tutorial lecture: "Ten Key Ideas for Reinforcement Learning and Optimal Control" can be found at Based on my book "Reinforcement Learning and Optimal Control", Athena Scientific, 2019.

0

6

19

Dimitri Bertsekas

@DBertsekas

6 years

#ReinforcementLearning An expanded (by 40%) 72-page survey/research paper "Feature-Based Aggregation and Deep Reinforcement Learning", at arXiv:1804.04577

In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their...

Feature Based Aggregation and Deep Reinforcement Learning

0

2

15

Dimitri Bertsekas

@DBertsekas

5 years

#ReinforcementLearning Just finished my Reinforcement Learning and Optimal Control course at ASU. Draft book, slides, and videolectures can be found at

0

2

17

Dimitri Bertsekas

@DBertsekas

1 year

@beenwrekt Policy improvement is a Newton step to solve the Bellman Eq: maps cost fn approx to cost fn of one-step lookahead policy. Cost fn error does not matter if within the convergence region of Newton's method: the most reliable performance predictor in #ReinforcementLearning since 90s

0

2

14

Dimitri Bertsekas

@DBertsekas

7 months

@findlayden43801 This EPUB KINDLE PDF EBOOK is an unauthorized fake. Please do not rea it or buy it

0

4

11

Dimitri Bertsekas

@DBertsekas

6 years

Bertsekas #ReinforcementLearning New on-line video on "Feature-Baded Aggregation and Deep Reinforcement Learning", ASU Lecture, 4/26/18 at

Lecture at Arizona State University, on 4/26/18. Slides at http://www.mit.edu/~dimitrib/Slides_Aggr_DeepRL.pdf. Paper at http://www.mit.edu/~dimitrib/Feature...

Neuro-Dynamic Programming by Dimitri Bertsekas, John N. Tsitsiklis - Books on Google Play

0

4

12

Dimitri Bertsekas

@DBertsekas

5 years

#ReinforcementLearning My Reinforcement Learning and Optimal Control book has been completed, and can be ordered from the publishing company, Athena Scientific, and from .

0

1

11

Dimitri Bertsekas

@DBertsekas

3 years

@Ferdous_nayan There is an ebook at

Neuro-Dynamic Programming - Ebook written by Dimitri Bertsekas, John N. Tsitsiklis. Read this book using Google Play Books app on your PC, android, iOS devices. Download for offline reading, highli...