Alexander Terenin @avt_im Twitter profile

Pinned Tweet

Alexander Terenin

27 days

As promised: our work on Cost-aware Bayesian optimization via the Pandora's Box Gittins index is now on arXiv! This is joint work with amazing Cornell PhD student Qian Xie as well as Raul Astudillo, @peter_i_frazier , and @ZivScully . Preprint link below!

2

8

55

Last Seen Profiles

@ClubDeluxe

@AnaMenaMusic

@vbIHMEHczj46557

@rikolagu

@emilymfg

@dangbozo

@futebolreii

@yepyapyes66063

@colingreenwood

@bc1plainview

@2facedfairyyyyy

@ibprodigy05

@InnertempleRES

@Jana_Raegan7769

@JayCockburn

@_Aytn34_

@SurajKharole

@abo3yasiir

@pjharvss

@Pika3yoro

@people_inside_

@zloy_odessit

@ActusDals

@Jana_Defi

@seinorfouedo

@LenyStrobel

@Luynhosaurus

@Jkt_Networks2

@JorgeGuardia_

@sitappa_shooter

@sujing110

@sakinol08

@Tante_Binal69

@sujing110

@stw_pdg

@snjin18607

Alexander Terenin

@avt_im

2 months

As a rule, I don’t dismiss other people’s research areas. People know all kinds of things I don’t, including technical reasons why an unpopular method might one day achieve performance no other method can. Consequently, I ask others not to dismiss mine. Let’s talk about why.

67

116

2K

Alexander Terenin

@avt_im

2 months

I'm an academic, I left the UK, and I brought a loved one with me to the US. After ten months here, I'm convinced the intellectual environment and resources here are decisively better than at the best places in the UK. If you're serious about science, come to the US!

Rishi Sunak

@RishiSunak

3 months

We’ve taken action to reduce migration. Student dependant applications are now down by 80%.

9K

2K

15K

30

111

1K

Alexander Terenin

@avt_im

2 years

Announcing: Geometric Kernels, a library for computing kernels on non-Euclidean spaces like graphs and manifolds! Now, it's much easier to try out some of the geometric Gaussian process models we've been developing - just install a Python package!

Geometric Kernels

A cross-framework package for kernels and Gaussian processes on manifolds, graphs, and meshes

geometric-kernels.github.io

12

185

995

Alexander Terenin

@avt_im

8 months

Seen at NeurIPS 👀

12

56

917

Alexander Terenin

@avt_im

3 years

I'm going to try an experiment, and open-source my PhD dissertation writing by making the thesis document visible to the public while I write it. Check it out and follow along!

GitHub - aterenin/phdthesis: Gaussian Processes and Statistical Decision-making in Non-Euclidean...

Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces - aterenin/phdthesis

github.com

14

62

607

Alexander Terenin

@avt_im

2 years

My PhD thesis "Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces" is now on arXiv! Check it out!

Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces

Bayesian learning using Gaussian processes provides a foundational framework for making decisions in a manner that balances what is known with what could be learned by gathering data. In this...

arxiv.org

11

63

560

Alexander Terenin

@avt_im

4 months

NeurIPS should not be doing a high school track. The main effect will be to give the most-privileged yet another tool with which to outcompete less-privileged kids on college admissions. We need more paths for getting into research at *later* stages of life, not earlier ones.

Talia Ringer 🟣 🎗️

@TaliaRinger

4 months

I don't say yes to high school students because I think that is a good way to skew our research community even more towards the most privileged. Exception is if it's deliberately a program at a high school that is emphatically not for students like that

2

3

84

18

37

500

Alexander Terenin

@avt_im

4 years

Got my first internship rejection of the application season, by DeepMind. Always wanted to see whether working at an industry research lab would be for me, and this is my last chance before graduating to find out. Is anyone looking for a machine learning research intern?

39

52

471

Alexander Terenin

@avt_im

2 years

Ever wanted a Gaussian process whose domain is a manifold important enough to actually have a name? If it's a Lie group or homogeneous space, we've worked out a general recipe for computing sq. exp. (heat, diffusion, ..) and Matérn kernels on it!

16

71

477

Alexander Terenin

@avt_im

3 years

Yesterday, I submitted my PhD thesis! My experiment in writing in an open-source format was a resounding success! I want to thank everyone who liked, commented, or gave me feedback or encouragement - you made the process so much more fun and exciting!

GitHub - aterenin/phdthesis: Gaussian Processes and Statistical Decision-making in Non-Euclidean...

Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces - aterenin/phdthesis

github.com

14

37

451

Alexander Terenin

@avt_im

8 months

Belated career announcement: I've joined Cornell as an Assistant Research Professor! This is a 3-year fancy research fellowship (non-tenure-track), during which I'm looking to shift my research to decision-making systems like Bayesian optimization, bandits, and online learning.

14

4

408

Alexander Terenin

@avt_im

3 years

Some news: I've accepted an offer to join the Machine Learning Group within Computational and Biological Learning at the University of Cambridge as a postdoc. Excited to be part of a research group that I have long respected for its astonishing quality of ideas! @CambridgeMLG

20

5

400

Alexander Terenin

@avt_im

2 months

So let’s learn from the scientific mistakes of the past, and not broadly dismiss *any* area of machine learning. Better to lose a tiny bit of taxpayer money but give scientists room to experiment. Else, valuable ideas might go undiscovered because everyone thinks the same way.

10

11

373

Alexander Terenin

@avt_im

4 months

The NeurIPS high school track has made its way to Chinese social media. The collaborator who sent me this said there is at least one example of a professor asking their PhD student to help write a paper for one of their kids so that it can help their college admissions abroad.

13

48

366

Alexander Terenin

@avt_im

2 months

I started working on machine learning in my mid-twenties, after simultaneous degrees in psychology and statistics - the latter with no linear algebra, no optimization, no decision theory, no rigorous proofs, and virtually no code. I didn't even know what gradient descent was.

Andrew Côté

@Andercot

2 months

Am I too late to get into deep-tech / hardware / physics / science / engineering? I didn't start my engineering degree until I was 24. Before that I was working in marketing and sales, I had a humanities degree, and for my entire life had been "not a math guy" So no. Go do it

89

172

2K

10

346

Alexander Terenin

@avt_im

4 years

Efficiently sampling functions from Gaussian process posteriors Time to showcase our ICML paper! Here, we develop techniques for linear-time sampling from GP posteriors, which also enables one to easily differentiate through GP sample paths. (1/n)

3

60

337

Alexander Terenin

@avt_im

3 years

Open-source PhD thesis update! Yesterday and today, I wrote a draft introduction for my thesis, which introduces Bayesian learning, statistical decision-making, and Gaussian processes at a broad, informal level, motivating why these concepts should be formalized. Check it out!

7

30

330

Alexander Terenin

@avt_im

3 years

Graph neural networks are one of the best-kept secrets of the deep learning revolution: a model class that both really works, to the point of creating actual commercial value, and gives a useful conceptual framework for thinking about deep learning.

Google DeepMind

@GoogleDeepMind

3 years

Our team partnered with @googlemaps to boost the accuracy of ETA estimation in major cities across the world by up to 50% using graph neural networks. Today we’re pleased to share the full paper, which has also been recently accepted @CIKM2021 . Paper: 1/

20

276

1K

2

48

329

Alexander Terenin

@avt_im

3 years

You've seen many descriptions of Bayes' Rule. But have you ever seen one that's this pretty? 😄

6

46

317

Alexander Terenin

@avt_im

1 year

Ever wanted a Gaussian process whose domain is a manifold important enough to have a name? We've worked out a general recipe for building kernels on non-compact manifolds which are symmetric spaces, incl. hyperbolic space and the space of SPD matrices!

7

46

295

Alexander Terenin

@avt_im

3 years

Wow, over 500 people decided my open-source PhD thesis writing idea was cool enough to give it a like! Thank you all for the encouragement! Today, I finished the section on multivariate Gaussians - introducing and reviewing their properties. Check it out!

7

10

270

Alexander Terenin

@avt_im

3 years

Time to showcase our group's final AISTATS paper, "Learning Contact Dynamics using Physically Structured Neural Networks" which explores modeling of non-smooth contact dynamics. Shoutout to first author Andreas Hochlehnert - this is his MSc work! @mpd37

5

38

268

Alexander Terenin

@avt_im

2 months

The people developing Bayesian learning are not stupid. There are things it can do other (general) methods can’t, my chief example - of commercial and scientific importance - being Bayesian optimization and other decision-theoretic problems where we want to minimize regret.

4

258

Alexander Terenin

@avt_im

2 years

Announcing the 2022 NeurIPS Workshop on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems! Please consider submitting a 4-page extended abstract by September 22nd! Thanks to amazing organizers @gpleiss , @liza_p_semenova , @ziwphd

1

47

256

Alexander Terenin

@avt_im

3 years

Passed my PhD viva with minor corrections! Thanks very much to the committee for examining me, and my advisors and collaborators for the support over the years! @StefanoErmon @mmbronstein @mpd37 @flaxter

17

1

254

Alexander Terenin

@avt_im

2 months

So what does our friend Valeriy, from the original tweet, want us to do? Go big on what is popular *today*, chiefly (I presume deep net based) conformal prediction. Other approaches are a “waste of taxpayer money”. People said the same thing about neural nets a decade ago.

1

5

244

Alexander Terenin

@avt_im

2 months

It’s easy to jump on the right bandwagon with the benefit of hindsight. What matters is how you act towards ideas that may not be quite there yet. Do you dismiss areas broadly, or do you get into nuanced technical discussion?

2

13

242

Alexander Terenin

@avt_im

1 month

A preview of a figure from a paper landing on arXiv very soon! Can't wait to tell you what I've been thinking about for the past year! Stay tuned..

9

7

222

Alexander Terenin

@avt_im

2 months

First: make no mistake - if you insult students I’ve worked with - complete with surface-level “Darwin College” jokes - you insult me. Still, instead of dwelling on this, let’s focus on the way of thinking that leads one here. We can learn something important from it.

4

1

214

Alexander Terenin

@avt_im

2 years

Announcing the virtual seminar series on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems! Time: Mondays at 16:00 UTC The inaugural keynote will be given by @ta_broderick on March 14th (pi day). Please register on the website!

Virtual Seminar Series on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems

gp-seminar-series.github.io

3

46

210

Alexander Terenin

@avt_im

2 months

Valeriy’s point is that Bayesian methods, Gaussian processes included, can give poorly calibrated results. Therefore: developing them is a waste of resources and should not be pursued. This point can be made without “Darwin” Worse: this kind of thinking harms the ML ecosystem.

4

2

211

Alexander Terenin

@avt_im

2 months

So let’s talk about it! First: imagine you’re Yann LeCun in the 90s. You’ve seen what neural nets can do on tiny problems by today’s standards. You’re convinced, if only there was more data and compute, they’d decisively solve various hard image processing problems.

4

3

205

Alexander Terenin

@avt_im

2 months

So what happens? Neural nets develop slowly, until ImageNet - at that point, the results became so strong anyone ignoring them was a fool. And there are plenty of folks who - still today - dismiss deep learning as some kind of non-rigorous mumbo-jumbo going away soon.

1

205

Alexander Terenin

@avt_im

2 years

If you are interested in AI research, and good at mathematics, but discouraged by advice to focus on code, ignore it. On some problems, progress will come from experiments and empirical work. On others, it will come from theorem and proof. The field needs both kinds of people.

Mo Bavarian

@mobav0

2 years

This is probably well-known in some circles but not everywhere. The most important skill for Research Scientists in AI (at least at @OpenAI ) is software engineering. Background in ML research is sometimes useful, but you can usually get away with a few landmark paper.

38

193

2K

8

14

204

Alexander Terenin

@avt_im

3 months

Today at ICLR, we're presenting "Stochastic Gradient Descent for Gaussian Processes Done Right". This work uses insights from optimization theory to improve performance of stochastic-gradient-descent based large-scale Gaussian processes training algorithms. More below:

3

18

199

Alexander Terenin

@avt_im

2 months

You try to convince your colleagues neural nets are promising. You get strong empirical results for the time and show them to others. And how do people react? Skepticism. “You can’t prove SGD converges, therefore it doesn’t” “We need more SVMs (& other ideas big at the time)”

3

2

197

Alexander Terenin

@avt_im

4 years

Got rejected for a junior research fellowship at Cambridge, after making it to the final interview round. While I’m disappointed to still be on the academic job market into next year, I’m incredibly honored to have made it to the end in a contest with <1% success rate.

6

2

184

Alexander Terenin

@avt_im

6 months

I recently wrote a blog post on what successful research might look like for me, in order to better orient my work at Cornell over the next several years. I'd love your thoughts on it!

On Successful Research

avt.im

2

21

184

Alexander Terenin

@avt_im

4 years

Variational Integrator Networks for Physically Structured Embeddings Time to showcase our AISTATS paper! We use ideas from neural ODEs to study data-efficient representation learning of physical dynamical systems. (1/n)

7

38

179

Alexander Terenin

@avt_im

1 year

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent New preprint - with some exciting and counterintuitive results - is now on arXiv! Joint work with: @JihaoAndreasLin @JaviAC7 @shreyaspadhy David Janz @jmhernandez233

2

30

173

Alexander Terenin

@avt_im

2 years

When working with a Gaussian process, have you ever wondered why Cholesky factorization failed, or a CG solve did not converge? Answer: it's because you've got redundant, overlapping data points. And that's just the starting point! On arXiv now!

Numerically Stable Sparse Gaussian Processes via Minimum...

Gaussian processes are frequently deployed as part of larger machine learning and decision-making systems, for instance in geospatial modeling, Bayesian optimization, or in latent Gaussian models....

arxiv.org

3

20

162

Alexander Terenin

@avt_im

17 days

Next Monday at UAI 2024, together with @vabor112 , we'll give a tutorial on Geometric Probabilistic Models! The key theme will be on principled geometry-aware handling of uncertainty. We're really excited to present, and will be making our slides and notebooks available online.

1

13

157

Alexander Terenin

@avt_im

3 years

I haven't yet tweeted about Chapter 3 of my thesis - time to tell you about that! This chapter studies properties of geometric Gaussian process models, such as the Matérn class. Here's what the covariance kernel of these processes looks like on a sphere and dragon manifold!

4

10

153

Alexander Terenin

@avt_im

2 years

There's no talk tomorrow for the Gaussian process seminar series due to the holiday, but check out the schedule for more upcoming seminars!

Virtual Seminar Series on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems

gp-seminar-series.github.io

0

11

150

Alexander Terenin

@avt_im

4 years

Very honored and excited to announce that our work "Efficiently sampling functions from Gaussian process posteriors" received an honorable mention for the ICML2020 outstanding paper award! (1/n)

Alexander Terenin

@avt_im

4 years

Efficiently sampling functions from Gaussian process posteriors Time to showcase our ICML paper! Here, we develop techniques for linear-time sampling from GP posteriors, which also enables one to easily differentiate through GP sample paths. (1/n)

3

60

337

1

15

148

Alexander Terenin

@avt_im

1 year

Announcement: with my Cambridge funding ending soon, I'm looking for postdoc opportunities in the United States or Canada! I'm interested in branching out from Gaussian processes and working on robotics-inspired ML such as planning, RL, and world models. Please reach out!

4

22

140

Alexander Terenin

@avt_im

2 years

My first submittable faculty application draft is done, so perhaps it's time to announce: I'm on the machine learning and computer science faculty job market! I study data-efficient learning and decision-making. If you want to know more, please reach out, and spread the word!

6

23

138

Alexander Terenin

@avt_im

2 years

Today, I graduated from Imperial College. Reflecting, what a journey it’s been - plenty of challenges along the way. Since I started, all I’ve wanted was to calmly and peacefully work on and solve the problems that interested me. I still haven’t gotten there. But I made it!

6

0

135

Alexander Terenin

@avt_im

2 years

Here's a teaser from a paper I've been working on this year - it's a squared exponential Gaussian process on a real projective space! And there's so much more.. Landing on arXiv hopefully in a few days!

3

5

134

Alexander Terenin

@avt_im

2 years

Academic career milestone: for the first time, somebody who I've never met or communicated with online, and who hasn't previously worked with one of my advisors, invited me to give a talk.

2

1

129

Alexander Terenin

@avt_im

3 years

Even more good news! Our paper "Pathwise Conditioning of Gaussian Processes" is accepted at the Journal of Machine Learning Research! This work explores some ideas following our ICML paper - blog post for that one below! @mpd37

2

18

130

Alexander Terenin

@avt_im

4 years

Matérn Gaussian processes on Riemannian manifolds Time to showcase our latest work! Here, we develop training techniques for Matérn Gaussian processes f : M -> R on Riemannian domains, including computing the kernels of said processes. (1/n)

3

18

124

Alexander Terenin

@avt_im

10 months

After a year or two of not submitting anything to machine learning conferences, my NeurIPS record this year is three for three! I'll tweet about the papers - and about the new job I just started - soon, but for now I'm just happy that my work is finally ready to share with you!

5

0

122

Alexander Terenin

@avt_im

3 years

“Physics isn’t mathematics, where major advances can be done with a pen and piece of paper. It’s an experimental science..” To dismiss mathematics is to dismiss reasoning itself. Doing so risks repeating the same class of mistakes that held neural networks back for years..

Jean de Nyandwi

@Jeande_d

3 years

Machine learning isn’t mathematics or physics, where major advances can be done with a pen and a piece of paper. It’s an engineering science - @fchollet

8

21

201

6

15

113

Alexander Terenin

@avt_im

3 years

Our recently-accepted JMLR paper "Pathwise Conditioning of Gaussian Processes" is now live! Check it out! @mpd37

1

22

119

Alexander Terenin

@avt_im

5 months

Call for new organizers - Seminar Series on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems! If you're interested in organizing the next season - inviting speakers, telling people about the series, maybe putting together a workshop - please get in touch!

1

13

115

Alexander Terenin

@avt_im

2 years

Today's NeurIPS reviews remind me that readers often focus on different things than authors. My submission - which to me felt like some of my most solid work yet - received the worst reviews I've gotten in years. Yet, this clearly isn't the reviewers' fault.. 1/3

2

108

Alexander Terenin

@avt_im

2 months

P.S. - you should also consider Switzerland! From talking to colleagues who study and work there, the scientific environment appears to be excellent, and may well be comparable to the one here.

4

3

106

Alexander Terenin

@avt_im

3 years

One week after surgery. Today, I managed to take a proper shower for the first time. Gradually starting to feel human again! It’s also a big day for a second reason: my PhD viva, which I insisted on not postponing, will take place in the evening. Wish me luck!

8

0

107

Alexander Terenin

@avt_im

3 years

Today, I finished a draft of the stochastic partial differential equation section in my thesis. These are used to define Gaussian processes on non-Euclidean spaces, and calculate their covariance kernels - one of the key contributions. I'm getting close to a full first draft!

1

104

Alexander Terenin

@avt_im

4 years

Excited to be organizing the NeurIPS workshop on Interpretable Inductive Biases and Physically Structured Learning! Check it out and consider submitting a 4-page workshop paper - extended abstract deadline is October 2nd. @_mlutter @cosmo_shirley @wangleiphy

2

21

98

Alexander Terenin

@avt_im

2 years

No, I am not doing okay today. There are so many things I want to talk about, but instead I'm at a loss for words, if not for my then for family's sake. So, I'm left with nothing but picking meaningless Twitter fights about Kullback-Leibler divergences.

0

1

96

Alexander Terenin

@avt_im

2 years

Happy NeurIPS deadline week! One of my collaborators just discovered a mistake in one of my proofs.. 😭 How's your week going?

4

1

95

Alexander Terenin

@avt_im

3 years

Let's talk about rejecting applicants who "did well in class". Every time we do that, we blanket exclude those not studying at top-tier research universities. Many excellent students study at universities without a single active machine learning researcher in their department.

Gautam Kamath

@thegautamkamath

3 years

Application season is coming up! I wrote a post on "How to Ask for a Letter of Recommendation." Comments welcome. Feel free to share with anyone applying for grad school or jobs. tl;dr: Ask ppl you've done research w/ early & be organized! @AcademicChatter

11

49

251

2

9

90

Alexander Terenin

@avt_im

4 months

Terrible for DEI. Imagine you're a smart kid who finds the @ml_collective Discord. You get into research, read some books, eventually write a NeurIPS workshop paper. Impressive? Not if people don't believe the work is real, because other kids' engineer parents do it for them.

Gautam Kamath

@thegautamkamath

4 months

NeurIPS 2024 will have a track for papers from high schoolers.

79

91

597

3

8

89

Alexander Terenin

@avt_im

3 years

Time to showcase one of our AISTATS papers! We study alignment of time series in different spaces - for instance, motion of a physical system in pixel space and coordinate space - using tools of optimal transport. @CohenSamuel13 @brandondamos @mpd37

1

15

89

Alexander Terenin

@avt_im

3 years

We've now got a blog post describing Matérn Gaussian Processes on Graphs, our AISTATS paper which won the "Best Student Paper" award, in detail. Check it out! @mpd37 @NicolasDurrande

0

16

89

Alexander Terenin

@avt_im

2 years

Very last last evening, I submitted the final version of my PhD thesis. I'll be sharing bits in the coming days - lots of pretty pictures to show! In the meantime, I wanted to thank everyone who's been here and offered encouragement. You made the process so much more fun!

2

1

87

Alexander Terenin

@avt_im

3 years

I wanted to share another figure from my thesis before NeurIPS reviews arrive! This one illustrates the class of pathwise-conditioning-based posterior Gaussian processes approximations we study, where the prior and update terms are approximated with finite basis expansions.

1

6

89

Alexander Terenin

@avt_im

3 months

Software package announcement! The Geometric Kernels package, which implements various geometric Gaussian process models from my papers - on manifolds, graphs, and similar - is now on PyPi! It can be installed with `pip install geometric_kernels`. Check it out!

Viacheslav Borovitskiy

@vabor112

3 months

Excited to announce a new release of GeometricKernels! You can now install the package via `pip install geometric_kernels`! Featuring new spaces, efficient GP sampling, and a much simpler interface, in addition to brand new tutorials & enhanced docs!

3

17

90

1

12

87

Alexander Terenin

@avt_im

3 years

Our paper “Matérn Gaussian Processes on Graphs” was selected for oral presentation at AISTATS! We’re still working on the blog post, but in the meantime check out the paper! @NicolasDurrande @mpd37

3

7

86

Alexander Terenin

@avt_im

8 months

I'm presenting three papers at NeurIPS - one spotlight, and one oral! (2) "Posterior Contraction Rates for Matérn Gaussian Processes on Riemannian Manifolds" - spotlight poster - with Paul Rosa, Viacheslav Borovitskiy, and Judith Rousseau. Short summary and links below!

1

2

79

Alexander Terenin

@avt_im

4 years

It’s definitely discouraging to not even land an interview once during the course of my PhD, in spite of having plenty of publications and even awards for my research. I guess my research area isn’t interesting to people there who are hiring - I hope it’s interesting to others.

3

1

78

Alexander Terenin

@avt_im

3 years

Last time, I showed you what the Matérn kernel defined on the surface of the dragon manifold looks like. Here's what a posterior Gaussian process with that kernel, conditioned on training data, looks like! Notice how the standard deviation grows as we move away from data!

2

7

74

Alexander Terenin

@avt_im

6 months

Averaging is a really good idea! This idea was crucial in getting SGD for GPs to work, both in its original () and refined () form, by removing learning rate schedules. Geometric averaging works very well with multiplicative noise!

Stochastic Gradient Descent for Gaussian Processes Done Right

As is well known, both sampling from the posterior and computing the mean of the posterior in Gaussian process regression reduces to solving a large linear system of equations. We study the use of...

arxiv.org

Konstantin Mishchenko

@konstmish

6 months

I often hear people who know optimization theory say "averaging works assuming convexity, so it doesn't make sense for deep learning." Yes it does, it just works. Often, it can replace learning rate scheduler. Just don't use uniform weights, use EMA or polynomial weights.

7

14

150

1

6

76

Alexander Terenin

@avt_im

8 months

I'm presenting three papers at NeurIPS - one spotlight and one oral! (3) "Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent" - oral presentation - check out the linked blog post on this work! Summary, authors, and links below!

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent

papers.avt.im

1

9

74

Alexander Terenin

@avt_im

2 months

@predict_addict Oh man, where do I begin: (1) My PhD was not funded by UK taxpayers. (2) Gaussian processes do work, with direct scientific and commercial value for the UK and elsewhere. They are the only model class with which you can reliably built a working Bayesian optimization algorithm.

4

0

74

Alexander Terenin

@avt_im

2 years

The only reason probability is mysterious is because the usual ways of teaching it aggressively avoid the concept of random variable because it is "too abstract". This is misguided. The concept is neither abstract, nor avoidable for purposes of understanding.

Lionel Page

@page_eco

2 years

The notion of “probability” is both familiar (we use it), and mysterious (we would typically struggle to define it). Here is a 🧵 to remove the mystery and make the meaning of “probability” more intuitive.

66

1K

5K

11

5

76

Alexander Terenin

@avt_im

3 years

Headed for surgery. Hoping for no surprises last minute. Wish me luck and see you all soon!

3

0

75

Alexander Terenin

@avt_im

3 years

The problem with filling spots with those “already published” is that this is an incredibly strong proxy for “Harvard, Stanford, MIT, ..” Undergraduates at institutions with no ML research labs, no advising, and no connection to the community have no chance. This isn’t right.

Peyman Milanfar

@docmilanfar

3 years

I've the sense many students are legitimately concerned that most of the spots in CS grad admission will be filled with applicants who have publications already. The relevant question therefore is whether having publications is an overwhelming advantage, even if not necessary

3

2

39

5

4

73

Alexander Terenin

@avt_im

3 years

Today, I gave a brief tutorial on multi-armed bandits at UCL. Slides are available here: . If you missed it, don't worry! I'm giving the same tutorial again tomorrow at 12:30 UTC, and will hopefully not forget to record it that time.

1

16

74

Alexander Terenin

@avt_im

3 years

The Journal of Machine Learning Research has clearly demonstrated that for-profit companies do not need to play a role of any kind in order to run a successful journal with no fees. I encourage everyone who is on an Elsevier editorial board to resign immediately.

Wim Thiery

@WimThiery

3 years

Dear @ResearchGate , Thank you for your message and the threat to suspend my account. To avoid this from happening in the future, my group will never submit a paper to an #Elsevier journal again. #OpenScience Kind regards, Prof. Wim Thiery

25

139

744

0

13

73

Alexander Terenin

@avt_im

1 year

Scott Aaronson: "like the Jesuit astronomers declining to look through Galileo’s telescope, what Chomsky and his followers are ultimately angry at is reality itself, for having the temerity to offer something up that they didn’t predict and that doesn’t fit their worldview."

2

8

73

Alexander Terenin

@avt_im

3 years

ML/Stats Twitter: does anyone understand why the convergence properties of K-means are so bad? Is there an alternative clustering algorithm which runs in O(N) and can guarantee that nearby points are clustered while different clusters are separated?

10

8

70

Alexander Terenin

@avt_im

3 years

As a postdoc starting soon at the University of Cambridge, I'm delighted to say that I'll be joining Sidney Sussex College as a College Research Associate! @SidneySussex @Cambridge_Uni @CambridgeMLG @Postdoc_Academy

1

2

68

Alexander Terenin

@avt_im

4 years

What I read: as a PhD applicant, the most important factor is which university you studied at - this determines both who writes your letters and what research opportunities, if any, you had available. But what about all the talented people who didn’t have these opportunities?

Fadel Adib

@fadeladib

4 years

I've been serving on grad admissions committees at MIT for 5 years - in EECS and Media Lab If you want to get into a PhD at a place like MIT, here's a thread with some advice based on my observations 1/13

34

550

2K

4

7

70

Alexander Terenin

@avt_im

2 years

Thanks to everyone who participated in the NeurIPS Workshop on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems! At our poster session, I counted approximately 140 people - a spectacular success! Lots of work, but well worth it.

3

1

69

Alexander Terenin

@avt_im

5 years

I've been interested in applying differential geometry to study machine learning problems with physical interpretations for some time now. These notes are the cleanest presentation I've seen yet of the geometric and physical aspects of this setup.

0

16

68

Alexander Terenin

@avt_im

3 years

The long-term consequence of big tech dismissing ethics and fairness concerns will be regulation and loss of control regarding what systems can be deployed and how data can be used. I wonder how much faster this will come as a result of @timnitGebru and @mmitchell_ai 's firing?

2

15

67

Alexander Terenin

@avt_im

2 years

Wow, this really blew up. I just want to make enough of a living to one day own a home and have the opportunity to start a family in a financially responsible manner. Everyone deserves to be able to do that. Is that really a lot to ask for, as a young academic?

4

66

Alexander Terenin

@avt_im

3 years

Yesterday was a really big day! I managed to make it through the NeurIPS abstract deadline, a JMLR camera-ready deadline, and a decision deadline all in one day. These are all great deadlines to have, but damn, that was a lot.

0

66

Alexander Terenin

@avt_im

3 years

My thesis has a total of 94 plots, shown within 34 distinct figure environments. As I get close to submitting, I get closer to being ready to share more of these with you. Here's a preview of the updated version of Figure 2.1 from earlier: . Stay tuned!

Alexander Terenin

@avt_im

3 years

I've begun making figures for my thesis, starting with the Gaussian process section! Here's the first one, which illustrates the process of sampling conditional multivariate Gaussians in the standard manner. This is to be contrasted with the pathwise view presented next.

1

5

51

3

63

Alexander Terenin

@avt_im

3 years

When we first wrote this paper, our goal was simply to synthesize prior work in the statistics community with modern tools such as automatic differentiation and inducing points, so that people were aware that graph Gaussian processes existed, and could use them more easily.

AISTATS Conference

@aistats_conf

3 years

Congratulations to Viacheslav Borovitskiy, Iskander Azangulov, Alexander Terenin, Peter Mostowsky, Marc Deisenroth, and Nicolas Durrande on winning AISTATS 2021's the Best Student Paper Award for their paper "Matérn Gaussian Processes on Graphs"

1

28

216

3

9

64

Alexander Terenin

@avt_im

3 years

It took me years of hard work in my mid twenties to overcome an American educational system which gave up on my ability to learn proper mathematics early on without giving me a chance to try. Where others laugh at this joke, I only feel sadness.

Brian Skinner

@gravity_levity

3 years

1/2 A Russian mathematician is hired by a math department in the US, and is assigned to teach Calculus 1. On the day before her first lecture, she asks a colleague: "what am I supposed to teach in this class?" The colleague says, "well, it's standard first-semester calculus...

12

114

753

4

0

63

Alexander Terenin

@avt_im

3 years

Absolutely remarkable. Graph neural networks seem astonishingly good at recognizing patterns in just about any kind of data one can imagine.

Petar Veličković

@PetarV_93

3 years

Our work has been published in @Nature !! (G)NNs can successfully guide the intuition of mathematicians & yield top-tier results -- in both representation theory & knot theory. See my 🧵 for more insight...

14

152

879

1

7

62

Alexander Terenin

@avt_im

3 years

Just finished my first in-person day as a postdoc at the University of Cambridge! I have a lot of reactions. I'm in a fascinating place, and it'll take time to get to know it. I'm incredibly happy to be back to research. There are so many ideas I can't wait to share with you!

2

0

63

Alexander Terenin

@avt_im

2 years

Today is both Valentine's day, and the day I'm due to submit my PhD thesis corrections. I'll be reading the thesis from start to finish to correct any last-minute typos, hopefully submitting either later today or tomorrow. I can't wait to share the final version with you.

1

0

62