(1/7) As I've been working on PAC-Bayes for a few years, often, students ask me what is an easy reference to begin with. My answer: "I know excellent and accurate books and papers on PAC-Bayes, but they're not easy". So I decided to write this tutorial.
The chairman of my session
@icmlconf
did not join, so I was asked to replace him and moderate questions to speakers.
But no other speaker joined either so there was no one to answer questions.
Fortunately, there was no question either.
...
...
🤣
(1/3) Personal news: in January 2023, I will join ESSEC Asia-Pacific (Singapore) as a Professor in the Department of Information Systems, Decision Sciences and Statistics
@ESSEC_AP
@essec
"User-friendly Introduction to PAC-Bayes bounds" is now published:
Thanks a lot for all the feedback on the first version I posted on arXiv some time ago 🙏
I'll be teaching a course on this topic at SSML 2024
@OISTedu
So proud of my PhD student Badr-Eddine Chérief-Abdellatif
@ENSAEparis
: his first paper ever submitted to
#ICML2020
is accepted, and he is is single author!
"Convergence Rates of Variational Inference in Sparse Deep Learning"
Le modèle scientifique "n'a rien à voir avec celui de la vie politique. Le problème, c'est que nous avons des directions de journaux qui proviennent toutes de formations de type Sciences Po (...) et voient la science et ses débats à travers ce prisme."
Desk reject
@NeurIPSConf
because "paper does not meet the formatting requirements". No explanations & they don't reply to messages, but it seems it's this recurring A4/letter format problem. Please reply if you had the same issue. This is not a valid reason for rejection.
Submitted in June 2017, revised, revised again, accepted, waiting to be published.... Until today ! Happy that our paper with James on the concentration of variational approximations just appeared in the Annals of Statistics ☺️
Sorry, I'll allow myself one more self-promotion message: I finally received my own copy of "User-friendly introduction to PAC-Bayes bounds". I can promise: your rabbits will really like to use it as a tent if they go camping 🏕️
Little update on "User-friendly introduction to PAC-Bayes bounds": I spent some time this year revising/fixing some chapters thanks to the thorough comments of two reviewers 🙏
It will appear in Foundations and Trends in Machine Learning in the first quarter of 2024.
(1/7) As I've been working on PAC-Bayes for a few years, often, students ask me what is an easy reference to begin with. My answer: "I know excellent and accurate books and papers on PAC-Bayes, but they're not easy". So I decided to write this tutorial.
Our paper on robust regression via MMD minimization with Mathieu Gerber from
@BristolUniMaths
accepted for publication in Biometrika 🥳
We will do our best to release a R package (+python hopefully) this summer. These estimators work
*well*.
Our paper on the robustess properties of the MMD estimator is now officially accepted for publication in the Bernoulli journal !
Should be 🍾🍾 time but hey, the city is in state of emergency, bars are closed, it will be 🧃🧃 tonight !
Badr-Eddine + I have been talking about it for weeks, it's finally ready! Further study ot MMD estimation (
@MarkGirolami
@fx_briol
&
@roydanroy
): robustness properties to misspecification and dependence:
Ismaël Castillo's lecture notes on Bayesian nonparametrics (Saint-Flour summer school on probability theory 2023) are now on arXiv.
I attended the summer school, the lecture was amazing. Highly recommended!!
Dear all,
I just arrived in Singapore. I was welcomed by many friendly herons. I will post more photos later. I wish you a very happy new year.
明けましておめでとうございます!!🥳 シンガポールに引っ越しした。ここも、サギがたくさんいる。
The special issue "Approximate Bayesian Inference"
@Entropy_MDPI
is now available as book/pdf, you can download it (it's free):
無料で「Entropy」学術誌の特別号の「近似ベイズ推論」をダウンロードできます。表紙の写真は中目黒のアオサギです :-)
We hired 2 post-docs in the first half of 2020 and they are still not allowed to enter Japan. Can't we allow them (and everyone in a similar situation) in before the Olympic athletes ? The Olympics are in 150 days, not next month!
Japan to begin allowing foreign Olympic athletes' entry next month: Nikkei: Athletes participating in this year's Olympic Games in Tokyo can begin entering Japan after it lifts its emergency declaration order next month, the Nikkei newspaper reported on…
Our paper with James Ridgway on consistency and concentration theorems for variational approximations in Bayesian inference is officially accepted today for publication in the Annals of Statistics 😊
The preprint is available on arXiv:
This I finally found a copy of Stéphane Boucheron's book published in... 1992. Contains pedagogical introductions to:
- Valiant's PAC learning,
- Vapnik-Cervonenkis bounds,
- neural networks,
- Gold's recursive learning...
I just noticed that the PhD thesis of Borja Rodríguez Gálvez is now online. Many interesting results, it's really something nice to read! "An Information-Theoretic Approach to Generalization Theory"
New preprint: "Non-exponentially weighted aggregation: regret bounds for unbounded loss functions".
- study of (1.4), extends Bayes & exponential weights (1.3),
- explicit formula (3.4),
- regret analysis, without boundedness of the loss.
Joining the team of moderators
@arxiv
this week. Happy to support
@arxiv
and open science!
(As a consequence, please inform Emmanuel Macron I'm not available for the currently vacant job of prime minister in France.)
@mathladyhazel
There is so much debate on whether 0°=1 or 0°=0 that we should decide 0°=1/2 to make it look like we found a compromise and make everyone unhappy.
After three years doing research full time, happy to teach again! My first class is tomorrow morning. As you can see I am almost stressed. Life is almost hard.
Just arrived at Toulouse School of Economics
@TSEinfo
I'll be staying there until Nov. 29 and teach a short course on PAC-Bayes and Mutual Information bounds.
All packed up. I'll be travelling to Europe tomorrow!
Busy schedule:
- research with my colleagues from the French campus
@essec
,
- talks at
@UCDMathStat
(30/11), Orsay (7/12), Jussieu (8/12) and
@stats_UCL
(14/12),
- attend a conference
@_CIRM
.
Hopefully see you there!
The Lebesgue measure λ(.) is defined on sets A⊆R that are Lebesgue-measurable. Under the axiom of choice, some sets are not.
However, it is possible to extend λ(.) to ALL subsets A⊆R, at the cost of replacing σ-additivity by finite additivity.
Banach proved it. In French 😅
I started my PhD in September 2003. For 20 years, I managed to keep my mail box clean. Never more than a few unread messages, each message answered with 1 or 2 days.
Well, I regret to inform you that this time will end soon.
Just wow. I saw a stork-billed kingfisher for the first time... on the way to the train station! It's beautiful, and unexpectedly big for a kingfisher!
Martin-chasseur gurial // コウハシショウビン。
Our paper with
@OtmaneSakhi
& Nicolas Chopin accepted
@icmlconf
2023: "PAC-Bayesian Offline Contextual Bandits With Guarantees".
Thanks a lot to my co-authors! I learnt a lot by working with them. My 1st publication with the word "bandit" in the title 😅
Just arrived in Japan. I'm in Atami, attending the Workshop on Bayes Duality
The hotel is amazing!
I'll be in Tokyo on Monday only to give a talk
@RIKEN_AIP_EN
(that's also part of the workshop).
I will give a talk at the Bayes Club on April 21. I will highlight some differences and similarities between PAC-Bayes bounds and contraction of the posterior, with consequences for variational approximations.
Markus Reiss on TBA (Theory Based on Applications): lower bounds for SPDE estimators. Amazed by how pedagogical the talk was given the technical nature of the topic.
@_CIRM
#MMS2023
The only thing that makes online conference bearable is that you can change your clothes each time you go from one poster room to the other.
#AISTATS2022
Still locked at home for a few days, but I can't imagine a better view to start 2022 !
Happy new year to all of you ! ❤️🎉
明けまして おめでとうございます‼️🍾🎊
Bonne année à tous ! 🐥🦩
Just arrived in Beijing. I will attend the International Conference on the Mathematical Theory of Deep Learning at the Chinese Academy of Sciences (and give a talk).
Once, in class, some students asked me how to ensure measurability in the proof of the delta-method... today I see that there is a clean way to do things :)
Our paper "Matrix factorization for multivariate time series analysis" written with Nicolas Marie
@ESMEsudria
/
@UParisNanterre
accepted for publication in the Electronic Journal of Statistics 😊
(1/8) New paper with Geoffrey Wolfer
@geowolfer
on arXiv today: "Optimistic Estimation of Convergence in Markov Chains with the Average Mixing Time"
@RIKEN_AIP_EN
@essec
"I wish I had trusted 🐵 from the beginning" is such a perfect definition of regret in online learning!
Lecture by Shinji Ito on online learning at
#MLSS2024
I'm glad
@Aistats2020
is popular but I'm not sure this is good news. We (as a community) submit and publish too many papers. We should seriously think about it.
Accepted for publication in the Annals of Statistics :)
P. Alquier, V. Cottet & G. Lecué, Estimation Bounds and Sharp Oracle Inequalities of Regularized Procedures with Lipschitz Loss Functions.
The preprint is available on arxiv:
I will give a talk at
@NUSingapore
seminar of statistics & data science on April 12.
Old & new results on "consistency of variational Bayes". I hope to connect with everyone interested in Bayes & PAC-Bayes here, and hopefully initiate new collaborations!
Lol. In France, shoplifting is not a crime, and we do it sometimes for fun. I thought people in Japan would enjoy it too, but thanks to this document, I am now aware that I could be in trouble, and I will not do it.
Our paper with Badr-Eddine Chérief-Abdellatif, Alexis Derumigny and Jean-David Fermanian accepted for publication in Journal of the American Statistical Association 🍾🍾🍾
Link to the paper:
Link to the R package:
Do you use variational approximations to estimate mixtures? Have you ever wondered about their consistency? Rate? Can you still sleep at night? Do you feel nervous?
Don't wait! This paper by my PhD student Badr-Eddine Chérief-Abdellatif is all you need!
Honoured & very happy that our paper was actually selected for the best paper award by the chairs of ACML 2019:
Congrats to my co-authors Badr-Eddine &
@EmtiyazKhan
and thanks to our institutions
@RIKEN_AIP_EN
and
@ENSAEparis
"A Generalization Bound for Online Variational Inference" by Badr Chérief-Abdellatif
@ENSAEparis
, Emti Khan
@EmtiyazKhan
@RIKEN_AIP
+ myself accepted at ACML2019 😀 see you in Nagoya 😎
New preprint: "MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy" with B.-E. Chérief-Abdellatif
@ENSAEparis
Bayesian version of MMD-based estimation of
@fx_briol
@MarkGirolami
&
@roydanroy
Quand des gens viennent, dans nos écoles, dire ça, on ne devrait pas laisser passer. C'est faux. Les étudiants doivent savoir que quand Google & Facebook ont ouvert leurs centres IA à Paris, ce sont des chercheurs, avec des thèses, qu'ils ont recruté.
Tomorrow, and Wednesday next week, I will give a lecture at Todai
@UTokyo_News_en
. It will be my first time teaching there, I am really honoured by this.
... just a bit strange that I have to do it from my own living room 😎
New preprint: "MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy" with B.-E. Chérief-Abdellatif
@ENSAEparis
Bayesian version of MMD-based estimation of
@fx_briol
@MarkGirolami
&
@roydanroy
This is now my flag (seriously). I want to say to the world that my France is not the France of Le Pen. It is the France of Henri Lebesgue and his measure.
My friend and co-author Charles Riou launched his own YouTube chanel. It's meant as an online reading group where he discusses new papers on ML and statistics. Please check it 🙏
@MLNewPapers
I'll give an online talk at the Center for Doctoral Training in Modern Statistics and Statistical Machine Learning at Imperial & Oxford
@StatMLIO
: Tuesday, Aug. 18, 10pm UK time.
Working on the slides right now :)
Based on: and
Announcing the ICLR 2025 Call for Papers!
Abstract submission: 11:59pm, Sept 27 (AoE)
Submission date: 11:59pm, Oct 1 (AoE)
Reviews released: Nov 12
Author/Reviewer Discussion: Nov 12-26
Author Last Day to Reply: Nov 27
Final Decisions: Jan 22 2025