Balázs Kégl Profile Banner
Balázs Kégl Profile
Balázs Kégl

@balazskegl

3,069
Followers
1,095
Following
711
Media
8,035
Statuses

Head of AI Research @HuaweiFr . Podcast: Blogs: , .

Orsay, France
Joined May 2011
Don't wanna be here? Send us removal request.
@balazskegl
Balázs Kégl
3 years
I feel angry that it is so hard to publish machine learning contributions on small data. Most problems are not images and NLP, and most data scientist work on relatively small tabular data. And most of the _value_ is there. And we know very little about that regime.
27
86
707
@balazskegl
Balázs Kégl
3 years
What do you think about this argument? GPT-3 is obviously not intelligent. It was fed by a nontrivial percentage of all text produced by humans which cannot grow by several orders of magnitude. Therefore the paradigm of feeding data into function learning is a dead end for AGI.
57
31
418
@balazskegl
Balázs Kégl
9 months
@ylecun Thanks Yann, I think your political role now is as important as your technical role.
4
3
228
@balazskegl
Balázs Kégl
7 months
@ylecun I don't think it's that simple. The most awesome property of living cognition is not the absorption of 1E15 bytes of video "data" the child sees but its filtering. How little it actually needs to "see" to build a world model. And the key is that a child is not a mere observer,
7
7
100
@balazskegl
Balázs Kégl
7 months
@ylecun You can only learn causal models by embodying the AI and let it manipulate the world, running experiments. Watching videos and reading text will not get you to causal world models, whatever the architecture and the task you define.
10
2
85
@balazskegl
Balázs Kégl
4 years
We are hiring at Huawei's Paris AI research! - PhD in 2020 or 2021 - permission to work in France - strong publication record - preferable domains: RL, bandits, physical systems, time series, AutoML, combinatorial and online optimization, transfer learning RT please
Tweet media one
0
21
60
@balazskegl
Balázs Kégl
7 years
Gender balance at a #NIPS2017 workshop panel, sooo refreshing #nips4creativity #creativeAI @douglas_eck
Tweet media one
1
14
59
@balazskegl
Balázs Kégl
8 years
The death of lecture. Why on earth should I prepare a course on #convnets other than a Q&A on this?Thanks @karpathy
0
8
60
@balazskegl
Balázs Kégl
10 months
@ylecun @EduardoSlonski @DrJimFan @RichardSSutton We also have 10s to 100s millions intero and exteroceptive neurons (estimate by GPT :) ). Also, we don't just given a set of data and learn a function, we live, the paradigm is way more complex than anything we do in ML. The rest of the body, may be as important as the network
4
1
56
@balazskegl
Balázs Kégl
1 year
@martinmbauer But I love the supercoductor discussion on twitter. It's a new category, halfway between poetry and science.
0
0
51
@balazskegl
Balázs Kégl
7 years
My slides: A historical introduction to #DeepLearning : hardware, data, and tricks #bigdata #datascience #ML
Tweet media one
Tweet media two
Tweet media three
Tweet media four
0
36
50
@balazskegl
Balázs Kégl
10 months
I think "Bayesian" is mostly a synonym of "subjective", probabilities unsupported by third-person scientific data. I think there is nothing wrong with quantifying feelings and even asking for validation, just let's be clear about what we're doing.
@ChristophMolnar
Christoph Molnar
10 months
Two types of Bayesians 1) Beer pub Bayesian: People who want to be seen as intelligent by using words like "Bayesian", "prior", "posterior", "updating" to refer to the fact that people have opinions and sometimes change their minds. 2) Actual Bayesian who analyzes data
22
30
365
5
7
49
@balazskegl
Balázs Kégl
3 years
@davidasboth The surprise of the last year for me that you can train neural nets on 200 points. OK, it's smooth low-dim data. But you _do_ need NNs, linear or forests do not work. And the experiments generated strange puzzles over heteroscedastic NN regression.
2
8
48
@balazskegl
Balázs Kégl
11 months
@tszzl Yes. Banks seems to overlook the spiritual way though that we also have a tendency to follow facing technocratization. Culture is changing, but so do we, individuals. With food and shelter assured, more and more people will have time to explore enlightenment, which will
9
5
43
@balazskegl
Balázs Kégl
3 years
A constructive idea: let's have a #5000 track or tag at conferences, in which papers only use training data with <= 5000 instances.
3
1
43
@balazskegl
Balázs Kégl
2 years
. @vervaeke_john proposes here an alternative to the Turing test wherein we let two copies of the subject to talk to each other. His scientific prediction is that within a minute the conversation, and I quote, "goes batshit crazy". I tested GPT-3. 👇
1
10
42
@balazskegl
Balázs Kégl
7 months
@GaryMarcus Given the enormous, unimaginable suffering in the 20th century, should we have stopped the Enlightenment? Had Luther foreseen the death of God, would he have gone ahead with his proposals?
4
0
42
@balazskegl
Balázs Kégl
2 years
I've been in this ML business for about 30 years now, and still can't get around the fact that the optimal random forest overfits like crazy. This is not how the world supposed to be.
Tweet media one
7
3
39
@balazskegl
Balázs Kégl
6 years
A major milestone: our #RAMP preprint paper is out. Don't hesitate to comment or discuss our vision on the openreview site if you are interested in #transparency and #reproducibility in scientific #datascience workflows.
Tweet media one
1
26
40
@balazskegl
Balázs Kégl
7 months
@ylecun True. What you fundamentally cannot learn by mere observation is what data you should learn from. What prediction is important/useful to care about. We are trying to "teach" AI everything to compensate this lack, and quantity covers it up superficially. But this is the main
2
2
38
@balazskegl
Balázs Kégl
6 years
We have concluded the #ASDchallenge of @SaclayCDS @institutpasteur @IESFfrance , 123 participants, 677 submissions, AUC moved from 0.68 to 0.81. Congrats to the winners and welcome to the collaborative phase! #datascience for #healthcare #MachineLearning
Tweet media one
2
22
38
@balazskegl
Balázs Kégl
1 year
I love that ICML accepts these kinds of papers, actually it's pretty surprising, we didn't even think about aiming. Just look at this plot and promise me never again using standard error as your confidence estimate on heavy-tail RL losses. Andrew Patterson, Samuel Neumann,
Tweet media one
4
7
34
@balazskegl
Balázs Kégl
3 years
Anybody else noticed that in reinforcement learning, performance across seeds is non-Gaussian? How do you cope with it? This means that reporting mean and even standard deviation is not that meaningful, we're averaging qualitatively different things.
Tweet media one
6
1
34
@balazskegl
Balázs Kégl
2 years
I don't know about "batshit crazy", but as much as I can tell, it is the right scientific term. More precisely, it's like Sheldon trying to pick up a woman in a bar.
Tweet media one
3
4
35
@balazskegl
Balázs Kégl
3 years
Let's celebrate the 30th anniversary of Rich Sutton's Dyna paper. Four pages, zero experiments, but one full page on limitations, 600 citations, and the founding of a sub-branch in reinforcement learning on which I am betting for AGI.
0
2
35
@balazskegl
Balázs Kégl
2 years
Forget ChatGPT, here is what's coming. As all devices will be equipped with self-optimizing AIs, they will naturally develop their personalities (since their experiences will differ). This means that engineering will go from designing "machines" to "shepherding" little AIs.
6
4
30
@balazskegl
Balázs Kégl
7 years
As long as it lives in a perfectly simulated world with a known and well defined objective.
@TheEconomist
The Economist
7 years
The latest AI can work things out without being taught
12
229
279
1
13
33
@balazskegl
Balázs Kégl
10 months
@roydanroy It's how you feel after someone defines "understanding" for you.
1
2
30
@balazskegl
Balázs Kégl
10 months
@ProfNoahGian Yes. Your wife would say that this is the nerd's way of asking for validation of our feelings.
@balazskegl
Balázs Kégl
10 months
I think "Bayesian" is mostly a synonym of "subjective", probabilities unsupported by third-person scientific data. I think there is nothing wrong with quantifying feelings and even asking for validation, just let's be clear about what we're doing.
5
7
49
1
3
30
@balazskegl
Balázs Kégl
9 months
@QiaochuYuan I like him, like his approach, and happy that he has success. He inspired me to start my podcast. You don't need to listen to him. You can also do what he does, follow your own instincts and manifest the way you think conversations should be done. It's enlightening.
1
0
27
@balazskegl
Balázs Kégl
1 year
The difference between planning in #reinforcementlearning and in #LLM is that in RL we check where we really are after each action, and replan. This means that the agent needs to be connected to the world. This rhymes with what @ylecun has been arguing.
4
3
24
@balazskegl
Balázs Kégl
2 years
Catchy title (GPT-3 is not a god to be believed in), but the article is hyper-precise. I think GPT-3 is a nice tool but missing grounding. Also nice to see how my vision articulates with @ylecun 's. IA : Pourquoi Huawei ne croit pas à GPT-3 via @zdnetfr
0
2
24
@balazskegl
Balázs Kégl
9 years
The #datascience ecosystem: actors, incentives, challenges. @SaclayCDS @U_ParisSaclay http://t.co/6tzYefS0D0
Tweet media one
0
23
23
@balazskegl
Balázs Kégl
8 years
Tired of ad optimization? Work w us @SaclayCDS on #DataScience in astro, health, ecolo w @agramfort @GaelVaroquaux
0
19
23
@balazskegl
Balázs Kégl
2 years
As we become less tribal, a major source of conflict is perceptual diversity. That we cannot imagine that others perceive the world differently from us. @anilkseth
0
5
23
@balazskegl
Balázs Kégl
2 years
This should be the default condition to write NeurIPS papers.
Tweet media one
3
0
21
@balazskegl
Balázs Kégl
10 months
"The immune system of a pregnant person is mindblowing" Amazing paper on how selves can be nested by @AnnaCiaunica @drmichaellevin @_fernando_rosas and Karl Friston Watch @AnnaCiaunica explaining the gist:
@AnnaCiaunica
Anna Ciaunica @annaciaunica.bsky.social
10 months
Nested Selves: Self-Organisation and Shared Markov Blankets in Prenatal Development in Humans via @OSFramework
2
16
65
1
8
22
@balazskegl
Balázs Kégl
3 years
Well, I think that's exactly the point of @GaryMarcus . Images that fool us need to be carefully engineered, and play no role in our survival (on the contrary, our "illusions" are probably necessary for survival), whereas deep nets have an arbitrary relevance landscape.
@ylecun
Yann LeCun
3 years
@GaryMarcus @savvyRL @scaleai @allen_ai @anh_ng8 @jeffclune @jasonyo Happy birthday human neural nets! You are nearly 1,000,000 years old now, and still utterly fooled by images like these.
11
24
221
2
3
22
@balazskegl
Balázs Kégl
9 years
. @agramfort is explaining his super cool fast Lasso at #icml2015 . Soon in @scikit_learn . http://t.co/ABfqlxfNyi
Tweet media one
1
18
21
@balazskegl
Balázs Kégl
8 years
Looking outside the #DataScience bubble. @lawrennd on the issues of sampling bias and data valuation.
0
5
22
@balazskegl
Balázs Kégl
11 months
I agree with @drmichaellevin that we have been producing high-level agential intelligences aka kids, without asking for guarantees about their alignment. That raises the following poll. Retweet if you're interested in the answer. Do you have
kids, worried AI alignmnt
55
kids, not worried
105
no kids, worried
64
no kids, not worried
209
18
8
22
@balazskegl
Balázs Kégl
6 years
My slides for the #DataScience colloquium @ENS_ULM today at noon in Jaurès Room (29 rue d’Ulm). Thanks @KrzakalaF @gabrielpeyre for the opportunity. #machinelearning #datachellenge #design
Tweet media one
Tweet media two
Tweet media three
Tweet media four
2
9
21
@balazskegl
Balázs Kégl
8 years
Why do some developers at strong companies like Google consider Agile development to be nonsense? by David Jeske
0
13
21
@balazskegl
Balázs Kégl
6 years
"Overall, there is a huge mismatch in what ML researchers consider “sexy” and what is important for ML practitioners!"
0
6
20
@balazskegl
Balázs Kégl
2 years
@BeingMIAkashs @ylecun Model-predictive control on a learned model _is_ reinforcement learning. It is called model-based reinforcement learning. This is like saying: "Abandon Machine Learning, in favor of optimization".
2
5
20
@balazskegl
Balázs Kégl
3 years
@OkbaLeftHanded That's true but in my experience e.g. small data reinforcement learning has a huge market, trillions per year. It's not the IT sector, more heavy (control) engineering and transportation so it's not the usual high-liquidity companies that dominate ML research today. But it's big.
2
2
20
@balazskegl
Balázs Kégl
2 years
@moreisdifferent @tdietterich Yes, we have the same experience in model-based reinforcement learning. I hate it because it's dumbly inefficient, but retrain from scratch on the full data each time a new data arrives is unbeatable for now.
1
0
20
@balazskegl
Balázs Kégl
10 months
If you find Sapolsky's no free will thesis appealing, read this dialog. Performative contradictions are best demonstrated through stories.
@drmichaellevin
Michael Levin
10 months
A short dialog between an applicant who doesn’t believe in free will and a hiring manager at a software company:
27
19
101
3
5
18
@balazskegl
Balázs Kégl
11 months
@casebash @drmichaellevin Best to listen to @vervaeke_john . We don't align kids by giving them and enforcing rules (purely), but by embodied stories and rituals, and by bringing them in contact both with others and with reality, so they bind to both.
1
3
19
@balazskegl
Balázs Kégl
6 years
On the ambitions of @scikit_learn , @GaelVaroquaux at the inauguration of the scikit-learn consortium @StationF @Inria
Tweet media one
0
5
18
@balazskegl
Balázs Kégl
6 years
The @SaclayCDS Solar wind #RAMP event of tomorrow is open. The goal is to detect variable-length events in multiple aligned time series. Don't hesitate to sign up if you would like to participate tomorrow or in the next two weeks.
Tweet media one
0
10
17
@balazskegl
Balázs Kégl
1 year
@CRSTNNSL @DavidDeutschOxf I think it may be purity vs mixing. There was less population density, languages developed "on their own" for 50K years. Then with the Indo-European explosion, there was much more mixing, and mixing meant to get rid of fragile complexities. You can see it in modenr-day English,
3
1
17
@balazskegl
Balázs Kégl
7 years
My slides for today's #NIPS2017 workshop talk on collaborative #datachallenge and #datascience process management. Room S1 in ten minutes.
Tweet media one
0
10
17
@balazskegl
Balázs Kégl
1 year
Happy Eastern!! Goulash a la cowboy.
Tweet media one
2
1
17
@balazskegl
Balázs Kégl
7 months
@ylecun Yes, it's the same category error: LLMs still cannot bike. Propositional and procedural knowing is not the same.
0
3
16
@balazskegl
Balázs Kégl
4 years
On the way to AGI, I think pure model-free RL is a dead end. Yes it is difficult for an agent to figure out what slice of the world matters for reaching its goals but this is not an excuse for not learning a model for that slice. Every living thing builds such a model. Pls refute
4
0
17
@balazskegl
Balázs Kégl
11 months
My conversation with @AnnaCiaunica about embodied cognition, feeling like a zombie, depersonalization, meditation, movement therapy, 1st person science, and much more. Pls give it thumbs up, share, comment, and sign up!
2
3
16
@balazskegl
Balázs Kégl
10 months
@TonyZador Hm, you put the genome in a test tube, it won't hunt. You put the colt in water, it won't stand. The point is that there are a lot of environmental and developmental affordances and constraints that produce the behavior, and those are also information.
2
0
14
@balazskegl
Balázs Kégl
1 year
@fchollet That's what we use in Hungary.
0
0
16
@balazskegl
Balázs Kégl
9 months
This is way scarier than AGI.
@pitdesi
Sheel Mohnot
9 months
I am often surprised by polling but this one scares the shit out of me. 1 out of 5 people 18-29 believe the holocaust is a myth. How did it get to be this way?
Tweet media one
944
619
5K
0
3
16
@balazskegl
Balázs Kégl
7 years
Congratulations to my and @AKAZAKCI 's PhD student @mehdidc defending his thesis on #DeepLearning and novelty generation. If you want to understand why we count letters generated by a model trained on digits when we don't even want to generate letters, ask him a copy of his thesis
Tweet media one
0
1
16
@balazskegl
Balázs Kégl
9 years
LSTM in ten lines of code #tensorflow #NIPS2015
0
4
15
@balazskegl
Balázs Kégl
10 months
DALLE is definitely out of sync :) "Realistic photo of a young tourist-looking guy smuggling weights of a large language model in his backpack at US airport customs in Chicago."
Tweet media one
6
8
16
@balazskegl
Balázs Kégl
2 years
@bernhardsson I have an upcoming preprint (brutally downscored by NeurIPS :) ) that shows that some of the Ray Tune engines beat random search by a factor of 2x to 3x. Best ones seem AX, BlendSearch, Hebo.
0
0
16
@balazskegl
Balázs Kégl
1 year
OK, I've seen stuff. But 6000 French CEOs celebrating my 17-year-old son with a standing ovation still feels surreal. What a day! I'm immensely proud, bravo Martin! Thank you @tweets_Apm !
@tweets_Apm
Apm
1 year
Martin a construit le coffre #Apm2023 , la fameuse capsule temporelle qui sera ouverte en 2050 !
Tweet media one
1
1
6
2
0
15
@balazskegl
Balázs Kégl
11 months
@DrJimFan @tegmark No debate, I agree :). What I would add is that "world" is a relative concept. It's what we sense. So LLM has implicit information about our world model. A WiFi antenna's world model will be quite different. In that sense LLMs will be helpful for an agent whose world model
0
1
15
@balazskegl
Balázs Kégl
7 years
How to build a #datascience pipeline. Start with the prediction target y. Involve #IT early.
1
9
14
@balazskegl
Balázs Kégl
6 years
The @SaclayCDS @institutpasteur @IESFfrance launch the Autism Spectrum Disorder #datachallenge . The competition will end on July 1st, followed by a collaborative phase. We award 9.5K€ in money prizes to the top ten participants. #ASDchallenge #datascience
0
14
15
@balazskegl
Balázs Kégl
7 years
My slides on #ramp workflow and kits for today's @ParisMLgroup meetup. Build your #datachallenge . Organize teamwork.
Tweet media one
0
7
15
@balazskegl
Balázs Kégl
4 years
The four reviews we got on our NeurIPS paper on model-based RL are excellent. The paper will be rejected but it's rather because of systemic issues and the lack of a clear scientific paradigm in ML than any misreading or not reading of the paper by the reviewers.
2
3
15
@balazskegl
Balázs Kégl
4 years
@vgr I built a radiator in my bedroom out of the cooling grid of an old fridge, a plastic pipe, and some corks during the 97 ice storm in Montreal. Electricity was out for a week but gas water heater worked in the bathroom.
0
2
13
@balazskegl
Balázs Kégl
2 years
@Aella_Girl I'm with David Deida on this. That desire is like the air I'm breathing, life energy that I can turn into creation. No shame to have it and no suppression. But I don't act on it because I'm monogamous and what we're building with my partner could not exist if we were poly.
1
0
13
@balazskegl
Balázs Kégl
11 months
@sentientist Think about them this way: you are their frontal lobes. They are born immature. So their behavior is a mirror of your toddler self. My worst rage is coming when they do something I was not allowed to. My inner toddler shrieks, IT'S NOT FAIR!!! Golden material to bring into
0
0
14
@balazskegl
Balázs Kégl
7 years
How to build a #datascience pipeline. Preparing my talk at #OReillyAI in the #AI @xprize @ibmwatson workshop.
0
14
14
@balazskegl
Balázs Kégl
7 years
We launched our first "Kaggle RAMP": a collaborative RAMP team to compete in a @kaggle challenge. Feel free to join!
Tweet media one
0
9
13
@balazskegl
Balázs Kégl
2 years
Entropy is ~ the log number of microscopic states that produces the same macroscopic state, right? So can we say that without postulating _levels_ (at least a micro and macroscopic level of analysis), entropy loses its grounding?
1
1
14
@balazskegl
Balázs Kégl
1 year
I'm super excited to announce my podcast: I, scientist, where we explore AI, body, and soul. My first guest is @_GPaolo whom we discuss agency in AI, reinforcement learning, and much more. Pls give it thumbs up, subscribe, comment, and share.
Tweet media one
2
2
14
@balazskegl
Balázs Kégl
6 years
Our next single-day @SaclayCDS #RAMP hackathon will be on October 10 at the INRIA Turing building on detecting Solar storms from spacecraft measurements (event detection in time series). If you would like to participate, please sign up before October 3:
0
15
13
@balazskegl
Balázs Kégl
8 years
Title page of my #nips2016 WS talk tomorrow in room 127-128 at 14h45. Title is written by a machine trained only on digits.
Tweet media one
0
3
14
@balazskegl
Balázs Kégl
7 years
The first ever scientific paper where optimization of the #datascience analysis pipeline was entirely #crowdsourced .
Tweet media one
Tweet media two
0
15
14
@balazskegl
Balázs Kégl
7 years
"Elements of causal inference" by Jonas Peters, Dominik Janzing and @bschoelkopf is out, #openaccess , at the top of my reading list.
0
1
14
@balazskegl
Balázs Kégl
8 years
Breaking news
2
25
14
@balazskegl
Balázs Kégl
2 years
@hardmaru @slashML It's painstakingly slow because there are so many moving parts and they all depend on each other. Also takes a couple of years to build a good experimental stack. Third: benchmarks are all over the place, experimental settings are not fixed.
2
0
14
@balazskegl
Balázs Kégl
6 months
Are we rushing ahead into the unknown, scaring people, while also producing fake shelters for them to retreat into artificial small worlds? Are we losing our grip on the world or realizing our humanness in the mirror of AI? Will AI take our agency away? Are we willingly giving it
1
3
14
@balazskegl
Balázs Kégl
7 years
My talk on collaborative #datachallenges and managing the #datascience process @franceisai
Tweet media one
0
6
13
@balazskegl
Balázs Kégl
4 years
Our #ICLR2021 paper on model-based RL with @albertcthomas and Gabriel Hurtado Blog: We compare generative models on Acrobot. Both multimodality and heteroscedasticity are crucial, even though the environment is noiseless. 1/n
1
3
12
@balazskegl
Balázs Kégl
7 months
We've just submitted a position paper on embodied AI. One of my realizations during writing is that social media AI is more embodied than LLMs, making them more natural to interact with, which may explain why there is less buzz about their alignment, even though their impact on
Tweet media one
Tweet media two
1
3
13
@balazskegl
Balázs Kégl
1 year
@fchollet You underestimate the reliability of the source when we judge information. Our whole brain is about judging epistemic authority. I can't check the veracity of every piece of information, it defeats the purpose of distributed intelligence.
1
0
13
@balazskegl
Balázs Kégl
2 years
@Philip_Goff Can you elaborate how theists deny suffering? My understanding would be more like they make suffering meaningful, thus bearable. Avoiding suffering is not at the top of the value hierarchy, but suffering exists.
1
0
12
@balazskegl
Balázs Kégl
1 year
@hubermanlab @lexfridman For what would he give the yearly Fridman prize and why?
0
0
13