Pikagi
Tim Duignan Profile Banner
Tim Duignan Profile
Tim Duignan

@TimothyDuignan

5,097
Followers
1,937
Following
199
Media
3,276
Statuses

Modelling and simulation of electrolyte solutions using quantum chemistry, stat mech and neural network potentials #compchem #theochem

Brisbane, Queensland
Joined February 2013
Don't wanna be here? Send us removal request.
Pinned Tweet
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This is the most surprising and exciting result of my career: we were running simulations of NaCl with a neural network potential that implicitly accounts for the effect of the water, ie a continuum solvent model (trained on normal MD) when Junji noticed something strange: 1/n
130
741
5K
Last Seen Profiles
Poyraz

@Berkbayda

uta

@utatsumugi1

BustyTribes

@BustyTribes

Massage Onie ๐Ÿ‡บ๐Ÿ‡ธ

@NoQuarterGivin1

ูุญู„ ุชุนุฒูŠ

@AlymnWadua62091

๐Ÿฆ‹๐“ˆ’๐“‚‚๐“ธ

@kasumi_nagi1101

ู…ู†ู‚ุจู‡ ุฌู…ูŠู„ู‡ุŒ๐Ÿ‡ฑ๐Ÿ‡ง๐Ÿ’ฏโŒ๐Ÿ’ฏ

@MnqbhJmylh

ู‡ู„ุงู„ูŠ ูˆุงู‚ุนูŠ

@gKvfC62yojqF4Kr

ู‡ู„ุงู„ ุงู„ู…ุฎูŠู†ูŠ

@abunoor8802

Perry Roth

@plroth

Woman, innit

@ahamrick

168 เธŠเธฒเนเธ™เธฅ

@sora_kub

eda yฤฑlmazer

@lipatilia

Reading Horror also @weirdfictionfan.bsky.social

@SnixelmixN22624

Vakaviti

@vakaviti

sissi & asiri siskoluktan ucamayan kus

@icarusvari

Ramskalyan

@Ramskalyan

Roch Ttm

@RochTtm82284

StanleyBurtonCentre

@UoLSBC

๐Ÿป

@peludo_gordito

Suzu ใ€ๆดพ้ฃๅฅณๅƒ•ใ€‘

@hakenmaid_jp

Center for Youth Political Participation

@RutgersCYPP

Bello Yusuf Olatunji

@BOlatunj

Kaveh Ghoreishi

@KavehGhoreishi

EGBB ๐Ÿ’™๐Ÿ€๐Ÿ’š

@Eagangirlsbball

Aguayitx๐Ÿ‡ต๐Ÿ‡ธ

@eatmyplanet

awkwardsnake

@kvnwayne

ู†ูˆุงู„ ุงู„ุญู„ูˆู‡

@AmFysl81266

ๅŠ ่—คๅคๅญ

@natsuko_kato111

andii

@andiipoops

New๐Ÿ‹

@Newsiito

Salva from Tunco Life ๐ŸŒด

@essalvatore

Tika dee5

@TikaDee5

Emine Yฤฑlmaz

@EmineYilmazee

Nirnay Dhale

@dhale_nirnay

The Hashslinging Slasher

@HashslingingThe

@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
A crystal was nucleating and growing! despite the NNP only being trained on solution data (4 M). Amazingly it has the correct crystal structure, i.e., FCC. This is a phase transitions, an emergent phenomena, totally out of distribution, supposedly where AI is no good.
12
50
1K
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Impressively this is actually more correct than the all-atom MD as the true solubility of this model is actually very low, all atom MD is just too slow to see it. I think this pretty conclusively disproves the idea that NNPs are only interpolating on the data they are given.
20
23
728
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
I think neural network potentials are the most important scientific tool of the next decade. The ability to simulate systems at the molecular scale starting from nothing but quantum mechanics will be transformative for a vast range of problems throughout biology and chemistry 1/n
@JPhysChem
The Journal of Physical Chemistry
@JPhysChem
7 months
The Potential of #Neural Network Potentials A perspective from Timothy Duignan @TimothyDuignan @Griffith_Uni ๐Ÿ”“ Open access in ACS Physical Chemistry Au ๐Ÿ‘‰
Tweet media one
1
34
191
12
101
666
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
Here's another pretty incredible example of neural network potentials extrapolating outside their training data in a way I wouldn't expect. We were simulating an electrolyte with an NNP when this happened:
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This is the most surprising and exciting result of my career: we were running simulations of NaCl with a neural network potential that implicitly accounts for the effect of the water, ie a continuum solvent model (trained on normal MD) when Junji noticed something strange: 1/n
130
741
5K
23
76
645
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
I have always assumed it would be impossible to study crystal nucleation etc with a continuum solvent model. Here it just dropped out for free. They're incredibly cheap to run too. ie. tens of cpu hours. More info on continuum solvent models here:
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
I want to explain a statistical mechanical concept known as coarse graining which I think might be useful for thinking about things like AF3. Especially a special case known as continuum or implicit solvent models.
7
44
363
4
19
590
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
The lattice parameter is a bit high compared to experiment, so we took a KCl coarse grained force field trained on quantum chemical solution MD data and showed that it could simulate crystal dissolution. It maintains the correct lattice spacing and appears to dissolve in the same
6
19
547
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
So the race is really heating up to build a truly universal force field. This is one of those powerful ideas that people in the field of molecular simulation have been dreaming about for decades. What exactly is it and how far away are we? 1/n
Tweet media one
20
110
482
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
You know a field is taking off when you regularly see multiple order of magnitude improvements in performance. Thats whats happening in deep learning for molecular simulation. This field is going to be transformed. Latest example: 1/n
Tweet media one
3
48
472
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Ok so what is a neural network potential concretely? It's just a very flexible function with many adjustable parameters that you fit to the 'potential energy surface.' This is just the energy as function of the position of the atoms in your system. 1/n
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
I think neural network potentials are the most important scientific tool of the next decade. The ability to simulate systems at the molecular scale starting from nothing but quantum mechanics will be transformative for a vast range of problems throughout biology and chemistry 1/n
12
101
666
7
52
403
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Check out the preprint here: Code etc.: Videos: and
Tweet card media
NaCl crystallisation

Video of the nucleation and crystal growth of NaCl simulated in water with a coarse-grained continuum solvent model trained on all atom classical molecular d...

www.youtube.com
5
38
398
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
We also simulated LiCl,LiBr and KCl at quantum chemical accuracy and reproduced the ion specific pairing affinities and matching experimental activity coefficient derivatives. Activity coefficients are the most important property of electrolyte solutions, they are ubiquitous
Tweet media one
Tweet media two
3
13
382
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
I want to explain a statistical mechanical concept known as coarse graining which I think might be useful for thinking about things like AF3. Especially a special case known as continuum or implicit solvent models.
@demishassabis
Demis Hassabis
@demishassabis
5 months
Thrilled to announce AlphaFold 3 which can predict the structures and interactions of nearly all of lifeโ€™s molecules with state-of-the-art accuracy including proteins, DNA and RNA. Biology is a complex dynamical system so modeling interactions is crucial
Tweet media one
204
2K
8K
7
44
363
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
If you want to know more about how NNPs and why they're so exciting check out this thread and paper:
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Ok so what is a neural network potential concretely? It's just a very flexible function with many adjustable parameters that you fit to the 'potential energy surface.' This is just the energy as function of the position of the atoms in your system. 1/n
Tweet media one
7
52
403
14
24
362
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
So isnโ€™t it strange that diffusion models and the human brain are the two best methods for conjuring up images out of nothing and they both happen to operate on the same fundamental algorithm ie molecular dynamics? How many ways of conjuring up images do we think the universe
26
24
356
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
It seems we're learning that deep learning is mostly about the data. If you want to know where it will really take off look to areas where you can continuously generate increasingly diverse but consistently high quality data. That leads you to quantum chemistry:
Tweet media one
Tweet media two
5
40
298
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Ok so the new AlphaFold model relies in large part on a "relatively standard diffusion approach" turns out you can think of this as just a special case of a neural network potential, it just uses experimental data not quantum chemistry to train on. 1/n
@demishassabis
Demis Hassabis
@demishassabis
5 months
Thrilled to announce AlphaFold 3 which can predict the structures and interactions of nearly all of lifeโ€™s molecules with state-of-the-art accuracy including proteins, DNA and RNA. Biology is a complex dynamical system so modeling interactions is crucial
Tweet media one
204
2K
8K
6
42
299
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
We also showed you can extract infinite dilution pairing free energies and diffusivities and get almost perfect radial distribution functions with the continuum solvent model and more.
1
4
231
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
So what are the most exciting potential applications of a universal forcefield? Weโ€™re already starting to see one emerge that could result in a more general and more useful form of alphafold3. 1/n
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
So the race is really heating up to build a truly universal force field. This is one of those powerful ideas that people in the field of molecular simulation have been dreaming about for decades. What exactly is it and how far away are we? 1/n
Tweet media one
20
110
482
3
30
225
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Another very impressive general purpose graph NN for molecular simulation. Feels like this field is really accelerating.
Tweet media one
4
39
211
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Wow this blows me away you can get roughly correct structure of pure electrolyte solutions out of this.
Tweet media one
Tweet media two
@demishassabis
Demis Hassabis
@demishassabis
5 months
Thrilled to announce AlphaFold 3 which can predict the structures and interactions of nearly all of lifeโ€™s molecules with state-of-the-art accuracy including proteins, DNA and RNA. Biology is a complex dynamical system so modeling interactions is crucial
Tweet media one
204
2K
8K
10
27
197
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
This is one of those beautiful ideas that took me a long time to see, but in retrospect seems obvious. If correct, it implies a massive step up in the number of problems molecular simulation can be fruitfully applied to.
Tweet media one
Tweet media two
5
20
194
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Wow this is incredibly motivating and rewarding to see how many people are excited about salt crystallisation and the potential for AI to accelerate basic science! Thank you so much! Some people asked about why this is an important topic to study? 1/n
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This is the most surprising and exciting result of my career: we were running simulations of NaCl with a neural network potential that implicitly accounts for the effect of the water, ie a continuum solvent model (trained on normal MD) when Junji noticed something strange: 1/n
130
741
5K
3
14
172
@TimothyDuignan
Tim Duignan
@TimothyDuignan
7 months
Clearest explanation of this beautiful idea that I've found so far.
Tweet media one
2
22
169
@TimothyDuignan
Tim Duignan
@TimothyDuignan
7 months
Tried to capture why Iโ€™m so excited about NNPs. I honestly think they are a profoundly useful tool.
@JPhysChem
The Journal of Physical Chemistry
@JPhysChem
7 months
The Potential of #Neural Network Potentials A perspective from Timothy Duignan @TimothyDuignan @Griffith_Uni ๐Ÿ”“ Open access in ACS Physical Chemistry Au ๐Ÿ‘‰
Tweet media one
1
34
191
8
17
161
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
More fascinating connections between statistical mechanics/phase transitions and the surprising ability of neural networks to generalise.
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 year
I think this is a profound paper โ€ฆ This is what โ€˜grokkingโ€™ is right? A sharp jump downward in energy/loss? Itโ€™s just a phase transition right? Stat. mech. must have the tools to explain the success of deep neural networks.
Tweet media one
3
7
59
3
18
150
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This paper nicely demonstrates the point I was making about AF3. It can be interpreted as a forcefield as it is learning the gradient of the log probabilities ie the score, but log probabilities are just free energies. So itโ€™s learning an approximation to the true free energy.
Tweet media one
@biorxiv_biophys
bioRxiv Biophysics
@biorxiv_biophys
5 months
AlphaFold3, a secret sauce for predicting mutational effects on protein-protein interactions #biorxiv_biophys
0
21
83
3
10
128
@TimothyDuignan
Tim Duignan
@TimothyDuignan
9 months
Is there a machine/deep learning textbook anywhere that teaches the Boltzmann/Gibbs distribution? Is it called something else? Have looked at three so far with no mention of it. It is the entropy maximising distribution! Surely it is important to know?
Tweet media one
26
16
123
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Couldnโ€™t agree more with this excellent piece by @gdefabritiis A pioneer in this field. These tools are already transformatively useful for molecular simulation and will therefore be important for many closely related fields. There are improvements that still need to be made
Tweet media one
@gklambauer
Gรผnter Klambauer
@gklambauer
2 months
Machine Learning Potentials: A Roadmap Toward Next-Generation Biomolecular Simulations Neural network potentials are very powerful, but they appear too slow to be ever used for macromolecules.. this work provides a possible roadmap... P:
Tweet media one
4
38
134
1
25
124
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Interesting how deep learning for generating equilibrium distributions seems to be converging back to molecular dynamics. Like this is just langevin dynamics with a learnt score. So just NNP-MD with many runs in parallel right? Or am I missing something?
Tweet media one
@PomaResearch
Adolfo Poma
@PomaResearch
5 months
Predicting equilibrium conformations in protein via #ML , what else soon, to predict non-equilibrium distributions. Thoughts?
2
10
67
8
11
120
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Wow Microsoft patented the coarse grained diffusion model work. Anyone know of any precedents/thoughts on what this means? So they can stop anyone from using this approach?
Tweet media one
15
21
115
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
Very exciting. Another excellent example of the power of combining quantum chemistry, statistical mechanics and neural network potentials. This is just the beginning.
@PaesaniLab
Paesani Lab
@PaesaniLab
4 months
Ever wondered if we could model #water to autoionize and correctly predict pH = 7? ๐Ÿค” Well, now we can! ๐Ÿ˜Ž In our latest @ChemRxiv preprint, we introduce a #deepneuralnetwork potential trained on density-corrected #DFT that predicts the autoionization constant of water to be Kw
Tweet media one
4
18
124
2
10
91
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 years
@LakeBrenden @jeremyphoward ChatGPTโ€™s response is lot better. The last line is a little suspicious though.
Tweet media one
5
1
89
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
Another nice recent papers showing the power of combining MD with diffusion models. This one from @therealpeterobi where they show you can use it to make umbrella sampling more efficient.
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
You know a field is taking off when you regularly see multiple order of magnitude improvements in performance. Thats whats happening in deep learning for molecular simulation. This field is going to be transformed. Latest example: 1/n
Tweet media one
3
48
472
4
13
89
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Yeah seems backward to me how much compute is spent using exact calculations to generate Boltzmann distributions, ie langevin dynamics for diffusion models/MD simulations when nature will give you one for free if you just provide it with an energy landscape and let the atoms move
@francoisfleuret
Franรงois Fleuret
@francoisfleuret
2 months
Saturday morning [not very original] thought: Given the incredible noise-resistance of deep models, it is extremely bizarre that specialized deep learning chips stick to the traditional "perfectly exact" computing paradigm, given the complexity and cost it involves.
28
8
161
5
7
86
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 year
So pleased to get this preprint out. Feel like weโ€™ve finally worked out how to do something Iโ€™ve been trying to do for 13 years since the start of my PhD: Build an accurate continuum solvent model of ion-ion interactions in solution.
Tweet card media
Scalable molecular simulation of electrolyte solutions with...

Unleashing the predictive power of molecular dynamics (MD), Neural Network Potentials (NNPs) trained on Density Functional Theory (DFT) calculations are revolutionizing our ability to simulate...

arxiv.org
5
11
82
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 year
I want to record a prediction: ML acceleration of molecular simulation will transform all of physical science. From quantum scale all the way up to climate. Justification: 1/n
@DaniloJRezende
Danilo J. Rezende
@DaniloJRezende
1 year
Yes, this is the ultimate way ML will help accelerate physical sciences. By constructing custom MCMC operators (eg proposal distributions) to accelerate traditional MD/MCMC simulations in combination with existing tools. This can be done while preserving all error bars.
3
10
118
3
3
80
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
This is self-ionisation! Water splitting apart into hydronium and hydroxide ions. Amazingly it somehow knows to put the hydronium in the correct pyramidal structure. There are no examples of this in the training data, yet it remains stable and then something even cooler happened:
10
1
79
@TimothyDuignan
Tim Duignan
@TimothyDuignan
11 days
Another very interesting NNP for electrolyte solutions paper. People have spent decades trying to build accurate classical force fields of calcium carbonate one of the most fundamental substances in existence. Turns out even very sophiticated polarisable models overestimate
Tweet media one
2
12
79
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Exactly, for example quantum chemistry simulations and ML go together like hand and glove. One gives accuracy and reliability, the other speed and scale.
@ylecun
Yann LeCun
@ylecun
2 months
Data generation through simulation is one reason why the whole idea of ML for science is so exciting.
81
164
1K
3
6
76
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
This is the Grotthuss mechanism! Chains of proton hopping events give rise to a much higher diffusivity of acid in water. The NNP has rediscovered that this is a plausible mechanism of ion transport, without even being asked to. Granted the rates of these processes are too high
4
0
71
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Interesting discussion. The idea that diffusion models are going to replace MD for generating probability distributions pops up again though. I do not get this: diffusion models are MD ie langevin dynamics on a free energy surface. If anything deep learning got replaced with MD.
@labenz
Nathan Labenz
@labenz
5 months
Even highly educated / plugged-in people, myself included, know very little about the latest biotechnology I hope this episode can help change that, because things are about to get crazy! Full episode is here:
3
7
47
6
9
71
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
One day (I have no idea when!) I think weโ€™ll simulate this whole thing from first principles. Is that crazy or obvious?
@DeryaTR_
Derya Unutmaz, MD
@DeryaTR_
5 months
This is just an amazing photo of a neuron! โ€œA single neuron is shown with 5,600 of the nerve fibers (blue) that connect to it. The synapses that make these connections are in green.โ€ Credit: Google Research & Lichtman Lab, Harvard University. Renderings by D. Berger, Harvard
Tweet media one
6
39
229
25
4
70
@TimothyDuignan
Tim Duignan
@TimothyDuignan
3 months
Interesting comment in a fascinating piece from @RuxandraTeslo . I agree with the essay overall. But I often see this idea expressed about biology: itโ€™s impossible to understand from first principles. I think itโ€™s become so ingrained no one questions it. But I think we should. I
Tweet media one
13
11
69
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 month
Very nice. This is the way to do it. Combine diffusion to give you the stability far from equilibrium with forces to give you the accuracy near equilibrium.
Tweet media one
@MarkNeumannnn
Mark Neumann
@MarkNeumannnn
2 months
Super excited to release a set of models for computational chemistry - my last 2 years of work @OrbMaterials . Post ELMo, I'm at some risk of becoming a "one trick pony" career wise, but we've managed to make pre-training work nicely for 3d crystal structures. More below!
1
16
69
0
5
69
@TimothyDuignan
Tim Duignan
@TimothyDuignan
11 days
And the Nobel prize commitee agrees
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
11 days
Another very interesting NNP for electrolyte solutions paper. People have spent decades trying to build accurate classical force fields of calcium carbonate one of the most fundamental substances in existence. Turns out even very sophiticated polarisable models overestimate
Tweet media one
2
12
79
1
8
66
@TimothyDuignan
Tim Duignan
@TimothyDuignan
8 months
Amazing how far ahead Behler and Parrinello were. There will be a Nobel prize for this paper one day imo:
Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces

The accurate description of chemical processes often requires the use of computationally demanding methods like density-functional theory (DFT), making long simulations of large systems unfeasible....

journals.aps.org
3
5
65
@TimothyDuignan
Tim Duignan
@TimothyDuignan
3 months
Exactly this is why Iโ€™m so excited about neural network potentials. They are the only approach where you can computationally generate your own high quality, targeted training data on the fly in an automated fashion.
@AllThingsApx
Kyle Tretina, Ph.D.
@AllThingsApx
3 months
*AI for BioChem is data-starved.* A theme of every session @icmlconf was that they were all trying to deal with this fact. e.g. improving model efficiency, data preparations from public sources and using synthetic data where possible
8
11
75
4
5
61
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 year
I think this is a profound paper โ€ฆ This is what โ€˜grokkingโ€™ is right? A sharp jump downward in energy/loss? Itโ€™s just a phase transition right? Stat. mech. must have the tools to explain the success of deep neural networks.
Tweet media one
3
7
59
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
Was thrilled to appear on Cognitive Revolution, one of my favourite podcasts, was a great discussion. Neural network potentials are set to significantly impact many areas of science and engineering imo. This is just the beginning.
Tweet card media
Fluid Intelligence: Simulating Solutions with Tim Duignan

Explore the fusion of AI and computational chemistry with University of Queensland's researcher, Tim Duignan. Watch Episode Here Read Episode Description Explore the fusion of AI and computational...

www.cognitiverevolution.ai
2
6
58
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Many of the most important scientific advances have followed a simple recipe: Adopt a tool developed in another field to your own. The most beautiful demonstration of this is actually diffusion models which have done this three times already! Firstly tools and ideas from
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Applying new tools from other fields is key.
Tweet media one
0
4
15
3
10
57
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
@carnot_cyclist Diffusion models use langevin dynamics which was an algorithm invented to simulate the behavior of molecules with time. The score is just a time dependent forcefield.
2
1
57
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
No great question. First task is to provide more accurate data for large scale models of chemical processes ie activities, diffusivities, reaction rates etc. This can be done with homogeneous single phase simulations. Second task is to provide direct physical insight into the
@erwkuhn
Erwin Kuhn
@erwkuhn
4 months
@TimothyDuignan Amateur question: what's the end state for ML-enabled molecular dynamics simulation? Is it to replace current activity coefficient models, achieve much more accurate simulations, or both?
1
2
7
3
7
56
@TimothyDuignan
Tim Duignan
@TimothyDuignan
9 months
Quantum computing experts claim computing properties of Femoco is impossible with classical computing and if you could do it you could revolutionize fertilizer synthesis. Turns out you can do it with DFT fine but almost no one cares.
Tweet media one
6
2
56
@TimothyDuignan
Tim Duignan
@TimothyDuignan
3 months
Awesome to drop by Rowan on my way back from the Gordon water conference in Holderness. Such an awesome tool theyโ€™re building canโ€™t wait to see what they do next!
@RowanSci
Rowan
@RowanSci
3 months
It was fantastic to host @TimothyDuignan today for our third Rowan Seminar and hear about how neural network potentials are revolutionizing electrolyte simulation and ab initio MD!
Tweet media one
0
1
20
1
5
54
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Ok for all the haters out there I managed to break it which was what I was originally trying to do/expecting. Here's NaCl. ๐Ÿคฃ
Tweet media one
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Wow this blows me away you can get roughly correct structure of pure electrolyte solutions out of this.
Tweet media one
Tweet media two
10
27
197
7
5
51
@TimothyDuignan
Tim Duignan
@TimothyDuignan
8 months
Love the flow of ideas back and forth between molecular simulation and deep learning. Diffusion models originally inspired by molecular dynamics algorithms (langevin dynamics) now inspiring new approaches to accelerate MD.
@HannesStaerk
Hannes Stรคrk
@HannesStaerk
8 months
Oke, the AlphaFlow paper is awesome: AlphaFold Meets Flow Matching for Generating Protein Ensembles Just watch how AlphaFlow's ensemble reproduces details of MD. Weights + code We have it in the reading group on Mon 11am EST! 1/2
15
153
763
2
10
50
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Ha cool. We found this back in 2021
Tweet media one
@duncan__c
Duncan S. Campbell
@duncan__c
2 months
Holy shit, is this real? Stanford, MIT, and Toyota found a 50% lithium-ion battery cycle life improvement simply by changing the power level of its first charge after manufacturing?
32
129
857
0
5
51
@TimothyDuignan
Tim Duignan
@TimothyDuignan
3 months
This is a compelling argument for investing in neural network potentials which donโ€™t suffer from the same data limitations. The problem with trying to use โ€˜wet lab innovationsโ€™ is that techniques for obtaining simultaneous femtosecond/picometer scale resolution experimental data
@owl_poster
owl (in sf 10/18 to 10/23)
@owl_poster
3 months
Wet-lab innovations will lead the AI revolution in biology i feel like i keep repeating this argument to people so i decided to just write it out 1.9k words, 9 minutes reading time, very short!
16
98
478
2
3
49
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Great piece and excellent point:
Tweet media one
@mackenziejem
Mackenzie Morehead
@mackenziejem
2 months
New post on the near frontier of Neural Net Potentials and the implication for company building and industry competitive dynamics These models are smaller than GPT1. Excited to see them scale Thoughts on: * role in computational stack * materials science vs drug discovery *
Tweet media one
4
9
58
1
7
46
@TimothyDuignan
Tim Duignan
@TimothyDuignan
8 months
Notice how similar to MD this is conceptually. It is actually mathematically essentially the same also. The only difference is the force field is learnt from the PDB where you know the forces are 0 because they are equilibrium states. Really its an implicit solvent force field.
@woodyahern
Woody Ahern
@woodyahern
1 year
RFdiffusionAA generating a small molecule binding protein against an experimental FXIa inhibitor (OQO), a ligand which is significantly different than any in its training dataset.
1
62
243
4
5
44
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Big tech companies are getting in on this now. Google has GNoME
Tweet card media
Scaling deep learning for materials discovery

Nature - A protocol using large-scale training of graph networks enables high-throughput discovery of novel stable structures and led to the identification of 2.2 million crystal structures, of...

www.nature.com
1
4
43
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
This is a beautiful paper from Aleksander Durumeric, Yaoyi Chen, @FrankNoeBerlin , @CecClementi where they combine denoising with forces to train a coarse grained neural network potential. This is an idea we are playing with too. It nicely demonstrates the deep connection between
Tweet media one
1
5
43
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
And then MACE-MP-0 which can simulate across multiple phases of matter despite being only trained on crystals.
Tweet card media
A foundation model for atomistic materials chemistry

Machine-learned force fields have transformed the atomistic modelling of materials by enabling simulations of ab initio quality on unprecedented time and length scales. However, they are currently...

arxiv.org
1
7
41
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
But thereโ€™s a new way people are starting to get very excited about using neural network potentials. Around 2017/2018 we saw things like ANI-1 and Tensormol-0.1 which could do this for a 4 atom types for a range of structures.
Tweet media one
Tweet media two
1
3
40
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Some people are not impressed by this. Maybe im just incompetent but I spent literally years trying to build continuum solvent models of this exact thing and couldnโ€™t do much better itโ€™s really hard to model without explicit water! ๐Ÿคฃ
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Wow this blows me away you can get roughly correct structure of pure electrolyte solutions out of this.
Tweet media one
Tweet media two
10
27
197
4
0
41
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
Excellent piece. Completely agree we should be trying to build scientific foundation models. I suspect we might need to new organizational structures to develop them though as they need large groups of dedicated full time experts which academia is not great for.
@kasratweets
kasra
@kasratweets
4 months
can we build GPT-4 but for scientific data? I wrote a primer on foundation models applied to science, featuring exciting recent work by @TimothyDuignan , @MilesCranmer , and @PolymathicAI
3
5
63
2
4
41
@TimothyDuignan
Tim Duignan
@TimothyDuignan
7 months
We see exactly the same thing for simple electrolytes. If you cannot get sodium chloride pairing free energy right you are not going to get protein folding right. I often donโ€™t point this out because I donโ€™t want to offend senior researchers.
Tweet media one
@JCIM_JCTC
JCIM & JCTC Journals
@JCIM_JCTC
7 months
Take a look at this #OpenAccess paper ๐Ÿ“ from the latest issue of Journal of Chemical Theory and Computation #JCTC ๐Ÿ”Ž The Role of Force Fields and Water Models in Protein Folding and Unfolding Dynamics ๐Ÿ’ฆ๐Ÿ”ฌ ๐Ÿ”“ #thermodynamics
Tweet media one
0
7
20
6
3
37
@TimothyDuignan
Tim Duignan
@TimothyDuignan
8 days
Well put.
Tweet media one
3
4
40
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
This is the future of generative AI for chemistry. It will merge with molecular dynamics as at the core they are doing the same thing. And there is only so much information you can get from minimum energy structures. The PDB is tapped out surely?
1
2
39
@TimothyDuignan
Tim Duignan
@TimothyDuignan
7 days
Tweet media one
1
2
38
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Good take as always. I donโ€™t think this axis makes sense really though. I would argue a diffusion model is more physics based than a lennard jones forcefield. Harmonic approximation about the minima is in every physics text book but Iโ€™ve never seen a 1/r^12 repulsion.
Tweet media one
@biogerontology
Alex Zhavoronkov, PhD (aka Aleksandrs Zavoronkovs)
@biogerontology
5 months
2
1
7
5
4
38
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 year
This is a beautiful clear explanation of diffusion models. The cool thing is they are actually really easy to understand if you know molecular simulation. There is a direct analog for almost every concept. 1/n
@sedielem
Sander Dieleman
@sedielem
1 year
New blog post about the geometry of diffusion guidance: This complements my previous blog post on the topic of guidance, but it has a lot of diagrams which I was too lazy to draw back then! Guest-starring Bundle, the cutest bunny in ML ๐Ÿ‡
9
77
352
1
5
38
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Neural network potentials can make these predictions thousands of times faster than possible with direct solution of Schrรถdinger equation.
Tweet media one
2
2
37
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This rapid progress is incredibly exciting to me and indicates that we may be approaching a point where we might have a tool we could reasonably call a true universal force field. There are still many challenges though but also lots of new ideas to try:
Tweet card media
Overcoming systematic softening in universal machine learning...

Machine learning interatomic potentials (MLIPs) have introduced a new paradigm for atomic simulations. Recent advancements have seen the emergence of universal MLIPs (uMLIPs) that are pre-trained...

arxiv.org
1
5
35
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Fascinating thread. Highlights exactly why PDB structures alone can only get you so far. So much is determined by dynamics. The central question is how to get data on the dynamics processes so you can train on it.
@philipcball
Philip Ball
@philipcball
5 months
Hereโ€™s a paper that will get zero press because it looks totally specialist, not to say obscure. Itโ€™s about how an important class of transcription factors regulate genes. But I think it's worth dissecting because it raises a wider question.Bear with me...
14
63
337
2
2
35
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Just imagine one day we will be able to go to a website like this and run accurate dynamics on any system of atoms we want. This will transform all of science and society. We will finally be masters of the molecular scale.
1
2
33
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Things are accelerating now though using more general training data sets like with: M3GNet, and CHGNet
Tweet card media
CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling

Nature Machine Intelligence - An outstanding challenge in materials science is doing large-scale simulations with complex electron interactions. Deng and colleagues introduce a universal graph...

www.nature.com
1
4
33
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
@Dr_Gingerballs Great questions. I didn't expect it because the continuum solvent free energy is highly concentration depedendent so I would not expect it to generalise to stable crystals. In some sense it is more accurate than the MD as it predicts the correct global minima (crystal) whereas
2
2
35
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 year
Awesome paper. Shows how we can train on many different levels of theory simultaneously will be very important as we make DFT databases bigger and bigger. We need to build a PDB equivalent but for quantum chemistry.
1
0
35
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Yeah this is very consistent with what I generally see. The NNPs are now so good that the error is determined by the accuracy of the underlying DFT. This means getting improved performance may just mean better fitting to the noise in the underlying data.
@CorinWagen
Corin Wagen
@CorinWagen
2 months
interesting new work from @johnkitchin & co-workers studying errors in the (now-ubiquitous) OC20 dataset - they argue that NNP errors are approaching 0.2 eV because that's the intrinsic error of the dataset.
Tweet media one
1
3
41
1
5
34
@TimothyDuignan
Tim Duignan
@TimothyDuignan
11 months
Another nice ion pairing paper on NaCl with NNPs. Look at the spread on those classical force fields in comparison! This is the fundamental medium in which all of biology occurs and we haven't been able to predict even its most basic properties until now!
Tweet media one
1
6
35
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
And most recently Microsoft announced one:
MatterSim: A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures - Microsoft...

Accurate and fast prediction of materials properties is central to the digital transformation of materials design. However, the vast design space and diverse operating conditions pose significant...

www.microsoft.com
1
6
33
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Ok Iโ€™ve been trying to get people excited about neural network potentials aka machine learning force fields. But what are the key challenges for using this tool to try and build the universal force field?: 1/n
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
So the race is really heating up to build a truly universal force field. This is one of those powerful ideas that people in the field of molecular simulation have been dreaming about for decades. What exactly is it and how far away are we? 1/n
Tweet media one
20
110
482
1
6
33
@TimothyDuignan
Tim Duignan
@TimothyDuignan
6 months
So byte dance have entered the universal machine learned force field race with a very impressive paper starting with the right problem imo: liquid electrolytes. I think this could be a critically important technology.
Tweet media one
1
7
34
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
NNPs are a particular class of machine learning potential. There are many types but a particularly important recent breakthrough has been equivariance which greatly reduced the amount of training data needed. (Alphafold2 used this idea)
1
2
32
@TimothyDuignan
Tim Duignan
@TimothyDuignan
3 months
Excellent piece by Scott Aaronson.
Tweet media one
2
2
34
@TimothyDuignan
Tim Duignan
@TimothyDuignan
2 months
Very nice explicit formulation of self attention as minimizing an energy function, something I alluded to in my perspectives article. So LLMs and diffusion models are essentially just NNPs?
Tweet media one
@vasud3vshyam
Vasu Shyam
@vasud3vshyam
2 months
Ever looked at the attention operation and said "hang on, that's a one-point function!"?
Tweet media one
19
129
1K
1
8
33
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This can be done to well below thermal noise for many small systems today but the problem is it's slow. Here NNPs come to the rescue, they predict E from the positions of all the atoms much faster. We train them on examples of Schrodingers equation we have solved previously.
Tweet media one
2
3
31
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
People have been trying to build this for a long time with point charges and Lennard Jones potentials. That approach has built some very useful tools but unfortunately in general you still have to do a lot of case by case fiddling and testing of parameters to get these working.
Tweet media one
2
2
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
The idea is that for any set of atoms in a given position there is a potential energy. You can get this by solving Schrรถdinger equation. If you could calculate this energy and its gradient (-force) accurately and fast enough it would be transformatively useful.
Tweet media one
1
3
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This is really important as the energy is so fundamental. It determines the probability of observing particles in a given arrangement and its gradient is the negative of the force so it can tell you how the particles move too.
1
2
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
8 months
Very cool. Diffusion models use a molecular simulation algorithm (thermally annealed langevin dynamics) so of course you see phase transitions directly analogous to the sudden changes that occur when you cool/heat a system of molecules, i.e., crystallization.
@StatMLPapers
Stat.ML Papers
@StatMLPapers
8 months
A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data
1
32
151
2
2
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
8 months
So cool! I assume this is the same thing that goes on at phase transition boundaries in stat mech: โ€˜Schramm-Loewner curves appear as domain boundaries between phases at second-order critical points like the critical Ising modelโ€™
@jaschasd
Jascha Sohl-Dickstein
@jaschasd
8 months
Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.
278
2K
10K
2
4
31
@TimothyDuignan
Tim Duignan
@TimothyDuignan
1 month
Nice. Rowan is the perfect tool for someone looking to pick up quantum chemistry I reckon, not just for experts, particularly with the excellent visualisations.
@AriWagen
Ari Wagen
@AriWagen
1 month
how'd i end up with hundreds of gifs in my downloads? funny you ask ... (video is of an xtb optimization i was struggling with, looking at this paper from @BroereDaniel on dicobalt complex spin states: )
2
1
11
2
3
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This is because you could run simulations to directly see what atoms are doing at the molecular scale for important systems and sample from the equilibrium probability distribution (Boltzmann distribution) to calculate important experimental quantities.
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
So at equilibrium the probability of observing a given configuration of atoms is given by the Boltzmann distribution:
Tweet media one
1
0
14
1
3
29
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
This means that Google's claim that they have "surpassed physics based tools" is kind of strange. In fact there is a ton of physics baked into it how diffusion models work!
Tweet media one
1
4
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
5 months
Caveat: The reality, as always, is obviously significantly more complicated then I've presented eg excited states, non adiabatic and quantum nuclear effects etc. Which will need to be included in various cases using additional tools.
4
2
29
@TimothyDuignan
Tim Duignan
@TimothyDuignan
11 months
Check out this really nice collaboration with @alisterpage and two awesome students where we show you can resample from DFTB MD, compute forces at a higher level of theory and run stable MD with equivariant neural network potentials.
Tweet card media
High-Throughput Aqueous Electrolyte Structure Prediction Using IonSolvR and Equivariant Graph...

Neural network potentials have recently emerged as an efficient and accurate tool for accelerating ab initio molecular dynamics (AIMD) in order to simulate complex condensed phases such as electrol...

pubs.acs.org
1
5
30
@TimothyDuignan
Tim Duignan
@TimothyDuignan
4 months
Link:
Tweet card media
Learning data efficient coarse-grained molecular dynamics from...

Machine-learned coarse-grained (MLCG) molecular dynamics is a promising option for modeling biomolecules. However, MLCG models currently require large amounts of data from reference atomistic...

arxiv.org
1
5
29