It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width....

The goal of this workshop is to try to understand the ongoing revolution in transformers and large language models (LLMs) through a wide lens (including neuroscience, physics, cognitive science, and...

In recent years, state-of-the-art methods in computer vision have utilized increasingly deep convolutional neural network architectures (CNNs), with some of the most successful models employing...

The choice of initial learning rate can have a profound effect on the performance of deep networks. We present a class of neural networks with solvable training dynamics, and confirm their...

The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset ...

Overview Modern machine learning models are often highly overparameterized. The prime examples of late are neural network architectures that can achieve state-of-the-art performance while having many...

Large language models (LLMs) have demonstrated an unprecedented ability to perform complex tasks in multiple domains, including mathematical and scientific reasoning. We demonstrate that with...

Updates (2019/07/16): Links to talk videos and slides are updated at the Schedule page

Updates (2019/07/16): Links to talk videos and slides are updated at the Schedule page

A longstanding goal in deep learning research has been to precisely characterize training and generalization. However, the often complex loss landscapes of neural networks have made a theory of...

0:00 Welcome by Rina Panigrahy11:40 Workshop's goal16:23 talk on "How to Augment Supervised Learning with Reasoning" by Leslie Valiant39:40:00 talk on "Langu...

Presented by Yasaman Bahri, Google BrainFull Title: "Understanding Deep Learning: Theoretical Building Blocks From The Study of Wide Networks"Talk Descriptio...

Main - Why SEDL? - Call for Papers - Organizers

It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width....

There are two cultures in the use of statistical modeling to reach conclusions from data. One assumes that the data are generated by a given stochastic data model. The other uses algorithmic models...

Similar to humans and animals, deep artificial neural networks exhibit critical periods during which a temporary stimulus deficit can impair the development of a skill. The extent of the...

AI is transforming our relationships with technology and with others, our senses of self, as well as our approaches to health care, banking, democracy, and the courts. But while AI in its many forms...

The story of how solid-state physics emerged in the postwar period and was eventually rebranded as condensed-matter physics illuminates some major shifts in the

This review covers several topics involving renormalization group ideas. The solution of the $s$-wave Kondo Hamiltonian, describing a single magnetic impurity in a nonmagnetic metal, is explained in...

The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset or the number of parameters in the network. We...

Deep convolutional networks have witnessed unprecedented success in various machine learning applications. Formal understanding on what makes these networks so successful is gradually unfolding,...

A central challenge in developing versatile machine learning systems is catastrophic forgetting: a model trained on tasks in sequence will suffer significant performance drops on earlier tasks....

The choice of initial learning rate can have a profound effect on the performance of deep networks. We present a class of neural networks with solvable training dynamics, and confirm their...

At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evolution of an ANN...