There are currently two parameterizations used to derive fixed kernels corresponding to infinite width neural networks, the NTK (Neural Tangent Kernel) parameterization and the naive standard parameterization. However, the extrapolation of both of th...

Document(s)

Title

There are currently two parameterizations used to derive fixed kernels corresponding to infinite width neural networks, the NTK (Neural Tangent Kernel) parameterization and the naive standard parameterization. However, the extrapolation of both of th...

A longstanding goal in deep learning research has been to precisely characterize training and generalization. However, the often complex loss landscapes of neural networks have made a theory of learning dynamics elusive. In this work, we show that fo...

Neural Tangents is a library designed to enable research into infinite-width neural networks. It provides a high-level API for specifying complex and hierarchical neural network architectures. These networks can then be trained and evaluated either a...

We study the behavior of the Iwasawa invariants of the Iwasawa modules which appear in Kato's main conjecture without $p$-adic $L$-functions under congruences. It generalizes the work of Greenberg-Vatsal, Emerton-Pollack-Weston, B.D. Kim, Greenberg-I...

We consider a one-dimensional lattice system of unbounded, real-valued spins with arbitrary strong, quadratic, finite-range interaction. We show the equivalence of the grand canonical ensemble (gce) and the canonical ensemble (ce), in the sense of ob...

Selecting an optimizer is a central step in the contemporary deep learning pipeline. In this paper, we demonstrate the sensitivity of optimizer comparisons to the metaparameter tuning protocol. Our findings suggest that the metaparameter search space...

There is a previously identified equivalence between wide fully connected neural networks (FCNs) and Gaussian processes (GPs). This equivalence enables, for instance, test set predictions that would have resulted from a fully Bayesian, infinitely wid...