When and how can an attention mechanism learn to selectively attend to informative tokens, thereby enabling detection of weak, rare, and sparsely located features? We address these questions theoretically in a sparse-token classification model in which positive samples embed a weak signal vector in a randomly chosen subset of tokens, whereas negative samples are pure noise. In the long-sequence limit, we show that a simple single-layer attention classifier can in principle achieve vanishing test error when the signal strength grows only logarithmically in the sequence length L, whereas linear classifiers require sqrt(L) scaling. Moving from representational power to learnability, we study training at finite in a high-dimensional regime, where sample size and embedding dimension grow proportionally. We prove that just two gradient updates suffice for the query weight vector of the attention classifier to acquire a nontrivial alignment with the hidden signal, inducing an attention map that selectively amplifies informative tokens. We further derive an exact asymptotic expression for the test error and training loss of the trained attention-based classifier, and quantify its capacity – the largest dataset size that is typically perfectly separable – thereby explaining the advantage of adaptive token selection over nonadaptive linear baselines. Joint work with Nicholas Barnfield and Yue M Lu.

Studying large scale dynamics of interacting many-particle systems is extremely demanding both analytical and numerical. Fortunately, there exist an effective asymptotic description, called hydrodynamics, based on averaging over local equilibrium states. Unfortunately, in general it is an open problem to understand its emergence, i.e. to connect the microscopic and the macroscopic description.

This motivates to study the problem of hydrodynamic emergence in simple models with exact microscopic solutions, such as the hard rods model (hard spheres in 1D). In arXiv:2408.04502, we had derived the diffusive-scale hydrodynamic equation. To our surprise the result was not of the expected Navier-Stokes type.

To investigate this, we propose a radical strategy: to study the evolution of a single deterministic configuration, instead of a statistical ensemble of them. First, I will explain how one can meaningfully define hydrodynamics in this scenario. Then, I will use the new to show that diffusion is in fact absent in hard rods and that hydrodynamics even works for “unphysical” configurations very far from local equilibrium. In addition, the method also provides error bounds on the quality of the hydrodynamic approximation.

Based on arXiv:2507.17827