Latent Lattice Language Models

Administrator 0 阅读

AI Digest - ArXiv AI

Latent Lattice Language Models

Language models typically increase per-token computation through deeper or looped layers and latent reasoning steps, while richer residual connections improve information flow across layers. We instead increase computational capacity: the number of active computational states a model maintains. We introduce the Lattice of Latents, a set of per-layer horizontal latent streams that run recurrently along time, alongside the vertical residual stream that flows through depth. Like hyper-connections,


Source: ArXiv AI