Recurrent Interface Networks (RINs) address the waste of applying the same amount of computation to every part of high-dimensional data, even though much of that data is redundant. Instead of running global self-attention across all image or video elements, a RIN concentrates it on a smaller set of latent tokens. Cross-attention lets these tokens gather information from the data and send information back. Multiple blocks build bottom-up and top-down feedback, allowing increasingly expressive information routing while keeping the main computation relatively independent of the data’s dimensions.
The work applies this architecture to diffusion models, whose repeated denoising steps offer a natural setting for recurrence: each step makes a gradual change to the generation task. RINs exploit this by initializing each step’s latent tokens with those from the previous step, a method called latent self-conditioning. The resulting models achieve state-of-the-art pixel-based diffusion performance for image and video generation, including 1024-by-1024 images without cascaded models or guidance. The authors also report that RINs are domain-agnostic and can be up to ten times more efficient than 2D and 3D U-Nets. The central contribution is therefore both an architecture for scaling attention and a way to reuse computation across iterative generation.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.