Proximal Vision Transformer

Abstract

Transformers have become a dominant architecture in computer vision due to their ability to capture long-range dependencies

however, standard self-attention remains sensitive to feature magnitudes and lacks intrinsic inductive biases such as shift and scale invariance, resulting in a strong dependence on large training datasets. In this work, we introduce Proximal Attention, a reformulation of self-attention in which query and key vectors are normalized before similarity computation. The proposed Proximal Block centers, scales, and ℓ2-normalizes each vector, producing feature representations invariant to affine transformations. We show that the resulting proximal similarity is mathematically equivalent to a scaled Pearson correlation, shifting attention from magnitude-based matching to correlation-based comparison. Proximal Attention is architecture agnostic and can be integrated into both linear and standard quadratic self-attention backbones. We instantiate it in two variants: the Proximal Vision Transformer (PViT), based on a linear-attention backbone, and the Compressed Proximal Vision Transformer (CPViT), based on a standard self-attention backbone with a token-compression module to reduce sequence length. Experiments on ImageNet, CIFAR-10, CIFAR-100, and SVHN demonstrate improved classification performance. On ImageNet, PViT achieves up to 83.9% Top-1 accuracy, while CPViT reaches 83.3% at lower FLOPs than comparable quadratic-attention models. © 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.