Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition
AuthorsGuray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Naser Damer, Fadi Boutros
AffiliationsTU Darmstadt, Darmstadt, Germany · Fraunhofer Institute for Computer Graphics Research IGD, Darmstadt, Germany
SD-ViT trains every ViT depth to produce useful face embeddings, enabling cheaper early-exit recognition without changing the gallery.
Key results
ViT-S/12 on WebFace4M at half depth.
Baseline comparison at the same depth and compute.
SD-ViT probe matched against a gallery enrolled at full depth.
Compared with 96.89% for conventional full-depth training.
ViT-S/12 training overhead on NVIDIA A100, from 65.5 to 68.6 GPU-hours.
What the paper found
Stochastic Deep Supervision for Gallery Consistent Early-Exit Face Recognition introduces SD-ViT, which makes intermediate Vision Transformer layers usable for face recognition without retraining a separate model for each compute budget. During training, it randomly samples an intermediate block, sends its features through the same embedding head as the final block, and applies the same AdaFace margin loss and identity prototypes. This shared supervision puts embeddings from different depths in a common space, so an early-exit probe can match a gallery enrolled once at full depth, with no gallery recomputation or per-exit adapters. On WebFace4M with ViT-S/12, SD-ViT reaches 95.11% IJB-C TAR at half depth, compared with 42.22% for standard training; at quarter depth, it scores 93.06% against a full-depth gallery, where the standard model scores 0.02%. Full-depth performance is 96.51% versus 96.89% for the conventional model. A single threshold calibrated at full depth also remains usable from quarter depth onward. The auxiliary training adds 4.7% in the ViT-S/12 run, measured in NVIDIA A100 GPU-hours. The key result is that one model can trade compute for speed while keeping its face embeddings compatible with a fixed gallery.
Original abstract
Vision Transformers (ViTs) perform strongly in face recognition (FR), yet their fixed-depth inference limits deployment in resource-constrained settings. We propose SD-ViT, a training framework that gives a usable exit at every block depth by making every depth produce discriminative embeddings, for budget-selected fixed-depth exits. Standard ViTs supervise intermediate layers only through backpropagation, so their intermediate representations are not directly optimized to form usable identity embeddings. SD-ViT adds a stochastic auxiliary signal: at each iteration a randomly selected intermediate block is passed through the same embedding head as the final layer and trained with a weighted margin-penalty softmax loss. The objective is thus stochastically sampled deep supervision through a shared classifier. Because both loss terms share that head and classifier, intermediate tokens are trained to map into the same embedding space as final-layer tokens. This matters for FR, where matching is by similarity: a probe processed at an early exit can be compared directly against full-depth gallery embeddings, with no gallery recomputation or adaptation layers. On established benchmarks, SD-ViT stays within 0.4 IJB-C points of the same backbone trained conventionally at full depth, while its early-exit probes recover over 96% of their own full-depth performance at 50% of the computational cost without modifying the reference gallery. A single verification threshold calibrated at full depth remains usable from quarter depth onward.
Read the original paperMore in Computer Vision
Browse all 82 papers →InstantHMR : Real-Time Human Mesh Recovery on Your Phone
Mohamed Adjel, Kahina Chalabi, Vincent Bonnet
InstantHMR reconstructs detailed 3D human meshes from images in milliseconds, including on a smartphone.
VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection
Ke Sun, Shuwei Li, Haiyang Mei, Mike Zheng Shou
A periodic trace left by video-generation VAEs could help spot AI-made videos even when their visible artifacts are gone.
CoLa3D: Composable Latent 3D Decomposition
Xudong Cai, Yongcai Wang, Deying Li, Chuanxia Zheng
CoLa3D turns a whole-scene representation into a set of coherent, reusable 3D object meshes.