NTH

Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition

AuthorsGuray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Naser Damer, Fadi Boutros

AffiliationsTU Darmstadt, Darmstadt, Germany · Fraunhofer Institute for Computer Graphics Research IGD, Darmstadt, Germany

November 19, 2026 2 min read
Watch on YouTube
The one-line take

SD-ViT trains every ViT depth to produce useful face embeddings, enabling cheaper early-exit recognition without changing the gallery.

Key results

95.11%
Half-depth SD-ViT IJB-C TAR

ViT-S/12 on WebFace4M at half depth.

42.22%
Half-depth standard-training IJB-C TAR

Baseline comparison at the same depth and compute.

93.06%
Quarter-depth gallery-fixed IJB-C TAR

SD-ViT probe matched against a gallery enrolled at full depth.

96.51%
Full-depth SD-ViT IJB-C TAR

Compared with 96.89% for conventional full-depth training.

4.7%
Additional training time

ViT-S/12 training overhead on NVIDIA A100, from 65.5 to 68.6 GPU-hours.

What the paper found

Stochastic Deep Supervision for Gallery Consistent Early-Exit Face Recognition introduces SD-ViT, which makes intermediate Vision Transformer layers usable for face recognition without retraining a separate model for each compute budget. During training, it randomly samples an intermediate block, sends its features through the same embedding head as the final block, and applies the same AdaFace margin loss and identity prototypes. This shared supervision puts embeddings from different depths in a common space, so an early-exit probe can match a gallery enrolled once at full depth, with no gallery recomputation or per-exit adapters. On WebFace4M with ViT-S/12, SD-ViT reaches 95.11% IJB-C TAR at half depth, compared with 42.22% for standard training; at quarter depth, it scores 93.06% against a full-depth gallery, where the standard model scores 0.02%. Full-depth performance is 96.51% versus 96.89% for the conventional model. A single threshold calibrated at full depth also remains usable from quarter depth onward. The auxiliary training adds 4.7% in the ViT-S/12 run, measured in NVIDIA A100 GPU-hours. The key result is that one model can trade compute for speed while keeping its face embeddings compatible with a fixed gallery.

Original abstract

Vision Transformers (ViTs) perform strongly in face recognition (FR), yet their fixed-depth inference limits deployment in resource-constrained settings. We propose SD-ViT, a training framework that gives a usable exit at every block depth by making every depth produce discriminative embeddings, for budget-selected fixed-depth exits. Standard ViTs supervise intermediate layers only through backpropagation, so their intermediate representations are not directly optimized to form usable identity embeddings. SD-ViT adds a stochastic auxiliary signal: at each iteration a randomly selected intermediate block is passed through the same embedding head as the final layer and trained with a weighted margin-penalty softmax loss. The objective is thus stochastically sampled deep supervision through a shared classifier. Because both loss terms share that head and classifier, intermediate tokens are trained to map into the same embedding space as final-layer tokens. This matters for FR, where matching is by similarity: a probe processed at an early exit can be compared directly against full-depth gallery embeddings, with no gallery recomputation or adaptation layers. On established benchmarks, SD-ViT stays within 0.4 IJB-C points of the same backbone trained conventionally at full depth, while its early-exit probes recover over 96% of their own full-depth performance at 50% of the computational cost without modifying the reference gallery. A single verification threshold calibrated at full depth remains usable from quarter depth onward.

Read the original paper

More in Computer Vision

Browse all 82 papers →