ElasticDino

Dense Semantic Features using Elastic Deformations

WACV 2027

Ulysse Mizrahi* Sean Zaretzky* Dov Danon Amit H. Bermano
Tel Aviv University
*Equal contribution

TL;DR

While vision transformers such as the "Dino" family are being used in various downstream tasks, they are limited in their output resolution due to the quadratic complexity of self-attention.

ElasticDino serves as a lightweight adapter to these models upscaling the output resolution through spatial deformations, using the original full resolution image for guidance. Through the upscaling process it also fixes various artifacts from the base models and creates high-resolution feature maps with sharp semantics beneficial for downstream tasks.

ElasticDino teaser showing DINOv2 features, ElasticDino features, and downstream dense tasks

Abstract

Vision transformers (ViTs) serve as the backbone for countless vision tasks, including dense prediction tasks such as segmentation and depth estimation. However, the quadratic complexity of self-attention limits semantic feature resolution and induces spatial artifacts. Consequently, dense prediction requires significant task-specific, computational and data collection efforts.

In this work, we introduce ElasticDino, an adapter for these backbones, which produces high-resolution feature maps that preserve the fine details of the input image. We observe, as a key insight, the consistency of artifacts across channels, which can be simultaneously resolved and matched to the input image structure. Using a lightweight, fully convolutional architecture that predicts only spatial deformations, ElasticDino produces pixel-aligned, high-resolution feature maps with artifacts greatly reduced. We specifically leverage deformations for their ability to efficiently apply consistent corrections across channels and strictly preserve sharp semantics, thereby, avoiding the smoothness biases inherent to alternative approaches.

Due to the absence of ground truth, we employ pixel-level linear heads to predict a set of downstream dense tasks, guiding the deformation process without leaking task specific semantics into the model.

ElasticDino enables efficient, high-performance training for dense tasks. We demonstrate superior performance over current methods and obtain state-of-the art results on semantic segmentation over a range of popular benchmarks. We argue that ElasticDino is recommended for many of the dense tasks currently employing pretrained attention-based backbones.

Motivation

DINO features are low resolution and prone to artifacts

Standard ViT backbones operate on image patches, often yielding 32x32 or 64x64 feature grids for images that downstream models need to reason about at full resolution. This introduces positional noise, feature bleeding across boundaries, and extra burden on task-specific decoders.

ViT feature artifacts across feature channels
Patch-level features show positional artifacts and thickened boundaries.
Cross-channel consistency of ViT artifacts
The artifacts are spatially consistent across channels, so one deformation field can correct all channels together.

Method

Task-guided Elastic Deformations

ElasticDino alternates bilinear upsampling with lightweight Feature Deformation layers. Each layer reads the current semantic feature map and the image at the same resolution, predicts a 2D deformation field with depthwise-separable convolutional blocks, and warps the features instead of directly rewriting their values.

ElasticDino method overview
Method overview. The input image is passed to the backbone for the low resolution, artifact-full features. These are successively passed to a sequence feature-fixing layers and then bilinearly upsampled by a factor 2. Feature Deformation layers project and concatenate both the image and the semantic features, which are fed to a series of depthwise residual blocks, that end up generating a deformation field that warps the feature map. Task heads induce the sole losses of the system. They are simple MLPs that operate on the individual pixel level. Each head is compared to the ground truth using a task-specific loss.
Shared Deformations

Each Feature Deformation layer predicts a 2D warp field from the image and current feature map. The same spatial warp is applied to all channels, matching the channel-consistent structure of ViT artifacts while keeping computation lightweight.

Semantic Preservation

The network warps existing features rather than directly rewriting feature values. Sampling from the original feature map preserves backbone semantics, discourages hallucinated task labels, and sharpens boundaries without smoothing away object identity.

Pixel-Level Heads

ElasticDino trains without ground-truth high-resolution DINO features. Instead, simple 1x1 task heads supervise every pixel for RGB, depth, semantic segmentation, normals, shading, albedo, residuals, and intrinsic decomposition.

Because the heads operate independently at each pixel, they cannot hide spatial mistakes with decoder context. The deformation layers must therefore align the frozen semantic features to image structure before the heads can solve the dense tasks.

Results

Qualitative Results

Quantitative Results

Semantic Segmentation

Model Cityscapes mIoU ↑ COCO mIoU ↑ ADE20k mIoU ↑
ElasticDino-L0.68750.62700.5299
OneFormer-sw-L0.64120.62640.5109
Mask2Former-sw-L0.59860.62390.4928

Segmentation and Depth downstream tasks comparison against state of the art upsamplers - DINOv2

Bold = best, underline = second best.

MethodSemantic SegmentationDepth Estimation
COCOVOCADE20KCityscapesCOCO
mIoU ↑Acc ↑mIoU ↑Acc ↑mIoU ↑Acc ↑mIoU ↑Acc ↑δ1 ↑RMSE ↓
Nearest56.1776.9776.4193.8037.2771.9154.0590.3658.080.70
Bilinear59.0379.0780.7095.1739.2373.6959.3792.4759.920.66
FeatUp60.1079.9581.0895.3238.8273.7456.0691.8661.690.64
LoftUp61.6281.1184.1896.2241.5175.4162.1393.5358.010.68
JAFAR60.3280.3084.1996.1840.1674.3961.5893.4560.050.65
AnyUp60.6880.5983.8196.1340.3974.7660.9993.3760.650.64
UPLift61.1080.8584.8296.4141.2975.0565.0294.3460.550.64
ED (ours)62.7281.7486.0096.7842.1076.0061.3393.4464.140.57

Segmentation and Depth downstream tasks comparison against state of the art upsamplers - DINOv3

Bold = best, underline = second best.

MethodSemantic SegmentationDepth Estimation
COCOVOCADE20KCityscapesCOCO
mIoU ↑Acc ↑mIoU ↑Acc ↑mIoU ↑Acc ↑mIoU ↑Acc ↑δ1 ↑RMSE ↓
JAFAR62.281.3882.995.9642.3175.757.2592.358.20.68
AnyUp62.7081.6983.8896.2142.6976.0258.3292.8159.430.66
UPLift63.5282.1484.8696.5844.1276.6262.3193.8259.290.66
ED (ours)64.0482.4386.2696.8944.5777.0362.9194.0361.100.62

Runtime and Memory

Run time comparisons of ElasticDino with competing feature upscaling works over DINOv2

Metric ED (ours) AnyUp LoftUp JAFAR UPLift
Avg Time (ms)108.41545.42418.89177.79147.27
Peak mem (GB)0.711.948.317.339.22

Citation

@misc{mizrahi2026elasticdino,
  title={ElasticDino: Dense Semantic Features using Elastic Deformations},
  author={Mizrahi, Ulysse and Zaretzky, Sean and Danon, Dov and Bermano, Amit H.},
  year={2026}
}