VibeEdit: Image Editing with Canvas Instructions
AuthorsJinjing Zhao, Fangyun Wei, Yitong Wang, Xiuyu Wu, Yunuo Chen, Yang Yue, Sirui Zhang, Wenbo Wang, Hongyang Zhang, Dong Chen, Yan Lu, Chang Xu
AffiliationsUniversity of Sydney · Microsoft Research · Fudan University · Nankai University · Shanghai Jiao Tong University · Tsinghua University · University of Science and Technology of China · University of Waterloo · Oct 2026 Project page: https://zhaojingjing713.github.io/VibeEdit/ Conventional Editing Canvas-Annotated Image Edited Image Image VibeEdit “Change the middle chair to blue” Text Prompt Write & Mar
VibeEdit lets users edit the exact objects they mean by marking them directly on an image and briefly describing the desired change.
Key results
Source–target pairs used to train VibeEdit.
Human-curated cases emphasizing target selection among similar objects.
Overall score on the VibeEdit benchmark.
PSNR in dB, measuring preservation outside the edit region.
Strongest text-instructed baseline’s overall benchmark score.
Baseline PSNR in dB outside the edit region.
What the paper found
VibeEdit replaces the separate text prompt in image editing with canvas instructions: users mark an image with circles, scribbles, arrows, and short notes to specify both the target and the requested change. It supports adding, removing, replacing, modifying, and moving objects. Built by adapting Qwen-Image-Edit, the system uses layer-decoupled conditioning: Qwen2.5-VL interprets the annotated image, while separate generative inputs encode the clean source image and canvas marks. Training combines region-weighted supervised fine-tuning with rubric-guided DiffusionNFT reinforcement learning, rewarding edit completion, local quality, and preservation of untouched regions. The training corpus contains 1,554,062 source–target edit pairs, and evaluation uses an independently curated 419-case benchmark focused on selecting the right object among similar instances. On that benchmark, VibeEdit scores 79.9% on the VLM rubric and 32.8 dB in outside-region PSNR, compared with 67.4% and 24.0 dB for FireRed, the strongest text-instructed baseline in the comparison. OpenAI’s GPT-5.6-sol serves as the benchmark judge. The results highlight the central contribution: spatial marks paired with brief notes can ground edits more precisely while reducing unwanted changes elsewhere in the image.
Original abstract
In text-guided image editing, describing the desired change is often straightforward, but identifying the intended object or region can be cumbersome, especially when several objects look alike. We introduce a new image editing interface that lets users place spatial marks and optional short notes directly on the image. Together, these annotations form a canvas instruction that specifies where to edit and what to change. Our editor, VibeEdit, follows these instructions to perform object addition, removal, replacement, attribute modification, and movement without a separate text prompt. We construct 1.55 million source-target edit pairs with object masks and structured edit descriptions, from which we render canvas instructions during training. We adapt Qwen-Image-Edit with layer-decoupled conditioning that separately encodes source images and canvas instructions for image editing. We train the model with region-weighted supervised fine-tuning, followed by rubric-guided reinforcement learning to improve edit completion, local edit quality, and preservation of unedited regions. We evaluate VibeEdit on an independently constructed, human-curated benchmark of 419 cases emphasizing target selection among similar objects. VibeEdit achieves a VLM rubric score of 79.9 and an outside-region PSNR of 32.8 dB, compared with 67.4 and 24.0 dB for FireRed, the highest-scoring text-instructed baseline in our evaluation.
Read the original paperMore in Computer Vision
Browse all 64 papers →LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
Suhwan Cho, Yonwoo Choi, Soongjin Kim, Jicheol Park, Taegyu Lim
LEGO replaces brittle depth-based camera lifting with a learned transformer view synthesizer that gives diffusion models better structure for generating first-person video.
NAMVIS: Next-Scale Autoregressive Multi-View Image Synthesis
Ramil Khafizov, Ilya Statsenko, Ruslan Rakhimov, Artem Komarichev, Peter Wonka, Evgeny Burnaev
NAMVIS makes multi-view image generation faster by using camera-aware autoregressive scaling instead of slow diffusion denoising.
JLD: Perceptual Distance Through A Jacobian Lens
Shreshth Saini, Balu Adsumilli, Alan C. Bovik
JLD uses the Jacobian of a frozen vision model to build a faster, more resolution-robust measure of how visually different two images appear.