SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing
By: Han Zou , Yan Zhang , Ruiqi Yu and more
Potential Business Impact:
Helps artists edit drawings with words and lines.
Sketch editing is central to digital illustration, yet existing image editing systems struggle to preserve the sparse, style-sensitive structure of line art while supporting both high-level semantic changes and precise local redrawing. We present SketchAssist, an interactive sketch drawing assistant that accelerates creation by unifying instruction-guided global edits with line-guided region redrawing, while keeping unrelated regions and overall composition intact. To enable this assistant at scale, we introduce a controllable data generation pipeline that (i) constructs attribute-addition sequences from attribute-free base sketches, (ii) forms multi-step edit chains via cross-sequence sampling, and (iii) expands stylistic coverage with a style-preserving attribute-removal model applied to diverse sketches. Building on this data, SketchAssist employs a unified sketch editing framework with minimal changes to DiT-based editors. We repurpose the RGB channels to encode the inputs, enabling seamless switching between instruction-guided edits and line-guided redrawing within a single input interface. To further specialize behavior across modes, we integrate a task-guided mixture-of-experts into LoRA layers, routing by text and visual cues to improve semantic controllability, structural fidelity, and style preservation. Extensive experiments show state-of-the-art results on both tasks, with superior instruction adherence and style/structure preservation compared to recent baselines. Together, our dataset and SketchAssist provide a practical, controllable assistant for sketch creation and revision.
Similar Papers
Real-Time Intuitive AI Drawing System for Collaboration: Enhancing Human Creativity through Formal and Contextual Intent Integration
CV and Pattern Recognition
Draws pictures from your doodles and ideas.
Simple Lines, Big Ideas: Towards Interpretable Assessment of Human Creativity from Drawings
CV and Pattern Recognition
Computer grades drawings for creativity automatically.
Stroke2Sketch: Harnessing Stroke Attributes for Training-Free Sketch Generation
CV and Pattern Recognition
Makes drawings look like a specific artist's style.