Sissi: Zero-shot Style-guided Image Synthesis via Semantic-style Integration
By: Yingying Deng , Xiangyu He , Fan Tang and more
Text-guided image generation has advanced rapidly with large-scale diffusion models, yet achieving precise stylization with visual exemplars remains difficult. Existing approaches often depend on task-specific retraining or expensive inversion procedures, which can compromise content integrity, reduce style fidelity, and lead to an unsatisfactory trade-off between semantic prompt adherence and style alignment. In this work, we introduce a training-free framework that reformulates style-guided synthesis as an in-context learning task. Guided by textual semantic prompts, our method concatenates a reference style image with a masked target image, leveraging a pretrained ReFlow-based inpainting model to seamlessly integrate semantic content with the desired style through multimodal attention fusion. We further analyze the imbalance and noise sensitivity inherent in multimodal attention fusion and propose a Dynamic Semantic-Style Integration (DSSI) mechanism that reweights attention between textual semantic and style visual tokens, effectively resolving guidance conflicts and enhancing output coherence. Experiments show that our approach achieves high-fidelity stylization with superior semantic-style balance and visual quality, offering a simple yet powerful alternative to complex, artifact-prone prior methods.
Similar Papers
Insert In Style: A Zero-Shot Generative Framework for Harmonious Cross-Domain Object Composition
CV and Pattern Recognition
Puts real things into cartoon worlds perfectly.
Text to Sketch Generation with Multi-Styles
CV and Pattern Recognition
Draws pictures in any chosen art style.
Generative Editing in the Joint Vision-Language Space for Zero-Shot Composed Image Retrieval
CV and Pattern Recognition
Find images using text and a picture.