Dynamic VLM-Guided Negative Prompting for Diffusion Models
By: Hoyeon Chang, Seungjin Kim, Yoonseok Choi
Potential Business Impact:
Makes AI art look better by guiding it away from bad ideas.
We propose a novel approach for dynamic negative prompting in diffusion models that leverages Vision-Language Models (VLMs) to adaptively generate negative prompts during the denoising process. Unlike traditional Negative Prompting methods that use fixed negative prompts, our method generates intermediate image predictions at specific denoising steps and queries a VLM to produce contextually appropriate negative prompts. We evaluate our approach on various benchmark datasets and demonstrate the trade-offs between negative guidance strength and text-image alignment.
Similar Papers
VLM-Guided Adaptive Negative Prompting for Creative Generation
Graphics
Makes computers invent surprising new pictures.
Generalizable Vision-Language Few-Shot Adaptation with Predictive Prompts and Negative Learning
CV and Pattern Recognition
Teaches computers to learn from few examples.
VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Models
CV and Pattern Recognition
Teaches computers to understand pictures and words better.