Structure-Accurate Medical Image Translation via Dynamic Frequency Balance and Knowledge Guidance
By: Jiahua Xu , Dawei Zhou , Lei Hu and more
Potential Business Impact:
Makes medical pictures clearer for better diagnoses.
Multimodal medical images play a crucial role in the precise and comprehensive clinical diagnosis. Diffusion model is a powerful strategy to synthesize the required medical images. However, existing approaches still suffer from the problem of anatomical structure distortion due to the overfitting of high-frequency information and the weakening of low-frequency information. Thus, we propose a novel method based on dynamic frequency balance and knowledge guidance. Specifically, we first extract the low-frequency and high-frequency components by decomposing the critical features of the model using wavelet transform. Then, a dynamic frequency balance module is designed to adaptively adjust frequency for enhancing global low-frequency features and effective high-frequency details as well as suppressing high-frequency noise. To further overcome the challenges posed by the large differences between different medical modalities, we construct a knowledge-guided mechanism that fuses the prior clinical knowledge from a visual language model with visual features, to facilitate the generation of accurate anatomical structures. Experimental evaluations on multiple datasets show the proposed method achieves significant improvements in qualitative and quantitative assessments, verifying its effectiveness and superiority.
Similar Papers
Frequency-domain Multi-modal Fusion for Language-guided Medical Image Segmentation
CV and Pattern Recognition
Helps doctors find lung sickness in X-rays.
Robust AI-Synthesized Image Detection via Multi-feature Frequency-aware Learning
Graphics
Finds fake pictures made by AI.
FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editing
CV and Pattern Recognition
Edits pictures precisely without messing them up.