Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generation

Token merging speeds up diffusion models without extra training by skipping token calculations that don’t add new information. Current methods focus mainly on the parts of an image that match the text prompt. This often throws away important background features and fine details that the prompt never mentions. The paper proposes Dynamics‑aware Token Merging (DaTMe), another training‑free approach that uses the model’s own diffusion dynamics to decide which tokens to drop. DaTMe builds a “dynamics‑aware importance map” with the Tweedie denoised estimate. The map combines two signals: * a temporal signal that measures how the predicted clean image changes from one diffusion step to the next, and * a structural

signal that picks up high‑frequency details like edges and boundaries. Based on a chosen compression ratio, DaTMe groups tokens adaptively for merging. For M.Tech projects or simulations, DaTMe gives a solid way to explore token redundancy in generative vision pipelines. When implementing it, you can compare how well DaTMe keeps structural coherence at different compression levels against baselines that only use text relevance. Keep in mind that the reported quality improvements are shown in the paper, but the actual speedup you see will depend on the diffusion model you use and your hardware setup.

Message: Hi FE, I need help with “Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generation” in “Signal & Image Processing”