Google MobileDiffusion

What is Google MobileDiffusion?

MobileDiffusion is a text-to-image model designed specifically for fast on-device generation on mobile phones. It is a compact latent diffusion model with only 520 million parameters, made up of three main components: a text encoder, a diffusion UNet, and an image decoder.

Its key efficiency feature is the use of DiffusionGAN for one-step image generation, allowing it to produce a high-quality 512x512 image in about half a second on newer iPhones and Android devices. This combination of speed and small size makes on-device text-to-image generation practical, with responsible AI practices in mind.

Who created Google MobileDiffusion?

MobileDiffusion was developed by a team at Google including Zhisheng Xiao, Yanwu Xu, Jiuqiang Tang, Haolin Jia, Lutz Justen, Daniel Fenner, Ronald Wotzlaw, Jianing Wei, Raman Sarokin, Juhyun Lee, Andrei Kulik, Chuo-Ling Chang, and Matthias Grundmann. Their goal was to build an efficient latent diffusion model capable of rapid text-to-image generation on mobile devices within a compact 520 million parameter footprint.

How Google MobileDiffusion works

• Architecture: MobileDiffusion uses a text encoder (CLIP-ViT/L14), a diffusion UNet, and an image decoder.
• Diffusion UNet: The UNet mixes transformer blocks and convolution blocks, with an emphasis on using transformer blocks efficiently.
• One-step generation: DiffusionGAN enables single-step sampling, which is what makes sub-second image generation possible.
• Training: The generator and discriminator are initialized from a pre-trained diffusion UNet, which speeds up training by reusing the existing model’s learned features.
• Performance: The model runs on both iOS and Android phones, generating 512x512 images in roughly half a second.
• Output quality: Generated images are high-quality and varied across different styles and subjects, and the project page includes sample images produced with DiffusionGAN’s one-step sampling.