https://i.imgur.com/lM3EjK9.png Problem ============= Label map (semantic segmentation) to realistic image using GANs. Contributions ========= 1. Coarse-to-fine generator 2. Multi-scale discriminator 3. Robust adversarial learning objective function Coarse-to-fine Generator ================= https://i.imgur.com/osEyGOj.png G1 - Global generator G2 - Local enhancer Global Generator: 1. convolutional front-end 2. set of residual blocks 3. transposed convolutional back-end A semantic label map is passed through the 3 components sequentially Local Enhancer: 1. convolutional front-end 2. set of residual blocks 3. transposed convolutional back-end Training scheme: 1. Train standalone global generator 2. Freeze global generator weights, train local enhancer 3. Fine-tune all weights together Multi scale Discriminator =================== https://i.imgur.com/hNP1cni.png To allow for global context but work at higher resolution as well, several discriminators are applied at different image scales. Robust adversarial learning objective function =============================== https://i.imgur.com/j7CIbV3.png * Compare original and generated images in feature space at different scales. * This is done to ensure more abstract resemblance, not just pixel-space resemblance. * For feature extraction the discriminator is used.