This paper introduces triangleGAN ($\triangle$GAN) that aims at crossdomain joint distribution matching: The model is shown below https://i.imgur.com/boIDOMu.png Having two domains of data $x$ and $y$, there are two generators: 1 $G_x(y)$ which takes $y$ and generates $\tilde{x}$ 2 $G_y(x)$ which takes $x$ and generates $\tilde{y}$ There are two discriminators in the model: 1 $D_1 (x,y)$ a discriminator that distinguishes between $(x, y)$ and either of $(x, \tilde{y})$ or $(\tilde{x}, y)$. 2 $D_2 (x,y)$ a discriminator that distinguishes between $(x, \tilde{y})$ and $(\tilde{x}, y)$. The second discriminator is ALI and can be used on unpaired sets of data. The first discriminator is equivalent to a conditional discriminator where the true paired data $(x, y)$ is compared to either $(x, \tilde{y})$ or $(\tilde{x}, y)$, where one element in the pair is sampled. This discriminator needs paired $(x, y)$ data for training. This model can be used for semisupervised settings, where a small set of paired data is provided. In this paper it is used for:  semisupervised image classification, where a small subset of CIFAR10 is labelled. $x$ and $y$ are images and class labels here.  image to image translation on edge2shoes dataset, where only a subset of dataset is paired.  attribute conditional image generation where $x$ and $y$ domains are image and attributes. CelebA and COCO datasets are used here. In one experiment testset images are projected to attributes and then given those attributes new images are generated: On celebA: https://i.imgur.com/EX5tDZ0.png On COCO: https://i.imgur.com/GRpvjGx.png In another experiment some attributes are chosen (as samples shown below in the first row with different noise) and then another feature is added (using the same noise) to generate the samples in the second row: https://i.imgur.com/KeHL8Ye.png The triangle gan demonstrates improved performance compared to triple gan in the experiments shown in the paper. It has been also compared with Disco gan (a model that can be trained on unpaired data) and shows improved performance when some percentage of paired data is provided. In an experiment they pair each MNIST digit with its transposed (as $x$, $y$ pairs). DiscoGAN cannot learn correct mapping between them, while triangleGAN can learn correct mapping since it leverages paired data. https://i.imgur.com/Vz9Zfhu.png In general this model is a useful approach for semisupervised crossdomain matching and can leverage unpaired data (using ALI) as well as paired data (using conditional discriminator).
Your comment:
