臺灣博碩士論文加值系統

在2014年， Goodfellow發表了一個全新的生成模型，生成對抗模型。 生成對抗模型讓兩個類神經網路在一個極小極大化比賽中相互對抗，試著去生成現實中的例子。 但是生成對抗模型的目標函數有不太好的性質導致無法收斂的問題。 為了解決這個問題， Arjovsky提出了Wasserstein距離以取代延森-山農散度而且他證明了Wasserstein距離有比延森-山農散度更好的性質。 而當Arjovsky使用Wasserstein距離時，他必須面對一個問題，如何滿足1-Lipschitz條件。 Arjovsky使用權重剪裁去解決這個問題但是這樣會產生一些奇怪的行為。 根據這個問題，Gulrajani使用了梯度懲罰去提升Wasserstein生成對抗網路且達到很有質量的生成。

In 2014, Goodfellow proposed a brand-new generative model, the generative adversarial net. The generative adversarial net lets two neural networks fight with each other in a minimax game and try to generate realistic samples. But the object function of the generative adversarial net has poor properties and then it would cause the non-convergence issue. For this, Arjovsky introduced the Wasserstein distance to replace the Jensen-Shannon divergence and he proved that the Wasserstein distance has better properties than the Jensen-Shannon divergence. And when Arjovsky used the Wasserstein distance, he faced a new problem, how to satisfy the 1-Lipschitz condition. Arjovsky solved it by using weight clipping but it would lead to some pathological behaviors. For this problem, Gulrajani made use of the gradient penalty to improve the Wasserstein generative adversarial net and achieved the high quality generations.

中文摘要
Abstract
誌謝 i
Content iii
List of Tables iv
List of Figures v
. 1  Introduction 1

. 2  Generative Adversarial Net 4
. 2.1  Framework ............... 4

. 2.2  The Objective of the Generative Adversarial Net. . . . . . . 4

. 2.3  The Training Procedure of the Generative Adversarial Net . . . . . . . . . . 7

. 3  The Problem of the Generative Adversarial Net 9

. 4  Wasserstein Generative Adversarial Net 13
. 4.1  Wasserstein Distance.................... 13

. 4.2  Properties of the Wasserstein Distance ................. 14

. 4.3  Comparison with Other Divergences ................... 17

. 4.4  Compute Wasserstein Distance with Linear Programming . . . . . . . . . . . 20

. 4.5  Wasserstein Generative Adversarial Net............... 23

. 4.6  Improved Wasserstein Generative Adversarial Net . . . . 26

. 5 Experiments 29
. 5.1 Mnist ....................................... 29

. 5.2 CIFAR-10 . . . . . . . . . . . . . . . . . . . . . . . . 31

. 5.3 Flowers ...................................... 33

. 5.4 LSUN Dataset .................................. 36

. 6 Conclusion 47

. Reference 48

. [1]  Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adver- sarial networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 214–223, International Convention Centre, Sydney, Australia, 06–11 Aug 2017. PMLR.

. [2]  Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2172– 2180, 2016.

. [3]  Robert O Ferguson. Linear programming, volume 19.

. [4]  Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
http://www.deeplearningbook.org.

. [5]  Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, edi- tors, Advances in Neural Information Processing Systems 27, pages 2672–2680. Curran Associates, Inc., 2014.

. [6]  Ian J. Goodfellow. NIPS 2016 tutorial: Generative adversarial networks. CoRR, abs/1701.00160, 2017.

. [7]  Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5767–5777. Curran Associates, Inc., 2017.

. [8]  Vincent Herrmann. Wasserstein gan and the kantorovich-rubinstein duality. 2017.

. [9]  G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. r. Mohamed, N. Jaitly, A. Senior, V. Van- houcke, P. Nguyen, T. N. Sainath, and B. Kingsbury. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82–97, Nov 2012.

. [10]  Geo rey E. Hinton, Terrence J. Sejnowski, and David H. Ackley. Boltzmann machines: Constraint satisfaction networks that learn. Technical Report CMU-CS-84-119, Com- puter Science Department, Carnegie Mellon University, Pittsburgh, PA, 1984.

. [11]  Sergey Io e and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, pages 448–456. JMLR.org, 2015.

. [12]  Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image transla- tion with conditional adversarial networks. arXiv preprint, 2017.

. [13]  Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

. [14]  Alex Krizhevsky and Geo rey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.

. [15]  Alex Krizhevsky, Ilya Sutskever, and Geo rey E Hinton. Imagenet classi cation with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012.

. [16]  Y. Lecun, L. Bottou, Y. Bengio, and P. Ha ner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov 1998.

. [17]  Yann LeCun, Bernhard E. Boser, John S. Denker, Donnie Henderson, R. E. Howard, Wayne E. Hubbard, and Lawrence D. Jackel. Handwritten digit recognition with a back-propagation network. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems 2, pages 396–404. Morgan-Kaufmann, 1990.

. [18]  Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010.

. [19]  Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint, 2017.

. [20]  Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. CoRR, abs/1611.02163, 2016.

. [21]  Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. CoRR, abs/1411.1784, 2014.

. [22]  Vinod Nair and Geo rey E. Hinton. Recti ed linear units improve restricted boltz- mann machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, pages 807–814, USA, 2010. Omnipress.

. [23]  Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In Advances in Neural In- formation Processing Systems, pages 271–279, 2016.

. [24]  Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. arXiv preprint arXiv:1605.05396, 2016.

. [25]  Sebastian Ruder. An overview of gradient descent optimization algorithms. CoRR, abs/1609.04747, 2016.

. [26]  Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neu- ral Information Processing Systems 29, pages 2234–2242. Curran Associates, Inc., 2016.

. [27]  Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.

. [28]  Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li. Empirical evaluation of recti ed activations in convolutional network. CoRR, abs/1505.00853, 2015.

. [29]  Fisher Yu, Yinda Zhang, Shuran Song, Ari Se , and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.

. [30]  Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional net- works. In European conference on computer vision, pages 818–833. Springer, 2014.

. [31]  Matthew D Zeiler, Graham W Taylor, and Rob Fergus. Adaptive deconvolutional net- works for mid and high level feature learning. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2018–2025. IEEE, 2011.

. [32]  Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image- to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593, 2017.