深度學習擁有卓越的能力來探索資料中背後的特徵。雖然深度學習在實務上有重大突破，其理論的了解卻甚少。近期文獻指出，受限玻爾茲曼機與變分重整化群有一對一的對應。然而，這對應是有爭議的，我們希望能建立更嚴謹的對應關係。

在這篇論文中，我們使用受限玻爾茲曼機用以優化重整化群。理論上，重整化群的描述需要無限多的耦合常數。因此，在實務上人們會引進變分參數來取代耦合常數。然而，最佳變分參數的選擇常破壞自洽性，而有效率的投影算符是問題相依的。因次，我們使用受限玻爾茲曼機來參數化投影算符，並以相對熵的最小化當作選擇變分參數的最佳準則。我們相信，本演算法可以做為優化重整化群的通用架構，並給出重整化群與深度學習對應的解釋。

Deep learning has yielded impressive results in difficult machine learning tasks due to its ability to learn relevant features from data. Despite the success of deep learning, relatively little is understood theoretically. It has been shown recently an exact mapping between the variational renormalization group and the deep neural networks based on the restricted Boltzmann machines. Since the discussions are not uncontroversial, it remains desirable to establish a more rigorous connection between renormalization group and deep learning.

In this work, we propose a general method for optimizing real-space renormalization-group transformation through divergence minimization. One of the main obstacle in real space renormalization group methods is that the renormalized Hamiltonian involves an infinity of coupling parameters. For this reason it is an old intention to improve the transformation by introducing variational parameters. However, the optimal criterion for choosing variational parmameter can lead to inherent inconsistency and the form of projection operators can be problem dependent. Therefore, we explore the structure of restricted Boltzmann machine to parameterize the projection operator and adopt the minimization of the Kullback-Leibler divergence between the normalizing factor and the Hamiltonian as the optimal criterion in choosing the variational parameter. It may serve as a general method for optimizing real-space renormalization-group transformation and shed light on the connection between renormalization group and deep learning.

口試委員會審定書 iii
摘要 v
Abstract v
1 Introduction 1
2 Real Space Renormalization Group 3
2.1 Phase Transition 3
2.1.1 Critical Exponents 4
2.1.2 Scaling Hypothesis 6
2.1.3 Kadanoff Construction 7
2.2 Definition of Transformation 8
2.2.1 Examples of Projection Operators 10
2.2.2 Eigenvalues and Critical Exponents 10
2.2.3 Equivalent Formulation of Transformation 14
2.3 Proliferation of Interactions 15
2.4 Non-Variational Renormalization Group 17
2.4.1 Finite Lattice Renormalization Group 17
2.4.2 Niemeijer-van Leeuwen Cumulant Approximation 20
2.5 Variational Renormalization Group 21
2.5.1 Migdal-Kadanoff Bond-Moving Approximation 22
2.5.2 Kadanoff’s Upper-Bound Approximation 23
2.5.3 Jan’s Approximation 28
2.5.4 Divergence Minimization 29
2.6 Monte Carlo Renormalization Group 32
2.6.1 Calculation of Critical Exponents 34
3 Boltzmann Machine 37
3.1 Graphical Models 37
3.2 Exponential Families 40
3.2.1 Mean Parametrization 41
3.3 Learning 43
3.4 Markov Chain Monte Carlo 46
3.4.1 Gibbs Sampling 48
3.4.2 Contrastive Divergence 49
4 Optimization of Renormalization Group Transformations 51
4.1 Convergence of Renormalization 51
4.2 Location of FixedP oints 52
4.2.1 Swendsen’s Optimization 52
4.2.2 Gausterer’s Optimization 54
4.3 Divergence Minimization 55
5 Experiment 57
5.1 Monte Carlo Renormalization Group 57
5.2 Learning Thermodynamics 60
5.2.1 Filters 62
5.3 Tuned Renormalization 65
5.4 Divergence Minimization 70
6 Conclusion 75
Bibliography 77

[1] Maciej Koch-Janusz and Zohar Ringel. Mutual information, neural networks and the renormalization group. Nature Physics, 14(6):578, 2018.
[2] Giuseppe Carleo and Matthias Troyer. Solving the quantum many-body prob- lem with artificial neural networks. Science, 355(6325):602–606, 2017.
[3] Elina Robeva and Anna Seigal. Duality of graphical models and tensor net- works. arXiv preprint arXiv:1710.01437, 2017.
[4] Pankaj Mehta and David J Schwab. An exact mapping between the variational renormalization group and deep learning. arXiv preprint arXiv:1410.3831, 2014.
[5] Leo P Kadanoff. Variational principles and approximate renormalization group calculations. Physical Review Letters, 34(16):1005, 1975.
[6] Robert H Swendsen. Optimization of real-space renormalization-group trans- formations. Physical review letters, 52(26):2321, 1984.
[7] H Gausterer and CB Lang. Operator oriented optimization of block spin transformations. Physics Letters B, 186(1):103–106, 1987.
[8] Dorit Ron, Achi Brandt, and Robert H Swendsen. Surprising convergence of the monte carlo renormalization group for the three-dimensional ising model. Physical Review E, 95(5):053305, 2017.
[9] H Eugene Stanley. Phase transitions and critical phenomena. Clarendon Press, Oxford, 1971.
[10] Rodney J Baxter. Exactly solved models in statistical mechanics. Elsevier, 2016.
[11] Julia M Yeomans. Statistical mechanics of phase transitions. Clarendon Press, 1992.
[12] Michael E Fisher. Renormalization group theory: Its basis and formulation in statistical physics. Reviews of Modern Physics, 70(2):653, 1998.
[13] Michael E Fisher. The theory of equilibrium critical phenomena. Reports on progress in physics, 30(2):615, 1967.
[14] Leo P Kadanoff. Scaling laws for ising models near tc. Physics Physique Fizika, 2(6):263, 1966.
[15] Franz J Wegner. Corrections to scaling laws. Physical Review B, 5(11):4529, 1972.
[16] Kenneth G Wilson. Renormalization group and critical phenomena. i. renor- malization group and the kadanoff scaling picture. Physical review B, 4(9):3174, 1971.
[17] Martin J Wainwright, Michael I Jordan, et al. Graphical models, exponen- tial families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2):1–305, 2008.
[18] Leo P Kadanoff and Anthony Houghton. Numerical evaluations of the critical properties of the two-dimensional ising model. Physical Review B, 11(1):377, 1975.
[19] Th Niemeijer and JMJ Van Leeuwen. Renormalization theory for ising like spin systems. Phase Transitions and Critical Phenomena, 6, 1976.
[20] HJ Hilhorst, M Schick, and JMJ Van Leeuwen. Differential form of real-space renormalization: Exact results for two-dimensional ising models. Physical Review Letters, 40(25):1605, 1978.
[21] HJ Hilhorst, M Schick, and JMJ van Leeuwen. Exact renormalization group equations for the two-dimensional ising model. Physical Review B, 19(5):2749, 1979.
[22] Kenneth G Wilson. The renormalization group: Critical phenomena and the kondo problem. Reviews of modern physics, 47(4):773, 1975.
[23] Thomas L Bell and Kenneth G Wilson. Finite-lattice approximations to renor- malization groups. Physical Review B, 11(9):3431, 1975.
[24] JA Tjon. Numerical study of the renormalization group equations in the four-cell approximation. Physics Letters A, 49(4):289–290, 1974.
[25] M Nauenberg and B Nienhuis. Critical surface for square ising spin lattice. Physical Review Letters, 33(16):944, 1974.
[26] Th Niemeijer and JMJ Van Leeuwen. Wilson theory for spin systems on a triangular lattice. Physical Review Letters, 31(23):1411, 1973.
[27] Th Niemeyer and JMJ Van Leeuwen. Wilson theory for 2-dimensional ising spin systems. Physica, 71(1):17–40, 1974.
[28] Mehran Kardar. Statistical physics of fields. Cambridge University Press, 2007.
[29] Alexander A Migdal. Phase transitions in gauge and spin-lattice systems. Soviet Journal of Experimental and Theoretical Physics, 42:743, 1976.
[30] Leo P Kadanoff. Notes on migdal’s recursion formulas. Annals of Physics, 100(1-2):359–394, 1976.
[31] Theodore W Burkhardt and JMJ van Leeuwen. Real-space renormalization, volume 30. Springer, 1982.
[32] W Van Saarloos, JMJ Van Leeuwen, and AMM Pruisken. Variational prin- ciples in renormalization theory. Physica A: Statistical Mechanics and its Applications, 92(3-4):323–342, 1978.
[33] Theodore W Burkhardt. Kadanoff’s lower-bound renormalization transfor- mation. Physical Review B, 13(7):3187, 1976.
[34] Efi Efrati, Zhe Wang, Amy Kolan, and Leo P Kadanoff. Real-space renor- malization in statistical mechanics. Reviews of Modern Physics, 86(2):647, 2014.
[35] Leo P Kadanoff, Anthony Houghton, and Mehmet C Yalabik. Variational ap- proximations for renormalization group transformations. Journal of Statistical Physics, 14(2):171–203, 1976.
[36] Robert H Swendsen. Monte carlo renormalization group. Physical Review Letters, 42(14):859, 1979.
[37] Clive F Baillie, Rajan Gupta, Kenneth A Hawick, and G Stuart Paw- ley. Monte carlo renormalization-group study of the three-dimensional ising model. Physical Review B, 45(18):10438, 1992.
[38] Steffen L Lauritzen. Graphical models, volume 17. Clarendon Press, 1996.
[39] Kevin P Murphy. Machine learning: a probabilistic perspective. MIT press,
2012.
[40] Kenneth G Wilson. Monte-carlo calculations for the lattice gauge theory. In
Recent developments in gauge theories, pages 363–402. Springer, 1980.
[41] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT
press, 2016.
[42] Geoffrey E Hinton. Training products of experts by minimizing contrastive
divergence. Neural computation, 14(8):1771–1800, 2002.
[43] Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh. A fast learning
algorithm for deep belief nets. Neural computation, 18(7):1527–1554, 2006.
[44] Kenneth G Wilson. The renormalization group and critical phenomena. Re- views of Modern Physics, 55(3):583, 1983.
[45] Michael E Fisher and Mohit Randeria. Location of renormalization-group fixed points. Physical review letters, 56(21):2332, 1986.
[46] Robert H Swendsen. Monte carlo calculation of renormalized coupling pa- rameters. Physical review letters, 52(14):1165, 1984.
[47] Robert B Griffiths and Paul A Pearce. Position-space renormalization-group transformations: Some proofs and some problems. Physical Review Letters, 41(14):917, 1978.
[48] Robert B Griffiths and Paul A Pearce. Mathematical properties of position- space renormalization-group transformations. Journal of Statistical Physics, 20(5):499–545, 1979.
[49] Robert B Griffiths. Mathematical properties of renormalization-group trans- formations. Physica A: Statistical Mechanics and its Applications, 106(1- 2):59–69, 1981.
[50] Ulli Wolff. Collective monte carlo updating for spin systems. Physical Review Letters, 62(4):361, 1989.
[51] Nicolas Le Roux and Yoshua Bengio. Representational power of re- stricted boltzmann machines and deep belief networks. Neural computation, 20(6):1631–1649, 2008.
[52] Giacomo Torlai and Roger G Melko. Learning thermodynamics with boltz- mann machines. Physical Review B, 94(16):165134, 2016.
[53] HWJ Blöte, JR Heringa, A Hoogland, EW Meyer, and TS Smit. Monte carlo renormalization of the 3d ising model: Analyticity and convergence. Physical review letters, 76(15):2613, 1996.
[54] Martin Hasenbusch. Finite size scaling study of lattice models in the three- dimensional ising universality class. Physical Review B, 82(17):174433, 2010.
[55] GS Pawley, RH Swendsen, DJ Wallace, and KG Wilson. Monte carlo renormalization-group calculations of critical behavior in the simple-cubic ising model. Physical Review B, 29(7):4030, 1984.
[56] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014.
[57] R. H. Swendsen. Monte carlo renormalization. In Topics in Current Physics, pages 57–86. Springer Berlin Heidelberg, 1982.