臺灣博碩士論文加值系統

人們對於生物體內如何運作很感興趣，因此數學家利用數學方式建立類神經網路演算法，來模仿生物體內神經元之間互相連結和傳遞訊息，最後利用類神經網路得到代表學習訊號的參數，拿它來做訊號辨識或分類；而此論文所要探討的內容：探討擴散網路(Diffusion Network)[1]之學習演算法實現於類比積體電路之可行性。
首先是修改和簡化擴散網路之學習演算法，修改學習演算法是為了易於實現成類比積體電路，簡化學習演算法的目標是為了讓電路較好設計，而學習理論部份最重要的是應用蒙地卡羅期望值最大化(Monte Carlo Expectation Maximization)來得到代表待學習訊號的擴散網路參數，接著再利用最佳化方法修改學習演算法，經過修改後之學習演算法可利用模擬軟體(MATLAB)來驗證，藉由不同的訓練資料來找尋擴散網路參數的範圍和解析度，來了解擴散網路要成功學習訊號，對於精確度要求程度高低，這些前置模擬工作對於電路規格有很大的關連性；接著探討電路實現之可行性，最後再利用實驗室虛擬儀器工程平（Laboratory Virtual Instrumentation Engineering Workbench）來模擬擴散網路學習過程。

People have always been interested in how neurons work in organisms, and therefore mathmeaticans built neural network algorithms with the use of mathematical method. A neural network algorithm can mimic behaviors that connect and communicate messages between neurons in organisms. By using neural network, we got the learning signal’s representative parameters that can apply to recognize and classify different signals. The theme of the thesis is ‘exploring the feasibility of training diffusion network with on-chip circuitry’.
First, modify and simplify the training algorithm of Diffusion Network. The modification of training algorithm of the diffusion network is for an easier implementation in analog integrated circuit. On the other hand, simplifying the training algorithm of the diffusion network enables the design of analog integrated circuit to become easier. The most important part of training theory is getting the parameters of the training signal by using the algorithm, ‘Monte Carlo Expectation Maximization.’ The training algorithm is modified by optimization method and can be verified by the mathematic simulation software (MATLAB). Searching the range and resolution of the parameters in the different training signals helps us to understand how necessary the accuracy of parameter is for the diffusion network to be successful. The simulation work is closely related to the circuit specification, then the study explores the feasibility of training diffusion network learning processes by the Laboratory Virtual Instrumentation Engineering Workbench (LABVIEW).

誌謝 I
摘要 II
Abstract III
章節目錄 IV
圖目錄 VI
表目錄 IX
第一章 內容介紹 1
1.1 研究動機與目標 2
1.2 研究結果貢獻 2
1.3 章節簡介 3
第二章 相關文獻回顧 4
2.1 擴散網路模型 4
2.2 擴散網路學習理論 7
2.2.1 蒙地卡羅權重取樣法 7
2.2.2 蒙地卡羅期望值最大化學習法 7
2.3 擴散網路重建理論 13
2.4 最佳化方法 14
2.5 類神經網路演算法實現於類比積體電路之考量 17
2.6 總結 18
第三章 基於易實現積體電路修改擴散網路學習演算法 19
3.1固定步階梯度驟降擴散網路演算法 19
3.1.1 利用梯度驟降法取得搜尋方向與定出步階尺寸 19
3.1.2 簡化固定步階梯度驟降擴散網路演算法 22
3.1.3 原擴散網路與固定步階梯度驟降擴散網路之演算法流程 23
3.2顯示固定步階梯度驟降演算法之模擬過程 28
3.2.1 學習參數無限制時之模擬過程 28
3.2.2 學習參數有限制時之模擬過程 33
3.3固定步階梯度驟降演算法學習訊號時學習參數之限制範圍和解析度 38
3.3.1 學習正弦波之學習參數的初始值、限制範圍與解析度 38
3.3.2 學習心跳圖之學習參數的初始值、限制範圍與解析度 45
3.3.3 學習希臘字母之學習參數的初始值、限制範圍與解析度 50
3.3.4 綜合以上條件列出學習參數限制表 54
3.4 總結 56
第四章 積分器電路之限制和準確度與擴散網路學習理論之關係 57
4.1 積分器電路架構 57
4.1.1 架構一：混合式電晶體差動對數域積分器 57
4.1.2 架構二：階層壓縮式對數域積分器 60
4.1.3 架構三：全差動電流模式積分 61
4.2 積分器電路之限制和準確度 63
4.3 總結 75
第五章 利用實驗室虛擬儀器工程平台來模擬擴散網路學習過程 77
5.1 擴散網路演算法轉換成Labview圖像型程式 77
5.2利用Labview控制波形產生器與訊號擷取器來模擬學習過程 77
5.2.1 實驗方式與學習過程 78
5.2.2 學習結果 79
5.3 總結 83
第六章 結論 84
6.1 研究總結 84
6.2 研究未來發展方向 85
參考文獻 87

[1] J. R. Movellan, P. Mineiro, and R. J. Williams, "A Monte Carlo EM approach for partially observable diffusion processes: Theory and applications to neural networks," Neural Computation, vol. 14, no. 7, pp. 1507-1544, 2002.

[2] H. Chen and A. F. Murray, "Continuous restricted Boltzmann machine with an implementable training algorithm," IEE Proceedings-Vision Image and Signal
Processing, vol. 150, no. 3, pp. 153-158, 2003.

[3] Y.-S. Hsu, "Biomedical Signal Recognition Using Diffusion Networks."
Master's Thesis of National Tsing Hua University,Taiwan, 2007.

[4] Edwin K. P. Chong and Statislaw H. Zak , An Introduction to Optimization, 2nd Edition, 2001.

[5] G. Cauwenberghs, "An analog VLSI recurrent neural network learning a
continuous-time trajectory," IEEE Transactions on Neural Networks, vol. 7, pp. 346-361, 1996.

[6] Antonio J. Montalvo, Ronald S. Gyurcsik, and John J. Paulos, "An analog VLSI neural network with on-chip perturbation learning," IEEE journal of solid-state circuits, vol. 32, pp. 535-543, 1997.

[7] G. Cauwenberghs and M. Bayoumi, Learning on silicon: Adaptive VLSI neural systems: Kluwer Academic Publishers, 1999.

[8] Mirko Gravati, Maurizio Valle, Giuseppe Ferri, Nicola Guerrini, Linder Reyes, " A novel current-mode very low power analog CMOS four quadrant multiplier, " In: Proceedings of ESSCIRC, Grenoble, France, 0-7803-9205 IEEE, 2005.

[9] G. Cauwenberghs, "An analog VLSI recurrent neural network learning a continuous-time trajectory," IEEE Transactions on Neural Networks, vol. 7, pp. 346-361, 1996.


[10] M. Valle, "Analog VLSI implementation of artificial neural networks with Supervised On-Chip Learning," Analog Integrated Circuits and Signal Processing, vol. 33, pp. 263-287, 2002.

[11] C.-H. Chien, "A Stochastic System on a Chip Basing on the Diffusion Networks." Master's Thesis of National Tsing Hua University,Taiwan, 2008.

[12] Y. Tsividis, "Externally linear integrators," IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 45, pp. 1181-1187, 1998.

[13] G. Groenewold, "Optimal dynamic range integrators," IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 39, pp. 614-627, 1992..

[14] J. Moreira and M. Silva, "Limits to the Dynamic Range of Low-Power Continuous-Time Integrators," IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 48, p. 805-817, 2001.

[15] E. El-Masry and J. Wu, "Fully Differential Class-AB Log-Domain Integrator," Analog Integrated Circuits and Signal Processing, vol. 25, pp. 35-46, 2000.

[16] Ippei Akita, Kazuyuki Wada, Yoshiaki Tadokoro, "A 0.6-V Dynamic Biasing Filter With 89-dB Dynamic Range in 0.18-μm CMOS," IEEE journal of
solid-state circuits, vol. 44, pp. 2790-2799, 2009.

[17] Ippei Akita, Kazuyuki Wada, and Yoshiaki Tadokoro, "Simplified Low-Voltage
CMOS Syllabic Companding Log Domain Filter," IEEE international Symposium on Circuits and Sysytmes, pp. 2244-2247, May 2007.

[18] S. L. Smith, and E. Sánchez-Sinencio, “Low voltage integrators for high-frequency CMOS filters using current mode techniques,” IEEE Trans. on Circuits Syst. II: Analog and Digital Signal Processing, vol. 43, no. 1, Jan. 1996, pp. 39 – 48