(54.236.58.220) 您好!臺灣時間:2021/03/05 05:17
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

我願授權國圖
: 
twitterline
研究生:沈于琳
研究生(外文):Shen, Yu-Lin
論文名稱:以電阻式記憶體突觸實現雙層感知器硬體與設計考量
論文名稱(外文):Hardware Implementation and Design Consideration of Two-layer Perceptron using RRAM Synapse
指導教授:侯拓宏
指導教授(外文):Hou, Tuo-Hung
口試委員:張添烜
口試委員(外文):Chang, Tian-Sheuan
口試日期:2017-10-12
學位類別:碩士
校院名稱:國立交通大學
系所名稱:電子研究所
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2017
畢業學年度:106
語文別:英文
論文頁數:46
中文關鍵詞:多層倒傳遞神經網路類比電阻式記憶體
外文關鍵詞:multi-layer back-propagationanalog resistive random access memory
相關次數:
  • 被引用被引用:0
  • 點閱點閱:430
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
本論文探討實現即時學習(online-trained)多層倒傳遞硬體神經網路(multi- layer back-propagation hardware neural network)的方法與在影像辨識之應用。目前利用軟體運算的類神經網路應用如影像、語音辨識等已逐漸成熟,但隨著資料量的大幅增加,軟體運算所需的高能耗可能造成其發展之障礙,目前有許多研究團隊已著手進行硬體神經網路的研究。在硬體神經網路中最關鍵的電子突觸元件(synaptic device)部分,類比電阻式記憶體(analog resistive random-access memory, RRAM)是最佳的候選人,因其結構簡單,可以達成高密度陣列;且其可調的阻值特性,可以簡化演算法的運算設計。在本論文所實現的硬體神經網路架構中,包括以leaky integrate-and-fire電路作為神經元、電阻式記憶體作為突觸,決定神經元之間的聯結強弱,而其強弱趨勢則由FPGA依照倒傳遞演算法以梯度下降方式(gradient descent method)進行更新。本研究有效結合元件特性量測與建模、電路板設計、演算法模擬,系統整合測試等跨域能力,驗證一元件到系統的完整設計流程。
  我們透過倒傳遞演算法中各項參數調整,配合既有電阻式記憶體元件的模型,可有效減少類比電阻式記憶體非理想特性以及電路誤差對辨識結果的影響。最後在本研究中,我們實現了兩層的倒傳遞神經網路的硬體架構,並展現正確的辨識成果。值得一提的是本研究中的兩層倒傳遞神經網路硬體因其複雜性,是第一次被成功實現,相信這將成為未來更深層的仿生運算研究與應用之重要參考。
This thesis discusses the implementation of online-trained multilayer back-propagation hardware neural network and its application in image recognition. At present, applications of software-based neural network such as image and speech recognition have gradually turned mature. However, because of the substantial increase in the amount of data, software computing required high energy consumption, which may become the major development obstacle. There are numerous research efforts have been dedicated to energy-efficient hardware neural networks in order to solve this issue. Analog resistive random-access memory (RRAM) is the best candidate for the most important electronic synaptic device in the hardware neural network. Because of its simple structure, RRAM-based synaptic device can be integrated into high density arrays. Moreover, its adjustable resistance values can simplify the algorithm design. In this thesis, the hardware network architecture includes leaky integrate-and-fire (LIF) circuits as neurons and RRAM as synapses to present the connecting strength between neurons. The conductance of RRAM is updated according to the gradient descent method implemented using a FPGA. This study accomplished a complete design flow from single components to the complete system by combining several cross-layer design capabilities, such as component characteristic measurement and modeling, printed circuit board design, algorithm simulation, and system test platform.
By adjusting the parameters in the algorithm, we can effectively reduce the non-ideal characteristics of the analog RRAM device and circuit variations on the final image classification result. Finally, we implemented two-layer back-propagation neural network hardware and show correct multi-class classification results. It is worth mentioning that this study shows the first demonstration of two-layer back-propagation neural network hardware, which is known to be difficult to realize because of the complexity. We believe that this study could serve as an important reference for the future research and applications of deep neuromorphic computing.
中文摘要 iii
Abstract iv
誌謝 vi
Contents vii
Figure Captions x
LIST OF TABLES xiii
Chapter 1 Introduction 1
1.1 Brain-inspired computing application 1
1.2 Limitation of von Neumann architecture 1
1.3 Hardware artificial neural network 2
1.3.1 STDP characteristic on RRAM 2
1.3.2 Waveform for weight update 4
1.3.3 One-layer back-propagation on RRAM 5
1.4 Motivation 6
Chapter 2 Two-layer back-propagation perceptron design 7
2.1 Introduction 7
2.1.1 Weight sum 7
2.1.2 Weight update according to gradient descent method 8
2.1.3 Hardware system introduction 9
2.2 Time diagram 11
2.3 Feed forward read 12
2.3.1 Read mode circuit 14
2.3.2 Analog input and weighted sum 15
2.3.3 Dummy and activation function 16
2.4 Back propagation read 17
2.4.1 LIF circuit implementation 19
2.4.2 Shared integrator and integrator all output in read parts 20
2.5 Weight update 21
2.5.1 Write mode time diagram 23
2.6 Summary 25
Chapter 3 Non-ideal effect on hardware design 27
3.1 Synaptic devices – RRAM 27
3.2 Neuron precision 28
3.2.1 Discharge time influence 28
3.3 Parasitic capacitance 29
3.4 Disturbance 30
3.4.1 Monitor training process to verify unexpected effect 30
3.4.2 Modify update waveform for decreasing disturbance 31
3.4.3 Monitor training process of modified waveform 32
3.4.4 Further monitor with analog multiplication of modified waveform 34
3.5 Summary 36
Chapter 4 Implementation discussion 37
4.1 Fixed parameter 37
4.1.1 Activation function scaling and binary threshold 37
4.1.2 Input pattern and corresponding weight map 38
4.2 Learning rate 40
4.3 Bounded activation function 40
4.4 Dummy value 41
4.5 Summary 43
Chapter 5 Conclusion and future work 44
References 45
[1] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
[2] J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431-3440.
[3] C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling," IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1915-1929, 2013.
[4] S. Ji, W. Xu, M. Yang, and K. Yu, "3D convolutional neural networks for human action recognition," IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 1, pp. 221-231, 2013.
[5] D. Ferrucci et al., "Building Watson: An overview of the DeepQA project," AI magazine, vol. 31, no. 3, pp. 59-79, 2010.
[6] Y. Taigman, M. Yang, M. A. Ranzato, and L. Wolf, "Deepface: Closing the gap to human-level performance in face verification," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1701-1708.
[7] D. Silver et al., "Mastering the game of Go with deep neural networks and tree search," Nature, vol. 529, no. 7587, pp. 484-489, 2016.
[8] W. Kuzmicz, "The future of CMOS: More Moore or the next big thing?," in Mixed Design of Integrated Circuits and Systems, 2017 MIXDES-24th International Conference, 2017, pp. 21-26: IEEE.
[9] Wang, A., et al., "More-Than-Moore: 3D heterogeneous integration into CMOS technologies." Nano/Micro Engineered and Molecular Systems (NEMS), 2017 IEEE 12th International Conference on. IEEE, 2017.
[10] A. Wang et al., "More-Than-Moore: 3D heterogeneous integration into CMOS technologies," in Nano/Micro Engineered and Molecular Systems (NEMS), 2017 IEEE 12th International Conference on, 2017, pp. 1-4: IEEE.
[11] J. Hsu, "Ibm's new brain [news]," IEEE spectrum, vol. 51, no. 10, pp. 17-19, 2014.
[12] B. V. Benjamin et al., "Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations," Proceedings of the IEEE, vol. 102, no. 5, pp. 699-716, 2014.
[13] J. Schemmel, D. Briiderle, A. Griibl, M. Hock, K. Meier, and S. Millner, "A wafer-scale neuromorphic hardware system for large-scale neural modeling," in Circuits and systems (ISCAS), proceedings of 2010 IEEE international symposium on, 2010, pp. 1947-1950: IEEE.
[14] S. Scholze et al., "A 32GBit/s communication SoC for a waferscale neuromorphic system," Integration, the VLSI Journal, vol. 45, no. 1, pp. 61-75, 2012.
[15] S. B. Furber, F. Galluppi, S. Temple, and L. A. Plana, "The spinnaker project," Proceedings of the IEEE, vol. 102, no. 5, pp. 652-665, 2014.
[16] G.-q. Bi and M.-m. Poo, "Synaptic modification by correlated activity: Hebb's postulate revisited," Annual review of neuroscience, vol. 24, no. 1, pp. 139-166, 2001.
[17] 黃國源 (2016)。 類神經網路。新北市: 全華圖書。
[18] I.-T. Wang, Y.-C. Lin, Y.-F. Wang, C.-W. Hsu, and T.-H. Hou, "3D synaptic architecture with ultralow sub-10 fJ energy per spike for neuromorphic computation," in Electron Devices Meeting (IEDM), 2014 IEEE International, 2014, pp. 28.5. 1-28.5. 4: IEEE.
[19] S. Lashkare, N. Panwar, P. Kumbhare, B. Das, and U. Ganguly, "PCMO-Based RRAM and NPN Bipolar Selector as Synapse for Energy Efficient STDP," IEEE Electron Device Letters, vol. 38, no. 9, pp. 1212-1215, 2017.
[20] A. Amirsoleimani, M. Ahmadi, and A. Ahmadi, "STDP-based unsupervised learning of memristive spiking neural network by Morris-Lecar model," in Neural Networks (IJCNN), 2017 International Joint Conference on, 2017, pp. 3409-3414: IEEE.
[21] M. Chu et al., "Neuromorphic hardware system for visual pattern recognition with memristor array and CMOS neuron," IEEE Transactions on Industrial Electronics, vol. 62, no. 4, pp. 2410-2419, 2015.
[22] F. Alibart, E. Zamanidoost, and D. B. Strukov, "Pattern classification by memristive crossbar circuits using ex situ and in situ training," Nature communications, vol. 4, p. 2072, 2013.
[23] S. Park et al., "RRAM-based synapse for neuromorphic system with pattern recognition function," in Electron Devices Meeting (IEDM), 2012 IEEE International, 2012, pp. 10.2. 1-10.2. 4: IEEE.
[24] X. Wu, V. Saxena, and K. Zhu, "Homogeneous spiking neuromorphic system for real-world pattern recognition," IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 5, no. 2, pp. 254-266, 2015.
[25] Chou, T., “Implementation of hardware neural network system based on analog resistive artificial synapse,” M.S. thesis, NCTU, 2016.
[26] L. Gao et al., "Fully parallel write/read in resistive synaptic array for accelerating on-chip learning," Nanotechnology, vol. 26, no. 45, p. 455204, 2015.
電子全文 電子全文(網際網路公開日期:20221012)
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關論文
 
無相關期刊
 
無相關點閱論文
 
系統版面圖檔 系統版面圖檔