跳到主要內容

臺灣博碩士論文加值系統

(18.97.14.80) 您好!臺灣時間:2025/01/25 23:19
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:范凱傑
研究生(外文):Fan, Kai-Jie
論文名稱:使用電阻式記憶體且具備時間步階調整的脈衝神經網路推論引擎
論文名稱(外文):RRAM-Based Time-Step Adjustable Spiking Neural Network Inference Engine
指導教授:侯拓宏
指導教授(外文):HOU, TUO-HUNG
口試委員:侯拓宏周世傑張添烜
口試委員(外文):Hou, Tuo-HungJou, Shyh-JyeChang, Tian-Sheuan
口試日期:2023-09-22
學位類別:碩士
校院名稱:國立陽明交通大學
系所名稱:電子研究所
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2023
畢業學年度:112
語文別:中文
論文頁數:59
中文關鍵詞:脈衝神經網路記憶體內運算非揮發式記憶體時間步階調整推 論引擎
外文關鍵詞:Spiking neural networkComputing-in-memorynon-volatile memorytime-step adjustable featureinference engine
相關次數:
  • 被引用被引用:0
  • 點閱點閱:7
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 The Internet-of-Things and Edge Computing . . . . . . . . . . . . . . . . . . . 1
1.2 The Emerging Hardware Architecture and Materials . . . . . . . . . . . . . . . 2
1.3 The Brain-Inspired Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 The Fundamentals of Spiking Neural Network . . . . . . . . . . . . . . . . . . 4
1.5 Hardware-Friendly SNN model . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Overview of Existing Hardware Architecture for SNN . . . . . . . . . . . . . . . 10
2.1 General category of SNN architecture . . . . . . . . . . . . . . . . . . . . . . 10
2.1.1 Large-scale Multi-core Neuromorphic Accelerator . . . . . . . . . . . 10
2.1.2 Low-power Single-core Inference Engine . . . . . . . . . . . . . . . . 11
2.2 Bionic Synapse Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Resistive-random-access-memory Operation . . . . . . . . . . . . . . 12
2.2.2 Single-ended and Differential Ternary Operation . . . . . . . . . . . . 14
2.3 Specialized feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.1 Current Amplifier and Voltage Clamper . . . . . . . . . . . . . . . . . 16
2.3.2 Other Common Variation of Feature . . . . . . . . . . . . . . . . . . . 17
2.4 LIF/IF neuron circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.1 Synchronous and Asynchronous Type of Neuron . . . . . . . . . . . . 20
2.4.2 The Leaky Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3 Proposed Spiking Neural Network Inference Engine . . . . . . . . . . . . . . . . 24
3.1 The Proposed Spiking Neural Network Hardware Implementation . . . . . . . 24
3.1.1 Membrane Threshold Setting and Current Accumulation . . . . . . . . 24
3.1.2 Temporal Dynamics of Synapse and Neuron Sensing Scheme . . . . . 25
3.2 Synchronous Neuron Design and the Layout Issues . . . . . . . . . . . . . . . 27
3.2.1 Design of Synchronous Neuron . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 Layout Limitation under CIM Architecture . . . . . . . . . . . . . . . 30
3.3 Time-step Adjustable Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3.1 Limitation of Membrane for Long Time-Step Operation . . . . . . . . 31
3.3.2 Membrane Detection Circuit . . . . . . . . . . . . . . . . . . . . . . . 32
3.4 The Floor Plan and the Rest of Peripheral Circuit . . . . . . . . . . . . . . . . 33
3.5 TNN Implementation with SNN Hardware . . . . . . . . . . . . . . . . . . . . 35
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4 Non-ideal Effect and the Analysis of Power and Area . . . . . . . . . . . . . . . . 37
4.1 The Variation Caused by the Neuron . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 Current Linearity Issue during MAC Operation . . . . . . . . . . . . . . . . . 39
4.3 Error Induced during the Process of Membrane Recharging . . . . . . . . . . . 41
4.4 Hardware Robustness Test by Practical Samples . . . . . . . . . . . . . . . . . 44
4.5 Analysis of Power and Area of the Proposed Macro . . . . . . . . . . . . . . . 46
4.5.1 Power Analysis of the Proposed Macro . . . . . . . . . . . . . . . . . 46
4.5.2 Area Analysis of the Proposed Macro . . . . . . . . . . . . . . . . . . 48
4.5.3 Benchmark of the Proposed Macro . . . . . . . . . . . . . . . . . . . 49
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2.1 Asynchronous Neuron under Long Time-step Operation . . . . . . . . 54
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
References
[1] L. Kong, J. Tan, J. Huang, G. Chen, S. Wang, X. Jin, P. Zeng, M. Khan, and S. K.
Das, “Edge-computing-driven internet of things: A survey,” ACM Comput. Surv., vol. 55,
no. 8, dec 2022. [Online]. Available: https://doi.org/10.1145/3555308
[2] S. J. Zidan, M.A. and W. Lu, “The future of electronics based on memristive systems,” in
Nat Electron 1, 2018, p. 22–29.
[3] W. Haensch, A. Raghunathan, K. Roy, B. Chakrabarti, C. M. Phatak, C. Wang, and
S. Guha, “Compute in-memory with non-volatile elements for neural networks: A
review from a co-design perspective,” Advanced Materials, vol. n/a, no. n/a, p. 2204944.
[Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/adma.202204944
[4] L. Deng, Y. Wu, X. Hu, L. Liang, Y. Ding, G. Li, G. Zhao, P. Li, and Y. Xie, “Rethinking
the performance comparison between snns and anns,” Neural Networks, vol. 121, pp.
294–307, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/
S0893608019302667
[5] E. Izhikevich, “Which model to use for cortical spiking neurons?” IEEE Transactions on
Neural Networks, vol. 15, no. 5, pp. 1063–1070, 2004.
[6] A. Basu, L. Deng, C. Frenkel, and X. Zhang, “Spiking neural network integrated circuits:
A review of trends and future directions,” in 2022 IEEE Custom Integrated Circuits Conference
(CICC). IEEE, 2022, pp. 1–8.
[7] D.-A. Nguyen, X.-T. Tran, and F. Iacopi, “A review of algorithms and hardware
implementations for spiking neural networks,” Journal of Low Power Electronics
and Applications, vol. 11, no. 2, 2021. [Online]. Available: https://www.mdpi.com/
2079-9268/11/2/23
[8] A. Singh, M. A. Lebdeh, A. Gebregiorgis, R. Bishnoi, R. V. Joshi, and S. Hamdioui, “Srif:
Scalable and reliable integrate and fire circuit adc for memristor-based cim architectures,”
IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 68, no. 5, pp. 1917–
1930, 2021.
[9] H. Jiang, S. Huang, X. Peng, and S. Yu, “Mint: Mixed-precision rram-based in-memory
training architecture,” in 2020 IEEE International Symposium on Circuits and Systems
(ISCAS), 2020, pp. 1–5.
[10] T. Xie, S. Yu, and S. Li, “A high-parallelism rram-based compute-in-memory macro with
intrinsic impedance boosting and in-adc computing,” IEEE Journal on Exploratory Solid-
State Computational Devices and Circuits, vol. 9, no. 1, pp. 38–46, 2023.
[11] C.-C. Liu, “Design of high-speed energy-efficient successive-approximation analog-todigital
converters,” D], National Cheng Kung University, Tainan, Taiwan, ROC, pp. 17–
32, 2010.
[12] L. Yao, M. Steyaert, and W. Sansen, “A 1-v 140-/spl mu/w 88-db audio sigma-delta modulator
in 90-nm cmos,” IEEE Journal of Solid-State Circuits, vol. 39, no. 11, pp. 1809–1818,
2004.
[13] S. Hussain, R. Kumar, and G. Trivedi, “Comparison of nmos and pmos input
driving dynamic comparator in 45nm technology,” IOP Conference Series: Materials
Science and Engineering, vol. 1020, no. 1, p. 012022, jan 2021. [Online]. Available:
https://dx.doi.org/10.1088/1757-899X/1020/1/012022
[14] J. Yeung and H. Mahmoodi, “Robust sense amplifier design under random dopant fluctuations
in nano-scale cmos technologies,” in 2006 IEEE International SOC Conference,
2006, pp. 261–264.
[15] B. Yan, Q. Yang, W.-H. Chen, K.-T. Chang, J.-W. Su, C.-H. Hsu, S.-H. Li, H.-Y. Lee,
S.-S. Sheu, M.-S. Ho, Q. Wu, M.-F. Chang, Y. Chen, and H. Li, “Rram-based spiking
nonvolatile computing-in-memory processing engine with precision-configurable in situ
nonlinear activation,” in 2019 Symposium on VLSI Technology, 2019, pp. T86–T87.
[16] M. Bouvier, A. Valentian, T. Mesquida, F. Rummens, M. Reyboz, E. Vianello, and
E. Beigne, “Spiking neural networks hardware implementations and challenges: A
survey,” J. Emerg. Technol. Comput. Syst., vol. 15, no. 2, apr 2019. [Online]. Available:
https://doi.org/10.1145/3304103
[17] E. Painkras, L. A. Plana, J. Garside, S. Temple, F. Galluppi, C. Patterson, D. R. Lester,
A. D. Brown, and S. B. Furber, “Spinnaker: A 1-w 18-core system-on-chip for massivelyparallel
neural network simulation,” IEEE Journal of Solid-State Circuits, vol. 48, no. 8,
pp. 1943–1953, 2013.
[18] F. Akopyan, J. Sawada, A. Cassidy, R. Alvarez-Icaza, J. Arthur, P. Merolla, N. Imam,
Y. Nakamura, P. Datta, G.-J. Nam, B. Taba, M. Beakes, B. Brezzo, J. B. Kuang,
R. Manohar, W. P. Risk, B. Jackson, and D. S. Modha, “Truenorth: Design and tool flow
of a 65 mw 1 million neuron programmable neurosynaptic chip,” IEEE Transactions on
Computer-Aided Design of Integrated Circuits and Systems, vol. 34, no. 10, pp. 1537–
1557, 2015.
[19] M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi,
N. Imam, S. Jain, Y. Liao, C.-K. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul,
J. Tse, G. Venkataramanan, Y.-H. Weng, A. Wild, Y. Yang, and H. Wang, “Loihi: A neuromorphic
manycore processor with on-chip learning,” IEEE Micro, vol. 38, no. 1, pp.
82–99, 2018.
[20] P. Jain, U. Arslan, M. Sekhar, B. C. Lin, L. Wei, T. Sahu, J. Alzate-vinasco, A. Vangapaty,
M. Meterelliyoz, N. Strutt, A. B. Chen, P. Hentges, P. A. Quintero, C. Connor, O. Golonzka,
K. Fischer, and F. Hamzaoglu, “13.2 a 3.6mb 10.1mb/mm2 embedded non-volatile reram
macro in 22nm finfet technology with adaptive forming/set/reset schemes yielding down
to 0.5v with sensing time of 5ns at 0.7v,” in 2019 IEEE International Solid- State Circuits
Conference - (ISSCC), 2019, pp. 212–214.
[21] J.-H. Yoon, M. Chang, W.-S. Khwa, Y.-D. Chih, M.-F. Chang, and A. Raychowdhury, “A
40-nm 118.44-tops/w voltage-sensing compute-in-memory rram macro with write verification
and multi-bit encoding,” IEEE Journal of Solid-State Circuits, vol. 57, no. 3, pp.
845–857, 2022.
[22] S.-T. Wei, B. Gao, D. Wu, J.-S. Tang, H. Qian, and H.-Q. Wu, “Trends and challenges in
the circuit and macro of rram-based computing-in-memory systems,” Chip, vol. 1, no. 1,
p. 100004, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/
S2709472322000028
[23] Q. Liu, B. Gao, P. Yao, D. Wu, J. Chen, Y. Pang, W. Zhang, Y. Liao, C.-X. Xue, W.-H.
Chen, J. Tang, Y. Wang, M.-F. Chang, H. Qian, and H. Wu, “33.2 a fully integrated analog
reram based 78.4tops/w compute-in-memory chip with fully parallel mac computing,” in
2020 IEEE International Solid- State Circuits Conference - (ISSCC), 2020, pp. 500–502.
[24] S. D. Spetalnick, M. Chang, B. Crafton, W.-S. Khwa, Y.-D. Chih, M.-F. Chang, and A. Raychowdhury,
“A 40nm 64kb 26.56tops/w 2.37mb/mm2rram binary/compute-in-memory
macro with 4.23x improvement in density and >75of sensing dynamic range,” in 2022
IEEE International Solid- State Circuits Conference (ISSCC), vol. 65, 2022, pp. 1–3.
[25] J. Shamsi, K. Mohammadi, and S. B. Shokouhi, “A low power circuit of a leaky integrate
and fire neuron with global reset,” in 2017 Iranian Conference on Electrical Engineering
(ICEE), 2017, pp. 366–369.
[26] M. Zare, E. Zafarkhah, and N. S. Anzabi-Nezhad, “An area and energy efficient lif
neuron model with spike frequency adaptation mechanism,” Neurocomputing, vol. 465,
pp. 350–358, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/
pii/S0925231221013588
[27] L. Han, P. Huang, Y. Wang, Z. Zhou, Y. Zhang, X. Liu, and J. Kang, “Efficient discrete
temporal coding spike-driven in-memory computing macro for deep neural network based
on nonvolatile memory,” IEEE Transactions on Circuits and Systems I: Regular Papers,
vol. 69, no. 11, pp. 4487–4498, 2022.
[28] R. Mao, B. Wen, M. Jiang, J. Chen, and C. Li, “Experimentally-validated crossbar model
for defect-aware training of neural networks,” IEEE Transactions on Circuits and Systems
II: Express Briefs, vol. 69, no. 5, pp. 2468–2472, 2022.
[29] H. M. Lehmann, J. Hille, C. Grassmann, and V. Issakov, “Leaky integrate-and-fire neuron
with a refractory period mechanism for invariant spikes,” in 2022 17th Conference on Ph.D
Research in Microelectronics and Electronics (PRIME), 2022, pp. 365–368.
[30] A. K. Sharma, M. Madhusudan, S. M. Burns, P. Mukherjee, S. Yaldiz, R. Harjani, and S. S.
Sapatnekar, “Common-centroid layouts for analog circuits: Advantages and limitations,”
in 2021 Design, Automation Test in Europe Conference Exhibition (DATE), 2021, pp.
1224–1229.
[31] G. Wegmann, E. Vittoz, and F. Rahali, “Charge injection in analog mos switches,” IEEE
Journal of Solid-State Circuits, vol. 22, no. 6, pp. 1091–1097, 1987.
[32] W. Xu and E. Friedman, “Clock feedthrough in cmos analog transmission gate switches,”
in 15th Annual IEEE International ASIC/SOC Conference, 2002, pp. 181–185.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊