臺灣博碩士論文加值系統

現今神經網路非常廣泛的被應用在各個領域。然而，大量的權重使其對記憶體的需求量大增，應用在嵌入式系統時需儲存在外部記憶體，且每個神經元在每次計算時都需要用到權重，意即運算時需要多次存取外部記憶體，增加大量的能耗。龐大的硬體成本及存取外部記憶體耗電的特性，使其應用在晶片設計時成本大幅提高。故此研究以團隊有的40奈米構音異常語音轉換系統晶片為研究平台進行架構及效能分析，分析出系統的瓶頸，重新排程並使效能提升38%，並提出以K-means量化方式在神經網路不重新訓練的前提下，能以對每層隱藏層或對每個神經元的方式對權重分群使參數壓縮，且品質損失較原先針對所有權重分群的方式有明顯改善，並綜合上述提到的分群方式及傳輸頻寬等條件，評估出最佳的設計方案。

Nowadays, neural networks are widely used in various fields. However, a large number of weights increase the requirement for memory, which needs to be stored in external memory when used in embedded systems. When each neuron calculating, they need to use weight in every computation, which means that the external memory needs to be accessed multiple times during the operation, it really increases energy consumption. The huge hardware cost and the power consumption of accessing external memory make the cost in chip design become significantly high. Therefore, this research uses 40nm dysarthria voice conversion SoC as a research platform, focus on architecture and performance analysis, find out the bottleneck of the system, and re-scheduling reduce 38% of conversion time. Without re-training neural network, I propose that per layer or per neuron K-means clustering can compress the parameters, and the quality loss is significantly improved compared with per network K-means. Considering the above-mentioned clustering method and transmission bandwidth and other conditions, the best design solution of DVC SoC is evaluated.

致謝 ii
Abstract iv
目錄 vi
圖目錄 ix
表目錄 xi
第一章 緒論 1
1.1. 研究動機 1
1.2. 研究貢獻 2
1.3. 論文架構 2
第二章 基於深層神經網路的構音異常語音轉換系統晶片 4
2.1. 構音異常語音轉換 4
2.2. 構音異常語音轉換演算法 6
2.3. 構音異常轉換系統晶片 10
2.4. 排程 16
2.5. 後續的優化課題 16
第三章 以DMA實現較佳排程 20
3.1. 加入DMA對DVC SoC排程的影響 21
3.2. 以frame為單位之DVC SoC排程 22
3.3. 以inference為單位之DVC SoC排程 25
3.4. 排程後續課題 31
第四章 以MI搭配權重量化實現高效率的外部記憶體存取 33
4.1. DNN計算與權重讀取 34
4.2. Flash讀取時間推導 35
4.3.可行的緩解方法 40
4.4.本文方法 44
4.5. K-means介紹 45
4.6. K-means在NN網路的應用 47
4.7. K-means最佳方案探討 50
第五章 優化結果 54
第六章 結論與未來展望 56
參考文獻 58

[1].S. Han, H. Mao, and W. J. Dally. (2016). Deep compression: Compressing
deep neural networks with pruning, trained quantization and Huffman coding [Online].
Available: https://arxiv.org/pdf/1510.00149.pdf
[2].K. Kobayashi and T. Toda, “Sprocket: open-source voice conversion software,” 2018
[3].R. L. MacDonald, et al., “Disordered speech data collection: lessons learned at 1 million utterances from project Euphonia,” in Proc. INTERSPEECH, 2021
[4].J. R. Green, et al., “Automatic speech recognition of disordered speech: personalized models outperforming human listeners on short phrases,” in Proc. INTERSPEECH, 2021
[5].T. -J. Lin et al., "A 40nm CMOS SoC for Real-Time Dysarthric Voice Conversion of Stroke Patients," in Proc. ASP-DAC, 2022, pp. 7-8.
[6].Y. H. Lai, et al., “A deep-learning-based voice conversion system for dysarthria speakers,” in Proc. ASHA, 2018
[7].吳政憲，2020。適用於構音異常語音轉換之低功耗神經網路加速器。碩士論文。嘉義，中正大學電機工程研究所
[8].王紹宇，2020。基於微控制器之構音異常語音轉換SoC。碩士論文。嘉義，中正大學電機工程研究所
[9].A. Zermini, et al., “Binaural and log-power spectra features with deep neural networks for speech-noise separation,” in Proc. MMSP, 2017
[10].M. Huang, “Development of taiwan mandarin hearing in noise test,” Department of speech language pathology and audiology, National Taipei University of Nursing and Health Science, 2005.
[11].D. Arthur, S. Vassilvitskii, “k-means++: The advantages of careful seeding," in
Proc. Symp. Discrete Algorithms, 2007.
[12].C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An algorithm for intelligibility prediction of time frequency weighted noisy speech,” in Proc. IEEE Trans. Audio, Speech, Lang, 2011.
[13]. MX25U1635E DATASHEET [Online].
Available: https://datasheetspdf.com/pdffile/792586/MACRONIX/MX25U1635E/1
[14].S. Seo and J. Kim, "Hybrid Approach for Efficient Quantization of Weights in Convolutional Neural Networks," in Proc. BigComp, 2018, pp. 638-641.
[15].Lei, Wang et al. “Compressing Deep Convolutional Networks Using K-means Based on Weights Distribution.” IIP'17 (2017).
[16].Y. Gong, L. Liu, M. Yang and L. Bourdev, “Compressing deep convolutional
networks using vector quantization,” in Proc. ICLR, 2015.
[17].E. Dupuis, D. Novo, I. O’Connor and A. Bosio, "Sensitivity Analysis and Compression Opportunities in DNNs Using Weight Sharing," in Proc. DDECS, 2020, pp. 1-6.
[18].S. Han, J. Pool, J. Tran, and W. J. Dally. (2015). Learning both weights and connections for efficient neural networks [Online].
Available: https://arxiv.org/pdf/1506.02626.pdf
[19].徐瑋程，2021。深層網路硬體之零值計算省略方法研究。碩士論文。嘉義，中正大學資訊工程研究所
[20].Van Leeuwen, Jan, “On the construction of huffman trees”. in Proc. ICALP, 1976, pp. 382–410.
[21].Denton, Emily, Zaremba, Wojciech, Bruna, Joan, LeCun, Yann, and Fergus, Rob. (2014). Exploiting linearstructure within convolutional networks for efficient evaluation[Online].
Avaliable：https://arxiv.org/pdf/1404.0736.pdf