臺灣博碩士論文加值系統

數位波導理論（Digital Waveguide Filters）已被應用於樂器聲音合成許多年，然其在設計濾波器係數以合成特定樂器之音色需要耗費相當大的計算量。在這方面，隨著神經網路的發展，一種結合機器學習的聲源與濾波器建模法（Source-filter model）在合成小提琴複雜且多樣化的演奏技巧上，已有非常著眼的表現。然而，此方法缺少對任何吹奏樂器都非常重要的雜音，以及氣音的合成。因此，在本論文中，我們以吹管樂器中的法國號及單簧管為例，根據數位波導理論為基礎來模擬管樂聲音的合成，符合吹管樂器的物理行為，並且應用長短期記憶網路模型（Long short-term memory, LSTM）預測重要參數，盡可能保留管樂應有的音色，充分結合了兩種合成方法的優勢。另外我們亦根據參數調整合成出具有不同雜音程度的聲音。在本文實驗中，我們以Real World Computing（RWC）數據庫中的預錄音檔作為訓練資料，來呈現我們方法的成效。結果顯示我們不僅能透過更動聲源（source）輕易地改變音高（pitch）與音長（duration），且能將聲音的動態變化即時呈現，亦能合成出伴隨管樂演奏的氣音，在合成處理計算效率高之下，聲音變得豐富、逼真。

Digital Waveguide Filter (DWF) has been applied to the synthesis of wind instruments for many years. To design the filter coefficients of synthesizing the timbre of a particular instrument usually takes lots of time and effort. In this respect, a novel method of combining source-filter model and a recurrent neural network using Long short-term memory (LSTM) for the synthesis of the violin has been successfully proposed. However, it lacks concerning the synthesis of the noise part which is important for playing a wind instrument. In this study, we use the DWF for simulating the noise part and apply the LSTM based source-filter model to preserve the timbre of the sound. We synthesize the sound of French horn and clarinet to demonstrate this work. The pre-recording tones from the Real-World Computing (RWC) database are used in our experiment. The result of the sound we synthesized is close to the real tone and accompanied by a pitch-synchronized noise. By altering the source, the pitch and duration of the sound we synthesized can be changed easily. Furthermore, the proposed method can be effectively implemented to different musical instruments and various playing techniques. It is proposed this synthesis method be applied to the VST plugin.

摘要 i
Abstract ii
Acknowledgements iii
Table of Contents iV
List of Figures V

Chapter 1. Introduction 1

Chapter 2. Related Work 3
2.1. Wind instrument 3
2.1.1. The French horn 3
2.1.2. The Clarinet 5
2.1.3. Common Synthesis Method 5
2.2. Physical Modeling in Wind Instrument 8
2.2.1. Digital Waveguide Filter 8
2.2.2. Physical Characteristic of Reed 11
2.3. Source-Filter Model Synthesis Using Recurrent Neural Networks 14
2.3.1. Source-Filter Model 14
2.3.2. Long Short-Term Memory Neural Network 15
2.3.3. LSTM Time-Varying Source-Filter Model 17

Chapter 3. Method 19
3.1. The Proposed Synthesis System 19
3.2. The Time-Varying Filter 22
3.3. The Synthesis of Blowing Noise 23
3.4. The Influence of the Reed Table 26

Chapter 4. Result 28
4.1. Experiments Setup 28
4.2. Synthesis Results 29
4.3. Discussion 36

Chapter 5. Conclusion 39
5.1. Conclusion 39
5.2. Future Works 40
References 41

[1] J.M.Chowning, "The synthesis of complex audio spectra by means of frequency modulation," Journal of the audio engineering society, vol. 21, no. 7, pp. 526–534, 1973.
[2] R. Bristow-Johnson, “Wavetable synthesis 101, a fundamental perspective,” in Audio Engineering Society Convention 101, Audio Engineering Society, 1996.
[3] E. D. Scheirer and L. Ray, “Algorithmic and wavetable synthesis in the MPEG-4 multimedia standard,” in Audio Engineering Society Convention 105, Audio Engineering Society, 1998.
[4] S.C.BassandT.W.Goeddel, "The efficient digital implementation of subtractive music synthesis," IEEE Micro, vol. 1, no. 3, pp. 24–37, 1981.
[5] T. G. Carpenter, “Developing an audio unit plugin using a digital waveguide model of a wind instrument,” 2012.
[6] S. Ystad, “Sound modeling applied to flute sounds,” Journal of the Audio Engineering Society, vol. 48, no. 9, pp. 810–825, 2000.
[7] J. Engel, C. Resnick, A. Roberts, S. Dieleman, M. Norouzi, D. Eck, and K. Simonyan, “Neural audio synthesis of musical notes with WaveNet autoencoders,” in Proceedings of the 34th International Conference on Machine Learning (D. Precup and Y. W. Teh, eds.), vol. 70 of Proceedings of Machine Learning Research, (International Convention Centre, Sydney, Australia), pp. 1068–1077, PMLR, 06–11 Aug 2017.
[8] A. Cichocki, R. Unbehauen, and R. W. Swiniarski, Neural networks for optimization and signal processing, vol. 253. Wiley New York, 1993.
[9] A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in 2013 IEEE international conference on acoustics, speech and signal processing, pp. 6645–6649, Ieee, 2013.
[10] A.SarroffandM.A.Casey, “Musical audio synthesis using autoencoding neural nets,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR2014), International Society for Music Information Retrieval, 2014.
[11] H.-C. Yang, Y. Lin, and A. Su, “A novel source filter model using lstm/k-means machine learning methods for the synthesis of bowed-string musical instruments,” in Audio Engineering Society Convention 148, Audio Engineering Society, 2020.
[12] YAMAHA, “Musical instrument guide,”
[13] A. Röbel, “Adaptive additive synthesis of sound,” Proceedings of the ICMC, vol. 2, no. 1, p. 6, 1999.
[14] V. Välimäki, “Physics-based modeling of musical instruments,” Acta Acustica united with Acustica, vol. 90, no. 4, pp. 611–617, 2004.
[15] A. Chaigne and A. Askenfelt, “Numerical simulations of piano strings. i. a physical model for a struck string using finite difference methods,” The Journal of the Acoustical Society of America, vol. 95, no. 2, pp. 1112–1118, 1994.
[16] D.P.Berners, "Acoustics and signal processing techniques for physical modeling of brass instruments." Ph.D. thesis, Citeseer, 1999.
[17] R.L.Harrison-Harsley, “Physical modeling of brass instruments using finite-difference time-domain methods,” 2018.
[18] J. O. Smith, “Physical modeling using digital waveguides,” Computer music journal, vol. 16, no. 4, pp. 74–91, 1992.
[19] A. W. Su and L. San-Fu, “Synthesis of plucked-string tones by physical modeling with recurrent neural networks,” in Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing, pp. 71–76, IEEE, 1997.
[20] S.-F. Liang, A. W. Su, and C.-T. Lin, “Model-based synthesis of plucked string instruments by using a class of scattering recurrent networks,” IEEE transactions on neural networks, vol. 11, no. 1, pp. 171–185, 2000.
[21] A. W. Su and L. San-Fu, “Physical modeling of acoustic instrument and its application on music synthesis,” 1995.
[22] G. P. Scavone, An acoustic analysis of single-reed woodwind instruments with an emphasis on design and performance issues and digital waveguide modeling techniques. Ph.D. thesis, Citeseer, 1997.
[23] C. S. Kao, “Piano synthesis and sustain pedal simulation based on digital waveguide,” 2020.
[24] N. H. Fletcher et al., “Nonlinear dynamics and chaos in musical instruments,” Complexity International, vol. 1, pp. 106–117, 1994.
[25] Z. Zhang, “Cause-effect relationship between vocal fold physiology and voice production in a three-dimensional phonation model,” The Journal of the Acoustical Society of America, vol. 139, no. 4, pp. 1493–1507, 2016.
[26] J. Pekonen, “Computationally efficient music synthesis–methods and sound design,”
Master of Science (Technology) thesis, TKK Helsinki University of Technology, Espoo, Finland, 2007.
[27] A. Klapuri, “Analysis of musical instrument sounds by source-filter-decay model,” in 2007 IEEE International Conference on Acoustics, Speech and Signal Processing- ICASSP’07, vol. 1, pp. I–53, IEEE, 2007.
[28] H. Hahn, A. Röbel, J. J. Burred, and S. Weinzierl, “Source-filter model for quasi- harmonic instruments,” in Digital Audio Effects (DAFx), pp. 1–1, 2010.
[29] M. Flohberger, “Source/filter–model,” Acoustic tube models.-Linear prediction.- Formant Synthesizer, Technische Unversität Graz, 2003.
[30] C. Vahidi, G. Fazekas, C. Saitis, and A. Palladini, “Timbre space representation of a subtractive synthesizer,” arXiv preprint arXiv:2009.11706, 2020.
[31] S.You-Cheng, “Detection of GTTM local boundary rules by using algorithmically generated music scores,” in Master's thesis, National Cheng Kung University, 2020.
[32] B. Thoshkahna, F. X. Nsabimana, and R. R. Kalpathi, “A transient detection algorithm for audio using iterative analysis of stft.,” in ISMIR, pp. 203–208, 2011.
[33] A. Freed, “Real-time inverse transform additive synthesis for additive and pitch synchronous noise and sound spatialization,” in Audio Engineering Society Convention 105, Audio Engineering Society, 1998.
[34] M. Goto et al., “Development of the rwc music database,” in Proceedings of the 18th International Congress on Acoustics (ICA 2004), vol. 1, pp. 553–556, 2004.