臺灣博碩士論文加值系統

在智慧型感知系統中，觀察訊號混合了許多未知的獨立訊號，因此，如何將未知獨立訊號有效分離出來是機器學習領域中的重要課題，而獨立成份分析(independent component analysis, ICA)是被廣泛應用於未知訊號分離(blind source separation, BSS)的問題上。在標準獨立成份分析中，假設所有的未知獨立訊號都是處於穩定的程序(Stationary Process)，然而，未知訊號可能會動態的改變位置或者處於啟動(active)以及未啟動(inactive)的狀態，因此標準的獨立成份分析在實際應用上是受到限制。為了在智慧型感知系統中提供更有效的訊號分離的效果，本研究擬提出非穩定性貝氏獨立成份分析演算法，透過線上貝氏學習機制，達到即時估測非穩定性未知訊號的啟動狀態與個數，並且改進文獻中相關演算法的計算複雜度。再者，在智慧型感知系統中，除了要能正確分離混合訊號之外，相關的應用，例如：語者認證、語音辨識及以語者分割與分群系統等應用上，如何將訊號斷點有效偵測出來是必要的。因此，本研究透過半馬可夫模型(semi-Markov model)來描述非穩定性貝氏獨立成份分析模型參數在區間長度(duration)上的資訊，進而估測出正確的非穩定未知訊號的斷點資訊。最後，我們模擬各種非穩定性的訊號混合並評估本論文提出的非穩定性貝氏獨立成份分析演算法的解混合效果，驗證本演算法在非穩定性未知訊號分離的優越性，在初步的實驗結果中我們提出的方法比文獻最新的方法達到較高的訊號干擾比(signal-to-inference ratio)，並且有較好的斷點偵測效果。

In an intelligent speech perception system, it is required to recover speech signals from the mixed signals where some unknown and independent sources are simultaneously acquired by the system microphones. As we known, the independent component analysis (ICA) is a popular approach for blind source separation (BSS) and is referred as an important issue in the fields of machine learning. Traditionally, the standard ICA assumes that the source signals are stationary. This assumption restricts the performance of ICA in real-world applications. Since the source signals may be moving or may be active or inactive as time goes on, we propose a nonstationary Bayesian ICA (NB-ICA) for dealing with the nonstationary blind source separation for an intelligent speech perception system. The proposed NB-ICA algorithm is based on the online Bayesian learning theory which identifies the source activity and estimates the number of source signals in real time. Also, comparing with the other nonstationary ICA algorithms, the computation cost of the proposed NB-ICA can be significantly reduced. In addition, the segmentation information of the source signals is important for the applications of intelligent speech perception such as the systems of speech recognition, speaker identification and speaker diarization. Accordingly, we incorporate a semi-Markov model for capturing the duration information for the estimated status of the source signals. The experimental results show that the proposed NB-ICA algorithm is efficient for the separation of nonstationary source signals in terms of signal-to-inference ratio and detection accuracy of signal segments.

章節目錄
中文摘要 I
ABSTRACT III
章節目錄 IV
圖目錄 VII
表目錄 IX
第一章 緒論 1
1.1 前言 1
1.2 研究動機與目的 2
1.3 研究方法簡介 4
1.4章節概要 5
第二章 獨立成份分析 6
2.1 獨立成份分析基本理論 7
2.1.1 中央極限定理 9
2.1.2 非高斯特性與峰態 10
2.1.3 集中化、白色化 12
2.2最佳化演算法 13
2.3 獨立成份分析演算法目標函數 14
2.3.1獨立成份分析基於負熵(Negentropy) 15
2.3.2獨立成份分析基於最大相似度函數 15
2.3.3 獨立成份分析基於交互資訊 16
第三章 貝氏和非穩定獨立成份分析 18
3.1 噪音的獨立成份分析模型(NOISY ICA MODEL) 19
3.2 隱藏式馬可夫模型(HIDDEN MARKOV MODEL) 23
3.3 切換式獨立成份分析(SWITCHING ICA) 24
3.4 線上變異性貝氏 (ONLINE VARIATIONAL BAYESIAN)27
3.5 非穩定性獨立成份分析(NON-STATIONARY ICA) 29
3.6 區塊調適獨立成份分析(BLOCK ADAPTIVE ICA) 30
第四章 線上貝氏學習於動態訊號分離 32
4.1 非穩定性之訊號混合 33
4.2 線上貝氏學習(ONLINE BAYESIAN LEARNING) 34
4.3 模型建立(MODEL CONSTRUCTION) 36
4.4 模型推論(MODEL INFERENCE) 39
第五章 實驗 46
5.1 實驗設定 46
5.2 未知訊號分離實驗 46
5.2.1模擬來源訊號狀態的改變 47
5.2.2 模擬語者狀態改變和位置移動 52
5.2.3 參數型混合矩陣 57
5.3 實驗討論 59
第六章 結論與未來研究方向 62
6.1結論 62
6.2 未來研究方向 63
參考文獻 66

參考文獻

[1]H. Attias, “A variational Bayesian framework for graphic model,” In Advances Neural Informationn on Processing Systems 12, S. A. Solla, T. K. Leen, and K. R. Muller Eds. Cambridge, MA: MIT Press, pp. 206-212, 2000.
[2]C. M. Bishop, Pattern Recognition and Machine Learning, 2006.
[3]C. M. Bishop and N. D. Lawrence, “Variational Bayesian independent component analysis,” Technical Report, University of Cambridge, 2000.
[4]C. M. Bishop, “Variational Principal components,” In Proceedings of ICANN’99, IEE, vol. 1, pp. 509-514, 1999.
[5]D. Burshtein, “Robust parametric modeling of durations in hidden Markov models,” IEEE Transactions on Speech and Audio Processing, vol. 4, no. 3, pp. 240-242, 1996.
[6]G. Box and G. Tiao, Bayesian Inference in Statistical Analysis, John Wiley and Sons, 1973.
[7]R. Boscolo, H. Pan, and V. P. Roychowdhury, “Non-Parametric ICA,” In Proceedings of the Third International Workshop on Independent Component Analysis and Blind Signal Separation, San Diego, USA, pp. 13-18, 2001.
[8]S. Bulek and N. Erdol, “Block Adaptive ICA with a Time Varying Mixing Matrix,” Independent Component Analysis and Blind Signal Separation, pp. 90-95, 2009.
[9]A. T. Cemgil, C. Fevotteand, and S. J. Godsill, “Variational and stochastic inference for Bayesian source separation,” Digital Signal Processing, vol. 17, no. 5, pp. 891-913, 2007.
[10]J.-T. Chien and G. H. Liao, “Transformation-based Bayesian predictive classification using online prior evolution,” IEEE Transactions on Speech and Audio Processing, vol. 9, no. 4, pp. 399-410, 2001.
[11]J.-T. Chien and C.-H. Huang, “Bayesian learning of speech duration models,” IEEE Transactions on Speech and Audio Processing, vol. 11, no. 6, pp. 558-567, 2003.
[12]J. Cardoso, “Informax and Maximum Likelihood for Source Separation,” IEEE Letters on Signal Processing, vol. 4, no. 4, pp. 112-114, 1997.
[13]K. Chan, T. W. Lee, and T. J. Sejnowski, “Variational learning of clusters of undercomplete nonsymmetric independent components,” Journal of Machine Learning Research, vol. 3, pp. 99-114, 2002.
[14]K. Chan, T.-W. Lee, and T. J. Sejnowski, “Variational Bayesian learning of ICA with missing data,” Neural Computation., vol. 15, pp. 1991-2011, 2003.
[15]P. Comon, “Independent component analysis, a new concept?” Signal Processing, vol. 36, pp. 287-314, 1994.
[16]R. A. Choudreyand and S. J. Roberts, “Flexible Bayesian independent component analysis for blind source separation,” In Proceedings International Conference Independent Component Anal. Signal Separation, pp. 90-95, 2001.
[17]R. A. Choudrey and S. J. Roberts, “Bayesian ICA with hidden Markov sources,” In Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation, pp. 809-814, 2003.
[18]J. Eriksson, J. Karvanen, and V. Koivunen, “Source Distribution Adaptive Maximum Likelihood Estimation of ICA Model,” In Proceedings of the Third International Workshop on Independent Component Analysis and Blind Signal Separation, San Diego, USA, vol. 2, pp. 411-430, 2001.
[19]J. Eriksson, A. Kanhainen, and V. Koivunen, “Novel Characteristic Function Based Criteria For ICA,” In Proceedings of the Third International Workshop on Independent Component Analysis and Blind Signal Separation, San Diego, USA , vol. 2, pp. 781-785, 2001.
[20]R. Everson and S. J. Roberts, “Non-stationary independent component analysis,” In Proceedings of ICANN, pp. 503-508, 1999.
[21]A. Hyvarinen, “Survey on Independent Component Analysis,” Neural Computing Surveys, vol. 2, pp. 94-128, 1999.
[22]A. Hyvarinen and E. Oja, “Independent Component Analysis: Algorithm and Application,” Neural Networks, vol. 13, pp. 411-430, 2001.
[23]A. Hyvarinen, J. Karhunen, and E. Oja, Independent Component Analysis, Wiley, New York, 2001.
[24]A. Honkela and H. Valpola, “On-line variational Bayesian learning”, In Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation, pp.803-808, 2003.
[25]A. Hyvarinen, “Fast and Robust Fixed-Point Algorithms for Independent Component Analysis,” IEEE Transactions on Neural Network, vol. 10, pp. 626-634, 1999.
[26]A. Hyvarinen and E. Oja, “A Fast Fixed-Point Algorithm for Independent Component Analysis,’ Neural Computation, vol. 9, pp. 1483-1492, 1997.
[27]J. Hirayama, S. Maeda and S. Ishii, “Markov and semi-Markov switching of source appearances for nonstationary independent component analysis,” IEEE Transactions on Neural Networks, vol. 18, no. 5, pp. 1326-1342, 2007.
[28]K. Hild, D. Erdogmus, and J. Principe, “Blind Source Separation Using Renyi’s Mutual Information,” IEEE Signal Processing Letters, vol. 8, no .6, pp. 174-176, 2001.
[29]X. D. Huang, Y. Ariki, and M. A. Jack, Hidden Markov Models for Speech Recognition. Edinburgh University Press, 1990.
[30]S. Ikeda, “Factor Analysis Preprocessing For ICA,” In Proceedings of the Third International Workshop on Independent Component Analysis and Blind Signal Separation, San Diego, USA, pp. 29-35, 2001.
[31]B. H. Juang, Pattern Recognition in Speech and Language Processing, CRC Press, 2003.
[32]H. Jiang, K. Hirose, and Q. Huo, “Robust Speech Recognition Based on a Bayesian Prediction Approach,” IEEE Transactions on Speech and Audio Processing, vol. 7, no. 4, 1999.
[33]M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, “An introduction to variational methods for graphical models,” Machine Learning, vol. 37, pp. 183-233, 1999.
[34]K. Kiviluoto and E. Oja, “Independent Component Analysis for Parallel Financial Time Series,”In Proceedings of the International Conference on Neural information Processing (ICONIP ’98), vol. 2, pp. 895-898, 1998.
[35]R. Lambert and A. Bell , “Blind Separation of Multiple Speakers in a multipath environment,” In Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, vol. 1, pp. 423-426, 1997.
[36]David J. C. MacKay, “Probable networks and plausible predictions - a review of practical Bayesian methods for supervised neural networks,” Network: Computation in Neural Systems, vol. 6, pp. 469-505, 1995.
[37]S. Makeig, A. Bell, T. Jung, and T. Sejnowski, “Independent Component Analysis of Electroencephalographic,” Advances in Neural Information Processing System, vol. 8, Cambridge, MA: MIT Press, pp. 145-151, 1996.
[38]W. D. Penny, R. M. Everson, and S. J. Roberts, “Hidden Markov Independent Components Analysis,” in Advances in Independent Component Analysis, pp. 3-22, 2000.
[39]L. R. Rabiner and B. H. Juang, Fundamentals of Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 1993.
[40]H. Saruwatari, K. Sawai, A. LEE, K. Shikano, A. Kaminuma, and M. Sakata, “Speech Enhancement and Recognition in Car Environment using Blind Source Separation and Subband Elimination Processing,” in Proceedings of the Fourth International Workshop on Independent Component Analysis and Blind Signal Separation, vol. 2, pp. 1135-1146, 2003.
[41]O. Winther and K. B. Petersen, “Flexible and efficient implementations of Bayesian independent component analysis,” Neurocomputing, vol. 71, pp. 221-233, 2007.
[42]S. Watanabe, Y. Minami, A. Nakamura, and N. Ueda, “Variational Bayesian Estimation and Clustering for Speech Recognition,” IEEE Transactions on Audio, Speech And Language Processing, vol. 12, no. 4, pp. 365-381, 2004.
[43]H. H. Yang and S.I. Amari, “Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information,” Neural Computation, vol. 9, pp. 1457-1482, 1997.
[44]K. Yu and M. J. F. Gales, “Bayesian Adaptive Inference and Adptive Training,” IEEE Transactions on Audio, Speech and Language Processing, vol. 15, no. 6, 2007.
[45]K. Yu and M. J. F. Gales, “Incremental adaptation using Bayesian inference”, In Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, pp. 217-220, 2006.