臺灣博碩士論文加值系統

當我們想在一堆大量的資料裡面，找出他們之間的相關性時，找出一個最佳的座標系統來表示這些資料，或一些資料的壓縮是必要的，比如資料分析，特徵擷取，信號處理或類神經網路的研究等等。為了要達到這個目的，我們往往需要對原始資料做一些線性轉換，比方說：主成分分析，因素分析，投影追蹤，以及未知信號認證等等。近來，對以上這些問題，我們發展出了一種強而有力的方法，它叫做獨立信號分析(Independent Component Analysis)。我們可以說這個方法是主成分分析的一種延伸，主成分分析只能找出資料二階的相關特性，無法處理更高階的統計特性，只能定義出正交的基底，而獨立元件分析則無此限制。
獨立元件分析可以應用在很多方面，例如腦電波圖的處理，去除信號雜訊，遙測影像分析，浮水印(資料加密解密)，在特徵擷取方面可以應用在人臉辨識上，或材質的識別，物件的分類等等。
在這篇論文裡一共包含五個章節。第一個章節會先做一個簡單的介紹，包括整個問題的概述，相關的研究，可應用的層面等等。第二章，我會先從主成分分析的原理開始介紹，接著是獨立信號分析的原理和演算法的介紹，在這裡，我們會介紹四種不同的演算法，並將這四種作一個比較和一個簡要的結論。第三章我會開始介紹一些在影像方面的應用，諸如浮水印，人臉辨識，去雜訊等等。而第四章介紹的是一些在聲音上面的應用，例如從一群人的吵雜聲音中分離出一個人的聲音，或者在一段背景音樂中分離出人的聲音。最後，在第五章會有個總結和未來仍在研究的內容和方向。

When we want to find some relationship in large amounts of data, such as data analysis, feature extraction, signal processing, de-noising, and neural network research, a suitable representation of these data or data compression is needed. To achieve these goals, we usually do some linear transformation on original data. For example, principle component analysis, factor analysis, projection pursuit, and blind identification, etc. Recently, a powerful linear transform method called independent component analysis (ICA) is developed. We can say that this is an extension of principle component analysis (PCA), which can only impose uncorrelated up to second order statistics, and thus, defines orthogonal directions. However, ICA does not have these constrains and can analyze higher-order statistics.
Independent component analysis can be applied on many applications such as electroencephalogram (EEG), de-noising, remote sensing, watermarking (data encryption or decryption), feature extraction such as facial detection and recognition, texture extraction, and object classification, etc.
This thesis consists of five chapters. There will be an introduction to independent component analysis in chapter 1, including problem formulation, related works, some applications and the organization of this thesis. In chapter 2, I will introduce the algorithm of principle component analysis first, and then the theories and algorithms of independent component analysis will be explained. Also I will compare these algorithms and a summary will be made. In chapter 3, I will introduce applications of ICA on image processing, such as watermark, face detection, de-noise, etc. In chapter 4, I will introduce applications of ICA on sound processing, such as separating human speeches or separating human voice from background music .Finally, a concluding remark and future research directions shall be given in chapter 5.

Contents
Chapter 1 Introduction………….…………………………....1
1.1 Problem Formulation...............................................................…...1
1.2 Related Works.........................................................................…....4
1.3 Applications....................................................................................7
1.4 Organization...................................................................................7
Chapter 2 Review of Independent Component Analysis.....................................................................9
2.1 Introduction....................................................................................9
2.2 Principle Component Analysis.....................................................10
2.2.1 Introduction..........................................................................10
2.2.2 PCA and ICA........................................................................12
2.3 Independent Component Analysis................................................13
2.3.1 What is Independence...........................................................13
2.3.2 Definition..............................................................................14
2.3.3 Assumption...........................................................................16
2.3.4 Ambiguities...........................................................................19
2.4 Principle of ICA Estimation.........................................................20
2.4.1 Nongaussianity.....................................................................20
2.4.2 Minimization of Mutual Information....................................23
2.4.3 Maximum Likelihood Estimation.........................................25
2.5Algorithms of ICA..........................................................................26
2.5.1 Preprocessing.......................................................................26
2.5.2 The JADE Algorithm............................................................28
2.5.3 The Fast ICA Algorithm.......................................................30
2.5.4 S.O.B.I..................................................................................32
2.5.5 The LinSep Algorithm...........................................................34
2.5.6 Comparison Between Algorithms.........................................35
2.6 Overcomplete ICA.......................................................................37
2.6.1 Introduction..........................................................................37
2.6.2 Virtual Sensors.....................................................................38
2.6.3 Algorithms for 3 Sources and 2 Sensors...............................40
Chapter 3 Applications on Image Processing.......................43
3.1 Introduction..................................................................................43
3.2 Basic Application: Extracting Images from Mixed Images.........44
3.2.1 Introduction..........................................................................44
3.2.2 Experiment Result.................................................................45
3.3 Digital Image Watermarking........................................................47
3.3.1 Introduction..........................................................................47
3.3.2 Detecting and Extracting Watermark...................................49
3.3.3 Image Cryptosystems with Dual Encryption........................52
3.4 Restoration of Archival Documents.............................................56
3.4.1 Introduction..........................................................................56
3.4.2 Scanning a Document...........................................................57
3.4.3 Handwriting Case................................................................60
3.5 De-noising....................................................................................62
3.5.1 Introduction..........................................................................62
3.5.2 De-noising in Digital Camera..............................................62
3.5.3 Discussion............................................................................67
3.6 Face Detection and Recognition...................................................67
3.6.1 Introduction..........................................................................67
3.6.2 Face Detection and Authentication......................................68
3.6.3 Face Localization.................................................................70
3.6.4 Experiment Result................................................................71
3.7 Color Space Choice......................................................................74
3.8 Conclusion....................................................................................81
Chapter 4 Applications on Audio Processing........................83
4.1 Introduction..................................................................................83
4.2 Solving the Cocktail-Party Problem.............................................84
4.2.1 Introduction..........................................................................84
4.2.2 Synthesized Mixing...............................................................84
4.2.3 Real Recording.....................................................................85
4.3 Conclusion....................................................................................90
Chapter 5 Conclusion and Future Works.............................93
5.1 Conclusion....................................................................................93
5.2 Future Works................................................................................94
Reference..................................................................................97

Independent Component Analysis Basic Theories
[1] Pierre Comon, “Independent component analysis, A new concept?”, Vol. 36, no 3, Special issue on High-Order Statistics, April 1994
[2] Aapo Hyvärinen and Erkki Oja, “Independent Component Analysis: Algorithms and Applications”, Neural Neworks, April 1999
[3] Aapo Hyvärinen, “Survey on Independent Component Analysis”
[4] J. F. Cardoso, “Source separation using higher order moments”, Proc. Internal. Conf. Acoust. Speech Signal Process, Glasgow, 1989, pp. 2109-2112.
[5] P. Comon, “independent component analysis”, Internal. Signal Processing Workshop out High-Order Statistics, Elsevier, Amsterdam 1992, pp. 29-38.
[6] M. Gaeta and J.L. Lacoume, “Source separation without a priori knowledge: The maximum likelihood solution”, in: Torres, Masgrau and Lagunas, eds., Proc EUSIPCO Conf., Barcelona, Elsevier, Amsterdam, 1990, pp. 621-624.
[7] Y. Inouye and T. Matsui, “cumulant based parameter estimation of linear systems”, Proc. Workshop Higher-Order Spectral Analysis, Vail, Colorado, June 1989, pp. 180-185.
[8] C. Jutten and J. Herault, “Blind separation of sources, Part I: An adaptive algorithm based on neuromimatic architecture”, Signal Processing, Vol. 24, No. 1, July 1991, pp. 1-10.
[9] L. Tong et al., “A necessary and sufficient condition of blind identification”, Internal. Signal Processing Workshop on High-Order Statistics, Chamrousse, France, 10-12 July 1991, pp. 261-264.
[10] J.C. Fort, “Stability of the JH sources separation algorithm”, Traitement du Signal, Vol. 8, No. 1, 1991, pp. 35-42
[11] Y. Bar-Ness, “Bootstrapping adaptive interference cancellers: Some practical limitations”, The Globecom Conf., Miami, November 1982, Paper F3.7, pp. 1251-1255.
[12] G. Giannakis, Y. Inouye and J.M. Mendel, “Cumulant based identification of multichannel moving average models”, IEEE Automat. Control, Vol. 34, July 1989, pp. 783-787
[13] J.L. Lacoume and P. Ruiz, “Extraction of independent components from correlated inputs, A solution based on cumulants”, Proc Workshop Higher-Order Spectral Analysis, Vail, Colorado, June 1989, pp. 146-151.
[14] M. Gaeta, Les Statistiques d`Ordre Supérieur Appliquées à la Séparation de Sources, Ph.D. Thesis, Université de Grenoble, July 1991.
[15] A. Souloumiac and J.F. Cardoso, “Comparaisons de Méthodes de Séparation de Sources”, XIIIth Coll. GRETSI, September 1991, pp. 661-664.
[16] J.F. Cardoso, “Super-symmetric decomposition of the fourth-order cumulant tensor, blind identification of more sources than sensors”, Proc. Internal. Conf. Acoust. Speech Signal Process. 91, 14-17 May 1991.
[17] J.F. Cardoso, “Iterative techniques for blind sources separation using only fourth order cumulants”, Conf. EUSIPCO, 1992, pp. 739-742.
[18] P. Comon, “Separation of stochastic processes”, Proc. Workshop Higher-Order Spectral Analysis, Vail, Colorado, June 1989, pp. 174-179
[19] G. Darmois, “Analyse Générale des Liaisons Stochastiques”, Rec. Inst. Internal. Stat., Vol. 21, 1953, pp. 2-8
[20] L. Fety, Methodes de traitement d’antenne adaptées aux radiocommunications, Doctorate Thesis, ENST, 1998.
[21] L. Tong, V.C. Soon and R. Liu, “AMUSE: A new blind identification algorithm”, Proc, Internal. Conf. Acoust. Soeech Signal Process., 1990, pp. 1783-1787.
[22] C. Jutten., “Calcul neuromimétique et traitement du signal, analyse en composantes independents”, PhD thesis, INPG, Univ. Grenoble, 1987.
[23] C. Jutten and J. Herault. “Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture.”, Signal Processing, 24:1-10, 1991.
[24] Cardoso, J.F., Souloumiac A., "Blind beamforming for non Gaussian signals ", IEEE Proceedings-F, vol. 140, no. 6, pp. 362-370, December 1993
[25] A. Belouchrani, K. Abed-Meraim, J.F. Cardoso and E. Moulines (February 1997). ”A blind source separation technique using second order statistics”, IEEE Trans. on Signal Processing, 45(2): 434-444
[26] T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley & Sons, 1991.
[27] A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 3rd edition, 1991.
[28] A. Hyvärinen. New approximations of differential entropy for independent component analysis and projection pursuit. In Advances in Neural Information Processing Systems, volume 10, pages 273-279. MIT Press, 1998.
[29] J. P. Nadal and N. Parga. Non-linear neurons in the low noise limit: a factorial code maximizes information transfer. Network, 5:565-581, 1994.
[30] J. F. Cardoso. Infomax and maximum likelihood for source separation. IEEE Letters on Signal Processing, 4:112-114, 1997.
[31] B. A. Pearlmutter an L. C. Parra. Maximum likelihood blind source separation: A context-sensitive generalization of ica. In Advances in Neural Information Processing Systems, volume 9, pages 613-619, 1997.
[32] M. S. Lewicki and T. J. Sejnowski. “Learning overcomplete representations.” Neural Computation 12, 337-365, 2000
[33] P. Comon and O. Grellier, “Non-linear inversion of underdetermined mixtures”, in First International Workshop on Independent Component Analysis and Signal Separation (ICA99), Aussois, France, 11-15, January 1999, pp. 461-465.
[34] L. De Lathauwer, P. Comon, B. De Moor, and J. Vandewalle. ICA algorithms for 3 sources and 2 sensors. IEEE Sig. Proc. Workshop on Higher-Order Statistics, June 14-16, 1999, Caesarea, Israel, Pages 116-120, 1999.
[35] F.J. Theis, E.W. Lang, Formalization of the Two-Step Approach to Overcomplete BSS, Proc. of SIP 2002, pp. 207-212 (2002).
[36] D. G. Luenberger. “Optimization by Vector Space Methods”, John Wiley & Sons, 1969.
[37] Martin-Clemente, R., Acha, J.I. “ New Equations and Iterative Algorithm for Blind Separation of Sources”, Signal Processing. Vol. 82. Num. 6. 2002. pp. 861-873
Applications on Image Processing
[38] Dan Yu, Farook Sattar, and Kai-Kuang Ma, “Watermark Detection and Extraction Using Independent Component Analysis Method”, 92-104, EURASIP Journal on Applied Signal Processing, 2002
[39] J. G. Proakis, “Digital Communications”, McGraw-Hill, New York, 2nd edition, 1989.
[40] Qui-Hua Lin and Fu-Liang Yin, “Blind source separation applied to image cryptosystems with dual encryption”, Electronics Letters 12th September 2002 Vol.38 No.19
[41] Chew Lim Tan, Ruini Cao, and Peiyi Shen “Restoration of Archival Documents Using a Wavelet Technique”
[42] H.-S. Don, “A Noise Attribute Thresholding Method for Document Image Binarization,” Proc. Third Int’l Conf. Document Analysis and Recognition, pp.231-234, Aug. 1995.
[43] G. Sharma, “Cancellation of Show-Through in Duplex Scanning,” Proc. Int’l Conf. Image Processing, vol.3 ,pp. 609-612, Sep. 2000.
[44] Mizuki HAGIWARA and Masayuki KAWAMATA, “Detection of Frame Displacement for Old Films Using Phase-Only Correlation”, International Symposium on Intelligent Signal Processing and Communication Systems, November 21-24, 2002.
[45] DENNING, D.E.R, “Cryptography and data security” (Addison-Wesley, Reading, MA, USA, 1982)
[46] PECORA, L.M., and CARROLL, Y.L., “Synchronization in chaotic systems”, Phys. Rev. Lett., 1990, 64, (8), pp. 821-823
[47] B. Deknuydt, J. Smolders, Luc Van Etcken, and André Oosterlinck, “Color Space Choice for nearly reversible Image Compression”, 1300/SPIE Vol. 1818 Visual Communications and Image Processing ’92.
Applications on Audio Processing
[48] Te-Won Lee, and Andreas Ziehe, “Combining time-delayed decorrelation and ICA: Towards solving the cocktail party problem”, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing , May 1998, Seattle, Vol 2, pp. 1249-1252.
[49] http://inc2.ucsd.edu/~tewon/