臺灣博碩士論文加值系統

人臉辨識是長期在電腦視覺領域研究的主題。然而在真實世界中，遮擋物是經常發生且對辨識能力造成阻礙。在有遮擋物的情形下，人臉有效的資訊部分減少。近來，有基於稀疏表示的分類方法被提出於強健性人臉辨識的問題，於隨機像素破壞的人臉影像辨識上有不凡的表現，然而此方法對於真實世界的遮擋情形卻是缺乏效率及有效性。
　　基於壓縮感知的發展如L1 norm最小化問題以及矩陣的秩最小化問題，我們採用一種稀疏低秩模型，對人臉影像取廣義的梯度方向後，將其套用於此回歸模型上。我們採用總共三階的梯度方向當作人臉影像的特徵，透過alternating direction method of multipliers (ADMM)我們可以將此稀疏低秩模型將以最佳化。我們觀察到梯度方向圖並不符合低秩矩陣的假設，然而最佳化方法仍可以成功的適用，我們稱此為弱低秩最佳化問題。實驗上，我們證明透過弱低秩問題的最佳化，我們會喪失還原上的空間資訊，但是對於辨識能力有大幅的提升。與近年來的最先進的方法比較，我們提出的方法有最好的效能。
　　接著，我們研究凸優化與非凸優化方法的行為於一些稀疏低秩模型上，Robust Principal Component Analysis (RPCA) 與 Low-Rank Representation (LRR) 是兩個著名的稀疏低秩模型。他們都可以透過ADMM步驟與凸函數代理來做優化。然而凸函數代理有時無法有效的近似原始問題，所以我們引入了非凸函數來做更佳的近似。首先我們提出一個新的非凸函數代理，於矩陣補完的問題上有更佳的表現。接著我們將此代理函數用於RPCA與LRR問題，形成非凸的稀疏低秩模型。於ADMM的步驟上，我們也提出了加上衝量項改良了對偶步驟的更新，我們稱此為對偶衝量，並利用此避免最佳化點卡於局部最佳值的問題，以及使收斂速度更快。我們寫出了完整的理論上的收斂性分析與證明了非凸方法的收斂率。實驗上我們也證明非凸方法與對偶衝量都能夠使最佳化的解收斂到更小的還原誤差的點上。在RPCA上我們以矩陣及影像去雜訊作為實驗，在LRR上我們以頻譜聚類與異常偵測作為實驗。我們也與其他基於RPCA及LRR的改良方法做比較，也以實驗證明我們的方法有最佳的表現。

Face recognition is a very popular research topic for computer vision. However, in the real-world scenario, occlusion is a frequently occurring obstacle for recognition. With the occlusion, the information for the face of an individual is diminished. Recently, sparse representation based classification has been proposed on robust the face recognition problem. They have extraordinary performance on randomly corrupted face images. However, for real-world occlusion, this method lacks efficiency and effectiveness.
Inspired by the techniques related to compressive sensing, such as L1-norm minimization and the rank minimization, we propose a novel sparse and low-rank model in this thesis. It performs regression for face images on the generalized gradient direction domain. We adopt the three orders gradient direction as features. Through the alternating direction method of multipliers (ADMM) procedure, we can optimize the sparse and low-rank model. We observe that the gradient direction map does not satisfy the low-rank assumption spatially. However, the optimization method still works well on the “weak low-rankness” optimization problem. In experiments, we show that, with the proposed method for solving the weak low-rank problem, the recognition rate can be improved greatly. Compared to the state-of-the-art methods, proposed method has the best performance.
Next, we study the behavior of the convex and nonconvex optimizations of different sparse and low-rank models. Robust Principal Component Analysis (RPCA) and Low-Rank Representation (LRR) are two well-known models. They can be optimized by ADMM with the convex surrogates. Nevertheless, the convex surrogates cannot approximate the original problem well. Therefore, we propose a nonconvex surrogate for better approximation. First, a novel nonconvex surrogate which has better performance on the matrix completion and image completion problem is proposed. Next, this surrogate is applied on the RPCA and LRR to form the nonconvex sparse and low-rank models. In the ADMM procedure, we proposed to revise the dual updates with the momentum term. We call this trick dual momentum, which can avoid the local optimal problem and boost the convergence speed. We give a complete theoretical convergence analysis and prove the convergence rate of the nonconvex approach. Experiments show that the nonconvex approach with dual momentum can converge to a point with a smaller recovery residual on extensive applications such as matrix and image denoising of RPCA and spectral clustering with outlier detection of LRR. We also compare proposed method to the other improvement methods based on the RPCA and LRR and show that proposed methods has the best performance.

口試委員會審定書 #
誌謝 i
中文摘要 ii
ABSTRACT iv
CONTENTS vi
LIST OF FIGURES xi
LIST OF TABLES xvii
Chapter 1 Introduction 1
1.1 Machine Learning and Face Recognition 1
1.2 Numerical Optimization 2
1.3 Contribution and Achievement of the Thesis 2
1.4 Organization of the Thesis 5
1.5 Notations 5
Chapter 2 The Fundamentals of Statistics and Data Modeling 7
2.1 Principal Component Analysis (PCA) 7
2.2 Probabilistic Principal Component Analysis 10
2.3 Two-Dimensional Principal Component Analysis 14
2.4 Kernel Principal Component Analysis 15
2.5 Schematic View of Supervised and Unsupervised Learning 17
2.6 Linear Discriminant Analysis 21
2.7 Manifold Learning 22
2.7.1 Multidimensional Scaling (MDS) 23
2.7.2 Locally Linear Embedding (LLE) 25
2.7.3 Laplacian Eigenmaps (LE) 26
2.8 Conclusion 28
Chapter 3 Compressive Sensing and Dictionary Learning 29
3.1 Sparse Signal Recovery 29
3.1.1 Basic theory of recoverability of sparse signal 29
3.1.2 Programming algorithm of L1 norm minimization 34
3.1.3 Hierarchical adaptive Lasso (HAL) 38
3.1.4 Exactness of recovery of sparse signal 41
3.2 Low-Rank Recovery 43
3.2.1 Basic theory of recoverability of low-rank signals 43
3.2.2 Programming algorithm of nuclear norm minimization 45
3.3 Sparse and Low-Rank Models 49
3.3.1 Robust Principal Component Analysis (RPCA) 49
3.3.2 Sparse Subspace Clustering (SSC) 53
3.3.3 Low-Rank Subspace Clustering (LRSC) 54
3.3.4 Low-Rank Representation (LRR) 54
3.4 Dictionary Learning 56
3.5 Conclusion 58
Chapter 4 Convex Optimization 59
4.1 Convexity 59
4.2 Lagrange Duality 62
4.3 Proximal Gradient 63
4.4 Augmented Lagrangian Method 65
4.4.1 Deciding the Step Size 66
4.5 Alternating Direction Method of Multiplers (ADMM) 66
4.5.1 Douglas-Rachford splitting 66
4.5.2 Douglas-Rachford method on dual problem 67
4.6 Conclusion 68
Chapter 5 Occluded Face Recognition with Dictionary Learning and Occlusion Pattern 69
5.1 Related Work 69
5.1.1 Sparse Representation Based Classification (SRC) 69
5.1.2 Maximum Correntropy Criterion 70
5.1.3 Half-Quadratic Minimization (HQ) 71
5.2 Problem Formulation 73
5.3 Proposed Occlusion Pattern-Based Sparse Representation and Classification (OPSRC) 73
5.4 Experiments and Discussion 79
5.4.1 YaleFace Face Database 80
5.4.2 AR Face Database 81
5.4.3 Discussion on the OPSRC and the comparison 84
5.5 Conclusion 85
Chapter 6 Occluded Face Recognition Using Sparse and Low-Rank Regression with Generalized Gradient Direction 87
6.1 Related Works and Background 87
6.1.1 Gradient Image 88
6.1.2 Sparse Representation 89
6.1.3 Low-rank Representation 91
6.2 Proposed Gradient Direction-based Hierarchical Adaptive Sparse and Low-Rank (GD-HASLR) Model 92
6.2.1 Generalized Image Gradient Direction 92
6.2.2 Sparse and Low-Rank Regularized Regression 95
6.2.3 Hierarchical Adaptive Sparse and Low-Rank Regression 96
6.2.4 Optimization 98
6.2.5 Residual Computing and Classification 99
6.2.6 Low-rankness in Image Gradient Direction Domain 101
6.2.7 Time Complexity Analysis 104
6.3 Experiments on Real Disguise and Block Occlusion 104
6.3.1 AR Database Real-world Occlusion 104
6.3.2 Comparison with CNN-Based Methods 107
6.3.3 Analysis 110
6.3.4 Synthetic Different Occlusion Patterns 114
6.4 Experiments on Illumination Occlusion 116
6.4.1 CMU-PIE Database for illumination occlusion 116
6.4.2 Extended Yale B Database for illumination occlusion 118
6.5 Conclusion 120
Chapter 7 Nonconvex Approach for Sparse and Low-Rank Constrained Model with Dual Momentum 122
7.1 Related Works and Background 123
7.1.1 Sparse and Low-Rank Models 123
7.1.2 Nuclear norm minimization and optimization 125
7.1.3 Nonconvex surrogate for rank minimization 126
7.2 Proposed piecewise linear regularizer 128
7.3 Nonconvex surrogate for RPCA and LRR 130
7.3.1 Propose to introduce nonconvex surrogate on RPCA 130
7.3.2 Propose to introduce nonconvex surrogate on LRR 132
7.3.3 Proposed momentum trick on dual variables 134
7.4 Convergence Analysis 138
7.5 Experimental results 150
7.5.1 Proposed nonconvex surrogate 150
7.5.2 Nonconvex RPCA with dual momentum 156
7.5.3 Nonconvex LRR with dual momentum 161
7.6 Conclusion 165
Chapter 8 Conclusion and Future Work 166
REFERENCE 169
誌謝 i
中文摘要 ii
ABSTRACT iii
CONTENTS v
LIST OF FIGURES x
LIST OF TABLES xvi
Chapter 1 Introduction 1
1.1 Machine Learning and Face Recognition 1
1.2 Numerical Optimization 2
1.3 Contribution and Achievement of the Thesis 2
1.4 Organization of the Thesis 4
1.5 Notations 5
Chapter 2 The Fundamentals of Statistics and Data Modeling 6
2.1 Principal Component Analysis (PCA) 6
2.2 Probabilistic Principal Component Analysis 9
2.3 Two-Dimensional Principal Component Analysis 13
2.4 Kernel Principal Component Analysis 14
2.5 Schematic View of Supervised and Unsupervised Learning 16
2.6 Linear Discriminant Analysis 19
2.7 Manifold Learning 21
2.7.1 Multidimensional Scaling (MDS) 21
2.7.2 Locally Linear Embedding (LLE) 23
2.7.3 Laplacian Eigenmaps (LE) 24
2.8 Conclusion 26
Chapter 3 Compressive Sensing and Dictionary Learning 27
3.1 Sparse Signal Recovery 27
3.1.1 Basic theory of recoverability of sparse signal 27
3.1.2 Programming algorithm of L1 norm minimization 31
3.1.3 Hierarchical adaptive Lasso (HAL) 36
3.1.4 Exactness of recovery of sparse signal 39
3.2 Low-Rank Recovery 40
3.2.1 Basic theory of recoverability of low-rank signal 41
3.2.2 Programming algorithm of nuclear norm minimization 42
3.3 Sparse and Low-Rank Model 46
3.3.1 Robust Principal Component Analysis (RPCA) 46
3.3.2 Sparse Subspace Clustering (SSC) 50
3.3.3 Low-Rank Subspace Clustering (LRSC) 50
3.3.4 Low-Rank Representation (LRR) 51
3.4 Dictionary Learning 52
3.5 Conclusion 54
Chapter 4 Convex Optimization 55
4.1 Convexity 55
4.2 Lagrange Duality 58
4.3 Proximal Gradient 59
4.4 Augmented Lagrangian Method 61
4.4.1 Deciding the Step Size 61
4.5 Alternating Direction Method of Multiplers (ADMM) 62
4.5.1 Douglas-Rachford splitting 62
4.5.2 Douglas-Rachford method on dual problem 62
4.6 Conclusion 63
Chapter 5 Occluded Face Recognition with Dictionary Learning and Occlusion Pattern 65
5.1 Related Work 65
5.1.1 Sparse Representation Based Classification (SRC) 65
5.1.2 Maximum Correntropy Criterion 66
5.1.3 Half-Quadratic Minimization (HQ) 67
5.2 Problem Formulation 69
5.3 Proposed Occlusion Pattern-Based Sparse Representation and Classification (OPSRC) 69
5.4 Experiments and Discussion 75
5.4.1 YaleFace Face Database 76
5.4.2 AR Face Database 77
5.4.3 Discussion on the OPSRC and the comparison 80
5.5 Conclusion 81
Chapter 6 Occluded Face Recognition Using Sparse and Low-Rank Regression with Generalized Gradient Direction 83
6.1 Related Works and Background 83
6.1.1 Gradient Image 84
6.1.2 Sparse Representation 85
6.1.3 Low-rank Representation 87
6.2 Proposed Gradient Direction-based Hierarchical Adaptive Sparse and Low-Rank (GD-HASLR) Model 88
6.2.1 Generalized Image Gradient Direction 88
6.2.2 Sparse and Low-Rank Regularized Regression 90
6.2.3 Hierarchical Adaptive Sparse and Low-Rank Regression 92
6.2.4 Optimization 93
6.2.5 Residual Computing and Classification 95
6.2.6 Low-rankness in Image Gradient Direction Domain 97
6.3 Experiments on Real Disguise and Block Occlusion 100
6.3.1 AR Database Real-world Occlusion 100
6.3.2 Comparison with CNN-Based Methods 102
6.3.3 Analysis 106
6.3.4 Synthetic Different Occlusion Patterns 110
6.4 Experiments on Illumination Occlusion 112
6.4.1 CMU-PIE Database for illumination occlusion 112
6.4.2 Extended Yale B Database for illumination occlusion 114
6.5 Conclusion 116
Chapter 7 Nonconvex Approach for Sparse and Low-Rank Constrained Model with Dual Momentum 118
7.1 Related Works and Background 119
7.1.1 Sparse and Low-Rank Models 119
7.1.2 Nuclear norm minimization and optimization 120
7.1.3 Nonconvex surrogate for rank minimization 121
7.2 Proposed piecewise linear regularizer 123
7.3 Nonconvex surrogate for RPCA and LRR 126
7.3.1 Introducing nonconvex surrogate on RPCA 126
7.3.2 Introducing nonconvex surrogate on LRR 128
7.3.3 Proposed momentum trick on dual variable 129
7.4 Convergence Analysis 133
7.5 Experimental results 143
7.5.1 Proposed nonconvex nonsmooth surrogate 143
7.5.2 Nonconvex RPCA with dual momentum 149
7.5.3 Nonconvex LRR with dual momentum 154
7.6 Conclusion 158
Chapter 8 Conclusion 159
REFERENCE 161

A.Introduction
[1]V. Bruce, “Influences of familiarity on the processing of faces,” Perception, vol. 15, pp. 387-397, 1986.
[2]H. D. Ellis, J. W. Shepherd, G. M. Davies, “Identification of familiar and unfamiliar faces from internal and external features: Some implications for theories of face recognition,” Perception, vol. 8, no. 4, pp. 431-439, 1979.
[3]D. Marr, E. Hildreth, “Theory of edge detection,” Proc. Roy. Soc. London, vol. B207, pp. 187-217, 1980.
[4]D. Marr, T. Poggio, “A computational theory of human stereo vision,” Proc. Roy. Soc. London, vol. B204, pp. 301-328, 1979.
[5]M. A. Turk, A. P. Pentland, “Face recognition using eigenfaces,” Proc. Int. Conf. on Patt. Recog., pp. 586-591, 1991.
[6]M. Kirby and L. Sirovich, “Application of the Karhunen-Loeve procedure for the characterization of human faces,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, pp. 103-108, 1990.
[7]R. Chellappa, C.L. Wilson, S. Sirohey, “Human and machine recognition of Faces: A survey,” Proc. IEEE, vol. 83, no. 5, pp. 705-740, 1995.
[8]L. Bottou, "Large-Scale machine learning with stochastic gradient descent", Proc. 19th Int''l Conf. Computational Statistics, 2010.
[9]D.P. Bertsekas, Nonlinear Programming., Belmont, MA, USA:Athena Scientific, 1995.
B.Statistics and data modeling
[10]C. Eckart, G. Young, “The approximation of one matrix by another of lower rank,” Psychometrika, vol. 1, no. 3, pp. 211-218, 1936.
[11]R. Vidal, Y. Ma, S. S. Sastry, Generalized Principal Component Analysis, Springer, New York, 2016.
[12]M.E. Tipping, C.M. Bishop, “Probabilistic principal components analysis,” J. Royal Statistical Soc. B, vol. 61, no. 3, pp. 611-622, 1999.
[13]J. Yang D. Zhang, A. F. Frangi, J. Y. Yang, “Two-dimensional PCA: A new approach to appearance-based face representation and recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no.1, pp. 131–137, 2004.
[14]B. Schlkopf, A. Smola, K.-R. Mller, “Kernel principal component analysis,” Proc. Int''l Conf. Artificial Neural Networks, pp. 583-588, 1997.
[15]K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, Cambridge, MA, 2012.
[16]S. Shalev-Shwartz, S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.
[17]L. Valiant, “A theory of the learnable,” Commun. ACM, vol. 27, pp. 1134-1142, Nov. 1984.
[18]V. N. Vapnik, “An overview of statistical learning theory,” IEEE Trans. Neural Networks, vol. 10, pp. 988-999, Sept. 1999.
[19]A. Blumer, A. Ehrenfeucht, D. Haussler, M. K. Warmuth, “Learnability and the Vapnik–Chervonenkis dimension”, J. ACM, vol. 36, no. 4, pp. 929-965, 1989.
[20]C. M. Bishop, Pattern Recognition and Machine Learning, Springer, New York, 2007.
[21]J. Friedman, R. Tibshirani, T. Hastie, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, 2009.
[22]C. Cortes, V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
[23]I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, “Generative adversarial nets,” Proc. Adv. Neural Inf. Process. Syst., pp. 2672-2680, 2014.
[24]P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no.7, pp. 711–720, 1997.
[25]J. B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika, vol. 29, no. 1, pp. 1-27, Mar. 1964.
[26]S.T. Roweis, L.K. Saul, “Nonlinear dimensionality reduction by local linear embedding,” Science, vol. 290, pp. 2323-2326, Dec. 2000.
[27]M. Belkin, P. Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Computation, vol. 15, no. 6, pp. 1373-1396, 2003.
C.Compressive Sensing and Dictionary Learning
[28]R. van Handel, “Probability in high dimension,” Princeton University, Jun. 2014.
[29]D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, 2006.
[30]R. Vershynin, “High dimensional probability,” 2016, to be published.
[31]S. Mallat, Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, 1993.
[32]G. Davis, S. Mallat, Z. Zhang, “Adaptive time-frequency decompositions,” Opt. Eng., vol. 33, no. 7, pp. 2183-91, 1994.
[33]W. J. Fu, “Penalized regression: The bridge versus the lasso,” J. Computat. Graph. Statist., vol. 7, pp. 397-416, 1998.
[34]B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, “Least angle regression,” Ann. Statist., vol. 32, no. 2, pp. 407-499, 2004.
[35]A. Lee, F. Caron, A. Doucet, and C. Holmes, “A hierarchical Bayesian framework for constructing sparsity-inducing priors,” Technical Report, University of Oxford, UK, pp. 1-18, 2010.
[36]D. Andrews, C. Mallows, “Scale mixtures of normal distributions,” J. R. Statist. Soc, vol. 36, pp. 99, 1974.
[37]A. Armagan, D. B. Dunson, J. Lee, “Generalized double Pareto shrinkage,” Technical Report, University of Duke, 2011.
[38]J. Griffin, P. Brown, “Bayesian adaptive lassos with non-convex penalization,” Technical Report, University of Warwick, 2007.
[39]M. Figueiredo, “Adaptive Sparseness for Supervised Learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, pp. 1150-1159, 2003.
[40]E. Candes, M. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag., vol. 25, no. 2, pp. 21-30, Mar. 2008.
[41]J. Cai, E. Candes, and Z. Shen, “A singular value thresholding algorithm for matrix completion,” SIAM J. Optim., vol. 20, no. 4, pp. 1956-1982, 2010.
[42]J. Wright, Y. Peng, Y. Ma, A. Ganesh, S. Rao, “Robust principal component analysis: exact recovery of corrupted low-rank matrices by convex optimization”, Proc. Adv. Neural Inf. Process. Syst., 2009.
[43]E. Candes, X. Li, Y. Ma, and J. Wright, “Robust principal component analysis?,” Journal of the ACM, vol. 58, issue 3, article 11, May 2011.
[44]E. Elhamifar, R. Vidal, “Sparse subspace clustering,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2790-2797, 2009.
[45]G. Liu, Z. Lin, Y. Yu, “Robust subspace segmentation by low-rank representation,” Proc. Int''l Conf. Mach. Learn., pp. 663-670, 2010.
[46]G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu and Y. Ma, “Robust recovery of subspace structures by low-rank representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 171-184, Jan. 2013.
D.Convex Optimization
[47]S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004.
[48]D. Bertsekas, Convex Optimization Algorithms, Belmont, MA, USA:Athena Scientific, 2015.
[49]Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, MA, Norwell:Kluwer, 2004.
[50]N. Parikh, S. Boyd, “Proximal algorithms,” Found. Trends Optim., vol. 1, no. 3, pp. 123-231, 2013.
[51]S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method of multipliers", Found. Trends Mach. Learn., vol. 8, no. 1, pp. 1-122, 2011.
[52]A. Beck, M. Teboulle, “A fast iterative shrinkage thresholding algorithm for linear inverse problems,” SIAM J. Imaging Sci., vol. 2, no. 1, pp. 183-202, 2009.
[53]T. Lin, S. Ma, S. Zhang, “On the global linear convergence of the ADMM with multiblock variables,” SIAM J. Optim., vol. 25, no. 3, pp. 1478-1497, 2015.
[54]Y. Wang, W. Yin, J. Zeng, “Global convergence of ADMM in nonconvex nonsmooth optimization,” in arXiv, 2015, [online] Available: https://arxiv.org/abs/1511.06324.
[55]M. Hong, Z.-Q. Lo, M. Razaviyayn, “Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems” Proc. IEEE Int. Conf. Acoust. Speech Signal Process., pp. 1-5, 2015.
E.OPSRC related works
[56]J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, issue. 2, pp. 210-227, Feb. 2009.
[57]L. Zhang, M. Yang, and X. Feng, “Sparse representation or collaborative representation: Which helps face recognition?” Proc. IEEE Int’l Conf. Computer Vision, pp. 471-478, 2011.
[58]P. J. Huber, “Robust estimation of a location parameter,” Ann. Math. Statist., vol. 35, no. 1, pp. 73-101, 1964.
[59]P. J. Huber, E. Ronchetti, Robust Statistics, USA, NY, New York:Wiley, 2009.
[60]R. He, W. S. Zheng, B. G. Hu, and X. W. Kong, “A regularized correntropy framework for robust pattern recognition,” Neural Computation, vol. 23, issue 8, pp. 2074-2100, 2011.
[61]R. He, W. S. Zheng, T. Tan, and Z. Sun, “Half-quadratic based iterative minimization for robust sparse representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, issue 2, pp. 261-275, 2014.
[62]X. T. Yuan and B. G. Hu, “Robust feature extraction via information theoretic learning,” Proc. Int’l Conf. Mach. Learn., pp. 1193-1200, 2009.
[63]R. He, W. S. Zheng, and B. G. Hu, “Maximum correntropy criterion for robust face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, issue 8, pp. 1561-1576, 2011.
[64]M. Nikolova and M. K. Ng, “Analysis of half-quadratic minimization methods for signal and image recovery,” SIAM J. Scientific Computing, vol. 27, issue 3, pp. 937-966, 2005.
[65]A. Y. Yang, S. S. Sastry, A. Ganesh, and Y. Ma, “Fast L1-Minimization algorithms and an application in robust face recognition: A review,” Proc. Int’l Conf. Image Process., pp. 1849-1852, 2010.
[66]R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, “Face detection in color images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, issue 5, pp. 696–706. 2002.
[67]A. S. Georghiades, P. N. Belhumeur and D. J. Kriegman, “From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, issue 6, pp. 643-660, June 2001.
[68]A. M. Martinez, “PCA versus LDA,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, issue 2, pp. 228-233, 2001.
F.GD-HASLR related works
[69]G. Hua, M. H. Yang, E. G. Learned-Miller, Y. Ma, M. Turk, D. J. Kriegman, and T. S. Huang, “Introduction to the special section on real-world face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 10, pp. 1921-1924, Oct. 2011.
[70]H. Jia and A. M. Martinez, “Support vector machines in face recognition with occlusions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 136-141, 2009
[71]Z. Zhou, A. Wagner, J. Wright, H. Mobahi and Y. Ma, “Face recognition with contiguous occlusion using Markov random fields,” Proc. IEEE Int''l Conf. Computer Vision, pp.1050-1057, 2009.
[72]X. X. Li, D. Q. Dai, X. F. Zhang, and C. X. Ren, “Structured sparse error coding for face recognition with occlusion,” IEEE Trans. Image Process., vol. 22, no. 5, pp. 1889-1900, May 2013.
[73]R. Liang and X. X. Li, “Mixed error coding for face recognition with mixed occlusions,” Proc. Int''l Joint Conf. Artificial Intelligence, pp. 3657-3663, 2015.
[74]S. Cai, L. Zhang, W. Zuo, and X. Feng, “A probabilistic collaborative representation based approach for pattern classification,” IEEE Conf. Computer Vision and Pattern Recognition, 2016.
[75]M. Yang, L. Zhang, J. Yang, and D. Zhang, “Robust sparse coding for face recognition,” IEEE Conf. Computer Vision and Pattern Recognition, pp. 625-632, 2011.
[76]M. Yang, L. Zhang, J. Yang, and D. Zhang, “Regularized robust coding for face recognition,” IEEE Trans. Image Process., vol. 22, no. 5, pp. 1753-1766, May 2013.
[77]M. Iliadis, H. Wang, R. Molina, and A. K. Katsaggelos, “Robust and low-rank representation for fast face identification with occlusions,” arXiv preprint arXiv: 1605.02266, 2016.
[78]D. Zhang, Y. Hu, J. Ye, X. Li and X. He, “Matrix completion by truncated nuclear norm regularization,” IEEE Conf. Computer Vision and Pattern Recognition, pp. 2192-2199, 2012.
[79]E. Candes and B. Recht, “Exact matrix completion via convex optimization,” Conf. Foundations of Computational Math., vol. 9, pp. 717-772, 2008.
[80]E. Elhamifar and R. Vidal, “Sparse subspace clustering: Algorithm theory and applications,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 2765-2781, Nov. 2013.
[81]G. Liu, H. Xu, and S. Yan, “Exact subspace segmentation and outlier detection by low-rank representation,” Proc. Int’l Conf. Artificial Intelligence and Statistics, pp. 703-711, 2012.
[82]L. Ma, C. Wang, B. Xiao, and W. Zhou, “Sparse representation for face recognition based on discriminative low-rank dictionary learning,” IEEE Conf. Computer Vision and Pattern Recognition, pp. 2586-2593, 2012.
[83]C. P. Wei, C. F. Chen, and Y. C. F. Wang, “Robust face recognition with structurally incoherent low-rank matrix decomposition,” IEEE Trans. Image Process., vol. 23, no. 8, pp. 3294-3307, Aug. 2014.
[84]J. Qian, L. Luo, J. Yang, F. Zhang, and Z. Lin, “Robust nuclear norm regularized regression for face recognition with occlusion,” Pattern Recognition, vol. 48, no. 10, pp. 3145-3159, Oct. 2015.
[85]R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd Edition, New Delhi: Pearson, 2008.
[86]X. Xiang, M. Dao, Gregory D. Hager, and T. D. Tran, “Hierarchical sparse and collaborative low-rank representation for emotion recognition,” Proc. IEEE Int’l Conf. Acoustics, Speech and Signal Processing, pp. 3811-3815, 2015.
[87]A. Lee, F. Caron, A. Doucet, and C. Holmes, “Bayesian sparsity-path-analysis of genetic association signal using generalized t priors,” Statistical Applications in Genetics and Molecular Biology, vol. 11, no. 2, pp. 1-29, 2012.
[88]Y. Taigman, M. Yang, M. A. Ranzato, and L. Wolf, “DeepFace: Closing the gap to human-level performance in face verification,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1701-1708, 2014.
[89]F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 815-823, 2015.
[90]O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” British Machine Vision Conference, pp. 1–12, 2015.
[91]G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments,” Technical Report 07-49 UMass, vol. 1, no. 2, pp. 1-11, 2007.
[92]L. Wolf, T. Hassner and I. Maoz, “Face recognition in unconstrained videos with matched background similarity,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.529-534, 2011.
[93]D. G. Lowe, “Object recognition from local scale-invariant features,” Proc. Int''l Conf. Computer Vision, pp. 1150-1157, 1999.
[94]G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, “Sparse representations of image gradient orientations for visual recognition and tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, pp. 26-33, 2012.
[95]O. E. Barndorff-Nielsen, “Normal inverse Gaussian distributions and stochastic volatility modeling,” Scand. J. Stat., vol. 24, pp. 1-13, Mar. 1997.
[96]K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” Proc. Int’l. Conf. Learn. Representations, pp. 1-14, 2015.
[97]M. M. Ghazi and H. K. Ekenel, “A Comprehensive analysis of deep learning based representation for face recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 102–109, 2016.
[98]X. Wu, R. He, and Z. Sun, “A lightened CNN for deep face representation,” arXiv preprint arXiv: 1511.02683, 2015.
[99]X. Wu, “Learning robust deep face representation,” arXiv preprint arXiv: 1507.04844, 2015.
[100]A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Proc. Adv. Neural Inf. Process. Syst., pp. 1106–1114, 2012.
[101]M. Lin, Q. Chen, and S. Yan, “Network in network,” Computing Research Repository, arXiv preprint arXiv: 1312.4400v3, 2013.
[102]D. L. Donoho, “Denoising by soft-thresholding,” IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 613-627, Mar. 1995.
[103]T. Sim, S. Baker, and M. Bsat, “The CMU pose, illumination, and expression (PIE) database,” Proc. IEEE Conf. Face and Gesture Recognition, pp. 46–51, 2002.
G.Nonconvex sparse and low-rank model related work
[104]R. Basri, D. Jacobs, “Lambertian reflection and linear subspaces,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 3, pp. 218-233, Mar. 2003.
[105]C. Eckart, G. Young, “The approximation of one matrix by another of lower rank,” Psychometrika, vol. 1, no. 3, pp. 211-218, 1936.
[106]Z. Lin, R. Liu, Z. Su, "Linearized alternating direction method with adaptive penalty for low-rank representation", Proc. Adv. Neural Inf. Process. Syst., pp. 1-7, 2011.
[107]X. Zhong, L. Xu, Y. Li, Z. Liu, E. Chen, “A nonconvex relaxation approach for rank minimization problems,” Proc. AAAI Conf. Artif. Intell., pp. 1980-1987, 2015.
[108]L. E. Frank, J. H. Friedman, “A statistical view of some chemometrics regression tools,” Technometrics, vol. 35, no. 2, pp. 109-135, 1993.
[109]J. Trzasko, A. Manduca, “Highly undersampled magnetic resonance image reconstruction via homotopic l0-minimization ,” IEEE Trans. Med. Imag., vol. 28, no. 1, pp. 106-121, Jan. 2009.
[110]C. H. Zhang, “Nearly unbiased variable selection under minimax concave penalty,” Ann. Statist., vol. 38, no. 2, pp. 894-942, Apr. 2010.
[111]T. Zhang, “Analysis of multi-stage convex relaxation for sparse regularization,” J. Mach. Learn. Res., vol. 11, pp. 1081-1107, Jan. 2010.
[112]J. Fan, R. Li, “Variable selection via nonconcave penalized likelihood and its oracle properties,” J. Amer. Statist. Assoc., vol. 96, no. 456, pp. 1348-1360, 2001.
[113]C. Gao, N. Wang, Q. Yu, and Z. Zhang, “A feasible nonconvex relaxation approach to feature selection,” Proc. AAAI Conf. Artif. Intell., pp. 356-361, 2011.
[114]D. Bertsekas, Convex Optimization Algorithms, Belmont, MA, USA:Athena Scientific, 2015.
[115]C. Lu, C. Zhu, C. Xu, S. Yan, Z. Lin, “Generalized singular value thresholding,” Proc. AAAI Conf. Artif. Intell., pp. 1805-1811, 2015.
[116]C. Lu, J. Tang, S. Yan, and Z. Lin, “Nonconvex nonsmooth low rank minimization via iteratively reweighted nuclear norm,” IEEE Trans. Image Process., vol. 25, no. 2, pp. 829-839, Feb. 2016.
[117]P. Gong, C. Zhang, Z. Lu, J. Huang, J. Ye, “A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems,” Proc. Int. Conf. Mach. Learn., pp. 37-45, 2013.
[118]Y. Hu, D. Zhang, J. Ye, X. Li, X. He, “Fast and accurate matrix completion via truncated nuclear norm regularization,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 9, pp. 2117-2130, Sep. 2013.
[119]Q. Liu, Z. Lai, Z. Zhou, F. Kuang, and Z. Jin, “A truncated nuclear norm regularization method based on weighted residual error for matrix completion,” IEEE Trans. Image Process., vol. 25, issue 1, pp. 316–330, Jan. 2016.
[120]D. Gabay, B. Mercier, “A dual algorithm for the solution of nonlinear variational problems via Finite-Element Approximations,” Computer Math. Applications, vol. 2, pp. 17-40, 1976.
[121]Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices", UIUC Technical Report, Aug. 2009.
[122]J. Bolte, S. Sabach, M. Teboulle, “Proximal alternating linearized minimization for nonconvex and nonsmooth problems,” Math. Progr., pp. 460-494, Jul. 2013.
[123]S. Ji, J. Ye, “An accelerated gradient method for trace norm minimization,” Proc. 26th ICML Conf., pp. 457–464, 2009.
[124]C. J. Hsieh, P. Olsen, “Nuclear norm minimization via active subspace selection,” Proc. 31th ICML Conf., pp. 575–583, 2014.
[125]J. Feng, H. Xu, S. Yan, “Online robust PCA via stochastic optimization,” Proc. Adv. Neural Inf. Process. Syst., pp. 404-412, 2013.
[126]Q. Zhao, D. Meng, Z. Xu, W. Zuo, L. Zhang, “Robust principal component analysis with complex noise,” Proc. 31th Int’l Conf. Mach. Learn., pp. 55-63, 2014.
[127]S. D. Babacan, M. Luessi, R. Molina, A. K. Katsaggelos, “Sparse Bayesian methods for low-rank matrix estimation,” IEEE Trans. Signal Process., vol. 60, no. 8, pp. 3964-3977, Aug. 2012.
[128]F.-F. Li, R. Fergus, P. Perona, “Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 178-188, 2004.
[129]Z. Lin, R. Liu, Z. Su, “Linearized alternating direction method with adaptive penalty for low-rank representation,” Proc. Adv. Neural Inf. Process. Syst., pp. 612-620, 2011.
[130]J. Shi, J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000..
[131]S. Xiao, M. Tan, D. Xu, “Weighted block-sparse low rank representation for face clustering in videos,” Proc. Eur. Conf. Comput. Vis., pp. 123-138, Sep. 2014.
[132]N. Halko, P. Martinsson, and J. Tropp, “Finding Structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions,” SIAM Rev., vol. 53, no. 2, pp. 217-288, 2011.
[133]J. Costeira and T. Kanade, “A multibody factorization method for independently moving objects,” Int’l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998.