在本論文中，我提出了一個通用的無參考式影片品質評估方法，本方法主要基於2維的卷積神經網絡(CNN)、多層感知機(MLP)和支持向量迴歸(SVR)模型的組合。首先，我分別沿著三個不同的維度，從影片中提取出三種切片，包括一個空間切片和兩個時空切片，而這些切片同時包含了空間和時間特性。接下來，為了使本架構更符合人類視覺系統的特性，對於擁有較多受損像素的特定幀(key frame)，或是前後幀內容較不連續的特定幀，進行提取，並通過使用一個預訓練(pre-trained)的2維CNN模型，從空間和時空的特定幀中提取深度的特徵，且為了取得更多元化的資訊，我從CNN的不同層中，提取出不同的特徵，並將這些空間和時空特徵輸入到每個MLP中生成質量分數。最後將每個從MLP得到的質量分數輸入至SVR中，進行分數的選擇，並得到最終的感知影片品質分數。我提出的方法分別在三種資料庫上進行測試，分別是LIVE Video Quality Database、CSIQ Video Quality Database和 IVPL Video Quality Database，而實驗的結果表明，我的方法與其他現有的無參考式方法相比，有著不錯的表現，而且比較起現有的全參考式方法，仍然具有競爭力。

In this paper, we propose a general-purpose no-reference (NR) video quality assessment (VQA) metric based on the cascade combination of 2D convolutional neural network (CNN), multi-layer perceptron (MLP), and support vector regression (SVR) model. First, we extract frames along three kinds of dimensions, including two spatial directions and one temporal direction, to consider spatial and temporal properties simultaneously. Then, the deep features are extracted from slices of both spatial and spatiotemporal domains by using a 2D pre-trained CNN. In addition, we also extract various features from different layers of CNN model for capturing more diverse contents of videos. These features can capture different aspects of video frames for predicting quality scores, and we take these features as inputs of MLP to obtain a few estimated quality scores on different perspectives. Finally, these estimated scores are selected on SVR fusion stage. And the final quality score is predicted by inputting estimated scores of the best combination into SVR model. The proposed method is evaluated on the well-known Video database of Laboratory for Image & Video Engineering (LIVE), Image & Video Processing Laboratory (IVPL), and CSIQ. And the experimental result demonstrates that our method is competitive with other state-of-the-art and full-reference (FR) and well-performing NR VQA metrics.

摘要 i
Abstract ii
目錄 iii
圖目錄 iv
表目錄 vi
第一章 緒論 1
1.1 前言 1
1.2 論文的動機與大綱 2
第二章 文獻探討與相關背景知識 4
2.1 文獻探討 4
2.2 相關背景知識 6
2.2.1 多層感知機 6
2.2.2 卷積神經網絡 8
2.2.3 支持向量迴歸 12
第三章 論文研究方法 14
3.1 空間與時間的影片切片 16
3.2 特定幀選取 17
3.3 特徵提取 20
3.4 預測品質分數 27
3.5 融合分數 39
第四章 實驗結果 43
4.1 資料庫介紹 43
4.2 實驗參數設定 51
4.3 實驗分析與結果討論 53
第五章 結論與未來展望 62
參考文獻 63

[1] S. W. "A perceputal distortion metric for digital color video," SPIE Human Vis. Electron. Imag. IV, vol. 3644, pp. 175-184, 1999.
[2] C. J. van den Branden Lambrecht, D. M. Costantini, G. L. Sicuranza,, "Quality assessment of motion rendition in video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 5, pp. 766-782, Aug. 1999.
[3] W. Lu, X. Li, X. Gao, W. Tang, J. Li, and D. Tao, "A video quality assessment metric based on human visual system," Cognit. Comput., vol. 2, no. 2, pp. 120-131, Jun. 2010.
[4] P. V. Vu, C. T. Vu, and D. M. Chandler, "A spatiotemporal mostapparent-distortion model for video quality assessment," in Proc. 18th IEEE Int. Conf. Image Process. (ICIP), pp. 2505-2508, Sep. 2011.
[5] P. V. Vu and D. M. Chandler, "ViS3: An algorithm for video quality assessment via analysis of spatial and spatiotemporal slices," J. Electron. Imag., vol. 23, no. 1, pp. 013016-1-013013-25, 2014.
[6] K. Seshadrinathan and A. C. Bovik, "Motion tuned spatio-temporal quality assessment of natural videos," IEEE Trans. Image Process., vol. 19, no. 2, p. 335–350, Feb. 2010.
[7] M. H. Pinson and S. Wolf, "A new standardized method for objectively measuring video quality," IEEE Trans. Broadcast., vol. 10, no. 3, p. 312–322, Sep. 2004.
[8] Z. Wang, L. Lu, and A. C. Bovik, "Video quality assessment based on structural distortion measurement," Signal Process., Image Commun., vol. 19, no. 2, p. 121–132, Feb. 2004.
[9] A. B. Watson, J. Hu, and J. F. McGowan, "DVQ: A digital video quality metric based on human vision," J. Electron. Imag., vol. 10, no. 1, p. 20–29, Jan. 2001.
[10] M. Barkowsky, J. Bialkowski, B. Eskofier, R. Bito, and A. Kaup, "Temporal trajectory aware video quality measure," IEEE J. Sel. Topics, vol. 3, no. 2, p. 266–279, Apr. 2009.
[11] A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba, "Considering temporal variations of spatial visual distortions in video quality assessment," IEEE J. Sel. Topics Signal Process., vol. 3, no. 2, pp. 253-265, Apr. 2009.
[12] Manasa, K., and Sumohana S. Channappayya., "An optical flow-based full reference video quality assessment algorithm," IEEE Trans. Image Process., vol. 25, no. 6, pp. 2480-2492, June 2016.
[13] R. Soundararajan and A. C. Bovik, "Video quality assessment by reduced reference spatio-temporal entropic differencing," IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 4, pp. 684-694, Apr. 2012.
[14] L. Ma, S. Li, and K. N. Ngan, "Reduced-Reference Video Quality Assessment of Compressed Video Sequences," IEEE Trans. Circuits and Systems for Video Technology, vol. 22, no. 10, pp. 1441-1456, Oct. 2012.
[15] M. Masry, S. S. Hemami, and Y. Sermadevi, "A scalable wavelet-based video distortion metric and applications," IEEE Trans. Circ. Syst. Video Technol., vol. 16, no. 2, pp. 260-273, Feb. 2006.
[16] G. Valenzise, S. Magni, M. Tagliasacchi, and S. Tubaro, "No-reference pixel video quality monitoring of channel-induced distortion," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 4, pp. 605-618, Apr. 2012.
[17] F. Zhang, W. Lin, Z. Chen, and K. N. Ngan, "Additive log-logistic model for networked video quality assessment," IEEE Trans. Image Process., vol. 22, no. 4, pp. 1536-1547, Apr. 2013.
[18] J. Wang, Z. Wang, F. Wang, T. Rahim, Z. Fei, "A no-reference video quality assessment method for VoIP applications," in 13th IEEE Int. Conf. Signal Processing. (ICSP), Nov. 2016.
[19] K. Zhu, C. Li, V. Asari, and D. Saupe, "No-Reference Video Quality Assessment Based on Artifact Measurement and Statistical Analysis," IEEE Trans. Circuits and Systems for Video Technology, vol. 25, no. 4, pp. 533-546, Apr. 2015.
[20] K.-C. Yang, C. C. Guest, K. El-Maleh, and P. K. Das, "Perceptual temporal quality metric for compressed video," IEEE Trans. Multimedia, vol. 9, no. 7, pp. 1528-1535, Nov. 2007.
[21] M. A. Saad, A. C. Bovik, and C. Charrier, "Blind prediction of natural video quality," IEEE Trans. Image Process., vol. 23, no. 3, pp. 1352-1365, Mar. 2014.
[22] A. Mittal, M. A. Saad, and A. C. Bovik, "A Completely Blind Video Integrity Oracle," IEEE Trans. Image Process., vol. 25, no. 1, pp. 289-300, Jan. 2016.
[23] Y. Li, L.-M. Po, C.-H. Cheung, X. Xu, L. Feng, F. Yuan, and K.-W. Cheung, "No reference video quality assessment with 3d shearlet transform and convolutional neural networks," IEEE Trans. Circuits and Systems for Video Technology, vol. 26, no. 6, pp. 1044-1057, Jun. 2016.
[24] X. Li, Q. Guo, and X. Lu, "Spatiotemporal Statistics for Video Quality Assessment," IEEE Trans. Image Process., vol. 25, no. 7, Jul. 2016.
[25] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A Large-Scale Hierarchical Image Database," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009.
[26] A. Krizhevsky, I. Sutskever, and G. E Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, pp. 1097-1105, 2012.
[27] K. Simonyan, and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," in 3rd Int. Conf. Learning Representations (ICLR), 2015.
[28] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2015.
[29] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, "Going Deeper with Convolutions," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2015.
[30] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z, "Rethinking the inception architecture for computer vision," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 2818-2826, 2016.
[31] C.-W. Ngo, T.-C. Pong, and H.-J. Zhang, "Motion analysis and segmentation through spatio-temporal slices processing," IEEE Trans. Image Process, vol. 12, no. 3, pp. 341-355, 2003.
[32] [Online]. Available: https://ai.googleblog.com/2016/08/improving-inception-and-image.html. [Accessed Dec. 2018].
[33] C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, p. 27:1–27:27, 2011.
[34] K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, "Study of subjective and objective quality assessment of video," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1427-1441, Jun 2010.
[35] Image & Video Processing Laboratory, The Chinese University of, "IVP subjective quality video database," [Online]. Available: http://ivp.ee.cuhk.edu.hk/research/database/subjective/index.shtml. [Accessed Dec. 2018].
[36] Z. Li, A. Aaron, I. Katsavounidis, A. Moorthy, and M. Manohara, "Toward A Practical Perceptual Video Quality Metric," [Online]. Available: http://techblog.netflix.com/2016/06/toward-practical-perceptual-video.html. [Accessed Dec. 2018].