臺灣博碩士論文加值系統

隨著科技的進步，顯示器解析度的大幅提升，傳統影像放大方法已經無法將目前主流的解析度(1080 p)有效的提升至4K甚至是8K的解析度了。近年來由於硬體的進步，深度學習相關的技術也有大幅的成長，開始有人將深度學習(Deep Learning)相關的技術用於超高解析度的領域上。實驗結果發現，相對於傳統甚至是其他機器學習相關的方法在影像恢復的品質上都有大幅度的提升，此種高倍率的影像放大問題便不再是遙不可及的夢想。爾後因為物件分類和偵測領域上，發現神經網路越深效果可以越好，超高解析度領域便開始有人將神經網路加深，實驗結果也證實其影像品質的恢復相對於一開始的淺層網路恢復要來的好。後來研究相關題材的人員，基本上都採取較深的神經網路，但是網路的加深，勢必代表著參數量的提升，導致內存占用加大，對於將訓練好的模型放在一些內存占用較少的系統(e.g. 嵌入式系統)上使用也變得越來越不可能。基於上述問題，本論文提出一個低複雜度(內存占用少)的神經網路架構用來解決超高解析度的問題，相對於其他基於深度學習相關技術實作的低複雜度(內存占用少)超高解析度架構能有更好的影像恢復品質。
本論文提出的多尺度二值化殘差網路有幾個特色: 1)藉由多尺度放大的架構，可以達到輸入一張低解析度圖片，同時得到三種不同放大倍率的高解析度圖片。由於這種多尺度的設計，神經網路中的某些參數還可以共用，達到參數減少的目的，進而降低內存的使用。2)使用了殘差結構，每個殘差結構裡的輸出能更專注的學習與輸入的差別，讓堆疊較深的神經網路能夠有效收斂。3)引進了參數二值化，讓神經網路中的大部分的參數都可以實行二值化，進而降低內存的使用。

With the advancement of technology, the resolution of the display has been greatly improved, and the conventional image Super-Resolution method is unable to effectively upsample the current mainstream resolution (1080p) to 4K or even 8K resolution. In recent years, due to the advancement of hardware, the technology related to deep learning has also grown substantially. Some people have begun to use deep learning related technology in the field of Super-Resolution. The experimental results show that compared with the traditional and even other machine learning related methods, the quality of image restoration has been greatly improved. This high-scale Super-Resolution problem is no longer an unreachable dream. Later, because of the field of image classification and detection, it was found that the deeper of the neural network, the better of the performance, and the Super-resolution field began to deepen the neural network. The experimental results also confirmed that the restoration of image quality is better than the initial shallow network recovery. Later, people who studied related topics basically took a deeper neural network, but the deepening of the network will inevitably represent an increase in the amount of parameters, resulting in an increase in memory usage, and becomes increasingly impossible to use a trained model on some systems with less memory footprint (e.g. embedded systems). Based on the above problems, this paper proposes a low complexity (less memory footprint) neural network architecture to solve the problem of Super-Resolution, compared to other low complexity (less memory footprint) neural network architecture which solve the problem of Super-Resolution architecture providing better image recovery quality.
The multi-scale binary residual network proposed in this paper has several features: 1) By adopting a multi-scale upsampling architecture, it is possible to input a low-resolution image and obtain three high-resolution images with different scales. Due to this multi-scale design, certain parameters in the neural network can also be shared, achieving the purpose of parameter reduction, thereby reducing memory usage. 2) Using the residual structure, the output in each residual structure can focus on learning the difference between input, allowing the deeper neural network to converge effectively. 3) The binary weights is introduced, so that most of the parameters of the neural network can be binarized, thereby reducing the use of memory.

摘要 I
Abstract II
誌謝 IV
Contents V
Figure Captions VII
Table VIII
Chapter 1. Introduction 1
1.1 Motivation 1
1.2 Background 1
1.3 Organization 3
Chapter 2. Related Work 4
2.1 Single-Image Super-Resolution 4
2.2 Residual Neural Network 4
2.2.1 ResNet 5
2.2.2 VDSR 5
2.3 Binary Weights 6
2.4 Multi-Scale 6
Chapter 3. Proposed Method 8
3.1 Proposed Network 8
3.1.1 Feature Extraction 8
3.1.2 Main Branch 9
3.1.3 Sub Branch 10
3.1.4 Upsampling 10
3.2 Residual Learning 11
3.3 Binary Weights 12
3.4 Multi-scale 14
Chapter 4. Experiments and Comparisons 17
4.1 Datasets 17
4.2 Implementation and Training Details 17
4.3 Study of Channel and Depth size 18
4.4 Study of Residual Structure 20
4.5 Comparison of Memory usage 21
4.6 Activation fucntion 22
4.7 Comparison of Other Models 23
Chapter 5. Conclusion and Future Work 30
References 31

[1] R. Keys (1981). Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing. 29 (6):1153–1160. doi:10. 1109/TASSP.1981.1163711.
[2] Numerical Recipes in C, 1988–92 Cambridge University Press, ISBN 0-521-43108-5, pp. 123–128.
[3] J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution via sparse representation. IEEE Transactions on image processing, 19(11):2861–2873, 2010.
[4] S. Schulter, C. Leistner, and H. Bischof. Fast and accurate image upscaling with super-resolution forests. In CVPR, 2015.
[5] C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295–307, 2015.
[6] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015.
[7] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaussian Denoiser: Residual learning of deep CNN for image denoising, IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, Jul. 2017.
[8] C. Dong, C. Loy, and X. Tang. Accelerating the super-resolution convolutional neural network. In ECCV, 2016.
[9] J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks, in IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[10] B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee. Enhanced deep residual networks for single image super-resolution. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.
[11] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, ´ A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802, 2016.
[12] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
[13] Courbariaux, M., Bengio, Y., David, J.P.: Binaryconnect: Training deep neural networks with binary weights during propagations. In: Advances in Neural Information Processing Systems. (2015) 3105–3113
[14] Courbariaux, M., Bengio, Y.: Binarynet: Training deep neural networks with weights and activations constrained to +1 or -1. CoRR (2016)
[15] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. Xnornet: Imagenet classification using binary convolutional neural networks. arXiv preprint arXiv:1603.05279, 2016.
[16] R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang, B. Lim, S. Son, H. Kim, S. Nah, K. M. Lee, et al. Ntire 2017 challenge on single image super-resolution: Methods and results. In CVPRW, 2017.
[17] M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. AlberiMorel. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVC, 2012.
[18] R. Zeyde, M. Elad, and M. Protter. On single image scale-up using sparse-representations. In Proc. 7th Int. Conf. Curves Surf., 2010.
[19] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, 2001.
[20] J.-B. Huang, A. Singh, and N. Ahuja. Single image superresolution from transformed self-exemplars. In CVPR, 2015.
[21] Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, T. Ogawa, T. Yamasaki, and K. Aizawa. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications, 2017.
[22] D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2014.
[23] He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: ICCV. (2015) 1026–1034
[24] P. C. Lin. Image Super-Resolution via Deep Level Residual Network, 2018
[25] J. Kim, J. K. Lee, and K. M. Lee. Deeply-recursive convolutional network for image super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[26] T. Tong, G. Li, X. Liu, and Q. Gao. Image super-resolution using dense skip connections. In ICCV, 2017.
[27] Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Fast and accurate image super-resolution with deep laplacian pyramid networks. arXiv:1710.01992 (2017)