臺灣博碩士論文加值系統

本論文主要探討將卷積神經網路(CNN)應用於水彩混色預測的模型，以及利用色彩感知誤差作為損失函式對於模型訓練的影響。目前顏料混色之研究多以Kubelka-Munk理論作為基礎，然而K-M理論本身的限制，無法完全符合半透明的水彩顏料預測之需求。因此，設計一個更符合的水彩顏料的混色模型，能夠對未來水彩模擬以及相關應用有所貢獻。本文用一多層卷積神經網路來學習兩顏料混色後的反射頻譜。訓練資料部分使用[Chen et al. 2018]之水彩資料庫，訓練目標是最小化混色後的反射頻譜在上與真實混色之色彩誤差。透過本文所提出的模型以及損失函式，混色預測在測試資料中，有88.7%的結果能夠達到∆ELab < 5的色彩誤差，也就是人眼無法輕易分辨其色差之程度。 此外，本研究發現一發生在預測結果上之特別現象。此模型透過色感誤差的損失函式訓練後所產生的結果在頻譜上，相較於真實混色，會有中幅度的波動，無法貼近真實頻譜。然而在色彩上，卻因為損失函式，而能在色彩上與真實混色貼近。

In the paper, we propose a model to predict the mixture of watercolor pigments using convolutional neural networks (CNN). With a watercolor dataset, we train our model to minimize the loss function of sRGB differences. In metric of color difference ∆ELab, our model achieves 88.7% of data that ∆ELab < 5 on the test set, which means the difference cannot easily be detected by the human eye. In addition, an interesting phenomenon is found; Even if the reflectance curve of the predicted color is not as smooth as the ground truth curve, the RGB color is still close to the ground truth.

誌謝 v
Acknowledgements vii
摘要 ix
Abstract xi
1 Introduction 1
2 RelatedWork 3
2.1 Kubelka-Munk Model for pigment color mixing . . . . . . . . . . . . . 3
2.2 Neuron Network for Color Mixing . . . . . . . . . . . . . . . . . . . . . 5
2.3 CIE Color Matching Functions . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 Development of Convolutional Neuron Network. . . . . . . . . . . . . . 9
3 Method 11
3.1 Convolutional Network for Pigment Mixing . . . . . . . . . . . . . . . . 12
3.2 Perceptual Loss Function to Color . . . . . . . . . . . . . . . . . . . . . 15
4 Experiments 21
4.1 Training Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5 Discussion 31
6 Conclusion 35
Bibliography 37

[1] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al. Tensorflow: A system for large-scale machine learning. In OSDI, volume 16, pages 265–283, 2016.
[2] O. Abdel-Hamid, A.-r. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu. Convolutional neural networks for speech recognition. IEEE/ACM Transactions on audio, speech, and language processing, 22(10):1533–1545, 2014.
[3] W. Baxter, J. Wendt, and M. C. Lin. Impasto: a realistic, interactive model for paint. In Proc. of the 3rd international symposium on Non-photorealistic animation and rendering, pages 45–148. ACM, 2004.
[4] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. In Advances in neural information processing systems, pages 153–160, 2007.
[5] C. M. Bishop and C. Roach. Fast curve fitting using neural networks. Review of scientific instruments, 63(10):4450–4456, 1992.
[6] K. Chellapilla, S. Puri, and P. Simard. High performance convolutional neural networks for document processing. In Tenth International Workshop on Frontiers in Handwriting Recognition. Suvisoft, 2006.
[7] M.-Y. Chen, C.-S. Yang, and M. Ouhyoung. A smart palette for helping novice painters to mix physical watercolor pigments. In E. Jain and J. Kosinka, editors, Proc. of EuroGraphics 2018, Posters, page April 2018. The Eurographics Association,2018.
[8] C. J. Curtis, S. E. Anderson, J. E. Seims, K. W. Fleischer, and D. H. Salesin. Computer-generated watercolor. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 421–430. ACM Press/Addison-Wesley Publishing Co., 1997.
[9] P. Edström. Examination of the revised kubelka-munk theory: considerations of modeling strategies. JOSA A, 24(2):548–556, 2007.
[10] J. Guild et al. The colorimetric properties of the spectrum. Phil. Trans. R. Soc. Lond. A, 230(681-693):149–187, 1931.
[11] C. S. Haase and G. W. Meyer. Modeling pigmented materials for realistic image synthesis. ACM Transactions on Graphics (TOG), 11(4):305–335, 1992.
[12] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proc. of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
[13] G. E. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527–1554, 2006.
[14] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[15] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
[16] P. Kubelka. New contributions to the optics of intensely light-scattering materials. part i. Josa, 38(5):448–457, 1948.
[17] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989.
[18] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
[19] W. Mokrzycki and M. Tatol. Colour differenceΔ e-a survey. MGV, 20(4):383–411, 2011.
[20] C. Poultney, S. Chopra, Y. L. Cun, et al. Efficient learning of sparse representations with an energy-based model. In Advances in neural information processing systems, pages 1137–1144, 2007.
[21] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[22] D. Steinkraus, I. Buck, and P. Simard. Using gpus for machine learning algorithms. In Document Analysis and Recognition, 2005. Proceedings. Eighth International Conference on, pages 1115–1120. IEEE, 2005.
[23] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, et al. Going deeper with convolutions. Cvpr, 2015.
[24] S. Westland, L. Iovine, and J. M. Bishop. Kubelka-munk or neural networks for computer colorant formulation? In 9th Congress of the International Colour Association, volume 4421, pages 745–749. International Society for Optics and Photonics, 2002.
[25] W. D. Wright. A re-determination of the trichromatic coefficients of the spectral colours. Transactions of the Optical Society, 30(4):141, 1929.
[26] S. Xu, H. Tan, X. Jiao, F. Lau, and Y. Pan. A generic pigment model for digital painting. In Computer Graphics Forum, volume 26, pages 609–618. Wiley Online Library, 2007.
[27] M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818–833. Springer, 2014.