臺灣博碩士論文加值系統

本論文探討近年來流行的磨課師(Massive Open Online Courses, MOOCs)課程及翻轉教室(Flipped classroom)的數位課程內容製作之拍攝課程技術，以如何實現即時渲染背景移除技術，藉以獲得較良好的數位內容製作品質。
在本研究提出的前景移除方法中，將擷取出來的PPT (Microsoft Power Point)簡報文字放置於前景，然後與攝影機擷取到的人物影像作為中景，且與自行定義的背景進行快速合成，這裡所提到的應用就是利用即時渲染背景移除技術，藉以實現多層場景之融合應用，本研究使用的軟體套件為OBS-Studio (Open Broadcaster Software)，其中提供色彩嵌空(Chroma Key)及色鍵(Color Key)兩種背景移除技術，這兩種方法基本上為取固定顏色的背景加以移除，只差在色彩嵌空多做了色溢出(Color Spill)的處理，除了能去掉綠色背景外，還可將殘留在物體邊緣的綠色去除，色彩嵌空跟顏色去背在OBS-Studio軟體項目裡屬於視訊特效濾鏡，其中包含8種可調整的影像處理項目，分別是關鍵顏色類型(Key Color Type)、相似性(Similarity)、平滑度(Smoothness)、關鍵色溢出減少(Key Color Spill Reduction)、不透明度(Opacity)、對比(Contrast)、亮度(Brightness)、及伽瑪(Gamma)。從中發現此技術的調整項目過多，參考網路及期刊發現有許多使用者，包括課程錄製教師反映，因為參數過多所以不好做調整，為了方便使用者快速可使用，另行開發可滿足使用者處理需求的外掛軟體。其中OBS-Studio軟體套件允許額外的開發者開發適合自己攝影情境下所使用的外掛功能，並在專門的論壇網站可以找到C++、Python、Lua等三種程式語言的外掛開發方式以及教學，在過濾器方面的開發只支援C++版本，可以配合建立在OBS-Studio專案裡的DirectX (DX)及OpenGL (Open Graphics Library) API (Application Programming Interface) 做函數調用，自製的外掛軟體包含幾項處理步驟，首先將輸入影像轉換成RGB到YUV的色彩空間，然後影像通過著色器(Shader)處理產生了紋理座標且貼影像到2維平面上，也就是對紋理座標取樣到U及V軸的平面上，再來通過取樣後的文字影像邊緣有可能產生不平順的鋸齒狀，所以再將影像做卷積過濾處理，然後由計算出的色度距離(Chroma Distance)與相似性做減法處理而產生基本遮罩，而後由基本遮罩除以平滑度再限制它的值在0到1的合理範圍內而得到完全遮罩，接下來色溢出的處理是由基本遮罩除以色溢出，再將得到的值限制在合理範圍內，外掛程式最後一步才是做亮度、對比、gamma的計算。其中的卷積處理可以替換掉不同的核心(Kernel)矩陣，進而得到效果不同的文字影像處理效果，可替換的效果有邊緣偵測(Edge detection)、銳利化(Sharpen)、平均值模糊(Box blur)、高斯模糊(Gaussian blur)。
最後在影像比對方面，為了得到一個合理又有效的色彩嵌空參數值，方便導播拍攝課程，將輸入及輸出的影像採用批次的方法擷取下來，然後擷取下來的輸入影像通過MATLAB製作的影像比對程式之後，將會得到最佳的實驗結果，且分析得到使用色彩嵌空時的最優參數設定，所採用的影像比對方法為許多論文所使用的客觀比對法，基於均方誤差、峰值信噪比、結構相似性，以上所寫到的技術及計算的詳細細節請參考本論文的內容。

This thesis explores the popular recording techniques for the Massive Open Online Courses (MOOCs) in order to achieve real-time rendering background removal technology and then obtain the good quality of digital content.
In the foreground removal method proposed in this thesis, the extracted Microsoft Power Point (PPT) presentation text is placed in the foreground, then the image captured by the camera is used as the medium scene, and the background is quickly synthesized with the self-defined background. The application mentioned here is to use the real-time rendering background removal technology to achieve the fusion application of multi-layer scenes. The software tool used in this thesis is Open Broadcaster Software Studio (OBS-Studio), which provides two background removal technologies, chroma key and color key. These two methods remove the background of the fixed color where only the difference in the chroma key works more than the color spill processing, except that the green background can be removed. It can also remove the green color left on the edge of the object. Chroma key and color key belong to video effect filter in OBS-Studio software project, which contains 8 kinds of adjustable image processing items including key color type, similarity, smoothness, key color spill reduction, opacity, contrast, brightness and gamma which have found that there are too many adjustment items for this technology. The reference network and journals found that there are many users including teachers who join in the mission to record the digital content. It is not easy to adjust because there are too many parameters. In order to facilitate the user quickly to use OBS-Studio, some plug-in functions are developed to solve the problem.
The software tool of OBS-Studio allows developers to develop plug-in software for situated learning, and provides plug-in development methods with tutorial lectures in C++, Python, and Lua on a dedicated forum website. However, the development of filters is only available in C++. It can be used to make function calls with the DirectX (DX) and Open Graphics Library (OpenGL) Application Programming Interfaces (APIs) built in the OBS-Studio project. The self-made plug-in software processing includes several steps. First, convert the input image from RGB format to YUV color space. Then, the image is passing into the shader. The process of shader produces texture coordinates and attaches the image to a 2-dimensional plane. That is, the texture coordinates are sampled onto the planes of the U and V axes, and then the edges of the sampled text image may have an uneven jagged. Therefore, the image is passing through the convolution filter, and then the calculated chroma distance and similarity are subtracted to produce a base mask. In addition, the base mask is divided by the smoothness. Afterward, its value is within a reasonable range of 0 to 1 to obtain a full mask. After that, the color spill processing is to divide the basic mask by the color spill and then limit the obtained value to a reasonable range. The final step is to make plug-ins brightness, contrast, and gamma to be calculated, where the convolution processing could replace different kernel matrices, and then get different effects of text and image processing. The alternative effects are edge detection, sharpen, and box blur, and Gaussian blur.
Finally, in the aspect of image matching to obtain a reasonable and effective chroma key parameter value, it is convenient to switch the shooting course, and the input and output images are extracted by batch method, and then the captured input image will get the best experimental results after the image matching program made by MATLAB. The analysis shows the optimal setting of parameter when using chroma key. The image matching method is the objective comparison method used in many thesis, based on the mean square error, the peak signal to noise ratio, structural similarity.

摘要
Abstract
LIST OF FIGURES
LIST OF TABLES
1.1 Overview
1.2 Motivation
1.3 Thesis Organization
Chapter 2 Background
2.1 Color Space
2.1.1 International Telecommunication Union Recommendation (ITU-R) BT. 709
2.1.2 YUV Common Format
2.1.3 Convert YUV to RGB
2.1.4 Convert RGB to YUV
2.2 Alpha Matting
2.2.1 Image Matting
2.2.2 Chroma Key
2.2.3 Sampling Strategies
2.2.4 Y Luminance Matting
2.2.5 Double Layer Mask
2.3 Convolution
2.3.1 Kernel Characteristics
2.3.2 Kernel Calculation Method
2.4 Image Matching
2.4.1 Computer Font
2.4.2 Mean Squared Error (MSE)
2.4.3 Peak Signal-to-Noise Ratio (PSNR)
2.4.3 Structural Similarity (SSIM)
2.5 Open Broadcaster Software Studio (OBS-Studio)
2.5.1 History
2.5.2 Real-Time Rendering Chroma Key Technology
2.5.3 Chroma Key Parameter
2.5.4 Chroma Key Similarity Parameter
2.5.5 Chroma Key Smoothness Parameter
2.5.6 Key Color Spill Reduction Parameter of Chroma Key
2.5.7 Chroma Key Opacity Parameter
2.5.8 Chroma Key Contrast Parameter
2.5.9 Chroma Key Brightness Parameter
2.5.10 Chroma Key Gamma Parameter
2.5.11 Plugin
2.6 Direct X
2.6.1 DX11 Texture Coordinates
2.6.2 Texture Filtering
2.6.3 Vertex Shader and Pixel Shader
2.7 MATLAB
2.7.1. Batch
2.7.2 Draw 3D Graph
Chapter 3 System Architecture
3.1 Introduction of System Architecture
3.1.1 Overview of System
3.1.2 OBS-Studio Graphic System Architecture
3.1.3 Single Scene Rendering Process
3.1.4 Input Source Rendering Process of Multiple Filter Iteration
3.1.5 Graphic API Encapsulation Processing
3.1.6 Mechanism of Image Rendering
3.2 System Requirement
3.2.1 OBS-Studio System Development Environment
3.3 OBS-Studio Filter Plugin Design
3.4 MSE, PSNR, and SSIM Design
3.4.1 Batch Process
3.4.2 Draw 3D Graph
3.4.3 Sample Power Point Presentation Test Strategy
Chapter 4 Implementation of the System
4.1 Implementation of Chroma Key Filter
4.1.1 The Implementation of Box Filter in Chroma Key
4.1.2 The Implementation of Edge Detection Filter in Chroma Key
4.1.3 The Implementation of Sharpen Filter in Chroma Key
4.1.4 The Implementation of Gaussian Blur Filter in Chroma Key
4.1.5 The Implementation of Image Matching Box Filter Resulted by MATLAB
4.1.6 The MSE Test Resulted through Chroma Key Box Filter is Drawn into a 3D DATA MAP
4.1.7 The Implementation of Image Matching Edge Detection Filter Resulted by MATLAB
4.1.8 The Implementation of Image Matching Gaussian Blur Filter Resulted by MATLAB
4.1.9 The Implementation of Image Matching Sharpen Filter Resulted by MATLAB
Chapter 5 Application of the System
5.1 Application Situation
5.1.1 Device Connection Method
5.2 Application Illustration
Chapter 6 Conclusion and Future Research
6.1 Conclusion
6.2 Future Work
References

[1]Educational technology, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Educational_technology [Accessed 15 3 2019]
[2]楊晰勛、陳世昌, “揭開磨課師製作團隊的面紗”, 數位開放教育的理論與實踐—磨課師, 科學發展(Science Development), 科技部, NO.549, 中華民國107年9月, pp.27-33
[3]MOOCS與OCW的不同?, “NCUx中央大學磨課師” [Online]. Available: http://ncux.ncu.edu.tw/?page_id=19 [Accessed 15 3 2019]
[4]Live streaming, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Live_streaming [Accessed 27 2 2019]
[5]Massive open online course, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Massive_open_online_course [Accessed 28 2 2019]
[6]Alvy Ray Smith and Jemes F. Blinn, “Blue Screen Matting”, SIGGRAPH ’96 Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, Microsoft Corporation, New York, USA, pp 259-268, 1996
[7]Ling Yin, “Automatic Stereoscopic 3D Chroma-Key Matting Using Perceptual Analysis and Prediction”, School of Electrical Engineering and Computer Science Faculty of Engineering, University of Ottawa, Canada, 2014
[8]Obs-studio/plugins/obs-filters/data/chroma_key_filter.effect, “GitHub” [Online]. Available: https://github.com/obsproject/obs-studio/blob/master/plugins/obs-filters/data/chroma_key_filter.effect [Accessed 21 1 2019]
[9]BT.709 : Parameter values for the HDTV standards for production and international programme exchange, “ITU, International Telecommunication Union” [Online]. https://www.itu.int/rec/R-REC-BT.709/ [Accessed 30 1 2018]
[10]International Telecommunication Union, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/International_Telecommunication_Union [Accessed 30 1 2018]
[11]YUV, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://zh.wikipedia.org/wiki/YUV [Accessed 30 12 2017]
[12]色彩深度與色彩取樣, “杰客森林, 單機攝影情報” [Online]. Available: https://www.jacksonlin.net/20170116-%E8%89%B2%E5%BD%A9%E6%B7%B1%E5%BA%A6%E8%88%87%E8%89%B2%E5%BD%A9%E5%8F%96%E6%A8%A3/ [Accessed 30 1 2018]
[13]影像去背, “維基百科, 自由的百科全書” [Online]. Available: https://zh.wikipedia.org/wiki/%E5%BD%B1%E5%83%8F%E5%8E%BB%E8%83%8C [Accessed 14 5 2019]
[14]Sample (DirectX HLSL Texture Object), “Microsoft”, Widows Dev Center [Online]. Available: https://docs.microsoft.com/en-us/windows/desktop/direct3dhlsl/dx-graphics-hlsl-to-sample [Accessed 25 1 2019]
[15]Kernel (image processing),” Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Kernel_(image_processing) [Accessed 21 12 2018]
[16]Image Filtering, “Machine Learning Guru” [Online]. Available: http://machinelearninguru.com/computer_vision/basics/convolution/image_convolution_1.html [Accessed 21 12 2018]
[17]TrueType, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia. org/wiki/TrueType [Accessed 2 2 2019]
[18]Computer font, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Computer_font [Accessed 2 2 2019]
[19]ClearType, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/ClearType [Accessed 15 2 2019]
[20]Mean squared error,” Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Mean_squared_error [Accessed 22 1 2019]
[21]Peak signal-to-noise ratio, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio [Accessed 17 1 2019]
[22]峰值信號雜訊比(Peak Signal Noise Ratio), “逍遙文工作室, Happy Man’s Studio”[Online]. Available: https://cg2010studio.com/2013/01/06/%E5%B3%B0%E5%80%BC%E4%BF%A1%E8%99%9F%E9%9B%9C%E8%A8%8A%E6%AF%94-peak-signal-to-noise-ratio/ [Accessed 18 1 2019]
[23]Zhou Wang and Alan C. Bovik, “Mean squared error: Love it or leave it? A new look at Signal Fidelity Measured”, IEEE Signal processing magazine, Jan 2009
[24]Structural similarity, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Structural_similarity [Accessed 18 1 2019]
[25]GitHub [Online]. Available: https://github.com/ [Accessed 21 1 2019]
[26]Open Broadcaster Software, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Open_Broadcaster_Software [Accessed 21 1 2019]
[27]CMake [Online]. Available: https://cmake.org/ [Accessed 21 1 2019]
[28]QT [Online]. Available: https://www.qt.io/ [Accessed 21 1 2019]
[29]Box blur, “Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Box_blur [Accessed 15 1 2019]
[30]Chapter 24. High-Quality Filtering, “GPU Gems” [Online]. Available: http://developer.download.nvidia.com/books/HTML/gpugems/gpugems_ch24.html [Accessed 3 12 2018]
[31]What is color spill? “DIGITAL ANARCHY, smart tools for creative minds” [Online]. Available: https://digitalanarchy.com/demos/chroma_color.html [Accessed 1 10 2018]
[32]Beato et al, US Patent No. 8,477,149 B2, Jul. 2, 2013
[33]Post Processing: Levels, Brightness & Contrast, “Computer Graphics” [Online]. Available: http://wes-uoit-comp-graphics.blogspot.com/2013/04/post-processing-levels-brightness.html [Accessed 21 1 2019]
[34]了解Gamma Correction, “LIN-DATA” [Online]. Available: http://aboutdada.com/?p=2171 [Accessed 10 11 2018]
[35]Gamma correction,” Wikipedia, The Free Encyclopedia” [Online]. Available: https://en.wikipedia.org/wiki/Gamma_correction [Accessed 18 1 2019]
[36]Welcome to OBS Studio’s documentation! “OBS Studio 20.1.0 documentation” [Online]. Available: https://obsproject.com/docs/ [Accessed 21 12 2018]
[37]Texturing and Lighting in DirectX 11, “3D Game Engine Programming” [Online]. Available: https://www.3dgep.com/texturing-lighting-directx-11/ [Accessed 21 15 2018]
[38]Shader Stages, “Microsoft | Docs, Previous Versions Documentation” [Online]. Available: https://docs.microsoft.com/en-us/previous-versions/bb205146(v=vs.85)#pixel-shader-stage [Accessed 21 15 2018]
[39]著色器(Shader), “逍遙文工作室, Happy Man’s Studio”, https://cg2010studio.com/2011/06/29/shader/ [Accessed 3 5 2019]
[40]開源直播工具OBS研究, “開發者頭條” [Online]. Available: https://toutiao.io/posts/mpdmtm/preview [Accessed 23 1 2019]